vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #314
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12client # Disk image does not exist, creating the virtualisation disk image...13client: QEMU running (pid 50)14worker2 # Disk image does not exist, creating the virtualisation disk image...15client # Formatting '/build/vm-state-client/tmp.OWyHgqeX0K', fmt=raw size=107374182416worker1 # Disk image does not exist, creating the virtualisation disk image...17worker1: QEMU running (pid 51)18client # mke2fs 1.47.4 (6-Mar-2025)19worker1 # Formatting '/build/vm-state-worker1/tmp.grU4wOI2it', fmt=raw size=107374182420worker2 # Formatting '/build/vm-state-worker2/tmp.Pd5C8dhnYF', fmt=raw size=107374182421lb # Disk image does not exist, creating the virtualisation disk image...22worker1 # mke2fs 1.47.4 (6-Mar-2025)23client # Discarding device blocks: 0/262144 done24worker2: QEMU running (pid 52)25lb # Formatting '/build/vm-state-lb/tmp.fFNtKttBgC', fmt=raw size=107374182426worker2 # mke2fs 1.47.4 (6-Mar-2025)27worker2 # Discarding device blocks: 0/262144 done28worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes29worker2 # Filesystem UUID: 5a98fc25-efc8-4bcf-bc69-de8b198663e230worker2 # Superblock backups stored on blocks:31worker2 # 32768, 98304, 163840, 22937632worker1 # Discarding device blocks: 0/262144 done33lb: QEMU running (pid 49)34worker2 # 35client # Creating filesystem with 262144 4k blocks and 65536 inodes36lb # mke2fs 1.47.4 (6-Mar-2025)37(finished: start all VMs, in 0.28 seconds)38worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes39worker2 # Allocating group tables: 0/8 done40worker2 # Writing inode tables: 0/8 done41client: waiting for unit network-addresses-eth1.service42client # Filesystem UUID: d81c2d8f-d3b8-4225-a3cf-d8a21ffbe55143lb # Discarding device blocks: 0/262144 done44client: waiting for the VM to finish booting45client # Superblock backups stored on blocks:46lb # Creating filesystem with 262144 4k blocks and 65536 inodes47worker1 # Filesystem UUID: 285ef510-549b-4e35-a2db-6acb9ecf8eb248lb # Filesystem UUID: 23d921c7-5853-484f-b198-055287288a2d49worker1 # Superblock backups stored on blocks:50client # 32768, 98304, 163840, 22937651worker2 # Creating journal (8192 blocks): done52lb # Superblock backups stored on blocks:53worker1 # 32768, 98304, 163840, 22937654worker1 # 55client # 56lb # 32768, 98304, 163840, 22937657worker2 # Writing superblocks and filesystem accounting information: 0/8 done58worker2 # 59client # Allocating group tables: 0/8 done60worker1 # Allocating group tables: 0/8 done61client # Writing inode tables: 0/8 done62lb # 63lb # Allocating group tables: 0/8 done64worker2 # Virtualisation disk image created.65worker1 # Writing inode tables: 0/8 done66lb # Writing inode tables: 0/8 done67client # Creating journal (8192 blocks): done68lb # Creating journal (8192 blocks): done69lb # Writing superblocks and filesystem accounting information: 0/8 done70worker1 # Creating journal (8192 blocks): done71lb # 72worker1 # Writing superblocks and filesystem accounting information: 0/8 done73lb # Virtualisation disk image created.74worker1 # 75client # Writing superblocks and filesystem accounting information: 0/8 done76worker1 # Virtualisation disk image created.77client # 78client # Virtualisation disk image created.79worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)80worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)81lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)82client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)83worker2 # 84worker2 # 85worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0086worker1 # 87worker1 # 88lb # 89lb # 90worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0091lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0092worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...93worker2 # 94worker2 # 95worker2 # 96worker2 # 97worker2 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB0098lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...99lb # 100lb # 101lb # 102worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...103lb # 104worker1 # 105worker1 # 106worker1 # 107worker1 # 108lb # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00109worker1 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00110worker2 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...111worker2 # 112worker2 # 113client # 114lb # Press Ctrl-B to configure iPXE (PCI 00:08.0)...115client # 116lb # 117worker1 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...118lb # 119worker1 # 120worker1 # 121client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00122client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...123client # 124client # 125client # 126client # 127client # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00128client # Press Ctrl-B to configure iPXE (PCI 00:08.0)...129client # 130client # 131lb # Booting from ROM...132worker2 # Booting from ROM...133worker1 # Booting from ROM...134client # Booting from ROM...135worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026136worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/g27xn1m1f16f9vvzlv5lxp45ik86lzk5-nixos-system-worker1-test/init regInfo=/nix/store/nz37j0fi8ibw0hm3wbnnfs2268ccybhn-closure-info/registration console=ttyS0,115200n8 console=tty0137worker1 # [ 0.000000] BIOS-provided physical RAM map:138lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026139worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable140worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved141worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved142worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable143lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0qln5mjy767qjp5wbmxjg8jay8k05dsm-nixos-system-lb-test/init regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration console=ttyS0,115200n8 console=tty0144worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved145lb # [ 0.000000] BIOS-provided physical RAM map:146worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved147lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable148worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved149lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved150worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved151lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved152lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable153worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved154lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved155worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved156lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved157worker1 # [ 0.000000] NX (Execute Disable) protection: active158lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved159worker1 # [ 0.000000] APIC: Static calls initialized160worker1 # [ 0.000000] SMBIOS 2.8 present.161lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved162lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved163worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014164lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved165worker1 # [ 0.000000] DMI: Memory slots populated: 1/1166lb # [ 0.000000] NX (Execute Disable) protection: active167worker1 # [ 0.000000] Hypervisor detected: KVM168lb # [ 0.000000] APIC: Static calls initialized169client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026170worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000171lb # [ 0.000000] SMBIOS 2.8 present.172worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00173lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014174worker1 # [ 0.000001] kvm-clock: using sched offset of 509913102 cycles175worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026176lb # [ 0.000000] DMI: Memory slots populated: 1/1177lb # [ 0.000000] Hypervisor detected: KVM178worker1 # [ 0.000003] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns179lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000180worker1 # [ 0.000007] tsc: Detected 2400.012 MHz processor181client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/6wlj2ay8s7brjjrdfwgnvhsxskfdjz0s-nixos-system-client-test/init regInfo=/nix/store/9qrjs3171pxvd0p653zlbv22i1fimhl2-closure-info/registration console=ttyS0,115200n8 console=tty0182lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00183worker1 # [ 0.000844] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000184client # [ 0.000000] BIOS-provided physical RAM map:185lb # [ 0.000000] kvm-clock: using sched offset of 509745253 cycles186client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable187worker1 # [ 0.000874] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs188lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns189client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved190worker1 # [ 0.000877] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT191worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/57k37pgcxfmcvcaikbpq6wxxg9c64rf7-nixos-system-worker2-test/init regInfo=/nix/store/8iq91pzaxdh07c3b8j2h7ipsghrhxcbk-closure-info/registration console=ttyS0,115200n8 console=tty0192lb # [ 0.000006] tsc: Detected 2400.012 MHz processor193worker2 # [ 0.000000] BIOS-provided physical RAM map:194worker1 # [ 0.002846] found SMP MP-table at [mem 0x000f5450-0x000f545f]195client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved196lb # [ 0.000829] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000197worker1 # [ 0.002863] Using GB pages for direct mapping198client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable199worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable200worker1 # [ 0.003016] RAMDISK: [mem 0x5e367000-0x5ffcffff]201lb # [ 0.000858] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs202client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved203worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved204worker1 # [ 0.003027] ACPI: Early table checksum verification disabled205lb # [ 0.000861] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT206client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved207worker1 # [ 0.003032] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )208worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved209lb # [ 0.002815] found SMP MP-table at [mem 0x000f5450-0x000f545f]210lb # [ 0.002831] Using GB pages for direct mapping211client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved212worker1 # [ 0.003036] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)213worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable214lb # [ 0.003007] RAMDISK: [mem 0x5e367000-0x5ffcffff]215client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved216lb # [ 0.003017] ACPI: Early table checksum verification disabled217worker1 # [ 0.003041] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)218worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved219client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved220lb # [ 0.003021] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )221worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved222worker1 # [ 0.003049] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)223client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved224lb # [ 0.003026] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)225worker1 # [ 0.003052] ACPI: FACS 0x000000005FFE0000 000040226client # [ 0.000000] NX (Execute Disable) protection: active227worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved228client # [ 0.000000] APIC: Static calls initialized229client # [ 0.000000] SMBIOS 2.8 present.230lb # [ 0.003031] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)231worker1 # [ 0.003053] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)232worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved233worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved234lb # [ 0.003039] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)235worker1 # [ 0.003055] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)236client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014237lb # [ 0.003041] ACPI: FACS 0x000000005FFE0000 000040238worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved239client # [ 0.000000] DMI: Memory slots populated: 1/1240client # [ 0.000000] Hypervisor detected: KVM241worker1 # [ 0.003057] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)242worker2 # [ 0.000000] NX (Execute Disable) protection: active243lb # [ 0.003042] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)244worker2 # [ 0.000000] APIC: Static calls initialized245client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000246worker2 # [ 0.000000] SMBIOS 2.8 present.247worker1 # [ 0.003058] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)248client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00249lb # [ 0.003044] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)250worker1 # [ 0.003060] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]251client # [ 0.000001] kvm-clock: using sched offset of 533485874 cycles252worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014253lb # [ 0.003046] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)254worker2 # [ 0.000000] DMI: Memory slots populated: 1/1255worker1 # [ 0.003061] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]256client # [ 0.000003] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns257worker2 # [ 0.000000] Hypervisor detected: KVM258lb # [ 0.003047] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)259worker1 # [ 0.003061] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]260client # [ 0.000007] tsc: Detected 2400.012 MHz processor261worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000262lb # [ 0.003049] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]263client # [ 0.000870] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000264worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00265worker1 # [ 0.003062] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]266lb # [ 0.003050] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]267worker2 # [ 0.000001] kvm-clock: using sched offset of 507326803 cycles268worker1 # [ 0.003062] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]269client # [ 0.000904] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs270lb # [ 0.003050] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]271client # [ 0.000908] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT272worker1 # [ 0.003063] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]273worker2 # [ 0.000004] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns274lb # [ 0.003051] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]275client # [ 0.003142] found SMP MP-table at [mem 0x000f5450-0x000f545f]276worker2 # [ 0.000007] tsc: Detected 2400.012 MHz processor277worker1 # [ 0.003063] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]278client # [ 0.003162] Using GB pages for direct mapping279worker1 # [ 0.003321] No NUMA configuration found280worker2 # [ 0.000908] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000281lb # [ 0.003051] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]282client # [ 0.003356] RAMDISK: [mem 0x5e366000-0x5ffcffff]283worker1 # [ 0.003322] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]284lb # [ 0.003052] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]285client # [ 0.003366] ACPI: Early table checksum verification disabled286worker2 # [ 0.000956] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs287worker1 # [ 0.003324] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]288lb # [ 0.003052] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]289client # [ 0.003370] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )290worker1 # [ 0.005763] Zone ranges:291worker2 # [ 0.000961] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT292lb # [ 0.003301] No NUMA configuration found293worker2 # [ 0.003112] found SMP MP-table at [mem 0x000f5450-0x000f545f]294worker1 # [ 0.005766] DMA [mem 0x0000000000001000-0x0000000000ffffff]295client # [ 0.003375] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)296lb # [ 0.003302] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]297worker2 # [ 0.003131] Using GB pages for direct mapping298worker1 # [ 0.005769] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]299worker2 # [ 0.003285] RAMDISK: [mem 0x5e367000-0x5ffcffff]300lb # [ 0.003305] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]301worker1 # [ 0.005770] Normal empty302client # [ 0.003380] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)303lb # [ 0.005711] Zone ranges:304worker1 # [ 0.005771] Device empty305worker2 # [ 0.003299] ACPI: Early table checksum verification disabled306worker1 # [ 0.005772] Movable zone start for each node307lb # [ 0.005712] DMA [mem 0x0000000000001000-0x0000000000ffffff]308client # [ 0.003389] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)309worker2 # [ 0.003304] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )310worker1 # [ 0.005773] Early memory node ranges311client # [ 0.003391] ACPI: FACS 0x000000005FFE0000 000040312lb # [ 0.005714] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]313lb # [ 0.005715] Normal empty314worker1 # [ 0.005773] node 0: [mem 0x0000000000001000-0x000000000009efff]315worker2 # [ 0.003310] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)316lb # [ 0.005716] Device empty317client # [ 0.003393] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)318worker1 # [ 0.005774] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]319lb # [ 0.005717] Movable zone start for each node320worker2 # [ 0.003315] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)321lb # [ 0.005718] Early memory node ranges322client # [ 0.003394] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)323worker1 # [ 0.005776] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]324lb # [ 0.005718] node 0: [mem 0x0000000000001000-0x000000000009efff]325worker2 # [ 0.003326] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)326worker1 # [ 0.005798] On node 0, zone DMA: 1 pages in unavailable ranges327client # [ 0.003396] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)328worker2 # [ 0.003328] ACPI: FACS 0x000000005FFE0000 000040329lb # [ 0.005719] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]330worker1 # [ 0.006073] On node 0, zone DMA: 97 pages in unavailable ranges331client # [ 0.003398] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)332lb # [ 0.005721] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]333worker2 # [ 0.003330] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)334worker1 # [ 0.035719] On node 0, zone DMA32: 40 pages in unavailable ranges335lb # [ 0.005742] On node 0, zone DMA: 1 pages in unavailable ranges336worker1 # [ 0.036280] ACPI: PM-Timer IO Port: 0x608337client # [ 0.003399] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]338worker2 # [ 0.003332] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)339lb # [ 0.006025] On node 0, zone DMA: 97 pages in unavailable ranges340worker1 # [ 0.036300] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])341client # [ 0.003400] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]342lb # [ 0.037308] On node 0, zone DMA32: 40 pages in unavailable ranges343worker2 # [ 0.003334] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)344worker1 # [ 0.036336] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23345client # [ 0.003401] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]346lb # [ 0.037818] ACPI: PM-Timer IO Port: 0x608347worker1 # [ 0.036339] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)348worker2 # [ 0.003336] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)349client # [ 0.003401] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]350lb # [ 0.037834] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])351worker1 # [ 0.036342] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)352worker2 # [ 0.003337] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]353lb # [ 0.037866] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23354client # [ 0.003402] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]355worker1 # [ 0.036343] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)356lb # [ 0.037869] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)357worker2 # [ 0.003339] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]358client # [ 0.003402] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]359worker1 # [ 0.036345] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)360lb # [ 0.037871] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)361worker2 # [ 0.003339] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]362client # [ 0.003403] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]363client # [ 0.003666] No NUMA configuration found364worker1 # [ 0.036346] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)365lb # [ 0.037872] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)366worker2 # [ 0.003340] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]367client # [ 0.003667] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]368worker1 # [ 0.036349] ACPI: Using ACPI (MADT) for SMP configuration information369lb # [ 0.037873] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)370worker2 # [ 0.003340] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]371client # [ 0.003670] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]372worker1 # [ 0.036350] ACPI: HPET id: 0x8086a201 base: 0xfed00000373client # [ 0.006261] Zone ranges:374lb # [ 0.037874] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)375worker2 # [ 0.003341] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]376worker1 # [ 0.036360] TSC deadline timer available377client # [ 0.006263] DMA [mem 0x0000000000001000-0x0000000000ffffff]378lb # [ 0.037877] ACPI: Using ACPI (MADT) for SMP configuration information379worker1 # [ 0.036367] CPU topo: Max. logical packages: 1380worker2 # [ 0.003341] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]381worker1 # [ 0.036368] CPU topo: Max. logical dies: 1382lb # [ 0.037878] ACPI: HPET id: 0x8086a201 base: 0xfed00000383client # [ 0.006264] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]384worker2 # [ 0.003648] No NUMA configuration found385client # [ 0.006266] Normal empty386lb # [ 0.037884] TSC deadline timer available387worker1 # [ 0.036368] CPU topo: Max. dies per package: 1388client # [ 0.006267] Device empty389worker2 # [ 0.003649] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]390lb # [ 0.037888] CPU topo: Max. logical packages: 1391worker1 # [ 0.036373] CPU topo: Max. threads per core: 1392client # [ 0.006267] Movable zone start for each node393lb # [ 0.037889] CPU topo: Max. logical dies: 1394worker2 # [ 0.003653] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]395worker1 # [ 0.036374] CPU topo: Num. cores per package: 1396client # [ 0.006268] Early memory node ranges397worker2 # [ 0.006700] Zone ranges:398lb # [ 0.037890] CPU topo: Max. dies per package: 1399worker1 # [ 0.036374] CPU topo: Num. threads per package: 1400client # [ 0.006269] node 0: [mem 0x0000000000001000-0x000000000009efff]401worker2 # [ 0.006702] DMA [mem 0x0000000000001000-0x0000000000ffffff]402lb # [ 0.037893] CPU topo: Max. threads per core: 1403worker1 # [ 0.036374] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs404client # [ 0.006270] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]405lb # [ 0.037894] CPU topo: Num. cores per package: 1406worker2 # [ 0.006704] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]407worker2 # [ 0.006706] Normal empty408lb # [ 0.037894] CPU topo: Num. threads per package: 1409worker1 # [ 0.036409] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()410client # [ 0.006271] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]411worker2 # [ 0.006707] Device empty412lb # [ 0.037895] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs413worker2 # [ 0.006708] Movable zone start for each node414client # [ 0.006291] On node 0, zone DMA: 1 pages in unavailable ranges415worker1 # [ 0.036460] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]416worker2 # [ 0.006709] Early memory node ranges417lb # [ 0.037921] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()418client # [ 0.006561] On node 0, zone DMA: 97 pages in unavailable ranges419worker1 # [ 0.036462] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]420worker2 # [ 0.006709] node 0: [mem 0x0000000000001000-0x000000000009efff]421client # [ 0.037260] On node 0, zone DMA32: 40 pages in unavailable ranges422lb # [ 0.037969] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]423worker2 # [ 0.006710] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]424worker1 # [ 0.036464] [mem 0x60000000-0xafffffff] available for PCI devices425client # [ 0.037801] ACPI: PM-Timer IO Port: 0x608426worker1 # [ 0.036467] Booting paravirtualized kernel on KVM427lb # [ 0.037971] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]428client # [ 0.037818] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])429worker2 # [ 0.006712] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]430lb # [ 0.037972] [mem 0x60000000-0xafffffff] available for PCI devices431worker2 # [ 0.006740] On node 0, zone DMA: 1 pages in unavailable ranges432client # [ 0.037857] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23433worker1 # [ 0.036473] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns434lb # [ 0.037974] Booting paravirtualized kernel on KVM435worker2 # [ 0.007066] On node 0, zone DMA: 97 pages in unavailable ranges436client # [ 0.037860] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)437worker1 # [ 0.041239] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1438worker2 # [ 0.040349] On node 0, zone DMA32: 40 pages in unavailable ranges439client # [ 0.037862] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)440lb # [ 0.037979] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns441worker2 # [ 0.040928] ACPI: PM-Timer IO Port: 0x608442worker1 # [ 0.043837] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152443client # [ 0.037863] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)444lb # [ 0.042656] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1445worker2 # [ 0.040945] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])446worker1 # [ 0.043885] kvm-guest: PV spinlocks disabled, single CPU447client # [ 0.037864] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)448lb # [ 0.045364] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152449worker2 # [ 0.040980] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23450lb # [ 0.045419] kvm-guest: PV spinlocks disabled, single CPU451client # [ 0.037865] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)452worker2 # [ 0.040984] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)453client # [ 0.037868] ACPI: Using ACPI (MADT) for SMP configuration information454worker2 # [ 0.040986] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)455client # [ 0.037869] ACPI: HPET id: 0x8086a201 base: 0xfed00000456worker2 # [ 0.040988] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)457client # [ 0.037876] TSC deadline timer available458client # [ 0.037881] CPU topo: Max. logical packages: 1459worker2 # [ 0.040989] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)460worker1 # [ 0.043887] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/g27xn1m1f16f9vvzlv5lxp45ik86lzk5-nixos-system-worker1-test/init regInfo=/nix/store/nz37j0fi8ibw0hm3wbnnfs2268ccybhn-closure-info/registration console=ttyS0,115200n8 console=tty0461client # [ 0.037882] CPU topo: Max. logical dies: 1462worker2 # [ 0.040990] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)463client # [ 0.037882] CPU topo: Max. dies per package: 1464lb # [ 0.045421] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0qln5mjy767qjp5wbmxjg8jay8k05dsm-nixos-system-lb-test/init regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration console=ttyS0,115200n8 console=tty0465client # [ 0.037886] CPU topo: Max. threads per core: 1466worker2 # [ 0.040992] ACPI: Using ACPI (MADT) for SMP configuration information467worker1 # [ 0.044000] Unknown kernel command line parameters "regInfo=/nix/store/nz37j0fi8ibw0hm3wbnnfs2268ccybhn-closure-info/registration", will be passed to user space.468client # [ 0.037887] CPU topo: Num. cores per package: 1469worker2 # [ 0.040994] ACPI: HPET id: 0x8086a201 base: 0xfed00000470worker1 # [ 0.044016] random: crng init done471client # [ 0.037887] CPU topo: Num. threads per package: 1472worker2 # [ 0.041001] TSC deadline timer available473lb # [ 0.045547] Unknown kernel command line parameters "regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration", will be passed to user space.474worker1 # [ 0.044017] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes475lb # [ 0.045563] random: crng init done476worker2 # [ 0.041006] CPU topo: Max. logical packages: 1477client # [ 0.037887] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs478worker2 # [ 0.041008] CPU topo: Max. logical dies: 1479lb # [ 0.045564] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes480worker1 # [ 0.046226] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)481client # [ 0.037915] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()482worker2 # [ 0.041008] CPU topo: Max. dies per package: 1483worker2 # [ 0.041012] CPU topo: Max. threads per core: 1484lb # [ 0.048076] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)485worker1 # [ 0.047342] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)486client # [ 0.037956] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]487worker2 # [ 0.041013] CPU topo: Num. cores per package: 1488worker1 # [ 0.047385] Fallback order for Node 0: 0489lb # [ 0.049254] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)490client # [ 0.037958] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]491worker2 # [ 0.041013] CPU topo: Num. threads per package: 1492lb # [ 0.049297] Fallback order for Node 0: 0493worker1 # [ 0.047389] Built 1 zonelists, mobility grouping on. Total pages: 393078494client # [ 0.037960] [mem 0x60000000-0xafffffff] available for PCI devices495worker2 # [ 0.041014] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs496worker1 # [ 0.047391] Policy zone: DMA32497lb # [ 0.049300] Built 1 zonelists, mobility grouping on. Total pages: 393078498client # [ 0.037962] Booting paravirtualized kernel on KVM499lb # [ 0.049302] Policy zone: DMA32500worker2 # [ 0.041043] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()501worker1 # [ 0.049979] mem auto-init: stack:all(zero), heap alloc:on, heap free:off502lb # [ 0.052459] mem auto-init: stack:all(zero), heap alloc:on, heap free:off503worker1 # [ 0.053775] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1504worker2 # [ 0.041088] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]505client # [ 0.037966] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns506worker1 # [ 0.057699] allocated 3145728 bytes of page_ext507lb # [ 0.056127] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1508client # [ 0.042772] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1509worker2 # [ 0.041090] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]510worker1 # [ 0.072326] ftrace: allocating 48731 entries in 192 pages511lb # [ 0.059709] allocated 3145728 bytes of page_ext512worker2 # [ 0.041092] [mem 0x60000000-0xafffffff] available for PCI devices513client # [ 0.045109] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152514worker1 # [ 0.072331] ftrace: allocated 192 pages with 2 groups515lb # [ 0.070790] ftrace: allocating 48731 entries in 192 pages516worker1 # [ 0.073410] Dynamic Preempt: lazy517worker2 # [ 0.041094] Booting paravirtualized kernel on KVM518client # [ 0.045165] kvm-guest: PV spinlocks disabled, single CPU519lb # [ 0.070794] ftrace: allocated 192 pages with 2 groups520lb # [ 0.071748] Dynamic Preempt: lazy521worker1 # [ 0.073599] rcu: Preemptible hierarchical RCU implementation.522worker2 # [ 0.041099] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns523worker1 # [ 0.073600] rcu: RCU event tracing is enabled.524lb # [ 0.071940] rcu: Preemptible hierarchical RCU implementation.525lb # [ 0.071941] rcu: RCU event tracing is enabled.526worker1 # [ 0.073601] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.527worker2 # [ 0.046077] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1528lb # [ 0.071942] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.529worker1 # [ 0.073603] Trampoline variant of Tasks RCU enabled.530worker2 # [ 0.048454] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152531lb # [ 0.071944] Trampoline variant of Tasks RCU enabled.532worker1 # [ 0.073604] Rude variant of Tasks RCU enabled.533worker2 # [ 0.048510] kvm-guest: PV spinlocks disabled, single CPU534lb # [ 0.071944] Rude variant of Tasks RCU enabled.535client # [ 0.045167] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/6wlj2ay8s7brjjrdfwgnvhsxskfdjz0s-nixos-system-client-test/init regInfo=/nix/store/9qrjs3171pxvd0p653zlbv22i1fimhl2-closure-info/registration console=ttyS0,115200n8 console=tty0536worker1 # [ 0.073604] Tracing variant of Tasks RCU enabled.537lb # [ 0.071945] Tracing variant of Tasks RCU enabled.538worker1 # [ 0.073605] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.539lb # [ 0.071946] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.540client # [ 0.045287] Unknown kernel command line parameters "regInfo=/nix/store/9qrjs3171pxvd0p653zlbv22i1fimhl2-closure-info/registration", will be passed to user space.541worker1 # [ 0.073606] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1542client # [ 0.045307] random: crng init done543lb # [ 0.071946] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1544worker1 # [ 0.073632] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.545client # [ 0.045308] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes546lb # [ 0.072024] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.547worker2 # [ 0.048511] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/57k37pgcxfmcvcaikbpq6wxxg9c64rf7-nixos-system-worker2-test/init regInfo=/nix/store/8iq91pzaxdh07c3b8j2h7ipsghrhxcbk-closure-info/registration console=ttyS0,115200n8 console=tty0548client # [ 0.047596] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)549worker1 # [ 0.073634] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.550lb # [ 0.072027] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.551client # [ 0.048816] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)552worker1 # [ 0.073635] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.553worker2 # [ 0.048618] Unknown kernel command line parameters "regInfo=/nix/store/8iq91pzaxdh07c3b8j2h7ipsghrhxcbk-closure-info/registration", will be passed to user space.554client # [ 0.048867] Fallback order for Node 0: 0555lb # [ 0.072028] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.556worker2 # [ 0.048633] random: crng init done557worker1 # [ 0.078502] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16558client # [ 0.048872] Built 1 zonelists, mobility grouping on. Total pages: 393078559lb # [ 0.077029] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16560worker2 # [ 0.048634] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes561client # [ 0.048874] Policy zone: DMA32562worker1 # [ 0.078815] rcu: srcu_init: Setting srcu_struct sizes based on contention.563lb # [ 0.077316] rcu: srcu_init: Setting srcu_struct sizes based on contention.564client # [ 0.051887] mem auto-init: stack:all(zero), heap alloc:on, heap free:off565worker2 # [ 0.050934] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)566worker1 # [ 0.078823] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns567client # [ 0.055498] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1568lb # [ 0.077324] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns569worker2 # [ 0.052217] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)570client # [ 0.059286] allocated 3145728 bytes of page_ext571worker2 # [ 0.052269] Fallback order for Node 0: 0572worker1 # [ 0.078946] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)573client # [ 0.070104] ftrace: allocating 48731 entries in 192 pages574lb # [ 0.077441] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)575worker1 # [ 0.082627] Console: colour VGA+ 80x25576worker2 # [ 0.052274] Built 1 zonelists, mobility grouping on. Total pages: 393078577lb # [ 0.081050] Console: colour VGA+ 80x25578client # [ 0.070108] ftrace: allocated 192 pages with 2 groups579worker2 # [ 0.052276] Policy zone: DMA32580worker1 # [ 0.082632] printk: legacy console [tty0] enabled581client # [ 0.071087] Dynamic Preempt: lazy582lb # [ 0.081054] printk: legacy console [tty0] enabled583worker2 # [ 0.055845] mem auto-init: stack:all(zero), heap alloc:on, heap free:off584worker1 # [ 0.112279] printk: legacy console [ttyS0] enabled585lb # [ 0.110846] printk: legacy console [ttyS0] enabled586client # [ 0.071232] rcu: Preemptible hierarchical RCU implementation.587client # [ 0.071232] rcu: RCU event tracing is enabled.588worker2 # [ 0.061017] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1589worker1 # [ 0.213808] ACPI: Core revision 20250807590lb # [ 0.208140] ACPI: Core revision 20250807591worker2 # [ 0.065266] allocated 3145728 bytes of page_ext592client # [ 0.071233] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.593worker2 # [ 0.075909] ftrace: allocating 48731 entries in 192 pages594client # [ 0.071235] Trampoline variant of Tasks RCU enabled.595lb # [ 0.209012] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns596worker2 # [ 0.075913] ftrace: allocated 192 pages with 2 groups597client # [ 0.071236] Rude variant of Tasks RCU enabled.598worker1 # [ 0.214813] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns599worker2 # [ 0.076869] Dynamic Preempt: lazy600client # [ 0.071236] Tracing variant of Tasks RCU enabled.601lb # [ 0.210511] APIC: Switch to symmetric I/O mode setup602worker1 # [ 0.216457] APIC: Switch to symmetric I/O mode setup603worker2 # [ 0.077060] rcu: Preemptible hierarchical RCU implementation.604lb # [ 0.211450] x2apic enabled605client # [ 0.071237] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.606worker2 # [ 0.077061] rcu: RCU event tracing is enabled.607worker1 # [ 0.217467] x2apic enabled608client # [ 0.071238] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1609worker2 # [ 0.077062] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.610lb # [ 0.212192] APIC: Switched APIC routing to: physical x2apic611worker1 # [ 0.218238] APIC: Switched APIC routing to: physical x2apic612worker2 # [ 0.077064] Trampoline variant of Tasks RCU enabled.613client # [ 0.071262] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.614worker2 # [ 0.077064] Rude variant of Tasks RCU enabled.615worker2 # [ 0.077064] Tracing variant of Tasks RCU enabled.616client # [ 0.071264] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.617lb # [ 0.214198] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1618worker1 # [ 0.220178] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1619worker2 # [ 0.077065] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.620client # [ 0.071264] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.621worker2 # [ 0.077066] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1622client # [ 0.075759] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16623lb # [ 0.215173] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns624worker1 # [ 0.221186] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns625worker2 # [ 0.077089] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.626client # [ 0.076065] rcu: srcu_init: Setting srcu_struct sizes based on contention.627lb # [ 0.216811] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)628worker2 # [ 0.077091] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.629worker1 # [ 0.222938] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)630client # [ 0.076075] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns631lb # [ 0.217897] x86/cpu: User Mode Instruction Prevention (UMIP) activated632worker2 # [ 0.077092] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.633worker1 # [ 0.224025] x86/cpu: User Mode Instruction Prevention (UMIP) activated634client # [ 0.076194] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)635worker2 # [ 0.081574] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16636client # [ 0.080054] Console: colour VGA+ 80x25637lb # [ 0.218942] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127638client # [ 0.080058] printk: legacy console [tty0] enabled639worker1 # [ 0.226074] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127640worker2 # [ 0.081860] rcu: srcu_init: Setting srcu_struct sizes based on contention.641lb # [ 0.220479] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0642client # [ 0.110227] printk: legacy console [ttyS0] enabled643worker1 # [ 0.226933] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0644worker2 # [ 0.081866] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns645client # [ 0.206053] ACPI: Core revision 20250807646lb # [ 0.221556] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto647worker2 # [ 0.081985] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)648worker1 # [ 0.227938] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto649worker2 # [ 0.085532] Console: colour VGA+ 80x25650client # [ 0.207000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns651worker2 # [ 0.085536] printk: legacy console [tty0] enabled652lb # [ 0.222808] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl653client # [ 0.208561] APIC: Switch to symmetric I/O mode setup654worker2 # [ 0.119993] printk: legacy console [ttyS0] enabled655worker1 # [ 0.228936] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl656lb # [ 0.223808] Transient Scheduler Attacks: Vulnerable: No microcode657client # [ 0.209505] x2apic enabled658worker2 # [ 0.218057] ACPI: Core revision 20250807659lb # [ 0.225509] Spectre V2 : Mitigation: Enhanced / Automatic IBRS660worker1 # [ 0.230934] Transient Scheduler Attacks: Vulnerable: No microcode661client # [ 0.210248] APIC: Switched APIC routing to: physical x2apic662lb # [ 0.225808] Speculative Return Stack Overflow: Mitigation: Safe RET663worker2 # [ 0.218976] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns664worker1 # [ 0.231933] Spectre V2 : Mitigation: Enhanced / Automatic IBRS665worker2 # [ 0.220479] APIC: Switch to symmetric I/O mode setup666worker1 # [ 0.232934] Speculative Return Stack Overflow: Mitigation: Safe RET667client # [ 0.212072] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1668worker2 # [ 0.221475] x2apic enabled669worker2 # [ 0.222262] APIC: Switched APIC routing to: physical x2apic670client # [ 0.213045] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns671lb # [ 0.226807] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization672client # [ 0.214687] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)673lb # [ 0.228815] Spectre V2 : Enabling IBPB for BPF674worker2 # [ 0.224062] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1675worker1 # [ 0.233933] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization676client # [ 0.215775] x86/cpu: User Mode Instruction Prevention (UMIP) activated677worker1 # [ 0.235944] Spectre V2 : Enabling IBPB for BPF678lb # [ 0.229808] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier679client # [ 0.216822] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127680worker2 # [ 0.225044] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns681lb # [ 0.230808] active return thunk: srso_alias_return_thunk682worker1 # [ 0.236935] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier683client # [ 0.217684] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0684worker2 # [ 0.226691] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)685lb # [ 0.231829] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'686worker1 # [ 0.237934] active return thunk: srso_alias_return_thunk687worker2 # [ 0.227776] x86/cpu: User Mode Instruction Prevention (UMIP) activated688client # [ 0.218688] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto689lb # [ 0.232807] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'690worker1 # [ 0.239623] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'691lb # [ 0.234535] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'692worker2 # [ 0.229761] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127693client # [ 0.219683] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl694worker1 # [ 0.240933] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'695lb # [ 0.234808] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'696worker2 # [ 0.230655] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0697client # [ 0.220684] Transient Scheduler Attacks: Vulnerable: No microcode698worker1 # [ 0.241933] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'699lb # [ 0.235807] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'700client # [ 0.221683] Spectre V2 : Mitigation: Enhanced / Automatic IBRS701worker1 # [ 0.242933] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'702worker2 # [ 0.231438] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto703lb # [ 0.236807] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'704client # [ 0.222683] Speculative Return Stack Overflow: Mitigation: Safe RET705worker1 # [ 0.243933] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'706worker2 # [ 0.232688] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl707lb # [ 0.237808] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'708worker1 # [ 0.244932] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'709worker2 # [ 0.233689] Transient Scheduler Attacks: Vulnerable: No microcode710lb # [ 0.239713] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256711worker2 # [ 0.234615] Spectre V2 : Mitigation: Enhanced / Automatic IBRS712worker1 # [ 0.245933] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'713lb # [ 0.240516] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64714worker2 # [ 0.235367] Speculative Return Stack Overflow: Mitigation: Safe RET715worker1 # [ 0.246934] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256716client # [ 0.223683] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization717lb # [ 0.241525] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512718client # [ 0.225692] Spectre V2 : Enabling IBPB for BPF719worker1 # [ 0.247936] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64720lb # [ 0.242512] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024721lb # [ 0.242808] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8722worker1 # [ 0.248933] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512723client # [ 0.226684] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier724worker1 # [ 0.249933] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024725client # [ 0.227684] active return thunk: srso_alias_return_thunk726lb # [ 0.243807] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.727worker2 # [ 0.236409] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization728worker1 # [ 0.250933] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8729client # [ 0.228706] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'730worker2 # [ 0.237695] Spectre V2 : Enabling IBPB for BPF731client # [ 0.229683] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'732worker1 # [ 0.251933] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.733worker2 # [ 0.238689] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier734client # [ 0.230683] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'735worker2 # [ 0.239689] active return thunk: srso_alias_return_thunk736client # [ 0.231683] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'737worker2 # [ 0.241338] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'738client # [ 0.232683] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'739worker2 # [ 0.242617] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'740client # [ 0.233683] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'741worker2 # [ 0.243688] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'742client # [ 0.234683] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'743worker2 # [ 0.244688] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'744client # [ 0.235684] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256745worker2 # [ 0.245688] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'746client # [ 0.236683] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64747worker2 # [ 0.246688] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'748client # [ 0.237683] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512749client # [ 0.238683] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024750worker2 # [ 0.247688] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'751client # [ 0.239683] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8752worker2 # [ 0.248688] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256753worker2 # [ 0.249688] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64754client # [ 0.240683] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.755worker2 # [ 0.250688] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512756worker2 # [ 0.251688] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024757worker2 # [ 0.252688] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8758worker2 # [ 0.253688] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.759lb # [ 0.280378] Freeing SMP alternatives memory: 44K760lb # [ 0.280811] pid_max: default: 32768 minimum: 301761lb # [ 0.281914] LSM: initializing lsm=capability,landlock,yama,bpf,ima762lb # [ 0.282915] landlock: Up and running.763lb # [ 0.283808] Yama: becoming mindful.764lb # [ 0.284609] LSM support for eBPF active765lb # [ 0.285379] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)766lb # [ 0.285849] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)767worker1 # [ 0.294357] Freeing SMP alternatives memory: 44K768lb # [ 0.289201] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)769worker1 # [ 0.294932] pid_max: default: 32768 minimum: 301770worker1 # [ 0.296036] LSM: initializing lsm=capability,landlock,yama,bpf,ima771lb # [ 0.290365] Performance Events: Fam17h+ core perfctr, AMD PMU driver.772lb # [ 0.290817] ... version: 2773worker1 # [ 0.297074] landlock: Up and running.774worker1 # [ 0.297934] Yama: becoming mindful.775lb # [ 0.291527] ... bit width: 48776lb # [ 0.291810] ... generic counters: 6777worker1 # [ 0.298816] LSM support for eBPF active778client # [ 0.277903] Freeing SMP alternatives memory: 44K779lb # [ 0.292494] ... generic bitmap: 000000000000003f780client # [ 0.278689] pid_max: default: 32768 minimum: 301781lb # [ 0.292810] ... fixed-purpose counters: 0782worker1 # [ 0.299539] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)783lb # [ 0.293501] ... fixed-purpose bitmap: 0000000000000000784client # [ 0.279507] LSM: initializing lsm=capability,landlock,yama,bpf,ima785worker1 # [ 0.300978] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)786client # [ 0.280498] landlock: Up and running.787lb # [ 0.293810] ... value mask: 0000ffffffffffff788client # [ 0.281107] Yama: becoming mindful.789worker2 # [ 0.289737] Freeing SMP alternatives memory: 44K790lb # [ 0.294663] ... max period: 00007fffffffffff791client # [ 0.281909] LSM support for eBPF active792worker2 # [ 0.290506] pid_max: default: 32768 minimum: 301793lb # [ 0.295454] ... global_ctrl mask: 000000000000003f794worker1 # [ 0.303423] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)795worker2 # [ 0.291323] LSM: initializing lsm=capability,landlock,yama,bpf,ima796lb # [ 0.295935] signal: max sigframe size: 3376797client # [ 0.282788] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)798worker2 # [ 0.291798] landlock: Up and running.799lb # [ 0.296703] rcu: Hierarchical SRCU implementation.800worker1 # [ 0.304534] Performance Events: Fam17h+ core perfctr, AMD PMU driver.801worker2 # [ 0.292689] Yama: becoming mindful.802client # [ 0.283720] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)803lb # [ 0.297401] rcu: Max phase no-delay instances is 400.804worker1 # [ 0.304942] ... version: 2805worker2 # [ 0.293907] LSM support for eBPF active806worker1 # [ 0.305647] ... bit width: 48807worker1 # [ 0.305935] ... generic counters: 6808worker2 # [ 0.294731] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)809client # [ 0.286303] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)810worker1 # [ 0.306608] ... generic bitmap: 000000000000003f811worker2 # [ 0.295724] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)812worker1 # [ 0.306936] ... fixed-purpose counters: 0813client # [ 0.287248] Performance Events: Fam17h+ core perfctr, AMD PMU driver.814worker1 # [ 0.307653] ... fixed-purpose bitmap: 0000000000000000815client # [ 0.287688] ... version: 2816worker1 # [ 0.307936] ... value mask: 0000ffffffffffff817client # [ 0.288390] ... bit width: 48818lb # [ 0.302486] smp: Bringing up secondary CPUs ...819worker2 # [ 0.298222] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)820client # [ 0.288686] ... generic counters: 6821worker1 # [ 0.308803] ... max period: 00007fffffffffff822lb # [ 0.302826] smp: Brought up 1 node, 1 CPU823worker1 # [ 0.309592] ... global_ctrl mask: 000000000000003f824lb # [ 0.303508] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)825worker2 # [ 0.299278] Performance Events: Fam17h+ core perfctr, AMD PMU driver.826client # [ 0.289360] ... generic bitmap: 000000000000003f827worker1 # [ 0.310046] signal: max sigframe size: 3376828worker2 # [ 0.299694] ... version: 2829worker1 # [ 0.310841] rcu: Hierarchical SRCU implementation.830worker2 # [ 0.300429] ... bit width: 48831client # [ 0.289775] ... fixed-purpose counters: 0832worker1 # [ 0.311517] rcu: Max phase no-delay instances is 400.833worker2 # [ 0.300691] ... generic counters: 6834lb # [ 0.304084] Memory: 1454568K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)835client # [ 0.290686] ... fixed-purpose bitmap: 0000000000000000836lb # [ 0.305072] devtmpfs: initialized837worker2 # [ 0.301361] ... generic bitmap: 000000000000003f838client # [ 0.291567] ... value mask: 0000ffffffffffff839worker2 # [ 0.301703] ... fixed-purpose counters: 0840lb # [ 0.305859] x86/mm: Memory block size: 128MB841client # [ 0.292430] ... max period: 00007fffffffffff842worker2 # [ 0.302405] ... fixed-purpose bitmap: 0000000000000000843client # [ 0.292688] ... global_ctrl mask: 000000000000003f844worker2 # [ 0.302690] ... value mask: 0000ffffffffffff845client # [ 0.293797] signal: max sigframe size: 3376846lb # [ 0.307676] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)847worker2 # [ 0.303623] ... max period: 00007fffffffffff848client # [ 0.294768] rcu: Hierarchical SRCU implementation.849worker2 # [ 0.304410] ... global_ctrl mask: 000000000000003f850lb # [ 0.308709] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).851worker1 # [ 0.316891] smp: Bringing up secondary CPUs ...852client # [ 0.295605] rcu: Max phase no-delay instances is 400.853worker2 # [ 0.304806] signal: max sigframe size: 3376854worker1 # [ 0.317526] smp: Brought up 1 node, 1 CPU855lb # [ 0.309901] pinctrl core: initialized pinctrl subsystem856worker2 # [ 0.305589] rcu: Hierarchical SRCU implementation.857worker1 # [ 0.317937] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)858lb # [ 0.311013] PM: RTC time: 12:20:16, date: 2026-09-20859worker2 # [ 0.306292] rcu: Max phase no-delay instances is 400.860worker1 # [ 0.319172] Memory: 1454564K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)861worker1 # [ 0.320295] devtmpfs: initialized862lb # [ 0.314695] NET: Registered PF_NETLINK/PF_ROUTE protocol family863worker1 # [ 0.321210] x86/mm: Memory block size: 128MB864client # [ 0.301605] smp: Bringing up secondary CPUs ...865lb # [ 0.315992] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations866client # [ 0.302333] smp: Brought up 1 node, 1 CPU867worker2 # [ 0.311417] smp: Bringing up secondary CPUs ...868client # [ 0.302690] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)869worker2 # [ 0.311707] smp: Brought up 1 node, 1 CPU870lb # [ 0.316833] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations871worker1 # [ 0.323136] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)872worker2 # [ 0.312342] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)873worker1 # [ 0.323970] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).874lb # [ 0.318089] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations875client # [ 0.303947] Memory: 1454600K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109348K reserved, 0K cma-reserved)876lb # [ 0.318820] audit: initializing netlink subsys (disabled)877worker1 # [ 0.325072] pinctrl core: initialized pinctrl subsystem878worker2 # [ 0.312901] Memory: 1454560K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)879client # [ 0.305007] devtmpfs: initialized880worker1 # [ 0.326213] PM: RTC time: 12:20:16, date: 2026-09-20881lb # [ 0.319947] thermal_sys: Registered thermal governor 'fair_share'882worker2 # [ 0.314013] devtmpfs: initialized883client # [ 0.305832] x86/mm: Memory block size: 128MB884lb # [ 0.319949] thermal_sys: Registered thermal governor 'bang_bang'885worker2 # [ 0.314806] x86/mm: Memory block size: 128MB886lb # [ 0.320812] thermal_sys: Registered thermal governor 'step_wise'887lb # [ 0.321742] thermal_sys: Registered thermal governor 'user_space'888client # [ 0.307833] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)889worker2 # [ 0.316622] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)890lb # [ 0.322528] audit: type=2000 audit(1789906817.598:1): state=initialized audit_enabled=0 res=1891worker1 # [ 0.330205] NET: Registered PF_NETLINK/PF_ROUTE protocol family892client # [ 0.308723] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).893worker2 # [ 0.317569] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).894lb # [ 0.323813] thermal_sys: Registered thermal governor 'power_allocator'895client # [ 0.309798] pinctrl core: initialized pinctrl subsystem896worker1 # [ 0.331458] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations897lb # [ 0.323827] cpuidle: using governor menu898worker2 # [ 0.318794] pinctrl core: initialized pinctrl subsystem899client # [ 0.310978] PM: RTC time: 12:20:16, date: 2026-09-20900worker2 # [ 0.319886] PM: RTC time: 12:20:16, date: 2026-09-20901worker1 # [ 0.331964] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations902lb # [ 0.326011] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5903worker1 # [ 0.333245] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations904lb # [ 0.327034] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]905worker1 # [ 0.333954] audit: initializing netlink subsys (disabled)906lb # [ 0.327814] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry907client # [ 0.314587] NET: Registered PF_NETLINK/PF_ROUTE protocol family908worker1 # [ 0.335110] thermal_sys: Registered thermal governor 'fair_share'909worker2 # [ 0.323538] NET: Registered PF_NETLINK/PF_ROUTE protocol family910lb # [ 0.328823] PCI: Using configuration type 1 for base access911worker1 # [ 0.335113] thermal_sys: Registered thermal governor 'bang_bang'912client # [ 0.315961] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations913lb # [ 0.329916] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.914worker2 # [ 0.324937] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations915worker1 # [ 0.335937] thermal_sys: Registered thermal governor 'step_wise'916client # [ 0.316712] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations917worker1 # [ 0.336890] thermal_sys: Registered thermal governor 'user_space'918worker2 # [ 0.325714] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations919worker1 # [ 0.337641] audit: type=2000 audit(1789906817.603:1): state=initialized audit_enabled=0 res=1920client # [ 0.317968] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations921worker2 # [ 0.326971] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations922client # [ 0.318702] audit: initializing netlink subsys (disabled)923worker1 # [ 0.338940] thermal_sys: Registered thermal governor 'power_allocator'924worker2 # [ 0.327704] audit: initializing netlink subsys (disabled)925worker1 # [ 0.338961] cpuidle: using governor menu926worker2 # [ 0.328770] thermal_sys: Registered thermal governor 'fair_share'927client # [ 0.319688] audit: type=2000 audit(1789906817.613:1): state=initialized audit_enabled=0 res=1928lb # [ 0.334901] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages929worker2 # [ 0.328772] thermal_sys: Registered thermal governor 'bang_bang'930worker1 # [ 0.341260] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5931lb # [ 0.335813] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page932client # [ 0.320920] thermal_sys: Registered thermal governor 'fair_share'933worker2 # [ 0.329692] thermal_sys: Registered thermal governor 'step_wise'934client # [ 0.320922] thermal_sys: Registered thermal governor 'bang_bang'935worker2 # [ 0.330592] thermal_sys: Registered thermal governor 'user_space'936worker1 # [ 0.342207] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]937client # [ 0.321687] thermal_sys: Registered thermal governor 'step_wise'938worker1 # [ 0.342942] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry939worker2 # [ 0.331386] audit: type=2000 audit(1789906817.608:1): state=initialized audit_enabled=0 res=1940client # [ 0.322649] thermal_sys: Registered thermal governor 'user_space'941worker1 # [ 0.343956] PCI: Using configuration type 1 for base access942worker2 # [ 0.332693] thermal_sys: Registered thermal governor 'power_allocator'943client # [ 0.323452] thermal_sys: Registered thermal governor 'power_allocator'944worker2 # [ 0.332709] cpuidle: using governor menu945client # [ 0.323706] cpuidle: using governor menu946lb # [ 0.340811] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages947worker1 # [ 0.345137] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.948lb # [ 0.341810] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page949worker2 # [ 0.334901] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5950client # [ 0.325928] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5951worker2 # [ 0.335919] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]952client # [ 0.326937] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]953worker2 # [ 0.336694] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry954client # [ 0.327691] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry955worker2 # [ 0.337703] PCI: Using configuration type 1 for base access956client # [ 0.328705] PCI: Using configuration type 1 for base access957worker1 # [ 0.350073] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages958worker2 # [ 0.338772] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.959worker1 # [ 0.350937] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page960client # [ 0.329849] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.961lb # [ 0.352181] ACPI: Added _OSI(Module Device)962worker1 # [ 0.355937] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages963lb # [ 0.352812] ACPI: Added _OSI(Processor Device)964worker1 # [ 0.356936] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page965lb # [ 0.353515] ACPI: Added _OSI(Processor Aggregator Device)966worker2 # [ 0.345929] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages967client # [ 0.337011] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages968worker2 # [ 0.346692] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page969client # [ 0.337688] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page970client # [ 0.338686] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages971worker2 # [ 0.351692] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages972worker2 # [ 0.352691] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page973lb # [ 0.362171] ACPI: 1 ACPI AML tables successfully acquired and loaded974worker1 # [ 0.367381] ACPI: Added _OSI(Module Device)975client # [ 0.345686] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page976worker1 # [ 0.367938] ACPI: Added _OSI(Processor Device)977worker1 # [ 0.368711] ACPI: Added _OSI(Processor Aggregator Device)978lb # [ 0.366240] ACPI: Interpreter enabled979lb # [ 0.366827] ACPI: PM: (supports S0 S3 S4 S5)980lb # [ 0.367537] ACPI: Using IOAPIC for interrupt routing981worker2 # [ 0.361374] ACPI: Added _OSI(Module Device)982worker2 # [ 0.361692] ACPI: Added _OSI(Processor Device)983worker2 # [ 0.362435] ACPI: Added _OSI(Processor Aggregator Device)984lb # [ 0.369899] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug985client # [ 0.354282] ACPI: Added _OSI(Module Device)986lb # [ 0.370811] PCI: Using E820 reservations for host bridge windows987client # [ 0.354688] ACPI: Added _OSI(Processor Device)988client # [ 0.355466] ACPI: Added _OSI(Processor Aggregator Device)989worker1 # [ 0.377402] ACPI: 1 ACPI AML tables successfully acquired and loaded990lb # [ 0.373910] ACPI: Enabled 2 GPEs in block 00 to 3F991worker1 # [ 0.381417] ACPI: Interpreter enabled992worker2 # [ 0.369282] ACPI: 1 ACPI AML tables successfully acquired and loaded993client # [ 0.360277] ACPI: 1 ACPI AML tables successfully acquired and loaded994worker1 # [ 0.381958] ACPI: PM: (supports S0 S3 S4 S5)995worker1 # [ 0.382713] ACPI: Using IOAPIC for interrupt routing996worker2 # [ 0.372722] ACPI: Interpreter enabled997client # [ 0.363714] ACPI: Interpreter enabled998worker2 # [ 0.373418] ACPI: PM: (supports S0 S3 S4 S5)999lb # [ 0.381554] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1000client # [ 0.364342] ACPI: PM: (supports S0 S3 S4 S5)1001worker1 # [ 0.385050] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1002worker2 # [ 0.373692] ACPI: Using IOAPIC for interrupt routing1003client # [ 0.364688] ACPI: Using IOAPIC for interrupt routing1004lb # [ 0.381818] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1005worker1 # [ 0.385937] PCI: Using E820 reservations for host bridge windows1006worker2 # [ 0.374502] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1007lb # [ 0.382900] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1008client # [ 0.365505] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1009lb # [ 0.383932] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1010worker1 # [ 0.389023] ACPI: Enabled 2 GPEs in block 00 to 3F1011lb # [ 0.385185] PCI host bridge to bus 0000:001012lb # [ 0.385815] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1013lb # [ 0.386810] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1014worker2 # [ 0.379691] PCI: Using E820 reservations for host bridge windows1015client # [ 0.370687] PCI: Using E820 reservations for host bridge windows1016lb # [ 0.387811] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1017worker2 # [ 0.380837] ACPI: Enabled 2 GPEs in block 00 to 3F1018client # [ 0.371811] ACPI: Enabled 2 GPEs in block 00 to 3F1019lb # [ 0.388810] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1020lb # [ 0.389810] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1021lb # [ 0.390810] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1022lb # [ 0.391811] pci_bus 0000:00: root bus resource [bus 00-ff]1023worker1 # [ 0.397110] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1024lb # [ 0.392897] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1025worker1 # [ 0.397946] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1026worker1 # [ 0.399026] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1027lb # [ 0.394274] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1028worker1 # [ 0.400060] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1029worker2 # [ 0.389496] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1030client # [ 0.380536] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1031worker1 # [ 0.401346] PCI host bridge to bus 0000:001032worker1 # [ 0.401941] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1033worker2 # [ 0.391131] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1034worker1 # [ 0.402936] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1035client # [ 0.382699] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1036worker2 # [ 0.391772] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1037worker1 # [ 0.403936] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1038worker2 # [ 0.392813] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1039client # [ 0.383775] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1040worker1 # [ 0.404935] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1041worker2 # [ 0.394058] PCI host bridge to bus 0000:001042client # [ 0.384809] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1043worker2 # [ 0.394695] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1044worker1 # [ 0.405936] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1045client # [ 0.386078] PCI host bridge to bus 0000:001046worker2 # [ 0.395692] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1047worker1 # [ 0.406935] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1048client # [ 0.386692] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1049lb # [ 0.396907] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1050worker1 # [ 0.407937] pci_bus 0000:00: root bus resource [bus 00-ff]1051worker2 # [ 0.396691] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1052lb # [ 0.397878] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1053client # [ 0.387687] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1054worker2 # [ 0.397690] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1055client # [ 0.388686] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1056lb # [ 0.398826] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1057worker1 # [ 0.408915] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1058worker2 # [ 0.398691] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1059client # [ 0.389687] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1060lb # [ 0.400191] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1061worker1 # [ 0.410425] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1062worker2 # [ 0.399691] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1063client # [ 0.390686] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1064worker2 # [ 0.400692] pci_bus 0000:00: root bus resource [bus 00-ff]1065client # [ 0.391686] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1066lb # [ 0.401627] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1067client # [ 0.392687] pci_bus 0000:00: root bus resource [bus 00-ff]1068worker2 # [ 0.401713] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1069client # [ 0.393740] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1070worker2 # [ 0.403146] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1071lb # [ 0.404844] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1072lb # [ 0.405703] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1073client # [ 0.395177] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1074lb # [ 0.406540] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1075lb # [ 0.406817] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1076lb # [ 0.408412] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1077worker1 # [ 0.413027] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1078worker1 # [ 0.413950] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1079worker2 # [ 0.405765] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1080lb # [ 0.410777] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1081worker1 # [ 0.414957] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1082worker2 # [ 0.406809] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1083lb # [ 0.411423] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1084worker2 # [ 0.407765] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1085worker1 # [ 0.416306] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1086lb # [ 0.411833] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1087worker2 # [ 0.409215] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1088lb # [ 0.413382] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1089worker1 # [ 0.417775] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1090client # [ 0.397785] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1091client # [ 0.398700] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1092worker2 # [ 0.410485] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1093client # [ 0.399649] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1094worker1 # [ 0.420976] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1095client # [ 0.400904] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1096lb # [ 0.416778] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1097worker1 # [ 0.421818] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1098lb # [ 0.417426] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1099worker2 # [ 0.413711] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1100worker1 # [ 0.422959] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1101client # [ 0.402532] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1102lb # [ 0.417832] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1103worker2 # [ 0.414546] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1104worker1 # [ 0.423944] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1105worker2 # [ 0.415387] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1106lb # [ 0.419365] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1107worker2 # [ 0.415697] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1108worker1 # [ 0.425516] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1109client # [ 0.405695] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1110worker2 # [ 0.417812] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1111lb # [ 0.421724] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1112client # [ 0.407123] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1113lb # [ 0.422448] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1114worker1 # [ 0.427968] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1115client # [ 0.407708] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1116worker1 # [ 0.428853] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1117worker2 # [ 0.419719] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1118client # [ 0.408692] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1119lb # [ 0.422832] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1120worker2 # [ 0.420579] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1121worker1 # [ 0.429669] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1122client # [ 0.410279] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1123lb # [ 0.424375] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1124worker2 # [ 0.421642] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1125worker1 # [ 0.430696] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1126worker2 # [ 0.423389] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1127lb # [ 0.426593] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1128client # [ 0.412718] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1129lb # [ 0.427434] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1130client # [ 0.413540] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1131lb # [ 0.427832] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1132client # [ 0.414420] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1133worker1 # [ 0.433965] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1134worker1 # [ 0.434944] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1135lb # [ 0.429430] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1136worker2 # [ 0.426701] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1137client # [ 0.416082] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1138worker1 # [ 0.435964] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1139worker2 # [ 0.427635] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1140worker2 # [ 0.428489] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1141lb # [ 0.432740] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1142worker1 # [ 0.437686] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1143client # [ 0.418722] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1144lb # [ 0.433425] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1145client # [ 0.419530] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1146worker2 # [ 0.430390] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1147lb # [ 0.433833] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1148client # [ 0.420417] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1149worker1 # [ 0.440968] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1150lb # [ 0.435430] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1151client # [ 0.421332] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1152worker2 # [ 0.433717] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1153worker1 # [ 0.441947] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1154worker1 # [ 0.442966] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1155lb # [ 0.436850] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1156worker2 # [ 0.434699] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1157lb # [ 0.437700] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1158worker2 # [ 0.435718] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1159worker1 # [ 0.444558] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1160client # [ 0.424693] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1161client # [ 0.425511] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1162lb # [ 0.438539] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1163worker2 # [ 0.437372] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1164client # [ 0.426396] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1165lb # [ 0.438817] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1166client # [ 0.427279] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1167lb # [ 0.440344] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1168worker1 # [ 0.447968] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1169worker2 # [ 0.440726] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1170worker1 # [ 0.448945] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1171worker2 # [ 0.441701] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1172lb # [ 0.441822] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1173client # [ 0.428719] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1174worker1 # [ 0.449967] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1175worker2 # [ 0.442717] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1176lb # [ 0.442775] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1177client # [ 0.429582] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1178lb # [ 0.444230] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1179worker1 # [ 0.451722] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1180client # [ 0.430444] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1181worker2 # [ 0.445133] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1182client # [ 0.431308] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1183lb # [ 0.445855] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1184worker2 # [ 0.447720] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1185worker1 # [ 0.454969] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1186lb # [ 0.447456] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1187worker2 # [ 0.448652] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1188client # [ 0.433673] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1189worker1 # [ 0.455950] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1190lb # [ 0.447832] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1191worker2 # [ 0.449469] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1192client # [ 0.434324] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1193worker1 # [ 0.456966] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1194client # [ 0.434709] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1195lb # [ 0.449372] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1196worker2 # [ 0.451314] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1197worker1 # [ 0.458646] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1198client # [ 0.436345] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1199lb # [ 0.450818] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1200lb # [ 0.451711] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1201worker1 # [ 0.461734] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1202lb # [ 0.452553] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1203worker2 # [ 0.453699] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1204worker1 # [ 0.462588] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1205worker2 # [ 0.454597] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1206client # [ 0.438722] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1207worker1 # [ 0.462959] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1208worker2 # [ 0.455433] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1209client # [ 0.439608] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1210lb # [ 0.453395] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1211worker2 # [ 0.455697] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1212client # [ 0.440426] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1213worker1 # [ 0.464258] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1214lb # [ 0.455616] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1215client # [ 0.440693] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1216worker2 # [ 0.457297] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1217worker1 # [ 0.465524] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1218lb # [ 0.456626] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1219client # [ 0.442324] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1220worker2 # [ 0.458703] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1221lb # [ 0.458504] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1222worker1 # [ 0.466968] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1223worker2 # [ 0.459690] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1224client # [ 0.443698] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1225worker1 # [ 0.467952] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1226client # [ 0.444657] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1227worker2 # [ 0.462014] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1228worker1 # [ 0.469492] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1229client # [ 0.446258] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1230worker2 # [ 0.463729] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1231worker2 # [ 0.464700] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1232worker1 # [ 0.471938] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1233client # [ 0.448568] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1234worker1 # [ 0.473663] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1235worker2 # [ 0.465717] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1236client # [ 0.449356] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1237worker1 # [ 0.473960] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1238client # [ 0.449709] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1239worker2 # [ 0.467346] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1240worker1 # [ 0.475507] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1241client # [ 0.451282] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1242lb # [ 0.459012] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1243worker1 # [ 0.477683] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1244lb # [ 0.461647] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1245worker2 # [ 0.469009] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1246worker1 # [ 0.477946] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1247worker2 # [ 0.470459] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1248worker1 # [ 0.478909] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1249lb # [ 0.462861] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1250client # [ 0.453070] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1251worker2 # [ 0.470713] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1252client # [ 0.454408] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1253lb # [ 0.464620] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1254worker2 # [ 0.472313] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1255client # [ 0.454710] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1256worker1 # [ 0.481427] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1257lb # [ 0.465810] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1258worker2 # [ 0.473437] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1259client # [ 0.456260] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1260worker1 # [ 0.482804] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1261lb # [ 0.467098] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1262worker2 # [ 0.473904] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1263client # [ 0.457583] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1264lb # [ 0.468057] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1265worker1 # [ 0.483798] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1266worker2 # [ 0.475361] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1267client # [ 0.458491] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1268worker1 # [ 0.485713] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1269lb # [ 0.469827] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1270client # [ 0.460429] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1271lb # [ 0.470672] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1272lb # [ 0.472151] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1273lb # [ 0.473619] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1274lb # [ 0.477523] ACPI: PCI: Interrupt link LNKA configured for IRQ 101275lb # [ 0.477924] ACPI: PCI: Interrupt link LNKB configured for IRQ 101276lb # [ 0.478912] ACPI: PCI: Interrupt link LNKC configured for IRQ 111277worker2 # [ 0.475890] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1278lb # [ 0.479932] ACPI: PCI: Interrupt link LNKD configured for IRQ 111279lb # [ 0.480910] ACPI: PCI: Interrupt link LNKE configured for IRQ 101280worker1 # [ 0.486169] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1281lb # [ 0.481907] ACPI: PCI: Interrupt link LNKF configured for IRQ 101282client # [ 0.461720] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1283lb # [ 0.482911] ACPI: PCI: Interrupt link LNKG configured for IRQ 111284worker1 # [ 0.488746] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1285worker2 # [ 0.479427] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1286lb # [ 0.483908] ACPI: PCI: Interrupt link LNKH configured for IRQ 111287worker1 # [ 0.489851] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1288worker2 # [ 0.479958] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1289lb # [ 0.484847] ACPI: PCI: Interrupt link GSIA configured for IRQ 161290lb # [ 0.485780] ACPI: PCI: Interrupt link GSIB configured for IRQ 171291lb # [ 0.486539] ACPI: PCI: Interrupt link GSIC configured for IRQ 181292worker2 # [ 0.481717] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1293worker1 # [ 0.492438] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1294lb # [ 0.486825] ACPI: PCI: Interrupt link GSID configured for IRQ 191295client # [ 0.465519] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1296worker2 # [ 0.482925] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1297worker1 # [ 0.493233] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1298lb # [ 0.487757] ACPI: PCI: Interrupt link GSIE configured for IRQ 201299client # [ 0.466605] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1300lb # [ 0.488701] ACPI: PCI: Interrupt link GSIF configured for IRQ 211301worker2 # [ 0.483979] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1302worker1 # [ 0.494254] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1303lb # [ 0.489530] ACPI: PCI: Interrupt link GSIG configured for IRQ 221304client # [ 0.468694] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1305lb # [ 0.489822] ACPI: PCI: Interrupt link GSIH configured for IRQ 231306worker1 # [ 0.495211] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1307client # [ 0.469932] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1308lb # [ 0.491854] iommu: Default domain type: Translated1309worker2 # [ 0.484960] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1310lb # [ 0.492712] iommu: DMA domain TLB invalidation policy: lazy mode1311client # [ 0.470983] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1312lb # [ 0.493798] ACPI: bus type USB registered1313lb # [ 0.494532] usbcore: registered new interface driver usbfs1314worker2 # [ 0.487721] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1315lb # [ 0.494829] usbcore: registered new interface driver hub1316worker2 # [ 0.488556] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1317lb # [ 0.495664] usbcore: registered new device driver usb1318worker2 # [ 0.489794] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1319lb # [ 0.497324] NetLabel: Initializing1320client # [ 0.471968] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1321worker2 # [ 0.491722] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1322lb # [ 0.497812] NetLabel: domain hash size = 1281323client # [ 0.474680] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1324lb # [ 0.498812] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1325worker1 # [ 0.499945] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1326client # [ 0.475573] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1327lb # [ 0.499792] NetLabel: unlabeled traffic allowed by default1328worker1 # [ 0.500827] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1329lb # [ 0.500480] PCI: Using ACPI for IRQ routing1330worker2 # [ 0.496378] ACPI: PCI: Interrupt link LNKA configured for IRQ 101331client # [ 0.476754] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1332worker1 # [ 0.502023] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1333worker2 # [ 0.496798] ACPI: PCI: Interrupt link LNKB configured for IRQ 101334worker2 # [ 0.497789] ACPI: PCI: Interrupt link LNKC configured for IRQ 111335client # [ 0.479127] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1336worker1 # [ 0.503964] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1337worker2 # [ 0.498791] ACPI: PCI: Interrupt link LNKD configured for IRQ 111338worker2 # [ 0.499791] ACPI: PCI: Interrupt link LNKE configured for IRQ 101339worker2 # [ 0.500787] ACPI: PCI: Interrupt link LNKF configured for IRQ 101340worker2 # [ 0.501793] ACPI: PCI: Interrupt link LNKG configured for IRQ 111341worker1 # [ 0.508087] ACPI: PCI: Interrupt link LNKA configured for IRQ 101342worker2 # [ 0.502791] ACPI: PCI: Interrupt link LNKH configured for IRQ 111343client # [ 0.483410] ACPI: PCI: Interrupt link LNKA configured for IRQ 101344worker2 # [ 0.503725] ACPI: PCI: Interrupt link GSIA configured for IRQ 161345client # [ 0.484596] ACPI: PCI: Interrupt link LNKB configured for IRQ 101346worker1 # [ 0.509066] ACPI: PCI: Interrupt link LNKB configured for IRQ 101347worker2 # [ 0.504706] ACPI: PCI: Interrupt link GSIB configured for IRQ 171348client # [ 0.485541] ACPI: PCI: Interrupt link LNKC configured for IRQ 111349worker1 # [ 0.510766] ACPI: PCI: Interrupt link LNKC configured for IRQ 111350worker2 # [ 0.505715] ACPI: PCI: Interrupt link GSIC configured for IRQ 181351client # [ 0.486513] ACPI: PCI: Interrupt link LNKD configured for IRQ 111352worker1 # [ 0.511751] ACPI: PCI: Interrupt link LNKD configured for IRQ 111353worker2 # [ 0.506692] ACPI: PCI: Interrupt link GSID configured for IRQ 191354client # [ 0.487508] ACPI: PCI: Interrupt link LNKE configured for IRQ 101355worker1 # [ 0.512737] ACPI: PCI: Interrupt link LNKE configured for IRQ 101356worker2 # [ 0.507656] ACPI: PCI: Interrupt link GSIE configured for IRQ 201357client # [ 0.488518] ACPI: PCI: Interrupt link LNKF configured for IRQ 101358worker2 # [ 0.508422] ACPI: PCI: Interrupt link GSIF configured for IRQ 211359worker1 # [ 0.513756] ACPI: PCI: Interrupt link LNKF configured for IRQ 101360client # [ 0.489510] ACPI: PCI: Interrupt link LNKG configured for IRQ 111361worker2 # [ 0.508707] ACPI: PCI: Interrupt link GSIG configured for IRQ 221362worker1 # [ 0.514796] ACPI: PCI: Interrupt link LNKG configured for IRQ 111363client # [ 0.490517] ACPI: PCI: Interrupt link LNKH configured for IRQ 111364worker2 # [ 0.509685] ACPI: PCI: Interrupt link GSIH configured for IRQ 231365worker1 # [ 0.515845] ACPI: PCI: Interrupt link LNKH configured for IRQ 111366client # [ 0.491456] ACPI: PCI: Interrupt link GSIA configured for IRQ 161367worker1 # [ 0.516684] ACPI: PCI: Interrupt link GSIA configured for IRQ 161368worker2 # [ 0.511493] iommu: Default domain type: Translated1369client # [ 0.491699] ACPI: PCI: Interrupt link GSIB configured for IRQ 171370worker1 # [ 0.516955] ACPI: PCI: Interrupt link GSIB configured for IRQ 171371worker2 # [ 0.512268] iommu: DMA domain TLB invalidation policy: lazy mode1372client # [ 0.492701] ACPI: PCI: Interrupt link GSIC configured for IRQ 181373worker1 # [ 0.517887] ACPI: PCI: Interrupt link GSIC configured for IRQ 181374worker2 # [ 0.512988] ACPI: bus type USB registered1375client # [ 0.493702] ACPI: PCI: Interrupt link GSID configured for IRQ 191376worker1 # [ 0.518898] ACPI: PCI: Interrupt link GSID configured for IRQ 191377worker2 # [ 0.513751] usbcore: registered new interface driver usbfs1378client # [ 0.494698] ACPI: PCI: Interrupt link GSIE configured for IRQ 201379worker1 # [ 0.519659] ACPI: PCI: Interrupt link GSIE configured for IRQ 201380worker2 # [ 0.514646] usbcore: registered new interface driver hub1381client # [ 0.495616] ACPI: PCI: Interrupt link GSIF configured for IRQ 211382worker1 # [ 0.519955] ACPI: PCI: Interrupt link GSIF configured for IRQ 211383worker2 # [ 0.515391] usbcore: registered new device driver usb1384client # [ 0.496431] ACPI: PCI: Interrupt link GSIG configured for IRQ 221385worker1 # [ 0.520944] ACPI: PCI: Interrupt link GSIG configured for IRQ 221386client # [ 0.496701] ACPI: PCI: Interrupt link GSIH configured for IRQ 231387worker2 # [ 0.516836] NetLabel: Initializing1388worker1 # [ 0.521899] ACPI: PCI: Interrupt link GSIH configured for IRQ 231389worker2 # [ 0.517442] NetLabel: domain hash size = 1281390worker2 # [ 0.517691] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1391client # [ 0.498996] iommu: Default domain type: Translated1392worker1 # [ 0.523979] iommu: Default domain type: Translated1393worker2 # [ 0.518685] NetLabel: unlabeled traffic allowed by default1394client # [ 0.499694] iommu: DMA domain TLB invalidation policy: lazy mode1395worker1 # [ 0.524751] iommu: DMA domain TLB invalidation policy: lazy mode1396worker2 # [ 0.519373] PCI: Using ACPI for IRQ routing1397client # [ 0.500939] ACPI: bus type USB registered1398worker1 # [ 0.525964] ACPI: bus type USB registered1399client # [ 0.501694] usbcore: registered new interface driver usbfs1400worker1 # [ 0.526696] usbcore: registered new interface driver usbfs1401client # [ 0.502612] usbcore: registered new interface driver hub1402worker1 # [ 0.526965] usbcore: registered new interface driver hub1403client # [ 0.503358] usbcore: registered new device driver usb1404worker1 # [ 0.527953] usbcore: registered new device driver usb1405client # [ 0.504638] NetLabel: Initializing1406client # [ 0.505175] NetLabel: domain hash size = 1281407worker1 # [ 0.529972] NetLabel: Initializing1408client # [ 0.505686] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1409worker1 # [ 0.530659] NetLabel: domain hash size = 1281410worker1 # [ 0.530936] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1411client # [ 0.506702] NetLabel: unlabeled traffic allowed by default1412client # [ 0.507651] PCI: Using ACPI for IRQ routing1413worker1 # [ 0.531904] NetLabel: unlabeled traffic allowed by default1414worker1 # [ 0.532631] PCI: Using ACPI for IRQ routing1415lb # [ 0.544420] pci 0000:00:01.0: vgaarb: setting as boot VGA device1416lb # [ 0.544806] pci 0000:00:01.0: vgaarb: bridge control possible1417lb # [ 0.544806] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1418lb # [ 0.544815] vgaarb: loaded1419lb # [ 0.545568] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01420lb # [ 0.546448] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1421lb # [ 0.550911] clocksource: Switched to clocksource kvm-clock1422lb # [ 0.554925] VFS: Disk quotas dquot_6.6.01423lb # [ 0.555789] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1424lb # [ 0.557340] pnp: PnP ACPI init1425lb # [ 0.558276] ACPI: IRQ 4 override to edge(!), high(!)1426lb # [ 0.559309] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1427lb # [ 0.561073] pnp: PnP ACPI: found 6 devices1428worker2 # [ 0.563417] pci 0000:00:01.0: vgaarb: setting as boot VGA device1429worker2 # [ 0.563687] pci 0000:00:01.0: vgaarb: bridge control possible1430lb # [ 0.569034] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1431lb # [ 0.570542] clocksource: Switched to clocksource acpi_pm1432worker2 # [ 0.563687] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1433worker2 # [ 0.563697] vgaarb: loaded1434lb # [ 0.571813] NET: Registered PF_INET protocol family1435worker2 # [ 0.564500] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01436worker2 # [ 0.565313] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1437lb # [ 0.573243] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1438client # [ 0.552200] pci 0000:00:01.0: vgaarb: setting as boot VGA device1439client # [ 0.552682] pci 0000:00:01.0: vgaarb: bridge control possible1440worker1 # [ 0.577223] pci 0000:00:01.0: vgaarb: setting as boot VGA device1441worker1 # [ 0.577932] pci 0000:00:01.0: vgaarb: bridge control possible1442client # [ 0.552682] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1443client # [ 0.552691] vgaarb: loaded1444worker1 # [ 0.577932] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1445worker2 # [ 0.571788] clocksource: Switched to clocksource kvm-clock1446client # [ 0.553433] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01447worker1 # [ 0.577942] vgaarb: loaded1448client # [ 0.553689] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1449worker1 # [ 0.578735] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01450worker1 # [ 0.579531] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1451worker2 # [ 0.575431] VFS: Disk quotas dquot_6.6.01452worker2 # [ 0.576176] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1453worker2 # [ 0.577556] pnp: PnP ACPI init1454worker2 # [ 0.578357] ACPI: IRQ 4 override to edge(!), high(!)1455worker1 # [ 0.583056] clocksource: Switched to clocksource kvm-clock1456client # [ 0.558816] clocksource: Switched to clocksource kvm-clock1457worker2 # [ 0.579341] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1458worker1 # [ 0.585061] VFS: Disk quotas dquot_6.6.01459client # [ 0.560517] VFS: Disk quotas dquot_6.6.01460worker2 # [ 0.580901] pnp: PnP ACPI: found 6 devices1461lb # [ 0.592093] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1462client # [ 0.563383] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1463worker1 # [ 0.588303] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1464client # [ 0.564991] pnp: PnP ACPI init1465lb # [ 0.593777] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1466worker1 # [ 0.590044] pnp: PnP ACPI init1467client # [ 0.565907] ACPI: IRQ 4 override to edge(!), high(!)1468worker1 # [ 0.591096] ACPI: IRQ 4 override to edge(!), high(!)1469lb # [ 0.595201] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1470client # [ 0.566979] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1471lb # [ 0.596658] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1472worker1 # [ 0.592419] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1473client # [ 0.568530] pnp: PnP ACPI: found 6 devices1474worker2 # [ 0.588615] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1475lb # [ 0.597953] TCP: Hash tables configured (established 16384 bind 16384)1476worker1 # [ 0.594259] pnp: PnP ACPI: found 6 devices1477worker2 # [ 0.590074] clocksource: Switched to clocksource acpi_pm1478lb # [ 0.599155] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1479worker2 # [ 0.591088] NET: Registered PF_INET protocol family1480lb # [ 0.600531] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1481worker2 # [ 0.592303] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1482lb # [ 0.601651] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1483lb # [ 0.602929] NET: Registered PF_UNIX/PF_LOCAL protocol family1484lb # [ 0.603978] NET: Registered PF_XDP protocol family1485lb # [ 0.604928] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1486client # [ 0.576338] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1487lb # [ 0.605962] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1488client # [ 0.577796] clocksource: Switched to clocksource acpi_pm1489lb # [ 0.607002] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1490worker1 # [ 0.602464] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1491client # [ 0.578777] NET: Registered PF_INET protocol family1492lb # [ 0.608116] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1493worker1 # [ 0.604300] clocksource: Switched to clocksource acpi_pm1494lb # [ 0.609241] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1495client # [ 0.579982] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1496worker1 # [ 0.605536] NET: Registered PF_INET protocol family1497lb # [ 0.610370] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1498worker1 # [ 0.607057] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1499lb # [ 0.612337] ACPI: \_SB_.GSIA: Enabled at IRQ 161500lb # [ 0.615561] ACPI: \_SB_.GSIB: Enabled at IRQ 171501lb # [ 0.617628] ACPI: \_SB_.GSIC: Enabled at IRQ 181502lb # [ 0.619930] ACPI: \_SB_.GSID: Enabled at IRQ 191503worker2 # [ 0.610436] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1504worker2 # [ 0.612075] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1505lb # [ 0.621808] PCI: CLS 0 bytes, default 641506worker2 # [ 0.613524] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1507lb # [ 0.622921] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1508worker2 # [ 0.615010] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1509lb # [ 0.624805] Trying to unpack rootfs image as initramfs...1510worker2 # [ 0.616333] TCP: Hash tables configured (established 16384 bind 16384)1511worker2 # [ 0.617544] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1512worker2 # [ 0.619006] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1513client # [ 0.598864] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1514worker2 # [ 0.620132] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1515client # [ 0.600430] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1516worker2 # [ 0.621398] NET: Registered PF_UNIX/PF_LOCAL protocol family1517worker2 # [ 0.622410] NET: Registered PF_XDP protocol family1518client # [ 0.601906] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1519worker2 # [ 0.623289] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1520client # [ 0.603334] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1521worker2 # [ 0.624295] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1522client # [ 0.604646] TCP: Hash tables configured (established 16384 bind 16384)1523worker1 # [ 0.629393] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1524worker2 # [ 0.625287] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1525client # [ 0.605942] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1526worker2 # [ 0.626387] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1527worker1 # [ 0.631064] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1528client # [ 0.607203] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1529worker2 # [ 0.627470] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1530worker1 # [ 0.632409] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1531client # [ 0.608317] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1532worker2 # [ 0.628596] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1533worker1 # [ 0.633782] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1534client # [ 0.609593] NET: Registered PF_UNIX/PF_LOCAL protocol family1535worker2 # [ 0.630349] ACPI: \_SB_.GSIA: Enabled at IRQ 161536worker1 # [ 0.635079] TCP: Hash tables configured (established 16384 bind 16384)1537client # [ 0.610661] NET: Registered PF_XDP protocol family1538client # [ 0.611522] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1539worker1 # [ 0.636368] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1540worker2 # [ 0.632704] ACPI: \_SB_.GSIB: Enabled at IRQ 171541client # [ 0.612682] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1542worker1 # [ 0.637728] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1543client # [ 0.613672] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1544worker1 # [ 0.638861] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1545client # [ 0.614793] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1546worker2 # [ 0.635300] ACPI: \_SB_.GSIC: Enabled at IRQ 181547worker1 # [ 0.640180] NET: Registered PF_UNIX/PF_LOCAL protocol family1548client # [ 0.615947] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1549worker1 # [ 0.641197] NET: Registered PF_XDP protocol family1550client # [ 0.617061] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1551worker1 # [ 0.642070] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1552worker2 # [ 0.637727] ACPI: \_SB_.GSID: Enabled at IRQ 191553worker1 # [ 0.643102] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1554client # [ 0.618829] ACPI: \_SB_.GSIA: Enabled at IRQ 161555worker2 # [ 0.639485] PCI: CLS 0 bytes, default 641556worker1 # [ 0.644154] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1557worker1 # [ 0.645273] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1558worker2 # [ 0.640364] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1559client # [ 0.621031] ACPI: \_SB_.GSIB: Enabled at IRQ 171560worker1 # [ 0.646320] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1561worker2 # [ 0.642011] Trying to unpack rootfs image as initramfs...1562worker1 # [ 0.647499] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1563client # [ 0.623249] ACPI: \_SB_.GSIC: Enabled at IRQ 181564worker1 # [ 0.649309] ACPI: \_SB_.GSIA: Enabled at IRQ 161565client # [ 0.625328] ACPI: \_SB_.GSID: Enabled at IRQ 191566worker1 # [ 0.651517] ACPI: \_SB_.GSIB: Enabled at IRQ 171567client # [ 0.627114] PCI: CLS 0 bytes, default 641568client # [ 0.628111] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1569worker1 # [ 0.653921] ACPI: \_SB_.GSIC: Enabled at IRQ 181570client # [ 0.629971] Trying to unpack rootfs image as initramfs...1571worker1 # [ 0.655922] ACPI: \_SB_.GSID: Enabled at IRQ 191572worker1 # [ 0.657707] PCI: CLS 0 bytes, default 641573worker1 # [ 0.658704] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1574worker1 # [ 0.660590] Trying to unpack rootfs image as initramfs...1575lb # [ 0.675262] Initialise system trusted keyrings1576lb # [ 0.676309] workingset: timestamp_bits=40 max_order=19 bucket_order=01577worker2 # [ 0.689680] Initialise system trusted keyrings1578worker2 # [ 0.692914] workingset: timestamp_bits=40 max_order=19 bucket_order=01579lb # [ 0.702595] Key type asymmetric registered1580lb # [ 0.703478] Asymmetric key parser 'x509' registered1581lb # [ 0.704405] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1582client # [ 0.677455] Initialise system trusted keyrings1583lb # [ 0.707711] io scheduler mq-deadline registered1584lb # [ 0.708461] io scheduler kyber registered1585client # [ 0.680691] workingset: timestamp_bits=40 max_order=19 bucket_order=01586lb # [ 0.712546] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1587worker1 # [ 0.708988] Initialise system trusted keyrings1588lb # [ 0.713888] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1589worker1 # [ 0.712195] workingset: timestamp_bits=40 max_order=19 bucket_order=01590lb # [ 0.719476] Linux agpgart interface v0.1031591lb # [ 0.720295] ACPI: bus type drm_connector registered1592lb # [ 0.724798] usbcore: registered new interface driver usbserial_generic1593lb # [ 0.725846] usbserial: USB Serial support registered for generic1594worker2 # [ 0.719813] Key type asymmetric registered1595worker2 # [ 0.720718] Asymmetric key parser 'x509' registered1596lb # [ 0.729524] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1597lb # [ 0.729524] Please enable it if your BIOS has the CPPC option.1598client # [ 0.702267] Key type asymmetric registered1599lb # [ 0.731830] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1600worker2 # [ 0.725950] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1601client # [ 0.706663] Asymmetric key parser 'x509' registered1602lb # [ 0.735739] drop_monitor: Initializing network drop monitor service1603lb # [ 0.736959] NET: Registered PF_INET6 protocol family1604client # [ 0.707521] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1605worker1 # [ 0.733586] Key type asymmetric registered1606worker2 # [ 0.729917] io scheduler mq-deadline registered1607worker2 # [ 0.730779] io scheduler kyber registered1608client # [ 0.710815] io scheduler mq-deadline registered1609lb # [ 0.740129] Segment Routing with IPv61610worker1 # [ 0.736151] Asymmetric key parser 'x509' registered1611client # [ 0.711564] io scheduler kyber registered1612worker1 # [ 0.737022] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1613lb # [ 0.742535] In-situ OAM (IOAM) with IPv61614worker2 # [ 0.733534] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1615client # [ 0.714292] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1616lb # [ 0.743841] IPI shorthand broadcast: enabled1617worker1 # [ 0.741322] io scheduler mq-deadline registered1618worker2 # [ 0.736884] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1619worker1 # [ 0.742079] io scheduler kyber registered1620client # [ 0.717653] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1621worker1 # [ 0.744765] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1622worker2 # [ 0.740543] Linux agpgart interface v0.1031623worker2 # [ 0.741325] ACPI: bus type drm_connector registered1624client # [ 0.721315] Linux agpgart interface v0.1031625worker1 # [ 0.746075] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1626client # [ 0.722078] ACPI: bus type drm_connector registered1627lb # [ 0.752778] sched_clock: Marking stable (618018979, 134483253)->(826415987, -73913755)1628worker2 # [ 0.744375] usbcore: registered new interface driver usbserial_generic1629worker2 # [ 0.745402] usbserial: USB Serial support registered for generic1630client # [ 0.725144] usbcore: registered new interface driver usbserial_generic1631client # [ 0.726239] usbserial: USB Serial support registered for generic1632worker1 # [ 0.751963] Linux agpgart interface v0.1031633worker1 # [ 0.752803] ACPI: bus type drm_connector registered1634client # [ 0.728648] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1635worker2 # [ 0.748852] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1636lb # [ 0.758835] registered taskstats version 11637worker2 # [ 0.748852] Please enable it if your BIOS has the CPPC option.1638client # [ 0.728648] Please enable it if your BIOS has the CPPC option.1639lb # [ 0.759808] Loading compiled-in X.509 certificates1640client # [ 0.730809] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1641worker2 # [ 0.751015] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1642worker1 # [ 0.757644] usbcore: registered new interface driver usbserial_generic1643worker1 # [ 0.758769] usbserial: USB Serial support registered for generic1644worker2 # [ 0.755043] drop_monitor: Initializing network drop monitor service1645client # [ 0.735822] drop_monitor: Initializing network drop monitor service1646worker2 # [ 0.756303] NET: Registered PF_INET6 protocol family1647worker1 # [ 0.761142] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1648client # [ 0.737028] NET: Registered PF_INET6 protocol family1649worker1 # [ 0.761142] Please enable it if your BIOS has the CPPC option.1650worker1 # [ 0.763450] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1651worker2 # [ 0.759398] Segment Routing with IPv61652client # [ 0.740164] Segment Routing with IPv61653client # [ 0.742656] In-situ OAM (IOAM) with IPv61654worker2 # [ 0.762859] In-situ OAM (IOAM) with IPv61655worker1 # [ 0.767315] drop_monitor: Initializing network drop monitor service1656client # [ 0.743724] IPI shorthand broadcast: enabled1657worker1 # [ 0.768528] NET: Registered PF_INET6 protocol family1658worker2 # [ 0.764041] IPI shorthand broadcast: enabled1659worker1 # [ 0.771699] Segment Routing with IPv61660lb # [ 0.778879] Demotion targets for Node 0: null1661worker1 # [ 0.775159] In-situ OAM (IOAM) with IPv61662lb # [ 0.779910] Key type .fscrypt registered1663worker1 # [ 0.776306] IPI shorthand broadcast: enabled1664client # [ 0.752006] sched_clock: Marking stable (617016166, 134607647)->(839352213, -87728400)1665worker2 # [ 0.772645] sched_clock: Marking stable (631284238, 140821296)->(848147670, -76042136)1666lb # [ 0.782511] Key type fscrypt-provisioning registered1667lb # [ 0.783449] ima: No TPM chip found, activating TPM-bypass!1668lb # [ 0.784337] ima: Allocated hash algorithm: sha11669worker2 # [ 0.776074] registered taskstats version 11670client # [ 0.756925] registered taskstats version 11671worker2 # [ 0.777076] Loading compiled-in X.509 certificates1672lb # [ 0.786544] ima: No architecture policies found1673client # [ 0.757899] Loading compiled-in X.509 certificates1674lb # [ 0.789525] PM: Magic number: 10:66:3301675worker1 # [ 0.784948] sched_clock: Marking stable (645015540, 139109779)->(868099969, -83974650)1676lb # [ 0.790240] tty tty51: hash matches1677lb # [ 0.794211] RAS: Correctable Errors collector initialized.1678worker1 # [ 0.790265] registered taskstats version 11679worker1 # [ 0.791292] Loading compiled-in X.509 certificates1680lb # [ 0.803281] clk: Disabling unused clocks1681worker2 # [ 0.796270] Demotion targets for Node 0: null1682lb # [ 0.805537] PM: genpd: Disabling unused power domains1683client # [ 0.776649] Demotion targets for Node 0: null1684client # [ 0.777522] Key type .fscrypt registered1685worker2 # [ 0.798913] Key type .fscrypt registered1686worker2 # [ 0.799617] Key type fscrypt-provisioning registered1687worker2 # [ 0.800646] ima: No TPM chip found, activating TPM-bypass!1688client # [ 0.780631] Key type fscrypt-provisioning registered1689client # [ 0.781639] ima: No TPM chip found, activating TPM-bypass!1690client # [ 0.782566] ima: Allocated hash algorithm: sha11691client # [ 0.783438] ima: No architecture policies found1692worker2 # [ 0.804848] ima: Allocated hash algorithm: sha11693worker1 # [ 0.810208] Demotion targets for Node 0: null1694worker2 # [ 0.805638] ima: No architecture policies found1695client # [ 0.785635] PM: Magic number: 10:66:3301696client # [ 0.786343] tty tty51: hash matches1697worker1 # [ 0.811218] Key type .fscrypt registered1698worker1 # [ 0.811901] Key type fscrypt-provisioning registered1699worker2 # [ 0.807854] PM: Magic number: 10:66:3301700worker2 # [ 0.808614] tty tty51: hash matches1701client # [ 0.790022] RAS: Correctable Errors collector initialized.1702worker1 # [ 0.815269] ima: No TPM chip found, activating TPM-bypass!1703worker1 # [ 0.816202] ima: Allocated hash algorithm: sha11704worker1 # [ 0.817041] ima: No architecture policies found1705worker2 # [ 0.812378] RAS: Correctable Errors collector initialized.1706worker1 # [ 0.821151] PM: Magic number: 10:66:3301707worker1 # [ 0.821910] tty tty51: hash matches1708client # [ 0.798832] clk: Disabling unused clocks1709worker1 # [ 0.825656] RAS: Correctable Errors collector initialized.1710worker2 # [ 0.821370] clk: Disabling unused clocks1711client # [ 0.799603] PM: genpd: Disabling unused power domains1712worker2 # [ 0.823864] PM: genpd: Disabling unused power domains1713worker1 # [ 0.834670] clk: Disabling unused clocks1714worker1 # [ 0.837144] PM: genpd: Disabling unused power domains1715lb # [ 0.995297] Freeing initrd memory: 29092K1716lb # [ 0.998801] Freeing unused decrypted memory: 2028K1717lb # [ 1.001680] Freeing unused kernel image (initmem) memory: 3644K1718lb # [ 1.002773] Write protecting the kernel read-only data: 32768k1719lb # [ 1.004772] Freeing unused kernel image (text/rodata gap) memory: 1208K1720lb # [ 1.006335] Freeing unused kernel image (rodata/data gap) memory: 752K1721client # [ 0.980009] Freeing initrd memory: 29096K1722client # [ 0.983403] Freeing unused decrypted memory: 2028K1723client # [ 0.986343] Freeing unused kernel image (initmem) memory: 3644K1724client # [ 0.987559] Write protecting the kernel read-only data: 32768k1725worker1 # [ 1.013429] Freeing initrd memory: 29092K1726client # [ 0.989579] Freeing unused kernel image (text/rodata gap) memory: 1208K1727client # [ 0.991203] Freeing unused kernel image (rodata/data gap) memory: 752K1728worker1 # [ 1.016958] Freeing unused decrypted memory: 2028K1729worker2 # [ 1.014541] Freeing initrd memory: 29092K1730worker1 # [ 1.019924] Freeing unused kernel image (initmem) memory: 3644K1731worker1 # [ 1.021104] Write protecting the kernel read-only data: 32768k1732worker2 # [ 1.017952] Freeing unused decrypted memory: 2028K1733worker1 # [ 1.023161] Freeing unused kernel image (text/rodata gap) memory: 1208K1734worker1 # [ 1.024721] Freeing unused kernel image (rodata/data gap) memory: 752K1735worker2 # [ 1.020620] Freeing unused kernel image (initmem) memory: 3644K1736worker2 # [ 1.021724] Write protecting the kernel read-only data: 32768k1737worker2 # [ 1.023623] Freeing unused kernel image (text/rodata gap) memory: 1208K1738worker2 # [ 1.025140] Freeing unused kernel image (rodata/data gap) memory: 752K1739lb # [ 1.058431] x86/mm: Checked W+X mappings: passed, no W+X pages found.1740lb # [ 1.059583] Run /init as init process1741lb # [ 1.070779] systemd[1]: Inserted module 'autofs4'1742client # [ 1.043139] x86/mm: Checked W+X mappings: passed, no W+X pages found.1743client # [ 1.044311] Run /init as init process1744worker1 # [ 1.076654] x86/mm: Checked W+X mappings: passed, no W+X pages found.1745worker1 # [ 1.077776] Run /init as init process1746client # [ 1.055395] systemd[1]: Inserted module 'autofs4'1747worker2 # [ 1.077229] x86/mm: Checked W+X mappings: passed, no W+X pages found.1748worker2 # [ 1.078434] Run /init as init process1749lb # [ 1.088088] fuse: init (API version 7.45)1750worker1 # [ 1.088719] systemd[1]: Inserted module 'autofs4'1751lb # [ 1.094338] ACPI: \_SB_.GSIG: Enabled at IRQ 221752lb # [ 1.097072] ACPI: \_SB_.GSIH: Enabled at IRQ 231753worker2 # [ 1.090077] systemd[1]: Inserted module 'autofs4'1754lb # [ 1.101390] ACPI: \_SB_.GSIE: Enabled at IRQ 201755client # [ 1.073048] fuse: init (API version 7.45)1756lb # [ 1.104331] ACPI: \_SB_.GSIF: Enabled at IRQ 211757client # [ 1.079128] ACPI: \_SB_.GSIG: Enabled at IRQ 221758worker1 # [ 1.105864] fuse: init (API version 7.45)1759client # [ 1.081951] ACPI: \_SB_.GSIH: Enabled at IRQ 231760client # [ 1.085182] ACPI: \_SB_.GSIE: Enabled at IRQ 201761worker1 # [ 1.111888] ACPI: \_SB_.GSIG: Enabled at IRQ 221762worker2 # [ 1.107575] fuse: init (API version 7.45)1763client # [ 1.088010] ACPI: \_SB_.GSIF: Enabled at IRQ 211764worker1 # [ 1.114463] ACPI: \_SB_.GSIH: Enabled at IRQ 231765worker1 # [ 1.117666] ACPI: \_SB_.GSIE: Enabled at IRQ 201766worker2 # [ 1.114391] ACPI: \_SB_.GSIG: Enabled at IRQ 221767worker1 # [ 1.120030] ACPI: \_SB_.GSIF: Enabled at IRQ 211768worker2 # [ 1.117153] ACPI: \_SB_.GSIH: Enabled at IRQ 231769worker2 # [ 1.121058] ACPI: \_SB_.GSIE: Enabled at IRQ 201770worker2 # [ 1.124034] ACPI: \_SB_.GSIF: Enabled at IRQ 211771lb # [ 1.133449] systemd[1]: Successfully made /usr/ read-only.1772client # [ 1.117751] systemd[1]: Successfully made /usr/ read-only.1773worker1 # [ 1.147182] systemd[1]: Successfully made /usr/ read-only.1774worker2 # [ 1.155033] systemd[1]: Successfully made /usr/ read-only.1775lb # [ 1.469008] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1776lb # [ 1.474446] systemd[1]: Detected virtualization kvm.1777lb # [ 1.475346] systemd[1]: Detected architecture x86-64.1778lb # [ 1.476298] systemd[1]: Running in initrd.1779lb # [ 1.477629] systemd[1]: Initializing machine ID from random generator.1780lb # [ 1.478902] systemd[1]: Hostname set to <lb>.1781worker1 # [ 1.483177] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1782worker1 # [ 1.488354] systemd[1]: Detected virtualization kvm.1783worker1 # [ 1.489221] systemd[1]: Detected architecture x86-64.1784worker1 # [ 1.490133] systemd[1]: Running in initrd.1785client # [ 1.454475] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1786worker1 # [ 1.491344] systemd[1]: Initializing machine ID from random generator.1787client # [ 1.466349] systemd[1]: Detected virtualization kvm.1788worker1 # [ 1.492545] systemd[1]: Hostname set to <worker1>.1789client # [ 1.468546] systemd[1]: Detected architecture x86-64.1790client # [ 1.470731] systemd[1]: Running in initrd.1791client # [ 1.473291] systemd[1]: Initializing machine ID from random generator.1792client # [ 1.476195] systemd[1]: Hostname set to <client>.1793worker2 # [ 1.491749] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1794worker2 # [ 1.503677] systemd[1]: Detected virtualization kvm.1795worker2 # [ 1.505872] systemd[1]: Detected architecture x86-64.1796worker2 # [ 1.508217] systemd[1]: Running in initrd.1797worker2 # [ 1.510908] systemd[1]: Initializing machine ID from random generator.1798worker2 # [ 1.514118] systemd[1]: Hostname set to <worker2>.1799lb # [ 1.684109] systemd[1]: bpf-restrict-fs: LSM BPF program attached1800worker1 # [ 1.707295] systemd[1]: bpf-restrict-fs: LSM BPF program attached1801client # [ 1.700220] systemd[1]: bpf-restrict-fs: LSM BPF program attached1802lb # [ 1.741185] systemd[1]: Queued start job for default target Initrd Default Target.1803lb # [ 1.746212] systemd[1]: Created slice Slice /system/modprobe.1804lb # [ 1.747612] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1805lb # [ 1.749307] systemd[1]: Expecting device /dev/disk/by-label/nixos...1806lb # [ 1.750504] systemd[1]: Reached target Path Units.1807lb # [ 1.751390] systemd[1]: Reached target Slice Units.1808lb # [ 1.752357] systemd[1]: Reached target Swaps.1809lb # [ 1.753204] systemd[1]: Reached target Timer Units.1810worker2 # [ 1.743056] systemd[1]: bpf-restrict-fs: LSM BPF program attached1811lb # [ 1.754260] systemd[1]: Listening on D-Bus System Message Bus Socket.1812lb # [ 1.755692] systemd[1]: Listening on Journal Socket (/dev/log).1813lb # [ 1.757023] systemd[1]: Listening on Journal Sockets.1814lb # [ 1.758258] systemd[1]: Listening on udev Control Socket.1815lb # [ 1.759454] systemd[1]: Listening on udev Kernel Socket.1816lb # [ 1.760550] systemd[1]: Reached target Socket Units.1817lb # [ 1.763028] systemd[1]: Starting Create List of Static Device Nodes...1818lb # [ 1.768597] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1819worker1 # [ 1.770421] systemd[1]: Queued start job for default target Initrd Default Target.1820lb # [ 1.776771] systemd[1]: Starting Load Kernel Module configfs...1821worker1 # [ 1.774976] systemd[1]: Created slice Slice /system/modprobe.1822client # [ 1.750652] systemd[1]: Queued start job for default target Initrd Default Target.1823worker1 # [ 1.776492] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1824worker1 # [ 1.778216] systemd[1]: Expecting device /dev/disk/by-label/nixos...1825client # [ 1.754088] systemd[1]: Created slice Slice /system/modprobe.1826worker1 # [ 1.779593] systemd[1]: Reached target Path Units.1827client # [ 1.755279] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1828worker1 # [ 1.780681] systemd[1]: Reached target Slice Units.1829client # [ 1.756654] systemd[1]: Expecting device /dev/disk/by-label/nixos...1830worker1 # [ 1.781819] systemd[1]: Reached target Swaps.1831client # [ 1.757727] systemd[1]: Reached target Path Units.1832worker1 # [ 1.782890] systemd[1]: Reached target Timer Units.1833client # [ 1.758540] systemd[1]: Reached target Slice Units.1834client # [ 1.759396] systemd[1]: Reached target Swaps.1835worker1 # [ 1.784328] systemd[1]: Listening on D-Bus System Message Bus Socket.1836client # [ 1.760214] systemd[1]: Reached target Timer Units.1837worker1 # [ 1.785855] systemd[1]: Listening on Journal Socket (/dev/log).1838client # [ 1.761241] systemd[1]: Listening on D-Bus System Message Bus Socket.1839client # [ 1.762530] systemd[1]: Listening on Journal Socket (/dev/log).1840worker1 # [ 1.787409] systemd[1]: Listening on Journal Sockets.1841client # [ 1.763711] systemd[1]: Listening on Journal Sockets.1842worker1 # [ 1.788686] systemd[1]: Listening on udev Control Socket.1843client # [ 1.764710] systemd[1]: Listening on udev Control Socket.1844worker1 # [ 1.789999] systemd[1]: Listening on udev Kernel Socket.1845lb # [ 1.794609] systemd[1]: Starting Journal Service...1846client # [ 1.765739] systemd[1]: Listening on udev Kernel Socket.1847worker1 # [ 1.791230] systemd[1]: Reached target Socket Units.1848client # [ 1.766660] systemd[1]: Reached target Socket Units.1849client # [ 1.768903] systemd[1]: Starting Create List of Static Device Nodes...1850worker1 # [ 1.793982] systemd[1]: Starting Create List of Static Device Nodes...1851client # [ 1.774712] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1852worker1 # [ 1.800215] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1853worker2 # [ 1.799398] systemd[1]: Queued start job for default target Initrd Default Target.1854worker1 # [ 1.805663] systemd[1]: Starting Load Kernel Module configfs...1855client # [ 1.781720] systemd[1]: Starting Load Kernel Module configfs...1856worker2 # [ 1.803348] systemd[1]: Created slice Slice /system/modprobe.1857worker2 # [ 1.804584] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1858worker2 # [ 1.806028] systemd[1]: Expecting device /dev/disk/by-label/nixos...1859worker2 # [ 1.807176] systemd[1]: Reached target Path Units.1860worker2 # [ 1.808079] systemd[1]: Reached target Slice Units.1861worker2 # [ 1.808976] systemd[1]: Reached target Swaps.1862worker2 # [ 1.809842] systemd[1]: Reached target Timer Units.1863worker2 # [ 1.810898] systemd[1]: Listening on D-Bus System Message Bus Socket.1864worker2 # [ 1.812156] systemd[1]: Listening on Journal Socket (/dev/log).1865lb # [ 1.808069] systemd[1]: Starting Load Kernel Modules...1866worker2 # [ 1.813308] systemd[1]: Listening on Journal Sockets.1867worker2 # [ 1.814385] systemd[1]: Listening on udev Control Socket.1868worker2 # [ 1.815450] systemd[1]: Listening on udev Kernel Socket.1869worker2 # [ 1.816416] systemd[1]: Reached target Socket Units.1870worker2 # [ 1.818548] systemd[1]: Starting Create List of Static Device Nodes...1871worker1 # [ 1.825246] systemd[1]: Starting Journal Service...1872client # [ 1.800737] systemd[1]: Starting Journal Service...1873worker2 # [ 1.822919] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1874lb # [ 1.832747] netfs: FS-Cache loaded1875worker2 # [ 1.828427] systemd[1]: Starting Load Kernel Module configfs...1876lb # [ 1.838682] 9pnet: Installing 9P2000 support1877lb # [ 1.849659] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1878worker2 # [ 1.844993] systemd[1]: Starting Journal Service...1879worker1 # [ 1.839212] systemd[1]: Starting Load Kernel Modules...1880client # [ 1.814736] systemd[1]: Starting Load Kernel Modules...1881lb # [ 1.858760] systemd[1]: Starting Coldplug All udev Devices...1882client # [ 1.832084] netfs: FS-Cache loaded1883worker1 # [ 1.857520] netfs: FS-Cache loaded1884client # [ 1.837549] 9pnet: Installing 9P2000 support1885worker1 # [ 1.862530] 9pnet: Installing 9P2000 support1886client # [ 1.845864] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1887worker1 # [ 1.870785] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1888lb # [ 1.876746] systemd[1]: Finished Create List of Static Device Nodes.1889worker2 # [ 1.858280] systemd[1]: Starting Load Kernel Modules...1890lb # [ 1.881274] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1891worker1 # [ 1.878221] systemd[1]: Starting Coldplug All udev Devices...1892client # [ 1.856902] systemd[1]: Starting Coldplug All udev Devices...1893lb # [ 1.889952] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1894worker2 # [ 1.882229] netfs: FS-Cache loaded1895lb # [ 1.891337] systemd-journald[67]: Collecting audit messages is disabled.1896lb # [ 1.897011] systemd[1]: modprobe@configfs.service: Deactivated successfully.1897worker2 # [ 1.889672] 9pnet: Installing 9P2000 support1898worker1 # [ 1.894444] systemd[1]: Finished Create List of Static Device Nodes.1899client # [ 1.872148] systemd[1]: Finished Create List of Static Device Nodes.1900lb # [ 1.903559] systemd[1]: Finished Load Kernel Module configfs.1901worker1 # [ 1.900828] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1902lb # [ 1.906005] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1903client # [ 1.878551] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1904worker2 # [ 1.900124] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1905lb # [ 1.910056] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1906worker1 # [ 1.908603] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1907lb # [ 1.916537] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1908client # [ 1.888150] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1909client # [ 1.889762] systemd-journald[67]: Collecting audit messages is disabled.1910worker1 # [ 1.915616] systemd[1]: modprobe@configfs.service: Deactivated successfully.1911worker2 # [ 1.912156] systemd[1]: Starting Coldplug All udev Devices...1912lb # [ 1.921767] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1913worker1 # [ 1.917824] systemd-journald[67]: Collecting audit messages is disabled.1914client # [ 1.895164] systemd[1]: modprobe@configfs.service: Deactivated successfully.1915worker1 # [ 1.922543] systemd[1]: Finished Load Kernel Module configfs.1916client # [ 1.901465] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1917worker1 # [ 1.927201] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1918client # [ 1.904011] systemd[1]: Finished Load Kernel Module configfs.1919worker1 # [ 1.929853] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1920worker2 # [ 1.928905] systemd[1]: Finished Create List of Static Device Nodes.1921client # [ 1.908207] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1922worker1 # [ 1.936153] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1923client # [ 1.912854] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1924worker2 # [ 1.936212] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1925worker1 # [ 1.941380] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1926lb # [ 1.949108] systemd[1]: Finished Load Kernel Modules.1927client # [ 1.921377] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1928worker2 # [ 1.941667] systemd-journald[67]: Collecting audit messages is disabled.1929worker2 # [ 1.947326] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1930lb # [ 1.957726] systemd[1]: Starting Apply Kernel Variables...1931worker2 # [ 1.950548] systemd[1]: modprobe@configfs.service: Deactivated successfully.1932worker2 # [ 1.957289] systemd[1]: Finished Load Kernel Module configfs.1933worker2 # [ 1.961597] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1934worker2 # [ 1.964184] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1935worker1 # [ 1.969735] systemd[1]: Finished Load Kernel Modules.1936client # [ 1.945499] systemd[1]: Finished Load Kernel Modules.1937lb # [ 1.977609] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1938worker2 # [ 1.971034] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1939client # [ 1.953907] systemd[1]: Starting Apply Kernel Variables...1940worker1 # [ 1.979338] systemd[1]: Starting Apply Kernel Variables...1941worker2 # [ 1.976065] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1942lb # [ 1.987849] systemd[1]: Starting Create Static Device Nodes in /dev...1943worker1 # [ 1.995454] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1944client # [ 1.973011] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1945lb # [ 2.002598] systemd[1]: Finished Apply Kernel Variables.1946worker1 # [ 2.005464] systemd[1]: Starting Create Static Device Nodes in /dev...1947worker2 # [ 2.004426] systemd[1]: Finished Load Kernel Modules.1948client # [ 1.985186] systemd[1]: Starting Create Static Device Nodes in /dev...1949lb # [ 2.017521] systemd[1]: Started Journal Service.1950lb # [ 1.885226] systemd-modules-load[68]: Inserted module 'dm_mod'1951lb # [ 1.887498] systemd-modules-load[68]: Inserted module 'virtio_balloon'1952worker2 # [ 2.013055] systemd[1]: Starting Apply Kernel Variables...1953worker1 # [ 2.020256] systemd[1]: Finished Apply Kernel Variables.1954lb # [ 1.890312] systemd-modules-load[68]: Inserted module 'virtio_gpu'1955client # [ 1.998789] systemd[1]: Finished Apply Kernel Variables.1956lb # [ 1.895071] systemd[1]: Finished Create Static Device Nodes in /dev.1957lb # [ 1.900105] systemd[1]: Reached target Preparation for Local File Systems.1958worker2 # [ 2.026423] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1959lb # [ 1.903127] systemd[1]: Reached target Local File Systems.1960lb # [ 1.906068] systemd[1]: Starting Create System Files and Directories...1961worker1 # [ 2.036895] systemd[1]: Started Journal Service.1962worker1 # [ 1.900231] systemd-modules-load[68]: Inserted module 'dm_mod'1963worker2 # [ 2.036111] systemd[1]: Starting Create Static Device Nodes in /dev...1964lb # [ 1.910861] systemd[1]: Starting Rule-based Manager for Device Events and Files...1965worker1 # [ 1.902463] systemd-modules-load[68]: Inserted module 'virtio_balloon'1966client # [ 1.882397] systemd-modules-load[68]: Inserted module 'dm_mod'[ 2.018270] systemd[1]: Started Journal Service.1967client # 1968worker1 # [ 1.906306] systemd-modules-load[68]: Inserted module 'virtio_gpu'1969client # [ 1.886347] systemd-modules-load[68]: Inserted module 'virtio_balloon'1970client # [ 1.887413] systemd-modules-load[68]: Inserted module 'virtio_gpu'1971worker1 # [ 1.912961] systemd[1]: Finished Create Static Device Nodes in /dev.1972worker1 # [ 1.915967] systemd[1]: Reached target Preparation for Local File Systems.1973worker1 # [ 1.919071] systemd[1]: Reached target Local File Systems.1974client # [ 1.900141] systemd[1]: Finished Create Static Device Nodes in /dev.1975worker1 # [ 1.922071] systemd[1]: Starting Create System Files and Directories...1976worker2 # [ 2.058525] systemd[1]: Finished Apply Kernel Variables.1977client # [ 1.904095] systemd[1]: Reached target Preparation for Local File Systems.1978client # [ 1.905168] systemd[1]: Reached target Local File Systems.1979worker1 # [ 1.926718] systemd[1]: Starting Rule-based Manager for Device Events and Files...1980client # [ 1.907918] systemd[1]: Starting Create System Files and Directories...1981lb # [ 1.938212] systemd[1]: Finished Create System Files and Directories.1982worker2 # [ 2.068515] systemd[1]: Started Journal Service.1983client # [ 1.915145] systemd[1]: Starting Rule-based Manager for Device Events and Files...1984worker2 # [ 1.930238] systemd-modules-load[68]: Inserted module 'dm_mod'1985worker2 # [ 1.931703] systemd-modules-load[68]: Inserted module 'virtio_balloon'1986worker2 # [ 1.934338] systemd-modules-load[68]: Inserted module 'virtio_gpu'1987worker2 # [ 1.946137] systemd[1]: Finished Create Static Device Nodes in /dev.1988worker2 # [ 1.947286] systemd[1]: Reached target Preparation for Local File Systems.1989worker2 # [ 1.948389] systemd[1]: Reached target Local File Systems.1990lb # [ 1.965410] systemd-udevd[83]: Using default interface naming scheme 'v261'.1991worker2 # [ 1.952330] systemd[1]: Starting Create System Files and Directories...1992worker1 # [ 1.963548] systemd[1]: Finished Create System Files and Directories.1993worker2 # [ 1.958380] systemd[1]: Starting Rule-based Manager for Device Events and Files...1994client # [ 1.949719] systemd[1]: Finished Create System Files and Directories.1995lb # [ 1.991076] systemd[1]: Started Rule-based Manager for Device Events and Files.1996worker1 # [ 1.984286] systemd-udevd[83]: Using default interface naming scheme 'v261'.1997client # [ 1.972315] systemd-udevd[83]: Using default interface naming scheme 'v261'.1998worker2 # [ 1.991848] systemd[1]: Finished Create System Files and Directories.1999worker1 # [ 2.011171] systemd[1]: Started Rule-based Manager for Device Events and Files.2000client # [ 1.996160] systemd[1]: Started Rule-based Manager for Device Events and Files.
loading 2,046 hidden lines…
4047lb # [ 20.421198] envoy[1009]: [2026-09-20 12:20:36.916][1009][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor4048lb # [ 20.423597] envoy[1009]: [2026-09-20 12:20:36.916][1009][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection4049lb # [ 20.429153] envoy[1009]: [2026-09-20 12:20:36.916][1009][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata4050lb # [ 20.431216] envoy[1009]: [2026-09-20 12:20:36.916][1009][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy4051lb # [ 20.433676] envoy[1009]: [2026-09-20 12:20:36.916][1009][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest4052lb # [ 20.438298] envoy[1009]: [2026-09-20 12:20:36.916][1009][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4053lb # [ 20.440122] envoy[1009]: [2026-09-20 12:20:36.916][1009][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal4054lb # [ 20.441670] envoy[1009]: [2026-09-20 12:20:36.916][1009][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default4055lb # [ 20.443391] envoy[1009]: [2026-09-20 12:20:36.916][1009][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche4056lb # [ 20.445482] envoy[1009]: [2026-09-20 12:20:36.916][1009][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin4057lb # [ 20.448604] envoy[1009]: [2026-09-20 12:20:36.916][1009][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource4058lb # [ 20.451613] envoy[1009]: [2026-09-20 12:20:36.916][1009][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso4059lb # [ 20.453631] envoy[1009]: [2026-09-20 12:20:36.916][1009][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm4060lb # [ 20.459720] envoy[1009]: [2026-09-20 12:20:36.916][1009][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert4061lb # [ 20.461640] envoy[1009]: [2026-09-20 12:20:36.916][1009][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options4062lb # [ 20.465768] envoy[1009]: [2026-09-20 12:20:36.916][1009][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default4063lb # [ 20.467865] envoy[1009]: [2026-09-20 12:20:36.916][1009][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory4064lb # [ 20.470696] envoy[1009]: [2026-09-20 12:20:36.919][1009][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san4065lb # [ 20.477116] envoy[1009]: [2026-09-20 12:20:36.919][1009][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator4066lb # [ 20.479391] envoy[1009]: [2026-09-20 12:20:36.919][1009][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed4067lb # [ 20.481285] envoy[1009]: [2026-09-20 12:20:36.919][1009][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls4068lb # [ 20.485481] envoy[1009]: [2026-09-20 12:20:36.919][1009][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua4069lb # [ 20.487242] envoy[1009]: [2026-09-20 12:20:36.919][1009][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation4070lb # [ 20.489447] envoy[1009]: [2026-09-20 12:20:36.919][1009][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic4071lb # [ 20.491248] envoy[1009]: [2026-09-20 12:20:36.919][1009][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default4072lb # [ 20.492918] envoy[1009]: [2026-09-20 12:20:36.919][1009][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher4073lb # [ 20.494897] envoy[1009]: [2026-09-20 12:20:36.919][1009][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4074lb # [ 20.498114] envoy[1009]: [2026-09-20 12:20:36.919][1009][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls4075lb # [ 20.503387] envoy[1009]: [2026-09-20 12:20:36.919][1009][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v84076lb # [ 20.505089] envoy[1009]: [2026-09-20 12:20:36.919][1009][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value4077lb # [ 20.508718] envoy[1009]: [2026-09-20 12:20:36.919][1009][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router4078lb # [ 20.535170] envoy[1009]: [2026-09-20 12:20:36.919][1009][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory4079lb # [ 20.538648] envoy[1009]: [2026-09-20 12:20:36.919][1009][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy4080lb # [ 20.541157] envoy[1009]: [2026-09-20 12:20:36.919][1009][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy4081lb # [ 20.552180] envoy[1009]: [2026-09-20 12:20:36.919][1009][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re24082lb # [ 20.553761] envoy[1009]: [2026-09-20 12:20:36.919][1009][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default4083lb # [ 20.555310] envoy[1009]: [2026-09-20 12:20:36.919][1009][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle4084lb # [ 20.557456] envoy[1009]: [2026-09-20 12:20:36.919][1009][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name4085lb # [ 20.560147] envoy[1009]: [2026-09-20 12:20:36.919][1009][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed4086lb # [ 20.562705] envoy[1009]: [2026-09-20 12:20:36.919][1009][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http14087lb # [ 20.564855] envoy[1009]: [2026-09-20 12:20:36.919][1009][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind4088lb # [ 20.566751] envoy[1009]: [2026-09-20 12:20:36.919][1009][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action4089lb # [ 20.568689] envoy[1009]: [2026-09-20 12:20:36.919][1009][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override4090lb # [ 20.571192] envoy[1009]: [2026-09-20 12:20:36.919][1009][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift4091lb # [ 20.574671] envoy[1009]: [2026-09-20 12:20:36.920][1009][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd4092lb # [ 20.579659] envoy[1009]: [2026-09-20 12:20:36.920][1009][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth24093lb # [ 20.581954] envoy[1009]: [2026-09-20 12:20:36.920][1009][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name4094lb # [ 20.584123] envoy[1009]: [2026-09-20 12:20:36.920][1009][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter4095lb # [ 20.586246] envoy[1009]: [2026-09-20 12:20:36.920][1009][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities4096lb # [ 20.588225] envoy[1009]: [2026-09-20 12:20:36.920][1009][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff4097lb # [ 20.590668] envoy[1009]: [2026-09-20 12:20:36.933][1009][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override4098lb # [ 20.593506] envoy[1009]: [2026-09-20 12:20:36.933][1009][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap4099lb # [ 20.598769] envoy[1009]: [2026-09-20 12:20:36.933][1009][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router4100lb # [ 20.601474] envoy[1009]: [2026-09-20 12:20:36.933][1009][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector4101lb # [ 20.607139] envoy[1009]: [2026-09-20 12:20:36.933][1009][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router4102lb # [ 20.609257] envoy[1009]: [2026-09-20 12:20:36.933][1009][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case4103lb # [ 20.611641] envoy[1009]: [2026-09-20 12:20:36.935][1009][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo4104lb # [ 20.613144] envoy[1009]: [2026-09-20 12:20:37.087][1009][info][main] [source/server/server.cc:531] HTTP header map info:4105lb # [ 20.614931] envoy[1009]: [2026-09-20 12:20:37.160][1009][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id4106lb # [ 20.627975] envoy[1009]: [2026-09-20 12:20:37.160][1009][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:4107lb # [ 20.629592] envoy[1009]: [2026-09-20 12:20:37.160][1009][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id4108lb # [ 20.637316] envoy[1009]: [2026-09-20 12:20:37.160][1009][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status4109lb # [ 20.684495] envoy[1009]: [2026-09-20 12:20:37.310][1009][info][main] [source/server/server.cc:968] runtime: {}4110lb # [ 20.692312] envoy[1009]: [2026-09-20 12:20:37.318][1009][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.4111lb # [ 20.695641] envoy[1009]: [2026-09-20 12:20:37.321][1009][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99014112lb # [ 20.698160] envoy[1009]: [2026-09-20 12:20:37.324][1009][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99014113lb # [ 20.701557] envoy[1009]: [2026-09-20 12:20:37.327][1009][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration4114lb # [ 20.703937] envoy[1009]: [2026-09-20 12:20:37.329][1009][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)4115lb # [ 20.705536] envoy[1009]: [2026-09-20 12:20:37.329][1009][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)4116lb # [ 20.759397] envoy[1009]: [2026-09-20 12:20:37.385][1009][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)4117worker1 # [ 20.752928] systemd[1]: Starting Nix worker-protocol over gRPC...4118worker2 # [ 20.749760] systemd[1]: Reached target Network is Online.4119worker2 # [ 20.758472] systemd[1]: Starting scheduler-token.service...4120lb # [ 20.783986] envoy[1009]: [2026-09-20 12:20:37.409][1009][info][config] [source/server/configuration_impl.cc:164] loading stats configuration4121lb # [ 20.786684] envoy[1009]: [2026-09-20 12:20:37.412][1009][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.4122lb # [ 20.789820] envoy[1009]: [2026-09-20 12:20:37.412][1009][info][main] [source/server/server.cc:1077] starting main dispatch loop4123worker2 # [ 20.877328] systemd[1]: scheduler-token.service: Deactivated successfully.4124worker2 # [ 20.880410] systemd[1]: Finished scheduler-token.service.4125worker2 # [ 20.881633] systemd[1]: scheduler-token.service: Consumed 44ms CPU time over 121ms wall clock time, 3.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4126worker2 # [ 20.887572] systemd[1]: Starting Nix worker-protocol over gRPC...4127worker1 # [ 20.950612] nix-grpc-daemon[852]: ts=2026-09-20T12:20:37Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4128worker1 # [ 20.961226] nix-grpc-daemon[852]: ts=2026-09-20T12:20:37Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4129worker1 # [ 20.967183] nix-grpc-daemon[852]: ts=2026-09-20T12:20:37Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14130worker1 # [ 21.012392] nix-grpc-daemon[852]: ts=2026-09-20T12:20:37Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4131worker1 # [ 21.017477] systemd[1]: Started Nix worker-protocol over gRPC.4132lb # [ 21.078498] envoy[1009]: [2026-09-20 12:20:37.704][1009][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization4133worker1 # [ 21.069664] nix-grpc-daemon[852]: ts=2026-09-20T12:20:37Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4134worker2 # [ 21.066616] nix-grpc-daemon[855]: ts=2026-09-20T12:20:37Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4135lb # [ 21.080908] envoy[1009]: [2026-09-20 12:20:37.704][1009][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized4136lb # [ 21.084426] envoy[1009]: [2026-09-20 12:20:37.704][1009][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager4137worker2 # [ 21.075777] nix-grpc-daemon[855]: ts=2026-09-20T12:20:37Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4138lb # [ 21.089552] envoy[1009]: [2026-09-20 12:20:37.706][1009][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers4139worker2 # [ 21.081159] nix-grpc-daemon[855]: ts=2026-09-20T12:20:37Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14140worker2 # [ 21.127174] nix-grpc-daemon[855]: ts=2026-09-20T12:20:37Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4141worker2 # [ 21.131428] systemd[1]: Started Nix worker-protocol over gRPC.4142worker2 # [ 21.187043] nix-grpc-daemon[855]: ts=2026-09-20T12:20:37Z level=info event=scheduler_connected addr=lb:500514143worker2 # [ 21.201195] nix-grpc-daemon[855]: ts=2026-09-20T12:20:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4144worker1 # [ 21.273858] nix-grpc-daemon[852]: ts=2026-09-20T12:20:37Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4145worker2 # [ 21.403963] nix-grpc-daemon[855]: ts=2026-09-20T12:20:37Z level=info event=scheduler_connected addr=lb:500514146worker2 # [ 21.408712] nix-grpc-daemon[855]: ts=2026-09-20T12:20:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4147worker2 # [ 21.609604] nix-grpc-daemon[855]: ts=2026-09-20T12:20:38Z level=info event=scheduler_connected addr=lb:500514148worker2 # [ 21.611963] nix-grpc-daemon[855]: ts=2026-09-20T12:20:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4149worker1 # [ 21.674242] nix-grpc-daemon[852]: ts=2026-09-20T12:20:38Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4150worker2 # [ 21.813660] nix-grpc-daemon[855]: ts=2026-09-20T12:20:38Z level=info event=scheduler_connected addr=lb:500514151worker2 # [ 21.817804] nix-grpc-daemon[855]: ts=2026-09-20T12:20:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4152worker2 # [ 22.019486] nix-grpc-daemon[855]: ts=2026-09-20T12:20:38Z level=info event=scheduler_connected addr=lb:500514153worker2 # [ 22.023478] nix-grpc-daemon[855]: ts=2026-09-20T12:20:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4154worker2 # [ 22.225721] nix-grpc-daemon[855]: ts=2026-09-20T12:20:38Z level=info event=scheduler_connected addr=lb:500514155worker2 # [ 22.227867] nix-grpc-daemon[855]: ts=2026-09-20T12:20:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4156worker2 # [ 22.429270] nix-grpc-daemon[855]: ts=2026-09-20T12:20:38Z level=info event=scheduler_connected addr=lb:500514157worker2 # [ 22.431862] nix-grpc-daemon[855]: ts=2026-09-20T12:20:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4158worker1 # [ 22.477336] nix-grpc-daemon[852]: ts=2026-09-20T12:20:38Z level=info event=scheduler_connected addr=lb:500514159worker1 # [ 22.480093] nix-grpc-daemon[852]: ts=2026-09-20T12:20:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4160worker2 # [ 22.633465] nix-grpc-daemon[855]: ts=2026-09-20T12:20:39Z level=info event=scheduler_connected addr=lb:500514161worker2 # [ 22.637736] nix-grpc-daemon[855]: ts=2026-09-20T12:20:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4162worker1 # [ 22.681507] nix-grpc-daemon[852]: ts=2026-09-20T12:20:39Z level=info event=scheduler_connected addr=lb:500514163worker1 # [ 22.685760] nix-grpc-daemon[852]: ts=2026-09-20T12:20:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4164worker2 # [ 22.839186] nix-grpc-daemon[855]: ts=2026-09-20T12:20:39Z level=info event=scheduler_connected addr=lb:500514165worker2 # [ 22.843537] nix-grpc-daemon[855]: ts=2026-09-20T12:20:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4166worker1 # [ 22.886476] nix-grpc-daemon[852]: ts=2026-09-20T12:20:39Z level=info event=scheduler_connected addr=lb:500514167worker1 # [ 22.888597] nix-grpc-daemon[852]: ts=2026-09-20T12:20:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4168worker2 # [ 23.044881] nix-grpc-daemon[855]: ts=2026-09-20T12:20:39Z level=info event=scheduler_connected addr=lb:500514169worker2 # [ 23.047547] nix-grpc-daemon[855]: ts=2026-09-20T12:20:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4170worker1 # [ 23.088875] nix-grpc-daemon[852]: ts=2026-09-20T12:20:39Z level=info event=scheduler_connected addr=lb:500514171worker1 # [ 23.090763] nix-grpc-daemon[852]: ts=2026-09-20T12:20:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4172worker2 # [ 23.249531] nix-grpc-daemon[855]: ts=2026-09-20T12:20:39Z level=info event=scheduler_connected addr=lb:500514173worker2 # [ 23.252120] nix-grpc-daemon[855]: ts=2026-09-20T12:20:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4174worker1 # [ 23.291454] nix-grpc-daemon[852]: ts=2026-09-20T12:20:39Z level=info event=scheduler_connected addr=lb:500514175worker1 # [ 23.293465] nix-grpc-daemon[852]: ts=2026-09-20T12:20:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4176worker2 # [ 23.453521] nix-grpc-daemon[855]: ts=2026-09-20T12:20:39Z level=info event=scheduler_connected addr=lb:500514177worker2 # [ 23.456152] nix-grpc-daemon[855]: ts=2026-09-20T12:20:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4178worker1 # [ 23.496588] nix-grpc-daemon[852]: ts=2026-09-20T12:20:39Z level=info event=scheduler_connected addr=lb:500514179worker1 # [ 23.500695] nix-grpc-daemon[852]: ts=2026-09-20T12:20:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4180worker2 # [ 23.657517] nix-grpc-daemon[855]: ts=2026-09-20T12:20:40Z level=info event=scheduler_connected addr=lb:500514181worker2 # [ 23.660892] nix-grpc-daemon[855]: ts=2026-09-20T12:20:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4182worker1 # [ 23.697174] nix-grpc-daemon[852]: ts=2026-09-20T12:20:40Z level=info event=scheduler_connected addr=lb:500514183worker1 # [ 23.699364] nix-grpc-daemon[852]: ts=2026-09-20T12:20:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4184worker2 # [ 23.862191] nix-grpc-daemon[855]: ts=2026-09-20T12:20:40Z level=info event=scheduler_connected addr=lb:500514185worker2 # [ 23.864142] nix-grpc-daemon[855]: ts=2026-09-20T12:20:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4186worker1 # [ 23.900621] nix-grpc-daemon[852]: ts=2026-09-20T12:20:40Z level=info event=scheduler_connected addr=lb:500514187worker1 # [ 23.904891] nix-grpc-daemon[852]: ts=2026-09-20T12:20:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4188worker2 # [ 24.064932] nix-grpc-daemon[855]: ts=2026-09-20T12:20:40Z level=info event=scheduler_connected addr=lb:500514189worker2 # [ 24.067338] nix-grpc-daemon[855]: ts=2026-09-20T12:20:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4190worker1 # [ 24.105801] nix-grpc-daemon[852]: ts=2026-09-20T12:20:40Z level=info event=scheduler_connected addr=lb:500514191worker1 # [ 24.109837] nix-grpc-daemon[852]: ts=2026-09-20T12:20:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4192worker2 # [ 24.269751] nix-grpc-daemon[855]: ts=2026-09-20T12:20:40Z level=info event=scheduler_connected addr=lb:500514193worker2 # [ 24.273516] nix-grpc-daemon[855]: ts=2026-09-20T12:20:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4194worker1 # [ 24.310445] nix-grpc-daemon[852]: ts=2026-09-20T12:20:40Z level=info event=scheduler_connected addr=lb:500514195worker1 # [ 24.312662] nix-grpc-daemon[852]: ts=2026-09-20T12:20:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4196worker2 # [ 24.474540] nix-grpc-daemon[855]: ts=2026-09-20T12:20:40Z level=info event=scheduler_connected addr=lb:500514197worker2 # [ 24.476568] nix-grpc-daemon[855]: ts=2026-09-20T12:20:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4198worker1 # [ 24.515756] nix-grpc-daemon[852]: ts=2026-09-20T12:20:41Z level=info event=scheduler_connected addr=lb:500514199worker1 # [ 24.519187] nix-grpc-daemon[852]: ts=2026-09-20T12:20:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4200worker2 # [ 24.678100] nix-grpc-daemon[855]: ts=2026-09-20T12:20:41Z level=info event=scheduler_connected addr=lb:500514201worker2 # [ 24.682358] nix-grpc-daemon[855]: ts=2026-09-20T12:20:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4202worker1 # [ 24.716193] nix-grpc-daemon[852]: ts=2026-09-20T12:20:41Z level=info event=scheduler_connected addr=lb:500514203worker1 # [ 24.718363] nix-grpc-daemon[852]: ts=2026-09-20T12:20:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4204worker2 # [ 24.883552] nix-grpc-daemon[855]: ts=2026-09-20T12:20:41Z level=info event=scheduler_connected addr=lb:500514205worker2 # [ 24.885528] nix-grpc-daemon[855]: ts=2026-09-20T12:20:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4206worker1 # [ 24.919310] nix-grpc-daemon[852]: ts=2026-09-20T12:20:41Z level=info event=scheduler_connected addr=lb:500514207worker1 # [ 24.921635] nix-grpc-daemon[852]: ts=2026-09-20T12:20:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4208worker2 # [ 25.086697] nix-grpc-daemon[855]: ts=2026-09-20T12:20:41Z level=info event=scheduler_connected addr=lb:500514209worker2 # [ 25.088799] nix-grpc-daemon[855]: ts=2026-09-20T12:20:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4210worker1 # [ 25.122341] nix-grpc-daemon[852]: ts=2026-09-20T12:20:41Z level=info event=scheduler_connected addr=lb:500514211worker1 # [ 25.124586] nix-grpc-daemon[852]: ts=2026-09-20T12:20:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4212worker2 # [ 25.291123] nix-grpc-daemon[855]: ts=2026-09-20T12:20:41Z level=info event=scheduler_connected addr=lb:500514213worker2 # [ 25.292876] nix-grpc-daemon[855]: ts=2026-09-20T12:20:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4214worker1 # [ 25.325518] nix-grpc-daemon[852]: ts=2026-09-20T12:20:41Z level=info event=scheduler_connected addr=lb:500514215worker1 # [ 25.328028] nix-grpc-daemon[852]: ts=2026-09-20T12:20:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4216worker2 # [ 25.492375] nix-grpc-daemon[855]: ts=2026-09-20T12:20:41Z level=info event=scheduler_connected addr=lb:500514217worker2 # [ 25.494663] nix-grpc-daemon[855]: ts=2026-09-20T12:20:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4218worker1 # [ 25.530762] nix-grpc-daemon[852]: ts=2026-09-20T12:20:42Z level=info event=scheduler_connected addr=lb:500514219worker1 # [ 25.534394] nix-grpc-daemon[852]: ts=2026-09-20T12:20:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4220worker2 # [ 25.695763] nix-grpc-daemon[855]: ts=2026-09-20T12:20:42Z level=info event=scheduler_connected addr=lb:500514221worker2 # [ 25.698168] nix-grpc-daemon[855]: ts=2026-09-20T12:20:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4222worker1 # [ 25.731384] nix-grpc-daemon[852]: ts=2026-09-20T12:20:42Z level=info event=scheduler_connected addr=lb:500514223worker1 # [ 25.734155] nix-grpc-daemon[852]: ts=2026-09-20T12:20:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4224worker2 # [ 25.899688] nix-grpc-daemon[855]: ts=2026-09-20T12:20:42Z level=info event=scheduler_connected addr=lb:500514225worker2 # [ 25.902363] nix-grpc-daemon[855]: ts=2026-09-20T12:20:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4226worker1 # [ 25.934800] nix-grpc-daemon[852]: ts=2026-09-20T12:20:42Z level=info event=scheduler_connected addr=lb:500514227worker1 # [ 25.937277] nix-grpc-daemon[852]: ts=2026-09-20T12:20:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4228worker2 # [ 26.103787] nix-grpc-daemon[855]: ts=2026-09-20T12:20:42Z level=info event=scheduler_connected addr=lb:500514229worker2 # [ 26.108737] nix-grpc-daemon[855]: ts=2026-09-20T12:20:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4230worker1 # [ 26.138315] nix-grpc-daemon[852]: ts=2026-09-20T12:20:42Z level=info event=scheduler_connected addr=lb:500514231worker1 # [ 26.140788] nix-grpc-daemon[852]: ts=2026-09-20T12:20:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4232worker2 # [ 26.310182] nix-grpc-daemon[855]: ts=2026-09-20T12:20:42Z level=info event=scheduler_connected addr=lb:500514233worker2 # [ 26.312223] nix-grpc-daemon[855]: ts=2026-09-20T12:20:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4234worker1 # [ 26.341638] nix-grpc-daemon[852]: ts=2026-09-20T12:20:42Z level=info event=scheduler_connected addr=lb:500514235worker1 # [ 26.344159] nix-grpc-daemon[852]: ts=2026-09-20T12:20:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4236worker2 # [ 26.511710] nix-grpc-daemon[855]: ts=2026-09-20T12:20:42Z level=info event=scheduler_connected addr=lb:500514237worker2 # [ 26.513767] nix-grpc-daemon[855]: ts=2026-09-20T12:20:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4238worker1 # [ 26.546665] nix-grpc-daemon[852]: ts=2026-09-20T12:20:43Z level=info event=scheduler_connected addr=lb:500514239worker1 # [ 26.548618] nix-grpc-daemon[852]: ts=2026-09-20T12:20:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4240worker2 # [ 26.715169] nix-grpc-daemon[855]: ts=2026-09-20T12:20:43Z level=info event=scheduler_connected addr=lb:500514241worker2 # [ 26.717457] nix-grpc-daemon[855]: ts=2026-09-20T12:20:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4242worker1 # [ 26.747059] nix-grpc-daemon[852]: ts=2026-09-20T12:20:43Z level=info event=scheduler_connected addr=lb:500514243worker1 # [ 26.749454] nix-grpc-daemon[852]: ts=2026-09-20T12:20:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4244worker2 # [ 26.918445] nix-grpc-daemon[855]: ts=2026-09-20T12:20:43Z level=info event=scheduler_connected addr=lb:500514245worker2 # [ 26.920711] nix-grpc-daemon[855]: ts=2026-09-20T12:20:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4246worker1 # [ 26.950347] nix-grpc-daemon[852]: ts=2026-09-20T12:20:43Z level=info event=scheduler_connected addr=lb:500514247worker1 # [ 26.952721] nix-grpc-daemon[852]: ts=2026-09-20T12:20:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4248worker2 # [ 27.121872] nix-grpc-daemon[855]: ts=2026-09-20T12:20:43Z level=info event=scheduler_connected addr=lb:500514249worker2 # [ 27.123987] nix-grpc-daemon[855]: ts=2026-09-20T12:20:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4250worker1 # [ 27.153350] nix-grpc-daemon[852]: ts=2026-09-20T12:20:43Z level=info event=scheduler_connected addr=lb:500514251worker1 # [ 27.155544] nix-grpc-daemon[852]: ts=2026-09-20T12:20:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4252worker2 # [ 27.326369] nix-grpc-daemon[855]: ts=2026-09-20T12:20:43Z level=info event=scheduler_connected addr=lb:500514253worker2 # [ 27.329201] nix-grpc-daemon[855]: ts=2026-09-20T12:20:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4254worker1 # [ 27.356516] nix-grpc-daemon[852]: ts=2026-09-20T12:20:43Z level=info event=scheduler_connected addr=lb:500514255worker1 # [ 27.358881] nix-grpc-daemon[852]: ts=2026-09-20T12:20:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4256worker2 # [ 27.529376] nix-grpc-daemon[855]: ts=2026-09-20T12:20:44Z level=info event=scheduler_connected addr=lb:500514257worker2 # [ 27.531441] nix-grpc-daemon[855]: ts=2026-09-20T12:20:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4258worker1 # [ 27.560354] nix-grpc-daemon[852]: ts=2026-09-20T12:20:44Z level=info event=scheduler_connected addr=lb:500514259worker1 # [ 27.562440] nix-grpc-daemon[852]: ts=2026-09-20T12:20:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4260worker2 # [ 27.732358] nix-grpc-daemon[855]: ts=2026-09-20T12:20:44Z level=info event=scheduler_connected addr=lb:500514261worker2 # [ 27.734577] nix-grpc-daemon[855]: ts=2026-09-20T12:20:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4262worker1 # [ 27.763412] nix-grpc-daemon[852]: ts=2026-09-20T12:20:44Z level=info event=scheduler_connected addr=lb:500514263worker1 # [ 27.765825] nix-grpc-daemon[852]: ts=2026-09-20T12:20:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4264worker2 # [ 27.936240] nix-grpc-daemon[855]: ts=2026-09-20T12:20:44Z level=info event=scheduler_connected addr=lb:500514265worker2 # [ 27.940655] nix-grpc-daemon[855]: ts=2026-09-20T12:20:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4266worker1 # [ 27.966626] nix-grpc-daemon[852]: ts=2026-09-20T12:20:44Z level=info event=scheduler_connected addr=lb:500514267worker1 # [ 27.968811] nix-grpc-daemon[852]: ts=2026-09-20T12:20:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4268worker2 # [ 28.141825] nix-grpc-daemon[855]: ts=2026-09-20T12:20:44Z level=info event=scheduler_connected addr=lb:500514269worker2 # [ 28.145749] nix-grpc-daemon[855]: ts=2026-09-20T12:20:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4270worker1 # [ 28.169482] nix-grpc-daemon[852]: ts=2026-09-20T12:20:44Z level=info event=scheduler_connected addr=lb:500514271worker1 # [ 28.171610] nix-grpc-daemon[852]: ts=2026-09-20T12:20:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4272worker2 # [ 28.348897] nix-grpc-daemon[855]: ts=2026-09-20T12:20:44Z level=info event=scheduler_connected addr=lb:500514273worker2 # [ 28.352714] nix-grpc-daemon[855]: ts=2026-09-20T12:20:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4274worker1 # [ 28.372179] nix-grpc-daemon[852]: ts=2026-09-20T12:20:44Z level=info event=scheduler_connected addr=lb:500514275worker1 # [ 28.374058] nix-grpc-daemon[852]: ts=2026-09-20T12:20:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4276worker2 # [ 28.553526] nix-grpc-daemon[855]: ts=2026-09-20T12:20:45Z level=info event=scheduler_connected addr=lb:500514277worker2 # [ 28.555667] nix-grpc-daemon[855]: ts=2026-09-20T12:20:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4278worker1 # [ 28.576749] nix-grpc-daemon[852]: ts=2026-09-20T12:20:45Z level=info event=scheduler_connected addr=lb:500514279worker1 # [ 28.579963] nix-grpc-daemon[852]: ts=2026-09-20T12:20:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4280worker2 # [ 28.756629] nix-grpc-daemon[855]: ts=2026-09-20T12:20:45Z level=info event=scheduler_connected addr=lb:500514281worker2 # [ 28.759370] nix-grpc-daemon[855]: ts=2026-09-20T12:20:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4282worker1 # [ 28.776887] nix-grpc-daemon[852]: ts=2026-09-20T12:20:45Z level=info event=scheduler_connected addr=lb:500514283worker1 # [ 28.778742] nix-grpc-daemon[852]: ts=2026-09-20T12:20:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4284lb # [ 28.897699] rustfs-bucket-start[1101]: mb s3://farm4285lb # [ 28.903919] systemd[1]: Finished rustfs-bucket.service.4286lb # [ 28.911590] systemd[1]: Starting niks3 server...4287worker2 # [ 28.960214] nix-grpc-daemon[855]: ts=2026-09-20T12:20:45Z level=info event=scheduler_connected addr=lb:500514288worker2 # [ 28.962642] nix-grpc-daemon[855]: ts=2026-09-20T12:20:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4289worker1 # [ 28.979428] nix-grpc-daemon[852]: ts=2026-09-20T12:20:45Z level=info event=scheduler_connected addr=lb:500514290worker1 # [ 28.981366] nix-grpc-daemon[852]: ts=2026-09-20T12:20:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4291lb # [ 29.152519] postgres[1118]: [1118] ERROR: relation "goose_db_version" does not exist at character 364292lb # [ 29.154273] postgres[1118]: [1118] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4293worker2 # [ 29.163397] nix-grpc-daemon[855]: ts=2026-09-20T12:20:45Z level=info event=scheduler_connected addr=lb:500514294worker2 # [ 29.165304] nix-grpc-daemon[855]: ts=2026-09-20T12:20:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4295worker1 # [ 29.181772] nix-grpc-daemon[852]: ts=2026-09-20T12:20:45Z level=info event=scheduler_connected addr=lb:500514296worker1 # [ 29.183671] nix-grpc-daemon[852]: ts=2026-09-20T12:20:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4297lb # [ 29.204187] niks3-server[1111]: 2026/09/20 12:20:45 OK 20241026095416_initial_model.sql (27.81ms)4298lb # [ 29.208495] niks3-server[1111]: 2026/09/20 12:20:45 OK 20251210153512_drop_unused_gin_index.sql (4.76ms)4299lb # [ 29.215086] niks3-server[1111]: 2026/09/20 12:20:45 OK 20251218171726_add_pins.sql (4.86ms)4300lb # [ 29.223586] niks3-server[1111]: 2026/09/20 12:20:45 OK 20260628120000_add_object_size_and_stats.sql (7.07ms)4301lb # [ 29.230785] niks3-server[1111]: 2026/09/20 12:20:45 OK 20260905000000_add_claims.sql (5.57ms)4302lb # [ 29.236758] niks3-server[1111]: 2026/09/20 12:20:45 OK 20260920000000_drop_claims.sql (5.84ms)4303lb # [ 29.239400] niks3-server[1111]: 2026/09/20 12:20:45 goose: successfully migrated database to version: 202609200000004304lb # [ 29.245246] niks3-server[1111]: 2026/09/20 12:20:45 OK 1_commit_pending_closure.sql (8.47ms)4305lb # [ 29.248249] niks3-server[1111]: 2026/09/20 12:20:45 OK 2_object_stats_trigger.sql (2.99ms)4306lb # [ 29.249377] niks3-server[1111]: 2026/09/20 12:20:45 goose: up to current file version: 24307lb # [ 29.257579] niks3-server[1111]: 2026/09/20 12:20:45 INFO Loaded signing key name=farm-test-1 path=/nix/store/c745bsy529madd2s6hz4z5is2436ni04-key4308lb # [ 29.344738] niks3-server[1111]: 2026/09/20 12:20:45 INFO Read proxy enabled — serving cache objects from S34309lb # [ 29.346803] niks3-server[1111]: 2026/09/20 12:20:45 INFO Using socket-activated listener address=0.0.0.0:57514310lb # [ 29.352618] systemd[1]: Started niks3 server.4311lb # [ 29.353516] niks3-server[1111]: 2026/09/20 12:20:45 INFO systemd watchdog enabled interval=15s4312lb # [ 29.355638] niks3-server[1111]: 2026/09/20 12:20:45 INFO Starting HTTP server address=0.0.0.0:57514313lb # [ 29.358187] systemd[1]: Reached target Multi-User System.4314lb # [ 29.360058] systemd[1]: Startup finished in 929ms (kernel) + 4.993s (initrd) + 23.437s (userspace) = 29.359s.4315lb # [ 29.365072] niks3-server[1111]: 2026/09/20 12:20:45 INFO lead: acquired remote=192.168.1.4:598984316worker2 # [ 29.358102] nix-grpc-daemon[855]: ts=2026-09-20T12:20:45Z level=info event=scheduler_take_over4317worker2 # [ 29.366586] nix-grpc-daemon[855]: ts=2026-09-20T12:20:45Z level=info event=scheduler_connected addr=lb:500514318worker2 # [ 29.371202] nix-grpc-daemon[855]: ts=2026-09-20T12:20:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4319worker1 # [ 29.384669] nix-grpc-daemon[852]: ts=2026-09-20T12:20:45Z level=info event=scheduler_connected addr=lb:500514320worker1 # [ 29.387157] nix-grpc-daemon[852]: ts=2026-09-20T12:20:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4321worker2 # [ 29.572358] nix-grpc-daemon[855]: ts=2026-09-20T12:20:46Z level=info event=scheduler_connected addr=lb:500514322worker2 # [ 29.576762] nix-grpc-daemon[855]: ts=2026-09-20T12:20:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4323worker1 # [ 29.589072] nix-grpc-daemon[852]: ts=2026-09-20T12:20:46Z level=info event=scheduler_connected addr=lb:500514324worker1 # [ 29.590684] nix-grpc-daemon[852]: ts=2026-09-20T12:20:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4325worker2 # [ 29.777811] nix-grpc-daemon[855]: ts=2026-09-20T12:20:46Z level=info event=scheduler_connected addr=lb:500514326worker2 # [ 29.780198] nix-grpc-daemon[855]: ts=2026-09-20T12:20:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4327worker1 # [ 29.789741] nix-grpc-daemon[852]: ts=2026-09-20T12:20:46Z level=info event=scheduler_connected addr=lb:500514328worker1 # [ 29.791911] nix-grpc-daemon[852]: ts=2026-09-20T12:20:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4329worker2 # [ 29.981083] nix-grpc-daemon[855]: ts=2026-09-20T12:20:46Z level=info event=scheduler_connected addr=lb:500514330worker2 # [ 29.982926] nix-grpc-daemon[855]: ts=2026-09-20T12:20:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4331worker1 # [ 29.993349] nix-grpc-daemon[852]: ts=2026-09-20T12:20:46Z level=info event=scheduler_connected addr=lb:500514332worker1 # [ 29.995284] nix-grpc-daemon[852]: ts=2026-09-20T12:20:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4333worker2 # [ 30.183775] nix-grpc-daemon[855]: ts=2026-09-20T12:20:46Z level=info event=scheduler_connected addr=lb:500514334worker2 # [ 30.186718] nix-grpc-daemon[855]: ts=2026-09-20T12:20:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4335worker1 # [ 30.194303] nix-grpc-daemon[852]: ts=2026-09-20T12:20:46Z level=info event=scheduler_connected addr=lb:500514336worker2 # [ 30.196727] nix-grpc-daemon[855]: ts=2026-09-20T12:20:46Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:558384337worker2 # [ 30.199139] nix-grpc-daemon[855]: ts=2026-09-20T12:20:46Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04338lb: (finished: waiting for unit niks3.service, in 18.70 seconds)4339lb: waiting for TCP port 5751 on localhost4340worker2 # [ 30.389876] nix-grpc-daemon[855]: ts=2026-09-20T12:20:46Z level=info event=scheduler_connected addr=lb:500514341lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!4342lb: (finished: waiting for TCP port 5751 on localhost, in 0.07 seconds)4343lb: waiting for unit envoy.service4344worker2 # [ 30.411669] nix-grpc-daemon[855]: ts=2026-09-20T12:20:46Z level=info event=oidc_jwks provider=mock keys=14345worker2 # [ 30.416425] nix-grpc-daemon[855]: ts=2026-09-20T12:20:46Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:558384346worker2 # [ 30.420527] nix-grpc-daemon[855]: ts=2026-09-20T12:20:46Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04347lb: (finished: waiting for unit envoy.service, in 0.05 seconds)4348lb: waiting for TCP port 50051 on localhost4349lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!4350lb # [ 30.493987] envoy[1009]: conn peer=[::1]:53770 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=24351lb: (finished: waiting for TCP port 50051 on localhost, in 0.03 seconds)4352worker2 # [ 30.579467] systemd[1]: Starting scheduler-token.service...4353worker2 # [ 30.643593] systemd[1]: scheduler-token.service: Deactivated successfully.4354worker2 # [ 30.645465] systemd[1]: Finished scheduler-token.service.4355worker2 # [ 30.647892] systemd[1]: scheduler-token.service: Consumed 23ms CPU time over 66ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4356lb: waiting until exactly no endpoints of x86_64-linux are unhealthy4357lb: must succeed: curl -sf localhost:9901/clusters4358lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4359lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.06 seconds)4360subtest: one node schedules and both builders hold a WorkerSession on it4361lb: must succeed: curl -sf localhost:9901/clusters4362lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4363worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4364worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4365lb: must succeed: curl -sf localhost:9901/clusters4366lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4367lb: must succeed: curl -sf localhost:9901/clusters4368lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4369worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'4370worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.04 seconds)4371(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.23 seconds)4372subtest: balancer auth: client cert, bearer token, nothing, unknown CN4373worker2 # [ 31.248116] nix-grpc-daemon[855]: ts=2026-09-20T12:20:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:558424374worker2 # [ 31.255354] systemd[1]: Started Nix Daemon.4375worker2 # [ 31.459144] nix-daemon[924]: accepted connection from pid 855, user nix-grpc-daemon (trusted)4376worker2 # [ 31.508576] nix-grpc-daemon[855]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 39 ms (attempt 1/5)4377worker2 # [ 31.550087] nix-grpc-daemon[855]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 174 ms (attempt 2/5)4378worker2 # [ 31.726843] nix-grpc-daemon[855]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 256 ms (attempt 3/5)4379worker2 # [ 31.985983] nix-grpc-daemon[855]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 389 ms (attempt 4/5)4380worker2 # [ 32.383301] nix-grpc-daemon[855]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4381worker1 # [ 32.427526] nix-grpc-daemon[852]: ts=2026-09-20T12:20:48Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:558304382worker1 # [ 32.433963] systemd[1]: Started Nix Daemon.4383worker1 # [ 32.624795] nix-daemon[909]: accepted connection from pid 852, user nix-grpc-daemon (trusted)4384worker1 # [ 32.650865] nix-grpc-daemon[852]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 72 ms (attempt 1/5)4385worker1 # [ 32.725906] nix-grpc-daemon[852]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 103 ms (attempt 2/5)4386worker1 # [ 32.831705] nix-grpc-daemon[852]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 276 ms (attempt 3/5)4387worker1 # [ 33.112291] nix-grpc-daemon[852]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 632 ms (attempt 4/5)4388worker1 # [ 33.748872] nix-grpc-daemon[852]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4389worker1 # [ 33.762125] nix-grpc-daemon[852]: ts=2026-09-20T12:20:50Z level=debug event=path_info_miss path=6wlj2ay8s7brjjrdfwgnvhsxskfdjz0s-nixos-system-client-test error="error: path '/nix/store/6wlj2ay8s7brjjrdfwgnvhsxskfdjz0s-nixos-system-client-test' is required, but there is no substituter that can build it"4390worker1 # [ 33.767272] nix-grpc-daemon[852]: ts=2026-09-20T12:20:50Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:558304391worker1 # [ 33.772597] nix-grpc-daemon[852]: ts=2026-09-20T12:20:50Z level=debug event=path_info_miss path=6wlj2ay8s7brjjrdfwgnvhsxskfdjz0s-nixos-system-client-test error="error: path '/nix/store/6wlj2ay8s7brjjrdfwgnvhsxskfdjz0s-nixos-system-client-test' is required, but there is no substituter that can build it"4392client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4393client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.07 seconds)4394client: must succeed: install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4395client: (finished: must succeed: install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds)4396worker2 # [ 34.119361] nix-grpc-daemon[855]: ts=2026-09-20T12:20:50Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:558424397worker1 # [ 34.148617] nix-grpc-daemon[852]: ts=2026-09-20T12:20:50Z level=info event=oidc_jwks provider=mock keys=14398worker1 # [ 34.151451] nix-grpc-daemon[852]: ts=2026-09-20T12:20:50Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:558304399worker1 # [ 34.158286] nix-grpc-daemon[852]: ts=2026-09-20T12:20:50Z level=debug event=path_info_miss path=6wlj2ay8s7brjjrdfwgnvhsxskfdjz0s-nixos-system-client-test error="error: path '/nix/store/6wlj2ay8s7brjjrdfwgnvhsxskfdjz0s-nixos-system-client-test' is required, but there is no substituter that can build it"4400worker2 # [ 34.157232] nix-grpc-daemon[855]: ts=2026-09-20T12:20:50Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:558424401worker2 # [ 34.173100] nix-grpc-daemon[855]: ts=2026-09-20T12:20:50Z level=debug event=path_info_miss path=6wlj2ay8s7brjjrdfwgnvhsxskfdjz0s-nixos-system-client-test error="error: path '/nix/store/6wlj2ay8s7brjjrdfwgnvhsxskfdjz0s-nixos-system-client-test' is required, but there is no substituter that can build it"4402client: must succeed: rm -r /var/lib/nix-grpc-store4403client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.03 seconds)4404worker1 # [ 36.435916] nix-grpc-daemon[852]: ts=2026-09-20T12:20:52Z level=info event=denied method=QueryMissing cn=- role=none4405worker2 # [ 36.621770] nix-grpc-daemon[855]: ts=2026-09-20T12:20:53Z level=info event=denied method=QueryMissing cn=stranger role=none4406(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 5.69 seconds)4407subtest: DAG build is scheduled across workers and lands in the cache4408client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24409worker1 # [ 36.840228] nix-grpc-daemon[852]: ts=2026-09-20T12:20:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:558304410client # don't know how to build these paths:4411client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4412worker2 # [ 36.852630] nix-grpc-daemon[855]: ts=2026-09-20T12:20:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:558424413worker2 # [ 36.889580] nix-grpc-daemon[855]: ts=2026-09-20T12:20:53Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04414worker2 # [ 36.894945] nix-grpc-daemon[855]: ts=2026-09-20T12:20:53Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.3:50051 clients=14415worker1 # [ 36.948612] nix-grpc-daemon[852]: ts=2026-09-20T12:20:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:558304416worker1 # [ 36.971635] nix-grpc-daemon[852]: ts=2026-09-20T12:20:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:558304417client # copying 1 paths to 'lb:50051'...4418worker1 # [ 36.986063] nix-grpc-daemon[852]: ts=2026-09-20T12:20:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:558304419client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4420worker1 # [ 37.019912] nix-daemon[909]: accepted connection from pid 852, user nix-grpc-daemon (trusted)4421worker1 # [ 37.052993] nix-grpc-daemon[852]: ts=2026-09-20T12:20:53Z level=info event=niks3_push_started pid=9204422worker1 # [ 37.331696] nix-grpc-daemon[920]: time=2026-09-20T12:20:53.830Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""4423worker1 # [ 37.521908] nix-daemon[909]: accepted connection from pid 932, user nix-grpc-daemon (trusted)4424worker1 # [ 37.539210] nix-daemon[909]: reaped child process 935, status = succeeded4425worker1 # [ 37.668044] nix-daemon[909]: accepted connection from pid 937, user nix-grpc-daemon (trusted)4426worker1 # [ 37.682902] nix-daemon[909]: reaped child process 940, status = succeeded4427lb # [ 37.706840] niks3-server[1111]: 2026/09/20 12:20:54 INFO Received uploads request method=POST path=/api/pending_closures4428worker1 # [ 37.725531] nix-grpc-daemon[920]: time=2026-09-20T12:20:54.228Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4429worker1 # [ 37.727916] nix-grpc-daemon[920]: time=2026-09-20T12:20:54.230Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4430lb # [ 37.789375] niks3-server[1111]: 2026/09/20 12:20:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4431lb # [ 37.798363] niks3-server[1111]: 2026/09/20 12:20:54 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4432lb # [ 37.815146] niks3-server[1111]: 2026/09/20 12:20:54 INFO Signed narinfos id=1 count=14433worker1 # [ 37.808571] nix-grpc-daemon[920]: time=2026-09-20T12:20:54.311Z level=INFO msg="Uploading 1 narinfos"4434lb # [ 37.833057] niks3-server[1111]: 2026/09/20 12:20:54 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4435lb # [ 37.838928] niks3-server[1111]: 2026/09/20 12:20:54 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4436lb # [ 37.844424] niks3-server[1111]: 2026/09/20 12:20:54 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo4437lb # [ 37.855192] niks3-server[1111]: 2026/09/20 12:20:54 INFO Completed upload id=14438worker1 # [ 37.846926] nix-grpc-daemon[920]: time=2026-09-20T12:20:54.349Z level=INFO msg="Upload complete. (498ms)"4439worker1 # [ 37.851472] nix-grpc-daemon[852]: ts=2026-09-20T12:20:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55830 duration_s=0 paths=1 nar_bytes_in=5044440worker1 # [ 37.854870] nix-grpc-daemon[852]: ts=2026-09-20T12:20:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:558304441worker1 # [ 37.862147] nix-daemon[909]: accepted connection from pid 852, user nix-grpc-daemon (trusted)4442worker1 # [ 37.868300] nix-daemon[909]: reaped child process 918, status = succeeded4443client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4444worker1 # [ 37.874221] nix-daemon[909]: accepted connection from pid 852, user nix-grpc-daemon (trusted)4445client # farm-a> node-a: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4446client # farm-a> LOG-farm-a-t14447worker1 # [ 39.092487] nix-daemon[909]: reaped child process 944, status = succeeded4448worker1 # [ 39.242352] nix-daemon[909]: accepted connection from pid 959, user nix-grpc-daemon (trusted)4449worker1 # [ 39.256149] nix-daemon[909]: reaped child process 962, status = succeeded4450lb # [ 39.279497] niks3-server[1111]: 2026/09/20 12:20:55 INFO Received uploads request method=POST path=/api/pending_closures4451worker1 # [ 39.403822] nix-grpc-daemon[920]: time=2026-09-20T12:20:55.905Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4452worker1 # [ 39.412263] nix-grpc-daemon[920]: time=2026-09-20T12:20:55.910Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4453lb # [ 39.466867] niks3-server[1111]: 2026/09/20 12:20:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4454lb # [ 39.478784] niks3-server[1111]: 2026/09/20 12:20:56 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4455lb # [ 39.487118] niks3-server[1111]: 2026/09/20 12:20:56 INFO Signed narinfos id=2 count=14456worker1 # [ 39.482965] nix-grpc-daemon[920]: time=2026-09-20T12:20:55.984Z level=INFO msg="Uploading 1 narinfos"4457lb # [ 39.495419] niks3-server[1111]: 2026/09/20 12:20:56 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4458lb # [ 39.500838] niks3-server[1111]: 2026/09/20 12:20:56 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4459lb # [ 39.510135] niks3-server[1111]: 2026/09/20 12:20:56 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4460lb # [ 39.520810] niks3-server[1111]: 2026/09/20 12:20:56 INFO Completed upload id=24461worker1 # [ 39.512633] nix-grpc-daemon[920]: time=2026-09-20T12:20:56.015Z level=INFO msg="Upload complete. (428ms)"4462worker1 # [ 39.515471] nix-grpc-daemon[852]: ts=2026-09-20T12:20:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55830 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14463lb # [ 39.525446] niks3-server[1111]: 2026/09/20 12:20:56 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4464worker1 # [ 39.528630] nix-daemon[909]: reaped child process 942, status = succeeded4465worker2 # [ 39.531175] nix-grpc-daemon[855]: ts=2026-09-20T12:20:56Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04466worker2 # [ 39.535448] nix-grpc-daemon[855]: ts=2026-09-20T12:20:56Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.3:50051 clients=14467worker1 # [ 39.581765] nix-grpc-daemon[852]: ts=2026-09-20T12:20:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:558304468worker1 # [ 39.591737] nix-grpc-daemon[852]: ts=2026-09-20T12:20:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:558304469worker1 # [ 39.612201] nix-grpc-daemon[920]: time=2026-09-20T12:20:56.115Z level=INFO msg="All 1 paths already cached"4470client # copying 1 paths to 'lb:50051'...4471client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4472worker1 # [ 39.620192] nix-grpc-daemon[852]: ts=2026-09-20T12:20:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:558304473worker1 # [ 39.633528] nix-daemon[909]: accepted connection from pid 852, user nix-grpc-daemon (trusted)4474worker1 # [ 39.795286] nix-daemon[909]: accepted connection from pid 968, user nix-grpc-daemon (trusted)4475worker1 # [ 39.810560] nix-daemon[909]: reaped child process 971, status = succeeded4476worker1 # [ 39.960613] nix-daemon[909]: accepted connection from pid 973, user nix-grpc-daemon (trusted)4477worker1 # [ 39.975371] nix-daemon[909]: reaped child process 976, status = succeeded4478lb # [ 39.996774] niks3-server[1111]: 2026/09/20 12:20:56 INFO Received uploads request method=POST path=/api/pending_closures4479worker1 # [ 40.002424] nix-grpc-daemon[920]: time=2026-09-20T12:20:56.504Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4480worker1 # [ 40.003918] nix-grpc-daemon[920]: time=2026-09-20T12:20:56.504Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4481lb # [ 40.033582] niks3-server[1111]: 2026/09/20 12:20:56 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4482lb # [ 40.049141] niks3-server[1111]: 2026/09/20 12:20:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4483lb # [ 40.054176] niks3-server[1111]: 2026/09/20 12:20:56 INFO Signed narinfos id=3 count=14484worker1 # [ 40.049532] nix-grpc-daemon[920]: time=2026-09-20T12:20:56.552Z level=INFO msg="Uploading 1 narinfos"4485lb # [ 40.061730] niks3-server[1111]: 2026/09/20 12:20:56 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4486lb # [ 40.072081] niks3-server[1111]: 2026/09/20 12:20:56 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4487lb # [ 40.082438] niks3-server[1111]: 2026/09/20 12:20:56 INFO Completed upload id=34488worker1 # [ 40.074093] nix-grpc-daemon[920]: time=2026-09-20T12:20:56.576Z level=INFO msg="Upload complete. (432ms)"4489worker1 # [ 40.079150] nix-grpc-daemon[852]: ts=2026-09-20T12:20:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55830 duration_s=0 paths=1 nar_bytes_in=6244490worker1 # [ 40.086405] nix-grpc-daemon[852]: ts=2026-09-20T12:20:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:558304491lb # [ 40.099875] niks3-server[1111]: 2026/09/20 12:20:56 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4492worker1 # [ 40.104724] nix-daemon[909]: accepted connection from pid 852, user nix-grpc-daemon (trusted)4493worker1 # [ 40.109964] nix-daemon[909]: reaped child process 965, status = succeeded4494client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4495worker1 # [ 40.113687] nix-daemon[909]: accepted connection from pid 852, user nix-grpc-daemon (trusted)4496client # farm-b> node-a: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4497lb # [ 40.768276] envoy[1009]: conn peer=[2001:db8:1::1]:34550 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=406 ms=25564498lb # [ 40.770166] envoy[1009]: conn peer=[2001:db8:1::1]:41534 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=674499lb # [ 40.771839] envoy[1009]: conn peer=[2001:db8:1::1]:41544 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=114500lb # [ 40.773404] envoy[1009]: conn peer=[2001:db8:1::1]:41554 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=74501client # farm-b> LOG-farm-b-t14502worker1 # [ 41.341554] nix-daemon[909]: reaped child process 980, status = succeeded4503worker1 # [ 41.496678] nix-daemon[909]: accepted connection from pid 995, user nix-grpc-daemon (trusted)4504worker1 # [ 41.511951] nix-daemon[909]: reaped child process 998, status = succeeded4505lb # [ 41.537842] niks3-server[1111]: 2026/09/20 12:20:58 INFO Received uploads request method=POST path=/api/pending_closures4506worker1 # [ 41.573614] nix-grpc-daemon[920]: time=2026-09-20T12:20:58.076Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4507worker1 # [ 41.576357] nix-grpc-daemon[920]: time=2026-09-20T12:20:58.078Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4508lb # [ 41.607750] niks3-server[1111]: 2026/09/20 12:20:58 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4509lb # [ 41.621975] niks3-server[1111]: 2026/09/20 12:20:58 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4510lb # [ 41.631138] niks3-server[1111]: 2026/09/20 12:20:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4511lb # [ 41.634076] niks3-server[1111]: 2026/09/20 12:20:58 INFO Signed narinfos id=4 count=14512worker1 # [ 41.625414] nix-grpc-daemon[920]: time=2026-09-20T12:20:58.128Z level=INFO msg="Uploading 1 narinfos"4513lb # [ 41.637918] niks3-server[1111]: 2026/09/20 12:20:58 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4514lb # [ 41.644933] niks3-server[1111]: 2026/09/20 12:20:58 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4515lb # [ 41.649500] niks3-server[1111]: 2026/09/20 12:20:58 INFO Completed upload id=44516worker1 # [ 41.640822] nix-grpc-daemon[920]: time=2026-09-20T12:20:58.143Z level=INFO msg="Upload complete. (318ms)"4517lb # [ 41.651249] niks3-server[1111]: 2026/09/20 12:20:58 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4518worker1 # [ 41.644451] nix-grpc-daemon[852]: ts=2026-09-20T12:20:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55830 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14519worker2 # [ 41.644720] nix-grpc-daemon[855]: ts=2026-09-20T12:20:58Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04520worker1 # [ 41.652640] nix-daemon[909]: reaped child process 978, status = succeeded4521worker2 # [ 41.647168] nix-grpc-daemon[855]: ts=2026-09-20T12:20:58Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.3:50051 clients=14522worker1 # [ 41.695712] nix-grpc-daemon[852]: ts=2026-09-20T12:20:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:558304523worker1 # [ 41.703539] nix-grpc-daemon[852]: ts=2026-09-20T12:20:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:558304524worker1 # [ 41.716408] nix-grpc-daemon[920]: time=2026-09-20T12:20:58.219Z level=INFO msg="All 2 paths already cached"4525client # copying 1 paths to 'lb:50051'...4526client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4527worker1 # [ 41.723616] nix-grpc-daemon[852]: ts=2026-09-20T12:20:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:558304528worker1 # [ 41.733410] nix-daemon[909]: accepted connection from pid 852, user nix-grpc-daemon (trusted)4529worker1 # [ 41.899871] nix-daemon[909]: accepted connection from pid 1002, user nix-grpc-daemon (trusted)4530worker1 # [ 41.914881] nix-daemon[909]: reaped child process 1005, status = succeeded4531worker1 # [ 42.089159] nix-daemon[909]: accepted connection from pid 1007, user nix-grpc-daemon (trusted)4532worker1 # [ 42.105872] nix-daemon[909]: reaped child process 1010, status = succeeded4533lb # [ 42.130797] niks3-server[1111]: 2026/09/20 12:20:58 INFO Received uploads request method=POST path=/api/pending_closures4534worker1 # [ 42.130234] nix-grpc-daemon[920]: time=2026-09-20T12:20:58.633Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4535worker1 # [ 42.131899] nix-grpc-daemon[920]: time=2026-09-20T12:20:58.634Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4536lb # [ 42.155036] niks3-server[1111]: 2026/09/20 12:20:58 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4537lb # [ 42.159803] niks3-server[1111]: 2026/09/20 12:20:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4538lb # [ 42.162265] niks3-server[1111]: 2026/09/20 12:20:58 INFO Signed narinfos id=5 count=14539worker1 # [ 42.153486] nix-grpc-daemon[920]: time=2026-09-20T12:20:58.656Z level=INFO msg="Uploading 1 narinfos"4540lb # [ 42.165555] niks3-server[1111]: 2026/09/20 12:20:58 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4541lb # [ 42.173552] niks3-server[1111]: 2026/09/20 12:20:58 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4542lb # [ 42.176834] niks3-server[1111]: 2026/09/20 12:20:58 INFO Completed upload id=54543worker1 # [ 42.168148] nix-grpc-daemon[920]: time=2026-09-20T12:20:58.671Z level=INFO msg="Upload complete. (422ms)"4544lb # [ 42.178725] niks3-server[1111]: 2026/09/20 12:20:58 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4545worker1 # [ 42.170361] nix-grpc-daemon[852]: ts=2026-09-20T12:20:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55830 duration_s=0 paths=1 nar_bytes_in=7524546worker1 # [ 42.174616] nix-grpc-daemon[852]: ts=2026-09-20T12:20:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:558304547worker1 # [ 42.180944] nix-daemon[909]: accepted connection from pid 852, user nix-grpc-daemon (trusted)4548worker1 # [ 42.183473] nix-daemon[909]: reaped child process 1000, status = succeeded4549client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4550worker1 # [ 42.189450] nix-daemon[909]: accepted connection from pid 852, user nix-grpc-daemon (trusted)4551client # farm-top> node-a: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4552client # farm-top> LOG-farm-top-t14553worker1 # [ 43.216349] nix-daemon[909]: reaped child process 1014, status = succeeded4554worker1 # [ 43.381108] nix-daemon[909]: accepted connection from pid 1030, user nix-grpc-daemon (trusted)4555worker1 # [ 43.394974] nix-daemon[909]: reaped child process 1033, status = succeeded4556lb # [ 43.421442] niks3-server[1111]: 2026/09/20 12:21:00 INFO Received uploads request method=POST path=/api/pending_closures4557worker1 # [ 43.424826] nix-grpc-daemon[920]: time=2026-09-20T12:20:59.927Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4558worker1 # [ 43.428262] nix-grpc-daemon[920]: time=2026-09-20T12:20:59.929Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4559lb # [ 43.474082] niks3-server[1111]: 2026/09/20 12:21:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4560lb # [ 43.477638] niks3-server[1111]: 2026/09/20 12:21:00 INFO Signed narinfos id=6 count=14561worker1 # [ 43.471895] nix-grpc-daemon[920]: time=2026-09-20T12:20:59.974Z level=INFO msg="Uploading 1 narinfos"4562lb # [ 43.487128] niks3-server[1111]: 2026/09/20 12:21:00 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4563lb # [ 43.491576] niks3-server[1111]: 2026/09/20 12:21:00 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4564lb # [ 43.502064] niks3-server[1111]: 2026/09/20 12:21:00 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4565lb # [ 43.509701] niks3-server[1111]: 2026/09/20 12:21:00 INFO Completed upload id=64566worker1 # [ 43.501070] nix-grpc-daemon[920]: time=2026-09-20T12:21:00.004Z level=INFO msg="Upload complete. (304ms)"4567lb # [ 43.511361] niks3-server[1111]: 2026/09/20 12:21:00 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4568lb # [ 43.514971] niks3-server[1111]: 2026/09/20 12:21:00 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4569worker1 # [ 43.505693] nix-grpc-daemon[852]: ts=2026-09-20T12:21:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55830 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14570worker1 # [ 43.514833] nix-daemon[909]: reaped child process 1012, status = succeeded4571client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 6.90 seconds)4572client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4573client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.22 seconds)4574client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4575client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.02 seconds)4576client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4577client # copying 3 paths...4578client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4579client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4580client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4581client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.29 seconds)4582(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 7.50 seconds)4583subtest: repeat is answered Cached by the scheduler without building4584worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4585worker1 # finding garbage collector roots...4586worker1 # deleting specified paths...4587worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4588worker1 # deleting unused links...4589worker1 # note: hard linking is currently saving 0.0 KiB4590worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.21 seconds)4591worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4592worker2 # finding garbage collector roots...4593worker2 # deleting specified paths...4594worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4595worker2 # deleting unused links...4596worker2 # note: hard linking is currently saving 0.0 KiB4597worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.25 seconds)4598client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24599worker2 # [ 44.819639] nix-grpc-daemon[855]: ts=2026-09-20T12:21:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:558424600client # don't know how to build these paths:4601client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4602worker2 # [ 44.828513] nix-grpc-daemon[855]: ts=2026-09-20T12:21:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:558424603worker2 # [ 44.851965] nix-grpc-daemon[855]: ts=2026-09-20T12:21:01Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14604client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 0.28 seconds)4605client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4606client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.16 seconds)4607lb: must succeed: curl -sf localhost:9901/clusters4608lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4609lb: must succeed: curl -sf localhost:9901/clusters4610lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4611worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4612worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4613(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.16 seconds)4614subtest: inputs already in the cache are not sent to the scheduler4615lb: must succeed: curl -sf localhost:9901/clusters4616lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4617lb: must succeed: curl -sf localhost:9901/clusters4618lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4619worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4620worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.03 seconds)4621client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&24622worker1 # [ 45.662767] nix-grpc-daemon[852]: ts=2026-09-20T12:21:02Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:558304623client # don't know how to build these paths:4624client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4625worker1 # [ 45.671113] nix-grpc-daemon[852]: ts=2026-09-20T12:21:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:558304626worker1 # [ 45.679946] nix-grpc-daemon[920]: time=2026-09-20T12:21:02.182Z level=INFO msg="All 2 paths already cached"4627worker2 # [ 45.687587] nix-grpc-daemon[855]: ts=2026-09-20T12:21:02Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04628worker2 # [ 45.692222] nix-grpc-daemon[855]: ts=2026-09-20T12:21:02Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.3:50051 clients=14629worker1 # [ 45.737814] nix-grpc-daemon[852]: ts=2026-09-20T12:21:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:558304630worker1 # [ 45.748350] nix-grpc-daemon[852]: ts=2026-09-20T12:21:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:558304631worker1 # [ 45.758517] nix-grpc-daemon[920]: time=2026-09-20T12:21:02.261Z level=INFO msg="All 2 paths already cached"4632client # copying 1 paths to 'lb:50051'...4633client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4634worker1 # [ 45.765767] nix-grpc-daemon[852]: ts=2026-09-20T12:21:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:558304635worker1 # [ 45.777435] nix-daemon[909]: accepted connection from pid 852, user nix-grpc-daemon (trusted)4636worker1 # [ 45.946897] nix-daemon[909]: accepted connection from pid 1050, user nix-grpc-daemon (trusted)4637worker1 # [ 45.960910] nix-daemon[909]: reaped child process 1053, status = succeeded4638worker1 # [ 46.117566] nix-daemon[909]: accepted connection from pid 1055, user nix-grpc-daemon (trusted)4639worker1 # [ 46.130716] nix-daemon[909]: reaped child process 1058, status = succeeded4640lb # [ 46.150959] niks3-server[1111]: 2026/09/20 12:21:02 INFO Received uploads request method=POST path=/api/pending_closures4641worker1 # [ 46.146315] nix-grpc-daemon[920]: time=2026-09-20T12:21:02.649Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4642worker1 # [ 46.147765] nix-grpc-daemon[920]: time=2026-09-20T12:21:02.650Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4643lb # [ 46.168521] niks3-server[1111]: 2026/09/20 12:21:02 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4644lb # [ 46.172205] niks3-server[1111]: 2026/09/20 12:21:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4645lb # [ 46.174384] niks3-server[1111]: 2026/09/20 12:21:02 INFO Signed narinfos id=7 count=14646worker1 # [ 46.165557] nix-grpc-daemon[920]: time=2026-09-20T12:21:02.668Z level=INFO msg="Uploading 1 narinfos"4647lb # [ 46.176650] niks3-server[1111]: 2026/09/20 12:21:02 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4648lb # [ 46.181967] niks3-server[1111]: 2026/09/20 12:21:02 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4649lb # [ 46.184746] niks3-server[1111]: 2026/09/20 12:21:02 INFO Completed upload id=74650worker1 # [ 46.175945] nix-grpc-daemon[920]: time=2026-09-20T12:21:02.678Z level=INFO msg="Upload complete. (385ms)"4651lb # [ 46.187204] niks3-server[1111]: 2026/09/20 12:21:02 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4652worker1 # [ 46.177910] nix-grpc-daemon[852]: ts=2026-09-20T12:21:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55830 duration_s=0 paths=1 nar_bytes_in=7524653worker1 # [ 46.180789] nix-grpc-daemon[852]: ts=2026-09-20T12:21:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:558304654worker1 # [ 46.186963] nix-daemon[909]: accepted connection from pid 852, user nix-grpc-daemon (trusted)4655worker1 # [ 46.190414] nix-daemon[909]: reaped child process 1048, status = succeeded4656worker1 # [ 46.195451] nix-daemon[909]: accepted connection from pid 852, user nix-grpc-daemon (trusted)4657client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4658client # farm-top> node-a: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4659worker1 # [ 47.411736] nix-daemon[909]: reaped child process 1062, status = succeeded4660client # farm-top> LOG-farm-top-t1b4661worker1 # [ 47.563123] nix-daemon[909]: accepted connection from pid 1077, user nix-grpc-daemon (trusted)4662worker1 # [ 47.578844] nix-daemon[909]: reaped child process 1080, status = succeeded4663lb # [ 47.601581] niks3-server[1111]: 2026/09/20 12:21:04 INFO Received uploads request method=POST path=/api/pending_closures4664worker1 # [ 47.612281] nix-grpc-daemon[920]: time=2026-09-20T12:21:04.114Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4665worker1 # [ 47.616533] nix-grpc-daemon[920]: time=2026-09-20T12:21:04.116Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4666lb # [ 47.645885] niks3-server[1111]: 2026/09/20 12:21:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4667worker1 # [ 47.641182] nix-grpc-daemon[920]: time=2026-09-20T12:21:04.143Z level=INFO msg="Uploading 1 narinfos"4668lb # [ 47.650033] niks3-server[1111]: 2026/09/20 12:21:04 INFO Signed narinfos id=8 count=14669lb # [ 47.651390] niks3-server[1111]: 2026/09/20 12:21:04 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4670lb # [ 47.654709] niks3-server[1111]: 2026/09/20 12:21:04 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4671lb # [ 47.658500] niks3-server[1111]: 2026/09/20 12:21:04 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4672lb # [ 47.663054] niks3-server[1111]: 2026/09/20 12:21:04 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4673lb # [ 47.667859] niks3-server[1111]: 2026/09/20 12:21:04 INFO Completed upload id=84674worker1 # [ 47.659186] nix-grpc-daemon[920]: time=2026-09-20T12:21:04.162Z level=INFO msg="Upload complete. (256ms)"4675lb # [ 47.669818] niks3-server[1111]: 2026/09/20 12:21:04 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4676worker1 # [ 47.662400] nix-grpc-daemon[852]: ts=2026-09-20T12:21:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55830 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14677worker1 # [ 47.670821] nix-daemon[909]: reaped child process 1060, status = succeeded4678client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.27 seconds)4679client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath4680client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.18 seconds)4681lb: must succeed: curl -sf localhost:9901/clusters4682lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4683lb: must succeed: curl -sf localhost:9901/clusters4684lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4685worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4686worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.03 seconds)4687(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.69 seconds)4688subtest: two clients wanting the same drv build it once4689client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4690client # [ 48.015621] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4691client # Running as unit: dedup1.service; invocation ID: 29d8cbc7c049430ba37e01d1d0b7533d4692client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 0.05 seconds)4693client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4694worker2 # [ 48.209904] nix-grpc-daemon[855]: ts=2026-09-20T12:21:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:558424695client # [ 48.196776] nix[900]: don't know how to build these paths:4696client # [ 48.198273] nix[900]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4697worker2 # [ 48.217767] nix-grpc-daemon[855]: ts=2026-09-20T12:21:04Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04698worker2 # [ 48.220146] nix-grpc-daemon[855]: ts=2026-09-20T12:21:04Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.4:50051 clients=14699worker2 # [ 48.223255] nix-grpc-daemon[855]: ts=2026-09-20T12:21:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:558424700worker2 # [ 48.234979] nix-grpc-daemon[855]: ts=2026-09-20T12:21:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:558424701client # [ 48.230908] nix[900]: copying 1 paths to 'lb:50051'...4702client # [ 48.231782] nix[900]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4703worker2 # [ 48.245807] nix-grpc-daemon[855]: ts=2026-09-20T12:21:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:558424704worker2 # [ 48.258924] nix-daemon[924]: accepted connection from pid 855, user nix-grpc-daemon (trusted)4705worker2 # [ 48.276481] nix-grpc-daemon[855]: ts=2026-09-20T12:21:04Z level=info event=niks3_push_started pid=9754706worker2 # [ 48.513237] nix-grpc-daemon[975]: time=2026-09-20T12:21:04.983Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""4707worker2 # [ 48.702403] nix-daemon[924]: accepted connection from pid 987, user nix-grpc-daemon (trusted)4708worker2 # [ 48.721185] nix-daemon[924]: reaped child process 990, status = succeeded4709worker2 # [ 48.880068] nix-daemon[924]: accepted connection from pid 993, user nix-grpc-daemon (trusted)4710worker2 # [ 48.894769] nix-daemon[924]: reaped child process 996, status = succeeded4711lb # [ 48.923914] niks3-server[1111]: 2026/09/20 12:21:05 INFO Received uploads request method=POST path=/api/pending_closures4712worker2 # [ 48.921948] nix-grpc-daemon[975]: time=2026-09-20T12:21:05.395Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4713worker2 # [ 48.924163] nix-grpc-daemon[975]: time=2026-09-20T12:21:05.397Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4714lb # [ 48.967765] niks3-server[1111]: 2026/09/20 12:21:05 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4715lb # [ 48.971765] niks3-server[1111]: 2026/09/20 12:21:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4716lb # [ 48.973768] niks3-server[1111]: 2026/09/20 12:21:05 INFO Signed narinfos id=9 count=14717worker2 # [ 48.960684] nix-grpc-daemon[975]: time=2026-09-20T12:21:05.433Z level=INFO msg="Uploading 1 narinfos"4718lb # [ 48.981032] niks3-server[1111]: 2026/09/20 12:21:05 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4719lb # [ 48.985302] niks3-server[1111]: 2026/09/20 12:21:05 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4720lb # [ 48.989637] niks3-server[1111]: 2026/09/20 12:21:05 INFO Completed upload id=94721worker2 # [ 48.975857] nix-grpc-daemon[975]: time=2026-09-20T12:21:05.449Z level=INFO msg="Upload complete. (449ms)"4722lb # [ 48.991691] niks3-server[1111]: 2026/09/20 12:21:05 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4723worker2 # [ 48.978357] nix-grpc-daemon[855]: ts=2026-09-20T12:21:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55842 duration_s=0 paths=1 nar_bytes_in=4324724worker2 # [ 48.981421] nix-grpc-daemon[855]: ts=2026-09-20T12:21:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:558424725worker2 # [ 48.990582] nix-daemon[924]: accepted connection from pid 855, user nix-grpc-daemon (trusted)4726worker2 # [ 48.992556] nix-daemon[924]: reaped child process 973, status = succeeded4727client # [ 48.982831] nix[900]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4728worker2 # [ 48.997718] nix-daemon[924]: accepted connection from pid 855, user nix-grpc-daemon (trusted)4729client # [ 49.080329] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4730client # Running as unit: dedup2.service; invocation ID: 93aa1d41752b48b2a9b9c556ad486f594731client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 1.06 seconds)4732worker1 # [ 49.335978] nix-grpc-daemon[852]: ts=2026-09-20T12:21:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:558304733client # [ 49.316787] nix[916]: don't know how to build these paths:4734client # [ 49.318358] nix[916]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4735worker2 # [ 49.339543] nix-grpc-daemon[855]: ts=2026-09-20T12:21:05Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04736worker2 # [ 49.344415] nix-grpc-daemon[855]: ts=2026-09-20T12:21:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:558424737client # [ 49.335872] nix[916]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4738client: must succeed: sleep 24739lb # [ 50.768824] envoy[1009]: conn peer=[2001:db8:1::1]:41566 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9524 tx=3858 ms=67024740lb # [ 50.770475] envoy[1009]: conn peer=[2001:db8:1::1]:43636 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=514741lb # [ 50.771847] envoy[1009]: conn peer=[2001:db8:1::1]:43650 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4139 tx=1705 ms=20124742client: (finished: must succeed: sleep 2, in 2.03 seconds)4743client: waiting for success: ! systemctl is-active dedup1 dedup24744worker2 # [ 52.626787] nix-daemon[924]: reaped child process 1000, status = succeeded4745worker2 # [ 52.786739] nix-daemon[924]: accepted connection from pid 1031, user nix-grpc-daemon (trusted)4746worker2 # [ 52.801265] nix-daemon[924]: reaped child process 1034, status = succeeded4747lb # [ 52.827455] niks3-server[1111]: 2026/09/20 12:21:09 INFO Received uploads request method=POST path=/api/pending_closures4748worker2 # [ 52.817270] nix-grpc-daemon[975]: time=2026-09-20T12:21:09.289Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4749worker2 # [ 52.819486] nix-grpc-daemon[975]: time=2026-09-20T12:21:09.289Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4750lb # [ 52.856107] niks3-server[1111]: 2026/09/20 12:21:09 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4751lb # [ 52.859061] niks3-server[1111]: 2026/09/20 12:21:09 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4752lb # [ 52.861967] niks3-server[1111]: 2026/09/20 12:21:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4753lb # [ 52.864894] niks3-server[1111]: 2026/09/20 12:21:09 INFO Signed narinfos id=10 count=14754worker2 # [ 52.850495] nix-grpc-daemon[975]: time=2026-09-20T12:21:09.323Z level=INFO msg="Uploading 1 narinfos"4755lb # [ 52.867981] niks3-server[1111]: 2026/09/20 12:21:09 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4756lb # [ 52.874070] niks3-server[1111]: 2026/09/20 12:21:09 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4757lb # [ 52.877352] niks3-server[1111]: 2026/09/20 12:21:09 INFO Completed upload id=104758worker2 # [ 52.862281] nix-grpc-daemon[975]: time=2026-09-20T12:21:09.335Z level=INFO msg="Upload complete. (250ms)"4759lb # [ 52.880062] niks3-server[1111]: 2026/09/20 12:21:09 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4760worker2 # [ 52.866451] nix-grpc-daemon[855]: ts=2026-09-20T12:21:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55842 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14761worker2 # [ 52.871211] nix-grpc-daemon[855]: ts=2026-09-20T12:21:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55842 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14762worker2 # [ 52.879320] nix-daemon[924]: reaped child process 998, status = succeeded4763client # [ 52.883186] systemd[1]: dedup2.service: Deactivated successfully.4764client # [ 52.885033] systemd[1]: dedup2.service: Consumed 122ms CPU time over 3.804s wall clock time, 31.2M memory peak, 7.4K incoming IP traffic, 6.3K outgoing IP traffic.4765client # [ 52.888329] systemd[1]: dedup1.service: Deactivated successfully.4766client # [ 52.889413] systemd[1]: dedup1.service: Consumed 103ms CPU time over 4.872s wall clock time, 39.3M memory peak, 8.6K incoming IP traffic, 9.1K outgoing IP traffic.4767client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.07 seconds)4768client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4769client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)4770worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4771worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.03 seconds)4772worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4773worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4774worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4775worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4776worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4777worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4778(finished: subtest: two clients wanting the same drv build it once, in 5.72 seconds)4779subtest: build hook: nix-daemon with builders = grpc://lb4780client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log4781client # [ 53.883548] systemd[1]: Started Nix Daemon.4782client # [ 54.060685] nix-daemon[958]: accepted connection from pid 955, user root (trusted)4783worker2 # [ 54.254629] nix-grpc-daemon[855]: ts=2026-09-20T12:21:10Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:558424784worker2 # [ 54.264897] nix-grpc-daemon[855]: ts=2026-09-20T12:21:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:558424785worker2 # [ 54.282180] nix-grpc-daemon[855]: ts=2026-09-20T12:21:10Z level=debug event=want client=6 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=04786worker2 # [ 54.284438] nix-grpc-daemon[855]: ts=2026-09-20T12:21:10Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.3:50051 clients=14787worker1 # [ 54.333967] nix-grpc-daemon[852]: ts=2026-09-20T12:21:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:558304788worker1 # [ 54.342890] nix-grpc-daemon[852]: ts=2026-09-20T12:21:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:558304789worker1 # [ 54.351374] nix-grpc-daemon[852]: ts=2026-09-20T12:21:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:558304790worker1 # [ 54.357080] nix-daemon[909]: accepted connection from pid 852, user nix-grpc-daemon (trusted)4791worker1 # [ 54.512950] nix-daemon[909]: accepted connection from pid 1105, user nix-grpc-daemon (trusted)4792worker1 # [ 54.527206] nix-daemon[909]: reaped child process 1108, status = succeeded4793worker1 # [ 54.654722] nix-daemon[909]: accepted connection from pid 1110, user nix-grpc-daemon (trusted)4794worker1 # [ 54.667500] nix-daemon[909]: reaped child process 1113, status = succeeded4795lb # [ 54.687143] niks3-server[1111]: 2026/09/20 12:21:11 INFO Received uploads request method=POST path=/api/pending_closures4796worker1 # [ 54.699882] nix-grpc-daemon[920]: time=2026-09-20T12:21:11.202Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4797worker1 # [ 54.701325] nix-grpc-daemon[920]: time=2026-09-20T12:21:11.202Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"4798lb # [ 54.719117] niks3-server[1111]: 2026/09/20 12:21:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4799lb # [ 54.721108] niks3-server[1111]: 2026/09/20 12:21:11 INFO Signed narinfos id=11 count=14800worker1 # [ 54.713763] nix-grpc-daemon[920]: time=2026-09-20T12:21:11.216Z level=INFO msg="Uploading 1 narinfos"4801lb # [ 54.727211] niks3-server[1111]: 2026/09/20 12:21:11 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4802lb # [ 54.841698] niks3-server[1111]: 2026/09/20 12:21:11 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst4803lb # [ 54.850701] niks3-server[1111]: 2026/09/20 12:21:11 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls4804lb # [ 54.860332] niks3-server[1111]: 2026/09/20 12:21:11 INFO Completed upload id=114805worker1 # [ 54.851716] nix-grpc-daemon[920]: time=2026-09-20T12:21:11.354Z level=INFO msg="Upload complete. (489ms)"4806worker1 # [ 54.854099] nix-grpc-daemon[852]: ts=2026-09-20T12:21:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55830 duration_s=0 paths=1 nar_bytes_in=5124807lb # [ 54.863689] niks3-server[1111]: 2026/09/20 12:21:11 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo4808worker1 # [ 54.857319] nix-grpc-daemon[852]: ts=2026-09-20T12:21:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:558304809worker1 # [ 54.864260] nix-daemon[909]: accepted connection from pid 852, user nix-grpc-daemon (trusted)4810worker1 # [ 54.866844] nix-daemon[909]: reaped child process 1103, status = succeeded4811worker1 # [ 54.872887] nix-daemon[909]: accepted connection from pid 852, user nix-grpc-daemon (trusted)4812worker1 # [ 56.302470] nix-daemon[909]: reaped child process 1117, status = succeeded4813worker1 # [ 56.427900] nix-daemon[909]: accepted connection from pid 1132, user nix-grpc-daemon (trusted)4814worker1 # [ 56.440775] nix-daemon[909]: reaped child process 1135, status = succeeded4815lb # [ 56.463913] niks3-server[1111]: 2026/09/20 12:21:13 INFO Received uploads request method=POST path=/api/pending_closures4816worker1 # [ 56.460261] nix-grpc-daemon[920]: time=2026-09-20T12:21:12.962Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4817worker1 # [ 56.463059] nix-grpc-daemon[920]: time=2026-09-20T12:21:12.964Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"4818lb # [ 56.491699] niks3-server[1111]: 2026/09/20 12:21:13 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv4819lb # [ 56.495539] niks3-server[1111]: 2026/09/20 12:21:13 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4820lb # [ 56.498720] niks3-server[1111]: 2026/09/20 12:21:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4821lb # [ 56.500645] niks3-server[1111]: 2026/09/20 12:21:13 INFO Signed narinfos id=12 count=14822worker1 # [ 56.493541] nix-grpc-daemon[920]: time=2026-09-20T12:21:12.996Z level=INFO msg="Uploading 1 narinfos"4823lb # [ 56.506551] niks3-server[1111]: 2026/09/20 12:21:13 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls4824lb # [ 56.510969] niks3-server[1111]: 2026/09/20 12:21:13 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4825lb # [ 56.514151] niks3-server[1111]: 2026/09/20 12:21:13 INFO Completed upload id=124826worker1 # [ 56.505461] nix-grpc-daemon[920]: time=2026-09-20T12:21:13.008Z level=INFO msg="Upload complete. (225ms)"4827lb # [ 56.517186] niks3-server[1111]: 2026/09/20 12:21:13 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo4828worker1 # [ 56.509329] nix-grpc-daemon[852]: ts=2026-09-20T12:21:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55830 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=6 outputs=14829worker1 # [ 56.515903] nix-daemon[909]: reaped child process 1115, status = succeeded4830worker2 # [ 56.508565] nix-grpc-daemon[855]: ts=2026-09-20T12:21:12Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:558424831worker2 # [ 56.521504] nix-grpc-daemon[855]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...4832worker1 # [ 56.564378] nix-grpc-daemon[852]: ts=2026-09-20T12:21:13Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:558304833worker1 # [ 56.567438] nix-grpc-daemon[852]: ts=2026-09-20T12:21:13Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:55830 duration_s=0 paths=1 nar_bytes_out=1284834worker2 # [ 56.779708] nix-grpc-daemon[855]: ts=2026-09-20T12:21:13Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:558424835worker2 # [ 56.786936] nix-grpc-daemon[855]: ts=2026-09-20T12:21:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:558424836worker2 # [ 56.904995] nix-grpc-daemon[975]: time=2026-09-20T12:21:13.377Z level=INFO msg="All 1 paths already cached"4837worker2 # [ 56.921343] nix-grpc-daemon[855]: ts=2026-09-20T12:21:13Z level=debug event=want client=7 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=04838worker2 # [ 56.925883] nix-grpc-daemon[855]: ts=2026-09-20T12:21:13Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.3:50051 clients=14839worker1 # [ 56.935200] nix-grpc-daemon[852]: ts=2026-09-20T12:21:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:558304840worker1 # [ 56.948705] nix-grpc-daemon[852]: ts=2026-09-20T12:21:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:558304841worker1 # [ 56.965434] nix-grpc-daemon[920]: time=2026-09-20T12:21:13.468Z level=INFO msg="All 1 paths already cached"4842worker1 # [ 56.971111] nix-grpc-daemon[852]: ts=2026-09-20T12:21:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:558304843worker1 # [ 56.977096] nix-daemon[909]: accepted connection from pid 852, user nix-grpc-daemon (trusted)4844worker1 # [ 57.150127] nix-daemon[909]: accepted connection from pid 1139, user nix-grpc-daemon (trusted)4845worker1 # [ 57.164804] nix-daemon[909]: reaped child process 1142, status = succeeded4846worker1 # [ 57.327854] nix-daemon[909]: accepted connection from pid 1144, user nix-grpc-daemon (trusted)4847worker1 # [ 57.341719] nix-daemon[909]: reaped child process 1147, status = succeeded4848lb # [ 57.361851] niks3-server[1111]: 2026/09/20 12:21:13 INFO Received uploads request method=POST path=/api/pending_closures4849worker1 # [ 57.362370] nix-grpc-daemon[920]: time=2026-09-20T12:21:13.865Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4850worker1 # [ 57.364135] nix-grpc-daemon[920]: time=2026-09-20T12:21:13.866Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"4851lb # [ 57.390993] niks3-server[1111]: 2026/09/20 12:21:14 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst4852lb # [ 57.403749] niks3-server[1111]: 2026/09/20 12:21:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4853worker1 # [ 57.400517] nix-grpc-daemon[920]: time=2026-09-20T12:21:13.903Z level=INFO msg="Uploading 1 narinfos"4854lb # [ 57.409144] niks3-server[1111]: 2026/09/20 12:21:14 INFO Signed narinfos id=13 count=14855lb # [ 57.416072] niks3-server[1111]: 2026/09/20 12:21:14 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls4856lb # [ 57.426345] niks3-server[1111]: 2026/09/20 12:21:14 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4857worker1 # [ 57.423375] nix-grpc-daemon[920]: time=2026-09-20T12:21:13.926Z level=INFO msg="Upload complete. (439ms)"4858lb # [ 57.431912] niks3-server[1111]: 2026/09/20 12:21:14 INFO Completed upload id=134859worker1 # [ 57.427089] nix-grpc-daemon[852]: ts=2026-09-20T12:21:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55830 duration_s=0 paths=1 nar_bytes_in=6244860lb # [ 57.437937] niks3-server[1111]: 2026/09/20 12:21:14 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo4861worker1 # [ 57.430255] nix-grpc-daemon[852]: ts=2026-09-20T12:21:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:558304862worker1 # [ 57.436137] nix-daemon[909]: accepted connection from pid 852, user nix-grpc-daemon (trusted)4863worker1 # [ 57.442718] nix-daemon[909]: reaped child process 1137, status = succeeded4864worker1 # [ 57.449087] nix-daemon[909]: accepted connection from pid 852, user nix-grpc-daemon (trusted)4865worker1 # [ 58.676697] nix-daemon[909]: reaped child process 1151, status = succeeded4866worker1 # [ 58.829583] nix-daemon[909]: accepted connection from pid 1166, user nix-grpc-daemon (trusted)4867worker1 # [ 58.842360] nix-daemon[909]: reaped child process 1169, status = succeeded4868lb # [ 58.867681] niks3-server[1111]: 2026/09/20 12:21:15 INFO Received uploads request method=POST path=/api/pending_closures4869worker1 # [ 58.976858] nix-grpc-daemon[920]: time=2026-09-20T12:21:15.479Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4870worker1 # [ 58.983482] nix-grpc-daemon[920]: time=2026-09-20T12:21:15.483Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"4871lb # [ 59.021863] niks3-server[1111]: 2026/09/20 12:21:15 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv4872lb # [ 59.032200] niks3-server[1111]: 2026/09/20 12:21:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4873lb # [ 59.038289] niks3-server[1111]: 2026/09/20 12:21:15 INFO Signed narinfos id=14 count=14874worker1 # [ 59.029836] nix-grpc-daemon[920]: time=2026-09-20T12:21:15.532Z level=INFO msg="Uploading 1 narinfos"4875lb # [ 59.049415] niks3-server[1111]: 2026/09/20 12:21:15 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst4876lb # [ 59.057556] niks3-server[1111]: 2026/09/20 12:21:15 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls4877lb # [ 59.063176] niks3-server[1111]: 2026/09/20 12:21:15 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4878lb # [ 59.102310] niks3-server[1111]: 2026/09/20 12:21:15 INFO Completed upload id=144879worker1 # [ 59.093928] nix-grpc-daemon[920]: time=2026-09-20T12:21:15.596Z level=INFO msg="Upload complete. (435ms)"4880lb # [ 59.104717] niks3-server[1111]: 2026/09/20 12:21:15 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo4881worker1 # [ 59.099985] nix-grpc-daemon[852]: ts=2026-09-20T12:21:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55830 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=7 outputs=14882worker1 # [ 59.107752] nix-grpc-daemon[852]: ts=2026-09-20T12:21:15Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:558304883worker1 # [ 59.118543] nix-daemon[909]: reaped child process 1149, status = succeeded4884worker1 # [ 59.122833] nix-grpc-daemon[852]: ts=2026-09-20T12:21:15Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:558304885worker1 # [ 59.126575] nix-grpc-daemon[852]: ts=2026-09-20T12:21:15Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:55830 duration_s=0 paths=1 nar_bytes_out=1764886worker2 # [ 59.361377] nix-grpc-daemon[855]: ts=2026-09-20T12:21:15Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:558424887worker2 # [ 59.367827] nix-grpc-daemon[855]: ts=2026-09-20T12:21:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:558424888worker2 # [ 59.384350] nix-grpc-daemon[975]: time=2026-09-20T12:21:15.856Z level=INFO msg="All 1 paths already cached"4889worker2 # [ 59.398624] nix-grpc-daemon[855]: ts=2026-09-20T12:21:15Z level=debug event=want client=8 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=04890worker1 # [ 59.408138] nix-grpc-daemon[852]: ts=2026-09-20T12:21:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:558304891worker2 # [ 59.403161] nix-grpc-daemon[855]: ts=2026-09-20T12:21:15Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.3:50051 clients=14892worker1 # [ 59.419141] nix-grpc-daemon[852]: ts=2026-09-20T12:21:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:558304893worker1 # [ 59.429435] nix-grpc-daemon[920]: time=2026-09-20T12:21:15.932Z level=INFO msg="All 2 paths already cached"4894worker1 # [ 59.436229] nix-grpc-daemon[852]: ts=2026-09-20T12:21:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:558304895worker1 # [ 59.445872] nix-daemon[909]: accepted connection from pid 852, user nix-grpc-daemon (trusted)4896worker1 # [ 59.619958] nix-daemon[909]: accepted connection from pid 1173, user nix-grpc-daemon (trusted)4897worker1 # [ 59.634161] nix-daemon[909]: reaped child process 1176, status = succeeded4898worker1 # [ 59.803961] nix-daemon[909]: accepted connection from pid 1178, user nix-grpc-daemon (trusted)4899worker1 # [ 59.817216] nix-daemon[909]: reaped child process 1181, status = succeeded4900lb # [ 59.840736] niks3-server[1111]: 2026/09/20 12:21:16 INFO Received uploads request method=POST path=/api/pending_closures4901worker1 # [ 59.839776] nix-grpc-daemon[920]: time=2026-09-20T12:21:16.342Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4902worker1 # [ 59.841296] nix-grpc-daemon[920]: time=2026-09-20T12:21:16.344Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"4903lb # [ 59.871142] niks3-server[1111]: 2026/09/20 12:21:16 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst4904lb # [ 59.878870] niks3-server[1111]: 2026/09/20 12:21:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4905lb # [ 59.883104] niks3-server[1111]: 2026/09/20 12:21:16 INFO Signed narinfos id=15 count=14906worker1 # [ 59.877417] nix-grpc-daemon[920]: time=2026-09-20T12:21:16.380Z level=INFO msg="Uploading 1 narinfos"4907lb # [ 59.895885] niks3-server[1111]: 2026/09/20 12:21:16 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls4908lb # [ 59.901458] niks3-server[1111]: 2026/09/20 12:21:16 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4909lb # [ 59.906324] niks3-server[1111]: 2026/09/20 12:21:16 INFO Completed upload id=154910worker1 # [ 59.900094] nix-grpc-daemon[920]: time=2026-09-20T12:21:16.403Z level=INFO msg="Upload complete. (439ms)"4911lb # [ 59.911659] niks3-server[1111]: 2026/09/20 12:21:16 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo4912worker1 # [ 59.902751] nix-grpc-daemon[852]: ts=2026-09-20T12:21:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55830 duration_s=0 paths=1 nar_bytes_in=7524913worker1 # [ 59.908390] nix-grpc-daemon[852]: ts=2026-09-20T12:21:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:558304914worker1 # [ 59.916434] nix-daemon[909]: accepted connection from pid 852, user nix-grpc-daemon (trusted)4915worker1 # [ 59.922075] nix-daemon[909]: reaped child process 1171, status = succeeded4916worker1 # [ 59.926729] nix-daemon[909]: accepted connection from pid 852, user nix-grpc-daemon (trusted)4917lb # [ 60.769864] envoy[1009]: conn peer=[2001:db8:1::1]:43654 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2972 tx=1372 ms=46694918lb # [ 60.771440] envoy[1009]: conn peer=[2001:db8:1::1]:43662 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1427 tx=1085 ms=35574919lb # [ 60.772873] envoy[1009]: conn peer=[2001:db8:1::1]:54424 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=264920lb # [ 60.774271] envoy[1009]: conn peer=[2001:db8:1::1]:54416 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3387 tx=1769 ms=23264921lb # [ 60.775672] envoy[1009]: conn peer=[2001:db8:1::1]:54432 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3897 tx=1894 ms=23874922lb # [ 60.777162] envoy[1009]: conn peer=[2001:db8:1::1]:54448 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=378 ms=514923worker1 # [ 61.349973] nix-daemon[909]: reaped child process 1185, status = succeeded4924worker1 # [ 61.510497] nix-daemon[909]: accepted connection from pid 1200, user nix-grpc-daemon (trusted)4925worker1 # [ 61.524280] nix-daemon[909]: reaped child process 1203, status = succeeded4926lb # [ 61.549774] niks3-server[1111]: 2026/09/20 12:21:18 INFO Received uploads request method=POST path=/api/pending_closures4927worker1 # [ 61.550040] nix-grpc-daemon[920]: time=2026-09-20T12:21:18.052Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4928worker1 # [ 61.551567] nix-grpc-daemon[920]: time=2026-09-20T12:21:18.054Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"4929lb # [ 61.588759] niks3-server[1111]: 2026/09/20 12:21:18 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst4930lb # [ 61.600260] niks3-server[1111]: 2026/09/20 12:21:18 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv4931lb # [ 61.608556] niks3-server[1111]: 2026/09/20 12:21:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4932lb # [ 61.613516] niks3-server[1111]: 2026/09/20 12:21:18 INFO Signed narinfos id=16 count=14933worker1 # [ 61.607354] nix-grpc-daemon[920]: time=2026-09-20T12:21:18.110Z level=INFO msg="Uploading 1 narinfos"4934lb # [ 61.618864] niks3-server[1111]: 2026/09/20 12:21:18 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls4935lb # [ 61.627115] niks3-server[1111]: 2026/09/20 12:21:18 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4936lb # [ 61.633971] niks3-server[1111]: 2026/09/20 12:21:18 INFO Completed upload id=164937worker1 # [ 61.625208] nix-grpc-daemon[920]: time=2026-09-20T12:21:18.128Z level=INFO msg="Upload complete. (290ms)"4938lb # [ 61.636848] niks3-server[1111]: 2026/09/20 12:21:18 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo4939worker1 # [ 61.628308] nix-grpc-daemon[852]: ts=2026-09-20T12:21:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55830 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=8 outputs=14940worker1 # [ 61.632089] nix-grpc-daemon[852]: ts=2026-09-20T12:21:18Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:558304941worker1 # [ 61.638430] nix-daemon[909]: reaped child process 1183, status = succeeded4942worker1 # [ 61.640388] nix-grpc-daemon[852]: ts=2026-09-20T12:21:18Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:558304943worker1 # [ 61.642129] nix-grpc-daemon[852]: ts=2026-09-20T12:21:18Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:55830 duration_s=0 paths=1 nar_bytes_out=2324944client # [ 61.714529] nix-daemon[958]: reaped child process 960, status = succeeded4945client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log, in 8.03 seconds)4946client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top4947client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.02 seconds)4948client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4949client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.03 seconds)4950client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4951client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.02 seconds)4952(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 8.10 seconds)4953subtest: an input only one worker has still reaches the builder4954client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only4955client # this derivation will be built:4956client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv4957client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...4958client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only, in 0.37 seconds)4959client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure4960client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.18 seconds)4961worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4962worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.23 seconds)4963worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only4964worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.03 seconds)4965client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&24966client # [ 62.781717] nix-daemon[958]: accepted connection from pid 1041, user root (trusted)4967client # this derivation will be built:4968client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv4969worker2 # [ 62.958112] nix-grpc-daemon[855]: ts=2026-09-20T12:21:19Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:558424970worker2 # [ 62.960811] nix-grpc-daemon[855]: ts=2026-09-20T12:21:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:558424971client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...4972client # copying 1 paths...4973client # copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' to 'grpc://lb:50051'...4974worker1 # [ 62.975033] nix-grpc-daemon[852]: ts=2026-09-20T12:21:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:558304975worker1 # [ 62.983227] nix-daemon[909]: accepted connection from pid 852, user nix-grpc-daemon (trusted)4976worker1 # [ 63.144364] nix-daemon[909]: accepted connection from pid 1212, user nix-grpc-daemon (trusted)4977worker1 # [ 63.157335] nix-daemon[909]: reaped child process 1215, status = succeeded4978lb # [ 63.182701] niks3-server[1111]: 2026/09/20 12:21:19 INFO Received uploads request method=POST path=/api/pending_closures4979worker1 # [ 63.182359] nix-grpc-daemon[920]: time=2026-09-20T12:21:19.685Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4980worker1 # [ 63.184298] nix-grpc-daemon[920]: time=2026-09-20T12:21:19.687Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"4981lb # [ 63.211761] niks3-server[1111]: 2026/09/20 12:21:19 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4982lb # [ 63.221226] niks3-server[1111]: 2026/09/20 12:21:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4983lb # [ 63.227605] niks3-server[1111]: 2026/09/20 12:21:19 INFO Signed narinfos id=17 count=14984worker1 # [ 63.222065] nix-grpc-daemon[920]: time=2026-09-20T12:21:19.724Z level=INFO msg="Uploading 1 narinfos"4985lb # [ 63.232385] niks3-server[1111]: 2026/09/20 12:21:19 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls4986lb # [ 63.243815] niks3-server[1111]: 2026/09/20 12:21:19 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4987worker1 # [ 63.241594] nix-grpc-daemon[920]: time=2026-09-20T12:21:19.744Z level=INFO msg="Upload complete. (250ms)"4988lb # [ 63.250284] niks3-server[1111]: 2026/09/20 12:21:19 INFO Completed upload id=174989worker1 # [ 63.243659] nix-grpc-daemon[852]: ts=2026-09-20T12:21:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55830 duration_s=0 paths=1 nar_bytes_in=1204990lb # [ 63.253755] niks3-server[1111]: 2026/09/20 12:21:19 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo4991worker1 # [ 63.250066] nix-daemon[909]: reaped child process 1210, status = succeeded4992worker2 # [ 63.251306] nix-grpc-daemon[855]: ts=2026-09-20T12:21:19Z level=debug event=want client=9 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=04993worker2 # [ 63.257248] nix-grpc-daemon[855]: ts=2026-09-20T12:21:19Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=14994worker2 # [ 63.299224] nix-grpc-daemon[855]: ts=2026-09-20T12:21:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:558424995worker2 # [ 63.309726] nix-grpc-daemon[855]: ts=2026-09-20T12:21:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:558424996client # copying 2 paths to 'lb:50051'...4997client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...4998worker2 # [ 63.324107] nix-grpc-daemon[855]: ts=2026-09-20T12:21:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:558424999client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5000worker2 # [ 63.338126] nix-daemon[924]: accepted connection from pid 855, user nix-grpc-daemon (trusted)5001worker2 # [ 63.500433] nix-daemon[924]: accepted connection from pid 1059, user nix-grpc-daemon (trusted)5002worker2 # [ 63.516142] nix-daemon[924]: reaped child process 1062, status = succeeded5003worker2 # [ 63.683108] nix-daemon[924]: accepted connection from pid 1064, user nix-grpc-daemon (trusted)5004worker2 # [ 63.695609] nix-daemon[924]: reaped child process 1067, status = succeeded5005lb # [ 63.726181] niks3-server[1111]: 2026/09/20 12:21:20 INFO Received uploads request method=POST path=/api/pending_closures5006lb # [ 63.733968] niks3-server[1111]: 2026/09/20 12:21:20 INFO Received uploads request method=POST path=/api/pending_closures5007worker2 # [ 63.726292] nix-grpc-daemon[975]: time=2026-09-20T12:21:20.199Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5008worker2 # [ 63.727922] nix-grpc-daemon[975]: time=2026-09-20T12:21:20.201Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5009worker2 # [ 63.729723] nix-grpc-daemon[975]: time=2026-09-20T12:21:20.202Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5010lb # [ 63.766511] niks3-server[1111]: 2026/09/20 12:21:20 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5011lb # [ 63.780065] niks3-server[1111]: 2026/09/20 12:21:20 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5012lb # [ 63.793329] niks3-server[1111]: 2026/09/20 12:21:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5013lb # [ 63.797226] niks3-server[1111]: 2026/09/20 12:21:20 INFO Signed narinfos id=19 count=25014lb # [ 63.803341] niks3-server[1111]: 2026/09/20 12:21:20 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5015lb # [ 63.805838] niks3-server[1111]: 2026/09/20 12:21:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5016lb # [ 63.808271] niks3-server[1111]: 2026/09/20 12:21:20 INFO Signed narinfos id=18 count=15017lb # [ 63.810274] niks3-server[1111]: 2026/09/20 12:21:20 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5018worker2 # [ 63.794149] nix-grpc-daemon[975]: time=2026-09-20T12:21:20.266Z level=INFO msg="Uploading 3 narinfos"5019lb # [ 63.822908] niks3-server[1111]: 2026/09/20 12:21:20 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5020lb # [ 63.825343] niks3-server[1111]: 2026/09/20 12:21:20 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5021lb # [ 63.828316] niks3-server[1111]: 2026/09/20 12:21:20 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5022lb # [ 63.831863] niks3-server[1111]: 2026/09/20 12:21:20 INFO Completed upload id=185023lb # [ 63.833439] niks3-server[1111]: 2026/09/20 12:21:20 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5024lb # [ 63.835527] niks3-server[1111]: 2026/09/20 12:21:20 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5025lb # [ 63.837656] niks3-server[1111]: 2026/09/20 12:21:20 INFO Completed upload id=195026worker2 # [ 63.823911] nix-grpc-daemon[975]: time=2026-09-20T12:21:20.296Z level=INFO msg="Upload complete. (470ms)"5027worker2 # [ 63.829200] nix-grpc-daemon[855]: ts=2026-09-20T12:21:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55842 duration_s=0 paths=2 nar_bytes_in=9765028worker2 # [ 63.834974] nix-grpc-daemon[855]: ts=2026-09-20T12:21:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:558425029worker2 # [ 63.847153] nix-daemon[924]: accepted connection from pid 855, user nix-grpc-daemon (trusted)5030worker2 # [ 63.858812] nix-daemon[924]: reaped child process 1057, status = succeeded5031worker2 # [ 63.863381] nix-grpc-daemon[855]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...5032client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5033worker2 # [ 63.872191] nix-daemon[924]: accepted connection from pid 855, user nix-grpc-daemon (trusted)5034client # uses-input-w1onlya> worker2: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5035worker2 # [ 64.486965] nix-daemon[924]: reaped child process 1072, status = succeeded5036worker2 # [ 64.605300] nix-daemon[924]: accepted connection from pid 1087, user nix-grpc-daemon (trusted)5037worker2 # [ 64.618799] nix-daemon[924]: reaped child process 1090, status = succeeded5038lb # [ 64.643711] niks3-server[1111]: 2026/09/20 12:21:21 INFO Received uploads request method=POST path=/api/pending_closures5039worker2 # [ 64.633583] nix-grpc-daemon[975]: time=2026-09-20T12:21:21.106Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5040lb # [ 64.659068] niks3-server[1111]: 2026/09/20 12:21:21 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5041lb # [ 64.661527] niks3-server[1111]: 2026/09/20 12:21:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5042lb # [ 64.664403] niks3-server[1111]: 2026/09/20 12:21:21 INFO Signed narinfos id=20 count=15043worker2 # [ 64.649023] nix-grpc-daemon[975]: time=2026-09-20T12:21:21.122Z level=INFO msg="Uploading 1 narinfos"5044lb # [ 64.670321] niks3-server[1111]: 2026/09/20 12:21:21 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5045lb # [ 64.677113] niks3-server[1111]: 2026/09/20 12:21:21 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5046lb # [ 64.680409] niks3-server[1111]: 2026/09/20 12:21:21 INFO Completed upload id=205047worker2 # [ 64.665215] nix-grpc-daemon[975]: time=2026-09-20T12:21:21.138Z level=INFO msg="Upload complete. (186ms)"5048lb # [ 64.683291] niks3-server[1111]: 2026/09/20 12:21:21 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5049worker2 # [ 64.669244] nix-grpc-daemon[855]: ts=2026-09-20T12:21:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55842 duration_s=0 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=9 outputs=15050worker2 # [ 64.675837] nix-daemon[924]: reaped child process 1069, status = succeeded5051client # copying 1 paths...5052worker2 # [ 64.711051] nix-grpc-daemon[855]: ts=2026-09-20T12:21:21Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:558425053client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5054worker1 # [ 64.728801] nix-grpc-daemon[852]: ts=2026-09-20T12:21:21Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:558305055worker1 # [ 64.736146] nix-grpc-daemon[852]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5056worker1 # [ 64.744691] nix-grpc-daemon[852]: ts=2026-09-20T12:21:21Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:55830 duration_s=0 paths=1 nar_bytes_out=1205057client # [ 64.830128] nix-daemon[958]: reaped child process 1044, status = succeeded5058client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.24 seconds)5059client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&25060client # [ 64.997475] nix-daemon[958]: accepted connection from pid 1063, user root (trusted)5061client # this derivation will be built:5062client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5063worker1 # [ 65.186503] nix-grpc-daemon[852]: ts=2026-09-20T12:21:21Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:558305064client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...5065worker1 # [ 65.192483] nix-grpc-daemon[852]: ts=2026-09-20T12:21:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:558305066worker1 # [ 65.197323] nix-grpc-daemon[920]: time=2026-09-20T12:21:21.699Z level=INFO msg="All 1 paths already cached"5067worker2 # [ 65.205920] nix-grpc-daemon[855]: ts=2026-09-20T12:21:21Z level=debug event=want client=10 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05068worker1 # [ 65.214773] nix-grpc-daemon[852]: ts=2026-09-20T12:21:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:558305069worker2 # [ 65.210586] nix-grpc-daemon[855]: ts=2026-09-20T12:21:21Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15070worker1 # [ 65.226185] nix-grpc-daemon[852]: ts=2026-09-20T12:21:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:558305071client # copying 1 paths to 'lb:50051'...5072client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5073worker1 # [ 65.238609] nix-grpc-daemon[852]: ts=2026-09-20T12:21:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:558305074worker1 # [ 65.250327] nix-daemon[909]: accepted connection from pid 852, user nix-grpc-daemon (trusted)5075worker1 # [ 65.265171] nix-grpc-daemon[852]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 41 ms (attempt 1/5)5076worker1 # [ 65.306406] nix-grpc-daemon[852]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 183 ms (attempt 2/5)5077worker1 # [ 65.495105] nix-grpc-daemon[852]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 364 ms (attempt 3/5)5078worker1 # [ 65.864893] nix-grpc-daemon[852]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 189 ms (attempt 4/5)5079worker1 # [ 66.059735] nix-grpc-daemon[852]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5080worker1 # [ 66.062335] nix-grpc-daemon[852]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5081worker1 # [ 66.237374] nix-daemon[909]: accepted connection from pid 1224, user nix-grpc-daemon (trusted)5082worker1 # [ 66.252911] nix-daemon[909]: reaped child process 1227, status = succeeded5083worker1 # [ 66.421555] nix-daemon[909]: accepted connection from pid 1229, user nix-grpc-daemon (trusted)5084worker1 # [ 66.436191] nix-daemon[909]: reaped child process 1232, status = succeeded5085lb # [ 66.462614] niks3-server[1111]: 2026/09/20 12:21:23 INFO Received uploads request method=POST path=/api/pending_closures5086worker1 # [ 66.458868] nix-grpc-daemon[920]: time=2026-09-20T12:21:22.961Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5087worker1 # [ 66.461296] nix-grpc-daemon[920]: time=2026-09-20T12:21:22.963Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5088lb # [ 66.480579] niks3-server[1111]: 2026/09/20 12:21:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5089lb # [ 66.483956] niks3-server[1111]: 2026/09/20 12:21:23 INFO Signed narinfos id=21 count=15090worker1 # [ 66.475084] nix-grpc-daemon[920]: time=2026-09-20T12:21:22.977Z level=INFO msg="Uploading 1 narinfos"5091lb # [ 66.485019] niks3-server[1111]: 2026/09/20 12:21:23 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5092lb # [ 66.488451] niks3-server[1111]: 2026/09/20 12:21:23 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5093lb # [ 66.493428] niks3-server[1111]: 2026/09/20 12:21:23 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5094lb # [ 66.497025] niks3-server[1111]: 2026/09/20 12:21:23 INFO Completed upload id=215095worker1 # [ 66.488333] nix-grpc-daemon[920]: time=2026-09-20T12:21:22.991Z level=INFO msg="Upload complete. (410ms)"5096lb # [ 66.498680] niks3-server[1111]: 2026/09/20 12:21:23 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5097worker1 # [ 66.490359] nix-grpc-daemon[852]: ts=2026-09-20T12:21:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55830 duration_s=1 paths=1 nar_bytes_in=5605098worker1 # [ 66.494991] nix-grpc-daemon[852]: ts=2026-09-20T12:21:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:558305099worker1 # [ 66.501550] nix-daemon[909]: accepted connection from pid 852, user nix-grpc-daemon (trusted)5100worker1 # [ 66.503728] nix-daemon[909]: reaped child process 1218, status = succeeded5101worker1 # [ 66.508939] nix-daemon[909]: accepted connection from pid 852, user nix-grpc-daemon (trusted)5102client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5103client # uses-input-w1onlyb> node-a: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5104worker1 # [ 68.136289] nix-daemon[909]: reaped child process 1236, status = succeeded5105worker1 # [ 68.305041] nix-daemon[909]: accepted connection from pid 1251, user nix-grpc-daemon (trusted)5106worker1 # [ 68.316874] nix-daemon[909]: reaped child process 1254, status = succeeded5107lb # [ 68.338385] niks3-server[1111]: 2026/09/20 12:21:24 INFO Received uploads request method=POST path=/api/pending_closures5108worker1 # [ 68.338024] nix-grpc-daemon[920]: time=2026-09-20T12:21:24.840Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5109lb # [ 68.369250] niks3-server[1111]: 2026/09/20 12:21:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5110worker1 # [ 68.365842] nix-grpc-daemon[920]: time=2026-09-20T12:21:24.868Z level=INFO msg="Uploading 1 narinfos"5111lb # [ 68.375475] niks3-server[1111]: 2026/09/20 12:21:25 INFO Signed narinfos id=22 count=15112lb # [ 68.381882] niks3-server[1111]: 2026/09/20 12:21:25 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5113lb # [ 68.391761] niks3-server[1111]: 2026/09/20 12:21:25 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5114lb # [ 68.399825] niks3-server[1111]: 2026/09/20 12:21:25 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5115lb # [ 68.406911] niks3-server[1111]: 2026/09/20 12:21:25 INFO Completed upload id=225116worker1 # [ 68.398319] nix-grpc-daemon[920]: time=2026-09-20T12:21:24.901Z level=INFO msg="Upload complete. (279ms)"5117lb # [ 68.409374] niks3-server[1111]: 2026/09/20 12:21:25 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5118worker1 # [ 68.404239] nix-grpc-daemon[852]: ts=2026-09-20T12:21:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55830 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=10 outputs=15119worker1 # [ 68.416584] nix-daemon[909]: reaped child process 1234, status = succeeded5120client # copying 1 paths...5121worker2 # [ 68.445181] nix-grpc-daemon[855]: ts=2026-09-20T12:21:24Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:558425122worker2 # [ 68.450041] nix-grpc-daemon[855]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5123client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5124worker2 # [ 68.476231] nix-grpc-daemon[855]: ts=2026-09-20T12:21:24Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:558425125worker2 # [ 68.478353] nix-grpc-daemon[855]: ts=2026-09-20T12:21:24Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:55842 duration_s=0 paths=1 nar_bytes_out=1205126client # [ 68.555663] nix-daemon[958]: reaped child process 1066, status = succeeded5127client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.73 seconds)5128client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&25129client # [ 68.720380] nix-daemon[958]: accepted connection from pid 1085, user root (trusted)5130client # this derivation will be built:5131client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5132worker1 # [ 68.913164] nix-grpc-daemon[852]: ts=2026-09-20T12:21:25Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:558305133worker2 # [ 68.914734] nix-grpc-daemon[855]: ts=2026-09-20T12:21:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:558425134client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...5135worker2 # [ 68.920904] nix-grpc-daemon[975]: time=2026-09-20T12:21:25.393Z level=INFO msg="All 1 paths already cached"5136worker2 # [ 68.935157] nix-grpc-daemon[855]: ts=2026-09-20T12:21:25Z level=debug event=want client=11 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05137worker2 # [ 68.941584] nix-grpc-daemon[855]: ts=2026-09-20T12:21:25Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15138worker2 # [ 68.946514] nix-grpc-daemon[855]: ts=2026-09-20T12:21:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:558425139worker2 # [ 68.956536] nix-grpc-daemon[855]: ts=2026-09-20T12:21:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:558425140worker2 # [ 68.967151] nix-grpc-daemon[975]: time=2026-09-20T12:21:25.440Z level=INFO msg="All 1 paths already cached"5141client # copying 1 paths to 'lb:50051'...5142client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5143worker2 # [ 68.973973] nix-grpc-daemon[855]: ts=2026-09-20T12:21:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:558425144worker2 # [ 68.985297] nix-daemon[924]: accepted connection from pid 855, user nix-grpc-daemon (trusted)5145worker2 # [ 69.164107] nix-daemon[924]: accepted connection from pid 1099, user nix-grpc-daemon (trusted)5146worker2 # [ 69.178340] nix-daemon[924]: reaped child process 1102, status = succeeded5147worker2 # [ 69.352061] nix-daemon[924]: accepted connection from pid 1104, user nix-grpc-daemon (trusted)5148worker2 # [ 69.365742] nix-daemon[924]: reaped child process 1107, status = succeeded5149lb # [ 69.397799] niks3-server[1111]: 2026/09/20 12:21:26 INFO Received uploads request method=POST path=/api/pending_closures5150worker2 # [ 69.389896] nix-grpc-daemon[975]: time=2026-09-20T12:21:25.863Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5151worker2 # [ 69.391504] nix-grpc-daemon[975]: time=2026-09-20T12:21:25.864Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5152lb # [ 69.423995] niks3-server[1111]: 2026/09/20 12:21:26 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5153lb # [ 69.437650] niks3-server[1111]: 2026/09/20 12:21:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5154worker2 # [ 69.426801] nix-grpc-daemon[975]: time=2026-09-20T12:21:25.899Z level=INFO msg="Uploading 1 narinfos"5155lb # [ 69.442623] niks3-server[1111]: 2026/09/20 12:21:26 INFO Signed narinfos id=23 count=15156lb # [ 69.449125] niks3-server[1111]: 2026/09/20 12:21:26 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5157lb # [ 69.459488] niks3-server[1111]: 2026/09/20 12:21:26 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5158worker2 # [ 69.449894] nix-grpc-daemon[975]: time=2026-09-20T12:21:25.922Z level=INFO msg="Upload complete. (454ms)"5159lb # [ 69.465157] niks3-server[1111]: 2026/09/20 12:21:26 INFO Completed upload id=235160lb # [ 69.468597] niks3-server[1111]: 2026/09/20 12:21:26 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5161worker2 # [ 69.454934] nix-grpc-daemon[855]: ts=2026-09-20T12:21:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55842 duration_s=0 paths=1 nar_bytes_in=5605162worker2 # [ 69.459710] nix-grpc-daemon[855]: ts=2026-09-20T12:21:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:558425163worker2 # [ 69.469388] nix-daemon[924]: accepted connection from pid 855, user nix-grpc-daemon (trusted)5164worker2 # [ 69.475299] nix-daemon[924]: reaped child process 1097, status = succeeded5165client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5166worker2 # [ 69.478344] nix-daemon[924]: accepted connection from pid 855, user nix-grpc-daemon (trusted)5167client # uses-input-w1onlyc> worker2: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5168worker2 # [ 70.305717] nix-daemon[924]: reaped child process 1111, status = succeeded5169worker2 # [ 70.443158] nix-daemon[924]: accepted connection from pid 1126, user nix-grpc-daemon (trusted)5170worker2 # [ 70.457132] nix-daemon[924]: reaped child process 1129, status = succeeded5171lb # [ 70.487755] niks3-server[1111]: 2026/09/20 12:21:27 INFO Received uploads request method=POST path=/api/pending_closures5172worker2 # [ 70.477796] nix-grpc-daemon[975]: time=2026-09-20T12:21:26.950Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5173lb # [ 70.505092] niks3-server[1111]: 2026/09/20 12:21:27 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5174lb # [ 70.508884] niks3-server[1111]: 2026/09/20 12:21:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5175worker2 # [ 70.496182] nix-grpc-daemon[975]: time=2026-09-20T12:21:26.969Z level=INFO msg="Uploading 1 narinfos"5176lb # [ 70.511554] niks3-server[1111]: 2026/09/20 12:21:27 INFO Signed narinfos id=24 count=15177lb # [ 70.513783] niks3-server[1111]: 2026/09/20 12:21:27 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5178lb # [ 70.519069] niks3-server[1111]: 2026/09/20 12:21:27 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5179lb # [ 70.522481] niks3-server[1111]: 2026/09/20 12:21:27 INFO Completed upload id=245180worker2 # [ 70.507410] nix-grpc-daemon[975]: time=2026-09-20T12:21:26.980Z level=INFO msg="Upload complete. (219ms)"5181lb # [ 70.525512] niks3-server[1111]: 2026/09/20 12:21:27 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5182worker2 # [ 70.510324] nix-grpc-daemon[855]: ts=2026-09-20T12:21:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55842 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=11 outputs=15183worker2 # [ 70.517353] nix-daemon[924]: reaped child process 1109, status = succeeded5184client # copying 1 paths...5185worker1 # [ 70.562733] nix-grpc-daemon[852]: ts=2026-09-20T12:21:27Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:558305186worker1 # [ 70.571824] nix-grpc-daemon[852]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5187client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5188worker1 # [ 70.600077] nix-grpc-daemon[852]: ts=2026-09-20T12:21:27Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:558305189worker1 # [ 70.605300] nix-grpc-daemon[852]: ts=2026-09-20T12:21:27Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:55830 duration_s=0 paths=1 nar_bytes_out=1205190client # [ 70.693252] nix-daemon[958]: reaped child process 1088, status = succeeded5191client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.13 seconds)5192(finished: subtest: an input only one worker has still reaches the builder, in 8.90 seconds)5193subtest: upload whose reference the worker lacks is completed from the cache5194client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache5195lb # [ 70.771124] envoy[1009]: conn peer=[2001:db8:1::1]:54460 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4328 tx=2109 ms=23285196lb # [ 70.774851] envoy[1009]: conn peer=[2001:db8:1::1]:54474 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=485197lb # [ 70.778334] envoy[1009]: conn peer=[2001:db8:1::1]:42060 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4193 tx=1730 ms=18305198lb # [ 70.781891] envoy[1009]: conn peer=[2001:db8:1::1]:42068 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=329 ms=635199lb # [ 70.785379] envoy[1009]: conn peer=[2001:db8:1::1]:42080 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3699 tx=1911 ms=33485200lb # [ 70.789082] envoy[1009]: conn peer=[2001:db8:1::1]:42082 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=545201lb # [ 70.792486] envoy[1009]: conn peer=[2001:db8:1::1]:42090 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3714 tx=1895 ms=17445202lb # [ 70.795942] envoy[1009]: conn peer=[2001:db8:1::1]:42096 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=575203client # this derivation will be built:5204client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5205client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5206client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache, in 0.37 seconds)5207client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache5208client # this derivation will be built:5209client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5210client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5211client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache, in 0.35 seconds)5212client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5213client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.18 seconds)5214worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5215worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.20 seconds)5216worker2 # [ 71.875570] systemd[1]: Stopping Nix worker-protocol over gRPC...5217worker2 # [ 72.154754] nix-grpc-daemon[855]: ts=2026-09-20T12:21:28Z level=info event=signal_exit5218worker2 # [ 72.397618] nix-grpc-daemon[855]: ts=2026-09-20T12:21:28Z level=info event=scheduler_yield5219worker2 # [ 72.402292] nix-grpc-daemon[855]: ts=2026-09-20T12:21:28Z level=info event=scheduler_restarting addr=lb:500515220lb # [ 72.416693] niks3-server[1111]: 2026/09/20 12:21:29 INFO lead: released remote=192.168.1.4:598985221worker1 # [ 72.415405] nix-grpc-daemon[852]: ts=2026-09-20T12:21:28Z level=info event=scheduler_restarting addr=lb:500515222worker2 # [ 72.408630] nix-grpc-daemon[855]: ts=2026-09-20T12:21:28Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:558385223worker2 # [ 72.413471] nix-grpc-daemon[855]: ts=2026-09-20T12:21:28Z level=info event=worker_gone addr=192.168.1.4:500515224worker2 # [ 72.416735] nix-grpc-daemon[855]: ts=2026-09-20T12:21:28Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:558385225worker2 # [ 72.420085] nix-grpc-daemon[855]: ts=2026-09-20T12:21:28Z level=info event=worker_gone addr=192.168.1.3:500515226worker2 # [ 72.603043] nix-grpc-daemon[855]: ts=2026-09-20T12:21:29Z level=info event=scheduler_connected addr=lb:500515227worker2 # [ 72.604871] nix-grpc-daemon[855]: ts=2026-09-20T12:21:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5228worker1 # [ 72.615974] nix-grpc-daemon[852]: ts=2026-09-20T12:21:29Z level=info event=scheduler_connected addr=lb:500515229worker1 # [ 72.618575] nix-grpc-daemon[852]: ts=2026-09-20T12:21:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5230worker2 # [ 72.805956] nix-grpc-daemon[855]: ts=2026-09-20T12:21:29Z level=info event=scheduler_connected addr=lb:500515231worker2 # [ 72.810065] nix-grpc-daemon[855]: ts=2026-09-20T12:21:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5232worker1 # [ 72.819734] nix-grpc-daemon[852]: ts=2026-09-20T12:21:29Z level=info event=scheduler_connected addr=lb:500515233worker1 # [ 72.824071] nix-grpc-daemon[852]: ts=2026-09-20T12:21:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5234worker2 # [ 73.011447] nix-grpc-daemon[855]: ts=2026-09-20T12:21:29Z level=info event=scheduler_connected addr=lb:500515235worker2 # [ 73.015604] nix-grpc-daemon[855]: ts=2026-09-20T12:21:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5236worker1 # [ 73.025848] nix-grpc-daemon[852]: ts=2026-09-20T12:21:29Z level=info event=scheduler_connected addr=lb:500515237worker1 # [ 73.028976] nix-grpc-daemon[852]: ts=2026-09-20T12:21:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5238worker2 # [ 73.217525] nix-grpc-daemon[855]: ts=2026-09-20T12:21:29Z level=info event=scheduler_connected addr=lb:500515239worker2 # [ 73.220587] nix-grpc-daemon[855]: ts=2026-09-20T12:21:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5240worker1 # [ 73.227158] nix-grpc-daemon[852]: ts=2026-09-20T12:21:29Z level=info event=scheduler_connected addr=lb:500515241worker1 # [ 73.231192] nix-grpc-daemon[852]: ts=2026-09-20T12:21:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5242worker2 # [ 73.402936] nix-grpc-daemon[855]: ts=2026-09-20T12:21:29Z level=info event=scheduler_restarting5243worker1 # [ 73.433519] nix-grpc-daemon[852]: ts=2026-09-20T12:21:29Z level=info event=scheduler_connected addr=lb:500515244worker2 # [ 73.430140] nix-daemon[924]: reaped child process 926, status = succeeded5245worker2 # [ 73.431232] nix-grpc-daemon[855]: ts=2026-09-20T12:21:29Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5246worker1 # [ 73.436824] nix-grpc-daemon[852]: ts=2026-09-20T12:21:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5247worker1 # [ 73.634475] nix-grpc-daemon[852]: ts=2026-09-20T12:21:30Z level=info event=scheduler_connected addr=lb:500515248worker1 # [ 73.638423] nix-grpc-daemon[852]: ts=2026-09-20T12:21:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5249worker2 # [ 73.757531] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5250worker2 # [ 73.758787] systemd[1]: Stopped Nix worker-protocol over gRPC.5251worker2 # [ 73.760247] systemd[1]: nix-grpc-daemon.service: Consumed 1.446s CPU time over 52.868s wall clock time, 49.8M memory peak, 53.7K incoming IP traffic, 132.4K outgoing IP traffic.5252worker2 # [ 73.764677] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5253worker2 # [ 73.766441] systemd[1]: Closed Nix worker-protocol over gRPC.5254worker2 # [ 73.767837] systemd[1]: nix-grpc-daemon.socket: Consumed 360us CPU time over 1min 3.516s wall clock time, 256K memory peak, 69.2K incoming IP traffic, 57.8K outgoing IP traffic.5255lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5256lb: must succeed: curl -sf localhost:9901/clusters5257worker1 # [ 73.839494] nix-grpc-daemon[852]: ts=2026-09-20T12:21:30Z level=info event=scheduler_connected addr=lb:500515258worker1 # [ 73.843377] nix-grpc-daemon[852]: ts=2026-09-20T12:21:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5259lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5260lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.07 seconds)5261client: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25262worker1 # [ 74.042443] nix-grpc-daemon[852]: ts=2026-09-20T12:21:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:558305263worker1 # [ 74.048281] nix-grpc-daemon[852]: ts=2026-09-20T12:21:30Z level=info event=scheduler_connected addr=lb:500515264client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5265worker1 # [ 74.052585] nix-grpc-daemon[852]: ts=2026-09-20T12:21:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5266worker1 # [ 74.056365] nix-grpc-daemon[852]: ts=2026-09-20T12:21:30Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:558305267client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.23 seconds)5268worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5269worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.01 seconds)5270worker2 # [ 74.122773] systemd[1]: Listening on Nix worker-protocol over gRPC.5271worker1 # [ 74.191561] systemd[1]: Stopping Nix worker-protocol over gRPC...5272worker1 # [ 74.248843] nix-grpc-daemon[852]: ts=2026-09-20T12:21:30Z level=info event=scheduler_connected addr=lb:500515273worker1 # [ 74.252741] nix-grpc-daemon[852]: ts=2026-09-20T12:21:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5274worker2 # [ 74.270951] systemd[1]: Starting scheduler-token.service...5275lb # [ 74.366386] niks3-server[1111]: 2026/09/20 12:21:30 INFO lead: acquired remote=192.168.1.3:453065276worker2 # [ 74.353656] systemd[1]: scheduler-token.service: Deactivated successfully.5277worker2 # [ 74.355552] systemd[1]: Finished scheduler-token.service.5278worker1 # [ 74.361118] nix-grpc-daemon[852]: ts=2026-09-20T12:21:30Z level=info event=scheduler_take_over5279worker2 # [ 74.356909] systemd[1]: scheduler-token.service: Consumed 35ms CPU time over 84ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5280worker2 # [ 74.361941] systemd[1]: Starting Nix worker-protocol over gRPC...5281worker1 # [ 74.454605] nix-grpc-daemon[852]: ts=2026-09-20T12:21:30Z level=info event=scheduler_connected addr=lb:500515282worker1 # [ 74.458421] nix-grpc-daemon[852]: ts=2026-09-20T12:21:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5283worker2 # [ 74.497785] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:30Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5284worker2 # [ 74.500320] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:30Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5285worker2 # [ 74.503572] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:30Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15286worker2 # [ 74.510781] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:30Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5287worker2 # [ 74.513240] systemd[1]: Started Nix worker-protocol over gRPC.5288worker2 # [ 74.525943] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:30Z level=info event=scheduler_connected addr=lb:500515289worker2 # [ 74.527447] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5290worker1 # [ 74.659619] nix-grpc-daemon[852]: ts=2026-09-20T12:21:31Z level=info event=scheduler_connected addr=lb:500515291worker1 # [ 74.663688] nix-grpc-daemon[852]: ts=2026-09-20T12:21:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5292worker2 # [ 74.728343] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:31Z level=info event=scheduler_connected addr=lb:500515293worker2 # [ 74.730143] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5294worker1 # [ 74.864629] nix-grpc-daemon[852]: ts=2026-09-20T12:21:31Z level=info event=scheduler_connected addr=lb:500515295worker1 # [ 74.868564] nix-grpc-daemon[852]: ts=2026-09-20T12:21:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5296worker2 # [ 74.930837] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:31Z level=info event=scheduler_connected addr=lb:500515297worker2 # [ 74.932725] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5298worker1 # [ 75.039491] nix-grpc-daemon[852]: ts=2026-09-20T12:21:31Z level=info event=signal_exit5299worker1 # [ 75.069663] nix-grpc-daemon[852]: ts=2026-09-20T12:21:31Z level=info event=scheduler_connected addr=lb:500515300worker1 # [ 75.073730] nix-grpc-daemon[852]: ts=2026-09-20T12:21:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5301worker2 # [ 75.134104] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:31Z level=info event=scheduler_connected addr=lb:500515302worker2 # [ 75.138088] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5303worker1 # [ 75.274395] nix-grpc-daemon[852]: ts=2026-09-20T12:21:31Z level=info event=scheduler_connected addr=lb:500515304worker1 # [ 75.276322] nix-grpc-daemon[852]: ts=2026-09-20T12:21:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5305worker1 # [ 75.307320] nix-grpc-daemon[852]: ts=2026-09-20T12:21:31Z level=info event=scheduler_yield5306lb # [ 75.319041] niks3-server[1111]: 2026/09/20 12:21:31 INFO lead: released remote=192.168.1.3:453065307worker2 # [ 75.338992] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:31Z level=info event=scheduler_connected addr=lb:500515308worker2 # [ 75.340767] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5309worker1 # [ 75.478329] nix-grpc-daemon[852]: ts=2026-09-20T12:21:31Z level=info event=scheduler_connected addr=lb:500515310worker1 # [ 75.482193] nix-grpc-daemon[852]: ts=2026-09-20T12:21:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5311worker2 # [ 75.543153] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:32Z level=info event=scheduler_connected addr=lb:500515312worker2 # [ 75.546530] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5313worker1 # [ 75.682931] nix-grpc-daemon[852]: ts=2026-09-20T12:21:32Z level=info event=scheduler_connected addr=lb:500515314worker1 # [ 75.685468] nix-grpc-daemon[852]: ts=2026-09-20T12:21:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5315worker2 # [ 75.744501] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:32Z level=info event=scheduler_connected addr=lb:500515316worker2 # [ 75.748331] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5317worker1 # [ 75.885893] nix-grpc-daemon[852]: ts=2026-09-20T12:21:32Z level=info event=scheduler_connected addr=lb:500515318worker1 # [ 75.890616] nix-grpc-daemon[852]: ts=2026-09-20T12:21:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5319worker2 # [ 75.949635] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:32Z level=info event=scheduler_connected addr=lb:500515320worker2 # [ 75.954115] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5321worker1 # [ 76.091383] nix-grpc-daemon[852]: ts=2026-09-20T12:21:32Z level=info event=scheduler_connected addr=lb:500515322worker1 # [ 76.095206] nix-grpc-daemon[852]: ts=2026-09-20T12:21:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5323worker2 # [ 76.155106] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:32Z level=info event=scheduler_connected addr=lb:500515324worker2 # [ 76.159507] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5325worker1 # [ 76.295672] nix-grpc-daemon[852]: ts=2026-09-20T12:21:32Z level=info event=scheduler_connected addr=lb:500515326worker1 # [ 76.297877] nix-grpc-daemon[852]: ts=2026-09-20T12:21:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5327worker1 # [ 76.307812] nix-grpc-daemon[852]: ts=2026-09-20T12:21:32Z level=info event=scheduler_restarting5328worker1 # [ 76.326624] nix-daemon[909]: reaped child process 911, status = succeeded5329worker2 # [ 76.360777] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:32Z level=info event=scheduler_connected addr=lb:500515330worker2 # [ 76.364838] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5331worker1 # [ 76.513716] nix-grpc-daemon[852]: ts=2026-09-20T12:21:33Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5332worker2 # [ 76.567387] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:33Z level=info event=scheduler_connected addr=lb:500515333worker2 # [ 76.570937] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5334worker2 # [ 76.768504] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:33Z level=info event=scheduler_connected addr=lb:500515335worker2 # [ 76.772799] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5336worker2 # [ 76.974299] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:33Z level=info event=scheduler_connected addr=lb:500515337worker2 # [ 76.978261] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5338worker1 # [ 77.166228] nix-grpc-daemon[852]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5339worker1 # [ 77.167840] nix-grpc-daemon[852]: E0000 00:00:1789906893.668655 895 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true5340worker1 # [ 77.169755] nix-grpc-daemon[852]: E0000 00:00:1789906893.669204 895 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5341worker2 # [ 77.179529] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:33Z level=info event=scheduler_connected addr=lb:500515342worker2 # [ 77.183673] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5343worker2 # [ 77.385138] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:33Z level=info event=scheduler_connected addr=lb:500515344worker2 # [ 77.389369] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5345worker2 # [ 77.590226] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:34Z level=info event=scheduler_connected addr=lb:500515346worker2 # [ 77.592345] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5347worker1 # [ 77.722274] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5348worker1 # [ 77.724415] systemd[1]: Stopped Nix worker-protocol over gRPC.5349worker1 # [ 77.726248] systemd[1]: nix-grpc-daemon.service: Consumed 2.760s CPU time over 56.969s wall clock time, 58.8M memory peak, 86.2K incoming IP traffic, 159.7K outgoing IP traffic.5350worker1 # [ 77.731090] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5351worker1 # [ 77.732856] systemd[1]: Closed Nix worker-protocol over gRPC.5352worker1 # [ 77.734514] systemd[1]: nix-grpc-daemon.socket: Consumed 305us CPU time over 1min 7.786s wall clock time, 256K memory peak, 76.6K incoming IP traffic, 63.2K outgoing IP traffic.5353lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5354lb: must succeed: curl -sf localhost:9901/clusters5355worker2 # [ 77.793216] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:34Z level=info event=scheduler_connected addr=lb:500515356worker2 # [ 77.795131] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5357lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5358lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.07 seconds)5359client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25360worker2 # [ 77.981618] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:569725361worker2 # [ 77.989233] nix-daemon[924]: accepted connection from pid 1153, user nix-grpc-daemon (trusted)5362worker2 # [ 77.995682] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:34Z level=info event=scheduler_connected addr=lb:500515363worker2 # [ 77.998444] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5364worker2 # [ 78.008866] nix-grpc-daemon[1153]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 17 ms (attempt 1/5)5365worker2 # [ 78.030420] nix-grpc-daemon[1153]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 119 ms (attempt 2/5)5366worker2 # [ 78.154996] nix-grpc-daemon[1153]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 253 ms (attempt 3/5)5367worker2 # [ 78.202893] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:34Z level=info event=scheduler_connected addr=lb:500515368worker2 # [ 78.206657] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5369worker2 # [ 78.408267] nix-grpc-daemon[1153]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 198 ms (attempt 4/5)5370worker2 # [ 78.411643] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:34Z level=info event=scheduler_connected addr=lb:500515371worker2 # [ 78.413329] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5372worker2 # [ 78.613842] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:35Z level=info event=scheduler_connected addr=lb:500515373worker2 # [ 78.616175] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5374worker2 # [ 78.618057] nix-grpc-daemon[1153]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5375client # copying 2 paths...5376client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5377worker2 # [ 78.628393] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:569725378client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5379worker2 # [ 78.633582] nix-daemon[924]: accepted connection from pid 1153, user nix-grpc-daemon (trusted)5380worker2 # [ 78.646706] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:35Z level=info event=niks3_push_started pid=12095381worker2 # [ 78.814224] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:35Z level=info event=scheduler_connected addr=lb:500515382worker2 # [ 78.816072] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5383worker2 # [ 78.820330] nix-grpc-daemon[1209]: time=2026-09-20T12:21:35.292Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""5384worker2 # [ 78.972387] nix-daemon[924]: accepted connection from pid 1222, user nix-grpc-daemon (trusted)5385worker2 # [ 78.985692] nix-daemon[924]: reaped child process 1225, status = succeeded5386lb # [ 79.019789] niks3-server[1111]: 2026/09/20 12:21:35 INFO Received uploads request method=POST path=/api/pending_closures5387lb # [ 79.028921] niks3-server[1111]: 2026/09/20 12:21:35 INFO Received uploads request method=POST path=/api/pending_closures5388worker2 # [ 79.016715] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:35Z level=info event=scheduler_connected addr=lb:500515389worker2 # [ 79.018457] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5390worker2 # [ 79.021809] nix-grpc-daemon[1209]: time=2026-09-20T12:21:35.495Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5391worker2 # [ 79.023947] nix-grpc-daemon[1209]: time=2026-09-20T12:21:35.496Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5392worker2 # [ 79.025546] nix-grpc-daemon[1209]: time=2026-09-20T12:21:35.496Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5393lb # [ 79.076481] niks3-server[1111]: 2026/09/20 12:21:35 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5394lb # [ 79.091093] niks3-server[1111]: 2026/09/20 12:21:35 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5395lb # [ 79.095507] niks3-server[1111]: 2026/09/20 12:21:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5396lb # [ 79.098142] niks3-server[1111]: 2026/09/20 12:21:35 INFO Signed narinfos id=25 count=15397lb # [ 79.100301] niks3-server[1111]: 2026/09/20 12:21:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5398lb # [ 79.104264] niks3-server[1111]: 2026/09/20 12:21:35 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5399lb # [ 79.107158] niks3-server[1111]: 2026/09/20 12:21:35 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5400lb # [ 79.112091] niks3-server[1111]: 2026/09/20 12:21:35 INFO Signed narinfos id=26 count=25401worker2 # [ 79.097923] nix-grpc-daemon[1209]: time=2026-09-20T12:21:35.570Z level=INFO msg="Uploading 3 narinfos"5402lb # [ 79.128101] niks3-server[1111]: 2026/09/20 12:21:35 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5403lb # [ 79.132043] niks3-server[1111]: 2026/09/20 12:21:35 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5404lb # [ 79.134290] niks3-server[1111]: 2026/09/20 12:21:35 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5405lb # [ 79.136398] niks3-server[1111]: 2026/09/20 12:21:35 INFO Completed upload id=265406lb # [ 79.137963] niks3-server[1111]: 2026/09/20 12:21:35 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5407lb # [ 79.140065] niks3-server[1111]: 2026/09/20 12:21:35 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5408lb # [ 79.142147] niks3-server[1111]: 2026/09/20 12:21:35 INFO Completed upload id=255409worker2 # [ 79.127169] nix-grpc-daemon[1209]: time=2026-09-20T12:21:35.600Z level=INFO msg="Upload complete. (303ms)"5410worker2 # [ 79.129554] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56972 duration_s=0 paths=2 nar_bytes_in=3125411worker2 # [ 79.136563] nix-daemon[924]: reaped child process 1207, status = succeeded5412client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.35 seconds)5413worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5414worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds)5415worker2 # [ 79.221218] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:35Z level=info event=scheduler_connected addr=lb:500515416worker2 # [ 79.225131] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5417worker1 # [ 79.244548] systemd[1]: Listening on Nix worker-protocol over gRPC.5418worker1 # [ 79.252140] systemd[1]: Starting Nix worker-protocol over gRPC...5419worker1 # [ 79.392416] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:35Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5420worker1 # [ 79.394904] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:35Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5421worker1 # [ 79.398795] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:35Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15422worker1 # [ 79.405792] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:35Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5423worker1 # [ 79.408332] systemd[1]: Started Nix worker-protocol over gRPC.5424worker1 # [ 79.414809] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:35Z level=info event=scheduler_take_over5425lb # [ 79.423786] niks3-server[1111]: 2026/09/20 12:21:36 INFO lead: acquired remote=192.168.1.3:469525426lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5427lb: must succeed: curl -sf localhost:9901/clusters5428worker1 # [ 79.424478] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:35Z level=info event=scheduler_connected addr=lb:500515429worker1 # [ 79.426123] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5430worker2 # [ 79.425566] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:35Z level=info event=scheduler_connected addr=lb:500515431worker2 # [ 79.427483] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5432lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5433worker1 # [ 79.627534] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:36Z level=info event=scheduler_connected addr=lb:500515434worker1 # [ 79.631450] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5435worker2 # [ 79.630183] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:36Z level=info event=scheduler_connected addr=lb:500515436worker2 # [ 79.633568] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5437worker1 # [ 79.832164] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:36Z level=info event=scheduler_connected addr=lb:500515438worker2 # [ 79.830254] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:36Z level=info event=scheduler_connected addr=lb:500515439worker1 # [ 79.839715] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:36Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:503145440worker1 # [ 79.846209] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:36Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05441worker1 # [ 79.856157] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:36Z level=info event=oidc_jwks provider=mock keys=15442worker1 # [ 79.860366] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:36Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:503145443worker1 # [ 79.864239] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:36Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05444lb: must succeed: curl -sf localhost:9901/clusters5445lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5446lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.14 seconds)5447(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 9.84 seconds)5448subtest: scheduler down: the other node takes the lock and keeps it5449lb: must succeed: curl -sf localhost:9901/clusters5450lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5451lb: must succeed: curl -sf localhost:9901/clusters5452lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5453lb: must succeed: curl -sf localhost:9901/clusters5454lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5455lb: must succeed: curl -sf localhost:9901/clusters5456lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5457lb # [ 80.770778] envoy[1009]: conn peer=[2001:db8:1::4]:60844 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=42231 tx=2136 ms=522625458lb # [ 80.773615] envoy[1009]: conn peer=[2001:db8:1::1]:34904 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=235459lb # [ 80.775176] envoy[1009]: conn peer=192.168.1.3:39474 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=13459 tx=2806 ms=546005460lb # [ 80.776657] envoy[1009]: conn peer=[2001:db8:1::1]:34906 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1006 tx=305 ms=11595461worker1 # [ 80.779530] systemd[1]: Stopping Nix worker-protocol over gRPC...5462worker1 # [ 81.410394] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:37Z level=info event=signal_exit5463worker1 # [ 81.615209] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:38Z level=info event=scheduler_yield5464lb # [ 81.626845] niks3-server[1111]: 2026/09/20 12:21:38 INFO lead: released remote=192.168.1.3:469525465worker2 # [ 81.617225] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:38Z level=info event=scheduler_restarting addr=lb:500515466worker1 # [ 81.624953] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:38Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:503145467worker1 # [ 81.631279] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:38Z level=info event=worker_gone addr=192.168.1.4:500515468worker1 # [ 81.634505] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:38Z level=info event=scheduler_restarting addr=lb:500515469worker1 # [ 81.637628] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:38Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:503145470worker1 # [ 81.641144] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:38Z level=info event=worker_gone addr=192.168.1.3:500515471worker2 # [ 81.817547] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:38Z level=info event=scheduler_connected addr=lb:500515472worker2 # [ 81.819490] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5473worker1 # [ 81.830827] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:38Z level=info event=scheduler_connected addr=lb:500515474worker1 # [ 81.832844] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5475worker2 # [ 82.020787] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:38Z level=info event=scheduler_connected addr=lb:500515476worker2 # [ 82.024670] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5477worker1 # [ 82.034161] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:38Z level=info event=scheduler_connected addr=lb:500515478worker1 # [ 82.038197] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5479worker2 # [ 82.225759] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:38Z level=info event=scheduler_connected addr=lb:500515480worker1 # [ 82.236301] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:38Z level=info event=scheduler_connected addr=lb:500515481worker2 # [ 82.229825] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5482worker1 # [ 82.240678] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5483worker2 # [ 82.431092] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:38Z level=info event=scheduler_connected addr=lb:500515484worker2 # [ 82.434959] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5485worker1 # [ 82.443221] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:38Z level=info event=scheduler_connected addr=lb:500515486worker1 # [ 82.446535] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5487worker1 # [ 82.620119] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:39Z level=info event=scheduler_restarting5488worker2 # [ 82.635961] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:39Z level=info event=scheduler_connected addr=lb:500515489worker2 # [ 82.637643] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5490worker1 # [ 82.727765] nix-grpc-daemon[1278]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5491worker1 # [ 82.729464] nix-grpc-daemon[1278]: E0000 00:00:1789906899.230535 1321 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true5492worker1 # [ 82.731851] nix-grpc-daemon[1278]: E0000 00:00:1789906899.230749 1321 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5493worker2 # [ 82.838478] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:39Z level=info event=scheduler_connected addr=lb:500515494worker2 # [ 82.840309] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5495worker2 # [ 83.041070] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:39Z level=info event=scheduler_connected addr=lb:500515496worker2 # [ 83.042827] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5497worker2 # [ 83.243635] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:39Z level=info event=scheduler_connected addr=lb:500515498worker2 # [ 83.245394] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5499worker1 # [ 83.425322] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5500worker1 # [ 83.427643] systemd[1]: Stopped Nix worker-protocol over gRPC.5501worker1 # [ 83.429206] systemd[1]: nix-grpc-daemon.service: Consumed 134ms CPU time over 4.173s wall clock time, 10.3M memory peak, 9.7K incoming IP traffic, 9.1K outgoing IP traffic.5502worker1 # [ 83.432921] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5503worker1 # [ 83.434867] systemd[1]: Closed Nix worker-protocol over gRPC.5504worker1 # [ 83.436276] systemd[1]: nix-grpc-daemon.socket: Consumed 540us CPU time over 4.197s wall clock time, 256K memory peak, 13.7K incoming IP traffic, 10.2K outgoing IP traffic.5505worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5506worker2 # [ 83.446630] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:39Z level=info event=scheduler_connected addr=lb:500515507worker2 # [ 83.448447] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5508worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.04 seconds)5509worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5510worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)5511worker2 # [ 83.649732] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:40Z level=info event=scheduler_connected addr=lb:500515512worker2 # [ 83.651409] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5513worker2 # [ 83.851053] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:40Z level=info event=scheduler_connected addr=lb:500515514worker2 # [ 83.852850] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5515worker2 # [ 84.054140] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:40Z level=info event=scheduler_connected addr=lb:500515516worker2 # [ 84.057948] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5517worker2 # [ 84.259158] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:40Z level=info event=scheduler_connected addr=lb:500515518worker2 # [ 84.263133] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5519worker2 # [ 84.464499] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:40Z level=info event=scheduler_connected addr=lb:500515520worker2 # [ 84.469335] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5521worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5522worker2 # [ 84.523214] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:40Z level=info event=scheduler_take_over5523lb # [ 84.538847] niks3-server[1111]: 2026/09/20 12:21:41 INFO lead: acquired remote=192.168.1.4:530845524worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5525worker2 # [ 84.674379] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:41Z level=info event=scheduler_connected addr=lb:500515526worker2 # [ 84.687152] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:41Z level=info event=oidc_jwks provider=mock keys=15527worker2 # [ 84.691361] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:41Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:342205528worker2 # [ 84.695330] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:41Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05529worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5530worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5531client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&25532worker2 # [ 85.807556] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:42Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:569725533client # don't know how to build these paths:5534client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5535worker2 # [ 85.816374] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:569725536worker2 # [ 85.837161] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:42Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05537worker2 # [ 85.842651] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:42Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.4:50051 clients=15538worker2 # [ 85.886668] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569725539worker2 # [ 85.900431] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:569725540client # copying 1 paths to 'lb:50051'...5541client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5542worker2 # [ 85.912909] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:569725543worker2 # [ 85.925222] nix-daemon[924]: accepted connection from pid 1153, user nix-grpc-daemon (trusted)5544worker2 # [ 86.097173] nix-daemon[924]: accepted connection from pid 1249, user nix-grpc-daemon (trusted)5545worker2 # [ 86.110937] nix-daemon[924]: reaped child process 1252, status = succeeded5546worker2 # [ 86.253111] nix-daemon[924]: accepted connection from pid 1254, user nix-grpc-daemon (trusted)5547worker2 # [ 86.266986] nix-daemon[924]: reaped child process 1257, status = succeeded5548lb # [ 86.293510] niks3-server[1111]: 2026/09/20 12:21:42 INFO Received uploads request method=POST path=/api/pending_closures5549worker2 # [ 86.283642] nix-grpc-daemon[1209]: time=2026-09-20T12:21:42.756Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5550worker2 # [ 86.285095] nix-grpc-daemon[1209]: time=2026-09-20T12:21:42.756Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5551lb # [ 86.308474] niks3-server[1111]: 2026/09/20 12:21:42 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5552lb # [ 86.312741] niks3-server[1111]: 2026/09/20 12:21:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5553lb # [ 86.314844] niks3-server[1111]: 2026/09/20 12:21:42 INFO Signed narinfos id=27 count=15554worker2 # [ 86.300039] nix-grpc-daemon[1209]: time=2026-09-20T12:21:42.773Z level=INFO msg="Uploading 1 narinfos"5555lb # [ 86.320669] niks3-server[1111]: 2026/09/20 12:21:42 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5556lb # [ 86.324574] niks3-server[1111]: 2026/09/20 12:21:42 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5557lb # [ 86.327875] niks3-server[1111]: 2026/09/20 12:21:42 INFO Completed upload id=275558worker2 # [ 86.312813] nix-grpc-daemon[1209]: time=2026-09-20T12:21:42.786Z level=INFO msg="Upload complete. (371ms)"5559worker2 # [ 86.314893] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56972 duration_s=0 paths=1 nar_bytes_in=5205560worker2 # [ 86.318179] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569725561worker2 # [ 86.322788] nix-daemon[924]: accepted connection from pid 1153, user nix-grpc-daemon (trusted)5562lb # [ 86.338783] niks3-server[1111]: 2026/09/20 12:21:42 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5563worker2 # [ 86.328310] nix-daemon[924]: reaped child process 1247, status = succeeded5564worker2 # [ 86.333136] nix-daemon[924]: accepted connection from pid 1153, user nix-grpc-daemon (trusted)5565client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5566client # farm-a> worker2: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5567client # farm-a> LOG-farm-a-on-standby5568worker2 # [ 87.748672] nix-daemon[924]: reaped child process 1261, status = succeeded5569worker2 # [ 87.896531] nix-daemon[924]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5570worker2 # [ 87.909981] nix-daemon[924]: reaped child process 1279, status = succeeded5571lb # [ 87.938779] niks3-server[1111]: 2026/09/20 12:21:44 INFO Received uploads request method=POST path=/api/pending_closures5572worker2 # [ 87.936413] nix-grpc-daemon[1209]: time=2026-09-20T12:21:44.409Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5573worker2 # [ 87.939097] nix-grpc-daemon[1209]: time=2026-09-20T12:21:44.411Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5574lb # [ 87.983480] niks3-server[1111]: 2026/09/20 12:21:44 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5575lb # [ 87.991947] niks3-server[1111]: 2026/09/20 12:21:44 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5576lb # [ 88.003203] niks3-server[1111]: 2026/09/20 12:21:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5577lb # [ 88.009125] niks3-server[1111]: 2026/09/20 12:21:44 INFO Signed narinfos id=28 count=15578worker2 # [ 87.993792] nix-grpc-daemon[1209]: time=2026-09-20T12:21:44.466Z level=INFO msg="Uploading 1 narinfos"5579lb # [ 88.012794] niks3-server[1111]: 2026/09/20 12:21:44 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5580lb # [ 88.020886] niks3-server[1111]: 2026/09/20 12:21:44 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5581lb # [ 88.025961] niks3-server[1111]: 2026/09/20 12:21:44 INFO Completed upload id=285582lb # [ 88.027501] niks3-server[1111]: 2026/09/20 12:21:44 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5583worker2 # [ 88.011183] nix-grpc-daemon[1209]: time=2026-09-20T12:21:44.484Z level=INFO msg="Upload complete. (270ms)"5584worker2 # [ 88.027133] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56972 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15585worker2 # [ 88.031774] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:44Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05586worker2 # [ 88.035214] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:44Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.4:50051 clients=15587worker2 # [ 88.044989] nix-daemon[924]: reaped child process 1259, status = succeeded5588worker2 # [ 88.069938] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569725589worker2 # [ 88.083394] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:569725590worker2 # [ 88.095832] nix-grpc-daemon[1209]: time=2026-09-20T12:21:44.568Z level=INFO msg="All 1 paths already cached"5591client # copying 1 paths to 'lb:50051'...5592client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5593worker2 # [ 88.106144] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:569725594worker2 # [ 88.116166] nix-daemon[924]: accepted connection from pid 1153, user nix-grpc-daemon (trusted)5595worker2 # [ 88.275650] nix-daemon[924]: accepted connection from pid 1285, user nix-grpc-daemon (trusted)5596worker2 # [ 88.289245] nix-daemon[924]: reaped child process 1288, status = succeeded5597worker2 # [ 88.425095] nix-daemon[924]: accepted connection from pid 1290, user nix-grpc-daemon (trusted)5598worker2 # [ 88.438713] nix-daemon[924]: reaped child process 1293, status = succeeded5599lb # [ 88.464439] niks3-server[1111]: 2026/09/20 12:21:45 INFO Received uploads request method=POST path=/api/pending_closures5600worker2 # [ 88.453359] nix-grpc-daemon[1209]: time=2026-09-20T12:21:44.926Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5601worker2 # [ 88.454964] nix-grpc-daemon[1209]: time=2026-09-20T12:21:44.928Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5602lb # [ 88.490929] niks3-server[1111]: 2026/09/20 12:21:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5603lb # [ 88.495597] niks3-server[1111]: 2026/09/20 12:21:45 INFO Signed narinfos id=29 count=15604worker2 # [ 88.480560] nix-grpc-daemon[1209]: time=2026-09-20T12:21:44.953Z level=INFO msg="Uploading 1 narinfos"5605lb # [ 88.501087] niks3-server[1111]: 2026/09/20 12:21:45 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5606lb # [ 88.509674] niks3-server[1111]: 2026/09/20 12:21:45 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5607lb # [ 88.513154] niks3-server[1111]: 2026/09/20 12:21:45 INFO Completed upload id=295608worker2 # [ 88.499217] nix-grpc-daemon[1209]: time=2026-09-20T12:21:44.972Z level=INFO msg="Upload complete. (372ms)"5609lb # [ 88.514886] niks3-server[1111]: 2026/09/20 12:21:45 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5610worker2 # [ 88.501499] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56972 duration_s=0 paths=1 nar_bytes_in=6405611worker2 # [ 88.505113] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569725612lb # [ 88.520572] niks3-server[1111]: 2026/09/20 12:21:45 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5613worker2 # [ 88.512168] nix-daemon[924]: accepted connection from pid 1153, user nix-grpc-daemon (trusted)5614worker2 # [ 88.514605] nix-daemon[924]: reaped child process 1283, status = succeeded5615client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5616client # farm-b> worker2: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5617worker2 # [ 88.520062] nix-daemon[924]: accepted connection from pid 1153, user nix-grpc-daemon (trusted)5618client # farm-b> LOG-farm-b-on-standby5619worker2 # [ 89.737405] nix-daemon[924]: reaped child process 1297, status = succeeded5620worker2 # [ 89.875171] nix-daemon[924]: accepted connection from pid 1312, user nix-grpc-daemon (trusted)5621worker2 # [ 89.887336] nix-daemon[924]: reaped child process 1315, status = succeeded5622lb # [ 89.919464] niks3-server[1111]: 2026/09/20 12:21:46 INFO Received uploads request method=POST path=/api/pending_closures5623worker2 # [ 89.913437] nix-grpc-daemon[1209]: time=2026-09-20T12:21:46.386Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5624worker2 # [ 89.914921] nix-grpc-daemon[1209]: time=2026-09-20T12:21:46.388Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"5625lb # [ 89.958457] niks3-server[1111]: 2026/09/20 12:21:46 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5626lb # [ 89.963721] niks3-server[1111]: 2026/09/20 12:21:46 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst5627lb # [ 89.977663] niks3-server[1111]: 2026/09/20 12:21:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5628lb # [ 89.984609] niks3-server[1111]: 2026/09/20 12:21:46 INFO Signed narinfos id=30 count=15629worker2 # [ 89.972651] nix-grpc-daemon[1209]: time=2026-09-20T12:21:46.445Z level=INFO msg="Uploading 1 narinfos"5630lb # [ 89.987872] niks3-server[1111]: 2026/09/20 12:21:46 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls5631lb # [ 89.999303] niks3-server[1111]: 2026/09/20 12:21:46 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5632lb # [ 90.004710] niks3-server[1111]: 2026/09/20 12:21:46 INFO Completed upload id=305633worker2 # [ 89.989724] nix-grpc-daemon[1209]: time=2026-09-20T12:21:46.462Z level=INFO msg="Upload complete. (261ms)"5634lb # [ 90.007515] niks3-server[1111]: 2026/09/20 12:21:46 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo5635worker2 # [ 89.993341] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56972 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=15636worker2 # [ 89.999027] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:46Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=05637worker2 # [ 90.003611] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:46Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.4:50051 clients=15638worker2 # [ 90.010180] nix-daemon[924]: reaped child process 1295, status = succeeded5639worker2 # [ 90.044700] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569725640worker2 # [ 90.056367] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:569725641worker2 # [ 90.069315] nix-grpc-daemon[1209]: time=2026-09-20T12:21:46.542Z level=INFO msg="All 2 paths already cached"5642client # copying 1 paths to 'lb:50051'...5643client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...5644worker2 # [ 90.077225] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:569725645worker2 # [ 90.087162] nix-daemon[924]: accepted connection from pid 1153, user nix-grpc-daemon (trusted)5646worker2 # [ 90.247134] nix-daemon[924]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)5647worker2 # [ 90.261241] nix-daemon[924]: reaped child process 1324, status = succeeded5648worker2 # [ 90.400662] nix-daemon[924]: accepted connection from pid 1326, user nix-grpc-daemon (trusted)5649worker2 # [ 90.413972] nix-daemon[924]: reaped child process 1329, status = succeeded5650lb # [ 90.438374] niks3-server[1111]: 2026/09/20 12:21:47 INFO Received uploads request method=POST path=/api/pending_closures5651worker2 # [ 90.427463] nix-grpc-daemon[1209]: time=2026-09-20T12:21:46.900Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5652worker2 # [ 90.428947] nix-grpc-daemon[1209]: time=2026-09-20T12:21:46.900Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"5653lb # [ 90.450483] niks3-server[1111]: 2026/09/20 12:21:47 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst5654lb # [ 90.454898] niks3-server[1111]: 2026/09/20 12:21:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5655lb # [ 90.456718] niks3-server[1111]: 2026/09/20 12:21:47 INFO Signed narinfos id=31 count=15656worker2 # [ 90.442811] nix-grpc-daemon[1209]: time=2026-09-20T12:21:46.915Z level=INFO msg="Uploading 1 narinfos"5657lb # [ 90.459055] niks3-server[1111]: 2026/09/20 12:21:47 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls5658lb # [ 90.464256] niks3-server[1111]: 2026/09/20 12:21:47 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5659lb # [ 90.468312] niks3-server[1111]: 2026/09/20 12:21:47 INFO Completed upload id=315660worker2 # [ 90.453086] nix-grpc-daemon[1209]: time=2026-09-20T12:21:46.926Z level=INFO msg="Upload complete. (353ms)"5661lb # [ 90.469844] niks3-server[1111]: 2026/09/20 12:21:47 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo5662worker2 # [ 90.455047] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56972 duration_s=0 paths=1 nar_bytes_in=7685663worker2 # [ 90.458620] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569725664worker2 # [ 90.464971] nix-daemon[924]: accepted connection from pid 1153, user nix-grpc-daemon (trusted)5665worker2 # [ 90.467320] nix-daemon[924]: reaped child process 1319, status = succeeded5666worker2 # [ 90.472449] nix-daemon[924]: accepted connection from pid 1153, user nix-grpc-daemon (trusted)5667client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...5668client # farm-top> worker2: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5669lb # [ 90.771993] envoy[1009]: conn peer=[2001:db8:1::3]:39602 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1746 tx=428 ms=32245670client # farm-top> LOG-farm-top-on-standby5671worker2 # [ 91.700179] nix-daemon[924]: reaped child process 1333, status = succeeded5672worker2 # [ 91.852614] nix-daemon[924]: accepted connection from pid 1348, user nix-grpc-daemon (trusted)5673worker2 # [ 91.866347] nix-daemon[924]: reaped child process 1351, status = succeeded5674lb # [ 91.898901] niks3-server[1111]: 2026/09/20 12:21:48 INFO Received uploads request method=POST path=/api/pending_closures5675worker2 # [ 91.894041] nix-grpc-daemon[1209]: time=2026-09-20T12:21:48.366Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5676worker2 # [ 91.895842] nix-grpc-daemon[1209]: time=2026-09-20T12:21:48.368Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"5677lb # [ 91.933785] niks3-server[1111]: 2026/09/20 12:21:48 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5678lb # [ 91.947377] niks3-server[1111]: 2026/09/20 12:21:48 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst5679lb # [ 91.957710] niks3-server[1111]: 2026/09/20 12:21:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5680lb # [ 91.961728] niks3-server[1111]: 2026/09/20 12:21:48 INFO Signed narinfos id=32 count=15681worker2 # [ 91.949310] nix-grpc-daemon[1209]: time=2026-09-20T12:21:48.422Z level=INFO msg="Uploading 1 narinfos"5682lb # [ 91.967783] niks3-server[1111]: 2026/09/20 12:21:48 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls5683lb # [ 91.972791] niks3-server[1111]: 2026/09/20 12:21:48 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5684lb # [ 91.976776] niks3-server[1111]: 2026/09/20 12:21:48 INFO Completed upload id=325685worker2 # [ 91.961688] nix-grpc-daemon[1209]: time=2026-09-20T12:21:48.434Z level=INFO msg="Upload complete. (282ms)"5686lb # [ 91.979076] niks3-server[1111]: 2026/09/20 12:21:48 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo5687worker2 # [ 91.964778] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56972 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=15688worker2 # [ 91.972719] nix-daemon[924]: reaped child process 1331, status = succeeded5689client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&2, in 6.40 seconds)5690client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath5691client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath, in 0.18 seconds)5692worker1 # [ 92.245559] systemd[1]: Listening on Nix worker-protocol over gRPC.5693worker1 # [ 92.249818] systemd[1]: Starting Nix worker-protocol over gRPC...5694worker1 # [ 92.417345] nix-grpc-daemon[1342]: ts=2026-09-20T12:21:48Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5695worker1 # [ 92.419409] nix-grpc-daemon[1342]: ts=2026-09-20T12:21:48Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5696worker1 # [ 92.423702] nix-grpc-daemon[1342]: ts=2026-09-20T12:21:48Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15697worker1 # [ 92.430934] nix-grpc-daemon[1342]: ts=2026-09-20T12:21:48Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5698worker1 # [ 92.435415] systemd[1]: Started Nix worker-protocol over gRPC.5699lb: must succeed: curl -sf localhost:9901/clusters5700worker1 # [ 92.459892] nix-grpc-daemon[1342]: ts=2026-09-20T12:21:48Z level=info event=scheduler_connected addr=lb:500515701worker2 # [ 92.456065] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:48Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:342205702worker2 # [ 92.459872] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:48Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05703lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5704worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5705worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5706lb: must succeed: curl -sf localhost:9901/clusters5707lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5708lb: must succeed: curl -sf localhost:9901/clusters5709lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5710worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5711worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5712(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.15 seconds)5713subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5714client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs5715client # [ 92.743348] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs.5716client # Running as unit: rs.service; invocation ID: 4786268a11fa427cb6e387068c15b4e55717client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs, in 0.07 seconds)5718client # [ 92.940669] nix[1188]: don't know how to build these paths:5719client # [ 92.941719] nix[1188]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5720worker2 # [ 92.951899] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:49Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:569725721worker2 # [ 92.964226] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:49Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=05722worker2 # [ 92.970243] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:49Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.4:50051 clients=15723worker2 # [ 93.012525] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569725724worker2 # [ 93.022940] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:569725725client # [ 93.017259] nix[1188]: copying 1 paths to 'lb:50051'...5726client # [ 93.021138] nix[1188]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...5727worker2 # [ 93.034728] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:569725728worker2 # [ 93.043863] nix-daemon[924]: accepted connection from pid 1153, user nix-grpc-daemon (trusted)5729worker2 # [ 93.211490] nix-daemon[924]: accepted connection from pid 1364, user nix-grpc-daemon (trusted)5730worker2 # [ 93.225225] nix-daemon[924]: reaped child process 1367, status = succeeded5731worker2 # [ 93.378668] nix-daemon[924]: accepted connection from pid 1369, user nix-grpc-daemon (trusted)5732worker2 # [ 93.392221] nix-daemon[924]: reaped child process 1372, status = succeeded5733lb # [ 93.422704] niks3-server[1111]: 2026/09/20 12:21:50 INFO Received uploads request method=POST path=/api/pending_closures5734worker2 # [ 93.415991] nix-grpc-daemon[1209]: time=2026-09-20T12:21:49.888Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5735worker2 # [ 93.417827] nix-grpc-daemon[1209]: time=2026-09-20T12:21:49.890Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"5736lb # [ 93.452759] niks3-server[1111]: 2026/09/20 12:21:50 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst5737lb # [ 93.462659] niks3-server[1111]: 2026/09/20 12:21:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5738worker2 # [ 93.453563] nix-grpc-daemon[1209]: time=2026-09-20T12:21:49.926Z level=INFO msg="Uploading 1 narinfos"5739lb # [ 93.469850] niks3-server[1111]: 2026/09/20 12:21:50 INFO Signed narinfos id=33 count=15740lb # [ 93.475826] niks3-server[1111]: 2026/09/20 12:21:50 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls5741lb # [ 93.486187] niks3-server[1111]: 2026/09/20 12:21:50 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5742worker2 # [ 93.475439] nix-grpc-daemon[1209]: time=2026-09-20T12:21:49.948Z level=INFO msg="Upload complete. (417ms)"5743lb # [ 93.490478] niks3-server[1111]: 2026/09/20 12:21:50 INFO Completed upload id=335744worker2 # [ 93.477881] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56972 duration_s=0 paths=1 nar_bytes_in=4165745worker2 # [ 93.479934] nix-grpc-daemon[1209]: time=2026-09-20T12:21:49.951Z level=WARN msg="Failed to register uploaded object" key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo error="server returned 404: object is not pending upload\n"5746worker2 # [ 93.483134] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569725747worker2 # [ 93.487983] nix-daemon[924]: accepted connection from pid 1153, user nix-grpc-daemon (trusted)5748worker2 # [ 93.494087] nix-daemon[924]: reaped child process 1362, status = succeeded5749worker2 # [ 93.499109] nix-daemon[924]: accepted connection from pid 1153, user nix-grpc-daemon (trusted)5750client # [ 93.483616] nix[1188]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...5751lb: must succeed: curl -sf localhost:9901/clusters5752lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5753lb: must succeed: curl -sf localhost:9901/clusters5754lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5755lb: must succeed: curl -sf localhost:9901/clusters5756lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5757lb: must succeed: curl -sf localhost:9901/clusters5758lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5759worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5760worker1 # [ 95.609552] systemd-journald[288]: Received client request to rotate journal, rotating.5761worker1 # [ 95.630371] systemd-journald[288]: Vacuuming done, freed 0B of archived journals from /var/log/journal/0946a2ebcdcf46f89965189f1fb115e8.5762worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.07 seconds)5763worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service5764worker2 # [ 95.537341] systemd[1]: Stopping Nix worker-protocol over gRPC...5765worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)5766client: must succeed: sleep 5.05767worker2 # [ 96.517946] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:52Z level=info event=draining5768client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5769lb # [ 100.773548] envoy[1009]: conn peer=[2001:db8:1::1]:55020 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9603 tx=3815 ms=61755770worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5771worker2 # [ 100.928930] nix-daemon[924]: reaped child process 1376, status = succeeded5772worker2 # [ 101.076940] nix-daemon[924]: accepted connection from pid 1416, user nix-grpc-daemon (trusted)5773worker2 # [ 101.090633] nix-daemon[924]: reaped child process 1419, status = succeeded5774lb # [ 101.123223] niks3-server[1111]: 2026/09/20 12:21:57 INFO Received uploads request method=POST path=/api/pending_closures5775worker2 # [ 101.114917] nix-grpc-daemon[1209]: time=2026-09-20T12:21:57.587Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5776worker2 # [ 101.116469] nix-grpc-daemon[1209]: time=2026-09-20T12:21:57.589Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"5777lb # [ 101.160092] niks3-server[1111]: 2026/09/20 12:21:57 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5778lb # [ 101.168686] niks3-server[1111]: 2026/09/20 12:21:57 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5779lb # [ 101.178461] niks3-server[1111]: 2026/09/20 12:21:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5780lb # [ 101.182625] niks3-server[1111]: 2026/09/20 12:21:57 INFO Signed narinfos id=34 count=15781worker2 # [ 101.170477] nix-grpc-daemon[1209]: time=2026-09-20T12:21:57.643Z level=INFO msg="Uploading 1 narinfos"5782lb # [ 101.190742] niks3-server[1111]: 2026/09/20 12:21:57 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls5783lb # [ 101.200092] niks3-server[1111]: 2026/09/20 12:21:57 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5784lb # [ 101.206093] niks3-server[1111]: 2026/09/20 12:21:57 INFO Completed upload id=345785worker2 # [ 101.191060] nix-grpc-daemon[1209]: time=2026-09-20T12:21:57.664Z level=INFO msg="Upload complete. (278ms)"5786lb # [ 101.208957] niks3-server[1111]: 2026/09/20 12:21:57 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo5787worker2 # [ 101.195388] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56972 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=15788worker2 # [ 101.201817] nix-daemon[924]: reaped child process 1374, status = succeeded5789client # [ 101.210901] systemd[1]: rs.service: Deactivated successfully.5790client # [ 101.212895] systemd[1]: rs.service: Consumed 109ms CPU time over 8.469s wall clock time, 39.1M memory peak, 8.3K incoming IP traffic, 8.9K outgoing IP traffic.5791worker2 # [ 101.521489] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:57Z level=info event=signal_exit5792worker2 # [ 101.547909] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:58Z level=info event=scheduler_yield5793worker1 # [ 101.558293] nix-grpc-daemon[1342]: ts=2026-09-20T12:21:58Z level=info event=scheduler_restarting addr=lb:500515794lb # [ 101.565693] niks3-server[1111]: 2026/09/20 12:21:58 INFO lead: released remote=192.168.1.4:530845795worker2 # [ 101.555153] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:58Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:342205796worker2 # [ 101.559265] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:58Z level=info event=worker_gone addr=192.168.1.3:500515797worker2 # [ 101.563459] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:58Z level=info event=scheduler_restarting addr=lb:500515798worker2 # [ 101.567934] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:58Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:342205799worker2 # [ 101.572124] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:58Z level=info event=worker_gone addr=192.168.1.4:500515800worker1 # [ 101.758821] nix-grpc-daemon[1342]: ts=2026-09-20T12:21:58Z level=info event=scheduler_connected addr=lb:500515801worker1 # [ 101.763229] nix-grpc-daemon[1342]: ts=2026-09-20T12:21:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5802worker2 # [ 101.760607] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:58Z level=info event=scheduler_connected addr=lb:500515803worker2 # [ 101.764569] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5804worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 1.09 seconds)5805client: waiting for success: ! systemctl is-active rs5806client: (finished: waiting for success: ! systemctl is-active rs, in 0.03 seconds)5807client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5808client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.03 seconds)5809client: must fail: journalctl -u rs | grep -q 'reconnecting'5810worker1 # [ 101.963901] nix-grpc-daemon[1342]: ts=2026-09-20T12:21:58Z level=info event=scheduler_connected addr=lb:500515811worker1 # [ 101.968087] nix-grpc-daemon[1342]: ts=2026-09-20T12:21:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5812worker2 # [ 101.963080] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:58Z level=info event=scheduler_connected addr=lb:500515813worker2 # [ 101.967309] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5814client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.02 seconds)5815lb: must succeed: curl -sf localhost:9901/clusters5816lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5817worker1 # [ 102.168879] nix-grpc-daemon[1342]: ts=2026-09-20T12:21:58Z level=info event=scheduler_connected addr=lb:500515818worker2 # [ 102.167481] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:58Z level=info event=scheduler_connected addr=lb:500515819worker2 # [ 102.169293] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5820worker1 # [ 102.173491] nix-grpc-daemon[1342]: ts=2026-09-20T12:21:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5821worker1 # [ 102.373871] nix-grpc-daemon[1342]: ts=2026-09-20T12:21:58Z level=info event=scheduler_connected addr=lb:500515822worker1 # [ 102.375701] nix-grpc-daemon[1342]: ts=2026-09-20T12:21:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5823worker2 # [ 102.372163] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:58Z level=info event=scheduler_connected addr=lb:500515824worker2 # [ 102.375524] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5825lb # [ 102.454170] niks3-server[1111]: 2026/09/20 12:21:59 INFO lead: acquired remote=192.168.1.3:465585826worker1 # [ 102.444876] nix-grpc-daemon[1342]: ts=2026-09-20T12:21:58Z level=info event=scheduler_take_over5827worker2 # [ 102.552203] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:59Z level=info event=scheduler_restarting5828worker2 # [ 102.570881] nix-grpc-daemon[1153]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5829worker1 # [ 102.577909] nix-grpc-daemon[1342]: ts=2026-09-20T12:21:59Z level=info event=scheduler_connected addr=lb:500515830worker1 # [ 102.581526] nix-grpc-daemon[1342]: ts=2026-09-20T12:21:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5831worker2 # [ 102.577924] nix-grpc-daemon[1153]: E0000 00:00:1789906919.042327 1196 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5832worker2 # [ 102.580440] nix-grpc-daemon[1153]: E0000 00:00:1789906919.044068 1196 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5833worker2 # [ 102.582702] nix-grpc-daemon[1153]: ts=2026-09-20T12:21:59Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5834worker2 # [ 102.584611] nix-daemon[924]: reaped child process 1202, status = succeeded5835worker1 # [ 102.782084] nix-grpc-daemon[1342]: ts=2026-09-20T12:21:59Z level=info event=scheduler_connected addr=lb:500515836worker1 # [ 102.786239] nix-grpc-daemon[1342]: ts=2026-09-20T12:21:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5837worker1 # [ 102.986660] nix-grpc-daemon[1342]: ts=2026-09-20T12:21:59Z level=info event=scheduler_connected addr=lb:500515838worker1 # [ 102.988703] nix-grpc-daemon[1342]: ts=2026-09-20T12:21:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5839lb: must succeed: curl -sf localhost:9901/clusters5840lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5841worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5842worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)5843worker1 # [ 103.189200] nix-grpc-daemon[1342]: ts=2026-09-20T12:21:59Z level=info event=scheduler_connected addr=lb:500515844worker1 # [ 103.191190] nix-grpc-daemon[1342]: ts=2026-09-20T12:21:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5845worker1 # [ 103.392076] nix-grpc-daemon[1342]: ts=2026-09-20T12:21:59Z level=info event=scheduler_connected addr=lb:500515846worker1 # [ 103.396533] nix-grpc-daemon[1342]: ts=2026-09-20T12:21:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5847worker1 # [ 103.597421] nix-grpc-daemon[1342]: ts=2026-09-20T12:22:00Z level=info event=scheduler_connected addr=lb:500515848worker1 # [ 103.601287] nix-grpc-daemon[1342]: ts=2026-09-20T12:22:00Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:607885849worker1 # [ 103.604401] nix-grpc-daemon[1342]: ts=2026-09-20T12:22:00Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05850lb: must succeed: curl -sf localhost:9901/clusters5851lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5852worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5853worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5854worker2 # [ 104.593317] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5855worker2 # [ 104.594615] systemd[1]: Stopped Nix worker-protocol over gRPC.5856worker2 # [ 104.597457] systemd[1]: nix-grpc-daemon.service: Consumed 1.417s CPU time over 30.230s wall clock time, 33.3M memory peak, 62K incoming IP traffic, 160.1K outgoing IP traffic.5857worker2 # [ 104.608779] systemd[1]: Starting scheduler-token.service...5858worker2 # [ 104.664758] systemd[1]: scheduler-token.service: Deactivated successfully.5859worker2 # [ 104.666255] systemd[1]: Finished scheduler-token.service.5860worker2 # [ 104.668111] systemd[1]: scheduler-token.service: Consumed 19ms CPU time over 56ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5861worker2 # [ 104.673428] systemd[1]: Starting Nix worker-protocol over gRPC...5862worker2 # [ 104.829138] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:01Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5863worker2 # [ 104.830908] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:01Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5864worker2 # [ 104.834674] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:01Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15865worker2 # [ 104.842119] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:01Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5866worker2 # [ 104.845902] systemd[1]: Started Nix worker-protocol over gRPC.5867worker2 # [ 104.857247] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:01Z level=info event=scheduler_connected addr=lb:500515868worker1 # [ 104.872863] nix-grpc-daemon[1342]: ts=2026-09-20T12:22:01Z level=info event=oidc_jwks provider=mock keys=15869worker1 # [ 104.876744] nix-grpc-daemon[1342]: ts=2026-09-20T12:22:01Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:607885870worker1 # [ 104.880684] nix-grpc-daemon[1342]: ts=2026-09-20T12:22:01Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05871lb: must succeed: curl -sf localhost:9901/clusters5872lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5873worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5874worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5875(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.64 seconds)5876subtest: scheduler restart mid-build: build finishes, no double build5877client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid5878client # [ 105.375758] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid.5879client # Running as unit: mid.service; invocation ID: b3b2c044afab4651a42f09cdd9f1669c5880client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid, in 0.07 seconds)5881worker1 # [ 105.581683] nix-grpc-daemon[1342]: ts=2026-09-20T12:22:02Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:608025882worker1 # [ 105.587350] nix-daemon[909]: accepted connection from pid 1342, user nix-grpc-daemon (trusted)5883client # [ 105.574120] nix[1224]: don't know how to build these paths:5884client # [ 105.575406] nix[1224]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5885worker1 # [ 105.599472] nix-grpc-daemon[1342]: ts=2026-09-20T12:22:02Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05886worker1 # [ 105.601316] nix-grpc-daemon[1342]: ts=2026-09-20T12:22:02Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=15887worker1 # [ 105.647358] nix-grpc-daemon[1342]: ts=2026-09-20T12:22:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608025888worker1 # [ 105.653955] nix-grpc-daemon[1342]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 72 ms (attempt 1/5)5889worker1 # [ 105.728412] nix-grpc-daemon[1342]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 133 ms (attempt 2/5)5890worker1 # [ 105.864400] nix-grpc-daemon[1342]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 168 ms (attempt 3/5)5891worker1 # [ 106.039290] nix-grpc-daemon[1342]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 183 ms (attempt 4/5)5892worker1 # [ 106.222771] nix-grpc-daemon[1342]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5893worker1 # [ 106.233793] nix-grpc-daemon[1342]: ts=2026-09-20T12:22:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:608025894client # [ 106.218788] nix[1224]: copying 1 paths to 'lb:50051'...5895worker1 # [ 106.243125] nix-grpc-daemon[1342]: ts=2026-09-20T12:22:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:608025896client # [ 106.222931] nix[1224]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...5897worker1 # [ 106.251264] nix-daemon[909]: accepted connection from pid 1342, user nix-grpc-daemon (trusted)5898worker1 # [ 106.260249] nix-grpc-daemon[1342]: ts=2026-09-20T12:22:02Z level=info event=niks3_push_started pid=14575899worker1 # [ 106.449704] nix-grpc-daemon[1457]: time=2026-09-20T12:22:02.952Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""5900worker1 # [ 106.601923] nix-daemon[909]: accepted connection from pid 1470, user nix-grpc-daemon (trusted)5901worker1 # [ 106.615168] nix-daemon[909]: reaped child process 1477, status = succeeded5902worker1 # [ 106.755387] nix-daemon[909]: accepted connection from pid 1479, user nix-grpc-daemon (trusted)5903worker1 # [ 106.768254] nix-daemon[909]: reaped child process 1482, status = succeeded5904lb # [ 106.790708] niks3-server[1111]: 2026/09/20 12:22:03 INFO Received uploads request method=POST path=/api/pending_closures5905worker1 # [ 106.787960] nix-grpc-daemon[1457]: time=2026-09-20T12:22:03.290Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5906worker1 # [ 106.791255] nix-grpc-daemon[1457]: time=2026-09-20T12:22:03.292Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"5907lb # [ 106.817285] niks3-server[1111]: 2026/09/20 12:22:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5908lb # [ 106.819955] niks3-server[1111]: 2026/09/20 12:22:03 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst5909worker1 # [ 106.816537] nix-grpc-daemon[1457]: time=2026-09-20T12:22:03.319Z level=INFO msg="Uploading 1 narinfos"5910lb # [ 106.826162] niks3-server[1111]: 2026/09/20 12:22:03 INFO Signed narinfos id=35 count=15911lb # [ 106.829458] niks3-server[1111]: 2026/09/20 12:22:03 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls5912lb # [ 106.834321] niks3-server[1111]: 2026/09/20 12:22:03 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5913lb # [ 106.841614] niks3-server[1111]: 2026/09/20 12:22:03 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo5914lb # [ 106.844719] niks3-server[1111]: 2026/09/20 12:22:03 INFO Completed upload id=355915worker1 # [ 106.836043] nix-grpc-daemon[1457]: time=2026-09-20T12:22:03.339Z level=INFO msg="Upload complete. (382ms)"5916worker1 # [ 106.838243] nix-grpc-daemon[1342]: ts=2026-09-20T12:22:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60802 duration_s=0 paths=1 nar_bytes_in=4245917worker1 # [ 106.841181] nix-grpc-daemon[1342]: ts=2026-09-20T12:22:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608025918worker1 # [ 106.846726] nix-daemon[909]: accepted connection from pid 1342, user nix-grpc-daemon (trusted)5919worker1 # [ 106.851413] nix-daemon[909]: reaped child process 1455, status = succeeded5920worker1 # [ 106.857496] nix-daemon[909]: accepted connection from pid 1342, user nix-grpc-daemon (trusted)5921client # [ 106.836339] nix[1224]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5922lb: must succeed: curl -sf localhost:9901/clusters5923lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5924lb: must succeed: curl -sf localhost:9901/clusters5925lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5926worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5927worker1 # [ 109.135589] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1342 (nix-grpc-daemon) on client request.5928worker1 # [ 109.141355] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5929lb # [ 109.157237] niks3-server[1111]: 2026/09/20 12:22:05 INFO lead: released remote=192.168.1.3:465585930client # [ 109.136395] nix[1224]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5931worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.06 seconds)5932worker2 # [ 109.149647] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:05Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5933client: must succeed: sleep 5.05934client # [ 109.140399] nix[1224]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5935worker1 # [ 109.170926] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5936worker1 # [ 109.172402] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5937worker1 # [ 109.173623] systemd[1]: nix-grpc-daemon.service: Consumed 433ms CPU time over 16.916s wall clock time, 31M memory peak, 17.6K incoming IP traffic, 25.8K outgoing IP traffic.5938worker1 # [ 109.179306] nix-daemon[909]: reaped child process 1484, status = succeeded5939worker1 # [ 109.180535] nix-daemon[909]: reaped child process 1448, status = succeeded5940worker1 # [ 109.188479] nix-daemon[1486]: unexpected Nix daemon error: error: interrupted by the user5941worker1 # [ 109.193203] nix-daemon[909]: reaped child process 1486, status = failed with exit code 15942worker2 # [ 109.350502] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:05Z level=info event=scheduler_connected addr=lb:500515943worker1 # [ 109.359719] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.5944worker1 # [ 109.364819] systemd[1]: Starting Nix worker-protocol over gRPC...5945worker1 # [ 109.521913] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:06Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5946worker1 # [ 109.523540] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:06Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5947worker1 # [ 109.526754] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:06Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15948worker1 # [ 109.533782] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:06Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5949worker1 # [ 109.535343] systemd[1]: Started Nix worker-protocol over gRPC.5950lb # [ 109.554639] niks3-server[1111]: 2026/09/20 12:22:06 INFO lead: acquired remote=192.168.1.3:553245951worker1 # [ 109.546057] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:06Z level=info event=scheduler_take_over5952worker1 # [ 109.551549] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:06Z level=info event=scheduler_connected addr=lb:500515953worker2 # [ 109.542758] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:06Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5954worker1 # [ 109.553091] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5955client # [ 109.638129] nix[1224]: scheduler at lb:50051: no healthy upstream, reconnecting5956worker2 # [ 109.744360] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:06Z level=info event=scheduler_connected addr=lb:500515957worker1 # [ 109.754222] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:06Z level=info event=scheduler_connected addr=lb:500515958worker2 # [ 109.748515] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5959worker1 # [ 109.757962] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5960worker2 # [ 109.949841] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:06Z level=info event=scheduler_connected addr=lb:500515961worker1 # [ 109.958846] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:06Z level=info event=scheduler_connected addr=lb:500515962worker2 # [ 109.954159] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5963worker1 # [ 109.963229] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5964worker2 # [ 110.156647] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:06Z level=info event=scheduler_connected addr=lb:500515965worker1 # [ 110.164085] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:06Z level=info event=scheduler_connected addr=lb:500515966worker2 # [ 110.159880] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5967worker1 # [ 110.168450] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5968worker2 # [ 110.357731] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:06Z level=info event=scheduler_connected addr=lb:500515969worker1 # [ 110.365582] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:06Z level=info event=scheduler_connected addr=lb:500515970worker1 # [ 110.367614] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5971worker2 # [ 110.361833] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5972worker1 # [ 110.566863] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:07Z level=info event=scheduler_connected addr=lb:500515973worker1 # [ 110.568448] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5974worker2 # [ 110.563533] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:07Z level=info event=scheduler_connected addr=lb:500515975worker2 # [ 110.566809] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5976client # [ 110.640621] nix[1224]: scheduler at lb:50051: no healthy upstream, reconnecting5977lb # [ 110.773530] envoy[1009]: conn peer=[2001:db8:1::1]:47054 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2930 tx=1308 ms=82545978worker1 # [ 110.768793] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:07Z level=info event=scheduler_connected addr=lb:500515979lb # [ 110.778474] envoy[1009]: conn peer=[2001:db8:1::4]:33254 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=39140 tx=2102 ms=280515980worker1 # [ 110.772579] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5981lb # [ 110.782377] envoy[1009]: conn peer=[2001:db8:1::3]:38516 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2585 tx=633 ms=167075982worker2 # [ 110.768211] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:07Z level=info event=scheduler_connected addr=lb:500515983worker2 # [ 110.772374] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5984worker1 # [ 110.973437] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:07Z level=info event=scheduler_connected addr=lb:500515985worker1 # [ 110.977397] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5986worker2 # [ 110.973395] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:07Z level=info event=scheduler_connected addr=lb:500515987worker2 # [ 110.975519] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5988worker1 # [ 111.178417] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:07Z level=info event=scheduler_connected addr=lb:500515989worker1 # [ 111.182338] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5990worker2 # [ 111.178130] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:07Z level=info event=scheduler_connected addr=lb:500515991worker2 # [ 111.181343] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5992worker1 # [ 111.382803] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:07Z level=info event=scheduler_connected addr=lb:500515993worker1 # [ 111.384596] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5994worker2 # [ 111.379071] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:07Z level=info event=scheduler_connected addr=lb:500515995worker2 # [ 111.383091] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5996worker1 # [ 111.586476] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:08Z level=info event=scheduler_connected addr=lb:500515997worker2 # [ 111.584499] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:08Z level=info event=scheduler_connected addr=lb:500515998worker1 # [ 111.593509] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:08Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:608305999worker1 # [ 111.597142] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:08Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06000worker1 # [ 111.609138] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:08Z level=info event=oidc_jwks provider=mock keys=16001worker1 # [ 111.612892] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:08Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:608306002worker1 # [ 111.616677] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:08Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06003worker1 # [ 112.673941] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:09Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06004worker1 # [ 112.678102] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:09Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=16005worker1 # [ 112.719139] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608426006worker1 # [ 112.730859] nix-daemon[909]: accepted connection from pid 1522, user nix-grpc-daemon (trusted)6007worker1 # [ 112.744179] nix-daemon[909]: accepted connection from pid 1522, user nix-grpc-daemon (trusted)6008client # [ 112.737050] nix[1224]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6009worker1 # [ 112.758818] nix-daemon[909]: accepted connection from pid 1522, user nix-grpc-daemon (trusted)6010client: (finished: must succeed: sleep 5.0, in 5.05 seconds)6011client: waiting for success: ! systemctl is-active mid6012worker1 # [ 114.577811] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:11Z level=info event=niks3_push_started pid=16016013worker1 # [ 114.608372] nix-daemon[909]: reaped child process 1577, status = succeeded6014worker1 # [ 114.797326] nix-grpc-daemon[1601]: time=2026-09-20T12:22:11.299Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""6015worker1 # [ 114.950379] nix-daemon[909]: accepted connection from pid 1614, user nix-grpc-daemon (trusted)6016worker1 # [ 114.961536] nix-daemon[909]: reaped child process 1617, status = succeeded6017lb # [ 114.988500] niks3-server[1111]: 2026/09/20 12:22:11 INFO Received uploads request method=POST path=/api/pending_closures6018worker1 # [ 114.988320] nix-grpc-daemon[1601]: time=2026-09-20T12:22:11.491Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6019worker1 # [ 114.989808] nix-grpc-daemon[1601]: time=2026-09-20T12:22:11.492Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6020lb # [ 115.050972] niks3-server[1111]: 2026/09/20 12:22:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign6021lb # [ 115.056063] niks3-server[1111]: 2026/09/20 12:22:11 INFO Signed narinfos id=36 count=16022worker1 # [ 115.046917] nix-grpc-daemon[1601]: time=2026-09-20T12:22:11.549Z level=INFO msg="Uploading 1 narinfos"6023lb # [ 115.067256] niks3-server[1111]: 2026/09/20 12:22:11 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete6024lb # [ 115.327935] niks3-server[1111]: 2026/09/20 12:22:11 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6025lb # [ 115.360344] niks3-server[1111]: 2026/09/20 12:22:11 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6026lb # [ 115.369482] niks3-server[1111]: 2026/09/20 12:22:11 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6027lb # [ 115.384877] niks3-server[1111]: 2026/09/20 12:22:12 INFO Completed upload id=366028worker1 # [ 115.376319] nix-grpc-daemon[1601]: time=2026-09-20T12:22:11.879Z level=INFO msg="Upload complete. (575ms)"6029worker1 # [ 115.383612] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60842 duration_s=2 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16030lb # [ 115.398789] niks3-server[1111]: 2026/09/20 12:22:12 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo6031worker1 # [ 115.403372] nix-daemon[909]: reaped child process 1575, status = succeeded6032client # [ 115.427564] systemd[1]: mid.service: Deactivated successfully.6033client # [ 115.431217] systemd[1]: mid.service: Consumed 131ms CPU time over 10.055s wall clock time, 37.3M memory peak, 10.2K incoming IP traffic, 11.7K outgoing IP traffic.6034client: (finished: waiting for success: ! systemctl is-active mid, in 1.11 seconds)6035client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6036client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.04 seconds)6037(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.27 seconds)6038subtest: interrupting the client stops the build on the worker6039client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr6040client # [ 115.620489] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr.6041client # Running as unit: intr.service; invocation ID: a89830a5f4a24b27b19de0581fe353ca6042client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr, in 0.04 seconds)6043worker2 # [ 115.820811] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:12Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:420926044worker2 # [ 115.826441] nix-daemon[924]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6045client # [ 115.819108] nix[1260]: don't know how to build these paths:6046client # [ 115.820070] nix[1260]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6047worker1 # [ 115.847756] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:12Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06048worker1 # [ 115.849566] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:12Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=16049worker2 # [ 115.886164] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:420926050worker2 # [ 115.899682] nix-grpc-daemon[1429]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 99 ms (attempt 1/5)6051worker2 # [ 116.004606] nix-grpc-daemon[1429]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 54 ms (attempt 2/5)6052worker2 # [ 116.064674] nix-grpc-daemon[1429]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 60 ms (attempt 3/5)6053worker2 # [ 116.130733] nix-grpc-daemon[1429]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 432 ms (attempt 4/5)6054worker2 # [ 116.569247] nix-grpc-daemon[1429]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6055worker2 # [ 116.587307] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:420926056client # [ 116.583457] nix[1260]: copying 1 paths to 'lb:50051'...6057client # [ 116.587144] nix[1260]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6058worker2 # [ 116.601958] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:420926059worker2 # [ 116.614156] nix-daemon[924]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6060worker2 # [ 116.634491] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:13Z level=info event=niks3_push_started pid=15176061worker2 # [ 116.825039] nix-grpc-daemon[1517]: time=2026-09-20T12:22:13.297Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""6062worker2 # [ 116.990883] nix-daemon[924]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)6063worker2 # [ 117.004184] nix-daemon[924]: reaped child process 1537, status = succeeded6064worker2 # [ 117.176353] nix-daemon[924]: accepted connection from pid 1539, user nix-grpc-daemon (trusted)6065worker2 # [ 117.189362] nix-daemon[924]: reaped child process 1542, status = succeeded6066lb # [ 117.220321] niks3-server[1111]: 2026/09/20 12:22:13 INFO Received uploads request method=POST path=/api/pending_closures6067worker2 # [ 117.217028] nix-grpc-daemon[1517]: time=2026-09-20T12:22:13.689Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6068worker2 # [ 117.218465] nix-grpc-daemon[1517]: time=2026-09-20T12:22:13.689Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6069lb # [ 117.254618] niks3-server[1111]: 2026/09/20 12:22:13 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6070lb # [ 117.263842] niks3-server[1111]: 2026/09/20 12:22:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign6071lb # [ 117.268770] niks3-server[1111]: 2026/09/20 12:22:13 INFO Signed narinfos id=37 count=16072worker2 # [ 117.256709] nix-grpc-daemon[1517]: time=2026-09-20T12:22:13.729Z level=INFO msg="Uploading 1 narinfos"6073lb # [ 117.279763] niks3-server[1111]: 2026/09/20 12:22:13 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6074lb # [ 117.288842] niks3-server[1111]: 2026/09/20 12:22:13 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete6075lb # [ 117.297406] niks3-server[1111]: 2026/09/20 12:22:13 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6076lb # [ 117.300728] niks3-server[1111]: 2026/09/20 12:22:13 INFO Completed upload id=376077worker2 # [ 117.285755] nix-grpc-daemon[1517]: time=2026-09-20T12:22:13.758Z level=INFO msg="Upload complete. (456ms)"6078worker2 # [ 117.288726] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42092 duration_s=0 paths=1 nar_bytes_in=4246079worker2 # [ 117.292855] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:420926080worker2 # [ 117.303065] nix-daemon[924]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6081worker2 # [ 117.306201] nix-daemon[924]: reaped child process 1515, status = succeeded6082worker2 # [ 117.311301] nix-daemon[924]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6083client # [ 117.297554] nix[1260]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6084client: must succeed: systemctl kill -s INT intr6085client # [ 119.120607] systemd[1]: intr.service: Sent signal SIGINT to main process 1260 (nix) on client request.6086client # [ 119.123091] nix[1260]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6087client: (finished: must succeed: systemctl kill -s INT intr, in 0.04 seconds)6088worker2 # [ 119.138270] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:15Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6089client # [ 119.126338] nix[1260]: error: interrupted by the user6090client # [ 119.153496] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6091client # [ 119.154773] systemd[1]: intr.service: Failed with result 'exit-code'.6092client # [ 119.156261] systemd[1]: intr.service: Consumed 102ms CPU time over 3.532s wall clock time, 39.3M memory peak, 7.8K incoming IP traffic, 8.7K outgoing IP traffic.6093worker2 # [ 119.329591] nix-daemon[924]: reaped child process 1544, status = succeeded6094worker2 # [ 119.334268] nix-daemon[1546]: unexpected Nix daemon error: error: interrupted by the user6095worker2 # [ 119.338379] nix-daemon[924]: reaped child process 1546, status = failed with exit code 16096(finished: subtest: interrupting the client stops the build on the worker, in 4.79 seconds)6097subtest: a deploy mid-build drains: the build finishes, the new generation takes over6098client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw6099client # [ 120.429519] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw.6100client # Running as unit: sw.service; invocation ID: 17f0dc7e35ad4e96ac9de644da63590c6101client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw, in 0.07 seconds)6102worker1 # [ 120.653644] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:608426103client # [ 120.637224] nix[1281]: don't know how to build these paths:6104client # [ 120.638194] nix[1281]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6105worker1 # [ 120.662044] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:17Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06106worker1 # [ 120.664774] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:17Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=16107worker1 # [ 120.705230] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608426108worker1 # [ 120.711718] nix-grpc-daemon[1522]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 60 ms (attempt 1/5)6109lb # [ 120.773287] envoy[1009]: conn peer=[2001:db8:1::1]:48890 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4222 tx=1794 ms=98406110lb # [ 120.774992] envoy[1009]: conn peer=[2001:db8:1::1]:60402 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2924 tx=753 ms=33376111worker1 # [ 120.777543] nix-grpc-daemon[1522]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 119 ms (attempt 2/5)6112worker1 # [ 120.902506] nix-grpc-daemon[1522]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 261 ms (attempt 3/5)6113worker1 # [ 121.169300] nix-grpc-daemon[1522]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 407 ms (attempt 4/5)6114worker1 # [ 121.579578] nix-grpc-daemon[1522]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6115worker1 # [ 121.593212] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:608426116client # [ 121.583688] nix[1281]: copying 1 paths to 'lb:50051'...6117client # [ 121.587141] nix[1281]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6118worker1 # [ 121.608791] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:608426119worker1 # [ 121.621847] nix-daemon[909]: accepted connection from pid 1522, user nix-grpc-daemon (trusted)6120worker1 # [ 121.797319] nix-daemon[909]: accepted connection from pid 1660, user nix-grpc-daemon (trusted)6121worker1 # [ 121.809968] nix-daemon[909]: reaped child process 1663, status = succeeded6122worker1 # [ 121.979583] nix-daemon[909]: accepted connection from pid 1665, user nix-grpc-daemon (trusted)6123worker1 # [ 121.992209] nix-daemon[909]: reaped child process 1668, status = succeeded6124lb # [ 122.017847] niks3-server[1111]: 2026/09/20 12:22:18 INFO Received uploads request method=POST path=/api/pending_closures6125worker1 # [ 122.019522] nix-grpc-daemon[1601]: time=2026-09-20T12:22:18.521Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6126worker1 # [ 122.021282] nix-grpc-daemon[1601]: time=2026-09-20T12:22:18.521Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6127lb # [ 122.046460] niks3-server[1111]: 2026/09/20 12:22:18 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6128lb # [ 122.058375] niks3-server[1111]: 2026/09/20 12:22:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6129lb # [ 122.064085] niks3-server[1111]: 2026/09/20 12:22:18 INFO Signed narinfos id=38 count=16130worker1 # [ 122.058511] nix-grpc-daemon[1601]: time=2026-09-20T12:22:18.561Z level=INFO msg="Uploading 1 narinfos"6131lb # [ 122.080252] niks3-server[1111]: 2026/09/20 12:22:18 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6132lb # [ 122.087384] niks3-server[1111]: 2026/09/20 12:22:18 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6133lb # [ 122.096663] niks3-server[1111]: 2026/09/20 12:22:18 INFO Completed upload id=386134worker1 # [ 122.087331] nix-grpc-daemon[1601]: time=2026-09-20T12:22:18.590Z level=INFO msg="Upload complete. (452ms)"6135lb # [ 122.100660] niks3-server[1111]: 2026/09/20 12:22:18 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6136worker1 # [ 122.093275] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60842 duration_s=0 paths=1 nar_bytes_in=4166137worker1 # [ 122.099573] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608426138worker1 # [ 122.109702] nix-daemon[909]: accepted connection from pid 1522, user nix-grpc-daemon (trusted)6139worker1 # [ 122.119257] nix-daemon[909]: reaped child process 1658, status = succeeded6140worker1 # [ 122.124291] nix-daemon[909]: accepted connection from pid 1522, user nix-grpc-daemon (trusted)6141client # [ 122.104396] nix[1281]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6142worker1: must succeed: systemctl reload nix-grpc-daemon.service6143worker1 # [ 124.094690] systemd[1]: Reloading Nix worker-protocol over gRPC...6144worker1 # [ 124.132266] systemd[1]: Reloaded Nix worker-protocol over gRPC.6145worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.08 seconds)6146worker1: must succeed: pgrep -f 'read -t [9]0 x'6147worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.04 seconds)6148client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&26149client # don't know how to build these paths:6150client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6151worker2 # [ 124.369636] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:20Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:420926152worker2 # [ 124.375294] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:420926153worker1 # [ 124.401637] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:20Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06154worker1 # [ 124.404146] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:20Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=16155worker2 # [ 124.442953] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:420926156worker2 # [ 124.455332] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:420926157client # copying 1 paths to 'lb:50051'...6158worker2 # [ 124.468198] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:420926159client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6160worker2 # [ 124.494668] nix-daemon[924]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6161worker1 # [ 124.539598] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:21Z level=info event=draining6162worker2 # [ 124.667453] nix-daemon[924]: accepted connection from pid 1600, user nix-grpc-daemon (trusted)6163worker2 # [ 124.681347] nix-daemon[924]: reaped child process 1603, status = succeeded6164worker2 # [ 124.844254] nix-daemon[924]: accepted connection from pid 1605, user nix-grpc-daemon (trusted)6165worker2 # [ 124.858116] nix-daemon[924]: reaped child process 1608, status = succeeded6166lb # [ 124.890815] niks3-server[1111]: 2026/09/20 12:22:21 INFO Received uploads request method=POST path=/api/pending_closures6167worker2 # [ 124.884840] nix-grpc-daemon[1517]: time=2026-09-20T12:22:21.357Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6168worker2 # [ 124.886397] nix-grpc-daemon[1517]: time=2026-09-20T12:22:21.359Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6169lb # [ 124.922987] niks3-server[1111]: 2026/09/20 12:22:21 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6170lb # [ 124.930880] niks3-server[1111]: 2026/09/20 12:22:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6171lb # [ 124.937070] niks3-server[1111]: 2026/09/20 12:22:21 INFO Signed narinfos id=39 count=16172worker2 # [ 124.924968] nix-grpc-daemon[1517]: time=2026-09-20T12:22:21.397Z level=INFO msg="Uploading 1 narinfos"6173lb # [ 124.942742] niks3-server[1111]: 2026/09/20 12:22:21 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6174lb # [ 124.951440] niks3-server[1111]: 2026/09/20 12:22:21 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6175worker2 # [ 124.941948] nix-grpc-daemon[1517]: time=2026-09-20T12:22:21.415Z level=INFO msg="Upload complete. (435ms)"6176lb # [ 124.957574] niks3-server[1111]: 2026/09/20 12:22:21 INFO Completed upload id=396177worker2 # [ 124.944933] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42092 duration_s=0 paths=1 nar_bytes_in=5286178lb # [ 124.962763] niks3-server[1111]: 2026/09/20 12:22:21 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6179worker2 # [ 124.950328] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:420926180worker2 # [ 124.960060] nix-daemon[924]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6181worker2 # [ 124.964781] nix-daemon[924]: reaped child process 1598, status = succeeded6182client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6183client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6184worker2 # [ 124.969976] nix-daemon[924]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6185client # farm-a> LOG-farm-a-during-drain6186worker2 # [ 126.398224] nix-daemon[924]: reaped child process 1612, status = succeeded6187worker2 # [ 126.543103] nix-daemon[924]: accepted connection from pid 1627, user nix-grpc-daemon (trusted)6188worker2 # [ 126.557453] nix-daemon[924]: reaped child process 1630, status = succeeded6189lb # [ 126.588965] niks3-server[1111]: 2026/09/20 12:22:23 INFO Received uploads request method=POST path=/api/pending_closures6190worker2 # [ 126.581888] nix-grpc-daemon[1517]: time=2026-09-20T12:22:23.054Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6191worker2 # [ 126.585067] nix-grpc-daemon[1517]: time=2026-09-20T12:22:23.057Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6192lb # [ 126.636852] niks3-server[1111]: 2026/09/20 12:22:23 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6193lb # [ 126.648145] niks3-server[1111]: 2026/09/20 12:22:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6194lb # [ 126.653506] niks3-server[1111]: 2026/09/20 12:22:23 INFO Signed narinfos id=40 count=16195worker2 # [ 126.641175] nix-grpc-daemon[1517]: time=2026-09-20T12:22:23.113Z level=INFO msg="Uploading 1 narinfos"6196lb # [ 126.656255] niks3-server[1111]: 2026/09/20 12:22:23 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6197lb # [ 126.672534] niks3-server[1111]: 2026/09/20 12:22:23 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6198lb # [ 126.676414] niks3-server[1111]: 2026/09/20 12:22:23 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6199lb # [ 126.679177] niks3-server[1111]: 2026/09/20 12:22:23 INFO Completed upload id=406200worker2 # [ 126.664293] nix-grpc-daemon[1517]: time=2026-09-20T12:22:23.137Z level=INFO msg="Upload complete. (285ms)"6201lb # [ 126.680821] niks3-server[1111]: 2026/09/20 12:22:23 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6202worker1 # [ 126.679400] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:23Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06203worker2 # [ 126.670312] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42092 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16204worker1 # [ 126.681258] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:23Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=16205worker2 # [ 126.681850] nix-daemon[924]: reaped child process 1610, status = succeeded6206worker2 # [ 126.722409] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:420926207worker2 # [ 126.734675] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:420926208worker2 # [ 126.747235] nix-grpc-daemon[1517]: time=2026-09-20T12:22:23.220Z level=INFO msg="All 1 paths already cached"6209client # copying 1 paths to 'lb:50051'...6210worker2 # [ 126.755167] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:420926211client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6212worker2 # [ 126.774264] nix-daemon[924]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6213worker2 # [ 126.954379] nix-daemon[924]: accepted connection from pid 1634, user nix-grpc-daemon (trusted)6214worker2 # [ 126.967796] nix-daemon[924]: reaped child process 1637, status = succeeded6215worker2 # [ 127.127718] nix-daemon[924]: accepted connection from pid 1639, user nix-grpc-daemon (trusted)6216worker2 # [ 127.141511] nix-daemon[924]: reaped child process 1642, status = succeeded6217lb # [ 127.172831] niks3-server[1111]: 2026/09/20 12:22:23 INFO Received uploads request method=POST path=/api/pending_closures6218worker2 # [ 127.165233] nix-grpc-daemon[1517]: time=2026-09-20T12:22:23.638Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6219worker2 # [ 127.167365] nix-grpc-daemon[1517]: time=2026-09-20T12:22:23.640Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6220lb # [ 127.200894] niks3-server[1111]: 2026/09/20 12:22:23 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6221lb # [ 127.210354] niks3-server[1111]: 2026/09/20 12:22:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6222lb # [ 127.214461] niks3-server[1111]: 2026/09/20 12:22:23 INFO Signed narinfos id=41 count=16223worker2 # [ 127.202620] nix-grpc-daemon[1517]: time=2026-09-20T12:22:23.675Z level=INFO msg="Uploading 1 narinfos"6224lb # [ 127.221171] niks3-server[1111]: 2026/09/20 12:22:23 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6225lb # [ 127.232506] niks3-server[1111]: 2026/09/20 12:22:23 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6226worker2 # [ 127.223636] nix-grpc-daemon[1517]: time=2026-09-20T12:22:23.696Z level=INFO msg="Upload complete. (436ms)"6227lb # [ 127.238685] niks3-server[1111]: 2026/09/20 12:22:23 INFO Completed upload id=416228worker2 # [ 127.225933] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42092 duration_s=0 paths=1 nar_bytes_in=6406229lb # [ 127.242541] niks3-server[1111]: 2026/09/20 12:22:23 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6230worker2 # [ 127.231291] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:420926231worker2 # [ 127.242232] nix-daemon[924]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6232worker2 # [ 127.246133] nix-daemon[924]: reaped child process 1632, status = succeeded6233worker2 # [ 127.249446] nix-daemon[924]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6234client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6235client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6236client # farm-b> LOG-farm-b-during-drain6237worker2 # [ 128.467333] nix-daemon[924]: reaped child process 1646, status = succeeded6238worker2 # [ 128.628546] nix-daemon[924]: accepted connection from pid 1661, user nix-grpc-daemon (trusted)6239worker2 # [ 128.643508] nix-daemon[924]: reaped child process 1664, status = succeeded6240lb # [ 128.672783] niks3-server[1111]: 2026/09/20 12:22:25 INFO Received uploads request method=POST path=/api/pending_closures6241worker2 # [ 128.662775] nix-grpc-daemon[1517]: time=2026-09-20T12:22:25.135Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6242worker2 # [ 128.664332] nix-grpc-daemon[1517]: time=2026-09-20T12:22:25.137Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6243lb # [ 128.694413] niks3-server[1111]: 2026/09/20 12:22:25 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6244lb # [ 128.698264] niks3-server[1111]: 2026/09/20 12:22:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6245worker2 # [ 128.686775] nix-grpc-daemon[1517]: time=2026-09-20T12:22:25.159Z level=INFO msg="Uploading 1 narinfos"6246lb # [ 128.702100] niks3-server[1111]: 2026/09/20 12:22:25 INFO Signed narinfos id=42 count=16247lb # [ 128.705046] niks3-server[1111]: 2026/09/20 12:22:25 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6248lb # [ 128.707899] niks3-server[1111]: 2026/09/20 12:22:25 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6249lb # [ 128.713491] niks3-server[1111]: 2026/09/20 12:22:25 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6250lb # [ 128.717241] niks3-server[1111]: 2026/09/20 12:22:25 INFO Completed upload id=426251worker2 # [ 128.702190] nix-grpc-daemon[1517]: time=2026-09-20T12:22:25.175Z level=INFO msg="Upload complete. (244ms)"6252lb # [ 128.718780] niks3-server[1111]: 2026/09/20 12:22:25 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6253worker2 # [ 128.704886] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42092 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16254worker2 # [ 128.712100] nix-daemon[924]: reaped child process 1644, status = succeeded6255worker1 # [ 128.718746] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:25Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06256worker1 # [ 128.720766] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:25Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=16257worker2 # [ 128.758402] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:420926258worker2 # [ 128.769243] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:420926259worker2 # [ 128.779890] nix-grpc-daemon[1517]: time=2026-09-20T12:22:25.252Z level=INFO msg="All 2 paths already cached"6260client # copying 1 paths to 'lb:50051'...6261client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6262worker2 # [ 128.788070] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:420926263worker2 # [ 128.796808] nix-daemon[924]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6264worker2 # [ 128.967637] nix-daemon[924]: accepted connection from pid 1668, user nix-grpc-daemon (trusted)6265worker2 # [ 128.980886] nix-daemon[924]: reaped child process 1671, status = succeeded6266worker2 # [ 129.154031] nix-daemon[924]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)6267worker2 # [ 129.167564] nix-daemon[924]: reaped child process 1676, status = succeeded6268lb # [ 129.198555] niks3-server[1111]: 2026/09/20 12:22:25 INFO Received uploads request method=POST path=/api/pending_closures6269worker2 # [ 129.191592] nix-grpc-daemon[1517]: time=2026-09-20T12:22:25.664Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6270worker2 # [ 129.193629] nix-grpc-daemon[1517]: time=2026-09-20T12:22:25.666Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6271lb # [ 129.229494] niks3-server[1111]: 2026/09/20 12:22:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6272lb # [ 129.234866] niks3-server[1111]: 2026/09/20 12:22:25 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6273worker2 # [ 129.225416] nix-grpc-daemon[1517]: time=2026-09-20T12:22:25.698Z level=INFO msg="Uploading 1 narinfos"6274lb # [ 129.240543] niks3-server[1111]: 2026/09/20 12:22:25 INFO Signed narinfos id=43 count=16275lb # [ 129.246970] niks3-server[1111]: 2026/09/20 12:22:25 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6276lb # [ 129.259102] niks3-server[1111]: 2026/09/20 12:22:25 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6277worker2 # [ 129.249982] nix-grpc-daemon[1517]: time=2026-09-20T12:22:25.723Z level=INFO msg="Upload complete. (441ms)"6278lb # [ 129.265193] niks3-server[1111]: 2026/09/20 12:22:25 INFO Completed upload id=436279worker2 # [ 129.252146] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42092 duration_s=0 paths=1 nar_bytes_in=7686280lb # [ 129.268674] niks3-server[1111]: 2026/09/20 12:22:25 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6281worker2 # [ 129.258570] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:420926282worker2 # [ 129.263615] nix-daemon[924]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6283worker2 # [ 129.265981] nix-daemon[924]: reaped child process 1666, status = succeeded6284worker2 # [ 129.271637] nix-daemon[924]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6285client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6286client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6287client # farm-top> LOG-farm-top-during-drain6288worker2 # [ 130.702770] nix-daemon[924]: reaped child process 1680, status = succeeded6289worker2 # [ 130.881685] nix-daemon[924]: accepted connection from pid 1695, user nix-grpc-daemon (trusted)6290worker2 # [ 130.895176] nix-daemon[924]: reaped child process 1698, status = succeeded6291lb # [ 130.926534] niks3-server[1111]: 2026/09/20 12:22:27 INFO Received uploads request method=POST path=/api/pending_closures6292worker2 # [ 130.920660] nix-grpc-daemon[1517]: time=2026-09-20T12:22:27.393Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6293worker2 # [ 130.923054] nix-grpc-daemon[1517]: time=2026-09-20T12:22:27.395Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6294lb # [ 130.957746] niks3-server[1111]: 2026/09/20 12:22:27 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6295lb # [ 130.970348] niks3-server[1111]: 2026/09/20 12:22:27 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6296lb # [ 130.981305] niks3-server[1111]: 2026/09/20 12:22:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6297worker2 # [ 130.973246] nix-grpc-daemon[1517]: time=2026-09-20T12:22:27.446Z level=INFO msg="Uploading 1 narinfos"6298lb # [ 130.987936] niks3-server[1111]: 2026/09/20 12:22:27 INFO Signed narinfos id=44 count=16299lb # [ 130.991447] niks3-server[1111]: 2026/09/20 12:22:27 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6300lb # [ 131.002748] niks3-server[1111]: 2026/09/20 12:22:27 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6301lb # [ 131.007922] niks3-server[1111]: 2026/09/20 12:22:27 INFO Completed upload id=446302worker2 # [ 130.992941] nix-grpc-daemon[1517]: time=2026-09-20T12:22:27.466Z level=INFO msg="Upload complete. (312ms)"6303lb # [ 131.009872] niks3-server[1111]: 2026/09/20 12:22:27 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6304worker2 # [ 130.996421] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42092 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16305worker2 # [ 131.005308] nix-daemon[924]: reaped child process 1678, status = succeeded6306client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&2, in 6.86 seconds)6307client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath6308client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath, in 0.17 seconds)6309worker1: must succeed: pkill -f 'read -t [9]0 x'6310worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6311client: waiting for success: ! systemctl is-active sw6312worker1 # [ 131.347291] nix-daemon[909]: reaped child process 1672, status = succeeded6313worker1 # [ 131.483674] nix-daemon[909]: accepted connection from pid 1718, user nix-grpc-daemon (trusted)6314worker1 # [ 131.496567] nix-daemon[909]: reaped child process 1721, status = succeeded6315lb # [ 131.522129] niks3-server[1111]: 2026/09/20 12:22:28 INFO Received uploads request method=POST path=/api/pending_closures6316worker1 # [ 131.522237] nix-grpc-daemon[1601]: time=2026-09-20T12:22:28.024Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6317worker1 # [ 131.524060] nix-grpc-daemon[1601]: time=2026-09-20T12:22:28.026Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6318lb # [ 131.561525] niks3-server[1111]: 2026/09/20 12:22:28 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6319lb # [ 131.573557] niks3-server[1111]: 2026/09/20 12:22:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6320lb # [ 131.578720] niks3-server[1111]: 2026/09/20 12:22:28 INFO Signed narinfos id=45 count=16321worker1 # [ 131.573695] nix-grpc-daemon[1601]: time=2026-09-20T12:22:28.076Z level=INFO msg="Uploading 1 narinfos"6322lb # [ 131.584358] niks3-server[1111]: 2026/09/20 12:22:28 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6323lb # [ 131.591057] niks3-server[1111]: 2026/09/20 12:22:28 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6324lb # [ 131.605671] niks3-server[1111]: 2026/09/20 12:22:28 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6325lb # [ 131.608974] niks3-server[1111]: 2026/09/20 12:22:28 INFO Completed upload id=456326lb # [ 131.610545] niks3-server[1111]: 2026/09/20 12:22:28 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6327worker1 # [ 131.603442] nix-grpc-daemon[1601]: time=2026-09-20T12:22:28.104Z level=INFO msg="Upload complete. (262ms)"6328worker1 # [ 131.610188] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60842 duration_s=9 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16329worker1 # [ 131.622634] nix-daemon[909]: reaped child process 1670, status = succeeded6330client # [ 131.621740] systemd[1]: sw.service: Deactivated successfully.6331client # [ 131.626100] systemd[1]: sw.service: Consumed 117ms CPU time over 11.194s wall clock time, 37.1M memory peak, 8.9K incoming IP traffic, 9K outgoing IP traffic.6332client: (finished: waiting for success: ! systemctl is-active sw, in 1.08 seconds)6333client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6334client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds)6335worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6336worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 0.05 seconds)6337lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6338lb: must succeed: curl -sf localhost:9901/clusters6339lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6340worker1 # [ 132.542632] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:29Z level=info event=signal_exit6341worker1 # [ 132.767873] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:29Z level=info event=scheduler_yield6342lb # [ 132.778341] niks3-server[1111]: 2026/09/20 12:22:29 INFO lead: released remote=192.168.1.3:553246343worker2 # [ 132.764520] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:29Z level=info event=scheduler_restarting addr=lb:500516344worker1 # [ 132.773419] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:29Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:608306345worker1 # [ 132.779330] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:29Z level=info event=worker_gone addr=192.168.1.4:500516346worker1 # [ 132.782432] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:29Z level=info event=scheduler_restarting addr=lb:500516347worker1 # [ 132.785391] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:29Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:608306348worker1 # [ 132.788640] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:29Z level=info event=worker_gone addr=192.168.1.3:500516349worker2 # [ 132.965294] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:29Z level=info event=scheduler_connected addr=lb:500516350worker2 # [ 132.969318] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6351worker1 # [ 132.978846] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:29Z level=info event=scheduler_connected addr=lb:500516352worker1 # [ 132.982924] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6353worker2 # [ 133.170091] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:29Z level=info event=scheduler_connected addr=lb:500516354worker2 # [ 133.172790] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6355worker1 # [ 133.183934] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:29Z level=info event=scheduler_connected addr=lb:500516356worker1 # [ 133.187924] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6357worker2 # [ 133.373331] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:29Z level=info event=scheduler_connected addr=lb:500516358worker2 # [ 133.375082] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6359worker1 # [ 133.388785] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:29Z level=info event=scheduler_connected addr=lb:500516360worker1 # [ 133.392795] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6361lb: must succeed: curl -sf localhost:9901/clusters6362lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6363worker2 # [ 133.576561] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:30Z level=info event=scheduler_connected addr=lb:500516364worker2 # [ 133.580988] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6365worker1 # [ 133.593295] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:30Z level=info event=scheduler_connected addr=lb:500516366worker1 # [ 133.597174] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6367worker1 # [ 133.768496] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:30Z level=info event=scheduler_restarting6368worker1 # [ 133.786810] nix-daemon[909]: reaped child process 1573, status = succeeded6369worker2 # [ 133.782117] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:30Z level=info event=scheduler_connected addr=lb:500516370worker2 # [ 133.783880] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6371worker1 # [ 133.807389] nix-grpc-daemon[1522]: ts=2026-09-20T12:22:30Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6372worker2 # [ 133.984881] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:30Z level=info event=scheduler_connected addr=lb:500516373worker2 # [ 133.986728] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6374worker2 # [ 134.188214] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:30Z level=info event=scheduler_connected addr=lb:500516375worker2 # [ 134.192510] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6376worker2 # [ 134.393930] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:30Z level=info event=scheduler_connected addr=lb:500516377worker2 # [ 134.398589] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6378lb: must succeed: curl -sf localhost:9901/clusters6379worker1 # [ 134.591152] nix-grpc-daemon[1522]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6380worker1 # [ 134.594557] nix-grpc-daemon[1522]: E0000 00:00:1789906951.093758 1565 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true6381worker1 # [ 134.599397] nix-grpc-daemon[1522]: E0000 00:00:1789906951.093937 1565 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6382worker2 # [ 134.599688] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:31Z level=info event=scheduler_connected addr=lb:500516383worker2 # [ 134.603624] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6384lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6385worker2 # [ 134.805452] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:31Z level=info event=scheduler_connected addr=lb:500516386worker2 # [ 134.809397] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6387worker2 # [ 134.851747] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:31Z level=info event=scheduler_take_over6388lb # [ 134.867895] niks3-server[1111]: 2026/09/20 12:22:31 INFO lead: acquired remote=192.168.1.4:354786389worker2 # [ 135.010548] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:31Z level=info event=scheduler_connected addr=lb:500516390worker2 # [ 135.014613] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6391worker2 # [ 135.216079] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:31Z level=info event=scheduler_connected addr=lb:500516392worker2 # [ 135.220228] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6393worker2 # [ 135.421448] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:31Z level=info event=scheduler_connected addr=lb:500516394worker2 # [ 135.425358] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6395worker1 # [ 135.579818] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6396worker1 # [ 135.583914] systemd[1]: nix-grpc-daemon.service: Consumed 691ms CPU time over 26.412s wall clock time, 31.6M memory peak, 28.3K incoming IP traffic, 38.7K outgoing IP traffic.6397lb: must succeed: curl -sf localhost:9901/clusters6398worker2 # [ 135.626545] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:32Z level=info event=scheduler_connected addr=lb:500516399worker2 # [ 135.630416] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6400lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6401worker2 # [ 135.832235] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:32Z level=info event=scheduler_connected addr=lb:500516402worker2 # [ 135.850190] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:32Z level=info event=oidc_jwks provider=mock keys=16403worker2 # [ 135.854504] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:32Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:399726404worker2 # [ 135.858824] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:32Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06405worker1 # [ 136.566192] systemd[1]: Starting Nix worker-protocol over gRPC...6406lb: must succeed: curl -sf localhost:9901/clusters6407worker1 # [ 136.725802] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:33Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6408worker1 # [ 136.728060] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:33Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6409worker1 # [ 136.732037] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:33Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16410worker1 # [ 136.739236] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:33Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6411worker1 # [ 136.741076] systemd[1]: Started Nix worker-protocol over gRPC.6412worker1 # [ 136.761715] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:33Z level=info event=scheduler_connected addr=lb:500516413worker2 # [ 136.757870] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:33Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:399726414lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6415worker2 # [ 136.762222] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:33Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06416lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 4.35 seconds)6417lb: must succeed: curl -sf localhost:9901/clusters6418lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6419worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6420worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6421(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 16.49 seconds)6422subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6423client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16424worker1 # [ 137.114842] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:33Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:415946425worker1 # [ 137.120508] nix-daemon[909]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6426worker2 # [ 137.122546] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:420926427worker2 # [ 137.164162] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:33Z level=debug event=want client=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06428worker2 # [ 137.166039] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:33Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16429worker2 # [ 137.167750] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:33Z level=debug event=want client=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06430worker2 # [ 137.169314] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:33Z level=debug event=want client=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06431worker1 # [ 137.214732] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415946432worker1 # [ 137.221494] nix-grpc-daemon[1733]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 82 ms (attempt 1/5)6433worker1 # [ 137.306359] nix-grpc-daemon[1733]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 131 ms (attempt 2/5)6434worker1 # [ 137.439991] nix-grpc-daemon[1733]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 197 ms (attempt 3/5)6435worker1 # [ 137.639760] nix-grpc-daemon[1733]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 163 ms (attempt 4/5)6436worker1 # [ 137.809803] nix-grpc-daemon[1733]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6437worker1 # [ 137.827077] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:415946438worker1 # [ 137.840331] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:415946439worker1 # [ 137.848638] nix-daemon[909]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6440worker1 # [ 137.871112] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:34Z level=info event=niks3_push_started pid=17896441worker1 # [ 138.053712] nix-grpc-daemon[1789]: time=2026-09-20T12:22:34.556Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""6442worker1 # [ 138.214668] nix-daemon[909]: accepted connection from pid 1802, user nix-grpc-daemon (trusted)6443worker1 # [ 138.228055] nix-daemon[909]: reaped child process 1805, status = succeeded6444worker1 # [ 138.389226] nix-daemon[909]: accepted connection from pid 1807, user nix-grpc-daemon (trusted)6445worker1 # [ 138.401704] nix-daemon[909]: reaped child process 1810, status = succeeded6446lb # [ 138.425242] niks3-server[1111]: 2026/09/20 12:22:35 INFO Received uploads request method=POST path=/api/pending_closures6447worker1 # [ 138.427084] nix-grpc-daemon[1789]: time=2026-09-20T12:22:34.929Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6448worker1 # [ 138.428516] nix-grpc-daemon[1789]: time=2026-09-20T12:22:34.929Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6449lb # [ 138.457972] niks3-server[1111]: 2026/09/20 12:22:35 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6450lb # [ 138.468722] niks3-server[1111]: 2026/09/20 12:22:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6451worker1 # [ 138.465101] nix-grpc-daemon[1789]: time=2026-09-20T12:22:34.967Z level=INFO msg="Uploading 1 narinfos"6452lb # [ 138.473536] niks3-server[1111]: 2026/09/20 12:22:35 INFO Signed narinfos id=46 count=16453lb # [ 138.485193] niks3-server[1111]: 2026/09/20 12:22:35 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6454lb # [ 138.491131] niks3-server[1111]: 2026/09/20 12:22:35 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6455lb # [ 138.498178] niks3-server[1111]: 2026/09/20 12:22:35 INFO Completed upload id=466456worker1 # [ 138.489552] nix-grpc-daemon[1789]: time=2026-09-20T12:22:34.992Z level=INFO msg="Upload complete. (432ms)"6457lb # [ 138.500628] niks3-server[1111]: 2026/09/20 12:22:35 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6458worker1 # [ 138.493545] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41594 duration_s=0 paths=1 nar_bytes_in=5046459worker1 # [ 138.498953] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415946460worker1 # [ 138.507143] nix-daemon[909]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6461worker1 # [ 138.514125] nix-daemon[909]: reaped child process 1787, status = succeeded6462worker1 # [ 138.517510] nix-daemon[909]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6463worker1 # [ 139.341524] nix-daemon[909]: reaped child process 1814, status = succeeded6464worker1 # [ 139.500800] nix-daemon[909]: accepted connection from pid 1829, user nix-grpc-daemon (trusted)6465worker1 # [ 139.513620] nix-daemon[909]: reaped child process 1832, status = succeeded6466lb # [ 139.536939] niks3-server[1111]: 2026/09/20 12:22:36 INFO Received uploads request method=POST path=/api/pending_closures6467worker1 # [ 139.532924] nix-grpc-daemon[1789]: time=2026-09-20T12:22:36.035Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6468worker1 # [ 139.536068] nix-grpc-daemon[1789]: time=2026-09-20T12:22:36.037Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6469lb # [ 139.558648] niks3-server[1111]: 2026/09/20 12:22:36 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6470lb # [ 139.565598] niks3-server[1111]: 2026/09/20 12:22:36 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6471lb # [ 139.570053] niks3-server[1111]: 2026/09/20 12:22:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6472lb # [ 139.572475] niks3-server[1111]: 2026/09/20 12:22:36 INFO Signed narinfos id=47 count=16473worker1 # [ 139.564314] nix-grpc-daemon[1789]: time=2026-09-20T12:22:36.066Z level=INFO msg="Uploading 1 narinfos"6474lb # [ 139.576149] niks3-server[1111]: 2026/09/20 12:22:36 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6475lb # [ 139.580110] niks3-server[1111]: 2026/09/20 12:22:36 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6476lb # [ 139.583852] niks3-server[1111]: 2026/09/20 12:22:36 INFO Completed upload id=476477worker1 # [ 139.576745] nix-grpc-daemon[1789]: time=2026-09-20T12:22:36.079Z level=INFO msg="Upload complete. (250ms)"6478lb # [ 139.586182] niks3-server[1111]: 2026/09/20 12:22:36 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6479worker1 # [ 139.579149] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41594 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=1 outputs=16480worker1 # [ 139.586036] nix-daemon[909]: reaped child process 1812, status = succeeded6481worker2 # [ 139.579687] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:36Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16482worker2 # [ 139.587314] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:36Z level=debug event=want client=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06483worker1 # [ 139.634766] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415946484worker1 # [ 139.645455] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:415946485worker1 # [ 139.656408] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:415946486worker1 # [ 139.664812] nix-daemon[909]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6487worker1 # [ 139.830572] nix-daemon[909]: accepted connection from pid 1836, user nix-grpc-daemon (trusted)6488worker1 # [ 139.843539] nix-daemon[909]: reaped child process 1839, status = succeeded6489worker1 # [ 139.993417] nix-daemon[909]: accepted connection from pid 1841, user nix-grpc-daemon (trusted)6490worker1 # [ 140.005908] nix-daemon[909]: reaped child process 1844, status = succeeded6491lb # [ 140.028176] niks3-server[1111]: 2026/09/20 12:22:36 INFO Received uploads request method=POST path=/api/pending_closures6492worker1 # [ 140.026280] nix-grpc-daemon[1789]: time=2026-09-20T12:22:36.529Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6493worker1 # [ 140.028242] nix-grpc-daemon[1789]: time=2026-09-20T12:22:36.530Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6494lb # [ 140.053771] niks3-server[1111]: 2026/09/20 12:22:36 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6495lb # [ 140.064880] niks3-server[1111]: 2026/09/20 12:22:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6496lb # [ 140.069065] niks3-server[1111]: 2026/09/20 12:22:36 INFO Signed narinfos id=48 count=16497worker1 # [ 140.065043] nix-grpc-daemon[1789]: time=2026-09-20T12:22:36.567Z level=INFO msg="Uploading 1 narinfos"6498lb # [ 140.075601] niks3-server[1111]: 2026/09/20 12:22:36 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6499lb # [ 140.087116] niks3-server[1111]: 2026/09/20 12:22:36 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6500lb # [ 140.395989] niks3-server[1111]: 2026/09/20 12:22:37 INFO Completed upload id=486501worker1 # [ 140.387599] nix-grpc-daemon[1789]: time=2026-09-20T12:22:36.890Z level=INFO msg="Upload complete. (711ms)"6502worker1 # [ 140.393175] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41594 duration_s=0 paths=1 nar_bytes_in=5046503worker1 # [ 140.398802] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415946504lb # [ 140.410224] niks3-server[1111]: 2026/09/20 12:22:37 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6505worker1 # [ 140.414204] nix-daemon[909]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6506worker1 # [ 140.425346] nix-daemon[909]: reaped child process 1834, status = succeeded6507worker1 # [ 140.432942] nix-daemon[909]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6508lb # [ 140.774136] envoy[1009]: conn peer=[2001:db8:1::1]:44728 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9628 tx=3891 ms=66426509lb # [ 140.778258] envoy[1009]: conn peer=[2001:db8:1::1]:60416 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2936 tx=1375 ms=109656510lb # [ 140.782882] envoy[1009]: conn peer=[2001:db8:1::3]:44818 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=3736 tx=865 ms=242496511worker1 # [ 141.264451] nix-daemon[909]: reaped child process 1848, status = succeeded6512worker1 # [ 141.431866] nix-daemon[909]: accepted connection from pid 1863, user nix-grpc-daemon (trusted)6513worker1 # [ 141.445448] nix-daemon[909]: reaped child process 1866, status = succeeded6514lb # [ 141.466750] niks3-server[1111]: 2026/09/20 12:22:38 INFO Received uploads request method=POST path=/api/pending_closures6515worker1 # [ 141.464992] nix-grpc-daemon[1789]: time=2026-09-20T12:22:37.967Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6516worker1 # [ 141.466486] nix-grpc-daemon[1789]: time=2026-09-20T12:22:37.967Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6517lb # [ 141.503595] niks3-server[1111]: 2026/09/20 12:22:38 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6518lb # [ 141.509110] niks3-server[1111]: 2026/09/20 12:22:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6519worker1 # [ 141.508685] nix-grpc-daemon[1789]: time=2026-09-20T12:22:38.011Z level=INFO msg="Uploading 1 narinfos"6520lb # [ 141.517595] niks3-server[1111]: 2026/09/20 12:22:38 INFO Signed narinfos id=49 count=16521lb # [ 141.522190] niks3-server[1111]: 2026/09/20 12:22:38 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6522lb # [ 141.526897] niks3-server[1111]: 2026/09/20 12:22:38 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6523lb # [ 141.536827] niks3-server[1111]: 2026/09/20 12:22:38 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6524lb # [ 141.541906] niks3-server[1111]: 2026/09/20 12:22:38 INFO Completed upload id=496525worker1 # [ 141.533439] nix-grpc-daemon[1789]: time=2026-09-20T12:22:38.036Z level=INFO msg="Upload complete. (287ms)"6526lb # [ 141.545531] niks3-server[1111]: 2026/09/20 12:22:38 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6527worker2 # [ 141.534531] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:38Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16528worker1 # [ 141.539993] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41594 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=2 outputs=16529worker2 # [ 141.540605] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:38Z level=debug event=want client=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06530worker1 # [ 141.546844] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415946531worker1 # [ 141.556403] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:415946532worker1 # [ 141.560051] nix-daemon[909]: reaped child process 1846, status = succeeded6533worker1 # [ 141.563508] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:415946534worker1 # [ 141.567215] nix-daemon[909]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6535worker1 # [ 141.719879] nix-daemon[909]: accepted connection from pid 1870, user nix-grpc-daemon (trusted)6536worker1 # [ 141.732465] nix-daemon[909]: reaped child process 1873, status = succeeded6537worker1 # [ 141.892103] nix-daemon[909]: accepted connection from pid 1875, user nix-grpc-daemon (trusted)6538worker1 # [ 141.904433] nix-daemon[909]: reaped child process 1878, status = succeeded6539lb # [ 141.927205] niks3-server[1111]: 2026/09/20 12:22:38 INFO Received uploads request method=POST path=/api/pending_closures6540worker1 # [ 141.930957] nix-grpc-daemon[1789]: time=2026-09-20T12:22:38.433Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6541worker1 # [ 141.932398] nix-grpc-daemon[1789]: time=2026-09-20T12:22:38.433Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6542lb # [ 141.956863] niks3-server[1111]: 2026/09/20 12:22:38 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6543lb # [ 141.967919] niks3-server[1111]: 2026/09/20 12:22:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6544worker1 # [ 141.964210] nix-grpc-daemon[1789]: time=2026-09-20T12:22:38.467Z level=INFO msg="Uploading 1 narinfos"6545lb # [ 141.973301] niks3-server[1111]: 2026/09/20 12:22:38 INFO Signed narinfos id=50 count=16546lb # [ 141.978279] niks3-server[1111]: 2026/09/20 12:22:38 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6547lb # [ 141.990081] niks3-server[1111]: 2026/09/20 12:22:38 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6548worker1 # [ 141.987723] nix-grpc-daemon[1789]: time=2026-09-20T12:22:38.490Z level=INFO msg="Upload complete. (414ms)"6549lb # [ 141.996511] niks3-server[1111]: 2026/09/20 12:22:38 INFO Completed upload id=506550worker1 # [ 141.990233] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41594 duration_s=0 paths=1 nar_bytes_in=5046551worker1 # [ 141.995099] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415946552lb # [ 142.007182] niks3-server[1111]: 2026/09/20 12:22:38 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6553worker1 # [ 142.003107] nix-daemon[909]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6554worker1 # [ 142.011667] nix-daemon[909]: reaped child process 1868, status = succeeded6555worker1 # [ 142.015723] nix-daemon[909]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6556worker1 # [ 143.245728] nix-daemon[909]: reaped child process 1882, status = succeeded6557worker1 # [ 143.404528] nix-daemon[909]: accepted connection from pid 1897, user nix-grpc-daemon (trusted)6558worker1 # [ 143.416449] nix-daemon[909]: reaped child process 1900, status = succeeded6559lb # [ 143.435475] niks3-server[1111]: 2026/09/20 12:22:40 INFO Received uploads request method=POST path=/api/pending_closures6560worker1 # [ 143.441456] nix-grpc-daemon[1789]: time=2026-09-20T12:22:39.944Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6561worker1 # [ 143.442987] nix-grpc-daemon[1789]: time=2026-09-20T12:22:39.945Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6562lb # [ 143.480746] niks3-server[1111]: 2026/09/20 12:22:40 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6563lb # [ 143.486535] niks3-server[1111]: 2026/09/20 12:22:40 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv6564lb # [ 143.498651] niks3-server[1111]: 2026/09/20 12:22:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6565lb # [ 143.504056] niks3-server[1111]: 2026/09/20 12:22:40 INFO Signed narinfos id=51 count=16566lb # [ 143.506647] niks3-server[1111]: 2026/09/20 12:22:40 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls6567worker1 # [ 143.498200] nix-grpc-daemon[1789]: time=2026-09-20T12:22:40.000Z level=INFO msg="Uploading 1 narinfos"6568lb # [ 143.517148] niks3-server[1111]: 2026/09/20 12:22:40 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6569lb # [ 143.520162] niks3-server[1111]: 2026/09/20 12:22:40 INFO Completed upload id=516570lb # [ 143.521949] niks3-server[1111]: 2026/09/20 12:22:40 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo6571worker1 # [ 143.512770] nix-grpc-daemon[1789]: time=2026-09-20T12:22:40.015Z level=INFO msg="Upload complete. (288ms)"6572worker1 # [ 143.519385] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41594 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=3 outputs=16573worker2 # [ 143.515885] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:39Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=16574worker2 # [ 143.522330] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:39Z level=debug event=want client=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=06575worker1 # [ 143.533506] nix-daemon[909]: reaped child process 1880, status = succeeded6576worker1 # [ 143.564133] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415946577worker1 # [ 143.576721] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:415946578worker1 # [ 143.587367] nix-grpc-daemon[1789]: time=2026-09-20T12:22:40.090Z level=INFO msg="All 1 paths already cached"6579worker1 # [ 143.593887] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:415946580worker1 # [ 143.604176] nix-daemon[909]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6581worker1 # [ 143.745452] nix-daemon[909]: accepted connection from pid 1904, user nix-grpc-daemon (trusted)6582worker1 # [ 143.759890] nix-daemon[909]: reaped child process 1907, status = succeeded6583worker1 # [ 143.908324] nix-daemon[909]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)6584worker1 # [ 143.921800] nix-daemon[909]: reaped child process 1912, status = succeeded6585lb # [ 143.943725] niks3-server[1111]: 2026/09/20 12:22:40 INFO Received uploads request method=POST path=/api/pending_closures6586worker1 # [ 143.943166] nix-grpc-daemon[1789]: time=2026-09-20T12:22:40.445Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6587worker1 # [ 143.944664] nix-grpc-daemon[1789]: time=2026-09-20T12:22:40.445Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"6588lb # [ 143.967830] niks3-server[1111]: 2026/09/20 12:22:40 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst6589lb # [ 143.979814] niks3-server[1111]: 2026/09/20 12:22:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6590lb # [ 143.985616] niks3-server[1111]: 2026/09/20 12:22:40 INFO Signed narinfos id=52 count=16591worker1 # [ 143.980055] nix-grpc-daemon[1789]: time=2026-09-20T12:22:40.482Z level=INFO msg="Uploading 1 narinfos"6592lb # [ 143.989138] niks3-server[1111]: 2026/09/20 12:22:40 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls6593lb # [ 144.001329] niks3-server[1111]: 2026/09/20 12:22:40 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6594lb # [ 144.005595] niks3-server[1111]: 2026/09/20 12:22:40 INFO Completed upload id=526595worker1 # [ 143.998886] nix-grpc-daemon[1789]: time=2026-09-20T12:22:40.501Z level=INFO msg="Upload complete. (383ms)"6596worker1 # [ 144.000937] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41594 duration_s=0 paths=1 nar_bytes_in=6246597lb # [ 144.011882] niks3-server[1111]: 2026/09/20 12:22:40 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo6598worker1 # [ 144.006993] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415946599worker1 # [ 144.011628] nix-daemon[909]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6600worker1 # [ 144.015063] nix-daemon[909]: reaped child process 1902, status = succeeded6601worker1 # [ 144.020229] nix-daemon[909]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6602worker1 # [ 145.235729] nix-daemon[909]: reaped child process 1916, status = succeeded6603worker1 # [ 145.368984] nix-daemon[909]: accepted connection from pid 1931, user nix-grpc-daemon (trusted)6604worker1 # [ 145.381506] nix-daemon[909]: reaped child process 1934, status = succeeded6605lb # [ 145.401663] niks3-server[1111]: 2026/09/20 12:22:42 INFO Received uploads request method=POST path=/api/pending_closures6606worker1 # [ 145.401457] nix-grpc-daemon[1789]: time=2026-09-20T12:22:41.904Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6607worker1 # [ 145.403150] nix-grpc-daemon[1789]: time=2026-09-20T12:22:41.906Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"6608lb # [ 145.439744] niks3-server[1111]: 2026/09/20 12:22:42 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv6609lb # [ 145.446884] niks3-server[1111]: 2026/09/20 12:22:42 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst6610lb # [ 145.458086] niks3-server[1111]: 2026/09/20 12:22:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6611worker1 # [ 145.454400] nix-grpc-daemon[1789]: time=2026-09-20T12:22:41.957Z level=INFO msg="Uploading 1 narinfos"6612lb # [ 145.464130] niks3-server[1111]: 2026/09/20 12:22:42 INFO Signed narinfos id=53 count=16613lb # [ 145.468460] niks3-server[1111]: 2026/09/20 12:22:42 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls6614lb # [ 145.478085] niks3-server[1111]: 2026/09/20 12:22:42 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6615lb # [ 145.484551] niks3-server[1111]: 2026/09/20 12:22:42 INFO Completed upload id=536616lb # [ 145.486635] niks3-server[1111]: 2026/09/20 12:22:42 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo6617worker1 # [ 145.475824] nix-grpc-daemon[1789]: time=2026-09-20T12:22:41.978Z level=INFO msg="Upload complete. (248ms)"6618worker1 # [ 145.480759] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41594 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=4 outputs=16619worker2 # [ 145.479526] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:41Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=16620worker1 # [ 145.489388] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415946621worker2 # [ 145.485460] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:41Z level=debug event=want client=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=06622worker1 # [ 145.500409] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:415946623worker1 # [ 145.502860] nix-daemon[909]: reaped child process 1914, status = succeeded6624worker1 # [ 145.508060] nix-grpc-daemon[1789]: time=2026-09-20T12:22:42.011Z level=INFO msg="All 1 paths already cached"6625worker1 # [ 145.511749] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:415946626worker1 # [ 145.515516] nix-daemon[909]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6627worker1 # [ 145.668044] nix-daemon[909]: accepted connection from pid 1938, user nix-grpc-daemon (trusted)6628worker1 # [ 145.681360] nix-daemon[909]: reaped child process 1941, status = succeeded6629worker1 # [ 145.845333] nix-daemon[909]: accepted connection from pid 1943, user nix-grpc-daemon (trusted)6630worker1 # [ 145.858394] nix-daemon[909]: reaped child process 1946, status = succeeded6631lb # [ 145.876890] niks3-server[1111]: 2026/09/20 12:22:42 INFO Received uploads request method=POST path=/api/pending_closures6632worker1 # [ 145.875718] nix-grpc-daemon[1789]: time=2026-09-20T12:22:42.378Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6633worker1 # [ 145.877236] nix-grpc-daemon[1789]: time=2026-09-20T12:22:42.380Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"6634lb # [ 145.902768] niks3-server[1111]: 2026/09/20 12:22:42 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst6635lb # [ 145.917124] niks3-server[1111]: 2026/09/20 12:22:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6636lb # [ 145.921723] niks3-server[1111]: 2026/09/20 12:22:42 INFO Signed narinfos id=54 count=16637worker1 # [ 145.915928] nix-grpc-daemon[1789]: time=2026-09-20T12:22:42.418Z level=INFO msg="Uploading 1 narinfos"6638lb # [ 145.929067] niks3-server[1111]: 2026/09/20 12:22:42 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls6639lb # [ 145.943872] niks3-server[1111]: 2026/09/20 12:22:42 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6640worker1 # [ 145.941549] nix-grpc-daemon[1789]: time=2026-09-20T12:22:42.444Z level=INFO msg="Upload complete. (420ms)"6641lb # [ 145.950230] niks3-server[1111]: 2026/09/20 12:22:42 INFO Completed upload id=546642worker1 # [ 145.943840] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41594 duration_s=0 paths=1 nar_bytes_in=6246643lb # [ 145.955367] niks3-server[1111]: 2026/09/20 12:22:42 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo6644worker1 # [ 145.946937] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415946645worker1 # [ 145.954826] nix-daemon[909]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6646worker1 # [ 145.961117] nix-daemon[909]: reaped child process 1936, status = succeeded6647worker1 # [ 145.964505] nix-daemon[909]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6648worker1 # [ 146.579415] nix-daemon[909]: reaped child process 1950, status = succeeded6649worker1 # [ 146.713553] nix-daemon[909]: accepted connection from pid 1965, user nix-grpc-daemon (trusted)6650worker1 # [ 146.726119] nix-daemon[909]: reaped child process 1968, status = succeeded6651lb # [ 146.749780] niks3-server[1111]: 2026/09/20 12:22:43 INFO Received uploads request method=POST path=/api/pending_closures6652worker1 # [ 146.748927] nix-grpc-daemon[1789]: time=2026-09-20T12:22:43.251Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6653worker1 # [ 146.750691] nix-grpc-daemon[1789]: time=2026-09-20T12:22:43.253Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"6654lb # [ 146.791095] niks3-server[1111]: 2026/09/20 12:22:43 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv6655lb # [ 146.796729] niks3-server[1111]: 2026/09/20 12:22:43 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst6656lb # [ 146.805358] niks3-server[1111]: 2026/09/20 12:22:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6657worker1 # [ 146.802242] nix-grpc-daemon[1789]: time=2026-09-20T12:22:43.305Z level=INFO msg="Uploading 1 narinfos"6658lb # [ 146.810970] niks3-server[1111]: 2026/09/20 12:22:43 INFO Signed narinfos id=55 count=16659lb # [ 146.814476] niks3-server[1111]: 2026/09/20 12:22:43 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls6660lb # [ 146.824597] niks3-server[1111]: 2026/09/20 12:22:43 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6661lb # [ 146.830622] niks3-server[1111]: 2026/09/20 12:22:43 INFO Completed upload id=556662worker1 # [ 146.821737] nix-grpc-daemon[1789]: time=2026-09-20T12:22:43.324Z level=INFO msg="Upload complete. (250ms)"6663lb # [ 146.833323] niks3-server[1111]: 2026/09/20 12:22:43 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo6664worker1 # [ 146.827277] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41594 duration_s=0 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=5 outputs=16665worker2 # [ 146.824817] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:43Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=16666worker1 # [ 146.839728] nix-daemon[909]: reaped child process 1948, status = succeeded6667worker2 # [ 146.831678] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:43Z level=debug event=want client=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=06668worker1 # [ 146.873987] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415946669worker1 # [ 146.886609] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:415946670worker1 # [ 146.899140] nix-grpc-daemon[1789]: time=2026-09-20T12:22:43.402Z level=INFO msg="All 1 paths already cached"6671worker1 # [ 146.906422] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:415946672worker1 # [ 146.914923] nix-daemon[909]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6673worker1 # [ 147.082823] nix-daemon[909]: accepted connection from pid 1972, user nix-grpc-daemon (trusted)6674worker1 # [ 147.095393] nix-daemon[909]: reaped child process 1975, status = succeeded6675worker1 # [ 147.245548] nix-daemon[909]: accepted connection from pid 1977, user nix-grpc-daemon (trusted)6676worker1 # [ 147.258094] nix-daemon[909]: reaped child process 1980, status = succeeded6677lb # [ 147.280819] niks3-server[1111]: 2026/09/20 12:22:43 INFO Received uploads request method=POST path=/api/pending_closures6678worker1 # [ 147.284676] nix-grpc-daemon[1789]: time=2026-09-20T12:22:43.787Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6679worker1 # [ 147.286128] nix-grpc-daemon[1789]: time=2026-09-20T12:22:43.787Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"6680lb # [ 147.309438] niks3-server[1111]: 2026/09/20 12:22:43 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst6681lb # [ 147.321494] niks3-server[1111]: 2026/09/20 12:22:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6682worker1 # [ 147.319191] nix-grpc-daemon[1789]: time=2026-09-20T12:22:43.822Z level=INFO msg="Uploading 1 narinfos"6683lb # [ 147.327852] niks3-server[1111]: 2026/09/20 12:22:43 INFO Signed narinfos id=56 count=16684lb # [ 147.331996] niks3-server[1111]: 2026/09/20 12:22:43 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls6685lb # [ 147.342758] niks3-server[1111]: 2026/09/20 12:22:43 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6686worker1 # [ 147.339752] nix-grpc-daemon[1789]: time=2026-09-20T12:22:43.842Z level=INFO msg="Upload complete. (411ms)"6687lb # [ 147.348707] niks3-server[1111]: 2026/09/20 12:22:43 INFO Completed upload id=566688lb # [ 147.352315] niks3-server[1111]: 2026/09/20 12:22:43 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo6689worker1 # [ 147.342695] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41594 duration_s=0 paths=1 nar_bytes_in=6246690worker1 # [ 147.347908] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415946691worker1 # [ 147.357904] nix-daemon[909]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6692worker1 # [ 147.363706] nix-daemon[909]: reaped child process 1970, status = succeeded6693worker1 # [ 147.366878] nix-daemon[909]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6694worker1 # [ 148.180402] nix-daemon[909]: reaped child process 1984, status = succeeded6695worker1 # [ 148.297161] nix-daemon[909]: accepted connection from pid 1999, user nix-grpc-daemon (trusted)6696worker1 # [ 148.309645] nix-daemon[909]: reaped child process 2002, status = succeeded6697lb # [ 148.329759] niks3-server[1111]: 2026/09/20 12:22:44 INFO Received uploads request method=POST path=/api/pending_closures6698worker1 # [ 148.329159] nix-grpc-daemon[1789]: time=2026-09-20T12:22:44.831Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6699worker1 # [ 148.330836] nix-grpc-daemon[1789]: time=2026-09-20T12:22:44.833Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"6700lb # [ 148.365699] niks3-server[1111]: 2026/09/20 12:22:44 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv6701lb # [ 148.373969] niks3-server[1111]: 2026/09/20 12:22:44 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst6702lb # [ 148.384131] niks3-server[1111]: 2026/09/20 12:22:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6703lb # [ 148.388741] niks3-server[1111]: 2026/09/20 12:22:45 INFO Signed narinfos id=57 count=16704worker1 # [ 148.382889] nix-grpc-daemon[1789]: time=2026-09-20T12:22:44.885Z level=INFO msg="Uploading 1 narinfos"6705lb # [ 148.394893] niks3-server[1111]: 2026/09/20 12:22:45 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls6706lb # [ 148.404089] niks3-server[1111]: 2026/09/20 12:22:45 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6707lb # [ 148.410283] niks3-server[1111]: 2026/09/20 12:22:45 INFO Completed upload id=576708worker1 # [ 148.401584] nix-grpc-daemon[1789]: time=2026-09-20T12:22:44.904Z level=INFO msg="Upload complete. (228ms)"6709lb # [ 148.412362] niks3-server[1111]: 2026/09/20 12:22:45 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo6710worker2 # [ 148.401363] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:44Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=16711worker1 # [ 148.408577] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41594 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=6 outputs=16712worker1 # [ 148.415320] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415946713worker2 # [ 148.411649] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:44Z level=debug event=want client=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=06714worker1 # [ 148.425532] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:415946715worker1 # [ 148.431817] nix-daemon[909]: reaped child process 1982, status = succeeded6716worker1 # [ 148.437878] nix-grpc-daemon[1789]: time=2026-09-20T12:22:44.940Z level=INFO msg="All 2 paths already cached"6717worker1 # [ 148.441901] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:415946718worker1 # [ 148.445634] nix-daemon[909]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6719worker1 # [ 148.585081] nix-daemon[909]: accepted connection from pid 2006, user nix-grpc-daemon (trusted)6720worker1 # [ 148.598296] nix-daemon[909]: reaped child process 2009, status = succeeded6721worker1 # [ 148.757070] nix-daemon[909]: accepted connection from pid 2011, user nix-grpc-daemon (trusted)6722worker1 # [ 148.769590] nix-daemon[909]: reaped child process 2014, status = succeeded6723lb # [ 148.792665] niks3-server[1111]: 2026/09/20 12:22:45 INFO Received uploads request method=POST path=/api/pending_closures6724worker1 # [ 148.792494] nix-grpc-daemon[1789]: time=2026-09-20T12:22:45.295Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6725worker1 # [ 148.794542] nix-grpc-daemon[1789]: time=2026-09-20T12:22:45.295Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"6726lb # [ 148.818900] niks3-server[1111]: 2026/09/20 12:22:45 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst6727lb # [ 148.829982] niks3-server[1111]: 2026/09/20 12:22:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6728lb # [ 148.834931] niks3-server[1111]: 2026/09/20 12:22:45 INFO Signed narinfos id=58 count=16729worker1 # [ 148.829305] nix-grpc-daemon[1789]: time=2026-09-20T12:22:45.332Z level=INFO msg="Uploading 1 narinfos"6730lb # [ 148.842285] niks3-server[1111]: 2026/09/20 12:22:45 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls6731lb # [ 148.852339] niks3-server[1111]: 2026/09/20 12:22:45 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6732worker1 # [ 148.849501] nix-grpc-daemon[1789]: time=2026-09-20T12:22:45.352Z level=INFO msg="Upload complete. (398ms)"6733lb # [ 148.858312] niks3-server[1111]: 2026/09/20 12:22:45 INFO Completed upload id=586734lb # [ 148.861445] niks3-server[1111]: 2026/09/20 12:22:45 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo6735worker1 # [ 148.851507] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41594 duration_s=0 paths=1 nar_bytes_in=7526736worker1 # [ 148.857101] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415946737worker1 # [ 148.865567] nix-daemon[909]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6738worker1 # [ 148.869323] nix-daemon[909]: reaped child process 2004, status = succeeded6739worker1 # [ 148.874182] nix-daemon[909]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6740worker1 # [ 149.700248] nix-daemon[909]: reaped child process 2018, status = succeeded6741worker1 # [ 149.859274] nix-daemon[909]: accepted connection from pid 2033, user nix-grpc-daemon (trusted)6742worker1 # [ 149.871994] nix-daemon[909]: reaped child process 2036, status = succeeded6743lb # [ 149.896497] niks3-server[1111]: 2026/09/20 12:22:46 INFO Received uploads request method=POST path=/api/pending_closures6744worker1 # [ 149.892612] nix-grpc-daemon[1789]: time=2026-09-20T12:22:46.395Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6745worker1 # [ 149.894208] nix-grpc-daemon[1789]: time=2026-09-20T12:22:46.397Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"6746lb # [ 149.921086] niks3-server[1111]: 2026/09/20 12:22:46 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv6747lb # [ 149.924331] niks3-server[1111]: 2026/09/20 12:22:46 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst6748lb # [ 149.929577] niks3-server[1111]: 2026/09/20 12:22:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6749lb # [ 149.931962] niks3-server[1111]: 2026/09/20 12:22:46 INFO Signed narinfos id=59 count=16750worker1 # [ 149.923170] nix-grpc-daemon[1789]: time=2026-09-20T12:22:46.426Z level=INFO msg="Uploading 1 narinfos"6751lb # [ 149.935235] niks3-server[1111]: 2026/09/20 12:22:46 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls6752lb # [ 149.940459] niks3-server[1111]: 2026/09/20 12:22:46 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6753lb # [ 149.944324] niks3-server[1111]: 2026/09/20 12:22:46 INFO Completed upload id=596754worker1 # [ 149.935446] nix-grpc-daemon[1789]: time=2026-09-20T12:22:46.438Z level=INFO msg="Upload complete. (254ms)"6755lb # [ 149.946388] niks3-server[1111]: 2026/09/20 12:22:46 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo6756worker1 # [ 149.939113] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41594 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=7 outputs=16757worker1 # [ 149.945451] nix-daemon[909]: reaped child process 2016, status = succeeded6758worker2 # [ 149.935487] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:46Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=16759worker1 # [ 149.984851] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415946760worker1 # [ 149.994225] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:415946761worker1 # [ 150.003756] nix-grpc-daemon[1789]: time=2026-09-20T12:22:46.506Z level=INFO msg="All 2 paths already cached"6762worker1 # [ 150.009332] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:415946763worker1 # [ 150.017980] nix-daemon[909]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6764worker1 # [ 150.177859] nix-daemon[909]: accepted connection from pid 2040, user nix-grpc-daemon (trusted)6765worker1 # [ 150.190705] nix-daemon[909]: reaped child process 2043, status = succeeded6766worker1 # [ 150.352720] nix-daemon[909]: accepted connection from pid 2045, user nix-grpc-daemon (trusted)6767worker1 # [ 150.365074] nix-daemon[909]: reaped child process 2048, status = succeeded6768lb # [ 150.390179] niks3-server[1111]: 2026/09/20 12:22:47 INFO Received uploads request method=POST path=/api/pending_closures6769worker1 # [ 150.391039] nix-grpc-daemon[1789]: time=2026-09-20T12:22:46.894Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6770worker1 # [ 150.393040] nix-grpc-daemon[1789]: time=2026-09-20T12:22:46.895Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"6771lb # [ 150.413081] niks3-server[1111]: 2026/09/20 12:22:47 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst6772lb # [ 150.414805] niks3-server[1111]: 2026/09/20 12:22:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6773lb # [ 150.417345] niks3-server[1111]: 2026/09/20 12:22:47 INFO Signed narinfos id=60 count=16774worker1 # [ 150.408785] nix-grpc-daemon[1789]: time=2026-09-20T12:22:46.911Z level=INFO msg="Uploading 1 narinfos"6775lb # [ 150.420436] niks3-server[1111]: 2026/09/20 12:22:47 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls6776lb # [ 150.425151] niks3-server[1111]: 2026/09/20 12:22:47 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6777lb # [ 150.430540] niks3-server[1111]: 2026/09/20 12:22:47 INFO Completed upload id=606778worker1 # [ 150.421837] nix-grpc-daemon[1789]: time=2026-09-20T12:22:46.924Z level=INFO msg="Upload complete. (395ms)"6779lb # [ 150.432144] niks3-server[1111]: 2026/09/20 12:22:47 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo6780worker1 # [ 150.423848] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41594 duration_s=0 paths=1 nar_bytes_in=7526781worker1 # [ 150.427845] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415946782worker1 # [ 150.433966] nix-daemon[909]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6783worker1 # [ 150.436286] nix-daemon[909]: reaped child process 2038, status = succeeded6784worker1 # [ 150.441572] nix-daemon[909]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6785worker1 # [ 151.657334] nix-daemon[909]: reaped child process 2052, status = succeeded6786worker1 # [ 151.804301] nix-daemon[909]: accepted connection from pid 2067, user nix-grpc-daemon (trusted)6787worker1 # [ 151.815306] nix-daemon[909]: reaped child process 2070, status = succeeded6788lb # [ 151.842714] niks3-server[1111]: 2026/09/20 12:22:48 INFO Received uploads request method=POST path=/api/pending_closures6789worker1 # [ 151.842449] nix-grpc-daemon[1789]: time=2026-09-20T12:22:48.345Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6790worker1 # [ 151.844434] nix-grpc-daemon[1789]: time=2026-09-20T12:22:48.347Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"6791lb # [ 151.869753] niks3-server[1111]: 2026/09/20 12:22:48 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv6792lb # [ 151.882464] niks3-server[1111]: 2026/09/20 12:22:48 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst6793lb # [ 151.889953] niks3-server[1111]: 2026/09/20 12:22:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6794worker1 # [ 151.884822] nix-grpc-daemon[1789]: time=2026-09-20T12:22:48.387Z level=INFO msg="Uploading 1 narinfos"6795lb # [ 151.893406] niks3-server[1111]: 2026/09/20 12:22:48 INFO Signed narinfos id=61 count=16796lb # [ 151.899293] niks3-server[1111]: 2026/09/20 12:22:48 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls6797lb # [ 151.903657] niks3-server[1111]: 2026/09/20 12:22:48 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6798lb # [ 151.906912] niks3-server[1111]: 2026/09/20 12:22:48 INFO Completed upload id=616799worker1 # [ 151.898142] nix-grpc-daemon[1789]: time=2026-09-20T12:22:48.401Z level=INFO msg="Upload complete. (248ms)"6800lb # [ 151.908667] niks3-server[1111]: 2026/09/20 12:22:48 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo6801worker1 # [ 151.900618] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41594 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=8 outputs=16802worker2 # [ 151.897892] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:48Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=16803worker1 # [ 151.907891] nix-daemon[909]: reaped child process 2050, status = succeeded6804worker1 # [ 151.946716] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415946805worker1 # [ 151.954848] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:415946806worker1 # [ 151.963242] nix-grpc-daemon[1789]: time=2026-09-20T12:22:48.466Z level=INFO msg="All 2 paths already cached"6807worker1 # [ 151.967779] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:415946808worker1 # [ 151.972119] nix-daemon[909]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6809worker1 # [ 152.131606] nix-daemon[909]: accepted connection from pid 2074, user nix-grpc-daemon (trusted)6810worker1 # [ 152.144373] nix-daemon[909]: reaped child process 2077, status = succeeded6811worker1 # [ 152.313163] nix-daemon[909]: accepted connection from pid 2079, user nix-grpc-daemon (trusted)6812worker1 # [ 152.325487] nix-daemon[909]: reaped child process 2082, status = succeeded6813lb # [ 152.352062] niks3-server[1111]: 2026/09/20 12:22:48 INFO Received uploads request method=POST path=/api/pending_closures6814worker1 # [ 152.350639] nix-grpc-daemon[1789]: time=2026-09-20T12:22:48.853Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6815worker1 # [ 152.352277] nix-grpc-daemon[1789]: time=2026-09-20T12:22:48.855Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"6816lb # [ 152.380075] niks3-server[1111]: 2026/09/20 12:22:49 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst6817lb # [ 152.389539] niks3-server[1111]: 2026/09/20 12:22:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6818lb # [ 152.393682] niks3-server[1111]: 2026/09/20 12:22:49 INFO Signed narinfos id=62 count=16819worker1 # [ 152.388999] nix-grpc-daemon[1789]: time=2026-09-20T12:22:48.892Z level=INFO msg="Uploading 1 narinfos"6820lb # [ 152.401730] niks3-server[1111]: 2026/09/20 12:22:49 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls6821lb # [ 152.412412] niks3-server[1111]: 2026/09/20 12:22:49 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6822worker1 # [ 152.408512] nix-grpc-daemon[1789]: time=2026-09-20T12:22:48.911Z level=INFO msg="Upload complete. (430ms)"6823lb # [ 152.417731] niks3-server[1111]: 2026/09/20 12:22:49 INFO Completed upload id=626824worker1 # [ 152.411339] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41594 duration_s=0 paths=1 nar_bytes_in=7526825lb # [ 152.423268] niks3-server[1111]: 2026/09/20 12:22:49 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo6826worker1 # [ 152.417275] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415946827worker1 # [ 152.425090] nix-daemon[909]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6828worker1 # [ 152.429691] nix-daemon[909]: reaped child process 2072, status = succeeded6829worker1 # [ 152.434907] nix-daemon[909]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6830worker1 # [ 153.857178] nix-daemon[909]: reaped child process 2086, status = succeeded6831worker1 # [ 154.001811] nix-daemon[909]: accepted connection from pid 2101, user nix-grpc-daemon (trusted)6832worker1 # [ 154.014363] nix-daemon[909]: reaped child process 2104, status = succeeded6833lb # [ 154.039740] niks3-server[1111]: 2026/09/20 12:22:50 INFO Received uploads request method=POST path=/api/pending_closures6834worker1 # [ 154.040322] nix-grpc-daemon[1789]: time=2026-09-20T12:22:50.541Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6835worker1 # [ 154.041755] nix-grpc-daemon[1789]: time=2026-09-20T12:22:50.541Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"6836lb # [ 154.081753] niks3-server[1111]: 2026/09/20 12:22:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6837worker1 # [ 154.077940] nix-grpc-daemon[1789]: time=2026-09-20T12:22:50.580Z level=INFO msg="Uploading 1 narinfos"6838lb # [ 154.088127] niks3-server[1111]: 2026/09/20 12:22:50 INFO Signed narinfos id=63 count=16839lb # [ 154.094278] niks3-server[1111]: 2026/09/20 12:22:50 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv6840lb # [ 154.102290] niks3-server[1111]: 2026/09/20 12:22:50 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst6841lb # [ 154.106897] niks3-server[1111]: 2026/09/20 12:22:50 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls6842lb # [ 154.115377] niks3-server[1111]: 2026/09/20 12:22:50 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6843lb # [ 154.120311] niks3-server[1111]: 2026/09/20 12:22:50 INFO Completed upload id=636844worker1 # [ 154.111903] nix-grpc-daemon[1789]: time=2026-09-20T12:22:50.614Z level=INFO msg="Upload complete. (269ms)"6845lb # [ 154.125063] niks3-server[1111]: 2026/09/20 12:22:50 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo6846worker1 # [ 154.117428] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41594 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=9 outputs=16847worker1 # [ 154.130578] nix-daemon[909]: reaped child process 2084, status = succeeded6848client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 17.28 seconds)6849worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6850worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6851client: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&16852worker2 # [ 154.426497] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:50Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:420926853worker1 # [ 154.441213] nix-grpc-daemon[1733]: ts=2026-09-20T12:22:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:415946854worker2 # [ 154.457561] nix-grpc-daemon[1429]: ts=2026-09-20T12:22:50Z level=debug event=want client=2 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=06855lb # [ 160.775590] envoy[1009]: conn peer=[2001:db8:1::1]:58446 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27072 tx=10697 ms=170236856client: (finished: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&1, in 30.26 seconds)6857(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 47.61 seconds)6858subtest: low disk drains a worker and builds go to the other6859worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6860worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.00 seconds)6861worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6862worker1 # [ 185.764936] nix-grpc-daemon[1733]: ts=2026-09-20T12:23:22Z level=info event=unhealthy reason=min_free6863worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.09 seconds)6864client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&26865worker2 # [ 186.761791] nix-grpc-daemon[1429]: ts=2026-09-20T12:23:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:420926866client # don't know how to build these paths:6867client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6868worker1 # [ 186.778467] nix-grpc-daemon[1733]: ts=2026-09-20T12:23:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:415946869worker2 # [ 186.793243] nix-grpc-daemon[1429]: ts=2026-09-20T12:23:23Z level=debug event=want client=3 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=06870worker2 # [ 186.798663] nix-grpc-daemon[1429]: ts=2026-09-20T12:23:23Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=16871worker2 # [ 186.837665] nix-grpc-daemon[1429]: ts=2026-09-20T12:23:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:420926872worker2 # [ 186.850704] nix-grpc-daemon[1429]: ts=2026-09-20T12:23:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:420926873client # copying 1 paths to 'lb:50051'...6874client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...6875worker2 # [ 186.863060] nix-grpc-daemon[1429]: ts=2026-09-20T12:23:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:420926876worker2 # [ 186.875165] nix-daemon[924]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6877worker2 # [ 187.049523] nix-daemon[924]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6878worker2 # [ 187.063663] nix-daemon[924]: reaped child process 1726, status = succeeded6879worker2 # [ 187.226342] nix-daemon[924]: accepted connection from pid 1728, user nix-grpc-daemon (trusted)6880worker2 # [ 187.239465] nix-daemon[924]: reaped child process 1731, status = succeeded6881lb # [ 187.271767] niks3-server[1111]: 2026/09/20 12:23:23 INFO Received uploads request method=POST path=/api/pending_closures6882worker2 # [ 187.266405] nix-grpc-daemon[1517]: time=2026-09-20T12:23:23.739Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6883worker2 # [ 187.267907] nix-grpc-daemon[1517]: time=2026-09-20T12:23:23.741Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"6884lb # [ 187.302510] niks3-server[1111]: 2026/09/20 12:23:23 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst6885lb # [ 187.313171] niks3-server[1111]: 2026/09/20 12:23:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6886worker2 # [ 187.306815] nix-grpc-daemon[1517]: time=2026-09-20T12:23:23.779Z level=INFO msg="Uploading 1 narinfos"6887lb # [ 187.321664] niks3-server[1111]: 2026/09/20 12:23:23 INFO Signed narinfos id=64 count=16888lb # [ 187.329554] niks3-server[1111]: 2026/09/20 12:23:23 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls6889lb # [ 187.339862] niks3-server[1111]: 2026/09/20 12:23:23 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6890lb # [ 187.346887] niks3-server[1111]: 2026/09/20 12:23:23 INFO Completed upload id=646891worker2 # [ 187.334459] nix-grpc-daemon[1517]: time=2026-09-20T12:23:23.807Z level=INFO msg="Upload complete. (447ms)"6892lb # [ 187.350267] niks3-server[1111]: 2026/09/20 12:23:23 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo6893worker2 # [ 187.336731] nix-grpc-daemon[1429]: ts=2026-09-20T12:23:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42092 duration_s=0 paths=1 nar_bytes_in=5126894worker2 # [ 187.341822] nix-grpc-daemon[1429]: ts=2026-09-20T12:23:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:420926895worker2 # [ 187.352107] nix-daemon[924]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6896worker2 # [ 187.359807] nix-daemon[924]: reaped child process 1721, status = succeeded6897worker2 # [ 187.364880] nix-daemon[924]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6898client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...6899client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6900client # farm-a> LOG-farm-a-drain6901worker2 # [ 188.391950] nix-daemon[924]: reaped child process 1735, status = succeeded6902worker2 # [ 188.559308] nix-daemon[924]: accepted connection from pid 1750, user nix-grpc-daemon (trusted)6903worker2 # [ 188.572411] nix-daemon[924]: reaped child process 1753, status = succeeded6904lb # [ 188.602970] niks3-server[1111]: 2026/09/20 12:23:25 INFO Received uploads request method=POST path=/api/pending_closures6905worker2 # [ 188.598035] nix-grpc-daemon[1517]: time=2026-09-20T12:23:25.070Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6906worker2 # [ 188.599513] nix-grpc-daemon[1517]: time=2026-09-20T12:23:25.072Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"6907lb # [ 188.637345] niks3-server[1111]: 2026/09/20 12:23:25 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6908lb # [ 188.652641] niks3-server[1111]: 2026/09/20 12:23:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6909worker2 # [ 188.643339] nix-grpc-daemon[1517]: time=2026-09-20T12:23:25.116Z level=INFO msg="Uploading 1 narinfos"6910lb # [ 188.658246] niks3-server[1111]: 2026/09/20 12:23:25 INFO Signed narinfos id=65 count=16911lb # [ 188.664273] niks3-server[1111]: 2026/09/20 12:23:25 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6912lb # [ 188.669274] niks3-server[1111]: 2026/09/20 12:23:25 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls6913lb # [ 188.674980] niks3-server[1111]: 2026/09/20 12:23:25 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6914lb # [ 188.678247] niks3-server[1111]: 2026/09/20 12:23:25 INFO Completed upload id=656915lb # [ 188.679963] niks3-server[1111]: 2026/09/20 12:23:25 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo6916worker2 # [ 188.664571] nix-grpc-daemon[1517]: time=2026-09-20T12:23:25.137Z level=INFO msg="Upload complete. (292ms)"6917worker2 # [ 188.670470] nix-grpc-daemon[1429]: ts=2026-09-20T12:23:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42092 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=10 outputs=16918worker2 # [ 188.681552] nix-grpc-daemon[1429]: ts=2026-09-20T12:23:25Z level=debug event=want client=3 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=06919worker2 # [ 188.685153] nix-grpc-daemon[1429]: ts=2026-09-20T12:23:25Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=16920worker2 # [ 188.692996] nix-daemon[924]: reaped child process 1733, status = succeeded6921worker2 # [ 188.725345] nix-grpc-daemon[1429]: ts=2026-09-20T12:23:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:420926922worker2 # [ 188.735856] nix-grpc-daemon[1429]: ts=2026-09-20T12:23:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:420926923worker2 # [ 188.746667] nix-grpc-daemon[1517]: time=2026-09-20T12:23:25.219Z level=INFO msg="All 1 paths already cached"6924client # copying 1 paths to 'lb:50051'...6925client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...6926worker2 # [ 188.754575] nix-grpc-daemon[1429]: ts=2026-09-20T12:23:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:420926927worker2 # [ 188.765108] nix-daemon[924]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6928worker2 # [ 188.931442] nix-daemon[924]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)6929worker2 # [ 188.946146] nix-daemon[924]: reaped child process 1762, status = succeeded6930worker2 # [ 189.094513] nix-daemon[924]: accepted connection from pid 1764, user nix-grpc-daemon (trusted)6931worker2 # [ 189.107656] nix-daemon[924]: reaped child process 1767, status = succeeded6932lb # [ 189.136598] niks3-server[1111]: 2026/09/20 12:23:25 INFO Received uploads request method=POST path=/api/pending_closures6933worker2 # [ 189.129721] nix-grpc-daemon[1517]: time=2026-09-20T12:23:25.602Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6934worker2 # [ 189.131574] nix-grpc-daemon[1517]: time=2026-09-20T12:23:25.604Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"6935lb # [ 189.162782] niks3-server[1111]: 2026/09/20 12:23:25 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst6936lb # [ 189.173923] niks3-server[1111]: 2026/09/20 12:23:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6937lb # [ 189.178171] niks3-server[1111]: 2026/09/20 12:23:25 INFO Signed narinfos id=66 count=16938worker2 # [ 189.166470] nix-grpc-daemon[1517]: time=2026-09-20T12:23:25.639Z level=INFO msg="Uploading 1 narinfos"6939lb # [ 189.184528] niks3-server[1111]: 2026/09/20 12:23:25 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls6940lb # [ 189.196192] niks3-server[1111]: 2026/09/20 12:23:25 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6941lb # [ 189.202765] niks3-server[1111]: 2026/09/20 12:23:25 INFO Completed upload id=666942worker2 # [ 189.187706] nix-grpc-daemon[1517]: time=2026-09-20T12:23:25.660Z level=INFO msg="Upload complete. (412ms)"6943worker2 # [ 189.191079] nix-grpc-daemon[1429]: ts=2026-09-20T12:23:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42092 duration_s=0 paths=1 nar_bytes_in=6246944lb # [ 189.205591] niks3-server[1111]: 2026/09/20 12:23:25 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo6945worker2 # [ 189.197422] nix-grpc-daemon[1429]: ts=2026-09-20T12:23:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:420926946worker2 # [ 189.201576] nix-daemon[924]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6947worker2 # [ 189.203963] nix-daemon[924]: reaped child process 1757, status = succeeded6948worker2 # [ 189.209181] nix-daemon[924]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6949client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...6950client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6951client # farm-b> LOG-farm-b-drain6952worker2 # [ 190.236512] nix-daemon[924]: reaped child process 1771, status = succeeded6953worker2 # [ 190.388625] nix-daemon[924]: accepted connection from pid 1786, user nix-grpc-daemon (trusted)6954worker2 # [ 190.402055] nix-daemon[924]: reaped child process 1789, status = succeeded6955lb # [ 190.431625] niks3-server[1111]: 2026/09/20 12:23:27 INFO Received uploads request method=POST path=/api/pending_closures6956worker2 # [ 190.424779] nix-grpc-daemon[1517]: time=2026-09-20T12:23:26.897Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6957worker2 # [ 190.426264] nix-grpc-daemon[1517]: time=2026-09-20T12:23:26.899Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"6958lb # [ 190.470398] niks3-server[1111]: 2026/09/20 12:23:27 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6959lb # [ 190.477628] niks3-server[1111]: 2026/09/20 12:23:27 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst6960lb # [ 190.488945] niks3-server[1111]: 2026/09/20 12:23:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6961lb # [ 190.495122] niks3-server[1111]: 2026/09/20 12:23:27 INFO Signed narinfos id=67 count=16962worker2 # [ 190.482616] nix-grpc-daemon[1517]: time=2026-09-20T12:23:26.955Z level=INFO msg="Uploading 1 narinfos"6963lb # [ 190.500039] niks3-server[1111]: 2026/09/20 12:23:27 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls6964lb # [ 190.511977] niks3-server[1111]: 2026/09/20 12:23:27 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6965lb # [ 190.517355] niks3-server[1111]: 2026/09/20 12:23:27 INFO Completed upload id=676966lb # [ 190.520400] niks3-server[1111]: 2026/09/20 12:23:27 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo6967worker2 # [ 190.504885] nix-grpc-daemon[1517]: time=2026-09-20T12:23:26.977Z level=INFO msg="Upload complete. (287ms)"6968worker2 # [ 190.512178] nix-grpc-daemon[1429]: ts=2026-09-20T12:23:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42092 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=11 outputs=16969worker2 # [ 190.520845] nix-grpc-daemon[1429]: ts=2026-09-20T12:23:26Z level=debug event=want client=3 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=06970worker2 # [ 190.524597] nix-grpc-daemon[1429]: ts=2026-09-20T12:23:26Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=16971worker2 # [ 190.533125] nix-daemon[924]: reaped child process 1769, status = succeeded6972worker2 # [ 190.564386] nix-grpc-daemon[1429]: ts=2026-09-20T12:23:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:420926973worker2 # [ 190.576317] nix-grpc-daemon[1429]: ts=2026-09-20T12:23:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:420926974worker2 # [ 190.588450] nix-grpc-daemon[1517]: time=2026-09-20T12:23:27.061Z level=INFO msg="All 2 paths already cached"6975client # copying 1 paths to 'lb:50051'...6976client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...6977worker2 # [ 190.595377] nix-grpc-daemon[1429]: ts=2026-09-20T12:23:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:420926978worker2 # [ 190.603777] nix-daemon[924]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6979lb # [ 190.776773] envoy[1009]: conn peer=[2001:db8:1::1]:55856 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=895 tx=498 ms=300476980worker2 # [ 190.765337] nix-daemon[924]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)6981worker2 # [ 190.780554] nix-daemon[924]: reaped child process 1798, status = succeeded6982worker2 # [ 190.929395] nix-daemon[924]: accepted connection from pid 1800, user nix-grpc-daemon (trusted)6983worker2 # [ 190.942429] nix-daemon[924]: reaped child process 1803, status = succeeded6984lb # [ 190.971300] niks3-server[1111]: 2026/09/20 12:23:27 INFO Received uploads request method=POST path=/api/pending_closures6985worker2 # [ 190.963517] nix-grpc-daemon[1517]: time=2026-09-20T12:23:27.436Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6986worker2 # [ 190.965456] nix-grpc-daemon[1517]: time=2026-09-20T12:23:27.438Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"6987lb # [ 190.997214] niks3-server[1111]: 2026/09/20 12:23:27 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst6988lb # [ 191.009064] niks3-server[1111]: 2026/09/20 12:23:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6989worker2 # [ 190.998940] nix-grpc-daemon[1517]: time=2026-09-20T12:23:27.471Z level=INFO msg="Uploading 1 narinfos"6990lb # [ 191.014133] niks3-server[1111]: 2026/09/20 12:23:27 INFO Signed narinfos id=68 count=16991lb # [ 191.019345] niks3-server[1111]: 2026/09/20 12:23:27 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls6992lb # [ 191.031753] niks3-server[1111]: 2026/09/20 12:23:27 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6993lb # [ 191.036422] niks3-server[1111]: 2026/09/20 12:23:27 INFO Completed upload id=686994worker2 # [ 191.021333] nix-grpc-daemon[1517]: time=2026-09-20T12:23:27.494Z level=INFO msg="Upload complete. (405ms)"6995lb # [ 191.041267] niks3-server[1111]: 2026/09/20 12:23:27 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo6996worker2 # [ 191.025516] nix-grpc-daemon[1429]: ts=2026-09-20T12:23:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42092 duration_s=0 paths=1 nar_bytes_in=7526997worker2 # [ 191.030671] nix-grpc-daemon[1429]: ts=2026-09-20T12:23:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:420926998worker2 # [ 191.039572] nix-daemon[924]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6999worker2 # [ 191.043879] nix-daemon[924]: reaped child process 1793, status = succeeded7000worker2 # [ 191.048965] nix-daemon[924]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)7001client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...7002client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7003client # farm-top> LOG-farm-top-drain7004worker2 # [ 192.269606] nix-daemon[924]: reaped child process 1807, status = succeeded7005worker2 # [ 192.399666] nix-daemon[924]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)7006worker2 # [ 192.411443] nix-daemon[924]: reaped child process 1825, status = succeeded7007lb # [ 192.439653] niks3-server[1111]: 2026/09/20 12:23:29 INFO Received uploads request method=POST path=/api/pending_closures7008worker2 # [ 192.429505] nix-grpc-daemon[1517]: time=2026-09-20T12:23:28.902Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7009worker2 # [ 192.431127] nix-grpc-daemon[1517]: time=2026-09-20T12:23:28.904Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"7010lb # [ 192.460663] niks3-server[1111]: 2026/09/20 12:23:29 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7011lb # [ 192.464621] niks3-server[1111]: 2026/09/20 12:23:29 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7012lb # [ 192.467346] niks3-server[1111]: 2026/09/20 12:23:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign7013lb # [ 192.468982] niks3-server[1111]: 2026/09/20 12:23:29 INFO Signed narinfos id=69 count=17014worker2 # [ 192.455235] nix-grpc-daemon[1517]: time=2026-09-20T12:23:28.928Z level=INFO msg="Uploading 1 narinfos"7015lb # [ 192.472823] niks3-server[1111]: 2026/09/20 12:23:29 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7016lb # [ 192.478060] niks3-server[1111]: 2026/09/20 12:23:29 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete7017lb # [ 192.481694] niks3-server[1111]: 2026/09/20 12:23:29 INFO Completed upload id=697018worker2 # [ 192.466428] nix-grpc-daemon[1517]: time=2026-09-20T12:23:28.939Z level=INFO msg="Upload complete. (210ms)"7019lb # [ 192.483335] niks3-server[1111]: 2026/09/20 12:23:29 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7020worker2 # [ 192.468744] nix-grpc-daemon[1429]: ts=2026-09-20T12:23:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42092 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=12 outputs=17021worker2 # [ 192.476054] nix-daemon[924]: reaped child process 1805, status = succeeded7022client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&2, in 5.97 seconds)7023client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath7024client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath, in 0.17 seconds)7025worker1: must succeed: rm /nix/.rw-store/fill7026worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.12 seconds)7027worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7028worker1 # [ 193.771570] nix-grpc-daemon[1733]: ts=2026-09-20T12:23:30Z level=info event=healthy reason=min_free7029worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.07 seconds)7030(finished: subtest: low disk drains a worker and builds go to the other, in 9.49 seconds)7031(finished: run the VM test script, in 194.96 seconds)7032test script finished in 195.17s7033cleanup7034kill QemuMachine (pid 50)7035client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7036kill QemuMachine (pid 49)7037lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7038kill QemuMachine (pid 51)7039worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7040kill QemuMachine (pid 52)7041worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7042(finished: cleanup, in 1.04 seconds)7043additionally exposed symbols:7044 client, lb, worker1, worker2,7045 vlan1,7046 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh