vm-test-run-nix-grpc-farm log

succeeded `vm-test-run-nix-grpc-farm` `checks.x86_64-linux.farm` · build #239 · raw

1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker2 # Disk image does not exist, creating the virtualisation disk image...13worker1 # Disk image does not exist, creating the virtualisation disk image...14worker2 # Formatting '/build/vm-state-worker2/tmp.OLrCMOLU8C', fmt=raw size=107374182415worker1: QEMU running (pid 52)16worker2: QEMU running (pid 51)17worker1 # Formatting '/build/vm-state-worker1/tmp.4U8mSGDVjH', fmt=raw size=107374182418worker2 # mke2fs 1.47.4 (6-Mar-2025)19worker1 # mke2fs 1.47.4 (6-Mar-2025)20client # Disk image does not exist, creating the virtualisation disk image...21client: QEMU running (pid 50)22worker2 # Discarding device blocks: 0/262144 done23worker1 # Discarding device blocks: 0/262144 done24worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes25worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes26worker2 # Filesystem UUID: 3cf0dd24-9088-4531-aacf-5b634dba4ce427worker1 # Filesystem UUID: 91cc112b-28b8-4fd4-a6ae-2df2e7b81a5f28worker2 # Superblock backups stored on blocks:29worker1 # Superblock backups stored on blocks:30worker2 # 32768, 98304, 163840, 22937631worker1 # 32768, 98304, 163840, 22937632worker2 # 33worker1 # 34worker2 # Allocating group tables: 0/8 done35worker1 # Allocating group tables: 0/8 done36worker2 # Writing inode tables: 0/8 done37worker1 # Writing inode tables: 0/8 done38worker2 # Creating journal (8192 blocks): done39worker1 # Creating journal (8192 blocks): done40client # Formatting '/build/vm-state-client/tmp.9gScfykLdJ', fmt=raw size=107374182441worker2 # Writing superblocks and filesystem accounting information: 0/8 done42worker1 # Writing superblocks and filesystem accounting information: 0/8 done43client # mke2fs 1.47.4 (6-Mar-2025)44worker2 # 45worker1 # 46client # Discarding device blocks: 0/262144 done47worker2 # Virtualisation disk image created.48worker1 # Virtualisation disk image created.49client # Creating filesystem with 262144 4k blocks and 65536 inodes50lb # Disk image does not exist, creating the virtualisation disk image...51client # Filesystem UUID: e796f030-2a07-4949-ac5a-bb910e0f032552lb # Formatting '/build/vm-state-lb/tmp.tUAOPrsUvx', fmt=raw size=107374182453client # Superblock backups stored on blocks:54lb # mke2fs 1.47.4 (6-Mar-2025)55client # 32768, 98304, 163840, 22937656lb # Discarding device blocks: 0/262144 done57client # 58lb # Creating filesystem with 262144 4k blocks and 65536 inodes59client # Allocating group tables: 0/8 done60lb # Filesystem UUID: 6db1eb38-5902-4b42-a10f-df0de5338e8a61client # Writing inode tables: 0/8 done62lb # Superblock backups stored on blocks:63client # Creating journal (8192 blocks): done64lb # 32768, 98304, 163840, 22937665client # Writing superblocks and filesystem accounting information: 0/8 done66lb # 67client # 68lb # Allocating group tables: 0/8 done69client # Virtualisation disk image created.70lb # Writing inode tables: 0/8 done71lb: QEMU running (pid 49)72lb # Creating journal (8192 blocks): done73lb # Writing superblocks and filesystem accounting information: 0/8 done74lb # 75lb # Virtualisation disk image created.76(finished: start all VMs, in 0.20 seconds)77client: waiting for unit network-addresses-eth1.service78client: waiting for the VM to finish booting79worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)80lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)81worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)82client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)83worker2 # 84worker2 # 85worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0086worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...87worker2 # 88worker2 # 89worker2 # 90worker2 # 91worker2 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB0092lb # 93lb # 94lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0095worker1 # 96worker2 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...97worker1 # 98worker2 # 99worker2 # 100lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...101lb # 102lb # 103lb # 104lb # 105worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00106lb # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00107worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...108worker1 # 109worker1 # 110worker1 # 111worker1 # 112worker1 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00113lb # Press Ctrl-B to configure iPXE (PCI 00:08.0)...114lb # 115lb # 116client # 117client # 118client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00119worker1 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...120worker1 # 121worker1 # 122client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...123client # 124client # 125client # 126client # 127client # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00128client # Press Ctrl-B to configure iPXE (PCI 00:08.0)...129client # 130client # 131worker2 # Booting from ROM...132lb # Booting from ROM...133worker1 # Booting from ROM...134client # Booting from ROM...135worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026136worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/axv4j5ipcz3md5gcs0myir2kw5zc8rx5-nixos-system-worker2-test/init regInfo=/nix/store/7xw1l1myi3ajs5d9h34ljmlbwcpmyijb-closure-info/registration console=ttyS0,115200n8 console=tty0137worker2 # [ 0.000000] BIOS-provided physical RAM map:138worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable139worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved140worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved141worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable142worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved143worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved144worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved145worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved146worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved147worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved148worker2 # [ 0.000000] NX (Execute Disable) protection: active149worker2 # [ 0.000000] APIC: Static calls initialized150worker2 # [ 0.000000] SMBIOS 2.8 present.151worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014152worker2 # [ 0.000000] DMI: Memory slots populated: 1/1153worker2 # [ 0.000000] Hypervisor detected: KVM154worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000155worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00156worker2 # [ 0.000000] kvm-clock: using sched offset of 452757866 cycles157worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns158worker2 # [ 0.000004] tsc: Detected 2400.012 MHz processor159worker2 # [ 0.000811] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000160worker2 # [ 0.000837] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs161worker2 # [ 0.000840] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT162worker2 # [ 0.002735] found SMP MP-table at [mem 0x000f5450-0x000f545f]163worker2 # [ 0.002746] Using GB pages for direct mapping164worker2 # [ 0.002812] RAMDISK: [mem 0x5e367000-0x5ffcffff]165worker2 # [ 0.002819] ACPI: Early table checksum verification disabled166worker2 # [ 0.002821] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )167lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026168worker2 # [ 0.002825] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)169worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026170worker2 # [ 0.002829] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)171client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026172worker2 # [ 0.002835] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)173worker2 # [ 0.002837] ACPI: FACS 0x000000005FFE0000 000040174lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/8y2rmlwiyr2iliam1yhl8i23qgaaknj8-nixos-system-lb-test/init regInfo=/nix/store/9qjmxix17wyd2infdr8gh5ii9mr60zqd-closure-info/registration console=ttyS0,115200n8 console=tty0175lb # [ 0.000000] BIOS-provided physical RAM map:176worker2 # [ 0.002839] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)177worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ybx76h9jv9dpawc3zds77wf04cv5ai7l-nixos-system-worker1-test/init regInfo=/nix/store/ymsg5phj9n8z8v1ff0hxanz4yfqahk69-closure-info/registration console=ttyS0,115200n8 console=tty0178lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable179worker1 # [ 0.000000] BIOS-provided physical RAM map:180worker2 # [ 0.002840] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)181client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/rwb9w0ssg6953fv1kadwpj7gi6289g64-nixos-system-client-test/init regInfo=/nix/store/drvr2wxmf52mip8j97fw99nxmanhvpns-closure-info/registration console=ttyS0,115200n8 console=tty0182lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved183worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable184client # [ 0.000000] BIOS-provided physical RAM map:185worker2 # [ 0.002842] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)186lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved187worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved188client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable189worker2 # [ 0.002843] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)190lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable191worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved192client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved193worker2 # [ 0.002845] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]194lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved195worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable196client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved197worker2 # [ 0.002846] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]198lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved199worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved200client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable201worker2 # [ 0.002846] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]202lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved203worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved204client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved205worker2 # [ 0.002847] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]206lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved207worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved208client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved209worker2 # [ 0.002847] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]210lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved211worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved212client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved213worker2 # [ 0.002848] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]214lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved215worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved216client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved217lb # [ 0.000000] NX (Execute Disable) protection: active218worker2 # [ 0.002848] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]219worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved220client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved221worker2 # [ 0.003069] No NUMA configuration found222lb # [ 0.000000] APIC: Static calls initialized223lb # [ 0.000000] SMBIOS 2.8 present.224worker1 # [ 0.000000] NX (Execute Disable) protection: active225client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved226worker1 # [ 0.000000] APIC: Static calls initialized227worker2 # [ 0.003070] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]228client # [ 0.000000] NX (Execute Disable) protection: active229worker1 # [ 0.000000] SMBIOS 2.8 present.230lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014231worker2 # [ 0.003072] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]232client # [ 0.000000] APIC: Static calls initialized233worker2 # [ 0.005214] Zone ranges:234client # [ 0.000000] SMBIOS 2.8 present.235lb # [ 0.000000] DMI: Memory slots populated: 1/1236worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014237lb # [ 0.000000] Hypervisor detected: KVM238worker2 # [ 0.005215] DMA [mem 0x0000000000001000-0x0000000000ffffff]239worker1 # [ 0.000000] DMI: Memory slots populated: 1/1240lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000241worker2 # [ 0.005216] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]242worker1 # [ 0.000000] Hypervisor detected: KVM243client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014244worker2 # [ 0.005217] Normal empty245lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00246worker2 # [ 0.005218] Device empty247worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000248client # [ 0.000000] DMI: Memory slots populated: 1/1249worker2 # [ 0.005219] Movable zone start for each node250client # [ 0.000000] Hypervisor detected: KVM251lb # [ 0.000000] kvm-clock: using sched offset of 459602151 cycles252worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00253worker2 # [ 0.005219] Early memory node ranges254client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000255worker1 # [ 0.000001] kvm-clock: using sched offset of 479221464 cycles256worker2 # [ 0.005220] node 0: [mem 0x0000000000001000-0x000000000009efff]257client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00258lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns259worker2 # [ 0.005221] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]260lb # [ 0.000005] tsc: Detected 2400.012 MHz processor261client # [ 0.000001] kvm-clock: using sched offset of 464690446 cycles262worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns263lb # [ 0.000811] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000264worker2 # [ 0.005222] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]265worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor266client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns267worker2 # [ 0.005240] On node 0, zone DMA: 1 pages in unavailable ranges268worker1 # [ 0.000804] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000269lb # [ 0.000838] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs270client # [ 0.000005] tsc: Detected 2400.012 MHz processor271worker2 # [ 0.005498] On node 0, zone DMA: 97 pages in unavailable ranges272client # [ 0.000826] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000273worker1 # [ 0.000831] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs274lb # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT275worker2 # [ 0.030501] On node 0, zone DMA32: 40 pages in unavailable ranges276lb # [ 0.002732] found SMP MP-table at [mem 0x000f5450-0x000f545f]277worker1 # [ 0.000834] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT278worker2 # [ 0.030953] ACPI: PM-Timer IO Port: 0x608279client # [ 0.000853] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs280lb # [ 0.002743] Using GB pages for direct mapping281worker1 # [ 0.002745] found SMP MP-table at [mem 0x000f5450-0x000f545f]282worker2 # [ 0.030963] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])283client # [ 0.000856] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT284lb # [ 0.002844] RAMDISK: [mem 0x5e367000-0x5ffcffff]285worker1 # [ 0.002757] Using GB pages for direct mapping286worker2 # [ 0.030990] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23287lb # [ 0.002853] ACPI: Early table checksum verification disabled288client # [ 0.002753] found SMP MP-table at [mem 0x000f5450-0x000f545f]289worker1 # [ 0.002899] RAMDISK: [mem 0x5e367000-0x5ffcffff]290client # [ 0.002764] Using GB pages for direct mapping291worker2 # [ 0.030992] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)292lb # [ 0.002856] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )293worker1 # [ 0.002908] ACPI: Early table checksum verification disabled294client # [ 0.002860] RAMDISK: [mem 0x5e366000-0x5ffcffff]295worker1 # [ 0.002911] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )296worker2 # [ 0.030994] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)297client # [ 0.002869] ACPI: Early table checksum verification disabled298lb # [ 0.002860] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)299worker2 # [ 0.030995] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)300client # [ 0.002871] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )301worker1 # [ 0.002914] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)302lb # [ 0.002864] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)303worker2 # [ 0.030996] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)304client # [ 0.002875] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)305worker1 # [ 0.002919] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)306worker2 # [ 0.030997] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)307lb # [ 0.002871] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)308worker1 # [ 0.002926] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)309client # [ 0.002879] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)310lb # [ 0.002873] ACPI: FACS 0x000000005FFE0000 000040311worker2 # [ 0.030999] ACPI: Using ACPI (MADT) for SMP configuration information312worker1 # [ 0.002928] ACPI: FACS 0x000000005FFE0000 000040313worker2 # [ 0.031000] ACPI: HPET id: 0x8086a201 base: 0xfed00000314client # [ 0.002886] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)315lb # [ 0.002874] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)316worker2 # [ 0.031003] TSC deadline timer available317client # [ 0.002888] ACPI: FACS 0x000000005FFE0000 000040318worker1 # [ 0.002930] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)319worker2 # [ 0.031008] CPU topo: Max. logical packages: 1320lb # [ 0.002876] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)321worker2 # [ 0.031009] CPU topo: Max. logical dies: 1322client # [ 0.002889] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)323worker1 # [ 0.002931] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)324worker2 # [ 0.031009] CPU topo: Max. dies per package: 1325lb # [ 0.002878] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)326client # [ 0.002891] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)327worker2 # [ 0.031012] CPU topo: Max. threads per core: 1328worker1 # [ 0.002933] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)329worker2 # [ 0.031013] CPU topo: Num. cores per package: 1330lb # [ 0.002879] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)331client # [ 0.002893] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)332worker1 # [ 0.002934] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)333worker2 # [ 0.031013] CPU topo: Num. threads per package: 1334lb # [ 0.002880] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]335worker1 # [ 0.002936] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]336worker2 # [ 0.031014] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs337client # [ 0.002894] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)338lb # [ 0.002881] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]339worker2 # [ 0.031030] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()340worker1 # [ 0.002937] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]341client # [ 0.002895] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]342lb # [ 0.002882] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]343worker1 # [ 0.002937] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]344client # [ 0.002896] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]345worker2 # [ 0.031059] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]346lb # [ 0.002882] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]347worker1 # [ 0.002938] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]348client # [ 0.002897] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]349worker2 # [ 0.031060] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]350lb # [ 0.002883] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]351worker1 # [ 0.002938] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]352client # [ 0.002898] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]353worker2 # [ 0.031061] [mem 0x60000000-0xafffffff] available for PCI devices354lb # [ 0.002883] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]355worker2 # [ 0.031063] Booting paravirtualized kernel on KVM356worker1 # [ 0.002939] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]357client # [ 0.002898] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]358lb # [ 0.002884] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]359client # [ 0.002898] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]360worker1 # [ 0.002939] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]361lb # [ 0.003111] No NUMA configuration found362worker2 # [ 0.031065] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns363worker1 # [ 0.003177] No NUMA configuration found364client # [ 0.002899] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]365lb # [ 0.003112] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]366worker2 # [ 0.035520] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1367worker1 # [ 0.003178] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]368client # [ 0.003124] No NUMA configuration found369lb # [ 0.003114] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]370worker2 # [ 0.037583] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152371lb # [ 0.005393] Zone ranges:372worker1 # [ 0.003180] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]373client # [ 0.003125] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]374worker1 # [ 0.005465] Zone ranges:375worker2 # [ 0.037621] kvm-guest: PV spinlocks disabled, single CPU376lb # [ 0.005395] DMA [mem 0x0000000000001000-0x0000000000ffffff]377client # [ 0.003127] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]378worker1 # [ 0.005466] DMA [mem 0x0000000000001000-0x0000000000ffffff]379client # [ 0.005324] Zone ranges:380lb # [ 0.005396] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]381worker1 # [ 0.005468] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]382lb # [ 0.005397] Normal empty383client # [ 0.005325] DMA [mem 0x0000000000001000-0x0000000000ffffff]384worker1 # [ 0.005469] Normal empty385lb # [ 0.005398] Device empty386worker1 # [ 0.005470] Device empty387client # [ 0.005327] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]388lb # [ 0.005399] Movable zone start for each node389client # [ 0.005328] Normal empty390worker1 # [ 0.005471] Movable zone start for each node391lb # [ 0.005400] Early memory node ranges392client # [ 0.005329] Device empty393worker1 # [ 0.005472] Early memory node ranges394client # [ 0.005329] Movable zone start for each node395lb # [ 0.005400] node 0: [mem 0x0000000000001000-0x000000000009efff]396worker1 # [ 0.005472] node 0: [mem 0x0000000000001000-0x000000000009efff]397worker2 # [ 0.037623] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/axv4j5ipcz3md5gcs0myir2kw5zc8rx5-nixos-system-worker2-test/init regInfo=/nix/store/7xw1l1myi3ajs5d9h34ljmlbwcpmyijb-closure-info/registration console=ttyS0,115200n8 console=tty0398client # [ 0.005330] Early memory node ranges399lb # [ 0.005401] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]400worker1 # [ 0.005473] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]401client # [ 0.005331] node 0: [mem 0x0000000000001000-0x000000000009efff]402lb # [ 0.005402] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]403worker1 # [ 0.005474] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]404client # [ 0.005332] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]405worker2 # [ 0.037712] Unknown kernel command line parameters "regInfo=/nix/store/7xw1l1myi3ajs5d9h34ljmlbwcpmyijb-closure-info/registration", will be passed to user space.406worker2 # [ 0.037725] random: crng init done407lb # [ 0.005436] On node 0, zone DMA: 1 pages in unavailable ranges408worker1 # [ 0.005493] On node 0, zone DMA: 1 pages in unavailable ranges409client # [ 0.005333] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]410lb # [ 0.005711] On node 0, zone DMA: 97 pages in unavailable ranges411worker1 # [ 0.005757] On node 0, zone DMA: 97 pages in unavailable ranges412worker2 # [ 0.037726] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes413client # [ 0.005357] On node 0, zone DMA: 1 pages in unavailable ranges414lb # [ 0.032433] On node 0, zone DMA32: 40 pages in unavailable ranges415worker1 # [ 0.032405] On node 0, zone DMA32: 40 pages in unavailable ranges416client # [ 0.005604] On node 0, zone DMA: 97 pages in unavailable ranges417worker2 # [ 0.039743] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)418lb # [ 0.032911] ACPI: PM-Timer IO Port: 0x608419worker1 # [ 0.032858] ACPI: PM-Timer IO Port: 0x608420client # [ 0.030792] On node 0, zone DMA32: 40 pages in unavailable ranges421lb # [ 0.032924] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])422worker1 # [ 0.032869] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])423worker2 # [ 0.040871] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)424client # [ 0.031256] ACPI: PM-Timer IO Port: 0x608425worker2 # [ 0.040902] Fallback order for Node 0: 0426lb # [ 0.032955] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23427worker1 # [ 0.032899] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23428client # [ 0.031266] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])429worker2 # [ 0.040905] Built 1 zonelists, mobility grouping on. Total pages: 393078430lb # [ 0.032958] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)431worker1 # [ 0.032902] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)432worker2 # [ 0.040906] Policy zone: DMA32433client # [ 0.031299] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23434lb # [ 0.032959] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)435worker1 # [ 0.032903] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)436worker2 # [ 0.043504] mem auto-init: stack:all(zero), heap alloc:on, heap free:off437client # [ 0.031302] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)438lb # [ 0.032960] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)439worker1 # [ 0.032904] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)440worker2 # [ 0.046443] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1441client # [ 0.031303] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)442worker2 # [ 0.049593] allocated 3145728 bytes of page_ext443lb # [ 0.032961] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)444worker1 # [ 0.032906] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)445client # [ 0.031304] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)446worker2 # [ 0.059079] ftrace: allocating 48731 entries in 192 pages447lb # [ 0.032962] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)448worker1 # [ 0.032906] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)449client # [ 0.031305] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)450worker2 # [ 0.059081] ftrace: allocated 192 pages with 2 groups451worker2 # [ 0.059873] Dynamic Preempt: lazy452lb # [ 0.032965] ACPI: Using ACPI (MADT) for SMP configuration information453worker1 # [ 0.032909] ACPI: Using ACPI (MADT) for SMP configuration information454client # [ 0.031306] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)455lb # [ 0.032965] ACPI: HPET id: 0x8086a201 base: 0xfed00000456worker2 # [ 0.059996] rcu: Preemptible hierarchical RCU implementation.457worker1 # [ 0.032910] ACPI: HPET id: 0x8086a201 base: 0xfed00000458client # [ 0.031308] ACPI: Using ACPI (MADT) for SMP configuration information459lb # [ 0.032969] TSC deadline timer available460worker1 # [ 0.032913] TSC deadline timer available461worker2 # [ 0.059997] rcu: RCU event tracing is enabled.462client # [ 0.031309] ACPI: HPET id: 0x8086a201 base: 0xfed00000463lb # [ 0.032973] CPU topo: Max. logical packages: 1464worker1 # [ 0.032917] CPU topo: Max. logical packages: 1465worker2 # [ 0.059998] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.466client # [ 0.031313] TSC deadline timer available467worker1 # [ 0.032918] CPU topo: Max. logical dies: 1468lb # [ 0.032974] CPU topo: Max. logical dies: 1469worker2 # [ 0.059999] Trampoline variant of Tasks RCU enabled.470client # [ 0.031317] CPU topo: Max. logical packages: 1471lb # [ 0.032974] CPU topo: Max. dies per package: 1472worker1 # [ 0.032918] CPU topo: Max. dies per package: 1473worker2 # [ 0.060000] Rude variant of Tasks RCU enabled.474client # [ 0.031318] CPU topo: Max. logical dies: 1475worker1 # [ 0.032922] CPU topo: Max. threads per core: 1476lb # [ 0.032978] CPU topo: Max. threads per core: 1477worker2 # [ 0.060000] Tracing variant of Tasks RCU enabled.478client # [ 0.031318] CPU topo: Max. dies per package: 1479worker1 # [ 0.032922] CPU topo: Num. cores per package: 1480lb # [ 0.032978] CPU topo: Num. cores per package: 1481client # [ 0.031322] CPU topo: Max. threads per core: 1482lb # [ 0.032978] CPU topo: Num. threads per package: 1483worker1 # [ 0.032923] CPU topo: Num. threads per package: 1484worker2 # [ 0.060001] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.485client # [ 0.031323] CPU topo: Num. cores per package: 1486worker1 # [ 0.032923] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs487lb # [ 0.032979] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs488client # [ 0.031323] CPU topo: Num. threads per package: 1489worker2 # [ 0.060001] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1490worker1 # [ 0.032942] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()491lb # [ 0.032996] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()492client # [ 0.031323] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs493worker2 # [ 0.060048] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.494client # [ 0.031340] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()495worker1 # [ 0.032977] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]496lb # [ 0.033028] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]497worker2 # [ 0.060050] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.498client # [ 0.031371] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]499worker1 # [ 0.032986] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]500lb # [ 0.033029] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]501worker1 # [ 0.032987] [mem 0x60000000-0xafffffff] available for PCI devices502worker2 # [ 0.060051] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.503lb # [ 0.033030] [mem 0x60000000-0xafffffff] available for PCI devices504client # [ 0.031373] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]505worker1 # [ 0.032988] Booting paravirtualized kernel on KVM506lb # [ 0.033032] Booting paravirtualized kernel on KVM507worker2 # [ 0.064477] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16508client # [ 0.031374] [mem 0x60000000-0xafffffff] available for PCI devices509client # [ 0.031375] Booting paravirtualized kernel on KVM510worker2 # [ 0.064735] rcu: srcu_init: Setting srcu_struct sizes based on contention.511worker1 # [ 0.032991] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns512lb # [ 0.033035] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns513worker1 # [ 0.037431] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1514worker2 # [ 0.064741] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns515client # [ 0.031378] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns516lb # [ 0.037497] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1517worker1 # [ 0.039642] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152518client # [ 0.035830] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1519lb # [ 0.039859] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152520worker2 # [ 0.064842] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)521worker1 # [ 0.039686] kvm-guest: PV spinlocks disabled, single CPU522worker2 # [ 0.068369] Console: colour VGA+ 80x25523lb # [ 0.039911] kvm-guest: PV spinlocks disabled, single CPU524client # [ 0.038021] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152525worker2 # [ 0.068372] printk: legacy console [tty0] enabled526client # [ 0.038062] kvm-guest: PV spinlocks disabled, single CPU527worker2 # [ 0.097927] printk: legacy console [ttyS0] enabled528worker2 # [ 0.196085] ACPI: Core revision 20250807529worker2 # [ 0.196976] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns530worker1 # [ 0.039688] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ybx76h9jv9dpawc3zds77wf04cv5ai7l-nixos-system-worker1-test/init regInfo=/nix/store/ymsg5phj9n8z8v1ff0hxanz4yfqahk69-closure-info/registration console=ttyS0,115200n8 console=tty0531lb # [ 0.039914] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/8y2rmlwiyr2iliam1yhl8i23qgaaknj8-nixos-system-lb-test/init regInfo=/nix/store/9qjmxix17wyd2infdr8gh5ii9mr60zqd-closure-info/registration console=ttyS0,115200n8 console=tty0532worker2 # [ 0.198502] APIC: Switch to symmetric I/O mode setup533client # [ 0.038063] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/rwb9w0ssg6953fv1kadwpj7gi6289g64-nixos-system-client-test/init regInfo=/nix/store/drvr2wxmf52mip8j97fw99nxmanhvpns-closure-info/registration console=ttyS0,115200n8 console=tty0534worker2 # [ 0.199457] x2apic enabled535worker1 # [ 0.039782] Unknown kernel command line parameters "regInfo=/nix/store/ymsg5phj9n8z8v1ff0hxanz4yfqahk69-closure-info/registration", will be passed to user space.536lb # [ 0.040019] Unknown kernel command line parameters "regInfo=/nix/store/9qjmxix17wyd2infdr8gh5ii9mr60zqd-closure-info/registration", will be passed to user space.537worker1 # [ 0.039794] random: crng init done538lb # [ 0.040033] random: crng init done539worker2 # [ 0.200187] APIC: Switched APIC routing to: physical x2apic540client # [ 0.038152] Unknown kernel command line parameters "regInfo=/nix/store/drvr2wxmf52mip8j97fw99nxmanhvpns-closure-info/registration", will be passed to user space.541worker1 # [ 0.039795] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes542client # [ 0.038164] random: crng init done543lb # [ 0.040034] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes544worker1 # [ 0.041927] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)545client # [ 0.038165] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes546lb # [ 0.042183] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)547worker2 # [ 0.201969] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1548worker1 # [ 0.043192] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)549client # [ 0.040221] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)550lb # [ 0.043258] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)551worker1 # [ 0.043227] Fallback order for Node 0: 0552lb # [ 0.043289] Fallback order for Node 0: 0553client # [ 0.041237] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)554worker1 # [ 0.043230] Built 1 zonelists, mobility grouping on. Total pages: 393078555client # [ 0.041267] Fallback order for Node 0: 0556worker1 # [ 0.043231] Policy zone: DMA32557lb # [ 0.043292] Built 1 zonelists, mobility grouping on. Total pages: 393078558worker2 # [ 0.203010] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns559lb # [ 0.043293] Policy zone: DMA32560client # [ 0.041269] Built 1 zonelists, mobility grouping on. Total pages: 393078561worker1 # [ 0.045743] mem auto-init: stack:all(zero), heap alloc:on, heap free:off562client # [ 0.041271] Policy zone: DMA32563worker2 # [ 0.204671] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)564lb # [ 0.045803] mem auto-init: stack:all(zero), heap alloc:on, heap free:off565worker1 # [ 0.048860] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1566lb # [ 0.049000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1567client # [ 0.043649] mem auto-init: stack:all(zero), heap alloc:on, heap free:off568worker1 # [ 0.052227] allocated 3145728 bytes of page_ext569worker2 # [ 0.205754] x86/cpu: User Mode Instruction Prevention (UMIP) activated570lb # [ 0.052557] allocated 3145728 bytes of page_ext571client # [ 0.046630] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1572worker1 # [ 0.062125] ftrace: allocating 48731 entries in 192 pages573lb # [ 0.062550] ftrace: allocating 48731 entries in 192 pages574client # [ 0.049915] allocated 3145728 bytes of page_ext575worker2 # [ 0.206795] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127576worker1 # [ 0.062127] ftrace: allocated 192 pages with 2 groups577lb # [ 0.062552] ftrace: allocated 192 pages with 2 groups578worker1 # [ 0.062958] Dynamic Preempt: lazy579client # [ 0.059616] ftrace: allocating 48731 entries in 192 pages580lb # [ 0.063408] Dynamic Preempt: lazy581worker2 # [ 0.207669] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0582worker1 # [ 0.063127] rcu: Preemptible hierarchical RCU implementation.583client # [ 0.059617] ftrace: allocated 192 pages with 2 groups584lb # [ 0.063551] rcu: Preemptible hierarchical RCU implementation.585client # [ 0.060445] Dynamic Preempt: lazy586worker1 # [ 0.063128] rcu: RCU event tracing is enabled.587lb # [ 0.063552] rcu: RCU event tracing is enabled.588client # [ 0.060574] rcu: Preemptible hierarchical RCU implementation.589worker1 # [ 0.063129] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.590worker2 # [ 0.208672] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto591client # [ 0.060574] rcu: RCU event tracing is enabled.592lb # [ 0.063553] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.593worker1 # [ 0.063130] Trampoline variant of Tasks RCU enabled.594lb # [ 0.063554] Trampoline variant of Tasks RCU enabled.595client # [ 0.060575] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.596worker1 # [ 0.063131] Rude variant of Tasks RCU enabled.597lb # [ 0.063555] Rude variant of Tasks RCU enabled.598worker2 # [ 0.209668] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl599worker1 # [ 0.063131] Tracing variant of Tasks RCU enabled.600client # [ 0.060576] Trampoline variant of Tasks RCU enabled.601lb # [ 0.063555] Tracing variant of Tasks RCU enabled.602client # [ 0.060577] Rude variant of Tasks RCU enabled.603worker2 # [ 0.210669] Transient Scheduler Attacks: Vulnerable: No microcode604worker1 # [ 0.063131] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.605client # [ 0.060577] Tracing variant of Tasks RCU enabled.606lb # [ 0.063556] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.607worker1 # [ 0.063132] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1608worker2 # [ 0.211668] Spectre V2 : Mitigation: Enhanced / Automatic IBRS609lb # [ 0.063557] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1610client # [ 0.060578] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.611worker1 # [ 0.063148] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.612worker2 # [ 0.212668] Speculative Return Stack Overflow: Mitigation: Safe RET613client # [ 0.060579] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1614lb # [ 0.063573] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.615worker1 # [ 0.063149] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.616client # [ 0.060633] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.617lb # [ 0.063575] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.618worker1 # [ 0.063150] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.619client # [ 0.060635] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.620lb # [ 0.063576] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.621worker1 # [ 0.067502] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16622lb # [ 0.067975] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16623client # [ 0.060636] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.624worker1 # [ 0.068032] rcu: srcu_init: Setting srcu_struct sizes based on contention.625worker2 # [ 0.213668] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization626lb # [ 0.068258] rcu: srcu_init: Setting srcu_struct sizes based on contention.627client # [ 0.064955] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16628worker2 # [ 0.215675] Spectre V2 : Enabling IBPB for BPF629worker1 # [ 0.068039] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns630client # [ 0.065221] rcu: srcu_init: Setting srcu_struct sizes based on contention.631lb # [ 0.068265] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns632worker2 # [ 0.216381] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier633client # [ 0.065227] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns634worker1 # [ 0.068158] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)635lb # [ 0.068370] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)636worker1 # [ 0.071803] Console: colour VGA+ 80x25637worker2 # [ 0.217668] active return thunk: srso_alias_return_thunk638lb # [ 0.072029] Console: colour VGA+ 80x25639worker1 # [ 0.071807] printk: legacy console [tty0] enabled640client # [ 0.065339] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)641lb # [ 0.072033] printk: legacy console [tty0] enabled642client # [ 0.069002] Console: colour VGA+ 80x25643worker1 # [ 0.101300] printk: legacy console [ttyS0] enabled644lb # [ 0.101772] printk: legacy console [ttyS0] enabled645worker2 # [ 0.218689] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'646client # [ 0.069006] printk: legacy console [tty0] enabled647worker1 # [ 0.197178] ACPI: Core revision 20250807648lb # [ 0.199144] ACPI: Core revision 20250807649client # [ 0.098580] printk: legacy console [ttyS0] enabled650worker2 # [ 0.219668] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'651client # [ 0.194618] ACPI: Core revision 20250807652worker2 # [ 0.220668] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'653worker1 # [ 0.198058] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns654lb # [ 0.200007] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns655worker2 # [ 0.221668] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'656worker1 # [ 0.199578] APIC: Switch to symmetric I/O mode setup657client # [ 0.195490] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns658lb # [ 0.201509] APIC: Switch to symmetric I/O mode setup659worker1 # [ 0.200558] x2apic enabled660worker2 # [ 0.222668] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'661lb # [ 0.202487] x2apic enabled662client # [ 0.197016] APIC: Switch to symmetric I/O mode setup663worker1 # [ 0.201333] APIC: Switched APIC routing to: physical x2apic664client # [ 0.197991] x2apic enabled665worker2 # [ 0.223668] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'666lb # [ 0.203264] APIC: Switched APIC routing to: physical x2apic667client # [ 0.198767] APIC: Switched APIC routing to: physical x2apic668worker2 # [ 0.224668] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'669worker1 # [ 0.203131] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1670lb # [ 0.205107] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1671worker2 # [ 0.225668] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256672client # [ 0.200608] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1673worker2 # [ 0.226668] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64674worker1 # [ 0.204120] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns675lb # [ 0.206099] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns676worker2 # [ 0.227668] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512677client # [ 0.201587] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns678worker1 # [ 0.205770] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)679worker2 # [ 0.228668] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024680lb # [ 0.207781] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)681worker1 # [ 0.206856] x86/cpu: User Mode Instruction Prevention (UMIP) activated682worker2 # [ 0.229668] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8683client # [ 0.203306] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)684lb # [ 0.208867] x86/cpu: User Mode Instruction Prevention (UMIP) activated685worker1 # [ 0.207957] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127686client # [ 0.204389] x86/cpu: User Mode Instruction Prevention (UMIP) activated687lb # [ 0.209907] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127688worker2 # [ 0.230668] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.689worker1 # [ 0.209457] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0690client # [ 0.205418] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127691lb # [ 0.211461] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0692client # [ 0.207006] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0693worker1 # [ 0.209772] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto694lb # [ 0.212562] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto695client # [ 0.208083] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto696worker1 # [ 0.211768] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl697lb # [ 0.213778] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl698worker1 # [ 0.212768] Transient Scheduler Attacks: Vulnerable: No microcode699client # [ 0.209304] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl700lb # [ 0.214778] Transient Scheduler Attacks: Vulnerable: No microcode701worker1 # [ 0.214490] Spectre V2 : Mitigation: Enhanced / Automatic IBRS702client # [ 0.210303] Transient Scheduler Attacks: Vulnerable: No microcode703lb # [ 0.216511] Spectre V2 : Mitigation: Enhanced / Automatic IBRS704worker1 # [ 0.215469] Speculative Return Stack Overflow: Mitigation: Safe RET705client # [ 0.212033] Spectre V2 : Mitigation: Enhanced / Automatic IBRS706lb # [ 0.217491] Speculative Return Stack Overflow: Mitigation: Safe RET707client # [ 0.213003] Speculative Return Stack Overflow: Mitigation: Safe RET708worker1 # [ 0.215768] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization709lb # [ 0.218537] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization710worker1 # [ 0.217776] Spectre V2 : Enabling IBPB for BPF711lb # [ 0.219786] Spectre V2 : Enabling IBPB for BPF712client # [ 0.214082] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization713client # [ 0.215311] Spectre V2 : Enabling IBPB for BPF714worker1 # [ 0.218769] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier715lb # [ 0.221324] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier716worker1 # [ 0.219769] active return thunk: srso_alias_return_thunk717client # [ 0.216304] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier718lb # [ 0.221779] active return thunk: srso_alias_return_thunk719client # [ 0.217304] active return thunk: srso_alias_return_thunk720worker1 # [ 0.221450] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'721lb # [ 0.223466] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'722worker1 # [ 0.222768] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'723client # [ 0.218967] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'724lb # [ 0.224778] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'725worker1 # [ 0.223768] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'726client # [ 0.220303] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'727lb # [ 0.225778] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'728worker1 # [ 0.224768] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'729client # [ 0.221289] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'730lb # [ 0.226778] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'731client # [ 0.222304] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'732worker1 # [ 0.225769] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'733lb # [ 0.227778] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'734client # [ 0.223303] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'735worker1 # [ 0.226768] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'736lb # [ 0.228779] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'737client # [ 0.224303] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'738worker1 # [ 0.227768] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'739lb # [ 0.229778] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'740worker1 # [ 0.228768] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256741client # [ 0.225303] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'742lb # [ 0.230778] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256743worker1 # [ 0.229768] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64744client # [ 0.226304] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256745lb # [ 0.232536] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64746worker1 # [ 0.230768] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512747client # [ 0.227303] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64748lb # [ 0.233552] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512749worker1 # [ 0.231767] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024750client # [ 0.228303] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512751lb # [ 0.234515] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024752client # [ 0.229304] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024753worker1 # [ 0.232768] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8754lb # [ 0.235532] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8755client # [ 0.230303] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8756worker1 # [ 0.233769] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.757lb # [ 0.236519] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.758client # [ 0.231303] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.759worker2 # [ 0.265149] Freeing SMP alternatives memory: 44K760worker2 # [ 0.265671] pid_max: default: 32768 minimum: 301761worker2 # [ 0.266755] LSM: initializing lsm=capability,landlock,yama,bpf,ima762worker2 # [ 0.267767] landlock: Up and running.763worker2 # [ 0.268416] Yama: becoming mindful.764worker2 # [ 0.269328] LSM support for eBPF active765worker2 # [ 0.270285] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)766worker2 # [ 0.270706] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)767worker2 # [ 0.273862] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)768worker2 # [ 0.275220] Performance Events: Fam17h+ core perfctr, AMD PMU driver.769worker2 # [ 0.275677] ... version: 2770worker2 # [ 0.276439] ... bit width: 48771worker2 # [ 0.276670] ... generic counters: 6772worker2 # [ 0.277402] ... generic bitmap: 000000000000003f773worker2 # [ 0.277670] ... fixed-purpose counters: 0774worker2 # [ 0.278456] ... fixed-purpose bitmap: 0000000000000000775worker2 # [ 0.278670] ... value mask: 0000ffffffffffff776worker2 # [ 0.279619] ... max period: 00007fffffffffff777worker2 # [ 0.280431] ... global_ctrl mask: 000000000000003f778worker2 # [ 0.280779] signal: max sigframe size: 3376779worker2 # [ 0.281619] rcu: Hierarchical SRCU implementation.780worker2 # [ 0.282329] rcu: Max phase no-delay instances is 400.781worker2 # [ 0.287196] smp: Bringing up secondary CPUs ...782worker2 # [ 0.287686] smp: Brought up 1 node, 1 CPU783worker2 # [ 0.288385] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)784lb # [ 0.271934] Freeing SMP alternatives memory: 44K785worker1 # [ 0.269148] Freeing SMP alternatives memory: 44K786client # [ 0.266542] Freeing SMP alternatives memory: 44K787lb # [ 0.272708] pid_max: default: 32768 minimum: 301788worker1 # [ 0.269771] pid_max: default: 32768 minimum: 301789client # [ 0.267308] pid_max: default: 32768 minimum: 301790worker2 # [ 0.288819] Memory: 1454576K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)791lb # [ 0.273435] LSM: initializing lsm=capability,landlock,yama,bpf,ima792worker1 # [ 0.270862] LSM: initializing lsm=capability,landlock,yama,bpf,ima793client # [ 0.268385] LSM: initializing lsm=capability,landlock,yama,bpf,ima794lb # [ 0.274583] landlock: Up and running.795worker2 # [ 0.289894] devtmpfs: initialized796worker1 # [ 0.271880] landlock: Up and running.797client # [ 0.269397] landlock: Up and running.798lb # [ 0.275217] Yama: becoming mindful.799worker1 # [ 0.272503] Yama: becoming mindful.800client # [ 0.270006] Yama: becoming mindful.801worker2 # [ 0.290708] x86/mm: Memory block size: 128MB802lb # [ 0.276002] LSM support for eBPF active803worker1 # [ 0.273406] LSM support for eBPF active804client # [ 0.270929] LSM support for eBPF active805lb # [ 0.276866] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)806client # [ 0.271398] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)807worker1 # [ 0.273930] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)808worker2 # [ 0.292430] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)809lb # [ 0.277826] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)810client # [ 0.272339] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)811worker2 # [ 0.293556] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).812worker1 # [ 0.274810] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)813worker2 # [ 0.294743] pinctrl core: initialized pinctrl subsystem814worker2 # [ 0.295888] PM: RTC time: 08:16:34, date: 2026-09-20815lb # [ 0.280505] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)816client # [ 0.275154] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)817worker1 # [ 0.277944] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)818lb # [ 0.281317] Performance Events: Fam17h+ core perfctr, AMD PMU driver.819client # [ 0.275797] Performance Events: Fam17h+ core perfctr, AMD PMU driver.820worker1 # [ 0.279348] Performance Events: Fam17h+ core perfctr, AMD PMU driver.821client # [ 0.276308] ... version: 2822lb # [ 0.281783] ... version: 2823worker1 # [ 0.279779] ... version: 2824lb # [ 0.282498] ... bit width: 48825client # [ 0.277016] ... bit width: 48826worker2 # [ 0.299304] NET: Registered PF_NETLINK/PF_ROUTE protocol family827worker1 # [ 0.280528] ... bit width: 48828client # [ 0.277310] ... generic counters: 6829lb # [ 0.282780] ... generic counters: 6830worker1 # [ 0.280770] ... generic counters: 6831client # [ 0.277986] ... generic bitmap: 000000000000003f832lb # [ 0.283448] ... generic bitmap: 000000000000003f833worker1 # [ 0.281462] ... generic bitmap: 000000000000003f834worker2 # [ 0.300164] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations835lb # [ 0.283831] ... fixed-purpose counters: 0836client # [ 0.278357] ... fixed-purpose counters: 0837worker1 # [ 0.281770] ... fixed-purpose counters: 0838lb # [ 0.284519] ... fixed-purpose bitmap: 0000000000000000839client # [ 0.279042] ... fixed-purpose bitmap: 0000000000000000840worker2 # [ 0.300690] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations841worker1 # [ 0.282451] ... fixed-purpose bitmap: 0000000000000000842lb # [ 0.284781] ... value mask: 0000ffffffffffff843client # [ 0.279306] ... value mask: 0000ffffffffffff844worker1 # [ 0.282770] ... value mask: 0000ffffffffffff845worker2 # [ 0.301935] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations846lb # [ 0.285654] ... max period: 00007fffffffffff847client # [ 0.280233] ... max period: 00007fffffffffff848worker1 # [ 0.283641] ... max period: 00007fffffffffff849worker2 # [ 0.302679] audit: initializing netlink subsys (disabled)850lb # [ 0.286428] ... global_ctrl mask: 000000000000003f851client # [ 0.280956] ... global_ctrl mask: 000000000000003f852worker1 # [ 0.284430] ... global_ctrl mask: 000000000000003f853lb # [ 0.286906] signal: max sigframe size: 3376854client # [ 0.281426] signal: max sigframe size: 3376855worker1 # [ 0.284902] signal: max sigframe size: 3376856worker2 # [ 0.303810] thermal_sys: Registered thermal governor 'fair_share'857lb # [ 0.287692] rcu: Hierarchical SRCU implementation.858client # [ 0.282184] rcu: Hierarchical SRCU implementation.859worker1 # [ 0.285665] rcu: Hierarchical SRCU implementation.860worker2 # [ 0.303812] thermal_sys: Registered thermal governor 'bang_bang'861lb # [ 0.288339] rcu: Max phase no-delay instances is 400.862client # [ 0.282865] rcu: Max phase no-delay instances is 400.863worker1 # [ 0.286324] rcu: Max phase no-delay instances is 400.864worker2 # [ 0.304671] thermal_sys: Registered thermal governor 'step_wise'865worker2 # [ 0.305599] thermal_sys: Registered thermal governor 'user_space'866worker2 # [ 0.306404] audit: type=2000 audit(1789892194.791:1): state=initialized audit_enabled=0 res=1867worker2 # [ 0.307673] thermal_sys: Registered thermal governor 'power_allocator'868worker2 # [ 0.307699] cpuidle: using governor menu869lb # [ 0.293258] smp: Bringing up secondary CPUs ...870client # [ 0.287793] smp: Bringing up secondary CPUs ...871worker1 # [ 0.291298] smp: Bringing up secondary CPUs ...872lb # [ 0.293793] smp: Brought up 1 node, 1 CPU873worker2 # [ 0.309860] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5874client # [ 0.288320] smp: Brought up 1 node, 1 CPU875worker1 # [ 0.291785] smp: Brought up 1 node, 1 CPU876lb # [ 0.294458] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)877client # [ 0.288972] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)878worker1 # [ 0.292433] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)879worker2 # [ 0.310879] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]880worker2 # [ 0.311673] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry881client # [ 0.289500] Memory: 1454560K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109348K reserved, 0K cma-reserved)882lb # [ 0.294955] Memory: 1454572K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)883worker2 # [ 0.312683] PCI: Using configuration type 1 for base access884worker1 # [ 0.292963] Memory: 1454556K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109348K reserved, 0K cma-reserved)885client # [ 0.290543] devtmpfs: initialized886lb # [ 0.296121] devtmpfs: initialized887worker1 # [ 0.294149] devtmpfs: initialized888client # [ 0.291340] x86/mm: Memory block size: 128MB889worker2 # [ 0.313761] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.890lb # [ 0.297032] x86/mm: Memory block size: 128MB891worker1 # [ 0.294955] x86/mm: Memory block size: 128MB892client # [ 0.293042] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)893lb # [ 0.298889] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)894worker1 # [ 0.296864] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)895client # [ 0.294317] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).896lb # [ 0.299812] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).897worker1 # [ 0.297799] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).898client # [ 0.295400] pinctrl core: initialized pinctrl subsystem899worker2 # [ 0.318922] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages900lb # [ 0.300865] pinctrl core: initialized pinctrl subsystem901worker1 # [ 0.298858] pinctrl core: initialized pinctrl subsystem902client # [ 0.296529] PM: RTC time: 08:16:34, date: 2026-09-20903worker2 # [ 0.319672] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page904lb # [ 0.301977] PM: RTC time: 08:16:34, date: 2026-09-20905worker1 # [ 0.299973] PM: RTC time: 08:16:34, date: 2026-09-20906client # [ 0.300071] NET: Registered PF_NETLINK/PF_ROUTE protocol family907lb # [ 0.305744] NET: Registered PF_NETLINK/PF_ROUTE protocol family908worker1 # [ 0.303507] NET: Registered PF_NETLINK/PF_ROUTE protocol family909client # [ 0.301631] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations910worker2 # [ 0.324672] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages911lb # [ 0.307006] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations912worker1 # [ 0.304304] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations913worker2 # [ 0.325670] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page914client # [ 0.302326] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations915lb # [ 0.307803] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations916worker1 # [ 0.304794] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations917client # [ 0.303570] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations918lb # [ 0.309055] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations919client # [ 0.304317] audit: initializing netlink subsys (disabled)920worker1 # [ 0.306064] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations921lb # [ 0.309794] audit: initializing netlink subsys (disabled)922worker1 # [ 0.306782] audit: initializing netlink subsys (disabled)923client # [ 0.305307] audit: type=2000 audit(1789892194.824:1): state=initialized audit_enabled=0 res=1924worker1 # [ 0.307933] thermal_sys: Registered thermal governor 'fair_share'925lb # [ 0.310782] audit: type=2000 audit(1789892194.823:1): state=initialized audit_enabled=0 res=1926client # [ 0.306467] thermal_sys: Registered thermal governor 'fair_share'927worker1 # [ 0.307935] thermal_sys: Registered thermal governor 'bang_bang'928lb # [ 0.311916] thermal_sys: Registered thermal governor 'fair_share'929client # [ 0.306468] thermal_sys: Registered thermal governor 'bang_bang'930worker1 # [ 0.308771] thermal_sys: Registered thermal governor 'step_wise'931lb # [ 0.311918] thermal_sys: Registered thermal governor 'bang_bang'932client # [ 0.307307] thermal_sys: Registered thermal governor 'step_wise'933lb # [ 0.312781] thermal_sys: Registered thermal governor 'step_wise'934worker1 # [ 0.309773] audit: type=2000 audit(1789892194.823:1): state=initialized audit_enabled=0 res=1935client # [ 0.308249] thermal_sys: Registered thermal governor 'user_space'936lb # [ 0.313751] thermal_sys: Registered thermal governor 'user_space'937worker1 # [ 0.311773] thermal_sys: Registered thermal governor 'user_space'938client # [ 0.309033] thermal_sys: Registered thermal governor 'power_allocator'939client # [ 0.309320] cpuidle: using governor menu940worker1 # [ 0.311774] thermal_sys: Registered thermal governor 'power_allocator'941lb # [ 0.314551] thermal_sys: Registered thermal governor 'power_allocator'942worker2 # [ 0.335998] ACPI: Added _OSI(Module Device)943worker1 # [ 0.312724] cpuidle: using governor menu944lb # [ 0.314794] cpuidle: using governor menu945worker2 # [ 0.336672] ACPI: Added _OSI(Processor Device)946client # [ 0.311531] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5947worker2 # [ 0.337391] ACPI: Added _OSI(Processor Aggregator Device)948worker1 # [ 0.314328] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5949lb # [ 0.317126] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5950client # [ 0.312572] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]951worker1 # [ 0.315021] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]952client # [ 0.313310] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry953lb # [ 0.318021] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]954worker1 # [ 0.315774] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry955client # [ 0.314318] PCI: Using configuration type 1 for base access956lb # [ 0.318783] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry957worker1 # [ 0.316782] PCI: Using configuration type 1 for base access958lb # [ 0.319793] PCI: Using configuration type 1 for base access959client # [ 0.315379] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.960worker1 # [ 0.317874] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.961lb # [ 0.321100] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.962worker2 # [ 0.345763] ACPI: 1 ACPI AML tables successfully acquired and loaded963worker1 # [ 0.323040] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages964lb # [ 0.326069] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages965client # [ 0.322323] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages966worker1 # [ 0.323771] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page967lb # [ 0.326782] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page968client # [ 0.323307] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page969worker2 # [ 0.350087] ACPI: Interpreter enabled970worker2 # [ 0.350686] ACPI: PM: (supports S0 S3 S4 S5)971worker2 # [ 0.351455] ACPI: Using IOAPIC for interrupt routing972worker2 # [ 0.351739] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug973worker1 # [ 0.328772] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages974worker2 # [ 0.352672] PCI: Using E820 reservations for host bridge windows975lb # [ 0.331782] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages976client # [ 0.328307] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages977worker1 # [ 0.329770] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page978client # [ 0.329307] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page979lb # [ 0.332781] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page980worker2 # [ 0.355742] ACPI: Enabled 2 GPEs in block 00 to 3F981worker1 # [ 0.340143] ACPI: Added _OSI(Module Device)982worker1 # [ 0.340772] ACPI: Added _OSI(Processor Device)983lb # [ 0.343150] ACPI: Added _OSI(Module Device)984worker2 # [ 0.364385] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])985lb # [ 0.343782] ACPI: Added _OSI(Processor Device)986worker1 # [ 0.341478] ACPI: Added _OSI(Processor Aggregator Device)987client # [ 0.340584] ACPI: Added _OSI(Module Device)988worker2 # [ 0.364677] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]989lb # [ 0.344541] ACPI: Added _OSI(Processor Aggregator Device)990client # [ 0.341270] ACPI: Added _OSI(Processor Device)991client # [ 0.341887] ACPI: Added _OSI(Processor Aggregator Device)992worker2 # [ 0.366531] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]993worker2 # [ 0.367760] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]994worker2 # [ 0.369016] PCI host bridge to bus 0000:00995worker2 # [ 0.369675] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]996worker2 # [ 0.370671] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]997client # [ 0.348075] ACPI: 1 ACPI AML tables successfully acquired and loaded998worker2 # [ 0.371670] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]999worker1 # [ 0.350298] ACPI: 1 ACPI AML tables successfully acquired and loaded1000lb # [ 0.352343] ACPI: 1 ACPI AML tables successfully acquired and loaded1001worker2 # [ 0.372671] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1002worker2 # [ 0.373671] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1003client # [ 0.352032] ACPI: Interpreter enabled1004worker2 # [ 0.374670] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1005client # [ 0.352321] ACPI: PM: (supports S0 S3 S4 S5)1006worker1 # [ 0.354046] ACPI: Interpreter enabled1007worker2 # [ 0.375671] pci_bus 0000:00: root bus resource [bus 00-ff]1008lb # [ 0.356205] ACPI: Interpreter enabled1009client # [ 0.353061] ACPI: Using IOAPIC for interrupt routing1010worker1 # [ 0.354702] ACPI: PM: (supports S0 S3 S4 S5)1011lb # [ 0.356796] ACPI: PM: (supports S0 S3 S4 S5)1012worker1 # [ 0.355273] ACPI: Using IOAPIC for interrupt routing1013worker2 # [ 0.376651] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1014lb # [ 0.357483] ACPI: Using IOAPIC for interrupt routing1015client # [ 0.353347] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1016worker1 # [ 0.355822] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1017worker2 # [ 0.378114] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1018worker1 # [ 0.356771] PCI: Using E820 reservations for host bridge windows1019lb # [ 0.359843] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1020lb # [ 0.360782] PCI: Using E820 reservations for host bridge windows1021client # [ 0.358306] PCI: Using E820 reservations for host bridge windows1022worker1 # [ 0.359915] ACPI: Enabled 2 GPEs in block 00 to 3F1023client # [ 0.359411] ACPI: Enabled 2 GPEs in block 00 to 3F1024lb # [ 0.363918] ACPI: Enabled 2 GPEs in block 00 to 3F1025worker2 # [ 0.380734] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1026worker2 # [ 0.381697] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1027worker2 # [ 0.382693] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1028worker2 # [ 0.383891] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1029client # [ 0.367350] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1030worker1 # [ 0.368613] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1031worker2 # [ 0.385381] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1032client # [ 0.368312] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1033worker1 # [ 0.369526] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1034lb # [ 0.372344] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1035client # [ 0.369386] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1036worker2 # [ 0.387678] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1037worker1 # [ 0.371447] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1038lb # [ 0.372787] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1039client # [ 0.370428] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1040worker2 # [ 0.388474] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1041worker1 # [ 0.371909] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1042lb # [ 0.373865] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1043client # [ 0.371669] PCI host bridge to bus 0000:001044worker2 # [ 0.389361] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1045worker1 # [ 0.373109] PCI host bridge to bus 0000:001046client # [ 0.372310] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1047lb # [ 0.374903] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1048worker2 # [ 0.389677] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1049worker1 # [ 0.373775] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1050client # [ 0.373306] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1051lb # [ 0.376137] PCI host bridge to bus 0000:001052worker1 # [ 0.374771] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1053worker2 # [ 0.391163] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1054client # [ 0.374306] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1055lb # [ 0.376786] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1056worker1 # [ 0.375771] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1057lb # [ 0.377781] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1058client # [ 0.375306] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1059worker1 # [ 0.376770] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1060lb # [ 0.378781] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1061client # [ 0.376306] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1062worker2 # [ 0.393671] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1063worker1 # [ 0.377771] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1064lb # [ 0.379780] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1065client # [ 0.377305] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1066worker2 # [ 0.394486] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1067worker1 # [ 0.378771] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1068client # [ 0.378306] pci_bus 0000:00: root bus resource [bus 00-ff]1069lb # [ 0.380780] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1070worker2 # [ 0.395390] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1071worker1 # [ 0.379771] pci_bus 0000:00: root bus resource [bus 00-ff]1072lb # [ 0.381781] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1073client # [ 0.379253] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1074worker1 # [ 0.380748] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1075worker2 # [ 0.396228] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1076lb # [ 0.382781] pci_bus 0000:00: root bus resource [bus 00-ff]1077client # [ 0.380743] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1078lb # [ 0.383810] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1079worker1 # [ 0.382198] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1080worker2 # [ 0.398520] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1081lb # [ 0.385225] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1082worker2 # [ 0.399334] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1083worker2 # [ 0.399693] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1084worker2 # [ 0.401230] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1085client # [ 0.383473] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1086client # [ 0.384319] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1087worker2 # [ 0.402685] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1088worker1 # [ 0.384846] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1089worker2 # [ 0.403485] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1090client # [ 0.385284] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1091worker1 # [ 0.385902] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1092worker2 # [ 0.404383] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1093lb # [ 0.387999] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1094worker1 # [ 0.386836] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1095client # [ 0.386448] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1096lb # [ 0.388793] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1097worker2 # [ 0.405222] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1098worker1 # [ 0.388088] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1099lb # [ 0.389824] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1100client # [ 0.388021] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1101worker1 # [ 0.389498] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1102lb # [ 0.391104] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1103worker2 # [ 0.406689] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1104worker2 # [ 0.407677] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1105client # [ 0.391338] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1106lb # [ 0.392531] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1107worker1 # [ 0.391795] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1108worker2 # [ 0.408601] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1109client # [ 0.392189] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1110worker1 # [ 0.392619] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1111client # [ 0.393030] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1112worker2 # [ 0.410037] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1113worker1 # [ 0.393545] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1114client # [ 0.393312] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1115lb # [ 0.395814] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1116worker1 # [ 0.394574] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1117lb # [ 0.396648] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1118worker2 # [ 0.411679] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1119lb # [ 0.397598] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1120client # [ 0.394864] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1121worker2 # [ 0.412522] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1122worker1 # [ 0.396109] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1123lb # [ 0.398634] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1124worker2 # [ 0.413357] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1125client # [ 0.397306] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1126worker1 # [ 0.397794] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1127lb # [ 0.400115] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1128worker2 # [ 0.414290] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1129client # [ 0.398191] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1130worker1 # [ 0.398627] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1131client # [ 0.399016] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1132worker1 # [ 0.399586] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1133lb # [ 0.402814] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1134worker2 # [ 0.415678] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1135worker2 # [ 0.416535] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1136lb # [ 0.403677] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1137client # [ 0.399880] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1138worker1 # [ 0.401199] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1139worker2 # [ 0.417365] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1140lb # [ 0.404519] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1141worker2 # [ 0.417676] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1142lb # [ 0.405389] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1143worker2 # [ 0.419200] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1144worker1 # [ 0.403802] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1145client # [ 0.402335] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1146worker1 # [ 0.404777] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1147client # [ 0.403314] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1148worker2 # [ 0.420683] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1149worker1 # [ 0.405757] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1150client # [ 0.404284] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1151worker2 # [ 0.421638] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1152lb # [ 0.407788] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1153worker1 # [ 0.407148] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1154lb # [ 0.409446] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1155client # [ 0.405709] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1156worker2 # [ 0.423221] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1157lb # [ 0.409802] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1158worker1 # [ 0.408793] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1159worker1 # [ 0.409691] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1160client # [ 0.408145] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1161worker2 # [ 0.424678] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1162lb # [ 0.411341] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1163worker2 # [ 0.425482] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1164client # [ 0.408919] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1165worker1 # [ 0.410492] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1166client # [ 0.409329] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1167worker2 # [ 0.426381] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1168lb # [ 0.412795] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1169worker1 # [ 0.411314] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1170lb # [ 0.413659] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1171worker2 # [ 0.427217] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1172lb # [ 0.414521] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1173client # [ 0.410875] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1174worker1 # [ 0.412778] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1175worker1 # [ 0.413681] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1176worker2 # [ 0.428679] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1177lb # [ 0.415341] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1178worker1 # [ 0.414601] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1179client # [ 0.413035] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1180worker2 # [ 0.429677] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1181client # [ 0.413315] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1182worker2 # [ 0.430611] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1183worker1 # [ 0.416175] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1184lb # [ 0.417789] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1185client # [ 0.414328] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1186lb # [ 0.418639] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1187worker2 # [ 0.432090] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1188client # [ 0.415941] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1189lb # [ 0.419559] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1190worker1 # [ 0.417803] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1191worker2 # [ 0.433222] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1192worker1 # [ 0.418646] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1193worker2 # [ 0.433887] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1194lb # [ 0.421253] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1195worker1 # [ 0.419770] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1196worker2 # [ 0.435208] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1197client # [ 0.418330] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1198worker1 # [ 0.421381] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1199client # [ 0.419313] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1200lb # [ 0.423795] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1201client # [ 0.420290] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1202worker1 # [ 0.422778] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1203lb # [ 0.424788] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1204worker1 # [ 0.423628] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1205lb # [ 0.425779] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1206client # [ 0.421760] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1207worker1 # [ 0.424486] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1208lb # [ 0.427365] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1209worker1 # [ 0.424776] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1210client # [ 0.424100] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1211client # [ 0.425030] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1212worker1 # [ 0.426353] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1213lb # [ 0.428787] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1214client # [ 0.425329] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1215lb # [ 0.429648] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1216worker1 # [ 0.427805] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1217lb # [ 0.430617] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1218client # [ 0.426312] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1219worker1 # [ 0.428776] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1220lb # [ 0.431651] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1221worker2 # [ 0.435883] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1222client # [ 0.427890] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1223worker1 # [ 0.430321] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1224lb # [ 0.433496] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1225client # [ 0.429319] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1226worker1 # [ 0.431809] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1227worker2 # [ 0.438701] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1228lb # [ 0.435794] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1229client # [ 0.430306] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1230worker1 # [ 0.432671] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1231lb # [ 0.436725] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1232worker2 # [ 0.439752] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1233worker1 # [ 0.433510] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1234client # [ 0.431864] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1235lb # [ 0.438174] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1236worker2 # [ 0.441615] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1237worker1 # [ 0.434319] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1238worker2 # [ 0.442686] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1239lb # [ 0.439803] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1240client # [ 0.434172] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1241worker1 # [ 0.436801] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1242client # [ 0.435312] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1243worker2 # [ 0.443958] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1244lb # [ 0.440664] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1245worker1 # [ 0.437615] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1246client # [ 0.436248] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1247lb # [ 0.441803] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1248worker1 # [ 0.438482] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1249worker2 # [ 0.444937] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1250client # [ 0.437673] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1251worker1 # [ 0.439370] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1252lb # [ 0.443355] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1253worker2 # [ 0.447168] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1254worker1 # [ 0.440350] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1255worker2 # [ 0.447678] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1256client # [ 0.439930] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1257lb # [ 0.444789] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1258worker1 # [ 0.440977] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1259client # [ 0.440313] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1260worker2 # [ 0.448951] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1261lb # [ 0.445680] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1262client # [ 0.441239] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1263lb # [ 0.446468] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1264worker2 # [ 0.450433] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1265worker1 # [ 0.442803] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1266lb # [ 0.447387] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1267client # [ 0.442788] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1268lb # [ 0.448793] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1269client # [ 0.444159] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1270worker2 # [ 0.453619] ACPI: PCI: Interrupt link LNKA configured for IRQ 101271lb # [ 0.449908] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1272client # [ 0.445156] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1273worker2 # [ 0.454512] ACPI: PCI: Interrupt link LNKB configured for IRQ 101274worker2 # [ 0.455756] ACPI: PCI: Interrupt link LNKC configured for IRQ 111275lb # [ 0.451478] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1276client # [ 0.446994] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1277worker2 # [ 0.456780] ACPI: PCI: Interrupt link LNKD configured for IRQ 111278worker2 # [ 0.457767] ACPI: PCI: Interrupt link LNKE configured for IRQ 101279worker2 # [ 0.458774] ACPI: PCI: Interrupt link LNKF configured for IRQ 101280worker2 # [ 0.459769] ACPI: PCI: Interrupt link LNKG configured for IRQ 111281worker2 # [ 0.460795] ACPI: PCI: Interrupt link LNKH configured for IRQ 111282worker2 # [ 0.461708] ACPI: PCI: Interrupt link GSIA configured for IRQ 161283worker1 # [ 0.443854] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1284worker2 # [ 0.462682] ACPI: PCI: Interrupt link GSIB configured for IRQ 171285worker1 # [ 0.446292] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1286worker2 # [ 0.463671] ACPI: PCI: Interrupt link GSIC configured for IRQ 181287worker2 # [ 0.464651] ACPI: PCI: Interrupt link GSID configured for IRQ 191288worker1 # [ 0.447017] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1289worker2 # [ 0.465431] ACPI: PCI: Interrupt link GSIE configured for IRQ 201290worker1 # [ 0.448292] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1291worker2 # [ 0.465687] ACPI: PCI: Interrupt link GSIF configured for IRQ 211292worker2 # [ 0.466682] ACPI: PCI: Interrupt link GSIG configured for IRQ 221293worker1 # [ 0.449049] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1294lb # [ 0.452008] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1295worker2 # [ 0.467628] ACPI: PCI: Interrupt link GSIH configured for IRQ 231296client # [ 0.447504] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1297worker1 # [ 0.450059] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1298lb # [ 0.454449] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1299client # [ 0.450042] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1300worker2 # [ 0.469436] iommu: Default domain type: Translated1301worker1 # [ 0.451027] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1302worker2 # [ 0.469684] iommu: DMA domain TLB invalidation policy: lazy mode1303lb # [ 0.455197] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1304client # [ 0.450595] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1305worker2 # [ 0.470851] ACPI: bus type USB registered1306worker1 # [ 0.452612] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1307worker2 # [ 0.471584] usbcore: registered new interface driver usbfs1308lb # [ 0.456667] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1309client # [ 0.452045] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1310worker1 # [ 0.453466] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1311worker2 # [ 0.472335] usbcore: registered new interface driver hub1312worker2 # [ 0.472681] usbcore: registered new device driver usb1313lb # [ 0.457752] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1314client # [ 0.452538] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1315worker1 # [ 0.454084] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1316worker2 # [ 0.474386] NetLabel: Initializing1317lb # [ 0.459089] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1318client # [ 0.453600] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1319worker1 # [ 0.455303] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1320worker2 # [ 0.474671] NetLabel: domain hash size = 1281321worker2 # [ 0.475391] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1322lb # [ 0.460041] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1323client # [ 0.454555] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1324worker2 # [ 0.475731] NetLabel: unlabeled traffic allowed by default1325worker2 # [ 0.476656] PCI: Using ACPI for IRQ routing1326lb # [ 0.461781] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1327client # [ 0.456396] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1328lb # [ 0.462700] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1329worker1 # [ 0.459124] ACPI: PCI: Interrupt link LNKA configured for IRQ 101330client # [ 0.457281] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1331worker1 # [ 0.459874] ACPI: PCI: Interrupt link LNKB configured for IRQ 101332lb # [ 0.463917] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1333worker1 # [ 0.460878] ACPI: PCI: Interrupt link LNKC configured for IRQ 111334client # [ 0.458626] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1335worker1 # [ 0.461866] ACPI: PCI: Interrupt link LNKD configured for IRQ 111336lb # [ 0.465484] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1337client # [ 0.460153] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1338worker1 # [ 0.462870] ACPI: PCI: Interrupt link LNKE configured for IRQ 101339worker1 # [ 0.463867] ACPI: PCI: Interrupt link LNKF configured for IRQ 101340worker1 # [ 0.464873] ACPI: PCI: Interrupt link LNKG configured for IRQ 111341worker1 # [ 0.465869] ACPI: PCI: Interrupt link LNKH configured for IRQ 111342lb # [ 0.469069] ACPI: PCI: Interrupt link LNKA configured for IRQ 101343client # [ 0.463918] ACPI: PCI: Interrupt link LNKA configured for IRQ 101344worker1 # [ 0.466807] ACPI: PCI: Interrupt link GSIA configured for IRQ 161345lb # [ 0.469890] ACPI: PCI: Interrupt link LNKB configured for IRQ 101346client # [ 0.464440] ACPI: PCI: Interrupt link LNKB configured for IRQ 101347worker1 # [ 0.467783] ACPI: PCI: Interrupt link GSIB configured for IRQ 171348lb # [ 0.470882] ACPI: PCI: Interrupt link LNKC configured for IRQ 111349client # [ 0.465405] ACPI: PCI: Interrupt link LNKC configured for IRQ 111350worker1 # [ 0.468784] ACPI: PCI: Interrupt link GSIC configured for IRQ 181351lb # [ 0.471884] ACPI: PCI: Interrupt link LNKD configured for IRQ 111352client # [ 0.466403] ACPI: PCI: Interrupt link LNKD configured for IRQ 111353worker1 # [ 0.469752] ACPI: PCI: Interrupt link GSID configured for IRQ 191354lb # [ 0.472901] ACPI: PCI: Interrupt link LNKE configured for IRQ 101355client # [ 0.467407] ACPI: PCI: Interrupt link LNKE configured for IRQ 101356worker1 # [ 0.470487] ACPI: PCI: Interrupt link GSIE configured for IRQ 201357lb # [ 0.473890] ACPI: PCI: Interrupt link LNKF configured for IRQ 101358worker1 # [ 0.470783] ACPI: PCI: Interrupt link GSIF configured for IRQ 211359client # [ 0.468408] ACPI: PCI: Interrupt link LNKF configured for IRQ 101360lb # [ 0.474885] ACPI: PCI: Interrupt link LNKG configured for IRQ 111361worker1 # [ 0.471752] ACPI: PCI: Interrupt link GSIG configured for IRQ 221362client # [ 0.469417] ACPI: PCI: Interrupt link LNKG configured for IRQ 111363worker1 # [ 0.472503] ACPI: PCI: Interrupt link GSIH configured for IRQ 231364lb # [ 0.475891] ACPI: PCI: Interrupt link LNKH configured for IRQ 111365client # [ 0.470404] ACPI: PCI: Interrupt link LNKH configured for IRQ 111366lb # [ 0.476821] ACPI: PCI: Interrupt link GSIA configured for IRQ 161367client # [ 0.471343] ACPI: PCI: Interrupt link GSIA configured for IRQ 161368worker1 # [ 0.473811] iommu: Default domain type: Translated1369lb # [ 0.477795] ACPI: PCI: Interrupt link GSIB configured for IRQ 171370client # [ 0.472321] ACPI: PCI: Interrupt link GSIB configured for IRQ 171371worker1 # [ 0.474629] iommu: DMA domain TLB invalidation policy: lazy mode1372lb # [ 0.478750] ACPI: PCI: Interrupt link GSIC configured for IRQ 181373client # [ 0.473261] ACPI: PCI: Interrupt link GSIC configured for IRQ 181374worker1 # [ 0.475778] ACPI: bus type USB registered1375lb # [ 0.479538] ACPI: PCI: Interrupt link GSID configured for IRQ 191376client # [ 0.474033] ACPI: PCI: Interrupt link GSID configured for IRQ 191377worker1 # [ 0.476533] usbcore: registered new interface driver usbfs1378lb # [ 0.479795] ACPI: PCI: Interrupt link GSIE configured for IRQ 201379client # [ 0.474321] ACPI: PCI: Interrupt link GSIE configured for IRQ 201380worker1 # [ 0.476788] usbcore: registered new interface driver hub1381lb # [ 0.480792] ACPI: PCI: Interrupt link GSIF configured for IRQ 211382client # [ 0.475279] ACPI: PCI: Interrupt link GSIF configured for IRQ 211383worker1 # [ 0.477676] usbcore: registered new device driver usb1384lb # [ 0.481793] ACPI: PCI: Interrupt link GSIG configured for IRQ 221385client # [ 0.476181] ACPI: PCI: Interrupt link GSIG configured for IRQ 221386worker1 # [ 0.479352] NetLabel: Initializing1387lb # [ 0.482792] ACPI: PCI: Interrupt link GSIH configured for IRQ 231388client # [ 0.477059] ACPI: PCI: Interrupt link GSIH configured for IRQ 231389worker1 # [ 0.479771] NetLabel: domain hash size = 1281390worker1 # [ 0.480547] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1391lb # [ 0.484861] iommu: Default domain type: Translated1392client # [ 0.478387] iommu: Default domain type: Translated1393worker1 # [ 0.480801] NetLabel: unlabeled traffic allowed by default1394client # [ 0.479196] iommu: DMA domain TLB invalidation policy: lazy mode1395lb # [ 0.485655] iommu: DMA domain TLB invalidation policy: lazy mode1396worker1 # [ 0.481720] PCI: Using ACPI for IRQ routing1397lb # [ 0.486781] ACPI: bus type USB registered1398client # [ 0.480295] ACPI: bus type USB registered1399lb # [ 0.487536] usbcore: registered new interface driver usbfs1400client # [ 0.480887] usbcore: registered new interface driver usbfs1401lb # [ 0.487798] usbcore: registered new interface driver hub1402client # [ 0.481326] usbcore: registered new interface driver hub1403lb # [ 0.488649] usbcore: registered new device driver usb1404client # [ 0.482217] usbcore: registered new device driver usb1405lb # [ 0.490273] NetLabel: Initializing1406client # [ 0.483875] NetLabel: Initializing1407lb # [ 0.490781] NetLabel: domain hash size = 1281408client # [ 0.484307] NetLabel: domain hash size = 1281409lb # [ 0.491598] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1410client # [ 0.485078] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1411lb # [ 0.492608] NetLabel: unlabeled traffic allowed by default1412client # [ 0.486134] NetLabel: unlabeled traffic allowed by default1413lb # [ 0.493582] PCI: Using ACPI for IRQ routing1414client # [ 0.487032] PCI: Using ACPI for IRQ routing1415worker2 # [ 0.519606] pci 0000:00:01.0: vgaarb: setting as boot VGA device1416worker2 # [ 0.519667] pci 0000:00:01.0: vgaarb: bridge control possible1417worker2 # [ 0.519667] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1418worker2 # [ 0.519685] vgaarb: loaded1419worker2 # [ 0.520406] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01420worker2 # [ 0.520671] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1421worker2 # [ 0.525760] clocksource: Switched to clocksource kvm-clock1422worker2 # [ 0.529214] VFS: Disk quotas dquot_6.6.01423worker2 # [ 0.529949] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1424worker2 # [ 0.531317] pnp: PnP ACPI init1425worker2 # [ 0.532145] ACPI: IRQ 4 override to edge(!), high(!)1426worker2 # [ 0.533138] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1427worker2 # [ 0.534750] pnp: PnP ACPI: found 6 devices1428worker2 # [ 0.542323] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1429worker2 # [ 0.543799] clocksource: Switched to clocksource acpi_pm1430worker1 # [ 0.525711] pci 0000:00:01.0: vgaarb: setting as boot VGA device1431worker2 # [ 0.544775] NET: Registered PF_INET protocol family1432worker1 # [ 0.525766] pci 0000:00:01.0: vgaarb: bridge control possible1433worker2 # [ 0.545903] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1434worker1 # [ 0.525766] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1435worker1 # [ 0.525775] vgaarb: loaded1436worker1 # [ 0.526519] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01437worker1 # [ 0.526770] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1438client # [ 0.530802] pci 0000:00:01.0: vgaarb: setting as boot VGA device1439worker1 # [ 0.530860] clocksource: Switched to clocksource kvm-clock1440lb # [ 0.537892] pci 0000:00:01.0: vgaarb: setting as boot VGA device1441client # [ 0.531302] pci 0000:00:01.0: vgaarb: bridge control possible1442lb # [ 0.538777] pci 0000:00:01.0: vgaarb: bridge control possible1443worker1 # [ 0.532510] VFS: Disk quotas dquot_6.6.01444client # [ 0.531302] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1445lb # [ 0.538777] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1446client # [ 0.531311] vgaarb: loaded1447lb # [ 0.538784] vgaarb: loaded1448client # [ 0.532083] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01449lb # [ 0.539522] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01450worker1 # [ 0.535119] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1451client # [ 0.532308] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1452lb # [ 0.539781] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1453worker1 # [ 0.536443] pnp: PnP ACPI init1454worker1 # [ 0.537240] ACPI: IRQ 4 override to edge(!), high(!)1455worker1 # [ 0.538274] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1456worker1 # [ 0.539908] pnp: PnP ACPI: found 6 devices1457client # [ 0.535398] clocksource: Switched to clocksource kvm-clock1458lb # [ 0.542872] clocksource: Switched to clocksource kvm-clock1459worker2 # [ 0.562103] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1460client # [ 0.537036] VFS: Disk quotas dquot_6.6.01461worker2 # [ 0.563467] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1462worker2 # [ 0.564729] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1463lb # [ 0.546350] VFS: Disk quotas dquot_6.6.01464worker2 # [ 0.566034] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1465client # [ 0.539546] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1466lb # [ 0.547106] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1467client # [ 0.540852] pnp: PnP ACPI init1468worker2 # [ 0.567214] TCP: Hash tables configured (established 16384 bind 16384)1469lb # [ 0.548429] pnp: PnP ACPI init1470client # [ 0.541662] ACPI: IRQ 4 override to edge(!), high(!)1471worker2 # [ 0.568285] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1472lb # [ 0.549230] ACPI: IRQ 4 override to edge(!), high(!)1473client # [ 0.542636] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1474worker2 # [ 0.569494] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1475lb # [ 0.550206] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1476worker1 # [ 0.547528] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1477client # [ 0.544147] pnp: PnP ACPI: found 6 devices1478worker2 # [ 0.570519] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1479lb # [ 0.551734] pnp: PnP ACPI: found 6 devices1480worker1 # [ 0.548932] clocksource: Switched to clocksource acpi_pm1481worker2 # [ 0.571654] NET: Registered PF_UNIX/PF_LOCAL protocol family1482worker1 # [ 0.549883] NET: Registered PF_INET protocol family1483worker2 # [ 0.572585] NET: Registered PF_XDP protocol family1484worker2 # [ 0.573394] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1485worker1 # [ 0.551082] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1486worker2 # [ 0.574355] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1487worker2 # [ 0.575318] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1488worker2 # [ 0.576371] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1489worker2 # [ 0.577428] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1490worker2 # [ 0.578478] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1491client # [ 0.551724] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1492lb # [ 0.559358] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1493client # [ 0.553155] clocksource: Switched to clocksource acpi_pm1494worker2 # [ 0.580421] ACPI: \_SB_.GSIA: Enabled at IRQ 161495lb # [ 0.560890] clocksource: Switched to clocksource acpi_pm1496client # [ 0.554192] NET: Registered PF_INET protocol family1497lb # [ 0.561906] NET: Registered PF_INET protocol family1498client # [ 0.555386] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1499worker2 # [ 0.582846] ACPI: \_SB_.GSIB: Enabled at IRQ 171500lb # [ 0.563124] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1501worker2 # [ 0.584682] ACPI: \_SB_.GSIC: Enabled at IRQ 181502worker2 # [ 0.586665] ACPI: \_SB_.GSID: Enabled at IRQ 191503worker2 # [ 0.588289] PCI: CLS 0 bytes, default 641504worker2 # [ 0.589137] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1505worker1 # [ 0.567676] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1506worker2 # [ 0.590759] Trying to unpack rootfs image as initramfs...1507worker1 # [ 0.569295] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1508worker1 # [ 0.570664] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1509worker1 # [ 0.572082] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1510worker1 # [ 0.573431] TCP: Hash tables configured (established 16384 bind 16384)1511worker1 # [ 0.574612] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1512worker1 # [ 0.575958] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1513client # [ 0.571877] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1514worker1 # [ 0.577036] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1515lb # [ 0.579811] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1516worker1 # [ 0.578208] NET: Registered PF_UNIX/PF_LOCAL protocol family1517client # [ 0.573397] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1518lb # [ 0.581296] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1519worker1 # [ 0.579142] NET: Registered PF_XDP protocol family1520client # [ 0.574705] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1521worker1 # [ 0.579964] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1522lb # [ 0.582591] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1523client # [ 0.576044] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1524worker1 # [ 0.580937] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1525lb # [ 0.583894] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1526client # [ 0.577275] TCP: Hash tables configured (established 16384 bind 16384)1527worker1 # [ 0.581898] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1528lb # [ 0.585100] TCP: Hash tables configured (established 16384 bind 16384)1529client # [ 0.578379] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1530worker1 # [ 0.582964] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1531lb # [ 0.586202] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1532worker1 # [ 0.584036] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1533client # [ 0.579635] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1534lb # [ 0.587503] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1535worker1 # [ 0.585073] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1536client # [ 0.580678] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1537lb # [ 0.588558] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1538client # [ 0.581823] NET: Registered PF_UNIX/PF_LOCAL protocol family1539worker1 # [ 0.586748] ACPI: \_SB_.GSIA: Enabled at IRQ 161540lb # [ 0.589768] NET: Registered PF_UNIX/PF_LOCAL protocol family1541client # [ 0.582770] NET: Registered PF_XDP protocol family1542lb # [ 0.590776] NET: Registered PF_XDP protocol family1543client # [ 0.583576] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1544worker1 # [ 0.588927] ACPI: \_SB_.GSIB: Enabled at IRQ 171545lb # [ 0.591657] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1546client # [ 0.584592] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1547lb # [ 0.592682] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1548client # [ 0.585591] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1549worker1 # [ 0.591137] ACPI: \_SB_.GSIC: Enabled at IRQ 181550lb # [ 0.593708] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1551client # [ 0.586702] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1552lb # [ 0.594819] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1553client # [ 0.587734] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1554worker1 # [ 0.593014] ACPI: \_SB_.GSID: Enabled at IRQ 191555lb # [ 0.595952] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1556client # [ 0.588833] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1557lb # [ 0.597036] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1558worker1 # [ 0.594603] PCI: CLS 0 bytes, default 641559client # [ 0.590527] ACPI: \_SB_.GSIA: Enabled at IRQ 161560lb # [ 0.598873] ACPI: \_SB_.GSIA: Enabled at IRQ 161561worker1 # [ 0.595522] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1562client # [ 0.592931] ACPI: \_SB_.GSIB: Enabled at IRQ 171563worker1 # [ 0.597364] Trying to unpack rootfs image as initramfs...1564lb # [ 0.601165] ACPI: \_SB_.GSIB: Enabled at IRQ 171565client # [ 0.594917] ACPI: \_SB_.GSIC: Enabled at IRQ 181566lb # [ 0.603470] ACPI: \_SB_.GSIC: Enabled at IRQ 181567client # [ 0.596856] ACPI: \_SB_.GSID: Enabled at IRQ 191568lb # [ 0.605569] ACPI: \_SB_.GSID: Enabled at IRQ 191569client # [ 0.598583] PCI: CLS 0 bytes, default 641570lb # [ 0.607292] PCI: CLS 0 bytes, default 641571client # [ 0.599487] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1572client # [ 0.601266] Trying to unpack rootfs image as initramfs...1573lb # [ 0.608264] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1574lb # [ 0.610060] Trying to unpack rootfs image as initramfs...1575worker2 # [ 0.635820] Initialise system trusted keyrings1576worker2 # [ 0.636732] workingset: timestamp_bits=40 max_order=19 bucket_order=01577worker2 # [ 0.660089] Key type asymmetric registered1578worker2 # [ 0.660816] Asymmetric key parser 'x509' registered1579worker2 # [ 0.661691] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1580worker1 # [ 0.643167] Initialise system trusted keyrings1581worker1 # [ 0.644115] workingset: timestamp_bits=40 max_order=19 bucket_order=01582worker2 # [ 0.667073] io scheduler mq-deadline registered1583worker2 # [ 0.667824] io scheduler kyber registered1584client # [ 0.644586] Initialise system trusted keyrings1585worker2 # [ 0.671477] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1586worker2 # [ 0.672723] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1587lb # [ 0.654718] Initialise system trusted keyrings1588client # [ 0.648162] workingset: timestamp_bits=40 max_order=19 bucket_order=01589worker2 # [ 0.676697] Linux agpgart interface v0.1031590lb # [ 0.657580] workingset: timestamp_bits=40 max_order=19 bucket_order=01591worker2 # [ 0.677469] ACPI: bus type drm_connector registered1592worker2 # [ 0.680371] usbcore: registered new interface driver usbserial_generic1593worker2 # [ 0.681488] usbserial: USB Serial support registered for generic1594worker2 # [ 0.684965] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1595worker1 # [ 0.663719] Key type asymmetric registered1596worker2 # [ 0.684965] Please enable it if your BIOS has the CPPC option.1597worker2 # [ 0.687199] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1598worker1 # [ 0.668786] Asymmetric key parser 'x509' registered1599worker2 # [ 0.691110] drop_monitor: Initializing network drop monitor service1600worker2 # [ 0.692268] NET: Registered PF_INET6 protocol family1601worker1 # [ 0.669654] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1602worker1 # [ 0.672883] io scheduler mq-deadline registered1603worker2 # [ 0.695382] Segment Routing with IPv61604worker1 # [ 0.673665] io scheduler kyber registered1605client # [ 0.669394] Key type asymmetric registered1606worker2 # [ 0.697964] In-situ OAM (IOAM) with IPv61607client # [ 0.671110] Asymmetric key parser 'x509' registered1608lb # [ 0.678815] Key type asymmetric registered1609worker2 # [ 0.698982] IPI shorthand broadcast: enabled1610lb # [ 0.679606] Asymmetric key parser 'x509' registered1611client # [ 0.671945] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1612worker1 # [ 0.677254] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1613worker1 # [ 0.678620] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1614client # [ 0.675296] io scheduler mq-deadline registered1615lb # [ 0.680502] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1616client # [ 0.678120] io scheduler kyber registered1617worker1 # [ 0.682529] Linux agpgart interface v0.1031618lb # [ 0.685606] io scheduler mq-deadline registered1619worker1 # [ 0.683300] ACPI: bus type drm_connector registered1620lb # [ 0.686401] io scheduler kyber registered1621client # [ 0.679358] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1622worker2 # [ 0.707090] sched_clock: Marking stable (571014565, 135926631)->(782524717, -75583521)1623client # [ 0.680657] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1624worker1 # [ 0.686194] usbcore: registered new interface driver usbserial_generic1625lb # [ 0.690126] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1626worker1 # [ 0.687318] usbserial: USB Serial support registered for generic1627lb # [ 0.691469] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1628worker2 # [ 0.712039] registered taskstats version 11629client # [ 0.685869] Linux agpgart interface v0.1031630worker1 # [ 0.689761] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1631worker2 # [ 0.713019] Loading compiled-in X.509 certificates1632client # [ 0.686634] ACPI: bus type drm_connector registered1633worker1 # [ 0.689761] Please enable it if your BIOS has the CPPC option.1634worker1 # [ 0.691927] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1635lb # [ 0.696831] Linux agpgart interface v0.1031636client # [ 0.689540] usbcore: registered new interface driver usbserial_generic1637lb # [ 0.697650] ACPI: bus type drm_connector registered1638client # [ 0.690615] usbserial: USB Serial support registered for generic1639worker1 # [ 0.695881] drop_monitor: Initializing network drop monitor service1640worker1 # [ 0.697101] NET: Registered PF_INET6 protocol family1641lb # [ 0.700945] usbcore: registered new interface driver usbserial_generic1642client # [ 0.694114] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1643lb # [ 0.702072] usbserial: USB Serial support registered for generic1644client # [ 0.694114] Please enable it if your BIOS has the CPPC option.1645client # [ 0.696365] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1646worker1 # [ 0.702199] Segment Routing with IPv61647lb # [ 0.705543] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1648lb # [ 0.705543] Please enable it if your BIOS has the CPPC option.1649worker1 # [ 0.704777] In-situ OAM (IOAM) with IPv61650lb # [ 0.707900] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1651worker1 # [ 0.705852] IPI shorthand broadcast: enabled1652client # [ 0.701260] drop_monitor: Initializing network drop monitor service1653client # [ 0.702446] NET: Registered PF_INET6 protocol family1654worker2 # [ 0.729402] Demotion targets for Node 0: null1655lb # [ 0.711685] drop_monitor: Initializing network drop monitor service1656worker2 # [ 0.731993] Key type .fscrypt registered1657client # [ 0.705578] Segment Routing with IPv61658lb # [ 0.712926] NET: Registered PF_INET6 protocol family1659worker2 # [ 0.732661] Key type fscrypt-provisioning registered1660client # [ 0.706260] In-situ OAM (IOAM) with IPv61661worker2 # [ 0.733593] ima: No TPM chip found, activating TPM-bypass!1662client # [ 0.708372] IPI shorthand broadcast: enabled1663lb # [ 0.715975] Segment Routing with IPv61664worker2 # [ 0.735961] ima: Allocated hash algorithm: sha11665worker2 # [ 0.736760] ima: No architecture policies found1666worker1 # [ 0.714045] sched_clock: Marking stable (580014349, 133728201)->(786206939, -72464389)1667lb # [ 0.718557] In-situ OAM (IOAM) with IPv61668lb # [ 0.719674] IPI shorthand broadcast: enabled1669worker2 # [ 0.740959] PM: Magic number: 10:89:2711670worker1 # [ 0.719001] registered taskstats version 11671worker1 # [ 0.719978] Loading compiled-in X.509 certificates1672worker2 # [ 0.742496] RAS: Correctable Errors collector initialized.1673client # [ 0.716570] sched_clock: Marking stable (582014048, 134075089)->(789273253, -73184116)1674lb # [ 0.727901] sched_clock: Marking stable (592014133, 135512302)->(798820996, -71294561)1675client # [ 0.722213] registered taskstats version 11676client # [ 0.723221] Loading compiled-in X.509 certificates1677worker2 # [ 0.751607] clk: Disabling unused clocks1678lb # [ 0.733803] registered taskstats version 11679worker2 # [ 0.753953] PM: genpd: Disabling unused power domains1680lb # [ 0.734859] Loading compiled-in X.509 certificates1681worker1 # [ 0.737763] Demotion targets for Node 0: null1682worker1 # [ 0.738800] Key type .fscrypt registered1683worker1 # [ 0.739499] Key type fscrypt-provisioning registered1684worker1 # [ 0.740498] ima: No TPM chip found, activating TPM-bypass!1685client # [ 0.739727] Demotion targets for Node 0: null1686worker1 # [ 0.745755] ima: Allocated hash algorithm: sha11687client # [ 0.742149] Key type .fscrypt registered1688worker1 # [ 0.746590] ima: No architecture policies found1689client # [ 0.742782] Key type fscrypt-provisioning registered1690client # [ 0.743689] ima: No TPM chip found, activating TPM-bypass!1691lb # [ 0.751541] Demotion targets for Node 0: null1692worker1 # [ 0.748761] PM: Magic number: 10:89:2711693client # [ 0.746113] ima: Allocated hash algorithm: sha11694lb # [ 0.753585] Key type .fscrypt registered1695worker1 # [ 0.750526] RAS: Correctable Errors collector initialized.1696client # [ 0.746875] ima: No architecture policies found1697lb # [ 0.754251] Key type fscrypt-provisioning registered1698lb # [ 0.755220] ima: No TPM chip found, activating TPM-bypass!1699client # [ 0.751130] PM: Magic number: 10:89:2711700lb # [ 0.758543] ima: Allocated hash algorithm: sha11701lb # [ 0.759349] ima: No architecture policies found1702client # [ 0.752705] RAS: Correctable Errors collector initialized.1703worker1 # [ 0.759414] clk: Disabling unused clocks1704lb # [ 0.763553] PM: Magic number: 10:89:2711705lb # [ 0.765249] RAS: Correctable Errors collector initialized.1706worker1 # [ 0.763762] PM: genpd: Disabling unused power domains1707client # [ 0.761805] clk: Disabling unused clocks1708client # [ 0.764126] PM: genpd: Disabling unused power domains1709lb # [ 0.774187] clk: Disabling unused clocks1710lb # [ 0.776547] PM: genpd: Disabling unused power domains1711worker2 # [ 0.907626] Freeing initrd memory: 29092K1712worker2 # [ 0.910695] Freeing unused decrypted memory: 2028K1713worker2 # [ 0.913243] Freeing unused kernel image (initmem) memory: 3644K1714worker2 # [ 0.914340] Write protecting the kernel read-only data: 32768k1715worker2 # [ 0.916237] Freeing unused kernel image (text/rodata gap) memory: 1208K1716worker2 # [ 0.917765] Freeing unused kernel image (rodata/data gap) memory: 752K1717worker1 # [ 0.925287] Freeing initrd memory: 29092K1718client # [ 0.921788] Freeing initrd memory: 29096K1719worker1 # [ 0.928464] Freeing unused decrypted memory: 2028K1720client # [ 0.924924] Freeing unused decrypted memory: 2028K1721worker1 # [ 0.931124] Freeing unused kernel image (initmem) memory: 3644K1722client # [ 0.927470] Freeing unused kernel image (initmem) memory: 3644K1723worker1 # [ 0.932248] Write protecting the kernel read-only data: 32768k1724client # [ 0.928572] Write protecting the kernel read-only data: 32768k1725worker1 # [ 0.934166] Freeing unused kernel image (text/rodata gap) memory: 1208K1726client # [ 0.930469] Freeing unused kernel image (text/rodata gap) memory: 1208K1727worker1 # [ 0.935748] Freeing unused kernel image (rodata/data gap) memory: 752K1728client # [ 0.931996] Freeing unused kernel image (rodata/data gap) memory: 752K1729lb # [ 0.942312] Freeing initrd memory: 29092K1730lb # [ 0.945577] Freeing unused decrypted memory: 2028K1731lb # [ 0.948206] Freeing unused kernel image (initmem) memory: 3644K1732lb # [ 0.949351] Write protecting the kernel read-only data: 32768k1733worker2 # [ 0.969054] x86/mm: Checked W+X mappings: passed, no W+X pages found.1734worker2 # [ 0.970130] Run /init as init process1735lb # [ 0.951258] Freeing unused kernel image (text/rodata gap) memory: 1208K1736lb # [ 0.952859] Freeing unused kernel image (rodata/data gap) memory: 752K1737worker2 # [ 0.979964] systemd[1]: Inserted module 'autofs4'1738worker2 # [ 0.995996] fuse: init (API version 7.45)1739worker2 # [ 1.002003] ACPI: \_SB_.GSIG: Enabled at IRQ 221740worker2 # [ 1.004484] ACPI: \_SB_.GSIH: Enabled at IRQ 231741worker2 # [ 1.008210] ACPI: \_SB_.GSIE: Enabled at IRQ 201742worker1 # [ 0.986998] x86/mm: Checked W+X mappings: passed, no W+X pages found.1743client # [ 0.983118] x86/mm: Checked W+X mappings: passed, no W+X pages found.1744worker1 # [ 0.988067] Run /init as init process1745client # [ 0.984197] Run /init as init process1746worker2 # [ 1.011009] ACPI: \_SB_.GSIF: Enabled at IRQ 211747worker1 # [ 0.998381] systemd[1]: Inserted module 'autofs4'1748client # [ 0.994124] systemd[1]: Inserted module 'autofs4'1749lb # [ 1.004127] x86/mm: Checked W+X mappings: passed, no W+X pages found.1750lb # [ 1.005210] Run /init as init process1751lb # [ 1.015649] systemd[1]: Inserted module 'autofs4'1752client # [ 1.010453] fuse: init (API version 7.45)1753worker1 # [ 1.014893] fuse: init (API version 7.45)1754client # [ 1.016337] ACPI: \_SB_.GSIG: Enabled at IRQ 221755worker1 # [ 1.020651] ACPI: \_SB_.GSIG: Enabled at IRQ 221756worker2 # [ 1.044068] systemd[1]: Successfully made /usr/ read-only.1757client # [ 1.018635] ACPI: \_SB_.GSIH: Enabled at IRQ 231758worker1 # [ 1.023127] ACPI: \_SB_.GSIH: Enabled at IRQ 231759worker1 # [ 1.026115] ACPI: \_SB_.GSIE: Enabled at IRQ 201760client # [ 1.022185] ACPI: \_SB_.GSIE: Enabled at IRQ 201761worker1 # [ 1.028323] ACPI: \_SB_.GSIF: Enabled at IRQ 211762lb # [ 1.031898] fuse: init (API version 7.45)1763client # [ 1.024842] ACPI: \_SB_.GSIF: Enabled at IRQ 211764lb # [ 1.037661] ACPI: \_SB_.GSIG: Enabled at IRQ 221765lb # [ 1.039998] ACPI: \_SB_.GSIH: Enabled at IRQ 231766lb # [ 1.043050] ACPI: \_SB_.GSIE: Enabled at IRQ 201767lb # [ 1.045322] ACPI: \_SB_.GSIF: Enabled at IRQ 211768worker1 # [ 1.058867] systemd[1]: Successfully made /usr/ read-only.1769client # [ 1.055858] systemd[1]: Successfully made /usr/ read-only.1770lb # [ 1.074698] systemd[1]: Successfully made /usr/ read-only.1771worker2 # [ 1.380750] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1772worker2 # [ 1.393565] systemd[1]: Detected virtualization kvm.1773worker2 # [ 1.395681] systemd[1]: Detected architecture x86-64.1774worker2 # [ 1.397806] systemd[1]: Running in initrd.1775worker2 # [ 1.400479] systemd[1]: Initializing machine ID from random generator.1776worker2 # [ 1.403327] systemd[1]: Hostname set to <worker2>.1777worker1 # [ 1.395563] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1778client # [ 1.392347] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1779worker1 # [ 1.407526] systemd[1]: Detected virtualization kvm.1780client # [ 1.404283] systemd[1]: Detected virtualization kvm.1781worker1 # [ 1.409652] systemd[1]: Detected architecture x86-64.1782client # [ 1.406397] systemd[1]: Detected architecture x86-64.1783lb # [ 1.410582] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1784worker1 # [ 1.411800] systemd[1]: Running in initrd.1785lb # [ 1.415945] systemd[1]: Detected virtualization kvm.1786client # [ 1.408502] systemd[1]: Running in initrd.1787lb # [ 1.416939] systemd[1]: Detected architecture x86-64.1788lb # [ 1.417912] systemd[1]: Running in initrd.1789worker1 # [ 1.414382] systemd[1]: Initializing machine ID from random generator.1790client # [ 1.410924] systemd[1]: Initializing machine ID from random generator.1791lb # [ 1.419177] systemd[1]: Initializing machine ID from random generator.1792worker1 # [ 1.417266] systemd[1]: Hostname set to <worker1>.1793lb # [ 1.421204] systemd[1]: Hostname set to <lb>.1794client # [ 1.413804] systemd[1]: Hostname set to <client>.1795worker2 # [ 1.636090] systemd[1]: bpf-restrict-fs: LSM BPF program attached1796client # [ 1.632053] systemd[1]: bpf-restrict-fs: LSM BPF program attached1797lb # [ 1.645723] systemd[1]: bpf-restrict-fs: LSM BPF program attached1798worker1 # [ 1.644169] systemd[1]: bpf-restrict-fs: LSM BPF program attached1799worker2 # [ 1.693407] systemd[1]: Queued start job for default target Initrd Default Target.1800worker2 # [ 1.697370] systemd[1]: Created slice Slice /system/modprobe.1801worker2 # [ 1.698602] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1802worker2 # [ 1.700093] systemd[1]: Expecting device /dev/disk/by-label/nixos...1803worker2 # [ 1.701240] systemd[1]: Reached target Path Units.1804worker2 # [ 1.702144] systemd[1]: Reached target Slice Units.1805worker2 # [ 1.703069] systemd[1]: Reached target Swaps.1806worker2 # [ 1.703860] systemd[1]: Reached target Timer Units.1807worker2 # [ 1.704949] systemd[1]: Listening on D-Bus System Message Bus Socket.1808worker2 # [ 1.706198] systemd[1]: Listening on Journal Socket (/dev/log).1809worker2 # [ 1.707438] systemd[1]: Listening on Journal Sockets.1810worker2 # [ 1.708562] systemd[1]: Listening on udev Control Socket.1811worker2 # [ 1.709680] systemd[1]: Listening on udev Kernel Socket.1812worker2 # [ 1.710682] systemd[1]: Reached target Socket Units.1813client # [ 1.685524] systemd[1]: Queued start job for default target Initrd Default Target.1814worker2 # [ 1.712634] systemd[1]: Starting Create List of Static Device Nodes...1815client # [ 1.689479] systemd[1]: Created slice Slice /system/modprobe.1816worker2 # [ 1.717018] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1817client # [ 1.690727] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1818client # [ 1.692149] systemd[1]: Expecting device /dev/disk/by-label/nixos...1819client # [ 1.693288] systemd[1]: Reached target Path Units.1820lb # [ 1.700580] systemd[1]: Queued start job for default target Initrd Default Target.1821client # [ 1.694160] systemd[1]: Reached target Slice Units.1822client # [ 1.695013] systemd[1]: Reached target Swaps.1823worker1 # [ 1.699298] systemd[1]: Queued start job for default target Initrd Default Target.1824client # [ 1.695807] systemd[1]: Reached target Timer Units.1825lb # [ 1.704060] systemd[1]: Created slice Slice /system/modprobe.1826client # [ 1.696819] systemd[1]: Listening on D-Bus System Message Bus Socket.1827worker2 # [ 1.723681] systemd[1]: Starting Load Kernel Module configfs...1828client # [ 1.698026] systemd[1]: Listening on Journal Socket (/dev/log).1829lb # [ 1.705242] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1830worker1 # [ 1.703176] systemd[1]: Created slice Slice /system/modprobe.1831client # [ 1.699246] systemd[1]: Listening on Journal Sockets.1832lb # [ 1.706714] systemd[1]: Expecting device /dev/disk/by-label/nixos...1833client # [ 1.700248] systemd[1]: Listening on udev Control Socket.1834lb # [ 1.707835] systemd[1]: Reached target Path Units.1835worker1 # [ 1.704427] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1836client # [ 1.701257] systemd[1]: Listening on udev Kernel Socket.1837lb # [ 1.708705] systemd[1]: Reached target Slice Units.1838worker1 # [ 1.705858] systemd[1]: Expecting device /dev/disk/by-label/nixos...1839client # [ 1.702221] systemd[1]: Reached target Socket Units.1840lb # [ 1.709616] systemd[1]: Reached target Swaps.1841worker1 # [ 1.706974] systemd[1]: Reached target Path Units.1842lb # [ 1.710414] systemd[1]: Reached target Timer Units.1843worker1 # [ 1.707836] systemd[1]: Reached target Slice Units.1844client # [ 1.703932] systemd[1]: Starting Create List of Static Device Nodes...1845worker1 # [ 1.708746] systemd[1]: Reached target Swaps.1846lb # [ 1.711459] systemd[1]: Listening on D-Bus System Message Bus Socket.1847worker1 # [ 1.709530] systemd[1]: Reached target Timer Units.1848lb # [ 1.712680] systemd[1]: Listening on Journal Socket (/dev/log).1849worker1 # [ 1.710544] systemd[1]: Listening on D-Bus System Message Bus Socket.1850lb # [ 1.713868] systemd[1]: Listening on Journal Sockets.1851worker1 # [ 1.711844] systemd[1]: Listening on Journal Socket (/dev/log).1852lb # [ 1.715060] systemd[1]: Listening on udev Control Socket.1853client # [ 1.708170] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1854worker1 # [ 1.713054] systemd[1]: Listening on Journal Sockets.1855lb # [ 1.716083] systemd[1]: Listening on udev Kernel Socket.1856worker1 # [ 1.714063] systemd[1]: Listening on udev Control Socket.1857lb # [ 1.717037] systemd[1]: Reached target Socket Units.1858worker1 # [ 1.715173] systemd[1]: Listening on udev Kernel Socket.1859worker1 # [ 1.716182] systemd[1]: Reached target Socket Units.1860lb # [ 1.718906] systemd[1]: Starting Create List of Static Device Nodes...1861worker1 # [ 1.718101] systemd[1]: Starting Create List of Static Device Nodes...1862worker2 # [ 1.740610] systemd[1]: Starting Journal Service...1863client # [ 1.716174] systemd[1]: Starting Load Kernel Module configfs...1864lb # [ 1.723599] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1865worker1 # [ 1.721822] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1866lb # [ 1.731601] systemd[1]: Starting Load Kernel Module configfs...1867worker2 # [ 1.752663] systemd[1]: Starting Load Kernel Modules...1868worker1 # [ 1.730830] systemd[1]: Starting Load Kernel Module configfs...1869client # [ 1.730695] systemd[1]: Starting Journal Service...1870lb # [ 1.746727] systemd[1]: Starting Journal Service...1871client # [ 1.743182] systemd[1]: Starting Load Kernel Modules...1872worker1 # [ 1.747949] systemd[1]: Starting Journal Service...1873worker2 # [ 1.760033] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1874worker2 # [ 1.776330] netfs: FS-Cache loaded1875lb # [ 1.759678] systemd[1]: Starting Load Kernel Modules...1876worker2 # [ 1.781339] 9pnet: Installing 9P2000 support1877worker1 # [ 1.762833] systemd[1]: Starting Load Kernel Modules...1878client # [ 1.750220] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1879worker2 # [ 1.792133] systemd[1]: Starting Coldplug All udev Devices...1880client # [ 1.766341] netfs: FS-Cache loaded1881client # [ 1.771037] 9pnet: Installing 9P2000 support1882lb # [ 1.766617] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1883worker1 # [ 1.766676] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1884lb # [ 1.783137] netfs: FS-Cache loaded1885worker1 # [ 1.782952] netfs: FS-Cache loaded1886client # [ 1.779348] systemd[1]: Starting Coldplug All udev Devices...1887worker2 # [ 1.807047] systemd[1]: Finished Create List of Static Device Nodes.1888lb # [ 1.788041] 9pnet: Installing 9P2000 support1889worker1 # [ 1.789423] 9pnet: Installing 9P2000 support1890worker2 # [ 1.812583] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1891lb # [ 1.796733] systemd[1]: Starting Coldplug All udev Devices...1892client # [ 1.793215] systemd[1]: Finished Create List of Static Device Nodes.1893worker2 # [ 1.821433] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1894client # [ 1.799677] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1895worker2 # [ 1.828436] systemd[1]: modprobe@configfs.service: Deactivated successfully.1896worker1 # [ 1.808151] systemd[1]: Starting Coldplug All udev Devices...1897worker2 # [ 1.831221] systemd-journald[67]: Collecting audit messages is disabled.1898lb # [ 1.813619] systemd[1]: Finished Create List of Static Device Nodes.1899client # [ 1.807673] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1900worker1 # [ 1.813717] systemd-journald[67]: Collecting audit messages is disabled.1901worker2 # [ 1.836306] systemd[1]: Finished Load Kernel Module configfs.1902lb # [ 1.819165] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1903client # [ 1.812510] systemd[1]: modprobe@configfs.service: Deactivated successfully.1904worker2 # [ 1.840467] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1905worker2 # [ 1.843413] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1906client # [ 1.817685] systemd-journald[67]: Collecting audit messages is disabled.1907worker1 # [ 1.822261] systemd[1]: Finished Create List of Static Device Nodes.1908client # [ 1.820454] systemd[1]: Finished Load Kernel Module configfs.1909lb # [ 1.828020] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1910worker1 # [ 1.827369] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1911client # [ 1.823845] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1912lb # [ 1.832317] systemd-journald[67]: Collecting audit messages is disabled.1913worker2 # [ 1.853046] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1914lb # [ 1.834999] systemd[1]: modprobe@configfs.service: Deactivated successfully.1915worker2 # [ 1.854404] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1916client # [ 1.828657] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1917worker1 # [ 1.836181] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1918lb # [ 1.841875] systemd[1]: Finished Load Kernel Module configfs.1919client # [ 1.838179] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1920worker1 # [ 1.843236] systemd[1]: modprobe@configfs.service: Deactivated successfully.1921lb # [ 1.846347] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1922client # [ 1.839622] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1923lb # [ 1.848972] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1924worker1 # [ 1.850243] systemd[1]: Finished Load Kernel Module configfs.1925lb # [ 1.858714] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1926worker1 # [ 1.856214] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1927lb # [ 1.860451] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1928worker2 # [ 1.884518] systemd[1]: Finished Load Kernel Modules.1929worker1 # [ 1.861321] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1930worker2 # [ 1.891157] systemd[1]: Starting Apply Kernel Variables...1931worker1 # [ 1.868983] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1932worker1 # [ 1.870533] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1933client # [ 1.868667] systemd[1]: Finished Load Kernel Modules.1934client # [ 1.876180] systemd[1]: Starting Apply Kernel Variables...1935worker2 # [ 1.906039] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1936lb # [ 1.892111] systemd[1]: Finished Load Kernel Modules.1937client # [ 1.886210] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1938worker2 # [ 1.914238] systemd[1]: Starting Create Static Device Nodes in /dev...1939lb # [ 1.899736] systemd[1]: Starting Apply Kernel Variables...1940worker1 # [ 1.900329] systemd[1]: Finished Load Kernel Modules.1941client # [ 1.895509] systemd[1]: Starting Create Static Device Nodes in /dev...1942lb # [ 1.910145] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1943worker1 # [ 1.910004] systemd[1]: Starting Apply Kernel Variables...1944worker2 # [ 1.935214] systemd[1]: Finished Apply Kernel Variables.1945lb # [ 1.920734] systemd[1]: Starting Create Static Device Nodes in /dev...1946client # [ 1.917227] systemd[1]: Finished Apply Kernel Variables.1947worker1 # [ 1.921866] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1948worker2 # [ 1.948671] systemd[1]: Started Journal Service.1949worker2 # [ 1.815231] systemd-modules-load[68]: Inserted module 'dm_mod'1950worker1 # [ 1.931442] systemd[1]: Starting Create Static Device Nodes in /dev...1951worker2 # [ 1.818436] systemd-modules-load[68]: Inserted module 'virtio_balloon'1952worker2 # [ 1.819491] systemd-modules-load[68]: Inserted module 'virtio_gpu'1953worker2 # [ 1.821269] systemd[1]: Finished Create Static Device Nodes in /dev.1954client # [ 1.931520] systemd[1]: Started Journal Service.1955client # [ 1.798463] systemd-modules-load[68]: Inserted module 'dm_mod'1956worker2 # [ 1.824240] systemd[1]: Reached target Preparation for Local File Systems.1957client # [ 1.800305] systemd-modules-load[68]: Inserted module 'virtio_balloon'1958worker2 # [ 1.825360] systemd[1]: Reached target Local File Systems.1959lb # [ 1.942843] systemd[1]: Finished Apply Kernel Variables.1960worker2 # [ 1.827248] systemd[1]: Starting Create System Files and Directories...1961worker1 # [ 1.806217] systemd-modules-load[68]: Inserted module 'dm_mod'1962client # [ 1.802847] systemd-modules-load[68]: Inserted module 'virtio_gpu'1963worker1 # [ 1.808272] systemd-modules-load[68]: Inserted module 'virtio_balloon'1964client # [ 1.805263] systemd[1]: Finished Create Static Device Nodes in /dev.1965worker1 # [ 1.811319] systemd-modules-load[68]: Inserted module 'virtio_gpu'1966worker1 # [ 1.947111] systemd[1]: Started Journal Service.1967worker2 # [ 1.834056] systemd[1]: Starting Rule-based Manager for Device Events and Files...1968lb # [ 1.955072] systemd[1]: Started Journal Service.1969client # [ 1.815110] systemd[1]: Reached target Preparation for Local File Systems.1970client # [ 1.816105] systemd[1]: Reached target Local File Systems.1971worker1 # [ 1.821202] systemd[1]: Finished Apply Kernel Variables.1972lb # [ 1.820628] systemd-modules-load[68]: Inserted module 'dm_mod'1973client # [ 1.817141] systemd[1]: Starting Create System Files and Directories...1974lb # [ 1.823743] systemd-modules-load[68]: Inserted module 'virtio_balloon'1975lb # [ 1.825501] systemd-modules-load[68]: Inserted module 'virtio_gpu'1976lb # [ 1.827491] systemd[1]: Finished Create Static Device Nodes in /dev.1977client # [ 1.823075] systemd[1]: Starting Rule-based Manager for Device Events and Files...1978lb # [ 1.833471] systemd[1]: Reached target Preparation for Local File Systems.1979lb # [ 1.834720] systemd[1]: Reached target Local File Systems.1980lb # [ 1.837474] systemd[1]: Starting Create System Files and Directories...1981worker2 # [ 1.858617] systemd[1]: Finished Create System Files and Directories.1982worker1 # [ 1.839277] systemd[1]: Finished Create Static Device Nodes in /dev.1983worker1 # [ 1.841435] systemd[1]: Reached target Preparation for Local File Systems.1984lb # [ 1.843070] systemd[1]: Starting Rule-based Manager for Device Events and Files...1985worker1 # [ 1.842471] systemd[1]: Reached target Local File Systems.1986worker1 # [ 1.846051] systemd[1]: Starting Create System Files and Directories...1987client # [ 1.845243] systemd[1]: Finished Create System Files and Directories.1988worker1 # [ 1.853071] systemd[1]: Starting Rule-based Manager for Device Events and Files...1989worker2 # [ 1.888321] systemd-udevd[76]: Using default interface naming scheme 'v261'.1990lb # [ 1.875675] systemd[1]: Finished Create System Files and Directories.1991client # [ 1.870418] systemd-udevd[83]: Using default interface naming scheme 'v261'.1992worker1 # [ 1.877165] systemd[1]: Finished Create System Files and Directories.1993worker2 # [ 1.913765] systemd[1]: Started Rule-based Manager for Device Events and Files.1994lb # [ 1.896919] systemd-udevd[80]: Using default interface naming scheme 'v261'.1995client # [ 1.893615] systemd[1]: Started Rule-based Manager for Device Events and Files.1996worker1 # [ 1.902228] systemd-udevd[83]: Using default interface naming scheme 'v261'.1997lb # [ 1.923358] systemd[1]: Started Rule-based Manager for Device Events and Files.1998worker1 # [ 1.928379] systemd[1]: Started Rule-based Manager for Device Events and Files.1999worker2 # [ 1.961197] systemd[1]: Finished Coldplug All udev Devices.2000worker2 # [ 1.963694] systemd[1]: Reached target System Initialization.

loading 1,845 hidden lines…

3846client # [ 10.681830] dhcpcd[564]: eth0: carrier acquired3847client # [ 10.692737] dhcpcd[564]: DUID 00:01:00:01:32:42:52:ed:52:54:00:12:34:563848client # [ 10.694282] dhcpcd[564]: eth0: IAID 00:12:34:563849client # [ 10.694985] dhcpcd[564]: eth0: adding address fe80::5054:ff:fe12:34563850client # [ 10.842690] 8021q: adding VLAN 0 to HW filter on device eth13851client # [ 10.722910] network-addresses-eth1-start[582]: adding address 192.168.1.1/24... done3852client # [ 10.735373] network-addresses-eth1-start[582]: adding address 2001:db8:1::1/64... done3853client # [ 10.751204] systemd[1]: Finished Address configuration of eth1.3854client # [ 10.775243] systemd[1]: Finished Extra networking commands..3855client # [ 10.777207] systemd[1]: Reached target Network.3856client # [ 10.779804] systemd[1]: Starting Permit User Sessions...3857client # [ 10.805085] systemd[1]: Finished Permit User Sessions.3858client # [ 10.808854] systemd[1]: Started Getty on tty1.3859client # [ 10.809837] systemd[1]: Reached target Login Prompts.3860lb # [ 11.119705] dhcpcd[713]: eth0: soliciting a DHCP lease3861lb # [ 11.269120] NET: Registered PF_PACKET protocol family3862lb # [ 11.138456] dhcpcd[713]: eth0: offered 10.0.2.15 from 10.0.2.23863lb # [ 11.141191] dhcpcd[713]: eth0: probing address 10.0.2.15/243864client: (finished: waiting for unit network-addresses-eth1.service, in 11.99 seconds)3865client: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; }3866client: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; }, in 0.02 seconds)3867lb: waiting for unit network-addresses-eth1.service3868lb: waiting for the VM to finish booting3869lb: Guest shell says: b'Spawning backdoor root shell...\n'3870lb: connected to guest root shell3871lb: (connecting took 0.00 seconds)3872lb: (finished: waiting for the VM to finish booting, in 0.00 seconds)3873worker2 # [ 11.414723] dhcpcd[691]: eth0: soliciting a DHCP lease3874lb: (finished: waiting for unit network-addresses-eth1.service, in 0.05 seconds)3875lb: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; }3876worker2 # [ 11.570845] NET: Registered PF_PACKET protocol family3877worker2 # [ 11.440816] dhcpcd[691]: eth0: offered 10.0.2.15 from 10.0.2.23878worker2 # [ 11.445342] dhcpcd[691]: eth0: probing address 10.0.2.15/243879lb: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; }, in 0.03 seconds)3880worker1: waiting for unit network-addresses-eth1.service3881worker1: waiting for the VM to finish booting3882worker1: Guest shell says: b'Spawning backdoor root shell...\n'3883worker1: connected to guest root shell3884worker1: (connecting took 0.00 seconds)3885worker1: (finished: waiting for the VM to finish booting, in 0.00 seconds)3886lb # [ 11.479588] dhcpcd[713]: eth0: soliciting an IPv6 router3887lb # [ 11.480817] dhcpcd[713]: eth0: Router Advertisement from fe80::23888lb # [ 11.481785] dhcpcd[713]: eth0: adding address fec0::5054:ff:fe12:3456/643889lb # [ 11.482695] dhcpcd[713]: eth0: adding route to fec0::/643890lb # [ 11.483581] dhcpcd[713]: eth0: adding default route via fe80::23891worker1: (finished: waiting for unit network-addresses-eth1.service, in 0.10 seconds)3892worker1: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; }3893worker1: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; }, in 0.03 seconds)3894worker2: waiting for unit network-addresses-eth1.service3895worker2: waiting for the VM to finish booting3896worker2: Guest shell says: b'Spawning backdoor root shell...\n'3897worker2: connected to guest root shell3898worker2: (connecting took 0.00 seconds)3899worker2: (finished: waiting for the VM to finish booting, in 0.00 seconds)3900worker2: (finished: waiting for unit network-addresses-eth1.service, in 0.08 seconds)3901worker2: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; }3902worker2: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; }, in 0.03 seconds)3903lb: waiting for unit niks3.service3904client # [ 11.863704] dhcpcd[564]: eth0: soliciting a DHCP lease3905client # [ 12.024256] NET: Registered PF_PACKET protocol family3906client # [ 11.896795] dhcpcd[564]: eth0: offered 10.0.2.15 from 10.0.2.23907client # [ 11.901368] dhcpcd[564]: eth0: probing address 10.0.2.15/243908worker1 # [ 12.047917] dhcpcd[692]: eth0: soliciting an IPv6 router3909worker1 # [ 12.050651] dhcpcd[692]: eth0: Router Advertisement from fe80::23910worker1 # [ 12.052743] dhcpcd[692]: eth0: adding address fec0::5054:ff:fe12:3456/643911worker1 # [ 12.054967] dhcpcd[692]: eth0: adding route to fec0::/643912worker1 # [ 12.056858] dhcpcd[692]: eth0: adding default route via fe80::23913lb # [ 12.179564] postgresql-pre-start[727]: selecting default time zone ... UTC3914lb # [ 12.184475] postgresql-pre-start[727]: creating configuration files ... ok3915worker1 # [ 12.269129] dhcpcd[692]: eth0: soliciting a DHCP lease3916worker1 # [ 12.425668] NET: Registered PF_PACKET protocol family3917worker1 # [ 12.298267] dhcpcd[692]: eth0: offered 10.0.2.15 from 10.0.2.23918worker1 # [ 12.301409] dhcpcd[692]: eth0: probing address 10.0.2.15/243919client # [ 12.315484] dhcpcd[564]: eth0: soliciting an IPv6 router3920client # [ 12.318180] dhcpcd[564]: eth0: Router Advertisement from fe80::23921client # [ 12.320483] dhcpcd[564]: eth0: adding address fec0::5054:ff:fe12:3456/643922client # [ 12.323127] dhcpcd[564]: eth0: adding route to fec0::/643923client # [ 12.325157] dhcpcd[564]: eth0: adding default route via fe80::23924lb # [ 12.421277] postgresql-pre-start[727]: running bootstrap script ... ok3925worker2 # [ 12.573837] dhcpcd[691]: eth0: soliciting an IPv6 router3926worker2 # [ 12.576646] dhcpcd[691]: eth0: Router Advertisement from fe80::23927worker2 # [ 12.578829] dhcpcd[691]: eth0: adding address fec0::5054:ff:fe12:3456/643928worker2 # [ 12.581118] dhcpcd[691]: eth0: adding route to fec0::/643929worker2 # [ 12.582997] dhcpcd[691]: eth0: adding default route via fe80::23930lb # [ 13.034327] postgresql-pre-start[727]: performing post-bootstrap initialization ... ok3931lb # [ 13.178557] postgresql-pre-start[727]: syncing data to disk ... ok3932lb # [ 13.180499] postgresql-pre-start[727]: initdb: warning: enabling "trust" authentication for local connections3933lb # [ 13.181721] postgresql-pre-start[727]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.3934lb # [ 13.183585] postgresql-pre-start[727]: Success. You can now start the database server using:3935lb # [ 13.184642] postgresql-pre-start[727]: pg_ctl -D /var/lib/postgresql/18 -l logfile start3936lb # [ 13.289679] postgres[798]: [798] LOG: starting PostgreSQL 18.6 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit3937lb # [ 13.293456] postgres[798]: [798] LOG: listening on IPv6 address "::1", port 54323938lb # [ 13.294529] postgres[798]: [798] LOG: listening on IPv4 address "127.0.0.1", port 54323939lb # [ 13.296801] postgres[798]: [798] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"3940lb # [ 13.310309] postgres[807]: [807] LOG: database system was shut down at 2026-09-20 08:16:47 GMT3941lb # [ 13.315174] postgres[798]: [798] LOG: database system is ready to accept connections3942lb # [ 13.319225] systemd[1]: Started PostgreSQL Server.3943lb # [ 13.326228] systemd[1]: Starting PostgreSQL Setup Scripts...3944lb # [ 13.500277] postgresql-setup-start[818]: CREATE DATABASE3945lb # [ 13.546755] postgresql-setup-start[823]: CREATE ROLE3946lb # [ 13.568384] postgresql-setup-start[825]: ALTER DATABASE3947lb # [ 13.574319] systemd[1]: Finished PostgreSQL Setup Scripts.3948lb # [ 13.576423] systemd[1]: Reached target PostgreSQL.3949worker2 # [ 15.794987] dhcpcd[691]: eth0: leased 10.0.2.15 for 86400 seconds3950worker2 # [ 15.796457] dhcpcd[691]: eth0: adding route to 10.0.2.0/243951worker2 # [ 15.797571] dhcpcd[691]: eth0: adding default route via 10.0.2.23952worker2 # [ 15.884648] systemd[1]: Started DHCP Client.3953worker2 # [ 15.886203] systemd[1]: Reached target Multi-User System.3954worker2 # [ 15.886951] systemd[1]: Startup finished in 837ms (kernel) + 4.165s (initrd) + 10.882s (userspace) = 15.885s.3955lb # [ 16.304636] dhcpcd[713]: eth0: leased 10.0.2.15 for 86400 seconds3956lb # [ 16.305682] dhcpcd[713]: eth0: adding route to 10.0.2.0/243957lb # [ 16.306523] dhcpcd[713]: eth0: adding default route via 10.0.2.23958lb # [ 16.413825] systemd[1]: Started DHCP Client.3959lb # [ 16.416506] systemd[1]: Reached target Network is Online.3960lb # [ 16.421236] systemd[1]: Started Envoy reverse proxy.3961lb # [ 16.425560] systemd[1]: Started mock-oidc.service.3962lb # [ 16.524550] mock-oidc-server[911]: Mock OIDC Server running3963lb # [ 16.525940] mock-oidc-server[911]: OIDC Address: 192.168.1.2:80803964lb # [ 16.526779] mock-oidc-server[911]: Issue Address: 0.0.0.0:80813965lb # [ 16.527757] mock-oidc-server[911]: Issuer: http://192.168.1.2:8080/oidc3966lb # [ 16.529163] mock-oidc-server[911]: JWKS: http://192.168.1.2:8080/oidc/.well-known/jwks.json3967lb # [ 16.530585] mock-oidc-server[911]: Discovery: http://192.168.1.2:8080/oidc/.well-known/openid-configuration3968lb # [ 16.532120] mock-oidc-server[911]: Issue tokens: http://0.0.0.0:8081/issue?sub=...3969client # [ 16.683658] dhcpcd[564]: eth0: leased 10.0.2.15 for 86400 seconds3970client # [ 16.686307] dhcpcd[564]: eth0: adding route to 10.0.2.0/243971client # [ 16.688538] dhcpcd[564]: eth0: adding default route via 10.0.2.23972client # [ 16.793305] systemd[1]: Started DHCP Client.3973client # [ 16.794956] systemd[1]: Reached target Multi-User System.3974client # [ 16.796220] systemd[1]: Startup finished in 853ms (kernel) + 5.064s (initrd) + 10.878s (userspace) = 16.795s.3975lb # [ 16.964576] envoy[910]: [2026-09-20 08:16:51.904][910][info][main] [source/server/server.cc:467] initializing epoch 0 (base id=0, hot restart version=11.104)3976lb # [ 16.966700] envoy[910]: [2026-09-20 08:16:51.904][910][info][main] [source/server/server.cc:469] statically linked extensions:3977lb # [ 16.968203] envoy[910]: [2026-09-20 08:16:51.904][910][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed3978lb # [ 16.970087] envoy[910]: [2026-09-20 08:16:51.904][910][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation3979lb # [ 16.972146] envoy[910]: [2026-09-20 08:16:51.904][910][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe3980lb # [ 16.974609] envoy[910]: [2026-09-20 08:16:51.904][910][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple3981lb # [ 16.977046] envoy[910]: [2026-09-20 08:16:51.904][910][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router3982lb # [ 16.979209] envoy[910]: [2026-09-20 08:16:51.904][910][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory3983lb # [ 16.981604] envoy[910]: [2026-09-20 08:16:51.906][910][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret3984lb # [ 16.983560] envoy[910]: [2026-09-20 08:16:51.906][910][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian23985lb # [ 16.985128] envoy[910]: [2026-09-20 08:16:51.906][910][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service3986lb # [ 16.989143] envoy[910]: [2026-09-20 08:16:51.906][910][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router3987lb # [ 17.023091] envoy[910]: [2026-09-20 08:16:51.906][910][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest3988lb # [ 17.027817] envoy[910]: [2026-09-20 08:16:51.908][910][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override3989lb # [ 17.030471] envoy[910]: [2026-09-20 08:16:51.908][910][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts3990lb # [ 17.033158] envoy[910]: [2026-09-20 08:16:51.910][910][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats3991lb # [ 17.035431] envoy[910]: [2026-09-20 08:16:51.912][910][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based3992lb # [ 17.039156] envoy[910]: [2026-09-20 08:16:51.912][910][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed3993lb # [ 17.041313] envoy[910]: [2026-09-20 08:16:51.912][910][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory3994lb # [ 17.044798] envoy[910]: [2026-09-20 08:16:51.912][910][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso3995lb # [ 17.046849] envoy[910]: [2026-09-20 08:16:51.912][910][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v83996lb # [ 17.048676] envoy[910]: [2026-09-20 08:16:51.912][910][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file3997lb # [ 17.050502] envoy[910]: [2026-09-20 08:16:51.912][910][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua3998lb # [ 17.052167] envoy[910]: [2026-09-20 08:16:51.914][910][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san3999lb # [ 17.058349] envoy[910]: [2026-09-20 08:16:51.914][910][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response4000lb # [ 17.060432] envoy[910]: [2026-09-20 08:16:51.917][910][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default4001lb # [ 17.062597] envoy[910]: [2026-09-20 08:16:51.917][910][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher4002lb # [ 17.064508] envoy[910]: [2026-09-20 08:16:51.917][910][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator4003lb # [ 17.066643] envoy[910]: [2026-09-20 08:16:51.917][910][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port4004lb # [ 17.068301] envoy[910]: [2026-09-20 08:16:51.917][910][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name4005lb # [ 17.071430] envoy[910]: [2026-09-20 08:16:51.917][910][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action4006lb # [ 17.073291] envoy[910]: [2026-09-20 08:16:51.917][910][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr4007lb # [ 17.074946] envoy[910]: [2026-09-20 08:16:51.917][910][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor4008lb # [ 17.077204] envoy[910]: [2026-09-20 08:16:51.917][910][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic4009lb # [ 17.078808] envoy[910]: [2026-09-20 08:16:51.917][910][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm4010lb # [ 17.084547] envoy[910]: [2026-09-20 08:16:51.917][910][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter4011lb # [ 17.086332] envoy[910]: [2026-09-20 08:16:51.918][910][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query4012lb # [ 17.089410] envoy[910]: [2026-09-20 08:16:51.918][910][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value4013lb # [ 17.093088] envoy[910]: [2026-09-20 08:16:51.918][910][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override4014lb # [ 17.095440] envoy[910]: [2026-09-20 08:16:51.918][910][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default4015lb # [ 17.097266] envoy[910]: [2026-09-20 08:16:51.918][910][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router4016lb # [ 17.099930] envoy[910]: [2026-09-20 08:16:51.918][910][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable4017lb # [ 17.101704] envoy[910]: [2026-09-20 08:16:51.918][910][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log4018lb # [ 17.108105] envoy[910]: [2026-09-20 08:16:51.918][910][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default4019lb # [ 17.109571] envoy[910]: [2026-09-20 08:16:51.918][910][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche4020lb # [ 17.111286] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap4021lb # [ 17.115421] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case4022lb # [ 17.117488] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple4023lb # [ 17.119487] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift4024lb # [ 17.122373] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection4025lb # [ 17.124183] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin4026lb # [ 17.127227] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection4027lb # [ 17.131769] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert4028lb # [ 17.133585] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule4029lb # [ 17.137075] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal4030lb # [ 17.138756] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid4031lb # [ 17.140214] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor4032lb # [ 17.142739] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata4033lb # [ 17.144857] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls4034lb # [ 17.150276] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default4035lb # [ 17.152209] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder4036lb # [ 17.154316] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd4037lb # [ 17.159417] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter4038lb # [ 17.161378] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls4039lb # [ 17.165651] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input4040lb # [ 17.167645] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header4041lb # [ 17.170320] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource4042lb # [ 17.173604] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth24043lb # [ 17.176030] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options4044lb # [ 17.180270] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name4045lb # [ 17.182710] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities4046lb # [ 17.184731] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy4047lb # [ 17.187673] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator4048lb # [ 17.190144] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4049lb # [ 17.192727] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind4050lb # [ 17.194543] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated4051lb # [ 17.196710] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain4052lb # [ 17.198634] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json4053lb # [ 17.200967] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re24054lb # [ 17.202708] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher4055lb # [ 17.205463] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default4056lb # [ 17.207275] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state4057lb # [ 17.210127] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default4058lb # [ 17.211958] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port4059lb # [ 17.219348] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default4060lb # [ 17.221425] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy4061lb # [ 17.233538] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction4062lb # [ 17.237916] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy4063lb # [ 17.240378] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme4064lb # [ 17.243377] envoy[910]: [2026-09-20 08:16:51.919][910][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4065lb # [ 17.245963] envoy[910]: [2026-09-20 08:16:51.955][910][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo4066lb # [ 17.247727] envoy[910]: [2026-09-20 08:16:51.955][910][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http14067lb # [ 17.249953] envoy[910]: [2026-09-20 08:16:51.955][910][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4068lb # [ 17.251956] envoy[910]: [2026-09-20 08:16:51.955][910][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default4069lb # [ 17.254085] envoy[910]: [2026-09-20 08:16:51.955][910][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality4070lb # [ 17.260615] envoy[910]: [2026-09-20 08:16:51.955][910][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit4071lb # [ 17.263386] envoy[910]: [2026-09-20 08:16:51.955][910][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params4072lb # [ 17.272678] envoy[910]: [2026-09-20 08:16:51.955][910][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector4073lb # [ 17.278354] envoy[910]: [2026-09-20 08:16:51.955][910][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input4074lb # [ 17.280622] envoy[910]: [2026-09-20 08:16:51.955][910][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based4075lb # [ 17.282639] envoy[910]: [2026-09-20 08:16:51.955][910][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router4076lb # [ 17.284375] envoy[910]: [2026-09-20 08:16:51.955][910][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff4077lb # [ 17.286633] envoy[910]: [2026-09-20 08:16:51.955][910][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config4078lb # [ 17.290219] envoy[910]: [2026-09-20 08:16:51.955][910][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation4079lb # [ 17.293089] envoy[910]: [2026-09-20 08:16:51.955][910][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default4080lb # [ 17.294788] envoy[910]: [2026-09-20 08:16:51.955][910][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle4081lb # [ 17.296621] envoy[910]: [2026-09-20 08:16:52.099][910][info][main] [source/server/server.cc:531] HTTP header map info:4082lb # [ 17.297928] envoy[910]: [2026-09-20 08:16:52.136][910][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id4083lb # [ 17.310456] envoy[910]: [2026-09-20 08:16:52.136][910][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:4084lb # [ 17.311873] envoy[910]: [2026-09-20 08:16:52.136][910][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id4085lb # [ 17.319153] envoy[910]: [2026-09-20 08:16:52.136][910][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status4086lb # [ 17.367045] envoy[910]: [2026-09-20 08:16:52.307][910][info][main] [source/server/server.cc:968] runtime: {}4087lb # [ 17.375637] envoy[910]: [2026-09-20 08:16:52.315][910][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.4088lb # [ 17.377755] envoy[910]: [2026-09-20 08:16:52.317][910][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99014089lb # [ 17.380881] envoy[910]: [2026-09-20 08:16:52.321][910][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99014090lb # [ 17.384670] envoy[910]: [2026-09-20 08:16:52.324][910][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration4091lb # [ 17.386445] envoy[910]: [2026-09-20 08:16:52.326][910][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)4092lb # [ 17.388098] envoy[910]: [2026-09-20 08:16:52.326][910][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)4093worker1 # [ 17.412911] dhcpcd[692]: eth0: leased 10.0.2.15 for 86400 seconds4094worker1 # [ 17.415825] dhcpcd[692]: eth0: adding route to 10.0.2.0/244095worker1 # [ 17.418161] dhcpcd[692]: eth0: adding default route via 10.0.2.24096lb # [ 17.435392] envoy[910]: [2026-09-20 08:16:52.375][910][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)4097worker1 # [ 17.442218] systemd[1]: Starting Nix worker-protocol over gRPC...4098worker2 # [ 17.462851] systemd[1]: Starting Nix worker-protocol over gRPC...4099lb # [ 17.459758] envoy[910]: [2026-09-20 08:16:52.399][910][info][config] [source/server/configuration_impl.cc:164] loading stats configuration4100lb # [ 17.462869] envoy[910]: [2026-09-20 08:16:52.403][910][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.4101lb # [ 17.466543] envoy[910]: [2026-09-20 08:16:52.403][910][info][main] [source/server/server.cc:1077] starting main dispatch loop4102worker1 # [ 17.558972] systemd[1]: Started DHCP Client.4103worker1 # [ 17.561218] systemd[1]: Reached target Multi-User System.4104worker1 # [ 17.615130] nix-grpc-daemon[732]: ts=2026-09-20T08:16:52Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4105worker1 # [ 17.626160] nix-grpc-daemon[732]: ts=2026-09-20T08:16:52Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14106worker2 # [ 17.676740] nix-grpc-daemon[784]: ts=2026-09-20T08:16:52Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4107worker1 # [ 17.661893] nix-grpc-daemon[732]: ts=2026-09-20T08:16:52Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4108worker1 # [ 17.665608] systemd[1]: Started Nix worker-protocol over gRPC.4109worker1 # [ 17.667795] systemd[1]: Startup finished in 857ms (kernel) + 4.490s (initrd) + 12.317s (userspace) = 17.666s.4110worker2 # [ 17.688748] nix-grpc-daemon[784]: ts=2026-09-20T08:16:52Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14111worker2 # [ 17.726040] nix-grpc-daemon[784]: ts=2026-09-20T08:16:52Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4112worker2 # [ 17.729259] systemd[1]: Started Nix worker-protocol over gRPC.4113lb # [ 17.719110] envoy[910]: [2026-09-20 08:16:52.658][910][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization4114lb # [ 17.723092] envoy[910]: [2026-09-20 08:16:52.659][910][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized4115lb # [ 17.728361] envoy[910]: [2026-09-20 08:16:52.659][910][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager4116worker1 # [ 17.729517] nix-grpc-daemon[732]: ts=2026-09-20T08:16:52Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4117lb # [ 17.733257] envoy[910]: [2026-09-20 08:16:52.667][910][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers4118worker2 # [ 17.797907] nix-grpc-daemon[784]: ts=2026-09-20T08:16:52Z level=info event=scheduler_connected addr=lb:500514119worker2 # [ 17.828523] nix-grpc-daemon[784]: ts=2026-09-20T08:16:52Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4120worker1 # [ 17.930616] nix-grpc-daemon[732]: ts=2026-09-20T08:16:52Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4121worker2 # [ 18.029758] nix-grpc-daemon[784]: ts=2026-09-20T08:16:53Z level=info event=scheduler_connected addr=lb:500514122worker2 # [ 18.039821] nix-grpc-daemon[784]: ts=2026-09-20T08:16:53Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4123worker2 # [ 18.240861] nix-grpc-daemon[784]: ts=2026-09-20T08:16:53Z level=info event=scheduler_connected addr=lb:500514124worker2 # [ 18.246497] nix-grpc-daemon[784]: ts=2026-09-20T08:16:53Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4125worker1 # [ 18.331447] nix-grpc-daemon[732]: ts=2026-09-20T08:16:53Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4126worker2 # [ 18.447325] nix-grpc-daemon[784]: ts=2026-09-20T08:16:53Z level=info event=scheduler_connected addr=lb:500514127worker2 # [ 18.451221] nix-grpc-daemon[784]: ts=2026-09-20T08:16:53Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4128worker2 # [ 18.651925] nix-grpc-daemon[784]: ts=2026-09-20T08:16:53Z level=info event=scheduler_connected addr=lb:500514129worker2 # [ 18.657319] nix-grpc-daemon[784]: ts=2026-09-20T08:16:53Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4130worker2 # [ 18.859257] nix-grpc-daemon[784]: ts=2026-09-20T08:16:53Z level=info event=scheduler_connected addr=lb:500514131worker2 # [ 18.863317] nix-grpc-daemon[784]: ts=2026-09-20T08:16:53Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4132worker2 # [ 19.064316] nix-grpc-daemon[784]: ts=2026-09-20T08:16:54Z level=info event=scheduler_connected addr=lb:500514133worker2 # [ 19.070106] nix-grpc-daemon[784]: ts=2026-09-20T08:16:54Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4134worker1 # [ 19.135486] nix-grpc-daemon[732]: ts=2026-09-20T08:16:53Z level=info event=scheduler_connected addr=lb:500514135worker1 # [ 19.140572] nix-grpc-daemon[732]: ts=2026-09-20T08:16:53Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4136worker2 # [ 19.270592] nix-grpc-daemon[784]: ts=2026-09-20T08:16:54Z level=info event=scheduler_connected addr=lb:500514137worker2 # [ 19.273158] nix-grpc-daemon[784]: ts=2026-09-20T08:16:54Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4138worker1 # [ 19.341976] nix-grpc-daemon[732]: ts=2026-09-20T08:16:54Z level=info event=scheduler_connected addr=lb:500514139worker1 # [ 19.346820] nix-grpc-daemon[732]: ts=2026-09-20T08:16:54Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4140worker2 # [ 19.474276] nix-grpc-daemon[784]: ts=2026-09-20T08:16:54Z level=info event=scheduler_connected addr=lb:500514141worker2 # [ 19.479715] nix-grpc-daemon[784]: ts=2026-09-20T08:16:54Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4142worker1 # [ 19.548177] nix-grpc-daemon[732]: ts=2026-09-20T08:16:54Z level=info event=scheduler_connected addr=lb:500514143worker1 # [ 19.553272] nix-grpc-daemon[732]: ts=2026-09-20T08:16:54Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4144worker2 # [ 19.680670] nix-grpc-daemon[784]: ts=2026-09-20T08:16:54Z level=info event=scheduler_connected addr=lb:500514145worker2 # [ 19.685654] nix-grpc-daemon[784]: ts=2026-09-20T08:16:54Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4146worker1 # [ 19.753697] nix-grpc-daemon[732]: ts=2026-09-20T08:16:54Z level=info event=scheduler_connected addr=lb:500514147worker1 # [ 19.758849] nix-grpc-daemon[732]: ts=2026-09-20T08:16:54Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4148worker2 # [ 19.887495] nix-grpc-daemon[784]: ts=2026-09-20T08:16:55Z level=info event=scheduler_connected addr=lb:500514149worker2 # [ 19.893161] nix-grpc-daemon[784]: ts=2026-09-20T08:16:55Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4150worker1 # [ 19.959421] nix-grpc-daemon[732]: ts=2026-09-20T08:16:54Z level=info event=scheduler_connected addr=lb:500514151worker1 # [ 19.962108] nix-grpc-daemon[732]: ts=2026-09-20T08:16:54Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4152worker2 # [ 20.094318] nix-grpc-daemon[784]: ts=2026-09-20T08:16:55Z level=info event=scheduler_connected addr=lb:500514153worker2 # [ 20.099975] nix-grpc-daemon[784]: ts=2026-09-20T08:16:55Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4154worker1 # [ 20.163260] nix-grpc-daemon[732]: ts=2026-09-20T08:16:54Z level=info event=scheduler_connected addr=lb:500514155worker1 # [ 20.165772] nix-grpc-daemon[732]: ts=2026-09-20T08:16:54Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4156worker2 # [ 20.301148] nix-grpc-daemon[784]: ts=2026-09-20T08:16:55Z level=info event=scheduler_connected addr=lb:500514157worker2 # [ 20.307157] nix-grpc-daemon[784]: ts=2026-09-20T08:16:55Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4158worker1 # [ 20.366554] nix-grpc-daemon[732]: ts=2026-09-20T08:16:55Z level=info event=scheduler_connected addr=lb:500514159worker1 # [ 20.369262] nix-grpc-daemon[732]: ts=2026-09-20T08:16:55Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4160worker2 # [ 20.507694] nix-grpc-daemon[784]: ts=2026-09-20T08:16:55Z level=info event=scheduler_connected addr=lb:500514161worker2 # [ 20.510390] nix-grpc-daemon[784]: ts=2026-09-20T08:16:55Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4162worker1 # [ 20.570267] nix-grpc-daemon[732]: ts=2026-09-20T08:16:55Z level=info event=scheduler_connected addr=lb:500514163worker1 # [ 20.574913] nix-grpc-daemon[732]: ts=2026-09-20T08:16:55Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4164worker2 # [ 20.711173] nix-grpc-daemon[784]: ts=2026-09-20T08:16:55Z level=info event=scheduler_connected addr=lb:500514165worker2 # [ 20.713743] nix-grpc-daemon[784]: ts=2026-09-20T08:16:55Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4166worker1 # [ 20.776151] nix-grpc-daemon[732]: ts=2026-09-20T08:16:55Z level=info event=scheduler_connected addr=lb:500514167worker1 # [ 20.781160] nix-grpc-daemon[732]: ts=2026-09-20T08:16:55Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4168worker2 # [ 20.915060] nix-grpc-daemon[784]: ts=2026-09-20T08:16:56Z level=info event=scheduler_connected addr=lb:500514169worker2 # [ 20.917851] nix-grpc-daemon[784]: ts=2026-09-20T08:16:56Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4170worker1 # [ 20.982156] nix-grpc-daemon[732]: ts=2026-09-20T08:16:55Z level=info event=scheduler_connected addr=lb:500514171worker1 # [ 20.987814] nix-grpc-daemon[732]: ts=2026-09-20T08:16:55Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4172worker2 # [ 21.118835] nix-grpc-daemon[784]: ts=2026-09-20T08:16:56Z level=info event=scheduler_connected addr=lb:500514173worker2 # [ 21.123802] nix-grpc-daemon[784]: ts=2026-09-20T08:16:56Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4174worker1 # [ 21.190498] nix-grpc-daemon[732]: ts=2026-09-20T08:16:55Z level=info event=scheduler_connected addr=lb:500514175worker1 # [ 21.194170] nix-grpc-daemon[732]: ts=2026-09-20T08:16:55Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4176worker2 # [ 21.324649] nix-grpc-daemon[784]: ts=2026-09-20T08:16:56Z level=info event=scheduler_connected addr=lb:500514177worker2 # [ 21.327564] nix-grpc-daemon[784]: ts=2026-09-20T08:16:56Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4178worker1 # [ 21.394466] nix-grpc-daemon[732]: ts=2026-09-20T08:16:56Z level=info event=scheduler_connected addr=lb:500514179worker1 # [ 21.397080] nix-grpc-daemon[732]: ts=2026-09-20T08:16:56Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4180worker2 # [ 21.528627] nix-grpc-daemon[784]: ts=2026-09-20T08:16:56Z level=info event=scheduler_connected addr=lb:500514181worker2 # [ 21.533684] nix-grpc-daemon[784]: ts=2026-09-20T08:16:56Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4182worker1 # [ 21.597763] nix-grpc-daemon[732]: ts=2026-09-20T08:16:56Z level=info event=scheduler_connected addr=lb:500514183worker1 # [ 21.600517] nix-grpc-daemon[732]: ts=2026-09-20T08:16:56Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4184worker2 # [ 21.734486] nix-grpc-daemon[784]: ts=2026-09-20T08:16:56Z level=info event=scheduler_connected addr=lb:500514185worker2 # [ 21.739305] nix-grpc-daemon[784]: ts=2026-09-20T08:16:56Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4186worker1 # [ 21.801194] nix-grpc-daemon[732]: ts=2026-09-20T08:16:56Z level=info event=scheduler_connected addr=lb:500514187worker1 # [ 21.803635] nix-grpc-daemon[732]: ts=2026-09-20T08:16:56Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4188worker2 # [ 21.941268] nix-grpc-daemon[784]: ts=2026-09-20T08:16:57Z level=info event=scheduler_connected addr=lb:500514189worker2 # [ 21.945766] nix-grpc-daemon[784]: ts=2026-09-20T08:16:57Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4190worker1 # [ 22.004467] nix-grpc-daemon[732]: ts=2026-09-20T08:16:56Z level=info event=scheduler_connected addr=lb:500514191worker1 # [ 22.007134] nix-grpc-daemon[732]: ts=2026-09-20T08:16:56Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4192worker2 # [ 22.146841] nix-grpc-daemon[784]: ts=2026-09-20T08:16:57Z level=info event=scheduler_connected addr=lb:500514193worker2 # [ 22.151541] nix-grpc-daemon[784]: ts=2026-09-20T08:16:57Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4194worker1 # [ 22.208248] nix-grpc-daemon[732]: ts=2026-09-20T08:16:56Z level=info event=scheduler_connected addr=lb:500514195worker1 # [ 22.210566] nix-grpc-daemon[732]: ts=2026-09-20T08:16:57Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4196worker2 # [ 22.352602] nix-grpc-daemon[784]: ts=2026-09-20T08:16:57Z level=info event=scheduler_connected addr=lb:500514197worker2 # [ 22.357666] nix-grpc-daemon[784]: ts=2026-09-20T08:16:57Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4198worker1 # [ 22.411715] nix-grpc-daemon[732]: ts=2026-09-20T08:16:57Z level=info event=scheduler_connected addr=lb:500514199worker1 # [ 22.416433] nix-grpc-daemon[732]: ts=2026-09-20T08:16:57Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4200worker2 # [ 22.558811] nix-grpc-daemon[784]: ts=2026-09-20T08:16:57Z level=info event=scheduler_connected addr=lb:500514201worker2 # [ 22.563581] nix-grpc-daemon[784]: ts=2026-09-20T08:16:57Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4202worker1 # [ 22.617105] nix-grpc-daemon[732]: ts=2026-09-20T08:16:57Z level=info event=scheduler_connected addr=lb:500514203worker1 # [ 22.619497] nix-grpc-daemon[732]: ts=2026-09-20T08:16:57Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4204worker2 # [ 22.765241] nix-grpc-daemon[784]: ts=2026-09-20T08:16:57Z level=info event=scheduler_connected addr=lb:500514205worker2 # [ 22.770197] nix-grpc-daemon[784]: ts=2026-09-20T08:16:57Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4206worker1 # [ 22.820740] nix-grpc-daemon[732]: ts=2026-09-20T08:16:57Z level=info event=scheduler_connected addr=lb:500514207worker1 # [ 22.826221] nix-grpc-daemon[732]: ts=2026-09-20T08:16:57Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4208worker2 # [ 22.971774] nix-grpc-daemon[784]: ts=2026-09-20T08:16:58Z level=info event=scheduler_connected addr=lb:500514209worker2 # [ 22.976657] nix-grpc-daemon[784]: ts=2026-09-20T08:16:58Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4210worker1 # [ 23.027245] nix-grpc-daemon[732]: ts=2026-09-20T08:16:57Z level=info event=scheduler_connected addr=lb:500514211worker1 # [ 23.032127] nix-grpc-daemon[732]: ts=2026-09-20T08:16:57Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4212worker2 # [ 23.177280] nix-grpc-daemon[784]: ts=2026-09-20T08:16:58Z level=info event=scheduler_connected addr=lb:500514213worker2 # [ 23.179829] nix-grpc-daemon[784]: ts=2026-09-20T08:16:58Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4214worker1 # [ 23.233860] nix-grpc-daemon[732]: ts=2026-09-20T08:16:58Z level=info event=scheduler_connected addr=lb:500514215worker1 # [ 23.238449] nix-grpc-daemon[732]: ts=2026-09-20T08:16:58Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4216worker2 # [ 23.380864] nix-grpc-daemon[784]: ts=2026-09-20T08:16:58Z level=info event=scheduler_connected addr=lb:500514217worker2 # [ 23.385766] nix-grpc-daemon[784]: ts=2026-09-20T08:16:58Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4218worker1 # [ 23.439103] nix-grpc-daemon[732]: ts=2026-09-20T08:16:58Z level=info event=scheduler_connected addr=lb:500514219worker1 # [ 23.443734] nix-grpc-daemon[732]: ts=2026-09-20T08:16:58Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4220worker2 # [ 23.586431] nix-grpc-daemon[784]: ts=2026-09-20T08:16:58Z level=info event=scheduler_connected addr=lb:500514221worker2 # [ 23.588990] nix-grpc-daemon[784]: ts=2026-09-20T08:16:58Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4222worker1 # [ 23.644503] nix-grpc-daemon[732]: ts=2026-09-20T08:16:58Z level=info event=scheduler_connected addr=lb:500514223worker1 # [ 23.646987] nix-grpc-daemon[732]: ts=2026-09-20T08:16:58Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4224worker2 # [ 23.789661] nix-grpc-daemon[784]: ts=2026-09-20T08:16:58Z level=info event=scheduler_connected addr=lb:500514225worker2 # [ 23.791921] nix-grpc-daemon[784]: ts=2026-09-20T08:16:58Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4226worker1 # [ 23.848066] nix-grpc-daemon[732]: ts=2026-09-20T08:16:58Z level=info event=scheduler_connected addr=lb:500514227worker1 # [ 23.852830] nix-grpc-daemon[732]: ts=2026-09-20T08:16:58Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4228worker2 # [ 23.995479] nix-grpc-daemon[784]: ts=2026-09-20T08:16:59Z level=info event=scheduler_connected addr=lb:500514229worker2 # [ 23.998872] nix-grpc-daemon[784]: ts=2026-09-20T08:16:59Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4230worker1 # [ 24.053455] nix-grpc-daemon[732]: ts=2026-09-20T08:16:58Z level=info event=scheduler_connected addr=lb:500514231worker1 # [ 24.056074] nix-grpc-daemon[732]: ts=2026-09-20T08:16:58Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4232worker2 # [ 24.196616] nix-grpc-daemon[784]: ts=2026-09-20T08:16:59Z level=info event=scheduler_connected addr=lb:500514233worker2 # [ 24.201690] nix-grpc-daemon[784]: ts=2026-09-20T08:16:59Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4234worker1 # [ 24.258544] nix-grpc-daemon[732]: ts=2026-09-20T08:16:59Z level=info event=scheduler_connected addr=lb:500514235worker1 # [ 24.262206] nix-grpc-daemon[732]: ts=2026-09-20T08:16:59Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4236worker2 # [ 24.402509] nix-grpc-daemon[784]: ts=2026-09-20T08:16:59Z level=info event=scheduler_connected addr=lb:500514237worker2 # [ 24.406444] nix-grpc-daemon[784]: ts=2026-09-20T08:16:59Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4238worker1 # [ 24.463067] nix-grpc-daemon[732]: ts=2026-09-20T08:16:59Z level=info event=scheduler_connected addr=lb:500514239worker1 # [ 24.467682] nix-grpc-daemon[732]: ts=2026-09-20T08:16:59Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4240worker2 # [ 24.607669] nix-grpc-daemon[784]: ts=2026-09-20T08:16:59Z level=info event=scheduler_connected addr=lb:500514241worker2 # [ 24.613120] nix-grpc-daemon[784]: ts=2026-09-20T08:16:59Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4242worker1 # [ 24.668501] nix-grpc-daemon[732]: ts=2026-09-20T08:16:59Z level=info event=scheduler_connected addr=lb:500514243worker1 # [ 24.673332] nix-grpc-daemon[732]: ts=2026-09-20T08:16:59Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4244worker2 # [ 24.813749] nix-grpc-daemon[784]: ts=2026-09-20T08:16:59Z level=info event=scheduler_connected addr=lb:500514245worker2 # [ 24.818656] nix-grpc-daemon[784]: ts=2026-09-20T08:16:59Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4246worker1 # [ 24.874504] nix-grpc-daemon[732]: ts=2026-09-20T08:16:59Z level=info event=scheduler_connected addr=lb:500514247worker1 # [ 24.879414] nix-grpc-daemon[732]: ts=2026-09-20T08:16:59Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4248worker2 # [ 25.019704] nix-grpc-daemon[784]: ts=2026-09-20T08:17:00Z level=info event=scheduler_connected addr=lb:500514249worker2 # [ 25.021953] nix-grpc-daemon[784]: ts=2026-09-20T08:17:00Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4250worker1 # [ 25.080548] nix-grpc-daemon[732]: ts=2026-09-20T08:16:59Z level=info event=scheduler_connected addr=lb:500514251worker1 # [ 25.085309] nix-grpc-daemon[732]: ts=2026-09-20T08:16:59Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4252worker2 # [ 25.222691] nix-grpc-daemon[784]: ts=2026-09-20T08:17:00Z level=info event=scheduler_connected addr=lb:500514253worker2 # [ 25.225183] nix-grpc-daemon[784]: ts=2026-09-20T08:17:00Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4254worker1 # [ 25.288281] nix-grpc-daemon[732]: ts=2026-09-20T08:17:00Z level=info event=scheduler_connected addr=lb:500514255worker1 # [ 25.292566] nix-grpc-daemon[732]: ts=2026-09-20T08:17:00Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4256worker2 # [ 25.426328] nix-grpc-daemon[784]: ts=2026-09-20T08:17:00Z level=info event=scheduler_connected addr=lb:500514257worker2 # [ 25.431238] nix-grpc-daemon[784]: ts=2026-09-20T08:17:00Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4258worker1 # [ 25.492932] nix-grpc-daemon[732]: ts=2026-09-20T08:17:00Z level=info event=scheduler_connected addr=lb:500514259worker1 # [ 25.495529] nix-grpc-daemon[732]: ts=2026-09-20T08:17:00Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4260worker2 # [ 25.632375] nix-grpc-daemon[784]: ts=2026-09-20T08:17:00Z level=info event=scheduler_connected addr=lb:500514261worker2 # [ 25.637438] nix-grpc-daemon[784]: ts=2026-09-20T08:17:00Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4262worker1 # [ 25.696744] nix-grpc-daemon[732]: ts=2026-09-20T08:17:00Z level=info event=scheduler_connected addr=lb:500514263worker1 # [ 25.701706] nix-grpc-daemon[732]: ts=2026-09-20T08:17:00Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4264worker2 # [ 25.838538] nix-grpc-daemon[784]: ts=2026-09-20T08:17:00Z level=info event=scheduler_connected addr=lb:500514265worker2 # [ 25.843784] nix-grpc-daemon[784]: ts=2026-09-20T08:17:00Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4266worker1 # [ 25.902271] nix-grpc-daemon[732]: ts=2026-09-20T08:17:00Z level=info event=scheduler_connected addr=lb:500514267worker1 # [ 25.904820] nix-grpc-daemon[732]: ts=2026-09-20T08:17:00Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4268worker2 # [ 26.044579] nix-grpc-daemon[784]: ts=2026-09-20T08:17:01Z level=info event=scheduler_connected addr=lb:500514269worker2 # [ 26.047261] nix-grpc-daemon[784]: ts=2026-09-20T08:17:01Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4270worker1 # [ 26.106353] nix-grpc-daemon[732]: ts=2026-09-20T08:17:00Z level=info event=scheduler_connected addr=lb:500514271worker1 # [ 26.112627] nix-grpc-daemon[732]: ts=2026-09-20T08:17:00Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4272worker2 # [ 26.248547] nix-grpc-daemon[784]: ts=2026-09-20T08:17:01Z level=info event=scheduler_connected addr=lb:500514273worker2 # [ 26.253524] nix-grpc-daemon[784]: ts=2026-09-20T08:17:01Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4274worker1 # [ 26.314296] nix-grpc-daemon[732]: ts=2026-09-20T08:17:01Z level=info event=scheduler_connected addr=lb:500514275worker1 # [ 26.319647] nix-grpc-daemon[732]: ts=2026-09-20T08:17:01Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4276worker2 # [ 26.454550] nix-grpc-daemon[784]: ts=2026-09-20T08:17:01Z level=info event=scheduler_connected addr=lb:500514277worker2 # [ 26.459819] nix-grpc-daemon[784]: ts=2026-09-20T08:17:01Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4278worker1 # [ 26.519896] nix-grpc-daemon[732]: ts=2026-09-20T08:17:01Z level=info event=scheduler_connected addr=lb:500514279worker1 # [ 26.524744] nix-grpc-daemon[732]: ts=2026-09-20T08:17:01Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4280worker2 # [ 26.660899] nix-grpc-daemon[784]: ts=2026-09-20T08:17:01Z level=info event=scheduler_connected addr=lb:500514281worker2 # [ 26.665797] nix-grpc-daemon[784]: ts=2026-09-20T08:17:01Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4282worker1 # [ 26.725693] nix-grpc-daemon[732]: ts=2026-09-20T08:17:01Z level=info event=scheduler_connected addr=lb:500514283worker1 # [ 26.732273] nix-grpc-daemon[732]: ts=2026-09-20T08:17:01Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4284worker2 # [ 26.866676] nix-grpc-daemon[784]: ts=2026-09-20T08:17:02Z level=info event=scheduler_connected addr=lb:500514285worker2 # [ 26.871672] nix-grpc-daemon[784]: ts=2026-09-20T08:17:02Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4286worker1 # [ 26.933162] nix-grpc-daemon[732]: ts=2026-09-20T08:17:01Z level=info event=scheduler_connected addr=lb:500514287worker1 # [ 26.938106] nix-grpc-daemon[732]: ts=2026-09-20T08:17:01Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4288worker2 # [ 27.073846] nix-grpc-daemon[784]: ts=2026-09-20T08:17:02Z level=info event=scheduler_connected addr=lb:500514289worker2 # [ 27.078686] nix-grpc-daemon[784]: ts=2026-09-20T08:17:02Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4290worker1 # [ 27.139164] nix-grpc-daemon[732]: ts=2026-09-20T08:17:01Z level=info event=scheduler_connected addr=lb:500514291worker1 # [ 27.143995] nix-grpc-daemon[732]: ts=2026-09-20T08:17:01Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4292worker2 # [ 27.279112] nix-grpc-daemon[784]: ts=2026-09-20T08:17:02Z level=info event=scheduler_connected addr=lb:500514293worker2 # [ 27.282137] nix-grpc-daemon[784]: ts=2026-09-20T08:17:02Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4294worker1 # [ 27.346475] nix-grpc-daemon[732]: ts=2026-09-20T08:17:02Z level=info event=scheduler_connected addr=lb:500514295worker1 # [ 27.350248] nix-grpc-daemon[732]: ts=2026-09-20T08:17:02Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4296worker2 # [ 27.482678] nix-grpc-daemon[784]: ts=2026-09-20T08:17:02Z level=info event=scheduler_connected addr=lb:500514297worker2 # [ 27.485630] nix-grpc-daemon[784]: ts=2026-09-20T08:17:02Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4298worker1 # [ 27.550879] nix-grpc-daemon[732]: ts=2026-09-20T08:17:02Z level=info event=scheduler_connected addr=lb:500514299worker1 # [ 27.555852] nix-grpc-daemon[732]: ts=2026-09-20T08:17:02Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4300worker2 # [ 27.686680] nix-grpc-daemon[784]: ts=2026-09-20T08:17:02Z level=info event=scheduler_connected addr=lb:500514301worker2 # [ 27.692558] nix-grpc-daemon[784]: ts=2026-09-20T08:17:02Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4302worker1 # [ 27.756749] nix-grpc-daemon[732]: ts=2026-09-20T08:17:02Z level=info event=scheduler_connected addr=lb:500514303worker1 # [ 27.762533] nix-grpc-daemon[732]: ts=2026-09-20T08:17:02Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4304worker2 # [ 27.893591] nix-grpc-daemon[784]: ts=2026-09-20T08:17:03Z level=info event=scheduler_connected addr=lb:500514305worker2 # [ 27.898796] nix-grpc-daemon[784]: ts=2026-09-20T08:17:03Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4306worker1 # [ 27.963648] nix-grpc-daemon[732]: ts=2026-09-20T08:17:02Z level=info event=scheduler_connected addr=lb:500514307worker1 # [ 27.968681] nix-grpc-daemon[732]: ts=2026-09-20T08:17:02Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4308worker2 # [ 28.099784] nix-grpc-daemon[784]: ts=2026-09-20T08:17:03Z level=info event=scheduler_connected addr=lb:500514309worker2 # [ 28.102116] nix-grpc-daemon[784]: ts=2026-09-20T08:17:03Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4310worker1 # [ 28.169867] nix-grpc-daemon[732]: ts=2026-09-20T08:17:02Z level=info event=scheduler_connected addr=lb:500514311worker1 # [ 28.174739] nix-grpc-daemon[732]: ts=2026-09-20T08:17:02Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4312worker2 # [ 28.303359] nix-grpc-daemon[784]: ts=2026-09-20T08:17:03Z level=info event=scheduler_connected addr=lb:500514313worker2 # [ 28.307980] nix-grpc-daemon[784]: ts=2026-09-20T08:17:03Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4314worker1 # [ 28.376660] nix-grpc-daemon[732]: ts=2026-09-20T08:17:03Z level=info event=scheduler_connected addr=lb:500514315worker1 # [ 28.381240] nix-grpc-daemon[732]: ts=2026-09-20T08:17:03Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4316worker2 # [ 28.508563] nix-grpc-daemon[784]: ts=2026-09-20T08:17:03Z level=info event=scheduler_connected addr=lb:500514317worker2 # [ 28.513669] nix-grpc-daemon[784]: ts=2026-09-20T08:17:03Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4318worker1 # [ 28.582304] nix-grpc-daemon[732]: ts=2026-09-20T08:17:03Z level=info event=scheduler_connected addr=lb:500514319worker1 # [ 28.587223] nix-grpc-daemon[732]: ts=2026-09-20T08:17:03Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4320worker2 # [ 28.714716] nix-grpc-daemon[784]: ts=2026-09-20T08:17:03Z level=info event=scheduler_connected addr=lb:500514321worker2 # [ 28.719743] nix-grpc-daemon[784]: ts=2026-09-20T08:17:03Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4322worker1 # [ 28.788157] nix-grpc-daemon[732]: ts=2026-09-20T08:17:03Z level=info event=scheduler_connected addr=lb:500514323worker1 # [ 28.793341] nix-grpc-daemon[732]: ts=2026-09-20T08:17:03Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4324worker2 # [ 28.920746] nix-grpc-daemon[784]: ts=2026-09-20T08:17:04Z level=info event=scheduler_connected addr=lb:500514325lb # [ 28.904345] rustfs-bucket-start[1030]: mb s3://farm4326lb # [ 28.908390] systemd[1]: Finished rustfs-bucket.service.4327worker2 # [ 28.925459] nix-grpc-daemon[784]: ts=2026-09-20T08:17:04Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4328lb # [ 28.914323] systemd[1]: Starting niks3 server...4329worker1 # [ 28.994691] nix-grpc-daemon[732]: ts=2026-09-20T08:17:03Z level=info event=scheduler_connected addr=lb:500514330worker1 # [ 28.999155] nix-grpc-daemon[732]: ts=2026-09-20T08:17:03Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4331worker2 # [ 29.127541] nix-grpc-daemon[784]: ts=2026-09-20T08:17:04Z level=info event=scheduler_connected addr=lb:500514332lb # [ 29.110594] postgres[1045]: [1045] ERROR: relation "goose_db_version" does not exist at character 364333lb # [ 29.112337] postgres[1045]: [1045] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4334worker2 # [ 29.131819] nix-grpc-daemon[784]: ts=2026-09-20T08:17:04Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4335lb # [ 29.149529] niks3-server[1039]: 2026/09/20 08:17:04 OK 20241026095416_initial_model.sql (19.54ms)4336lb # [ 29.152702] niks3-server[1039]: 2026/09/20 08:17:04 OK 20251210153512_drop_unused_gin_index.sql (3.22ms)4337lb # [ 29.157658] niks3-server[1039]: 2026/09/20 08:17:04 OK 20251218171726_add_pins.sql (4.8ms)4338lb # [ 29.163703] niks3-server[1039]: 2026/09/20 08:17:04 OK 20260628120000_add_object_size_and_stats.sql (6.08ms)4339lb # [ 29.168819] niks3-server[1039]: 2026/09/20 08:17:04 OK 20260905000000_add_claims.sql (5.1ms)4340lb # [ 29.174211] niks3-server[1039]: 2026/09/20 08:17:04 OK 20260920000000_drop_claims.sql (5.31ms)4341lb # [ 29.175367] niks3-server[1039]: 2026/09/20 08:17:04 goose: successfully migrated database to version: 202609200000004342lb # [ 29.181333] niks3-server[1039]: 2026/09/20 08:17:04 OK 1_commit_pending_closure.sql (7.09ms)4343lb # [ 29.183699] niks3-server[1039]: 2026/09/20 08:17:04 OK 2_object_stats_trigger.sql (2.38ms)4344lb # [ 29.186151] niks3-server[1039]: 2026/09/20 08:17:04 goose: up to current file version: 24345lb # [ 29.192102] niks3-server[1039]: 2026/09/20 08:17:04 INFO Loaded signing key name=farm-test-1 path=/nix/store/c745bsy529madd2s6hz4z5is2436ni04-key4346worker1 # [ 29.200229] nix-grpc-daemon[732]: ts=2026-09-20T08:17:03Z level=info event=scheduler_connected addr=lb:500514347worker1 # [ 29.204831] nix-grpc-daemon[732]: ts=2026-09-20T08:17:03Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4348lb # [ 29.271275] niks3-server[1039]: 2026/09/20 08:17:04 INFO Read proxy enabled — serving cache objects from S34349lb # [ 29.273356] niks3-server[1039]: 2026/09/20 08:17:04 INFO Using socket-activated listener address=0.0.0.0:57514350lb # [ 29.280038] niks3-server[1039]: 2026/09/20 08:17:04 INFO systemd watchdog enabled interval=15s4351lb # [ 29.281225] systemd[1]: Started niks3 server.4352lb # [ 29.281824] systemd[1]: Reached target Multi-User System.4353lb # [ 29.284444] systemd[1]: Startup finished in 873ms (kernel) + 4.121s (initrd) + 24.289s (userspace) = 29.284s.4354lb # [ 29.287278] niks3-server[1039]: 2026/09/20 08:17:04 INFO Starting HTTP server address=0.0.0.0:57514355lb # [ 29.295137] niks3-server[1039]: 2026/09/20 08:17:04 INFO lead: acquired remote=192.168.1.4:554544356worker2 # [ 29.320596] nix-grpc-daemon[784]: ts=2026-09-20T08:17:04Z level=info event=scheduler_take_over4357worker2 # [ 29.332735] nix-grpc-daemon[784]: ts=2026-09-20T08:17:04Z level=info event=scheduler_connected addr=lb:500514358worker2 # [ 29.338248] nix-grpc-daemon[784]: ts=2026-09-20T08:17:04Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04359lb: (finished: waiting for unit niks3.service, in 17.68 seconds)4360lb: waiting for TCP port 5751 on localhost4361worker1 # [ 29.408182] nix-grpc-daemon[732]: ts=2026-09-20T08:17:04Z level=info event=scheduler_connected addr=lb:500514362lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!4363worker1 # [ 29.411268] nix-grpc-daemon[732]: ts=2026-09-20T08:17:04Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4364lb: (finished: waiting for TCP port 5751 on localhost, in 0.06 seconds)4365lb: waiting for unit envoy.service4366lb: (finished: waiting for unit envoy.service, in 0.05 seconds)4367lb: waiting for TCP port 50051 on localhost4368lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!4369lb # [ 29.499940] envoy[910]: conn peer=[::1]:59200 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=24370lb: (finished: waiting for TCP port 50051 on localhost, in 0.04 seconds)4371worker1 # [ 29.608407] nix-grpc-daemon[732]: ts=2026-09-20T08:17:04Z level=info event=scheduler_connected addr=lb:500514372worker2 # [ 29.630516] nix-grpc-daemon[784]: ts=2026-09-20T08:17:04Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04373lb: waiting until exactly no endpoints of x86_64-linux are unhealthy4374??? Warning (UserWarning): retry(): The 'timeout_seconds' argument is deprecated. Use 'timeout' instead.4375 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 1284376lb: must succeed: curl -sf localhost:9901/clusters4377lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4378lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.08 seconds)4379subtest: one node schedules and both builders hold a WorkerSession on it4380lb: must succeed: curl -sf localhost:9901/clusters4381lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4382lb: must succeed: curl -sf localhost:9901/clusters4383lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4384worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true4385worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.09 seconds)4386(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 1.19 seconds)4387subtest: balancer auth: client cert, bearer token, nothing, unknown CN4388worker1 # [ 31.237949] nix-grpc-daemon[732]: ts=2026-09-20T08:17:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:555564389worker1 # [ 31.250567] systemd[1]: Started Nix Daemon.4390worker1 # [ 31.454571] nix-daemon[841]: accepted connection from pid 732, user nix-grpc-daemon (trusted)4391worker1 # [ 31.506402] nix-grpc-daemon[732]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 8 ms (attempt 1/5)4392worker1 # [ 31.516752] nix-grpc-daemon[732]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 60 ms (attempt 2/5)4393worker1 # [ 31.579520] nix-grpc-daemon[732]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 324 ms (attempt 3/5)4394worker1 # [ 31.909455] nix-grpc-daemon[732]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 70 ms (attempt 4/5)4395worker1 # [ 31.984492] nix-grpc-daemon[732]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4396worker2 # [ 32.040583] nix-grpc-daemon[784]: ts=2026-09-20T08:17:07Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:531324397worker2 # [ 32.052772] systemd[1]: Started Nix Daemon.4398worker2 # [ 32.263082] nix-daemon[847]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4399worker2 # [ 32.288655] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 56 ms (attempt 1/5)4400worker2 # [ 32.347237] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 17 ms (attempt 2/5)4401worker2 # [ 32.367458] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 47 ms (attempt 3/5)4402worker2 # [ 32.414544] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 427 ms (attempt 4/5)4403worker2 # [ 32.850187] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4404worker2 # [ 32.870568] nix-grpc-daemon[784]: ts=2026-09-20T08:17:08Z level=debug event=path_info_miss path=rwb9w0ssg6953fv1kadwpj7gi6289g64-nixos-system-client-test error="error: path '/nix/store/rwb9w0ssg6953fv1kadwpj7gi6289g64-nixos-system-client-test' is required, but there is no substituter that can build it"4405worker1 # [ 32.858645] nix-grpc-daemon[732]: ts=2026-09-20T08:17:07Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:555564406worker1 # [ 32.874913] nix-grpc-daemon[732]: ts=2026-09-20T08:17:07Z level=debug event=path_info_miss path=rwb9w0ssg6953fv1kadwpj7gi6289g64-nixos-system-client-test error="error: path '/nix/store/rwb9w0ssg6953fv1kadwpj7gi6289g64-nixos-system-client-test' is required, but there is no substituter that can build it"4407client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4408client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.07 seconds)4409client: must succeed: install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4410client: (finished: must succeed: install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds)4411worker2 # [ 33.244346] nix-grpc-daemon[784]: ts=2026-09-20T08:17:08Z level=info event=oidc_jwks provider=mock keys=14412worker2 # [ 33.247749] nix-grpc-daemon[784]: ts=2026-09-20T08:17:08Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:531324413worker1 # [ 33.257850] nix-grpc-daemon[732]: ts=2026-09-20T08:17:08Z level=info event=oidc_jwks provider=mock keys=14414worker1 # [ 33.261469] nix-grpc-daemon[732]: ts=2026-09-20T08:17:08Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:555564415worker1 # [ 33.270973] nix-grpc-daemon[732]: ts=2026-09-20T08:17:08Z level=debug event=path_info_miss path=rwb9w0ssg6953fv1kadwpj7gi6289g64-nixos-system-client-test error="error: path '/nix/store/rwb9w0ssg6953fv1kadwpj7gi6289g64-nixos-system-client-test' is required, but there is no substituter that can build it"4416worker1 # [ 33.280278] nix-grpc-daemon[732]: ts=2026-09-20T08:17:08Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:555564417worker1 # [ 33.289380] nix-grpc-daemon[732]: ts=2026-09-20T08:17:08Z level=debug event=path_info_miss path=rwb9w0ssg6953fv1kadwpj7gi6289g64-nixos-system-client-test error="error: path '/nix/store/rwb9w0ssg6953fv1kadwpj7gi6289g64-nixos-system-client-test' is required, but there is no substituter that can build it"4418client: must succeed: rm -r /var/lib/nix-grpc-store4419client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.02 seconds)4420worker2 # [ 33.519899] nix-grpc-daemon[784]: ts=2026-09-20T08:17:08Z level=info event=denied method=QueryMissing cn=- role=none4421worker1 # [ 33.683190] nix-grpc-daemon[732]: ts=2026-09-20T08:17:08Z level=info event=denied method=QueryMissing cn=stranger role=none4422(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 2.80 seconds)4423subtest: DAG build is scheduled across workers and lands in the cache4424client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag t1 >&24425client # don't know how to build these paths:4426worker1 # [ 33.894312] nix-grpc-daemon[732]: ts=2026-09-20T08:17:08Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:555564427client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4428worker2 # [ 33.939362] nix-grpc-daemon[784]: ts=2026-09-20T08:17:09Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04429worker2 # [ 33.944683] nix-grpc-daemon[784]: ts=2026-09-20T08:17:09Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.4:50051 clients=14430worker2 # [ 33.993969] nix-grpc-daemon[784]: ts=2026-09-20T08:17:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531324431worker2 # [ 34.011568] nix-grpc-daemon[784]: ts=2026-09-20T08:17:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:531324432client # copying 1 paths to 'lb:50051'...4433worker2 # [ 34.022727] nix-grpc-daemon[784]: ts=2026-09-20T08:17:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:531324434client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4435worker2 # [ 34.045519] nix-daemon[847]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4436worker2 # [ 34.060884] nix-grpc-daemon[784]: ts=2026-09-20T08:17:09Z level=info event=niks3_push_started pid=8594437worker2 # [ 34.466751] nix-daemon[847]: accepted connection from pid 870, user nix-grpc-daemon (trusted)4438worker2 # [ 34.481549] nix-daemon[847]: reaped child process 874, status = succeeded4439worker2 # [ 34.633666] nix-daemon[847]: accepted connection from pid 876, user nix-grpc-daemon (trusted)4440worker2 # [ 34.645208] nix-daemon[847]: reaped child process 879, status = succeeded4441lb # [ 34.641642] niks3-server[1039]: 2026/09/20 08:17:09 INFO Received uploads request method=POST path=/api/pending_closures4442worker2 # [ 34.690086] nix-grpc-daemon[859]: time=2026-09-20T08:17:09.824Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4443worker2 # [ 34.693059] nix-grpc-daemon[859]: time=2026-09-20T08:17:09.827Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4444lb # [ 34.711155] niks3-server[1039]: 2026/09/20 08:17:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4445lb # [ 34.734666] niks3-server[1039]: 2026/09/20 08:17:09 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4446lb # [ 34.742187] niks3-server[1039]: 2026/09/20 08:17:09 INFO Signed narinfos id=1 count=14447worker2 # [ 34.763406] nix-grpc-daemon[859]: time=2026-09-20T08:17:09.898Z level=INFO msg="Uploading 1 narinfos"4448lb # [ 34.753246] niks3-server[1039]: 2026/09/20 08:17:09 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4449lb # [ 34.759266] niks3-server[1039]: 2026/09/20 08:17:09 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4450lb # [ 34.768895] niks3-server[1039]: 2026/09/20 08:17:09 INFO Completed upload id=14451worker2 # [ 34.788472] nix-grpc-daemon[859]: time=2026-09-20T08:17:09.923Z level=INFO msg="Upload complete. (479ms)"4452lb # [ 34.776157] niks3-server[1039]: 2026/09/20 08:17:09 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo4453worker2 # [ 34.793929] nix-grpc-daemon[784]: ts=2026-09-20T08:17:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53132 duration_s=0 paths=1 nar_bytes_in=5044454worker2 # [ 34.802531] nix-grpc-daemon[784]: ts=2026-09-20T08:17:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531324455worker2 # [ 34.813279] nix-daemon[847]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4456worker2 # [ 34.823104] nix-daemon[847]: reaped child process 857, status = succeeded4457client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4458worker2 # [ 34.831762] nix-daemon[847]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4459client # farm-a> worker2: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4460client # farm-a> LOG-farm-a-t14461worker2 # [ 35.444974] nix-daemon[847]: reaped child process 883, status = succeeded4462worker2 # [ 35.566641] nix-daemon[847]: accepted connection from pid 898, user nix-grpc-daemon (trusted)4463worker2 # [ 35.579125] nix-daemon[847]: reaped child process 901, status = succeeded4464lb # [ 35.576682] niks3-server[1039]: 2026/09/20 08:17:10 INFO Received uploads request method=POST path=/api/pending_closures4465worker2 # [ 35.604206] nix-grpc-daemon[859]: time=2026-09-20T08:17:10.739Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4466worker2 # [ 35.605719] nix-grpc-daemon[859]: time=2026-09-20T08:17:10.741Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4467lb # [ 35.624360] niks3-server[1039]: 2026/09/20 08:17:10 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4468lb # [ 35.630198] niks3-server[1039]: 2026/09/20 08:17:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4469lb # [ 35.633174] niks3-server[1039]: 2026/09/20 08:17:10 INFO Signed narinfos id=2 count=14470worker2 # [ 35.653525] nix-grpc-daemon[859]: time=2026-09-20T08:17:10.788Z level=INFO msg="Uploading 1 narinfos"4471lb # [ 35.638110] niks3-server[1039]: 2026/09/20 08:17:10 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4472lb # [ 35.642656] niks3-server[1039]: 2026/09/20 08:17:10 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4473lb # [ 35.649860] niks3-server[1039]: 2026/09/20 08:17:10 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4474lb # [ 35.653721] niks3-server[1039]: 2026/09/20 08:17:10 INFO Completed upload id=24475worker2 # [ 35.673036] nix-grpc-daemon[859]: time=2026-09-20T08:17:10.808Z level=INFO msg="Upload complete. (233ms)"4476lb # [ 35.655779] niks3-server[1039]: 2026/09/20 08:17:10 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4477worker2 # [ 35.675521] nix-grpc-daemon[784]: ts=2026-09-20T08:17:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53132 duration_s=0 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14478worker2 # [ 35.685236] nix-daemon[847]: reaped child process 881, status = succeeded4479worker2 # [ 35.690709] nix-grpc-daemon[784]: ts=2026-09-20T08:17:10Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04480worker2 # [ 35.692787] nix-grpc-daemon[784]: ts=2026-09-20T08:17:10Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.4:50051 clients=14481worker2 # [ 35.734789] nix-grpc-daemon[784]: ts=2026-09-20T08:17:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531324482worker2 # [ 35.742343] nix-grpc-daemon[784]: ts=2026-09-20T08:17:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:531324483worker2 # [ 35.751047] nix-grpc-daemon[859]: time=2026-09-20T08:17:10.886Z level=INFO msg="All 1 paths already cached"4484client # copying 1 paths to 'lb:50051'...4485worker2 # [ 35.756076] nix-grpc-daemon[784]: ts=2026-09-20T08:17:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:531324486client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4487worker2 # [ 35.774128] nix-daemon[847]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4488worker2 # [ 35.934994] nix-daemon[847]: accepted connection from pid 909, user nix-grpc-daemon (trusted)4489worker2 # [ 35.947617] nix-daemon[847]: reaped child process 912, status = succeeded4490worker2 # [ 36.095535] nix-daemon[847]: accepted connection from pid 914, user nix-grpc-daemon (trusted)4491worker2 # [ 36.107061] nix-daemon[847]: reaped child process 917, status = succeeded4492lb # [ 36.098303] niks3-server[1039]: 2026/09/20 08:17:11 INFO Received uploads request method=POST path=/api/pending_closures4493worker2 # [ 36.128482] nix-grpc-daemon[859]: time=2026-09-20T08:17:11.263Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4494worker2 # [ 36.130417] nix-grpc-daemon[859]: time=2026-09-20T08:17:11.265Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4495lb # [ 36.128349] niks3-server[1039]: 2026/09/20 08:17:11 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4496lb # [ 36.143152] niks3-server[1039]: 2026/09/20 08:17:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4497worker2 # [ 36.170203] nix-grpc-daemon[859]: time=2026-09-20T08:17:11.305Z level=INFO msg="Uploading 1 narinfos"4498lb # [ 36.150849] niks3-server[1039]: 2026/09/20 08:17:11 INFO Signed narinfos id=3 count=14499lb # [ 36.155136] niks3-server[1039]: 2026/09/20 08:17:11 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4500lb # [ 36.166575] niks3-server[1039]: 2026/09/20 08:17:11 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4501worker2 # [ 36.192454] nix-grpc-daemon[859]: time=2026-09-20T08:17:11.327Z level=INFO msg="Upload complete. (408ms)"4502lb # [ 36.173330] niks3-server[1039]: 2026/09/20 08:17:11 INFO Completed upload id=34503worker2 # [ 36.194701] nix-grpc-daemon[784]: ts=2026-09-20T08:17:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53132 duration_s=0 paths=1 nar_bytes_in=6244504lb # [ 36.177673] niks3-server[1039]: 2026/09/20 08:17:11 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4505worker2 # [ 36.200732] nix-grpc-daemon[784]: ts=2026-09-20T08:17:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531324506worker2 # [ 36.210069] nix-daemon[847]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4507worker2 # [ 36.215074] nix-daemon[847]: reaped child process 906, status = succeeded4508client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4509worker2 # [ 36.218893] nix-daemon[847]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4510client # farm-b> worker2: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4511client # farm-b> LOG-farm-b-t14512worker2 # [ 37.446194] nix-daemon[847]: reaped child process 921, status = succeeded4513lb # [ 37.444331] envoy[910]: conn peer=[2001:db8:1::1]:40918 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=16874514lb # [ 37.448259] envoy[910]: conn peer=[2001:db8:1::1]:56322 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=974515lb # [ 37.451525] envoy[910]: conn peer=[2001:db8:1::1]:56338 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=74516lb # [ 37.454714] envoy[910]: conn peer=[2001:db8:1::1]:56350 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=124517worker2 # [ 37.567472] nix-daemon[847]: accepted connection from pid 936, user nix-grpc-daemon (trusted)4518worker2 # [ 37.577975] nix-daemon[847]: reaped child process 939, status = succeeded4519lb # [ 37.570626] niks3-server[1039]: 2026/09/20 08:17:12 INFO Received uploads request method=POST path=/api/pending_closures4520worker2 # [ 37.599237] nix-grpc-daemon[859]: time=2026-09-20T08:17:12.734Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4521worker2 # [ 37.600743] nix-grpc-daemon[859]: time=2026-09-20T08:17:12.736Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4522lb # [ 37.609077] niks3-server[1039]: 2026/09/20 08:17:12 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4523lb # [ 37.619669] niks3-server[1039]: 2026/09/20 08:17:12 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4524lb # [ 37.629314] niks3-server[1039]: 2026/09/20 08:17:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4525worker2 # [ 37.652432] nix-grpc-daemon[859]: time=2026-09-20T08:17:12.787Z level=INFO msg="Uploading 1 narinfos"4526lb # [ 37.633174] niks3-server[1039]: 2026/09/20 08:17:12 INFO Signed narinfos id=4 count=14527lb # [ 37.641776] niks3-server[1039]: 2026/09/20 08:17:12 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4528lb # [ 37.649216] niks3-server[1039]: 2026/09/20 08:17:12 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4529lb # [ 37.655049] niks3-server[1039]: 2026/09/20 08:17:12 INFO Completed upload id=44530worker2 # [ 37.674343] nix-grpc-daemon[859]: time=2026-09-20T08:17:12.809Z level=INFO msg="Upload complete. (247ms)"4531lb # [ 37.658072] niks3-server[1039]: 2026/09/20 08:17:12 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4532worker2 # [ 37.683780] nix-grpc-daemon[784]: ts=2026-09-20T08:17:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53132 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14533worker2 # [ 37.692491] nix-grpc-daemon[784]: ts=2026-09-20T08:17:12Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04534worker2 # [ 37.697349] nix-grpc-daemon[784]: ts=2026-09-20T08:17:12Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.4:50051 clients=14535worker2 # [ 37.705783] nix-daemon[847]: reaped child process 919, status = succeeded4536worker2 # [ 37.736684] nix-grpc-daemon[784]: ts=2026-09-20T08:17:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531324537worker2 # [ 37.748921] nix-grpc-daemon[784]: ts=2026-09-20T08:17:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:531324538worker2 # [ 37.762464] nix-grpc-daemon[859]: time=2026-09-20T08:17:12.897Z level=INFO msg="All 2 paths already cached"4539client # copying 1 paths to 'lb:50051'...4540worker2 # [ 37.770401] nix-grpc-daemon[784]: ts=2026-09-20T08:17:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:531324541client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4542worker2 # [ 37.786353] nix-daemon[847]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4543worker2 # [ 37.925935] nix-daemon[847]: accepted connection from pid 945, user nix-grpc-daemon (trusted)4544worker2 # [ 37.938929] nix-daemon[847]: reaped child process 948, status = succeeded4545worker2 # [ 38.059676] nix-daemon[847]: accepted connection from pid 950, user nix-grpc-daemon (trusted)4546worker2 # [ 38.072034] nix-daemon[847]: reaped child process 953, status = succeeded4547lb # [ 38.060699] niks3-server[1039]: 2026/09/20 08:17:13 INFO Received uploads request method=POST path=/api/pending_closures4548worker2 # [ 38.087758] nix-grpc-daemon[859]: time=2026-09-20T08:17:13.222Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4549worker2 # [ 38.089180] nix-grpc-daemon[859]: time=2026-09-20T08:17:13.222Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4550lb # [ 38.079075] niks3-server[1039]: 2026/09/20 08:17:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4551lb # [ 38.081205] niks3-server[1039]: 2026/09/20 08:17:13 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4552lb # [ 38.084414] niks3-server[1039]: 2026/09/20 08:17:13 INFO Signed narinfos id=5 count=14553worker2 # [ 38.103695] nix-grpc-daemon[859]: time=2026-09-20T08:17:13.239Z level=INFO msg="Uploading 1 narinfos"4554lb # [ 38.087638] niks3-server[1039]: 2026/09/20 08:17:13 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4555lb # [ 38.092060] niks3-server[1039]: 2026/09/20 08:17:13 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4556lb # [ 38.095979] niks3-server[1039]: 2026/09/20 08:17:13 INFO Completed upload id=54557worker2 # [ 38.115121] nix-grpc-daemon[859]: time=2026-09-20T08:17:13.250Z level=INFO msg="Upload complete. (316ms)"4558lb # [ 38.097565] niks3-server[1039]: 2026/09/20 08:17:13 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4559worker2 # [ 38.116969] nix-grpc-daemon[784]: ts=2026-09-20T08:17:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53132 duration_s=0 paths=1 nar_bytes_in=7524560worker2 # [ 38.120805] nix-grpc-daemon[784]: ts=2026-09-20T08:17:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531324561worker2 # [ 38.127381] nix-daemon[847]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4562worker2 # [ 38.128714] nix-daemon[847]: reaped child process 943, status = succeeded4563client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4564worker2 # [ 38.133727] nix-daemon[847]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4565client # farm-top> worker2: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4566client # farm-top> LOG-farm-top-t14567worker2 # [ 38.762125] nix-daemon[847]: reaped child process 957, status = succeeded4568worker2 # [ 38.906740] nix-daemon[847]: accepted connection from pid 972, user nix-grpc-daemon (trusted)4569worker2 # [ 38.917921] nix-daemon[847]: reaped child process 976, status = succeeded4570lb # [ 38.908784] niks3-server[1039]: 2026/09/20 08:17:13 INFO Received uploads request method=POST path=/api/pending_closures4571worker2 # [ 38.935886] nix-grpc-daemon[859]: time=2026-09-20T08:17:14.071Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4572worker2 # [ 38.937686] nix-grpc-daemon[859]: time=2026-09-20T08:17:14.072Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4573lb # [ 38.938832] niks3-server[1039]: 2026/09/20 08:17:13 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4574lb # [ 38.950442] niks3-server[1039]: 2026/09/20 08:17:13 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4575lb # [ 38.955962] niks3-server[1039]: 2026/09/20 08:17:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4576lb # [ 38.958609] niks3-server[1039]: 2026/09/20 08:17:13 INFO Signed narinfos id=6 count=14577worker2 # [ 38.979257] nix-grpc-daemon[859]: time=2026-09-20T08:17:14.114Z level=INFO msg="Uploading 1 narinfos"4578lb # [ 38.960926] niks3-server[1039]: 2026/09/20 08:17:13 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4579lb # [ 38.967381] niks3-server[1039]: 2026/09/20 08:17:13 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4580lb # [ 38.969776] niks3-server[1039]: 2026/09/20 08:17:13 INFO Completed upload id=64581worker2 # [ 38.989067] nix-grpc-daemon[859]: time=2026-09-20T08:17:14.124Z level=INFO msg="Upload complete. (248ms)"4582lb # [ 38.971719] niks3-server[1039]: 2026/09/20 08:17:13 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4583worker2 # [ 38.991499] nix-grpc-daemon[784]: ts=2026-09-20T08:17:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53132 duration_s=0 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14584worker2 # [ 38.999078] nix-daemon[847]: reaped child process 955, status = succeeded4585client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag t1 >&2, in 5.30 seconds)4586client: must succeed: nix eval --raw -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag t1 outPath4587client: (finished: must succeed: nix eval --raw -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag t1 outPath, in 0.16 seconds)4588client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4589client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.02 seconds)4590client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4591client # copying 3 paths...4592client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4593client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4594client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4595client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.29 seconds)4596(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 5.83 seconds)4597subtest: repeat is answered Cached by the scheduler without building4598worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4599worker1 # finding garbage collector roots...4600worker1 # deleting specified paths...4601worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4602worker1 # deleting unused links...4603worker1 # note: hard linking is currently saving 0.0 KiB4604worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.27 seconds)4605worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4606worker2 # finding garbage collector roots...4607worker2 # deleting specified paths...4608worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4609worker2 # deleting unused links...4610worker2 # note: hard linking is currently saving 0.0 KiB4611worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.25 seconds)4612client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag t1 >&24613worker1 # [ 40.267624] nix-grpc-daemon[732]: ts=2026-09-20T08:17:15Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:555564614client # don't know how to build these paths:4615client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4616worker2 # [ 40.300672] nix-grpc-daemon[784]: ts=2026-09-20T08:17:15Z level=debug event=want client=2 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=14617worker2 # [ 40.308215] nix-grpc-daemon[784]: ts=2026-09-20T08:17:15Z level=debug event=want client=2 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=14618worker2 # [ 40.317536] nix-grpc-daemon[784]: ts=2026-09-20T08:17:15Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14619client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag t1 >&2, in 0.27 seconds)4620client: must succeed: nix eval --raw -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag t1 outPath4621client: (finished: must succeed: nix eval --raw -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag t1 outPath, in 0.19 seconds)4622lb: must succeed: curl -sf localhost:9901/clusters4623lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4624lb: must succeed: curl -sf localhost:9901/clusters4625lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4626worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="cached"} ' || true4627worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="cached"} ' || true, in 0.07 seconds)4628(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.23 seconds)4629subtest: two clients wanting the same drv build it once4630client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4631client # [ 40.816342] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4632client # Running as unit: dedup1.service; invocation ID: 75204356c8bf48858f63cd0430ddda5c4633client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 0.07 seconds)4634client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4635client # [ 41.038037] nix[864]: don't know how to build these paths:4636client # [ 41.038929] nix[864]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4637worker2 # [ 41.061714] nix-grpc-daemon[784]: ts=2026-09-20T08:17:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:531324638worker2 # [ 41.071704] nix-grpc-daemon[784]: ts=2026-09-20T08:17:16Z level=debug event=want client=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04639worker2 # [ 41.075920] nix-grpc-daemon[784]: ts=2026-09-20T08:17:16Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.3:50051 clients=14640worker1 # [ 41.096104] nix-grpc-daemon[732]: ts=2026-09-20T08:17:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555564641worker1 # [ 41.111536] nix-grpc-daemon[732]: ts=2026-09-20T08:17:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:555564642client # [ 41.115156] nix[864]: copying 1 paths to 'lb:50051'...4643worker1 # [ 41.120923] nix-grpc-daemon[732]: ts=2026-09-20T08:17:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:555564644client # [ 41.117939] nix[864]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4645worker1 # [ 41.134957] nix-daemon[841]: accepted connection from pid 732, user nix-grpc-daemon (trusted)4646worker1 # [ 41.158225] nix-grpc-daemon[732]: ts=2026-09-20T08:17:15Z level=info event=niks3_push_started pid=8664647worker1 # [ 41.545380] nix-daemon[841]: accepted connection from pid 877, user nix-grpc-daemon (trusted)4648worker1 # [ 41.560611] nix-daemon[841]: reaped child process 881, status = succeeded4649worker1 # [ 41.693274] nix-daemon[841]: accepted connection from pid 883, user nix-grpc-daemon (trusted)4650worker1 # [ 41.705152] nix-daemon[841]: reaped child process 886, status = succeeded4651lb # [ 41.722414] niks3-server[1039]: 2026/09/20 08:17:16 INFO Received uploads request method=POST path=/api/pending_closures4652worker1 # [ 41.732123] nix-grpc-daemon[866]: time=2026-09-20T08:17:16.520Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4653worker1 # [ 41.734726] nix-grpc-daemon[866]: time=2026-09-20T08:17:16.524Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4654lb # [ 41.762657] niks3-server[1039]: 2026/09/20 08:17:16 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4655lb # [ 41.773065] niks3-server[1039]: 2026/09/20 08:17:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4656lb # [ 41.779369] niks3-server[1039]: 2026/09/20 08:17:16 INFO Signed narinfos id=7 count=14657worker1 # [ 41.781810] nix-grpc-daemon[866]: time=2026-09-20T08:17:16.572Z level=INFO msg="Uploading 1 narinfos"4658lb # [ 41.788135] niks3-server[1039]: 2026/09/20 08:17:16 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4659lb # [ 41.798720] niks3-server[1039]: 2026/09/20 08:17:16 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4660lb # [ 41.806596] niks3-server[1039]: 2026/09/20 08:17:16 INFO Completed upload id=74661worker1 # [ 41.805696] nix-grpc-daemon[866]: time=2026-09-20T08:17:16.595Z level=INFO msg="Upload complete. (401ms)"4662lb # [ 41.808542] niks3-server[1039]: 2026/09/20 08:17:16 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4663worker1 # [ 41.807832] nix-grpc-daemon[732]: ts=2026-09-20T08:17:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55556 duration_s=0 paths=1 nar_bytes_in=4324664worker1 # [ 41.813900] nix-grpc-daemon[732]: ts=2026-09-20T08:17:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555564665worker1 # [ 41.819635] nix-daemon[841]: accepted connection from pid 732, user nix-grpc-daemon (trusted)4666worker1 # [ 41.821880] nix-daemon[841]: reaped child process 864, status = succeeded4667worker1 # [ 41.827817] nix-daemon[841]: accepted connection from pid 732, user nix-grpc-daemon (trusted)4668client # [ 41.821974] nix[864]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4669client # [ 41.899640] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4670client # Running as unit: dedup2.service; invocation ID: 0fe5899cdfa24b38b97ae98469f3ebe94671client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 1.09 seconds)4672worker2 # [ 42.202141] nix-grpc-daemon[784]: ts=2026-09-20T08:17:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:531324673client # [ 42.183442] nix[879]: don't know how to build these paths:4674client # [ 42.184448] nix[879]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4675worker2 # [ 42.214939] nix-grpc-daemon[784]: ts=2026-09-20T08:17:17Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04676worker1 # [ 42.200699] nix-grpc-daemon[732]: ts=2026-09-20T08:17:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555564677client # [ 42.201362] nix[879]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4678client: must succeed: sleep 24679client: (finished: must succeed: sleep 2, in 2.03 seconds)4680worker1 # [ 46.444549] nix-daemon[841]: reaped child process 890, status = succeeded4681??? Warning (UserWarning): wait_until_succeeds(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4682 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394683client: waiting for success: ! systemctl is-active dedup1 dedup24684??? Warning (UserWarning): execute(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4685 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394686worker1 # [ 46.586412] nix-daemon[841]: accepted connection from pid 924, user nix-grpc-daemon (trusted)4687worker1 # [ 46.596891] nix-daemon[841]: reaped child process 927, status = succeeded4688lb # [ 46.612609] niks3-server[1039]: 2026/09/20 08:17:21 INFO Received uploads request method=POST path=/api/pending_closures4689worker1 # [ 46.617311] nix-grpc-daemon[866]: time=2026-09-20T08:17:21.407Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4690worker1 # [ 46.618976] nix-grpc-daemon[866]: time=2026-09-20T08:17:21.409Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4691lb # [ 46.637625] niks3-server[1039]: 2026/09/20 08:17:21 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4692lb # [ 46.642826] niks3-server[1039]: 2026/09/20 08:17:21 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4693lb # [ 46.645915] niks3-server[1039]: 2026/09/20 08:17:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4694lb # [ 46.648363] niks3-server[1039]: 2026/09/20 08:17:21 INFO Signed narinfos id=8 count=14695lb # [ 46.650070] niks3-server[1039]: 2026/09/20 08:17:21 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4696worker1 # [ 46.649300] nix-grpc-daemon[866]: time=2026-09-20T08:17:21.438Z level=INFO msg="Uploading 1 narinfos"4697lb # [ 46.655150] niks3-server[1039]: 2026/09/20 08:17:21 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4698lb # [ 46.658936] niks3-server[1039]: 2026/09/20 08:17:21 INFO Completed upload id=84699worker1 # [ 46.658108] nix-grpc-daemon[866]: time=2026-09-20T08:17:21.448Z level=INFO msg="Upload complete. (217ms)"4700lb # [ 46.660679] niks3-server[1039]: 2026/09/20 08:17:21 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4701worker1 # [ 46.660985] nix-grpc-daemon[732]: ts=2026-09-20T08:17:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55556 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=4 outputs=14702worker1 # [ 46.664703] nix-grpc-daemon[732]: ts=2026-09-20T08:17:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55556 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=4 outputs=14703worker1 # [ 46.670148] nix-daemon[841]: reaped child process 888, status = succeeded4704client # [ 46.684418] systemd[1]: dedup2.service: Deactivated successfully.4705client # [ 46.686280] systemd[1]: dedup2.service: Consumed 130ms CPU time over 4.786s wall clock time, 31.2M memory peak, 7.5K incoming IP traffic, 6.4K outgoing IP traffic.4706client # [ 46.723345] systemd[1]: dedup1.service: Deactivated successfully.4707client # [ 46.724867] systemd[1]: dedup1.service: Consumed 113ms CPU time over 5.908s wall clock time, 37.3M memory peak, 8.4K incoming IP traffic, 8.9K outgoing IP traffic.4708lb # [ 47.444581] envoy[910]: conn peer=[2001:db8:1::1]:56356 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9312 tx=3741 ms=51174709lb # [ 47.448547] envoy[910]: conn peer=[2001:db8:1::1]:56370 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1158 tx=481 ms=404710lb # [ 47.452065] envoy[910]: conn peer=[2001:db8:1::1]:56382 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1427 tx=1085 ms=44944711lb # [ 47.455625] envoy[910]: conn peer=[2001:db8:1::1]:56378 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2955 tx=1353 ms=56744712client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.09 seconds)4713client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4714client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)4715worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4716worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4717worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4718worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.06 seconds)4719worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="attached"} ' || true4720worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="attached"} ' || true, in 0.08 seconds)4721worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="attached"} ' || true4722worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="attached"} ' || true, in 0.07 seconds)4723(finished: subtest: two clients wanting the same drv build it once, in 7.08 seconds)4724subtest: build hook: nix-daemon with builders = grpc://lb4725client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag hook 2>/tmp/hook.log4726client # [ 48.049736] systemd[1]: Started Nix Daemon.4727client # [ 48.234382] nix-daemon[921]: accepted connection from pid 918, user root (trusted)4728worker1 # [ 48.441638] nix-grpc-daemon[732]: ts=2026-09-20T08:17:23Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:555564729worker2 # [ 48.468592] nix-grpc-daemon[784]: ts=2026-09-20T08:17:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:531324730worker2 # [ 48.484529] nix-grpc-daemon[784]: ts=2026-09-20T08:17:23Z level=debug event=want client=5 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=04731worker2 # [ 48.490562] nix-grpc-daemon[784]: ts=2026-09-20T08:17:23Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.4:50051 clients=14732worker2 # [ 48.495279] nix-grpc-daemon[784]: ts=2026-09-20T08:17:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531324733worker2 # [ 48.502921] nix-grpc-daemon[784]: ts=2026-09-20T08:17:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:531324734worker2 # [ 48.513750] nix-grpc-daemon[784]: ts=2026-09-20T08:17:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:531324735worker2 # [ 48.524532] nix-daemon[847]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4736worker2 # [ 48.694200] nix-daemon[847]: accepted connection from pid 1038, user nix-grpc-daemon (trusted)4737worker2 # [ 48.706582] nix-daemon[847]: reaped child process 1041, status = succeeded4738worker2 # [ 48.873226] nix-daemon[847]: accepted connection from pid 1043, user nix-grpc-daemon (trusted)4739worker2 # [ 48.883775] nix-daemon[847]: reaped child process 1046, status = succeeded4740lb # [ 48.879801] niks3-server[1039]: 2026/09/20 08:17:23 INFO Received uploads request method=POST path=/api/pending_closures4741worker2 # [ 48.907326] nix-grpc-daemon[859]: time=2026-09-20T08:17:24.042Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4742worker2 # [ 48.908967] nix-grpc-daemon[859]: time=2026-09-20T08:17:24.044Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"4743lb # [ 48.908714] niks3-server[1039]: 2026/09/20 08:17:23 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst4744lb # [ 48.923133] niks3-server[1039]: 2026/09/20 08:17:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4745worker2 # [ 48.947212] nix-grpc-daemon[859]: time=2026-09-20T08:17:24.082Z level=INFO msg="Uploading 1 narinfos"4746lb # [ 48.928511] niks3-server[1039]: 2026/09/20 08:17:23 INFO Signed narinfos id=9 count=14747lb # [ 48.935488] niks3-server[1039]: 2026/09/20 08:17:23 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls4748lb # [ 48.945792] niks3-server[1039]: 2026/09/20 08:17:23 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4749lb # [ 48.952777] niks3-server[1039]: 2026/09/20 08:17:23 INFO Completed upload id=94750worker2 # [ 48.971806] nix-grpc-daemon[859]: time=2026-09-20T08:17:24.107Z level=INFO msg="Upload complete. (436ms)"4751lb # [ 48.956667] niks3-server[1039]: 2026/09/20 08:17:23 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo4752worker2 # [ 48.975656] nix-grpc-daemon[784]: ts=2026-09-20T08:17:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53132 duration_s=0 paths=1 nar_bytes_in=5124753worker2 # [ 48.981551] nix-grpc-daemon[784]: ts=2026-09-20T08:17:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531324754worker2 # [ 48.990467] nix-daemon[847]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4755worker2 # [ 48.999489] nix-daemon[847]: reaped child process 1036, status = succeeded4756worker2 # [ 49.004848] nix-daemon[847]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4757worker2 # [ 50.028417] nix-daemon[847]: reaped child process 1050, status = succeeded4758worker2 # [ 50.197312] nix-daemon[847]: accepted connection from pid 1065, user nix-grpc-daemon (trusted)4759worker2 # [ 50.207666] nix-daemon[847]: reaped child process 1068, status = succeeded4760lb # [ 50.205496] niks3-server[1039]: 2026/09/20 08:17:25 INFO Received uploads request method=POST path=/api/pending_closures4761worker2 # [ 50.229416] nix-grpc-daemon[859]: time=2026-09-20T08:17:25.364Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4762worker2 # [ 50.230873] nix-grpc-daemon[859]: time=2026-09-20T08:17:25.366Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"4763lb # [ 50.233075] niks3-server[1039]: 2026/09/20 08:17:25 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv4764lb # [ 50.247220] niks3-server[1039]: 2026/09/20 08:17:25 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4765lb # [ 50.259467] niks3-server[1039]: 2026/09/20 08:17:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4766worker2 # [ 50.282411] nix-grpc-daemon[859]: time=2026-09-20T08:17:25.417Z level=INFO msg="Uploading 1 narinfos"4767lb # [ 50.263564] niks3-server[1039]: 2026/09/20 08:17:25 INFO Signed narinfos id=10 count=14768lb # [ 50.268799] niks3-server[1039]: 2026/09/20 08:17:25 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls4769lb # [ 50.278322] niks3-server[1039]: 2026/09/20 08:17:25 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4770lb # [ 50.281254] niks3-server[1039]: 2026/09/20 08:17:25 INFO Completed upload id=104771worker2 # [ 50.300672] nix-grpc-daemon[859]: time=2026-09-20T08:17:25.436Z level=INFO msg="Upload complete. (289ms)"4772lb # [ 50.283831] niks3-server[1039]: 2026/09/20 08:17:25 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo4773worker2 # [ 50.306544] nix-grpc-daemon[784]: ts=2026-09-20T08:17:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53132 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=5 outputs=14774worker2 # [ 50.318973] nix-daemon[847]: reaped child process 1048, status = succeeded4775worker1 # [ 50.296166] nix-grpc-daemon[732]: ts=2026-09-20T08:17:25Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:555564776worker1 # [ 50.309217] nix-grpc-daemon[732]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...4777worker2 # [ 50.361346] nix-grpc-daemon[784]: ts=2026-09-20T08:17:25Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:531324778worker2 # [ 50.367117] nix-grpc-daemon[784]: ts=2026-09-20T08:17:25Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53132 duration_s=0 paths=1 nar_bytes_out=1284779worker2 # [ 50.609195] nix-grpc-daemon[784]: ts=2026-09-20T08:17:25Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:531324780worker1 # [ 50.594630] nix-grpc-daemon[732]: ts=2026-09-20T08:17:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:555564781worker1 # [ 50.604424] nix-grpc-daemon[866]: time=2026-09-20T08:17:25.394Z level=INFO msg="All 1 paths already cached"4782worker2 # [ 50.637313] nix-grpc-daemon[784]: ts=2026-09-20T08:17:25Z level=debug event=want client=6 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=04783worker2 # [ 50.639323] nix-grpc-daemon[784]: ts=2026-09-20T08:17:25Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.4:50051 clients=14784worker2 # [ 50.682615] nix-grpc-daemon[784]: ts=2026-09-20T08:17:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531324785worker2 # [ 50.696363] nix-grpc-daemon[784]: ts=2026-09-20T08:17:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:531324786worker2 # [ 50.709487] nix-grpc-daemon[859]: time=2026-09-20T08:17:25.844Z level=INFO msg="All 1 paths already cached"4787worker2 # [ 50.718236] nix-grpc-daemon[784]: ts=2026-09-20T08:17:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:531324788worker2 # [ 50.727348] nix-daemon[847]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4789worker2 # [ 50.903132] nix-daemon[847]: accepted connection from pid 1074, user nix-grpc-daemon (trusted)4790worker2 # [ 50.915984] nix-daemon[847]: reaped child process 1077, status = succeeded4791worker2 # [ 51.071327] nix-daemon[847]: accepted connection from pid 1079, user nix-grpc-daemon (trusted)4792worker2 # [ 51.083408] nix-daemon[847]: reaped child process 1082, status = succeeded4793lb # [ 51.078396] niks3-server[1039]: 2026/09/20 08:17:26 INFO Received uploads request method=POST path=/api/pending_closures4794worker2 # [ 51.105450] nix-grpc-daemon[859]: time=2026-09-20T08:17:26.240Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4795worker2 # [ 51.106934] nix-grpc-daemon[859]: time=2026-09-20T08:17:26.242Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"4796lb # [ 51.106842] niks3-server[1039]: 2026/09/20 08:17:26 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst4797lb # [ 51.116269] niks3-server[1039]: 2026/09/20 08:17:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4798worker2 # [ 51.141737] nix-grpc-daemon[859]: time=2026-09-20T08:17:26.277Z level=INFO msg="Uploading 1 narinfos"4799lb # [ 51.123157] niks3-server[1039]: 2026/09/20 08:17:26 INFO Signed narinfos id=11 count=14800lb # [ 51.129607] niks3-server[1039]: 2026/09/20 08:17:26 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls4801lb # [ 51.140326] niks3-server[1039]: 2026/09/20 08:17:26 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4802lb # [ 51.146866] niks3-server[1039]: 2026/09/20 08:17:26 INFO Completed upload id=114803worker2 # [ 51.166408] nix-grpc-daemon[859]: time=2026-09-20T08:17:26.301Z level=INFO msg="Upload complete. (424ms)"4804lb # [ 51.149169] niks3-server[1039]: 2026/09/20 08:17:26 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo4805worker2 # [ 51.171128] nix-grpc-daemon[784]: ts=2026-09-20T08:17:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53132 duration_s=0 paths=1 nar_bytes_in=6244806worker2 # [ 51.175956] nix-grpc-daemon[784]: ts=2026-09-20T08:17:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531324807worker2 # [ 51.184824] nix-daemon[847]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4808worker2 # [ 51.189768] nix-daemon[847]: reaped child process 1072, status = succeeded4809worker2 # [ 51.194912] nix-daemon[847]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4810worker2 # [ 52.619850] nix-daemon[847]: reaped child process 1086, status = succeeded4811worker2 # [ 52.784310] nix-daemon[847]: accepted connection from pid 1101, user nix-grpc-daemon (trusted)4812worker2 # [ 52.794939] nix-daemon[847]: reaped child process 1104, status = succeeded4813lb # [ 52.791682] niks3-server[1039]: 2026/09/20 08:17:27 INFO Received uploads request method=POST path=/api/pending_closures4814worker2 # [ 52.820428] nix-grpc-daemon[859]: time=2026-09-20T08:17:27.955Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4815worker2 # [ 52.822240] nix-grpc-daemon[859]: time=2026-09-20T08:17:27.957Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"4816lb # [ 52.839746] niks3-server[1039]: 2026/09/20 08:17:27 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst4817lb # [ 52.847549] niks3-server[1039]: 2026/09/20 08:17:27 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv4818lb # [ 52.854755] niks3-server[1039]: 2026/09/20 08:17:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4819worker2 # [ 52.879091] nix-grpc-daemon[859]: time=2026-09-20T08:17:28.014Z level=INFO msg="Uploading 1 narinfos"4820lb # [ 52.859780] niks3-server[1039]: 2026/09/20 08:17:27 INFO Signed narinfos id=12 count=14821lb # [ 52.864960] niks3-server[1039]: 2026/09/20 08:17:27 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls4822lb # [ 52.874347] niks3-server[1039]: 2026/09/20 08:17:27 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4823lb # [ 52.879542] niks3-server[1039]: 2026/09/20 08:17:27 INFO Completed upload id=124824lb # [ 52.882819] niks3-server[1039]: 2026/09/20 08:17:27 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo4825worker2 # [ 52.900564] nix-grpc-daemon[859]: time=2026-09-20T08:17:28.035Z level=INFO msg="Upload complete. (297ms)"4826worker2 # [ 52.907509] nix-grpc-daemon[784]: ts=2026-09-20T08:17:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53132 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=6 outputs=14827worker1 # [ 52.896519] nix-grpc-daemon[732]: ts=2026-09-20T08:17:27Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:555564828worker2 # [ 52.919834] nix-daemon[847]: reaped child process 1084, status = succeeded4829worker1 # [ 52.901466] nix-grpc-daemon[732]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...4830worker2 # [ 52.944187] nix-grpc-daemon[784]: ts=2026-09-20T08:17:28Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:531324831worker2 # [ 52.949374] nix-grpc-daemon[784]: ts=2026-09-20T08:17:28Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53132 duration_s=0 paths=1 nar_bytes_out=1764832worker1 # [ 53.201780] nix-grpc-daemon[732]: ts=2026-09-20T08:17:27Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:555564833worker1 # [ 53.208374] nix-grpc-daemon[732]: ts=2026-09-20T08:17:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:555564834worker1 # [ 53.217261] nix-grpc-daemon[866]: time=2026-09-20T08:17:28.006Z level=INFO msg="All 2 paths already cached"4835worker2 # [ 53.250301] nix-grpc-daemon[784]: ts=2026-09-20T08:17:28Z level=debug event=want client=7 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=04836worker2 # [ 53.256786] nix-grpc-daemon[784]: ts=2026-09-20T08:17:28Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.4:50051 clients=14837worker2 # [ 53.261592] nix-grpc-daemon[784]: ts=2026-09-20T08:17:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531324838worker2 # [ 53.271139] nix-grpc-daemon[784]: ts=2026-09-20T08:17:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:531324839worker2 # [ 53.280473] nix-grpc-daemon[859]: time=2026-09-20T08:17:28.415Z level=INFO msg="All 2 paths already cached"4840worker2 # [ 53.286859] nix-grpc-daemon[784]: ts=2026-09-20T08:17:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:531324841worker2 # [ 53.297102] nix-daemon[847]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4842worker2 # [ 53.466764] nix-daemon[847]: accepted connection from pid 1110, user nix-grpc-daemon (trusted)4843worker2 # [ 53.479724] nix-daemon[847]: reaped child process 1113, status = succeeded4844worker2 # [ 53.606456] nix-daemon[847]: accepted connection from pid 1115, user nix-grpc-daemon (trusted)4845worker2 # [ 53.617380] nix-daemon[847]: reaped child process 1118, status = succeeded4846lb # [ 53.609343] niks3-server[1039]: 2026/09/20 08:17:28 INFO Received uploads request method=POST path=/api/pending_closures4847worker2 # [ 53.641914] nix-grpc-daemon[859]: time=2026-09-20T08:17:28.777Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4848worker2 # [ 53.643365] nix-grpc-daemon[859]: time=2026-09-20T08:17:28.778Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"4849lb # [ 53.644633] niks3-server[1039]: 2026/09/20 08:17:28 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst4850lb # [ 53.653964] niks3-server[1039]: 2026/09/20 08:17:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4851worker2 # [ 53.680220] nix-grpc-daemon[859]: time=2026-09-20T08:17:28.815Z level=INFO msg="Uploading 1 narinfos"4852lb # [ 53.661060] niks3-server[1039]: 2026/09/20 08:17:28 INFO Signed narinfos id=13 count=14853lb # [ 53.665836] niks3-server[1039]: 2026/09/20 08:17:28 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls4854lb # [ 53.678482] niks3-server[1039]: 2026/09/20 08:17:28 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4855lb # [ 53.683494] niks3-server[1039]: 2026/09/20 08:17:28 INFO Completed upload id=134856worker2 # [ 53.702693] nix-grpc-daemon[859]: time=2026-09-20T08:17:28.838Z level=INFO msg="Upload complete. (394ms)"4857lb # [ 53.688182] niks3-server[1039]: 2026/09/20 08:17:28 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo4858worker2 # [ 53.706395] nix-grpc-daemon[784]: ts=2026-09-20T08:17:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53132 duration_s=0 paths=1 nar_bytes_in=7524859worker2 # [ 53.711837] nix-grpc-daemon[784]: ts=2026-09-20T08:17:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531324860worker2 # [ 53.721576] nix-daemon[847]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4861worker2 # [ 53.727442] nix-daemon[847]: reaped child process 1108, status = succeeded4862worker2 # [ 53.731492] nix-daemon[847]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4863worker2 # [ 54.544234] nix-daemon[847]: reaped child process 1122, status = succeeded4864worker2 # [ 54.686056] nix-daemon[847]: accepted connection from pid 1137, user nix-grpc-daemon (trusted)4865worker2 # [ 54.697680] nix-daemon[847]: reaped child process 1140, status = succeeded4866lb # [ 54.693732] niks3-server[1039]: 2026/09/20 08:17:29 INFO Received uploads request method=POST path=/api/pending_closures4867worker2 # [ 54.721891] nix-grpc-daemon[859]: time=2026-09-20T08:17:29.856Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4868worker2 # [ 54.723296] nix-grpc-daemon[859]: time=2026-09-20T08:17:29.856Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"4869lb # [ 54.730688] niks3-server[1039]: 2026/09/20 08:17:29 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst4870lb # [ 54.741144] niks3-server[1039]: 2026/09/20 08:17:29 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv4871lb # [ 54.749389] niks3-server[1039]: 2026/09/20 08:17:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4872worker2 # [ 54.774835] nix-grpc-daemon[859]: time=2026-09-20T08:17:29.910Z level=INFO msg="Uploading 1 narinfos"4873lb # [ 54.755265] niks3-server[1039]: 2026/09/20 08:17:29 INFO Signed narinfos id=14 count=14874lb # [ 54.759107] niks3-server[1039]: 2026/09/20 08:17:29 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls4875lb # [ 54.767790] niks3-server[1039]: 2026/09/20 08:17:29 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4876lb # [ 54.770846] niks3-server[1039]: 2026/09/20 08:17:29 INFO Completed upload id=144877worker2 # [ 54.789988] nix-grpc-daemon[859]: time=2026-09-20T08:17:29.925Z level=INFO msg="Upload complete. (252ms)"4878lb # [ 54.774051] niks3-server[1039]: 2026/09/20 08:17:29 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo4879worker2 # [ 54.792344] nix-grpc-daemon[784]: ts=2026-09-20T08:17:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53132 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=7 outputs=14880worker1 # [ 54.778608] nix-grpc-daemon[732]: ts=2026-09-20T08:17:29Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:555564881worker2 # [ 54.799461] nix-daemon[847]: reaped child process 1120, status = succeeded4882worker1 # [ 54.783478] nix-grpc-daemon[732]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...4883worker1 # [ 54.799974] nix-grpc-daemon[732]: ts=2026-09-20T08:17:29Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:555564884worker1 # [ 54.802701] nix-grpc-daemon[732]: ts=2026-09-20T08:17:29Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:55556 duration_s=0 paths=1 nar_bytes_out=2324885client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.06 seconds)4886client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top4887client # [ 54.929862] nix-daemon[921]: reaped child process 923, status = succeeded4888client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.03 seconds)4889client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4890client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.04 seconds)4891client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4892client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4893(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.16 seconds)4894subtest: an input only one worker has still reaches the builder4895client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only4896client # this derivation will be built:4897client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv4898client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...4899client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only, in 0.41 seconds)4900client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure4901client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.20 seconds)4902worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4903worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.20 seconds)4904worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only4905worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)4906client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&24907client # [ 56.024624] nix-daemon[921]: accepted connection from pid 1001, user root (trusted)4908client # this derivation will be built:4909client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv4910worker2 # [ 56.231762] nix-grpc-daemon[784]: ts=2026-09-20T08:17:31Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:531324911client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux'...4912worker1 # [ 56.216326] nix-grpc-daemon[732]: ts=2026-09-20T08:17:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:555564913worker1 # [ 56.359929] nix-daemon[841]: accepted connection from pid 949, user nix-grpc-daemon (trusted)4914worker1 # [ 56.370915] nix-daemon[841]: reaped child process 952, status = succeeded4915lb # [ 56.384681] niks3-server[1039]: 2026/09/20 08:17:31 INFO Received uploads request method=POST path=/api/pending_closures4916worker1 # [ 56.450063] nix-grpc-daemon[866]: time=2026-09-20T08:17:31.239Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4917worker1 # [ 56.452106] nix-grpc-daemon[866]: time=2026-09-20T08:17:31.240Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"4918lb # [ 56.461966] niks3-server[1039]: 2026/09/20 08:17:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4919lb # [ 56.464301] niks3-server[1039]: 2026/09/20 08:17:31 INFO Signed narinfos id=15 count=14920worker1 # [ 56.464206] nix-grpc-daemon[866]: time=2026-09-20T08:17:31.253Z level=INFO msg="Uploading 1 narinfos"4921lb # [ 56.466424] niks3-server[1039]: 2026/09/20 08:17:31 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4922lb # [ 56.473138] niks3-server[1039]: 2026/09/20 08:17:31 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4923lb # [ 56.475760] niks3-server[1039]: 2026/09/20 08:17:31 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls4924lb # [ 56.483022] niks3-server[1039]: 2026/09/20 08:17:31 INFO Completed upload id=154925worker1 # [ 56.482448] nix-grpc-daemon[866]: time=2026-09-20T08:17:31.272Z level=INFO msg="Upload complete. (266ms)"4926lb # [ 56.489198] niks3-server[1039]: 2026/09/20 08:17:31 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo4927worker2 # [ 56.518074] nix-grpc-daemon[784]: ts=2026-09-20T08:17:31Z level=debug event=want client=8 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=04928worker2 # [ 56.520060] nix-grpc-daemon[784]: ts=2026-09-20T08:17:31Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=14929worker1 # [ 56.543402] nix-grpc-daemon[732]: ts=2026-09-20T08:17:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555564930worker1 # [ 56.554727] nix-grpc-daemon[732]: ts=2026-09-20T08:17:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:555564931client # copying 2 paths to 'lb:50051'...4932worker1 # [ 56.567957] nix-grpc-daemon[732]: ts=2026-09-20T08:17:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:555564933client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...4934client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...4935worker1 # [ 56.588205] nix-daemon[841]: accepted connection from pid 732, user nix-grpc-daemon (trusted)4936worker1 # [ 56.756412] nix-daemon[841]: accepted connection from pid 956, user nix-grpc-daemon (trusted)4937worker1 # [ 56.769106] nix-daemon[841]: reaped child process 959, status = succeeded4938worker1 # [ 56.902658] nix-daemon[841]: accepted connection from pid 961, user nix-grpc-daemon (trusted)4939worker1 # [ 56.916334] nix-daemon[841]: reaped child process 964, status = succeeded4940lb # [ 56.930465] niks3-server[1039]: 2026/09/20 08:17:31 INFO Received uploads request method=POST path=/api/pending_closures4941lb # [ 56.936197] niks3-server[1039]: 2026/09/20 08:17:31 INFO Received uploads request method=POST path=/api/pending_closures4942worker1 # [ 56.939589] nix-grpc-daemon[866]: time=2026-09-20T08:17:31.729Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4943worker1 # [ 56.940938] nix-grpc-daemon[866]: time=2026-09-20T08:17:31.729Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"4944worker1 # [ 56.942509] nix-grpc-daemon[866]: time=2026-09-20T08:17:31.729Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"4945lb # [ 56.953320] niks3-server[1039]: 2026/09/20 08:17:31 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst4946lb # [ 56.960823] niks3-server[1039]: 2026/09/20 08:17:31 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst4947lb # [ 56.963599] niks3-server[1039]: 2026/09/20 08:17:31 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls4948lb # [ 56.968535] niks3-server[1039]: 2026/09/20 08:17:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4949lb # [ 56.970530] niks3-server[1039]: 2026/09/20 08:17:31 INFO Signed narinfos id=16 count=14950lb # [ 56.972839] niks3-server[1039]: 2026/09/20 08:17:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4951lb # [ 56.975482] niks3-server[1039]: 2026/09/20 08:17:31 INFO Signed narinfos id=17 count=24952worker1 # [ 56.975077] nix-grpc-daemon[866]: time=2026-09-20T08:17:31.764Z level=INFO msg="Uploading 3 narinfos"4953lb # [ 56.978794] niks3-server[1039]: 2026/09/20 08:17:31 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls4954lb # [ 56.992409] niks3-server[1039]: 2026/09/20 08:17:31 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4955lb # [ 56.994906] niks3-server[1039]: 2026/09/20 08:17:31 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo4956lb # [ 56.999403] niks3-server[1039]: 2026/09/20 08:17:31 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo4957lb # [ 57.002487] niks3-server[1039]: 2026/09/20 08:17:31 INFO Completed upload id=164958lb # [ 57.003964] niks3-server[1039]: 2026/09/20 08:17:31 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo4959lb # [ 57.005723] niks3-server[1039]: 2026/09/20 08:17:31 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4960lb # [ 57.008268] niks3-server[1039]: 2026/09/20 08:17:31 INFO Completed upload id=174961worker1 # [ 57.007388] nix-grpc-daemon[866]: time=2026-09-20T08:17:31.797Z level=INFO msg="Upload complete. (400ms)"4962worker1 # [ 57.012124] nix-grpc-daemon[732]: ts=2026-09-20T08:17:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55556 duration_s=0 paths=2 nar_bytes_in=9764963worker1 # [ 57.014723] nix-grpc-daemon[732]: ts=2026-09-20T08:17:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555564964worker1 # [ 57.019295] nix-daemon[841]: accepted connection from pid 732, user nix-grpc-daemon (trusted)4965worker1 # [ 57.024195] nix-daemon[841]: reaped child process 954, status = succeeded4966worker1 # [ 57.028874] nix-daemon[841]: accepted connection from pid 732, user nix-grpc-daemon (trusted)4967client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...4968client # uses-input-w1onlya> node-a: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv4969lb # [ 57.444668] envoy[910]: conn peer=[2001:db8:1::1]:51638 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3397 tx=1775 ms=19274970lb # [ 57.446216] envoy[910]: conn peer=[2001:db8:1::1]:51648 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=244971lb # [ 57.447623] envoy[910]: conn peer=[2001:db8:1::1]:51654 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3880 tx=1969 ms=23934972lb # [ 57.449055] envoy[910]: conn peer=[2001:db8:1::1]:35886 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=378 ms=594973lb # [ 57.450394] envoy[910]: conn peer=[2001:db8:1::1]:35888 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4328 tx=2221 ms=16494974lb # [ 57.451874] envoy[910]: conn peer=[2001:db8:1::1]:35904 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=504975worker1 # [ 58.457580] nix-daemon[841]: reaped child process 968, status = succeeded4976worker1 # [ 58.602821] nix-daemon[841]: accepted connection from pid 983, user nix-grpc-daemon (trusted)4977worker1 # [ 58.613678] nix-daemon[841]: reaped child process 986, status = succeeded4978lb # [ 58.629649] niks3-server[1039]: 2026/09/20 08:17:33 INFO Received uploads request method=POST path=/api/pending_closures4979worker1 # [ 58.637235] nix-grpc-daemon[866]: time=2026-09-20T08:17:33.427Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4980lb # [ 58.659212] niks3-server[1039]: 2026/09/20 08:17:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4981lb # [ 58.664235] niks3-server[1039]: 2026/09/20 08:17:33 INFO Signed narinfos id=18 count=14982worker1 # [ 58.666218] nix-grpc-daemon[866]: time=2026-09-20T08:17:33.456Z level=INFO msg="Uploading 1 narinfos"4983lb # [ 58.672827] niks3-server[1039]: 2026/09/20 08:17:33 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv4984lb # [ 58.687356] niks3-server[1039]: 2026/09/20 08:17:33 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4985lb # [ 58.980873] niks3-server[1039]: 2026/09/20 08:17:33 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls4986lb # [ 58.997438] niks3-server[1039]: 2026/09/20 08:17:33 INFO Completed upload id=184987worker1 # [ 58.996358] nix-grpc-daemon[866]: time=2026-09-20T08:17:33.786Z level=INFO msg="Upload complete. (558ms)"4988worker1 # [ 59.001914] nix-grpc-daemon[732]: ts=2026-09-20T08:17:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55556 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=8 outputs=14989lb # [ 59.012208] niks3-server[1039]: 2026/09/20 08:17:33 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo4990worker1 # [ 59.016596] nix-daemon[841]: reaped child process 966, status = succeeded4991client # copying 1 paths...4992worker1 # [ 59.048896] nix-grpc-daemon[732]: ts=2026-09-20T08:17:33Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:555564993client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...4994worker2 # [ 59.090930] nix-grpc-daemon[784]: ts=2026-09-20T08:17:34Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:531324995worker2 # [ 59.107691] nix-grpc-daemon[784]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...4996worker2 # [ 59.126872] nix-grpc-daemon[784]: ts=2026-09-20T08:17:34Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53132 duration_s=0 paths=1 nar_bytes_out=1204997client # [ 59.207294] nix-daemon[921]: reaped child process 1004, status = succeeded4998client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&2, in 3.39 seconds)4999client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&25000client # [ 59.389785] nix-daemon[921]: accepted connection from pid 1022, user root (trusted)5001client # this derivation will be built:5002client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5003worker2 # [ 59.606190] nix-grpc-daemon[784]: ts=2026-09-20T08:17:34Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:531325004client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux'...5005worker2 # [ 59.611353] nix-grpc-daemon[784]: ts=2026-09-20T08:17:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:531325006worker2 # [ 59.632811] nix-grpc-daemon[784]: ts=2026-09-20T08:17:34Z level=debug event=want client=9 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05007worker2 # [ 59.638325] nix-grpc-daemon[784]: ts=2026-09-20T08:17:34Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=15008worker2 # [ 59.677074] nix-grpc-daemon[784]: ts=2026-09-20T08:17:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531325009worker2 # [ 59.689638] nix-grpc-daemon[784]: ts=2026-09-20T08:17:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:531325010client # copying 1 paths to 'lb:50051'...5011client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5012worker2 # [ 59.709241] nix-grpc-daemon[784]: ts=2026-09-20T08:17:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:531325013worker2 # [ 59.720149] nix-daemon[847]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5014worker2 # [ 59.735632] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 56 ms (attempt 1/5)5015worker2 # [ 59.797450] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 193 ms (attempt 2/5)5016worker2 # [ 59.996574] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 189 ms (attempt 3/5)5017worker2 # [ 60.192199] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 624 ms (attempt 4/5)5018worker2 # [ 60.822295] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5019worker2 # [ 60.827834] nix-grpc-daemon[784]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5020worker2 # [ 61.032268] nix-daemon[847]: accepted connection from pid 1156, user nix-grpc-daemon (trusted)5021worker2 # [ 61.043778] nix-daemon[847]: reaped child process 1159, status = succeeded5022worker2 # [ 61.205349] nix-daemon[847]: accepted connection from pid 1161, user nix-grpc-daemon (trusted)5023worker2 # [ 61.217347] nix-daemon[847]: reaped child process 1164, status = succeeded5024lb # [ 61.215390] niks3-server[1039]: 2026/09/20 08:17:36 INFO Received uploads request method=POST path=/api/pending_closures5025worker2 # [ 61.385528] nix-grpc-daemon[859]: time=2026-09-20T08:17:36.519Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5026worker2 # [ 61.389069] nix-grpc-daemon[859]: time=2026-09-20T08:17:36.519Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5027lb # [ 61.392398] niks3-server[1039]: 2026/09/20 08:17:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5028lb # [ 61.398146] niks3-server[1039]: 2026/09/20 08:17:36 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5029lb # [ 61.404553] niks3-server[1039]: 2026/09/20 08:17:36 INFO Signed narinfos id=19 count=15030worker2 # [ 61.423533] nix-grpc-daemon[859]: time=2026-09-20T08:17:36.558Z level=INFO msg="Uploading 1 narinfos"5031lb # [ 61.418185] niks3-server[1039]: 2026/09/20 08:17:36 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5032lb # [ 61.424177] niks3-server[1039]: 2026/09/20 08:17:36 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5033lb # [ 61.433060] niks3-server[1039]: 2026/09/20 08:17:36 INFO Completed upload id=195034worker2 # [ 61.454745] nix-grpc-daemon[859]: time=2026-09-20T08:17:36.590Z level=INFO msg="Upload complete. (607ms)"5035lb # [ 61.436741] niks3-server[1039]: 2026/09/20 08:17:36 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5036worker2 # [ 61.459958] nix-grpc-daemon[784]: ts=2026-09-20T08:17:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53132 duration_s=1 paths=1 nar_bytes_in=5605037worker2 # [ 61.467766] nix-grpc-daemon[784]: ts=2026-09-20T08:17:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531325038worker2 # [ 61.476632] nix-daemon[847]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5039worker2 # [ 61.492382] nix-daemon[847]: reaped child process 1150, status = succeeded5040worker2 # [ 61.495887] nix-grpc-daemon[784]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...5041client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5042worker2 # [ 61.510833] nix-daemon[847]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5043client # uses-input-w1onlyb> worker2: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5044worker2 # [ 62.527587] nix-daemon[847]: reaped child process 1169, status = succeeded5045worker2 # [ 62.681597] nix-daemon[847]: accepted connection from pid 1184, user nix-grpc-daemon (trusted)5046worker2 # [ 62.693496] nix-daemon[847]: reaped child process 1187, status = succeeded5047lb # [ 62.692548] niks3-server[1039]: 2026/09/20 08:17:37 INFO Received uploads request method=POST path=/api/pending_closures5048worker2 # [ 62.721881] nix-grpc-daemon[859]: time=2026-09-20T08:17:37.855Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5049lb # [ 62.721076] niks3-server[1039]: 2026/09/20 08:17:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5050lb # [ 62.729566] niks3-server[1039]: 2026/09/20 08:17:37 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5051lb # [ 62.735784] niks3-server[1039]: 2026/09/20 08:17:37 INFO Signed narinfos id=20 count=15052worker2 # [ 62.758203] nix-grpc-daemon[859]: time=2026-09-20T08:17:37.893Z level=INFO msg="Uploading 1 narinfos"5053lb # [ 62.740788] niks3-server[1039]: 2026/09/20 08:17:37 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5054lb # [ 62.752685] niks3-server[1039]: 2026/09/20 08:17:37 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5055worker2 # [ 62.777516] nix-grpc-daemon[859]: time=2026-09-20T08:17:37.912Z level=INFO msg="Upload complete. (255ms)"5056lb # [ 62.758814] niks3-server[1039]: 2026/09/20 08:17:37 INFO Completed upload id=205057worker2 # [ 62.779784] nix-grpc-daemon[784]: ts=2026-09-20T08:17:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53132 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=9 outputs=15058lb # [ 62.764441] niks3-server[1039]: 2026/09/20 08:17:37 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5059worker2 # [ 62.786444] nix-daemon[847]: reaped child process 1166, status = succeeded5060client # copying 1 paths...5061worker1 # [ 62.807489] nix-grpc-daemon[732]: ts=2026-09-20T08:17:37Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:555565062worker1 # [ 62.819357] nix-grpc-daemon[732]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5063client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5064worker1 # [ 62.852193] nix-grpc-daemon[732]: ts=2026-09-20T08:17:37Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:555565065worker1 # [ 62.856902] nix-grpc-daemon[732]: ts=2026-09-20T08:17:37Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:55556 duration_s=0 paths=1 nar_bytes_out=1205066client # [ 62.957734] nix-daemon[921]: reaped child process 1025, status = succeeded5067client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.75 seconds)5068client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&25069client # [ 63.130408] nix-daemon[921]: accepted connection from pid 1043, user root (trusted)5070client # this derivation will be built:5071client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5072worker1 # [ 63.311229] nix-grpc-daemon[732]: ts=2026-09-20T08:17:38Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:555565073client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux'...5074worker2 # [ 63.334482] nix-grpc-daemon[784]: ts=2026-09-20T08:17:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:531325075worker2 # [ 63.342297] nix-grpc-daemon[859]: time=2026-09-20T08:17:38.477Z level=INFO msg="All 1 paths already cached"5076worker2 # [ 63.356504] nix-grpc-daemon[784]: ts=2026-09-20T08:17:38Z level=debug event=want client=10 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05077worker1 # [ 63.338553] nix-grpc-daemon[732]: ts=2026-09-20T08:17:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555565078worker2 # [ 63.361174] nix-grpc-daemon[784]: ts=2026-09-20T08:17:38Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=15079worker1 # [ 63.350829] nix-grpc-daemon[732]: ts=2026-09-20T08:17:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:555565080worker1 # [ 63.364118] nix-grpc-daemon[866]: time=2026-09-20T08:17:38.153Z level=INFO msg="All 1 paths already cached"5081client # copying 1 paths to 'lb:50051'...5082client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5083worker1 # [ 63.370831] nix-grpc-daemon[732]: ts=2026-09-20T08:17:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:555565084worker1 # [ 63.379633] nix-daemon[841]: accepted connection from pid 732, user nix-grpc-daemon (trusted)5085worker1 # [ 63.521378] nix-daemon[841]: accepted connection from pid 991, user nix-grpc-daemon (trusted)5086worker1 # [ 63.534479] nix-daemon[841]: reaped child process 994, status = succeeded5087worker1 # [ 63.671930] nix-daemon[841]: accepted connection from pid 996, user nix-grpc-daemon (trusted)5088worker1 # [ 63.684290] nix-daemon[841]: reaped child process 999, status = succeeded5089lb # [ 63.699366] niks3-server[1039]: 2026/09/20 08:17:38 INFO Received uploads request method=POST path=/api/pending_closures5090worker1 # [ 63.707113] nix-grpc-daemon[866]: time=2026-09-20T08:17:38.497Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5091worker1 # [ 63.708469] nix-grpc-daemon[866]: time=2026-09-20T08:17:38.497Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5092lb # [ 63.727160] niks3-server[1039]: 2026/09/20 08:17:38 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5093lb # [ 63.736334] niks3-server[1039]: 2026/09/20 08:17:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5094lb # [ 63.743502] niks3-server[1039]: 2026/09/20 08:17:38 INFO Signed narinfos id=21 count=15095worker1 # [ 63.745539] nix-grpc-daemon[866]: time=2026-09-20T08:17:38.535Z level=INFO msg="Uploading 1 narinfos"5096lb # [ 63.748409] niks3-server[1039]: 2026/09/20 08:17:38 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5097lb # [ 63.759913] niks3-server[1039]: 2026/09/20 08:17:38 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5098worker1 # [ 63.765767] nix-grpc-daemon[866]: time=2026-09-20T08:17:38.556Z level=INFO msg="Upload complete. (372ms)"5099lb # [ 63.766539] niks3-server[1039]: 2026/09/20 08:17:38 INFO Completed upload id=215100worker1 # [ 63.767891] nix-grpc-daemon[732]: ts=2026-09-20T08:17:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55556 duration_s=0 paths=1 nar_bytes_in=5605101lb # [ 63.770116] niks3-server[1039]: 2026/09/20 08:17:38 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5102worker1 # [ 63.773405] nix-grpc-daemon[732]: ts=2026-09-20T08:17:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555565103worker1 # [ 63.778378] nix-daemon[841]: accepted connection from pid 732, user nix-grpc-daemon (trusted)5104worker1 # [ 63.780510] nix-daemon[841]: reaped child process 989, status = succeeded5105worker1 # [ 63.785894] nix-daemon[841]: accepted connection from pid 732, user nix-grpc-daemon (trusted)5106client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5107client # uses-input-w1onlyc> node-a: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5108worker1 # [ 65.212919] nix-daemon[841]: reaped child process 1003, status = succeeded5109worker1 # [ 65.366091] nix-daemon[841]: accepted connection from pid 1018, user nix-grpc-daemon (trusted)5110worker1 # [ 65.377747] nix-daemon[841]: reaped child process 1021, status = succeeded5111lb # [ 65.394674] niks3-server[1039]: 2026/09/20 08:17:40 INFO Received uploads request method=POST path=/api/pending_closures5112worker1 # [ 65.403426] nix-grpc-daemon[866]: time=2026-09-20T08:17:40.193Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5113lb # [ 65.430231] niks3-server[1039]: 2026/09/20 08:17:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5114worker1 # [ 65.437633] nix-grpc-daemon[866]: time=2026-09-20T08:17:40.227Z level=INFO msg="Uploading 1 narinfos"5115lb # [ 65.438473] niks3-server[1039]: 2026/09/20 08:17:40 INFO Signed narinfos id=22 count=15116lb # [ 65.443644] niks3-server[1039]: 2026/09/20 08:17:40 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5117lb # [ 65.449743] niks3-server[1039]: 2026/09/20 08:17:40 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5118lb # [ 65.458049] niks3-server[1039]: 2026/09/20 08:17:40 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5119lb # [ 65.463907] niks3-server[1039]: 2026/09/20 08:17:40 INFO Completed upload id=225120worker1 # [ 65.462973] nix-grpc-daemon[866]: time=2026-09-20T08:17:40.253Z level=WARN msg="Failed to register uploaded object" key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo error="server returned 404: object is not pending upload\n"5121worker1 # [ 65.472510] nix-grpc-daemon[866]: time=2026-09-20T08:17:40.260Z level=INFO msg="Upload complete. (276ms)"5122client # copying 1 paths...5123worker1 # [ 65.475764] nix-grpc-daemon[732]: ts=2026-09-20T08:17:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55556 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=10 outputs=15124worker1 # [ 65.482964] nix-grpc-daemon[732]: ts=2026-09-20T08:17:40Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:555565125client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5126worker1 # [ 65.491117] nix-daemon[841]: reaped child process 1001, status = succeeded5127worker2 # [ 65.521519] nix-grpc-daemon[784]: ts=2026-09-20T08:17:40Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:531325128worker2 # [ 65.533162] nix-grpc-daemon[784]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5129worker2 # [ 65.548071] nix-grpc-daemon[784]: ts=2026-09-20T08:17:40Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53132 duration_s=0 paths=1 nar_bytes_out=1205130client # [ 65.613337] nix-daemon[921]: reaped child process 1046, status = succeeded5131client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.65 seconds)5132(finished: subtest: an input only one worker has still reaches the builder, in 10.63 seconds)5133subtest: upload whose reference the worker lacks is completed from the cache5134client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache5135client # this derivation will be built:5136client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5137client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5138client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache, in 0.33 seconds)5139client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache5140client # this derivation will be built:5141client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5142client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5143client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache, in 0.33 seconds)5144client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5145client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.14 seconds)5146worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5147worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.17 seconds)5148worker2 # [ 66.682053] systemd[1]: Stopping Nix worker-protocol over gRPC...5149worker2 # [ 66.753811] nix-grpc-daemon[784]: ts=2026-09-20T08:17:41Z level=info event=signal_exit5150worker2 # [ 67.002492] nix-grpc-daemon[784]: ts=2026-09-20T08:17:42Z level=info event=scheduler_yield5151worker1 # [ 66.985170] nix-grpc-daemon[732]: ts=2026-09-20T08:17:41Z level=info event=scheduler_restarting addr=lb:500515152worker2 # [ 67.007969] nix-grpc-daemon[784]: ts=2026-09-20T08:17:42Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:423565153lb # [ 66.991126] niks3-server[1039]: 2026/09/20 08:17:41 INFO lead: released remote=192.168.1.4:554545154worker2 # [ 67.012619] nix-grpc-daemon[784]: ts=2026-09-20T08:17:42Z level=info event=worker_gone addr=192.168.1.3:500515155worker2 # [ 67.015734] nix-grpc-daemon[784]: ts=2026-09-20T08:17:42Z level=info event=scheduler_restarting addr=lb:500515156worker2 # [ 67.018662] nix-grpc-daemon[784]: ts=2026-09-20T08:17:42Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:423565157worker2 # [ 67.021879] nix-grpc-daemon[784]: ts=2026-09-20T08:17:42Z level=info event=worker_gone addr=192.168.1.4:500515158worker1 # [ 67.185582] nix-grpc-daemon[732]: ts=2026-09-20T08:17:41Z level=info event=scheduler_connected addr=lb:500515159worker2 # [ 67.209157] nix-grpc-daemon[784]: ts=2026-09-20T08:17:42Z level=info event=scheduler_connected addr=lb:500515160worker1 # [ 67.191406] nix-grpc-daemon[732]: ts=2026-09-20T08:17:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5161worker2 # [ 67.221730] nix-grpc-daemon[784]: ts=2026-09-20T08:17:42Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5162worker1 # [ 67.392633] nix-grpc-daemon[732]: ts=2026-09-20T08:17:42Z level=info event=scheduler_connected addr=lb:500515163worker1 # [ 67.397668] nix-grpc-daemon[732]: ts=2026-09-20T08:17:42Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5164worker2 # [ 67.422341] nix-grpc-daemon[784]: ts=2026-09-20T08:17:42Z level=info event=scheduler_connected addr=lb:500515165worker2 # [ 67.425268] nix-grpc-daemon[784]: ts=2026-09-20T08:17:42Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5166lb # [ 67.446590] envoy[910]: conn peer=[2001:db8:1::1]:35914 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=329 ms=905167lb # [ 67.450370] envoy[910]: conn peer=[2001:db8:1::1]:35912 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3834 tx=1740 ms=29495168lb # [ 67.454503] envoy[910]: conn peer=[2001:db8:1::1]:35916 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3701 tx=1943 ms=33225169lb # [ 67.458603] envoy[910]: conn peer=[2001:db8:1::1]:45738 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=585170lb # [ 67.462579] envoy[910]: conn peer=[2001:db8:1::1]:45750 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3686 tx=1801 ms=22695171lb # [ 67.466313] envoy[910]: conn peer=[2001:db8:1::1]:45762 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=329 ms=805172worker1 # [ 67.598501] nix-grpc-daemon[732]: ts=2026-09-20T08:17:42Z level=info event=scheduler_connected addr=lb:500515173worker1 # [ 67.602966] nix-grpc-daemon[732]: ts=2026-09-20T08:17:42Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5174worker2 # [ 67.626440] nix-grpc-daemon[784]: ts=2026-09-20T08:17:42Z level=info event=scheduler_connected addr=lb:500515175worker2 # [ 67.631460] nix-grpc-daemon[784]: ts=2026-09-20T08:17:42Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5176worker1 # [ 67.803674] nix-grpc-daemon[732]: ts=2026-09-20T08:17:42Z level=info event=scheduler_connected addr=lb:500515177worker1 # [ 67.808391] nix-grpc-daemon[732]: ts=2026-09-20T08:17:42Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5178worker2 # [ 67.834066] nix-grpc-daemon[784]: ts=2026-09-20T08:17:42Z level=info event=scheduler_connected addr=lb:500515179worker2 # [ 67.837862] nix-grpc-daemon[784]: ts=2026-09-20T08:17:42Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5180worker2 # [ 68.003177] nix-grpc-daemon[784]: ts=2026-09-20T08:17:43Z level=info event=scheduler_restarting5181worker2 # [ 68.028774] nix-daemon[847]: reaped child process 849, status = succeeded5182worker1 # [ 68.011089] nix-grpc-daemon[732]: ts=2026-09-20T08:17:42Z level=info event=scheduler_connected addr=lb:500515183worker1 # [ 68.014820] nix-grpc-daemon[732]: ts=2026-09-20T08:17:42Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5184worker2 # [ 68.049318] nix-grpc-daemon[784]: ts=2026-09-20T08:17:43Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5185worker1 # [ 68.215623] nix-grpc-daemon[732]: ts=2026-09-20T08:17:43Z level=info event=scheduler_connected addr=lb:500515186worker2 # [ 68.239183] nix-grpc-daemon[784]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5187worker2 # [ 68.242469] nix-grpc-daemon[784]: E0000 00:00:1789892263.373930 827 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 10, started true, shutdown true5188worker2 # [ 68.876692] nix-grpc-daemon[784]: E0000 00:00:1789892264.011254 827 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5189worker2 # [ 68.881659] nix-grpc-daemon[784]: E0000 00:00:1789892264.011372 827 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 8, started true, shutdown true5190worker1 # [ 69.292592] nix-grpc-daemon[732]: ts=2026-09-20T08:17:44Z level=info event=scheduler_take_over5191lb # [ 69.293625] niks3-server[1039]: 2026/09/20 08:17:44 INFO lead: acquired remote=192.168.1.3:329625192worker1 # [ 69.874280] nix-grpc-daemon[732]: ts=2026-09-20T08:17:44Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: remote connection failure"5193worker2 # [ 69.912533] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5194worker2 # [ 69.916880] systemd[1]: Stopped Nix worker-protocol over gRPC.5195worker2 # [ 69.920533] systemd[1]: nix-grpc-daemon.service: Consumed 2.330s CPU time over 52.450s wall clock time, 60.1M memory peak, 84K incoming IP traffic, 165.3K outgoing IP traffic.5196worker2 # [ 69.930796] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5197worker2 # [ 69.935089] systemd[1]: Closed Nix worker-protocol over gRPC.5198worker2 # [ 69.937461] systemd[1]: nix-grpc-daemon.socket: Consumed 291us CPU time over 1min 634ms wall clock time, 256K memory peak, 111.8K incoming IP traffic, 87K outgoing IP traffic.5199lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5200lb: must succeed: curl -sf localhost:9901/clusters5201lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5202lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.07 seconds)5203client: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25204worker1 # [ 70.075432] nix-grpc-daemon[732]: ts=2026-09-20T08:17:44Z level=info event=scheduler_connected addr=lb:500515205worker1 # [ 70.080457] nix-grpc-daemon[732]: ts=2026-09-20T08:17:44Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05206worker1 # [ 70.205604] nix-grpc-daemon[732]: ts=2026-09-20T08:17:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:555565207worker1 # [ 70.208570] nix-grpc-daemon[732]: ts=2026-09-20T08:17:44Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:555565208client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5209client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.24 seconds)5210worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5211worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.03 seconds)5212worker2 # [ 70.334341] systemd[1]: Listening on Nix worker-protocol over gRPC.5213worker1 # [ 70.352298] systemd[1]: Stopping Nix worker-protocol over gRPC...5214worker1 # [ 70.687978] nix-grpc-daemon[732]: ts=2026-09-20T08:17:45Z level=info event=signal_exit5215worker2 # [ 70.900231] systemd[1]: Starting Nix worker-protocol over gRPC...5216worker1 # [ 70.941329] nix-grpc-daemon[732]: ts=2026-09-20T08:17:45Z level=info event=scheduler_yield5217lb # [ 70.947357] niks3-server[1039]: 2026/09/20 08:17:45 INFO lead: released remote=192.168.1.3:329625218worker1 # [ 70.946969] nix-grpc-daemon[732]: ts=2026-09-20T08:17:45Z level=info event=scheduler_restarting addr=lb:500515219worker1 # [ 70.950140] nix-grpc-daemon[732]: ts=2026-09-20T08:17:45Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:500085220worker1 # [ 70.953430] nix-grpc-daemon[732]: ts=2026-09-20T08:17:45Z level=info event=worker_gone addr=192.168.1.3:500515221worker2 # [ 71.085937] nix-grpc-daemon[1211]: ts=2026-09-20T08:17:46Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5222worker2 # [ 71.089860] nix-grpc-daemon[1211]: ts=2026-09-20T08:17:46Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15223worker2 # [ 71.095885] nix-grpc-daemon[1211]: ts=2026-09-20T08:17:46Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5224worker2 # [ 71.097704] systemd[1]: Started Nix worker-protocol over gRPC.5225worker2 # [ 71.109526] nix-grpc-daemon[1211]: ts=2026-09-20T08:17:46Z level=info event=scheduler_take_over5226lb # [ 71.089930] niks3-server[1039]: 2026/09/20 08:17:46 INFO lead: acquired remote=192.168.1.4:539065227worker2 # [ 71.128548] nix-grpc-daemon[1211]: ts=2026-09-20T08:17:46Z level=info event=scheduler_connected addr=lb:500515228worker2 # [ 71.135939] nix-grpc-daemon[1211]: ts=2026-09-20T08:17:46Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5229worker1 # [ 71.145377] nix-grpc-daemon[732]: ts=2026-09-20T08:17:45Z level=info event=scheduler_connected addr=lb:500515230worker2 # [ 71.172266] nix-grpc-daemon[1211]: ts=2026-09-20T08:17:46Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05231worker2 # [ 71.336967] nix-grpc-daemon[1211]: ts=2026-09-20T08:17:46Z level=info event=scheduler_connected addr=lb:500515232worker2 # [ 71.341294] nix-grpc-daemon[1211]: ts=2026-09-20T08:17:46Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5233worker2 # [ 71.542198] nix-grpc-daemon[1211]: ts=2026-09-20T08:17:46Z level=info event=scheduler_connected addr=lb:500515234worker2 # [ 71.547119] nix-grpc-daemon[1211]: ts=2026-09-20T08:17:46Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05235worker1 # [ 71.946165] nix-grpc-daemon[732]: ts=2026-09-20T08:17:46Z level=info event=scheduler_restarting5236worker2 # [ 71.972064] nix-grpc-daemon[1211]: ts=2026-09-20T08:17:47Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:599785237worker1 # [ 71.956716] nix-grpc-daemon[732]: ts=2026-09-20T08:17:46Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5238worker2 # [ 71.975482] nix-grpc-daemon[1211]: ts=2026-09-20T08:17:47Z level=info event=worker_gone addr=192.168.1.3:500515239worker1 # [ 71.968707] nix-daemon[841]: reaped child process 843, status = succeeded5240worker1 # [ 72.807212] nix-grpc-daemon[732]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5241worker1 # [ 72.808678] nix-grpc-daemon[732]: E0000 00:00:1789892267.597154 826 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 10, started true, shutdown true5242worker1 # [ 72.810570] nix-grpc-daemon[732]: E0000 00:00:1789892267.597497 826 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5243worker1 # [ 73.831366] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5244worker1 # [ 73.833865] systemd[1]: Stopped Nix worker-protocol over gRPC.5245worker1 # [ 73.836504] systemd[1]: nix-grpc-daemon.service: Consumed 1.452s CPU time over 56.387s wall clock time, 50.5M memory peak, 51.7K incoming IP traffic, 93.8K outgoing IP traffic.5246worker1 # [ 73.847249] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5247worker1 # [ 73.851491] systemd[1]: Closed Nix worker-protocol over gRPC.5248worker1 # [ 73.855175] systemd[1]: nix-grpc-daemon.socket: Consumed 286us CPU time over 1min 4.262s wall clock time, 256K memory peak, 93.6K incoming IP traffic, 79.6K outgoing IP traffic.5249lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5250lb: must succeed: curl -sf localhost:9901/clusters5251lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5252lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.06 seconds)5253client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25254worker2 # [ 74.163289] nix-grpc-daemon[1211]: ts=2026-09-20T08:17:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:495705255worker2 # [ 74.175103] nix-daemon[847]: accepted connection from pid 1211, user nix-grpc-daemon (trusted)5256worker2 # [ 74.191416] nix-grpc-daemon[1211]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 4 ms (attempt 1/5)5257worker2 # [ 74.201223] nix-grpc-daemon[1211]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 111 ms (attempt 2/5)5258worker2 # [ 74.318148] nix-grpc-daemon[1211]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 129 ms (attempt 3/5)5259worker2 # [ 74.453243] nix-grpc-daemon[1211]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 163 ms (attempt 4/5)5260worker2 # [ 74.622550] nix-grpc-daemon[1211]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5261client # copying 2 paths...5262client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5263worker2 # [ 74.645471] nix-grpc-daemon[1211]: ts=2026-09-20T08:17:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:495705264client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5265worker2 # [ 74.658102] nix-daemon[847]: accepted connection from pid 1211, user nix-grpc-daemon (trusted)5266worker2 # [ 74.677674] nix-grpc-daemon[1211]: ts=2026-09-20T08:17:49Z level=info event=niks3_push_started pid=12675267worker2 # [ 75.018912] nix-daemon[847]: accepted connection from pid 1280, user nix-grpc-daemon (trusted)5268worker2 # [ 75.031234] nix-daemon[847]: reaped child process 1283, status = succeeded5269lb # [ 75.027760] niks3-server[1039]: 2026/09/20 08:17:49 INFO Received uploads request method=POST path=/api/pending_closures5270lb # [ 75.036941] niks3-server[1039]: 2026/09/20 08:17:49 INFO Received uploads request method=POST path=/api/pending_closures5271worker2 # [ 75.064319] nix-grpc-daemon[1267]: time=2026-09-20T08:17:50.198Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5272worker2 # [ 75.065678] nix-grpc-daemon[1267]: time=2026-09-20T08:17:50.198Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5273worker2 # [ 75.069384] nix-grpc-daemon[1267]: time=2026-09-20T08:17:50.198Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5274lb # [ 75.088141] niks3-server[1039]: 2026/09/20 08:17:50 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5275lb # [ 75.111460] niks3-server[1039]: 2026/09/20 08:17:50 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5276lb # [ 75.115058] niks3-server[1039]: 2026/09/20 08:17:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5277lb # [ 75.118541] niks3-server[1039]: 2026/09/20 08:17:50 INFO Signed narinfos id=23 count=15278lb # [ 75.120241] niks3-server[1039]: 2026/09/20 08:17:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5279lb # [ 75.123571] niks3-server[1039]: 2026/09/20 08:17:50 INFO Signed narinfos id=24 count=25280lb # [ 75.124702] niks3-server[1039]: 2026/09/20 08:17:50 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5281worker2 # [ 75.144130] nix-grpc-daemon[1267]: time=2026-09-20T08:17:50.278Z level=INFO msg="Uploading 3 narinfos"5282lb # [ 75.143495] niks3-server[1039]: 2026/09/20 08:17:50 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5283lb # [ 75.146251] niks3-server[1039]: 2026/09/20 08:17:50 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5284lb # [ 75.151414] niks3-server[1039]: 2026/09/20 08:17:50 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5285lb # [ 75.153538] niks3-server[1039]: 2026/09/20 08:17:50 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5286lb # [ 75.158338] niks3-server[1039]: 2026/09/20 08:17:50 INFO Completed upload id=235287lb # [ 75.159586] niks3-server[1039]: 2026/09/20 08:17:50 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5288lb # [ 75.162069] niks3-server[1039]: 2026/09/20 08:17:50 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5289lb # [ 75.164054] niks3-server[1039]: 2026/09/20 08:17:50 INFO Completed upload id=245290worker2 # [ 75.184649] nix-grpc-daemon[1267]: time=2026-09-20T08:17:50.319Z level=INFO msg="Upload complete. (322ms)"5291worker2 # [ 75.190603] nix-grpc-daemon[1211]: ts=2026-09-20T08:17:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49570 duration_s=0 paths=2 nar_bytes_in=3125292worker2 # [ 75.203216] nix-daemon[847]: reaped child process 1265, status = succeeded5293client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.28 seconds)5294worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5295worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.03 seconds)5296worker1 # [ 75.293579] systemd[1]: Listening on Nix worker-protocol over gRPC.5297worker1 # [ 75.301086] systemd[1]: Starting Nix worker-protocol over gRPC...5298worker1 # [ 75.446112] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:50Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5299worker1 # [ 75.449102] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:50Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15300worker1 # [ 75.454945] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:50Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5301worker1 # [ 75.458269] systemd[1]: Started Nix worker-protocol over gRPC.5302lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5303lb: must succeed: curl -sf localhost:9901/clusters5304worker1 # [ 75.486180] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:50Z level=info event=scheduler_connected addr=lb:500515305worker2 # [ 75.509192] nix-grpc-daemon[1211]: ts=2026-09-20T08:17:50Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05306lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5307lb: must succeed: curl -sf localhost:9901/clusters5308lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5309lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.15 seconds)5310(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.00 seconds)5311subtest: scheduler down: the other node takes the lock and keeps it5312lb: must succeed: curl -sf localhost:9901/clusters5313lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5314lb: must succeed: curl -sf localhost:9901/clusters5315lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5316lb: must succeed: curl -sf localhost:9901/clusters5317lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5318lb: must succeed: curl -sf localhost:9901/clusters5319lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5320worker2 # [ 76.839784] systemd[1]: Stopping Nix worker-protocol over gRPC...5321worker2 # [ 77.101602] nix-grpc-daemon[1211]: ts=2026-09-20T08:17:52Z level=info event=signal_exit5322worker2 # [ 77.313361] nix-grpc-daemon[1211]: ts=2026-09-20T08:17:52Z level=info event=scheduler_yield5323lb # [ 77.297759] niks3-server[1039]: 2026/09/20 08:17:52 INFO lead: released remote=192.168.1.4:539065324worker1 # [ 77.296685] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:52Z level=info event=scheduler_restarting addr=lb:500515325worker2 # [ 77.320386] nix-grpc-daemon[1211]: ts=2026-09-20T08:17:52Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:599785326worker2 # [ 77.326594] nix-grpc-daemon[1211]: ts=2026-09-20T08:17:52Z level=info event=worker_gone addr=192.168.1.3:500515327worker2 # [ 77.330145] nix-grpc-daemon[1211]: ts=2026-09-20T08:17:52Z level=info event=scheduler_restarting addr=lb:500515328worker2 # [ 77.333521] nix-grpc-daemon[1211]: ts=2026-09-20T08:17:52Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:599785329worker2 # [ 77.337305] nix-grpc-daemon[1211]: ts=2026-09-20T08:17:52Z level=info event=worker_gone addr=192.168.1.4:500515330lb # [ 77.446597] envoy[910]: conn peer=[2001:db8:1::4]:56028 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=13556 tx=2819 ms=502615331lb # [ 77.450546] envoy[910]: conn peer=[2001:db8:1::1]:45770 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=145332lb # [ 77.454103] envoy[910]: conn peer=192.168.1.3:48312 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=12990 tx=2561 ms=533315333lb # [ 77.457673] envoy[910]: conn peer=[2001:db8:1::1]:59672 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1005 tx=282 ms=10415334worker1 # [ 77.496968] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:52Z level=info event=scheduler_connected addr=lb:500515335worker1 # [ 77.504824] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:52Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5336worker2 # [ 77.526216] nix-grpc-daemon[1211]: ts=2026-09-20T08:17:52Z level=info event=scheduler_connected addr=lb:500515337worker2 # [ 77.537727] nix-grpc-daemon[1211]: ts=2026-09-20T08:17:52Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5338worker1 # [ 77.705860] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:52Z level=info event=scheduler_connected addr=lb:500515339worker1 # [ 77.711291] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:52Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5340worker2 # [ 77.738664] nix-grpc-daemon[1211]: ts=2026-09-20T08:17:52Z level=info event=scheduler_connected addr=lb:500515341worker2 # [ 77.744226] nix-grpc-daemon[1211]: ts=2026-09-20T08:17:52Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5342worker1 # [ 77.912226] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:52Z level=info event=scheduler_connected addr=lb:500515343worker1 # [ 77.916987] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:52Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5344worker2 # [ 77.945150] nix-grpc-daemon[1211]: ts=2026-09-20T08:17:53Z level=info event=scheduler_connected addr=lb:500515345worker2 # [ 77.950333] nix-grpc-daemon[1211]: ts=2026-09-20T08:17:53Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5346worker1 # [ 78.118131] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:52Z level=info event=scheduler_connected addr=lb:500515347worker1 # [ 78.123373] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:52Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5348worker2 # [ 78.151765] nix-grpc-daemon[1211]: ts=2026-09-20T08:17:53Z level=info event=scheduler_connected addr=lb:500515349worker2 # [ 78.156581] nix-grpc-daemon[1211]: ts=2026-09-20T08:17:53Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5350worker2 # [ 78.317872] nix-grpc-daemon[1211]: ts=2026-09-20T08:17:53Z level=info event=scheduler_restarting5351worker2 # [ 78.338236] nix-daemon[847]: reaped child process 1260, status = succeeded5352worker1 # [ 78.324977] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:53Z level=info event=scheduler_connected addr=lb:500515353worker1 # [ 78.329460] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:53Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5354worker2 # [ 78.367881] nix-grpc-daemon[1211]: ts=2026-09-20T08:17:53Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5355worker1 # [ 78.530377] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:53Z level=info event=scheduler_connected addr=lb:500515356worker2 # [ 78.552656] nix-grpc-daemon[1211]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5357worker2 # [ 78.554115] nix-grpc-daemon[1211]: E0000 00:00:1789892273.687969 1254 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 10, started true, shutdown true5358worker1 # [ 79.084174] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:53Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: remote connection failure"5359worker2 # [ 79.120725] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5360worker2 # [ 79.124770] systemd[1]: Stopped Nix worker-protocol over gRPC.5361worker2 # [ 79.127399] systemd[1]: nix-grpc-daemon.service: Consumed 363ms CPU time over 8.220s wall clock time, 32.8M memory peak, 18K incoming IP traffic, 23.1K outgoing IP traffic.5362worker2 # [ 79.139251] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5363worker2 # [ 79.143437] systemd[1]: Closed Nix worker-protocol over gRPC.5364worker2 # [ 79.146431] systemd[1]: nix-grpc-daemon.socket: Consumed 285us CPU time over 8.814s wall clock time, 256K memory peak, 27.3K incoming IP traffic, 23.3K outgoing IP traffic.5365worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5366worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.06 seconds)5367worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5368worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.04 seconds)5369worker1 # [ 79.284953] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:54Z level=info event=scheduler_connected addr=lb:500515370worker1 # [ 79.288229] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:54Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5371worker1 # [ 79.489869] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:54Z level=info event=scheduler_connected addr=lb:500515372worker1 # [ 79.493919] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:54Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: remote connection failure"5373worker1 # [ 79.694803] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:54Z level=info event=scheduler_connected addr=lb:500515374worker1 # [ 79.699841] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:54Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5375worker1 # [ 79.900910] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:54Z level=info event=scheduler_connected addr=lb:500515376worker1 # [ 79.905125] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:54Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: remote connection failure"5377worker1 # [ 80.106068] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:54Z level=info event=scheduler_connected addr=lb:500515378worker1 # [ 80.110988] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:54Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5379worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5380worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.04 seconds)5381worker1 # [ 80.311520] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:55Z level=info event=scheduler_connected addr=lb:500515382worker1 # [ 80.313755] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:55Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: remote connection failure"5383worker1 # [ 80.464625] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:55Z level=info event=scheduler_take_over5384lb # [ 80.465848] niks3-server[1039]: 2026/09/20 08:17:55 INFO lead: acquired remote=192.168.1.3:537225385worker1 # [ 80.514684] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:55Z level=info event=scheduler_connected addr=lb:500515386worker1 # [ 80.516539] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:55Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05387worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5388worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.03 seconds)5389client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag on-standby >&25390worker1 # [ 81.528386] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:56Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:494285391worker1 # [ 81.539438] nix-daemon[841]: accepted connection from pid 1044, user nix-grpc-daemon (trusted)5392client # don't know how to build these paths:5393client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5394worker1 # [ 81.558935] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:56Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05395worker1 # [ 81.563363] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:56Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=15396worker1 # [ 81.607530] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494285397worker1 # [ 81.618691] nix-grpc-daemon[1044]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 64 ms (attempt 1/5)5398worker1 # [ 81.688516] nix-grpc-daemon[1044]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 147 ms (attempt 2/5)5399worker1 # [ 81.841388] nix-grpc-daemon[1044]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 342 ms (attempt 3/5)5400worker1 # [ 82.189344] nix-grpc-daemon[1044]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 60 ms (attempt 4/5)5401worker1 # [ 82.255627] nix-grpc-daemon[1044]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5402worker1 # [ 82.274345] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:494285403client # copying 1 paths to 'lb:50051'...5404client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5405worker1 # [ 82.287353] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:494285406worker1 # [ 82.297156] nix-daemon[841]: accepted connection from pid 1044, user nix-grpc-daemon (trusted)5407worker1 # [ 82.316386] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:57Z level=info event=niks3_push_started pid=11235408worker1 # [ 82.638416] nix-daemon[841]: accepted connection from pid 1136, user nix-grpc-daemon (trusted)5409worker1 # [ 82.650286] nix-daemon[841]: reaped child process 1139, status = succeeded5410worker1 # [ 82.796310] nix-daemon[841]: accepted connection from pid 1141, user nix-grpc-daemon (trusted)5411worker1 # [ 82.808480] nix-daemon[841]: reaped child process 1144, status = succeeded5412lb # [ 82.822779] niks3-server[1039]: 2026/09/20 08:17:57 INFO Received uploads request method=POST path=/api/pending_closures5413worker1 # [ 82.833349] nix-grpc-daemon[1123]: time=2026-09-20T08:17:57.623Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5414worker1 # [ 82.835423] nix-grpc-daemon[1123]: time=2026-09-20T08:17:57.625Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5415lb # [ 82.863717] niks3-server[1039]: 2026/09/20 08:17:57 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5416lb # [ 82.874783] niks3-server[1039]: 2026/09/20 08:17:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5417lb # [ 82.881141] niks3-server[1039]: 2026/09/20 08:17:57 INFO Signed narinfos id=25 count=15418worker1 # [ 82.884298] nix-grpc-daemon[1123]: time=2026-09-20T08:17:57.674Z level=INFO msg="Uploading 1 narinfos"5419lb # [ 82.892557] niks3-server[1039]: 2026/09/20 08:17:57 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5420lb # [ 82.900568] niks3-server[1039]: 2026/09/20 08:17:57 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5421lb # [ 82.909364] niks3-server[1039]: 2026/09/20 08:17:57 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5422lb # [ 82.912612] niks3-server[1039]: 2026/09/20 08:17:57 INFO Completed upload id=255423worker1 # [ 82.912069] nix-grpc-daemon[1123]: time=2026-09-20T08:17:57.702Z level=INFO msg="Upload complete. (417ms)"5424worker1 # [ 82.918471] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49428 duration_s=0 paths=1 nar_bytes_in=5205425worker1 # [ 82.924460] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494285426worker1 # [ 82.933203] nix-daemon[841]: accepted connection from pid 1044, user nix-grpc-daemon (trusted)5427worker1 # [ 82.941277] nix-daemon[841]: reaped child process 1121, status = succeeded5428client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5429worker1 # [ 82.946469] nix-daemon[841]: accepted connection from pid 1044, user nix-grpc-daemon (trusted)5430client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5431client # farm-a> LOG-farm-a-on-standby5432worker1 # [ 83.971457] nix-daemon[841]: reaped child process 1148, status = succeeded5433worker1 # [ 84.107334] nix-daemon[841]: accepted connection from pid 1163, user nix-grpc-daemon (trusted)5434worker1 # [ 84.118045] nix-daemon[841]: reaped child process 1166, status = succeeded5435lb # [ 84.132315] niks3-server[1039]: 2026/09/20 08:17:59 INFO Received uploads request method=POST path=/api/pending_closures5436worker1 # [ 84.142752] nix-grpc-daemon[1123]: time=2026-09-20T08:17:58.932Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5437worker1 # [ 84.144325] nix-grpc-daemon[1123]: time=2026-09-20T08:17:58.934Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5438lb # [ 84.169337] niks3-server[1039]: 2026/09/20 08:17:59 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5439lb # [ 84.183311] niks3-server[1039]: 2026/09/20 08:17:59 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5440lb # [ 84.192917] niks3-server[1039]: 2026/09/20 08:17:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5441lb # [ 84.198936] niks3-server[1039]: 2026/09/20 08:17:59 INFO Signed narinfos id=26 count=15442worker1 # [ 84.198822] nix-grpc-daemon[1123]: time=2026-09-20T08:17:58.988Z level=INFO msg="Uploading 1 narinfos"5443lb # [ 84.202983] niks3-server[1039]: 2026/09/20 08:17:59 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5444lb # [ 84.211726] niks3-server[1039]: 2026/09/20 08:17:59 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5445lb # [ 84.217801] niks3-server[1039]: 2026/09/20 08:17:59 INFO Completed upload id=265446worker1 # [ 84.216984] nix-grpc-daemon[1123]: time=2026-09-20T08:17:59.007Z level=INFO msg="Upload complete. (263ms)"5447lb # [ 84.219099] niks3-server[1039]: 2026/09/20 08:17:59 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5448worker1 # [ 84.229158] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49428 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15449worker1 # [ 84.241867] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:59Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05450worker1 # [ 84.244324] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:59Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=15451worker1 # [ 84.247976] nix-daemon[841]: reaped child process 1146, status = succeeded5452worker1 # [ 84.286444] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494285453worker1 # [ 84.299742] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:494285454worker1 # [ 84.313279] nix-grpc-daemon[1123]: time=2026-09-20T08:17:59.103Z level=INFO msg="All 1 paths already cached"5455client # copying 1 paths to 'lb:50051'...5456client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5457worker1 # [ 84.321299] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:494285458worker1 # [ 84.329867] nix-daemon[841]: accepted connection from pid 1044, user nix-grpc-daemon (trusted)5459worker1 # [ 84.476597] nix-daemon[841]: accepted connection from pid 1172, user nix-grpc-daemon (trusted)5460worker1 # [ 84.488801] nix-daemon[841]: reaped child process 1175, status = succeeded5461worker1 # [ 84.626303] nix-daemon[841]: accepted connection from pid 1177, user nix-grpc-daemon (trusted)5462worker1 # [ 84.638100] nix-daemon[841]: reaped child process 1180, status = succeeded5463lb # [ 84.652308] niks3-server[1039]: 2026/09/20 08:17:59 INFO Received uploads request method=POST path=/api/pending_closures5464worker1 # [ 84.660197] nix-grpc-daemon[1123]: time=2026-09-20T08:17:59.450Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5465worker1 # [ 84.662249] nix-grpc-daemon[1123]: time=2026-09-20T08:17:59.452Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5466lb # [ 84.680534] niks3-server[1039]: 2026/09/20 08:17:59 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5467lb # [ 84.692974] niks3-server[1039]: 2026/09/20 08:17:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5468lb # [ 84.698513] niks3-server[1039]: 2026/09/20 08:17:59 INFO Signed narinfos id=27 count=15469worker1 # [ 84.700682] nix-grpc-daemon[1123]: time=2026-09-20T08:17:59.490Z level=INFO msg="Uploading 1 narinfos"5470lb # [ 84.705643] niks3-server[1039]: 2026/09/20 08:17:59 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5471lb # [ 84.717294] niks3-server[1039]: 2026/09/20 08:17:59 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5472lb # [ 84.722131] niks3-server[1039]: 2026/09/20 08:17:59 INFO Completed upload id=275473worker1 # [ 84.723705] nix-grpc-daemon[1123]: time=2026-09-20T08:17:59.513Z level=INFO msg="Upload complete. (381ms)"5474lb # [ 84.725321] niks3-server[1039]: 2026/09/20 08:17:59 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5475worker1 # [ 84.726190] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49428 duration_s=0 paths=1 nar_bytes_in=6405476worker1 # [ 84.732190] nix-grpc-daemon[1044]: ts=2026-09-20T08:17:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494285477worker1 # [ 84.740479] nix-daemon[841]: accepted connection from pid 1044, user nix-grpc-daemon (trusted)5478worker1 # [ 84.747459] nix-daemon[841]: reaped child process 1170, status = succeeded5479client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5480worker1 # [ 84.753063] nix-daemon[841]: accepted connection from pid 1044, user nix-grpc-daemon (trusted)5481client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5482client # farm-b> LOG-farm-b-on-standby5483worker1 # [ 85.375371] nix-daemon[841]: reaped child process 1184, status = succeeded5484worker1 # [ 85.512865] nix-daemon[841]: accepted connection from pid 1199, user nix-grpc-daemon (trusted)5485worker1 # [ 85.524327] nix-daemon[841]: reaped child process 1202, status = succeeded5486lb # [ 85.542610] niks3-server[1039]: 2026/09/20 08:18:00 INFO Received uploads request method=POST path=/api/pending_closures5487worker1 # [ 85.550240] nix-grpc-daemon[1123]: time=2026-09-20T08:18:00.340Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5488worker1 # [ 85.551918] nix-grpc-daemon[1123]: time=2026-09-20T08:18:00.342Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"5489lb # [ 85.576500] niks3-server[1039]: 2026/09/20 08:18:00 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5490lb # [ 85.588075] niks3-server[1039]: 2026/09/20 08:18:00 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst5491lb # [ 85.597733] niks3-server[1039]: 2026/09/20 08:18:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5492worker1 # [ 85.601429] nix-grpc-daemon[1123]: time=2026-09-20T08:18:00.391Z level=INFO msg="Uploading 1 narinfos"5493lb # [ 85.602559] niks3-server[1039]: 2026/09/20 08:18:00 INFO Signed narinfos id=28 count=15494lb # [ 85.606059] niks3-server[1039]: 2026/09/20 08:18:00 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls5495lb # [ 85.616383] niks3-server[1039]: 2026/09/20 08:18:00 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5496lb # [ 85.621408] niks3-server[1039]: 2026/09/20 08:18:00 INFO Completed upload id=285497lb # [ 85.623073] niks3-server[1039]: 2026/09/20 08:18:00 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo5498worker1 # [ 85.622062] nix-grpc-daemon[1123]: time=2026-09-20T08:18:00.412Z level=INFO msg="Upload complete. (262ms)"5499worker1 # [ 85.632289] nix-grpc-daemon[1044]: ts=2026-09-20T08:18:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49428 duration_s=0 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=15500worker1 # [ 85.640460] nix-grpc-daemon[1044]: ts=2026-09-20T08:18:00Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=05501worker1 # [ 85.644324] nix-grpc-daemon[1044]: ts=2026-09-20T08:18:00Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=15502worker1 # [ 85.653182] nix-daemon[841]: reaped child process 1182, status = succeeded5503worker1 # [ 85.685406] nix-grpc-daemon[1044]: ts=2026-09-20T08:18:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494285504worker1 # [ 85.697941] nix-grpc-daemon[1044]: ts=2026-09-20T08:18:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:494285505worker1 # [ 85.710064] nix-grpc-daemon[1123]: time=2026-09-20T08:18:00.500Z level=INFO msg="All 2 paths already cached"5506client # copying 1 paths to 'lb:50051'...5507client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...5508worker1 # [ 85.717894] nix-grpc-daemon[1044]: ts=2026-09-20T08:18:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:494285509worker1 # [ 85.727172] nix-daemon[841]: accepted connection from pid 1044, user nix-grpc-daemon (trusted)5510worker1 # [ 85.890612] nix-daemon[841]: accepted connection from pid 1208, user nix-grpc-daemon (trusted)5511worker1 # [ 85.902968] nix-daemon[841]: reaped child process 1211, status = succeeded5512worker1 # [ 86.051990] nix-daemon[841]: accepted connection from pid 1213, user nix-grpc-daemon (trusted)5513worker1 # [ 86.063769] nix-daemon[841]: reaped child process 1216, status = succeeded5514lb # [ 86.078168] niks3-server[1039]: 2026/09/20 08:18:01 INFO Received uploads request method=POST path=/api/pending_closures5515worker1 # [ 86.085282] nix-grpc-daemon[1123]: time=2026-09-20T08:18:00.875Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5516worker1 # [ 86.086794] nix-grpc-daemon[1123]: time=2026-09-20T08:18:00.877Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"5517lb # [ 86.108254] niks3-server[1039]: 2026/09/20 08:18:01 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst5518lb # [ 86.116385] niks3-server[1039]: 2026/09/20 08:18:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5519lb # [ 86.120944] niks3-server[1039]: 2026/09/20 08:18:01 INFO Signed narinfos id=29 count=15520worker1 # [ 86.122828] nix-grpc-daemon[1123]: time=2026-09-20T08:18:00.913Z level=INFO msg="Uploading 1 narinfos"5521lb # [ 86.129776] niks3-server[1039]: 2026/09/20 08:18:01 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls5522lb # [ 86.142268] niks3-server[1039]: 2026/09/20 08:18:01 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5523lb # [ 86.148755] niks3-server[1039]: 2026/09/20 08:18:01 INFO Completed upload id=295524worker1 # [ 86.147774] nix-grpc-daemon[1123]: time=2026-09-20T08:18:00.938Z level=INFO msg="Upload complete. (408ms)"5525lb # [ 86.151500] niks3-server[1039]: 2026/09/20 08:18:01 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo5526worker1 # [ 86.152131] nix-grpc-daemon[1044]: ts=2026-09-20T08:18:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49428 duration_s=0 paths=1 nar_bytes_in=7685527worker1 # [ 86.156777] nix-grpc-daemon[1044]: ts=2026-09-20T08:18:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494285528worker1 # [ 86.165464] nix-daemon[841]: accepted connection from pid 1044, user nix-grpc-daemon (trusted)5529worker1 # [ 86.169705] nix-daemon[841]: reaped child process 1206, status = succeeded5530worker1 # [ 86.174575] nix-daemon[841]: accepted connection from pid 1044, user nix-grpc-daemon (trusted)5531client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...5532client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5533client # farm-top> LOG-farm-top-on-standby5534worker1 # [ 87.189119] nix-daemon[841]: reaped child process 1220, status = succeeded5535worker1 # [ 87.327361] nix-daemon[841]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)5536worker1 # [ 87.339737] nix-daemon[841]: reaped child process 1238, status = succeeded5537lb # [ 87.355334] niks3-server[1039]: 2026/09/20 08:18:02 INFO Received uploads request method=POST path=/api/pending_closures5538worker1 # [ 87.363094] nix-grpc-daemon[1123]: time=2026-09-20T08:18:02.153Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5539worker1 # [ 87.364594] nix-grpc-daemon[1123]: time=2026-09-20T08:18:02.154Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"5540lb # [ 87.390583] niks3-server[1039]: 2026/09/20 08:18:02 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst5541lb # [ 87.400666] niks3-server[1039]: 2026/09/20 08:18:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5542worker1 # [ 87.404659] nix-grpc-daemon[1123]: time=2026-09-20T08:18:02.194Z level=INFO msg="Uploading 1 narinfos"5543lb # [ 87.406126] niks3-server[1039]: 2026/09/20 08:18:02 INFO Signed narinfos id=30 count=15544lb # [ 87.415476] niks3-server[1039]: 2026/09/20 08:18:02 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5545lb # [ 87.424148] niks3-server[1039]: 2026/09/20 08:18:02 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls5546lb # [ 87.429296] niks3-server[1039]: 2026/09/20 08:18:02 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5547lb # [ 87.434553] niks3-server[1039]: 2026/09/20 08:18:02 INFO Completed upload id=305548worker1 # [ 87.433703] nix-grpc-daemon[1123]: time=2026-09-20T08:18:02.223Z level=INFO msg="Upload complete. (252ms)"5549lb # [ 87.437511] niks3-server[1039]: 2026/09/20 08:18:02 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo5550worker1 # [ 87.440453] nix-grpc-daemon[1044]: ts=2026-09-20T08:18:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49428 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=15551lb # [ 87.445977] envoy[910]: conn peer=[2001:db8:1::4]:37280 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1711 tx=441 ms=72345552lb # [ 87.447609] envoy[910]: conn peer=[2001:db8:1::1]:59674 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9464 tx=3802 ms=59285553worker1 # [ 87.450958] nix-daemon[841]: reaped child process 1218, status = succeeded5554client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag on-standby >&2, in 6.17 seconds)5555client: must succeed: nix eval --raw -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag on-standby outPath5556client: (finished: must succeed: nix eval --raw -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag on-standby outPath, in 0.19 seconds)5557worker2 # [ 87.748503] systemd[1]: Listening on Nix worker-protocol over gRPC.5558worker2 # [ 87.755390] systemd[1]: Starting Nix worker-protocol over gRPC...5559worker2 # [ 87.900979] nix-grpc-daemon[1304]: ts=2026-09-20T08:18:03Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5560worker2 # [ 87.903826] nix-grpc-daemon[1304]: ts=2026-09-20T08:18:03Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15561worker2 # [ 87.908958] nix-grpc-daemon[1304]: ts=2026-09-20T08:18:03Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5562worker2 # [ 87.911945] systemd[1]: Started Nix worker-protocol over gRPC.5563lb: must succeed: curl -sf localhost:9901/clusters5564worker2 # [ 87.940132] nix-grpc-daemon[1304]: ts=2026-09-20T08:18:03Z level=info event=scheduler_connected addr=lb:500515565worker1 # [ 87.922461] nix-grpc-daemon[1044]: ts=2026-09-20T08:18:02Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05566lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5567worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5568worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.07 seconds)5569lb: must succeed: curl -sf localhost:9901/clusters5570lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5571lb: must succeed: curl -sf localhost:9901/clusters5572lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5573worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5574worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.05 seconds)5575(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 11.56 seconds)5576subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5577client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs5578client # [ 88.229497] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs.5579client # Running as unit: rs.service; invocation ID: 58db4f2d240e47aebb954656d10647005580client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs, in 0.07 seconds)5581client # [ 88.442723] nix[1145]: don't know how to build these paths:5582client # [ 88.443680] nix[1145]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5583worker1 # [ 88.446783] nix-grpc-daemon[1044]: ts=2026-09-20T08:18:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:494285584worker1 # [ 88.457064] nix-grpc-daemon[1044]: ts=2026-09-20T08:18:03Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=05585worker1 # [ 88.463887] nix-grpc-daemon[1044]: ts=2026-09-20T08:18:03Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=15586worker1 # [ 88.469353] nix-grpc-daemon[1044]: ts=2026-09-20T08:18:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494285587worker1 # [ 88.477232] nix-grpc-daemon[1044]: ts=2026-09-20T08:18:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:494285588client # [ 88.482127] nix[1145]: copying 1 paths to 'lb:50051'...5589client # [ 88.483476] nix[1145]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...5590worker1 # [ 88.487996] nix-grpc-daemon[1044]: ts=2026-09-20T08:18:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:494285591worker1 # [ 88.499185] nix-daemon[841]: accepted connection from pid 1044, user nix-grpc-daemon (trusted)5592worker1 # [ 88.661512] nix-daemon[841]: accepted connection from pid 1253, user nix-grpc-daemon (trusted)5593worker1 # [ 88.674518] nix-daemon[841]: reaped child process 1256, status = succeeded5594worker1 # [ 88.804629] nix-daemon[841]: accepted connection from pid 1258, user nix-grpc-daemon (trusted)5595worker1 # [ 88.816739] nix-daemon[841]: reaped child process 1261, status = succeeded5596lb # [ 88.828634] niks3-server[1039]: 2026/09/20 08:18:03 INFO Received uploads request method=POST path=/api/pending_closures5597worker1 # [ 88.836384] nix-grpc-daemon[1123]: time=2026-09-20T08:18:03.626Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5598worker1 # [ 88.837711] nix-grpc-daemon[1123]: time=2026-09-20T08:18:03.626Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"5599lb # [ 88.856838] niks3-server[1039]: 2026/09/20 08:18:03 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst5600lb # [ 88.863796] niks3-server[1039]: 2026/09/20 08:18:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5601lb # [ 88.867504] niks3-server[1039]: 2026/09/20 08:18:03 INFO Signed narinfos id=31 count=15602worker1 # [ 88.869831] nix-grpc-daemon[1123]: time=2026-09-20T08:18:03.659Z level=INFO msg="Uploading 1 narinfos"5603lb # [ 88.878974] niks3-server[1039]: 2026/09/20 08:18:03 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls5604lb # [ 88.884254] niks3-server[1039]: 2026/09/20 08:18:03 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5605lb # [ 88.888489] niks3-server[1039]: 2026/09/20 08:18:03 INFO Completed upload id=315606worker1 # [ 88.887444] nix-grpc-daemon[1123]: time=2026-09-20T08:18:03.677Z level=INFO msg="Upload complete. (378ms)"5607lb # [ 88.890032] niks3-server[1039]: 2026/09/20 08:18:03 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo5608worker1 # [ 88.889396] nix-grpc-daemon[1044]: ts=2026-09-20T08:18:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49428 duration_s=0 paths=1 nar_bytes_in=4165609worker1 # [ 88.893084] nix-grpc-daemon[1044]: ts=2026-09-20T08:18:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494285610worker1 # [ 88.899522] nix-daemon[841]: accepted connection from pid 1044, user nix-grpc-daemon (trusted)5611worker1 # [ 88.901795] nix-daemon[841]: reaped child process 1251, status = succeeded5612worker1 # [ 88.906869] nix-daemon[841]: accepted connection from pid 1044, user nix-grpc-daemon (trusted)5613client # [ 88.901492] nix[1145]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...5614lb: must succeed: curl -sf localhost:9901/clusters5615lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5616lb: must succeed: curl -sf localhost:9901/clusters5617lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5618lb: must succeed: curl -sf localhost:9901/clusters5619lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5620lb: must succeed: curl -sf localhost:9901/clusters5621lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5622worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5623worker2 # [ 91.061202] systemd-journald[288]: Received client request to rotate journal, rotating.5624worker2 # [ 91.080222] systemd-journald[288]: Vacuuming done, freed 0B of archived journals from /var/log/journal/0589f12d8ba749f99939a41902c2ef11.5625worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.07 seconds)5626worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5627worker1 # [ 90.977100] systemd[1]: Stopping Nix worker-protocol over gRPC...5628worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)5629??? Warning (UserWarning): sleep(): The 'secs' argument is deprecated. Use 'duration' instead.5630 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 5735631client: must succeed: sleep 5.05632worker1 # [ 91.462716] nix-grpc-daemon[1044]: ts=2026-09-20T08:18:06Z level=info event=draining5633client: (finished: must succeed: sleep 5.0, in 5.04 seconds)5634worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5635worker1 # [ 96.338797] nix-daemon[841]: reaped child process 1265, status = succeeded5636worker1 # [ 96.484554] nix-daemon[841]: accepted connection from pid 1305, user nix-grpc-daemon (trusted)5637worker1 # [ 96.496605] nix-daemon[841]: reaped child process 1308, status = succeeded5638lb # [ 96.512733] niks3-server[1039]: 2026/09/20 08:18:11 INFO Received uploads request method=POST path=/api/pending_closures5639worker1 # [ 96.522253] nix-grpc-daemon[1123]: time=2026-09-20T08:18:11.312Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5640worker1 # [ 96.523749] nix-grpc-daemon[1123]: time=2026-09-20T08:18:11.314Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"5641lb # [ 96.555215] niks3-server[1039]: 2026/09/20 08:18:11 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5642lb # [ 96.562303] niks3-server[1039]: 2026/09/20 08:18:11 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5643lb # [ 96.574287] niks3-server[1039]: 2026/09/20 08:18:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5644lb # [ 96.580358] niks3-server[1039]: 2026/09/20 08:18:11 INFO Signed narinfos id=32 count=15645worker1 # [ 96.582395] nix-grpc-daemon[1123]: time=2026-09-20T08:18:11.372Z level=INFO msg="Uploading 1 narinfos"5646lb # [ 96.583204] niks3-server[1039]: 2026/09/20 08:18:11 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls5647lb # [ 96.594582] niks3-server[1039]: 2026/09/20 08:18:11 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5648lb # [ 96.600173] niks3-server[1039]: 2026/09/20 08:18:11 INFO Completed upload id=325649worker1 # [ 96.600473] nix-grpc-daemon[1123]: time=2026-09-20T08:18:11.390Z level=INFO msg="Upload complete. (279ms)"5650lb # [ 96.602167] niks3-server[1039]: 2026/09/20 08:18:11 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo5651worker1 # [ 96.604254] nix-grpc-daemon[1044]: ts=2026-09-20T08:18:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49428 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=15652worker1 # [ 96.612272] nix-daemon[841]: reaped child process 1263, status = succeeded5653client # [ 96.644483] systemd[1]: rs.service: Deactivated successfully.5654client # [ 96.647813] systemd[1]: rs.service: Consumed 114ms CPU time over 8.417s wall clock time, 37.1M memory peak, 8.5K incoming IP traffic, 9K outgoing IP traffic.5655lb # [ 97.447158] envoy[910]: conn peer=[2001:db8:1::1]:58568 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2934 tx=1347 ms=81755656worker1 # [ 97.464965] nix-grpc-daemon[1044]: ts=2026-09-20T08:18:12Z level=info event=signal_exit5657worker1 # [ 97.686824] nix-grpc-daemon[1044]: ts=2026-09-20T08:18:12Z level=info event=scheduler_yield5658lb # [ 97.690529] niks3-server[1039]: 2026/09/20 08:18:12 INFO lead: released remote=192.168.1.3:537225659worker1 # [ 97.690623] nix-grpc-daemon[1044]: ts=2026-09-20T08:18:12Z level=info event=scheduler_restarting addr=lb:500515660worker2 # [ 97.710970] nix-grpc-daemon[1304]: ts=2026-09-20T08:18:12Z level=info event=scheduler_restarting addr=lb:500515661worker1 # [ 97.695450] nix-grpc-daemon[1044]: ts=2026-09-20T08:18:12Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:494125662worker1 # [ 97.698774] nix-grpc-daemon[1044]: ts=2026-09-20T08:18:12Z level=info event=worker_gone addr=192.168.1.4:500515663worker1 # [ 97.701666] nix-grpc-daemon[1044]: ts=2026-09-20T08:18:12Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:494125664worker1 # [ 97.704904] nix-grpc-daemon[1044]: ts=2026-09-20T08:18:12Z level=info event=worker_gone addr=192.168.1.3:500515665worker2 # [ 97.911441] nix-grpc-daemon[1304]: ts=2026-09-20T08:18:13Z level=info event=scheduler_connected addr=lb:500515666worker1 # [ 97.891479] nix-grpc-daemon[1044]: ts=2026-09-20T08:18:12Z level=info event=scheduler_connected addr=lb:500515667lb # [ 97.901177] niks3-server[1039]: 2026/09/20 08:18:12 INFO lead: acquired remote=192.168.1.4:534185668worker2 # [ 97.920432] nix-grpc-daemon[1304]: ts=2026-09-20T08:18:13Z level=info event=scheduler_take_over5669worker1 # [ 97.903515] nix-grpc-daemon[1044]: ts=2026-09-20T08:18:12Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5670worker2 # [ 97.924246] nix-grpc-daemon[1304]: ts=2026-09-20T08:18:13Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5671worker1 # [ 98.104565] nix-grpc-daemon[1044]: ts=2026-09-20T08:18:12Z level=info event=scheduler_connected addr=lb:500515672worker2 # [ 98.124694] nix-grpc-daemon[1304]: ts=2026-09-20T08:18:13Z level=info event=scheduler_connected addr=lb:500515673worker2 # [ 98.129716] nix-grpc-daemon[1304]: ts=2026-09-20T08:18:13Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=0 running=05674worker2 # [ 98.135774] nix-grpc-daemon[1304]: ts=2026-09-20T08:18:13Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5675worker2 # [ 98.336845] nix-grpc-daemon[1304]: ts=2026-09-20T08:18:13Z level=info event=scheduler_connected addr=lb:500515676worker2 # [ 98.340937] nix-grpc-daemon[1304]: ts=2026-09-20T08:18:13Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05677worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.14 seconds)5678client: waiting for success: ! systemctl is-active rs5679client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds)5680client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5681client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.04 seconds)5682client: must fail: journalctl -u rs | grep -q 'reconnecting'5683client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5684lb: must succeed: curl -sf localhost:9901/clusters5685lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5686worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5687worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.07 seconds)5688(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 10.44 seconds)5689subtest: scheduler restart mid-build: build finishes, no double build5690client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid5691client # [ 98.668934] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid.5692client # Running as unit: mid.service; invocation ID: 79c6afafb8f340d4b4814bc1b1f4f5725693client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid, in 0.07 seconds)5694worker2 # [ 98.712703] nix-grpc-daemon[1304]: ts=2026-09-20T08:18:13Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:447285695worker1 # [ 98.695407] nix-grpc-daemon[1044]: ts=2026-09-20T08:18:13Z level=info event=scheduler_restarting5696worker2 # [ 98.716781] nix-grpc-daemon[1304]: ts=2026-09-20T08:18:13Z level=info event=worker_gone addr=192.168.1.3:500515697worker1 # [ 98.709205] nix-grpc-daemon[1044]: ts=2026-09-20T08:18:13Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5698worker1 # [ 98.734973] nix-daemon[841]: reaped child process 1114, status = succeeded5699worker1 # [ 98.866494] nix-grpc-daemon[1044]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5700worker1 # [ 98.869977] nix-grpc-daemon[1044]: E0000 00:00:1789892293.656467 1087 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 10, started true, shutdown true5701worker2 # [ 98.897069] nix-grpc-daemon[1304]: ts=2026-09-20T08:18:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:447365702worker1 # [ 98.874967] nix-grpc-daemon[1044]: E0000 00:00:1789892293.656627 1087 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5703worker2 # [ 98.902924] nix-daemon[847]: accepted connection from pid 1304, user nix-grpc-daemon (trusted)5704client # [ 98.884430] nix[1180]: don't know how to build these paths:5705client # [ 98.886094] nix[1180]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5706worker1 # [ 98.892243] nix-grpc-daemon[1044]: E0000 00:00:1789892293.682486 1087 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 8, started true, shutdown true5707client # [ 100.057541] nix[1180]: scheduler at lb:50051: upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, reconnecting5708worker1 # [ 100.081719] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5709worker1 # [ 100.085364] systemd[1]: Stopped Nix worker-protocol over gRPC.5710worker1 # [ 100.087483] systemd[1]: nix-grpc-daemon.service: Consumed 1.267s CPU time over 24.780s wall clock time, 35.5M memory peak, 46.2K incoming IP traffic, 88.1K outgoing IP traffic.5711worker1 # [ 100.102845] systemd[1]: Starting Nix worker-protocol over gRPC...5712worker1 # [ 100.268774] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:15Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5713worker1 # [ 100.271981] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:15Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15714worker1 # [ 100.279055] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:15Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5715worker1 # [ 100.282330] systemd[1]: Started Nix worker-protocol over gRPC.5716worker1 # [ 100.306191] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:15Z level=info event=scheduler_connected addr=lb:500515717worker2 # [ 100.329129] nix-grpc-daemon[1304]: ts=2026-09-20T08:18:15Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05718worker2 # [ 100.590882] nix-grpc-daemon[1304]: ts=2026-09-20T08:18:15Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05719worker2 # [ 100.596574] nix-grpc-daemon[1304]: ts=2026-09-20T08:18:15Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=15720worker2 # [ 100.635768] nix-grpc-daemon[1304]: ts=2026-09-20T08:18:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:447365721worker2 # [ 100.647406] nix-grpc-daemon[1304]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 12 ms (attempt 1/5)5722worker2 # [ 100.665248] nix-grpc-daemon[1304]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 23 ms (attempt 2/5)5723worker2 # [ 100.688551] nix-grpc-daemon[1304]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 319 ms (attempt 3/5)5724worker2 # [ 101.013528] nix-grpc-daemon[1304]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 275 ms (attempt 4/5)5725worker2 # [ 101.295553] nix-grpc-daemon[1304]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5726worker2 # [ 101.313569] nix-grpc-daemon[1304]: ts=2026-09-20T08:18:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:447365727client # [ 101.301319] nix[1180]: copying 1 paths to 'lb:50051'...5728worker2 # [ 101.327312] nix-grpc-daemon[1304]: ts=2026-09-20T08:18:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:447365729client # [ 101.304700] nix[1180]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...5730worker2 # [ 101.336683] nix-daemon[847]: accepted connection from pid 1304, user nix-grpc-daemon (trusted)5731worker2 # [ 101.356417] nix-grpc-daemon[1304]: ts=2026-09-20T08:18:16Z level=info event=niks3_push_started pid=14315732worker2 # [ 101.703126] nix-daemon[847]: accepted connection from pid 1444, user nix-grpc-daemon (trusted)5733worker2 # [ 101.713588] nix-daemon[847]: reaped child process 1447, status = succeeded5734worker2 # [ 101.876037] nix-daemon[847]: accepted connection from pid 1449, user nix-grpc-daemon (trusted)5735worker2 # [ 101.887576] nix-daemon[847]: reaped child process 1452, status = succeeded5736lb # [ 101.884713] niks3-server[1039]: 2026/09/20 08:18:16 INFO Received uploads request method=POST path=/api/pending_closures5737worker2 # [ 101.913306] nix-grpc-daemon[1431]: time=2026-09-20T08:18:17.047Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5738worker2 # [ 101.914686] nix-grpc-daemon[1431]: time=2026-09-20T08:18:17.047Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"5739lb # [ 101.916278] niks3-server[1039]: 2026/09/20 08:18:16 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst5740lb # [ 101.927527] niks3-server[1039]: 2026/09/20 08:18:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5741lb # [ 101.932822] niks3-server[1039]: 2026/09/20 08:18:16 INFO Signed narinfos id=33 count=15742worker2 # [ 101.954794] nix-grpc-daemon[1431]: time=2026-09-20T08:18:17.089Z level=INFO msg="Uploading 1 narinfos"5743lb # [ 101.945055] niks3-server[1039]: 2026/09/20 08:18:16 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls5744lb # [ 101.952688] niks3-server[1039]: 2026/09/20 08:18:16 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5745lb # [ 101.960867] niks3-server[1039]: 2026/09/20 08:18:16 INFO Completed upload id=335746worker2 # [ 101.980114] nix-grpc-daemon[1431]: time=2026-09-20T08:18:17.115Z level=INFO msg="Upload complete. (429ms)"5747worker2 # [ 101.983076] nix-grpc-daemon[1304]: ts=2026-09-20T08:18:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44736 duration_s=0 paths=1 nar_bytes_in=4245748worker2 # [ 101.985552] nix-grpc-daemon[1431]: time=2026-09-20T08:18:17.120Z level=WARN msg="Failed to register uploaded object" key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo error="server returned 404: object is not pending upload\n"5749worker2 # [ 101.990716] nix-grpc-daemon[1304]: ts=2026-09-20T08:18:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:447365750worker2 # [ 101.996145] nix-daemon[847]: accepted connection from pid 1304, user nix-grpc-daemon (trusted)5751worker2 # [ 102.000192] nix-daemon[847]: reaped child process 1429, status = succeeded5752worker2 # [ 102.005214] nix-daemon[847]: accepted connection from pid 1304, user nix-grpc-daemon (trusted)5753client # [ 101.980133] nix[1180]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5754lb: must succeed: curl -sf localhost:9901/clusters5755lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5756lb: must succeed: curl -sf localhost:9901/clusters5757lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5758worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5759worker2 # [ 103.627924] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1304 (nix-grpc-daemon) on client request.5760worker2 # [ 103.629804] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5761lb # [ 103.622399] niks3-server[1039]: 2026/09/20 08:18:18 INFO lead: released remote=192.168.1.4:534185762client # [ 103.616334] nix[1180]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5763client # [ 103.617974] nix[1180]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5764worker1 # [ 103.620585] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:18Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5765worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.06 seconds)5766client: must succeed: sleep 5.05767worker2 # [ 103.663311] nix-daemon[847]: reaped child process 1454, status = succeeded5768worker2 # [ 103.664483] nix-daemon[847]: reaped child process 1414, status = succeeded5769worker2 # [ 103.669839] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5770worker2 # [ 103.671397] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5771worker2 # [ 103.673157] systemd[1]: nix-grpc-daemon.service: Consumed 417ms CPU time over 15.914s wall clock time, 33.1M memory peak, 13.1K incoming IP traffic, 18K outgoing IP traffic.5772worker2 # [ 103.681257] nix-daemon[1456]: unexpected Nix daemon error: error: interrupted by the user5773worker2 # [ 103.685187] nix-daemon[847]: reaped child process 1456, status = failed with exit code 15774worker1 # [ 103.820958] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:18Z level=info event=scheduler_connected addr=lb:500515775worker2 # [ 103.844720] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.5776worker2 # [ 103.854678] systemd[1]: Starting Nix worker-protocol over gRPC...5777worker2 # [ 104.034145] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:19Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5778worker2 # [ 104.037746] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:19Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15779worker2 # [ 104.044818] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:19Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5780worker2 # [ 104.046424] systemd[1]: Started Nix worker-protocol over gRPC.5781worker2 # [ 104.057172] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:19Z level=info event=scheduler_take_over5782lb # [ 104.037627] niks3-server[1039]: 2026/09/20 08:18:18 INFO lead: acquired remote=192.168.1.4:601485783worker1 # [ 104.040948] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:18Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5784worker2 # [ 104.073835] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:19Z level=info event=scheduler_connected addr=lb:500515785worker2 # [ 104.075577] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:19Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05786worker2 # [ 104.148756] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:19Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05787worker2 # [ 104.154195] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:19Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=15788worker2 # [ 104.202579] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:563785789worker2 # [ 104.212161] nix-daemon[847]: accepted connection from pid 1492, user nix-grpc-daemon (trusted)5790worker2 # [ 104.224397] nix-daemon[847]: accepted connection from pid 1492, user nix-grpc-daemon (trusted)5791worker2 # [ 104.235387] nix-daemon[847]: accepted connection from pid 1492, user nix-grpc-daemon (trusted)5792client # [ 104.210172] nix[1180]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5793worker1 # [ 104.241974] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:19Z level=info event=scheduler_connected addr=lb:500515794worker2 # [ 104.263339] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:19Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05795lb # [ 107.448424] envoy[910]: conn peer=[2001:db8:1::3]:56890 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=4391 tx=1156 ms=232165796lb # [ 107.452553] envoy[910]: conn peer=[2001:db8:1::4]:58956 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1172 tx=417 ms=157055797client: (finished: must succeed: sleep 5.0, in 5.04 seconds)5798client: waiting for success: ! systemctl is-active mid5799worker2 # [ 109.049425] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:24Z level=info event=niks3_push_started pid=15715800worker2 # [ 109.074873] nix-daemon[847]: reaped child process 1547, status = succeeded5801worker2 # [ 109.348691] nix-daemon[847]: accepted connection from pid 1584, user nix-grpc-daemon (trusted)5802worker2 # [ 109.360163] nix-daemon[847]: reaped child process 1587, status = succeeded5803lb # [ 109.355569] niks3-server[1039]: 2026/09/20 08:18:24 INFO Received uploads request method=POST path=/api/pending_closures5804worker2 # [ 109.384303] nix-grpc-daemon[1571]: time=2026-09-20T08:18:24.519Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5805worker2 # [ 109.385980] nix-grpc-daemon[1571]: time=2026-09-20T08:18:24.521Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"5806lb # [ 109.388807] niks3-server[1039]: 2026/09/20 08:18:24 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5807lb # [ 109.391916] niks3-server[1039]: 2026/09/20 08:18:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5808lb # [ 109.393901] niks3-server[1039]: 2026/09/20 08:18:24 INFO Signed narinfos id=34 count=15809worker2 # [ 109.412738] nix-grpc-daemon[1571]: time=2026-09-20T08:18:24.548Z level=INFO msg="Uploading 1 narinfos"5810lb # [ 109.400251] niks3-server[1039]: 2026/09/20 08:18:24 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5811lb # [ 109.406058] niks3-server[1039]: 2026/09/20 08:18:24 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5812lb # [ 109.411415] niks3-server[1039]: 2026/09/20 08:18:24 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls5813lb # [ 109.420599] niks3-server[1039]: 2026/09/20 08:18:24 INFO Completed upload id=345814worker2 # [ 109.439825] nix-grpc-daemon[1571]: time=2026-09-20T08:18:24.575Z level=INFO msg="Upload complete. (229ms)"5815worker2 # [ 109.442224] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56378 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=15816lb # [ 109.424783] niks3-server[1039]: 2026/09/20 08:18:24 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo5817worker2 # [ 109.450275] nix-daemon[847]: reaped child process 1545, status = succeeded5818client # [ 109.453046] systemd[1]: mid.service: Deactivated successfully.5819client # [ 109.454642] systemd[1]: mid.service: Consumed 133ms CPU time over 10.785s wall clock time, 39.1M memory peak, 9.9K incoming IP traffic, 10.9K outgoing IP traffic.5820client: (finished: waiting for success: ! systemctl is-active mid, in 1.10 seconds)5821client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5822client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.03 seconds)5823(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 11.42 seconds)5824subtest: interrupting the client stops the build on the worker5825client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr5826client # [ 110.065447] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr.5827client # Running as unit: intr.service; invocation ID: 47aa8ffc37b34b15b7e06064b4611d785828client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr, in 0.04 seconds)5829worker1 # [ 110.286515] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:542545830worker1 # [ 110.297776] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)5831client # [ 110.305310] nix[1215]: don't know how to build these paths:5832client # [ 110.306673] nix[1215]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv5833worker2 # [ 110.341790] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:25Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=05834worker2 # [ 110.346249] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:25Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.3:50051 clients=15835worker1 # [ 110.366404] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:542545836worker1 # [ 110.378297] nix-grpc-daemon[1323]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 37 ms (attempt 1/5)5837worker1 # [ 110.421280] nix-grpc-daemon[1323]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 117 ms (attempt 2/5)5838worker1 # [ 110.544413] nix-grpc-daemon[1323]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 72 ms (attempt 3/5)5839worker1 # [ 110.622302] nix-grpc-daemon[1323]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 705 ms (attempt 4/5)5840worker1 # [ 111.334237] nix-grpc-daemon[1323]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5841worker1 # [ 111.349157] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:542545842client # [ 111.353631] nix[1215]: copying 1 paths to 'lb:50051'...5843worker1 # [ 111.361935] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:542545844client # [ 111.357976] nix[1215]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...5845worker1 # [ 111.370860] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)5846worker1 # [ 111.391449] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:26Z level=info event=niks3_push_started pid=14115847worker1 # [ 111.719295] nix-daemon[841]: accepted connection from pid 1424, user nix-grpc-daemon (trusted)5848worker1 # [ 111.731204] nix-daemon[841]: reaped child process 1427, status = succeeded5849worker1 # [ 111.882437] nix-daemon[841]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)5850worker1 # [ 111.893964] nix-daemon[841]: reaped child process 1432, status = succeeded5851lb # [ 111.911728] niks3-server[1039]: 2026/09/20 08:18:26 INFO Received uploads request method=POST path=/api/pending_closures5852worker1 # [ 111.923282] nix-grpc-daemon[1411]: time=2026-09-20T08:18:26.713Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5853worker1 # [ 111.926436] nix-grpc-daemon[1411]: time=2026-09-20T08:18:26.715Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"5854lb # [ 111.950659] niks3-server[1039]: 2026/09/20 08:18:26 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst5855lb # [ 111.958613] niks3-server[1039]: 2026/09/20 08:18:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5856lb # [ 111.964781] niks3-server[1039]: 2026/09/20 08:18:26 INFO Signed narinfos id=35 count=15857worker1 # [ 111.967153] nix-grpc-daemon[1411]: time=2026-09-20T08:18:26.757Z level=INFO msg="Uploading 1 narinfos"5858lb # [ 111.979077] niks3-server[1039]: 2026/09/20 08:18:26 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls5859lb # [ 111.982799] niks3-server[1039]: 2026/09/20 08:18:26 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5860lb # [ 111.992252] niks3-server[1039]: 2026/09/20 08:18:26 INFO Completed upload id=355861worker1 # [ 111.991486] nix-grpc-daemon[1411]: time=2026-09-20T08:18:26.781Z level=INFO msg="Upload complete. (417ms)"5862lb # [ 111.994139] niks3-server[1039]: 2026/09/20 08:18:26 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo5863worker1 # [ 111.994624] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54254 duration_s=0 paths=1 nar_bytes_in=4245864worker1 # [ 111.999760] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:542545865worker1 # [ 112.007477] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)5866worker1 # [ 112.015146] nix-daemon[841]: reaped child process 1409, status = succeeded5867worker1 # [ 112.018289] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)5868client # [ 112.013224] nix[1215]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...5869client: must succeed: systemctl kill -s INT intr5870client # [ 113.580943] nix[1215]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5871client # [ 113.589440] systemd[1]: intr.service: Sent signal SIGINT to main process 1215 (nix) on client request.5872client # [ 113.593782] nix[1215]: error: interrupted by the user5873client: (finished: must succeed: systemctl kill -s INT intr, in 0.06 seconds)5874client # [ 113.623467] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5875client # [ 113.625370] systemd[1]: intr.service: Failed with result 'exit-code'.5876client # [ 113.626954] systemd[1]: intr.service: Consumed 115ms CPU time over 3.559s wall clock time, 39.1M memory peak, 7.7K incoming IP traffic, 8.7K outgoing IP traffic.5877worker1 # [ 113.638389] nix-daemon[841]: reaped child process 1434, status = succeeded5878worker1 # [ 113.644245] nix-daemon[1436]: unexpected Nix daemon error: error: interrupted by the user5879worker1 # [ 113.650168] nix-daemon[841]: reaped child process 1436, status = failed with exit code 15880(finished: subtest: interrupting the client stops the build on the worker, in 3.68 seconds)5881subtest: a deploy mid-build drains: the build finishes, the new generation takes over5882client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw5883client # [ 113.748792] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw.5884client # Running as unit: sw.service; invocation ID: 7885db53c7c547229e0ab21b7bce349d5885client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw, in 0.04 seconds)5886worker1 # [ 113.944598] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:542545887client # [ 113.946046] nix[1235]: don't know how to build these paths:5888client # [ 113.946936] nix[1235]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv5889worker2 # [ 113.981746] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:29Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=05890worker2 # [ 113.985803] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:29Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.4:50051 clients=15891worker2 # [ 114.023738] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:563785892worker2 # [ 114.036121] nix-grpc-daemon[1492]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 97 ms (attempt 1/5)5893worker2 # [ 114.138970] nix-grpc-daemon[1492]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 42 ms (attempt 2/5)5894worker2 # [ 114.186874] nix-grpc-daemon[1492]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 156 ms (attempt 3/5)5895worker2 # [ 114.348880] nix-grpc-daemon[1492]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 686 ms (attempt 4/5)5896worker2 # [ 115.041866] nix-grpc-daemon[1492]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5897worker2 # [ 115.055450] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:563785898client # [ 115.039731] nix[1235]: copying 1 paths to 'lb:50051'...5899worker2 # [ 115.068210] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:563785900client # [ 115.044151] nix[1235]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...5901worker2 # [ 115.077217] nix-daemon[847]: accepted connection from pid 1492, user nix-grpc-daemon (trusted)5902worker2 # [ 115.242553] nix-daemon[847]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)5903worker2 # [ 115.254226] nix-daemon[847]: reaped child process 1629, status = succeeded5904worker2 # [ 115.424683] nix-daemon[847]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)5905worker2 # [ 115.436269] nix-daemon[847]: reaped child process 1634, status = succeeded5906lb # [ 115.434841] niks3-server[1039]: 2026/09/20 08:18:30 INFO Received uploads request method=POST path=/api/pending_closures5907worker2 # [ 115.465264] nix-grpc-daemon[1571]: time=2026-09-20T08:18:30.600Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5908worker2 # [ 115.467200] nix-grpc-daemon[1571]: time=2026-09-20T08:18:30.602Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"5909lb # [ 115.466702] niks3-server[1039]: 2026/09/20 08:18:30 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst5910lb # [ 115.475205] niks3-server[1039]: 2026/09/20 08:18:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5911worker2 # [ 115.500733] nix-grpc-daemon[1571]: time=2026-09-20T08:18:30.635Z level=INFO msg="Uploading 1 narinfos"5912lb # [ 115.481410] niks3-server[1039]: 2026/09/20 08:18:30 INFO Signed narinfos id=36 count=15913lb # [ 115.486658] niks3-server[1039]: 2026/09/20 08:18:30 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls5914lb # [ 115.497255] niks3-server[1039]: 2026/09/20 08:18:30 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5915lb # [ 115.499926] niks3-server[1039]: 2026/09/20 08:18:30 INFO Completed upload id=365916worker2 # [ 115.520330] nix-grpc-daemon[1571]: time=2026-09-20T08:18:30.655Z level=INFO msg="Upload complete. (438ms)"5917lb # [ 115.502261] niks3-server[1039]: 2026/09/20 08:18:30 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo5918worker2 # [ 115.523992] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56378 duration_s=0 paths=1 nar_bytes_in=4165919worker2 # [ 115.529556] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:563785920worker2 # [ 115.537829] nix-daemon[847]: accepted connection from pid 1492, user nix-grpc-daemon (trusted)5921worker2 # [ 115.542951] nix-daemon[847]: reaped child process 1624, status = succeeded5922worker2 # [ 115.547519] nix-daemon[847]: accepted connection from pid 1492, user nix-grpc-daemon (trusted)5923client # [ 115.522101] nix[1235]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...5924worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&25925lb # [ 117.449550] envoy[910]: conn peer=[2001:db8:1::1]:55732 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4033 tx=1786 ms=105645926lb # [ 117.453400] envoy[910]: conn peer=[2001:db8:1::1]:53034 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2924 tx=755 ms=33345927worker2 # Checking switch inhibitors... done5928worker2 # [ 118.129448] nixos[1669]: switching to system configuration /nix/store/h5sbvfp8h8c9c7k729zfw5h7pifnx129-nixos-system-worker2-test5929worker2 # [ 118.132826] systemd[1]: Stopped target Local File Systems.5930worker2 # activating the configuration...5931worker2 # [ 118.135490] systemd[1]: Stopped target Remote File Systems.5932worker2 # setting up /etc...5933worker2 # [ 118.814134] systemd[1]: Reload requested from client PID 1669 ('.switch-to-conf') (unit backdoor.service)...5934worker2 # [ 118.815752] systemd[1]: Reloading...5935worker2 # [ 119.514489] systemd[1]: Reloading finished in 695 ms.5936worker2 # restarting sysinit-reactivation.target5937worker2 # [ 119.573087] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....5938worker2 # [ 119.717192] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..5939worker2 # [ 119.720596] systemd[1]: Reached target Reactivate sysinit units.5940worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service5941worker2 # [ 119.725512] dbus-broker-launch[534]: Caught SIGHUP, trigger reload.5942worker2 # [ 119.728529] systemd[1]: Reloading D-Bus System Message Bus...5943worker2 # [ 119.730936] systemd[1]: Reloading Nix worker-protocol over gRPC...5944worker2 # [ 119.755602] dbus-broker-launch[534]: Looking up NSS user entry for 'systemd-timesync'...5945worker2 # [ 119.760153] dbus-broker-launch[534]: NSS returned no entry for 'systemd-timesync'5946worker2 # [ 119.762134] dbus-broker-launch[534]: Invalid user-name in /nix/store/nl02z4vnl25dc1df60lqifjnw6c89zpg-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"5947worker2 # [ 119.776915] systemd[1]: Reloaded Nix worker-protocol over gRPC.5948worker2 # [ 119.793284] systemd[1]: Reloaded D-Bus System Message Bus.5949worker2 # [ 119.798383] systemd[1]: Reached target Local File Systems.5950worker2 # [ 119.803816] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os5951worker2 # [ 119.805622] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki5952worker2 # [ 119.808240] systemd[1]: Reached target Remote File Systems.5953worker2 # [ 119.814154] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore5954worker2 # [ 119.820751] systemd[1]: Starting Create SUID/SGID Wrappers...5955worker2 # [ 119.821683] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.5956worker2 # [ 119.823408] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc675957worker2 # [ 119.826482] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore5958worker2 # [ 119.828613] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os5959worker2 # [ 119.829927] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os5960worker2 # [ 120.052456] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:35Z level=info event=draining5961worker2 # [ 120.257427] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.5962worker2 # [ 120.263453] systemd[1]: Finished Create SUID/SGID Wrappers.5963worker2 # [ 120.265741] systemd[1]: Starting linger-users.service...5964worker2 # [ 120.310449] systemd[1]: linger-users.service: Deactivated successfully.5965worker2 # [ 120.311451] systemd[1]: Finished linger-users.service.5966worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service5967worker2 # [ 120.565293] nixos[1669]: finished switching to system configuration /nix/store/h5sbvfp8h8c9c7k729zfw5h7pifnx129-nixos-system-worker2-test5968worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 3.17 seconds)5969worker2: must succeed: pgrep -f 'read -t [9]0 x'5970worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)5971client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag during-drain >&25972worker1 # [ 120.821404] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:35Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:542545973client # don't know how to build these paths:5974client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv5975worker2 # [ 120.855870] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:35Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=05976worker2 # [ 120.857733] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:35Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.3:50051 clients=15977worker1 # [ 120.883913] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:542545978worker1 # [ 120.896459] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:542545979client # copying 1 paths to 'lb:50051'...5980worker1 # [ 120.909219] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:542545981client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...5982worker1 # [ 120.929487] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)5983worker1 # [ 121.091226] nix-daemon[841]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)5984worker1 # [ 121.103090] nix-daemon[841]: reaped child process 1489, status = succeeded5985worker1 # [ 121.252086] nix-daemon[841]: accepted connection from pid 1491, user nix-grpc-daemon (trusted)5986worker1 # [ 121.262741] nix-daemon[841]: reaped child process 1494, status = succeeded5987lb # [ 121.279269] niks3-server[1039]: 2026/09/20 08:18:36 INFO Received uploads request method=POST path=/api/pending_closures5988worker1 # [ 121.286799] nix-grpc-daemon[1411]: time=2026-09-20T08:18:36.076Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5989worker1 # [ 121.288543] nix-grpc-daemon[1411]: time=2026-09-20T08:18:36.078Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"5990lb # [ 121.310688] niks3-server[1039]: 2026/09/20 08:18:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5991lb # [ 121.318412] niks3-server[1039]: 2026/09/20 08:18:36 INFO Signed narinfos id=37 count=15992worker1 # [ 121.320623] nix-grpc-daemon[1411]: time=2026-09-20T08:18:36.110Z level=INFO msg="Uploading 1 narinfos"5993lb # [ 121.325474] niks3-server[1039]: 2026/09/20 08:18:36 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst5994lb # [ 121.338521] niks3-server[1039]: 2026/09/20 08:18:36 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls5995lb # [ 121.343818] niks3-server[1039]: 2026/09/20 08:18:36 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5996worker1 # [ 121.348843] nix-grpc-daemon[1411]: time=2026-09-20T08:18:36.139Z level=INFO msg="Upload complete. (407ms)"5997lb # [ 121.349750] niks3-server[1039]: 2026/09/20 08:18:36 INFO Completed upload id=375998worker1 # [ 121.350973] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54254 duration_s=0 paths=1 nar_bytes_in=5285999lb # [ 121.352923] niks3-server[1039]: 2026/09/20 08:18:36 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6000worker1 # [ 121.356728] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:542546001worker1 # [ 121.364584] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6002worker1 # [ 121.369029] nix-daemon[841]: reaped child process 1484, status = succeeded6003worker1 # [ 121.373949] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6004client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6005client # farm-a> node-a: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6006client # farm-a> LOG-farm-a-during-drain6007worker1 # [ 122.801167] nix-daemon[841]: reaped child process 1498, status = succeeded6008worker1 # [ 122.962583] nix-daemon[841]: accepted connection from pid 1513, user nix-grpc-daemon (trusted)6009worker1 # [ 122.973298] nix-daemon[841]: reaped child process 1516, status = succeeded6010lb # [ 122.994122] niks3-server[1039]: 2026/09/20 08:18:37 INFO Received uploads request method=POST path=/api/pending_closures6011worker1 # [ 122.999782] nix-grpc-daemon[1411]: time=2026-09-20T08:18:37.789Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6012worker1 # [ 123.001331] nix-grpc-daemon[1411]: time=2026-09-20T08:18:37.791Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6013lb # [ 123.031959] niks3-server[1039]: 2026/09/20 08:18:37 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6014lb # [ 123.039462] niks3-server[1039]: 2026/09/20 08:18:37 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6015lb # [ 123.052218] niks3-server[1039]: 2026/09/20 08:18:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6016lb # [ 123.056969] niks3-server[1039]: 2026/09/20 08:18:37 INFO Signed narinfos id=38 count=16017worker1 # [ 123.058484] nix-grpc-daemon[1411]: time=2026-09-20T08:18:37.848Z level=INFO msg="Uploading 1 narinfos"6018lb # [ 123.061512] niks3-server[1039]: 2026/09/20 08:18:38 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6019lb # [ 123.073129] niks3-server[1039]: 2026/09/20 08:18:38 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6020lb # [ 123.077600] niks3-server[1039]: 2026/09/20 08:18:38 INFO Completed upload id=386021lb # [ 123.078833] niks3-server[1039]: 2026/09/20 08:18:38 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6022worker1 # [ 123.076736] nix-grpc-daemon[1411]: time=2026-09-20T08:18:37.867Z level=INFO msg="Upload complete. (292ms)"6023worker2 # [ 123.105821] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:38Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06024worker1 # [ 123.083237] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54254 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16025worker2 # [ 123.107529] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:38Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.3:50051 clients=16026worker1 # [ 123.095890] nix-daemon[841]: reaped child process 1496, status = succeeded6027worker1 # [ 123.129474] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:542546028worker1 # [ 123.147611] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:542546029worker1 # [ 123.160711] nix-grpc-daemon[1411]: time=2026-09-20T08:18:37.950Z level=INFO msg="All 1 paths already cached"6030client # copying 1 paths to 'lb:50051'...6031worker1 # [ 123.168538] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:542546032client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6033worker1 # [ 123.187166] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6034worker1 # [ 123.330228] nix-daemon[841]: accepted connection from pid 1520, user nix-grpc-daemon (trusted)6035worker1 # [ 123.342451] nix-daemon[841]: reaped child process 1523, status = succeeded6036worker1 # [ 123.473472] nix-daemon[841]: accepted connection from pid 1525, user nix-grpc-daemon (trusted)6037worker1 # [ 123.485610] nix-daemon[841]: reaped child process 1528, status = succeeded6038lb # [ 123.498108] niks3-server[1039]: 2026/09/20 08:18:38 INFO Received uploads request method=POST path=/api/pending_closures6039worker1 # [ 123.506280] nix-grpc-daemon[1411]: time=2026-09-20T08:18:38.296Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6040worker1 # [ 123.507721] nix-grpc-daemon[1411]: time=2026-09-20T08:18:38.296Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6041lb # [ 123.529988] niks3-server[1039]: 2026/09/20 08:18:38 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6042lb # [ 123.537673] niks3-server[1039]: 2026/09/20 08:18:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6043lb # [ 123.542363] niks3-server[1039]: 2026/09/20 08:18:38 INFO Signed narinfos id=39 count=16044worker1 # [ 123.545002] nix-grpc-daemon[1411]: time=2026-09-20T08:18:38.335Z level=INFO msg="Uploading 1 narinfos"6045lb # [ 123.555587] niks3-server[1039]: 2026/09/20 08:18:38 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6046lb # [ 123.561608] niks3-server[1039]: 2026/09/20 08:18:38 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6047lb # [ 123.566438] niks3-server[1039]: 2026/09/20 08:18:38 INFO Completed upload id=396048worker1 # [ 123.567701] nix-grpc-daemon[1411]: time=2026-09-20T08:18:38.358Z level=INFO msg="Upload complete. (368ms)"6049worker1 # [ 123.569709] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54254 duration_s=0 paths=1 nar_bytes_in=6406050lb # [ 123.571729] niks3-server[1039]: 2026/09/20 08:18:38 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6051worker1 # [ 123.575419] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:542546052worker1 # [ 123.580048] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6053worker1 # [ 123.582244] nix-daemon[841]: reaped child process 1518, status = succeeded6054worker1 # [ 123.587678] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6055client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6056client # farm-b> node-a: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6057client # farm-b> LOG-farm-b-during-drain6058worker1 # [ 124.602030] nix-daemon[841]: reaped child process 1532, status = succeeded6059worker1 # [ 124.714532] nix-daemon[841]: accepted connection from pid 1547, user nix-grpc-daemon (trusted)6060worker1 # [ 124.726271] nix-daemon[841]: reaped child process 1550, status = succeeded6061lb # [ 124.738550] niks3-server[1039]: 2026/09/20 08:18:39 INFO Received uploads request method=POST path=/api/pending_closures6062worker1 # [ 124.742324] nix-grpc-daemon[1411]: time=2026-09-20T08:18:39.532Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6063worker1 # [ 124.743776] nix-grpc-daemon[1411]: time=2026-09-20T08:18:39.534Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6064lb # [ 124.759760] niks3-server[1039]: 2026/09/20 08:18:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6065lb # [ 124.761948] niks3-server[1039]: 2026/09/20 08:18:39 INFO Signed narinfos id=40 count=16066worker1 # [ 124.762372] nix-grpc-daemon[1411]: time=2026-09-20T08:18:39.552Z level=INFO msg="Uploading 1 narinfos"6067lb # [ 124.763334] niks3-server[1039]: 2026/09/20 08:18:39 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6068lb # [ 124.765957] niks3-server[1039]: 2026/09/20 08:18:39 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6069lb # [ 124.772436] niks3-server[1039]: 2026/09/20 08:18:39 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6070lb # [ 124.773973] niks3-server[1039]: 2026/09/20 08:18:39 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6071lb # [ 124.776404] niks3-server[1039]: 2026/09/20 08:18:39 INFO Completed upload id=406072worker1 # [ 124.775270] nix-grpc-daemon[1411]: time=2026-09-20T08:18:39.565Z level=INFO msg="Upload complete. (181ms)"6073lb # [ 124.779377] niks3-server[1039]: 2026/09/20 08:18:39 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6074worker1 # [ 124.777635] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54254 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16075worker2 # [ 124.803669] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:39Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06076worker1 # [ 124.784331] nix-daemon[841]: reaped child process 1530, status = succeeded6077worker2 # [ 124.805569] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:39Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.3:50051 clients=16078worker1 # [ 124.826588] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:542546079worker1 # [ 124.834129] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:542546080worker1 # [ 124.842204] nix-grpc-daemon[1411]: time=2026-09-20T08:18:39.632Z level=INFO msg="All 2 paths already cached"6081client # copying 1 paths to 'lb:50051'...6082worker1 # [ 124.846464] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:542546083client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6084worker1 # [ 124.854876] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6085worker1 # [ 124.999073] nix-daemon[841]: accepted connection from pid 1554, user nix-grpc-daemon (trusted)6086worker1 # [ 125.010973] nix-daemon[841]: reaped child process 1557, status = succeeded6087worker1 # [ 125.161333] nix-daemon[841]: accepted connection from pid 1559, user nix-grpc-daemon (trusted)6088worker1 # [ 125.173191] nix-daemon[841]: reaped child process 1562, status = succeeded6089lb # [ 125.189883] niks3-server[1039]: 2026/09/20 08:18:40 INFO Received uploads request method=POST path=/api/pending_closures6090worker1 # [ 125.197270] nix-grpc-daemon[1411]: time=2026-09-20T08:18:39.987Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6091worker1 # [ 125.198683] nix-grpc-daemon[1411]: time=2026-09-20T08:18:39.987Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6092lb # [ 125.215656] niks3-server[1039]: 2026/09/20 08:18:40 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6093lb # [ 125.228296] niks3-server[1039]: 2026/09/20 08:18:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6094lb # [ 125.233099] niks3-server[1039]: 2026/09/20 08:18:40 INFO Signed narinfos id=41 count=16095worker1 # [ 125.234591] nix-grpc-daemon[1411]: time=2026-09-20T08:18:40.024Z level=INFO msg="Uploading 1 narinfos"6096lb # [ 125.238806] niks3-server[1039]: 2026/09/20 08:18:40 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6097lb # [ 125.247461] niks3-server[1039]: 2026/09/20 08:18:40 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6098lb # [ 125.253323] niks3-server[1039]: 2026/09/20 08:18:40 INFO Completed upload id=416099worker1 # [ 125.252324] nix-grpc-daemon[1411]: time=2026-09-20T08:18:40.042Z level=INFO msg="Upload complete. (391ms)"6100worker1 # [ 125.254311] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54254 duration_s=0 paths=1 nar_bytes_in=7686101lb # [ 125.256214] niks3-server[1039]: 2026/09/20 08:18:40 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6102worker1 # [ 125.257334] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:542546103worker1 # [ 125.263069] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6104worker1 # [ 125.267116] nix-daemon[841]: reaped child process 1552, status = succeeded6105client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6106client # farm-top> node-a: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6107worker1 # [ 125.271712] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6108client # farm-top> LOG-farm-top-during-drain6109worker1 # [ 126.297391] nix-daemon[841]: reaped child process 1566, status = succeeded6110worker1 # [ 126.452913] nix-daemon[841]: accepted connection from pid 1581, user nix-grpc-daemon (trusted)6111worker1 # [ 126.465194] nix-daemon[841]: reaped child process 1584, status = succeeded6112lb # [ 126.480654] niks3-server[1039]: 2026/09/20 08:18:41 INFO Received uploads request method=POST path=/api/pending_closures6113worker1 # [ 126.488152] nix-grpc-daemon[1411]: time=2026-09-20T08:18:41.278Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6114worker1 # [ 126.490920] nix-grpc-daemon[1411]: time=2026-09-20T08:18:41.279Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6115lb # [ 126.510058] niks3-server[1039]: 2026/09/20 08:18:41 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6116lb # [ 126.527829] niks3-server[1039]: 2026/09/20 08:18:41 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6117lb # [ 126.532087] niks3-server[1039]: 2026/09/20 08:18:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6118lb # [ 126.537061] niks3-server[1039]: 2026/09/20 08:18:41 INFO Signed narinfos id=42 count=16119worker1 # [ 126.539292] nix-grpc-daemon[1411]: time=2026-09-20T08:18:41.329Z level=INFO msg="Uploading 1 narinfos"6120lb # [ 126.549081] niks3-server[1039]: 2026/09/20 08:18:41 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6121lb # [ 126.554763] niks3-server[1039]: 2026/09/20 08:18:41 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6122lb # [ 126.560564] niks3-server[1039]: 2026/09/20 08:18:41 INFO Completed upload id=426123lb # [ 126.562315] niks3-server[1039]: 2026/09/20 08:18:41 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6124worker1 # [ 126.561082] nix-grpc-daemon[1411]: time=2026-09-20T08:18:41.351Z level=INFO msg="Upload complete. (280ms)"6125worker1 # [ 126.567484] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54254 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16126worker1 # [ 126.580142] nix-daemon[841]: reaped child process 1564, status = succeeded6127client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag during-drain >&2, in 6.00 seconds)6128client: must succeed: nix eval --raw -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag during-drain outPath6129client: (finished: must succeed: nix eval --raw -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag during-drain outPath, in 0.21 seconds)6130worker2: must succeed: pkill -f 'read -t [9]0 x'6131worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6132client: waiting for success: ! systemctl is-active sw6133worker2 # [ 126.981470] nix-daemon[847]: reaped child process 1638, status = succeeded6134worker2 # [ 127.130578] nix-daemon[847]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6135worker2 # [ 127.141079] nix-daemon[847]: reaped child process 1818, status = succeeded6136lb # [ 127.136847] niks3-server[1039]: 2026/09/20 08:18:42 INFO Received uploads request method=POST path=/api/pending_closures6137worker2 # [ 127.164391] nix-grpc-daemon[1571]: time=2026-09-20T08:18:42.299Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6138worker2 # [ 127.165910] nix-grpc-daemon[1571]: time=2026-09-20T08:18:42.301Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6139lb # [ 127.168695] niks3-server[1039]: 2026/09/20 08:18:42 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6140lb # [ 127.178794] niks3-server[1039]: 2026/09/20 08:18:42 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6141lb # [ 127.191160] niks3-server[1039]: 2026/09/20 08:18:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6142lb # [ 127.197955] niks3-server[1039]: 2026/09/20 08:18:42 INFO Signed narinfos id=43 count=16143worker2 # [ 127.217522] nix-grpc-daemon[1571]: time=2026-09-20T08:18:42.352Z level=INFO msg="Uploading 1 narinfos"6144lb # [ 127.201309] niks3-server[1039]: 2026/09/20 08:18:42 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6145lb # [ 127.211465] niks3-server[1039]: 2026/09/20 08:18:42 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6146lb # [ 127.217085] niks3-server[1039]: 2026/09/20 08:18:42 INFO Completed upload id=436147worker2 # [ 127.236244] nix-grpc-daemon[1571]: time=2026-09-20T08:18:42.371Z level=INFO msg="Upload complete. (267ms)"6148lb # [ 127.219636] niks3-server[1039]: 2026/09/20 08:18:42 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6149worker2 # [ 127.243609] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56378 duration_s=11 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16150worker2 # [ 127.255142] nix-daemon[847]: reaped child process 1636, status = succeeded6151client # [ 127.243770] systemd[1]: sw.service: Deactivated successfully.6152client # [ 127.245629] systemd[1]: sw.service: Consumed 107ms CPU time over 13.496s wall clock time, 39.1M memory peak, 8.7K incoming IP traffic, 9K outgoing IP traffic.6153lb # [ 127.450931] envoy[910]: conn peer=[2001:db8:1::1]:38262 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9472 tx=3782 ms=57646154lb # [ 127.454635] envoy[910]: conn peer=[2001:db8:1::1]:38254 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2949 tx=1375 ms=132876155client: (finished: waiting for success: ! systemctl is-active sw, in 1.07 seconds)6156client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6157client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds)6158worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6159worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 0.06 seconds)6160lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6161lb: must succeed: curl -sf localhost:9901/clusters6162worker2 # [ 128.053669] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:43Z level=info event=signal_exit6163lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6164worker2 # [ 128.281297] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:43Z level=info event=scheduler_yield6165worker1 # [ 128.262776] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:43Z level=info event=scheduler_restarting addr=lb:500516166worker2 # [ 128.283053] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:43Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:563366167worker2 # [ 128.284380] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:43Z level=info event=worker_gone addr=192.168.1.3:500516168lb # [ 128.264598] niks3-server[1039]: 2026/09/20 08:18:43 INFO lead: released remote=192.168.1.4:601486169worker2 # [ 128.287846] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:43Z level=info event=scheduler_restarting addr=lb:500516170worker2 # [ 128.289662] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:43Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:563366171worker2 # [ 128.291208] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:43Z level=info event=worker_gone addr=192.168.1.4:500516172worker1 # [ 128.463082] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:43Z level=info event=scheduler_connected addr=lb:500516173worker2 # [ 128.488353] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:43Z level=info event=scheduler_connected addr=lb:500516174worker1 # [ 128.467916] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:43Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6175worker2 # [ 128.496292] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:43Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6176worker1 # [ 128.668839] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:43Z level=info event=scheduler_connected addr=lb:500516177worker1 # [ 128.673154] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:43Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6178worker2 # [ 128.697092] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:43Z level=info event=scheduler_connected addr=lb:500516179worker2 # [ 128.701330] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:43Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6180worker1 # [ 128.874266] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:43Z level=info event=scheduler_connected addr=lb:500516181worker1 # [ 128.878255] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:43Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6182worker2 # [ 128.901662] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:44Z level=info event=scheduler_connected addr=lb:500516183worker2 # [ 128.905602] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:44Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6184worker1 # [ 129.079187] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:43Z level=info event=scheduler_connected addr=lb:500516185worker2 # [ 129.105717] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:44Z level=info event=scheduler_connected addr=lb:500516186worker1 # [ 129.083065] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:43Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6187worker2 # [ 129.108240] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:44Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6188lb: must succeed: curl -sf localhost:9901/clusters6189lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6190worker2 # [ 129.283725] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:44Z level=info event=scheduler_restarting6191worker2 # [ 129.304447] nix-daemon[847]: reaped child process 1543, status = succeeded6192worker1 # [ 129.284675] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:44Z level=info event=scheduler_connected addr=lb:500516193worker2 # [ 129.307389] nix-grpc-daemon[1492]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6194worker2 # [ 129.319438] nix-grpc-daemon[1492]: E0000 00:00:1789892324.442252 1535 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true6195worker2 # [ 129.326822] nix-grpc-daemon[1492]: ts=2026-09-20T08:18:44Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6196worker1 # [ 130.051933] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:44Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: remote connection failure"6197worker2 # [ 130.092697] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6198worker2 # [ 130.095484] systemd[1]: nix-grpc-daemon.service: Consumed 656ms CPU time over 26.421s wall clock time, 33.2M memory peak, 24K incoming IP traffic, 32.8K outgoing IP traffic.6199lb: must succeed: curl -sf localhost:9901/clusters6200worker2 # [ 130.239824] systemd[1]: Starting Nix worker-protocol over gRPC...6201lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6202worker1 # [ 130.252588] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:45Z level=info event=scheduler_connected addr=lb:500516203worker1 # [ 130.255421] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:45Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6204worker1 # [ 130.289149] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:45Z level=info event=scheduler_take_over6205lb # [ 130.290320] niks3-server[1039]: 2026/09/20 08:18:45 INFO lead: acquired remote=192.168.1.3:493846206worker2 # [ 130.424194] nix-grpc-daemon[1830]: ts=2026-09-20T08:18:45Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6207worker2 # [ 130.428093] nix-grpc-daemon[1830]: ts=2026-09-20T08:18:45Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16208worker2 # [ 130.434058] nix-grpc-daemon[1830]: ts=2026-09-20T08:18:45Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6209worker2 # [ 130.437168] systemd[1]: Started Nix worker-protocol over gRPC.6210worker2 # [ 130.465080] nix-grpc-daemon[1830]: ts=2026-09-20T08:18:45Z level=info event=scheduler_connected addr=lb:500516211worker2 # [ 130.468917] nix-grpc-daemon[1830]: ts=2026-09-20T08:18:45Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6212worker1 # [ 130.456168] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:45Z level=info event=scheduler_connected addr=lb:500516213worker1 # [ 130.458434] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:45Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06214worker2 # [ 130.669971] nix-grpc-daemon[1830]: ts=2026-09-20T08:18:45Z level=info event=scheduler_connected addr=lb:500516215worker2 # [ 130.675378] nix-grpc-daemon[1830]: ts=2026-09-20T08:18:45Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6216worker2 # [ 130.876289] nix-grpc-daemon[1830]: ts=2026-09-20T08:18:46Z level=info event=scheduler_connected addr=lb:500516217worker1 # [ 130.858084] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:45Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06218lb: must succeed: curl -sf localhost:9901/clusters6219lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6220lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 3.28 seconds)6221lb: must succeed: curl -sf localhost:9901/clusters6222lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6223worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true6224worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.04 seconds)6225(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 17.66 seconds)6226subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6227client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16228worker1 # [ 131.606816] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:46Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:542546229worker1 # [ 131.619732] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:46Z level=debug event=want client=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06230worker1 # [ 131.625607] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:46Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16231worker1 # [ 131.629917] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:46Z level=debug event=want client=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06232worker1 # [ 131.633687] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:46Z level=debug event=want client=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06233worker1 # [ 131.662855] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:542546234worker1 # [ 131.672755] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:542546235worker1 # [ 131.682305] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:542546236worker1 # [ 131.693202] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6237worker1 # [ 131.850581] nix-daemon[841]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6238worker1 # [ 131.862931] nix-daemon[841]: reaped child process 1598, status = succeeded6239worker1 # [ 132.006101] nix-daemon[841]: accepted connection from pid 1600, user nix-grpc-daemon (trusted)6240worker1 # [ 132.017873] nix-daemon[841]: reaped child process 1603, status = succeeded6241lb # [ 132.035314] niks3-server[1039]: 2026/09/20 08:18:46 INFO Received uploads request method=POST path=/api/pending_closures6242worker1 # [ 132.046484] nix-grpc-daemon[1411]: time=2026-09-20T08:18:46.836Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6243worker1 # [ 132.047929] nix-grpc-daemon[1411]: time=2026-09-20T08:18:46.836Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6244lb # [ 132.065971] niks3-server[1039]: 2026/09/20 08:18:47 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6245lb # [ 132.075842] niks3-server[1039]: 2026/09/20 08:18:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6246lb # [ 132.081558] niks3-server[1039]: 2026/09/20 08:18:47 INFO Signed narinfos id=44 count=16247worker1 # [ 132.084359] nix-grpc-daemon[1411]: time=2026-09-20T08:18:46.874Z level=INFO msg="Uploading 1 narinfos"6248lb # [ 132.087953] niks3-server[1039]: 2026/09/20 08:18:47 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6249lb # [ 132.099833] niks3-server[1039]: 2026/09/20 08:18:47 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6250worker1 # [ 132.104135] nix-grpc-daemon[1411]: time=2026-09-20T08:18:46.894Z level=INFO msg="Upload complete. (399ms)"6251lb # [ 132.105501] niks3-server[1039]: 2026/09/20 08:18:47 INFO Completed upload id=446252lb # [ 132.108994] niks3-server[1039]: 2026/09/20 08:18:47 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6253worker1 # [ 132.107132] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54254 duration_s=0 paths=1 nar_bytes_in=5046254worker1 # [ 132.112701] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:542546255worker1 # [ 132.120969] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6256worker1 # [ 132.126606] nix-daemon[841]: reaped child process 1593, status = succeeded6257worker1 # [ 132.131312] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6258worker1 # [ 133.147226] nix-daemon[841]: reaped child process 1607, status = succeeded6259worker1 # [ 133.267414] nix-daemon[841]: accepted connection from pid 1622, user nix-grpc-daemon (trusted)6260worker1 # [ 133.277810] nix-daemon[841]: reaped child process 1625, status = succeeded6261lb # [ 133.294738] niks3-server[1039]: 2026/09/20 08:18:48 INFO Received uploads request method=POST path=/api/pending_closures6262worker1 # [ 133.304450] nix-grpc-daemon[1411]: time=2026-09-20T08:18:48.094Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6263worker1 # [ 133.305938] nix-grpc-daemon[1411]: time=2026-09-20T08:18:48.096Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6264lb # [ 133.338220] niks3-server[1039]: 2026/09/20 08:18:48 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6265lb # [ 133.344243] niks3-server[1039]: 2026/09/20 08:18:48 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6266lb # [ 133.356682] niks3-server[1039]: 2026/09/20 08:18:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6267lb # [ 133.363113] niks3-server[1039]: 2026/09/20 08:18:48 INFO Signed narinfos id=45 count=16268worker1 # [ 133.365093] nix-grpc-daemon[1411]: time=2026-09-20T08:18:48.155Z level=INFO msg="Uploading 1 narinfos"6269lb # [ 133.366087] niks3-server[1039]: 2026/09/20 08:18:48 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6270lb # [ 133.377585] niks3-server[1039]: 2026/09/20 08:18:48 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6271lb # [ 133.383509] niks3-server[1039]: 2026/09/20 08:18:48 INFO Completed upload id=456272lb # [ 133.385293] niks3-server[1039]: 2026/09/20 08:18:48 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6273worker1 # [ 133.384121] nix-grpc-daemon[1411]: time=2026-09-20T08:18:48.174Z level=INFO msg="Upload complete. (243ms)"6274worker1 # [ 133.393560] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54254 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=1 outputs=16275worker1 # [ 133.402490] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:48Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16276worker1 # [ 133.407232] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:48Z level=debug event=want client=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06277worker1 # [ 133.415868] nix-daemon[841]: reaped child process 1605, status = succeeded6278worker1 # [ 133.443112] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:542546279worker1 # [ 133.454503] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:542546280worker1 # [ 133.465848] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:542546281worker1 # [ 133.475157] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6282worker1 # [ 133.632170] nix-daemon[841]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)6283worker1 # [ 133.644991] nix-daemon[841]: reaped child process 1634, status = succeeded6284worker1 # [ 133.795707] nix-daemon[841]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)6285worker1 # [ 133.807773] nix-daemon[841]: reaped child process 1639, status = succeeded6286lb # [ 133.822666] niks3-server[1039]: 2026/09/20 08:18:48 INFO Received uploads request method=POST path=/api/pending_closures6287worker1 # [ 133.831347] nix-grpc-daemon[1411]: time=2026-09-20T08:18:48.621Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6288worker1 # [ 133.832763] nix-grpc-daemon[1411]: time=2026-09-20T08:18:48.621Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6289lb # [ 133.850477] niks3-server[1039]: 2026/09/20 08:18:48 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6290lb # [ 133.860240] niks3-server[1039]: 2026/09/20 08:18:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6291worker1 # [ 133.863685] nix-grpc-daemon[1411]: time=2026-09-20T08:18:48.653Z level=INFO msg="Uploading 1 narinfos"6292lb # [ 133.865126] niks3-server[1039]: 2026/09/20 08:18:48 INFO Signed narinfos id=46 count=16293lb # [ 133.871336] niks3-server[1039]: 2026/09/20 08:18:48 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6294lb # [ 133.883466] niks3-server[1039]: 2026/09/20 08:18:48 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6295lb # [ 133.888167] niks3-server[1039]: 2026/09/20 08:18:48 INFO Completed upload id=466296worker1 # [ 133.889512] nix-grpc-daemon[1411]: time=2026-09-20T08:18:48.679Z level=INFO msg="Upload complete. (403ms)"6297lb # [ 133.891219] niks3-server[1039]: 2026/09/20 08:18:48 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6298worker1 # [ 133.891474] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54254 duration_s=0 paths=1 nar_bytes_in=5046299worker1 # [ 133.896693] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:542546300worker1 # [ 133.907060] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6301worker1 # [ 133.911152] nix-daemon[841]: reaped child process 1629, status = succeeded6302worker1 # [ 133.915837] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6303worker1 # [ 134.930106] nix-daemon[841]: reaped child process 1643, status = succeeded6304worker1 # [ 135.046825] nix-daemon[841]: accepted connection from pid 1658, user nix-grpc-daemon (trusted)6305worker1 # [ 135.058106] nix-daemon[841]: reaped child process 1661, status = succeeded6306lb # [ 135.068245] niks3-server[1039]: 2026/09/20 08:18:50 INFO Received uploads request method=POST path=/api/pending_closures6307worker1 # [ 135.075939] nix-grpc-daemon[1411]: time=2026-09-20T08:18:49.866Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6308worker1 # [ 135.078057] nix-grpc-daemon[1411]: time=2026-09-20T08:18:49.867Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6309lb # [ 135.108120] niks3-server[1039]: 2026/09/20 08:18:50 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6310lb # [ 135.117089] niks3-server[1039]: 2026/09/20 08:18:50 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6311lb # [ 135.126188] niks3-server[1039]: 2026/09/20 08:18:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6312worker1 # [ 135.130855] nix-grpc-daemon[1411]: time=2026-09-20T08:18:49.920Z level=INFO msg="Uploading 1 narinfos"6313lb # [ 135.131758] niks3-server[1039]: 2026/09/20 08:18:50 INFO Signed narinfos id=47 count=16314lb # [ 135.138469] niks3-server[1039]: 2026/09/20 08:18:50 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6315lb # [ 135.147898] niks3-server[1039]: 2026/09/20 08:18:50 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6316lb # [ 135.153974] niks3-server[1039]: 2026/09/20 08:18:50 INFO Completed upload id=476317worker1 # [ 135.153131] nix-grpc-daemon[1411]: time=2026-09-20T08:18:49.943Z level=INFO msg="Upload complete. (230ms)"6318lb # [ 135.156639] niks3-server[1039]: 2026/09/20 08:18:50 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6319worker1 # [ 135.157187] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54254 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=2 outputs=16320worker1 # [ 135.159670] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:49Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16321worker1 # [ 135.162491] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:542546322worker1 # [ 135.165830] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:49Z level=debug event=want client=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06323worker1 # [ 135.171438] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:542546324worker1 # [ 135.173236] nix-daemon[841]: reaped child process 1641, status = succeeded6325worker1 # [ 135.177984] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:542546326worker1 # [ 135.182641] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6327worker1 # [ 135.307241] nix-daemon[841]: accepted connection from pid 1667, user nix-grpc-daemon (trusted)6328worker1 # [ 135.320391] nix-daemon[841]: reaped child process 1670, status = succeeded6329worker1 # [ 135.440637] nix-daemon[841]: accepted connection from pid 1672, user nix-grpc-daemon (trusted)6330worker1 # [ 135.453031] nix-daemon[841]: reaped child process 1675, status = succeeded6331lb # [ 135.462941] niks3-server[1039]: 2026/09/20 08:18:50 INFO Received uploads request method=POST path=/api/pending_closures6332worker1 # [ 135.470092] nix-grpc-daemon[1411]: time=2026-09-20T08:18:50.260Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6333worker1 # [ 135.471548] nix-grpc-daemon[1411]: time=2026-09-20T08:18:50.261Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6334lb # [ 135.491137] niks3-server[1039]: 2026/09/20 08:18:50 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6335lb # [ 135.502189] niks3-server[1039]: 2026/09/20 08:18:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6336worker1 # [ 135.505711] nix-grpc-daemon[1411]: time=2026-09-20T08:18:50.296Z level=INFO msg="Uploading 1 narinfos"6337lb # [ 135.507243] niks3-server[1039]: 2026/09/20 08:18:50 INFO Signed narinfos id=48 count=16338lb # [ 135.513565] niks3-server[1039]: 2026/09/20 08:18:50 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6339lb # [ 135.522594] niks3-server[1039]: 2026/09/20 08:18:50 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6340worker1 # [ 135.527888] nix-grpc-daemon[1411]: time=2026-09-20T08:18:50.318Z level=INFO msg="Upload complete. (340ms)"6341lb # [ 135.528791] niks3-server[1039]: 2026/09/20 08:18:50 INFO Completed upload id=486342worker1 # [ 135.529846] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54254 duration_s=0 paths=1 nar_bytes_in=5046343lb # [ 135.533476] niks3-server[1039]: 2026/09/20 08:18:50 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6344worker1 # [ 135.536481] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:542546345worker1 # [ 135.538445] nix-daemon[841]: reaped child process 1665, status = succeeded6346worker1 # [ 135.542035] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6347worker1 # [ 135.547285] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6348worker1 # [ 136.974903] nix-daemon[841]: reaped child process 1679, status = succeeded6349worker1 # [ 137.103538] nix-daemon[841]: accepted connection from pid 1694, user nix-grpc-daemon (trusted)6350worker1 # [ 137.115493] nix-daemon[841]: reaped child process 1697, status = succeeded6351lb # [ 137.130205] niks3-server[1039]: 2026/09/20 08:18:52 INFO Received uploads request method=POST path=/api/pending_closures6352worker1 # [ 137.140523] nix-grpc-daemon[1411]: time=2026-09-20T08:18:51.930Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6353worker1 # [ 137.143064] nix-grpc-daemon[1411]: time=2026-09-20T08:18:51.932Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6354lb # [ 137.169383] niks3-server[1039]: 2026/09/20 08:18:52 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6355lb # [ 137.178897] niks3-server[1039]: 2026/09/20 08:18:52 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv6356lb # [ 137.186706] niks3-server[1039]: 2026/09/20 08:18:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6357lb # [ 137.193545] niks3-server[1039]: 2026/09/20 08:18:52 INFO Signed narinfos id=49 count=16358worker1 # [ 137.195298] nix-grpc-daemon[1411]: time=2026-09-20T08:18:51.985Z level=INFO msg="Uploading 1 narinfos"6359lb # [ 137.199508] niks3-server[1039]: 2026/09/20 08:18:52 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls6360lb # [ 137.208214] niks3-server[1039]: 2026/09/20 08:18:52 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6361lb # [ 137.213225] niks3-server[1039]: 2026/09/20 08:18:52 INFO Completed upload id=496362lb # [ 137.216247] niks3-server[1039]: 2026/09/20 08:18:52 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo6363worker1 # [ 137.213913] nix-grpc-daemon[1411]: time=2026-09-20T08:18:52.004Z level=INFO msg="Upload complete. (255ms)"6364worker1 # [ 137.226443] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54254 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=3 outputs=16365worker1 # [ 137.231223] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:52Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=16366worker1 # [ 137.235294] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:542546367worker1 # [ 137.238722] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:52Z level=debug event=want client=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=06368worker1 # [ 137.249310] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:542546369worker1 # [ 137.251260] nix-daemon[841]: reaped child process 1677, status = succeeded6370worker1 # [ 137.258446] nix-grpc-daemon[1411]: time=2026-09-20T08:18:52.048Z level=INFO msg="All 1 paths already cached"6371worker1 # [ 137.263141] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:542546372worker1 # [ 137.268075] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6373worker1 # [ 137.415039] nix-daemon[841]: accepted connection from pid 1703, user nix-grpc-daemon (trusted)6374worker1 # [ 137.426925] nix-daemon[841]: reaped child process 1706, status = succeeded6375lb # [ 137.452938] envoy[910]: conn peer=[2001:db8:1::4]:57008 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1721 tx=552 ms=252426376worker1 # [ 137.577195] nix-daemon[841]: accepted connection from pid 1708, user nix-grpc-daemon (trusted)6377worker1 # [ 137.588895] nix-daemon[841]: reaped child process 1711, status = succeeded6378lb # [ 137.600208] niks3-server[1039]: 2026/09/20 08:18:52 INFO Received uploads request method=POST path=/api/pending_closures6379worker1 # [ 137.607576] nix-grpc-daemon[1411]: time=2026-09-20T08:18:52.397Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6380worker1 # [ 137.608932] nix-grpc-daemon[1411]: time=2026-09-20T08:18:52.397Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"6381lb # [ 137.624328] niks3-server[1039]: 2026/09/20 08:18:52 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst6382lb # [ 137.635540] niks3-server[1039]: 2026/09/20 08:18:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6383lb # [ 137.641860] niks3-server[1039]: 2026/09/20 08:18:52 INFO Signed narinfos id=50 count=16384worker1 # [ 137.644130] nix-grpc-daemon[1411]: time=2026-09-20T08:18:52.434Z level=INFO msg="Uploading 1 narinfos"6385lb # [ 137.646857] niks3-server[1039]: 2026/09/20 08:18:52 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls6386lb # [ 137.659604] niks3-server[1039]: 2026/09/20 08:18:52 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6387lb # [ 137.664396] niks3-server[1039]: 2026/09/20 08:18:52 INFO Completed upload id=506388worker1 # [ 137.665724] nix-grpc-daemon[1411]: time=2026-09-20T08:18:52.456Z level=INFO msg="Upload complete. (392ms)"6389worker1 # [ 137.667671] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54254 duration_s=0 paths=1 nar_bytes_in=6246390lb # [ 137.667200] niks3-server[1039]: 2026/09/20 08:18:52 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo6391worker1 # [ 137.672599] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:542546392worker1 # [ 137.679058] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6393worker1 # [ 137.681226] nix-daemon[841]: reaped child process 1701, status = succeeded6394worker1 # [ 137.686222] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6395worker1 # [ 138.714317] nix-daemon[841]: reaped child process 1715, status = succeeded6396worker1 # [ 138.834321] nix-daemon[841]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)6397worker1 # [ 138.844960] nix-daemon[841]: reaped child process 1733, status = succeeded6398lb # [ 138.856495] niks3-server[1039]: 2026/09/20 08:18:53 INFO Received uploads request method=POST path=/api/pending_closures6399worker1 # [ 138.863178] nix-grpc-daemon[1411]: time=2026-09-20T08:18:53.653Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6400worker1 # [ 138.864830] nix-grpc-daemon[1411]: time=2026-09-20T08:18:53.655Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"6401lb # [ 138.888797] niks3-server[1039]: 2026/09/20 08:18:53 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv6402lb # [ 138.904996] niks3-server[1039]: 2026/09/20 08:18:53 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst6403lb # [ 138.911809] niks3-server[1039]: 2026/09/20 08:18:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6404lb # [ 138.916491] niks3-server[1039]: 2026/09/20 08:18:53 INFO Signed narinfos id=51 count=16405worker1 # [ 138.918590] nix-grpc-daemon[1411]: time=2026-09-20T08:18:53.708Z level=INFO msg="Uploading 1 narinfos"6406lb # [ 138.930801] niks3-server[1039]: 2026/09/20 08:18:53 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6407worker1 # [ 138.935498] nix-grpc-daemon[1411]: time=2026-09-20T08:18:53.725Z level=INFO msg="Upload complete. (241ms)"6408lb # [ 138.936414] niks3-server[1039]: 2026/09/20 08:18:53 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls6409lb # [ 138.939249] niks3-server[1039]: 2026/09/20 08:18:53 INFO Completed upload id=516410worker1 # [ 138.941375] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54254 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=4 outputs=16411lb # [ 138.949483] niks3-server[1039]: 2026/09/20 08:18:53 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo6412worker1 # [ 138.951774] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:53Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=16413worker1 # [ 138.956354] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:542546414worker1 # [ 138.959735] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:53Z level=debug event=want client=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=06415worker1 # [ 138.967448] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:542546416worker1 # [ 138.971714] nix-daemon[841]: reaped child process 1713, status = succeeded6417worker1 # [ 138.973839] nix-grpc-daemon[1411]: time=2026-09-20T08:18:53.764Z level=INFO msg="All 1 paths already cached"6418worker1 # [ 138.977477] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:542546419worker1 # [ 138.982348] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6420worker1 # [ 139.106548] nix-daemon[841]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)6421worker1 # [ 139.120096] nix-daemon[841]: reaped child process 1742, status = succeeded6422worker1 # [ 139.263482] nix-daemon[841]: accepted connection from pid 1744, user nix-grpc-daemon (trusted)6423worker1 # [ 139.275632] nix-daemon[841]: reaped child process 1747, status = succeeded6424lb # [ 139.286489] niks3-server[1039]: 2026/09/20 08:18:54 INFO Received uploads request method=POST path=/api/pending_closures6425worker1 # [ 139.296080] nix-grpc-daemon[1411]: time=2026-09-20T08:18:54.086Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6426worker1 # [ 139.297551] nix-grpc-daemon[1411]: time=2026-09-20T08:18:54.087Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"6427lb # [ 139.317538] niks3-server[1039]: 2026/09/20 08:18:54 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst6428lb # [ 139.329410] niks3-server[1039]: 2026/09/20 08:18:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6429lb # [ 139.335568] niks3-server[1039]: 2026/09/20 08:18:54 INFO Signed narinfos id=52 count=16430worker1 # [ 139.337718] nix-grpc-daemon[1411]: time=2026-09-20T08:18:54.128Z level=INFO msg="Uploading 1 narinfos"6431lb # [ 139.339231] niks3-server[1039]: 2026/09/20 08:18:54 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls6432lb # [ 139.351331] niks3-server[1039]: 2026/09/20 08:18:54 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6433lb # [ 139.359668] niks3-server[1039]: 2026/09/20 08:18:54 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo6434lb # [ 139.364639] niks3-server[1039]: 2026/09/20 08:18:54 INFO Completed upload id=526435worker1 # [ 139.363744] nix-grpc-daemon[1411]: time=2026-09-20T08:18:54.153Z level=INFO msg="Upload complete. (376ms)"6436worker1 # [ 139.367281] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54254 duration_s=0 paths=1 nar_bytes_in=6246437worker1 # [ 139.372518] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:542546438worker1 # [ 139.382105] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6439worker1 # [ 139.389118] nix-daemon[841]: reaped child process 1737, status = succeeded6440worker1 # [ 139.393937] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6441worker1 # [ 140.421529] nix-daemon[841]: reaped child process 1751, status = succeeded6442worker1 # [ 140.537682] nix-daemon[841]: accepted connection from pid 1766, user nix-grpc-daemon (trusted)6443worker1 # [ 140.549759] nix-daemon[841]: reaped child process 1769, status = succeeded6444lb # [ 140.565524] niks3-server[1039]: 2026/09/20 08:18:55 INFO Received uploads request method=POST path=/api/pending_closures6445worker1 # [ 140.575423] nix-grpc-daemon[1411]: time=2026-09-20T08:18:55.365Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6446worker1 # [ 140.577047] nix-grpc-daemon[1411]: time=2026-09-20T08:18:55.367Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"6447lb # [ 140.612114] niks3-server[1039]: 2026/09/20 08:18:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6448lb # [ 140.617613] niks3-server[1039]: 2026/09/20 08:18:55 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst6449lb # [ 140.623388] niks3-server[1039]: 2026/09/20 08:18:55 INFO Signed narinfos id=53 count=16450worker1 # [ 140.625744] nix-grpc-daemon[1411]: time=2026-09-20T08:18:55.415Z level=INFO msg="Uploading 1 narinfos"6451lb # [ 140.629739] niks3-server[1039]: 2026/09/20 08:18:55 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv6452lb # [ 140.634920] niks3-server[1039]: 2026/09/20 08:18:55 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls6453lb # [ 140.644103] niks3-server[1039]: 2026/09/20 08:18:55 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6454lb # [ 140.649937] niks3-server[1039]: 2026/09/20 08:18:55 INFO Completed upload id=536455lb # [ 140.652988] niks3-server[1039]: 2026/09/20 08:18:55 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo6456worker1 # [ 140.650691] nix-grpc-daemon[1411]: time=2026-09-20T08:18:55.440Z level=INFO msg="Upload complete. (249ms)"6457worker1 # [ 140.663357] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54254 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=5 outputs=16458worker1 # [ 140.669310] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:55Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=16459worker1 # [ 140.673611] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:542546460worker1 # [ 140.677255] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:55Z level=debug event=want client=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=06461worker1 # [ 140.687778] nix-daemon[841]: reaped child process 1749, status = succeeded6462worker1 # [ 140.689764] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:542546463worker1 # [ 140.697772] nix-grpc-daemon[1411]: time=2026-09-20T08:18:55.488Z level=INFO msg="All 1 paths already cached"6464worker1 # [ 140.702603] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:542546465worker1 # [ 140.706611] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6466worker1 # [ 140.852809] nix-daemon[841]: accepted connection from pid 1775, user nix-grpc-daemon (trusted)6467worker1 # [ 140.864558] nix-daemon[841]: reaped child process 1778, status = succeeded6468worker1 # [ 141.018837] nix-daemon[841]: accepted connection from pid 1780, user nix-grpc-daemon (trusted)6469worker1 # [ 141.031052] nix-daemon[841]: reaped child process 1783, status = succeeded6470lb # [ 141.046284] niks3-server[1039]: 2026/09/20 08:18:55 INFO Received uploads request method=POST path=/api/pending_closures6471worker1 # [ 141.052652] nix-grpc-daemon[1411]: time=2026-09-20T08:18:55.842Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6472worker1 # [ 141.054195] nix-grpc-daemon[1411]: time=2026-09-20T08:18:55.844Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"6473lb # [ 141.076972] niks3-server[1039]: 2026/09/20 08:18:56 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst6474lb # [ 141.086775] niks3-server[1039]: 2026/09/20 08:18:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6475lb # [ 141.093404] niks3-server[1039]: 2026/09/20 08:18:56 INFO Signed narinfos id=54 count=16476worker1 # [ 141.095536] nix-grpc-daemon[1411]: time=2026-09-20T08:18:55.885Z level=INFO msg="Uploading 1 narinfos"6477lb # [ 141.098222] niks3-server[1039]: 2026/09/20 08:18:56 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls6478lb # [ 141.110683] niks3-server[1039]: 2026/09/20 08:18:56 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6479lb # [ 141.115163] niks3-server[1039]: 2026/09/20 08:18:56 INFO Completed upload id=546480worker1 # [ 141.114125] nix-grpc-daemon[1411]: time=2026-09-20T08:18:55.904Z level=INFO msg="Upload complete. (401ms)"6481lb # [ 141.117460] niks3-server[1039]: 2026/09/20 08:18:56 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo6482worker1 # [ 141.117911] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54254 duration_s=0 paths=1 nar_bytes_in=6246483worker1 # [ 141.122891] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:542546484worker1 # [ 141.133057] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6485worker1 # [ 141.137080] nix-daemon[841]: reaped child process 1773, status = succeeded6486worker1 # [ 141.141707] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6487worker1 # [ 142.367638] nix-daemon[841]: reaped child process 1787, status = succeeded6488worker1 # [ 142.513119] nix-daemon[841]: accepted connection from pid 1802, user nix-grpc-daemon (trusted)6489worker1 # [ 142.524332] nix-daemon[841]: reaped child process 1805, status = succeeded6490lb # [ 142.540425] niks3-server[1039]: 2026/09/20 08:18:57 INFO Received uploads request method=POST path=/api/pending_closures6491worker1 # [ 142.548311] nix-grpc-daemon[1411]: time=2026-09-20T08:18:57.338Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6492worker1 # [ 142.551710] nix-grpc-daemon[1411]: time=2026-09-20T08:18:57.340Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"6493lb # [ 142.574795] niks3-server[1039]: 2026/09/20 08:18:57 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst6494lb # [ 142.584483] niks3-server[1039]: 2026/09/20 08:18:57 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv6495lb # [ 142.593824] niks3-server[1039]: 2026/09/20 08:18:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6496worker1 # [ 142.599364] nix-grpc-daemon[1411]: time=2026-09-20T08:18:57.389Z level=INFO msg="Uploading 1 narinfos"6497lb # [ 142.600286] niks3-server[1039]: 2026/09/20 08:18:57 INFO Signed narinfos id=55 count=16498lb # [ 142.606227] niks3-server[1039]: 2026/09/20 08:18:57 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls6499lb # [ 142.615206] niks3-server[1039]: 2026/09/20 08:18:57 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6500lb # [ 142.621375] niks3-server[1039]: 2026/09/20 08:18:57 INFO Completed upload id=556501lb # [ 142.623420] niks3-server[1039]: 2026/09/20 08:18:57 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo6502worker1 # [ 142.622056] nix-grpc-daemon[1411]: time=2026-09-20T08:18:57.412Z level=INFO msg="Upload complete. (272ms)"6503worker1 # [ 142.630433] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54254 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=6 outputs=16504worker1 # [ 142.635927] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:57Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=16505worker1 # [ 142.646510] nix-daemon[841]: reaped child process 1785, status = succeeded6506worker1 # [ 142.672426] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:542546507worker1 # [ 142.682568] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:57Z level=debug event=want client=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=06508worker1 # [ 142.690776] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:542546509worker1 # [ 142.702878] nix-grpc-daemon[1411]: time=2026-09-20T08:18:57.493Z level=INFO msg="All 2 paths already cached"6510worker1 # [ 142.709752] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:542546511worker1 # [ 142.719129] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6512worker1 # [ 142.852269] nix-daemon[841]: accepted connection from pid 1811, user nix-grpc-daemon (trusted)6513worker1 # [ 142.864270] nix-daemon[841]: reaped child process 1814, status = succeeded6514worker1 # [ 142.992240] nix-daemon[841]: accepted connection from pid 1816, user nix-grpc-daemon (trusted)6515worker1 # [ 143.003869] nix-daemon[841]: reaped child process 1819, status = succeeded6516lb # [ 143.015416] niks3-server[1039]: 2026/09/20 08:18:57 INFO Received uploads request method=POST path=/api/pending_closures6517worker1 # [ 143.022457] nix-grpc-daemon[1411]: time=2026-09-20T08:18:57.812Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6518worker1 # [ 143.023885] nix-grpc-daemon[1411]: time=2026-09-20T08:18:57.812Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"6519lb # [ 143.039780] niks3-server[1039]: 2026/09/20 08:18:57 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst6520lb # [ 143.051205] niks3-server[1039]: 2026/09/20 08:18:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6521lb # [ 143.056249] niks3-server[1039]: 2026/09/20 08:18:57 INFO Signed narinfos id=56 count=16522worker1 # [ 143.058385] nix-grpc-daemon[1411]: time=2026-09-20T08:18:57.848Z level=INFO msg="Uploading 1 narinfos"6523lb # [ 143.063314] niks3-server[1039]: 2026/09/20 08:18:58 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls6524lb # [ 143.072429] niks3-server[1039]: 2026/09/20 08:18:58 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6525worker1 # [ 143.077281] nix-grpc-daemon[1411]: time=2026-09-20T08:18:57.867Z level=INFO msg="Upload complete. (349ms)"6526lb # [ 143.078274] niks3-server[1039]: 2026/09/20 08:18:58 INFO Completed upload id=566527worker1 # [ 143.079708] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54254 duration_s=0 paths=1 nar_bytes_in=7526528lb # [ 143.083982] niks3-server[1039]: 2026/09/20 08:18:58 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo6529worker1 # [ 143.085494] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:542546530worker1 # [ 143.090373] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6531worker1 # [ 143.095293] nix-daemon[841]: reaped child process 1809, status = succeeded6532worker1 # [ 143.100240] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6533worker1 # [ 144.120543] nix-daemon[841]: reaped child process 1823, status = succeeded6534worker1 # [ 144.275390] nix-daemon[841]: accepted connection from pid 1838, user nix-grpc-daemon (trusted)6535worker1 # [ 144.286240] nix-daemon[841]: reaped child process 1841, status = succeeded6536lb # [ 144.304655] niks3-server[1039]: 2026/09/20 08:18:59 INFO Received uploads request method=POST path=/api/pending_closures6537worker1 # [ 144.311863] nix-grpc-daemon[1411]: time=2026-09-20T08:18:59.101Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6538worker1 # [ 144.313404] nix-grpc-daemon[1411]: time=2026-09-20T08:18:59.103Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"6539lb # [ 144.329670] niks3-server[1039]: 2026/09/20 08:18:59 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst6540lb # [ 144.332939] niks3-server[1039]: 2026/09/20 08:18:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6541lb # [ 144.334870] niks3-server[1039]: 2026/09/20 08:18:59 INFO Signed narinfos id=57 count=16542worker1 # [ 144.335275] nix-grpc-daemon[1411]: time=2026-09-20T08:18:59.125Z level=INFO msg="Uploading 1 narinfos"6543lb # [ 144.337344] niks3-server[1039]: 2026/09/20 08:18:59 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls6544lb # [ 144.340120] niks3-server[1039]: 2026/09/20 08:18:59 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv6545lb # [ 144.346194] niks3-server[1039]: 2026/09/20 08:18:59 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6546lb # [ 144.348693] niks3-server[1039]: 2026/09/20 08:18:59 INFO Completed upload id=576547worker1 # [ 144.348902] nix-grpc-daemon[1411]: time=2026-09-20T08:18:59.139Z level=INFO msg="Upload complete. (242ms)"6548lb # [ 144.350905] niks3-server[1039]: 2026/09/20 08:18:59 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo6549worker1 # [ 144.351391] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54254 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=7 outputs=16550worker1 # [ 144.355550] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:59Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=16551worker1 # [ 144.357812] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:542546552worker1 # [ 144.364177] nix-daemon[841]: reaped child process 1821, status = succeeded6553worker1 # [ 144.401593] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:542546554worker1 # [ 144.412544] nix-grpc-daemon[1411]: time=2026-09-20T08:18:59.202Z level=INFO msg="All 2 paths already cached"6555worker1 # [ 144.418927] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:542546556worker1 # [ 144.425826] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6557worker1 # [ 144.587221] nix-daemon[841]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)6558worker1 # [ 144.599687] nix-daemon[841]: reaped child process 1848, status = succeeded6559worker1 # [ 144.753701] nix-daemon[841]: accepted connection from pid 1850, user nix-grpc-daemon (trusted)6560worker1 # [ 144.765244] nix-daemon[841]: reaped child process 1853, status = succeeded6561lb # [ 144.782573] niks3-server[1039]: 2026/09/20 08:18:59 INFO Received uploads request method=POST path=/api/pending_closures6562worker1 # [ 144.789401] nix-grpc-daemon[1411]: time=2026-09-20T08:18:59.579Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6563worker1 # [ 144.790852] nix-grpc-daemon[1411]: time=2026-09-20T08:18:59.579Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"6564lb # [ 144.809224] niks3-server[1039]: 2026/09/20 08:18:59 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst6565lb # [ 144.817529] niks3-server[1039]: 2026/09/20 08:18:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6566lb # [ 144.823911] niks3-server[1039]: 2026/09/20 08:18:59 INFO Signed narinfos id=58 count=16567worker1 # [ 144.826048] nix-grpc-daemon[1411]: time=2026-09-20T08:18:59.616Z level=INFO msg="Uploading 1 narinfos"6568lb # [ 144.828579] niks3-server[1039]: 2026/09/20 08:18:59 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls6569lb # [ 144.840623] niks3-server[1039]: 2026/09/20 08:18:59 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6570lb # [ 144.847099] niks3-server[1039]: 2026/09/20 08:18:59 INFO Completed upload id=586571worker1 # [ 144.848358] nix-grpc-daemon[1411]: time=2026-09-20T08:18:59.638Z level=INFO msg="Upload complete. (411ms)"6572lb # [ 144.850303] niks3-server[1039]: 2026/09/20 08:18:59 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo6573worker1 # [ 144.851827] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54254 duration_s=0 paths=1 nar_bytes_in=7526574worker1 # [ 144.857288] nix-grpc-daemon[1323]: ts=2026-09-20T08:18:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:542546575worker1 # [ 144.865262] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6576worker1 # [ 144.870347] nix-daemon[841]: reaped child process 1843, status = succeeded6577worker1 # [ 144.876207] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6578worker1 # [ 146.088330] nix-daemon[841]: reaped child process 1857, status = succeeded6579worker1 # [ 146.214943] nix-daemon[841]: accepted connection from pid 1872, user nix-grpc-daemon (trusted)6580worker1 # [ 146.227270] nix-daemon[841]: reaped child process 1875, status = succeeded6581lb # [ 146.239239] niks3-server[1039]: 2026/09/20 08:19:01 INFO Received uploads request method=POST path=/api/pending_closures6582worker1 # [ 146.246763] nix-grpc-daemon[1411]: time=2026-09-20T08:19:01.036Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6583worker1 # [ 146.249751] nix-grpc-daemon[1411]: time=2026-09-20T08:19:01.038Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"6584lb # [ 146.275519] niks3-server[1039]: 2026/09/20 08:19:01 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst6585lb # [ 146.282603] niks3-server[1039]: 2026/09/20 08:19:01 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv6586lb # [ 146.291119] niks3-server[1039]: 2026/09/20 08:19:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6587worker1 # [ 146.295072] nix-grpc-daemon[1411]: time=2026-09-20T08:19:01.085Z level=INFO msg="Uploading 1 narinfos"6588lb # [ 146.296244] niks3-server[1039]: 2026/09/20 08:19:01 INFO Signed narinfos id=59 count=16589lb # [ 146.299503] niks3-server[1039]: 2026/09/20 08:19:01 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls6590lb # [ 146.308057] niks3-server[1039]: 2026/09/20 08:19:01 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6591lb # [ 146.310801] niks3-server[1039]: 2026/09/20 08:19:01 INFO Completed upload id=596592worker1 # [ 146.309917] nix-grpc-daemon[1411]: time=2026-09-20T08:19:01.100Z level=INFO msg="Upload complete. (227ms)"6593lb # [ 146.312433] niks3-server[1039]: 2026/09/20 08:19:01 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo6594worker1 # [ 146.315740] nix-grpc-daemon[1323]: ts=2026-09-20T08:19:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54254 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=8 outputs=16595worker1 # [ 146.321532] nix-grpc-daemon[1323]: ts=2026-09-20T08:19:01Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=16596worker1 # [ 146.326229] nix-grpc-daemon[1323]: ts=2026-09-20T08:19:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:542546597worker1 # [ 146.334303] nix-grpc-daemon[1323]: ts=2026-09-20T08:19:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:542546598worker1 # [ 146.336115] nix-daemon[841]: reaped child process 1855, status = succeeded6599worker1 # [ 146.341639] nix-grpc-daemon[1411]: time=2026-09-20T08:19:01.131Z level=INFO msg="All 2 paths already cached"6600worker1 # [ 146.345696] nix-grpc-daemon[1323]: ts=2026-09-20T08:19:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:542546601worker1 # [ 146.349372] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6602worker1 # [ 146.491259] nix-daemon[841]: accepted connection from pid 1879, user nix-grpc-daemon (trusted)6603worker1 # [ 146.503453] nix-daemon[841]: reaped child process 1882, status = succeeded6604worker1 # [ 146.659259] nix-daemon[841]: accepted connection from pid 1884, user nix-grpc-daemon (trusted)6605worker1 # [ 146.671236] nix-daemon[841]: reaped child process 1887, status = succeeded6606lb # [ 146.686986] niks3-server[1039]: 2026/09/20 08:19:01 INFO Received uploads request method=POST path=/api/pending_closures6607worker1 # [ 146.694695] nix-grpc-daemon[1411]: time=2026-09-20T08:19:01.484Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6608worker1 # [ 146.696171] nix-grpc-daemon[1411]: time=2026-09-20T08:19:01.484Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"6609lb # [ 146.715176] niks3-server[1039]: 2026/09/20 08:19:01 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst6610lb # [ 146.725905] niks3-server[1039]: 2026/09/20 08:19:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6611worker1 # [ 146.729683] nix-grpc-daemon[1411]: time=2026-09-20T08:19:01.519Z level=INFO msg="Uploading 1 narinfos"6612lb # [ 146.730599] niks3-server[1039]: 2026/09/20 08:19:01 INFO Signed narinfos id=60 count=16613lb # [ 146.735929] niks3-server[1039]: 2026/09/20 08:19:01 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls6614lb # [ 146.746569] niks3-server[1039]: 2026/09/20 08:19:01 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6615lb # [ 146.749365] niks3-server[1039]: 2026/09/20 08:19:01 INFO Completed upload id=606616worker1 # [ 146.748409] nix-grpc-daemon[1411]: time=2026-09-20T08:19:01.538Z level=INFO msg="Upload complete. (393ms)"6617lb # [ 146.751281] niks3-server[1039]: 2026/09/20 08:19:01 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo6618worker1 # [ 146.753136] nix-grpc-daemon[1323]: ts=2026-09-20T08:19:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54254 duration_s=0 paths=1 nar_bytes_in=7526619worker1 # [ 146.757654] nix-grpc-daemon[1323]: ts=2026-09-20T08:19:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:542546620worker1 # [ 146.765862] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6621worker1 # [ 146.770452] nix-daemon[841]: reaped child process 1877, status = succeeded6622worker1 # [ 146.776259] nix-daemon[841]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6623worker1 # [ 147.988928] nix-daemon[841]: reaped child process 1891, status = succeeded6624worker1 # [ 148.105707] nix-daemon[841]: accepted connection from pid 1906, user nix-grpc-daemon (trusted)6625worker1 # [ 148.116642] nix-daemon[841]: reaped child process 1909, status = succeeded6626lb # [ 148.129364] niks3-server[1039]: 2026/09/20 08:19:03 INFO Received uploads request method=POST path=/api/pending_closures6627worker1 # [ 148.136380] nix-grpc-daemon[1411]: time=2026-09-20T08:19:02.926Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6628worker1 # [ 148.138442] nix-grpc-daemon[1411]: time=2026-09-20T08:19:02.928Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"6629lb # [ 148.165941] niks3-server[1039]: 2026/09/20 08:19:03 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv6630lb # [ 148.174624] niks3-server[1039]: 2026/09/20 08:19:03 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst6631lb # [ 148.183078] niks3-server[1039]: 2026/09/20 08:19:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6632lb # [ 148.189589] niks3-server[1039]: 2026/09/20 08:19:03 INFO Signed narinfos id=61 count=16633worker1 # [ 148.191657] nix-grpc-daemon[1411]: time=2026-09-20T08:19:02.981Z level=INFO msg="Uploading 1 narinfos"6634lb # [ 148.195153] niks3-server[1039]: 2026/09/20 08:19:03 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls6635lb # [ 148.204743] niks3-server[1039]: 2026/09/20 08:19:03 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6636lb # [ 148.210257] niks3-server[1039]: 2026/09/20 08:19:03 INFO Completed upload id=616637lb # [ 148.211794] niks3-server[1039]: 2026/09/20 08:19:03 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo6638worker1 # [ 148.209544] nix-grpc-daemon[1411]: time=2026-09-20T08:19:02.999Z level=INFO msg="Upload complete. (226ms)"6639worker1 # [ 148.217302] nix-grpc-daemon[1323]: ts=2026-09-20T08:19:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54254 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=9 outputs=16640worker1 # [ 148.229439] nix-daemon[841]: reaped child process 1889, status = succeeded6641client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 16.87 seconds)6642worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -E 'nix_grpc_build_info\{.*features="[^"]*vip[^"]*".*worker="node-a"\} 1'6643worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -E 'nix_grpc_build_info\{.*features="[^"]*vip[^"]*".*worker="node-a"\} 1', in 0.06 seconds)6644client: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&16645worker1 # [ 148.512107] nix-grpc-daemon[1323]: ts=2026-09-20T08:19:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:542546646worker1 # [ 148.525306] nix-grpc-daemon[1323]: ts=2026-09-20T08:19:03Z level=debug event=want client=2 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=06647client: (finished: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&1, in 0.24 seconds)6648(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 17.17 seconds)6649subtest: low disk drains a worker and builds go to the other6650worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6651worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.96 seconds)6652worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6653worker1 # [ 150.298959] nix-grpc-daemon[1323]: ts=2026-09-20T08:19:05Z level=info event=unhealthy reason=min_free6654worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.10 seconds)6655client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag drain >&26656worker2 # [ 150.811774] nix-grpc-daemon[1830]: ts=2026-09-20T08:19:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:560166657worker2 # [ 150.823454] nix-daemon[847]: accepted connection from pid 1830, user nix-grpc-daemon (trusted)6658client # don't know how to build these paths:6659client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6660worker1 # [ 150.824590] nix-grpc-daemon[1323]: ts=2026-09-20T08:19:05Z level=debug event=want client=3 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=06661worker1 # [ 150.828715] nix-grpc-daemon[1323]: ts=2026-09-20T08:19:05Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=16662worker2 # [ 150.888990] nix-grpc-daemon[1830]: ts=2026-09-20T08:19:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:560166663worker2 # [ 150.900814] nix-grpc-daemon[1830]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 49 ms (attempt 1/5)6664worker2 # [ 150.955462] nix-grpc-daemon[1830]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 51 ms (attempt 2/5)6665worker2 # [ 151.012488] nix-grpc-daemon[1830]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 152 ms (attempt 3/5)6666worker2 # [ 151.170589] nix-grpc-daemon[1830]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 400 ms (attempt 4/5)6667worker2 # [ 151.576888] nix-grpc-daemon[1830]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6668worker2 # [ 151.592101] nix-grpc-daemon[1830]: ts=2026-09-20T08:19:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:560166669client # copying 1 paths to 'lb:50051'...6670client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...6671worker2 # [ 151.603072] nix-grpc-daemon[1830]: ts=2026-09-20T08:19:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:560166672worker2 # [ 151.613158] nix-daemon[847]: accepted connection from pid 1830, user nix-grpc-daemon (trusted)6673worker2 # [ 151.632314] nix-grpc-daemon[1830]: ts=2026-09-20T08:19:06Z level=info event=niks3_push_started pid=18866674worker2 # [ 151.951274] nix-daemon[847]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6675worker2 # [ 151.963311] nix-daemon[847]: reaped child process 1902, status = succeeded6676worker2 # [ 152.133890] nix-daemon[847]: accepted connection from pid 1904, user nix-grpc-daemon (trusted)6677worker2 # [ 152.144757] nix-daemon[847]: reaped child process 1907, status = succeeded6678lb # [ 152.144803] niks3-server[1039]: 2026/09/20 08:19:07 INFO Received uploads request method=POST path=/api/pending_closures6679worker2 # [ 152.172347] nix-grpc-daemon[1886]: time=2026-09-20T08:19:07.307Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6680worker2 # [ 152.174398] nix-grpc-daemon[1886]: time=2026-09-20T08:19:07.309Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"6681lb # [ 152.177087] niks3-server[1039]: 2026/09/20 08:19:07 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst6682lb # [ 152.187824] niks3-server[1039]: 2026/09/20 08:19:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6683lb # [ 152.192767] niks3-server[1039]: 2026/09/20 08:19:07 INFO Signed narinfos id=62 count=16684worker2 # [ 152.215323] nix-grpc-daemon[1886]: time=2026-09-20T08:19:07.350Z level=INFO msg="Uploading 1 narinfos"6685lb # [ 152.205768] niks3-server[1039]: 2026/09/20 08:19:07 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls6686lb # [ 152.214264] niks3-server[1039]: 2026/09/20 08:19:07 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6687worker2 # [ 152.240883] nix-grpc-daemon[1886]: time=2026-09-20T08:19:07.376Z level=INFO msg="Upload complete. (429ms)"6688lb # [ 152.221689] niks3-server[1039]: 2026/09/20 08:19:07 INFO Completed upload id=626689lb # [ 152.224879] niks3-server[1039]: 2026/09/20 08:19:07 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo6690worker2 # [ 152.244316] nix-grpc-daemon[1830]: ts=2026-09-20T08:19:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56016 duration_s=0 paths=1 nar_bytes_in=5126691worker2 # [ 152.249548] nix-grpc-daemon[1830]: ts=2026-09-20T08:19:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:560166692worker2 # [ 152.260189] nix-daemon[847]: accepted connection from pid 1830, user nix-grpc-daemon (trusted)6693worker2 # [ 152.261794] nix-daemon[847]: reaped child process 1884, status = succeeded6694worker2 # [ 152.266844] nix-daemon[847]: accepted connection from pid 1830, user nix-grpc-daemon (trusted)6695client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...6696client # farm-a> worker2-next: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6697client # farm-a> LOG-farm-a-drain6698worker2 # [ 152.894851] nix-daemon[847]: reaped child process 1911, status = succeeded6699worker2 # [ 153.066494] nix-daemon[847]: accepted connection from pid 1926, user nix-grpc-daemon (trusted)6700worker2 # [ 153.080550] nix-daemon[847]: reaped child process 1929, status = succeeded6701lb # [ 153.079660] niks3-server[1039]: 2026/09/20 08:19:08 INFO Received uploads request method=POST path=/api/pending_closures6702worker2 # [ 153.103371] nix-grpc-daemon[1886]: time=2026-09-20T08:19:08.238Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6703worker2 # [ 153.104956] nix-grpc-daemon[1886]: time=2026-09-20T08:19:08.240Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"6704lb # [ 153.108633] niks3-server[1039]: 2026/09/20 08:19:08 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6705lb # [ 153.122448] niks3-server[1039]: 2026/09/20 08:19:08 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6706lb # [ 153.134719] niks3-server[1039]: 2026/09/20 08:19:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6707lb # [ 153.141369] niks3-server[1039]: 2026/09/20 08:19:08 INFO Signed narinfos id=63 count=16708worker2 # [ 153.163746] nix-grpc-daemon[1886]: time=2026-09-20T08:19:08.298Z level=INFO msg="Uploading 1 narinfos"6709lb # [ 153.144815] niks3-server[1039]: 2026/09/20 08:19:08 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls6710lb # [ 153.156285] niks3-server[1039]: 2026/09/20 08:19:08 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6711lb # [ 153.160538] niks3-server[1039]: 2026/09/20 08:19:08 INFO Completed upload id=636712worker2 # [ 153.182412] nix-grpc-daemon[1886]: time=2026-09-20T08:19:08.317Z level=INFO msg="Upload complete. (308ms)"6713lb # [ 153.164410] niks3-server[1039]: 2026/09/20 08:19:08 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo6714worker2 # [ 153.188308] nix-grpc-daemon[1830]: ts=2026-09-20T08:19:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56016 duration_s=0 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=10 outputs=16715worker1 # [ 153.174917] nix-grpc-daemon[1323]: ts=2026-09-20T08:19:07Z level=debug event=want client=3 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=06716worker1 # [ 153.179138] nix-grpc-daemon[1323]: ts=2026-09-20T08:19:07Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=16717worker2 # [ 153.202148] nix-daemon[847]: reaped child process 1909, status = succeeded6718worker2 # [ 153.240257] nix-grpc-daemon[1830]: ts=2026-09-20T08:19:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:560166719worker2 # [ 153.253695] nix-grpc-daemon[1830]: ts=2026-09-20T08:19:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:560166720worker2 # [ 153.265732] nix-grpc-daemon[1886]: time=2026-09-20T08:19:08.401Z level=INFO msg="All 1 paths already cached"6721client # copying 1 paths to 'lb:50051'...6722client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...6723worker2 # [ 153.274539] nix-grpc-daemon[1830]: ts=2026-09-20T08:19:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:560166724worker2 # [ 153.283441] nix-daemon[847]: accepted connection from pid 1830, user nix-grpc-daemon (trusted)6725worker2 # [ 153.475751] nix-daemon[847]: accepted connection from pid 1933, user nix-grpc-daemon (trusted)6726worker2 # [ 153.490899] nix-daemon[847]: reaped child process 1936, status = succeeded6727worker2 # [ 153.668630] nix-daemon[847]: accepted connection from pid 1938, user nix-grpc-daemon (trusted)6728worker2 # [ 153.680724] nix-daemon[847]: reaped child process 1941, status = succeeded6729lb # [ 153.677958] niks3-server[1039]: 2026/09/20 08:19:08 INFO Received uploads request method=POST path=/api/pending_closures6730worker2 # [ 153.706998] nix-grpc-daemon[1886]: time=2026-09-20T08:19:08.842Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6731worker2 # [ 153.708968] nix-grpc-daemon[1886]: time=2026-09-20T08:19:08.843Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"6732lb # [ 153.707119] niks3-server[1039]: 2026/09/20 08:19:08 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst6733lb # [ 153.717784] niks3-server[1039]: 2026/09/20 08:19:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6734worker2 # [ 153.743945] nix-grpc-daemon[1886]: time=2026-09-20T08:19:08.879Z level=INFO msg="Uploading 1 narinfos"6735lb # [ 153.724629] niks3-server[1039]: 2026/09/20 08:19:08 INFO Signed narinfos id=64 count=16736lb # [ 153.729598] niks3-server[1039]: 2026/09/20 08:19:08 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls6737lb # [ 153.740997] niks3-server[1039]: 2026/09/20 08:19:08 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6738lb # [ 153.748209] niks3-server[1039]: 2026/09/20 08:19:08 INFO Completed upload id=646739worker2 # [ 153.767356] nix-grpc-daemon[1886]: time=2026-09-20T08:19:08.902Z level=INFO msg="Upload complete. (470ms)"6740lb # [ 153.752084] niks3-server[1039]: 2026/09/20 08:19:08 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo6741worker2 # [ 153.770446] nix-grpc-daemon[1830]: ts=2026-09-20T08:19:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56016 duration_s=0 paths=1 nar_bytes_in=6246742worker2 # [ 153.775686] nix-grpc-daemon[1830]: ts=2026-09-20T08:19:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:560166743worker2 # [ 153.787065] nix-daemon[847]: accepted connection from pid 1830, user nix-grpc-daemon (trusted)6744worker2 # [ 153.790425] nix-daemon[847]: reaped child process 1931, status = succeeded6745worker2 # [ 153.795868] nix-daemon[847]: accepted connection from pid 1830, user nix-grpc-daemon (trusted)6746client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...6747client # farm-b> worker2-next: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6748client # farm-b> LOG-farm-b-drain6749worker2 # [ 154.824951] nix-daemon[847]: reaped child process 1945, status = succeeded6750worker2 # [ 154.984836] nix-daemon[847]: accepted connection from pid 1960, user nix-grpc-daemon (trusted)6751worker2 # [ 154.995640] nix-daemon[847]: reaped child process 1963, status = succeeded6752lb # [ 154.993962] niks3-server[1039]: 2026/09/20 08:19:09 INFO Received uploads request method=POST path=/api/pending_closures6753worker2 # [ 155.021557] nix-grpc-daemon[1886]: time=2026-09-20T08:19:10.156Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6754worker2 # [ 155.023166] nix-grpc-daemon[1886]: time=2026-09-20T08:19:10.158Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"6755lb # [ 155.026119] niks3-server[1039]: 2026/09/20 08:19:09 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6756lb # [ 155.042388] niks3-server[1039]: 2026/09/20 08:19:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6757lb # [ 155.048800] niks3-server[1039]: 2026/09/20 08:19:09 INFO Signed narinfos id=65 count=16758worker2 # [ 155.071150] nix-grpc-daemon[1886]: time=2026-09-20T08:19:10.206Z level=INFO msg="Uploading 1 narinfos"6759lb # [ 155.053289] niks3-server[1039]: 2026/09/20 08:19:09 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst6760lb # [ 155.063667] niks3-server[1039]: 2026/09/20 08:19:10 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls6761lb # [ 155.072701] niks3-server[1039]: 2026/09/20 08:19:10 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6762lb # [ 155.076829] niks3-server[1039]: 2026/09/20 08:19:10 INFO Completed upload id=656763lb # [ 155.078346] niks3-server[1039]: 2026/09/20 08:19:10 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo6764worker2 # [ 155.097470] nix-grpc-daemon[1886]: time=2026-09-20T08:19:10.232Z level=INFO msg="Upload complete. (294ms)"6765worker2 # [ 155.103767] nix-grpc-daemon[1830]: ts=2026-09-20T08:19:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56016 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=11 outputs=16766worker1 # [ 155.088757] nix-grpc-daemon[1323]: ts=2026-09-20T08:19:09Z level=debug event=want client=3 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=06767worker1 # [ 155.093182] nix-grpc-daemon[1323]: ts=2026-09-20T08:19:09Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=16768worker2 # [ 155.117140] nix-daemon[847]: reaped child process 1943, status = succeeded6769worker2 # [ 155.152585] nix-grpc-daemon[1830]: ts=2026-09-20T08:19:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:560166770worker2 # [ 155.164508] nix-grpc-daemon[1830]: ts=2026-09-20T08:19:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:560166771worker2 # [ 155.177783] nix-grpc-daemon[1886]: time=2026-09-20T08:19:10.313Z level=INFO msg="All 2 paths already cached"6772client # copying 1 paths to 'lb:50051'...6773client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...6774worker2 # [ 155.185190] nix-grpc-daemon[1830]: ts=2026-09-20T08:19:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:560166775worker2 # [ 155.195121] nix-daemon[847]: accepted connection from pid 1830, user nix-grpc-daemon (trusted)6776worker2 # [ 155.371503] nix-daemon[847]: accepted connection from pid 1967, user nix-grpc-daemon (trusted)6777worker2 # [ 155.383606] nix-daemon[847]: reaped child process 1970, status = succeeded6778worker2 # [ 155.550209] nix-daemon[847]: accepted connection from pid 1972, user nix-grpc-daemon (trusted)6779worker2 # [ 155.562154] nix-daemon[847]: reaped child process 1975, status = succeeded6780lb # [ 155.559835] niks3-server[1039]: 2026/09/20 08:19:10 INFO Received uploads request method=POST path=/api/pending_closures6781worker2 # [ 155.588690] nix-grpc-daemon[1886]: time=2026-09-20T08:19:10.724Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6782worker2 # [ 155.590322] nix-grpc-daemon[1886]: time=2026-09-20T08:19:10.725Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"6783lb # [ 155.589228] niks3-server[1039]: 2026/09/20 08:19:10 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst6784lb # [ 155.600502] niks3-server[1039]: 2026/09/20 08:19:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6785worker2 # [ 155.624416] nix-grpc-daemon[1886]: time=2026-09-20T08:19:10.759Z level=INFO msg="Uploading 1 narinfos"6786lb # [ 155.606390] niks3-server[1039]: 2026/09/20 08:19:10 INFO Signed narinfos id=66 count=16787lb # [ 155.610647] niks3-server[1039]: 2026/09/20 08:19:10 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls6788lb # [ 155.622872] niks3-server[1039]: 2026/09/20 08:19:10 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6789lb # [ 155.629544] niks3-server[1039]: 2026/09/20 08:19:10 INFO Completed upload id=666790worker2 # [ 155.651288] nix-grpc-daemon[1886]: time=2026-09-20T08:19:10.786Z level=INFO msg="Upload complete. (444ms)"6791lb # [ 155.632445] niks3-server[1039]: 2026/09/20 08:19:10 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo6792worker2 # [ 155.655233] nix-grpc-daemon[1830]: ts=2026-09-20T08:19:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56016 duration_s=0 paths=1 nar_bytes_in=7526793worker2 # [ 155.660247] nix-grpc-daemon[1830]: ts=2026-09-20T08:19:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:560166794worker2 # [ 155.669205] nix-daemon[847]: accepted connection from pid 1830, user nix-grpc-daemon (trusted)6795worker2 # [ 155.678145] nix-daemon[847]: reaped child process 1965, status = succeeded6796client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...6797worker2 # [ 155.681225] nix-daemon[847]: accepted connection from pid 1830, user nix-grpc-daemon (trusted)6798client # farm-top> worker2-next: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6799client # farm-top> LOG-farm-top-drain6800worker2 # [ 156.494303] nix-daemon[847]: reaped child process 1979, status = succeeded6801worker2 # [ 156.641512] nix-daemon[847]: accepted connection from pid 1994, user nix-grpc-daemon (trusted)6802worker2 # [ 156.653946] nix-daemon[847]: reaped child process 1997, status = succeeded6803lb # [ 156.651477] niks3-server[1039]: 2026/09/20 08:19:11 INFO Received uploads request method=POST path=/api/pending_closures6804worker2 # [ 156.679544] nix-grpc-daemon[1886]: time=2026-09-20T08:19:11.814Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6805worker2 # [ 156.681331] nix-grpc-daemon[1886]: time=2026-09-20T08:19:11.816Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"6806lb # [ 156.686535] niks3-server[1039]: 2026/09/20 08:19:11 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6807lb # [ 156.693696] niks3-server[1039]: 2026/09/20 08:19:11 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst6808lb # [ 156.705922] niks3-server[1039]: 2026/09/20 08:19:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6809lb # [ 156.711071] niks3-server[1039]: 2026/09/20 08:19:11 INFO Signed narinfos id=67 count=16810worker2 # [ 156.733279] nix-grpc-daemon[1886]: time=2026-09-20T08:19:11.868Z level=INFO msg="Uploading 1 narinfos"6811lb # [ 156.716713] niks3-server[1039]: 2026/09/20 08:19:11 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls6812lb # [ 156.724970] niks3-server[1039]: 2026/09/20 08:19:11 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6813lb # [ 156.731245] niks3-server[1039]: 2026/09/20 08:19:11 INFO Completed upload id=676814worker2 # [ 156.750365] nix-grpc-daemon[1886]: time=2026-09-20T08:19:11.885Z level=INFO msg="Upload complete. (263ms)"6815lb # [ 156.734085] niks3-server[1039]: 2026/09/20 08:19:11 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo6816worker2 # [ 156.756328] nix-grpc-daemon[1830]: ts=2026-09-20T08:19:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56016 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=12 outputs=16817worker2 # [ 156.768470] nix-daemon[847]: reaped child process 1977, status = succeeded6818client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag drain >&2, in 6.15 seconds)6819client: must succeed: nix eval --raw -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag drain outPath6820client: (finished: must succeed: nix eval --raw -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag drain outPath, in 0.20 seconds)6821worker1: must succeed: rm /nix/.rw-store/fill6822worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.12 seconds)6823worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6824worker1 # [ 157.303556] nix-grpc-daemon[1323]: ts=2026-09-20T08:19:12Z level=info event=healthy reason=min_free6825lb # [ 157.453150] envoy[910]: conn peer=[2001:db8:1::1]:51922 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=26765 tx=10617 ms=166226826lb # [ 157.456863] envoy[910]: conn peer=[2001:db8:1::1]:60660 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=708 tx=438 ms=316827lb # [ 157.460282] envoy[910]: conn peer=[2001:db8:1::1]:60668 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9321 tx=3827 ms=59626828worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.10 seconds)6829(finished: subtest: low disk drains a worker and builds go to the other, in 9.72 seconds)6830(finished: run the VM test script, in 159.14 seconds)6831test script finished in 159.34s6832cleanup6833kill QemuMachine (pid 50)6834client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6835kill QemuMachine (pid 49)6836lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6837kill QemuMachine (pid 52)6838worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6839kill QemuMachine (pid 51)6840worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6841(finished: cleanup, in 1.01 seconds)6842additionally exposed symbols:6843 client, lb, worker1, worker2,6844 vlan1,6845 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh

succeeded vm-test-run-nix-grpc-farm checks.x86_64-linux.farm · build #239 · raw

succeeded `vm-test-run-nix-grpc-farm` `checks.x86_64-linux.farm` · build #239 · raw