vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #397
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12client # Disk image does not exist, creating the virtualisation disk image...13client # Formatting '/build/vm-state-client/tmp.QNXncuxWjW', fmt=raw size=107374182414lb # Disk image does not exist, creating the virtualisation disk image...15client: QEMU running (pid 49)16lb: QEMU running (pid 50)17lb # Formatting '/build/vm-state-lb/tmp.iEQ8xPAGMI', fmt=raw size=107374182418worker1 # Disk image does not exist, creating the virtualisation disk image...19client # mke2fs 1.47.4 (6-Mar-2025)20worker1: QEMU running (pid 51)21worker1 # Formatting '/build/vm-state-worker1/tmp.5Ql9yKYnsW', fmt=raw size=107374182422lb # mke2fs 1.47.4 (6-Mar-2025)23lb # Discarding device blocks: 0/262144 done24worker2: QEMU running (pid 52)25worker1 # mke2fs 1.47.4 (6-Mar-2025)26client # Discarding device blocks: 0/262144 done27lb # Creating filesystem with 262144 4k blocks and 65536 inodes28worker2 # Disk image does not exist, creating the virtualisation disk image...29worker1 # Discarding device blocks: 0/262144 done30client # Creating filesystem with 262144 4k blocks and 65536 inodes31worker2 # Formatting '/build/vm-state-worker2/tmp.dzVhpk68b0', fmt=raw size=107374182432worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes33client # Filesystem UUID: e4ad23d5-30db-4da7-9ecd-a2d0f291b34834worker2 # mke2fs 1.47.4 (6-Mar-2025)35worker1 # Filesystem UUID: d479f543-70dc-4bba-ad91-3e02f7b5111136client # Superblock backups stored on blocks:37worker2 # Discarding device blocks: 0/262144 done38worker1 # Superblock backups stored on blocks:39client # 32768, 98304, 163840, 22937640worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes41worker1 # 32768, 98304, 163840, 22937642worker1 # 43client # 44(finished: start all VMs, in 0.24 seconds)45worker2 # Filesystem UUID: dd1dc788-4674-4330-88c5-d2b3bc545c7446client: waiting for unit network-addresses-eth1.service47client # Allocating group tables: 0/8 done48worker2 # Superblock backups stored on blocks:49client: waiting for the VM to finish booting50client # Writing inode tables: 0/8 done51worker2 # 32768, 98304, 163840, 22937652worker1 # Allocating group tables: 0/8 done53lb # Filesystem UUID: 995c1016-4210-4a2b-a959-6ad98a85b55454client # Creating journal (8192 blocks): done55worker2 # 56worker1 # Writing inode tables: 0/8 done57lb # Superblock backups stored on blocks:58client # Writing superblocks and filesystem accounting information: 0/8 done59worker2 # Allocating group tables: 0/8 done60worker1 # Creating journal (8192 blocks): done61client # 62lb # 32768, 98304, 163840, 22937663worker2 # Writing inode tables: 0/8 done64worker1 # Writing superblocks and filesystem accounting information: 0/8 done65client # Virtualisation disk image created.66lb # 67worker2 # Creating journal (8192 blocks): done68worker2 # Writing superblocks and filesystem accounting information: 0/8 done69worker2 # 70worker1 # 71lb # Allocating group tables: 0/8 done72worker1 # Virtualisation disk image created.73lb # Writing inode tables: 0/8 done74lb # Creating journal (8192 blocks): done75worker2 # Virtualisation disk image created.76lb # Writing superblocks and filesystem accounting information: 0/8 done77lb # 78lb # Virtualisation disk image created.79worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)80client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)81worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)82lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)83worker2 # 84worker2 # 85worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0086worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...87worker2 # 88worker2 # 89worker2 # 90worker2 # 91worker2 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB0092client # 93client # 94worker2 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...95worker2 # 96worker2 # 97client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0098worker1 # 99worker1 # 100lb # 101lb # 102worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00103lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00104client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...105client # 106client # 107client # 108client # 109client # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00110worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...111worker1 # 112worker1 # 113lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...114worker1 # 115lb # 116worker1 # 117lb # 118lb # 119lb # 120worker1 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00121lb # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00122client # Press Ctrl-B to configure iPXE (PCI 00:08.0)...123client # 124client # 125worker1 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...126worker1 # 127worker1 # 128lb # Press Ctrl-B to configure iPXE (PCI 00:08.0)...129lb # 130lb # 131worker2 # Booting from ROM...132client # Booting from ROM...133worker1 # Booting from ROM...134lb # Booting from ROM...135worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026136worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/lmg54zm73d3fkgh5l2hyy342zxg079h1-nixos-system-worker2-test/init regInfo=/nix/store/5k92nyq4mwpmcxd2y9ya0qyaxrvqcb9c-closure-info/registration console=ttyS0,115200n8 console=tty0137worker2 # [ 0.000000] BIOS-provided physical RAM map:138worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable139worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved140worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved141worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable142worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved143worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved144worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved145worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved146worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved147worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved148worker2 # [ 0.000000] NX (Execute Disable) protection: active149worker2 # [ 0.000000] APIC: Static calls initialized150worker2 # [ 0.000000] SMBIOS 2.8 present.151worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014152worker2 # [ 0.000000] DMI: Memory slots populated: 1/1153worker2 # [ 0.000000] Hypervisor detected: KVM154worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000155worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00156client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026157worker2 # [ 0.000000] kvm-clock: using sched offset of 466034630 cycles158worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns159worker2 # [ 0.000004] tsc: Detected 2400.012 MHz processor160worker2 # [ 0.000814] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000161client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/95v1d82m9fafgfkbb6biz3wcfj11j9p6-nixos-system-client-test/init regInfo=/nix/store/gqjbl5xrdwlfrvpm6fwl8laj1m1n38ap-closure-info/registration console=ttyS0,115200n8 console=tty0162worker2 # [ 0.000841] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs163client # [ 0.000000] BIOS-provided physical RAM map:164worker2 # [ 0.000843] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT165client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable166worker2 # [ 0.002736] found SMP MP-table at [mem 0x000f5450-0x000f545f]167client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved168worker2 # [ 0.002746] Using GB pages for direct mapping169client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved170worker2 # [ 0.002859] RAMDISK: [mem 0x5e367000-0x5ffcffff]171worker2 # [ 0.002868] ACPI: Early table checksum verification disabled172client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable173worker2 # [ 0.002871] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )174client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved175worker2 # [ 0.002875] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)176client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved177client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved178worker2 # [ 0.002879] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)179client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved180worker2 # [ 0.002885] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)181client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved182worker2 # [ 0.002887] ACPI: FACS 0x000000005FFE0000 000040183client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved184worker2 # [ 0.002889] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)185client # [ 0.000000] NX (Execute Disable) protection: active186client # [ 0.000000] APIC: Static calls initialized187worker2 # [ 0.002890] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)188client # [ 0.000000] SMBIOS 2.8 present.189worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026190worker2 # [ 0.002892] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)191client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014192client # [ 0.000000] DMI: Memory slots populated: 1/1193worker2 # [ 0.002894] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)194lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026195client # [ 0.000000] Hypervisor detected: KVM196worker2 # [ 0.002895] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]197client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000198client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00199worker2 # [ 0.002896] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]200client # [ 0.000000] kvm-clock: using sched offset of 468655147 cycles201worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/yslql6ccjgpwc34v09asjzmml5yiafmj-nixos-system-worker1-test/init regInfo=/nix/store/n9axs0iwhn6nwffv8f5c7najnc77ppfp-closure-info/registration console=ttyS0,115200n8 console=tty0202worker2 # [ 0.002897] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]203worker1 # [ 0.000000] BIOS-provided physical RAM map:204worker2 # [ 0.002897] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]205client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns206worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable207lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0qln5mjy767qjp5wbmxjg8jay8k05dsm-nixos-system-lb-test/init regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration console=ttyS0,115200n8 console=tty0208client # [ 0.000005] tsc: Detected 2400.012 MHz processor209worker2 # [ 0.002897] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]210worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved211lb # [ 0.000000] BIOS-provided physical RAM map:212client # [ 0.000813] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000213worker2 # [ 0.002898] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]214worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved215lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable216client # [ 0.000839] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs217worker2 # [ 0.002898] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]218worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable219lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved220worker2 # [ 0.003125] No NUMA configuration found221client # [ 0.000842] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT222worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved223lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved224worker2 # [ 0.003126] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]225client # [ 0.002765] found SMP MP-table at [mem 0x000f5450-0x000f545f]226worker2 # [ 0.003128] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]227lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable228client # [ 0.002776] Using GB pages for direct mapping229worker2 # [ 0.005225] Zone ranges:230worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved231client # [ 0.002903] RAMDISK: [mem 0x5e366000-0x5ffcffff]232worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved233lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved234worker2 # [ 0.005226] DMA [mem 0x0000000000001000-0x0000000000ffffff]235client # [ 0.002913] ACPI: Early table checksum verification disabled236lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved237worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved238worker2 # [ 0.005227] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]239client # [ 0.002915] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )240worker2 # [ 0.005228] Normal empty241worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved242worker2 # [ 0.005229] Device empty243lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved244worker2 # [ 0.005229] Movable zone start for each node245client # [ 0.002919] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)246lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved247worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved248worker2 # [ 0.005230] Early memory node ranges249worker1 # [ 0.000000] NX (Execute Disable) protection: active250client # [ 0.002923] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)251lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved252worker2 # [ 0.005230] node 0: [mem 0x0000000000001000-0x000000000009efff]253worker1 # [ 0.000000] APIC: Static calls initialized254worker1 # [ 0.000000] SMBIOS 2.8 present.255worker2 # [ 0.005231] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]256lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved257client # [ 0.002930] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)258lb # [ 0.000000] NX (Execute Disable) protection: active259client # [ 0.002932] ACPI: FACS 0x000000005FFE0000 000040260worker2 # [ 0.005232] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]261worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014262lb # [ 0.000000] APIC: Static calls initialized263lb # [ 0.000000] SMBIOS 2.8 present.264worker2 # [ 0.005251] On node 0, zone DMA: 1 pages in unavailable ranges265worker1 # [ 0.000000] DMI: Memory slots populated: 1/1266client # [ 0.002933] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)267worker1 # [ 0.000000] Hypervisor detected: KVM268worker2 # [ 0.005501] On node 0, zone DMA: 97 pages in unavailable ranges269lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014270worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000271client # [ 0.002935] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)272worker2 # [ 0.030312] On node 0, zone DMA32: 40 pages in unavailable ranges273lb # [ 0.000000] DMI: Memory slots populated: 1/1274worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00275worker2 # [ 0.030768] ACPI: PM-Timer IO Port: 0x608276lb # [ 0.000000] Hypervisor detected: KVM277client # [ 0.002936] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)278worker1 # [ 0.000000] kvm-clock: using sched offset of 497592473 cycles279worker2 # [ 0.030779] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])280lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000281client # [ 0.002938] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)282lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00283worker2 # [ 0.030805] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23284worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns285lb # [ 0.000001] kvm-clock: using sched offset of 501800401 cycles286client # [ 0.002939] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]287worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor288worker2 # [ 0.030808] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)289worker1 # [ 0.000810] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000290client # [ 0.002940] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]291worker2 # [ 0.030810] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)292lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns293worker2 # [ 0.030811] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)294client # [ 0.002941] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]295lb # [ 0.000005] tsc: Detected 2400.012 MHz processor296worker1 # [ 0.000837] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs297lb # [ 0.000804] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000298worker2 # [ 0.030812] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)299worker1 # [ 0.000840] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT300client # [ 0.002941] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]301worker1 # [ 0.002742] found SMP MP-table at [mem 0x000f5450-0x000f545f]302worker2 # [ 0.030812] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)303client # [ 0.002942] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]304lb # [ 0.000832] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs305worker1 # [ 0.002753] Using GB pages for direct mapping306worker2 # [ 0.030815] ACPI: Using ACPI (MADT) for SMP configuration information307client # [ 0.002942] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]308lb # [ 0.000834] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT309worker1 # [ 0.002854] RAMDISK: [mem 0x5e367000-0x5ffcffff]310worker2 # [ 0.030816] ACPI: HPET id: 0x8086a201 base: 0xfed00000311lb # [ 0.002729] found SMP MP-table at [mem 0x000f5450-0x000f545f]312worker1 # [ 0.002863] ACPI: Early table checksum verification disabled313client # [ 0.002943] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]314worker2 # [ 0.030819] TSC deadline timer available315lb # [ 0.002741] Using GB pages for direct mapping316worker1 # [ 0.002866] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )317worker2 # [ 0.030823] CPU topo: Max. logical packages: 1318client # [ 0.003171] No NUMA configuration found319lb # [ 0.002899] RAMDISK: [mem 0x5e367000-0x5ffcffff]320client # [ 0.003172] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]321worker2 # [ 0.030823] CPU topo: Max. logical dies: 1322worker1 # [ 0.002870] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)323lb # [ 0.002909] ACPI: Early table checksum verification disabled324worker2 # [ 0.030824] CPU topo: Max. dies per package: 1325client # [ 0.003175] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]326client # [ 0.005307] Zone ranges:327lb # [ 0.002912] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )328worker2 # [ 0.030827] CPU topo: Max. threads per core: 1329worker1 # [ 0.002874] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)330worker2 # [ 0.030828] CPU topo: Num. cores per package: 1331client # [ 0.005308] DMA [mem 0x0000000000001000-0x0000000000ffffff]332lb # [ 0.002916] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)333worker2 # [ 0.030828] CPU topo: Num. threads per package: 1334worker1 # [ 0.002883] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)335client # [ 0.005310] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]336client # [ 0.005311] Normal empty337worker1 # [ 0.002885] ACPI: FACS 0x000000005FFE0000 000040338worker2 # [ 0.030829] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs339lb # [ 0.002921] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)340client # [ 0.005312] Device empty341client # [ 0.005313] Movable zone start for each node342worker2 # [ 0.030846] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()343worker1 # [ 0.002886] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)344lb # [ 0.002928] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)345client # [ 0.005313] Early memory node ranges346lb # [ 0.002930] ACPI: FACS 0x000000005FFE0000 000040347worker2 # [ 0.030875] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]348client # [ 0.005314] node 0: [mem 0x0000000000001000-0x000000000009efff]349worker1 # [ 0.002888] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)350client # [ 0.005315] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]351lb # [ 0.002932] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)352worker2 # [ 0.030877] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]353worker1 # [ 0.002889] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)354worker2 # [ 0.030878] [mem 0x60000000-0xafffffff] available for PCI devices355client # [ 0.005316] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]356lb # [ 0.002933] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)357worker1 # [ 0.002891] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)358worker2 # [ 0.030879] Booting paravirtualized kernel on KVM359client # [ 0.005335] On node 0, zone DMA: 1 pages in unavailable ranges360lb # [ 0.002935] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)361worker1 # [ 0.002892] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]362client # [ 0.005588] On node 0, zone DMA: 97 pages in unavailable ranges363worker2 # [ 0.030882] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns364client # [ 0.031165] On node 0, zone DMA32: 40 pages in unavailable ranges365lb # [ 0.002936] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)366worker1 # [ 0.002893] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]367client # [ 0.031635] ACPI: PM-Timer IO Port: 0x608368worker2 # [ 0.035329] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1369worker1 # [ 0.002894] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]370lb # [ 0.002938] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]371client # [ 0.031645] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])372worker2 # [ 0.037392] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152373lb # [ 0.002939] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]374worker1 # [ 0.002894] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]375client # [ 0.031671] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23376worker2 # [ 0.037435] kvm-guest: PV spinlocks disabled, single CPU377lb # [ 0.002939] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]378worker1 # [ 0.002894] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]379client # [ 0.031673] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)380worker1 # [ 0.002895] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]381lb # [ 0.002940] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]382client # [ 0.031675] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)383worker1 # [ 0.002895] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]384lb # [ 0.002940] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]385client # [ 0.031676] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)386worker1 # [ 0.003128] No NUMA configuration found387lb # [ 0.002941] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]388client # [ 0.031677] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)389worker1 # [ 0.003129] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]390worker2 # [ 0.037436] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/lmg54zm73d3fkgh5l2hyy342zxg079h1-nixos-system-worker2-test/init regInfo=/nix/store/5k92nyq4mwpmcxd2y9ya0qyaxrvqcb9c-closure-info/registration console=ttyS0,115200n8 console=tty0391lb # [ 0.002941] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]392client # [ 0.031678] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)393worker1 # [ 0.003132] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]394lb # [ 0.003175] No NUMA configuration found395worker1 # [ 0.005570] Zone ranges:396client # [ 0.031680] ACPI: Using ACPI (MADT) for SMP configuration information397lb # [ 0.003176] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]398worker1 # [ 0.005571] DMA [mem 0x0000000000001000-0x0000000000ffffff]399client # [ 0.031681] ACPI: HPET id: 0x8086a201 base: 0xfed00000400worker2 # [ 0.037525] Unknown kernel command line parameters "regInfo=/nix/store/5k92nyq4mwpmcxd2y9ya0qyaxrvqcb9c-closure-info/registration", will be passed to user space.401client # [ 0.031685] TSC deadline timer available402lb # [ 0.003178] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]403worker2 # [ 0.037538] random: crng init done404worker1 # [ 0.005573] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]405lb # [ 0.005493] Zone ranges:406client # [ 0.031689] CPU topo: Max. logical packages: 1407worker1 # [ 0.005574] Normal empty408worker1 # [ 0.005575] Device empty409worker2 # [ 0.037539] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes410client # [ 0.031690] CPU topo: Max. logical dies: 1411lb # [ 0.005494] DMA [mem 0x0000000000001000-0x0000000000ffffff]412worker1 # [ 0.005575] Movable zone start for each node413client # [ 0.031690] CPU topo: Max. dies per package: 1414worker1 # [ 0.005576] Early memory node ranges415lb # [ 0.005495] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]416worker2 # [ 0.039547] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)417client # [ 0.031693] CPU topo: Max. threads per core: 1418lb # [ 0.005497] Normal empty419lb # [ 0.005498] Device empty420worker1 # [ 0.005576] node 0: [mem 0x0000000000001000-0x000000000009efff]421client # [ 0.031694] CPU topo: Num. cores per package: 1422worker2 # [ 0.040676] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)423lb # [ 0.005498] Movable zone start for each node424worker1 # [ 0.005577] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]425worker2 # [ 0.040707] Fallback order for Node 0: 0426client # [ 0.031694] CPU topo: Num. threads per package: 1427lb # [ 0.005499] Early memory node ranges428worker1 # [ 0.005578] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]429worker2 # [ 0.040710] Built 1 zonelists, mobility grouping on. Total pages: 393078430client # [ 0.031695] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs431lb # [ 0.005499] node 0: [mem 0x0000000000001000-0x000000000009efff]432worker2 # [ 0.040711] Policy zone: DMA32433worker1 # [ 0.005600] On node 0, zone DMA: 1 pages in unavailable ranges434lb # [ 0.005500] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]435client # [ 0.031712] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()436worker2 # [ 0.043087] mem auto-init: stack:all(zero), heap alloc:on, heap free:off437worker1 # [ 0.005881] On node 0, zone DMA: 97 pages in unavailable ranges438lb # [ 0.005501] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]439client # [ 0.031741] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]440worker2 # [ 0.045965] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1441worker1 # [ 0.032757] On node 0, zone DMA32: 40 pages in unavailable ranges442lb # [ 0.005522] On node 0, zone DMA: 1 pages in unavailable ranges443worker2 # [ 0.049074] allocated 3145728 bytes of page_ext444client # [ 0.031743] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]445worker1 # [ 0.033206] ACPI: PM-Timer IO Port: 0x608446lb # [ 0.005791] On node 0, zone DMA: 97 pages in unavailable ranges447worker2 # [ 0.058449] ftrace: allocating 48731 entries in 192 pages448worker1 # [ 0.033216] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])449client # [ 0.031744] [mem 0x60000000-0xafffffff] available for PCI devices450lb # [ 0.032637] On node 0, zone DMA32: 40 pages in unavailable ranges451worker2 # [ 0.058451] ftrace: allocated 192 pages with 2 groups452client # [ 0.031746] Booting paravirtualized kernel on KVM453worker1 # [ 0.033244] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23454worker2 # [ 0.059242] Dynamic Preempt: lazy455lb # [ 0.033097] ACPI: PM-Timer IO Port: 0x608456worker2 # [ 0.059410] rcu: Preemptible hierarchical RCU implementation.457worker1 # [ 0.033247] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)458lb # [ 0.033108] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])459client # [ 0.031748] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns460worker2 # [ 0.059411] rcu: RCU event tracing is enabled.461worker1 # [ 0.033249] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)462lb # [ 0.033134] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23463client # [ 0.036191] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1464worker2 # [ 0.059411] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.465worker1 # [ 0.033250] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)466lb # [ 0.033137] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)467worker2 # [ 0.059412] Trampoline variant of Tasks RCU enabled.468client # [ 0.038277] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152469lb # [ 0.033138] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)470worker1 # [ 0.033251] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)471worker2 # [ 0.059413] Rude variant of Tasks RCU enabled.472client # [ 0.038327] kvm-guest: PV spinlocks disabled, single CPU473worker1 # [ 0.033252] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)474worker2 # [ 0.059413] Tracing variant of Tasks RCU enabled.475lb # [ 0.033139] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)476worker1 # [ 0.033254] ACPI: Using ACPI (MADT) for SMP configuration information477lb # [ 0.033140] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)478worker2 # [ 0.059414] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.479worker1 # [ 0.033255] ACPI: HPET id: 0x8086a201 base: 0xfed00000480lb # [ 0.033141] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)481worker2 # [ 0.059414] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1482worker1 # [ 0.033258] TSC deadline timer available483worker1 # [ 0.033262] CPU topo: Max. logical packages: 1484lb # [ 0.033143] ACPI: Using ACPI (MADT) for SMP configuration information485worker2 # [ 0.059430] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.486worker1 # [ 0.033263] CPU topo: Max. logical dies: 1487lb # [ 0.033144] ACPI: HPET id: 0x8086a201 base: 0xfed00000488client # [ 0.038329] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/95v1d82m9fafgfkbb6biz3wcfj11j9p6-nixos-system-client-test/init regInfo=/nix/store/gqjbl5xrdwlfrvpm6fwl8laj1m1n38ap-closure-info/registration console=ttyS0,115200n8 console=tty0489lb # [ 0.033148] TSC deadline timer available490worker1 # [ 0.033264] CPU topo: Max. dies per package: 1491worker2 # [ 0.059431] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.492lb # [ 0.033152] CPU topo: Max. logical packages: 1493worker1 # [ 0.033267] CPU topo: Max. threads per core: 1494lb # [ 0.033153] CPU topo: Max. logical dies: 1495worker1 # [ 0.033267] CPU topo: Num. cores per package: 1496worker2 # [ 0.059432] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.497lb # [ 0.033153] CPU topo: Max. dies per package: 1498client # [ 0.038429] Unknown kernel command line parameters "regInfo=/nix/store/gqjbl5xrdwlfrvpm6fwl8laj1m1n38ap-closure-info/registration", will be passed to user space.499worker1 # [ 0.033268] CPU topo: Num. threads per package: 1500client # [ 0.038442] random: crng init done501worker2 # [ 0.063876] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16502lb # [ 0.033156] CPU topo: Max. threads per core: 1503worker1 # [ 0.033268] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs504lb # [ 0.033157] CPU topo: Num. cores per package: 1505worker2 # [ 0.064136] rcu: srcu_init: Setting srcu_struct sizes based on contention.506client # [ 0.038442] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes507worker1 # [ 0.033285] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()508lb # [ 0.033158] CPU topo: Num. threads per package: 1509client # [ 0.040446] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)510lb # [ 0.033158] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs511worker2 # [ 0.064142] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns512worker1 # [ 0.033315] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]513client # [ 0.041692] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)514lb # [ 0.033177] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()515worker1 # [ 0.033317] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]516worker2 # [ 0.064240] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)517client # [ 0.041724] Fallback order for Node 0: 0518worker2 # [ 0.067779] Console: colour VGA+ 80x25519lb # [ 0.033208] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]520worker1 # [ 0.033318] [mem 0x60000000-0xafffffff] available for PCI devices521client # [ 0.041727] Built 1 zonelists, mobility grouping on. Total pages: 393078522worker2 # [ 0.067782] printk: legacy console [tty0] enabled523worker1 # [ 0.033319] Booting paravirtualized kernel on KVM524client # [ 0.041728] Policy zone: DMA32525lb # [ 0.033209] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]526worker2 # [ 0.097220] printk: legacy console [ttyS0] enabled527client # [ 0.044098] mem auto-init: stack:all(zero), heap alloc:on, heap free:off528lb # [ 0.033210] [mem 0x60000000-0xafffffff] available for PCI devices529worker1 # [ 0.033321] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns530worker2 # [ 0.196704] ACPI: Core revision 20250807531lb # [ 0.033212] Booting paravirtualized kernel on KVM532client # [ 0.047066] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1533worker1 # [ 0.037762] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1534client # [ 0.050211] allocated 3145728 bytes of page_ext535lb # [ 0.033216] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns536worker1 # [ 0.039922] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152537client # [ 0.059726] ftrace: allocating 48731 entries in 192 pages538worker2 # [ 0.197596] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns539worker1 # [ 0.039965] kvm-guest: PV spinlocks disabled, single CPU540client # [ 0.059729] ftrace: allocated 192 pages with 2 groups541lb # [ 0.037659] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1542worker2 # [ 0.199105] APIC: Switch to symmetric I/O mode setup543client # [ 0.060556] Dynamic Preempt: lazy544worker2 # [ 0.200042] x2apic enabled545lb # [ 0.040051] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152546client # [ 0.060728] rcu: Preemptible hierarchical RCU implementation.547client # [ 0.060728] rcu: RCU event tracing is enabled.548lb # [ 0.040095] kvm-guest: PV spinlocks disabled, single CPU549worker2 # [ 0.200783] APIC: Switched APIC routing to: physical x2apic550client # [ 0.060729] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.551client # [ 0.060731] Trampoline variant of Tasks RCU enabled.552client # [ 0.060731] Rude variant of Tasks RCU enabled.553worker1 # [ 0.039966] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/yslql6ccjgpwc34v09asjzmml5yiafmj-nixos-system-worker1-test/init regInfo=/nix/store/n9axs0iwhn6nwffv8f5c7najnc77ppfp-closure-info/registration console=ttyS0,115200n8 console=tty0554worker2 # [ 0.202609] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1555client # [ 0.060731] Tracing variant of Tasks RCU enabled.556client # [ 0.060732] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.557lb # [ 0.040096] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0qln5mjy767qjp5wbmxjg8jay8k05dsm-nixos-system-lb-test/init regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration console=ttyS0,115200n8 console=tty0558worker1 # [ 0.040057] Unknown kernel command line parameters "regInfo=/nix/store/n9axs0iwhn6nwffv8f5c7najnc77ppfp-closure-info/registration", will be passed to user space.559worker2 # [ 0.203579] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns560worker1 # [ 0.040069] random: crng init done561client # [ 0.060733] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1562worker1 # [ 0.040070] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes563worker2 # [ 0.205210] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)564lb # [ 0.040192] Unknown kernel command line parameters "regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration", will be passed to user space.565client # [ 0.060753] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.566lb # [ 0.040205] random: crng init done567worker1 # [ 0.042200] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)568worker2 # [ 0.206293] x86/cpu: User Mode Instruction Prevention (UMIP) activated569client # [ 0.060754] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.570lb # [ 0.040206] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes571worker1 # [ 0.043454] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)572worker1 # [ 0.043490] Fallback order for Node 0: 0573worker2 # [ 0.207338] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127574client # [ 0.060755] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.575lb # [ 0.042387] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)576worker1 # [ 0.043493] Built 1 zonelists, mobility grouping on. Total pages: 393078577client # [ 0.065172] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16578lb # [ 0.043453] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)579worker2 # [ 0.208208] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0580worker1 # [ 0.043494] Policy zone: DMA32581lb # [ 0.043487] Fallback order for Node 0: 0582client # [ 0.065438] rcu: srcu_init: Setting srcu_struct sizes based on contention.583worker1 # [ 0.046043] mem auto-init: stack:all(zero), heap alloc:on, heap free:off584lb # [ 0.043491] Built 1 zonelists, mobility grouping on. Total pages: 393078585lb # [ 0.043492] Policy zone: DMA32586worker1 # [ 0.049250] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1587worker2 # [ 0.209211] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto588client # [ 0.065444] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns589worker1 # [ 0.052577] allocated 3145728 bytes of page_ext590lb # [ 0.046024] mem auto-init: stack:all(zero), heap alloc:on, heap free:off591worker1 # [ 0.062367] ftrace: allocating 48731 entries in 192 pages592client # [ 0.065547] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)593worker2 # [ 0.210208] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl594lb # [ 0.049310] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1595worker1 # [ 0.062369] ftrace: allocated 192 pages with 2 groups596client # [ 0.069041] Console: colour VGA+ 80x25597lb # [ 0.052856] allocated 3145728 bytes of page_ext598worker1 # [ 0.063211] Dynamic Preempt: lazy599client # [ 0.069044] printk: legacy console [tty0] enabled600worker2 # [ 0.211208] Transient Scheduler Attacks: Vulnerable: No microcode601lb # [ 0.062767] ftrace: allocating 48731 entries in 192 pages602worker1 # [ 0.063342] rcu: Preemptible hierarchical RCU implementation.603client # [ 0.098959] printk: legacy console [ttyS0] enabled604lb # [ 0.062769] ftrace: allocated 192 pages with 2 groups605worker2 # [ 0.212207] Spectre V2 : Mitigation: Enhanced / Automatic IBRS606worker1 # [ 0.063343] rcu: RCU event tracing is enabled.607lb # [ 0.063655] Dynamic Preempt: lazy608client # [ 0.197360] ACPI: Core revision 20250807609worker1 # [ 0.063343] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.610lb # [ 0.063836] rcu: Preemptible hierarchical RCU implementation.611worker2 # [ 0.213207] Speculative Return Stack Overflow: Mitigation: Safe RET612worker1 # [ 0.063344] Trampoline variant of Tasks RCU enabled.613lb # [ 0.063837] rcu: RCU event tracing is enabled.614client # [ 0.198243] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns615worker1 # [ 0.063345] Rude variant of Tasks RCU enabled.616lb # [ 0.063838] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.617worker1 # [ 0.063345] Tracing variant of Tasks RCU enabled.618client # [ 0.199816] APIC: Switch to symmetric I/O mode setup619lb # [ 0.063840] Trampoline variant of Tasks RCU enabled.620lb # [ 0.063840] Rude variant of Tasks RCU enabled.621worker1 # [ 0.063346] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.622client # [ 0.200786] x2apic enabled623lb # [ 0.063840] Tracing variant of Tasks RCU enabled.624worker1 # [ 0.063347] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1625client # [ 0.201584] APIC: Switched APIC routing to: physical x2apic626worker2 # [ 0.214208] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization627lb # [ 0.063841] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.628worker1 # [ 0.063402] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.629worker2 # [ 0.216214] Spectre V2 : Enabling IBPB for BPF630lb # [ 0.063842] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1631worker1 # [ 0.063404] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.632client # [ 0.203456] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1633lb # [ 0.063865] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.634worker2 # [ 0.217208] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier635worker1 # [ 0.063405] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.636lb # [ 0.063867] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.637worker2 # [ 0.218208] active return thunk: srso_alias_return_thunk638worker1 # [ 0.068056] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16639client # [ 0.204462] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns640lb # [ 0.063868] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.641worker1 # [ 0.068351] rcu: srcu_init: Setting srcu_struct sizes based on contention.642worker2 # [ 0.219229] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'643lb # [ 0.068242] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16644client # [ 0.206139] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)645worker2 # [ 0.220207] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'646worker1 # [ 0.068358] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns647lb # [ 0.068523] rcu: srcu_init: Setting srcu_struct sizes based on contention.648client # [ 0.207222] x86/cpu: User Mode Instruction Prevention (UMIP) activated649worker2 # [ 0.221208] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'650worker1 # [ 0.068463] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)651lb # [ 0.068530] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns652worker1 # [ 0.072034] Console: colour VGA+ 80x25653client # [ 0.209051] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127654worker2 # [ 0.222207] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'655worker1 # [ 0.072037] printk: legacy console [tty0] enabled656lb # [ 0.068636] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)657client # [ 0.209817] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0658worker1 # [ 0.101626] printk: legacy console [ttyS0] enabled659worker2 # [ 0.223207] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'660lb # [ 0.072173] Console: colour VGA+ 80x25661worker1 # [ 0.198066] ACPI: Core revision 20250807662lb # [ 0.072176] printk: legacy console [tty0] enabled663worker2 # [ 0.224207] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'664lb # [ 0.101523] printk: legacy console [ttyS0] enabled665client # [ 0.210910] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto666lb # [ 0.197789] ACPI: Core revision 20250807667worker2 # [ 0.225207] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'668worker1 # [ 0.198927] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns669client # [ 0.212137] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl670worker1 # [ 0.200494] APIC: Switch to symmetric I/O mode setup671worker2 # [ 0.226208] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256672lb # [ 0.198661] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns673worker1 # [ 0.201433] x2apic enabled674client # [ 0.213137] Transient Scheduler Attacks: Vulnerable: No microcode675worker2 # [ 0.227923] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64676lb # [ 0.200173] APIC: Switch to symmetric I/O mode setup677worker1 # [ 0.202188] APIC: Switched APIC routing to: physical x2apic678client # [ 0.214137] Spectre V2 : Mitigation: Enhanced / Automatic IBRS679worker2 # [ 0.228207] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512680lb # [ 0.201135] x2apic enabled681worker2 # [ 0.229207] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024682client # [ 0.215136] Speculative Return Stack Overflow: Mitigation: Safe RET683lb # [ 0.201913] APIC: Switched APIC routing to: physical x2apic684worker1 # [ 0.203969] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1685worker2 # [ 0.230207] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8686lb # [ 0.203735] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1687worker1 # [ 0.204977] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns688worker2 # [ 0.231207] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.689worker1 # [ 0.206647] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)690lb # [ 0.204718] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns691client # [ 0.216136] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization692client # [ 0.218144] Spectre V2 : Enabling IBPB for BPF693worker1 # [ 0.207732] x86/cpu: User Mode Instruction Prevention (UMIP) activated694lb # [ 0.206404] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)695worker1 # [ 0.208783] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127696client # [ 0.218883] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier697lb # [ 0.207490] x86/cpu: User Mode Instruction Prevention (UMIP) activated698worker1 # [ 0.210344] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0699client # [ 0.220136] active return thunk: srso_alias_return_thunk700lb # [ 0.208582] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127701client # [ 0.221158] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'702lb # [ 0.210120] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0703worker1 # [ 0.210649] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto704client # [ 0.222136] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'705worker1 # [ 0.212645] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl706lb # [ 0.211173] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto707client # [ 0.223137] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'708worker1 # [ 0.213645] Transient Scheduler Attacks: Vulnerable: No microcode709client # [ 0.224136] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'710lb # [ 0.212401] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl711worker1 # [ 0.214645] Spectre V2 : Mitigation: Enhanced / Automatic IBRS712client # [ 0.225136] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'713lb # [ 0.213402] Transient Scheduler Attacks: Vulnerable: No microcode714worker1 # [ 0.215645] Speculative Return Stack Overflow: Mitigation: Safe RET715lb # [ 0.215153] Spectre V2 : Mitigation: Enhanced / Automatic IBRS716client # [ 0.226136] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'717lb # [ 0.216123] Speculative Return Stack Overflow: Mitigation: Safe RET718client # [ 0.227137] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'719client # [ 0.229137] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256720worker1 # [ 0.216645] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization721client # [ 0.230136] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64722worker1 # [ 0.218654] Spectre V2 : Enabling IBPB for BPF723client # [ 0.231140] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512724lb # [ 0.217180] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization725worker1 # [ 0.219646] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier726client # [ 0.232132] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024727lb # [ 0.218409] Spectre V2 : Enabling IBPB for BPF728worker1 # [ 0.220645] active return thunk: srso_alias_return_thunk729client # [ 0.233137] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8730lb # [ 0.219938] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier731worker1 # [ 0.221666] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'732client # [ 0.234136] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.733lb # [ 0.220402] active return thunk: srso_alias_return_thunk734worker1 # [ 0.222644] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'735lb # [ 0.221263] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'736worker1 # [ 0.224387] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'737lb # [ 0.222400] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'738worker1 # [ 0.225411] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'739lb # [ 0.223401] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'740worker1 # [ 0.226446] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'741lb # [ 0.224401] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'742worker1 # [ 0.227420] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'743lb # [ 0.225400] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'744worker1 # [ 0.228450] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'745lb # [ 0.226401] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'746worker1 # [ 0.229645] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256747worker1 # [ 0.230644] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64748lb # [ 0.227401] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'749worker1 # [ 0.231645] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512750lb # [ 0.228401] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256751worker1 # [ 0.232644] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024752lb # [ 0.230147] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64753worker1 # [ 0.233644] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8754lb # [ 0.231146] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512755lb # [ 0.232148] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024756worker1 # [ 0.234644] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.757lb # [ 0.233137] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8758worker2 # [ 0.265960] Freeing SMP alternatives memory: 44K759lb # [ 0.234172] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.760worker2 # [ 0.266210] pid_max: default: 32768 minimum: 301761worker2 # [ 0.267299] LSM: initializing lsm=capability,landlock,yama,bpf,ima762worker2 # [ 0.268308] landlock: Up and running.763worker2 # [ 0.269208] Yama: becoming mindful.764worker2 # [ 0.270362] LSM support for eBPF active765worker2 # [ 0.271131] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)766worker2 # [ 0.272248] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)767worker2 # [ 0.274649] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)768worker2 # [ 0.275726] Performance Events: Fam17h+ core perfctr, AMD PMU driver.769worker2 # [ 0.276213] ... version: 2770worker2 # [ 0.276954] ... bit width: 48771worker2 # [ 0.277210] ... generic counters: 6772worker2 # [ 0.277925] ... generic bitmap: 000000000000003f773worker2 # [ 0.278220] ... fixed-purpose counters: 0774worker2 # [ 0.278954] ... fixed-purpose bitmap: 0000000000000000775worker2 # [ 0.279210] ... value mask: 0000ffffffffffff776worker2 # [ 0.280133] ... max period: 00007fffffffffff777worker2 # [ 0.280934] ... global_ctrl mask: 000000000000003f778worker2 # [ 0.281325] signal: max sigframe size: 3376779worker2 # [ 0.282141] rcu: Hierarchical SRCU implementation.780client # [ 0.269439] Freeing SMP alternatives memory: 44K781client # [ 0.270139] pid_max: default: 32768 minimum: 301782worker2 # [ 0.282798] rcu: Max phase no-delay instances is 400.783client # [ 0.271157] LSM: initializing lsm=capability,landlock,yama,bpf,ima784client # [ 0.272241] landlock: Up and running.785client # [ 0.272876] Yama: becoming mindful.786client # [ 0.273791] LSM support for eBPF active787client # [ 0.274232] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)788worker2 # [ 0.287766] smp: Bringing up secondary CPUs ...789client # [ 0.275171] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)790worker2 # [ 0.288224] smp: Brought up 1 node, 1 CPU791worker2 # [ 0.288889] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)792client # [ 0.278043] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)793worker2 # [ 0.289408] Memory: 1454560K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)794client # [ 0.278657] Performance Events: Fam17h+ core perfctr, AMD PMU driver.795worker2 # [ 0.290429] devtmpfs: initialized796client # [ 0.279142] ... version: 2797worker1 # [ 0.269399] Freeing SMP alternatives memory: 44K798worker2 # [ 0.291124] x86/mm: Memory block size: 128MB799client # [ 0.279840] ... bit width: 48800worker1 # [ 0.269648] pid_max: default: 32768 minimum: 301801client # [ 0.280139] ... generic counters: 6802lb # [ 0.268565] Freeing SMP alternatives memory: 44K803worker1 # [ 0.270749] LSM: initializing lsm=capability,landlock,yama,bpf,ima804client # [ 0.280870] ... generic bitmap: 000000000000003f805worker2 # [ 0.292745] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)806worker1 # [ 0.271749] landlock: Up and running.807lb # [ 0.269338] pid_max: default: 32768 minimum: 301808client # [ 0.281186] ... fixed-purpose counters: 0809worker1 # [ 0.272645] Yama: becoming mindful.810lb # [ 0.270015] LSM: initializing lsm=capability,landlock,yama,bpf,ima811client # [ 0.281875] ... fixed-purpose bitmap: 0000000000000000812worker2 # [ 0.293232] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).813worker1 # [ 0.273432] LSM support for eBPF active814lb # [ 0.270506] landlock: Up and running.815client # [ 0.282139] ... value mask: 0000ffffffffffff816worker2 # [ 0.294290] pinctrl core: initialized pinctrl subsystem817lb # [ 0.271821] Yama: becoming mindful.818worker1 # [ 0.274220] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)819client # [ 0.282993] ... max period: 00007fffffffffff820lb # [ 0.272616] LSM support for eBPF active821worker2 # [ 0.295349] PM: RTC time: 13:27:39, date: 2026-09-20822client # [ 0.283789] ... global_ctrl mask: 000000000000003f823worker1 # [ 0.274686] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)824lb # [ 0.273411] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)825client # [ 0.284259] signal: max sigframe size: 3376826client # [ 0.285071] rcu: Hierarchical SRCU implementation.827lb # [ 0.274443] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)828worker1 # [ 0.277707] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)829client # [ 0.285725] rcu: Max phase no-delay instances is 400.830worker2 # [ 0.298741] NET: Registered PF_NETLINK/PF_ROUTE protocol family831worker1 # [ 0.279244] Performance Events: Fam17h+ core perfctr, AMD PMU driver.832lb # [ 0.276841] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)833worker2 # [ 0.299685] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations834worker1 # [ 0.279654] ... version: 2835worker1 # [ 0.280330] ... bit width: 48836worker2 # [ 0.300230] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations837lb # [ 0.277969] Performance Events: Fam17h+ core perfctr, AMD PMU driver.838worker1 # [ 0.280647] ... generic counters: 6839lb # [ 0.278412] ... version: 2840worker1 # [ 0.281328] ... generic bitmap: 000000000000003f841worker2 # [ 0.301469] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations842lb # [ 0.279108] ... bit width: 48843worker1 # [ 0.281647] ... fixed-purpose counters: 0844client # [ 0.290710] smp: Bringing up secondary CPUs ...845lb # [ 0.279403] ... generic counters: 6846worker2 # [ 0.302219] audit: initializing netlink subsys (disabled)847worker1 # [ 0.282316] ... fixed-purpose bitmap: 0000000000000000848client # [ 0.291155] smp: Brought up 1 node, 1 CPU849lb # [ 0.280067] ... generic bitmap: 000000000000003f850worker2 # [ 0.303333] thermal_sys: Registered thermal governor 'fair_share'851worker1 # [ 0.282650] ... value mask: 0000ffffffffffff852lb # [ 0.280403] ... fixed-purpose counters: 0853client # [ 0.291829] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)854worker2 # [ 0.303336] thermal_sys: Registered thermal governor 'bang_bang'855lb # [ 0.281096] ... fixed-purpose bitmap: 0000000000000000856worker1 # [ 0.283532] ... max period: 00007fffffffffff857worker2 # [ 0.304211] thermal_sys: Registered thermal governor 'step_wise'858lb # [ 0.281403] ... value mask: 0000ffffffffffff859worker1 # [ 0.284296] ... global_ctrl mask: 000000000000003f860client # [ 0.292291] Memory: 1454576K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109348K reserved, 0K cma-reserved)861worker2 # [ 0.305127] thermal_sys: Registered thermal governor 'user_space'862worker1 # [ 0.284774] signal: max sigframe size: 3376863lb # [ 0.282226] ... max period: 00007fffffffffff864client # [ 0.293364] devtmpfs: initialized865worker1 # [ 0.285564] rcu: Hierarchical SRCU implementation.866lb # [ 0.283046] ... global_ctrl mask: 000000000000003f867worker2 # [ 0.305888] audit: type=2000 audit(1789910859.812:1): state=initialized audit_enabled=0 res=1868client # [ 0.294179] x86/mm: Memory block size: 128MB869worker1 # [ 0.286187] rcu: Max phase no-delay instances is 400.870lb # [ 0.283513] signal: max sigframe size: 3376871worker2 # [ 0.307212] thermal_sys: Registered thermal governor 'power_allocator'872lb # [ 0.284282] rcu: Hierarchical SRCU implementation.873worker2 # [ 0.307226] cpuidle: using governor menu874lb # [ 0.284981] rcu: Max phase no-delay instances is 400.875client # [ 0.295901] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)876worker2 # [ 0.309410] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5877client # [ 0.297054] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).878client # [ 0.298220] pinctrl core: initialized pinctrl subsystem879worker2 # [ 0.310436] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]880worker1 # [ 0.291145] smp: Bringing up secondary CPUs ...881worker2 # [ 0.311213] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry882client # [ 0.299326] PM: RTC time: 13:27:39, date: 2026-09-20883worker1 # [ 0.291661] smp: Brought up 1 node, 1 CPU884worker2 # [ 0.312222] PCI: Using configuration type 1 for base access885worker1 # [ 0.292308] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)886lb # [ 0.290008] smp: Bringing up secondary CPUs ...887lb # [ 0.290418] smp: Brought up 1 node, 1 CPU888worker2 # [ 0.313280] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.889lb # [ 0.291060] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)890client # [ 0.302745] NET: Registered PF_NETLINK/PF_ROUTE protocol family891worker1 # [ 0.292800] Memory: 1454556K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)892worker1 # [ 0.293902] devtmpfs: initialized893client # [ 0.303619] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations894worker1 # [ 0.294652] x86/mm: Memory block size: 128MB895lb # [ 0.291563] Memory: 1454560K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)896lb # [ 0.292656] devtmpfs: initialized897client # [ 0.304158] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations898lb # [ 0.293469] x86/mm: Memory block size: 128MB899worker1 # [ 0.296403] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)900worker2 # [ 0.318485] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages901client # [ 0.305400] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations902worker2 # [ 0.319211] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page903worker1 # [ 0.297520] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).904client # [ 0.306149] audit: initializing netlink subsys (disabled)905lb # [ 0.295205] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)906worker2 # [ 0.320189] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages907worker1 # [ 0.298719] pinctrl core: initialized pinctrl subsystem908client # [ 0.307141] audit: type=2000 audit(1789910859.834:1): state=initialized audit_enabled=0 res=1909lb # [ 0.296274] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).910worker1 # [ 0.299790] PM: RTC time: 13:27:39, date: 2026-09-20911client # [ 0.308287] thermal_sys: Registered thermal governor 'fair_share'912lb # [ 0.297486] pinctrl core: initialized pinctrl subsystem913client # [ 0.308290] thermal_sys: Registered thermal governor 'bang_bang'914lb # [ 0.298592] PM: RTC time: 13:27:39, date: 2026-09-20915client # [ 0.309140] thermal_sys: Registered thermal governor 'step_wise'916client # [ 0.310111] thermal_sys: Registered thermal governor 'user_space'917worker1 # [ 0.303352] NET: Registered PF_NETLINK/PF_ROUTE protocol family918client # [ 0.310893] thermal_sys: Registered thermal governor 'power_allocator'919worker1 # [ 0.304132] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations920client # [ 0.311154] cpuidle: using governor menu921lb # [ 0.302115] NET: Registered PF_NETLINK/PF_ROUTE protocol family922worker1 # [ 0.304671] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations923client # [ 0.313376] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5924lb # [ 0.303120] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations925worker1 # [ 0.305925] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations926worker2 # [ 0.329210] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page927lb # [ 0.303427] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations928client # [ 0.314398] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]929worker1 # [ 0.306657] audit: initializing netlink subsys (disabled)930client # [ 0.315142] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry931worker2 # [ 0.331184] ACPI: Added _OSI(Module Device)932lb # [ 0.304677] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations933worker1 # [ 0.307808] thermal_sys: Registered thermal governor 'fair_share'934worker2 # [ 0.331692] ACPI: Added _OSI(Processor Device)935client # [ 0.316152] PCI: Using configuration type 1 for base access936lb # [ 0.305414] audit: initializing netlink subsys (disabled)937worker1 # [ 0.307811] thermal_sys: Registered thermal governor 'bang_bang'938worker2 # [ 0.332210] ACPI: Added _OSI(Processor Aggregator Device)939worker1 # [ 0.308648] thermal_sys: Registered thermal governor 'step_wise'940lb # [ 0.306491] thermal_sys: Registered thermal governor 'fair_share'941client # [ 0.317233] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.942worker1 # [ 0.309573] thermal_sys: Registered thermal governor 'user_space'943lb # [ 0.306493] thermal_sys: Registered thermal governor 'bang_bang'944lb # [ 0.307405] thermal_sys: Registered thermal governor 'step_wise'945worker1 # [ 0.310383] audit: type=2000 audit(1789910859.849:1): state=initialized audit_enabled=0 res=1946lb # [ 0.308329] thermal_sys: Registered thermal governor 'user_space'947worker1 # [ 0.311651] thermal_sys: Registered thermal governor 'power_allocator'948worker1 # [ 0.311675] cpuidle: using governor menu949lb # [ 0.309109] audit: type=2000 audit(1789910859.850:1): state=initialized audit_enabled=0 res=1950lb # [ 0.310406] thermal_sys: Registered thermal governor 'power_allocator'951worker1 # [ 0.313867] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5952lb # [ 0.310432] cpuidle: using governor menu953client # [ 0.324175] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages954worker1 # [ 0.314940] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]955lb # [ 0.312584] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5956client # [ 0.325141] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page957worker1 # [ 0.315650] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry958lb # [ 0.313714] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]959worker1 # [ 0.316659] PCI: Using configuration type 1 for base access960lb # [ 0.314407] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry961worker2 # [ 0.343014] ACPI: 1 ACPI AML tables successfully acquired and loaded962worker1 # [ 0.317768] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.963lb # [ 0.315416] PCI: Using configuration type 1 for base access964lb # [ 0.316506] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.965client # [ 0.330141] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages966client # [ 0.331140] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page967worker1 # [ 0.322942] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages968worker2 # [ 0.349465] ACPI: Interpreter enabled969worker2 # [ 0.350161] ACPI: PM: (supports S0 S3 S4 S5)970worker1 # [ 0.323648] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page971lb # [ 0.321670] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages972worker2 # [ 0.350728] ACPI: Using IOAPIC for interrupt routing973lb # [ 0.322405] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page974worker2 # [ 0.351289] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug975worker2 # [ 0.352211] PCI: Using E820 reservations for host bridge windows976worker1 # [ 0.328649] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages977worker2 # [ 0.355286] ACPI: Enabled 2 GPEs in block 00 to 3F978worker1 # [ 0.329647] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page979lb # [ 0.327404] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages980lb # [ 0.328403] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page981client # [ 0.342411] ACPI: Added _OSI(Module Device)982client # [ 0.343141] ACPI: Added _OSI(Processor Device)983client # [ 0.343885] ACPI: Added _OSI(Processor Aggregator Device)984client # [ 0.348013] ACPI: 1 ACPI AML tables successfully acquired and loaded985worker2 # [ 0.363865] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])986worker2 # [ 0.364216] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]987worker1 # [ 0.340038] ACPI: Added _OSI(Module Device)988worker1 # [ 0.340649] ACPI: Added _OSI(Processor Device)989lb # [ 0.338849] ACPI: Added _OSI(Module Device)990worker1 # [ 0.341401] ACPI: Added _OSI(Processor Aggregator Device)991lb # [ 0.339405] ACPI: Added _OSI(Processor Device)992worker2 # [ 0.366416] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]993lb # [ 0.340136] ACPI: Added _OSI(Processor Aggregator Device)994worker2 # [ 0.367339] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]995client # [ 0.354132] ACPI: Interpreter enabled996worker2 # [ 0.368571] PCI host bridge to bus 0000:00997client # [ 0.354791] ACPI: PM: (supports S0 S3 S4 S5)998worker2 # [ 0.369215] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]999client # [ 0.355140] ACPI: Using IOAPIC for interrupt routing1000worker2 # [ 0.370210] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1001client # [ 0.356050] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1002worker2 # [ 0.371210] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1003worker2 # [ 0.372210] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1004worker2 # [ 0.373210] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1005worker1 # [ 0.350224] ACPI: 1 ACPI AML tables successfully acquired and loaded1006worker2 # [ 0.374210] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1007client # [ 0.361140] PCI: Using E820 reservations for host bridge windows1008lb # [ 0.348967] ACPI: 1 ACPI AML tables successfully acquired and loaded1009worker2 # [ 0.375210] pci_bus 0000:00: root bus resource [bus 00-ff]1010client # [ 0.362243] ACPI: Enabled 2 GPEs in block 00 to 3F1011worker2 # [ 0.376200] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1012worker1 # [ 0.354066] ACPI: Interpreter enabled1013worker2 # [ 0.377673] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1014worker1 # [ 0.354664] ACPI: PM: (supports S0 S3 S4 S5)1015lb # [ 0.352824] ACPI: Interpreter enabled1016worker1 # [ 0.355358] ACPI: Using IOAPIC for interrupt routing1017lb # [ 0.353421] ACPI: PM: (supports S0 S3 S4 S5)1018lb # [ 0.354093] ACPI: Using IOAPIC for interrupt routing1019worker1 # [ 0.355681] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1020worker1 # [ 0.356647] PCI: Using E820 reservations for host bridge windows1021lb # [ 0.354455] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1022lb # [ 0.355404] PCI: Using E820 reservations for host bridge windows1023client # [ 0.370215] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1024worker1 # [ 0.359709] ACPI: Enabled 2 GPEs in block 00 to 3F1025worker2 # [ 0.380277] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1026client # [ 0.371145] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1027lb # [ 0.358453] ACPI: Enabled 2 GPEs in block 00 to 3F1028worker2 # [ 0.381222] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1029client # [ 0.372217] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1030worker2 # [ 0.382175] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1031client # [ 0.373265] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1032worker2 # [ 0.383234] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1033client # [ 0.374500] PCI host bridge to bus 0000:001034client # [ 0.375145] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1035client # [ 0.376140] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1036worker2 # [ 0.384988] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1037client # [ 0.377140] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1038worker1 # [ 0.368298] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1039client # [ 0.378140] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1040lb # [ 0.367002] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1041worker1 # [ 0.368654] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1042client # [ 0.379140] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1043worker2 # [ 0.388229] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1044client # [ 0.380139] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1045lb # [ 0.367410] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1046worker2 # [ 0.389077] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1047worker1 # [ 0.370231] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1048client # [ 0.381140] pci_bus 0000:00: root bus resource [bus 00-ff]1049worker2 # [ 0.389915] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1050worker1 # [ 0.370775] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1051lb # [ 0.368844] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1052client # [ 0.382090] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1053worker2 # [ 0.390216] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1054worker1 # [ 0.372059] PCI host bridge to bus 0000:001055lb # [ 0.369528] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1056worker1 # [ 0.372652] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1057lb # [ 0.370782] PCI host bridge to bus 0000:001058client # [ 0.383553] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1059worker2 # [ 0.391908] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1060worker1 # [ 0.373647] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1061lb # [ 0.371408] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1062lb # [ 0.372403] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1063worker1 # [ 0.374647] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1064lb # [ 0.373403] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1065worker1 # [ 0.375647] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1066worker1 # [ 0.376647] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1067lb # [ 0.374403] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1068worker2 # [ 0.395327] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1069worker1 # [ 0.377647] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1070lb # [ 0.375403] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1071worker2 # [ 0.396865] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1072worker1 # [ 0.378648] pci_bus 0000:00: root bus resource [bus 00-ff]1073worker2 # [ 0.397231] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1074lb # [ 0.376403] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1075client # [ 0.386259] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1076lb # [ 0.377404] pci_bus 0000:00: root bus resource [bus 00-ff]1077worker1 # [ 0.379583] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1078client # [ 0.387194] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1079worker2 # [ 0.398771] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1080lb # [ 0.378395] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1081worker1 # [ 0.381046] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1082client # [ 0.388130] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1083lb # [ 0.379915] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1084client # [ 0.389416] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1085worker2 # [ 0.401232] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1086worker2 # [ 0.402103] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1087client # [ 0.390843] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1088worker2 # [ 0.402980] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1089client # [ 0.393146] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1090worker2 # [ 0.405144] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1091client # [ 0.393992] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1092worker1 # [ 0.383765] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1093client # [ 0.394879] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1094worker1 # [ 0.384672] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1095lb # [ 0.382481] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1096client # [ 0.395145] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1097worker2 # [ 0.408038] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1098worker1 # [ 0.385591] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1099lb # [ 0.383416] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1100worker2 # [ 0.408845] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1101lb # [ 0.384344] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1102client # [ 0.396634] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1103worker2 # [ 0.409232] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1104worker1 # [ 0.386524] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1105lb # [ 0.385364] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1106worker2 # [ 0.410753] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1107worker1 # [ 0.388559] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1108client # [ 0.399140] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1109lb # [ 0.387140] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1110client # [ 0.399979] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1111client # [ 0.400889] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1112worker1 # [ 0.391690] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1113worker2 # [ 0.413170] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1114worker1 # [ 0.392539] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1115lb # [ 0.390421] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1116client # [ 0.401680] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1117worker2 # [ 0.413862] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1118lb # [ 0.391224] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1119worker1 # [ 0.393364] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1120worker2 # [ 0.414233] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1121worker1 # [ 0.393653] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1122lb # [ 0.392117] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1123client # [ 0.403146] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1124lb # [ 0.392410] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1125worker2 # [ 0.415771] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1126client # [ 0.403995] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1127worker1 # [ 0.395191] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1128client # [ 0.404866] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1129lb # [ 0.393928] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1130worker2 # [ 0.418145] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1131client # [ 0.406258] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1132worker2 # [ 0.418893] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1133worker1 # [ 0.397678] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1134lb # [ 0.396344] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1135worker1 # [ 0.398512] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1136worker2 # [ 0.419232] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1137lb # [ 0.397002] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1138client # [ 0.408147] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1139worker1 # [ 0.399364] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1140lb # [ 0.397425] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1141client # [ 0.408997] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1142worker1 # [ 0.400188] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1143worker2 # [ 0.421350] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1144client # [ 0.409889] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1145lb # [ 0.398989] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1146client # [ 0.410704] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1147worker2 # [ 0.423869] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1148worker2 # [ 0.424217] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1149worker1 # [ 0.402946] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1150worker2 # [ 0.425232] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1151lb # [ 0.401429] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1152client # [ 0.412147] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1153worker1 # [ 0.404263] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1154worker2 # [ 0.426216] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1155lb # [ 0.402410] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1156client # [ 0.413148] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1157worker1 # [ 0.404669] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1158lb # [ 0.403414] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1159client # [ 0.414088] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1160worker2 # [ 0.427722] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1161worker1 # [ 0.406246] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1162lb # [ 0.404974] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1163client # [ 0.415550] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1164worker2 # [ 0.429223] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1165worker1 # [ 0.408490] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1166client # [ 0.417150] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1167worker2 # [ 0.431002] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1168lb # [ 0.407368] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1169worker1 # [ 0.409284] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1170client # [ 0.418002] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1171lb # [ 0.408003] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1172worker1 # [ 0.409670] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1173worker2 # [ 0.432624] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1174client # [ 0.418885] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1175lb # [ 0.408426] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1176client # [ 0.419718] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1177worker1 # [ 0.411180] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1178worker2 # [ 0.435041] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1179lb # [ 0.410023] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1180worker2 # [ 0.435838] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1181client # [ 0.421153] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1182worker2 # [ 0.436232] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1183worker1 # [ 0.414483] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1184client # [ 0.422007] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1185lb # [ 0.412237] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1186worker1 # [ 0.415282] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1187client # [ 0.422860] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1188lb # [ 0.413029] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1189worker1 # [ 0.415669] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1190client # [ 0.423147] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1191worker2 # [ 0.437740] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1192lb # [ 0.413426] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1193worker1 # [ 0.417279] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1194client # [ 0.424695] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1195lb # [ 0.414968] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1196worker2 # [ 0.439218] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1197worker2 # [ 0.440074] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1198client # [ 0.426151] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1199worker2 # [ 0.440993] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1200client # [ 0.427126] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1201worker1 # [ 0.419664] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1202worker2 # [ 0.442636] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1203lb # [ 0.417434] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1204worker1 # [ 0.421268] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1205client # [ 0.428689] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1206lb # [ 0.419045] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1207worker1 # [ 0.421669] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1208worker2 # [ 0.443936] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1209lb # [ 0.419426] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1210client # [ 0.430164] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1211worker2 # [ 0.444415] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1212worker1 # [ 0.423196] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1213lb # [ 0.420999] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1214client # [ 0.431006] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1215client # [ 0.431900] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1216worker2 # [ 0.446717] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1217worker1 # [ 0.424655] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1218lb # [ 0.423384] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1219worker1 # [ 0.425493] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1220client # [ 0.432712] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1221lb # [ 0.424018] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1222worker1 # [ 0.426358] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1223lb # [ 0.424433] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1224worker1 # [ 0.426653] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1225client # [ 0.434148] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1226lb # [ 0.425409] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1227client # [ 0.435146] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1228worker1 # [ 0.428210] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1229client # [ 0.436124] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1230lb # [ 0.426974] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1231worker1 # [ 0.429683] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1232client # [ 0.437552] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1233lb # [ 0.428416] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1234worker1 # [ 0.430639] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1235client # [ 0.438675] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1236lb # [ 0.429364] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1237worker1 # [ 0.432202] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1238client # [ 0.439358] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1239lb # [ 0.430785] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1240worker2 # [ 0.447417] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1241client # [ 0.440663] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1242worker1 # [ 0.433686] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1243worker2 # [ 0.450037] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1244lb # [ 0.433340] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1245worker1 # [ 0.434511] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1246worker2 # [ 0.451097] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1247lb # [ 0.434410] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1248worker1 # [ 0.435670] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1249lb # [ 0.435352] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1250worker2 # [ 0.452851] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1251worker1 # [ 0.437249] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1252lb # [ 0.436763] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1253worker2 # [ 0.453477] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1254worker2 # [ 0.454810] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1255worker1 # [ 0.439379] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1256lb # [ 0.439211] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1257worker1 # [ 0.439655] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1258worker2 # [ 0.455466] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1259lb # [ 0.440010] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1260worker1 # [ 0.440603] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1261lb # [ 0.440426] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1262worker2 # [ 0.457239] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1263worker1 # [ 0.442080] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1264client # [ 0.441359] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1265worker2 # [ 0.458083] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1266lb # [ 0.441974] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1267worker2 # [ 0.459238] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1268worker1 # [ 0.443698] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1269lb # [ 0.443247] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1270worker2 # [ 0.460903] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1271worker1 # [ 0.444780] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1272lb # [ 0.444221] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1273client # [ 0.444720] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1274worker1 # [ 0.446392] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1275lb # [ 0.446107] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1276client # [ 0.445392] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1277client # [ 0.447106] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1278worker2 # [ 0.464483] ACPI: PCI: Interrupt link LNKA configured for IRQ 101279worker2 # [ 0.465319] ACPI: PCI: Interrupt link LNKB configured for IRQ 101280client # [ 0.448109] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1281worker2 # [ 0.466306] ACPI: PCI: Interrupt link LNKC configured for IRQ 111282client # [ 0.449433] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1283worker2 # [ 0.467304] ACPI: PCI: Interrupt link LNKD configured for IRQ 111284worker2 # [ 0.468306] ACPI: PCI: Interrupt link LNKE configured for IRQ 101285client # [ 0.450387] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1286worker2 # [ 0.469311] ACPI: PCI: Interrupt link LNKF configured for IRQ 101287worker2 # [ 0.470311] ACPI: PCI: Interrupt link LNKG configured for IRQ 111288client # [ 0.452620] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1289worker2 # [ 0.471315] ACPI: PCI: Interrupt link LNKH configured for IRQ 111290client # [ 0.453145] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1291worker2 # [ 0.472248] ACPI: PCI: Interrupt link GSIA configured for IRQ 161292client # [ 0.454396] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1293worker2 # [ 0.473226] ACPI: PCI: Interrupt link GSIB configured for IRQ 171294lb # [ 0.446599] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1295worker1 # [ 0.446871] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1296worker2 # [ 0.474215] ACPI: PCI: Interrupt link GSIC configured for IRQ 181297client # [ 0.455839] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1298worker1 # [ 0.449391] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1299worker2 # [ 0.475215] ACPI: PCI: Interrupt link GSID configured for IRQ 191300lb # [ 0.449214] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1301worker2 # [ 0.476167] ACPI: PCI: Interrupt link GSIE configured for IRQ 201302lb # [ 0.450269] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1303worker1 # [ 0.450048] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1304worker2 # [ 0.476978] ACPI: PCI: Interrupt link GSIF configured for IRQ 211305worker2 # [ 0.477226] ACPI: PCI: Interrupt link GSIG configured for IRQ 221306client # [ 0.459070] ACPI: PCI: Interrupt link LNKA configured for IRQ 101307lb # [ 0.452102] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1308worker1 # [ 0.451415] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1309worker2 # [ 0.478215] ACPI: PCI: Interrupt link GSIH configured for IRQ 231310client # [ 0.459954] ACPI: PCI: Interrupt link LNKB configured for IRQ 101311lb # [ 0.452680] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1312worker1 # [ 0.451878] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1313client # [ 0.461209] ACPI: PCI: Interrupt link LNKC configured for IRQ 111314worker2 # [ 0.480197] iommu: Default domain type: Translated1315client # [ 0.462235] ACPI: PCI: Interrupt link LNKD configured for IRQ 111316lb # [ 0.453694] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1317worker1 # [ 0.452959] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1318worker2 # [ 0.480818] iommu: DMA domain TLB invalidation policy: lazy mode1319client # [ 0.463236] ACPI: PCI: Interrupt link LNKE configured for IRQ 101320worker2 # [ 0.481487] ACPI: bus type USB registered1321lb # [ 0.454685] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1322client # [ 0.464234] ACPI: PCI: Interrupt link LNKF configured for IRQ 101323worker2 # [ 0.482224] usbcore: registered new interface driver usbfs1324worker1 # [ 0.453906] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1325client # [ 0.465236] ACPI: PCI: Interrupt link LNKG configured for IRQ 111326worker2 # [ 0.483135] usbcore: registered new interface driver hub1327lb # [ 0.456432] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1328worker1 # [ 0.455702] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1329worker2 # [ 0.483858] usbcore: registered new device driver usb1330client # [ 0.466244] ACPI: PCI: Interrupt link LNKH configured for IRQ 111331lb # [ 0.457265] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1332worker1 # [ 0.456508] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1333client # [ 0.467174] ACPI: PCI: Interrupt link GSIA configured for IRQ 161334worker2 # [ 0.485054] NetLabel: Initializing1335worker2 # [ 0.485647] NetLabel: domain hash size = 1281336client # [ 0.468116] ACPI: PCI: Interrupt link GSIB configured for IRQ 171337client # [ 0.468864] ACPI: PCI: Interrupt link GSIC configured for IRQ 181338worker2 # [ 0.486210] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1339worker1 # [ 0.457684] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1340lb # [ 0.458600] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1341worker2 # [ 0.487114] NetLabel: unlabeled traffic allowed by default1342client # [ 0.469151] ACPI: PCI: Interrupt link GSID configured for IRQ 191343lb # [ 0.460252] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1344worker2 # [ 0.487886] PCI: Using ACPI for IRQ routing1345worker1 # [ 0.460385] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1346client # [ 0.470154] ACPI: PCI: Interrupt link GSIE configured for IRQ 201347client # [ 0.471113] ACPI: PCI: Interrupt link GSIF configured for IRQ 211348client # [ 0.471902] ACPI: PCI: Interrupt link GSIG configured for IRQ 221349client # [ 0.472150] ACPI: PCI: Interrupt link GSIH configured for IRQ 231350lb # [ 0.464050] ACPI: PCI: Interrupt link LNKA configured for IRQ 101351worker1 # [ 0.463672] ACPI: PCI: Interrupt link LNKA configured for IRQ 101352lb # [ 0.464509] ACPI: PCI: Interrupt link LNKB configured for IRQ 101353client # [ 0.474113] iommu: Default domain type: Translated1354worker1 # [ 0.464753] ACPI: PCI: Interrupt link LNKB configured for IRQ 101355lb # [ 0.465506] ACPI: PCI: Interrupt link LNKC configured for IRQ 111356client # [ 0.474731] iommu: DMA domain TLB invalidation policy: lazy mode1357worker1 # [ 0.465747] ACPI: PCI: Interrupt link LNKC configured for IRQ 111358client # [ 0.475381] ACPI: bus type USB registered1359worker1 # [ 0.466746] ACPI: PCI: Interrupt link LNKD configured for IRQ 111360lb # [ 0.466500] ACPI: PCI: Interrupt link LNKD configured for IRQ 111361client # [ 0.476111] usbcore: registered new interface driver usbfs1362lb # [ 0.467503] ACPI: PCI: Interrupt link LNKE configured for IRQ 101363worker1 # [ 0.467745] ACPI: PCI: Interrupt link LNKE configured for IRQ 101364client # [ 0.476833] usbcore: registered new interface driver hub1365lb # [ 0.468500] ACPI: PCI: Interrupt link LNKF configured for IRQ 101366client # [ 0.477149] usbcore: registered new device driver usb1367worker1 # [ 0.468745] ACPI: PCI: Interrupt link LNKF configured for IRQ 101368lb # [ 0.469503] ACPI: PCI: Interrupt link LNKG configured for IRQ 111369worker1 # [ 0.469746] ACPI: PCI: Interrupt link LNKG configured for IRQ 111370client # [ 0.478831] NetLabel: Initializing1371lb # [ 0.470503] ACPI: PCI: Interrupt link LNKH configured for IRQ 111372worker1 # [ 0.470748] ACPI: PCI: Interrupt link LNKH configured for IRQ 111373client # [ 0.479140] NetLabel: domain hash size = 1281374lb # [ 0.471441] ACPI: PCI: Interrupt link GSIA configured for IRQ 161375client # [ 0.479857] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1376worker1 # [ 0.471684] ACPI: PCI: Interrupt link GSIA configured for IRQ 161377lb # [ 0.472380] ACPI: PCI: Interrupt link GSIB configured for IRQ 171378client # [ 0.480194] NetLabel: unlabeled traffic allowed by default1379worker1 # [ 0.472654] ACPI: PCI: Interrupt link GSIB configured for IRQ 171380lb # [ 0.473123] ACPI: PCI: Interrupt link GSIC configured for IRQ 181381client # [ 0.481074] PCI: Using ACPI for IRQ routing1382worker1 # [ 0.473604] ACPI: PCI: Interrupt link GSIC configured for IRQ 181383lb # [ 0.473416] ACPI: PCI: Interrupt link GSID configured for IRQ 191384worker1 # [ 0.474412] ACPI: PCI: Interrupt link GSID configured for IRQ 191385lb # [ 0.474361] ACPI: PCI: Interrupt link GSIE configured for IRQ 201386worker1 # [ 0.474659] ACPI: PCI: Interrupt link GSIE configured for IRQ 201387lb # [ 0.475135] ACPI: PCI: Interrupt link GSIF configured for IRQ 211388worker1 # [ 0.475637] ACPI: PCI: Interrupt link GSIF configured for IRQ 211389lb # [ 0.475416] ACPI: PCI: Interrupt link GSIG configured for IRQ 221390worker1 # [ 0.476438] ACPI: PCI: Interrupt link GSIG configured for IRQ 221391lb # [ 0.476366] ACPI: PCI: Interrupt link GSIH configured for IRQ 231392worker1 # [ 0.477449] ACPI: PCI: Interrupt link GSIH configured for IRQ 231393lb # [ 0.478256] iommu: Default domain type: Translated1394worker1 # [ 0.479590] iommu: Default domain type: Translated1395lb # [ 0.479049] iommu: DMA domain TLB invalidation policy: lazy mode1396worker1 # [ 0.480248] iommu: DMA domain TLB invalidation policy: lazy mode1397lb # [ 0.479669] ACPI: bus type USB registered1398worker1 # [ 0.480911] ACPI: bus type USB registered1399lb # [ 0.480419] usbcore: registered new interface driver usbfs1400lb # [ 0.481295] usbcore: registered new interface driver hub1401worker1 # [ 0.481652] usbcore: registered new interface driver usbfs1402lb # [ 0.482056] usbcore: registered new device driver usb1403worker1 # [ 0.482560] usbcore: registered new interface driver hub1404worker1 # [ 0.483355] usbcore: registered new device driver usb1405lb # [ 0.483552] NetLabel: Initializing1406worker1 # [ 0.484507] NetLabel: Initializing1407lb # [ 0.484249] NetLabel: domain hash size = 1281408worker1 # [ 0.485072] NetLabel: domain hash size = 1281409lb # [ 0.485005] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1410worker1 # [ 0.485647] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1411lb # [ 0.485464] NetLabel: unlabeled traffic allowed by default1412worker1 # [ 0.486689] NetLabel: unlabeled traffic allowed by default1413lb # [ 0.486418] PCI: Using ACPI for IRQ routing1414worker1 # [ 0.487581] PCI: Using ACPI for IRQ routing1415worker2 # [ 0.532370] pci 0000:00:01.0: vgaarb: setting as boot VGA device1416worker2 # [ 0.533206] pci 0000:00:01.0: vgaarb: bridge control possible1417worker2 # [ 0.533206] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1418worker2 # [ 0.533215] vgaarb: loaded1419worker2 # [ 0.533978] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01420worker2 # [ 0.534212] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1421worker2 # [ 0.538293] clocksource: Switched to clocksource kvm-clock1422worker2 # [ 0.539972] VFS: Disk quotas dquot_6.6.01423client # [ 0.524603] pci 0000:00:01.0: vgaarb: setting as boot VGA device1424worker2 # [ 0.542415] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1425worker2 # [ 0.543780] pnp: PnP ACPI init1426client # [ 0.525135] pci 0000:00:01.0: vgaarb: bridge control possible1427worker2 # [ 0.544584] ACPI: IRQ 4 override to edge(!), high(!)1428client # [ 0.525135] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1429client # [ 0.525144] vgaarb: loaded1430worker2 # [ 0.545582] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1431client # [ 0.525866] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01432worker2 # [ 0.547175] pnp: PnP ACPI: found 6 devices1433client # [ 0.526139] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1434client # [ 0.530226] clocksource: Switched to clocksource kvm-clock1435worker2 # [ 0.554815] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1436client # [ 0.533582] VFS: Disk quotas dquot_6.6.01437worker2 # [ 0.556340] clocksource: Switched to clocksource acpi_pm1438client # [ 0.534288] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1439worker2 # [ 0.557313] NET: Registered PF_INET protocol family1440client # [ 0.535610] pnp: PnP ACPI init1441client # [ 0.536394] ACPI: IRQ 4 override to edge(!), high(!)1442worker2 # [ 0.558484] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1443lb # [ 0.530640] pci 0000:00:01.0: vgaarb: setting as boot VGA device1444client # [ 0.537351] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1445lb # [ 0.531400] pci 0000:00:01.0: vgaarb: bridge control possible1446worker1 # [ 0.531928] pci 0000:00:01.0: vgaarb: setting as boot VGA device1447client # [ 0.538862] pnp: PnP ACPI: found 6 devices1448worker1 # [ 0.532643] pci 0000:00:01.0: vgaarb: bridge control possible1449lb # [ 0.531400] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1450lb # [ 0.531408] vgaarb: loaded1451worker1 # [ 0.532643] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1452lb # [ 0.532148] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01453worker1 # [ 0.532652] vgaarb: loaded1454lb # [ 0.532405] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1455worker1 # [ 0.533392] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01456worker1 # [ 0.533649] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1457worker1 # [ 0.536741] clocksource: Switched to clocksource kvm-clock1458client # [ 0.546420] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1459lb # [ 0.537493] clocksource: Switched to clocksource kvm-clock1460client # [ 0.547871] clocksource: Switched to clocksource acpi_pm1461client # [ 0.548827] NET: Registered PF_INET protocol family1462worker1 # [ 0.540206] VFS: Disk quotas dquot_6.6.01463client # [ 0.549991] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1464lb # [ 0.541032] VFS: Disk quotas dquot_6.6.01465worker1 # [ 0.540928] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1466worker1 # [ 0.542259] pnp: PnP ACPI init1467lb # [ 0.541716] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1468worker2 # [ 0.574510] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1469lb # [ 0.543025] pnp: PnP ACPI init1470worker1 # [ 0.543036] ACPI: IRQ 4 override to edge(!), high(!)1471lb # [ 0.543827] ACPI: IRQ 4 override to edge(!), high(!)1472worker2 # [ 0.576028] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1473worker1 # [ 0.544032] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1474lb # [ 0.544745] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1475worker1 # [ 0.545549] pnp: PnP ACPI: found 6 devices1476worker2 # [ 0.577364] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1477lb # [ 0.546250] pnp: PnP ACPI: found 6 devices1478worker2 # [ 0.578657] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1479worker2 # [ 0.579929] TCP: Hash tables configured (established 16384 bind 16384)1480worker2 # [ 0.581107] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1481worker2 # [ 0.582410] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1482worker2 # [ 0.583540] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1483worker2 # [ 0.584801] NET: Registered PF_UNIX/PF_LOCAL protocol family1484worker2 # [ 0.585819] NET: Registered PF_XDP protocol family1485worker1 # [ 0.553150] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1486worker2 # [ 0.586725] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1487lb # [ 0.553861] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1488worker1 # [ 0.554802] clocksource: Switched to clocksource acpi_pm1489worker2 # [ 0.587680] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1490lb # [ 0.555320] clocksource: Switched to clocksource acpi_pm1491worker1 # [ 0.555897] NET: Registered PF_INET protocol family1492client # [ 0.566015] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1493worker2 # [ 0.588657] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1494lb # [ 0.556270] NET: Registered PF_INET protocol family1495worker2 # [ 0.589682] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1496worker1 # [ 0.557089] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1497client # [ 0.567462] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1498lb # [ 0.557405] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1499worker2 # [ 0.590702] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1500client # [ 0.568680] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1501worker2 # [ 0.591772] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1502client # [ 0.570006] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1503worker2 # [ 0.593458] ACPI: \_SB_.GSIA: Enabled at IRQ 161504client # [ 0.571234] TCP: Hash tables configured (established 16384 bind 16384)1505client # [ 0.572325] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1506worker2 # [ 0.596026] ACPI: \_SB_.GSIB: Enabled at IRQ 171507client # [ 0.573573] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1508client # [ 0.574628] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1509worker2 # [ 0.597886] ACPI: \_SB_.GSIC: Enabled at IRQ 181510client # [ 0.575846] NET: Registered PF_UNIX/PF_LOCAL protocol family1511client # [ 0.576796] NET: Registered PF_XDP protocol family1512worker2 # [ 0.599767] ACPI: \_SB_.GSID: Enabled at IRQ 191513client # [ 0.577605] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1514client # [ 0.578590] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1515worker2 # [ 0.601372] PCI: CLS 0 bytes, default 641516client # [ 0.579580] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1517worker2 # [ 0.602292] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1518client # [ 0.580724] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1519worker2 # [ 0.604023] Trying to unpack rootfs image as initramfs...1520client # [ 0.581810] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1521client # [ 0.582874] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1522client # [ 0.584549] ACPI: \_SB_.GSIA: Enabled at IRQ 161523worker1 # [ 0.574084] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1524lb # [ 0.574067] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1525lb # [ 0.575448] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1526worker1 # [ 0.575676] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1527client # [ 0.587064] ACPI: \_SB_.GSIB: Enabled at IRQ 171528lb # [ 0.576766] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1529worker1 # [ 0.576989] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1530lb # [ 0.578088] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1531client # [ 0.588870] ACPI: \_SB_.GSIC: Enabled at IRQ 181532worker1 # [ 0.578335] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1533worker1 # [ 0.579560] TCP: Hash tables configured (established 16384 bind 16384)1534lb # [ 0.579312] TCP: Hash tables configured (established 16384 bind 16384)1535client # [ 0.590730] ACPI: \_SB_.GSID: Enabled at IRQ 191536lb # [ 0.580469] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1537worker1 # [ 0.580702] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1538client # [ 0.592110] PCI: CLS 0 bytes, default 641539lb # [ 0.581723] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1540worker1 # [ 0.581962] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1541lb # [ 0.582770] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1542worker1 # [ 0.583029] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1543client # [ 0.592955] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1544lb # [ 0.583961] NET: Registered PF_UNIX/PF_LOCAL protocol family1545worker1 # [ 0.584206] NET: Registered PF_UNIX/PF_LOCAL protocol family1546client # [ 0.594628] Trying to unpack rootfs image as initramfs...1547lb # [ 0.584907] NET: Registered PF_XDP protocol family1548worker1 # [ 0.585140] NET: Registered PF_XDP protocol family1549lb # [ 0.585710] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1550worker1 # [ 0.586006] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1551lb # [ 0.586710] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1552worker1 # [ 0.587023] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1553lb # [ 0.587732] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1554worker1 # [ 0.588043] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1555lb # [ 0.588843] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1556worker1 # [ 0.589141] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1557lb # [ 0.589971] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1558worker1 # [ 0.590265] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1559lb # [ 0.591085] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1560worker1 # [ 0.591363] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1561lb # [ 0.592865] ACPI: \_SB_.GSIA: Enabled at IRQ 161562worker1 # [ 0.593104] ACPI: \_SB_.GSIA: Enabled at IRQ 161563lb # [ 0.595459] ACPI: \_SB_.GSIB: Enabled at IRQ 171564worker1 # [ 0.595835] ACPI: \_SB_.GSIB: Enabled at IRQ 171565lb # [ 0.597396] ACPI: \_SB_.GSIC: Enabled at IRQ 181566worker1 # [ 0.597852] ACPI: \_SB_.GSIC: Enabled at IRQ 181567lb # [ 0.599365] ACPI: \_SB_.GSID: Enabled at IRQ 191568worker1 # [ 0.599782] ACPI: \_SB_.GSID: Enabled at IRQ 191569lb # [ 0.601053] PCI: CLS 0 bytes, default 641570worker1 # [ 0.601499] PCI: CLS 0 bytes, default 641571lb # [ 0.602004] Trying to unpack rootfs image as initramfs...1572worker1 # [ 0.602439] Trying to unpack rootfs image as initramfs...1573worker1 # [ 0.606631] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1574lb # [ 0.606309] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1575worker2 # [ 0.650509] Initialise system trusted keyrings1576worker2 # [ 0.651483] workingset: timestamp_bits=40 max_order=19 bucket_order=01577client # [ 0.636165] Initialise system trusted keyrings1578client # [ 0.639759] workingset: timestamp_bits=40 max_order=19 bucket_order=01579worker2 # [ 0.675313] Key type asymmetric registered1580worker2 # [ 0.676048] Asymmetric key parser 'x509' registered1581worker2 # [ 0.676965] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1582lb # [ 0.647572] Initialise system trusted keyrings1583worker2 # [ 0.680202] io scheduler mq-deadline registered1584worker2 # [ 0.680964] io scheduler kyber registered1585client # [ 0.659040] Key type asymmetric registered1586worker1 # [ 0.649477] Initialise system trusted keyrings1587lb # [ 0.650942] workingset: timestamp_bits=40 max_order=19 bucket_order=01588worker2 # [ 0.684644] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1589worker1 # [ 0.652354] workingset: timestamp_bits=40 max_order=19 bucket_order=01590client # [ 0.663747] Asymmetric key parser 'x509' registered1591worker2 # [ 0.685938] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1592client # [ 0.664584] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1593client # [ 0.667791] io scheduler mq-deadline registered1594client # [ 0.668539] io scheduler kyber registered1595worker2 # [ 0.691225] Linux agpgart interface v0.1031596worker2 # [ 0.692001] ACPI: bus type drm_connector registered1597client # [ 0.671892] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1598worker2 # [ 0.695283] usbcore: registered new interface driver usbserial_generic1599client # [ 0.673157] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1600worker2 # [ 0.696373] usbserial: USB Serial support registered for generic1601worker2 # [ 0.699103] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1602client # [ 0.678126] Linux agpgart interface v0.1031603worker2 # [ 0.699103] Please enable it if your BIOS has the CPPC option.1604client # [ 0.678893] ACPI: bus type drm_connector registered1605worker2 # [ 0.701428] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1606lb # [ 0.672131] Key type asymmetric registered1607client # [ 0.682184] usbcore: registered new interface driver usbserial_generic1608lb # [ 0.672834] Asymmetric key parser 'x509' registered1609worker2 # [ 0.705246] drop_monitor: Initializing network drop monitor service1610client # [ 0.683264] usbserial: USB Serial support registered for generic1611worker1 # [ 0.673610] Key type asymmetric registered1612worker2 # [ 0.706395] NET: Registered PF_INET6 protocol family1613worker1 # [ 0.674323] Asymmetric key parser 'x509' registered1614worker1 # [ 0.675166] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1615client # [ 0.685722] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1616lb # [ 0.675956] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1617client # [ 0.685722] Please enable it if your BIOS has the CPPC option.1618worker2 # [ 0.709497] Segment Routing with IPv61619client # [ 0.687899] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1620lb # [ 0.678956] io scheduler mq-deadline registered1621worker2 # [ 0.712109] In-situ OAM (IOAM) with IPv61622lb # [ 0.679703] io scheduler kyber registered1623worker1 # [ 0.680435] io scheduler mq-deadline registered1624worker2 # [ 0.713119] IPI shorthand broadcast: enabled1625client # [ 0.690829] drop_monitor: Initializing network drop monitor service1626worker1 # [ 0.681209] io scheduler kyber registered1627client # [ 0.691950] NET: Registered PF_INET6 protocol family1628lb # [ 0.682476] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1629lb # [ 0.683723] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1630client # [ 0.695179] Segment Routing with IPv61631worker1 # [ 0.684917] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1632worker1 # [ 0.686223] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1633client # [ 0.697747] In-situ OAM (IOAM) with IPv61634lb # [ 0.687689] Linux agpgart interface v0.1031635client # [ 0.698778] IPI shorthand broadcast: enabled1636lb # [ 0.688427] ACPI: bus type drm_connector registered1637worker2 # [ 0.721206] sched_clock: Marking stable (584015885, 137065135)->(791322913, -70241893)1638worker1 # [ 0.690173] Linux agpgart interface v0.1031639worker1 # [ 0.690993] ACPI: bus type drm_connector registered1640worker2 # [ 0.725274] registered taskstats version 11641lb # [ 0.693334] usbcore: registered new interface driver usbserial_generic1642worker2 # [ 0.726311] Loading compiled-in X.509 certificates1643lb # [ 0.694415] usbserial: USB Serial support registered for generic1644worker1 # [ 0.695775] usbcore: registered new interface driver usbserial_generic1645client # [ 0.706048] sched_clock: Marking stable (570013827, 135691177)->(781323133, -75618129)1646worker1 # [ 0.696887] usbserial: USB Serial support registered for generic1647lb # [ 0.697909] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1648lb # [ 0.697909] Please enable it if your BIOS has the CPPC option.1649worker1 # [ 0.699337] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1650worker1 # [ 0.699337] Please enable it if your BIOS has the CPPC option.1651lb # [ 0.700158] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1652client # [ 0.711909] registered taskstats version 11653lb # [ 0.701527] drop_monitor: Initializing network drop monitor service1654worker1 # [ 0.701528] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1655client # [ 0.712847] Loading compiled-in X.509 certificates1656lb # [ 0.702664] NET: Registered PF_INET6 protocol family1657worker1 # [ 0.705492] drop_monitor: Initializing network drop monitor service1658worker1 # [ 0.706779] NET: Registered PF_INET6 protocol family1659lb # [ 0.707356] Segment Routing with IPv61660lb # [ 0.708064] In-situ OAM (IOAM) with IPv61661worker1 # [ 0.711414] Segment Routing with IPv61662lb # [ 0.711196] IPI shorthand broadcast: enabled1663worker2 # [ 0.744097] Demotion targets for Node 0: null1664worker1 # [ 0.712093] In-situ OAM (IOAM) with IPv61665worker1 # [ 0.713202] IPI shorthand broadcast: enabled1666worker2 # [ 0.746133] Key type .fscrypt registered1667worker2 # [ 0.746802] Key type fscrypt-provisioning registered1668worker2 # [ 0.747770] ima: No TPM chip found, activating TPM-bypass!1669worker2 # [ 0.752095] ima: Allocated hash algorithm: sha11670lb # [ 0.719499] sched_clock: Marking stable (585014246, 133870711)->(790401198, -71516241)1671worker2 # [ 0.752862] ima: No architecture policies found1672client # [ 0.730720] Demotion targets for Node 0: null1673client # [ 0.731581] Key type .fscrypt registered1674worker1 # [ 0.721473] sched_clock: Marking stable (587014178, 134285698)->(794323248, -73023372)1675client # [ 0.732276] Key type fscrypt-provisioning registered1676worker2 # [ 0.755099] PM: Magic number: 10:825:4831677client # [ 0.733192] ima: No TPM chip found, activating TPM-bypass!1678lb # [ 0.724095] registered taskstats version 11679worker2 # [ 0.756667] RAS: Correctable Errors collector initialized.1680lb # [ 0.725130] Loading compiled-in X.509 certificates1681worker1 # [ 0.726563] registered taskstats version 11682worker1 # [ 0.727553] Loading compiled-in X.509 certificates1683client # [ 0.737722] ima: Allocated hash algorithm: sha11684client # [ 0.738504] ima: No architecture policies found1685client # [ 0.739450] PM: Magic number: 10:825:4831686client # [ 0.743050] RAS: Correctable Errors collector initialized.1687worker2 # [ 0.765734] clk: Disabling unused clocks1688worker2 # [ 0.768092] PM: genpd: Disabling unused power domains1689client # [ 0.751676] clk: Disabling unused clocks1690lb # [ 0.742904] Demotion targets for Node 0: null1691client # [ 0.753737] PM: genpd: Disabling unused power domains1692lb # [ 0.743798] Key type .fscrypt registered1693worker1 # [ 0.746320] Demotion targets for Node 0: null1694lb # [ 0.746894] Key type fscrypt-provisioning registered1695worker1 # [ 0.747328] Key type .fscrypt registered1696worker1 # [ 0.748067] Key type fscrypt-provisioning registered1697lb # [ 0.747811] ima: No TPM chip found, activating TPM-bypass!1698lb # [ 0.748688] ima: Allocated hash algorithm: sha11699worker1 # [ 0.749043] ima: No TPM chip found, activating TPM-bypass!1700lb # [ 0.749495] ima: No architecture policies found1701lb # [ 0.751898] PM: Magic number: 10:825:4831702worker1 # [ 0.752329] ima: Allocated hash algorithm: sha11703worker1 # [ 0.753111] ima: No architecture policies found1704lb # [ 0.753557] RAS: Correctable Errors collector initialized.1705worker1 # [ 0.757316] PM: Magic number: 10:825:4831706worker1 # [ 0.759052] RAS: Correctable Errors collector initialized.1707lb # [ 0.763654] clk: Disabling unused clocks1708lb # [ 0.765902] PM: genpd: Disabling unused power domains1709worker1 # [ 0.768078] clk: Disabling unused clocks1710worker1 # [ 0.772328] PM: genpd: Disabling unused power domains1711worker2 # [ 0.923333] Freeing initrd memory: 29092K1712worker2 # [ 0.926392] Freeing unused decrypted memory: 2028K1713worker2 # [ 0.928865] Freeing unused kernel image (initmem) memory: 3644K1714worker2 # [ 0.929902] Write protecting the kernel read-only data: 32768k1715worker2 # [ 0.931704] Freeing unused kernel image (text/rodata gap) memory: 1208K1716worker2 # [ 0.933166] Freeing unused kernel image (rodata/data gap) memory: 752K1717client # [ 0.913071] Freeing initrd memory: 29096K1718client # [ 0.916171] Freeing unused decrypted memory: 2028K1719client # [ 0.918669] Freeing unused kernel image (initmem) memory: 3644K1720client # [ 0.919786] Write protecting the kernel read-only data: 32768k1721client # [ 0.921599] Freeing unused kernel image (text/rodata gap) memory: 1208K1722client # [ 0.923101] Freeing unused kernel image (rodata/data gap) memory: 752K1723worker1 # [ 0.932491] Freeing initrd memory: 29092K1724worker1 # [ 0.935828] Freeing unused decrypted memory: 2028K1725lb # [ 0.937019] Freeing initrd memory: 29092K1726worker1 # [ 0.938706] Freeing unused kernel image (initmem) memory: 3644K1727worker1 # [ 0.939801] Write protecting the kernel read-only data: 32768k1728lb # [ 0.940165] Freeing unused decrypted memory: 2028K1729worker1 # [ 0.941776] Freeing unused kernel image (text/rodata gap) memory: 1208K1730lb # [ 0.942767] Freeing unused kernel image (initmem) memory: 3644K1731worker1 # [ 0.943379] Freeing unused kernel image (rodata/data gap) memory: 752K1732lb # [ 0.943853] Write protecting the kernel read-only data: 32768k1733lb # [ 0.945746] Freeing unused kernel image (text/rodata gap) memory: 1208K1734lb # [ 0.947329] Freeing unused kernel image (rodata/data gap) memory: 752K1735worker2 # [ 0.984212] x86/mm: Checked W+X mappings: passed, no W+X pages found.1736worker2 # [ 0.985260] Run /init as init process1737worker2 # [ 0.995207] systemd[1]: Inserted module 'autofs4'1738client # [ 0.974219] x86/mm: Checked W+X mappings: passed, no W+X pages found.1739client # [ 0.975308] Run /init as init process1740client # [ 0.985285] systemd[1]: Inserted module 'autofs4'1741worker2 # [ 1.011398] fuse: init (API version 7.45)1742worker2 # [ 1.017112] ACPI: \_SB_.GSIG: Enabled at IRQ 221743worker2 # [ 1.019711] ACPI: \_SB_.GSIH: Enabled at IRQ 231744worker2 # [ 1.023422] ACPI: \_SB_.GSIE: Enabled at IRQ 201745client # [ 1.001451] fuse: init (API version 7.45)1746worker2 # [ 1.025829] ACPI: \_SB_.GSIF: Enabled at IRQ 211747worker1 # [ 0.994648] x86/mm: Checked W+X mappings: passed, no W+X pages found.1748worker1 # [ 0.995737] Run /init as init process1749client # [ 1.007240] ACPI: \_SB_.GSIG: Enabled at IRQ 221750lb # [ 0.998595] x86/mm: Checked W+X mappings: passed, no W+X pages found.1751client # [ 1.009828] ACPI: \_SB_.GSIH: Enabled at IRQ 231752lb # [ 0.999745] Run /init as init process1753client # [ 1.012947] ACPI: \_SB_.GSIE: Enabled at IRQ 201754client # [ 1.015259] ACPI: \_SB_.GSIF: Enabled at IRQ 211755worker1 # [ 1.006181] systemd[1]: Inserted module 'autofs4'1756lb # [ 1.010157] systemd[1]: Inserted module 'autofs4'1757worker1 # [ 1.023133] fuse: init (API version 7.45)1758worker2 # [ 1.056838] systemd[1]: Successfully made /usr/ read-only.1759lb # [ 1.026629] fuse: init (API version 7.45)1760worker1 # [ 1.029143] ACPI: \_SB_.GSIG: Enabled at IRQ 221761worker1 # [ 1.031757] ACPI: \_SB_.GSIH: Enabled at IRQ 231762lb # [ 1.032451] ACPI: \_SB_.GSIG: Enabled at IRQ 221763client # [ 1.043878] systemd[1]: Successfully made /usr/ read-only.1764lb # [ 1.035049] ACPI: \_SB_.GSIH: Enabled at IRQ 231765worker1 # [ 1.035265] ACPI: \_SB_.GSIE: Enabled at IRQ 201766worker1 # [ 1.037922] ACPI: \_SB_.GSIF: Enabled at IRQ 211767lb # [ 1.038150] ACPI: \_SB_.GSIE: Enabled at IRQ 201768lb # [ 1.040348] ACPI: \_SB_.GSIF: Enabled at IRQ 211769worker1 # [ 1.070251] systemd[1]: Successfully made /usr/ read-only.1770lb # [ 1.073053] systemd[1]: Successfully made /usr/ read-only.1771worker2 # [ 1.392469] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1772worker2 # [ 1.398009] systemd[1]: Detected virtualization kvm.1773worker2 # [ 1.398972] systemd[1]: Detected architecture x86-64.1774worker2 # [ 1.399942] systemd[1]: Running in initrd.1775worker2 # [ 1.401217] systemd[1]: Initializing machine ID from random generator.1776worker2 # [ 1.402522] systemd[1]: Hostname set to <worker2>.1777client # [ 1.380176] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1778client # [ 1.389809] systemd[1]: Detected virtualization kvm.1779client # [ 1.391461] systemd[1]: Detected architecture x86-64.1780client # [ 1.393159] systemd[1]: Running in initrd.1781client # [ 1.395308] systemd[1]: Initializing machine ID from random generator.1782client # [ 1.397601] systemd[1]: Hostname set to <client>.1783worker1 # [ 1.406821] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1784worker1 # [ 1.418728] systemd[1]: Detected virtualization kvm.1785lb # [ 1.409658] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1786worker1 # [ 1.420827] systemd[1]: Detected architecture x86-64.1787lb # [ 1.421675] systemd[1]: Detected virtualization kvm.1788worker1 # [ 1.422967] systemd[1]: Running in initrd.1789lb # [ 1.423855] systemd[1]: Detected architecture x86-64.1790lb # [ 1.426023] systemd[1]: Running in initrd.1791worker1 # [ 1.425546] systemd[1]: Initializing machine ID from random generator.1792worker1 # [ 1.428588] systemd[1]: Hostname set to <worker1>.1793lb # [ 1.428642] systemd[1]: Initializing machine ID from random generator.1794lb # [ 1.431538] systemd[1]: Hostname set to <lb>.1795worker2 # [ 1.596248] systemd[1]: bpf-restrict-fs: LSM BPF program attached1796worker2 # [ 1.634796] systemd[1]: Queued start job for default target Initrd Default Target.1797worker2 # [ 1.639416] systemd[1]: Created slice Slice /system/modprobe.1798worker2 # [ 1.640602] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1799client # [ 1.617805] systemd[1]: bpf-restrict-fs: LSM BPF program attached1800worker2 # [ 1.642013] systemd[1]: Expecting device /dev/disk/by-label/nixos...1801worker2 # [ 1.643184] systemd[1]: Reached target Path Units.1802worker2 # [ 1.644116] systemd[1]: Reached target Slice Units.1803worker2 # [ 1.645009] systemd[1]: Reached target Swaps.1804worker2 # [ 1.645825] systemd[1]: Reached target Timer Units.1805worker2 # [ 1.646834] systemd[1]: Listening on D-Bus System Message Bus Socket.1806worker2 # [ 1.648159] systemd[1]: Listening on Journal Socket (/dev/log).1807worker2 # [ 1.649328] systemd[1]: Listening on Journal Sockets.1808worker2 # [ 1.650376] systemd[1]: Listening on udev Control Socket.1809worker2 # [ 1.651453] systemd[1]: Listening on udev Kernel Socket.1810worker2 # [ 1.652457] systemd[1]: Reached target Socket Units.1811worker2 # [ 1.654305] systemd[1]: Starting Create List of Static Device Nodes...1812worker2 # [ 1.658189] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1813worker2 # [ 1.665160] systemd[1]: Starting Load Kernel Module configfs...1814worker2 # [ 1.679262] systemd[1]: Starting Journal Service...1815worker1 # [ 1.651069] systemd[1]: bpf-restrict-fs: LSM BPF program attached1816client # [ 1.667070] systemd[1]: Queued start job for default target Initrd Default Target.1817worker2 # [ 1.691191] systemd[1]: Starting Load Kernel Modules...1818client # [ 1.671142] systemd[1]: Created slice Slice /system/modprobe.1819lb # [ 1.660307] systemd[1]: bpf-restrict-fs: LSM BPF program attached1820client # [ 1.672294] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1821client # [ 1.673633] systemd[1]: Expecting device /dev/disk/by-label/nixos...1822client # [ 1.674728] systemd[1]: Reached target Path Units.1823client # [ 1.675555] systemd[1]: Reached target Slice Units.1824client # [ 1.676410] systemd[1]: Reached target Swaps.1825client # [ 1.677190] systemd[1]: Reached target Timer Units.1826client # [ 1.678194] systemd[1]: Listening on D-Bus System Message Bus Socket.1827client # [ 1.679375] systemd[1]: Listening on Journal Socket (/dev/log).1828client # [ 1.680499] systemd[1]: Listening on Journal Sockets.1829client # [ 1.681475] systemd[1]: Listening on udev Control Socket.1830client # [ 1.682518] systemd[1]: Listening on udev Kernel Socket.1831client # [ 1.683445] systemd[1]: Reached target Socket Units.1832client # [ 1.685276] systemd[1]: Starting Create List of Static Device Nodes...1833worker2 # [ 1.696158] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1834client # [ 1.689790] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1835worker2 # [ 1.714200] netfs: FS-Cache loaded1836client # [ 1.696417] systemd[1]: Starting Load Kernel Module configfs...1837worker2 # [ 1.718938] 9pnet: Installing 9P2000 support1838worker2 # [ 1.728329] systemd[1]: Starting Coldplug All udev Devices...1839client # [ 1.712029] systemd[1]: Starting Journal Service...1840worker1 # [ 1.704774] systemd[1]: Queued start job for default target Initrd Default Target.1841worker1 # [ 1.708619] systemd[1]: Created slice Slice /system/modprobe.1842worker2 # [ 1.742128] systemd[1]: Finished Create List of Static Device Nodes.1843worker1 # [ 1.709830] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1844worker1 # [ 1.711244] systemd[1]: Expecting device /dev/disk/by-label/nixos...1845worker1 # [ 1.712451] systemd[1]: Reached target Path Units.1846lb # [ 1.712365] systemd[1]: Queued start job for default target Initrd Default Target.1847worker1 # [ 1.713303] systemd[1]: Reached target Slice Units.1848worker1 # [ 1.714170] systemd[1]: Reached target Swaps.1849worker1 # [ 1.715033] systemd[1]: Reached target Timer Units.1850client # [ 1.725593] systemd[1]: Starting Load Kernel Modules...1851worker1 # [ 1.716047] systemd[1]: Listening on D-Bus System Message Bus Socket.1852lb # [ 1.716257] systemd[1]: Created slice Slice /system/modprobe.1853worker2 # [ 1.748658] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1854worker1 # [ 1.717246] systemd[1]: Listening on Journal Socket (/dev/log).1855lb # [ 1.717444] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1856worker1 # [ 1.718434] systemd[1]: Listening on Journal Sockets.1857lb # [ 1.718824] systemd[1]: Expecting device /dev/disk/by-label/nixos...1858worker1 # [ 1.719446] systemd[1]: Listening on udev Control Socket.1859lb # [ 1.719926] systemd[1]: Reached target Path Units.1860worker1 # [ 1.720454] systemd[1]: Listening on udev Kernel Socket.1861lb # [ 1.720774] systemd[1]: Reached target Slice Units.1862worker1 # [ 1.721398] systemd[1]: Reached target Socket Units.1863lb # [ 1.721605] systemd[1]: Reached target Swaps.1864lb # [ 1.722393] systemd[1]: Reached target Timer Units.1865worker1 # [ 1.723153] systemd[1]: Starting Create List of Static Device Nodes...1866lb # [ 1.723387] systemd[1]: Listening on D-Bus System Message Bus Socket.1867worker2 # [ 1.756450] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1868lb # [ 1.724620] systemd[1]: Listening on Journal Socket (/dev/log).1869lb # [ 1.725803] systemd[1]: Listening on Journal Sockets.1870lb # [ 1.726888] systemd[1]: Listening on udev Control Socket.1871worker1 # [ 1.727132] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1872lb # [ 1.727919] systemd[1]: Listening on udev Kernel Socket.1873lb # [ 1.728903] systemd[1]: Reached target Socket Units.1874worker2 # [ 1.761492] systemd[1]: modprobe@configfs.service: Deactivated successfully.1875lb # [ 1.730763] systemd[1]: Starting Create List of Static Device Nodes...1876client # [ 1.730802] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1877worker2 # [ 1.764787] systemd-journald[67]: Collecting audit messages is disabled.1878lb # [ 1.734754] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1879worker2 # [ 1.767453] systemd[1]: Finished Load Kernel Module configfs.1880worker1 # [ 1.735366] systemd[1]: Starting Load Kernel Module configfs...1881client # [ 1.748547] netfs: FS-Cache loaded1882lb # [ 1.740985] systemd[1]: Starting Load Kernel Module configfs...1883worker2 # [ 1.774520] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1884client # [ 1.755983] 9pnet: Installing 9P2000 support1885worker2 # [ 1.777443] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1886worker1 # [ 1.747971] systemd[1]: Starting Journal Service...1887worker2 # [ 1.785146] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1888lb # [ 1.755236] systemd[1]: Starting Journal Service...1889worker2 # [ 1.788130] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1890client # [ 1.767964] systemd[1]: Starting Coldplug All udev Devices...1891worker1 # [ 1.763385] systemd[1]: Starting Load Kernel Modules...1892lb # [ 1.764955] systemd[1]: Starting Load Kernel Modules...1893client # [ 1.779259] systemd-journald[67]: Collecting audit messages is disabled.1894client # [ 1.782800] systemd[1]: Finished Create List of Static Device Nodes.1895client # [ 1.787499] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1896worker1 # [ 1.767380] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1897lb # [ 1.771970] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1898worker1 # [ 1.784895] netfs: FS-Cache loaded1899worker2 # [ 1.817549] systemd[1]: Finished Load Kernel Modules.1900client # [ 1.796129] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1901worker1 # [ 1.789700] 9pnet: Installing 9P2000 support1902client # [ 1.802123] systemd[1]: modprobe@configfs.service: Deactivated successfully.1903lb # [ 1.792184] netfs: FS-Cache loaded1904worker2 # [ 1.824947] systemd[1]: Starting Apply Kernel Variables...1905lb # [ 1.796988] 9pnet: Installing 9P2000 support1906client # [ 1.809088] systemd[1]: Finished Load Kernel Module configfs.1907worker1 # [ 1.801425] systemd[1]: Starting Coldplug All udev Devices...1908worker2 # [ 1.835190] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1909client # [ 1.813160] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1910client # [ 1.816835] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1911lb # [ 1.807990] systemd[1]: Starting Coldplug All udev Devices...1912worker2 # [ 1.844547] systemd[1]: Starting Create Static Device Nodes in /dev...1913client # [ 1.823738] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1914client # [ 1.826955] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1915worker1 # [ 1.820573] systemd[1]: Finished Create List of Static Device Nodes.1916lb # [ 1.824963] systemd[1]: Finished Create List of Static Device Nodes.1917worker1 # [ 1.827417] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1918lb # [ 1.830488] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1919worker2 # [ 1.864450] systemd[1]: Finished Apply Kernel Variables.1920worker1 # [ 1.835386] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1921worker1 # [ 1.836740] systemd-journald[67]: Collecting audit messages is disabled.1922lb # [ 1.837317] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1923worker1 # [ 1.839704] systemd[1]: modprobe@configfs.service: Deactivated successfully.1924lb # [ 1.841594] systemd-journald[67]: Collecting audit messages is disabled.1925lb # [ 1.843195] systemd[1]: modprobe@configfs.service: Deactivated successfully.1926client # [ 1.855248] systemd[1]: Finished Load Kernel Modules.1927worker1 # [ 1.846743] systemd[1]: Finished Load Kernel Module configfs.1928worker2 # [ 1.879606] systemd[1]: Finished Create Static Device Nodes in /dev.1929worker2 # [ 1.744084] systemd-modules-load[68]: Inserted module 'dm_mod'1930worker2 # [ 1.746390] systemd-modules-load[68]: Inserted module 'virtio_balloon'1931worker1 # [ 1.849870] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1932lb # [ 1.851060] systemd[1]: Finished Load Kernel Module configfs.1933worker2 # [ 1.884509] systemd[1]: Started Journal Service.1934client # [ 1.862923] systemd[1]: Starting Apply Kernel Variables...1935worker2 # [ 1.748557] systemd-modules-load[68]: Inserted module 'virtio_gpu'1936worker1 # [ 1.854782] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1937lb # [ 1.855384] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1938worker1 # [ 1.861495] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1939lb # [ 1.861279] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1940worker2 # [ 1.759045] systemd[1]: Reached target Preparation for Local File Systems.1941client # [ 1.876193] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1942worker2 # [ 1.761974] systemd[1]: Reached target Local File Systems.1943worker1 # [ 1.866388] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1944lb # [ 1.866737] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1945worker2 # [ 1.764879] systemd[1]: Starting Create System Files and Directories...1946lb # [ 1.869919] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1947client # [ 1.886925] systemd[1]: Starting Create Static Device Nodes in /dev...1948worker2 # [ 1.772064] systemd[1]: Starting Rule-based Manager for Device Events and Files...1949client # [ 1.899822] systemd[1]: Finished Apply Kernel Variables.1950client # [ 1.765529] systemd-modules-load[68]: Inserted module 'dm_mod'1951worker1 # [ 1.892850] systemd[1]: Finished Load Kernel Modules.1952client # [ 1.768406] systemd-modules-load[68]: Inserted module 'virtio_balloon'1953client # [ 1.769453] systemd-modules-load[68]: Inserted module 'virtio_gpu'1954client # [ 1.907396] systemd[1]: Started Journal Service.1955lb # [ 1.899394] systemd[1]: Finished Load Kernel Modules.1956worker1 # [ 1.901492] systemd[1]: Starting Apply Kernel Variables...1957worker2 # [ 1.797631] systemd[1]: Finished Create System Files and Directories.1958lb # [ 1.907481] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1959client # [ 1.789136] systemd[1]: Finished Create Static Device Nodes in /dev.1960lb # [ 1.915092] systemd[1]: Starting Apply Kernel Variables...1961worker1 # [ 1.915391] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1962client # [ 1.790470] systemd[1]: Reached target Preparation for Local File Systems.1963client # [ 1.792739] systemd[1]: Reached target Local File Systems.1964client # [ 1.794339] systemd[1]: Starting Create System Files and Directories...1965worker1 # [ 1.924486] systemd[1]: Starting Create Static Device Nodes in /dev...1966worker2 # [ 1.820213] systemd-udevd[84]: Using default interface naming scheme 'v261'.1967lb # [ 1.924957] systemd[1]: Starting Create Static Device Nodes in /dev...1968client # [ 1.802141] systemd[1]: Starting Rule-based Manager for Device Events and Files...1969worker1 # [ 1.937419] systemd[1]: Finished Apply Kernel Variables.1970worker2 # [ 1.845122] systemd[1]: Started Rule-based Manager for Device Events and Files.1971worker1 # [ 1.953028] systemd[1]: Started Journal Service.1972lb # [ 1.955420] systemd[1]: Finished Apply Kernel Variables.1973client # [ 1.830517] systemd[1]: Finished Create System Files and Directories.1974worker1 # [ 1.821208] systemd-modules-load[68]: Inserted module 'dm_mod'1975worker1 # [ 1.824241] systemd-modules-load[68]: Inserted module 'virtio_balloon'1976worker1 # [ 1.825285] systemd-modules-load[68]: Inserted module 'virtio_gpu'1977lb # [ 1.960123] systemd[1]: Started Journal Service.1978lb # [ 1.824731] systemd-modules-load[68]: Inserted module 'dm_mod'1979lb # [ 1.828965] systemd-modules-load[68]: Inserted module 'virtio_balloon'1980lb # [ 1.829993] systemd-modules-load[68]: Inserted module 'virtio_gpu'1981worker1 # [ 1.831879] systemd[1]: Finished Create Static Device Nodes in /dev.1982lb # [ 1.833898] systemd[1]: Finished Create Static Device Nodes in /dev.1983worker1 # [ 1.836879] systemd[1]: Reached target Preparation for Local File Systems.1984worker1 # [ 1.838073] systemd[1]: Reached target Local File Systems.1985worker1 # [ 1.841168] systemd[1]: Starting Create System Files and Directories...1986lb # [ 1.841729] systemd[1]: Reached target Preparation for Local File Systems.1987client # [ 1.850284] systemd-udevd[83]: Using default interface naming scheme 'v261'.1988lb # [ 1.842796] systemd[1]: Reached target Local File Systems.1989lb # [ 1.845367] systemd[1]: Starting Create System Files and Directories...1990worker1 # [ 1.845905] systemd[1]: Starting Rule-based Manager for Device Events and Files...1991lb # [ 1.850558] systemd[1]: Starting Rule-based Manager for Device Events and Files...1992worker2 # [ 1.890795] systemd[1]: Finished Coldplug All udev Devices.1993worker2 # [ 1.891735] systemd[1]: Reached target System Initialization.1994worker2 # [ 1.892620] systemd[1]: Reached target Basic System.1995client # [ 1.873814] systemd[1]: Started Rule-based Manager for Device Events and Files.1996worker1 # [ 1.871434] systemd[1]: Finished Create System Files and Directories.1997lb # [ 1.876198] systemd[1]: Finished Create System Files and Directories.1998worker1 # [ 1.895250] systemd-udevd[83]: Using default interface naming scheme 'v261'.1999lb # [ 1.898235] systemd-udevd[83]: Using default interface naming scheme 'v261'.2000worker1 # [ 1.919879] systemd[1]: Started Rule-based Manager for Device Events and Files.
loading 1,902 hidden lines…
3903worker2 # [ 11.684197] 8021q: adding VLAN 0 to HW filter on device eth03904worker2 # [ 11.548380] dhcpcd[749]: eth0: waiting for carrier3905worker2 # [ 11.549241] dhcpcd[749]: eth0: carrier acquired3906worker2 # [ 11.556648] dhcpcd[749]: DUID 00:01:00:01:32:42:9b:d7:52:54:00:12:34:563907worker2 # [ 11.557753] dhcpcd[749]: eth0: IAID 00:12:34:563908worker2 # [ 11.558578] dhcpcd[749]: eth0: adding address fe80::5054:ff:fe12:34563909client # [ 11.728058] dhcpcd[569]: eth0: soliciting an IPv6 router3910client # [ 11.729391] dhcpcd[569]: eth0: Router Advertisement from fe80::23911client # [ 11.730212] dhcpcd[569]: eth0: adding address fec0::5054:ff:fe12:3456/643912client # [ 11.731056] dhcpcd[569]: eth0: adding route to fec0::/643913client # [ 11.731740] dhcpcd[569]: eth0: adding default route via fe80::23914worker1 # [ 11.999353] dhcpcd[749]: eth0: soliciting an IPv6 router3915worker1 # [ 12.000634] dhcpcd[749]: eth0: Router Advertisement from fe80::23916worker1 # [ 12.001501] dhcpcd[749]: eth0: adding address fec0::5054:ff:fe12:3456/643917worker1 # [ 12.002388] dhcpcd[749]: eth0: adding route to fec0::/643918worker1 # [ 12.003222] dhcpcd[749]: eth0: adding default route via fe80::23919worker2 # [ 12.303543] dhcpcd[749]: eth0: soliciting a DHCP lease3920worker2 # [ 12.467837] NET: Registered PF_PACKET protocol family3921worker2 # [ 12.338669] dhcpcd[749]: eth0: offered 10.0.2.15 from 10.0.2.23922worker2 # [ 12.342355] dhcpcd[749]: eth0: probing address 10.0.2.15/243923lb # [ 12.704902] dhcpcd[712]: eth0: soliciting a DHCP lease3924lb # [ 12.853359] NET: Registered PF_PACKET protocol family3925lb # [ 12.723450] dhcpcd[712]: eth0: offered 10.0.2.15 from 10.0.2.23926lb # [ 12.726162] dhcpcd[712]: eth0: probing address 10.0.2.15/243927lb # [ 13.133135] postgresql-pre-start[730]: selecting default time zone ... UTC3928lb # [ 13.137437] postgresql-pre-start[730]: creating configuration files ... ok3929lb # [ 13.216878] dhcpcd[712]: eth0: soliciting an IPv6 router3930lb # [ 13.218248] dhcpcd[712]: eth0: Router Advertisement from fe80::23931lb # [ 13.219288] dhcpcd[712]: eth0: adding address fec0::5054:ff:fe12:3456/643932lb # [ 13.220363] dhcpcd[712]: eth0: adding route to fec0::/643933lb # [ 13.221315] dhcpcd[712]: eth0: adding default route via fe80::23934lb # [ 13.377433] postgresql-pre-start[730]: running bootstrap script ... ok3935lb # [ 13.955127] postgresql-pre-start[730]: performing post-bootstrap initialization ... ok3936worker2 # [ 14.027803] dhcpcd[749]: eth0: soliciting an IPv6 router3937worker2 # [ 14.029203] dhcpcd[749]: eth0: Router Advertisement from fe80::23938worker2 # [ 14.029992] dhcpcd[749]: eth0: adding address fec0::5054:ff:fe12:3456/643939worker2 # [ 14.030841] dhcpcd[749]: eth0: adding route to fec0::/643940worker2 # [ 14.031631] dhcpcd[749]: eth0: adding default route via fe80::23941lb # [ 14.173874] postgresql-pre-start[730]: syncing data to disk ... ok3942lb # [ 14.174761] postgresql-pre-start[730]: initdb: warning: enabling "trust" authentication for local connections3943lb # [ 14.175964] postgresql-pre-start[730]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.3944lb # [ 14.177851] postgresql-pre-start[730]: Success. You can now start the database server using:3945lb # [ 14.178914] postgresql-pre-start[730]: pg_ctl -D /var/lib/postgresql/18 -l logfile start3946lb # [ 14.292815] postgres[803]: [803] LOG: starting PostgreSQL 18.6 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit3947lb # [ 14.296928] postgres[803]: [803] LOG: listening on IPv6 address "::1", port 54323948lb # [ 14.298019] postgres[803]: [803] LOG: listening on IPv4 address "127.0.0.1", port 54323949lb # [ 14.300477] postgres[803]: [803] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"3950lb # [ 14.313312] postgres[812]: [812] LOG: database system was shut down at 2026-09-20 13:27:53 GMT3951lb # [ 14.318375] postgres[803]: [803] LOG: database system is ready to accept connections3952lb # [ 14.325284] systemd[1]: Started PostgreSQL Server.3953lb # [ 14.330606] systemd[1]: Starting PostgreSQL Setup Scripts...3954lb # [ 14.512787] postgresql-setup-start[823]: CREATE DATABASE3955lb # [ 14.553859] postgresql-setup-start[828]: CREATE ROLE3956lb # [ 14.572684] postgresql-setup-start[830]: ALTER DATABASE3957lb # [ 14.578199] systemd[1]: Finished PostgreSQL Setup Scripts.3958lb # [ 14.580220] systemd[1]: Reached target PostgreSQL.3959worker1 # [ 15.522285] dhcpcd[749]: eth0: leased 10.0.2.15 for 86400 seconds3960worker1 # [ 15.524669] dhcpcd[749]: eth0: adding route to 10.0.2.0/243961worker1 # [ 15.526713] dhcpcd[749]: eth0: adding default route via 10.0.2.23962worker1 # [ 15.654934] systemd[1]: Started DHCP Client.3963worker1 # [ 15.657080] systemd[1]: Reached target Multi-User System.3964worker1 # [ 15.658151] systemd[1]: Startup finished in 864ms (kernel) + 4.178s (initrd) + 10.612s (userspace) = 15.656s.3965client # [ 15.679932] dhcpcd[569]: eth0: leased 10.0.2.15 for 86400 seconds3966client # [ 15.683550] dhcpcd[569]: eth0: adding route to 10.0.2.0/243967client # [ 15.686267] dhcpcd[569]: eth0: adding default route via 10.0.2.23968client # [ 15.800803] systemd[1]: Started DHCP Client.3969client # [ 15.803420] systemd[1]: Reached target Multi-User System.3970client # [ 15.804456] systemd[1]: Startup finished in 842ms (kernel) + 4.094s (initrd) + 10.866s (userspace) = 15.804s.3971worker2 # [ 17.720306] dhcpcd[749]: eth0: leased 10.0.2.15 for 86400 seconds3972worker2 # [ 17.722808] dhcpcd[749]: eth0: adding route to 10.0.2.0/243973worker2 # [ 17.725211] dhcpcd[749]: eth0: adding default route via 10.0.2.23974worker2 # [ 17.849953] systemd[1]: Started DHCP Client.3975worker2 # [ 17.852070] systemd[1]: Reached target Multi-User System.3976worker2 # [ 17.852798] systemd[1]: Startup finished in 851ms (kernel) + 5.060s (initrd) + 11.940s (userspace) = 17.851s.3977lb # [ 17.944537] dhcpcd[712]: eth0: leased 10.0.2.15 for 86400 seconds3978lb # [ 17.947256] dhcpcd[712]: eth0: adding route to 10.0.2.0/243979lb # [ 17.949679] dhcpcd[712]: eth0: adding default route via 10.0.2.23980lb # [ 18.103315] systemd[1]: Started DHCP Client.3981lb # [ 18.105083] systemd[1]: Reached target Network is Online.3982lb # [ 18.109295] systemd[1]: Started Envoy reverse proxy.3983lb # [ 18.112681] systemd[1]: Started mock-oidc.service.3984lb # [ 18.209535] mock-oidc-server[931]: Mock OIDC Server running3985lb # [ 18.210646] mock-oidc-server[931]: OIDC Address: 192.168.1.2:80803986lb # [ 18.211803] mock-oidc-server[931]: Issue Address: 0.0.0.0:80813987lb # [ 18.212727] mock-oidc-server[931]: Issuer: http://192.168.1.2:8080/oidc3988lb # [ 18.213729] mock-oidc-server[931]: JWKS: http://192.168.1.2:8080/oidc/.well-known/jwks.json3989lb # [ 18.214948] mock-oidc-server[931]: Discovery: http://192.168.1.2:8080/oidc/.well-known/openid-configuration3990lb # [ 18.216302] mock-oidc-server[931]: Issue tokens: http://0.0.0.0:8081/issue?sub=...3991lb # [ 18.648903] envoy[930]: [2026-09-20 13:27:58.672][930][info][main] [source/server/server.cc:467] initializing epoch 0 (base id=0, hot restart version=11.104)3992lb # [ 18.650642] envoy[930]: [2026-09-20 13:27:58.672][930][info][main] [source/server/server.cc:469] statically linked extensions:3993lb # [ 18.652199] envoy[930]: [2026-09-20 13:27:58.672][930][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor3994lb # [ 18.654887] envoy[930]: [2026-09-20 13:27:58.672][930][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header3995lb # [ 18.657603] envoy[930]: [2026-09-20 13:27:58.672][930][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts3996lb # [ 18.660535] envoy[930]: [2026-09-20 13:27:58.672][930][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state3997lb # [ 18.663317] envoy[930]: [2026-09-20 13:27:58.672][930][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based3998lb # [ 18.665071] envoy[930]: [2026-09-20 13:27:58.672][930][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router3999lb # [ 18.666638] envoy[930]: [2026-09-20 13:27:58.672][930][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian24000lb # [ 18.668130] envoy[930]: [2026-09-20 13:27:58.672][930][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json4001lb # [ 18.669611] envoy[930]: [2026-09-20 13:27:58.672][930][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4002lb # [ 18.671828] envoy[930]: [2026-09-20 13:27:58.672][930][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher4003lb # [ 18.674056] envoy[930]: [2026-09-20 13:27:58.672][930][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation4004lb # [ 18.675876] envoy[930]: [2026-09-20 13:27:58.672][930][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain4005lb # [ 18.677820] envoy[930]: [2026-09-20 13:27:58.672][930][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params4006lb # [ 18.695653] envoy[930]: [2026-09-20 13:27:58.672][930][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port4007lb # [ 18.697570] envoy[930]: [2026-09-20 13:27:58.676][930][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default4008lb # [ 18.699334] envoy[930]: [2026-09-20 13:27:58.676][930][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default4009lb # [ 18.701112] envoy[930]: [2026-09-20 13:27:58.676][930][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log4010lb # [ 18.707184] envoy[930]: [2026-09-20 13:27:58.676][930][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection4011lb # [ 18.708950] envoy[930]: [2026-09-20 13:27:58.676][930][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit4012lb # [ 18.711749] envoy[930]: [2026-09-20 13:27:58.676][930][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input4013lb # [ 18.713521] envoy[930]: [2026-09-20 13:27:58.676][930][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple4014lb # [ 18.715534] envoy[930]: [2026-09-20 13:27:58.679][930][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4015lb # [ 18.717403] envoy[930]: [2026-09-20 13:27:58.679][930][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default4016lb # [ 18.719134] envoy[930]: [2026-09-20 13:27:58.679][930][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service4017lb # [ 18.723046] envoy[930]: [2026-09-20 13:27:58.679][930][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default4018lb # [ 18.724793] envoy[930]: [2026-09-20 13:27:58.679][930][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter4019lb # [ 18.727090] envoy[930]: [2026-09-20 13:27:58.679][930][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule4020lb # [ 18.729546] envoy[930]: [2026-09-20 13:27:58.679][930][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr4021lb # [ 18.731202] envoy[930]: [2026-09-20 13:27:58.679][930][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file4022lb # [ 18.732872] envoy[930]: [2026-09-20 13:27:58.679][930][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based4023lb # [ 18.736474] envoy[930]: [2026-09-20 13:27:58.679][930][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable4024lb # [ 18.738283] envoy[930]: [2026-09-20 13:27:58.679][930][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator4025lb # [ 18.740488] envoy[930]: [2026-09-20 13:27:58.679][930][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid4026lb # [ 18.743122] envoy[930]: [2026-09-20 13:27:58.679][930][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple4027lb # [ 18.746088] envoy[930]: [2026-09-20 13:27:58.679][930][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder4028lb # [ 18.748164] envoy[930]: [2026-09-20 13:27:58.679][930][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default4029lb # [ 18.749806] envoy[930]: [2026-09-20 13:27:58.681][930][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query4030lb # [ 18.752563] envoy[930]: [2026-09-20 13:27:58.681][930][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction4031lb # [ 18.756533] envoy[930]: [2026-09-20 13:27:58.681][930][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config4032lb # [ 18.759690] envoy[930]: [2026-09-20 13:27:58.681][930][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe4033lb # [ 18.762429] envoy[930]: [2026-09-20 13:27:58.681][930][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response4034lb # [ 18.764457] envoy[930]: [2026-09-20 13:27:58.681][930][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme4035lb # [ 18.767284] envoy[930]: [2026-09-20 13:27:58.681][930][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated4036lb # [ 18.768958] envoy[930]: [2026-09-20 13:27:58.681][930][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input4037lb # [ 18.770653] envoy[930]: [2026-09-20 13:27:58.681][930][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality4038lb # [ 18.777361] envoy[930]: [2026-09-20 13:27:58.684][930][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats4039lb # [ 18.779986] envoy[930]: [2026-09-20 13:27:58.684][930][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port4040lb # [ 18.787987] envoy[930]: [2026-09-20 13:27:58.684][930][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor4041lb # [ 18.790585] envoy[930]: [2026-09-20 13:27:58.684][930][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection4042lb # [ 18.795281] envoy[930]: [2026-09-20 13:27:58.684][930][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata4043lb # [ 18.797342] envoy[930]: [2026-09-20 13:27:58.684][930][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy4044lb # [ 18.799825] envoy[930]: [2026-09-20 13:27:58.684][930][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest4045lb # [ 18.804461] envoy[930]: [2026-09-20 13:27:58.684][930][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4046lb # [ 18.806423] envoy[930]: [2026-09-20 13:27:58.684][930][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal4047lb # [ 18.808066] envoy[930]: [2026-09-20 13:27:58.684][930][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default4048lb # [ 18.809834] envoy[930]: [2026-09-20 13:27:58.684][930][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche4049lb # [ 18.811686] envoy[930]: [2026-09-20 13:27:58.684][930][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin4050lb # [ 18.814820] envoy[930]: [2026-09-20 13:27:58.684][930][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource4051lb # [ 18.817722] envoy[930]: [2026-09-20 13:27:58.684][930][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso4052lb # [ 18.819719] envoy[930]: [2026-09-20 13:27:58.684][930][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm4053lb # [ 18.825655] envoy[930]: [2026-09-20 13:27:58.684][930][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert4054lb # [ 18.827574] envoy[930]: [2026-09-20 13:27:58.684][930][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options4055lb # [ 18.831626] envoy[930]: [2026-09-20 13:27:58.684][930][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default4056lb # [ 18.833723] envoy[930]: [2026-09-20 13:27:58.684][930][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory4057lb # [ 18.836741] envoy[930]: [2026-09-20 13:27:58.689][930][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san4058lb # [ 18.843148] envoy[930]: [2026-09-20 13:27:58.689][930][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator4059lb # [ 18.845470] envoy[930]: [2026-09-20 13:27:58.689][930][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed4060lb # [ 18.847318] envoy[930]: [2026-09-20 13:27:58.689][930][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls4061lb # [ 18.851556] envoy[930]: [2026-09-20 13:27:58.689][930][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua4062lb # [ 18.853504] envoy[930]: [2026-09-20 13:27:58.689][930][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation4063lb # [ 18.856096] envoy[930]: [2026-09-20 13:27:58.689][930][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic4064lb # [ 18.858580] envoy[930]: [2026-09-20 13:27:58.689][930][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default4065lb # [ 18.860359] envoy[930]: [2026-09-20 13:27:58.689][930][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher4066lb # [ 18.862506] envoy[930]: [2026-09-20 13:27:58.689][930][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4067lb # [ 18.865211] envoy[930]: [2026-09-20 13:27:58.689][930][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls4068lb # [ 18.871343] envoy[930]: [2026-09-20 13:27:58.689][930][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v84069lb # [ 18.873328] envoy[930]: [2026-09-20 13:27:58.689][930][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value4070lb # [ 18.877128] envoy[930]: [2026-09-20 13:27:58.689][930][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router4071lb # [ 18.906650] envoy[930]: [2026-09-20 13:27:58.689][930][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory4072lb # [ 18.909546] envoy[930]: [2026-09-20 13:27:58.689][930][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy4073lb # [ 18.912428] envoy[930]: [2026-09-20 13:27:58.689][930][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy4074lb # [ 18.925075] envoy[930]: [2026-09-20 13:27:58.701][930][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re24075lb # [ 18.926871] envoy[930]: [2026-09-20 13:27:58.701][930][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default4076lb # [ 18.928633] envoy[930]: [2026-09-20 13:27:58.701][930][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle4077lb # [ 18.930809] envoy[930]: [2026-09-20 13:27:58.701][930][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name4078lb # [ 18.933299] envoy[930]: [2026-09-20 13:27:58.701][930][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed4079lb # [ 18.935786] envoy[930]: [2026-09-20 13:27:58.701][930][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http14080lb # [ 18.938047] envoy[930]: [2026-09-20 13:27:58.701][930][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind4081lb # [ 18.939832] envoy[930]: [2026-09-20 13:27:58.701][930][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action4082lb # [ 18.941925] envoy[930]: [2026-09-20 13:27:58.701][930][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override4083lb # [ 18.944784] envoy[930]: [2026-09-20 13:27:58.701][930][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift4084lb # [ 18.948261] envoy[930]: [2026-09-20 13:27:58.701][930][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd4085lb # [ 18.953271] envoy[930]: [2026-09-20 13:27:58.701][930][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth24086lb # [ 18.955668] envoy[930]: [2026-09-20 13:27:58.701][930][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name4087lb # [ 18.958099] envoy[930]: [2026-09-20 13:27:58.701][930][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter4088lb # [ 18.960252] envoy[930]: [2026-09-20 13:27:58.701][930][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities4089lb # [ 18.962392] envoy[930]: [2026-09-20 13:27:58.701][930][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff4090lb # [ 18.964793] envoy[930]: [2026-09-20 13:27:58.702][930][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override4091lb # [ 18.967738] envoy[930]: [2026-09-20 13:27:58.702][930][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap4092lb # [ 18.972572] envoy[930]: [2026-09-20 13:27:58.702][930][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router4093lb # [ 18.975893] envoy[930]: [2026-09-20 13:27:58.702][930][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector4094lb # [ 18.982457] envoy[930]: [2026-09-20 13:27:58.702][930][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router4095lb # [ 18.984243] envoy[930]: [2026-09-20 13:27:58.702][930][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case4096lb # [ 18.986424] envoy[930]: [2026-09-20 13:27:58.711][930][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo4097lb # [ 18.987945] envoy[930]: [2026-09-20 13:27:58.819][930][info][main] [source/server/server.cc:531] HTTP header map info:4098lb # [ 18.989288] envoy[930]: [2026-09-20 13:27:58.871][930][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id4099lb # [ 19.003274] envoy[930]: [2026-09-20 13:27:58.871][930][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:4100lb # [ 19.004751] envoy[930]: [2026-09-20 13:27:58.871][930][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id4101lb # [ 19.013061] envoy[930]: [2026-09-20 13:27:58.871][930][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status4102lb # [ 19.050158] envoy[930]: [2026-09-20 13:27:59.074][930][info][main] [source/server/server.cc:968] runtime: {}4103lb # [ 19.056245] envoy[930]: [2026-09-20 13:27:59.080][930][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.4104lb # [ 19.058657] envoy[930]: [2026-09-20 13:27:59.082][930][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99014105lb # [ 19.061540] envoy[930]: [2026-09-20 13:27:59.085][930][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99014106lb # [ 19.064434] envoy[930]: [2026-09-20 13:27:59.088][930][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration4107lb # [ 19.066203] envoy[930]: [2026-09-20 13:27:59.090][930][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)4108lb # [ 19.067867] envoy[930]: [2026-09-20 13:27:59.090][930][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)4109lb # [ 19.124155] envoy[930]: [2026-09-20 13:27:59.148][930][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)4110worker1 # [ 19.125236] systemd[1]: Starting Nix worker-protocol over gRPC...4111worker2 # [ 19.154740] systemd[1]: Reached target Network is Online.4112worker2 # [ 19.163369] systemd[1]: Starting scheduler-token.service...4113lb # [ 19.153031] envoy[930]: [2026-09-20 13:27:59.177][930][info][config] [source/server/configuration_impl.cc:164] loading stats configuration4114lb # [ 19.156077] envoy[930]: [2026-09-20 13:27:59.180][930][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.4115lb # [ 19.158918] envoy[930]: [2026-09-20 13:27:59.180][930][info][main] [source/server/server.cc:1077] starting main dispatch loop4116worker2 # [ 19.274692] systemd[1]: scheduler-token.service: Deactivated successfully.4117worker2 # [ 19.276564] systemd[1]: Finished scheduler-token.service.4118worker2 # [ 19.277608] systemd[1]: scheduler-token.service: Consumed 34ms CPU time over 113ms wall clock time, 3.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4119worker2 # [ 19.282846] systemd[1]: Starting Nix worker-protocol over gRPC...4120worker1 # [ 19.356789] nix-grpc-daemon[848]: ts=2026-09-20T13:27:58Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4121worker1 # [ 19.365988] nix-grpc-daemon[848]: ts=2026-09-20T13:27:58Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4122worker1 # [ 19.371088] nix-grpc-daemon[848]: ts=2026-09-20T13:27:58Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14123worker1 # [ 19.410208] nix-grpc-daemon[848]: ts=2026-09-20T13:27:59Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4124worker1 # [ 19.412986] systemd[1]: Started Nix worker-protocol over gRPC.4125worker2 # [ 19.449406] nix-grpc-daemon[847]: ts=2026-09-20T13:27:59Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4126worker2 # [ 19.458053] nix-grpc-daemon[847]: ts=2026-09-20T13:27:59Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4127worker2 # [ 19.462488] nix-grpc-daemon[847]: ts=2026-09-20T13:27:59Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14128lb # [ 19.460433] envoy[930]: [2026-09-20 13:27:59.484][930][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization4129worker1 # [ 19.460707] nix-grpc-daemon[848]: ts=2026-09-20T13:27:59Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4130lb # [ 19.462240] envoy[930]: [2026-09-20 13:27:59.486][930][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized4131lb # [ 19.463942] envoy[930]: [2026-09-20 13:27:59.486][930][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager4132lb # [ 19.465566] envoy[930]: [2026-09-20 13:27:59.488][930][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers4133worker2 # [ 19.501173] nix-grpc-daemon[847]: ts=2026-09-20T13:27:59Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4134worker2 # [ 19.504239] systemd[1]: Started Nix worker-protocol over gRPC.4135worker2 # [ 19.549303] nix-grpc-daemon[847]: ts=2026-09-20T13:27:59Z level=info event=scheduler_connected addr=lb:500514136worker2 # [ 19.557727] nix-grpc-daemon[847]: ts=2026-09-20T13:27:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4137worker1 # [ 19.662392] nix-grpc-daemon[848]: ts=2026-09-20T13:27:59Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4138worker2 # [ 19.759375] nix-grpc-daemon[847]: ts=2026-09-20T13:27:59Z level=info event=scheduler_connected addr=lb:500514139worker2 # [ 19.761162] nix-grpc-daemon[847]: ts=2026-09-20T13:27:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4140worker2 # [ 19.961838] nix-grpc-daemon[847]: ts=2026-09-20T13:27:59Z level=info event=scheduler_connected addr=lb:500514141worker2 # [ 19.963857] nix-grpc-daemon[847]: ts=2026-09-20T13:27:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4142worker1 # [ 20.063115] nix-grpc-daemon[848]: ts=2026-09-20T13:27:59Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4143worker2 # [ 20.164615] nix-grpc-daemon[847]: ts=2026-09-20T13:27:59Z level=info event=scheduler_connected addr=lb:500514144worker2 # [ 20.166329] nix-grpc-daemon[847]: ts=2026-09-20T13:27:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4145worker2 # [ 20.367613] nix-grpc-daemon[847]: ts=2026-09-20T13:28:00Z level=info event=scheduler_connected addr=lb:500514146worker2 # [ 20.371498] nix-grpc-daemon[847]: ts=2026-09-20T13:28:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4147worker2 # [ 20.572443] nix-grpc-daemon[847]: ts=2026-09-20T13:28:00Z level=info event=scheduler_connected addr=lb:500514148worker2 # [ 20.574308] nix-grpc-daemon[847]: ts=2026-09-20T13:28:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4149worker2 # [ 20.775601] nix-grpc-daemon[847]: ts=2026-09-20T13:28:00Z level=info event=scheduler_connected addr=lb:500514150worker2 # [ 20.780447] nix-grpc-daemon[847]: ts=2026-09-20T13:28:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4151worker1 # [ 20.867626] nix-grpc-daemon[848]: ts=2026-09-20T13:28:00Z level=info event=scheduler_connected addr=lb:500514152worker1 # [ 20.872576] nix-grpc-daemon[848]: ts=2026-09-20T13:28:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4153worker2 # [ 20.981386] nix-grpc-daemon[847]: ts=2026-09-20T13:28:00Z level=info event=scheduler_connected addr=lb:500514154worker2 # [ 20.985117] nix-grpc-daemon[847]: ts=2026-09-20T13:28:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4155worker1 # [ 21.073817] nix-grpc-daemon[848]: ts=2026-09-20T13:28:00Z level=info event=scheduler_connected addr=lb:500514156worker1 # [ 21.077844] nix-grpc-daemon[848]: ts=2026-09-20T13:28:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4157worker2 # [ 21.185537] nix-grpc-daemon[847]: ts=2026-09-20T13:28:00Z level=info event=scheduler_connected addr=lb:500514158worker2 # [ 21.187306] nix-grpc-daemon[847]: ts=2026-09-20T13:28:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4159worker1 # [ 21.278845] nix-grpc-daemon[848]: ts=2026-09-20T13:28:00Z level=info event=scheduler_connected addr=lb:500514160worker1 # [ 21.283090] nix-grpc-daemon[848]: ts=2026-09-20T13:28:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4161worker2 # [ 21.388736] nix-grpc-daemon[847]: ts=2026-09-20T13:28:01Z level=info event=scheduler_connected addr=lb:500514162worker2 # [ 21.392631] nix-grpc-daemon[847]: ts=2026-09-20T13:28:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4163worker1 # [ 21.483704] nix-grpc-daemon[848]: ts=2026-09-20T13:28:01Z level=info event=scheduler_connected addr=lb:500514164worker1 # [ 21.487799] nix-grpc-daemon[848]: ts=2026-09-20T13:28:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4165worker2 # [ 21.594437] nix-grpc-daemon[847]: ts=2026-09-20T13:28:01Z level=info event=scheduler_connected addr=lb:500514166worker2 # [ 21.598067] nix-grpc-daemon[847]: ts=2026-09-20T13:28:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4167worker1 # [ 21.688611] nix-grpc-daemon[848]: ts=2026-09-20T13:28:01Z level=info event=scheduler_connected addr=lb:500514168worker1 # [ 21.692926] nix-grpc-daemon[848]: ts=2026-09-20T13:28:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4169worker2 # [ 21.799234] nix-grpc-daemon[847]: ts=2026-09-20T13:28:01Z level=info event=scheduler_connected addr=lb:500514170worker2 # [ 21.803183] nix-grpc-daemon[847]: ts=2026-09-20T13:28:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4171worker1 # [ 21.894495] nix-grpc-daemon[848]: ts=2026-09-20T13:28:01Z level=info event=scheduler_connected addr=lb:500514172worker1 # [ 21.898269] nix-grpc-daemon[848]: ts=2026-09-20T13:28:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4173worker2 # [ 22.004250] nix-grpc-daemon[847]: ts=2026-09-20T13:28:01Z level=info event=scheduler_connected addr=lb:500514174worker2 # [ 22.007937] nix-grpc-daemon[847]: ts=2026-09-20T13:28:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4175worker1 # [ 22.099296] nix-grpc-daemon[848]: ts=2026-09-20T13:28:01Z level=info event=scheduler_connected addr=lb:500514176worker1 # [ 22.103408] nix-grpc-daemon[848]: ts=2026-09-20T13:28:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4177worker2 # [ 22.209142] nix-grpc-daemon[847]: ts=2026-09-20T13:28:01Z level=info event=scheduler_connected addr=lb:500514178worker2 # [ 22.212927] nix-grpc-daemon[847]: ts=2026-09-20T13:28:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4179worker1 # [ 22.304201] nix-grpc-daemon[848]: ts=2026-09-20T13:28:01Z level=info event=scheduler_connected addr=lb:500514180worker1 # [ 22.308238] nix-grpc-daemon[848]: ts=2026-09-20T13:28:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4181worker2 # [ 22.414233] nix-grpc-daemon[847]: ts=2026-09-20T13:28:02Z level=info event=scheduler_connected addr=lb:500514182worker2 # [ 22.418325] nix-grpc-daemon[847]: ts=2026-09-20T13:28:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4183worker1 # [ 22.508803] nix-grpc-daemon[848]: ts=2026-09-20T13:28:02Z level=info event=scheduler_connected addr=lb:500514184worker1 # [ 22.512925] nix-grpc-daemon[848]: ts=2026-09-20T13:28:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4185worker2 # [ 22.620567] nix-grpc-daemon[847]: ts=2026-09-20T13:28:02Z level=info event=scheduler_connected addr=lb:500514186worker2 # [ 22.623688] nix-grpc-daemon[847]: ts=2026-09-20T13:28:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4187worker1 # [ 22.714081] nix-grpc-daemon[848]: ts=2026-09-20T13:28:02Z level=info event=scheduler_connected addr=lb:500514188worker1 # [ 22.716138] nix-grpc-daemon[848]: ts=2026-09-20T13:28:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4189worker2 # [ 22.821741] nix-grpc-daemon[847]: ts=2026-09-20T13:28:02Z level=info event=scheduler_connected addr=lb:500514190worker2 # [ 22.825678] nix-grpc-daemon[847]: ts=2026-09-20T13:28:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4191worker1 # [ 22.917099] nix-grpc-daemon[848]: ts=2026-09-20T13:28:02Z level=info event=scheduler_connected addr=lb:500514192worker1 # [ 22.919039] nix-grpc-daemon[848]: ts=2026-09-20T13:28:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4193worker2 # [ 23.026776] nix-grpc-daemon[847]: ts=2026-09-20T13:28:02Z level=info event=scheduler_connected addr=lb:500514194worker2 # [ 23.030680] nix-grpc-daemon[847]: ts=2026-09-20T13:28:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4195worker1 # [ 23.119777] nix-grpc-daemon[848]: ts=2026-09-20T13:28:02Z level=info event=scheduler_connected addr=lb:500514196worker1 # [ 23.121686] nix-grpc-daemon[848]: ts=2026-09-20T13:28:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4197worker2 # [ 23.231667] nix-grpc-daemon[847]: ts=2026-09-20T13:28:02Z level=info event=scheduler_connected addr=lb:500514198worker2 # [ 23.235519] nix-grpc-daemon[847]: ts=2026-09-20T13:28:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4199worker1 # [ 23.322471] nix-grpc-daemon[848]: ts=2026-09-20T13:28:02Z level=info event=scheduler_connected addr=lb:500514200worker1 # [ 23.324432] nix-grpc-daemon[848]: ts=2026-09-20T13:28:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4201worker2 # [ 23.436657] nix-grpc-daemon[847]: ts=2026-09-20T13:28:03Z level=info event=scheduler_connected addr=lb:500514202worker2 # [ 23.440544] nix-grpc-daemon[847]: ts=2026-09-20T13:28:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4203worker1 # [ 23.525064] nix-grpc-daemon[848]: ts=2026-09-20T13:28:03Z level=info event=scheduler_connected addr=lb:500514204worker1 # [ 23.528369] nix-grpc-daemon[848]: ts=2026-09-20T13:28:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4205worker2 # [ 23.642704] nix-grpc-daemon[847]: ts=2026-09-20T13:28:03Z level=info event=scheduler_connected addr=lb:500514206worker2 # [ 23.645888] nix-grpc-daemon[847]: ts=2026-09-20T13:28:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4207worker1 # [ 23.729181] nix-grpc-daemon[848]: ts=2026-09-20T13:28:03Z level=info event=scheduler_connected addr=lb:500514208worker1 # [ 23.733283] nix-grpc-daemon[848]: ts=2026-09-20T13:28:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4209worker2 # [ 23.843668] nix-grpc-daemon[847]: ts=2026-09-20T13:28:03Z level=info event=scheduler_connected addr=lb:500514210worker2 # [ 23.847495] nix-grpc-daemon[847]: ts=2026-09-20T13:28:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4211worker1 # [ 23.935622] nix-grpc-daemon[848]: ts=2026-09-20T13:28:03Z level=info event=scheduler_connected addr=lb:500514212worker1 # [ 23.938767] nix-grpc-daemon[848]: ts=2026-09-20T13:28:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4213worker2 # [ 24.048554] nix-grpc-daemon[847]: ts=2026-09-20T13:28:03Z level=info event=scheduler_connected addr=lb:500514214worker2 # [ 24.052387] nix-grpc-daemon[847]: ts=2026-09-20T13:28:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4215worker1 # [ 24.136503] nix-grpc-daemon[848]: ts=2026-09-20T13:28:03Z level=info event=scheduler_connected addr=lb:500514216worker1 # [ 24.140716] nix-grpc-daemon[848]: ts=2026-09-20T13:28:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4217worker2 # [ 24.253383] nix-grpc-daemon[847]: ts=2026-09-20T13:28:04Z level=info event=scheduler_connected addr=lb:500514218worker2 # [ 24.257309] nix-grpc-daemon[847]: ts=2026-09-20T13:28:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4219worker1 # [ 24.341519] nix-grpc-daemon[848]: ts=2026-09-20T13:28:03Z level=info event=scheduler_connected addr=lb:500514220worker1 # [ 24.345581] nix-grpc-daemon[848]: ts=2026-09-20T13:28:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4221worker2 # [ 24.457791] nix-grpc-daemon[847]: ts=2026-09-20T13:28:04Z level=info event=scheduler_connected addr=lb:500514222worker2 # [ 24.459754] nix-grpc-daemon[847]: ts=2026-09-20T13:28:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4223worker1 # [ 24.546473] nix-grpc-daemon[848]: ts=2026-09-20T13:28:04Z level=info event=scheduler_connected addr=lb:500514224worker1 # [ 24.550409] nix-grpc-daemon[848]: ts=2026-09-20T13:28:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4225worker2 # [ 24.662233] nix-grpc-daemon[847]: ts=2026-09-20T13:28:04Z level=info event=scheduler_connected addr=lb:500514226worker2 # [ 24.665405] nix-grpc-daemon[847]: ts=2026-09-20T13:28:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4227worker1 # [ 24.750910] nix-grpc-daemon[848]: ts=2026-09-20T13:28:04Z level=info event=scheduler_connected addr=lb:500514228worker1 # [ 24.754759] nix-grpc-daemon[848]: ts=2026-09-20T13:28:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4229worker2 # [ 24.862925] nix-grpc-daemon[847]: ts=2026-09-20T13:28:04Z level=info event=scheduler_connected addr=lb:500514230worker2 # [ 24.866713] nix-grpc-daemon[847]: ts=2026-09-20T13:28:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4231worker1 # [ 24.956641] nix-grpc-daemon[848]: ts=2026-09-20T13:28:04Z level=info event=scheduler_connected addr=lb:500514232worker1 # [ 24.960286] nix-grpc-daemon[848]: ts=2026-09-20T13:28:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4233worker2 # [ 25.067357] nix-grpc-daemon[847]: ts=2026-09-20T13:28:04Z level=info event=scheduler_connected addr=lb:500514234worker2 # [ 25.069248] nix-grpc-daemon[847]: ts=2026-09-20T13:28:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4235worker1 # [ 25.161148] nix-grpc-daemon[848]: ts=2026-09-20T13:28:04Z level=info event=scheduler_connected addr=lb:500514236worker1 # [ 25.165153] nix-grpc-daemon[848]: ts=2026-09-20T13:28:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4237worker2 # [ 25.269837] nix-grpc-daemon[847]: ts=2026-09-20T13:28:05Z level=info event=scheduler_connected addr=lb:500514238worker2 # [ 25.271597] nix-grpc-daemon[847]: ts=2026-09-20T13:28:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4239worker1 # [ 25.365917] nix-grpc-daemon[848]: ts=2026-09-20T13:28:04Z level=info event=scheduler_connected addr=lb:500514240worker1 # [ 25.369994] nix-grpc-daemon[848]: ts=2026-09-20T13:28:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4241worker2 # [ 25.472305] nix-grpc-daemon[847]: ts=2026-09-20T13:28:05Z level=info event=scheduler_connected addr=lb:500514242worker2 # [ 25.474119] nix-grpc-daemon[847]: ts=2026-09-20T13:28:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4243worker1 # [ 25.570908] nix-grpc-daemon[848]: ts=2026-09-20T13:28:05Z level=info event=scheduler_connected addr=lb:500514244worker1 # [ 25.574965] nix-grpc-daemon[848]: ts=2026-09-20T13:28:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4245worker2 # [ 25.676416] nix-grpc-daemon[847]: ts=2026-09-20T13:28:05Z level=info event=scheduler_connected addr=lb:500514246worker2 # [ 25.679444] nix-grpc-daemon[847]: ts=2026-09-20T13:28:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4247worker1 # [ 25.775939] nix-grpc-daemon[848]: ts=2026-09-20T13:28:05Z level=info event=scheduler_connected addr=lb:500514248worker1 # [ 25.779984] nix-grpc-daemon[848]: ts=2026-09-20T13:28:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4249worker2 # [ 25.877816] nix-grpc-daemon[847]: ts=2026-09-20T13:28:05Z level=info event=scheduler_connected addr=lb:500514250worker2 # [ 25.881576] nix-grpc-daemon[847]: ts=2026-09-20T13:28:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4251worker1 # [ 25.981925] nix-grpc-daemon[848]: ts=2026-09-20T13:28:05Z level=info event=scheduler_connected addr=lb:500514252worker1 # [ 25.986547] nix-grpc-daemon[848]: ts=2026-09-20T13:28:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4253worker2 # [ 26.082544] nix-grpc-daemon[847]: ts=2026-09-20T13:28:05Z level=info event=scheduler_connected addr=lb:500514254worker2 # [ 26.086579] nix-grpc-daemon[847]: ts=2026-09-20T13:28:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4255lb # [ 26.177497] rustfs-bucket-start[1012]: mb s3://farm4256lb # [ 26.182852] systemd[1]: Finished rustfs-bucket.service.4257lb # [ 26.189430] systemd[1]: Starting niks3 server...4258worker1 # [ 26.187761] nix-grpc-daemon[848]: ts=2026-09-20T13:28:05Z level=info event=scheduler_connected addr=lb:500514259worker1 # [ 26.192562] nix-grpc-daemon[848]: ts=2026-09-20T13:28:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4260worker2 # [ 26.287396] nix-grpc-daemon[847]: ts=2026-09-20T13:28:06Z level=info event=scheduler_connected addr=lb:500514261worker2 # [ 26.291207] nix-grpc-daemon[847]: ts=2026-09-20T13:28:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4262worker1 # [ 26.392912] nix-grpc-daemon[848]: ts=2026-09-20T13:28:06Z level=info event=scheduler_connected addr=lb:500514263worker1 # [ 26.394836] nix-grpc-daemon[848]: ts=2026-09-20T13:28:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4264lb # [ 26.412218] postgres[1027]: [1027] ERROR: relation "goose_db_version" does not exist at character 364265lb # [ 26.413401] postgres[1027]: [1027] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4266lb # [ 26.448978] niks3-server[1021]: 2026/09/20 13:28:06 OK 20241026095416_initial_model.sql (19.07ms)4267lb # [ 26.452821] niks3-server[1021]: 2026/09/20 13:28:06 OK 20251210153512_drop_unused_gin_index.sql (2.12ms)4268lb # [ 26.458271] niks3-server[1021]: 2026/09/20 13:28:06 OK 20251218171726_add_pins.sql (5.43ms)4269lb # [ 26.464056] niks3-server[1021]: 2026/09/20 13:28:06 OK 20260628120000_add_object_size_and_stats.sql (5.76ms)4270worker2 # [ 26.492105] nix-grpc-daemon[847]: ts=2026-09-20T13:28:06Z level=info event=scheduler_connected addr=lb:500514271lb # [ 26.468392] niks3-server[1021]: 2026/09/20 13:28:06 OK 20260905000000_add_claims.sql (4.26ms)4272worker2 # [ 26.496107] nix-grpc-daemon[847]: ts=2026-09-20T13:28:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4273lb # [ 26.473223] niks3-server[1021]: 2026/09/20 13:28:06 OK 20260920000000_drop_claims.sql (4.55ms)4274lb # [ 26.474352] niks3-server[1021]: 2026/09/20 13:28:06 goose: successfully migrated database to version: 202609200000004275lb # [ 26.479638] niks3-server[1021]: 2026/09/20 13:28:06 OK 1_commit_pending_closure.sql (6.64ms)4276lb # [ 26.482312] niks3-server[1021]: 2026/09/20 13:28:06 OK 2_object_stats_trigger.sql (1.41ms)4277lb # [ 26.483639] niks3-server[1021]: 2026/09/20 13:28:06 goose: up to current file version: 24278lb # [ 26.489152] niks3-server[1021]: 2026/09/20 13:28:06 INFO Loaded signing key name=farm-test-1 path=/nix/store/c745bsy529madd2s6hz4z5is2436ni04-key4279lb # [ 26.555761] niks3-server[1021]: 2026/09/20 13:28:06 INFO Read proxy enabled — serving cache objects from S34280lb # [ 26.557540] niks3-server[1021]: 2026/09/20 13:28:06 INFO Using socket-activated listener address=0.0.0.0:57514281lb # [ 26.562105] niks3-server[1021]: 2026/09/20 13:28:06 INFO systemd watchdog enabled interval=15s4282lb # [ 26.564425] systemd[1]: Started niks3 server.4283lb # [ 26.565133] systemd[1]: Reached target Multi-User System.4284lb # [ 26.567644] systemd[1]: Startup finished in 869ms (kernel) + 5.038s (initrd) + 20.659s (userspace) = 26.567s.4285lb # [ 26.569894] niks3-server[1021]: 2026/09/20 13:28:06 INFO Starting HTTP server address=0.0.0.0:57514286lb # [ 26.576402] niks3-server[1021]: 2026/09/20 13:28:06 INFO lead: acquired remote=192.168.1.4:426564287worker2 # [ 26.611263] nix-grpc-daemon[847]: ts=2026-09-20T13:28:06Z level=info event=scheduler_take_over4288worker1 # [ 26.595933] nix-grpc-daemon[848]: ts=2026-09-20T13:28:06Z level=info event=scheduler_connected addr=lb:500514289worker1 # [ 26.600177] nix-grpc-daemon[848]: ts=2026-09-20T13:28:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4290worker2 # [ 26.696736] nix-grpc-daemon[847]: ts=2026-09-20T13:28:06Z level=info event=scheduler_connected addr=lb:500514291worker2 # [ 26.698739] nix-grpc-daemon[847]: ts=2026-09-20T13:28:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4292worker1 # [ 26.800764] nix-grpc-daemon[848]: ts=2026-09-20T13:28:06Z level=info event=scheduler_connected addr=lb:500514293worker1 # [ 26.803928] nix-grpc-daemon[848]: ts=2026-09-20T13:28:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4294lb: (finished: waiting for unit niks3.service, in 15.32 seconds)4295lb: waiting for TCP port 5751 on localhost4296worker2 # [ 26.899627] nix-grpc-daemon[847]: ts=2026-09-20T13:28:06Z level=info event=scheduler_connected addr=lb:500514297worker2 # [ 26.901723] nix-grpc-daemon[847]: ts=2026-09-20T13:28:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4298lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!4299lb: (finished: waiting for TCP port 5751 on localhost, in 0.08 seconds)4300lb: waiting for unit envoy.service4301lb: (finished: waiting for unit envoy.service, in 0.05 seconds)4302lb: waiting for TCP port 50051 on localhost4303lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!4304lb # [ 26.976829] envoy[930]: conn peer=[::1]:35276 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=44305lb: (finished: waiting for TCP port 50051 on localhost, in 0.03 seconds)4306worker1 # [ 27.006669] nix-grpc-daemon[848]: ts=2026-09-20T13:28:06Z level=info event=scheduler_connected addr=lb:500514307worker1 # [ 27.010975] nix-grpc-daemon[848]: ts=2026-09-20T13:28:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4308worker2 # [ 27.102540] nix-grpc-daemon[847]: ts=2026-09-20T13:28:06Z level=info event=scheduler_connected addr=lb:500514309worker2 # [ 27.104941] nix-grpc-daemon[847]: ts=2026-09-20T13:28:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4310worker2 # [ 27.137181] systemd[1]: Starting scheduler-token.service...4311worker2 # [ 27.208845] systemd[1]: scheduler-token.service: Deactivated successfully.4312worker2 # [ 27.209885] systemd[1]: Finished scheduler-token.service.4313worker2 # [ 27.211438] systemd[1]: scheduler-token.service: Consumed 20ms CPU time over 73ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4314lb: waiting until exactly no endpoints of x86_64-linux are unhealthy4315??? Warning (UserWarning): retry(): The 'timeout_seconds' argument is deprecated. Use 'timeout' instead.4316 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 1284317lb: must succeed: curl -sf localhost:9901/clusters4318worker1 # [ 27.206985] nix-grpc-daemon[848]: ts=2026-09-20T13:28:06Z level=info event=scheduler_connected addr=lb:500514319worker1 # [ 27.211232] nix-grpc-daemon[848]: ts=2026-09-20T13:28:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4320lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4321lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.08 seconds)4322subtest: one node schedules and both builders hold a WorkerSession on it4323lb: must succeed: curl -sf localhost:9901/clusters4324worker2 # [ 27.305841] nix-grpc-daemon[847]: ts=2026-09-20T13:28:07Z level=info event=scheduler_connected addr=lb:500514325worker2 # [ 27.309618] nix-grpc-daemon[847]: ts=2026-09-20T13:28:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4326lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4327worker1 # [ 27.411947] nix-grpc-daemon[848]: ts=2026-09-20T13:28:07Z level=info event=scheduler_connected addr=lb:500514328worker1 # [ 27.416136] nix-grpc-daemon[848]: ts=2026-09-20T13:28:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4329worker2 # [ 27.511925] nix-grpc-daemon[847]: ts=2026-09-20T13:28:07Z level=info event=scheduler_connected addr=lb:500514330worker2 # [ 27.516153] nix-grpc-daemon[847]: ts=2026-09-20T13:28:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4331worker1 # [ 27.616659] nix-grpc-daemon[848]: ts=2026-09-20T13:28:07Z level=info event=scheduler_connected addr=lb:500514332worker2 # [ 27.659324] nix-grpc-daemon[847]: ts=2026-09-20T13:28:07Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:330884333worker2 # [ 27.664293] nix-grpc-daemon[847]: ts=2026-09-20T13:28:07Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04334worker2 # [ 27.716441] nix-grpc-daemon[847]: ts=2026-09-20T13:28:07Z level=info event=scheduler_connected addr=lb:500514335worker2 # [ 27.734086] nix-grpc-daemon[847]: ts=2026-09-20T13:28:07Z level=info event=oidc_jwks provider=mock keys=14336worker2 # [ 27.738342] nix-grpc-daemon[847]: ts=2026-09-20T13:28:07Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:330884337worker2 # [ 27.742151] nix-grpc-daemon[847]: ts=2026-09-20T13:28:07Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04338lb: must succeed: curl -sf localhost:9901/clusters4339lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4340worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4341worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4342lb: must succeed: curl -sf localhost:9901/clusters4343lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4344lb: must succeed: curl -sf localhost:9901/clusters4345lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4346worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'4347worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.05 seconds)4348(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 1.28 seconds)4349subtest: balancer auth: client cert, bearer token, nothing, unknown CN4350worker1 # [ 28.848390] nix-grpc-daemon[848]: ts=2026-09-20T13:28:08Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:489064351worker1 # [ 28.855346] systemd[1]: Started Nix Daemon.4352worker1 # [ 29.033682] nix-daemon[905]: accepted connection from pid 848, user nix-grpc-daemon (trusted)4353worker1 # [ 29.085852] nix-grpc-daemon[848]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 28 ms (attempt 1/5)4354worker1 # [ 29.116285] nix-grpc-daemon[848]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 174 ms (attempt 2/5)4355worker1 # [ 29.293200] nix-grpc-daemon[848]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 195 ms (attempt 3/5)4356worker1 # [ 29.494073] nix-grpc-daemon[848]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 162 ms (attempt 4/5)4357worker1 # [ 29.664893] nix-grpc-daemon[848]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4358worker1 # [ 29.698128] nix-grpc-daemon[848]: ts=2026-09-20T13:28:09Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:489064359worker1 # [ 29.717341] nix-grpc-daemon[848]: ts=2026-09-20T13:28:09Z level=debug event=path_info_miss path=95v1d82m9fafgfkbb6biz3wcfj11j9p6-nixos-system-client-test error="error: path '/nix/store/95v1d82m9fafgfkbb6biz3wcfj11j9p6-nixos-system-client-test' is required, but there is no substituter that can build it"4360worker2 # [ 29.764197] nix-grpc-daemon[847]: ts=2026-09-20T13:28:09Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:330904361worker2 # [ 29.770897] systemd[1]: Started Nix Daemon.4362worker2 # [ 29.967337] nix-daemon[915]: accepted connection from pid 847, user nix-grpc-daemon (trusted)4363worker2 # [ 29.993895] nix-grpc-daemon[847]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 38 ms (attempt 1/5)4364worker2 # [ 30.034492] nix-grpc-daemon[847]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 102 ms (attempt 2/5)4365worker2 # [ 30.139129] nix-grpc-daemon[847]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 296 ms (attempt 3/5)4366worker2 # [ 30.435438] nix-grpc-daemon[847]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 419 ms (attempt 4/5)4367worker2 # [ 30.862914] nix-grpc-daemon[847]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4368worker2 # [ 30.875473] nix-grpc-daemon[847]: ts=2026-09-20T13:28:10Z level=debug event=path_info_miss path=95v1d82m9fafgfkbb6biz3wcfj11j9p6-nixos-system-client-test error="error: path '/nix/store/95v1d82m9fafgfkbb6biz3wcfj11j9p6-nixos-system-client-test' is required, but there is no substituter that can build it"4369client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4370client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.09 seconds)4371client: must succeed: install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4372client: (finished: must succeed: install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds)4373worker1 # [ 31.210193] nix-grpc-daemon[848]: ts=2026-09-20T13:28:10Z level=info event=oidc_jwks provider=mock keys=14374worker1 # [ 31.211808] nix-grpc-daemon[848]: ts=2026-09-20T13:28:10Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:489064375worker2 # [ 31.252260] nix-grpc-daemon[847]: ts=2026-09-20T13:28:10Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:330904376worker2 # [ 31.262951] nix-grpc-daemon[847]: ts=2026-09-20T13:28:11Z level=debug event=path_info_miss path=95v1d82m9fafgfkbb6biz3wcfj11j9p6-nixos-system-client-test error="error: path '/nix/store/95v1d82m9fafgfkbb6biz3wcfj11j9p6-nixos-system-client-test' is required, but there is no substituter that can build it"4377worker2 # [ 31.273620] nix-grpc-daemon[847]: ts=2026-09-20T13:28:11Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:330904378worker2 # [ 31.283419] nix-grpc-daemon[847]: ts=2026-09-20T13:28:11Z level=debug event=path_info_miss path=95v1d82m9fafgfkbb6biz3wcfj11j9p6-nixos-system-client-test error="error: path '/nix/store/95v1d82m9fafgfkbb6biz3wcfj11j9p6-nixos-system-client-test' is required, but there is no substituter that can build it"4379client: must succeed: rm -r /var/lib/nix-grpc-store4380client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.03 seconds)4381worker2 # [ 31.527746] nix-grpc-daemon[847]: ts=2026-09-20T13:28:11Z level=info event=denied method=QueryMissing cn=- role=none4382worker2 # [ 31.737173] nix-grpc-daemon[847]: ts=2026-09-20T13:28:11Z level=info event=denied method=QueryMissing cn=stranger role=none4383(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.19 seconds)4384subtest: DAG build is scheduled across workers and lands in the cache4385client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24386worker2 # [ 31.971899] nix-grpc-daemon[847]: ts=2026-09-20T13:28:11Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:330904387client # don't know how to build these paths:4388client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4389worker1 # [ 31.962963] nix-grpc-daemon[848]: ts=2026-09-20T13:28:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:489064390worker2 # [ 32.019267] nix-grpc-daemon[847]: ts=2026-09-20T13:28:11Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04391worker2 # [ 32.023817] nix-grpc-daemon[847]: ts=2026-09-20T13:28:11Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.3:50051 clients=14392worker1 # [ 32.047299] nix-grpc-daemon[848]: ts=2026-09-20T13:28:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:489064393worker1 # [ 32.067979] nix-grpc-daemon[848]: ts=2026-09-20T13:28:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:489064394client # copying 1 paths to 'lb:50051'...4395worker1 # [ 32.080081] nix-grpc-daemon[848]: ts=2026-09-20T13:28:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:489064396client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4397worker1 # [ 32.109377] nix-daemon[905]: accepted connection from pid 848, user nix-grpc-daemon (trusted)4398worker1 # [ 32.135227] nix-grpc-daemon[848]: ts=2026-09-20T13:28:11Z level=info event=niks3_push_started pid=9154399worker1 # [ 32.433746] nix-grpc-daemon[915]: time=2026-09-20T13:28:12.043Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""4400worker1 # [ 32.641459] nix-daemon[905]: accepted connection from pid 928, user nix-grpc-daemon (trusted)4401worker1 # [ 32.658625] nix-daemon[905]: reaped child process 931, status = succeeded4402worker1 # [ 32.828062] nix-daemon[905]: accepted connection from pid 933, user nix-grpc-daemon (trusted)4403worker1 # [ 32.840990] nix-daemon[905]: reaped child process 936, status = succeeded4404lb # [ 32.856661] niks3-server[1021]: 2026/09/20 13:28:12 INFO Received uploads request method=POST path=/api/pending_closures4405worker1 # [ 32.879301] nix-grpc-daemon[915]: time=2026-09-20T13:28:12.492Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4406worker1 # [ 32.881670] nix-grpc-daemon[915]: time=2026-09-20T13:28:12.494Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4407lb # [ 32.924073] niks3-server[1021]: 2026/09/20 13:28:12 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4408lb # [ 32.927167] niks3-server[1021]: 2026/09/20 13:28:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4409lb # [ 32.940385] niks3-server[1021]: 2026/09/20 13:28:12 INFO Signed narinfos id=1 count=14410worker1 # [ 32.940446] nix-grpc-daemon[915]: time=2026-09-20T13:28:12.553Z level=INFO msg="Uploading 1 narinfos"4411lb # [ 32.953639] niks3-server[1021]: 2026/09/20 13:28:12 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4412lb # [ 32.957731] niks3-server[1021]: 2026/09/20 13:28:12 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4413lb # [ 32.963797] niks3-server[1021]: 2026/09/20 13:28:12 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo4414lb # [ 32.967894] niks3-server[1021]: 2026/09/20 13:28:12 INFO Completed upload id=14415worker1 # [ 32.969318] nix-grpc-daemon[915]: time=2026-09-20T13:28:12.582Z level=INFO msg="Upload complete. (508ms)"4416worker1 # [ 32.974613] nix-grpc-daemon[848]: ts=2026-09-20T13:28:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48906 duration_s=0 paths=1 nar_bytes_in=5044417worker1 # [ 32.981940] nix-grpc-daemon[848]: ts=2026-09-20T13:28:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:489064418worker1 # [ 32.993195] nix-daemon[905]: accepted connection from pid 848, user nix-grpc-daemon (trusted)4419worker1 # [ 33.001736] nix-daemon[905]: reaped child process 913, status = succeeded4420client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4421worker1 # [ 33.010899] nix-daemon[905]: accepted connection from pid 848, user nix-grpc-daemon (trusted)4422client # farm-a> node-a: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4423client # farm-a> LOG-farm-a-t14424worker1 # [ 34.439310] nix-daemon[905]: reaped child process 940, status = succeeded4425worker1 # [ 34.590363] nix-daemon[905]: accepted connection from pid 956, user nix-grpc-daemon (trusted)4426worker1 # [ 34.603843] nix-daemon[905]: reaped child process 959, status = succeeded4427lb # [ 34.620527] niks3-server[1021]: 2026/09/20 13:28:14 INFO Received uploads request method=POST path=/api/pending_closures4428worker1 # [ 34.636339] nix-grpc-daemon[915]: time=2026-09-20T13:28:14.249Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4429worker1 # [ 34.638043] nix-grpc-daemon[915]: time=2026-09-20T13:28:14.251Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4430lb # [ 34.668190] niks3-server[1021]: 2026/09/20 13:28:14 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4431lb # [ 34.681367] niks3-server[1021]: 2026/09/20 13:28:14 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4432lb # [ 34.688439] niks3-server[1021]: 2026/09/20 13:28:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4433lb # [ 34.692382] niks3-server[1021]: 2026/09/20 13:28:14 INFO Signed narinfos id=2 count=14434worker1 # [ 34.695208] nix-grpc-daemon[915]: time=2026-09-20T13:28:14.307Z level=INFO msg="Uploading 1 narinfos"4435lb # [ 34.704065] niks3-server[1021]: 2026/09/20 13:28:14 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4436lb # [ 34.715481] niks3-server[1021]: 2026/09/20 13:28:14 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4437lb # [ 34.718168] niks3-server[1021]: 2026/09/20 13:28:14 INFO Completed upload id=24438worker1 # [ 34.718316] nix-grpc-daemon[915]: time=2026-09-20T13:28:14.331Z level=INFO msg="Upload complete. (296ms)"4439lb # [ 34.723207] niks3-server[1021]: 2026/09/20 13:28:14 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4440worker1 # [ 34.724540] nix-grpc-daemon[848]: ts=2026-09-20T13:28:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48906 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14441worker2 # [ 34.765909] nix-grpc-daemon[847]: ts=2026-09-20T13:28:14Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04442worker2 # [ 34.768302] nix-grpc-daemon[847]: ts=2026-09-20T13:28:14Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.3:50051 clients=14443worker1 # [ 34.738828] nix-daemon[905]: reaped child process 938, status = succeeded4444worker1 # [ 34.781445] nix-grpc-daemon[848]: ts=2026-09-20T13:28:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:489064445worker1 # [ 34.793946] nix-grpc-daemon[848]: ts=2026-09-20T13:28:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:489064446worker1 # [ 34.808674] nix-grpc-daemon[915]: time=2026-09-20T13:28:14.421Z level=INFO msg="All 1 paths already cached"4447client # copying 1 paths to 'lb:50051'...4448worker1 # [ 34.816479] nix-grpc-daemon[848]: ts=2026-09-20T13:28:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:489064449client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4450worker1 # [ 34.836370] nix-daemon[905]: accepted connection from pid 848, user nix-grpc-daemon (trusted)4451worker1 # [ 35.022378] nix-daemon[905]: accepted connection from pid 963, user nix-grpc-daemon (trusted)4452worker1 # [ 35.035546] nix-daemon[905]: reaped child process 966, status = succeeded4453worker1 # [ 35.191585] nix-daemon[905]: accepted connection from pid 968, user nix-grpc-daemon (trusted)4454worker1 # [ 35.204177] nix-daemon[905]: reaped child process 971, status = succeeded4455lb # [ 35.218943] niks3-server[1021]: 2026/09/20 13:28:15 INFO Received uploads request method=POST path=/api/pending_closures4456worker1 # [ 35.225080] nix-grpc-daemon[915]: time=2026-09-20T13:28:14.838Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4457worker1 # [ 35.226695] nix-grpc-daemon[915]: time=2026-09-20T13:28:14.839Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4458lb # [ 35.236971] niks3-server[1021]: 2026/09/20 13:28:15 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4459lb # [ 35.241639] niks3-server[1021]: 2026/09/20 13:28:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4460lb # [ 35.244628] niks3-server[1021]: 2026/09/20 13:28:15 INFO Signed narinfos id=3 count=14461worker1 # [ 35.244606] nix-grpc-daemon[915]: time=2026-09-20T13:28:14.857Z level=INFO msg="Uploading 1 narinfos"4462lb # [ 35.245897] niks3-server[1021]: 2026/09/20 13:28:15 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4463lb # [ 35.251451] niks3-server[1021]: 2026/09/20 13:28:15 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4464lb # [ 35.254925] niks3-server[1021]: 2026/09/20 13:28:15 INFO Completed upload id=34465worker1 # [ 35.254907] nix-grpc-daemon[915]: time=2026-09-20T13:28:14.868Z level=INFO msg="Upload complete. (403ms)"4466lb # [ 35.256877] niks3-server[1021]: 2026/09/20 13:28:15 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4467worker1 # [ 35.257095] nix-grpc-daemon[848]: ts=2026-09-20T13:28:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48906 duration_s=0 paths=1 nar_bytes_in=6244468worker1 # [ 35.261246] nix-grpc-daemon[848]: ts=2026-09-20T13:28:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:489064469worker1 # [ 35.266921] nix-daemon[905]: accepted connection from pid 848, user nix-grpc-daemon (trusted)4470worker1 # [ 35.269081] nix-daemon[905]: reaped child process 961, status = succeeded4471worker1 # [ 35.274353] nix-daemon[905]: accepted connection from pid 848, user nix-grpc-daemon (trusted)4472client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4473client # farm-b> node-a: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4474client # farm-b> LOG-farm-b-t14475worker1 # [ 36.702747] nix-daemon[905]: reaped child process 975, status = succeeded4476worker1 # [ 36.864561] nix-daemon[905]: accepted connection from pid 991, user nix-grpc-daemon (trusted)4477worker1 # [ 36.877373] nix-daemon[905]: reaped child process 994, status = succeeded4478lb # [ 36.894774] niks3-server[1021]: 2026/09/20 13:28:16 INFO Received uploads request method=POST path=/api/pending_closures4479worker1 # [ 36.904389] nix-grpc-daemon[915]: time=2026-09-20T13:28:16.516Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4480worker1 # [ 36.905847] nix-grpc-daemon[915]: time=2026-09-20T13:28:16.516Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4481lb # [ 36.924251] niks3-server[1021]: 2026/09/20 13:28:16 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4482lb # [ 36.936897] niks3-server[1021]: 2026/09/20 13:28:16 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4483lb # [ 36.946515] niks3-server[1021]: 2026/09/20 13:28:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4484lb # [ 36.951508] niks3-server[1021]: 2026/09/20 13:28:16 INFO Signed narinfos id=4 count=14485worker1 # [ 36.954289] nix-grpc-daemon[915]: time=2026-09-20T13:28:16.567Z level=INFO msg="Uploading 1 narinfos"4486lb # [ 36.960238] niks3-server[1021]: 2026/09/20 13:28:16 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4487lb # [ 36.967773] niks3-server[1021]: 2026/09/20 13:28:16 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4488lb # [ 36.976043] niks3-server[1021]: 2026/09/20 13:28:17 INFO Completed upload id=44489lb # [ 36.977724] niks3-server[1021]: 2026/09/20 13:28:17 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4490worker1 # [ 36.976306] nix-grpc-daemon[915]: time=2026-09-20T13:28:16.589Z level=INFO msg="Upload complete. (292ms)"4491worker1 # [ 36.982651] nix-grpc-daemon[848]: ts=2026-09-20T13:28:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48906 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14492worker2 # [ 37.018371] nix-grpc-daemon[847]: ts=2026-09-20T13:28:16Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04493worker2 # [ 37.022532] nix-grpc-daemon[847]: ts=2026-09-20T13:28:16Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.3:50051 clients=14494worker1 # [ 36.997120] nix-daemon[905]: reaped child process 973, status = succeeded4495worker1 # [ 37.033366] nix-grpc-daemon[848]: ts=2026-09-20T13:28:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:489064496worker1 # [ 37.046076] nix-grpc-daemon[848]: ts=2026-09-20T13:28:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:489064497worker1 # [ 37.059157] nix-grpc-daemon[915]: time=2026-09-20T13:28:16.672Z level=INFO msg="All 2 paths already cached"4498client # copying 1 paths to 'lb:50051'...4499client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4500worker1 # [ 37.067844] nix-grpc-daemon[848]: ts=2026-09-20T13:28:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:489064501worker1 # [ 37.080958] nix-daemon[905]: accepted connection from pid 848, user nix-grpc-daemon (trusted)4502worker1 # [ 37.251766] nix-daemon[905]: accepted connection from pid 998, user nix-grpc-daemon (trusted)4503worker1 # [ 37.265534] nix-daemon[905]: reaped child process 1001, status = succeeded4504worker1 # [ 37.415243] nix-daemon[905]: accepted connection from pid 1003, user nix-grpc-daemon (trusted)4505worker1 # [ 37.428673] nix-daemon[905]: reaped child process 1006, status = succeeded4506lb # [ 37.441881] niks3-server[1021]: 2026/09/20 13:28:17 INFO Received uploads request method=POST path=/api/pending_closures4507worker1 # [ 37.450778] nix-grpc-daemon[915]: time=2026-09-20T13:28:17.064Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4508worker1 # [ 37.452369] nix-grpc-daemon[915]: time=2026-09-20T13:28:17.065Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4509lb # [ 37.469891] niks3-server[1021]: 2026/09/20 13:28:17 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4510lb # [ 37.481906] niks3-server[1021]: 2026/09/20 13:28:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4511worker1 # [ 37.487142] nix-grpc-daemon[915]: time=2026-09-20T13:28:17.100Z level=INFO msg="Uploading 1 narinfos"4512lb # [ 37.487134] niks3-server[1021]: 2026/09/20 13:28:17 INFO Signed narinfos id=5 count=14513lb # [ 37.493807] niks3-server[1021]: 2026/09/20 13:28:17 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4514lb # [ 37.504231] niks3-server[1021]: 2026/09/20 13:28:17 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4515lb # [ 37.544248] niks3-server[1021]: 2026/09/20 13:28:17 INFO Completed upload id=54516worker1 # [ 37.544459] nix-grpc-daemon[915]: time=2026-09-20T13:28:17.157Z level=INFO msg="Upload complete. (447ms)"4517lb # [ 37.547216] niks3-server[1021]: 2026/09/20 13:28:17 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4518worker1 # [ 37.549213] nix-grpc-daemon[848]: ts=2026-09-20T13:28:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48906 duration_s=0 paths=1 nar_bytes_in=7524519worker1 # [ 37.555261] nix-grpc-daemon[848]: ts=2026-09-20T13:28:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:489064520worker1 # [ 37.565150] nix-daemon[905]: accepted connection from pid 848, user nix-grpc-daemon (trusted)4521worker1 # [ 37.579815] nix-daemon[905]: reaped child process 996, status = succeeded4522client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4523client # farm-top> node-a: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4524worker1 # [ 37.586150] nix-daemon[905]: accepted connection from pid 848, user nix-grpc-daemon (trusted)4525client # farm-top> LOG-farm-top-t14526worker1 # [ 39.013748] nix-daemon[905]: reaped child process 1010, status = succeeded4527lb # [ 39.133376] envoy[930]: conn peer=[2001:db8:1::1]:56978 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=749 tx=423 ms=20444528lb # [ 39.135148] envoy[930]: conn peer=[2001:db8:1::1]:56984 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=634529lb # [ 39.136474] envoy[930]: conn peer=[2001:db8:1::1]:56994 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=134530lb # [ 39.137873] envoy[930]: conn peer=[2001:db8:1::1]:57004 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=114531worker1 # [ 39.157386] nix-daemon[905]: accepted connection from pid 1025, user nix-grpc-daemon (trusted)4532worker1 # [ 39.171377] nix-daemon[905]: reaped child process 1028, status = succeeded4533lb # [ 39.186527] niks3-server[1021]: 2026/09/20 13:28:19 INFO Received uploads request method=POST path=/api/pending_closures4534worker1 # [ 39.191069] nix-grpc-daemon[915]: time=2026-09-20T13:28:18.804Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4535worker1 # [ 39.194060] nix-grpc-daemon[915]: time=2026-09-20T13:28:18.805Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4536lb # [ 39.205282] niks3-server[1021]: 2026/09/20 13:28:19 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4537lb # [ 39.209039] niks3-server[1021]: 2026/09/20 13:28:19 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4538lb # [ 39.214403] niks3-server[1021]: 2026/09/20 13:28:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4539lb # [ 39.216513] niks3-server[1021]: 2026/09/20 13:28:19 INFO Signed narinfos id=6 count=14540worker1 # [ 39.217821] nix-grpc-daemon[915]: time=2026-09-20T13:28:18.830Z level=INFO msg="Uploading 1 narinfos"4541lb # [ 39.222030] niks3-server[1021]: 2026/09/20 13:28:19 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4542lb # [ 39.224120] niks3-server[1021]: 2026/09/20 13:28:19 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4543lb # [ 39.228395] niks3-server[1021]: 2026/09/20 13:28:19 INFO Completed upload id=64544worker1 # [ 39.229655] nix-grpc-daemon[915]: time=2026-09-20T13:28:18.842Z level=INFO msg="Upload complete. (230ms)"4545lb # [ 39.230150] niks3-server[1021]: 2026/09/20 13:28:19 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4546worker1 # [ 39.232318] nix-grpc-daemon[848]: ts=2026-09-20T13:28:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48906 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14547worker1 # [ 39.238951] nix-daemon[905]: reaped child process 1008, status = succeeded4548client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 7.53 seconds)4549client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4550client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.18 seconds)4551client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4552client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.02 seconds)4553client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4554client # copying 3 paths...4555client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4556client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4557client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4558client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.25 seconds)4559(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 8.06 seconds)4560subtest: repeat is answered Cached by the scheduler without building4561worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4562worker1 # finding garbage collector roots...4563worker1 # deleting specified paths...4564worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4565worker1 # deleting unused links...4566worker1 # note: hard linking is currently saving 0.0 KiB4567worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.23 seconds)4568worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4569worker2 # finding garbage collector roots...4570worker2 # deleting specified paths...4571worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4572worker2 # deleting unused links...4573worker2 # note: hard linking is currently saving 0.0 KiB4574worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.26 seconds)4575client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24576client # don't know how to build these paths:4577worker2 # [ 40.517981] nix-grpc-daemon[847]: ts=2026-09-20T13:28:20Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:330904578client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4579worker1 # [ 40.494500] nix-grpc-daemon[848]: ts=2026-09-20T13:28:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:489064580worker1 # [ 40.506769] nix-grpc-daemon[915]: time=2026-09-20T13:28:20.119Z level=INFO msg="All 2 paths already cached"4581worker2 # [ 40.547864] nix-grpc-daemon[847]: ts=2026-09-20T13:28:20Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14582client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 0.27 seconds)4583client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4584client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.18 seconds)4585lb: must succeed: curl -sf localhost:9901/clusters4586lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4587lb: must succeed: curl -sf localhost:9901/clusters4588lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4589worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4590worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4591(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.16 seconds)4592subtest: inputs already in the cache are not sent to the scheduler4593lb: must succeed: curl -sf localhost:9901/clusters4594lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4595lb: must succeed: curl -sf localhost:9901/clusters4596lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4597worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4598worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.03 seconds)4599client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&24600worker1 # [ 41.277553] nix-grpc-daemon[848]: ts=2026-09-20T13:28:20Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:489064601client # don't know how to build these paths:4602client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4603worker1 # [ 41.283049] nix-grpc-daemon[848]: ts=2026-09-20T13:28:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:489064604worker1 # [ 41.288854] nix-grpc-daemon[915]: time=2026-09-20T13:28:20.902Z level=INFO msg="All 2 paths already cached"4605worker2 # [ 41.327362] nix-grpc-daemon[847]: ts=2026-09-20T13:28:21Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04606worker2 # [ 41.329579] nix-grpc-daemon[847]: ts=2026-09-20T13:28:21Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.3:50051 clients=14607worker1 # [ 41.341856] nix-grpc-daemon[848]: ts=2026-09-20T13:28:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:489064608worker1 # [ 41.351145] nix-grpc-daemon[848]: ts=2026-09-20T13:28:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:489064609worker1 # [ 41.359472] nix-grpc-daemon[915]: time=2026-09-20T13:28:20.972Z level=INFO msg="All 2 paths already cached"4610client # copying 1 paths to 'lb:50051'...4611worker1 # [ 41.365195] nix-grpc-daemon[848]: ts=2026-09-20T13:28:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:489064612client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4613worker1 # [ 41.372704] nix-daemon[905]: accepted connection from pid 848, user nix-grpc-daemon (trusted)4614worker1 # [ 41.535393] nix-daemon[905]: accepted connection from pid 1046, user nix-grpc-daemon (trusted)4615worker1 # [ 41.548681] nix-daemon[905]: reaped child process 1049, status = succeeded4616worker1 # [ 41.722080] nix-daemon[905]: accepted connection from pid 1051, user nix-grpc-daemon (trusted)4617worker1 # [ 41.734708] nix-daemon[905]: reaped child process 1054, status = succeeded4618lb # [ 41.751505] niks3-server[1021]: 2026/09/20 13:28:21 INFO Received uploads request method=POST path=/api/pending_closures4619worker1 # [ 41.761714] nix-grpc-daemon[915]: time=2026-09-20T13:28:21.374Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4620worker1 # [ 41.763153] nix-grpc-daemon[915]: time=2026-09-20T13:28:21.374Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4621lb # [ 41.780783] niks3-server[1021]: 2026/09/20 13:28:21 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4622lb # [ 41.791136] niks3-server[1021]: 2026/09/20 13:28:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4623lb # [ 41.796703] niks3-server[1021]: 2026/09/20 13:28:21 INFO Signed narinfos id=7 count=14624worker1 # [ 41.799602] nix-grpc-daemon[915]: time=2026-09-20T13:28:21.412Z level=INFO msg="Uploading 1 narinfos"4625lb # [ 41.801258] niks3-server[1021]: 2026/09/20 13:28:21 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4626lb # [ 41.813761] niks3-server[1021]: 2026/09/20 13:28:21 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4627worker1 # [ 41.819956] nix-grpc-daemon[915]: time=2026-09-20T13:28:21.433Z level=INFO msg="Upload complete. (442ms)"4628lb # [ 41.820280] niks3-server[1021]: 2026/09/20 13:28:21 INFO Completed upload id=74629lb # [ 41.823669] niks3-server[1021]: 2026/09/20 13:28:21 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4630worker1 # [ 41.822851] nix-grpc-daemon[848]: ts=2026-09-20T13:28:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48906 duration_s=0 paths=1 nar_bytes_in=7524631worker1 # [ 41.827960] nix-grpc-daemon[848]: ts=2026-09-20T13:28:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:489064632worker1 # [ 41.837225] nix-daemon[905]: accepted connection from pid 848, user nix-grpc-daemon (trusted)4633worker1 # [ 41.844627] nix-daemon[905]: reaped child process 1044, status = succeeded4634worker1 # [ 41.847868] nix-daemon[905]: accepted connection from pid 848, user nix-grpc-daemon (trusted)4635client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4636client # farm-top> node-a: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4637client # farm-top> LOG-farm-top-t1b4638worker1 # [ 42.865350] nix-daemon[905]: reaped child process 1058, status = succeeded4639worker1 # [ 43.018892] nix-daemon[905]: accepted connection from pid 1074, user nix-grpc-daemon (trusted)4640worker1 # [ 43.032507] nix-daemon[905]: reaped child process 1077, status = succeeded4641lb # [ 43.049505] niks3-server[1021]: 2026/09/20 13:28:23 INFO Received uploads request method=POST path=/api/pending_closures4642worker1 # [ 43.056501] nix-grpc-daemon[915]: time=2026-09-20T13:28:22.669Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4643worker1 # [ 43.058430] nix-grpc-daemon[915]: time=2026-09-20T13:28:22.671Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4644lb # [ 43.080450] niks3-server[1021]: 2026/09/20 13:28:23 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4645lb # [ 43.082950] niks3-server[1021]: 2026/09/20 13:28:23 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4646lb # [ 43.086985] niks3-server[1021]: 2026/09/20 13:28:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4647lb # [ 43.088951] niks3-server[1021]: 2026/09/20 13:28:23 INFO Signed narinfos id=8 count=14648worker1 # [ 43.089107] nix-grpc-daemon[915]: time=2026-09-20T13:28:22.702Z level=INFO msg="Uploading 1 narinfos"4649lb # [ 43.093705] niks3-server[1021]: 2026/09/20 13:28:23 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4650lb # [ 43.098106] niks3-server[1021]: 2026/09/20 13:28:23 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4651worker1 # [ 43.426823] nix-grpc-daemon[915]: time=2026-09-20T13:28:23.040Z level=INFO msg="Upload complete. (570ms)"4652lb # [ 43.426661] niks3-server[1021]: 2026/09/20 13:28:23 INFO Completed upload id=84653worker1 # [ 43.431301] nix-grpc-daemon[848]: ts=2026-09-20T13:28:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48906 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14654lb # [ 43.433716] niks3-server[1021]: 2026/09/20 13:28:23 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4655worker1 # [ 43.437868] nix-daemon[905]: reaped child process 1056, status = succeeded4656client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.39 seconds)4657client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath4658client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.21 seconds)4659lb: must succeed: curl -sf localhost:9901/clusters4660lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4661lb: must succeed: curl -sf localhost:9901/clusters4662lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4663worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4664worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4665(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.88 seconds)4666subtest: two clients wanting the same drv build it once4667client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4668client # [ 43.916208] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4669client # Running as unit: dedup1.service; invocation ID: b4d9c8314df14e2ca11b86e2c41e78964670client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 0.08 seconds)4671client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4672worker1 # [ 44.133704] nix-grpc-daemon[848]: ts=2026-09-20T13:28:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:489064673client # [ 44.149835] nix[887]: don't know how to build these paths:4674client # [ 44.150792] nix[887]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4675worker2 # [ 44.178904] nix-grpc-daemon[847]: ts=2026-09-20T13:28:23Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04676worker2 # [ 44.184605] nix-grpc-daemon[847]: ts=2026-09-20T13:28:23Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.4:50051 clients=14677worker2 # [ 44.191124] nix-grpc-daemon[847]: ts=2026-09-20T13:28:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330904678worker2 # [ 44.208744] nix-grpc-daemon[847]: ts=2026-09-20T13:28:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:330904679client # [ 44.198154] nix[887]: copying 1 paths to 'lb:50051'...4680client # [ 44.199183] nix[887]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4681worker2 # [ 44.219982] nix-grpc-daemon[847]: ts=2026-09-20T13:28:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:330904682worker2 # [ 44.232253] nix-daemon[915]: accepted connection from pid 847, user nix-grpc-daemon (trusted)4683worker2 # [ 44.255318] nix-grpc-daemon[847]: ts=2026-09-20T13:28:24Z level=info event=niks3_push_started pid=9674684worker2 # [ 44.548100] nix-grpc-daemon[967]: time=2026-09-20T13:28:24.291Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""4685worker2 # [ 44.742458] nix-daemon[915]: accepted connection from pid 980, user nix-grpc-daemon (trusted)4686worker2 # [ 44.758622] nix-daemon[915]: reaped child process 983, status = succeeded4687worker2 # [ 44.926916] nix-daemon[915]: accepted connection from pid 985, user nix-grpc-daemon (trusted)4688worker2 # [ 44.938830] nix-daemon[915]: reaped child process 988, status = succeeded4689lb # [ 44.927543] niks3-server[1021]: 2026/09/20 13:28:24 INFO Received uploads request method=POST path=/api/pending_closures4690worker2 # [ 44.966618] nix-grpc-daemon[967]: time=2026-09-20T13:28:24.713Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4691worker2 # [ 44.969063] nix-grpc-daemon[967]: time=2026-09-20T13:28:24.716Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4692lb # [ 44.975348] niks3-server[1021]: 2026/09/20 13:28:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4693lb # [ 44.981258] niks3-server[1021]: 2026/09/20 13:28:25 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4694worker2 # [ 45.018106] nix-grpc-daemon[967]: time=2026-09-20T13:28:24.765Z level=INFO msg="Uploading 1 narinfos"4695lb # [ 44.988413] niks3-server[1021]: 2026/09/20 13:28:25 INFO Signed narinfos id=9 count=14696lb # [ 44.992489] niks3-server[1021]: 2026/09/20 13:28:25 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4697client # [ 45.001701] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4698lb # [ 45.003198] niks3-server[1021]: 2026/09/20 13:28:25 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4699client # Running as unit: dedup2.service; invocation ID: 0977b4ef805c4acf88db790d0214a6184700lb # [ 45.010628] niks3-server[1021]: 2026/09/20 13:28:25 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4701lb # [ 45.013612] niks3-server[1021]: 2026/09/20 13:28:25 INFO Completed upload id=94702worker2 # [ 45.043198] nix-grpc-daemon[967]: time=2026-09-20T13:28:24.790Z level=INFO msg="Upload complete. (480ms)"4703client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 1.09 seconds)4704worker2 # [ 45.045598] nix-grpc-daemon[847]: ts=2026-09-20T13:28:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33090 duration_s=0 paths=1 nar_bytes_in=4324705worker2 # [ 45.053206] nix-grpc-daemon[847]: ts=2026-09-20T13:28:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330904706worker2 # [ 45.063399] nix-daemon[915]: accepted connection from pid 847, user nix-grpc-daemon (trusted)4707worker2 # [ 45.065332] nix-daemon[915]: reaped child process 965, status = succeeded4708worker2 # [ 45.071030] nix-daemon[915]: accepted connection from pid 847, user nix-grpc-daemon (trusted)4709client # [ 45.049318] nix[887]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4710client # [ 45.284873] nix[902]: don't know how to build these paths:4711client # [ 45.286125] nix[902]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4712worker1 # [ 45.276177] nix-grpc-daemon[848]: ts=2026-09-20T13:28:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:489064713worker2 # [ 45.317588] nix-grpc-daemon[847]: ts=2026-09-20T13:28:25Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04714client # [ 45.302628] nix[902]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4715worker2 # [ 45.323273] nix-grpc-daemon[847]: ts=2026-09-20T13:28:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330904716client: must succeed: sleep 24717lb # [ 49.134620] envoy[930]: conn peer=[2001:db8:1::1]:57020 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9537 tx=3853 ms=73294718lb # [ 49.138413] envoy[930]: conn peer=[2001:db8:1::1]:47634 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=434719lb # [ 49.141890] envoy[930]: conn peer=[2001:db8:1::1]:47644 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4182 tx=1700 ms=21704720client: (finished: must succeed: sleep 2, in 2.03 seconds)4721??? Warning (UserWarning): wait_until_succeeds(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4722 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394723client: waiting for success: ! systemctl is-active dedup1 dedup24724??? Warning (UserWarning): execute(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4725 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394726worker2 # [ 49.705933] nix-daemon[915]: reaped child process 992, status = succeeded4727worker2 # [ 49.863453] nix-daemon[915]: accepted connection from pid 1027, user nix-grpc-daemon (trusted)4728worker2 # [ 49.874283] nix-daemon[915]: reaped child process 1030, status = succeeded4729lb # [ 49.862156] niks3-server[1021]: 2026/09/20 13:28:29 INFO Received uploads request method=POST path=/api/pending_closures4730worker2 # [ 49.900602] nix-grpc-daemon[967]: time=2026-09-20T13:28:29.647Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4731worker2 # [ 49.902209] nix-grpc-daemon[967]: time=2026-09-20T13:28:29.649Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4732lb # [ 49.906776] niks3-server[1021]: 2026/09/20 13:28:29 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4733lb # [ 49.918785] niks3-server[1021]: 2026/09/20 13:28:29 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4734lb # [ 49.929564] niks3-server[1021]: 2026/09/20 13:28:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4735worker2 # [ 49.965114] nix-grpc-daemon[967]: time=2026-09-20T13:28:29.711Z level=INFO msg="Uploading 1 narinfos"4736lb # [ 49.934920] niks3-server[1021]: 2026/09/20 13:28:29 INFO Signed narinfos id=10 count=14737lb # [ 49.944864] niks3-server[1021]: 2026/09/20 13:28:29 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4738lb # [ 49.954386] niks3-server[1021]: 2026/09/20 13:28:29 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4739lb # [ 49.958354] niks3-server[1021]: 2026/09/20 13:28:29 INFO Completed upload id=104740worker2 # [ 49.987833] nix-grpc-daemon[967]: time=2026-09-20T13:28:29.734Z level=INFO msg="Upload complete. (302ms)"4741lb # [ 49.959930] niks3-server[1021]: 2026/09/20 13:28:29 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4742worker2 # [ 49.995688] nix-grpc-daemon[847]: ts=2026-09-20T13:28:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33090 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14743worker2 # [ 50.002945] nix-grpc-daemon[847]: ts=2026-09-20T13:28:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33090 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14744worker2 # [ 50.014415] nix-daemon[915]: reaped child process 990, status = succeeded4745client # [ 50.021311] systemd[1]: dedup1.service: Deactivated successfully.4746client # [ 50.028744] systemd[1]: dedup1.service: Consumed 128ms CPU time over 6.109s wall clock time, 37.3M memory peak, 8.5K incoming IP traffic, 9.1K outgoing IP traffic.4747client # [ 50.066947] systemd[1]: dedup2.service: Deactivated successfully.4748client # [ 50.068646] systemd[1]: dedup2.service: Consumed 145ms CPU time over 5.067s wall clock time, 31.2M memory peak, 7.6K incoming IP traffic, 6.4K outgoing IP traffic.4749client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.12 seconds)4750client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4751client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.05 seconds)4752worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4753worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.06 seconds)4754worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4755worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4756worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4757worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4758worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4759worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4760(finished: subtest: two clients wanting the same drv build it once, in 7.09 seconds)4761subtest: build hook: nix-daemon with builders = grpc://lb4762client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log4763client # [ 51.152514] systemd[1]: Started Nix Daemon.4764client # [ 51.329208] nix-daemon[943]: accepted connection from pid 940, user root (trusted)4765worker1 # [ 51.519912] nix-grpc-daemon[848]: ts=2026-09-20T13:28:31Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:489064766worker1 # [ 51.527943] nix-grpc-daemon[848]: ts=2026-09-20T13:28:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:489064767worker2 # [ 51.570571] nix-grpc-daemon[847]: ts=2026-09-20T13:28:31Z level=debug event=want client=6 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=04768worker2 # [ 51.574857] nix-grpc-daemon[847]: ts=2026-09-20T13:28:31Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.3:50051 clients=14769worker1 # [ 51.585098] nix-grpc-daemon[848]: ts=2026-09-20T13:28:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:489064770worker1 # [ 51.598856] nix-grpc-daemon[848]: ts=2026-09-20T13:28:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:489064771worker1 # [ 51.610500] nix-grpc-daemon[848]: ts=2026-09-20T13:28:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:489064772worker1 # [ 51.634530] nix-daemon[905]: accepted connection from pid 848, user nix-grpc-daemon (trusted)4773worker1 # [ 51.823869] nix-daemon[905]: accepted connection from pid 1106, user nix-grpc-daemon (trusted)4774worker1 # [ 51.840316] nix-daemon[905]: reaped child process 1109, status = succeeded4775worker1 # [ 52.003493] nix-daemon[905]: accepted connection from pid 1111, user nix-grpc-daemon (trusted)4776worker1 # [ 52.016423] nix-daemon[905]: reaped child process 1114, status = succeeded4777lb # [ 52.033809] niks3-server[1021]: 2026/09/20 13:28:32 INFO Received uploads request method=POST path=/api/pending_closures4778worker1 # [ 52.043232] nix-grpc-daemon[915]: time=2026-09-20T13:28:31.655Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4779worker1 # [ 52.044705] nix-grpc-daemon[915]: time=2026-09-20T13:28:31.656Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"4780lb # [ 52.062512] niks3-server[1021]: 2026/09/20 13:28:32 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst4781lb # [ 52.072123] niks3-server[1021]: 2026/09/20 13:28:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4782worker1 # [ 52.077413] nix-grpc-daemon[915]: time=2026-09-20T13:28:31.690Z level=INFO msg="Uploading 1 narinfos"4783lb # [ 52.077711] niks3-server[1021]: 2026/09/20 13:28:32 INFO Signed narinfos id=11 count=14784lb # [ 52.084849] niks3-server[1021]: 2026/09/20 13:28:32 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls4785lb # [ 52.093835] niks3-server[1021]: 2026/09/20 13:28:32 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4786lb # [ 52.100430] niks3-server[1021]: 2026/09/20 13:28:32 INFO Completed upload id=114787worker1 # [ 52.100482] nix-grpc-daemon[915]: time=2026-09-20T13:28:31.713Z level=INFO msg="Upload complete. (453ms)"4788lb # [ 52.104889] niks3-server[1021]: 2026/09/20 13:28:32 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo4789worker1 # [ 52.104110] nix-grpc-daemon[848]: ts=2026-09-20T13:28:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48906 duration_s=0 paths=1 nar_bytes_in=5124790worker1 # [ 52.109238] nix-grpc-daemon[848]: ts=2026-09-20T13:28:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:489064791worker1 # [ 52.118159] nix-daemon[905]: accepted connection from pid 848, user nix-grpc-daemon (trusted)4792worker1 # [ 52.125797] nix-daemon[905]: reaped child process 1104, status = succeeded4793worker1 # [ 52.128754] nix-daemon[905]: accepted connection from pid 848, user nix-grpc-daemon (trusted)4794worker1 # [ 53.151472] nix-daemon[905]: reaped child process 1118, status = succeeded4795worker1 # [ 53.316848] nix-daemon[905]: accepted connection from pid 1133, user nix-grpc-daemon (trusted)4796worker1 # [ 53.330108] nix-daemon[905]: reaped child process 1136, status = succeeded4797lb # [ 53.347412] niks3-server[1021]: 2026/09/20 13:28:33 INFO Received uploads request method=POST path=/api/pending_closures4798worker1 # [ 53.355860] nix-grpc-daemon[915]: time=2026-09-20T13:28:32.968Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4799worker1 # [ 53.359060] nix-grpc-daemon[915]: time=2026-09-20T13:28:32.970Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"4800lb # [ 53.383359] niks3-server[1021]: 2026/09/20 13:28:33 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4801lb # [ 53.396754] niks3-server[1021]: 2026/09/20 13:28:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4802lb # [ 53.404111] niks3-server[1021]: 2026/09/20 13:28:33 INFO Signed narinfos id=12 count=14803worker1 # [ 53.407567] nix-grpc-daemon[915]: time=2026-09-20T13:28:33.020Z level=INFO msg="Uploading 1 narinfos"4804lb # [ 53.410608] niks3-server[1021]: 2026/09/20 13:28:33 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls4805lb # [ 53.414859] niks3-server[1021]: 2026/09/20 13:28:33 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv4806lb # [ 53.425247] niks3-server[1021]: 2026/09/20 13:28:33 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4807lb # [ 53.430222] niks3-server[1021]: 2026/09/20 13:28:33 INFO Completed upload id=124808worker1 # [ 53.430580] nix-grpc-daemon[915]: time=2026-09-20T13:28:33.043Z level=INFO msg="Upload complete. (294ms)"4809lb # [ 53.432749] niks3-server[1021]: 2026/09/20 13:28:33 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo4810worker1 # [ 53.435934] nix-grpc-daemon[848]: ts=2026-09-20T13:28:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48906 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=6 outputs=14811worker1 # [ 53.444454] nix-grpc-daemon[848]: ts=2026-09-20T13:28:33Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:489064812worker1 # [ 53.453356] nix-daemon[905]: reaped child process 1116, status = succeeded4813worker2 # [ 53.484338] nix-grpc-daemon[847]: ts=2026-09-20T13:28:33Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:330904814worker2 # [ 53.494890] nix-grpc-daemon[847]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...4815worker2 # [ 53.514918] nix-grpc-daemon[847]: ts=2026-09-20T13:28:33Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:33090 duration_s=0 paths=1 nar_bytes_out=1284816worker1 # [ 53.739880] nix-grpc-daemon[848]: ts=2026-09-20T13:28:33Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:489064817worker1 # [ 53.747328] nix-grpc-daemon[848]: ts=2026-09-20T13:28:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:489064818worker1 # [ 53.786739] nix-grpc-daemon[915]: time=2026-09-20T13:28:33.399Z level=INFO msg="All 1 paths already cached"4819worker2 # [ 53.831758] nix-grpc-daemon[847]: ts=2026-09-20T13:28:33Z level=debug event=want client=7 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=04820worker1 # [ 53.804458] nix-grpc-daemon[848]: ts=2026-09-20T13:28:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:489064821worker2 # [ 53.836139] nix-grpc-daemon[847]: ts=2026-09-20T13:28:33Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.3:50051 clients=14822worker1 # [ 53.816158] nix-grpc-daemon[848]: ts=2026-09-20T13:28:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:489064823worker1 # [ 53.834887] nix-grpc-daemon[915]: time=2026-09-20T13:28:33.448Z level=INFO msg="All 1 paths already cached"4824worker1 # [ 53.842339] nix-grpc-daemon[848]: ts=2026-09-20T13:28:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:489064825worker1 # [ 53.853184] nix-daemon[905]: accepted connection from pid 848, user nix-grpc-daemon (trusted)4826worker1 # [ 54.017341] nix-daemon[905]: accepted connection from pid 1140, user nix-grpc-daemon (trusted)4827worker1 # [ 54.031952] nix-daemon[905]: reaped child process 1143, status = succeeded4828worker1 # [ 54.195649] nix-daemon[905]: accepted connection from pid 1145, user nix-grpc-daemon (trusted)4829worker1 # [ 54.208464] nix-daemon[905]: reaped child process 1148, status = succeeded4830lb # [ 54.222584] niks3-server[1021]: 2026/09/20 13:28:34 INFO Received uploads request method=POST path=/api/pending_closures4831worker1 # [ 54.227470] nix-grpc-daemon[915]: time=2026-09-20T13:28:33.840Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4832worker1 # [ 54.229248] nix-grpc-daemon[915]: time=2026-09-20T13:28:33.840Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"4833lb # [ 54.236217] niks3-server[1021]: 2026/09/20 13:28:34 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst4834lb # [ 54.240110] niks3-server[1021]: 2026/09/20 13:28:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4835lb # [ 54.242433] niks3-server[1021]: 2026/09/20 13:28:34 INFO Signed narinfos id=13 count=14836worker1 # [ 54.243950] nix-grpc-daemon[915]: time=2026-09-20T13:28:33.856Z level=INFO msg="Uploading 1 narinfos"4837lb # [ 54.246287] niks3-server[1021]: 2026/09/20 13:28:34 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls4838lb # [ 54.251780] niks3-server[1021]: 2026/09/20 13:28:34 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4839lb # [ 54.254169] niks3-server[1021]: 2026/09/20 13:28:34 INFO Completed upload id=134840worker1 # [ 54.255496] nix-grpc-daemon[915]: time=2026-09-20T13:28:33.868Z level=INFO msg="Upload complete. (389ms)"4841lb # [ 54.256116] niks3-server[1021]: 2026/09/20 13:28:34 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo4842worker1 # [ 54.258192] nix-grpc-daemon[848]: ts=2026-09-20T13:28:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48906 duration_s=0 paths=1 nar_bytes_in=6244843worker1 # [ 54.261540] nix-grpc-daemon[848]: ts=2026-09-20T13:28:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:489064844worker1 # [ 54.268603] nix-daemon[905]: accepted connection from pid 848, user nix-grpc-daemon (trusted)4845worker1 # [ 54.271946] nix-daemon[905]: reaped child process 1138, status = succeeded4846worker1 # [ 54.277157] nix-daemon[905]: accepted connection from pid 848, user nix-grpc-daemon (trusted)4847worker1 # [ 55.505879] nix-daemon[905]: reaped child process 1152, status = succeeded4848worker1 # [ 55.649165] nix-daemon[905]: accepted connection from pid 1167, user nix-grpc-daemon (trusted)4849worker1 # [ 55.663633] nix-daemon[905]: reaped child process 1170, status = succeeded4850lb # [ 55.682081] niks3-server[1021]: 2026/09/20 13:28:35 INFO Received uploads request method=POST path=/api/pending_closures4851worker1 # [ 55.687207] nix-grpc-daemon[915]: time=2026-09-20T13:28:35.300Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4852worker1 # [ 55.691051] nix-grpc-daemon[915]: time=2026-09-20T13:28:35.301Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"4853lb # [ 55.702283] niks3-server[1021]: 2026/09/20 13:28:35 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv4854lb # [ 55.705048] niks3-server[1021]: 2026/09/20 13:28:35 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst4855lb # [ 55.709960] niks3-server[1021]: 2026/09/20 13:28:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4856lb # [ 55.712048] niks3-server[1021]: 2026/09/20 13:28:35 INFO Signed narinfos id=14 count=14857worker1 # [ 55.713625] nix-grpc-daemon[915]: time=2026-09-20T13:28:35.326Z level=INFO msg="Uploading 1 narinfos"4858lb # [ 55.716287] niks3-server[1021]: 2026/09/20 13:28:35 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls4859lb # [ 55.721156] niks3-server[1021]: 2026/09/20 13:28:35 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4860lb # [ 55.724976] niks3-server[1021]: 2026/09/20 13:28:35 INFO Completed upload id=144861worker1 # [ 55.725183] nix-grpc-daemon[915]: time=2026-09-20T13:28:35.338Z level=INFO msg="Upload complete. (239ms)"4862lb # [ 55.726625] niks3-server[1021]: 2026/09/20 13:28:35 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo4863worker1 # [ 55.727801] nix-grpc-daemon[848]: ts=2026-09-20T13:28:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48906 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=7 outputs=14864worker1 # [ 55.734884] nix-daemon[905]: reaped child process 1150, status = succeeded4865worker2 # [ 55.764372] nix-grpc-daemon[847]: ts=2026-09-20T13:28:35Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:330904866worker2 # [ 55.769324] nix-grpc-daemon[847]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...4867worker1 # [ 55.760120] nix-grpc-daemon[848]: ts=2026-09-20T13:28:35Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:489064868worker1 # [ 55.763551] nix-grpc-daemon[848]: ts=2026-09-20T13:28:35Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48906 duration_s=0 paths=1 nar_bytes_out=1764869worker2 # [ 56.065871] nix-grpc-daemon[847]: ts=2026-09-20T13:28:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:330904870worker1 # [ 56.033896] nix-grpc-daemon[848]: ts=2026-09-20T13:28:35Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:489064871worker2 # [ 56.415392] nix-grpc-daemon[967]: time=2026-09-20T13:28:36.160Z level=INFO msg="All 2 paths already cached"4872worker2 # [ 56.431206] nix-grpc-daemon[847]: ts=2026-09-20T13:28:36Z level=debug event=want client=8 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=04873worker2 # [ 56.435498] nix-grpc-daemon[847]: ts=2026-09-20T13:28:36Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.3:50051 clients=14874worker1 # [ 56.445837] nix-grpc-daemon[848]: ts=2026-09-20T13:28:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:489064875worker1 # [ 56.456914] nix-grpc-daemon[848]: ts=2026-09-20T13:28:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:489064876worker1 # [ 56.473526] nix-grpc-daemon[915]: time=2026-09-20T13:28:36.086Z level=INFO msg="All 2 paths already cached"4877worker1 # [ 56.480312] nix-grpc-daemon[848]: ts=2026-09-20T13:28:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:489064878worker1 # [ 56.488701] nix-daemon[905]: accepted connection from pid 848, user nix-grpc-daemon (trusted)4879worker1 # [ 56.664116] nix-daemon[905]: accepted connection from pid 1174, user nix-grpc-daemon (trusted)4880worker1 # [ 56.677666] nix-daemon[905]: reaped child process 1177, status = succeeded4881worker1 # [ 56.855550] nix-daemon[905]: accepted connection from pid 1179, user nix-grpc-daemon (trusted)4882worker1 # [ 56.870954] nix-daemon[905]: reaped child process 1182, status = succeeded4883lb # [ 56.883760] niks3-server[1021]: 2026/09/20 13:28:36 INFO Received uploads request method=POST path=/api/pending_closures4884worker1 # [ 56.888974] nix-grpc-daemon[915]: time=2026-09-20T13:28:36.502Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4885worker1 # [ 56.890527] nix-grpc-daemon[915]: time=2026-09-20T13:28:36.503Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"4886lb # [ 56.900956] niks3-server[1021]: 2026/09/20 13:28:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4887lb # [ 56.903871] niks3-server[1021]: 2026/09/20 13:28:36 INFO Signed narinfos id=15 count=14888worker1 # [ 56.904976] nix-grpc-daemon[915]: time=2026-09-20T13:28:36.518Z level=INFO msg="Uploading 1 narinfos"4889lb # [ 56.907610] niks3-server[1021]: 2026/09/20 13:28:36 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls4890lb # [ 56.910247] niks3-server[1021]: 2026/09/20 13:28:36 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst4891lb # [ 56.914189] niks3-server[1021]: 2026/09/20 13:28:36 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4892lb # [ 56.917127] niks3-server[1021]: 2026/09/20 13:28:36 INFO Completed upload id=154893worker1 # [ 56.918274] nix-grpc-daemon[915]: time=2026-09-20T13:28:36.531Z level=INFO msg="Upload complete. (415ms)"4894lb # [ 56.918787] niks3-server[1021]: 2026/09/20 13:28:36 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo4895worker1 # [ 56.920723] nix-grpc-daemon[848]: ts=2026-09-20T13:28:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48906 duration_s=0 paths=1 nar_bytes_in=7524896worker1 # [ 56.923487] nix-grpc-daemon[848]: ts=2026-09-20T13:28:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:489064897worker1 # [ 56.929245] nix-daemon[905]: accepted connection from pid 848, user nix-grpc-daemon (trusted)4898worker1 # [ 56.934235] nix-daemon[905]: reaped child process 1172, status = succeeded4899worker1 # [ 56.938653] nix-daemon[905]: accepted connection from pid 848, user nix-grpc-daemon (trusted)4900worker1 # [ 57.564372] nix-daemon[905]: reaped child process 1186, status = succeeded4901worker1 # [ 57.717223] nix-daemon[905]: accepted connection from pid 1201, user nix-grpc-daemon (trusted)4902worker1 # [ 57.731100] nix-daemon[905]: reaped child process 1204, status = succeeded4903lb # [ 57.747522] niks3-server[1021]: 2026/09/20 13:28:37 INFO Received uploads request method=POST path=/api/pending_closures4904worker1 # [ 57.751756] nix-grpc-daemon[915]: time=2026-09-20T13:28:37.365Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4905worker1 # [ 57.753314] nix-grpc-daemon[915]: time=2026-09-20T13:28:37.366Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"4906lb # [ 57.771585] niks3-server[1021]: 2026/09/20 13:28:37 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv4907lb # [ 57.775057] niks3-server[1021]: 2026/09/20 13:28:37 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst4908lb # [ 57.777964] niks3-server[1021]: 2026/09/20 13:28:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4909lb # [ 57.779944] niks3-server[1021]: 2026/09/20 13:28:37 INFO Signed narinfos id=16 count=14910worker1 # [ 57.780205] nix-grpc-daemon[915]: time=2026-09-20T13:28:37.393Z level=INFO msg="Uploading 1 narinfos"4911lb # [ 57.782968] niks3-server[1021]: 2026/09/20 13:28:37 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls4912lb # [ 57.787056] niks3-server[1021]: 2026/09/20 13:28:37 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4913lb # [ 57.791568] niks3-server[1021]: 2026/09/20 13:28:37 INFO Completed upload id=164914worker1 # [ 57.791556] nix-grpc-daemon[915]: time=2026-09-20T13:28:37.404Z level=INFO msg="Upload complete. (246ms)"4915lb # [ 57.793220] niks3-server[1021]: 2026/09/20 13:28:37 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo4916worker1 # [ 57.793711] nix-grpc-daemon[848]: ts=2026-09-20T13:28:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48906 duration_s=0 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=8 outputs=14917worker1 # [ 57.800567] nix-daemon[905]: reaped child process 1184, status = succeeded4918worker2 # [ 57.831042] nix-grpc-daemon[847]: ts=2026-09-20T13:28:37Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:330904919worker2 # [ 57.835793] nix-grpc-daemon[847]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...4920worker2 # [ 57.857165] nix-grpc-daemon[847]: ts=2026-09-20T13:28:37Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:330904921worker2 # [ 57.859168] nix-grpc-daemon[847]: ts=2026-09-20T13:28:37Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:33090 duration_s=0 paths=1 nar_bytes_out=2324922client # [ 57.952532] nix-daemon[943]: reaped child process 945, status = succeeded4923client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.02 seconds)4924client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top4925client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.02 seconds)4926client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4927client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.04 seconds)4928client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4929client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4930(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.11 seconds)4931subtest: an input only one worker has still reaches the builder4932client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only4933client # this derivation will be built:4934client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv4935client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...4936client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only, in 0.41 seconds)4937client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure4938client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.16 seconds)4939worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4940worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.19 seconds)4941worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only4942worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)4943client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&24944client # [ 59.031461] nix-daemon[943]: accepted connection from pid 1023, user root (trusted)4945client # this derivation will be built:4946client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv4947lb # [ 59.134635] envoy[930]: conn peer=[2001:db8:1::1]:59906 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2972 tx=1372 ms=58544948lb # [ 59.138387] envoy[930]: conn peer=[2001:db8:1::1]:59914 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1427 tx=1082 ms=47574949lb # [ 59.141829] envoy[930]: conn peer=[2001:db8:1::1]:58346 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=334 ms=814950lb # [ 59.145242] envoy[930]: conn peer=[2001:db8:1::1]:59918 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3388 tx=1636 ms=20214951lb # [ 59.148690] envoy[930]: conn peer=[2001:db8:1::1]:58348 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3893 tx=1990 ms=20764952lb # [ 59.152244] envoy[930]: conn peer=[2001:db8:1::1]:58360 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=378 ms=554953lb # [ 59.155642] envoy[930]: conn peer=[2001:db8:1::1]:58388 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=544954lb # [ 59.159085] envoy[930]: conn peer=[2001:db8:1::1]:58372 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4341 tx=2221 ms=18504955worker2 # [ 59.245319] nix-grpc-daemon[847]: ts=2026-09-20T13:28:38Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:330904956client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...4957worker2 # [ 59.250837] nix-grpc-daemon[847]: ts=2026-09-20T13:28:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:330904958client # copying 1 paths...4959client # copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' to 'grpc://lb:50051'...4960worker2 # [ 59.261535] nix-grpc-daemon[847]: ts=2026-09-20T13:28:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:330904961worker2 # [ 59.272284] nix-daemon[915]: accepted connection from pid 847, user nix-grpc-daemon (trusted)4962worker2 # [ 59.461566] nix-daemon[915]: accepted connection from pid 1055, user nix-grpc-daemon (trusted)4963worker2 # [ 59.477100] nix-daemon[915]: reaped child process 1058, status = succeeded4964lb # [ 59.464785] niks3-server[1021]: 2026/09/20 13:28:39 INFO Received uploads request method=POST path=/api/pending_closures4965worker2 # [ 59.501042] nix-grpc-daemon[967]: time=2026-09-20T13:28:39.247Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4966worker2 # [ 59.502425] nix-grpc-daemon[967]: time=2026-09-20T13:28:39.247Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"4967lb # [ 59.489606] niks3-server[1021]: 2026/09/20 13:28:39 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4968lb # [ 59.498250] niks3-server[1021]: 2026/09/20 13:28:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4969worker2 # [ 59.532869] nix-grpc-daemon[967]: time=2026-09-20T13:28:39.279Z level=INFO msg="Uploading 1 narinfos"4970lb # [ 59.503125] niks3-server[1021]: 2026/09/20 13:28:39 INFO Signed narinfos id=17 count=14971lb # [ 59.509690] niks3-server[1021]: 2026/09/20 13:28:39 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls4972lb # [ 59.519468] niks3-server[1021]: 2026/09/20 13:28:39 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4973worker2 # [ 59.555226] nix-grpc-daemon[967]: time=2026-09-20T13:28:39.302Z level=INFO msg="Upload complete. (271ms)"4974lb # [ 59.525710] niks3-server[1021]: 2026/09/20 13:28:39 INFO Completed upload id=174975worker2 # [ 59.557427] nix-grpc-daemon[847]: ts=2026-09-20T13:28:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33090 duration_s=0 paths=1 nar_bytes_in=1204976lb # [ 59.530651] niks3-server[1021]: 2026/09/20 13:28:39 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo4977worker2 # [ 59.565790] nix-daemon[915]: reaped child process 1053, status = succeeded4978worker2 # [ 59.569336] nix-grpc-daemon[847]: ts=2026-09-20T13:28:39Z level=debug event=want client=9 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=04979worker2 # [ 59.571247] nix-grpc-daemon[847]: ts=2026-09-20T13:28:39Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=14980worker2 # [ 59.616454] nix-grpc-daemon[847]: ts=2026-09-20T13:28:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330904981worker2 # [ 59.625073] nix-grpc-daemon[847]: ts=2026-09-20T13:28:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:330904982client # copying 2 paths to 'lb:50051'...4983client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...4984worker2 # [ 59.636333] nix-grpc-daemon[847]: ts=2026-09-20T13:28:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:330904985client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...4986worker2 # [ 59.642186] nix-daemon[915]: accepted connection from pid 847, user nix-grpc-daemon (trusted)4987worker2 # [ 59.797931] nix-daemon[915]: accepted connection from pid 1064, user nix-grpc-daemon (trusted)4988worker2 # [ 59.812853] nix-daemon[915]: reaped child process 1067, status = succeeded4989worker2 # [ 59.961840] nix-daemon[915]: accepted connection from pid 1069, user nix-grpc-daemon (trusted)4990worker2 # [ 59.976354] nix-daemon[915]: reaped child process 1072, status = succeeded4991lb # [ 59.960282] niks3-server[1021]: 2026/09/20 13:28:39 INFO Received uploads request method=POST path=/api/pending_closures4992lb # [ 59.968809] niks3-server[1021]: 2026/09/20 13:28:39 INFO Received uploads request method=POST path=/api/pending_closures4993worker2 # [ 60.006915] nix-grpc-daemon[967]: time=2026-09-20T13:28:39.753Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4994worker2 # [ 60.008289] nix-grpc-daemon[967]: time=2026-09-20T13:28:39.753Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"4995worker2 # [ 60.009813] nix-grpc-daemon[967]: time=2026-09-20T13:28:39.753Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"4996lb # [ 60.004926] niks3-server[1021]: 2026/09/20 13:28:40 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst4997lb # [ 60.007831] niks3-server[1021]: 2026/09/20 13:28:40 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst4998lb # [ 60.013287] niks3-server[1021]: 2026/09/20 13:28:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4999lb # [ 60.015031] niks3-server[1021]: 2026/09/20 13:28:40 INFO Signed narinfos id=19 count=25000lb # [ 60.017554] niks3-server[1021]: 2026/09/20 13:28:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5001lb # [ 60.020327] niks3-server[1021]: 2026/09/20 13:28:40 INFO Signed narinfos id=18 count=15002lb # [ 60.021777] niks3-server[1021]: 2026/09/20 13:28:40 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5003worker2 # [ 60.051869] nix-grpc-daemon[967]: time=2026-09-20T13:28:39.798Z level=INFO msg="Uploading 3 narinfos"5004lb # [ 60.024782] niks3-server[1021]: 2026/09/20 13:28:40 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5005lb # [ 60.035141] niks3-server[1021]: 2026/09/20 13:28:40 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5006lb # [ 60.039308] niks3-server[1021]: 2026/09/20 13:28:40 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5007lb # [ 60.041818] niks3-server[1021]: 2026/09/20 13:28:40 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5008lb # [ 60.045832] niks3-server[1021]: 2026/09/20 13:28:40 INFO Completed upload id=185009lb # [ 60.047158] niks3-server[1021]: 2026/09/20 13:28:40 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5010lb # [ 60.049726] niks3-server[1021]: 2026/09/20 13:28:40 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5011lb # [ 60.051896] niks3-server[1021]: 2026/09/20 13:28:40 INFO Completed upload id=195012worker2 # [ 60.081622] nix-grpc-daemon[967]: time=2026-09-20T13:28:39.828Z level=INFO msg="Upload complete. (427ms)"5013worker2 # [ 60.085863] nix-grpc-daemon[847]: ts=2026-09-20T13:28:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33090 duration_s=0 paths=2 nar_bytes_in=9765014worker2 # [ 60.091306] nix-grpc-daemon[847]: ts=2026-09-20T13:28:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330905015worker2 # [ 60.099577] nix-daemon[915]: accepted connection from pid 847, user nix-grpc-daemon (trusted)5016worker2 # [ 60.105604] nix-daemon[915]: reaped child process 1062, status = succeeded5017worker2 # [ 60.111504] nix-daemon[915]: accepted connection from pid 847, user nix-grpc-daemon (trusted)5018client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5019client # uses-input-w1onlya> worker2: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5020worker2 # [ 60.725307] nix-daemon[915]: reaped child process 1076, status = succeeded5021worker2 # [ 60.862846] nix-daemon[915]: accepted connection from pid 1091, user nix-grpc-daemon (trusted)5022worker2 # [ 60.875996] nix-daemon[915]: reaped child process 1094, status = succeeded5023lb # [ 60.856877] niks3-server[1021]: 2026/09/20 13:28:40 INFO Received uploads request method=POST path=/api/pending_closures5024worker2 # [ 60.894564] nix-grpc-daemon[967]: time=2026-09-20T13:28:40.641Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5025lb # [ 60.885582] niks3-server[1021]: 2026/09/20 13:28:40 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5026lb # [ 60.893346] niks3-server[1021]: 2026/09/20 13:28:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5027worker2 # [ 60.927221] nix-grpc-daemon[967]: time=2026-09-20T13:28:40.674Z level=INFO msg="Uploading 1 narinfos"5028lb # [ 60.898174] niks3-server[1021]: 2026/09/20 13:28:40 INFO Signed narinfos id=20 count=15029lb # [ 60.904193] niks3-server[1021]: 2026/09/20 13:28:40 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5030lb # [ 60.914986] niks3-server[1021]: 2026/09/20 13:28:40 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5031lb # [ 60.919579] niks3-server[1021]: 2026/09/20 13:28:40 INFO Completed upload id=205032worker2 # [ 60.949020] nix-grpc-daemon[967]: time=2026-09-20T13:28:40.696Z level=INFO msg="Upload complete. (231ms)"5033worker2 # [ 60.952594] nix-grpc-daemon[847]: ts=2026-09-20T13:28:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33090 duration_s=0 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=9 outputs=15034client # copying 1 paths...5035worker2 # [ 60.958278] nix-grpc-daemon[967]: time=2026-09-20T13:28:40.699Z level=WARN msg="Failed to register uploaded object" key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo error="server returned 404: object is not pending upload\n"5036worker2 # [ 60.964323] nix-daemon[915]: reaped child process 1074, status = succeeded5037worker1 # [ 60.932717] nix-grpc-daemon[848]: ts=2026-09-20T13:28:40Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:489065038worker1 # [ 60.947450] nix-grpc-daemon[848]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5039client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5040worker1 # [ 60.977656] nix-grpc-daemon[848]: ts=2026-09-20T13:28:40Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:489065041worker1 # [ 60.982877] nix-grpc-daemon[848]: ts=2026-09-20T13:28:40Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48906 duration_s=0 paths=1 nar_bytes_out=1205042client # [ 61.102121] nix-daemon[943]: reaped child process 1026, status = succeeded5043client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.28 seconds)5044client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&25045client # [ 61.282551] nix-daemon[943]: accepted connection from pid 1044, user root (trusted)5046client # this derivation will be built:5047client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5048worker1 # [ 61.658607] nix-grpc-daemon[848]: ts=2026-09-20T13:28:41Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:489065049client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...5050worker1 # [ 61.664776] nix-grpc-daemon[848]: ts=2026-09-20T13:28:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:489065051worker1 # [ 61.672103] nix-grpc-daemon[915]: time=2026-09-20T13:28:41.283Z level=INFO msg="All 1 paths already cached"5052worker2 # [ 61.709280] nix-grpc-daemon[847]: ts=2026-09-20T13:28:41Z level=debug event=want client=10 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05053worker2 # [ 61.711385] nix-grpc-daemon[847]: ts=2026-09-20T13:28:41Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15054worker1 # [ 61.724411] nix-grpc-daemon[848]: ts=2026-09-20T13:28:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:489065055worker1 # [ 61.737073] nix-grpc-daemon[848]: ts=2026-09-20T13:28:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:489065056client # copying 1 paths to 'lb:50051'...5057client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5058worker1 # [ 61.754995] nix-grpc-daemon[848]: ts=2026-09-20T13:28:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:489065059worker1 # [ 61.766153] nix-daemon[905]: accepted connection from pid 848, user nix-grpc-daemon (trusted)5060worker1 # [ 61.782444] nix-grpc-daemon[848]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 100 ms (attempt 1/5)5061worker1 # [ 61.888471] nix-grpc-daemon[848]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 125 ms (attempt 2/5)5062worker1 # [ 62.020235] nix-grpc-daemon[848]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 310 ms (attempt 3/5)5063worker1 # [ 62.330147] nix-grpc-daemon[848]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 467 ms (attempt 4/5)5064worker1 # [ 62.803626] nix-grpc-daemon[848]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5065worker1 # [ 62.809159] nix-grpc-daemon[848]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5066worker1 # [ 63.020859] nix-daemon[905]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5067worker1 # [ 63.033342] nix-daemon[905]: reaped child process 1221, status = succeeded5068worker1 # [ 63.202581] nix-daemon[905]: accepted connection from pid 1223, user nix-grpc-daemon (trusted)5069worker1 # [ 63.214878] nix-daemon[905]: reaped child process 1226, status = succeeded5070lb # [ 63.233232] niks3-server[1021]: 2026/09/20 13:28:43 INFO Received uploads request method=POST path=/api/pending_closures5071worker1 # [ 63.241802] nix-grpc-daemon[915]: time=2026-09-20T13:28:42.854Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5072worker1 # [ 63.243302] nix-grpc-daemon[915]: time=2026-09-20T13:28:42.856Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5073lb # [ 63.261491] niks3-server[1021]: 2026/09/20 13:28:43 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5074lb # [ 63.272892] niks3-server[1021]: 2026/09/20 13:28:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5075worker1 # [ 63.277600] nix-grpc-daemon[915]: time=2026-09-20T13:28:42.890Z level=INFO msg="Uploading 1 narinfos"5076lb # [ 63.277912] niks3-server[1021]: 2026/09/20 13:28:43 INFO Signed narinfos id=21 count=15077lb # [ 63.284242] niks3-server[1021]: 2026/09/20 13:28:43 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5078lb # [ 63.294417] niks3-server[1021]: 2026/09/20 13:28:43 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5079worker1 # [ 63.300249] nix-grpc-daemon[915]: time=2026-09-20T13:28:42.913Z level=INFO msg="Upload complete. (474ms)"5080lb # [ 63.300494] niks3-server[1021]: 2026/09/20 13:28:43 INFO Completed upload id=215081worker1 # [ 63.302566] nix-grpc-daemon[848]: ts=2026-09-20T13:28:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48906 duration_s=1 paths=1 nar_bytes_in=5605082lb # [ 63.303736] niks3-server[1021]: 2026/09/20 13:28:43 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5083worker1 # [ 63.307695] nix-grpc-daemon[848]: ts=2026-09-20T13:28:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:489065084worker1 # [ 63.313376] nix-daemon[905]: accepted connection from pid 848, user nix-grpc-daemon (trusted)5085worker1 # [ 63.315650] nix-daemon[905]: reaped child process 1212, status = succeeded5086worker1 # [ 63.321041] nix-daemon[905]: accepted connection from pid 848, user nix-grpc-daemon (trusted)5087client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5088client # uses-input-w1onlyb> node-a: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5089worker1 # [ 63.739908] nix-daemon[905]: reaped child process 1230, status = succeeded5090worker1 # [ 63.899461] nix-daemon[905]: accepted connection from pid 1245, user nix-grpc-daemon (trusted)5091worker1 # [ 63.912333] nix-daemon[905]: reaped child process 1248, status = succeeded5092lb # [ 63.928859] niks3-server[1021]: 2026/09/20 13:28:43 INFO Received uploads request method=POST path=/api/pending_closures5093worker1 # [ 63.936691] nix-grpc-daemon[915]: time=2026-09-20T13:28:43.549Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5094lb # [ 63.955750] niks3-server[1021]: 2026/09/20 13:28:43 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5095lb # [ 63.964249] niks3-server[1021]: 2026/09/20 13:28:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5096lb # [ 63.968401] niks3-server[1021]: 2026/09/20 13:28:43 INFO Signed narinfos id=22 count=15097worker1 # [ 63.972415] nix-grpc-daemon[915]: time=2026-09-20T13:28:43.585Z level=INFO msg="Uploading 1 narinfos"5098lb # [ 63.977071] niks3-server[1021]: 2026/09/20 13:28:44 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5099lb # [ 63.989874] niks3-server[1021]: 2026/09/20 13:28:44 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5100lb # [ 63.994151] niks3-server[1021]: 2026/09/20 13:28:44 INFO Completed upload id=225101worker1 # [ 63.994043] nix-grpc-daemon[915]: time=2026-09-20T13:28:43.607Z level=INFO msg="Upload complete. (265ms)"5102lb # [ 63.998799] niks3-server[1021]: 2026/09/20 13:28:44 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5103worker1 # [ 63.997461] nix-grpc-daemon[848]: ts=2026-09-20T13:28:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48906 duration_s=0 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=10 outputs=15104client # copying 1 paths...5105worker1 # [ 64.009120] nix-daemon[905]: reaped child process 1228, status = succeeded5106worker2 # [ 64.037467] nix-grpc-daemon[847]: ts=2026-09-20T13:28:43Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:330905107worker2 # [ 64.047505] nix-grpc-daemon[847]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5108client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5109worker1 # [ 64.049575] nix-grpc-daemon[848]: ts=2026-09-20T13:28:43Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:489065110worker1 # [ 64.054562] nix-grpc-daemon[848]: ts=2026-09-20T13:28:43Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48906 duration_s=0 paths=1 nar_bytes_out=1205111client # [ 64.127298] nix-daemon[943]: reaped child process 1047, status = succeeded5112client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.03 seconds)5113client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&25114client # [ 64.316925] nix-daemon[943]: accepted connection from pid 1065, user root (trusted)5115client # this derivation will be built:5116client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5117worker2 # [ 64.510500] nix-grpc-daemon[847]: ts=2026-09-20T13:28:44Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:330905118client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...5119worker2 # [ 64.516849] nix-grpc-daemon[847]: ts=2026-09-20T13:28:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:330905120worker2 # [ 64.522256] nix-grpc-daemon[967]: time=2026-09-20T13:28:44.267Z level=INFO msg="All 1 paths already cached"5121worker2 # [ 64.531829] nix-grpc-daemon[847]: ts=2026-09-20T13:28:44Z level=debug event=want client=11 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05122worker2 # [ 64.538204] nix-grpc-daemon[847]: ts=2026-09-20T13:28:44Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15123worker2 # [ 64.543258] nix-grpc-daemon[847]: ts=2026-09-20T13:28:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330905124worker2 # [ 64.554456] nix-grpc-daemon[847]: ts=2026-09-20T13:28:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:330905125worker2 # [ 64.566524] nix-grpc-daemon[967]: time=2026-09-20T13:28:44.313Z level=INFO msg="All 1 paths already cached"5126client # copying 1 paths to 'lb:50051'...5127client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5128worker2 # [ 64.573707] nix-grpc-daemon[847]: ts=2026-09-20T13:28:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:330905129worker2 # [ 64.584756] nix-daemon[915]: accepted connection from pid 847, user nix-grpc-daemon (trusted)5130worker2 # [ 64.752309] nix-daemon[915]: accepted connection from pid 1103, user nix-grpc-daemon (trusted)5131worker2 # [ 64.765239] nix-daemon[915]: reaped child process 1106, status = succeeded5132worker2 # [ 64.926142] nix-daemon[915]: accepted connection from pid 1108, user nix-grpc-daemon (trusted)5133worker2 # [ 64.937857] nix-daemon[915]: reaped child process 1111, status = succeeded5134lb # [ 64.925384] niks3-server[1021]: 2026/09/20 13:28:44 INFO Received uploads request method=POST path=/api/pending_closures5135worker2 # [ 64.963976] nix-grpc-daemon[967]: time=2026-09-20T13:28:44.711Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5136worker2 # [ 64.965452] nix-grpc-daemon[967]: time=2026-09-20T13:28:44.712Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5137lb # [ 64.958852] niks3-server[1021]: 2026/09/20 13:28:44 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5138lb # [ 64.964501] niks3-server[1021]: 2026/09/20 13:28:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5139lb # [ 64.970590] niks3-server[1021]: 2026/09/20 13:28:44 INFO Signed narinfos id=23 count=15140worker2 # [ 65.003084] nix-grpc-daemon[967]: time=2026-09-20T13:28:44.750Z level=INFO msg="Uploading 1 narinfos"5141lb # [ 64.976982] niks3-server[1021]: 2026/09/20 13:28:45 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5142lb # [ 64.988142] niks3-server[1021]: 2026/09/20 13:28:45 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5143worker2 # [ 65.022071] nix-grpc-daemon[967]: time=2026-09-20T13:28:44.769Z level=INFO msg="Upload complete. (426ms)"5144lb # [ 64.992361] niks3-server[1021]: 2026/09/20 13:28:45 INFO Completed upload id=235145worker2 # [ 65.024451] nix-grpc-daemon[847]: ts=2026-09-20T13:28:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33090 duration_s=0 paths=1 nar_bytes_in=5605146worker2 # [ 65.030305] nix-grpc-daemon[967]: time=2026-09-20T13:28:44.775Z level=WARN msg="Failed to register uploaded object" key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo error="server returned 404: object is not pending upload\n"5147worker2 # [ 65.034632] nix-grpc-daemon[847]: ts=2026-09-20T13:28:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330905148worker2 # [ 65.041122] nix-daemon[915]: accepted connection from pid 847, user nix-grpc-daemon (trusted)5149worker2 # [ 65.046933] nix-daemon[915]: reaped child process 1101, status = succeeded5150worker2 # [ 65.051098] nix-daemon[915]: accepted connection from pid 847, user nix-grpc-daemon (trusted)5151client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5152client # uses-input-w1onlyc> worker2: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5153worker2 # [ 66.476737] nix-daemon[915]: reaped child process 1115, status = succeeded5154worker2 # [ 66.626164] nix-daemon[915]: accepted connection from pid 1130, user nix-grpc-daemon (trusted)5155worker2 # [ 66.636913] nix-daemon[915]: reaped child process 1133, status = succeeded5156lb # [ 66.623480] niks3-server[1021]: 2026/09/20 13:28:46 INFO Received uploads request method=POST path=/api/pending_closures5157worker2 # [ 66.660991] nix-grpc-daemon[967]: time=2026-09-20T13:28:46.407Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5158lb # [ 66.653825] niks3-server[1021]: 2026/09/20 13:28:46 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5159lb # [ 66.660923] niks3-server[1021]: 2026/09/20 13:28:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5160worker2 # [ 66.695160] nix-grpc-daemon[967]: time=2026-09-20T13:28:46.442Z level=INFO msg="Uploading 1 narinfos"5161lb # [ 66.665809] niks3-server[1021]: 2026/09/20 13:28:46 INFO Signed narinfos id=24 count=15162lb # [ 66.671303] niks3-server[1021]: 2026/09/20 13:28:46 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5163lb # [ 66.685070] niks3-server[1021]: 2026/09/20 13:28:46 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5164lb # [ 66.689875] niks3-server[1021]: 2026/09/20 13:28:46 INFO Completed upload id=245165worker2 # [ 66.719480] nix-grpc-daemon[967]: time=2026-09-20T13:28:46.466Z level=INFO msg="Upload complete. (261ms)"5166lb # [ 66.692537] niks3-server[1021]: 2026/09/20 13:28:46 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5167worker2 # [ 66.723392] nix-grpc-daemon[847]: ts=2026-09-20T13:28:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33090 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=11 outputs=15168worker2 # [ 66.735655] nix-daemon[915]: reaped child process 1113, status = succeeded5169client # copying 1 paths...5170worker1 # [ 66.744483] nix-grpc-daemon[848]: ts=2026-09-20T13:28:46Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:489065171worker1 # [ 66.753394] nix-grpc-daemon[848]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5172client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5173worker1 # [ 66.785174] nix-grpc-daemon[848]: ts=2026-09-20T13:28:46Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:489065174worker1 # [ 66.788970] nix-grpc-daemon[848]: ts=2026-09-20T13:28:46Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48906 duration_s=0 paths=1 nar_bytes_out=1205175client # [ 66.889263] nix-daemon[943]: reaped child process 1068, status = succeeded5176client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.76 seconds)5177(finished: subtest: an input only one worker has still reaches the builder, in 8.85 seconds)5178subtest: upload whose reference the worker lacks is completed from the cache5179client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache5180client # this derivation will be built:5181client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5182client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5183client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache, in 0.37 seconds)5184client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache5185client # this derivation will be built:5186client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5187client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5188client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache, in 0.37 seconds)5189client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5190client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.18 seconds)5191worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5192worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.20 seconds)5193worker2 # [ 68.083474] systemd[1]: Stopping Nix worker-protocol over gRPC...5194worker2 # [ 68.527734] nix-grpc-daemon[847]: ts=2026-09-20T13:28:48Z level=info event=signal_exit5195worker2 # [ 68.766695] nix-grpc-daemon[847]: ts=2026-09-20T13:28:48Z level=info event=scheduler_yield5196worker2 # [ 68.771236] nix-grpc-daemon[847]: ts=2026-09-20T13:28:48Z level=info event=scheduler_restarting addr=lb:500515197lb # [ 68.740823] niks3-server[1021]: 2026/09/20 13:28:48 INFO lead: released remote=192.168.1.4:426565198worker2 # [ 68.776382] nix-grpc-daemon[847]: ts=2026-09-20T13:28:48Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:330885199worker1 # [ 68.748518] nix-grpc-daemon[848]: ts=2026-09-20T13:28:48Z level=info event=scheduler_restarting addr=lb:500515200worker2 # [ 68.781112] nix-grpc-daemon[847]: ts=2026-09-20T13:28:48Z level=info event=worker_gone addr=192.168.1.4:500515201worker2 # [ 68.784210] nix-grpc-daemon[847]: ts=2026-09-20T13:28:48Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:330885202worker2 # [ 68.787471] nix-grpc-daemon[847]: ts=2026-09-20T13:28:48Z level=info event=worker_gone addr=192.168.1.3:500515203worker2 # [ 68.972675] nix-grpc-daemon[847]: ts=2026-09-20T13:28:48Z level=info event=scheduler_connected addr=lb:500515204worker2 # [ 68.976930] nix-grpc-daemon[847]: ts=2026-09-20T13:28:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5205worker1 # [ 68.948800] nix-grpc-daemon[848]: ts=2026-09-20T13:28:48Z level=info event=scheduler_connected addr=lb:500515206worker1 # [ 68.952790] nix-grpc-daemon[848]: ts=2026-09-20T13:28:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5207lb # [ 69.135716] envoy[930]: conn peer=[2001:db8:1::1]:58404 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4195 tx=1842 ms=18215208lb # [ 69.139475] envoy[930]: conn peer=[2001:db8:1::1]:58410 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=565209lb # [ 69.142945] envoy[930]: conn peer=[2001:db8:1::1]:58422 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3701 tx=1891 ms=24075210worker2 # [ 69.177493] nix-grpc-daemon[847]: ts=2026-09-20T13:28:48Z level=info event=scheduler_connected addr=lb:500515211lb # [ 69.146486] envoy[930]: conn peer=[2001:db8:1::1]:54162 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=185212worker2 # [ 69.180443] nix-grpc-daemon[847]: ts=2026-09-20T13:28:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5213lb # [ 69.151114] envoy[930]: conn peer=[2001:db8:1::1]:54174 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3715 tx=1894 ms=23565214worker1 # [ 69.153823] nix-grpc-daemon[848]: ts=2026-09-20T13:28:48Z level=info event=scheduler_connected addr=lb:500515215lb # [ 69.155178] envoy[930]: conn peer=[2001:db8:1::1]:54182 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=525216worker1 # [ 69.159434] nix-grpc-daemon[848]: ts=2026-09-20T13:28:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5217worker2 # [ 69.381252] nix-grpc-daemon[847]: ts=2026-09-20T13:28:49Z level=info event=scheduler_connected addr=lb:500515218worker2 # [ 69.382904] nix-grpc-daemon[847]: ts=2026-09-20T13:28:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5219worker1 # [ 69.360653] nix-grpc-daemon[848]: ts=2026-09-20T13:28:48Z level=info event=scheduler_connected addr=lb:500515220worker1 # [ 69.364673] nix-grpc-daemon[848]: ts=2026-09-20T13:28:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5221worker2 # [ 69.584170] nix-grpc-daemon[847]: ts=2026-09-20T13:28:49Z level=info event=scheduler_connected addr=lb:500515222worker2 # [ 69.585518] nix-grpc-daemon[847]: ts=2026-09-20T13:28:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5223worker1 # [ 69.565967] nix-grpc-daemon[848]: ts=2026-09-20T13:28:49Z level=info event=scheduler_connected addr=lb:500515224worker1 # [ 69.569168] nix-grpc-daemon[848]: ts=2026-09-20T13:28:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5225worker2 # [ 69.767540] nix-grpc-daemon[847]: ts=2026-09-20T13:28:49Z level=info event=scheduler_restarting5226worker2 # [ 69.795277] nix-grpc-daemon[847]: ts=2026-09-20T13:28:49Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5227worker1 # [ 69.768537] nix-grpc-daemon[848]: ts=2026-09-20T13:28:49Z level=info event=scheduler_connected addr=lb:500515228worker1 # [ 69.771796] nix-grpc-daemon[848]: ts=2026-09-20T13:28:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5229worker2 # [ 69.806641] nix-daemon[915]: reaped child process 917, status = succeeded5230worker1 # [ 69.969460] nix-grpc-daemon[848]: ts=2026-09-20T13:28:49Z level=info event=scheduler_connected addr=lb:500515231worker1 # [ 69.973347] nix-grpc-daemon[848]: ts=2026-09-20T13:28:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5232worker1 # [ 70.174317] nix-grpc-daemon[848]: ts=2026-09-20T13:28:49Z level=info event=scheduler_connected addr=lb:500515233worker1 # [ 70.178194] nix-grpc-daemon[848]: ts=2026-09-20T13:28:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5234worker1 # [ 70.379084] nix-grpc-daemon[848]: ts=2026-09-20T13:28:49Z level=info event=scheduler_connected addr=lb:500515235worker1 # [ 70.383207] nix-grpc-daemon[848]: ts=2026-09-20T13:28:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5236worker1 # [ 70.583735] nix-grpc-daemon[848]: ts=2026-09-20T13:28:50Z level=info event=scheduler_connected addr=lb:500515237worker1 # [ 70.587706] nix-grpc-daemon[848]: ts=2026-09-20T13:28:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5238worker2 # [ 70.622790] nix-grpc-daemon[847]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5239worker2 # [ 70.626336] nix-grpc-daemon[847]: E0000 00:00:1789910930.368546 890 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5240worker2 # [ 70.630920] nix-grpc-daemon[847]: E0000 00:00:1789910930.369062 890 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5241worker1 # [ 70.790338] nix-grpc-daemon[848]: ts=2026-09-20T13:28:50Z level=info event=scheduler_connected addr=lb:500515242worker1 # [ 70.793633] nix-grpc-daemon[848]: ts=2026-09-20T13:28:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5243worker2 # [ 71.010847] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5244worker2 # [ 71.012961] systemd[1]: Stopped Nix worker-protocol over gRPC.5245worker2 # [ 71.013848] systemd[1]: nix-grpc-daemon.service: Consumed 1.417s CPU time over 51.727s wall clock time, 50.3M memory peak, 56.1K incoming IP traffic, 129.4K outgoing IP traffic.5246worker2 # [ 71.018254] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5247worker2 # [ 71.020128] systemd[1]: Closed Nix worker-protocol over gRPC.5248worker2 # [ 71.020890] systemd[1]: nix-grpc-daemon.socket: Consumed 300us CPU time over 1min 275ms wall clock time, 256K memory peak, 71.8K incoming IP traffic, 60.1K outgoing IP traffic.5249worker1 # [ 70.991855] nix-grpc-daemon[848]: ts=2026-09-20T13:28:50Z level=info event=scheduler_connected addr=lb:500515250worker1 # [ 70.995858] nix-grpc-daemon[848]: ts=2026-09-20T13:28:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5251lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5252lb: must succeed: curl -sf localhost:9901/clusters5253lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5254lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.07 seconds)5255client: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25256worker1 # [ 71.196824] nix-grpc-daemon[848]: ts=2026-09-20T13:28:50Z level=info event=scheduler_connected addr=lb:500515257worker1 # [ 71.200791] nix-grpc-daemon[848]: ts=2026-09-20T13:28:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5258worker1 # [ 71.271072] nix-grpc-daemon[848]: ts=2026-09-20T13:28:50Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:489065259worker1 # [ 71.278373] nix-grpc-daemon[848]: ts=2026-09-20T13:28:50Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:489065260client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5261client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.24 seconds)5262worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5263worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds)5264worker2 # [ 71.396214] systemd[1]: Listening on Nix worker-protocol over gRPC.5265worker1 # [ 71.401853] nix-grpc-daemon[848]: ts=2026-09-20T13:28:51Z level=info event=scheduler_connected addr=lb:500515266worker1 # [ 71.406251] nix-grpc-daemon[848]: ts=2026-09-20T13:28:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5267worker1 # [ 71.437133] systemd[1]: Stopping Nix worker-protocol over gRPC...5268worker1 # [ 71.439862] nix-grpc-daemon[848]: ts=2026-09-20T13:28:51Z level=info event=signal_exit5269lb # [ 71.575184] niks3-server[1021]: 2026/09/20 13:28:51 INFO lead: acquired remote=192.168.1.3:523165270worker1 # [ 71.578064] nix-grpc-daemon[848]: ts=2026-09-20T13:28:51Z level=info event=scheduler_take_over5271worker1 # [ 71.579568] nix-grpc-daemon[848]: ts=2026-09-20T13:28:51Z level=info event=scheduler_yield5272lb # [ 71.583201] niks3-server[1021]: 2026/09/20 13:28:51 INFO lead: released remote=192.168.1.3:523165273worker1 # [ 71.606221] nix-grpc-daemon[848]: ts=2026-09-20T13:28:51Z level=info event=scheduler_connected addr=lb:500515274worker1 # [ 71.607824] nix-grpc-daemon[848]: ts=2026-09-20T13:28:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5275worker1 # [ 71.808874] nix-grpc-daemon[848]: ts=2026-09-20T13:28:51Z level=info event=scheduler_connected addr=lb:500515276worker1 # [ 71.810628] nix-grpc-daemon[848]: ts=2026-09-20T13:28:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5277worker2 # [ 72.003652] systemd[1]: Starting scheduler-token.service...5278worker1 # [ 72.011225] nix-grpc-daemon[848]: ts=2026-09-20T13:28:51Z level=info event=scheduler_connected addr=lb:500515279worker1 # [ 72.013092] nix-grpc-daemon[848]: ts=2026-09-20T13:28:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5280worker2 # [ 72.096871] systemd[1]: scheduler-token.service: Deactivated successfully.5281worker2 # [ 72.100334] systemd[1]: Finished scheduler-token.service.5282worker2 # [ 72.105124] systemd[1]: scheduler-token.service: Consumed 35ms CPU time over 95ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5283worker2 # [ 72.108476] systemd[1]: Starting Nix worker-protocol over gRPC...5284worker1 # [ 72.213820] nix-grpc-daemon[848]: ts=2026-09-20T13:28:51Z level=info event=scheduler_connected addr=lb:500515285worker1 # [ 72.215658] nix-grpc-daemon[848]: ts=2026-09-20T13:28:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5286worker2 # [ 72.268827] nix-grpc-daemon[1157]: ts=2026-09-20T13:28:52Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5287worker2 # [ 72.270580] nix-grpc-daemon[1157]: ts=2026-09-20T13:28:52Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5288worker2 # [ 72.273764] nix-grpc-daemon[1157]: ts=2026-09-20T13:28:52Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15289worker2 # [ 72.280462] nix-grpc-daemon[1157]: ts=2026-09-20T13:28:52Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5290worker2 # [ 72.285447] systemd[1]: Started Nix worker-protocol over gRPC.5291worker2 # [ 72.290339] nix-grpc-daemon[1157]: ts=2026-09-20T13:28:52Z level=info event=scheduler_take_over5292lb # [ 72.260823] niks3-server[1021]: 2026/09/20 13:28:52 INFO lead: acquired remote=192.168.1.4:471305293worker2 # [ 72.303421] nix-grpc-daemon[1157]: ts=2026-09-20T13:28:52Z level=info event=scheduler_connected addr=lb:500515294worker2 # [ 72.309981] nix-grpc-daemon[1157]: ts=2026-09-20T13:28:52Z level=info event=oidc_jwks provider=mock keys=15295worker2 # [ 72.311336] nix-grpc-daemon[1157]: ts=2026-09-20T13:28:52Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:570445296worker2 # [ 72.313068] nix-grpc-daemon[1157]: ts=2026-09-20T13:28:52Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05297worker1 # [ 72.416529] nix-grpc-daemon[848]: ts=2026-09-20T13:28:52Z level=info event=scheduler_connected addr=lb:500515298worker2 # [ 72.447930] nix-grpc-daemon[1157]: ts=2026-09-20T13:28:52Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:570445299worker2 # [ 72.452467] nix-grpc-daemon[1157]: ts=2026-09-20T13:28:52Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05300worker1 # [ 72.580425] nix-grpc-daemon[848]: ts=2026-09-20T13:28:52Z level=info event=scheduler_restarting5301worker2 # [ 72.633624] nix-grpc-daemon[1157]: ts=2026-09-20T13:28:52Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:570445302worker2 # [ 72.637174] nix-grpc-daemon[1157]: ts=2026-09-20T13:28:52Z level=info event=worker_gone addr=192.168.1.3:500515303worker1 # [ 72.617518] nix-grpc-daemon[848]: ts=2026-09-20T13:28:52Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5304worker1 # [ 72.635532] nix-daemon[905]: reaped child process 907, status = succeeded5305worker1 # [ 72.932969] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5306worker1 # [ 72.936490] systemd[1]: Stopped Nix worker-protocol over gRPC.5307worker1 # [ 72.940598] systemd[1]: nix-grpc-daemon.service: Consumed 2.672s CPU time over 53.808s wall clock time, 58.6M memory peak, 84.9K incoming IP traffic, 156.2K outgoing IP traffic.5308worker1 # [ 72.951862] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5309worker1 # [ 72.956170] systemd[1]: Closed Nix worker-protocol over gRPC.5310worker1 # [ 72.958993] systemd[1]: nix-grpc-daemon.socket: Consumed 317us CPU time over 1min 2.986s wall clock time, 256K memory peak, 73.5K incoming IP traffic, 61.7K outgoing IP traffic.5311lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5312lb: must succeed: curl -sf localhost:9901/clusters5313lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5314lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.07 seconds)5315client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25316worker2 # [ 73.273416] nix-grpc-daemon[1157]: ts=2026-09-20T13:28:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:586705317worker2 # [ 73.278681] nix-daemon[915]: accepted connection from pid 1157, user nix-grpc-daemon (trusted)5318worker2 # [ 73.288133] nix-grpc-daemon[1157]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 23 ms (attempt 1/5)5319worker2 # [ 73.313837] nix-grpc-daemon[1157]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 38 ms (attempt 2/5)5320worker2 # [ 73.357877] nix-grpc-daemon[1157]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 363 ms (attempt 3/5)5321worker2 # [ 73.726476] nix-grpc-daemon[1157]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 641 ms (attempt 4/5)5322worker2 # [ 74.371654] nix-grpc-daemon[1157]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5323client # copying 2 paths...5324worker2 # [ 74.389228] nix-grpc-daemon[1157]: ts=2026-09-20T13:28:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:586705325client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5326client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5327worker2 # [ 74.400445] nix-daemon[915]: accepted connection from pid 1157, user nix-grpc-daemon (trusted)5328worker2 # [ 74.414347] nix-grpc-daemon[1157]: ts=2026-09-20T13:28:54Z level=info event=niks3_push_started pid=12135329worker2 # [ 74.596545] nix-grpc-daemon[1213]: time=2026-09-20T13:28:54.343Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""5330worker2 # [ 74.755796] nix-daemon[915]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5331worker2 # [ 74.766527] nix-daemon[915]: reaped child process 1229, status = succeeded5332lb # [ 74.755665] niks3-server[1021]: 2026/09/20 13:28:54 INFO Received uploads request method=POST path=/api/pending_closures5333lb # [ 74.764397] niks3-server[1021]: 2026/09/20 13:28:54 INFO Received uploads request method=POST path=/api/pending_closures5334worker2 # [ 74.800827] nix-grpc-daemon[1213]: time=2026-09-20T13:28:54.547Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5335worker2 # [ 74.802339] nix-grpc-daemon[1213]: time=2026-09-20T13:28:54.549Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5336worker2 # [ 74.804284] nix-grpc-daemon[1213]: time=2026-09-20T13:28:54.551Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5337lb # [ 74.817787] niks3-server[1021]: 2026/09/20 13:28:54 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5338lb # [ 74.825451] niks3-server[1021]: 2026/09/20 13:28:54 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5339lb # [ 74.833705] niks3-server[1021]: 2026/09/20 13:28:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5340lb # [ 74.845218] niks3-server[1021]: 2026/09/20 13:28:54 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5341lb # [ 74.846939] niks3-server[1021]: 2026/09/20 13:28:54 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5342lb # [ 74.849894] niks3-server[1021]: 2026/09/20 13:28:54 INFO Signed narinfos id=25 count=15343lb # [ 74.851309] niks3-server[1021]: 2026/09/20 13:28:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5344lb # [ 74.853205] niks3-server[1021]: 2026/09/20 13:28:54 INFO Signed narinfos id=26 count=25345worker2 # [ 74.884181] nix-grpc-daemon[1213]: time=2026-09-20T13:28:54.630Z level=INFO msg="Uploading 3 narinfos"5346lb # [ 74.869175] niks3-server[1021]: 2026/09/20 13:28:54 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5347lb # [ 74.872437] niks3-server[1021]: 2026/09/20 13:28:54 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5348lb # [ 74.879546] niks3-server[1021]: 2026/09/20 13:28:54 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5349lb # [ 74.881841] niks3-server[1021]: 2026/09/20 13:28:54 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5350lb # [ 74.885220] niks3-server[1021]: 2026/09/20 13:28:54 INFO Completed upload id=255351lb # [ 74.886548] niks3-server[1021]: 2026/09/20 13:28:54 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5352lb # [ 74.889616] niks3-server[1021]: 2026/09/20 13:28:54 INFO Completed upload id=265353worker2 # [ 74.919554] nix-grpc-daemon[1213]: time=2026-09-20T13:28:54.666Z level=INFO msg="Upload complete. (318ms)"5354worker2 # [ 74.925156] nix-grpc-daemon[1157]: ts=2026-09-20T13:28:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58670 duration_s=0 paths=2 nar_bytes_in=3125355worker2 # [ 74.936619] nix-daemon[915]: reaped child process 1211, status = succeeded5356client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.90 seconds)5357worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5358worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.03 seconds)5359worker1 # [ 75.021698] systemd[1]: Listening on Nix worker-protocol over gRPC.5360worker1 # [ 75.028894] systemd[1]: Starting Nix worker-protocol over gRPC...5361worker1 # [ 75.184537] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:54Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5362worker1 # [ 75.186332] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:54Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5363worker1 # [ 75.189939] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:54Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15364worker1 # [ 75.196722] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:54Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5365worker1 # [ 75.198291] systemd[1]: Started Nix worker-protocol over gRPC.5366lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5367lb: must succeed: curl -sf localhost:9901/clusters5368worker1 # [ 75.217323] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:54Z level=info event=scheduler_connected addr=lb:500515369worker2 # [ 75.249400] nix-grpc-daemon[1157]: ts=2026-09-20T13:28:54Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:570445370worker2 # [ 75.253421] nix-grpc-daemon[1157]: ts=2026-09-20T13:28:54Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05371lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5372lb: must succeed: curl -sf localhost:9901/clusters5373lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5374lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.14 seconds)5375(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 9.45 seconds)5376subtest: scheduler down: the other node takes the lock and keeps it5377lb: must succeed: curl -sf localhost:9901/clusters5378lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5379lb: must succeed: curl -sf localhost:9901/clusters5380lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5381lb: must succeed: curl -sf localhost:9901/clusters5382lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5383lb: must succeed: curl -sf localhost:9901/clusters5384lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5385worker2 # [ 76.577850] systemd[1]: Stopping Nix worker-protocol over gRPC...5386worker2 # [ 77.287508] nix-grpc-daemon[1157]: ts=2026-09-20T13:28:57Z level=info event=signal_exit5387worker2 # [ 77.290704] nix-grpc-daemon[1157]: ts=2026-09-20T13:28:57Z level=info event=scheduler_yield5388lb # [ 77.263801] niks3-server[1021]: 2026/09/20 13:28:57 INFO lead: released remote=192.168.1.4:471305389worker1 # [ 77.268449] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:56Z level=info event=scheduler_restarting addr=lb:500515390worker2 # [ 77.298197] nix-grpc-daemon[1157]: ts=2026-09-20T13:28:57Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:570445391worker2 # [ 77.300137] nix-grpc-daemon[1157]: ts=2026-09-20T13:28:57Z level=info event=worker_gone addr=192.168.1.3:500515392worker2 # [ 77.301794] nix-grpc-daemon[1157]: ts=2026-09-20T13:28:57Z level=info event=scheduler_restarting addr=lb:500515393worker2 # [ 77.303035] nix-grpc-daemon[1157]: ts=2026-09-20T13:28:57Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:570445394worker2 # [ 77.304385] nix-grpc-daemon[1157]: ts=2026-09-20T13:28:57Z level=info event=worker_gone addr=192.168.1.4:500515395worker1 # [ 77.468979] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:57Z level=info event=scheduler_connected addr=lb:500515396worker2 # [ 77.500843] nix-grpc-daemon[1157]: ts=2026-09-20T13:28:57Z level=info event=scheduler_connected addr=lb:500515397worker2 # [ 77.502616] nix-grpc-daemon[1157]: ts=2026-09-20T13:28:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5398worker1 # [ 77.473310] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5399worker2 # [ 77.703299] nix-grpc-daemon[1157]: ts=2026-09-20T13:28:57Z level=info event=scheduler_connected addr=lb:500515400worker2 # [ 77.705034] nix-grpc-daemon[1157]: ts=2026-09-20T13:28:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5401worker1 # [ 77.673929] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:57Z level=info event=scheduler_connected addr=lb:500515402worker1 # [ 77.678216] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5403worker2 # [ 77.905638] nix-grpc-daemon[1157]: ts=2026-09-20T13:28:57Z level=info event=scheduler_connected addr=lb:500515404worker2 # [ 77.907367] nix-grpc-daemon[1157]: ts=2026-09-20T13:28:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5405worker1 # [ 77.878510] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:57Z level=info event=scheduler_connected addr=lb:500515406worker1 # [ 77.880293] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5407worker2 # [ 78.108676] nix-grpc-daemon[1157]: ts=2026-09-20T13:28:57Z level=info event=scheduler_connected addr=lb:500515408worker1 # [ 78.081472] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:57Z level=info event=scheduler_connected addr=lb:500515409worker2 # [ 78.112676] nix-grpc-daemon[1157]: ts=2026-09-20T13:28:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5410worker1 # [ 78.085455] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5411worker2 # [ 78.291743] nix-grpc-daemon[1157]: ts=2026-09-20T13:28:58Z level=info event=scheduler_restarting5412worker2 # [ 78.302805] nix-daemon[915]: reaped child process 1206, status = succeeded5413worker2 # [ 78.311444] nix-grpc-daemon[1157]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5414worker1 # [ 78.283348] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:57Z level=info event=scheduler_connected addr=lb:500515415worker2 # [ 78.313280] nix-grpc-daemon[1157]: E0000 00:00:1789910938.058532 1200 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5416worker1 # [ 78.285066] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5417worker2 # [ 78.315212] nix-grpc-daemon[1157]: E0000 00:00:1789910938.060344 1200 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5418worker2 # [ 78.321217] nix-grpc-daemon[1157]: ts=2026-09-20T13:28:58Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5419worker1 # [ 78.486195] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:58Z level=info event=scheduler_connected addr=lb:500515420worker1 # [ 78.490276] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5421worker1 # [ 78.690988] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:58Z level=info event=scheduler_connected addr=lb:500515422worker1 # [ 78.695219] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5423worker1 # [ 78.895486] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:58Z level=info event=scheduler_connected addr=lb:500515424worker1 # [ 78.897353] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5425worker1 # [ 79.097921] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:58Z level=info event=scheduler_connected addr=lb:500515426worker1 # [ 79.099994] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5427lb # [ 79.136756] envoy[930]: conn peer=[2001:db8:1::4]:43110 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=38106 tx=1973 ms=502545428lb # [ 79.140635] envoy[930]: conn peer=[2001:db8:1::1]:54184 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=215429lb # [ 79.144060] envoy[930]: conn peer=[2001:db8:1::3]:39454 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=12278 tx=2587 ms=522495430lb # [ 79.147635] envoy[930]: conn peer=[2001:db8:1::1]:39664 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1007 tx=282 ms=16635431lb # [ 79.151097] envoy[930]: conn peer=[2001:db8:1::4]:53938 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=4478 tx=326 ms=60195432worker1 # [ 79.301076] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:58Z level=info event=scheduler_connected addr=lb:500515433worker1 # [ 79.302901] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5434worker1 # [ 79.503475] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:59Z level=info event=scheduler_connected addr=lb:500515435worker1 # [ 79.505331] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5436worker1 # [ 79.706281] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:59Z level=info event=scheduler_connected addr=lb:500515437worker1 # [ 79.710284] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5438worker1 # [ 79.910966] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:59Z level=info event=scheduler_connected addr=lb:500515439worker1 # [ 79.914910] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5440worker1 # [ 80.115715] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:59Z level=info event=scheduler_connected addr=lb:500515441worker1 # [ 80.120308] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5442lb # [ 80.207205] niks3-server[1021]: 2026/09/20 13:29:00 INFO lead: acquired remote=192.168.1.3:453945443worker1 # [ 80.210386] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:59Z level=info event=scheduler_take_over5444worker2 # [ 80.311521] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5445worker2 # [ 80.313867] systemd[1]: Stopped Nix worker-protocol over gRPC.5446worker2 # [ 80.315824] systemd[1]: nix-grpc-daemon.service: Consumed 287ms CPU time over 8.202s wall clock time, 30.6M memory peak, 19.3K incoming IP traffic, 26.1K outgoing IP traffic.5447worker2 # [ 80.320360] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5448worker2 # [ 80.322075] systemd[1]: Closed Nix worker-protocol over gRPC.5449worker2 # [ 80.323352] systemd[1]: nix-grpc-daemon.socket: Consumed 311us CPU time over 8.930s wall clock time, 256K memory peak, 19.4K incoming IP traffic, 13.9K outgoing IP traffic.5450worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5451worker1 # [ 80.323284] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:59Z level=info event=scheduler_connected addr=lb:500515452worker1 # [ 80.326890] nix-grpc-daemon[1272]: ts=2026-09-20T13:28:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5453worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.07 seconds)5454worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5455worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5456worker1 # [ 80.523323] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:00Z level=info event=scheduler_connected addr=lb:500515457worker1 # [ 80.525297] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5458worker1 # [ 80.726310] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:00Z level=info event=scheduler_connected addr=lb:500515459worker1 # [ 80.733720] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:00Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:544285460worker1 # [ 80.738083] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:00Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05461worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5462worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5463client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&25464worker1 # [ 81.659516] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:544305465worker1 # [ 81.670240] nix-daemon[905]: accepted connection from pid 1272, user nix-grpc-daemon (trusted)5466client # don't know how to build these paths:5467client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5468worker1 # [ 81.684809] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544305469worker1 # [ 81.695513] nix-grpc-daemon[1272]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 82 ms (attempt 1/5)5470worker1 # [ 81.783307] nix-grpc-daemon[1272]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 97 ms (attempt 2/5)5471worker1 # [ 81.886437] nix-grpc-daemon[1272]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 170 ms (attempt 3/5)5472worker1 # [ 82.062607] nix-grpc-daemon[1272]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 195 ms (attempt 4/5)5473worker1 # [ 82.263936] nix-grpc-daemon[1272]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5474worker1 # [ 82.291556] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:01Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05475worker1 # [ 82.296827] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:01Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=15476worker1 # [ 82.336478] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544305477worker1 # [ 82.349081] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544305478client # copying 1 paths to 'lb:50051'...5479client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5480worker1 # [ 82.360469] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544305481worker1 # [ 82.371101] nix-daemon[905]: accepted connection from pid 1272, user nix-grpc-daemon (trusted)5482worker1 # [ 82.390212] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:02Z level=info event=niks3_push_started pid=13425483worker1 # [ 82.547317] nix-grpc-daemon[1342]: time=2026-09-20T13:29:02.160Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""5484worker1 # [ 82.693363] nix-daemon[905]: accepted connection from pid 1355, user nix-grpc-daemon (trusted)5485worker1 # [ 82.704840] nix-daemon[905]: reaped child process 1358, status = succeeded5486worker1 # [ 82.848209] nix-daemon[905]: accepted connection from pid 1360, user nix-grpc-daemon (trusted)5487worker1 # [ 82.860590] nix-daemon[905]: reaped child process 1363, status = succeeded5488lb # [ 82.874222] niks3-server[1021]: 2026/09/20 13:29:02 INFO Received uploads request method=POST path=/api/pending_closures5489worker1 # [ 82.883075] nix-grpc-daemon[1342]: time=2026-09-20T13:29:02.495Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5490worker1 # [ 82.884812] nix-grpc-daemon[1342]: time=2026-09-20T13:29:02.495Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5491lb # [ 82.914659] niks3-server[1021]: 2026/09/20 13:29:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5492lb # [ 82.921956] niks3-server[1021]: 2026/09/20 13:29:02 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5493lb # [ 82.932866] niks3-server[1021]: 2026/09/20 13:29:02 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5494lb # [ 82.942375] niks3-server[1021]: 2026/09/20 13:29:02 INFO Signed narinfos id=27 count=15495worker1 # [ 82.945488] nix-grpc-daemon[1342]: time=2026-09-20T13:29:02.558Z level=INFO msg="Uploading 1 narinfos"5496lb # [ 82.952700] niks3-server[1021]: 2026/09/20 13:29:02 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5497lb # [ 82.959829] niks3-server[1021]: 2026/09/20 13:29:02 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5498lb # [ 82.965044] niks3-server[1021]: 2026/09/20 13:29:02 INFO Completed upload id=275499worker1 # [ 82.965057] nix-grpc-daemon[1342]: time=2026-09-20T13:29:02.578Z level=INFO msg="Upload complete. (413ms)"5500worker1 # [ 82.967474] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54430 duration_s=0 paths=1 nar_bytes_in=5205501worker1 # [ 82.971163] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544305502worker1 # [ 82.975700] nix-daemon[905]: accepted connection from pid 1272, user nix-grpc-daemon (trusted)5503worker1 # [ 82.980451] nix-daemon[905]: reaped child process 1340, status = succeeded5504client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5505client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5506worker1 # [ 82.985332] nix-daemon[905]: accepted connection from pid 1272, user nix-grpc-daemon (trusted)5507client # farm-a> LOG-farm-a-on-standby5508worker1 # [ 84.199199] nix-daemon[905]: reaped child process 1367, status = succeeded5509worker1 # [ 84.348643] nix-daemon[905]: accepted connection from pid 1382, user nix-grpc-daemon (trusted)5510worker1 # [ 84.361814] nix-daemon[905]: reaped child process 1385, status = succeeded5511lb # [ 84.378534] niks3-server[1021]: 2026/09/20 13:29:04 INFO Received uploads request method=POST path=/api/pending_closures5512worker1 # [ 84.389628] nix-grpc-daemon[1342]: time=2026-09-20T13:29:04.002Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5513worker1 # [ 84.391198] nix-grpc-daemon[1342]: time=2026-09-20T13:29:04.004Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5514lb # [ 84.421723] niks3-server[1021]: 2026/09/20 13:29:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5515worker1 # [ 84.427537] nix-grpc-daemon[1342]: time=2026-09-20T13:29:04.040Z level=INFO msg="Uploading 1 narinfos"5516lb # [ 84.427769] niks3-server[1021]: 2026/09/20 13:29:04 INFO Signed narinfos id=28 count=15517lb # [ 84.440273] niks3-server[1021]: 2026/09/20 13:29:04 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5518lb # [ 84.445265] niks3-server[1021]: 2026/09/20 13:29:04 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5519lb # [ 84.449471] niks3-server[1021]: 2026/09/20 13:29:04 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5520lb # [ 84.457602] niks3-server[1021]: 2026/09/20 13:29:04 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5521lb # [ 84.462384] niks3-server[1021]: 2026/09/20 13:29:04 INFO Completed upload id=285522worker1 # [ 84.462530] nix-grpc-daemon[1342]: time=2026-09-20T13:29:04.075Z level=INFO msg="Upload complete. (270ms)"5523lb # [ 84.465316] niks3-server[1021]: 2026/09/20 13:29:04 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5524worker1 # [ 84.471781] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54430 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15525worker1 # [ 84.486710] nix-daemon[905]: reaped child process 1365, status = succeeded5526worker1 # [ 84.488704] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:04Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05527worker1 # [ 84.490787] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:04Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=15528worker1 # [ 84.532865] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544305529worker1 # [ 84.545144] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544305530worker1 # [ 84.558054] nix-grpc-daemon[1342]: time=2026-09-20T13:29:04.171Z level=INFO msg="All 1 paths already cached"5531client # copying 1 paths to 'lb:50051'...5532client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5533worker1 # [ 84.566061] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544305534worker1 # [ 84.576181] nix-daemon[905]: accepted connection from pid 1272, user nix-grpc-daemon (trusted)5535worker1 # [ 84.742781] nix-daemon[905]: accepted connection from pid 1391, user nix-grpc-daemon (trusted)5536worker1 # [ 84.756203] nix-daemon[905]: reaped child process 1394, status = succeeded5537worker1 # [ 84.908475] nix-daemon[905]: accepted connection from pid 1396, user nix-grpc-daemon (trusted)5538worker1 # [ 84.921867] nix-daemon[905]: reaped child process 1399, status = succeeded5539lb # [ 84.936696] niks3-server[1021]: 2026/09/20 13:29:04 INFO Received uploads request method=POST path=/api/pending_closures5540worker1 # [ 84.944917] nix-grpc-daemon[1342]: time=2026-09-20T13:29:04.558Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5541worker1 # [ 84.946854] nix-grpc-daemon[1342]: time=2026-09-20T13:29:04.559Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5542lb # [ 84.965584] niks3-server[1021]: 2026/09/20 13:29:04 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5543lb # [ 84.978151] niks3-server[1021]: 2026/09/20 13:29:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5544lb # [ 84.985065] niks3-server[1021]: 2026/09/20 13:29:05 INFO Signed narinfos id=29 count=15545worker1 # [ 84.988547] nix-grpc-daemon[1342]: time=2026-09-20T13:29:04.601Z level=INFO msg="Uploading 1 narinfos"5546lb # [ 84.990282] niks3-server[1021]: 2026/09/20 13:29:05 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5547lb # [ 85.001700] niks3-server[1021]: 2026/09/20 13:29:05 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5548worker1 # [ 85.008821] nix-grpc-daemon[1342]: time=2026-09-20T13:29:04.621Z level=INFO msg="Upload complete. (421ms)"5549lb # [ 85.008826] niks3-server[1021]: 2026/09/20 13:29:05 INFO Completed upload id=295550worker1 # [ 85.011206] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54430 duration_s=0 paths=1 nar_bytes_in=6405551lb # [ 85.011933] niks3-server[1021]: 2026/09/20 13:29:05 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5552worker1 # [ 85.016672] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544305553worker1 # [ 85.026561] nix-daemon[905]: accepted connection from pid 1272, user nix-grpc-daemon (trusted)5554worker1 # [ 85.029572] nix-daemon[905]: reaped child process 1389, status = succeeded5555client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5556worker1 # [ 85.036379] nix-daemon[905]: accepted connection from pid 1272, user nix-grpc-daemon (trusted)5557client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5558worker1 # [ 85.849536] nix-daemon[905]: reaped child process 1403, status = succeeded5559client # farm-b> LOG-farm-b-on-standby5560worker1 # [ 85.991825] nix-daemon[905]: accepted connection from pid 1418, user nix-grpc-daemon (trusted)5561worker1 # [ 86.005634] nix-daemon[905]: reaped child process 1421, status = succeeded5562lb # [ 86.021732] niks3-server[1021]: 2026/09/20 13:29:06 INFO Received uploads request method=POST path=/api/pending_closures5563worker1 # [ 86.029967] nix-grpc-daemon[1342]: time=2026-09-20T13:29:05.643Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5564worker1 # [ 86.031617] nix-grpc-daemon[1342]: time=2026-09-20T13:29:05.644Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"5565lb # [ 86.064406] niks3-server[1021]: 2026/09/20 13:29:06 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst5566lb # [ 86.071147] niks3-server[1021]: 2026/09/20 13:29:06 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5567lb # [ 86.079972] niks3-server[1021]: 2026/09/20 13:29:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5568worker1 # [ 86.086715] nix-grpc-daemon[1342]: time=2026-09-20T13:29:05.699Z level=INFO msg="Uploading 1 narinfos"5569lb # [ 86.086616] niks3-server[1021]: 2026/09/20 13:29:06 INFO Signed narinfos id=30 count=15570lb # [ 86.090509] niks3-server[1021]: 2026/09/20 13:29:06 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls5571lb # [ 86.101480] niks3-server[1021]: 2026/09/20 13:29:06 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5572lb # [ 86.107211] niks3-server[1021]: 2026/09/20 13:29:06 INFO Completed upload id=305573worker1 # [ 86.107361] nix-grpc-daemon[1342]: time=2026-09-20T13:29:05.720Z level=INFO msg="Upload complete. (265ms)"5574lb # [ 86.109700] niks3-server[1021]: 2026/09/20 13:29:06 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo5575worker1 # [ 86.118205] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54430 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=15576worker1 # [ 86.126313] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:05Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=05577worker1 # [ 86.130235] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:05Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=15578worker1 # [ 86.139666] nix-daemon[905]: reaped child process 1401, status = succeeded5579worker1 # [ 86.169476] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544305580worker1 # [ 86.182148] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544305581worker1 # [ 86.195701] nix-grpc-daemon[1342]: time=2026-09-20T13:29:05.808Z level=INFO msg="All 2 paths already cached"5582client # copying 1 paths to 'lb:50051'...5583client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...5584worker1 # [ 86.204127] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544305585worker1 # [ 86.214149] nix-daemon[905]: accepted connection from pid 1272, user nix-grpc-daemon (trusted)5586worker1 # [ 86.373750] nix-daemon[905]: accepted connection from pid 1427, user nix-grpc-daemon (trusted)5587worker1 # [ 86.387573] nix-daemon[905]: reaped child process 1430, status = succeeded5588worker1 # [ 86.542666] nix-daemon[905]: accepted connection from pid 1432, user nix-grpc-daemon (trusted)5589worker1 # [ 86.556107] nix-daemon[905]: reaped child process 1435, status = succeeded5590lb # [ 86.571361] niks3-server[1021]: 2026/09/20 13:29:06 INFO Received uploads request method=POST path=/api/pending_closures5591worker1 # [ 86.583570] nix-grpc-daemon[1342]: time=2026-09-20T13:29:06.196Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5592worker1 # [ 86.584928] nix-grpc-daemon[1342]: time=2026-09-20T13:29:06.196Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"5593lb # [ 86.600416] niks3-server[1021]: 2026/09/20 13:29:06 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst5594lb # [ 86.614243] niks3-server[1021]: 2026/09/20 13:29:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5595lb # [ 86.621100] niks3-server[1021]: 2026/09/20 13:29:06 INFO Signed narinfos id=31 count=15596worker1 # [ 86.623914] nix-grpc-daemon[1342]: time=2026-09-20T13:29:06.236Z level=INFO msg="Uploading 1 narinfos"5597lb # [ 86.623917] niks3-server[1021]: 2026/09/20 13:29:06 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls5598lb # [ 86.635132] niks3-server[1021]: 2026/09/20 13:29:06 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5599lb # [ 86.641570] niks3-server[1021]: 2026/09/20 13:29:06 INFO Completed upload id=315600worker1 # [ 86.641180] nix-grpc-daemon[1342]: time=2026-09-20T13:29:06.254Z level=INFO msg="Upload complete. (416ms)"5601lb # [ 86.643169] niks3-server[1021]: 2026/09/20 13:29:06 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo5602worker1 # [ 86.644795] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54430 duration_s=0 paths=1 nar_bytes_in=7685603worker1 # [ 86.650314] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544305604worker1 # [ 86.657390] nix-daemon[905]: accepted connection from pid 1272, user nix-grpc-daemon (trusted)5605worker1 # [ 86.665736] nix-daemon[905]: reaped child process 1425, status = succeeded5606worker1 # [ 86.668958] nix-daemon[905]: accepted connection from pid 1272, user nix-grpc-daemon (trusted)5607client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...5608client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5609client # farm-top> LOG-farm-top-on-standby5610worker1 # [ 87.294382] nix-daemon[905]: reaped child process 1439, status = succeeded5611worker1 # [ 87.451095] nix-daemon[905]: accepted connection from pid 1454, user nix-grpc-daemon (trusted)5612worker1 # [ 87.463840] nix-daemon[905]: reaped child process 1457, status = succeeded5613lb # [ 87.480366] niks3-server[1021]: 2026/09/20 13:29:07 INFO Received uploads request method=POST path=/api/pending_closures5614worker1 # [ 87.488742] nix-grpc-daemon[1342]: time=2026-09-20T13:29:07.101Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5615worker1 # [ 87.490222] nix-grpc-daemon[1342]: time=2026-09-20T13:29:07.103Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"5616lb # [ 87.516985] niks3-server[1021]: 2026/09/20 13:29:07 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5617lb # [ 87.522210] niks3-server[1021]: 2026/09/20 13:29:07 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst5618lb # [ 87.535274] niks3-server[1021]: 2026/09/20 13:29:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5619lb # [ 87.540950] niks3-server[1021]: 2026/09/20 13:29:07 INFO Signed narinfos id=32 count=15620worker1 # [ 87.543691] nix-grpc-daemon[1342]: time=2026-09-20T13:29:07.156Z level=INFO msg="Uploading 1 narinfos"5621lb # [ 87.543924] niks3-server[1021]: 2026/09/20 13:29:07 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls5622lb # [ 87.555463] niks3-server[1021]: 2026/09/20 13:29:07 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5623lb # [ 87.562242] niks3-server[1021]: 2026/09/20 13:29:07 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo5624lb # [ 87.565256] niks3-server[1021]: 2026/09/20 13:29:07 INFO Completed upload id=325625worker1 # [ 87.565365] nix-grpc-daemon[1342]: time=2026-09-20T13:29:07.178Z level=INFO msg="Upload complete. (289ms)"5626worker1 # [ 87.573354] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54430 duration_s=0 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=15627worker1 # [ 87.586253] nix-daemon[905]: reaped child process 1437, status = succeeded5628client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&2, in 6.14 seconds)5629client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath5630client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath, in 0.17 seconds)5631worker2 # [ 87.872669] systemd[1]: Listening on Nix worker-protocol over gRPC.5632worker2 # [ 87.879263] systemd[1]: Starting scheduler-token.service...5633worker2 # [ 87.950102] systemd[1]: scheduler-token.service: Deactivated successfully.5634worker2 # [ 87.951459] systemd[1]: Finished scheduler-token.service.5635worker2 # [ 87.954101] systemd[1]: scheduler-token.service: Consumed 21ms CPU time over 71ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5636worker2 # [ 87.958957] systemd[1]: Starting Nix worker-protocol over gRPC...5637worker2 # [ 88.117119] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:07Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5638worker2 # [ 88.118950] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:07Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5639worker2 # [ 88.122338] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:07Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15640worker2 # [ 88.130817] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:07Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5641worker2 # [ 88.136171] systemd[1]: Started Nix worker-protocol over gRPC.5642lb: must succeed: curl -sf localhost:9901/clusters5643worker2 # [ 88.153151] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:07Z level=info event=scheduler_connected addr=lb:500515644worker1 # [ 88.133210] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:07Z level=info event=oidc_jwks provider=mock keys=15645worker1 # [ 88.137199] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:07Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:544285646worker1 # [ 88.141114] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:07Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05647lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5648worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5649worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5650lb: must succeed: curl -sf localhost:9901/clusters5651lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5652lb: must succeed: curl -sf localhost:9901/clusters5653lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5654worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5655worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5656(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.05 seconds)5657subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5658client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs5659client # [ 88.458721] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs.5660client # Running as unit: rs.service; invocation ID: f800769eef564fb68e5a7f7b84569ad35661client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs, in 0.07 seconds)5662worker2 # [ 88.665102] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:08Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:552485663worker2 # [ 88.671342] nix-daemon[915]: accepted connection from pid 1253, user nix-grpc-daemon (trusted)5664client # [ 88.656897] nix[1167]: don't know how to build these paths:5665client # [ 88.657882] nix[1167]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5666worker1 # [ 88.658198] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:08Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=05667worker1 # [ 88.662181] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:08Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=15668worker1 # [ 88.701412] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544305669worker1 # [ 88.713920] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544305670client # [ 88.733320] nix[1167]: copying 1 paths to 'lb:50051'...5671worker1 # [ 88.725947] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544305672client # [ 88.735959] nix[1167]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...5673worker1 # [ 88.738861] nix-daemon[905]: accepted connection from pid 1272, user nix-grpc-daemon (trusted)5674worker1 # [ 88.915130] nix-daemon[905]: accepted connection from pid 1470, user nix-grpc-daemon (trusted)5675worker1 # [ 88.927822] nix-daemon[905]: reaped child process 1473, status = succeeded5676worker1 # [ 89.085625] nix-daemon[905]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)5677worker1 # [ 89.098076] nix-daemon[905]: reaped child process 1478, status = succeeded5678lb # [ 89.109991] niks3-server[1021]: 2026/09/20 13:29:09 INFO Received uploads request method=POST path=/api/pending_closures5679worker1 # [ 89.118502] nix-grpc-daemon[1342]: time=2026-09-20T13:29:08.731Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5680worker1 # [ 89.120051] nix-grpc-daemon[1342]: time=2026-09-20T13:29:08.733Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"5681lb # [ 89.137771] envoy[930]: conn peer=[2001:db8:1::1]:39672 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9607 tx=3859 ms=59295682lb # [ 89.145722] niks3-server[1021]: 2026/09/20 13:29:09 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst5683lb # [ 89.155627] niks3-server[1021]: 2026/09/20 13:29:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5684worker1 # [ 89.164875] nix-grpc-daemon[1342]: time=2026-09-20T13:29:08.777Z level=INFO msg="Uploading 1 narinfos"5685lb # [ 89.164828] niks3-server[1021]: 2026/09/20 13:29:09 INFO Signed narinfos id=33 count=15686lb # [ 89.171525] niks3-server[1021]: 2026/09/20 13:29:09 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls5687lb # [ 89.184830] niks3-server[1021]: 2026/09/20 13:29:09 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5688lb # [ 89.190356] niks3-server[1021]: 2026/09/20 13:29:09 INFO Completed upload id=335689worker1 # [ 89.190572] nix-grpc-daemon[1342]: time=2026-09-20T13:29:08.803Z level=INFO msg="Upload complete. (439ms)"5690lb # [ 89.192129] niks3-server[1021]: 2026/09/20 13:29:09 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo5691worker1 # [ 89.194901] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54430 duration_s=0 paths=1 nar_bytes_in=4165692worker1 # [ 89.201316] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544305693worker1 # [ 89.210531] nix-daemon[905]: accepted connection from pid 1272, user nix-grpc-daemon (trusted)5694worker1 # [ 89.222172] nix-daemon[905]: reaped child process 1468, status = succeeded5695worker1 # [ 89.225196] nix-daemon[905]: accepted connection from pid 1272, user nix-grpc-daemon (trusted)5696client # [ 89.233538] nix[1167]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...5697lb: must succeed: curl -sf localhost:9901/clusters5698lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5699lb: must succeed: curl -sf localhost:9901/clusters5700lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5701lb: must succeed: curl -sf localhost:9901/clusters5702lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5703lb: must succeed: curl -sf localhost:9901/clusters5704lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5705worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5706worker2 # [ 91.328470] systemd-journald[286]: Received client request to rotate journal, rotating.5707worker2 # [ 91.348373] systemd-journald[286]: Vacuuming done, freed 0B of archived journals from /var/log/journal/002810d9aa2249079862513cd9323d7d.5708worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.07 seconds)5709worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5710worker1 # [ 91.238733] systemd[1]: Stopping Nix worker-protocol over gRPC...5711worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)5712??? Warning (UserWarning): sleep(): The 'secs' argument is deprecated. Use 'duration' instead.5713 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 5735714client: must succeed: sleep 5.05715worker1 # [ 92.210894] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:11Z level=info event=draining5716client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5717worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5718worker1 # [ 96.657230] nix-daemon[905]: reaped child process 1482, status = succeeded5719worker1 # [ 96.819409] nix-daemon[905]: accepted connection from pid 1522, user nix-grpc-daemon (trusted)5720worker1 # [ 96.831621] nix-daemon[905]: reaped child process 1525, status = succeeded5721lb # [ 96.845546] niks3-server[1021]: 2026/09/20 13:29:16 INFO Received uploads request method=POST path=/api/pending_closures5722worker1 # [ 96.856211] nix-grpc-daemon[1342]: time=2026-09-20T13:29:16.469Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5723worker1 # [ 96.857788] nix-grpc-daemon[1342]: time=2026-09-20T13:29:16.470Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"5724lb # [ 96.892525] niks3-server[1021]: 2026/09/20 13:29:16 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5725lb # [ 96.897525] niks3-server[1021]: 2026/09/20 13:29:16 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5726lb # [ 96.907509] niks3-server[1021]: 2026/09/20 13:29:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5727lb # [ 96.912353] niks3-server[1021]: 2026/09/20 13:29:16 INFO Signed narinfos id=34 count=15728worker1 # [ 96.915297] nix-grpc-daemon[1342]: time=2026-09-20T13:29:16.528Z level=INFO msg="Uploading 1 narinfos"5729lb # [ 96.918918] niks3-server[1021]: 2026/09/20 13:29:16 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls5730lb # [ 96.924163] niks3-server[1021]: 2026/09/20 13:29:16 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5731lb # [ 96.928566] niks3-server[1021]: 2026/09/20 13:29:16 INFO Completed upload id=345732lb # [ 96.929768] niks3-server[1021]: 2026/09/20 13:29:16 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo5733worker1 # [ 96.928619] nix-grpc-daemon[1342]: time=2026-09-20T13:29:16.541Z level=INFO msg="Upload complete. (289ms)"5734worker1 # [ 96.935940] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54430 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=15735worker1 # [ 96.948802] nix-daemon[905]: reaped child process 1480, status = succeeded5736client # [ 96.980293] systemd[1]: rs.service: Deactivated successfully.5737client # [ 96.984415] systemd[1]: rs.service: Consumed 109ms CPU time over 8.524s wall clock time, 37.1M memory peak, 8.4K incoming IP traffic, 8.9K outgoing IP traffic.5738worker1 # [ 97.212782] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:16Z level=info event=signal_exit5739worker1 # [ 97.227158] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:16Z level=info event=scheduler_yield5740lb # [ 97.229667] niks3-server[1021]: 2026/09/20 13:29:17 INFO lead: released remote=192.168.1.3:453945741worker1 # [ 97.231165] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:16Z level=info event=scheduler_restarting addr=lb:500515742worker2 # [ 97.261540] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:17Z level=info event=scheduler_restarting addr=lb:500515743worker1 # [ 97.235630] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:16Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:544285744worker1 # [ 97.238943] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:16Z level=info event=worker_gone addr=192.168.1.3:500515745worker1 # [ 97.241984] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:16Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:544285746worker1 # [ 97.245327] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:16Z level=info event=worker_gone addr=192.168.1.4:500515747worker1 # [ 97.432132] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:17Z level=info event=scheduler_connected addr=lb:500515748worker2 # [ 97.462407] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:17Z level=info event=scheduler_connected addr=lb:500515749worker2 # [ 97.467167] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:17Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5750worker1 # [ 97.438130] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:17Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5751worker2 # [ 97.668661] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:17Z level=info event=scheduler_connected addr=lb:500515752worker1 # [ 97.638870] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:17Z level=info event=scheduler_connected addr=lb:500515753worker2 # [ 97.671772] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5754worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 1.10 seconds)5755worker1 # [ 97.643178] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5756client: waiting for success: ! systemctl is-active rs5757client: (finished: waiting for success: ! systemctl is-active rs, in 0.03 seconds)5758client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5759client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.03 seconds)5760client: must fail: journalctl -u rs | grep -q 'reconnecting'5761client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.02 seconds)5762lb: must succeed: curl -sf localhost:9901/clusters5763lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5764worker2 # [ 97.872791] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:17Z level=info event=scheduler_connected addr=lb:500515765worker1 # [ 97.843956] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:17Z level=info event=scheduler_connected addr=lb:500515766worker1 # [ 97.848059] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5767worker2 # [ 97.877661] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5768worker1 # [ 98.049811] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:17Z level=info event=scheduler_connected addr=lb:500515769worker2 # [ 98.078895] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:17Z level=info event=scheduler_connected addr=lb:500515770worker2 # [ 98.083387] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5771worker1 # [ 98.053725] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5772worker2 # [ 98.142303] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:17Z level=info event=scheduler_take_over5773lb # [ 98.112811] niks3-server[1021]: 2026/09/20 13:29:18 INFO lead: acquired remote=192.168.1.4:344585774worker1 # [ 98.227805] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:17Z level=info event=scheduler_restarting5775worker1 # [ 98.240942] nix-daemon[905]: reaped child process 1333, status = succeeded5776worker2 # [ 98.285740] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:18Z level=info event=scheduler_connected addr=lb:500515777worker1 # [ 98.258754] nix-grpc-daemon[1272]: ts=2026-09-20T13:29:17Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5778worker2 # [ 98.289137] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5779worker2 # [ 98.486676] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:18Z level=info event=scheduler_connected addr=lb:500515780worker2 # [ 98.488503] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5781worker1 # [ 98.616627] nix-grpc-daemon[1272]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5782worker1 # [ 98.618096] nix-grpc-daemon[1272]: E0000 00:00:1789910958.229426 1315 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5783worker1 # [ 98.619970] nix-grpc-daemon[1272]: E0000 00:00:1789910958.229867 1315 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5784worker2 # [ 98.690083] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:18Z level=info event=scheduler_connected addr=lb:500515785worker2 # [ 98.694166] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5786lb: must succeed: curl -sf localhost:9901/clusters5787lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5788worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5789worker2 # [ 98.895615] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:18Z level=info event=scheduler_connected addr=lb:500515790worker2 # [ 98.900655] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5791worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5792worker2 # [ 99.101738] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:18Z level=info event=scheduler_connected addr=lb:500515793worker2 # [ 99.105772] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5794lb # [ 99.137651] envoy[930]: conn peer=[2001:db8:1::1]:34656 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2930 tx=1331 ms=83145795lb # [ 99.141361] envoy[930]: conn peer=[2001:db8:1::3]:52998 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=5568 tx=1298 ms=230445796worker2 # [ 99.308325] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:19Z level=info event=scheduler_connected addr=lb:500515797worker2 # [ 99.311885] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5798worker2 # [ 99.509416] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:19Z level=info event=scheduler_connected addr=lb:500515799worker2 # [ 99.522138] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:19Z level=info event=oidc_jwks provider=mock keys=15800worker2 # [ 99.525884] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:19Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:376705801worker2 # [ 99.529912] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:19Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05802lb: must succeed: curl -sf localhost:9901/clusters5803lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5804worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5805worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5806worker1 # [ 100.289145] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5807worker1 # [ 100.293483] systemd[1]: Stopped Nix worker-protocol over gRPC.5808worker1 # [ 100.295517] systemd[1]: nix-grpc-daemon.service: Consumed 1.308s CPU time over 25.260s wall clock time, 33.6M memory peak, 48.9K incoming IP traffic, 90.1K outgoing IP traffic.5809worker1 # [ 100.311387] systemd[1]: Starting Nix worker-protocol over gRPC...5810worker1 # [ 100.472863] nix-grpc-daemon[1532]: ts=2026-09-20T13:29:20Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5811worker1 # [ 100.474729] nix-grpc-daemon[1532]: ts=2026-09-20T13:29:20Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5812worker1 # [ 100.477867] nix-grpc-daemon[1532]: ts=2026-09-20T13:29:20Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15813worker1 # [ 100.484563] nix-grpc-daemon[1532]: ts=2026-09-20T13:29:20Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5814worker1 # [ 100.487982] systemd[1]: Started Nix worker-protocol over gRPC.5815worker1 # [ 100.501676] nix-grpc-daemon[1532]: ts=2026-09-20T13:29:20Z level=info event=scheduler_connected addr=lb:500515816worker2 # [ 100.534819] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:20Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:376705817worker2 # [ 100.538624] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:20Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05818lb: must succeed: curl -sf localhost:9901/clusters5819lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5820worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5821worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)5822(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.75 seconds)5823subtest: scheduler restart mid-build: build finishes, no double build5824client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid5825client # [ 101.208627] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid.5826client # Running as unit: mid.service; invocation ID: 8e24023444194fb6a91840e45b2aa3d85827client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid, in 0.07 seconds)5828worker1 # [ 101.394962] nix-grpc-daemon[1532]: ts=2026-09-20T13:29:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:471705829worker1 # [ 101.403814] nix-daemon[905]: accepted connection from pid 1532, user nix-grpc-daemon (trusted)5830client # [ 101.428370] nix[1202]: don't know how to build these paths:5831client # [ 101.429333] nix[1202]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5832worker2 # [ 101.455871] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:21Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05833worker2 # [ 101.458272] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:21Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=15834worker2 # [ 101.499860] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:552485835worker2 # [ 101.506844] nix-grpc-daemon[1253]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 67 ms (attempt 1/5)5836worker2 # [ 101.576758] nix-grpc-daemon[1253]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 122 ms (attempt 2/5)5837worker2 # [ 101.701588] nix-grpc-daemon[1253]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 76 ms (attempt 3/5)5838worker2 # [ 101.783783] nix-grpc-daemon[1253]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 326 ms (attempt 4/5)5839worker2 # [ 102.110804] nix-grpc-daemon[1253]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5840worker2 # [ 102.128422] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:552485841client # [ 102.117473] nix[1202]: copying 1 paths to 'lb:50051'...5842client # [ 102.121155] nix[1202]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...5843worker2 # [ 102.142060] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:552485844worker2 # [ 102.151881] nix-daemon[915]: accepted connection from pid 1253, user nix-grpc-daemon (trusted)5845worker2 # [ 102.174181] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:21Z level=info event=niks3_push_started pid=13685846worker2 # [ 102.364955] nix-grpc-daemon[1368]: time=2026-09-20T13:29:22.111Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""5847worker2 # [ 102.526649] nix-daemon[915]: accepted connection from pid 1381, user nix-grpc-daemon (trusted)5848worker2 # [ 102.538854] nix-daemon[915]: reaped child process 1388, status = succeeded5849worker2 # [ 102.691228] nix-daemon[915]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)5850worker2 # [ 102.705165] nix-daemon[915]: reaped child process 1393, status = succeeded5851lb # [ 102.690149] niks3-server[1021]: 2026/09/20 13:29:22 INFO Received uploads request method=POST path=/api/pending_closures5852worker2 # [ 102.729790] nix-grpc-daemon[1368]: time=2026-09-20T13:29:22.476Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5853worker2 # [ 102.731344] nix-grpc-daemon[1368]: time=2026-09-20T13:29:22.478Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"5854lb # [ 102.724283] niks3-server[1021]: 2026/09/20 13:29:22 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst5855lb # [ 102.734557] niks3-server[1021]: 2026/09/20 13:29:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5856lb # [ 102.740713] niks3-server[1021]: 2026/09/20 13:29:22 INFO Signed narinfos id=35 count=15857worker2 # [ 102.772860] nix-grpc-daemon[1368]: time=2026-09-20T13:29:22.519Z level=INFO msg="Uploading 1 narinfos"5858lb # [ 102.748737] niks3-server[1021]: 2026/09/20 13:29:22 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls5859lb # [ 102.757145] niks3-server[1021]: 2026/09/20 13:29:22 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5860lb # [ 102.761738] niks3-server[1021]: 2026/09/20 13:29:22 INFO Completed upload id=355861worker2 # [ 102.791449] nix-grpc-daemon[1368]: time=2026-09-20T13:29:22.538Z level=INFO msg="Upload complete. (423ms)"5862lb # [ 102.763271] niks3-server[1021]: 2026/09/20 13:29:22 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo5863worker2 # [ 102.794396] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55248 duration_s=0 paths=1 nar_bytes_in=4245864worker2 # [ 102.797552] nix-grpc-daemon[1253]: ts=2026-09-20T13:29:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:552485865worker2 # [ 102.805066] nix-daemon[915]: accepted connection from pid 1253, user nix-grpc-daemon (trusted)5866worker2 # [ 102.809673] nix-daemon[915]: reaped child process 1366, status = succeeded5867worker2 # [ 102.814964] nix-daemon[915]: accepted connection from pid 1253, user nix-grpc-daemon (trusted)5868client # [ 102.794270] nix[1202]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5869lb: must succeed: curl -sf localhost:9901/clusters5870lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5871lb: must succeed: curl -sf localhost:9901/clusters5872lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5873worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5874worker2 # [ 103.841312] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1253 (nix-grpc-daemon) on client request.5875worker2 # [ 103.843740] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5876lb # [ 103.817475] niks3-server[1021]: 2026/09/20 13:29:23 INFO lead: released remote=192.168.1.4:344585877worker1 # [ 103.823821] nix-grpc-daemon[1532]: ts=2026-09-20T13:29:23Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5878client # [ 103.832972] nix[1202]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5879client # [ 103.835114] nix[1202]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5880worker2 # [ 103.861781] systemd[1]: Starting scheduler-token.service...5881worker2 # [ 103.877438] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5882worker2 # [ 103.879654] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5883worker2 # [ 103.880869] systemd[1]: nix-grpc-daemon.service: Consumed 467ms CPU time over 15.917s wall clock time, 31.1M memory peak, 17.4K incoming IP traffic, 33.7K outgoing IP traffic.5884worker2 # [ 103.886358] nix-daemon[915]: reaped child process 1395, status = succeeded5885worker2 # [ 103.887864] nix-daemon[915]: reaped child process 1309, status = succeeded5886worker2 # [ 103.897237] nix-daemon[1397]: unexpected Nix daemon error: error: interrupted by the user5887worker2 # [ 103.902069] nix-daemon[915]: reaped child process 1397, status = failed with exit code 15888worker2 # [ 103.942420] systemd[1]: scheduler-token.service: Deactivated successfully.5889worker2 # [ 103.943611] systemd[1]: Finished scheduler-token.service.5890worker2 # [ 103.945763] systemd[1]: scheduler-token.service: Consumed 17ms CPU time over 80ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5891worker2 # [ 103.947625] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.5892worker2 # [ 103.951953] systemd[1]: Starting scheduler-token.service...5893worker2 # [ 104.014675] systemd[1]: scheduler-token.service: Deactivated successfully.5894worker2 # [ 104.016407] systemd[1]: Finished scheduler-token.service.5895worker2 # [ 104.020140] systemd[1]: scheduler-token.service: Consumed 17ms CPU time over 64ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5896worker2 # [ 104.025329] systemd[1]: Starting Nix worker-protocol over gRPC...5897worker1 # [ 104.025094] nix-grpc-daemon[1532]: ts=2026-09-20T13:29:23Z level=info event=scheduler_connected addr=lb:500515898worker2 # [ 104.180939] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:23Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5899worker2 # [ 104.183253] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:23Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5900worker2 # [ 104.186612] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:23Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15901worker2 # [ 104.192958] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:23Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5902worker2 # [ 104.197199] systemd[1]: Started Nix worker-protocol over gRPC.5903worker2 # [ 104.202310] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:23Z level=info event=scheduler_take_over5904lb # [ 104.173187] niks3-server[1021]: 2026/09/20 13:29:24 INFO lead: acquired remote=192.168.1.4:607445905worker2 # [ 104.205906] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:23Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:406905906worker2 # [ 104.207396] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:23Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05907worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.40 seconds)5908client: must succeed: sleep 5.05909worker2 # [ 104.214833] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:23Z level=info event=scheduler_connected addr=lb:500515910worker2 # [ 104.219061] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:23Z level=info event=oidc_jwks provider=mock keys=15911worker2 # [ 104.220384] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:23Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:406905912worker2 # [ 104.221995] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:23Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05913worker2 # [ 104.362888] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:24Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05914worker2 # [ 104.367905] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:24Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=15915worker1 # [ 104.377410] nix-grpc-daemon[1532]: ts=2026-09-20T13:29:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471705916worker1 # [ 104.384290] nix-grpc-daemon[1532]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 8 ms (attempt 1/5)5917worker1 # [ 104.394607] nix-grpc-daemon[1532]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 124 ms (attempt 2/5)5918worker1 # [ 104.521163] nix-grpc-daemon[1532]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 15 ms (attempt 3/5)5919worker1 # [ 104.536505] nix-grpc-daemon[1532]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 779 ms (attempt 4/5)5920worker1 # [ 105.322517] nix-grpc-daemon[1532]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5921worker1 # [ 105.336732] nix-grpc-daemon[1532]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' from 'http://lb:5751'...5922worker1 # [ 105.362714] nix-daemon[905]: accepted connection from pid 1532, user nix-grpc-daemon (trusted)5923client # [ 105.385695] nix[1202]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5924worker1 # [ 105.378075] nix-daemon[905]: accepted connection from pid 1532, user nix-grpc-daemon (trusted)5925lb # [ 109.137531] envoy[930]: conn peer=[2001:db8:1::4]:55940 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=10296 tx=702 ms=157065926client: (finished: must succeed: sleep 5.0, in 5.04 seconds)5927worker1 # [ 109.397794] nix-grpc-daemon[1532]: ts=2026-09-20T13:29:29Z level=info event=niks3_push_started pid=16295928worker1 # [ 109.411985] nix-daemon[905]: reaped child process 1605, status = succeeded5929client: waiting for success: ! systemctl is-active mid5930worker1 # [ 109.585345] nix-grpc-daemon[1629]: time=2026-09-20T13:29:29.197Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""5931worker1 # [ 109.739648] nix-daemon[905]: accepted connection from pid 1642, user nix-grpc-daemon (trusted)5932worker1 # [ 109.754777] nix-daemon[905]: reaped child process 1645, status = succeeded5933lb # [ 109.766491] niks3-server[1021]: 2026/09/20 13:29:29 INFO Received uploads request method=POST path=/api/pending_closures5934worker1 # [ 109.772062] nix-grpc-daemon[1629]: time=2026-09-20T13:29:29.385Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5935worker1 # [ 109.773679] nix-grpc-daemon[1629]: time=2026-09-20T13:29:29.386Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"5936lb # [ 109.799823] niks3-server[1021]: 2026/09/20 13:29:29 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5937lb # [ 109.801503] niks3-server[1021]: 2026/09/20 13:29:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5938worker1 # [ 109.804465] nix-grpc-daemon[1629]: time=2026-09-20T13:29:29.417Z level=INFO msg="Uploading 1 narinfos"5939lb # [ 109.805371] niks3-server[1021]: 2026/09/20 13:29:29 INFO Signed narinfos id=36 count=15940lb # [ 109.821378] niks3-server[1021]: 2026/09/20 13:29:29 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5941lb # [ 109.823680] niks3-server[1021]: 2026/09/20 13:29:29 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5942lb # [ 109.826864] niks3-server[1021]: 2026/09/20 13:29:29 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls5943lb # [ 109.829389] niks3-server[1021]: 2026/09/20 13:29:29 INFO Completed upload id=365944worker1 # [ 109.829171] nix-grpc-daemon[1629]: time=2026-09-20T13:29:29.442Z level=INFO msg="Upload complete. (240ms)"5945lb # [ 109.831684] niks3-server[1021]: 2026/09/20 13:29:29 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo5946worker1 # [ 109.832320] nix-grpc-daemon[1532]: ts=2026-09-20T13:29:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47170 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=15947worker1 # [ 109.840755] nix-daemon[905]: reaped child process 1603, status = succeeded5948client # [ 109.883874] systemd[1]: mid.service: Deactivated successfully.5949client # [ 109.889561] systemd[1]: mid.service: Consumed 132ms CPU time over 8.678s wall clock time, 39.1M memory peak, 9.7K incoming IP traffic, 10.6K outgoing IP traffic.5950client: (finished: waiting for success: ! systemctl is-active mid, in 1.10 seconds)5951client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5952client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.03 seconds)5953(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 9.42 seconds)5954subtest: interrupting the client stops the build on the worker5955client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr5956client # [ 110.609620] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr.5957client # Running as unit: intr.service; invocation ID: 4eecaf8c02e44b51a3e5aaa0aff52c7f5958client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr, in 0.04 seconds)5959worker2 # [ 110.829427] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:407145960worker2 # [ 110.842797] nix-daemon[915]: accepted connection from pid 1435, user nix-grpc-daemon (trusted)5961client # [ 110.836828] nix[1237]: don't know how to build these paths:5962client # [ 110.837735] nix[1237]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv5963worker2 # [ 110.867854] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:30Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=05964worker2 # [ 110.872679] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:30Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=15965worker2 # [ 110.917150] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:407145966worker2 # [ 110.934189] nix-grpc-daemon[1435]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 22 ms (attempt 1/5)5967worker2 # [ 110.961923] nix-grpc-daemon[1435]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 20 ms (attempt 2/5)5968worker2 # [ 110.987623] nix-grpc-daemon[1435]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 280 ms (attempt 3/5)5969worker2 # [ 111.273588] nix-grpc-daemon[1435]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 559 ms (attempt 4/5)5970worker2 # [ 111.840729] nix-grpc-daemon[1435]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5971worker2 # [ 111.851378] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:407145972client # [ 111.838272] nix[1237]: copying 1 paths to 'lb:50051'...5973worker2 # [ 111.862578] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:407145974client # [ 111.841924] nix[1237]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...5975worker2 # [ 111.873300] nix-daemon[915]: accepted connection from pid 1435, user nix-grpc-daemon (trusted)5976worker2 # [ 111.885118] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:31Z level=info event=niks3_push_started pid=15115977worker2 # [ 112.031234] nix-grpc-daemon[1511]: time=2026-09-20T13:29:31.777Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""5978worker2 # [ 112.155563] nix-daemon[915]: accepted connection from pid 1524, user nix-grpc-daemon (trusted)5979worker2 # [ 112.170875] nix-daemon[915]: reaped child process 1527, status = succeeded5980worker2 # [ 112.292551] nix-daemon[915]: accepted connection from pid 1529, user nix-grpc-daemon (trusted)5981worker2 # [ 112.304871] nix-daemon[915]: reaped child process 1532, status = succeeded5982lb # [ 112.288531] niks3-server[1021]: 2026/09/20 13:29:32 INFO Received uploads request method=POST path=/api/pending_closures5983worker2 # [ 112.326252] nix-grpc-daemon[1511]: time=2026-09-20T13:29:32.073Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5984worker2 # [ 112.329039] nix-grpc-daemon[1511]: time=2026-09-20T13:29:32.075Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"5985lb # [ 112.325182] niks3-server[1021]: 2026/09/20 13:29:32 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst5986lb # [ 112.330343] niks3-server[1021]: 2026/09/20 13:29:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5987lb # [ 112.335538] niks3-server[1021]: 2026/09/20 13:29:32 INFO Signed narinfos id=37 count=15988worker2 # [ 112.368530] nix-grpc-daemon[1511]: time=2026-09-20T13:29:32.115Z level=INFO msg="Uploading 1 narinfos"5989lb # [ 112.346299] niks3-server[1021]: 2026/09/20 13:29:32 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls5990lb # [ 112.355568] niks3-server[1021]: 2026/09/20 13:29:32 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5991lb # [ 112.365920] niks3-server[1021]: 2026/09/20 13:29:32 INFO Completed upload id=375992worker2 # [ 112.395495] nix-grpc-daemon[1511]: time=2026-09-20T13:29:32.142Z level=INFO msg="Upload complete. (360ms)"5993lb # [ 112.367301] niks3-server[1021]: 2026/09/20 13:29:32 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo5994worker2 # [ 112.397770] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40714 duration_s=0 paths=1 nar_bytes_in=4245995worker2 # [ 112.400944] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:407145996worker2 # [ 112.405766] nix-daemon[915]: accepted connection from pid 1435, user nix-grpc-daemon (trusted)5997worker2 # [ 112.410114] nix-daemon[915]: reaped child process 1509, status = succeeded5998client # [ 112.393654] nix[1237]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...5999worker2 # [ 112.414953] nix-daemon[915]: accepted connection from pid 1435, user nix-grpc-daemon (trusted)6000client: must succeed: systemctl kill -s INT intr6001client # [ 114.036846] systemd[1]: intr.service: Sent signal SIGINT to main process 1237 (nix) on client request.6002client # [ 114.040627] nix[1237]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6003client: (finished: must succeed: systemctl kill -s INT intr, in 0.04 seconds)6004client # [ 114.044618] nix[1237]: error: interrupted by the user6005client # [ 114.073544] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6006client # [ 114.074789] systemd[1]: intr.service: Failed with result 'exit-code'.6007client # [ 114.076503] systemd[1]: intr.service: Consumed 121ms CPU time over 3.463s wall clock time, 39.1M memory peak, 7.6K incoming IP traffic, 8.7K outgoing IP traffic.6008worker2 # [ 114.231923] nix-daemon[1536]: unexpected Nix daemon error: error: interrupted by the user6009worker2 # [ 114.235092] nix-daemon[915]: reaped child process 1534, status = succeeded6010worker2 # [ 114.238455] nix-daemon[915]: reaped child process 1536, status = failed with exit code 16011(finished: subtest: interrupting the client stops the build on the worker, in 4.71 seconds)6012subtest: a deploy mid-build drains: the build finishes, the new generation takes over6013client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw6014client # [ 115.338565] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw.6015client # Running as unit: sw.service; invocation ID: 9e20d4398120419096de858e0b1b34b56016client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw, in 0.07 seconds)6017worker1 # [ 115.530178] nix-grpc-daemon[1532]: ts=2026-09-20T13:29:35Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:471706018client # [ 115.542906] nix[1257]: don't know how to build these paths:6019client # [ 115.543935] nix[1257]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6020worker2 # [ 115.569739] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:35Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06021worker2 # [ 115.571894] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:35Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=16022worker1 # [ 115.582872] nix-grpc-daemon[1532]: ts=2026-09-20T13:29:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471706023worker1 # [ 115.592688] nix-grpc-daemon[1532]: ts=2026-09-20T13:29:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471706024client # [ 115.608357] nix[1257]: copying 1 paths to 'lb:50051'...6025worker1 # [ 115.600242] nix-grpc-daemon[1532]: ts=2026-09-20T13:29:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:471706026client # [ 115.611190] nix[1257]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6027worker1 # [ 115.609626] nix-daemon[905]: accepted connection from pid 1532, user nix-grpc-daemon (trusted)6028worker1 # [ 115.758893] nix-daemon[905]: accepted connection from pid 1677, user nix-grpc-daemon (trusted)6029worker1 # [ 115.773421] nix-daemon[905]: reaped child process 1680, status = succeeded6030worker1 # [ 115.942925] nix-daemon[905]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6031worker1 # [ 115.956834] nix-daemon[905]: reaped child process 1685, status = succeeded6032lb # [ 115.974270] niks3-server[1021]: 2026/09/20 13:29:35 INFO Received uploads request method=POST path=/api/pending_closures6033worker1 # [ 115.979065] nix-grpc-daemon[1629]: time=2026-09-20T13:29:35.592Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6034worker1 # [ 115.980575] nix-grpc-daemon[1629]: time=2026-09-20T13:29:35.593Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6035lb # [ 115.990326] niks3-server[1021]: 2026/09/20 13:29:36 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6036lb # [ 115.995637] niks3-server[1021]: 2026/09/20 13:29:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6037lb # [ 115.997737] niks3-server[1021]: 2026/09/20 13:29:36 INFO Signed narinfos id=38 count=16038worker1 # [ 115.999573] nix-grpc-daemon[1629]: time=2026-09-20T13:29:35.612Z level=INFO msg="Uploading 1 narinfos"6039lb # [ 116.002628] niks3-server[1021]: 2026/09/20 13:29:36 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6040lb # [ 116.007454] niks3-server[1021]: 2026/09/20 13:29:36 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6041lb # [ 116.011459] niks3-server[1021]: 2026/09/20 13:29:36 INFO Completed upload id=386042worker1 # [ 116.011276] nix-grpc-daemon[1629]: time=2026-09-20T13:29:35.624Z level=INFO msg="Upload complete. (394ms)"6043lb # [ 116.013267] niks3-server[1021]: 2026/09/20 13:29:36 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6044worker1 # [ 116.013844] nix-grpc-daemon[1532]: ts=2026-09-20T13:29:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47170 duration_s=0 paths=1 nar_bytes_in=4166045worker1 # [ 116.017666] nix-grpc-daemon[1532]: ts=2026-09-20T13:29:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471706046worker1 # [ 116.025052] nix-daemon[905]: accepted connection from pid 1532, user nix-grpc-daemon (trusted)6047worker1 # [ 116.027669] nix-daemon[905]: reaped child process 1675, status = succeeded6048client # [ 116.040813] nix[1257]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6049worker1 # [ 116.033132] nix-daemon[905]: accepted connection from pid 1532, user nix-grpc-daemon (trusted)6050worker1: must succeed: systemctl reload nix-grpc-daemon.service6051worker1 # [ 117.868901] systemd[1]: Reloading Nix worker-protocol over gRPC...6052worker1 # [ 117.915095] systemd[1]: Reloaded Nix worker-protocol over gRPC.6053worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.09 seconds)6054worker1: must succeed: pgrep -f 'read -t [9]0 x'6055worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)6056client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&26057client # don't know how to build these paths:6058client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6059worker2 # [ 118.163366] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:407146060worker2 # [ 118.165987] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:407146061worker2 # [ 118.181994] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:37Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06062worker2 # [ 118.184121] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:37Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=16063worker2 # [ 118.227926] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:407146064worker2 # [ 118.235836] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:407146065client # copying 1 paths to 'lb:50051'...6066worker2 # [ 118.243600] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:407146067client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6068worker2 # [ 118.254543] nix-daemon[915]: accepted connection from pid 1435, user nix-grpc-daemon (trusted)6069worker2 # [ 118.389532] nix-daemon[915]: accepted connection from pid 1590, user nix-grpc-daemon (trusted)6070worker2 # [ 118.403114] nix-daemon[915]: reaped child process 1593, status = succeeded6071worker1 # [ 118.491679] nix-grpc-daemon[1532]: ts=2026-09-20T13:29:38Z level=info event=draining6072worker2 # [ 118.563053] nix-daemon[915]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6073worker2 # [ 118.574525] nix-daemon[915]: reaped child process 1598, status = succeeded6074lb # [ 118.558802] niks3-server[1021]: 2026/09/20 13:29:38 INFO Received uploads request method=POST path=/api/pending_closures6075worker2 # [ 118.599949] nix-grpc-daemon[1511]: time=2026-09-20T13:29:38.346Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6076worker2 # [ 118.601841] nix-grpc-daemon[1511]: time=2026-09-20T13:29:38.348Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6077lb # [ 118.591357] niks3-server[1021]: 2026/09/20 13:29:38 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6078lb # [ 118.600474] niks3-server[1021]: 2026/09/20 13:29:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6079lb # [ 118.605252] niks3-server[1021]: 2026/09/20 13:29:38 INFO Signed narinfos id=39 count=16080worker2 # [ 118.639945] nix-grpc-daemon[1511]: time=2026-09-20T13:29:38.386Z level=INFO msg="Uploading 1 narinfos"6081lb # [ 118.614263] niks3-server[1021]: 2026/09/20 13:29:38 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6082lb # [ 118.626830] niks3-server[1021]: 2026/09/20 13:29:38 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6083lb # [ 118.635608] niks3-server[1021]: 2026/09/20 13:29:38 INFO Completed upload id=396084worker2 # [ 118.665408] nix-grpc-daemon[1511]: time=2026-09-20T13:29:38.412Z level=INFO msg="Upload complete. (404ms)"6085lb # [ 118.636864] niks3-server[1021]: 2026/09/20 13:29:38 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6086worker2 # [ 118.667669] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40714 duration_s=0 paths=1 nar_bytes_in=5286087worker2 # [ 118.670778] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:407146088worker2 # [ 118.675740] nix-daemon[915]: accepted connection from pid 1435, user nix-grpc-daemon (trusted)6089worker2 # [ 118.680559] nix-daemon[915]: reaped child process 1588, status = succeeded6090client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6091worker2 # [ 118.686523] nix-daemon[915]: accepted connection from pid 1435, user nix-grpc-daemon (trusted)6092client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6093lb # [ 119.139110] envoy[930]: conn peer=[2001:db8:1::1]:53030 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3835 tx=1661 ms=84606094lb # [ 119.142823] envoy[930]: conn peer=[2001:db8:1::1]:38446 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2924 tx=757 ms=32526095client # farm-a> LOG-farm-a-during-drain6096worker2 # [ 119.900375] nix-daemon[915]: reaped child process 1602, status = succeeded6097worker2 # [ 120.019097] nix-daemon[915]: accepted connection from pid 1617, user nix-grpc-daemon (trusted)6098worker2 # [ 120.030445] nix-daemon[915]: reaped child process 1620, status = succeeded6099lb # [ 120.017279] niks3-server[1021]: 2026/09/20 13:29:40 INFO Received uploads request method=POST path=/api/pending_closures6100worker2 # [ 120.052700] nix-grpc-daemon[1511]: time=2026-09-20T13:29:39.799Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6101worker2 # [ 120.054249] nix-grpc-daemon[1511]: time=2026-09-20T13:29:39.801Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6102lb # [ 120.061772] niks3-server[1021]: 2026/09/20 13:29:40 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6103lb # [ 120.067866] niks3-server[1021]: 2026/09/20 13:29:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6104lb # [ 120.072482] niks3-server[1021]: 2026/09/20 13:29:40 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6105lb # [ 120.079120] niks3-server[1021]: 2026/09/20 13:29:40 INFO Signed narinfos id=40 count=16106worker2 # [ 120.111538] nix-grpc-daemon[1511]: time=2026-09-20T13:29:39.858Z level=INFO msg="Uploading 1 narinfos"6107lb # [ 120.086182] niks3-server[1021]: 2026/09/20 13:29:40 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6108lb # [ 120.096048] niks3-server[1021]: 2026/09/20 13:29:40 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6109lb # [ 120.100962] niks3-server[1021]: 2026/09/20 13:29:40 INFO Completed upload id=406110worker2 # [ 120.131867] nix-grpc-daemon[1511]: time=2026-09-20T13:29:39.878Z level=INFO msg="Upload complete. (239ms)"6111lb # [ 120.103632] niks3-server[1021]: 2026/09/20 13:29:40 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6112worker2 # [ 120.134263] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40714 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16113worker2 # [ 120.141190] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:39Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06114worker2 # [ 120.145327] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:39Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=16115worker2 # [ 120.147738] nix-daemon[915]: reaped child process 1600, status = succeeded6116worker2 # [ 120.186665] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:407146117worker2 # [ 120.195179] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:407146118worker2 # [ 120.204631] nix-grpc-daemon[1511]: time=2026-09-20T13:29:39.951Z level=INFO msg="All 1 paths already cached"6119client # copying 1 paths to 'lb:50051'...6120worker2 # [ 120.209247] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:407146121client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6122worker2 # [ 120.217891] nix-daemon[915]: accepted connection from pid 1435, user nix-grpc-daemon (trusted)6123worker2 # [ 120.361943] nix-daemon[915]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)6124worker2 # [ 120.375489] nix-daemon[915]: reaped child process 1629, status = succeeded6125worker2 # [ 120.506473] nix-daemon[915]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)6126worker2 # [ 120.519199] nix-daemon[915]: reaped child process 1634, status = succeeded6127lb # [ 120.500475] niks3-server[1021]: 2026/09/20 13:29:40 INFO Received uploads request method=POST path=/api/pending_closures6128worker2 # [ 120.535617] nix-grpc-daemon[1511]: time=2026-09-20T13:29:40.282Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6129worker2 # [ 120.537088] nix-grpc-daemon[1511]: time=2026-09-20T13:29:40.282Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6130lb # [ 120.514986] niks3-server[1021]: 2026/09/20 13:29:40 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6131lb # [ 120.517793] niks3-server[1021]: 2026/09/20 13:29:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6132lb # [ 120.520884] niks3-server[1021]: 2026/09/20 13:29:40 INFO Signed narinfos id=41 count=16133worker2 # [ 120.550605] nix-grpc-daemon[1511]: time=2026-09-20T13:29:40.297Z level=INFO msg="Uploading 1 narinfos"6134lb # [ 120.524419] niks3-server[1021]: 2026/09/20 13:29:40 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6135lb # [ 120.528394] niks3-server[1021]: 2026/09/20 13:29:40 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6136lb # [ 120.531873] niks3-server[1021]: 2026/09/20 13:29:40 INFO Completed upload id=416137worker2 # [ 120.561299] nix-grpc-daemon[1511]: time=2026-09-20T13:29:40.308Z level=INFO msg="Upload complete. (338ms)"6138lb # [ 120.533508] niks3-server[1021]: 2026/09/20 13:29:40 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6139worker2 # [ 120.563223] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40714 duration_s=0 paths=1 nar_bytes_in=6406140worker2 # [ 120.567442] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:407146141worker2 # [ 120.573534] nix-daemon[915]: accepted connection from pid 1435, user nix-grpc-daemon (trusted)6142worker2 # [ 120.575903] nix-daemon[915]: reaped child process 1624, status = succeeded6143worker2 # [ 120.581049] nix-daemon[915]: accepted connection from pid 1435, user nix-grpc-daemon (trusted)6144client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6145client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6146client # farm-b> LOG-farm-b-during-drain6147worker2 # [ 121.395911] nix-daemon[915]: reaped child process 1638, status = succeeded6148worker2 # [ 121.530613] nix-daemon[915]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)6149worker2 # [ 121.543419] nix-daemon[915]: reaped child process 1656, status = succeeded6150lb # [ 121.526291] niks3-server[1021]: 2026/09/20 13:29:41 INFO Received uploads request method=POST path=/api/pending_closures6151worker2 # [ 121.560901] nix-grpc-daemon[1511]: time=2026-09-20T13:29:41.307Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6152worker2 # [ 121.562517] nix-grpc-daemon[1511]: time=2026-09-20T13:29:41.309Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6153lb # [ 121.544144] niks3-server[1021]: 2026/09/20 13:29:41 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6154worker2 # [ 121.583060] nix-grpc-daemon[1511]: time=2026-09-20T13:29:41.329Z level=INFO msg="Uploading 1 narinfos"6155lb # [ 121.553375] niks3-server[1021]: 2026/09/20 13:29:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6156lb # [ 121.555427] niks3-server[1021]: 2026/09/20 13:29:41 INFO Signed narinfos id=42 count=16157lb # [ 121.556512] niks3-server[1021]: 2026/09/20 13:29:41 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6158lb # [ 121.560350] niks3-server[1021]: 2026/09/20 13:29:41 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6159lb # [ 121.565582] niks3-server[1021]: 2026/09/20 13:29:41 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6160lb # [ 121.570543] niks3-server[1021]: 2026/09/20 13:29:41 INFO Completed upload id=426161worker2 # [ 121.600114] nix-grpc-daemon[1511]: time=2026-09-20T13:29:41.347Z level=INFO msg="Upload complete. (212ms)"6162lb # [ 121.572100] niks3-server[1021]: 2026/09/20 13:29:41 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6163worker2 # [ 121.602655] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40714 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16164worker2 # [ 121.609138] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:41Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06165worker2 # [ 121.611125] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:41Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=16166worker2 # [ 121.615672] nix-daemon[915]: reaped child process 1636, status = succeeded6167worker2 # [ 121.654146] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:407146168worker2 # [ 121.663215] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:407146169worker2 # [ 121.673184] nix-grpc-daemon[1511]: time=2026-09-20T13:29:41.420Z level=INFO msg="All 2 paths already cached"6170client # copying 1 paths to 'lb:50051'...6171client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6172worker2 # [ 121.678670] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:407146173worker2 # [ 121.687376] nix-daemon[915]: accepted connection from pid 1435, user nix-grpc-daemon (trusted)6174worker2 # [ 121.835200] nix-daemon[915]: accepted connection from pid 1662, user nix-grpc-daemon (trusted)6175worker2 # [ 121.848288] nix-daemon[915]: reaped child process 1665, status = succeeded6176worker2 # [ 122.007384] nix-daemon[915]: accepted connection from pid 1667, user nix-grpc-daemon (trusted)6177worker2 # [ 122.020146] nix-daemon[915]: reaped child process 1670, status = succeeded6178lb # [ 122.007256] niks3-server[1021]: 2026/09/20 13:29:42 INFO Received uploads request method=POST path=/api/pending_closures6179worker2 # [ 122.045431] nix-grpc-daemon[1511]: time=2026-09-20T13:29:41.792Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6180worker2 # [ 122.047280] nix-grpc-daemon[1511]: time=2026-09-20T13:29:41.794Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6181lb # [ 122.043100] niks3-server[1021]: 2026/09/20 13:29:42 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6182lb # [ 122.052547] niks3-server[1021]: 2026/09/20 13:29:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6183lb # [ 122.057756] niks3-server[1021]: 2026/09/20 13:29:42 INFO Signed narinfos id=43 count=16184worker2 # [ 122.089955] nix-grpc-daemon[1511]: time=2026-09-20T13:29:41.837Z level=INFO msg="Uploading 1 narinfos"6185lb # [ 122.065488] niks3-server[1021]: 2026/09/20 13:29:42 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6186lb # [ 122.075821] niks3-server[1021]: 2026/09/20 13:29:42 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6187lb # [ 122.079880] niks3-server[1021]: 2026/09/20 13:29:42 INFO Completed upload id=436188worker2 # [ 122.109581] nix-grpc-daemon[1511]: time=2026-09-20T13:29:41.856Z level=INFO msg="Upload complete. (411ms)"6189lb # [ 122.082282] niks3-server[1021]: 2026/09/20 13:29:42 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6190worker2 # [ 122.114748] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40714 duration_s=0 paths=1 nar_bytes_in=7686191worker2 # [ 122.121209] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:407146192worker2 # [ 122.129519] nix-daemon[915]: accepted connection from pid 1435, user nix-grpc-daemon (trusted)6193worker2 # [ 122.134645] nix-daemon[915]: reaped child process 1660, status = succeeded6194worker2 # [ 122.139956] nix-daemon[915]: accepted connection from pid 1435, user nix-grpc-daemon (trusted)6195client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6196client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6197client # farm-top> LOG-farm-top-during-drain6198worker2 # [ 123.153790] nix-daemon[915]: reaped child process 1674, status = succeeded6199worker2 # [ 123.291846] nix-daemon[915]: accepted connection from pid 1689, user nix-grpc-daemon (trusted)6200worker2 # [ 123.305927] nix-daemon[915]: reaped child process 1692, status = succeeded6201lb # [ 123.286468] niks3-server[1021]: 2026/09/20 13:29:43 INFO Received uploads request method=POST path=/api/pending_closures6202worker2 # [ 123.321283] nix-grpc-daemon[1511]: time=2026-09-20T13:29:43.068Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6203worker2 # [ 123.324299] nix-grpc-daemon[1511]: time=2026-09-20T13:29:43.069Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6204lb # [ 123.305687] niks3-server[1021]: 2026/09/20 13:29:43 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6205lb # [ 123.310344] niks3-server[1021]: 2026/09/20 13:29:43 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6206lb # [ 123.314097] niks3-server[1021]: 2026/09/20 13:29:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6207lb # [ 123.316098] niks3-server[1021]: 2026/09/20 13:29:43 INFO Signed narinfos id=44 count=16208worker2 # [ 123.345895] nix-grpc-daemon[1511]: time=2026-09-20T13:29:43.092Z level=INFO msg="Uploading 1 narinfos"6209lb # [ 123.319276] niks3-server[1021]: 2026/09/20 13:29:43 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6210lb # [ 123.324403] niks3-server[1021]: 2026/09/20 13:29:43 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6211lb # [ 123.327736] niks3-server[1021]: 2026/09/20 13:29:43 INFO Completed upload id=446212worker2 # [ 123.357249] nix-grpc-daemon[1511]: time=2026-09-20T13:29:43.104Z level=INFO msg="Upload complete. (210ms)"6213lb # [ 123.329347] niks3-server[1021]: 2026/09/20 13:29:43 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6214worker2 # [ 123.360820] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40714 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16215worker2 # [ 123.368251] nix-daemon[915]: reaped child process 1672, status = succeeded6216client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&2, in 5.40 seconds)6217client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath6218client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath, in 0.16 seconds)6219worker1: must succeed: pkill -f 'read -t [9]0 x'6220worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6221client: waiting for success: ! systemctl is-active sw6222worker1 # [ 123.655414] nix-daemon[905]: reaped child process 1689, status = succeeded6223worker1 # [ 123.795761] nix-daemon[905]: accepted connection from pid 1729, user nix-grpc-daemon (trusted)6224worker1 # [ 123.810687] nix-daemon[905]: reaped child process 1732, status = succeeded6225lb # [ 123.825730] niks3-server[1021]: 2026/09/20 13:29:43 INFO Received uploads request method=POST path=/api/pending_closures6226worker1 # [ 123.908973] nix-grpc-daemon[1629]: time=2026-09-20T13:29:43.521Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6227worker1 # [ 123.911670] nix-grpc-daemon[1629]: time=2026-09-20T13:29:43.522Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6228lb # [ 123.943330] niks3-server[1021]: 2026/09/20 13:29:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6229lb # [ 123.950833] niks3-server[1021]: 2026/09/20 13:29:43 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6230lb # [ 123.958497] niks3-server[1021]: 2026/09/20 13:29:43 INFO Signed narinfos id=45 count=16231worker1 # [ 123.962509] nix-grpc-daemon[1629]: time=2026-09-20T13:29:43.575Z level=INFO msg="Uploading 1 narinfos"6232lb # [ 123.964611] niks3-server[1021]: 2026/09/20 13:29:43 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6233lb # [ 123.973704] niks3-server[1021]: 2026/09/20 13:29:43 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6234lb # [ 123.978788] niks3-server[1021]: 2026/09/20 13:29:44 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6235lb # [ 123.990375] niks3-server[1021]: 2026/09/20 13:29:44 INFO Completed upload id=456236worker1 # [ 123.990425] nix-grpc-daemon[1629]: time=2026-09-20T13:29:43.603Z level=INFO msg="Upload complete. (343ms)"6237worker1 # [ 123.994373] nix-grpc-daemon[1532]: ts=2026-09-20T13:29:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47170 duration_s=7 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16238lb # [ 123.997955] niks3-server[1021]: 2026/09/20 13:29:44 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6239worker1 # [ 124.003065] nix-daemon[905]: reaped child process 1687, status = succeeded6240client # [ 124.028272] systemd[1]: sw.service: Deactivated successfully.6241client # [ 124.030490] systemd[1]: sw.service: Consumed 109ms CPU time over 8.691s wall clock time, 39.1M memory peak, 8.2K incoming IP traffic, 8.9K outgoing IP traffic.6242worker1 # [ 124.494499] nix-grpc-daemon[1532]: ts=2026-09-20T13:29:44Z level=info event=signal_exit6243client: (finished: waiting for success: ! systemctl is-active sw, in 1.07 seconds)6244client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6245client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds)6246worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6247worker2 # [ 125.751590] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:45Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:406906248worker2 # [ 125.753613] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:45Z level=info event=worker_gone addr=192.168.1.3:500516249worker1 # [ 125.722816] nix-grpc-daemon[1532]: ts=2026-09-20T13:29:45Z level=info event=scheduler_restarting6250worker1 # [ 125.736596] nix-grpc-daemon[1532]: ts=2026-09-20T13:29:45Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6251worker1 # [ 125.755614] nix-daemon[905]: reaped child process 1585, status = succeeded6252worker1 # [ 126.326967] nix-grpc-daemon[1532]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6253worker1 # [ 126.330641] nix-grpc-daemon[1532]: E0000 00:00:1789910985.939844 1575 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true6254worker1 # [ 126.335337] nix-grpc-daemon[1532]: E0000 00:00:1789910985.940125 1575 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6255worker1 # [ 126.522641] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6256worker1 # [ 126.523753] systemd[1]: nix-grpc-daemon.service: Consumed 640ms CPU time over 26.210s wall clock time, 30.1M memory peak, 21K incoming IP traffic, 27.7K outgoing IP traffic.6257worker1 # [ 126.531316] systemd[1]: Starting Nix worker-protocol over gRPC...6258worker1 # [ 126.674525] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:46Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6259worker1 # [ 126.676357] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:46Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6260worker1 # [ 126.679921] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:46Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16261worker1 # [ 126.687069] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:46Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6262worker1 # [ 126.693752] systemd[1]: Started Nix worker-protocol over gRPC.6263worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.03 seconds)6264lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6265lb: must succeed: curl -sf localhost:9901/clusters6266worker1 # [ 126.707549] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:46Z level=info event=scheduler_connected addr=lb:500516267worker2 # [ 126.739252] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:46Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:406906268worker2 # [ 126.741068] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:46Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06269lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6270lb: must succeed: curl -sf localhost:9901/clusters6271lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6272lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.11 seconds)6273lb: must succeed: curl -sf localhost:9901/clusters6274lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6275worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6276worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6277(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 12.63 seconds)6278subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6279client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16280worker2 # [ 128.108333] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:407146281worker1 # [ 128.085151] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:495306282worker1 # [ 128.094097] nix-daemon[905]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6283worker1 # [ 128.105697] nix-grpc-daemon[1746]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 38 ms (attempt 1/5)6284worker1 # [ 128.149174] nix-grpc-daemon[1746]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 90 ms (attempt 2/5)6285worker1 # [ 128.245519] nix-grpc-daemon[1746]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 33 ms (attempt 3/5)6286worker1 # [ 128.279139] nix-grpc-daemon[1746]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 673 ms (attempt 4/5)6287worker1 # [ 128.958495] nix-grpc-daemon[1746]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6288worker2 # [ 129.018659] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:48Z level=debug event=want client=5 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06289worker2 # [ 129.020442] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:48Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16290worker2 # [ 129.022314] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:48Z level=debug event=want client=5 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06291worker2 # [ 129.023836] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:48Z level=debug event=want client=5 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06292worker1 # [ 129.034650] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:495306293worker1 # [ 129.046317] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:495306294worker1 # [ 129.056474] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:495306295worker1 # [ 129.065601] nix-daemon[905]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6296worker1 # [ 129.091841] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:48Z level=info event=niks3_push_started pid=18026297lb # [ 129.138934] envoy[930]: conn peer=[2001:db8:1::1]:37906 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9572 tx=3858 ms=52146298lb # [ 129.140458] envoy[930]: conn peer=[2001:db8:1::1]:37902 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2932 tx=1307 ms=84746299lb # [ 129.141891] envoy[930]: conn peer=[2001:db8:1::3]:46634 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1114 tx=448 ms=252296300worker1 # [ 129.274763] nix-grpc-daemon[1802]: time=2026-09-20T13:29:48.886Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""6301worker1 # [ 129.429828] nix-daemon[905]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6302worker1 # [ 129.444294] nix-daemon[905]: reaped child process 1818, status = succeeded6303worker1 # [ 129.613668] nix-daemon[905]: accepted connection from pid 1820, user nix-grpc-daemon (trusted)6304worker1 # [ 129.628567] nix-daemon[905]: reaped child process 1823, status = succeeded6305lb # [ 129.645885] niks3-server[1021]: 2026/09/20 13:29:49 INFO Received uploads request method=POST path=/api/pending_closures6306worker1 # [ 129.656084] nix-grpc-daemon[1802]: time=2026-09-20T13:29:49.268Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6307worker1 # [ 129.657553] nix-grpc-daemon[1802]: time=2026-09-20T13:29:49.268Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6308lb # [ 129.672109] niks3-server[1021]: 2026/09/20 13:29:49 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6309lb # [ 129.675056] niks3-server[1021]: 2026/09/20 13:29:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6310lb # [ 129.678193] niks3-server[1021]: 2026/09/20 13:29:49 INFO Signed narinfos id=46 count=16311worker1 # [ 129.678477] nix-grpc-daemon[1802]: time=2026-09-20T13:29:49.291Z level=INFO msg="Uploading 1 narinfos"6312lb # [ 129.680811] niks3-server[1021]: 2026/09/20 13:29:49 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6313lb # [ 129.685036] niks3-server[1021]: 2026/09/20 13:29:49 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6314lb # [ 129.688463] niks3-server[1021]: 2026/09/20 13:29:49 INFO Completed upload id=466315worker1 # [ 129.688339] nix-grpc-daemon[1802]: time=2026-09-20T13:29:49.301Z level=INFO msg="Upload complete. (413ms)"6316lb # [ 129.690906] niks3-server[1021]: 2026/09/20 13:29:49 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6317worker1 # [ 129.690584] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49530 duration_s=0 paths=1 nar_bytes_in=5046318worker1 # [ 129.694151] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:495306319worker1 # [ 129.699491] nix-daemon[905]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6320worker1 # [ 129.705869] nix-daemon[905]: reaped child process 1800, status = succeeded6321worker1 # [ 129.710394] nix-daemon[905]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6322worker1 # [ 131.136573] nix-daemon[905]: reaped child process 1827, status = succeeded6323worker1 # [ 131.288389] nix-daemon[905]: accepted connection from pid 1842, user nix-grpc-daemon (trusted)6324worker1 # [ 131.300138] nix-daemon[905]: reaped child process 1845, status = succeeded6325lb # [ 131.318161] niks3-server[1021]: 2026/09/20 13:29:51 INFO Received uploads request method=POST path=/api/pending_closures6326worker1 # [ 131.328873] nix-grpc-daemon[1802]: time=2026-09-20T13:29:50.941Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6327worker1 # [ 131.331024] nix-grpc-daemon[1802]: time=2026-09-20T13:29:50.943Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6328lb # [ 131.358149] niks3-server[1021]: 2026/09/20 13:29:51 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6329lb # [ 131.365818] niks3-server[1021]: 2026/09/20 13:29:51 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6330lb # [ 131.378423] niks3-server[1021]: 2026/09/20 13:29:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6331lb # [ 131.384504] niks3-server[1021]: 2026/09/20 13:29:51 INFO Signed narinfos id=47 count=16332worker1 # [ 131.385110] nix-grpc-daemon[1802]: time=2026-09-20T13:29:50.997Z level=INFO msg="Uploading 1 narinfos"6333lb # [ 131.390401] niks3-server[1021]: 2026/09/20 13:29:51 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6334lb # [ 131.400286] niks3-server[1021]: 2026/09/20 13:29:51 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6335lb # [ 131.404501] niks3-server[1021]: 2026/09/20 13:29:51 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6336lb # [ 131.406800] niks3-server[1021]: 2026/09/20 13:29:51 INFO Completed upload id=476337worker1 # [ 131.406941] nix-grpc-daemon[1802]: time=2026-09-20T13:29:51.020Z level=INFO msg="Upload complete. (288ms)"6338worker2 # [ 131.442306] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:51Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16339worker1 # [ 131.413973] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49530 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=7 outputs=16340worker2 # [ 131.448961] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:51Z level=debug event=want client=5 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06341worker1 # [ 131.422473] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:495306342worker1 # [ 131.434987] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:495306343worker1 # [ 131.439408] nix-daemon[905]: reaped child process 1825, status = succeeded6344worker1 # [ 131.445377] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:495306345worker1 # [ 131.452298] nix-daemon[905]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6346worker1 # [ 131.595814] nix-daemon[905]: accepted connection from pid 1849, user nix-grpc-daemon (trusted)6347worker1 # [ 131.609157] nix-daemon[905]: reaped child process 1852, status = succeeded6348worker1 # [ 131.764370] nix-daemon[905]: accepted connection from pid 1854, user nix-grpc-daemon (trusted)6349worker1 # [ 131.777785] nix-daemon[905]: reaped child process 1857, status = succeeded6350lb # [ 131.793203] niks3-server[1021]: 2026/09/20 13:29:51 INFO Received uploads request method=POST path=/api/pending_closures6351worker1 # [ 131.799728] nix-grpc-daemon[1802]: time=2026-09-20T13:29:51.412Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6352worker1 # [ 131.801127] nix-grpc-daemon[1802]: time=2026-09-20T13:29:51.412Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6353lb # [ 131.817248] niks3-server[1021]: 2026/09/20 13:29:51 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6354lb # [ 131.830587] niks3-server[1021]: 2026/09/20 13:29:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6355lb # [ 131.835801] niks3-server[1021]: 2026/09/20 13:29:51 INFO Signed narinfos id=48 count=16356worker1 # [ 131.838787] nix-grpc-daemon[1802]: time=2026-09-20T13:29:51.451Z level=INFO msg="Uploading 1 narinfos"6357lb # [ 131.839820] niks3-server[1021]: 2026/09/20 13:29:51 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6358lb # [ 131.848716] niks3-server[1021]: 2026/09/20 13:29:51 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6359lb # [ 131.853615] niks3-server[1021]: 2026/09/20 13:29:51 INFO Completed upload id=486360worker1 # [ 131.853612] nix-grpc-daemon[1802]: time=2026-09-20T13:29:51.466Z level=INFO msg="Upload complete. (396ms)"6361lb # [ 131.855469] niks3-server[1021]: 2026/09/20 13:29:51 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6362worker1 # [ 131.855731] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49530 duration_s=0 paths=1 nar_bytes_in=5046363worker1 # [ 131.859519] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:495306364worker1 # [ 131.866594] nix-daemon[905]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6365worker1 # [ 131.868600] nix-daemon[905]: reaped child process 1847, status = succeeded6366worker1 # [ 131.874871] nix-daemon[905]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6367worker1 # [ 132.893534] nix-daemon[905]: reaped child process 1861, status = succeeded6368worker1 # [ 133.024194] nix-daemon[905]: accepted connection from pid 1876, user nix-grpc-daemon (trusted)6369worker1 # [ 133.037900] nix-daemon[905]: reaped child process 1879, status = succeeded6370lb # [ 133.049132] niks3-server[1021]: 2026/09/20 13:29:53 INFO Received uploads request method=POST path=/api/pending_closures6371worker1 # [ 133.053446] nix-grpc-daemon[1802]: time=2026-09-20T13:29:52.666Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6372worker1 # [ 133.054919] nix-grpc-daemon[1802]: time=2026-09-20T13:29:52.668Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6373lb # [ 133.068489] niks3-server[1021]: 2026/09/20 13:29:53 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6374lb # [ 133.071180] niks3-server[1021]: 2026/09/20 13:29:53 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6375lb # [ 133.077569] niks3-server[1021]: 2026/09/20 13:29:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6376worker1 # [ 133.082371] nix-grpc-daemon[1802]: time=2026-09-20T13:29:52.695Z level=INFO msg="Uploading 1 narinfos"6377lb # [ 133.082221] niks3-server[1021]: 2026/09/20 13:29:53 INFO Signed narinfos id=49 count=16378lb # [ 133.089209] niks3-server[1021]: 2026/09/20 13:29:53 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6379lb # [ 133.093525] niks3-server[1021]: 2026/09/20 13:29:53 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6380lb # [ 133.096240] niks3-server[1021]: 2026/09/20 13:29:53 INFO Completed upload id=496381worker1 # [ 133.096166] nix-grpc-daemon[1802]: time=2026-09-20T13:29:52.709Z level=INFO msg="Upload complete. (214ms)"6382lb # [ 133.098907] niks3-server[1021]: 2026/09/20 13:29:53 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6383worker2 # [ 133.129425] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:52Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16384worker1 # [ 133.100176] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49530 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=8 outputs=16385worker1 # [ 133.103825] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:495306386worker2 # [ 133.135309] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:52Z level=debug event=want client=5 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06387worker1 # [ 133.110774] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:495306388worker1 # [ 133.112994] nix-daemon[905]: reaped child process 1859, status = succeeded6389worker1 # [ 133.116324] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:495306390worker1 # [ 133.120433] nix-daemon[905]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6391worker1 # [ 133.270682] nix-daemon[905]: accepted connection from pid 1883, user nix-grpc-daemon (trusted)6392worker1 # [ 133.285109] nix-daemon[905]: reaped child process 1886, status = succeeded6393worker1 # [ 133.422963] nix-daemon[905]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)6394worker1 # [ 133.437185] nix-daemon[905]: reaped child process 1891, status = succeeded6395lb # [ 133.453255] niks3-server[1021]: 2026/09/20 13:29:53 INFO Received uploads request method=POST path=/api/pending_closures6396worker1 # [ 133.460752] nix-grpc-daemon[1802]: time=2026-09-20T13:29:53.074Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6397worker1 # [ 133.462298] nix-grpc-daemon[1802]: time=2026-09-20T13:29:53.075Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6398lb # [ 133.480716] niks3-server[1021]: 2026/09/20 13:29:53 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6399lb # [ 133.488328] niks3-server[1021]: 2026/09/20 13:29:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6400worker1 # [ 133.492747] nix-grpc-daemon[1802]: time=2026-09-20T13:29:53.105Z level=INFO msg="Uploading 1 narinfos"6401lb # [ 133.493158] niks3-server[1021]: 2026/09/20 13:29:53 INFO Signed narinfos id=50 count=16402lb # [ 133.495550] niks3-server[1021]: 2026/09/20 13:29:53 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6403lb # [ 133.500247] niks3-server[1021]: 2026/09/20 13:29:53 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6404lb # [ 133.503767] niks3-server[1021]: 2026/09/20 13:29:53 INFO Completed upload id=506405worker1 # [ 133.503818] nix-grpc-daemon[1802]: time=2026-09-20T13:29:53.117Z level=INFO msg="Upload complete. (377ms)"6406lb # [ 133.505241] niks3-server[1021]: 2026/09/20 13:29:53 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6407worker1 # [ 133.505979] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49530 duration_s=0 paths=1 nar_bytes_in=5046408worker1 # [ 133.509038] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:495306409worker1 # [ 133.515031] nix-daemon[905]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6410worker1 # [ 133.520514] nix-daemon[905]: reaped child process 1881, status = succeeded6411worker1 # [ 133.525320] nix-daemon[905]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6412worker1 # [ 134.155485] nix-daemon[905]: reaped child process 1895, status = succeeded6413worker1 # [ 134.299092] nix-daemon[905]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)6414worker1 # [ 134.313904] nix-daemon[905]: reaped child process 1913, status = succeeded6415lb # [ 134.325388] niks3-server[1021]: 2026/09/20 13:29:54 INFO Received uploads request method=POST path=/api/pending_closures6416worker1 # [ 134.336099] nix-grpc-daemon[1802]: time=2026-09-20T13:29:53.949Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6417worker1 # [ 134.337738] nix-grpc-daemon[1802]: time=2026-09-20T13:29:53.950Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6418lb # [ 134.366469] niks3-server[1021]: 2026/09/20 13:29:54 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv6419lb # [ 134.375459] niks3-server[1021]: 2026/09/20 13:29:54 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6420lb # [ 134.382118] niks3-server[1021]: 2026/09/20 13:29:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6421worker1 # [ 134.388877] nix-grpc-daemon[1802]: time=2026-09-20T13:29:54.001Z level=INFO msg="Uploading 1 narinfos"6422lb # [ 134.388695] niks3-server[1021]: 2026/09/20 13:29:54 INFO Signed narinfos id=51 count=16423lb # [ 134.393779] niks3-server[1021]: 2026/09/20 13:29:54 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls6424lb # [ 134.406066] niks3-server[1021]: 2026/09/20 13:29:54 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6425lb # [ 134.411957] niks3-server[1021]: 2026/09/20 13:29:54 INFO Completed upload id=516426worker1 # [ 134.411864] nix-grpc-daemon[1802]: time=2026-09-20T13:29:54.024Z level=INFO msg="Upload complete. (277ms)"6427lb # [ 134.413954] niks3-server[1021]: 2026/09/20 13:29:54 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo6428worker2 # [ 134.447514] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:54Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=16429worker2 # [ 134.454184] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:54Z level=debug event=want client=5 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=06430worker1 # [ 134.421253] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49530 duration_s=0 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=9 outputs=16431worker1 # [ 134.427350] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:495306432worker1 # [ 134.440798] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:495306433worker1 # [ 134.446707] nix-daemon[905]: reaped child process 1893, status = succeeded6434worker1 # [ 134.452886] nix-grpc-daemon[1802]: time=2026-09-20T13:29:54.066Z level=INFO msg="All 1 paths already cached"6435worker1 # [ 134.457594] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:495306436worker1 # [ 134.463082] nix-daemon[905]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6437worker1 # [ 134.608388] nix-daemon[905]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)6438worker1 # [ 134.625412] nix-daemon[905]: reaped child process 1920, status = succeeded6439worker1 # [ 134.768200] nix-daemon[905]: accepted connection from pid 1922, user nix-grpc-daemon (trusted)6440worker1 # [ 134.784375] nix-daemon[905]: reaped child process 1925, status = succeeded6441lb # [ 134.795923] niks3-server[1021]: 2026/09/20 13:29:54 INFO Received uploads request method=POST path=/api/pending_closures6442worker1 # [ 134.804214] nix-grpc-daemon[1802]: time=2026-09-20T13:29:54.417Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6443worker1 # [ 134.805897] nix-grpc-daemon[1802]: time=2026-09-20T13:29:54.419Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"6444lb # [ 134.826527] niks3-server[1021]: 2026/09/20 13:29:54 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst6445lb # [ 134.841087] niks3-server[1021]: 2026/09/20 13:29:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6446worker1 # [ 134.845768] nix-grpc-daemon[1802]: time=2026-09-20T13:29:54.458Z level=INFO msg="Uploading 1 narinfos"6447lb # [ 134.846277] niks3-server[1021]: 2026/09/20 13:29:54 INFO Signed narinfos id=52 count=16448lb # [ 134.853451] niks3-server[1021]: 2026/09/20 13:29:54 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls6449lb # [ 134.864714] niks3-server[1021]: 2026/09/20 13:29:54 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6450lb # [ 134.874819] niks3-server[1021]: 2026/09/20 13:29:54 INFO Completed upload id=526451worker1 # [ 134.875182] nix-grpc-daemon[1802]: time=2026-09-20T13:29:54.488Z level=INFO msg="Upload complete. (405ms)"6452lb # [ 134.877734] niks3-server[1021]: 2026/09/20 13:29:54 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo6453worker1 # [ 134.879090] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49530 duration_s=0 paths=1 nar_bytes_in=6246454worker1 # [ 134.884099] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:495306455worker1 # [ 134.892533] nix-daemon[905]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6456worker1 # [ 134.895756] nix-daemon[905]: reaped child process 1915, status = succeeded6457worker1 # [ 134.904056] nix-daemon[905]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6458worker1 # [ 136.128229] nix-daemon[905]: reaped child process 1929, status = succeeded6459worker1 # [ 136.263355] nix-daemon[905]: accepted connection from pid 1944, user nix-grpc-daemon (trusted)6460worker1 # [ 136.279103] nix-daemon[905]: reaped child process 1947, status = succeeded6461lb # [ 136.289416] niks3-server[1021]: 2026/09/20 13:29:56 INFO Received uploads request method=POST path=/api/pending_closures6462worker1 # [ 136.294432] nix-grpc-daemon[1802]: time=2026-09-20T13:29:55.907Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6463worker1 # [ 136.298726] nix-grpc-daemon[1802]: time=2026-09-20T13:29:55.909Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"6464lb # [ 136.327719] niks3-server[1021]: 2026/09/20 13:29:56 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv6465lb # [ 136.338744] niks3-server[1021]: 2026/09/20 13:29:56 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst6466lb # [ 136.350461] niks3-server[1021]: 2026/09/20 13:29:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6467worker1 # [ 136.357099] nix-grpc-daemon[1802]: time=2026-09-20T13:29:55.970Z level=INFO msg="Uploading 1 narinfos"6468lb # [ 136.356260] niks3-server[1021]: 2026/09/20 13:29:56 INFO Signed narinfos id=53 count=16469lb # [ 136.363233] niks3-server[1021]: 2026/09/20 13:29:56 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls6470lb # [ 136.373428] niks3-server[1021]: 2026/09/20 13:29:56 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6471lb # [ 136.380414] niks3-server[1021]: 2026/09/20 13:29:56 INFO Completed upload id=536472worker1 # [ 136.380725] nix-grpc-daemon[1802]: time=2026-09-20T13:29:55.993Z level=INFO msg="Upload complete. (269ms)"6473lb # [ 136.383225] niks3-server[1021]: 2026/09/20 13:29:56 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo6474worker1 # [ 136.384365] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49530 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=10 outputs=16475worker1 # [ 136.392228] nix-daemon[905]: reaped child process 1927, status = succeeded6476worker2 # [ 136.416872] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:56Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=16477worker2 # [ 136.428537] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:56Z level=debug event=want client=5 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=06478worker1 # [ 136.430824] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:495306479worker1 # [ 136.441213] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:495306480worker1 # [ 136.453816] nix-grpc-daemon[1802]: time=2026-09-20T13:29:56.067Z level=INFO msg="All 1 paths already cached"6481worker1 # [ 136.459248] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:495306482worker1 # [ 136.465258] nix-daemon[905]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6483worker1 # [ 136.617141] nix-daemon[905]: accepted connection from pid 1951, user nix-grpc-daemon (trusted)6484worker1 # [ 136.632711] nix-daemon[905]: reaped child process 1954, status = succeeded6485worker1 # [ 136.782533] nix-daemon[905]: accepted connection from pid 1956, user nix-grpc-daemon (trusted)6486worker1 # [ 136.797228] nix-daemon[905]: reaped child process 1959, status = succeeded6487lb # [ 136.812506] niks3-server[1021]: 2026/09/20 13:29:56 INFO Received uploads request method=POST path=/api/pending_closures6488worker1 # [ 136.820754] nix-grpc-daemon[1802]: time=2026-09-20T13:29:56.433Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6489worker1 # [ 136.822201] nix-grpc-daemon[1802]: time=2026-09-20T13:29:56.433Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"6490lb # [ 136.835886] niks3-server[1021]: 2026/09/20 13:29:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6491lb # [ 136.840875] niks3-server[1021]: 2026/09/20 13:29:56 INFO Signed narinfos id=54 count=16492worker1 # [ 136.841203] nix-grpc-daemon[1802]: time=2026-09-20T13:29:56.454Z level=INFO msg="Uploading 1 narinfos"6493lb # [ 136.844441] niks3-server[1021]: 2026/09/20 13:29:56 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst6494lb # [ 136.848097] niks3-server[1021]: 2026/09/20 13:29:56 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls6495lb # [ 136.855039] niks3-server[1021]: 2026/09/20 13:29:56 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6496worker1 # [ 136.859989] nix-grpc-daemon[1802]: time=2026-09-20T13:29:56.473Z level=INFO msg="Upload complete. (383ms)"6497lb # [ 136.860359] niks3-server[1021]: 2026/09/20 13:29:56 INFO Completed upload id=546498lb # [ 136.863283] niks3-server[1021]: 2026/09/20 13:29:56 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo6499worker1 # [ 136.862591] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49530 duration_s=0 paths=1 nar_bytes_in=6246500worker1 # [ 136.866275] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:495306501worker1 # [ 136.873250] nix-daemon[905]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6502worker1 # [ 136.875749] nix-daemon[905]: reaped child process 1949, status = succeeded6503worker1 # [ 136.881326] nix-daemon[905]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6504worker1 # [ 138.098769] nix-daemon[905]: reaped child process 1963, status = succeeded6505worker1 # [ 138.256903] nix-daemon[905]: accepted connection from pid 1978, user nix-grpc-daemon (trusted)6506worker1 # [ 138.272716] nix-daemon[905]: reaped child process 1981, status = succeeded6507lb # [ 138.286964] niks3-server[1021]: 2026/09/20 13:29:58 INFO Received uploads request method=POST path=/api/pending_closures6508worker1 # [ 138.294465] nix-grpc-daemon[1802]: time=2026-09-20T13:29:57.907Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6509worker1 # [ 138.297200] nix-grpc-daemon[1802]: time=2026-09-20T13:29:57.909Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"6510lb # [ 138.317952] niks3-server[1021]: 2026/09/20 13:29:58 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst6511lb # [ 138.323205] niks3-server[1021]: 2026/09/20 13:29:58 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv6512lb # [ 138.329500] niks3-server[1021]: 2026/09/20 13:29:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6513lb # [ 138.332720] niks3-server[1021]: 2026/09/20 13:29:58 INFO Signed narinfos id=55 count=16514worker1 # [ 138.333030] nix-grpc-daemon[1802]: time=2026-09-20T13:29:57.946Z level=INFO msg="Uploading 1 narinfos"6515lb # [ 138.337773] niks3-server[1021]: 2026/09/20 13:29:58 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls6516lb # [ 138.345432] niks3-server[1021]: 2026/09/20 13:29:58 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6517lb # [ 138.350978] niks3-server[1021]: 2026/09/20 13:29:58 INFO Completed upload id=556518worker1 # [ 138.351145] nix-grpc-daemon[1802]: time=2026-09-20T13:29:57.964Z level=INFO msg="Upload complete. (261ms)"6519lb # [ 138.353247] niks3-server[1021]: 2026/09/20 13:29:58 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo6520worker1 # [ 138.355479] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49530 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=11 outputs=16521worker2 # [ 138.387303] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:58Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=16522worker1 # [ 138.363655] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:495306523worker2 # [ 138.395598] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:58Z level=debug event=want client=5 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=06524worker1 # [ 138.368083] nix-daemon[905]: reaped child process 1961, status = succeeded6525worker1 # [ 138.373635] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:495306526worker1 # [ 138.381990] nix-grpc-daemon[1802]: time=2026-09-20T13:29:57.995Z level=INFO msg="All 1 paths already cached"6527worker1 # [ 138.387877] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:495306528worker1 # [ 138.394628] nix-daemon[905]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6529worker1 # [ 138.543632] nix-daemon[905]: accepted connection from pid 1985, user nix-grpc-daemon (trusted)6530worker1 # [ 138.559301] nix-daemon[905]: reaped child process 1988, status = succeeded6531worker1 # [ 138.726830] nix-daemon[905]: accepted connection from pid 1990, user nix-grpc-daemon (trusted)6532worker1 # [ 138.746264] nix-daemon[905]: reaped child process 1993, status = succeeded6533lb # [ 138.760584] niks3-server[1021]: 2026/09/20 13:29:58 INFO Received uploads request method=POST path=/api/pending_closures6534worker1 # [ 138.765919] nix-grpc-daemon[1802]: time=2026-09-20T13:29:58.379Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6535worker1 # [ 138.767716] nix-grpc-daemon[1802]: time=2026-09-20T13:29:58.380Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"6536lb # [ 138.783450] niks3-server[1021]: 2026/09/20 13:29:58 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst6537lb # [ 138.789203] niks3-server[1021]: 2026/09/20 13:29:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6538lb # [ 138.792472] niks3-server[1021]: 2026/09/20 13:29:58 INFO Signed narinfos id=56 count=16539worker1 # [ 138.792481] nix-grpc-daemon[1802]: time=2026-09-20T13:29:58.405Z level=INFO msg="Uploading 1 narinfos"6540lb # [ 138.797962] niks3-server[1021]: 2026/09/20 13:29:58 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls6541lb # [ 138.805169] niks3-server[1021]: 2026/09/20 13:29:58 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6542lb # [ 138.810180] niks3-server[1021]: 2026/09/20 13:29:58 INFO Completed upload id=566543worker1 # [ 138.810271] nix-grpc-daemon[1802]: time=2026-09-20T13:29:58.423Z level=INFO msg="Upload complete. (407ms)"6544worker1 # [ 138.813092] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49530 duration_s=0 paths=1 nar_bytes_in=6246545lb # [ 138.813549] niks3-server[1021]: 2026/09/20 13:29:58 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo6546worker1 # [ 138.816939] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:495306547worker1 # [ 138.825486] nix-daemon[905]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6548worker1 # [ 138.826995] nix-daemon[905]: reaped child process 1983, status = succeeded6549worker1 # [ 138.832905] nix-daemon[905]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6550worker1 # [ 139.851844] nix-daemon[905]: reaped child process 1997, status = succeeded6551worker1 # [ 139.994544] nix-daemon[905]: accepted connection from pid 2012, user nix-grpc-daemon (trusted)6552worker1 # [ 140.010081] nix-daemon[905]: reaped child process 2015, status = succeeded6553lb # [ 140.023380] niks3-server[1021]: 2026/09/20 13:30:00 INFO Received uploads request method=POST path=/api/pending_closures6554worker1 # [ 140.030924] nix-grpc-daemon[1802]: time=2026-09-20T13:29:59.643Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6555worker1 # [ 140.034052] nix-grpc-daemon[1802]: time=2026-09-20T13:29:59.645Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"6556lb # [ 140.050941] niks3-server[1021]: 2026/09/20 13:30:00 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst6557lb # [ 140.058442] niks3-server[1021]: 2026/09/20 13:30:00 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv6558lb # [ 140.063374] niks3-server[1021]: 2026/09/20 13:30:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6559lb # [ 140.066708] niks3-server[1021]: 2026/09/20 13:30:00 INFO Signed narinfos id=57 count=16560worker1 # [ 140.067183] nix-grpc-daemon[1802]: time=2026-09-20T13:29:59.680Z level=INFO msg="Uploading 1 narinfos"6561lb # [ 140.072650] niks3-server[1021]: 2026/09/20 13:30:00 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls6562lb # [ 140.081203] niks3-server[1021]: 2026/09/20 13:30:00 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6563lb # [ 140.084778] niks3-server[1021]: 2026/09/20 13:30:00 INFO Completed upload id=576564worker1 # [ 140.086913] nix-grpc-daemon[1802]: time=2026-09-20T13:29:59.700Z level=INFO msg="Upload complete. (246ms)"6565lb # [ 140.087918] niks3-server[1021]: 2026/09/20 13:30:00 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo6566worker1 # [ 140.090605] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49530 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=12 outputs=16567worker2 # [ 140.122529] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:59Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=16568worker1 # [ 140.098662] nix-daemon[905]: reaped child process 1995, status = succeeded6569worker2 # [ 140.129859] nix-grpc-daemon[1435]: ts=2026-09-20T13:29:59Z level=debug event=want client=5 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=06570worker1 # [ 140.141441] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:495306571worker1 # [ 140.154775] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:495306572worker1 # [ 140.167758] nix-grpc-daemon[1802]: time=2026-09-20T13:29:59.780Z level=INFO msg="All 2 paths already cached"6573worker1 # [ 140.176176] nix-grpc-daemon[1746]: ts=2026-09-20T13:29:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:495306574worker1 # [ 140.186373] nix-daemon[905]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6575worker1 # [ 140.345654] nix-daemon[905]: accepted connection from pid 2019, user nix-grpc-daemon (trusted)6576worker1 # [ 140.361572] nix-daemon[905]: reaped child process 2022, status = succeeded6577worker1 # [ 140.520652] nix-daemon[905]: accepted connection from pid 2024, user nix-grpc-daemon (trusted)6578worker1 # [ 140.535656] nix-daemon[905]: reaped child process 2027, status = succeeded6579lb # [ 140.550073] niks3-server[1021]: 2026/09/20 13:30:00 INFO Received uploads request method=POST path=/api/pending_closures6580worker1 # [ 140.555343] nix-grpc-daemon[1802]: time=2026-09-20T13:30:00.168Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6581worker1 # [ 140.557141] nix-grpc-daemon[1802]: time=2026-09-20T13:30:00.170Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"6582lb # [ 140.572705] niks3-server[1021]: 2026/09/20 13:30:00 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst6583lb # [ 140.579902] niks3-server[1021]: 2026/09/20 13:30:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6584lb # [ 140.583512] niks3-server[1021]: 2026/09/20 13:30:00 INFO Signed narinfos id=58 count=16585worker1 # [ 140.583559] nix-grpc-daemon[1802]: time=2026-09-20T13:30:00.196Z level=INFO msg="Uploading 1 narinfos"6586lb # [ 140.587616] niks3-server[1021]: 2026/09/20 13:30:00 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls6587lb # [ 140.596476] niks3-server[1021]: 2026/09/20 13:30:00 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6588lb # [ 140.601617] niks3-server[1021]: 2026/09/20 13:30:00 INFO Completed upload id=586589worker1 # [ 140.603924] nix-grpc-daemon[1802]: time=2026-09-20T13:30:00.217Z level=INFO msg="Upload complete. (405ms)"6590lb # [ 140.604604] niks3-server[1021]: 2026/09/20 13:30:00 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo6591worker1 # [ 140.607291] nix-grpc-daemon[1746]: ts=2026-09-20T13:30:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49530 duration_s=0 paths=1 nar_bytes_in=7526592worker1 # [ 140.611331] nix-grpc-daemon[1746]: ts=2026-09-20T13:30:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:495306593worker1 # [ 140.618302] nix-daemon[905]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6594worker1 # [ 140.625544] nix-daemon[905]: reaped child process 2017, status = succeeded6595worker1 # [ 140.633213] nix-daemon[905]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6596worker1 # [ 141.853575] nix-daemon[905]: reaped child process 2031, status = succeeded6597worker1 # [ 141.980155] nix-daemon[905]: accepted connection from pid 2046, user nix-grpc-daemon (trusted)6598worker1 # [ 141.998941] nix-daemon[905]: reaped child process 2049, status = succeeded6599lb # [ 142.011737] niks3-server[1021]: 2026/09/20 13:30:02 INFO Received uploads request method=POST path=/api/pending_closures6600worker1 # [ 142.021519] nix-grpc-daemon[1802]: time=2026-09-20T13:30:01.634Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6601worker1 # [ 142.025609] nix-grpc-daemon[1802]: time=2026-09-20T13:30:01.636Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"6602lb # [ 142.053898] niks3-server[1021]: 2026/09/20 13:30:02 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv6603lb # [ 142.057374] niks3-server[1021]: 2026/09/20 13:30:02 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst6604lb # [ 142.063290] niks3-server[1021]: 2026/09/20 13:30:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6605lb # [ 142.066867] niks3-server[1021]: 2026/09/20 13:30:02 INFO Signed narinfos id=59 count=16606worker1 # [ 142.066948] nix-grpc-daemon[1802]: time=2026-09-20T13:30:01.679Z level=INFO msg="Uploading 1 narinfos"6607lb # [ 142.071716] niks3-server[1021]: 2026/09/20 13:30:02 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls6608lb # [ 142.079900] niks3-server[1021]: 2026/09/20 13:30:02 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6609lb # [ 142.084289] niks3-server[1021]: 2026/09/20 13:30:02 INFO Completed upload id=596610worker1 # [ 142.084348] nix-grpc-daemon[1802]: time=2026-09-20T13:30:01.697Z level=INFO msg="Upload complete. (240ms)"6611lb # [ 142.088806] niks3-server[1021]: 2026/09/20 13:30:02 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo6612worker2 # [ 142.118769] nix-grpc-daemon[1435]: ts=2026-09-20T13:30:01Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=16613worker1 # [ 142.088477] nix-grpc-daemon[1746]: ts=2026-09-20T13:30:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49530 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=13 outputs=16614worker1 # [ 142.094564] nix-grpc-daemon[1746]: ts=2026-09-20T13:30:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:495306615worker1 # [ 142.100716] nix-daemon[905]: reaped child process 2029, status = succeeded6616worker1 # [ 142.107048] nix-grpc-daemon[1746]: ts=2026-09-20T13:30:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:495306617worker1 # [ 142.115678] nix-grpc-daemon[1802]: time=2026-09-20T13:30:01.728Z level=INFO msg="All 2 paths already cached"6618worker1 # [ 142.121576] nix-grpc-daemon[1746]: ts=2026-09-20T13:30:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:495306619worker1 # [ 142.125727] nix-daemon[905]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6620worker1 # [ 142.281887] nix-daemon[905]: accepted connection from pid 2053, user nix-grpc-daemon (trusted)6621worker1 # [ 142.297853] nix-daemon[905]: reaped child process 2056, status = succeeded6622worker1 # [ 142.461994] nix-daemon[905]: accepted connection from pid 2058, user nix-grpc-daemon (trusted)6623worker1 # [ 142.478588] nix-daemon[905]: reaped child process 2061, status = succeeded6624lb # [ 142.491037] niks3-server[1021]: 2026/09/20 13:30:02 INFO Received uploads request method=POST path=/api/pending_closures6625worker1 # [ 142.496893] nix-grpc-daemon[1802]: time=2026-09-20T13:30:02.110Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6626worker1 # [ 142.498793] nix-grpc-daemon[1802]: time=2026-09-20T13:30:02.112Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"6627lb # [ 142.518527] niks3-server[1021]: 2026/09/20 13:30:02 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst6628lb # [ 142.521285] niks3-server[1021]: 2026/09/20 13:30:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6629lb # [ 142.524181] niks3-server[1021]: 2026/09/20 13:30:02 INFO Signed narinfos id=60 count=16630worker1 # [ 142.526530] nix-grpc-daemon[1802]: time=2026-09-20T13:30:02.139Z level=INFO msg="Uploading 1 narinfos"6631lb # [ 142.530900] niks3-server[1021]: 2026/09/20 13:30:02 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls6632lb # [ 142.538550] niks3-server[1021]: 2026/09/20 13:30:02 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6633lb # [ 142.543469] niks3-server[1021]: 2026/09/20 13:30:02 INFO Completed upload id=606634worker1 # [ 142.544083] nix-grpc-daemon[1802]: time=2026-09-20T13:30:02.157Z level=INFO msg="Upload complete. (411ms)"6635worker1 # [ 142.547090] nix-grpc-daemon[1746]: ts=2026-09-20T13:30:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49530 duration_s=0 paths=1 nar_bytes_in=7526636lb # [ 142.549166] niks3-server[1021]: 2026/09/20 13:30:02 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo6637worker1 # [ 142.551892] nix-grpc-daemon[1746]: ts=2026-09-20T13:30:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:495306638worker1 # [ 142.558748] nix-daemon[905]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6639worker1 # [ 142.562429] nix-daemon[905]: reaped child process 2051, status = succeeded6640worker1 # [ 142.569150] nix-daemon[905]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6641worker1 # [ 143.390908] nix-daemon[905]: reaped child process 2065, status = succeeded6642worker1 # [ 143.537269] nix-daemon[905]: accepted connection from pid 2080, user nix-grpc-daemon (trusted)6643worker1 # [ 143.553755] nix-daemon[905]: reaped child process 2083, status = succeeded6644lb # [ 143.567488] niks3-server[1021]: 2026/09/20 13:30:03 INFO Received uploads request method=POST path=/api/pending_closures6645worker1 # [ 143.573523] nix-grpc-daemon[1802]: time=2026-09-20T13:30:03.186Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6646worker1 # [ 143.576392] nix-grpc-daemon[1802]: time=2026-09-20T13:30:03.188Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"6647lb # [ 143.593059] niks3-server[1021]: 2026/09/20 13:30:03 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst6648lb # [ 143.597577] niks3-server[1021]: 2026/09/20 13:30:03 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv6649lb # [ 143.603269] niks3-server[1021]: 2026/09/20 13:30:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6650lb # [ 143.606170] niks3-server[1021]: 2026/09/20 13:30:03 INFO Signed narinfos id=61 count=16651worker1 # [ 143.608236] nix-grpc-daemon[1802]: time=2026-09-20T13:30:03.221Z level=INFO msg="Uploading 1 narinfos"6652lb # [ 143.615184] niks3-server[1021]: 2026/09/20 13:30:03 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls6653lb # [ 143.619903] niks3-server[1021]: 2026/09/20 13:30:03 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6654lb # [ 143.623578] niks3-server[1021]: 2026/09/20 13:30:03 INFO Completed upload id=616655worker1 # [ 143.623633] nix-grpc-daemon[1802]: time=2026-09-20T13:30:03.236Z level=INFO msg="Upload complete. (245ms)"6656worker1 # [ 143.627393] nix-grpc-daemon[1746]: ts=2026-09-20T13:30:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49530 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=14 outputs=16657lb # [ 143.628269] niks3-server[1021]: 2026/09/20 13:30:03 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo6658worker2 # [ 143.658391] nix-grpc-daemon[1435]: ts=2026-09-20T13:30:03Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=16659worker1 # [ 143.632881] nix-grpc-daemon[1746]: ts=2026-09-20T13:30:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:495306660worker1 # [ 143.637907] nix-daemon[905]: reaped child process 2063, status = succeeded6661worker1 # [ 143.643381] nix-grpc-daemon[1746]: ts=2026-09-20T13:30:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:495306662worker1 # [ 143.650815] nix-grpc-daemon[1802]: time=2026-09-20T13:30:03.264Z level=INFO msg="All 2 paths already cached"6663worker1 # [ 143.656278] nix-grpc-daemon[1746]: ts=2026-09-20T13:30:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:495306664worker1 # [ 143.660361] nix-daemon[905]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6665worker1 # [ 143.810119] nix-daemon[905]: accepted connection from pid 2087, user nix-grpc-daemon (trusted)6666worker1 # [ 143.826230] nix-daemon[905]: reaped child process 2090, status = succeeded6667worker1 # [ 143.974715] nix-daemon[905]: accepted connection from pid 2092, user nix-grpc-daemon (trusted)6668worker1 # [ 143.990460] nix-daemon[905]: reaped child process 2095, status = succeeded6669lb # [ 144.001097] niks3-server[1021]: 2026/09/20 13:30:04 INFO Received uploads request method=POST path=/api/pending_closures6670worker1 # [ 144.006152] nix-grpc-daemon[1802]: time=2026-09-20T13:30:03.619Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6671worker1 # [ 144.007698] nix-grpc-daemon[1802]: time=2026-09-20T13:30:03.619Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"6672lb # [ 144.017713] niks3-server[1021]: 2026/09/20 13:30:04 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst6673lb # [ 144.022988] niks3-server[1021]: 2026/09/20 13:30:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6674lb # [ 144.026162] niks3-server[1021]: 2026/09/20 13:30:04 INFO Signed narinfos id=62 count=16675worker1 # [ 144.025978] nix-grpc-daemon[1802]: time=2026-09-20T13:30:03.639Z level=INFO msg="Uploading 1 narinfos"6676lb # [ 144.029156] niks3-server[1021]: 2026/09/20 13:30:04 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls6677lb # [ 144.035269] niks3-server[1021]: 2026/09/20 13:30:04 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6678lb # [ 144.040236] niks3-server[1021]: 2026/09/20 13:30:04 INFO Completed upload id=626679worker1 # [ 144.040265] nix-grpc-daemon[1802]: time=2026-09-20T13:30:03.653Z level=INFO msg="Upload complete. (372ms)"6680worker1 # [ 144.043091] nix-grpc-daemon[1746]: ts=2026-09-20T13:30:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49530 duration_s=0 paths=1 nar_bytes_in=7526681lb # [ 144.043970] niks3-server[1021]: 2026/09/20 13:30:04 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo6682worker1 # [ 144.046644] nix-grpc-daemon[1746]: ts=2026-09-20T13:30:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:495306683worker1 # [ 144.053707] nix-daemon[905]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6684worker1 # [ 144.057539] nix-daemon[905]: reaped child process 2085, status = succeeded6685worker1 # [ 144.063877] nix-daemon[905]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6686worker1 # [ 144.696114] nix-daemon[905]: reaped child process 2099, status = succeeded6687worker1 # [ 144.832098] nix-daemon[905]: accepted connection from pid 2114, user nix-grpc-daemon (trusted)6688worker1 # [ 144.845368] nix-daemon[905]: reaped child process 2117, status = succeeded6689lb # [ 144.857391] niks3-server[1021]: 2026/09/20 13:30:04 INFO Received uploads request method=POST path=/api/pending_closures6690worker1 # [ 144.862675] nix-grpc-daemon[1802]: time=2026-09-20T13:30:04.475Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6691worker1 # [ 144.865060] nix-grpc-daemon[1802]: time=2026-09-20T13:30:04.477Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"6692lb # [ 144.885994] niks3-server[1021]: 2026/09/20 13:30:04 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv6693lb # [ 144.891358] niks3-server[1021]: 2026/09/20 13:30:04 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst6694lb # [ 144.896263] niks3-server[1021]: 2026/09/20 13:30:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6695lb # [ 144.899209] niks3-server[1021]: 2026/09/20 13:30:04 INFO Signed narinfos id=63 count=16696worker1 # [ 144.899304] nix-grpc-daemon[1802]: time=2026-09-20T13:30:04.512Z level=INFO msg="Uploading 1 narinfos"6697lb # [ 144.902744] niks3-server[1021]: 2026/09/20 13:30:04 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls6698lb # [ 144.910075] niks3-server[1021]: 2026/09/20 13:30:04 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6699lb # [ 144.915053] niks3-server[1021]: 2026/09/20 13:30:04 INFO Completed upload id=636700worker1 # [ 144.915041] nix-grpc-daemon[1802]: time=2026-09-20T13:30:04.528Z level=INFO msg="Upload complete. (241ms)"6701lb # [ 144.918423] niks3-server[1021]: 2026/09/20 13:30:04 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo6702worker1 # [ 144.918410] nix-grpc-daemon[1746]: ts=2026-09-20T13:30:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49530 duration_s=0 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=15 outputs=16703worker1 # [ 144.925346] nix-daemon[905]: reaped child process 2097, status = succeeded6704client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 17.07 seconds)6705worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6706worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6707client: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&16708worker1 # [ 145.254074] nix-grpc-daemon[1746]: ts=2026-09-20T13:30:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:495306709worker2 # [ 145.290160] nix-grpc-daemon[1435]: ts=2026-09-20T13:30:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:407146710worker2 # [ 145.308603] nix-grpc-daemon[1435]: ts=2026-09-20T13:30:05Z level=debug event=want client=6 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=06711client: (finished: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&1, in 0.27 seconds)6712(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 17.39 seconds)6713subtest: low disk drains a worker and builds go to the other6714worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6715worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.00 seconds)6716worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6717worker1 # [ 146.701497] nix-grpc-daemon[1746]: ts=2026-09-20T13:30:06Z level=info event=unhealthy reason=min_free6718worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.10 seconds)6719client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&26720client # don't know how to build these paths:6721worker1 # [ 147.582588] nix-grpc-daemon[1746]: ts=2026-09-20T13:30:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:495306722client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6723worker2 # [ 147.618670] nix-grpc-daemon[1435]: ts=2026-09-20T13:30:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:407146724worker2 # [ 147.636126] nix-grpc-daemon[1435]: ts=2026-09-20T13:30:07Z level=debug event=want client=7 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=06725worker2 # [ 147.638987] nix-grpc-daemon[1435]: ts=2026-09-20T13:30:07Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=16726worker2 # [ 147.681126] nix-grpc-daemon[1435]: ts=2026-09-20T13:30:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:407146727worker2 # [ 147.692113] nix-grpc-daemon[1435]: ts=2026-09-20T13:30:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:407146728client # copying 1 paths to 'lb:50051'...6729client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...6730worker2 # [ 147.701871] nix-grpc-daemon[1435]: ts=2026-09-20T13:30:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:407146731worker2 # [ 147.707879] nix-daemon[915]: accepted connection from pid 1435, user nix-grpc-daemon (trusted)6732worker2 # [ 147.877578] nix-daemon[915]: accepted connection from pid 1717, user nix-grpc-daemon (trusted)6733worker2 # [ 147.895923] nix-daemon[915]: reaped child process 1720, status = succeeded6734worker2 # [ 148.054322] nix-daemon[915]: accepted connection from pid 1722, user nix-grpc-daemon (trusted)6735worker2 # [ 148.072521] nix-daemon[915]: reaped child process 1725, status = succeeded6736lb # [ 148.056445] niks3-server[1021]: 2026/09/20 13:30:08 INFO Received uploads request method=POST path=/api/pending_closures6737worker2 # [ 148.093142] nix-grpc-daemon[1511]: time=2026-09-20T13:30:07.839Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6738worker2 # [ 148.095453] nix-grpc-daemon[1511]: time=2026-09-20T13:30:07.841Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"6739lb # [ 148.076237] niks3-server[1021]: 2026/09/20 13:30:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6740lb # [ 148.078196] niks3-server[1021]: 2026/09/20 13:30:08 INFO Signed narinfos id=64 count=16741worker2 # [ 148.109298] nix-grpc-daemon[1511]: time=2026-09-20T13:30:07.856Z level=INFO msg="Uploading 1 narinfos"6742lb # [ 148.082392] niks3-server[1021]: 2026/09/20 13:30:08 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst6743lb # [ 148.085162] niks3-server[1021]: 2026/09/20 13:30:08 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls6744lb # [ 148.090159] niks3-server[1021]: 2026/09/20 13:30:08 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6745lb # [ 148.093222] niks3-server[1021]: 2026/09/20 13:30:08 INFO Completed upload id=646746worker2 # [ 148.122874] nix-grpc-daemon[1511]: time=2026-09-20T13:30:07.869Z level=INFO msg="Upload complete. (405ms)"6747lb # [ 148.096624] niks3-server[1021]: 2026/09/20 13:30:08 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo6748worker2 # [ 148.126099] nix-grpc-daemon[1435]: ts=2026-09-20T13:30:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40714 duration_s=0 paths=1 nar_bytes_in=5126749worker2 # [ 148.129144] nix-grpc-daemon[1435]: ts=2026-09-20T13:30:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:407146750worker2 # [ 148.137088] nix-daemon[915]: accepted connection from pid 1435, user nix-grpc-daemon (trusted)6751worker2 # [ 148.141677] nix-daemon[915]: reaped child process 1715, status = succeeded6752client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...6753worker2 # [ 148.147308] nix-daemon[915]: accepted connection from pid 1435, user nix-grpc-daemon (trusted)6754client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6755lb # [ 149.140175] envoy[930]: conn peer=[2001:db8:1::1]:44330 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27128 tx=10675 ms=168586756lb # [ 149.141811] envoy[930]: conn peer=[2001:db8:1::1]:57482 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=878 tx=481 ms=406757client # farm-a> LOG-farm-a-drain6758worker2 # [ 149.363626] nix-daemon[915]: reaped child process 1729, status = succeeded6759worker2 # [ 149.496382] nix-daemon[915]: accepted connection from pid 1744, user nix-grpc-daemon (trusted)6760worker2 # [ 149.509343] nix-daemon[915]: reaped child process 1747, status = succeeded6761lb # [ 149.496102] niks3-server[1021]: 2026/09/20 13:30:09 INFO Received uploads request method=POST path=/api/pending_closures6762worker2 # [ 149.533524] nix-grpc-daemon[1511]: time=2026-09-20T13:30:09.280Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6763worker2 # [ 149.535087] nix-grpc-daemon[1511]: time=2026-09-20T13:30:09.282Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"6764lb # [ 149.522302] niks3-server[1021]: 2026/09/20 13:30:09 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6765lb # [ 149.525974] niks3-server[1021]: 2026/09/20 13:30:09 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6766lb # [ 149.529458] niks3-server[1021]: 2026/09/20 13:30:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6767lb # [ 149.531177] niks3-server[1021]: 2026/09/20 13:30:09 INFO Signed narinfos id=65 count=16768worker2 # [ 149.562085] nix-grpc-daemon[1511]: time=2026-09-20T13:30:09.308Z level=INFO msg="Uploading 1 narinfos"6769lb # [ 149.537627] niks3-server[1021]: 2026/09/20 13:30:09 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls6770lb # [ 149.539118] niks3-server[1021]: 2026/09/20 13:30:09 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6771lb # [ 149.543336] niks3-server[1021]: 2026/09/20 13:30:09 INFO Completed upload id=656772worker2 # [ 149.572881] nix-grpc-daemon[1511]: time=2026-09-20T13:30:09.319Z level=INFO msg="Upload complete. (217ms)"6773lb # [ 149.545087] niks3-server[1021]: 2026/09/20 13:30:09 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo6774worker2 # [ 149.575624] nix-grpc-daemon[1435]: ts=2026-09-20T13:30:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40714 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=16 outputs=16775worker2 # [ 149.582653] nix-grpc-daemon[1435]: ts=2026-09-20T13:30:09Z level=debug event=want client=7 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=06776worker2 # [ 149.584765] nix-grpc-daemon[1435]: ts=2026-09-20T13:30:09Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=16777worker2 # [ 149.589145] nix-daemon[915]: reaped child process 1727, status = succeeded6778worker2 # [ 149.626641] nix-grpc-daemon[1435]: ts=2026-09-20T13:30:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:407146779worker2 # [ 149.633513] nix-grpc-daemon[1435]: ts=2026-09-20T13:30:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:407146780worker2 # [ 149.639704] nix-grpc-daemon[1511]: time=2026-09-20T13:30:09.386Z level=INFO msg="All 1 paths already cached"6781client # copying 1 paths to 'lb:50051'...6782client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...6783worker2 # [ 149.643979] nix-grpc-daemon[1435]: ts=2026-09-20T13:30:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:407146784worker2 # [ 149.649379] nix-daemon[915]: accepted connection from pid 1435, user nix-grpc-daemon (trusted)6785worker2 # [ 149.794186] nix-daemon[915]: accepted connection from pid 1753, user nix-grpc-daemon (trusted)6786worker2 # [ 149.808685] nix-daemon[915]: reaped child process 1756, status = succeeded6787worker2 # [ 149.962574] nix-daemon[915]: accepted connection from pid 1758, user nix-grpc-daemon (trusted)6788worker2 # [ 149.975348] nix-daemon[915]: reaped child process 1761, status = succeeded6789lb # [ 149.956858] niks3-server[1021]: 2026/09/20 13:30:09 INFO Received uploads request method=POST path=/api/pending_closures6790worker2 # [ 149.992374] nix-grpc-daemon[1511]: time=2026-09-20T13:30:09.739Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6791worker2 # [ 149.993780] nix-grpc-daemon[1511]: time=2026-09-20T13:30:09.739Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"6792lb # [ 149.975420] niks3-server[1021]: 2026/09/20 13:30:09 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst6793lb # [ 149.987539] niks3-server[1021]: 2026/09/20 13:30:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6794worker2 # [ 150.024253] nix-grpc-daemon[1511]: time=2026-09-20T13:30:09.771Z level=INFO msg="Uploading 1 narinfos"6795lb # [ 149.994345] niks3-server[1021]: 2026/09/20 13:30:10 INFO Signed narinfos id=66 count=16796lb # [ 150.002336] niks3-server[1021]: 2026/09/20 13:30:10 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls6797lb # [ 150.012269] niks3-server[1021]: 2026/09/20 13:30:10 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6798lb # [ 150.019049] niks3-server[1021]: 2026/09/20 13:30:10 INFO Completed upload id=666799worker2 # [ 150.048589] nix-grpc-daemon[1511]: time=2026-09-20T13:30:09.795Z level=INFO msg="Upload complete. (394ms)"6800lb # [ 150.021697] niks3-server[1021]: 2026/09/20 13:30:10 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo6801worker2 # [ 150.050704] nix-grpc-daemon[1435]: ts=2026-09-20T13:30:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40714 duration_s=0 paths=1 nar_bytes_in=6246802worker2 # [ 150.053678] nix-grpc-daemon[1435]: ts=2026-09-20T13:30:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:407146803worker2 # [ 150.060063] nix-daemon[915]: accepted connection from pid 1435, user nix-grpc-daemon (trusted)6804worker2 # [ 150.063814] nix-daemon[915]: reaped child process 1751, status = succeeded6805client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...6806worker2 # [ 150.069047] nix-daemon[915]: accepted connection from pid 1435, user nix-grpc-daemon (trusted)6807client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6808client # farm-b> LOG-farm-b-drain6809worker2 # [ 151.284662] nix-daemon[915]: reaped child process 1765, status = succeeded6810worker2 # [ 151.418775] nix-daemon[915]: accepted connection from pid 1780, user nix-grpc-daemon (trusted)6811worker2 # [ 151.431497] nix-daemon[915]: reaped child process 1783, status = succeeded6812lb # [ 151.417495] niks3-server[1021]: 2026/09/20 13:30:11 INFO Received uploads request method=POST path=/api/pending_closures6813worker2 # [ 151.452571] nix-grpc-daemon[1511]: time=2026-09-20T13:30:11.199Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6814worker2 # [ 151.454135] nix-grpc-daemon[1511]: time=2026-09-20T13:30:11.201Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"6815lb # [ 151.443377] niks3-server[1021]: 2026/09/20 13:30:11 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6816lb # [ 151.446326] niks3-server[1021]: 2026/09/20 13:30:11 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst6817lb # [ 151.451277] niks3-server[1021]: 2026/09/20 13:30:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6818lb # [ 151.453526] niks3-server[1021]: 2026/09/20 13:30:11 INFO Signed narinfos id=67 count=16819worker2 # [ 151.484846] nix-grpc-daemon[1511]: time=2026-09-20T13:30:11.231Z level=INFO msg="Uploading 1 narinfos"6820lb # [ 151.456732] niks3-server[1021]: 2026/09/20 13:30:11 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls6821lb # [ 151.461921] niks3-server[1021]: 2026/09/20 13:30:11 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6822lb # [ 151.465107] niks3-server[1021]: 2026/09/20 13:30:11 INFO Completed upload id=676823worker2 # [ 151.494648] nix-grpc-daemon[1511]: time=2026-09-20T13:30:11.241Z level=INFO msg="Upload complete. (218ms)"6824lb # [ 151.468151] niks3-server[1021]: 2026/09/20 13:30:11 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo6825worker2 # [ 151.499082] nix-grpc-daemon[1435]: ts=2026-09-20T13:30:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40714 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=17 outputs=16826worker2 # [ 151.504983] nix-grpc-daemon[1435]: ts=2026-09-20T13:30:11Z level=debug event=want client=7 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=06827worker2 # [ 151.506996] nix-grpc-daemon[1435]: ts=2026-09-20T13:30:11Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=16828worker2 # [ 151.512641] nix-daemon[915]: reaped child process 1763, status = succeeded6829worker2 # [ 151.549108] nix-grpc-daemon[1435]: ts=2026-09-20T13:30:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:407146830worker2 # [ 151.555913] nix-grpc-daemon[1435]: ts=2026-09-20T13:30:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:407146831worker2 # [ 151.562869] nix-grpc-daemon[1511]: time=2026-09-20T13:30:11.309Z level=INFO msg="All 2 paths already cached"6832client # copying 1 paths to 'lb:50051'...6833client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...6834worker2 # [ 151.568570] nix-grpc-daemon[1435]: ts=2026-09-20T13:30:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:407146835worker2 # [ 151.572616] nix-daemon[915]: accepted connection from pid 1435, user nix-grpc-daemon (trusted)6836worker2 # [ 151.710855] nix-daemon[915]: accepted connection from pid 1789, user nix-grpc-daemon (trusted)6837worker2 # [ 151.729059] nix-daemon[915]: reaped child process 1792, status = succeeded6838worker2 # [ 151.872428] nix-daemon[915]: accepted connection from pid 1794, user nix-grpc-daemon (trusted)6839worker2 # [ 151.887634] nix-daemon[915]: reaped child process 1797, status = succeeded6840lb # [ 151.873054] niks3-server[1021]: 2026/09/20 13:30:11 INFO Received uploads request method=POST path=/api/pending_closures6841worker2 # [ 151.907230] nix-grpc-daemon[1511]: time=2026-09-20T13:30:11.654Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6842worker2 # [ 151.908835] nix-grpc-daemon[1511]: time=2026-09-20T13:30:11.655Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"6843lb # [ 151.890542] niks3-server[1021]: 2026/09/20 13:30:11 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst6844lb # [ 151.893813] niks3-server[1021]: 2026/09/20 13:30:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6845lb # [ 151.896908] niks3-server[1021]: 2026/09/20 13:30:11 INFO Signed narinfos id=68 count=16846worker2 # [ 151.926492] nix-grpc-daemon[1511]: time=2026-09-20T13:30:11.673Z level=INFO msg="Uploading 1 narinfos"6847lb # [ 151.903377] niks3-server[1021]: 2026/09/20 13:30:11 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6848lb # [ 151.916335] niks3-server[1021]: 2026/09/20 13:30:11 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls6849lb # [ 151.948293] niks3-server[1021]: 2026/09/20 13:30:11 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo6850lb # [ 151.964289] niks3-server[1021]: 2026/09/20 13:30:11 INFO Completed upload id=686851worker2 # [ 151.994054] nix-grpc-daemon[1511]: time=2026-09-20T13:30:11.741Z level=INFO msg="Upload complete. (413ms)"6852worker2 # [ 151.996784] nix-grpc-daemon[1435]: ts=2026-09-20T13:30:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40714 duration_s=0 paths=1 nar_bytes_in=7526853worker2 # [ 152.000171] nix-grpc-daemon[1435]: ts=2026-09-20T13:30:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:407146854worker2 # [ 152.005275] nix-daemon[915]: accepted connection from pid 1435, user nix-grpc-daemon (trusted)6855worker2 # [ 152.014239] nix-daemon[915]: reaped child process 1787, status = succeeded6856client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...6857worker2 # [ 152.017733] nix-daemon[915]: accepted connection from pid 1435, user nix-grpc-daemon (trusted)6858client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6859client # farm-top> LOG-farm-top-drain6860worker2 # [ 153.436888] nix-daemon[915]: reaped child process 1801, status = succeeded6861worker2 # [ 153.586980] nix-daemon[915]: accepted connection from pid 1816, user nix-grpc-daemon (trusted)6862worker2 # [ 153.601663] nix-daemon[915]: reaped child process 1819, status = succeeded6863lb # [ 153.583059] niks3-server[1021]: 2026/09/20 13:30:13 INFO Received uploads request method=POST path=/api/pending_closures6864worker2 # [ 153.616826] nix-grpc-daemon[1511]: time=2026-09-20T13:30:13.363Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6865worker2 # [ 153.618383] nix-grpc-daemon[1511]: time=2026-09-20T13:30:13.365Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"6866lb # [ 153.603376] niks3-server[1021]: 2026/09/20 13:30:13 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6867lb # [ 153.606028] niks3-server[1021]: 2026/09/20 13:30:13 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst6868lb # [ 153.611700] niks3-server[1021]: 2026/09/20 13:30:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6869lb # [ 153.614312] niks3-server[1021]: 2026/09/20 13:30:13 INFO Signed narinfos id=69 count=16870worker2 # [ 153.645404] nix-grpc-daemon[1511]: time=2026-09-20T13:30:13.392Z level=INFO msg="Uploading 1 narinfos"6871lb # [ 153.616703] niks3-server[1021]: 2026/09/20 13:30:13 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls6872lb # [ 153.623455] niks3-server[1021]: 2026/09/20 13:30:13 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6873lb # [ 153.626962] niks3-server[1021]: 2026/09/20 13:30:13 INFO Completed upload id=696874worker2 # [ 153.656474] nix-grpc-daemon[1511]: time=2026-09-20T13:30:13.403Z level=INFO msg="Upload complete. (230ms)"6875lb # [ 153.629074] niks3-server[1021]: 2026/09/20 13:30:13 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo6876worker2 # [ 153.660044] nix-grpc-daemon[1435]: ts=2026-09-20T13:30:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40714 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=18 outputs=16877worker2 # [ 153.667959] nix-daemon[915]: reaped child process 1799, status = succeeded6878client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&2, in 6.30 seconds)6879client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath6880client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath, in 0.18 seconds)6881worker1: must succeed: rm /nix/.rw-store/fill6882worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.11 seconds)6883worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6884worker1 # [ 154.708085] nix-grpc-daemon[1746]: ts=2026-09-20T13:30:14Z level=info event=healthy reason=min_free6885worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.10 seconds)6886(finished: subtest: low disk drains a worker and builds go to the other, in 9.84 seconds)6887(finished: run the VM test script, in 156.06 seconds)6888test script finished in 156.20s6889cleanup6890kill QemuMachine (pid 49)6891client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6892kill QemuMachine (pid 50)6893lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6894kill QemuMachine (pid 51)6895worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6896kill QemuMachine (pid 52)6897worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6898(finished: cleanup, in 1.08 seconds)6899additionally exposed symbols:6900 client, lb, worker1, worker2,6901 vlan1,6902 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh