vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #420
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker1 # Disk image does not exist, creating the virtualisation disk image...13lb # Disk image does not exist, creating the virtualisation disk image...14worker1: QEMU running (pid 49)15worker1 # Formatting '/build/vm-state-worker1/tmp.5EhJH8wi2L', fmt=raw size=107374182416lb: QEMU running (pid 50)17worker1 # mke2fs 1.47.4 (6-Mar-2025)18lb # Formatting '/build/vm-state-lb/tmp.ipuX8evPat', fmt=raw size=107374182419worker1 # Discarding device blocks: 0/262144 done20lb # mke2fs 1.47.4 (6-Mar-2025)21client # Disk image does not exist, creating the virtualisation disk image...22worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes23client: QEMU running (pid 51)24lb # Discarding device blocks: 0/262144 done25client # Formatting '/build/vm-state-client/tmp.T6iyHfmCy4', fmt=raw size=107374182426lb # Creating filesystem with 262144 4k blocks and 65536 inodes27client # mke2fs 1.47.4 (6-Mar-2025)28lb # Filesystem UUID: 60ececdb-aeee-4912-a169-b0c6e1e407a329client # Discarding device blocks: 0/262144 done30lb # Superblock backups stored on blocks:31client # Creating filesystem with 262144 4k blocks and 65536 inodes32lb # 32768, 98304, 163840, 22937633client # Filesystem UUID: 8933627b-7980-4360-acaf-bd24ebb1868134lb # 35client # Superblock backups stored on blocks:36lb # Allocating group tables: 0/8 done37client # 32768, 98304, 163840, 22937638lb # Writing inode tables: 0/8 done39client # 40lb # Creating journal (8192 blocks): done41client # Allocating group tables: 0/8 done42lb # Writing superblocks and filesystem accounting information: 0/8 done43client # Writing inode tables: 0/8 done44lb # 45client # Creating journal (8192 blocks): done46lb # Virtualisation disk image created.47client # Writing superblocks and filesystem accounting information: 0/8 done48worker1 # Filesystem UUID: e463b681-a438-48f4-b982-c7a242630d9649client # 50worker1 # Superblock backups stored on blocks:51client # Virtualisation disk image created.52worker1 # 32768, 98304, 163840, 22937653worker1 # 54worker1 # Allocating group tables: 0/8 done55worker1 # Writing inode tables: 0/8 done56worker1 # Creating journal (8192 blocks): done57worker1 # Writing superblocks and filesystem accounting information: 0/8 done58worker1 # 59worker1 # Virtualisation disk image created.60worker2 # Disk image does not exist, creating the virtualisation disk image...61worker2: QEMU running (pid 52)62worker2 # Formatting '/build/vm-state-worker2/tmp.RewIWmLiBR', fmt=raw size=107374182463worker2 # mke2fs 1.47.4 (6-Mar-2025)64worker2 # Discarding device blocks: 0/262144 done65worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes66(finished: start all VMs, in 0.22 seconds)67worker2 # Filesystem UUID: cab73aa8-c5a6-4263-b468-f2e8e3244f7168client: waiting for unit network-addresses-eth1.service69worker2 # Superblock backups stored on blocks:70client: waiting for the VM to finish booting71worker2 # 32768, 98304, 163840, 22937672worker2 # 73worker2 # Allocating group tables: 0/8 done74worker2 # Writing inode tables: 0/8 done75worker2 # Creating journal (8192 blocks): done76worker2 # Writing superblocks and filesystem accounting information: 0/8 done77worker2 # 78worker2 # Virtualisation disk image created.79client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)80worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)81lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)82worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)83client # 84client # 85client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0086client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...87client # 88client # 89client # 90client # 91client # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB0092client # Press Ctrl-B to configure iPXE (PCI 00:08.0)...93worker1 # 94client # 95worker1 # 96client # 97worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0098lb # 99lb # 100worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...101worker1 # 102worker1 # 103worker1 # 104worker1 # 105lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00106worker1 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00107lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...108lb # 109lb # 110lb # 111lb # 112worker1 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...113lb # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00114worker1 # 115worker1 # 116lb # Press Ctrl-B to configure iPXE (PCI 00:08.0)...117lb # 118lb # 119worker2 # 120worker2 # 121worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00122client # Booting from ROM...123worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...124worker2 # 125worker2 # 126worker2 # 127worker2 # 128worker2 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00129worker2 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...130worker2 # 131worker2 # 132worker1 # Booting from ROM...133lb # Booting from ROM...134worker2 # Booting from ROM...135client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026136client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/8rqmizrzfz4wllvr68b1lm9xv8fjj8kq-nixos-system-client-test/init regInfo=/nix/store/2l4hi571xbq2m4iwvjkpzvhb8b10rcak-closure-info/registration console=ttyS0,115200n8 console=tty0137client # [ 0.000000] BIOS-provided physical RAM map:138client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable139client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved140client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved141client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable142client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved143client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved144client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved145client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved146client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved147client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved148client # [ 0.000000] NX (Execute Disable) protection: active149client # [ 0.000000] APIC: Static calls initialized150client # [ 0.000000] SMBIOS 2.8 present.151worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026152client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014153client # [ 0.000000] DMI: Memory slots populated: 1/1154client # [ 0.000000] Hypervisor detected: KVM155client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000156client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00157client # [ 0.000000] kvm-clock: using sched offset of 451723481 cycles158worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/gv3xz4nw7pwvgpdjpqcinnhx0qmm2mya-nixos-system-worker1-test/init regInfo=/nix/store/2bya3w5gn9rywqs4dmjkmdcipcnishg5-closure-info/registration console=ttyS0,115200n8 console=tty0159client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns160worker1 # [ 0.000000] BIOS-provided physical RAM map:161client # [ 0.000004] tsc: Detected 2400.012 MHz processor162worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable163client # [ 0.000804] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000164worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved165client # [ 0.000831] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs166worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved167client # [ 0.000833] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT168worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable169lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026170client # [ 0.002727] found SMP MP-table at [mem 0x000f5450-0x000f545f]171client # [ 0.002738] Using GB pages for direct mapping172worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved173client # [ 0.002824] RAMDISK: [mem 0x5e366000-0x5ffcffff]174worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved175client # [ 0.002832] ACPI: Early table checksum verification disabled176worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved177client # [ 0.002834] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )178worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved179client # [ 0.002838] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)180lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0qln5mjy767qjp5wbmxjg8jay8k05dsm-nixos-system-lb-test/init regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration console=ttyS0,115200n8 console=tty0181lb # [ 0.000000] BIOS-provided physical RAM map:182worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved183client # [ 0.002842] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)184lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable185worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved186client # [ 0.002848] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)187worker1 # [ 0.000000] NX (Execute Disable) protection: active188lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved189client # [ 0.002850] ACPI: FACS 0x000000005FFE0000 000040190worker1 # [ 0.000000] APIC: Static calls initialized191lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved192worker1 # [ 0.000000] SMBIOS 2.8 present.193client # [ 0.002852] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)194lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable195worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014196client # [ 0.002853] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)197lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved198worker1 # [ 0.000000] DMI: Memory slots populated: 1/1199worker1 # [ 0.000000] Hypervisor detected: KVM200lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved201client # [ 0.002855] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)202worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000203lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved204client # [ 0.002856] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)205worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00206lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved207client # [ 0.002858] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]208worker1 # [ 0.000000] kvm-clock: using sched offset of 470340058 cycles209lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved210client # [ 0.002859] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]211lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved212client # [ 0.002859] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]213worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns214lb # [ 0.000000] NX (Execute Disable) protection: active215worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor216lb # [ 0.000000] APIC: Static calls initialized217client # [ 0.002860] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]218lb # [ 0.000000] SMBIOS 2.8 present.219worker1 # [ 0.000814] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000220client # [ 0.002860] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]221lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014222client # [ 0.002861] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]223worker1 # [ 0.000841] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs224lb # [ 0.000000] DMI: Memory slots populated: 1/1225client # [ 0.002861] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]226lb # [ 0.000000] Hypervisor detected: KVM227worker1 # [ 0.000844] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT228client # [ 0.003093] No NUMA configuration found229lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000230worker1 # [ 0.002738] found SMP MP-table at [mem 0x000f5450-0x000f545f]231client # [ 0.003094] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]232lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00233worker1 # [ 0.002750] Using GB pages for direct mapping234lb # [ 0.000000] kvm-clock: using sched offset of 471586372 cycles235client # [ 0.003096] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]236client # [ 0.005336] Zone ranges:237worker1 # [ 0.002832] RAMDISK: [mem 0x5e367000-0x5ffcffff]238client # [ 0.005337] DMA [mem 0x0000000000001000-0x0000000000ffffff]239worker1 # [ 0.002840] ACPI: Early table checksum verification disabled240lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns241lb # [ 0.000004] tsc: Detected 2400.012 MHz processor242client # [ 0.005338] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]243worker1 # [ 0.002843] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )244client # [ 0.005340] Normal empty245lb # [ 0.000815] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000246client # [ 0.005340] Device empty247client # [ 0.005341] Movable zone start for each node248worker1 # [ 0.002847] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)249client # [ 0.005342] Early memory node ranges250lb # [ 0.000842] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs251client # [ 0.005342] node 0: [mem 0x0000000000001000-0x000000000009efff]252lb # [ 0.000845] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT253worker1 # [ 0.002852] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)254client # [ 0.005343] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]255lb # [ 0.002749] found SMP MP-table at [mem 0x000f5450-0x000f545f]256lb # [ 0.002760] Using GB pages for direct mapping257worker1 # [ 0.002860] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)258client # [ 0.005344] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]259lb # [ 0.002915] RAMDISK: [mem 0x5e367000-0x5ffcffff]260worker1 # [ 0.002862] ACPI: FACS 0x000000005FFE0000 000040261client # [ 0.005364] On node 0, zone DMA: 1 pages in unavailable ranges262lb # [ 0.002923] ACPI: Early table checksum verification disabled263worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026264worker1 # [ 0.002863] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)265client # [ 0.005633] On node 0, zone DMA: 97 pages in unavailable ranges266lb # [ 0.002925] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )267client # [ 0.032566] On node 0, zone DMA32: 40 pages in unavailable ranges268client # [ 0.033023] ACPI: PM-Timer IO Port: 0x608269worker1 # [ 0.002865] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)270lb # [ 0.002930] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)271client # [ 0.033034] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])272lb # [ 0.002934] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)273worker1 # [ 0.002866] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)274client # [ 0.033060] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23275lb # [ 0.002941] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)276client # [ 0.033063] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)277worker1 # [ 0.002868] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)278worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/zi95182s0r7r9dy0k5gafi46cyal071y-nixos-system-worker2-test/init regInfo=/nix/store/grzww4bbh5fzl9fk54zm1fhc18k3xb06-closure-info/registration console=ttyS0,115200n8 console=tty0279lb # [ 0.002943] ACPI: FACS 0x000000005FFE0000 000040280worker2 # [ 0.000000] BIOS-provided physical RAM map:281client # [ 0.033064] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)282worker1 # [ 0.002869] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]283worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable284lb # [ 0.002944] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)285client # [ 0.033065] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)286worker1 # [ 0.002870] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]287worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved288client # [ 0.033066] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)289worker1 # [ 0.002871] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]290lb # [ 0.002946] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)291worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved292client # [ 0.033067] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)293worker1 # [ 0.002871] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]294lb # [ 0.002947] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)295client # [ 0.033070] ACPI: Using ACPI (MADT) for SMP configuration information296worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable297worker1 # [ 0.002872] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]298client # [ 0.033071] ACPI: HPET id: 0x8086a201 base: 0xfed00000299lb # [ 0.002949] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)300worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved301client # [ 0.033074] TSC deadline timer available302worker1 # [ 0.002872] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]303lb # [ 0.002950] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]304client # [ 0.033078] CPU topo: Max. logical packages: 1305worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved306client # [ 0.033079] CPU topo: Max. logical dies: 1307worker1 # [ 0.002873] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]308lb # [ 0.002951] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]309worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved310client # [ 0.033079] CPU topo: Max. dies per package: 1311worker1 # [ 0.003100] No NUMA configuration found312client # [ 0.033083] CPU topo: Max. threads per core: 1313lb # [ 0.002952] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]314worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved315worker1 # [ 0.003101] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]316client # [ 0.033083] CPU topo: Num. cores per package: 1317worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved318lb # [ 0.002952] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]319worker1 # [ 0.003103] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]320client # [ 0.033084] CPU topo: Num. threads per package: 1321worker1 # [ 0.005594] Zone ranges:322worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved323lb # [ 0.002953] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]324client # [ 0.033084] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs325worker1 # [ 0.005595] DMA [mem 0x0000000000001000-0x0000000000ffffff]326worker2 # [ 0.000000] NX (Execute Disable) protection: active327lb # [ 0.002953] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]328client # [ 0.033101] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()329worker2 # [ 0.000000] APIC: Static calls initialized330worker1 # [ 0.005597] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]331worker2 # [ 0.000000] SMBIOS 2.8 present.332worker1 # [ 0.005598] Normal empty333lb # [ 0.002954] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]334client # [ 0.033131] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]335worker1 # [ 0.005599] Device empty336lb # [ 0.003178] No NUMA configuration found337worker1 # [ 0.005599] Movable zone start for each node338worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014339client # [ 0.033133] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]340worker1 # [ 0.005600] Early memory node ranges341lb # [ 0.003178] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]342worker2 # [ 0.000000] DMI: Memory slots populated: 1/1343client # [ 0.033135] [mem 0x60000000-0xafffffff] available for PCI devices344worker1 # [ 0.005600] node 0: [mem 0x0000000000001000-0x000000000009efff]345worker2 # [ 0.000000] Hypervisor detected: KVM346lb # [ 0.003181] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]347lb # [ 0.005437] Zone ranges:348client # [ 0.033136] Booting paravirtualized kernel on KVM349worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000350worker1 # [ 0.005602] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]351lb # [ 0.005438] DMA [mem 0x0000000000001000-0x0000000000ffffff]352worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00353worker1 # [ 0.005603] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]354client # [ 0.033139] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns355lb # [ 0.005439] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]356worker2 # [ 0.000000] kvm-clock: using sched offset of 480167299 cycles357lb # [ 0.005440] Normal empty358worker1 # [ 0.005622] On node 0, zone DMA: 1 pages in unavailable ranges359lb # [ 0.005441] Device empty360client # [ 0.037586] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1361worker1 # [ 0.005883] On node 0, zone DMA: 97 pages in unavailable ranges362lb # [ 0.005442] Movable zone start for each node363worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns364client # [ 0.039736] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152365lb # [ 0.005442] Early memory node ranges366worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor367worker1 # [ 0.032981] On node 0, zone DMA32: 40 pages in unavailable ranges368client # [ 0.039779] kvm-guest: PV spinlocks disabled, single CPU369lb # [ 0.005443] node 0: [mem 0x0000000000001000-0x000000000009efff]370worker1 # [ 0.033449] ACPI: PM-Timer IO Port: 0x608371worker2 # [ 0.000813] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000372lb # [ 0.005444] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]373worker1 # [ 0.033462] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])374worker2 # [ 0.000841] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs375lb # [ 0.005445] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]376worker1 # [ 0.033489] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23377worker2 # [ 0.000844] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT378lb # [ 0.005469] On node 0, zone DMA: 1 pages in unavailable ranges379worker1 # [ 0.033492] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)380worker2 # [ 0.002744] found SMP MP-table at [mem 0x000f5450-0x000f545f]381lb # [ 0.005743] On node 0, zone DMA: 97 pages in unavailable ranges382worker2 # [ 0.002756] Using GB pages for direct mapping383worker1 # [ 0.033493] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)384client # [ 0.039780] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/8rqmizrzfz4wllvr68b1lm9xv8fjj8kq-nixos-system-client-test/init regInfo=/nix/store/2l4hi571xbq2m4iwvjkpzvhb8b10rcak-closure-info/registration console=ttyS0,115200n8 console=tty0385lb # [ 0.032774] On node 0, zone DMA32: 40 pages in unavailable ranges386worker2 # [ 0.002900] RAMDISK: [mem 0x5e367000-0x5ffcffff]387lb # [ 0.033239] ACPI: PM-Timer IO Port: 0x608388worker1 # [ 0.033495] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)389worker2 # [ 0.002908] ACPI: Early table checksum verification disabled390lb # [ 0.033250] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])391worker2 # [ 0.002912] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )392client # [ 0.039879] Unknown kernel command line parameters "regInfo=/nix/store/2l4hi571xbq2m4iwvjkpzvhb8b10rcak-closure-info/registration", will be passed to user space.393worker1 # [ 0.033496] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)394client # [ 0.039891] random: crng init done395lb # [ 0.033276] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23396worker2 # [ 0.002916] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)397worker1 # [ 0.033496] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)398client # [ 0.039892] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes399lb # [ 0.033279] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)400lb # [ 0.033280] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)401worker2 # [ 0.002920] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)402worker1 # [ 0.033499] ACPI: Using ACPI (MADT) for SMP configuration information403client # [ 0.042046] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)404worker1 # [ 0.033500] ACPI: HPET id: 0x8086a201 base: 0xfed00000405lb # [ 0.033281] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)406worker2 # [ 0.002928] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)407client # [ 0.043366] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)408worker1 # [ 0.033505] TSC deadline timer available409worker2 # [ 0.002930] ACPI: FACS 0x000000005FFE0000 000040410lb # [ 0.033282] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)411client # [ 0.043398] Fallback order for Node 0: 0412worker1 # [ 0.033510] CPU topo: Max. logical packages: 1413lb # [ 0.033283] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)414client # [ 0.043401] Built 1 zonelists, mobility grouping on. Total pages: 393078415worker2 # [ 0.002932] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)416worker1 # [ 0.033510] CPU topo: Max. logical dies: 1417client # [ 0.043402] Policy zone: DMA32418lb # [ 0.033285] ACPI: Using ACPI (MADT) for SMP configuration information419worker1 # [ 0.033511] CPU topo: Max. dies per package: 1420worker2 # [ 0.002933] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)421client # [ 0.045920] mem auto-init: stack:all(zero), heap alloc:on, heap free:off422lb # [ 0.033286] ACPI: HPET id: 0x8086a201 base: 0xfed00000423worker1 # [ 0.033514] CPU topo: Max. threads per core: 1424lb # [ 0.033290] TSC deadline timer available425client # [ 0.049050] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1426worker2 # [ 0.002935] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)427worker1 # [ 0.033515] CPU topo: Num. cores per package: 1428lb # [ 0.033294] CPU topo: Max. logical packages: 1429client # [ 0.052394] allocated 3145728 bytes of page_ext430worker1 # [ 0.033515] CPU topo: Num. threads per package: 1431lb # [ 0.033295] CPU topo: Max. logical dies: 1432worker2 # [ 0.002936] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)433client # [ 0.062437] ftrace: allocating 48731 entries in 192 pages434lb # [ 0.033295] CPU topo: Max. dies per package: 1435worker1 # [ 0.033516] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs436client # [ 0.062438] ftrace: allocated 192 pages with 2 groups437lb # [ 0.033299] CPU topo: Max. threads per core: 1438worker2 # [ 0.002938] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]439client # [ 0.063281] Dynamic Preempt: lazy440lb # [ 0.033299] CPU topo: Num. cores per package: 1441worker1 # [ 0.033539] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()442client # [ 0.063447] rcu: Preemptible hierarchical RCU implementation.443worker2 # [ 0.002939] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]444lb # [ 0.033300] CPU topo: Num. threads per package: 1445client # [ 0.063447] rcu: RCU event tracing is enabled.446worker2 # [ 0.002939] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]447worker1 # [ 0.033575] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]448lb # [ 0.033300] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs449client # [ 0.063448] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.450worker2 # [ 0.002940] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]451lb # [ 0.033317] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()452worker1 # [ 0.033576] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]453client # [ 0.063450] Trampoline variant of Tasks RCU enabled.454client # [ 0.063450] Rude variant of Tasks RCU enabled.455worker2 # [ 0.002940] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]456worker1 # [ 0.033578] [mem 0x60000000-0xafffffff] available for PCI devices457lb # [ 0.033347] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]458client # [ 0.063450] Tracing variant of Tasks RCU enabled.459worker1 # [ 0.033580] Booting paravirtualized kernel on KVM460worker2 # [ 0.002941] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]461lb # [ 0.033349] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]462client # [ 0.063451] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.463worker2 # [ 0.002941] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]464lb # [ 0.033350] [mem 0x60000000-0xafffffff] available for PCI devices465worker1 # [ 0.033583] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns466worker2 # [ 0.003167] No NUMA configuration found467client # [ 0.063452] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1468lb # [ 0.033351] Booting paravirtualized kernel on KVM469worker2 # [ 0.003168] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]470worker1 # [ 0.038058] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1471client # [ 0.063472] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.472worker2 # [ 0.003170] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]473lb # [ 0.033354] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns474worker1 # [ 0.040578] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152475worker2 # [ 0.005403] Zone ranges:476client # [ 0.063474] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.477worker1 # [ 0.040618] kvm-guest: PV spinlocks disabled, single CPU478lb # [ 0.037828] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1479worker2 # [ 0.005404] DMA [mem 0x0000000000001000-0x0000000000ffffff]480lb # [ 0.040033] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152481client # [ 0.063475] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.482worker2 # [ 0.005406] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]483worker2 # [ 0.005407] Normal empty484client # [ 0.068111] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16485lb # [ 0.040077] kvm-guest: PV spinlocks disabled, single CPU486worker2 # [ 0.005408] Device empty487worker2 # [ 0.005408] Movable zone start for each node488client # [ 0.068405] rcu: srcu_init: Setting srcu_struct sizes based on contention.489worker2 # [ 0.005409] Early memory node ranges490worker2 # [ 0.005409] node 0: [mem 0x0000000000001000-0x000000000009efff]491client # [ 0.068412] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns492worker1 # [ 0.040620] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/gv3xz4nw7pwvgpdjpqcinnhx0qmm2mya-nixos-system-worker1-test/init regInfo=/nix/store/2bya3w5gn9rywqs4dmjkmdcipcnishg5-closure-info/registration console=ttyS0,115200n8 console=tty0493worker2 # [ 0.005410] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]494client # [ 0.068513] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)495worker2 # [ 0.005411] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]496client # [ 0.072032] Console: colour VGA+ 80x25497lb # [ 0.040079] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0qln5mjy767qjp5wbmxjg8jay8k05dsm-nixos-system-lb-test/init regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration console=ttyS0,115200n8 console=tty0498worker2 # [ 0.005431] On node 0, zone DMA: 1 pages in unavailable ranges499worker1 # [ 0.040716] Unknown kernel command line parameters "regInfo=/nix/store/2bya3w5gn9rywqs4dmjkmdcipcnishg5-closure-info/registration", will be passed to user space.500client # [ 0.072035] printk: legacy console [tty0] enabled501worker1 # [ 0.040730] random: crng init done502client # [ 0.101613] printk: legacy console [ttyS0] enabled503worker2 # [ 0.005695] On node 0, zone DMA: 97 pages in unavailable ranges504lb # [ 0.040171] Unknown kernel command line parameters "regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration", will be passed to user space.505worker2 # [ 0.032607] On node 0, zone DMA32: 40 pages in unavailable ranges506client # [ 0.197888] ACPI: Core revision 20250807507worker1 # [ 0.040731] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes508lb # [ 0.040183] random: crng init done509worker2 # [ 0.033065] ACPI: PM-Timer IO Port: 0x608510worker2 # [ 0.033079] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])511lb # [ 0.040184] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes512worker1 # [ 0.042892] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)513client # [ 0.198727] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns514worker2 # [ 0.033104] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23515lb # [ 0.042350] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)516worker1 # [ 0.043971] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)517client # [ 0.200237] APIC: Switch to symmetric I/O mode setup518worker2 # [ 0.033107] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)519worker1 # [ 0.044004] Fallback order for Node 0: 0520lb # [ 0.043700] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)521client # [ 0.201171] x2apic enabled522lb # [ 0.043731] Fallback order for Node 0: 0523worker2 # [ 0.033109] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)524worker1 # [ 0.044006] Built 1 zonelists, mobility grouping on. Total pages: 393078525client # [ 0.201930] APIC: Switched APIC routing to: physical x2apic526lb # [ 0.043733] Built 1 zonelists, mobility grouping on. Total pages: 393078527worker1 # [ 0.044007] Policy zone: DMA32528worker2 # [ 0.033110] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)529lb # [ 0.043735] Policy zone: DMA32530worker1 # [ 0.046789] mem auto-init: stack:all(zero), heap alloc:on, heap free:off531worker2 # [ 0.033111] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)532lb # [ 0.046268] mem auto-init: stack:all(zero), heap alloc:on, heap free:off533client # [ 0.203693] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1534worker2 # [ 0.033112] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)535lb # [ 0.049376] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1536worker1 # [ 0.050077] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1537lb # [ 0.052730] allocated 3145728 bytes of page_ext538worker1 # [ 0.053424] allocated 3145728 bytes of page_ext539worker2 # [ 0.033114] ACPI: Using ACPI (MADT) for SMP configuration information540lb # [ 0.062707] ftrace: allocating 48731 entries in 192 pages541worker2 # [ 0.033115] ACPI: HPET id: 0x8086a201 base: 0xfed00000542worker1 # [ 0.063549] ftrace: allocating 48731 entries in 192 pages543client # [ 0.204661] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns544worker2 # [ 0.033120] TSC deadline timer available545lb # [ 0.062709] ftrace: allocated 192 pages with 2 groups546worker1 # [ 0.063551] ftrace: allocated 192 pages with 2 groups547lb # [ 0.063531] Dynamic Preempt: lazy548worker2 # [ 0.033125] CPU topo: Max. logical packages: 1549worker1 # [ 0.064395] Dynamic Preempt: lazy550client # [ 0.206303] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)551lb # [ 0.063682] rcu: Preemptible hierarchical RCU implementation.552worker2 # [ 0.033126] CPU topo: Max. logical dies: 1553worker1 # [ 0.064563] rcu: Preemptible hierarchical RCU implementation.554worker2 # [ 0.033126] CPU topo: Max. dies per package: 1555lb # [ 0.063683] rcu: RCU event tracing is enabled.556worker1 # [ 0.064564] rcu: RCU event tracing is enabled.557worker2 # [ 0.033130] CPU topo: Max. threads per core: 1558client # [ 0.207385] x86/cpu: User Mode Instruction Prevention (UMIP) activated559lb # [ 0.063684] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.560worker2 # [ 0.033131] CPU topo: Num. cores per package: 1561worker1 # [ 0.064564] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.562lb # [ 0.063685] Trampoline variant of Tasks RCU enabled.563worker2 # [ 0.033131] CPU topo: Num. threads per package: 1564client # [ 0.208466] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127565worker1 # [ 0.064566] Trampoline variant of Tasks RCU enabled.566lb # [ 0.063685] Rude variant of Tasks RCU enabled.567worker1 # [ 0.064566] Rude variant of Tasks RCU enabled.568worker2 # [ 0.033131] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs569lb # [ 0.063685] Tracing variant of Tasks RCU enabled.570client # [ 0.209301] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0571worker1 # [ 0.064566] Tracing variant of Tasks RCU enabled.572worker2 # [ 0.033162] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()573lb # [ 0.063686] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.574worker1 # [ 0.064567] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.575worker2 # [ 0.033197] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]576client # [ 0.210304] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto577lb # [ 0.063687] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1578worker1 # [ 0.064568] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1579worker2 # [ 0.033198] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]580lb # [ 0.063700] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.581client # [ 0.211300] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl582worker1 # [ 0.064577] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.583worker2 # [ 0.033200] [mem 0x60000000-0xafffffff] available for PCI devices584worker2 # [ 0.033202] Booting paravirtualized kernel on KVM585lb # [ 0.063702] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.586client # [ 0.212301] Transient Scheduler Attacks: Vulnerable: No microcode587worker1 # [ 0.064579] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.588worker2 # [ 0.033205] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns589lb # [ 0.063703] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.590client # [ 0.213300] Spectre V2 : Mitigation: Enhanced / Automatic IBRS591worker1 # [ 0.064580] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.592lb # [ 0.068423] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16593worker2 # [ 0.037641] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1594worker1 # [ 0.068919] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16595client # [ 0.215170] Speculative Return Stack Overflow: Mitigation: Safe RET596lb # [ 0.068704] rcu: srcu_init: Setting srcu_struct sizes based on contention.597worker2 # [ 0.040065] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152598worker1 # [ 0.069192] rcu: srcu_init: Setting srcu_struct sizes based on contention.599worker2 # [ 0.040106] kvm-guest: PV spinlocks disabled, single CPU600lb # [ 0.068710] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns601worker1 # [ 0.069198] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns602lb # [ 0.068810] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)603worker1 # [ 0.069305] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)604lb # [ 0.072348] Console: colour VGA+ 80x25605worker1 # [ 0.072819] Console: colour VGA+ 80x25606lb # [ 0.072351] printk: legacy console [tty0] enabled607worker1 # [ 0.072822] printk: legacy console [tty0] enabled608lb # [ 0.102424] printk: legacy console [ttyS0] enabled609worker1 # [ 0.102184] printk: legacy console [ttyS0] enabled610worker2 # [ 0.040107] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/zi95182s0r7r9dy0k5gafi46cyal071y-nixos-system-worker2-test/init regInfo=/nix/store/grzww4bbh5fzl9fk54zm1fhc18k3xb06-closure-info/registration console=ttyS0,115200n8 console=tty0611lb # [ 0.199165] ACPI: Core revision 20250807612worker1 # [ 0.207226] ACPI: Core revision 20250807613client # [ 0.216301] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization614worker2 # [ 0.040209] Unknown kernel command line parameters "regInfo=/nix/store/grzww4bbh5fzl9fk54zm1fhc18k3xb06-closure-info/registration", will be passed to user space.615client # [ 0.218311] Spectre V2 : Enabling IBPB for BPF616lb # [ 0.200054] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns617worker2 # [ 0.040222] random: crng init done618worker1 # [ 0.208111] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns619lb # [ 0.201657] APIC: Switch to symmetric I/O mode setup620worker2 # [ 0.040223] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes621worker1 # [ 0.209676] APIC: Switch to symmetric I/O mode setup622client # [ 0.219302] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier623lb # [ 0.202600] x2apic enabled624worker1 # [ 0.210614] x2apic enabled625worker2 # [ 0.042378] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)626client # [ 0.220302] active return thunk: srso_alias_return_thunk627lb # [ 0.203364] APIC: Switched APIC routing to: physical x2apic628worker2 # [ 0.043453] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)629worker1 # [ 0.211367] APIC: Switched APIC routing to: physical x2apic630worker2 # [ 0.043486] Fallback order for Node 0: 0631worker2 # [ 0.043489] Built 1 zonelists, mobility grouping on. Total pages: 393078632client # [ 0.221328] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'633worker2 # [ 0.043490] Policy zone: DMA32634lb # [ 0.205162] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1635worker1 # [ 0.213150] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1636worker2 # [ 0.046012] mem auto-init: stack:all(zero), heap alloc:on, heap free:off637client # [ 0.222301] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'638worker2 # [ 0.049252] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1639lb # [ 0.206126] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns640client # [ 0.223301] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'641worker1 # [ 0.214120] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns642worker2 # [ 0.052837] allocated 3145728 bytes of page_ext643worker2 # [ 0.062745] ftrace: allocating 48731 entries in 192 pages644lb # [ 0.207748] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)645client # [ 0.225131] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'646worker2 # [ 0.062747] ftrace: allocated 192 pages with 2 groups647worker1 # [ 0.215758] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)648worker2 # [ 0.063602] Dynamic Preempt: lazy649lb # [ 0.208831] x86/cpu: User Mode Instruction Prevention (UMIP) activated650client # [ 0.226300] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'651worker2 # [ 0.063778] rcu: Preemptible hierarchical RCU implementation.652worker1 # [ 0.216842] x86/cpu: User Mode Instruction Prevention (UMIP) activated653worker2 # [ 0.063778] rcu: RCU event tracing is enabled.654lb # [ 0.209905] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127655client # [ 0.227301] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'656worker1 # [ 0.217940] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127657worker2 # [ 0.063779] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.658lb # [ 0.210745] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0659worker2 # [ 0.063780] Trampoline variant of Tasks RCU enabled.660worker1 # [ 0.218757] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0661client # [ 0.228301] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'662worker2 # [ 0.063781] Rude variant of Tasks RCU enabled.663worker2 # [ 0.063781] Tracing variant of Tasks RCU enabled.664client # [ 0.229301] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256665lb # [ 0.211749] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto666worker1 # [ 0.219760] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto667worker2 # [ 0.063782] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.668client # [ 0.231092] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64669worker2 # [ 0.063782] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1670lb # [ 0.212745] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl671worker1 # [ 0.221756] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl672client # [ 0.232102] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512673worker2 # [ 0.063798] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.674lb # [ 0.213745] Transient Scheduler Attacks: Vulnerable: No microcode675worker1 # [ 0.222756] Transient Scheduler Attacks: Vulnerable: No microcode676client # [ 0.233098] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024677worker2 # [ 0.063800] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.678lb # [ 0.214744] Spectre V2 : Mitigation: Enhanced / Automatic IBRS679worker1 # [ 0.223755] Spectre V2 : Mitigation: Enhanced / Automatic IBRS680lb # [ 0.215744] Speculative Return Stack Overflow: Mitigation: Safe RET681client # [ 0.234300] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8682worker2 # [ 0.063801] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.683worker1 # [ 0.224756] Speculative Return Stack Overflow: Mitigation: Safe RET684worker2 # [ 0.068208] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16685worker2 # [ 0.068478] rcu: srcu_init: Setting srcu_struct sizes based on contention.686client # [ 0.235300] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.687worker2 # [ 0.068484] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns688lb # [ 0.216744] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization689worker2 # [ 0.068589] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)690worker1 # [ 0.225755] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization691worker2 # [ 0.072083] Console: colour VGA+ 80x25692lb # [ 0.218752] Spectre V2 : Enabling IBPB for BPF693worker2 # [ 0.072086] printk: legacy console [tty0] enabled694worker1 # [ 0.227764] Spectre V2 : Enabling IBPB for BPF695worker2 # [ 0.101493] printk: legacy console [ttyS0] enabled696lb # [ 0.219746] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier697worker1 # [ 0.228756] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier698worker2 # [ 0.198176] ACPI: Core revision 20250807699lb # [ 0.220745] active return thunk: srso_alias_return_thunk700worker1 # [ 0.229756] active return thunk: srso_alias_return_thunk701lb # [ 0.221770] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'702worker1 # [ 0.230778] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'703worker2 # [ 0.199138] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns704lb # [ 0.222747] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'705worker2 # [ 0.201347] APIC: Switch to symmetric I/O mode setup706worker1 # [ 0.231757] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'707worker2 # [ 0.202291] x2apic enabled708lb # [ 0.224501] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'709worker1 # [ 0.232757] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'710worker2 # [ 0.203029] APIC: Switched APIC routing to: physical x2apic711lb # [ 0.225504] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'712worker1 # [ 0.234603] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'713lb # [ 0.226511] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'714worker1 # [ 0.235756] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'715worker2 # [ 0.204824] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1716lb # [ 0.227506] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'717worker1 # [ 0.236756] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'718lb # [ 0.228544] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'719worker2 # [ 0.205771] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns720worker1 # [ 0.237756] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'721lb # [ 0.229745] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256722worker2 # [ 0.207379] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)723worker1 # [ 0.238757] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256724lb # [ 0.230745] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64725worker2 # [ 0.208463] x86/cpu: User Mode Instruction Prevention (UMIP) activated726worker1 # [ 0.239756] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64727lb # [ 0.231744] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512728worker2 # [ 0.209493] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127729worker1 # [ 0.240756] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512730lb # [ 0.232744] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024731worker2 # [ 0.210376] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0732worker1 # [ 0.241756] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024733lb # [ 0.233745] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8734worker1 # [ 0.242756] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8735worker2 # [ 0.211380] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto736lb # [ 0.234744] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.737worker2 # [ 0.212376] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl738worker1 # [ 0.243756] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.739worker2 # [ 0.213376] Transient Scheduler Attacks: Vulnerable: No microcode740worker2 # [ 0.214376] Spectre V2 : Mitigation: Enhanced / Automatic IBRS741worker2 # [ 0.215376] Speculative Return Stack Overflow: Mitigation: Safe RET742worker2 # [ 0.216376] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization743worker2 # [ 0.218383] Spectre V2 : Enabling IBPB for BPF744worker2 # [ 0.219087] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier745worker2 # [ 0.220376] active return thunk: srso_alias_return_thunk746worker2 # [ 0.221221] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'747worker2 # [ 0.222376] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'748worker2 # [ 0.223376] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'749worker2 # [ 0.224318] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'750worker2 # [ 0.225130] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'751worker2 # [ 0.226109] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'752client # [ 0.272581] Freeing SMP alternatives memory: 44K753client # [ 0.273304] pid_max: default: 32768 minimum: 301754worker2 # [ 0.227151] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'755worker2 # [ 0.228376] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256756client # [ 0.274321] LSM: initializing lsm=capability,landlock,yama,bpf,ima757client # [ 0.275390] landlock: Up and running.758worker2 # [ 0.229376] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64759client # [ 0.276013] Yama: becoming mindful.760worker2 # [ 0.230298] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512761client # [ 0.276516] LSM support for eBPF active762worker2 # [ 0.231086] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024763client # [ 0.277416] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)764worker2 # [ 0.232082] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8765client # [ 0.278338] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)766worker2 # [ 0.233076] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.767client # [ 0.281045] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)768client # [ 0.281823] Performance Events: Fam17h+ core perfctr, AMD PMU driver.769client # [ 0.282305] ... version: 2770client # [ 0.283015] ... bit width: 48771client # [ 0.283302] ... generic counters: 6772client # [ 0.284009] ... generic bitmap: 000000000000003f773client # [ 0.284348] ... fixed-purpose counters: 0774client # [ 0.285049] ... fixed-purpose bitmap: 0000000000000000775client # [ 0.285303] ... value mask: 0000ffffffffffff776lb # [ 0.269499] Freeing SMP alternatives memory: 44K777client # [ 0.286193] ... max period: 00007fffffffffff778lb # [ 0.269747] pid_max: default: 32768 minimum: 301779client # [ 0.286977] ... global_ctrl mask: 000000000000003f780client # [ 0.287423] signal: max sigframe size: 3376781lb # [ 0.270832] LSM: initializing lsm=capability,landlock,yama,bpf,ima782lb # [ 0.271865] landlock: Up and running.783client # [ 0.288201] rcu: Hierarchical SRCU implementation.784lb # [ 0.272745] Yama: becoming mindful.785client # [ 0.288896] rcu: Max phase no-delay instances is 400.786worker1 # [ 0.282143] Freeing SMP alternatives memory: 44K787lb # [ 0.273776] LSM support for eBPF active788worker1 # [ 0.282761] pid_max: default: 32768 minimum: 301789lb # [ 0.274535] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)790worker1 # [ 0.283870] LSM: initializing lsm=capability,landlock,yama,bpf,ima791worker1 # [ 0.284864] landlock: Up and running.792lb # [ 0.275782] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)793worker1 # [ 0.285514] Yama: becoming mindful.794worker1 # [ 0.286438] LSM support for eBPF active795client # [ 0.293945] smp: Bringing up secondary CPUs ...796lb # [ 0.277957] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)797worker1 # [ 0.286866] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)798client # [ 0.294317] smp: Brought up 1 node, 1 CPU799client # [ 0.294978] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)800lb # [ 0.279298] Performance Events: Fam17h+ core perfctr, AMD PMU driver.801worker1 # [ 0.287793] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)802lb # [ 0.279753] ... version: 2803lb # [ 0.280446] ... bit width: 48804lb # [ 0.280747] ... generic counters: 6805client # [ 0.295466] Memory: 1454536K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109348K reserved, 0K cma-reserved)806worker1 # [ 0.290801] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)807lb # [ 0.281450] ... generic bitmap: 000000000000003f808client # [ 0.296573] devtmpfs: initialized809lb # [ 0.281747] ... fixed-purpose counters: 0810client # [ 0.297405] x86/mm: Memory block size: 128MB811worker1 # [ 0.292632] Performance Events: Fam17h+ core perfctr, AMD PMU driver.812lb # [ 0.282441] ... fixed-purpose bitmap: 0000000000000000813worker1 # [ 0.293598] ... version: 2814lb # [ 0.282747] ... value mask: 0000ffffffffffff815worker1 # [ 0.294318] ... bit width: 48816client # [ 0.299136] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)817lb # [ 0.283629] ... max period: 00007fffffffffff818worker1 # [ 0.294771] ... generic counters: 6819lb # [ 0.284426] ... global_ctrl mask: 000000000000003f820worker1 # [ 0.295472] ... generic bitmap: 000000000000003f821client # [ 0.300235] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).822lb # [ 0.284850] signal: max sigframe size: 3376823worker1 # [ 0.295758] ... fixed-purpose counters: 0824client # [ 0.301396] pinctrl core: initialized pinctrl subsystem825lb # [ 0.285711] rcu: Hierarchical SRCU implementation.826worker1 # [ 0.296488] ... fixed-purpose bitmap: 0000000000000000827client # [ 0.302531] PM: RTC time: 13:33:23, date: 2026-09-20828lb # [ 0.286387] rcu: Max phase no-delay instances is 400.829worker1 # [ 0.296760] ... value mask: 0000ffffffffffff830worker1 # [ 0.297641] ... max period: 00007fffffffffff831worker1 # [ 0.298416] ... global_ctrl mask: 000000000000003f832worker1 # [ 0.298872] signal: max sigframe size: 3376833client # [ 0.306048] NET: Registered PF_NETLINK/PF_ROUTE protocol family834worker2 # [ 0.268205] Freeing SMP alternatives memory: 44K835worker1 # [ 0.299650] rcu: Hierarchical SRCU implementation.836worker2 # [ 0.268922] pid_max: default: 32768 minimum: 301837worker1 # [ 0.300313] rcu: Max phase no-delay instances is 400.838client # [ 0.306833] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations839lb # [ 0.291231] smp: Bringing up secondary CPUs ...840worker2 # [ 0.269462] LSM: initializing lsm=capability,landlock,yama,bpf,ima841lb # [ 0.291761] smp: Brought up 1 node, 1 CPU842worker2 # [ 0.270479] landlock: Up and running.843client # [ 0.307324] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations844worker2 # [ 0.271377] Yama: becoming mindful.845lb # [ 0.292435] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)846worker2 # [ 0.272181] LSM support for eBPF active847client # [ 0.308582] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations848client # [ 0.309313] audit: initializing netlink subsys (disabled)849worker2 # [ 0.272972] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)850lb # [ 0.292920] Memory: 1454568K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)851worker1 # [ 0.305403] smp: Bringing up secondary CPUs ...852lb # [ 0.293982] devtmpfs: initialized853worker2 # [ 0.274274] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)854client # [ 0.310304] audit: type=2000 audit(1789911204.459:1): state=initialized audit_enabled=0 res=1855worker1 # [ 0.305774] smp: Brought up 1 node, 1 CPU856lb # [ 0.294791] x86/mm: Memory block size: 128MB857client # [ 0.311443] thermal_sys: Registered thermal governor 'fair_share'858worker1 # [ 0.306432] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)859client # [ 0.311445] thermal_sys: Registered thermal governor 'bang_bang'860worker2 # [ 0.276573] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)861lb # [ 0.296623] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)862client # [ 0.312304] thermal_sys: Registered thermal governor 'step_wise'863worker1 # [ 0.306909] Memory: 1454596K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)864worker2 # [ 0.277960] Performance Events: Fam17h+ core perfctr, AMD PMU driver.865client # [ 0.313271] thermal_sys: Registered thermal governor 'user_space'866worker1 # [ 0.308004] devtmpfs: initialized867lb # [ 0.297639] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).868worker2 # [ 0.278386] ... version: 2869client # [ 0.314031] thermal_sys: Registered thermal governor 'power_allocator'870worker1 # [ 0.308764] x86/mm: Memory block size: 128MB871lb # [ 0.298851] pinctrl core: initialized pinctrl subsystem872worker2 # [ 0.279084] ... bit width: 48873client # [ 0.314315] cpuidle: using governor menu874worker2 # [ 0.279379] ... generic counters: 6875lb # [ 0.299917] PM: RTC time: 13:33:23, date: 2026-09-20876worker2 # [ 0.280086] ... generic bitmap: 000000000000003f877client # [ 0.316504] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5878worker1 # [ 0.310529] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)879worker2 # [ 0.280380] ... fixed-purpose counters: 0880worker2 # [ 0.281077] ... fixed-purpose bitmap: 0000000000000000881worker1 # [ 0.311615] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).882client # [ 0.317527] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]883worker2 # [ 0.281378] ... value mask: 0000ffffffffffff884worker1 # [ 0.312854] pinctrl core: initialized pinctrl subsystem885client # [ 0.318305] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry886lb # [ 0.303459] NET: Registered PF_NETLINK/PF_ROUTE protocol family887worker2 # [ 0.282230] ... max period: 00007fffffffffff888worker1 # [ 0.313928] PM: RTC time: 13:33:23, date: 2026-09-20889client # [ 0.319314] PCI: Using configuration type 1 for base access890worker2 # [ 0.283031] ... global_ctrl mask: 000000000000003f891lb # [ 0.304239] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations892worker2 # [ 0.283479] signal: max sigframe size: 3376893client # [ 0.320400] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.894worker2 # [ 0.284293] rcu: Hierarchical SRCU implementation.895lb # [ 0.304771] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations896worker2 # [ 0.284948] rcu: Max phase no-delay instances is 400.897lb # [ 0.306025] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations898worker1 # [ 0.317557] NET: Registered PF_NETLINK/PF_ROUTE protocol family899lb # [ 0.306755] audit: initializing netlink subsys (disabled)900worker1 # [ 0.318988] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations901lb # [ 0.307894] thermal_sys: Registered thermal governor 'fair_share'902lb # [ 0.307896] thermal_sys: Registered thermal governor 'bang_bang'903worker1 # [ 0.319788] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations904lb # [ 0.308748] thermal_sys: Registered thermal governor 'step_wise'905worker2 # [ 0.290039] smp: Bringing up secondary CPUs ...906lb # [ 0.309695] thermal_sys: Registered thermal governor 'user_space'907client # [ 0.327327] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages908worker1 # [ 0.321113] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations909worker2 # [ 0.290395] smp: Brought up 1 node, 1 CPU910client # [ 0.328303] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page911worker1 # [ 0.321775] audit: initializing netlink subsys (disabled)912lb # [ 0.310462] audit: type=2000 audit(1789911204.477:1): state=initialized audit_enabled=0 res=1913worker2 # [ 0.291127] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)914lb # [ 0.311753] thermal_sys: Registered thermal governor 'power_allocator'915worker1 # [ 0.323199] thermal_sys: Registered thermal governor 'fair_share'916lb # [ 0.311768] cpuidle: using governor menu917worker2 # [ 0.291554] Memory: 1454564K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)918worker1 # [ 0.323202] thermal_sys: Registered thermal governor 'bang_bang'919lb # [ 0.313952] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5920worker2 # [ 0.292619] devtmpfs: initialized921worker1 # [ 0.323762] thermal_sys: Registered thermal governor 'step_wise'922worker2 # [ 0.293397] x86/mm: Memory block size: 128MB923client # [ 0.333303] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages924lb # [ 0.314978] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]925worker1 # [ 0.324763] audit: type=2000 audit(1789911204.478:1): state=initialized audit_enabled=0 res=1926client # [ 0.334303] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page927lb # [ 0.315750] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry928worker1 # [ 0.326764] thermal_sys: Registered thermal governor 'user_space'929worker2 # [ 0.295186] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)930lb # [ 0.316759] PCI: Using configuration type 1 for base access931worker1 # [ 0.326766] thermal_sys: Registered thermal governor 'power_allocator'932worker1 # [ 0.327780] cpuidle: using governor menu933worker2 # [ 0.296295] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).934lb # [ 0.317850] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.935worker2 # [ 0.297484] pinctrl core: initialized pinctrl subsystem936worker1 # [ 0.330063] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5937worker2 # [ 0.298717] PM: RTC time: 13:33:23, date: 2026-09-20938worker1 # [ 0.331134] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]939worker1 # [ 0.331762] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry940worker1 # [ 0.332772] PCI: Using configuration type 1 for base access941lb # [ 0.323039] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages942worker2 # [ 0.302122] NET: Registered PF_NETLINK/PF_ROUTE protocol family943lb # [ 0.323748] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page944worker1 # [ 0.333862] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.945worker2 # [ 0.302872] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations946client # [ 0.345542] ACPI: Added _OSI(Module Device)947worker2 # [ 0.303400] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations948client # [ 0.346295] ACPI: Added _OSI(Processor Device)949client # [ 0.347109] ACPI: Added _OSI(Processor Aggregator Device)950worker2 # [ 0.304641] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations951worker2 # [ 0.305389] audit: initializing netlink subsys (disabled)952lb # [ 0.328748] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages953worker2 # [ 0.306549] thermal_sys: Registered thermal governor 'fair_share'954lb # [ 0.329747] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page955worker2 # [ 0.306551] thermal_sys: Registered thermal governor 'bang_bang'956worker1 # [ 0.340884] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages957worker2 # [ 0.307379] thermal_sys: Registered thermal governor 'step_wise'958worker1 # [ 0.341760] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page959worker2 # [ 0.308337] thermal_sys: Registered thermal governor 'user_space'960worker2 # [ 0.309113] audit: type=2000 audit(1789911204.501:1): state=initialized audit_enabled=0 res=1961client # [ 0.353871] ACPI: 1 ACPI AML tables successfully acquired and loaded962worker2 # [ 0.310383] thermal_sys: Registered thermal governor 'power_allocator'963worker2 # [ 0.310406] cpuidle: using governor menu964worker1 # [ 0.346759] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages965worker2 # [ 0.312635] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5966worker1 # [ 0.347761] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page967client # [ 0.357706] ACPI: Interpreter enabled968worker2 # [ 0.313655] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]969client # [ 0.358335] ACPI: PM: (supports S0 S3 S4 S5)970lb # [ 0.340101] ACPI: Added _OSI(Module Device)971client # [ 0.359050] ACPI: Using IOAPIC for interrupt routing972worker2 # [ 0.314381] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry973lb # [ 0.340749] ACPI: Added _OSI(Processor Device)974worker2 # [ 0.315391] PCI: Using configuration type 1 for base access975lb # [ 0.341463] ACPI: Added _OSI(Processor Aggregator Device)976worker2 # [ 0.316505] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.977client # [ 0.361360] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug978client # [ 0.362303] PCI: Using E820 reservations for host bridge windows979client # [ 0.365402] ACPI: Enabled 2 GPEs in block 00 to 3F980worker1 # [ 0.357561] ACPI: Added _OSI(Module Device)981worker2 # [ 0.321516] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages982worker1 # [ 0.358242] ACPI: Added _OSI(Processor Device)983worker2 # [ 0.322380] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page984worker1 # [ 0.358760] ACPI: Added _OSI(Processor Aggregator Device)985lb # [ 0.349947] ACPI: 1 ACPI AML tables successfully acquired and loaded986worker2 # [ 0.327380] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages987client # [ 0.372695] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])988lb # [ 0.354193] ACPI: Interpreter enabled989worker2 # [ 0.328379] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page990lb # [ 0.354763] ACPI: PM: (supports S0 S3 S4 S5)991client # [ 0.373308] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]992lb # [ 0.355478] ACPI: Using IOAPIC for interrupt routing993client # [ 0.374380] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]994lb # [ 0.355799] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug995client # [ 0.375472] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]996worker1 # [ 0.368222] ACPI: 1 ACPI AML tables successfully acquired and loaded997lb # [ 0.356747] PCI: Using E820 reservations for host bridge windows998client # [ 0.376656] PCI host bridge to bus 0000:00999client # [ 0.377307] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1000lb # [ 0.359821] ACPI: Enabled 2 GPEs in block 00 to 3F1001client # [ 0.378302] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1002worker1 # [ 0.372117] ACPI: Interpreter enabled1003client # [ 0.379303] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1004worker1 # [ 0.372752] ACPI: PM: (supports S0 S3 S4 S5)1005client # [ 0.380302] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1006worker1 # [ 0.373443] ACPI: Using IOAPIC for interrupt routing1007worker2 # [ 0.338744] ACPI: Added _OSI(Module Device)1008client # [ 0.381303] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1009worker1 # [ 0.373793] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1010worker2 # [ 0.339381] ACPI: Added _OSI(Processor Device)1011worker1 # [ 0.374758] PCI: Using E820 reservations for host bridge windows1012client # [ 0.382302] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1013worker2 # [ 0.340170] ACPI: Added _OSI(Processor Aggregator Device)1014client # [ 0.383303] pci_bus 0000:00: root bus resource [bus 00-ff]1015client # [ 0.384264] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1016worker1 # [ 0.377843] ACPI: Enabled 2 GPEs in block 00 to 3F1017lb # [ 0.368518] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1018client # [ 0.385743] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1019lb # [ 0.368754] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1020lb # [ 0.370576] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1021lb # [ 0.371813] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1022lb # [ 0.373124] PCI host bridge to bus 0000:001023worker2 # [ 0.349997] ACPI: 1 ACPI AML tables successfully acquired and loaded1024lb # [ 0.373751] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1025worker1 # [ 0.386029] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1026lb # [ 0.374747] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1027worker1 # [ 0.386764] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1028client # [ 0.388414] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1029lb # [ 0.375748] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1030worker1 # [ 0.387840] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1031client # [ 0.389315] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1032worker2 # [ 0.353778] ACPI: Interpreter enabled1033lb # [ 0.376747] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1034worker2 # [ 0.354396] ACPI: PM: (supports S0 S3 S4 S5)1035worker1 # [ 0.388886] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1036client # [ 0.390247] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1037lb # [ 0.377747] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1038worker2 # [ 0.355106] ACPI: Using IOAPIC for interrupt routing1039worker1 # [ 0.390144] PCI host bridge to bus 0000:001040lb # [ 0.378747] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1041client # [ 0.391613] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1042worker1 # [ 0.390764] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1043lb # [ 0.379752] pci_bus 0000:00: root bus resource [bus 00-ff]1044worker1 # [ 0.391759] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1045client # [ 0.393012] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1046lb # [ 0.380735] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1047worker2 # [ 0.357418] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1048worker1 # [ 0.392758] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1049worker2 # [ 0.358379] PCI: Using E820 reservations for host bridge windows1050worker1 # [ 0.393760] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1051lb # [ 0.382203] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1052worker1 # [ 0.394760] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1053client # [ 0.396323] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1054worker2 # [ 0.361483] ACPI: Enabled 2 GPEs in block 00 to 3F1055client # [ 0.397180] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1056worker1 # [ 0.395765] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1057client # [ 0.398027] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1058worker1 # [ 0.396786] pci_bus 0000:00: root bus resource [bus 00-ff]1059client # [ 0.398308] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1060worker1 # [ 0.397761] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1061worker1 # [ 0.399220] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1062client # [ 0.400325] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1063lb # [ 0.384883] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1064client # [ 0.403320] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1065worker2 # [ 0.368952] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1066lb # [ 0.385762] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1067client # [ 0.404159] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1068worker2 # [ 0.369385] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1069lb # [ 0.386716] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1070client # [ 0.405033] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1071worker2 # [ 0.370457] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1072lb # [ 0.387914] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1073client # [ 0.405873] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1074worker2 # [ 0.371500] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1075worker2 # [ 0.372723] PCI host bridge to bus 0000:001076lb # [ 0.389449] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1077worker2 # [ 0.373383] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1078worker1 # [ 0.401872] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1079worker1 # [ 0.402771] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1080worker2 # [ 0.374379] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1081client # [ 0.408312] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1082worker1 # [ 0.403759] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1083worker2 # [ 0.375378] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1084client # [ 0.409186] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1085lb # [ 0.392770] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1086lb # [ 0.393621] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1087client # [ 0.410085] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1088worker2 # [ 0.376379] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1089worker1 # [ 0.404935] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1090lb # [ 0.394490] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1091worker2 # [ 0.377378] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1092client # [ 0.411683] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1093lb # [ 0.394754] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1094worker1 # [ 0.406558] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1095worker2 # [ 0.378378] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1096worker2 # [ 0.379383] pci_bus 0000:00: root bus resource [bus 00-ff]1097lb # [ 0.396315] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1098client # [ 0.414303] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1099worker2 # [ 0.380455] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1100client # [ 0.415148] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1101worker1 # [ 0.409785] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1102worker2 # [ 0.381852] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1103worker1 # [ 0.410675] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1104lb # [ 0.397775] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1105client # [ 0.416047] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1106lb # [ 0.398624] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1107worker1 # [ 0.411521] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1108lb # [ 0.399480] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1109worker1 # [ 0.412638] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1110client # [ 0.416857] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1111lb # [ 0.400351] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1112worker1 # [ 0.414083] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1113client # [ 0.419153] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1114client # [ 0.420064] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1115worker1 # [ 0.415778] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1116client # [ 0.420326] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1117worker1 # [ 0.416743] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1118lb # [ 0.402775] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1119worker2 # [ 0.384441] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1120worker1 # [ 0.417569] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1121client # [ 0.421884] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1122lb # [ 0.403759] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1123worker2 # [ 0.385391] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1124lb # [ 0.404778] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1125worker1 # [ 0.419156] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1126worker2 # [ 0.386348] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1127worker2 # [ 0.387553] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1128lb # [ 0.406444] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1129client # [ 0.425231] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1130client # [ 0.425931] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1131worker1 # [ 0.421767] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1132worker2 # [ 0.389106] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1133client # [ 0.426324] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1134worker1 # [ 0.422635] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1135worker1 # [ 0.423526] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1136client # [ 0.427903] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1137lb # [ 0.409786] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1138worker2 # [ 0.392406] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1139worker1 # [ 0.425294] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1140lb # [ 0.410760] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1141worker2 # [ 0.393293] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1142client # [ 0.429338] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1143lb # [ 0.411779] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1144worker2 # [ 0.394141] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1145client # [ 0.430190] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1146worker1 # [ 0.426788] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1147worker2 # [ 0.394386] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1148worker1 # [ 0.427642] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1149lb # [ 0.413441] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1150client # [ 0.431076] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1151worker2 # [ 0.395974] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1152client # [ 0.432329] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1153worker1 # [ 0.428526] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1154lb # [ 0.416758] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1155client # [ 0.433855] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1156worker1 # [ 0.430322] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1157worker2 # [ 0.397408] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1158lb # [ 0.417757] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1159worker2 # [ 0.398246] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1160worker1 # [ 0.431778] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1161client # [ 0.435342] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1162worker2 # [ 0.399158] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1163lb # [ 0.418778] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1164worker1 # [ 0.432616] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1165client # [ 0.436313] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1166worker1 # [ 0.433486] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1167worker2 # [ 0.400768] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1168lb # [ 0.420467] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1169client # [ 0.437841] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1170worker1 # [ 0.434359] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1171worker2 # [ 0.403566] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1172lb # [ 0.423761] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1173client # [ 0.440239] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1174worker2 # [ 0.404386] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1175lb # [ 0.424760] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1176client # [ 0.440943] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1177worker1 # [ 0.437012] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1178worker2 # [ 0.405353] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1179worker1 # [ 0.437766] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1180lb # [ 0.425777] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1181client # [ 0.441324] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1182worker1 # [ 0.438676] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1183worker2 # [ 0.406736] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1184client # [ 0.442872] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1185lb # [ 0.427468] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1186worker1 # [ 0.440138] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1187worker2 # [ 0.408387] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1188client # [ 0.445033] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1189worker2 # [ 0.409280] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1190worker1 # [ 0.441768] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1191client # [ 0.445310] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1192worker2 # [ 0.410165] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1193lb # [ 0.430695] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1194worker1 # [ 0.442637] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1195client # [ 0.446257] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1196lb # [ 0.431507] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1197worker1 # [ 0.443488] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1198worker1 # [ 0.443764] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1199worker2 # [ 0.411791] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1200lb # [ 0.432631] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1201client # [ 0.448105] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1202lb # [ 0.433758] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1203worker1 # [ 0.445796] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1204worker2 # [ 0.414388] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1205client # [ 0.450123] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1206worker2 # [ 0.415285] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1207worker1 # [ 0.447629] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1208client # [ 0.451151] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1209lb # [ 0.435536] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1210worker2 # [ 0.416141] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1211worker1 # [ 0.448501] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1212client # [ 0.452999] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1213worker2 # [ 0.417808] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1214lb # [ 0.437786] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1215worker1 # [ 0.449325] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1216lb # [ 0.438777] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1217worker1 # [ 0.450766] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1218worker2 # [ 0.420408] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1219worker1 # [ 0.451623] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1220worker2 # [ 0.421387] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1221lb # [ 0.440472] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1222worker1 # [ 0.452495] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1223worker2 # [ 0.422401] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1224worker1 # [ 0.454152] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1225worker2 # [ 0.424033] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1226lb # [ 0.442795] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1227worker1 # [ 0.455779] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1228lb # [ 0.443754] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1229worker1 # [ 0.456653] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1230worker2 # [ 0.425427] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1231lb # [ 0.444763] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1232worker2 # [ 0.426327] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1233worker1 # [ 0.457517] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1234client # [ 0.453532] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1235lb # [ 0.446327] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1236worker2 # [ 0.427154] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1237worker1 # [ 0.459192] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1238client # [ 0.456050] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1239worker2 # [ 0.428221] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1240lb # [ 0.447754] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1241worker1 # [ 0.460481] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1242client # [ 0.456527] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1243lb # [ 0.448621] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1244worker2 # [ 0.429935] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1245lb # [ 0.449470] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1246worker1 # [ 0.460974] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1247client # [ 0.457991] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1248worker2 # [ 0.431386] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1249worker1 # [ 0.462324] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1250client # [ 0.458573] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1251lb # [ 0.450348] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1252worker2 # [ 0.432379] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1253client # [ 0.459596] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1254lb # [ 0.451429] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1255worker2 # [ 0.433933] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1256client # [ 0.460566] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1257lb # [ 0.451958] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1258lb # [ 0.453356] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1259client # [ 0.462330] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1260worker2 # [ 0.435418] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1261client # [ 0.463166] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1262worker2 # [ 0.436316] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1263worker2 # [ 0.437402] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1264client # [ 0.464634] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1265client # [ 0.466105] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1266worker2 # [ 0.438974] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1267worker1 # [ 0.462949] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1268worker2 # [ 0.440403] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1269worker2 # [ 0.441290] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1270worker1 # [ 0.465331] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1271client # [ 0.469898] ACPI: PCI: Interrupt link LNKA configured for IRQ 101272worker2 # [ 0.442156] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1273client # [ 0.470408] ACPI: PCI: Interrupt link LNKB configured for IRQ 101274worker1 # [ 0.466029] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1275client # [ 0.471403] ACPI: PCI: Interrupt link LNKC configured for IRQ 111276client # [ 0.472400] ACPI: PCI: Interrupt link LNKD configured for IRQ 111277worker1 # [ 0.467279] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1278worker2 # [ 0.443845] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1279lb # [ 0.453944] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1280client # [ 0.473407] ACPI: PCI: Interrupt link LNKE configured for IRQ 101281worker1 # [ 0.467981] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1282worker2 # [ 0.445065] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1283lb # [ 0.456381] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1284client # [ 0.474398] ACPI: PCI: Interrupt link LNKF configured for IRQ 101285client # [ 0.475403] ACPI: PCI: Interrupt link LNKG configured for IRQ 111286worker1 # [ 0.469047] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1287worker2 # [ 0.445579] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1288lb # [ 0.456970] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1289client # [ 0.476398] ACPI: PCI: Interrupt link LNKH configured for IRQ 111290worker2 # [ 0.446987] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1291worker1 # [ 0.470015] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1292lb # [ 0.458393] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1293client # [ 0.477340] ACPI: PCI: Interrupt link GSIA configured for IRQ 161294client # [ 0.478268] ACPI: PCI: Interrupt link GSIB configured for IRQ 171295worker1 # [ 0.471686] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1296lb # [ 0.459036] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1297client # [ 0.479067] ACPI: PCI: Interrupt link GSIC configured for IRQ 181298worker1 # [ 0.472457] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1299client # [ 0.479317] ACPI: PCI: Interrupt link GSID configured for IRQ 191300lb # [ 0.460031] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1301client # [ 0.480266] ACPI: PCI: Interrupt link GSIE configured for IRQ 201302worker1 # [ 0.473089] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1303lb # [ 0.461010] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1304client # [ 0.481168] ACPI: PCI: Interrupt link GSIF configured for IRQ 211305client # [ 0.482048] ACPI: PCI: Interrupt link GSIG configured for IRQ 221306worker1 # [ 0.474325] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1307lb # [ 0.462739] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1308client # [ 0.482314] ACPI: PCI: Interrupt link GSIH configured for IRQ 231309lb # [ 0.463618] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1310client # [ 0.484347] iommu: Default domain type: Translated1311lb # [ 0.464846] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1312client # [ 0.485169] iommu: DMA domain TLB invalidation policy: lazy mode1313worker1 # [ 0.478524] ACPI: PCI: Interrupt link LNKA configured for IRQ 101314client # [ 0.486328] ACPI: bus type USB registered1315lb # [ 0.466407] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1316worker1 # [ 0.479676] ACPI: PCI: Interrupt link LNKB configured for IRQ 101317client # [ 0.487089] usbcore: registered new interface driver usbfs1318client # [ 0.487972] usbcore: registered new interface driver hub1319worker2 # [ 0.447596] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1320worker1 # [ 0.480750] ACPI: PCI: Interrupt link LNKC configured for IRQ 111321client # [ 0.488313] usbcore: registered new device driver usb1322worker1 # [ 0.481879] ACPI: PCI: Interrupt link LNKD configured for IRQ 111323worker2 # [ 0.450048] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1324client # [ 0.490118] NetLabel: Initializing1325worker1 # [ 0.482897] ACPI: PCI: Interrupt link LNKE configured for IRQ 101326lb # [ 0.469782] ACPI: PCI: Interrupt link LNKA configured for IRQ 101327client # [ 0.490764] NetLabel: domain hash size = 1281328worker2 # [ 0.450620] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1329lb # [ 0.470827] ACPI: PCI: Interrupt link LNKB configured for IRQ 101330worker1 # [ 0.483882] ACPI: PCI: Interrupt link LNKF configured for IRQ 101331client # [ 0.491302] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1332lb # [ 0.471847] ACPI: PCI: Interrupt link LNKC configured for IRQ 111333worker1 # [ 0.484875] ACPI: PCI: Interrupt link LNKG configured for IRQ 111334worker2 # [ 0.451995] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1335client # [ 0.492338] NetLabel: unlabeled traffic allowed by default1336lb # [ 0.472848] ACPI: PCI: Interrupt link LNKD configured for IRQ 111337client # [ 0.493219] PCI: Using ACPI for IRQ routing1338worker1 # [ 0.485864] ACPI: PCI: Interrupt link LNKH configured for IRQ 111339worker2 # [ 0.452658] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1340worker1 # [ 0.486795] ACPI: PCI: Interrupt link GSIA configured for IRQ 161341lb # [ 0.473843] ACPI: PCI: Interrupt link LNKE configured for IRQ 101342worker1 # [ 0.487776] ACPI: PCI: Interrupt link GSIB configured for IRQ 171343lb # [ 0.474848] ACPI: PCI: Interrupt link LNKF configured for IRQ 101344worker2 # [ 0.453676] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1345worker1 # [ 0.488765] ACPI: PCI: Interrupt link GSIC configured for IRQ 181346lb # [ 0.475846] ACPI: PCI: Interrupt link LNKG configured for IRQ 111347worker1 # [ 0.489727] ACPI: PCI: Interrupt link GSID configured for IRQ 191348worker2 # [ 0.454627] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1349lb # [ 0.476843] ACPI: PCI: Interrupt link LNKH configured for IRQ 111350worker1 # [ 0.490486] ACPI: PCI: Interrupt link GSIE configured for IRQ 201351lb # [ 0.477789] ACPI: PCI: Interrupt link GSIA configured for IRQ 161352worker1 # [ 0.490770] ACPI: PCI: Interrupt link GSIF configured for IRQ 211353worker2 # [ 0.456380] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1354lb # [ 0.478760] ACPI: PCI: Interrupt link GSIB configured for IRQ 171355worker1 # [ 0.491732] ACPI: PCI: Interrupt link GSIG configured for IRQ 221356worker2 # [ 0.457241] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1357lb # [ 0.479721] ACPI: PCI: Interrupt link GSIC configured for IRQ 181358worker1 # [ 0.492515] ACPI: PCI: Interrupt link GSIH configured for IRQ 231359lb # [ 0.480481] ACPI: PCI: Interrupt link GSID configured for IRQ 191360worker2 # [ 0.458446] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1361lb # [ 0.480760] ACPI: PCI: Interrupt link GSIE configured for IRQ 201362worker1 # [ 0.493914] iommu: Default domain type: Translated1363lb # [ 0.481718] ACPI: PCI: Interrupt link GSIF configured for IRQ 211364worker2 # [ 0.460078] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1365worker1 # [ 0.494711] iommu: DMA domain TLB invalidation policy: lazy mode1366lb # [ 0.482527] ACPI: PCI: Interrupt link GSIG configured for IRQ 221367worker1 # [ 0.495750] ACPI: bus type USB registered1368lb # [ 0.482758] ACPI: PCI: Interrupt link GSIH configured for IRQ 231369worker1 # [ 0.496501] usbcore: registered new interface driver usbfs1370worker1 # [ 0.496772] usbcore: registered new interface driver hub1371lb # [ 0.484911] iommu: Default domain type: Translated1372worker1 # [ 0.497603] usbcore: registered new device driver usb1373worker2 # [ 0.463459] ACPI: PCI: Interrupt link LNKA configured for IRQ 101374lb # [ 0.485687] iommu: DMA domain TLB invalidation policy: lazy mode1375worker2 # [ 0.464489] ACPI: PCI: Interrupt link LNKB configured for IRQ 101376worker1 # [ 0.499296] NetLabel: Initializing1377lb # [ 0.486777] ACPI: bus type USB registered1378worker2 # [ 0.465486] ACPI: PCI: Interrupt link LNKC configured for IRQ 111379lb # [ 0.487501] usbcore: registered new interface driver usbfs1380worker1 # [ 0.499778] NetLabel: domain hash size = 1281381lb # [ 0.487765] usbcore: registered new interface driver hub1382worker2 # [ 0.466480] ACPI: PCI: Interrupt link LNKD configured for IRQ 111383lb # [ 0.488649] usbcore: registered new device driver usb1384worker1 # [ 0.500775] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1385worker2 # [ 0.467480] ACPI: PCI: Interrupt link LNKE configured for IRQ 101386worker1 # [ 0.501950] NetLabel: unlabeled traffic allowed by default1387worker2 # [ 0.468476] ACPI: PCI: Interrupt link LNKF configured for IRQ 101388worker1 # [ 0.502774] PCI: Using ACPI for IRQ routing1389worker2 # [ 0.469486] ACPI: PCI: Interrupt link LNKG configured for IRQ 111390lb # [ 0.491088] NetLabel: Initializing1391worker2 # [ 0.470487] ACPI: PCI: Interrupt link LNKH configured for IRQ 111392lb # [ 0.491748] NetLabel: domain hash size = 1281393worker2 # [ 0.471414] ACPI: PCI: Interrupt link GSIA configured for IRQ 161394lb # [ 0.492517] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1395worker2 # [ 0.472378] ACPI: PCI: Interrupt link GSIB configured for IRQ 171396lb # [ 0.492791] NetLabel: unlabeled traffic allowed by default1397worker2 # [ 0.473322] ACPI: PCI: Interrupt link GSIC configured for IRQ 181398lb # [ 0.493731] PCI: Using ACPI for IRQ routing1399worker2 # [ 0.474135] ACPI: PCI: Interrupt link GSID configured for IRQ 191400worker2 # [ 0.474391] ACPI: PCI: Interrupt link GSIE configured for IRQ 201401worker2 # [ 0.475370] ACPI: PCI: Interrupt link GSIF configured for IRQ 211402worker2 # [ 0.476268] ACPI: PCI: Interrupt link GSIG configured for IRQ 221403worker2 # [ 0.477163] ACPI: PCI: Interrupt link GSIH configured for IRQ 231404worker2 # [ 0.479228] iommu: Default domain type: Translated1405worker2 # [ 0.479940] iommu: DMA domain TLB invalidation policy: lazy mode1406worker2 # [ 0.480652] ACPI: bus type USB registered1407worker2 # [ 0.481393] usbcore: registered new interface driver usbfs1408worker2 # [ 0.483128] usbcore: registered new interface driver hub1409worker2 # [ 0.483410] usbcore: registered new device driver usb1410worker2 # [ 0.485423] NetLabel: Initializing1411worker2 # [ 0.486104] NetLabel: domain hash size = 1281412worker2 # [ 0.486380] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1413worker2 # [ 0.487360] NetLabel: unlabeled traffic allowed by default1414worker2 # [ 0.488106] PCI: Using ACPI for IRQ routing1415client # [ 0.536619] pci 0000:00:01.0: vgaarb: setting as boot VGA device1416client # [ 0.537299] pci 0000:00:01.0: vgaarb: bridge control possible1417client # [ 0.537299] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1418client # [ 0.537307] vgaarb: loaded1419client # [ 0.538058] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01420client # [ 0.538303] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1421client # [ 0.544384] clocksource: Switched to clocksource kvm-clock1422client # [ 0.547944] VFS: Disk quotas dquot_6.6.01423client # [ 0.548685] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1424client # [ 0.550095] pnp: PnP ACPI init1425client # [ 0.550946] ACPI: IRQ 4 override to edge(!), high(!)1426worker1 # [ 0.546715] pci 0000:00:01.0: vgaarb: setting as boot VGA device1427client # [ 0.551949] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1428worker1 # [ 0.546754] pci 0000:00:01.0: vgaarb: bridge control possible1429client # [ 0.553468] pnp: PnP ACPI: found 6 devices1430worker1 # [ 0.546754] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1431worker1 # [ 0.546762] vgaarb: loaded1432worker1 # [ 0.547534] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01433lb # [ 0.537038] pci 0000:00:01.0: vgaarb: setting as boot VGA device1434worker1 # [ 0.547758] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1435lb # [ 0.537743] pci 0000:00:01.0: vgaarb: bridge control possible1436lb # [ 0.537743] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1437lb # [ 0.537751] vgaarb: loaded1438lb # [ 0.538494] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01439lb # [ 0.538749] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1440client # [ 0.561097] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1441worker1 # [ 0.552850] clocksource: Switched to clocksource kvm-clock1442client # [ 0.562571] clocksource: Switched to clocksource acpi_pm1443worker1 # [ 0.554524] VFS: Disk quotas dquot_6.6.01444client # [ 0.563578] NET: Registered PF_INET protocol family1445client # [ 0.564751] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1446worker1 # [ 0.557061] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1447lb # [ 0.543833] clocksource: Switched to clocksource kvm-clock1448worker1 # [ 0.558478] pnp: PnP ACPI init1449worker1 # [ 0.559288] ACPI: IRQ 4 override to edge(!), high(!)1450worker1 # [ 0.560329] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1451lb # [ 0.547279] VFS: Disk quotas dquot_6.6.01452worker1 # [ 0.561931] pnp: PnP ACPI: found 6 devices1453lb # [ 0.548011] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1454lb # [ 0.549417] pnp: PnP ACPI init1455lb # [ 0.550261] ACPI: IRQ 4 override to edge(!), high(!)1456lb # [ 0.551285] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1457lb # [ 0.552849] pnp: PnP ACPI: found 6 devices1458worker2 # [ 0.532090] pci 0000:00:01.0: vgaarb: setting as boot VGA device1459worker2 # [ 0.532375] pci 0000:00:01.0: vgaarb: bridge control possible1460worker2 # [ 0.532375] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1461worker2 # [ 0.532383] vgaarb: loaded1462worker1 # [ 0.569553] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1463worker2 # [ 0.533096] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01464worker1 # [ 0.571035] clocksource: Switched to clocksource acpi_pm1465worker2 # [ 0.533380] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1466worker1 # [ 0.571995] NET: Registered PF_INET protocol family1467client # [ 0.581089] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1468worker1 # [ 0.573119] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1469client # [ 0.582586] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1470lb # [ 0.560502] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1471client # [ 0.583970] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1472lb # [ 0.561959] clocksource: Switched to clocksource acpi_pm1473client # [ 0.585327] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1474lb # [ 0.562945] NET: Registered PF_INET protocol family1475worker2 # [ 0.538465] clocksource: Switched to clocksource kvm-clock1476client # [ 0.586607] TCP: Hash tables configured (established 16384 bind 16384)1477lb # [ 0.564171] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1478client # [ 0.587929] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1479client # [ 0.589244] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1480worker2 # [ 0.541869] VFS: Disk quotas dquot_6.6.01481client # [ 0.590388] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1482worker2 # [ 0.542600] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1483client # [ 0.591633] NET: Registered PF_UNIX/PF_LOCAL protocol family1484worker2 # [ 0.544002] pnp: PnP ACPI init1485client # [ 0.592649] NET: Registered PF_XDP protocol family1486worker2 # [ 0.544778] ACPI: IRQ 4 override to edge(!), high(!)1487client # [ 0.593533] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1488worker2 # [ 0.545764] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1489client # [ 0.594573] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1490worker2 # [ 0.547336] pnp: PnP ACPI: found 6 devices1491client # [ 0.595607] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1492client # [ 0.596784] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1493client # [ 0.597949] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1494worker1 # [ 0.589465] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1495client # [ 0.599125] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1496worker1 # [ 0.591022] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1497client # [ 0.600861] ACPI: \_SB_.GSIA: Enabled at IRQ 161498worker1 # [ 0.592354] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1499worker1 # [ 0.593771] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1500client # [ 0.603685] ACPI: \_SB_.GSIB: Enabled at IRQ 171501worker2 # [ 0.554997] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1502worker1 # [ 0.595074] TCP: Hash tables configured (established 16384 bind 16384)1503lb # [ 0.580791] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1504worker2 # [ 0.556482] clocksource: Switched to clocksource acpi_pm1505client # [ 0.605603] ACPI: \_SB_.GSIC: Enabled at IRQ 181506worker1 # [ 0.596175] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1507lb # [ 0.582302] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1508worker2 # [ 0.557454] NET: Registered PF_INET protocol family1509worker1 # [ 0.597402] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1510lb # [ 0.583557] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1511client # [ 0.607511] ACPI: \_SB_.GSID: Enabled at IRQ 191512worker2 # [ 0.558592] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1513worker1 # [ 0.598504] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1514lb # [ 0.584900] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1515worker1 # [ 0.599702] NET: Registered PF_UNIX/PF_LOCAL protocol family1516client # [ 0.609186] PCI: CLS 0 bytes, default 641517lb # [ 0.586179] TCP: Hash tables configured (established 16384 bind 16384)1518worker1 # [ 0.600690] NET: Registered PF_XDP protocol family1519lb # [ 0.587301] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1520client # [ 0.610057] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1521worker1 # [ 0.601536] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1522worker1 # [ 0.602530] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1523client # [ 0.611734] Trying to unpack rootfs image as initramfs...1524lb # [ 0.588533] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1525worker1 # [ 0.603514] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1526lb # [ 0.589637] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1527worker1 # [ 0.604651] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1528lb # [ 0.590842] NET: Registered PF_UNIX/PF_LOCAL protocol family1529lb # [ 0.591767] NET: Registered PF_XDP protocol family1530worker1 # [ 0.605801] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1531lb # [ 0.592619] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1532worker1 # [ 0.606929] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1533lb # [ 0.593612] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1534worker1 # [ 0.608657] ACPI: \_SB_.GSIA: Enabled at IRQ 161535lb # [ 0.594630] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1536lb # [ 0.595732] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1537worker1 # [ 0.611017] ACPI: \_SB_.GSIB: Enabled at IRQ 171538lb # [ 0.596879] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1539lb # [ 0.598065] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1540worker1 # [ 0.613251] ACPI: \_SB_.GSIC: Enabled at IRQ 181541lb # [ 0.599812] ACPI: \_SB_.GSIA: Enabled at IRQ 161542worker2 # [ 0.575110] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1543worker1 # [ 0.615291] ACPI: \_SB_.GSID: Enabled at IRQ 191544lb # [ 0.601913] ACPI: \_SB_.GSIB: Enabled at IRQ 171545worker2 # [ 0.576676] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1546worker1 # [ 0.616987] PCI: CLS 0 bytes, default 641547worker2 # [ 0.578013] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1548lb # [ 0.603908] ACPI: \_SB_.GSIC: Enabled at IRQ 181549worker1 # [ 0.617949] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1550worker2 # [ 0.579338] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1551worker1 # [ 0.619654] Trying to unpack rootfs image as initramfs...1552lb # [ 0.605878] ACPI: \_SB_.GSID: Enabled at IRQ 191553worker2 # [ 0.580598] TCP: Hash tables configured (established 16384 bind 16384)1554worker2 # [ 0.581683] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1555lb # [ 0.607572] PCI: CLS 0 bytes, default 641556worker2 # [ 0.583034] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1557lb # [ 0.608516] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1558worker2 # [ 0.584169] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1559lb # [ 0.610261] Trying to unpack rootfs image as initramfs...1560worker2 # [ 0.585425] NET: Registered PF_UNIX/PF_LOCAL protocol family1561worker2 # [ 0.586450] NET: Registered PF_XDP protocol family1562worker2 # [ 0.587328] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1563worker2 # [ 0.588467] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1564worker2 # [ 0.589515] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1565worker2 # [ 0.590649] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1566worker2 # [ 0.591808] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1567worker2 # [ 0.592986] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1568worker2 # [ 0.594811] ACPI: \_SB_.GSIA: Enabled at IRQ 161569worker2 # [ 0.596999] ACPI: \_SB_.GSIB: Enabled at IRQ 171570worker2 # [ 0.599001] ACPI: \_SB_.GSIC: Enabled at IRQ 181571worker2 # [ 0.601124] ACPI: \_SB_.GSID: Enabled at IRQ 191572worker2 # [ 0.602832] PCI: CLS 0 bytes, default 641573worker2 # [ 0.603781] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1574worker2 # [ 0.605593] Trying to unpack rootfs image as initramfs...1575client # [ 0.656340] Initialise system trusted keyrings1576client # [ 0.659958] workingset: timestamp_bits=40 max_order=19 bucket_order=01577worker1 # [ 0.665955] Initialise system trusted keyrings1578worker1 # [ 0.666912] workingset: timestamp_bits=40 max_order=19 bucket_order=01579lb # [ 0.654009] Initialise system trusted keyrings1580lb # [ 0.654986] workingset: timestamp_bits=40 max_order=19 bucket_order=01581client # [ 0.681276] Key type asymmetric registered1582client # [ 0.683954] Asymmetric key parser 'x509' registered1583client # [ 0.684933] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1584client # [ 0.688043] io scheduler mq-deadline registered1585client # [ 0.688850] io scheduler kyber registered1586client # [ 0.691530] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1587client # [ 0.692847] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1588client # [ 0.698724] Linux agpgart interface v0.1031589client # [ 0.699573] ACPI: bus type drm_connector registered1590worker1 # [ 0.690671] Key type asymmetric registered1591worker1 # [ 0.691385] Asymmetric key parser 'x509' registered1592worker2 # [ 0.652426] Initialise system trusted keyrings1593lb # [ 0.678167] Key type asymmetric registered1594worker1 # [ 0.692277] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1595lb # [ 0.678881] Asymmetric key parser 'x509' registered1596worker2 # [ 0.653399] workingset: timestamp_bits=40 max_order=19 bucket_order=01597client # [ 0.702366] usbcore: registered new interface driver usbserial_generic1598lb # [ 0.679693] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1599client # [ 0.703436] usbserial: USB Serial support registered for generic1600worker1 # [ 0.696709] io scheduler mq-deadline registered1601worker1 # [ 0.697505] io scheduler kyber registered1602lb # [ 0.685211] io scheduler mq-deadline registered1603client # [ 0.707915] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1604lb # [ 0.685975] io scheduler kyber registered1605client # [ 0.707915] Please enable it if your BIOS has the CPPC option.1606worker1 # [ 0.701126] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1607client # [ 0.710121] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1608lb # [ 0.688612] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1609worker1 # [ 0.702388] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1610client # [ 0.713050] drop_monitor: Initializing network drop monitor service1611lb # [ 0.689855] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1612client # [ 0.714183] NET: Registered PF_INET6 protocol family1613worker1 # [ 0.706737] Linux agpgart interface v0.1031614worker1 # [ 0.707493] ACPI: bus type drm_connector registered1615client # [ 0.717383] Segment Routing with IPv61616lb # [ 0.695819] Linux agpgart interface v0.1031617lb # [ 0.696600] ACPI: bus type drm_connector registered1618client # [ 0.720231] In-situ OAM (IOAM) with IPv61619worker1 # [ 0.710825] usbcore: registered new interface driver usbserial_generic1620worker1 # [ 0.711907] usbserial: USB Serial support registered for generic1621client # [ 0.721263] IPI shorthand broadcast: enabled1622lb # [ 0.699459] usbcore: registered new interface driver usbserial_generic1623worker2 # [ 0.674842] Key type asymmetric registered1624lb # [ 0.700570] usbserial: USB Serial support registered for generic1625worker1 # [ 0.714592] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1626worker1 # [ 0.714592] Please enable it if your BIOS has the CPPC option.1627worker2 # [ 0.677925] Asymmetric key parser 'x509' registered1628worker1 # [ 0.716936] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1629lb # [ 0.703076] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1630lb # [ 0.703076] Please enable it if your BIOS has the CPPC option.1631worker2 # [ 0.678771] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1632worker2 # [ 0.680144] io scheduler mq-deadline registered1633lb # [ 0.705283] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1634client # [ 0.729439] sched_clock: Marking stable (595014551, 133887170)->(804896355, -75994634)1635worker1 # [ 0.721724] drop_monitor: Initializing network drop monitor service1636worker2 # [ 0.682932] io scheduler kyber registered1637worker1 # [ 0.722967] NET: Registered PF_INET6 protocol family1638lb # [ 0.709192] drop_monitor: Initializing network drop monitor service1639client # [ 0.733023] registered taskstats version 11640lb # [ 0.710390] NET: Registered PF_INET6 protocol family1641client # [ 0.734032] Loading compiled-in X.509 certificates1642worker2 # [ 0.685950] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1643worker1 # [ 0.726034] Segment Routing with IPv61644worker2 # [ 0.687247] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1645worker1 # [ 0.728591] In-situ OAM (IOAM) with IPv61646lb # [ 0.715161] Segment Routing with IPv61647worker1 # [ 0.729719] IPI shorthand broadcast: enabled1648lb # [ 0.715869] In-situ OAM (IOAM) with IPv61649worker2 # [ 0.692773] Linux agpgart interface v0.1031650worker2 # [ 0.693537] ACPI: bus type drm_connector registered1651lb # [ 0.719203] IPI shorthand broadcast: enabled1652worker2 # [ 0.696387] usbcore: registered new interface driver usbserial_generic1653worker2 # [ 0.697446] usbserial: USB Serial support registered for generic1654worker1 # [ 0.737931] sched_clock: Marking stable (595015277, 142557147)->(808128089, -70555665)1655lb # [ 0.725632] sched_clock: Marking stable (590015124, 135033985)->(801902010, -76852901)1656worker2 # [ 0.700922] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1657worker2 # [ 0.700922] Please enable it if your BIOS has the CPPC option.1658worker1 # [ 0.742660] registered taskstats version 11659worker2 # [ 0.703119] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1660worker1 # [ 0.743633] Loading compiled-in X.509 certificates1661lb # [ 0.730135] registered taskstats version 11662client # [ 0.753917] Demotion targets for Node 0: null1663lb # [ 0.731145] Loading compiled-in X.509 certificates1664worker2 # [ 0.706050] drop_monitor: Initializing network drop monitor service1665client # [ 0.754824] Key type .fscrypt registered1666client # [ 0.755569] Key type fscrypt-provisioning registered1667worker2 # [ 0.707204] NET: Registered PF_INET6 protocol family1668client # [ 0.756561] ima: No TPM chip found, activating TPM-bypass!1669worker2 # [ 0.710367] Segment Routing with IPv61670client # [ 0.760919] ima: Allocated hash algorithm: sha11671worker2 # [ 0.712925] In-situ OAM (IOAM) with IPv61672client # [ 0.761829] ima: No architecture policies found1673worker2 # [ 0.713947] IPI shorthand broadcast: enabled1674client # [ 0.763923] PM: Magic number: 10:928:5841675client # [ 0.765562] RAS: Correctable Errors collector initialized.1676worker1 # [ 0.761586] Demotion targets for Node 0: null1677worker2 # [ 0.722269] sched_clock: Marking stable (587014563, 134891435)->(798626907, -76720909)1678worker1 # [ 0.762464] Key type .fscrypt registered1679worker1 # [ 0.763193] Key type fscrypt-provisioning registered1680lb # [ 0.750066] Demotion targets for Node 0: null1681worker1 # [ 0.764149] ima: No TPM chip found, activating TPM-bypass!1682lb # [ 0.750960] Key type .fscrypt registered1683client # [ 0.774586] clk: Disabling unused clocks1684lb # [ 0.751666] Key type fscrypt-provisioning registered1685worker2 # [ 0.727114] registered taskstats version 11686lb # [ 0.752584] ima: No TPM chip found, activating TPM-bypass!1687worker2 # [ 0.728082] Loading compiled-in X.509 certificates1688worker1 # [ 0.768592] ima: Allocated hash algorithm: sha11689worker1 # [ 0.769439] ima: No architecture policies found1690client # [ 0.778928] PM: genpd: Disabling unused power domains1691lb # [ 0.757064] ima: Allocated hash algorithm: sha11692worker1 # [ 0.771586] PM: Magic number: 10:928:5841693lb # [ 0.757880] ima: No architecture policies found1694lb # [ 0.758829] PM: Magic number: 10:928:5841695worker1 # [ 0.773243] RAS: Correctable Errors collector initialized.1696lb # [ 0.762487] RAS: Correctable Errors collector initialized.1697worker1 # [ 0.782301] clk: Disabling unused clocks1698lb # [ 0.771117] clk: Disabling unused clocks1699worker2 # [ 0.745924] Demotion targets for Node 0: null1700lb # [ 0.771859] PM: genpd: Disabling unused power domains1701worker1 # [ 0.786589] PM: genpd: Disabling unused power domains1702worker2 # [ 0.747950] Key type .fscrypt registered1703worker2 # [ 0.748661] Key type fscrypt-provisioning registered1704worker2 # [ 0.749623] ima: No TPM chip found, activating TPM-bypass!1705worker2 # [ 0.753920] ima: Allocated hash algorithm: sha11706worker2 # [ 0.754744] ima: No architecture policies found1707worker2 # [ 0.756926] PM: Magic number: 10:928:5841708worker2 # [ 0.758580] RAS: Correctable Errors collector initialized.1709worker2 # [ 0.767563] clk: Disabling unused clocks1710worker2 # [ 0.770932] PM: genpd: Disabling unused power domains1711client # [ 0.941410] Freeing initrd memory: 29096K1712client # [ 0.944564] Freeing unused decrypted memory: 2028K1713client # [ 0.947216] Freeing unused kernel image (initmem) memory: 3644K1714client # [ 0.948262] Write protecting the kernel read-only data: 32768k1715client # [ 0.950127] Freeing unused kernel image (text/rodata gap) memory: 1208K1716client # [ 0.951587] Freeing unused kernel image (rodata/data gap) memory: 752K1717worker1 # [ 0.944210] Freeing initrd memory: 29092K1718worker1 # [ 0.947369] Freeing unused decrypted memory: 2028K1719worker1 # [ 0.950008] Freeing unused kernel image (initmem) memory: 3644K1720worker1 # [ 0.951052] Write protecting the kernel read-only data: 32768k1721worker1 # [ 0.952870] Freeing unused kernel image (text/rodata gap) memory: 1208K1722worker1 # [ 0.954346] Freeing unused kernel image (rodata/data gap) memory: 752K1723lb # [ 0.940942] Freeing initrd memory: 29092K1724lb # [ 0.944081] Freeing unused decrypted memory: 2028K1725lb # [ 0.946628] Freeing unused kernel image (initmem) memory: 3644K1726lb # [ 0.947679] Write protecting the kernel read-only data: 32768k1727lb # [ 0.949496] Freeing unused kernel image (text/rodata gap) memory: 1208K1728lb # [ 0.950971] Freeing unused kernel image (rodata/data gap) memory: 752K1729worker2 # [ 0.939936] Freeing initrd memory: 29092K1730worker2 # [ 0.943093] Freeing unused decrypted memory: 2028K1731worker2 # [ 0.945720] Freeing unused kernel image (initmem) memory: 3644K1732worker2 # [ 0.946844] Write protecting the kernel read-only data: 32768k1733worker2 # [ 0.948772] Freeing unused kernel image (text/rodata gap) memory: 1208K1734worker2 # [ 0.950350] Freeing unused kernel image (rodata/data gap) memory: 752K1735client # [ 1.002882] x86/mm: Checked W+X mappings: passed, no W+X pages found.1736client # [ 1.004024] Run /init as init process1737client # [ 1.014183] systemd[1]: Inserted module 'autofs4'1738worker1 # [ 1.005367] x86/mm: Checked W+X mappings: passed, no W+X pages found.1739worker1 # [ 1.006401] Run /init as init process1740lb # [ 1.002165] x86/mm: Checked W+X mappings: passed, no W+X pages found.1741worker1 # [ 1.016662] systemd[1]: Inserted module 'autofs4'1742lb # [ 1.003227] Run /init as init process1743client # [ 1.030518] fuse: init (API version 7.45)1744client # [ 1.036371] ACPI: \_SB_.GSIG: Enabled at IRQ 221745lb # [ 1.013657] systemd[1]: Inserted module 'autofs4'1746client # [ 1.039053] ACPI: \_SB_.GSIH: Enabled at IRQ 231747worker1 # [ 1.032900] fuse: init (API version 7.45)1748client # [ 1.043120] ACPI: \_SB_.GSIE: Enabled at IRQ 201749client # [ 1.045774] ACPI: \_SB_.GSIF: Enabled at IRQ 211750worker1 # [ 1.038670] ACPI: \_SB_.GSIG: Enabled at IRQ 221751worker1 # [ 1.040963] ACPI: \_SB_.GSIH: Enabled at IRQ 231752worker2 # [ 1.001639] x86/mm: Checked W+X mappings: passed, no W+X pages found.1753worker2 # [ 1.002689] Run /init as init process1754lb # [ 1.029914] fuse: init (API version 7.45)1755worker1 # [ 1.044515] ACPI: \_SB_.GSIE: Enabled at IRQ 201756worker1 # [ 1.047339] ACPI: \_SB_.GSIF: Enabled at IRQ 211757lb # [ 1.035722] ACPI: \_SB_.GSIG: Enabled at IRQ 221758lb # [ 1.038155] ACPI: \_SB_.GSIH: Enabled at IRQ 231759worker2 # [ 1.012843] systemd[1]: Inserted module 'autofs4'1760lb # [ 1.040908] ACPI: \_SB_.GSIE: Enabled at IRQ 201761lb # [ 1.043087] ACPI: \_SB_.GSIF: Enabled at IRQ 211762worker2 # [ 1.029136] fuse: init (API version 7.45)1763client # [ 1.078197] systemd[1]: Successfully made /usr/ read-only.1764worker2 # [ 1.034958] ACPI: \_SB_.GSIG: Enabled at IRQ 221765worker1 # [ 1.075368] systemd[1]: Successfully made /usr/ read-only.1766worker2 # [ 1.037345] ACPI: \_SB_.GSIH: Enabled at IRQ 231767worker2 # [ 1.040306] ACPI: \_SB_.GSIE: Enabled at IRQ 201768worker2 # [ 1.042579] ACPI: \_SB_.GSIF: Enabled at IRQ 211769lb # [ 1.069740] systemd[1]: Successfully made /usr/ read-only.1770worker2 # [ 1.068691] systemd[1]: Successfully made /usr/ read-only.1771client # [ 1.415064] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1772client # [ 1.427010] systemd[1]: Detected virtualization kvm.1773client # [ 1.429105] systemd[1]: Detected architecture x86-64.1774worker1 # [ 1.411864] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1775client # [ 1.431189] systemd[1]: Running in initrd.1776lb # [ 1.405698] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1777worker1 # [ 1.423781] systemd[1]: Detected virtualization kvm.1778lb # [ 1.410661] systemd[1]: Detected virtualization kvm.1779client # [ 1.433618] systemd[1]: Initializing machine ID from random generator.1780worker1 # [ 1.425867] systemd[1]: Detected architecture x86-64.1781lb # [ 1.412798] systemd[1]: Detected architecture x86-64.1782client # [ 1.436478] systemd[1]: Hostname set to <client>.1783worker1 # [ 1.427940] systemd[1]: Running in initrd.1784lb # [ 1.414980] systemd[1]: Running in initrd.1785worker1 # [ 1.430357] systemd[1]: Initializing machine ID from random generator.1786lb # [ 1.417713] systemd[1]: Initializing machine ID from random generator.1787worker1 # [ 1.433242] systemd[1]: Hostname set to <worker1>.1788lb # [ 1.420712] systemd[1]: Hostname set to <lb>.1789worker2 # [ 1.405231] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1790worker2 # [ 1.419011] systemd[1]: Detected virtualization kvm.1791worker2 # [ 1.421521] systemd[1]: Detected architecture x86-64.1792worker2 # [ 1.424022] systemd[1]: Running in initrd.1793worker2 # [ 1.426736] systemd[1]: Initializing machine ID from random generator.1794worker2 # [ 1.430129] systemd[1]: Hostname set to <worker2>.1795client # [ 1.645150] systemd[1]: bpf-restrict-fs: LSM BPF program attached1796lb # [ 1.630725] systemd[1]: bpf-restrict-fs: LSM BPF program attached1797worker1 # [ 1.648075] systemd[1]: bpf-restrict-fs: LSM BPF program attached1798client # [ 1.683301] systemd[1]: Queued start job for default target Initrd Default Target.1799client # [ 1.687177] systemd[1]: Created slice Slice /system/modprobe.1800client # [ 1.688358] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1801client # [ 1.689789] systemd[1]: Expecting device /dev/disk/by-label/nixos...1802client # [ 1.690966] systemd[1]: Reached target Path Units.1803client # [ 1.691853] systemd[1]: Reached target Slice Units.1804client # [ 1.692782] systemd[1]: Reached target Swaps.1805client # [ 1.693633] systemd[1]: Reached target Timer Units.1806client # [ 1.694636] systemd[1]: Listening on D-Bus System Message Bus Socket.1807client # [ 1.695970] systemd[1]: Listening on Journal Socket (/dev/log).1808worker2 # [ 1.648104] systemd[1]: bpf-restrict-fs: LSM BPF program attached1809client # [ 1.697148] systemd[1]: Listening on Journal Sockets.1810client # [ 1.698121] systemd[1]: Listening on udev Control Socket.1811client # [ 1.699138] systemd[1]: Listening on udev Kernel Socket.1812client # [ 1.700101] systemd[1]: Reached target Socket Units.1813client # [ 1.701814] systemd[1]: Starting Create List of Static Device Nodes...1814lb # [ 1.679723] systemd[1]: Queued start job for default target Initrd Default Target.1815client # [ 1.705705] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1816lb # [ 1.684449] systemd[1]: Created slice Slice /system/modprobe.1817lb # [ 1.685726] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1818lb # [ 1.687238] systemd[1]: Expecting device /dev/disk/by-label/nixos...1819worker1 # [ 1.701128] systemd[1]: Queued start job for default target Initrd Default Target.1820lb # [ 1.688380] systemd[1]: Reached target Path Units.1821lb # [ 1.689301] systemd[1]: Reached target Slice Units.1822client # [ 1.713082] systemd[1]: Starting Load Kernel Module configfs...1823lb # [ 1.690222] systemd[1]: Reached target Swaps.1824worker1 # [ 1.704877] systemd[1]: Created slice Slice /system/modprobe.1825lb # [ 1.691052] systemd[1]: Reached target Timer Units.1826lb # [ 1.692157] systemd[1]: Listening on D-Bus System Message Bus Socket.1827worker1 # [ 1.706114] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1828lb # [ 1.693414] systemd[1]: Listening on Journal Socket (/dev/log).1829worker1 # [ 1.707472] systemd[1]: Expecting device /dev/disk/by-label/nixos...1830worker1 # [ 1.708605] systemd[1]: Reached target Path Units.1831lb # [ 1.694620] systemd[1]: Listening on Journal Sockets.1832worker1 # [ 1.709488] systemd[1]: Reached target Slice Units.1833lb # [ 1.695622] systemd[1]: Listening on udev Control Socket.1834worker1 # [ 1.710384] systemd[1]: Reached target Swaps.1835lb # [ 1.696626] systemd[1]: Listening on udev Kernel Socket.1836worker1 # [ 1.711197] systemd[1]: Reached target Timer Units.1837lb # [ 1.697572] systemd[1]: Reached target Socket Units.1838worker1 # [ 1.712183] systemd[1]: Listening on D-Bus System Message Bus Socket.1839worker1 # [ 1.713417] systemd[1]: Listening on Journal Socket (/dev/log).1840lb # [ 1.699482] systemd[1]: Starting Create List of Static Device Nodes...1841worker1 # [ 1.714617] systemd[1]: Listening on Journal Sockets.1842worker1 # [ 1.715640] systemd[1]: Listening on udev Control Socket.1843worker1 # [ 1.716737] systemd[1]: Listening on udev Kernel Socket.1844client # [ 1.726127] systemd[1]: Starting Journal Service...1845worker1 # [ 1.717723] systemd[1]: Reached target Socket Units.1846lb # [ 1.704117] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1847worker1 # [ 1.719488] systemd[1]: Starting Create List of Static Device Nodes...1848worker1 # [ 1.723409] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1849lb # [ 1.710786] systemd[1]: Starting Load Kernel Module configfs...1850worker2 # [ 1.689724] systemd[1]: Queued start job for default target Initrd Default Target.1851worker1 # [ 1.730748] systemd[1]: Starting Load Kernel Module configfs...1852client # [ 1.740025] systemd[1]: Starting Load Kernel Modules...1853worker2 # [ 1.694325] systemd[1]: Created slice Slice /system/modprobe.1854worker2 # [ 1.695481] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1855worker2 # [ 1.696836] systemd[1]: Expecting device /dev/disk/by-label/nixos...1856worker2 # [ 1.697916] systemd[1]: Reached target Path Units.1857worker2 # [ 1.698747] systemd[1]: Reached target Slice Units.1858worker2 # [ 1.699640] systemd[1]: Reached target Swaps.1859worker2 # [ 1.700397] systemd[1]: Reached target Timer Units.1860lb # [ 1.726126] systemd[1]: Starting Journal Service...1861worker2 # [ 1.701388] systemd[1]: Listening on D-Bus System Message Bus Socket.1862worker2 # [ 1.702572] systemd[1]: Listening on Journal Socket (/dev/log).1863worker2 # [ 1.703697] systemd[1]: Listening on Journal Sockets.1864worker1 # [ 1.743646] systemd[1]: Starting Journal Service...1865worker2 # [ 1.704658] systemd[1]: Listening on udev Control Socket.1866worker2 # [ 1.705631] systemd[1]: Listening on udev Kernel Socket.1867worker2 # [ 1.706534] systemd[1]: Reached target Socket Units.1868worker2 # [ 1.708401] systemd[1]: Starting Create List of Static Device Nodes...1869client # [ 1.744976] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1870worker2 # [ 1.712974] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1871lb # [ 1.739122] systemd[1]: Starting Load Kernel Modules...1872client # [ 1.762737] netfs: FS-Cache loaded1873worker1 # [ 1.755213] systemd[1]: Starting Load Kernel Modules...1874client # [ 1.767483] 9pnet: Installing 9P2000 support1875worker2 # [ 1.719505] systemd[1]: Starting Load Kernel Module configfs...1876client # [ 1.776177] systemd[1]: Starting Coldplug All udev Devices...1877lb # [ 1.744147] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1878worker2 # [ 1.735565] systemd[1]: Starting Journal Service...1879worker1 # [ 1.762654] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1880lb # [ 1.762950] netfs: FS-Cache loaded1881worker1 # [ 1.781250] netfs: FS-Cache loaded1882client # [ 1.790023] systemd[1]: Finished Create List of Static Device Nodes.1883lb # [ 1.767871] 9pnet: Installing 9P2000 support1884worker1 # [ 1.785935] 9pnet: Installing 9P2000 support1885client # [ 1.796467] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1886worker2 # [ 1.748987] systemd[1]: Starting Load Kernel Modules...1887client # [ 1.803337] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1888lb # [ 1.782177] systemd[1]: Starting Coldplug All udev Devices...1889worker1 # [ 1.798915] systemd[1]: Starting Coldplug All udev Devices...1890client # [ 1.810300] systemd[1]: modprobe@configfs.service: Deactivated successfully.1891worker2 # [ 1.752967] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1892client # [ 1.813464] systemd-journald[67]: Collecting audit messages is disabled.1893client # [ 1.815942] systemd[1]: Finished Load Kernel Module configfs.1894worker2 # [ 1.771388] netfs: FS-Cache loaded1895client # [ 1.822326] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1896lb # [ 1.800172] systemd[1]: Finished Create List of Static Device Nodes.1897worker2 # [ 1.776247] 9pnet: Installing 9P2000 support1898client # [ 1.825396] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1899worker1 # [ 1.820663] systemd[1]: Finished Create List of Static Device Nodes.1900lb # [ 1.808975] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1901client # [ 1.832965] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1902worker2 # [ 1.786984] systemd[1]: Starting Coldplug All udev Devices...1903client # [ 1.835930] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1904worker1 # [ 1.827944] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1905lb # [ 1.817151] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1906worker1 # [ 1.836690] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1907lb # [ 1.822664] systemd-journald[67]: Collecting audit messages is disabled.1908lb # [ 1.825669] systemd[1]: modprobe@configfs.service: Deactivated successfully.1909worker2 # [ 1.802076] systemd[1]: Finished Create List of Static Device Nodes.1910worker1 # [ 1.843466] systemd[1]: modprobe@configfs.service: Deactivated successfully.1911lb # [ 1.832423] systemd[1]: Finished Load Kernel Module configfs.1912worker2 # [ 1.807599] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1913worker1 # [ 1.848959] systemd-journald[67]: Collecting audit messages is disabled.1914worker1 # [ 1.850513] systemd[1]: Finished Load Kernel Module configfs.1915lb # [ 1.836198] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1916lb # [ 1.838879] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1917client # [ 1.864399] systemd[1]: Finished Load Kernel Modules.1918worker2 # [ 1.816320] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1919worker1 # [ 1.858087] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1920worker2 # [ 1.821341] systemd[1]: modprobe@configfs.service: Deactivated successfully.1921worker2 # [ 1.822689] systemd-journald[67]: Collecting audit messages is disabled.1922client # [ 1.871716] systemd[1]: Starting Apply Kernel Variables...1923lb # [ 1.848099] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1924worker1 # [ 1.862540] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1925lb # [ 1.851284] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1926worker2 # [ 1.827300] systemd[1]: Finished Load Kernel Module configfs.1927worker1 # [ 1.870856] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1928client # [ 1.881034] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1929worker2 # [ 1.834381] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1930worker1 # [ 1.874746] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1931worker2 # [ 1.839264] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1932client # [ 1.890974] systemd[1]: Starting Create Static Device Nodes in /dev...1933worker2 # [ 1.844811] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1934worker2 # [ 1.848934] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1935lb # [ 1.879642] systemd[1]: Finished Load Kernel Modules.1936client # [ 1.908058] systemd[1]: Finished Apply Kernel Variables.1937lb # [ 1.887270] systemd[1]: Starting Apply Kernel Variables...1938worker1 # [ 1.906138] systemd[1]: Finished Load Kernel Modules.1939worker1 # [ 1.913765] systemd[1]: Starting Apply Kernel Variables...1940client # [ 1.923774] systemd[1]: Started Journal Service.1941client # [ 1.791239] systemd-modules-load[68]: Inserted module 'dm_mod'1942client # [ 1.793432] systemd-modules-load[68]: Inserted module 'virtio_balloon'1943lb # [ 1.904131] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1944worker2 # [ 1.880008] systemd[1]: Finished Load Kernel Modules.1945client # [ 1.795284] systemd-modules-load[68]: Inserted module 'virtio_gpu'1946client # [ 1.797306] systemd[1]: Finished Create Static Device Nodes in /dev.1947worker1 # [ 1.923727] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1948lb # [ 1.913270] systemd[1]: Starting Create Static Device Nodes in /dev...1949client # [ 1.803117] systemd[1]: Reached target Preparation for Local File Systems.1950worker2 # [ 1.889114] systemd[1]: Starting Apply Kernel Variables...1951client # [ 1.804109] systemd[1]: Reached target Local File Systems.1952client # [ 1.806966] systemd[1]: Starting Create System Files and Directories...1953worker1 # [ 1.932778] systemd[1]: Starting Create Static Device Nodes in /dev...1954client # [ 1.812129] systemd[1]: Starting Rule-based Manager for Device Events and Files...1955worker2 # [ 1.898054] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1956lb # [ 1.926168] systemd[1]: Finished Apply Kernel Variables.1957worker2 # [ 1.909173] systemd[1]: Starting Create Static Device Nodes in /dev...1958worker1 # [ 1.949179] systemd[1]: Finished Apply Kernel Variables.1959lb # [ 1.942919] systemd[1]: Started Journal Service.1960lb # [ 1.810217] systemd-modules-load[68]: Inserted module 'dm_mod'1961lb # [ 1.813559] systemd-modules-load[68]: Inserted module 'virtio_balloon'1962client # [ 1.840564] systemd[1]: Finished Create System Files and Directories.1963lb # [ 1.816322] systemd-modules-load[68]: Inserted module 'virtio_gpu'1964worker1 # [ 1.965775] systemd[1]: Started Journal Service.1965lb # [ 1.817409] systemd[1]: Finished Create Static Device Nodes in /dev.1966worker1 # [ 1.824256] systemd-modules-load[68]: Inserted module 'dm_mod'1967lb # [ 1.819352] systemd[1]: Reached target Preparation for Local File Systems.1968worker1 # [ 1.826312] systemd-modules-load[68]: Inserted module 'virtio_balloon'1969worker2 # [ 1.929937] systemd[1]: Finished Apply Kernel Variables.1970lb # [ 1.820482] systemd[1]: Reached target Local File Systems.1971worker1 # [ 1.827959] systemd-modules-load[68]: Inserted module 'virtio_gpu'1972worker1 # [ 1.829409] systemd[1]: Finished Create Static Device Nodes in /dev.1973lb # [ 1.824103] systemd[1]: Starting Create System Files and Directories...1974worker1 # [ 1.831351] systemd[1]: Reached target Preparation for Local File Systems.1975lb # [ 1.829081] systemd[1]: Starting Rule-based Manager for Device Events and Files...1976worker2 # [ 1.939751] systemd[1]: Started Journal Service.1977worker1 # [ 1.837071] systemd[1]: Reached target Local File Systems.1978worker1 # [ 1.839881] systemd[1]: Starting Create System Files and Directories...1979worker2 # [ 1.808373] systemd-modules-load[68]: Inserted module 'dm_mod'1980worker1 # [ 1.842929] systemd[1]: Starting Rule-based Manager for Device Events and Files...1981worker2 # [ 1.811607] systemd-modules-load[68]: Inserted module 'virtio_balloon'1982client # [ 1.862323] systemd-udevd[80]: Using default interface naming scheme 'v261'.1983worker2 # [ 1.814233] systemd-modules-load[68]: Inserted module 'virtio_gpu'1984worker2 # [ 1.815296] systemd[1]: Finished Create Static Device Nodes in /dev.1985worker2 # [ 1.817351] systemd[1]: Reached target Preparation for Local File Systems.1986worker2 # [ 1.818469] systemd[1]: Reached target Local File Systems.1987worker2 # [ 1.820278] systemd[1]: Starting Create System Files and Directories...1988worker2 # [ 1.827127] systemd[1]: Starting Rule-based Manager for Device Events and Files...1989client # [ 1.885701] systemd[1]: Started Rule-based Manager for Device Events and Files.1990worker1 # [ 1.868393] systemd[1]: Finished Create System Files and Directories.1991lb # [ 1.863202] systemd[1]: Finished Create System Files and Directories.1992worker2 # [ 1.850621] systemd[1]: Finished Create System Files and Directories.1993lb # [ 1.886649] systemd-udevd[77]: Using default interface naming scheme 'v261'.1994worker1 # [ 1.896360] systemd-udevd[78]: Using default interface naming scheme 'v261'.1995worker2 # [ 1.883645] systemd-udevd[77]: Using default interface naming scheme 'v261'.1996client # [ 1.934607] systemd[1]: Finished Coldplug All udev Devices.1997lb # [ 1.910922] systemd[1]: Started Rule-based Manager for Device Events and Files.1998client # [ 1.935510] systemd[1]: Reached target System Initialization.1999client # [ 1.936279] systemd[1]: Reached target Basic System.2000worker1 # [ 1.919780] systemd[1]: Started Rule-based Manager for Device Events and Files.
loading 2,118 hidden lines…
4119worker2 # [ 17.550522] systemd[1]: Finished scheduler-token.service.4120worker2 # [ 17.551871] systemd[1]: scheduler-token.service: Consumed 27ms CPU time over 107ms wall clock time, 3.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4121worker2 # [ 17.557826] systemd[1]: Starting Nix worker-protocol over gRPC...4122worker1 # [ 17.629401] nix-grpc-daemon[780]: ts=2026-09-20T13:33:40Z level=info event=scheduler_connected addr=lb:500514123worker1 # [ 17.642956] nix-grpc-daemon[780]: ts=2026-09-20T13:33:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4124worker2 # [ 17.759144] nix-grpc-daemon[787]: ts=2026-09-20T13:33:41Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4125client # [ 17.818162] dhcpcd[567]: eth0: leased 10.0.2.15 for 86400 seconds4126worker2 # [ 17.768163] nix-grpc-daemon[787]: ts=2026-09-20T13:33:41Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4127client # [ 17.819184] dhcpcd[567]: eth0: adding route to 10.0.2.0/244128client # [ 17.819895] dhcpcd[567]: eth0: adding default route via 10.0.2.24129worker2 # [ 17.773451] nix-grpc-daemon[787]: ts=2026-09-20T13:33:41Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14130worker1 # [ 17.843713] nix-grpc-daemon[780]: ts=2026-09-20T13:33:41Z level=info event=scheduler_connected addr=lb:500514131worker1 # [ 17.845473] nix-grpc-daemon[780]: ts=2026-09-20T13:33:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4132worker2 # [ 17.820127] nix-grpc-daemon[787]: ts=2026-09-20T13:33:41Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4133worker2 # [ 17.825351] systemd[1]: Started Nix worker-protocol over gRPC.4134worker2 # [ 17.828149] systemd[1]: Startup finished in 871ms (kernel) + 4.944s (initrd) + 12.008s (userspace) = 17.824s.4135client # [ 17.913789] systemd[1]: Started DHCP Client.4136client # [ 17.916165] systemd[1]: Reached target Multi-User System.4137client # [ 17.917211] systemd[1]: Startup finished in 873ms (kernel) + 5.034s (initrd) + 12.008s (userspace) = 17.916s.4138worker2 # [ 17.878503] nix-grpc-daemon[787]: ts=2026-09-20T13:33:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4139worker1 # [ 18.046057] nix-grpc-daemon[780]: ts=2026-09-20T13:33:41Z level=info event=scheduler_connected addr=lb:500514140worker1 # [ 18.047793] nix-grpc-daemon[780]: ts=2026-09-20T13:33:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4141worker2 # [ 18.080766] nix-grpc-daemon[787]: ts=2026-09-20T13:33:41Z level=info event=scheduler_connected addr=lb:500514142worker2 # [ 18.082444] nix-grpc-daemon[787]: ts=2026-09-20T13:33:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4143worker1 # [ 18.248173] nix-grpc-daemon[780]: ts=2026-09-20T13:33:41Z level=info event=scheduler_connected addr=lb:500514144worker1 # [ 18.250043] nix-grpc-daemon[780]: ts=2026-09-20T13:33:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4145worker2 # [ 18.283691] nix-grpc-daemon[787]: ts=2026-09-20T13:33:41Z level=info event=scheduler_connected addr=lb:500514146worker2 # [ 18.287560] nix-grpc-daemon[787]: ts=2026-09-20T13:33:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4147worker1 # [ 18.450635] nix-grpc-daemon[780]: ts=2026-09-20T13:33:41Z level=info event=scheduler_connected addr=lb:500514148worker1 # [ 18.452354] nix-grpc-daemon[780]: ts=2026-09-20T13:33:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4149worker2 # [ 18.489060] nix-grpc-daemon[787]: ts=2026-09-20T13:33:41Z level=info event=scheduler_connected addr=lb:500514150worker2 # [ 18.492833] nix-grpc-daemon[787]: ts=2026-09-20T13:33:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4151worker1 # [ 18.653331] nix-grpc-daemon[780]: ts=2026-09-20T13:33:41Z level=info event=scheduler_connected addr=lb:500514152worker1 # [ 18.654969] nix-grpc-daemon[780]: ts=2026-09-20T13:33:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4153worker2 # [ 18.694242] nix-grpc-daemon[787]: ts=2026-09-20T13:33:42Z level=info event=scheduler_connected addr=lb:500514154worker2 # [ 18.698161] nix-grpc-daemon[787]: ts=2026-09-20T13:33:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4155worker1 # [ 18.855664] nix-grpc-daemon[780]: ts=2026-09-20T13:33:42Z level=info event=scheduler_connected addr=lb:500514156worker1 # [ 18.857480] nix-grpc-daemon[780]: ts=2026-09-20T13:33:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4157worker2 # [ 18.900494] nix-grpc-daemon[787]: ts=2026-09-20T13:33:42Z level=info event=scheduler_connected addr=lb:500514158worker2 # [ 18.903832] nix-grpc-daemon[787]: ts=2026-09-20T13:33:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4159worker1 # [ 19.058438] nix-grpc-daemon[780]: ts=2026-09-20T13:33:42Z level=info event=scheduler_connected addr=lb:500514160worker1 # [ 19.062497] nix-grpc-daemon[780]: ts=2026-09-20T13:33:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4161worker2 # [ 19.100971] nix-grpc-daemon[787]: ts=2026-09-20T13:33:42Z level=info event=scheduler_connected addr=lb:500514162worker2 # [ 19.102675] nix-grpc-daemon[787]: ts=2026-09-20T13:33:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4163worker1 # [ 19.262777] nix-grpc-daemon[780]: ts=2026-09-20T13:33:42Z level=info event=scheduler_connected addr=lb:500514164worker1 # [ 19.264442] nix-grpc-daemon[780]: ts=2026-09-20T13:33:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4165worker2 # [ 19.303570] nix-grpc-daemon[787]: ts=2026-09-20T13:33:42Z level=info event=scheduler_connected addr=lb:500514166worker2 # [ 19.305331] nix-grpc-daemon[787]: ts=2026-09-20T13:33:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4167worker1 # [ 19.465072] nix-grpc-daemon[780]: ts=2026-09-20T13:33:42Z level=info event=scheduler_connected addr=lb:500514168worker1 # [ 19.466752] nix-grpc-daemon[780]: ts=2026-09-20T13:33:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4169worker2 # [ 19.506129] nix-grpc-daemon[787]: ts=2026-09-20T13:33:42Z level=info event=scheduler_connected addr=lb:500514170worker2 # [ 19.508109] nix-grpc-daemon[787]: ts=2026-09-20T13:33:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4171worker1 # [ 19.668590] nix-grpc-daemon[780]: ts=2026-09-20T13:33:42Z level=info event=scheduler_connected addr=lb:500514172worker1 # [ 19.671070] nix-grpc-daemon[780]: ts=2026-09-20T13:33:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4173worker2 # [ 19.708865] nix-grpc-daemon[787]: ts=2026-09-20T13:33:43Z level=info event=scheduler_connected addr=lb:500514174worker2 # [ 19.710517] nix-grpc-daemon[787]: ts=2026-09-20T13:33:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4175worker1 # [ 19.869428] nix-grpc-daemon[780]: ts=2026-09-20T13:33:43Z level=info event=scheduler_connected addr=lb:500514176worker1 # [ 19.873338] nix-grpc-daemon[780]: ts=2026-09-20T13:33:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4177worker2 # [ 19.911383] nix-grpc-daemon[787]: ts=2026-09-20T13:33:43Z level=info event=scheduler_connected addr=lb:500514178worker2 # [ 19.912978] nix-grpc-daemon[787]: ts=2026-09-20T13:33:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4179worker1 # [ 20.074607] nix-grpc-daemon[780]: ts=2026-09-20T13:33:43Z level=info event=scheduler_connected addr=lb:500514180worker1 # [ 20.078796] nix-grpc-daemon[780]: ts=2026-09-20T13:33:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4181worker2 # [ 20.113690] nix-grpc-daemon[787]: ts=2026-09-20T13:33:43Z level=info event=scheduler_connected addr=lb:500514182worker2 # [ 20.115542] nix-grpc-daemon[787]: ts=2026-09-20T13:33:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4183worker1 # [ 20.279623] nix-grpc-daemon[780]: ts=2026-09-20T13:33:43Z level=info event=scheduler_connected addr=lb:500514184worker1 # [ 20.283438] nix-grpc-daemon[780]: ts=2026-09-20T13:33:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4185worker2 # [ 20.316846] nix-grpc-daemon[787]: ts=2026-09-20T13:33:43Z level=info event=scheduler_connected addr=lb:500514186worker2 # [ 20.320683] nix-grpc-daemon[787]: ts=2026-09-20T13:33:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4187worker1 # [ 20.484293] nix-grpc-daemon[780]: ts=2026-09-20T13:33:43Z level=info event=scheduler_connected addr=lb:500514188worker1 # [ 20.488185] nix-grpc-daemon[780]: ts=2026-09-20T13:33:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4189worker2 # [ 20.521906] nix-grpc-daemon[787]: ts=2026-09-20T13:33:43Z level=info event=scheduler_connected addr=lb:500514190worker2 # [ 20.525746] nix-grpc-daemon[787]: ts=2026-09-20T13:33:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4191worker1 # [ 20.690352] nix-grpc-daemon[780]: ts=2026-09-20T13:33:43Z level=info event=scheduler_connected addr=lb:500514192worker1 # [ 20.693482] nix-grpc-daemon[780]: ts=2026-09-20T13:33:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4193worker2 # [ 20.726335] nix-grpc-daemon[787]: ts=2026-09-20T13:33:44Z level=info event=scheduler_connected addr=lb:500514194worker2 # [ 20.728062] nix-grpc-daemon[787]: ts=2026-09-20T13:33:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4195worker1 # [ 20.891123] nix-grpc-daemon[780]: ts=2026-09-20T13:33:44Z level=info event=scheduler_connected addr=lb:500514196worker1 # [ 20.895223] nix-grpc-daemon[780]: ts=2026-09-20T13:33:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4197worker2 # [ 20.931274] nix-grpc-daemon[787]: ts=2026-09-20T13:33:44Z level=info event=scheduler_connected addr=lb:500514198worker2 # [ 20.934626] nix-grpc-daemon[787]: ts=2026-09-20T13:33:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4199worker1 # [ 21.095673] nix-grpc-daemon[780]: ts=2026-09-20T13:33:44Z level=info event=scheduler_connected addr=lb:500514200worker1 # [ 21.099577] nix-grpc-daemon[780]: ts=2026-09-20T13:33:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4201worker2 # [ 21.131490] nix-grpc-daemon[787]: ts=2026-09-20T13:33:44Z level=info event=scheduler_connected addr=lb:500514202worker2 # [ 21.133233] nix-grpc-daemon[787]: ts=2026-09-20T13:33:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4203worker1 # [ 21.300441] nix-grpc-daemon[780]: ts=2026-09-20T13:33:44Z level=info event=scheduler_connected addr=lb:500514204worker1 # [ 21.304117] nix-grpc-daemon[780]: ts=2026-09-20T13:33:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4205worker2 # [ 21.334587] nix-grpc-daemon[787]: ts=2026-09-20T13:33:44Z level=info event=scheduler_connected addr=lb:500514206worker2 # [ 21.338389] nix-grpc-daemon[787]: ts=2026-09-20T13:33:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4207worker1 # [ 21.504912] nix-grpc-daemon[780]: ts=2026-09-20T13:33:44Z level=info event=scheduler_connected addr=lb:500514208worker1 # [ 21.508714] nix-grpc-daemon[780]: ts=2026-09-20T13:33:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4209worker2 # [ 21.539668] nix-grpc-daemon[787]: ts=2026-09-20T13:33:44Z level=info event=scheduler_connected addr=lb:500514210worker2 # [ 21.543678] nix-grpc-daemon[787]: ts=2026-09-20T13:33:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4211worker1 # [ 21.710955] nix-grpc-daemon[780]: ts=2026-09-20T13:33:44Z level=info event=scheduler_connected addr=lb:500514212worker1 # [ 21.714141] nix-grpc-daemon[780]: ts=2026-09-20T13:33:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4213worker2 # [ 21.744772] nix-grpc-daemon[787]: ts=2026-09-20T13:33:45Z level=info event=scheduler_connected addr=lb:500514214worker2 # [ 21.748499] nix-grpc-daemon[787]: ts=2026-09-20T13:33:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4215worker1 # [ 21.911298] nix-grpc-daemon[780]: ts=2026-09-20T13:33:45Z level=info event=scheduler_connected addr=lb:500514216worker1 # [ 21.913081] nix-grpc-daemon[780]: ts=2026-09-20T13:33:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4217worker2 # [ 21.949355] nix-grpc-daemon[787]: ts=2026-09-20T13:33:45Z level=info event=scheduler_connected addr=lb:500514218worker2 # [ 21.951067] nix-grpc-daemon[787]: ts=2026-09-20T13:33:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4219worker1 # [ 22.113652] nix-grpc-daemon[780]: ts=2026-09-20T13:33:45Z level=info event=scheduler_connected addr=lb:500514220worker1 # [ 22.115508] nix-grpc-daemon[780]: ts=2026-09-20T13:33:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4221worker2 # [ 22.152460] nix-grpc-daemon[787]: ts=2026-09-20T13:33:45Z level=info event=scheduler_connected addr=lb:500514222worker2 # [ 22.156309] nix-grpc-daemon[787]: ts=2026-09-20T13:33:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4223worker1 # [ 22.316094] nix-grpc-daemon[780]: ts=2026-09-20T13:33:45Z level=info event=scheduler_connected addr=lb:500514224worker1 # [ 22.318035] nix-grpc-daemon[780]: ts=2026-09-20T13:33:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4225worker2 # [ 22.357285] nix-grpc-daemon[787]: ts=2026-09-20T13:33:45Z level=info event=scheduler_connected addr=lb:500514226worker2 # [ 22.361198] nix-grpc-daemon[787]: ts=2026-09-20T13:33:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4227worker1 # [ 22.518646] nix-grpc-daemon[780]: ts=2026-09-20T13:33:45Z level=info event=scheduler_connected addr=lb:500514228worker1 # [ 22.520371] nix-grpc-daemon[780]: ts=2026-09-20T13:33:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4229worker2 # [ 22.561781] nix-grpc-daemon[787]: ts=2026-09-20T13:33:45Z level=info event=scheduler_connected addr=lb:500514230worker2 # [ 22.563435] nix-grpc-daemon[787]: ts=2026-09-20T13:33:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4231worker1 # [ 22.721362] nix-grpc-daemon[780]: ts=2026-09-20T13:33:45Z level=info event=scheduler_connected addr=lb:500514232worker1 # [ 22.723128] nix-grpc-daemon[780]: ts=2026-09-20T13:33:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4233worker2 # [ 22.764572] nix-grpc-daemon[787]: ts=2026-09-20T13:33:46Z level=info event=scheduler_connected addr=lb:500514234worker2 # [ 22.768357] nix-grpc-daemon[787]: ts=2026-09-20T13:33:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4235worker1 # [ 22.923702] nix-grpc-daemon[780]: ts=2026-09-20T13:33:46Z level=info event=scheduler_connected addr=lb:500514236worker1 # [ 22.925421] nix-grpc-daemon[780]: ts=2026-09-20T13:33:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4237worker2 # [ 22.969355] nix-grpc-daemon[787]: ts=2026-09-20T13:33:46Z level=info event=scheduler_connected addr=lb:500514238worker2 # [ 22.970959] nix-grpc-daemon[787]: ts=2026-09-20T13:33:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4239worker1 # [ 23.125927] nix-grpc-daemon[780]: ts=2026-09-20T13:33:46Z level=info event=scheduler_connected addr=lb:500514240worker1 # [ 23.127640] nix-grpc-daemon[780]: ts=2026-09-20T13:33:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4241worker2 # [ 23.172345] nix-grpc-daemon[787]: ts=2026-09-20T13:33:46Z level=info event=scheduler_connected addr=lb:500514242worker2 # [ 23.175975] nix-grpc-daemon[787]: ts=2026-09-20T13:33:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4243worker1 # [ 23.328659] nix-grpc-daemon[780]: ts=2026-09-20T13:33:46Z level=info event=scheduler_connected addr=lb:500514244worker1 # [ 23.332533] nix-grpc-daemon[780]: ts=2026-09-20T13:33:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4245worker2 # [ 23.377308] nix-grpc-daemon[787]: ts=2026-09-20T13:33:46Z level=info event=scheduler_connected addr=lb:500514246worker2 # [ 23.381137] nix-grpc-daemon[787]: ts=2026-09-20T13:33:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4247worker1 # [ 23.533079] nix-grpc-daemon[780]: ts=2026-09-20T13:33:46Z level=info event=scheduler_connected addr=lb:500514248worker1 # [ 23.535055] nix-grpc-daemon[780]: ts=2026-09-20T13:33:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4249worker2 # [ 23.581516] nix-grpc-daemon[787]: ts=2026-09-20T13:33:46Z level=info event=scheduler_connected addr=lb:500514250worker2 # [ 23.583211] nix-grpc-daemon[787]: ts=2026-09-20T13:33:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4251worker1 # [ 23.735977] nix-grpc-daemon[780]: ts=2026-09-20T13:33:46Z level=info event=scheduler_connected addr=lb:500514252worker1 # [ 23.737603] nix-grpc-daemon[780]: ts=2026-09-20T13:33:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4253worker2 # [ 23.784099] nix-grpc-daemon[787]: ts=2026-09-20T13:33:47Z level=info event=scheduler_connected addr=lb:500514254worker2 # [ 23.785800] nix-grpc-daemon[787]: ts=2026-09-20T13:33:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4255worker1 # [ 23.938219] nix-grpc-daemon[780]: ts=2026-09-20T13:33:47Z level=info event=scheduler_connected addr=lb:500514256worker1 # [ 23.939874] nix-grpc-daemon[780]: ts=2026-09-20T13:33:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4257worker2 # [ 23.988361] nix-grpc-daemon[787]: ts=2026-09-20T13:33:47Z level=info event=scheduler_connected addr=lb:500514258worker2 # [ 23.991445] nix-grpc-daemon[787]: ts=2026-09-20T13:33:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4259worker1 # [ 24.140467] nix-grpc-daemon[780]: ts=2026-09-20T13:33:47Z level=info event=scheduler_connected addr=lb:500514260worker1 # [ 24.142203] nix-grpc-daemon[780]: ts=2026-09-20T13:33:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4261worker2 # [ 24.189154] nix-grpc-daemon[787]: ts=2026-09-20T13:33:47Z level=info event=scheduler_connected addr=lb:500514262worker2 # [ 24.190811] nix-grpc-daemon[787]: ts=2026-09-20T13:33:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4263worker1 # [ 24.342786] nix-grpc-daemon[780]: ts=2026-09-20T13:33:47Z level=info event=scheduler_connected addr=lb:500514264worker1 # [ 24.344482] nix-grpc-daemon[780]: ts=2026-09-20T13:33:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4265worker2 # [ 24.391388] nix-grpc-daemon[787]: ts=2026-09-20T13:33:47Z level=info event=scheduler_connected addr=lb:500514266worker2 # [ 24.393100] nix-grpc-daemon[787]: ts=2026-09-20T13:33:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4267worker1 # [ 24.544986] nix-grpc-daemon[780]: ts=2026-09-20T13:33:47Z level=info event=scheduler_connected addr=lb:500514268worker1 # [ 24.546695] nix-grpc-daemon[780]: ts=2026-09-20T13:33:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4269worker2 # [ 24.594338] nix-grpc-daemon[787]: ts=2026-09-20T13:33:47Z level=info event=scheduler_connected addr=lb:500514270worker2 # [ 24.598082] nix-grpc-daemon[787]: ts=2026-09-20T13:33:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4271worker1 # [ 24.747864] nix-grpc-daemon[780]: ts=2026-09-20T13:33:47Z level=info event=scheduler_connected addr=lb:500514272worker1 # [ 24.749605] nix-grpc-daemon[780]: ts=2026-09-20T13:33:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4273worker2 # [ 24.798538] nix-grpc-daemon[787]: ts=2026-09-20T13:33:48Z level=info event=scheduler_connected addr=lb:500514274worker2 # [ 24.800229] nix-grpc-daemon[787]: ts=2026-09-20T13:33:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4275worker1 # [ 24.948799] nix-grpc-daemon[780]: ts=2026-09-20T13:33:48Z level=info event=scheduler_connected addr=lb:500514276worker1 # [ 24.950485] nix-grpc-daemon[780]: ts=2026-09-20T13:33:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4277worker2 # [ 25.002737] nix-grpc-daemon[787]: ts=2026-09-20T13:33:48Z level=info event=scheduler_connected addr=lb:500514278worker2 # [ 25.006206] nix-grpc-daemon[787]: ts=2026-09-20T13:33:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4279worker1 # [ 25.150914] nix-grpc-daemon[780]: ts=2026-09-20T13:33:48Z level=info event=scheduler_connected addr=lb:500514280worker1 # [ 25.152585] nix-grpc-daemon[780]: ts=2026-09-20T13:33:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4281worker2 # [ 25.207197] nix-grpc-daemon[787]: ts=2026-09-20T13:33:48Z level=info event=scheduler_connected addr=lb:500514282worker2 # [ 25.210931] nix-grpc-daemon[787]: ts=2026-09-20T13:33:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4283worker1 # [ 25.353311] nix-grpc-daemon[780]: ts=2026-09-20T13:33:48Z level=info event=scheduler_connected addr=lb:500514284worker1 # [ 25.355171] nix-grpc-daemon[780]: ts=2026-09-20T13:33:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4285worker2 # [ 25.412090] nix-grpc-daemon[787]: ts=2026-09-20T13:33:48Z level=info event=scheduler_connected addr=lb:500514286worker2 # [ 25.415896] nix-grpc-daemon[787]: ts=2026-09-20T13:33:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4287worker1 # [ 25.555839] nix-grpc-daemon[780]: ts=2026-09-20T13:33:48Z level=info event=scheduler_connected addr=lb:500514288worker1 # [ 25.558171] nix-grpc-daemon[780]: ts=2026-09-20T13:33:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4289worker2 # [ 25.616833] nix-grpc-daemon[787]: ts=2026-09-20T13:33:48Z level=info event=scheduler_connected addr=lb:500514290worker2 # [ 25.620590] nix-grpc-daemon[787]: ts=2026-09-20T13:33:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4291worker1 # [ 25.759812] nix-grpc-daemon[780]: ts=2026-09-20T13:33:48Z level=info event=scheduler_connected addr=lb:500514292worker1 # [ 25.761503] nix-grpc-daemon[780]: ts=2026-09-20T13:33:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4293worker2 # [ 25.821749] nix-grpc-daemon[787]: ts=2026-09-20T13:33:49Z level=info event=scheduler_connected addr=lb:500514294worker2 # [ 25.825631] nix-grpc-daemon[787]: ts=2026-09-20T13:33:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4295worker1 # [ 25.960329] nix-grpc-daemon[780]: ts=2026-09-20T13:33:49Z level=info event=scheduler_connected addr=lb:500514296worker1 # [ 25.962496] nix-grpc-daemon[780]: ts=2026-09-20T13:33:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4297worker2 # [ 26.027962] nix-grpc-daemon[787]: ts=2026-09-20T13:33:49Z level=info event=scheduler_connected addr=lb:500514298worker2 # [ 26.031270] nix-grpc-daemon[787]: ts=2026-09-20T13:33:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4299worker1 # [ 26.162993] nix-grpc-daemon[780]: ts=2026-09-20T13:33:49Z level=info event=scheduler_connected addr=lb:500514300worker1 # [ 26.164833] nix-grpc-daemon[780]: ts=2026-09-20T13:33:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4301worker2 # [ 26.228457] nix-grpc-daemon[787]: ts=2026-09-20T13:33:49Z level=info event=scheduler_connected addr=lb:500514302worker2 # [ 26.230235] nix-grpc-daemon[787]: ts=2026-09-20T13:33:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4303worker1 # [ 26.365298] nix-grpc-daemon[780]: ts=2026-09-20T13:33:49Z level=info event=scheduler_connected addr=lb:500514304worker1 # [ 26.366986] nix-grpc-daemon[780]: ts=2026-09-20T13:33:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4305worker2 # [ 26.430956] nix-grpc-daemon[787]: ts=2026-09-20T13:33:49Z level=info event=scheduler_connected addr=lb:500514306worker2 # [ 26.432732] nix-grpc-daemon[787]: ts=2026-09-20T13:33:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4307worker1 # [ 26.567569] nix-grpc-daemon[780]: ts=2026-09-20T13:33:49Z level=info event=scheduler_connected addr=lb:500514308worker1 # [ 26.569277] nix-grpc-daemon[780]: ts=2026-09-20T13:33:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4309worker2 # [ 26.633378] nix-grpc-daemon[787]: ts=2026-09-20T13:33:49Z level=info event=scheduler_connected addr=lb:500514310worker2 # [ 26.635067] nix-grpc-daemon[787]: ts=2026-09-20T13:33:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4311worker1 # [ 26.771671] nix-grpc-daemon[780]: ts=2026-09-20T13:33:49Z level=info event=scheduler_connected addr=lb:500514312worker1 # [ 26.774699] nix-grpc-daemon[780]: ts=2026-09-20T13:33:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4313worker2 # [ 26.835859] nix-grpc-daemon[787]: ts=2026-09-20T13:33:50Z level=info event=scheduler_connected addr=lb:500514314worker2 # [ 26.839289] nix-grpc-daemon[787]: ts=2026-09-20T13:33:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4315worker1 # [ 26.972532] nix-grpc-daemon[780]: ts=2026-09-20T13:33:50Z level=info event=scheduler_connected addr=lb:500514316worker1 # [ 26.976425] nix-grpc-daemon[780]: ts=2026-09-20T13:33:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4317worker2 # [ 27.041562] nix-grpc-daemon[787]: ts=2026-09-20T13:33:50Z level=info event=scheduler_connected addr=lb:500514318worker2 # [ 27.045193] nix-grpc-daemon[787]: ts=2026-09-20T13:33:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4319worker1 # [ 27.176791] nix-grpc-daemon[780]: ts=2026-09-20T13:33:50Z level=info event=scheduler_connected addr=lb:500514320worker1 # [ 27.178764] nix-grpc-daemon[780]: ts=2026-09-20T13:33:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4321worker2 # [ 27.245578] nix-grpc-daemon[787]: ts=2026-09-20T13:33:50Z level=info event=scheduler_connected addr=lb:500514322worker2 # [ 27.247406] nix-grpc-daemon[787]: ts=2026-09-20T13:33:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4323worker1 # [ 27.379553] nix-grpc-daemon[780]: ts=2026-09-20T13:33:50Z level=info event=scheduler_connected addr=lb:500514324worker1 # [ 27.383361] nix-grpc-daemon[780]: ts=2026-09-20T13:33:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4325worker2 # [ 27.448098] nix-grpc-daemon[787]: ts=2026-09-20T13:33:50Z level=info event=scheduler_connected addr=lb:500514326worker2 # [ 27.449849] nix-grpc-daemon[787]: ts=2026-09-20T13:33:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4327worker1 # [ 27.583861] nix-grpc-daemon[780]: ts=2026-09-20T13:33:50Z level=info event=scheduler_connected addr=lb:500514328worker1 # [ 27.585885] nix-grpc-daemon[780]: ts=2026-09-20T13:33:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4329worker2 # [ 27.650557] nix-grpc-daemon[787]: ts=2026-09-20T13:33:50Z level=info event=scheduler_connected addr=lb:500514330worker2 # [ 27.652472] nix-grpc-daemon[787]: ts=2026-09-20T13:33:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4331worker1 # [ 27.786911] nix-grpc-daemon[780]: ts=2026-09-20T13:33:51Z level=info event=scheduler_connected addr=lb:500514332worker1 # [ 27.788720] nix-grpc-daemon[780]: ts=2026-09-20T13:33:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4333worker2 # [ 27.852983] nix-grpc-daemon[787]: ts=2026-09-20T13:33:51Z level=info event=scheduler_connected addr=lb:500514334worker2 # [ 27.854729] nix-grpc-daemon[787]: ts=2026-09-20T13:33:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4335worker1 # [ 27.989311] nix-grpc-daemon[780]: ts=2026-09-20T13:33:51Z level=info event=scheduler_connected addr=lb:500514336worker1 # [ 27.991191] nix-grpc-daemon[780]: ts=2026-09-20T13:33:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4337worker2 # [ 28.056622] nix-grpc-daemon[787]: ts=2026-09-20T13:33:51Z level=info event=scheduler_connected addr=lb:500514338worker2 # [ 28.060345] nix-grpc-daemon[787]: ts=2026-09-20T13:33:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4339worker1 # [ 28.191640] nix-grpc-daemon[780]: ts=2026-09-20T13:33:51Z level=info event=scheduler_connected addr=lb:500514340worker1 # [ 28.193499] nix-grpc-daemon[780]: ts=2026-09-20T13:33:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4341worker2 # [ 28.258208] nix-grpc-daemon[787]: ts=2026-09-20T13:33:51Z level=info event=scheduler_connected addr=lb:500514342worker2 # [ 28.260060] nix-grpc-daemon[787]: ts=2026-09-20T13:33:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4343worker1 # [ 28.394297] nix-grpc-daemon[780]: ts=2026-09-20T13:33:51Z level=info event=scheduler_connected addr=lb:500514344worker1 # [ 28.396474] nix-grpc-daemon[780]: ts=2026-09-20T13:33:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4345worker2 # [ 28.461236] nix-grpc-daemon[787]: ts=2026-09-20T13:33:51Z level=info event=scheduler_connected addr=lb:500514346worker2 # [ 28.465108] nix-grpc-daemon[787]: ts=2026-09-20T13:33:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4347worker1 # [ 28.597446] nix-grpc-daemon[780]: ts=2026-09-20T13:33:51Z level=info event=scheduler_connected addr=lb:500514348worker1 # [ 28.601449] nix-grpc-daemon[780]: ts=2026-09-20T13:33:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4349worker2 # [ 28.665521] nix-grpc-daemon[787]: ts=2026-09-20T13:33:52Z level=info event=scheduler_connected addr=lb:500514350worker2 # [ 28.669504] nix-grpc-daemon[787]: ts=2026-09-20T13:33:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4351worker1 # [ 28.802225] nix-grpc-daemon[780]: ts=2026-09-20T13:33:52Z level=info event=scheduler_connected addr=lb:500514352worker1 # [ 28.804060] nix-grpc-daemon[780]: ts=2026-09-20T13:33:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4353worker2 # [ 28.869907] nix-grpc-daemon[787]: ts=2026-09-20T13:33:52Z level=info event=scheduler_connected addr=lb:500514354worker2 # [ 28.871610] nix-grpc-daemon[787]: ts=2026-09-20T13:33:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4355worker1 # [ 29.005167] nix-grpc-daemon[780]: ts=2026-09-20T13:33:52Z level=info event=scheduler_connected addr=lb:500514356worker1 # [ 29.009150] nix-grpc-daemon[780]: ts=2026-09-20T13:33:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4357worker2 # [ 29.074141] nix-grpc-daemon[787]: ts=2026-09-20T13:33:52Z level=info event=scheduler_connected addr=lb:500514358worker2 # [ 29.077582] nix-grpc-daemon[787]: ts=2026-09-20T13:33:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4359worker1 # [ 29.209637] nix-grpc-daemon[780]: ts=2026-09-20T13:33:52Z level=info event=scheduler_connected addr=lb:500514360worker1 # [ 29.213547] nix-grpc-daemon[780]: ts=2026-09-20T13:33:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4361worker2 # [ 29.275078] nix-grpc-daemon[787]: ts=2026-09-20T13:33:52Z level=info event=scheduler_connected addr=lb:500514362worker2 # [ 29.278222] nix-grpc-daemon[787]: ts=2026-09-20T13:33:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4363worker1 # [ 29.413541] nix-grpc-daemon[780]: ts=2026-09-20T13:33:52Z level=info event=scheduler_connected addr=lb:500514364worker1 # [ 29.417493] nix-grpc-daemon[780]: ts=2026-09-20T13:33:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4365worker2 # [ 29.478780] nix-grpc-daemon[787]: ts=2026-09-20T13:33:52Z level=info event=scheduler_connected addr=lb:500514366worker2 # [ 29.480561] nix-grpc-daemon[787]: ts=2026-09-20T13:33:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4367worker1 # [ 29.617880] nix-grpc-daemon[780]: ts=2026-09-20T13:33:52Z level=info event=scheduler_connected addr=lb:500514368worker1 # [ 29.619789] nix-grpc-daemon[780]: ts=2026-09-20T13:33:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4369worker2 # [ 29.681669] nix-grpc-daemon[787]: ts=2026-09-20T13:33:53Z level=info event=scheduler_connected addr=lb:500514370worker2 # [ 29.685335] nix-grpc-daemon[787]: ts=2026-09-20T13:33:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4371worker1 # [ 29.822294] nix-grpc-daemon[780]: ts=2026-09-20T13:33:53Z level=info event=scheduler_connected addr=lb:500514372worker1 # [ 29.825518] nix-grpc-daemon[780]: ts=2026-09-20T13:33:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4373worker2 # [ 29.885540] nix-grpc-daemon[787]: ts=2026-09-20T13:33:53Z level=info event=scheduler_connected addr=lb:500514374worker2 # [ 29.887339] nix-grpc-daemon[787]: ts=2026-09-20T13:33:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4375worker1 # [ 30.022679] nix-grpc-daemon[780]: ts=2026-09-20T13:33:53Z level=info event=scheduler_connected addr=lb:500514376worker1 # [ 30.024617] nix-grpc-daemon[780]: ts=2026-09-20T13:33:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4377worker2 # [ 30.089821] nix-grpc-daemon[787]: ts=2026-09-20T13:33:53Z level=info event=scheduler_connected addr=lb:500514378worker2 # [ 30.092947] nix-grpc-daemon[787]: ts=2026-09-20T13:33:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4379worker1 # [ 30.225191] nix-grpc-daemon[780]: ts=2026-09-20T13:33:53Z level=info event=scheduler_connected addr=lb:500514380worker1 # [ 30.226869] nix-grpc-daemon[780]: ts=2026-09-20T13:33:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4381lb # [ 30.265513] rustfs-bucket-start[1048]: mb s3://farm4382lb # [ 30.271618] systemd[1]: Finished rustfs-bucket.service.4383lb # [ 30.278437] systemd[1]: Starting niks3 server...4384worker2 # [ 30.290749] nix-grpc-daemon[787]: ts=2026-09-20T13:33:53Z level=info event=scheduler_connected addr=lb:500514385worker2 # [ 30.294665] nix-grpc-daemon[787]: ts=2026-09-20T13:33:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4386worker1 # [ 30.427647] nix-grpc-daemon[780]: ts=2026-09-20T13:33:53Z level=info event=scheduler_connected addr=lb:500514387worker1 # [ 30.429431] nix-grpc-daemon[780]: ts=2026-09-20T13:33:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4388lb # [ 30.471298] postgres[1064]: [1064] ERROR: relation "goose_db_version" does not exist at character 364389lb # [ 30.472831] postgres[1064]: [1064] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4390lb # [ 30.509986] niks3-server[1058]: 2026/09/20 13:33:53 OK 20241026095416_initial_model.sql (18.81ms)4391lb # [ 30.513678] niks3-server[1058]: 2026/09/20 13:33:53 OK 20251210153512_drop_unused_gin_index.sql (2.26ms)4392lb # [ 30.518352] niks3-server[1058]: 2026/09/20 13:33:53 OK 20251218171726_add_pins.sql (3.23ms)4393worker2 # [ 30.495266] nix-grpc-daemon[787]: ts=2026-09-20T13:33:53Z level=info event=scheduler_connected addr=lb:500514394worker2 # [ 30.496891] nix-grpc-daemon[787]: ts=2026-09-20T13:33:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4395lb # [ 30.523210] niks3-server[1058]: 2026/09/20 13:33:53 OK 20260628120000_add_object_size_and_stats.sql (4.96ms)4396lb # [ 30.527343] niks3-server[1058]: 2026/09/20 13:33:53 OK 20260905000000_add_claims.sql (2.78ms)4397lb # [ 30.530888] niks3-server[1058]: 2026/09/20 13:33:53 OK 20260920000000_drop_claims.sql (3.41ms)4398lb # [ 30.532019] niks3-server[1058]: 2026/09/20 13:33:53 goose: successfully migrated database to version: 202609200000004399lb # [ 30.536431] niks3-server[1058]: 2026/09/20 13:33:53 OK 1_commit_pending_closure.sql (5.64ms)4400lb # [ 30.538798] niks3-server[1058]: 2026/09/20 13:33:53 OK 2_object_stats_trigger.sql (1.22ms)4401lb # [ 30.539935] niks3-server[1058]: 2026/09/20 13:33:53 goose: up to current file version: 24402lb # [ 30.544150] niks3-server[1058]: 2026/09/20 13:33:53 INFO Loaded signing key name=farm-test-1 path=/nix/store/c745bsy529madd2s6hz4z5is2436ni04-key4403lb # [ 30.592117] niks3-server[1058]: 2026/09/20 13:33:53 INFO Read proxy enabled — serving cache objects from S34404lb # [ 30.594073] niks3-server[1058]: 2026/09/20 13:33:53 INFO Using socket-activated listener address=0.0.0.0:57514405lb # [ 30.600335] systemd[1]: Started niks3 server.4406lb # [ 30.600977] niks3-server[1058]: 2026/09/20 13:33:53 INFO systemd watchdog enabled interval=15s4407lb # [ 30.602772] systemd[1]: Reached target Multi-User System.4408lb # [ 30.604128] systemd[1]: Startup finished in 871ms (kernel) + 3.940s (initrd) + 25.791s (userspace) = 30.603s.4409lb # [ 30.605975] niks3-server[1058]: 2026/09/20 13:33:53 INFO Starting HTTP server address=0.0.0.0:57514410lb # [ 30.610896] niks3-server[1058]: 2026/09/20 13:33:53 INFO lead: acquired remote=192.168.1.4:345004411worker2 # [ 30.589868] nix-grpc-daemon[787]: ts=2026-09-20T13:33:53Z level=info event=scheduler_take_over4412worker1 # [ 30.629944] nix-grpc-daemon[780]: ts=2026-09-20T13:33:53Z level=info event=scheduler_connected addr=lb:500514413worker1 # [ 30.631734] nix-grpc-daemon[780]: ts=2026-09-20T13:33:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4414worker2 # [ 30.697664] nix-grpc-daemon[787]: ts=2026-09-20T13:33:54Z level=info event=scheduler_connected addr=lb:500514415worker2 # [ 30.701493] nix-grpc-daemon[787]: ts=2026-09-20T13:33:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4416worker1 # [ 30.832788] nix-grpc-daemon[780]: ts=2026-09-20T13:33:54Z level=info event=scheduler_connected addr=lb:500514417worker1 # [ 30.834497] nix-grpc-daemon[780]: ts=2026-09-20T13:33:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4418worker2 # [ 30.902487] nix-grpc-daemon[787]: ts=2026-09-20T13:33:54Z level=info event=scheduler_connected addr=lb:500514419worker2 # [ 30.925662] nix-grpc-daemon[787]: ts=2026-09-20T13:33:54Z level=info event=oidc_jwks provider=mock keys=14420worker2 # [ 30.930175] nix-grpc-daemon[787]: ts=2026-09-20T13:33:54Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:450704421worker2 # [ 30.935276] nix-grpc-daemon[787]: ts=2026-09-20T13:33:54Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04422worker2 # [ 31.004969] nix-grpc-daemon[787]: ts=2026-09-20T13:33:54Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:450704423worker1 # [ 31.035567] nix-grpc-daemon[780]: ts=2026-09-20T13:33:54Z level=info event=scheduler_connected addr=lb:500514424worker2 # [ 31.006486] nix-grpc-daemon[787]: ts=2026-09-20T13:33:54Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04425lb: (finished: waiting for unit niks3.service, in 19.50 seconds)4426lb: waiting for TCP port 5751 on localhost4427lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!4428lb: (finished: waiting for TCP port 5751 on localhost, in 0.08 seconds)4429lb: waiting for unit envoy.service4430lb: (finished: waiting for unit envoy.service, in 0.05 seconds)4431lb: waiting for TCP port 50051 on localhost4432lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!4433lb # [ 31.331464] envoy[891]: conn peer=[::1]:52398 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=24434lb: (finished: waiting for TCP port 50051 on localhost, in 0.03 seconds)4435worker2 # [ 31.432114] systemd[1]: Starting scheduler-token.service...4436worker2 # [ 31.486388] systemd[1]: scheduler-token.service: Deactivated successfully.4437worker2 # [ 31.487909] systemd[1]: Finished scheduler-token.service.4438worker2 # [ 31.489210] systemd[1]: scheduler-token.service: Consumed 18ms CPU time over 57ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4439lb: waiting until exactly no endpoints of x86_64-linux are unhealthy4440lb: must succeed: curl -sf localhost:9901/clusters4441lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4442lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.10 seconds)4443subtest: one node schedules and both builders hold a WorkerSession on it4444lb: must succeed: curl -sf localhost:9901/clusters4445lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4446worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4447worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4448lb: must succeed: curl -sf localhost:9901/clusters4449lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4450lb: must succeed: curl -sf localhost:9901/clusters4451lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4452lb: must succeed: curl -sf localhost:9901/clusters4453lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4454lb: must succeed: curl -sf localhost:9901/clusters4455lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4456worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'4457worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.05 seconds)4458worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4459worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4460lb: must succeed: curl -sf localhost:9901/clusters4461lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4462lb: must succeed: curl -sf localhost:9901/clusters4463lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4464worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4465worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4466worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4467worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4468worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4469worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4470worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4471worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4472(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.57 seconds)4473subtest: balancer auth: client cert, bearer token, nothing, unknown CN4474worker2 # [ 32.472411] nix-grpc-daemon[787]: ts=2026-09-20T13:33:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:450804475worker2 # [ 32.481810] systemd[1]: Started Nix Daemon.4476worker2 # [ 32.701351] nix-daemon[868]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4477worker2 # [ 32.754642] nix-grpc-daemon[787]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 20 ms (attempt 1/5)4478worker2 # [ 32.777142] nix-grpc-daemon[787]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 34 ms (attempt 2/5)4479worker2 # [ 32.814230] nix-grpc-daemon[787]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 80 ms (attempt 3/5)4480worker2 # [ 32.899567] nix-grpc-daemon[787]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 337 ms (attempt 4/5)4481worker2 # [ 33.243445] nix-grpc-daemon[787]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4482worker1 # [ 33.311701] nix-grpc-daemon[780]: ts=2026-09-20T13:33:56Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:374584483worker1 # [ 33.324749] systemd[1]: Started Nix Daemon.4484worker1 # [ 33.551510] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4485worker1 # [ 33.578508] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 43 ms (attempt 1/5)4486worker1 # [ 33.623963] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 180 ms (attempt 2/5)4487worker1 # [ 33.806892] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 234 ms (attempt 3/5)4488worker1 # [ 34.044045] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 90 ms (attempt 4/5)4489worker1 # [ 34.142197] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4490worker1 # [ 34.160148] nix-grpc-daemon[780]: ts=2026-09-20T13:33:57Z level=debug event=path_info_miss path=8rqmizrzfz4wllvr68b1lm9xv8fjj8kq-nixos-system-client-test error="error: path '/nix/store/8rqmizrzfz4wllvr68b1lm9xv8fjj8kq-nixos-system-client-test' is required, but there is no substituter that can build it"4491worker2 # [ 34.143919] nix-grpc-daemon[787]: ts=2026-09-20T13:33:57Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:450804492worker2 # [ 34.162627] nix-grpc-daemon[787]: ts=2026-09-20T13:33:57Z level=debug event=path_info_miss path=8rqmizrzfz4wllvr68b1lm9xv8fjj8kq-nixos-system-client-test error="error: path '/nix/store/8rqmizrzfz4wllvr68b1lm9xv8fjj8kq-nixos-system-client-test' is required, but there is no substituter that can build it"4493client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4494client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.08 seconds)4495client: must succeed: install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4496client: (finished: must succeed: install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds)4497worker1 # [ 34.549724] nix-grpc-daemon[780]: ts=2026-09-20T13:33:57Z level=info event=oidc_jwks provider=mock keys=14498worker1 # [ 34.553102] nix-grpc-daemon[780]: ts=2026-09-20T13:33:57Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:374584499worker2 # [ 34.540661] nix-grpc-daemon[787]: ts=2026-09-20T13:33:57Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:450804500worker2 # [ 34.546689] nix-grpc-daemon[787]: ts=2026-09-20T13:33:57Z level=debug event=path_info_miss path=8rqmizrzfz4wllvr68b1lm9xv8fjj8kq-nixos-system-client-test error="error: path '/nix/store/8rqmizrzfz4wllvr68b1lm9xv8fjj8kq-nixos-system-client-test' is required, but there is no substituter that can build it"4501worker1 # [ 34.582384] nix-grpc-daemon[780]: ts=2026-09-20T13:33:57Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:374584502worker1 # [ 34.592770] nix-grpc-daemon[780]: ts=2026-09-20T13:33:57Z level=debug event=path_info_miss path=8rqmizrzfz4wllvr68b1lm9xv8fjj8kq-nixos-system-client-test error="error: path '/nix/store/8rqmizrzfz4wllvr68b1lm9xv8fjj8kq-nixos-system-client-test' is required, but there is no substituter that can build it"4503client: must succeed: rm -r /var/lib/nix-grpc-store4504client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.03 seconds)4505worker2 # [ 34.784967] nix-grpc-daemon[787]: ts=2026-09-20T13:33:58Z level=info event=denied method=QueryMissing cn=- role=none4506worker1 # [ 35.020187] nix-grpc-daemon[780]: ts=2026-09-20T13:33:58Z level=info event=denied method=QueryMissing cn=stranger role=none4507(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 2.86 seconds)4508subtest: DAG build is scheduled across workers and lands in the cache4509client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24510worker2 # [ 35.223651] nix-grpc-daemon[787]: ts=2026-09-20T13:33:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:450804511client # don't know how to build these paths:4512client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4513worker1 # [ 35.267568] nix-grpc-daemon[780]: ts=2026-09-20T13:33:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374584514worker2 # [ 35.282816] nix-grpc-daemon[787]: ts=2026-09-20T13:33:58Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04515worker2 # [ 35.287354] nix-grpc-daemon[787]: ts=2026-09-20T13:33:58Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.4:50051 clients=14516worker2 # [ 35.333341] nix-grpc-daemon[787]: ts=2026-09-20T13:33:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:450804517worker2 # [ 35.353753] nix-grpc-daemon[787]: ts=2026-09-20T13:33:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:450804518client # copying 1 paths to 'lb:50051'...4519worker2 # [ 35.365377] nix-grpc-daemon[787]: ts=2026-09-20T13:33:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:450804520client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4521worker2 # [ 35.392838] nix-daemon[868]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4522worker2 # [ 35.416209] nix-grpc-daemon[787]: ts=2026-09-20T13:33:58Z level=info event=niks3_push_started pid=8804523worker2 # [ 35.674171] nix-grpc-daemon[880]: time=2026-09-20T13:33:59.005Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""4524worker2 # [ 35.855929] nix-daemon[868]: accepted connection from pid 893, user nix-grpc-daemon (trusted)4525worker2 # [ 35.869948] nix-daemon[868]: reaped child process 896, status = succeeded4526lb # [ 35.894546] envoy[891]: conn peer=[2001:db8:1::1]:56972 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=17434527lb # [ 35.896304] envoy[891]: conn peer=[2001:db8:1::1]:50840 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=363 ms=704528lb # [ 35.897842] envoy[891]: conn peer=[2001:db8:1::1]:50846 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=104529lb # [ 35.899443] envoy[891]: conn peer=[2001:db8:1::1]:50854 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=154530worker2 # [ 36.031628] nix-daemon[868]: accepted connection from pid 898, user nix-grpc-daemon (trusted)4531worker2 # [ 36.043896] nix-daemon[868]: reaped child process 901, status = succeeded4532lb # [ 36.084291] niks3-server[1058]: 2026/09/20 13:33:59 INFO Received uploads request method=POST path=/api/pending_closures4533worker2 # [ 36.094719] nix-grpc-daemon[880]: time=2026-09-20T13:33:59.429Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4534worker2 # [ 36.097247] nix-grpc-daemon[880]: time=2026-09-20T13:33:59.431Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4535lb # [ 36.154239] niks3-server[1058]: 2026/09/20 13:33:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4536lb # [ 36.166806] niks3-server[1058]: 2026/09/20 13:33:59 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4537lb # [ 36.175537] niks3-server[1058]: 2026/09/20 13:33:59 INFO Signed narinfos id=1 count=14538worker2 # [ 36.151851] nix-grpc-daemon[880]: time=2026-09-20T13:33:59.486Z level=INFO msg="Uploading 1 narinfos"4539lb # [ 36.184043] niks3-server[1058]: 2026/09/20 13:33:59 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4540lb # [ 36.187862] niks3-server[1058]: 2026/09/20 13:33:59 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4541lb # [ 36.196572] niks3-server[1058]: 2026/09/20 13:33:59 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo4542lb # [ 36.201331] niks3-server[1058]: 2026/09/20 13:33:59 INFO Completed upload id=14543worker2 # [ 36.176303] nix-grpc-daemon[880]: time=2026-09-20T13:33:59.510Z level=INFO msg="Upload complete. (489ms)"4544worker2 # [ 36.178472] nix-grpc-daemon[787]: ts=2026-09-20T13:33:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45080 duration_s=0 paths=1 nar_bytes_in=5044545worker2 # [ 36.181162] nix-grpc-daemon[787]: ts=2026-09-20T13:33:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:450804546worker2 # [ 36.189089] nix-daemon[868]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4547worker2 # [ 36.190321] nix-daemon[868]: reaped child process 878, status = succeeded4548client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4549client # farm-a> worker2: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4550worker2 # [ 36.195326] nix-daemon[868]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4551client # farm-a> LOG-farm-a-t14552worker2 # [ 37.825936] nix-daemon[868]: reaped child process 905, status = succeeded4553worker2 # [ 37.954878] nix-daemon[868]: accepted connection from pid 921, user nix-grpc-daemon (trusted)4554worker2 # [ 37.966655] nix-daemon[868]: reaped child process 924, status = succeeded4555lb # [ 38.006050] niks3-server[1058]: 2026/09/20 13:34:01 INFO Received uploads request method=POST path=/api/pending_closures4556worker2 # [ 38.145956] nix-grpc-daemon[880]: time=2026-09-20T13:34:01.479Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4557worker2 # [ 38.151729] nix-grpc-daemon[880]: time=2026-09-20T13:34:01.483Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4558lb # [ 38.222329] niks3-server[1058]: 2026/09/20 13:34:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4559lb # [ 38.228209] niks3-server[1058]: 2026/09/20 13:34:01 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4560lb # [ 38.233323] niks3-server[1058]: 2026/09/20 13:34:01 INFO Signed narinfos id=2 count=14561worker2 # [ 38.212148] nix-grpc-daemon[880]: time=2026-09-20T13:34:01.546Z level=INFO msg="Uploading 1 narinfos"4562lb # [ 38.239624] niks3-server[1058]: 2026/09/20 13:34:01 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4563lb # [ 38.254948] niks3-server[1058]: 2026/09/20 13:34:01 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4564lb # [ 38.263416] niks3-server[1058]: 2026/09/20 13:34:01 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4565lb # [ 38.272245] niks3-server[1058]: 2026/09/20 13:34:01 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4566lb # [ 38.281044] niks3-server[1058]: 2026/09/20 13:34:01 INFO Completed upload id=24567worker2 # [ 38.256092] nix-grpc-daemon[880]: time=2026-09-20T13:34:01.590Z level=INFO msg="Upload complete. (449ms)"4568worker2 # [ 38.260282] nix-grpc-daemon[787]: ts=2026-09-20T13:34:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45080 duration_s=2 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14569worker2 # [ 38.268912] nix-daemon[868]: reaped child process 903, status = succeeded4570worker2 # [ 38.273129] nix-grpc-daemon[787]: ts=2026-09-20T13:34:01Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04571worker2 # [ 38.274873] nix-grpc-daemon[787]: ts=2026-09-20T13:34:01Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.4:50051 clients=14572worker2 # [ 38.318203] nix-grpc-daemon[787]: ts=2026-09-20T13:34:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:450804573worker2 # [ 38.330240] nix-grpc-daemon[787]: ts=2026-09-20T13:34:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:450804574worker2 # [ 38.519120] nix-grpc-daemon[880]: time=2026-09-20T13:34:01.853Z level=INFO msg="All 1 paths already cached"4575client # copying 1 paths to 'lb:50051'...4576client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4577worker2 # [ 38.526193] nix-grpc-daemon[787]: ts=2026-09-20T13:34:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:450804578worker2 # [ 38.535463] nix-daemon[868]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4579worker2 # [ 38.714217] nix-daemon[868]: accepted connection from pid 930, user nix-grpc-daemon (trusted)4580worker2 # [ 38.726846] nix-daemon[868]: reaped child process 933, status = succeeded4581worker2 # [ 38.886051] nix-daemon[868]: accepted connection from pid 935, user nix-grpc-daemon (trusted)4582worker2 # [ 38.898456] nix-daemon[868]: reaped child process 938, status = succeeded4583lb # [ 38.936920] niks3-server[1058]: 2026/09/20 13:34:02 INFO Received uploads request method=POST path=/api/pending_closures4584worker2 # [ 38.923976] nix-grpc-daemon[880]: time=2026-09-20T13:34:02.258Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4585worker2 # [ 38.926473] nix-grpc-daemon[880]: time=2026-09-20T13:34:02.260Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4586lb # [ 38.967616] niks3-server[1058]: 2026/09/20 13:34:02 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4587lb # [ 38.984390] niks3-server[1058]: 2026/09/20 13:34:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4588lb # [ 38.989781] niks3-server[1058]: 2026/09/20 13:34:02 INFO Signed narinfos id=3 count=14589worker2 # [ 38.967785] nix-grpc-daemon[880]: time=2026-09-20T13:34:02.302Z level=INFO msg="Uploading 1 narinfos"4590lb # [ 38.996381] niks3-server[1058]: 2026/09/20 13:34:02 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4591lb # [ 39.004218] niks3-server[1058]: 2026/09/20 13:34:02 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4592lb # [ 39.011840] niks3-server[1058]: 2026/09/20 13:34:02 INFO Completed upload id=34593worker2 # [ 38.988568] nix-grpc-daemon[880]: time=2026-09-20T13:34:02.323Z level=INFO msg="Upload complete. (438ms)"4594lb # [ 39.014869] niks3-server[1058]: 2026/09/20 13:34:02 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4595worker2 # [ 38.990947] nix-grpc-daemon[787]: ts=2026-09-20T13:34:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45080 duration_s=0 paths=1 nar_bytes_in=6244596worker2 # [ 38.994143] nix-grpc-daemon[787]: ts=2026-09-20T13:34:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:450804597worker2 # [ 39.000482] nix-daemon[868]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4598worker2 # [ 39.009283] nix-daemon[868]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4599client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4600client # farm-b> worker2: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4601worker2 # [ 39.011669] nix-daemon[868]: reaped child process 928, status = succeeded4602client # farm-b> LOG-farm-b-t14603worker2 # [ 40.624887] nix-daemon[868]: reaped child process 942, status = succeeded4604worker2 # [ 40.761915] nix-daemon[868]: accepted connection from pid 958, user nix-grpc-daemon (trusted)4605worker2 # [ 40.772624] nix-daemon[868]: reaped child process 961, status = succeeded4606lb # [ 40.813704] niks3-server[1058]: 2026/09/20 13:34:04 INFO Received uploads request method=POST path=/api/pending_closures4607worker2 # [ 40.934439] nix-grpc-daemon[880]: time=2026-09-20T13:34:04.268Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4608worker2 # [ 40.940598] nix-grpc-daemon[880]: time=2026-09-20T13:34:04.272Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4609lb # [ 40.979102] niks3-server[1058]: 2026/09/20 13:34:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4610lb # [ 40.984157] niks3-server[1058]: 2026/09/20 13:34:04 INFO Signed narinfos id=4 count=14611worker2 # [ 40.959768] nix-grpc-daemon[880]: time=2026-09-20T13:34:04.293Z level=INFO msg="Uploading 1 narinfos"4612lb # [ 40.988632] niks3-server[1058]: 2026/09/20 13:34:04 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4613lb # [ 40.993676] niks3-server[1058]: 2026/09/20 13:34:04 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4614lb # [ 40.996479] niks3-server[1058]: 2026/09/20 13:34:04 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4615lb # [ 40.999864] niks3-server[1058]: 2026/09/20 13:34:04 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4616lb # [ 41.006294] niks3-server[1058]: 2026/09/20 13:34:04 INFO Completed upload id=44617worker2 # [ 40.981499] nix-grpc-daemon[880]: time=2026-09-20T13:34:04.316Z level=INFO msg="Upload complete. (362ms)"4618lb # [ 41.008716] niks3-server[1058]: 2026/09/20 13:34:04 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4619worker2 # [ 40.989662] nix-grpc-daemon[787]: ts=2026-09-20T13:34:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45080 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14620worker2 # [ 40.999549] nix-grpc-daemon[787]: ts=2026-09-20T13:34:04Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04621worker2 # [ 41.004360] nix-grpc-daemon[787]: ts=2026-09-20T13:34:04Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.4:50051 clients=14622worker2 # [ 41.015274] nix-daemon[868]: reaped child process 940, status = succeeded4623worker2 # [ 41.044237] nix-grpc-daemon[787]: ts=2026-09-20T13:34:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:450804624worker2 # [ 41.055306] nix-grpc-daemon[787]: ts=2026-09-20T13:34:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:450804625worker2 # [ 41.067059] nix-grpc-daemon[880]: time=2026-09-20T13:34:04.401Z level=INFO msg="All 2 paths already cached"4626client # copying 1 paths to 'lb:50051'...4627worker2 # [ 41.075085] nix-grpc-daemon[787]: ts=2026-09-20T13:34:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:450804628client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4629worker2 # [ 41.095164] nix-daemon[868]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4630worker2 # [ 41.263580] nix-daemon[868]: accepted connection from pid 967, user nix-grpc-daemon (trusted)4631worker2 # [ 41.275708] nix-daemon[868]: reaped child process 970, status = succeeded4632worker2 # [ 41.433357] nix-daemon[868]: accepted connection from pid 972, user nix-grpc-daemon (trusted)4633worker2 # [ 41.444983] nix-daemon[868]: reaped child process 975, status = succeeded4634lb # [ 41.485661] niks3-server[1058]: 2026/09/20 13:34:04 INFO Received uploads request method=POST path=/api/pending_closures4635worker2 # [ 41.473021] nix-grpc-daemon[880]: time=2026-09-20T13:34:04.807Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4636worker2 # [ 41.474579] nix-grpc-daemon[880]: time=2026-09-20T13:34:04.809Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4637lb # [ 41.526501] niks3-server[1058]: 2026/09/20 13:34:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4638lb # [ 41.531553] niks3-server[1058]: 2026/09/20 13:34:04 INFO Signed narinfos id=5 count=14639worker2 # [ 41.509462] nix-grpc-daemon[880]: time=2026-09-20T13:34:04.844Z level=INFO msg="Uploading 1 narinfos"4640lb # [ 41.540187] niks3-server[1058]: 2026/09/20 13:34:04 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4641lb # [ 41.545170] niks3-server[1058]: 2026/09/20 13:34:04 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4642lb # [ 41.550386] niks3-server[1058]: 2026/09/20 13:34:04 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4643lb # [ 41.557147] niks3-server[1058]: 2026/09/20 13:34:04 INFO Completed upload id=54644worker2 # [ 41.532333] nix-grpc-daemon[880]: time=2026-09-20T13:34:04.866Z level=INFO msg="Upload complete. (424ms)"4645lb # [ 41.558541] niks3-server[1058]: 2026/09/20 13:34:04 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4646worker2 # [ 41.536053] nix-grpc-daemon[787]: ts=2026-09-20T13:34:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45080 duration_s=0 paths=1 nar_bytes_in=7524647worker2 # [ 41.542086] nix-grpc-daemon[787]: ts=2026-09-20T13:34:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:450804648worker2 # [ 41.549500] nix-daemon[868]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4649worker2 # [ 41.559112] nix-daemon[868]: reaped child process 965, status = succeeded4650client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4651worker2 # [ 41.562940] nix-daemon[868]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4652client # farm-top> worker2: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4653client # farm-top> LOG-farm-top-t14654worker2 # [ 42.587268] nix-daemon[868]: reaped child process 979, status = succeeded4655worker2 # [ 42.735713] nix-daemon[868]: accepted connection from pid 994, user nix-grpc-daemon (trusted)4656worker2 # [ 42.748818] nix-daemon[868]: reaped child process 997, status = succeeded4657lb # [ 42.793620] niks3-server[1058]: 2026/09/20 13:34:06 INFO Received uploads request method=POST path=/api/pending_closures4658worker2 # [ 42.785153] nix-grpc-daemon[880]: time=2026-09-20T13:34:06.118Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4659worker2 # [ 42.787622] nix-grpc-daemon[880]: time=2026-09-20T13:34:06.118Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4660lb # [ 42.837208] niks3-server[1058]: 2026/09/20 13:34:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4661worker2 # [ 42.822966] nix-grpc-daemon[880]: time=2026-09-20T13:34:06.157Z level=INFO msg="Uploading 1 narinfos"4662lb # [ 42.848208] niks3-server[1058]: 2026/09/20 13:34:06 INFO Signed narinfos id=6 count=14663lb # [ 42.854894] niks3-server[1058]: 2026/09/20 13:34:06 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4664lb # [ 42.865228] niks3-server[1058]: 2026/09/20 13:34:06 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4665lb # [ 42.869336] niks3-server[1058]: 2026/09/20 13:34:06 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4666lb # [ 42.876714] niks3-server[1058]: 2026/09/20 13:34:06 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4667lb # [ 42.884453] niks3-server[1058]: 2026/09/20 13:34:06 INFO Completed upload id=64668worker2 # [ 42.859511] nix-grpc-daemon[880]: time=2026-09-20T13:34:06.194Z level=INFO msg="Upload complete. (290ms)"4669worker2 # [ 42.862668] nix-grpc-daemon[787]: ts=2026-09-20T13:34:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45080 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14670worker2 # [ 42.869202] nix-daemon[868]: reaped child process 977, status = succeeded4671lb # [ 42.892435] niks3-server[1058]: 2026/09/20 13:34:06 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4672client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 7.88 seconds)4673client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4674client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.17 seconds)4675client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4676client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.01 seconds)4677client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4678client # copying 3 paths...4679client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4680client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4681client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4682client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.22 seconds)4683(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 8.35 seconds)4684subtest: repeat is answered Cached by the scheduler without building4685worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4686worker1 # finding garbage collector roots...4687worker1 # deleting specified paths...4688worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4689worker1 # deleting unused links...4690worker1 # note: hard linking is currently saving 0.0 KiB4691worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.23 seconds)4692worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4693worker2 # finding garbage collector roots...4694worker2 # deleting specified paths...4695worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4696worker2 # deleting unused links...4697worker2 # note: hard linking is currently saving 0.0 KiB4698worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.18 seconds)4699client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24700client # don't know how to build these paths:4701client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4702worker1 # [ 43.978120] nix-grpc-daemon[780]: ts=2026-09-20T13:34:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:374584703worker1 # [ 43.983476] nix-grpc-daemon[780]: ts=2026-09-20T13:34:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374584704worker2 # [ 43.975798] nix-grpc-daemon[787]: ts=2026-09-20T13:34:07Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14705client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 0.22 seconds)4706client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4707client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.16 seconds)4708lb: must succeed: curl -sf localhost:9901/clusters4709lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4710lb: must succeed: curl -sf localhost:9901/clusters4711lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4712worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4713worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4714(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.03 seconds)4715subtest: inputs already in the cache are not sent to the scheduler4716lb: must succeed: curl -sf localhost:9901/clusters4717lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4718lb: must succeed: curl -sf localhost:9901/clusters4719lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4720worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4721worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4722client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&24723worker2 # [ 44.695462] nix-grpc-daemon[787]: ts=2026-09-20T13:34:08Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:450804724client # don't know how to build these paths:4725client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4726worker2 # [ 44.703195] nix-grpc-daemon[787]: ts=2026-09-20T13:34:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:450804727worker2 # [ 44.710648] nix-grpc-daemon[880]: time=2026-09-20T13:34:08.044Z level=INFO msg="All 2 paths already cached"4728worker2 # [ 44.721157] nix-grpc-daemon[787]: ts=2026-09-20T13:34:08Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04729worker2 # [ 44.726138] nix-grpc-daemon[787]: ts=2026-09-20T13:34:08Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.4:50051 clients=14730worker2 # [ 44.768703] nix-grpc-daemon[787]: ts=2026-09-20T13:34:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:450804731worker2 # [ 44.778740] nix-grpc-daemon[787]: ts=2026-09-20T13:34:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:450804732worker2 # [ 44.788419] nix-grpc-daemon[880]: time=2026-09-20T13:34:08.122Z level=INFO msg="All 2 paths already cached"4733client # copying 1 paths to 'lb:50051'...4734client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4735worker2 # [ 44.795882] nix-grpc-daemon[787]: ts=2026-09-20T13:34:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:450804736worker2 # [ 44.807359] nix-daemon[868]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4737worker2 # [ 44.961083] nix-daemon[868]: accepted connection from pid 1027, user nix-grpc-daemon (trusted)4738worker2 # [ 44.973888] nix-daemon[868]: reaped child process 1030, status = succeeded4739worker2 # [ 45.113942] nix-daemon[868]: accepted connection from pid 1032, user nix-grpc-daemon (trusted)4740worker2 # [ 45.125568] nix-daemon[868]: reaped child process 1035, status = succeeded4741lb # [ 45.167129] niks3-server[1058]: 2026/09/20 13:34:08 INFO Received uploads request method=POST path=/api/pending_closures4742worker2 # [ 45.149803] nix-grpc-daemon[880]: time=2026-09-20T13:34:08.484Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4743worker2 # [ 45.151605] nix-grpc-daemon[880]: time=2026-09-20T13:34:08.485Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4744lb # [ 45.194210] niks3-server[1058]: 2026/09/20 13:34:08 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4745lb # [ 45.206446] niks3-server[1058]: 2026/09/20 13:34:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4746worker2 # [ 45.187460] nix-grpc-daemon[880]: time=2026-09-20T13:34:08.521Z level=INFO msg="Uploading 1 narinfos"4747lb # [ 45.212595] niks3-server[1058]: 2026/09/20 13:34:08 INFO Signed narinfos id=7 count=14748lb # [ 45.218059] niks3-server[1058]: 2026/09/20 13:34:08 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4749lb # [ 45.228719] niks3-server[1058]: 2026/09/20 13:34:08 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4750worker2 # [ 45.210301] nix-grpc-daemon[880]: time=2026-09-20T13:34:08.544Z level=INFO msg="Upload complete. (391ms)"4751lb # [ 45.235564] niks3-server[1058]: 2026/09/20 13:34:08 INFO Completed upload id=74752lb # [ 45.238893] niks3-server[1058]: 2026/09/20 13:34:08 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4753worker2 # [ 45.212562] nix-grpc-daemon[787]: ts=2026-09-20T13:34:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45080 duration_s=0 paths=1 nar_bytes_in=7524754worker2 # [ 45.217155] nix-grpc-daemon[787]: ts=2026-09-20T13:34:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:450804755worker2 # [ 45.226069] nix-daemon[868]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4756worker2 # [ 45.230121] nix-daemon[868]: reaped child process 1025, status = succeeded4757worker2 # [ 45.234714] nix-daemon[868]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4758client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4759client # farm-top> worker2: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4760lb # [ 45.894811] envoy[891]: conn peer=[2001:db8:1::1]:50856 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9537 tx=3844 ms=76954761lb # [ 45.898190] envoy[891]: conn peer=[2001:db8:1::1]:40122 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=414762client # farm-top> LOG-farm-top-t1b4763worker2 # [ 46.459352] nix-daemon[868]: reaped child process 1039, status = succeeded4764worker2 # [ 46.602767] nix-daemon[868]: accepted connection from pid 1054, user nix-grpc-daemon (trusted)4765worker2 # [ 46.615059] nix-daemon[868]: reaped child process 1057, status = succeeded4766lb # [ 46.652127] niks3-server[1058]: 2026/09/20 13:34:09 INFO Received uploads request method=POST path=/api/pending_closures4767worker2 # [ 46.634332] nix-grpc-daemon[880]: time=2026-09-20T13:34:09.968Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4768worker2 # [ 46.635823] nix-grpc-daemon[880]: time=2026-09-20T13:34:09.970Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4769lb # [ 46.684872] niks3-server[1058]: 2026/09/20 13:34:10 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4770lb # [ 46.697574] niks3-server[1058]: 2026/09/20 13:34:10 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4771lb # [ 46.709329] niks3-server[1058]: 2026/09/20 13:34:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4772lb # [ 46.713096] niks3-server[1058]: 2026/09/20 13:34:10 INFO Signed narinfos id=8 count=14773worker2 # [ 46.690928] nix-grpc-daemon[880]: time=2026-09-20T13:34:10.025Z level=INFO msg="Uploading 1 narinfos"4774lb # [ 46.718057] niks3-server[1058]: 2026/09/20 13:34:10 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4775lb # [ 46.727848] niks3-server[1058]: 2026/09/20 13:34:10 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4776lb # [ 46.732674] niks3-server[1058]: 2026/09/20 13:34:10 INFO Completed upload id=84777worker2 # [ 46.707830] nix-grpc-daemon[880]: time=2026-09-20T13:34:10.042Z level=INFO msg="Upload complete. (265ms)"4778lb # [ 46.735063] niks3-server[1058]: 2026/09/20 13:34:10 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4779worker2 # [ 46.715101] nix-grpc-daemon[787]: ts=2026-09-20T13:34:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45080 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14780worker2 # [ 46.724705] nix-daemon[868]: reaped child process 1037, status = succeeded4781client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.21 seconds)4782client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath4783client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.19 seconds)4784lb: must succeed: curl -sf localhost:9901/clusters4785lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4786lb: must succeed: curl -sf localhost:9901/clusters4787lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4788worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4789worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4790(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.70 seconds)4791subtest: two clients wanting the same drv build it once4792client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4793client # [ 47.198758] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4794client # Running as unit: dedup1.service; invocation ID: 49523d8e75c64483bce38a22426ec4f94795client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 0.07 seconds)4796client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4797worker1 # [ 47.381500] nix-grpc-daemon[780]: ts=2026-09-20T13:34:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:374584798client # [ 47.405582] nix[887]: don't know how to build these paths:4799client # [ 47.406494] nix[887]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4800worker2 # [ 47.361955] nix-grpc-daemon[787]: ts=2026-09-20T13:34:10Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04801worker2 # [ 47.364266] nix-grpc-daemon[787]: ts=2026-09-20T13:34:10Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.3:50051 clients=14802worker1 # [ 47.396976] nix-grpc-daemon[780]: ts=2026-09-20T13:34:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374584803worker1 # [ 47.412715] nix-grpc-daemon[780]: ts=2026-09-20T13:34:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374584804client # [ 47.440072] nix[887]: copying 1 paths to 'lb:50051'...4805client # [ 47.440851] nix[887]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4806worker1 # [ 47.423738] nix-grpc-daemon[780]: ts=2026-09-20T13:34:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:374584807worker1 # [ 47.436368] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4808worker1 # [ 47.459438] nix-grpc-daemon[780]: ts=2026-09-20T13:34:10Z level=info event=niks3_push_started pid=8704809worker1 # [ 47.714267] nix-grpc-daemon[870]: time=2026-09-20T13:34:10.924Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""4810worker1 # [ 47.902928] nix-daemon[845]: accepted connection from pid 882, user nix-grpc-daemon (trusted)4811worker1 # [ 47.917753] nix-daemon[845]: reaped child process 885, status = succeeded4812worker1 # [ 48.047620] nix-daemon[845]: accepted connection from pid 887, user nix-grpc-daemon (trusted)4813worker1 # [ 48.060469] nix-daemon[845]: reaped child process 890, status = succeeded4814lb # [ 48.067286] niks3-server[1058]: 2026/09/20 13:34:11 INFO Received uploads request method=POST path=/api/pending_closures4815worker1 # [ 48.104066] nix-grpc-daemon[870]: time=2026-09-20T13:34:11.317Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4816worker1 # [ 48.107062] nix-grpc-daemon[870]: time=2026-09-20T13:34:11.320Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4817lb # [ 48.127863] niks3-server[1058]: 2026/09/20 13:34:11 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4818lb # [ 48.138283] niks3-server[1058]: 2026/09/20 13:34:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4819lb # [ 48.142793] niks3-server[1058]: 2026/09/20 13:34:11 INFO Signed narinfos id=9 count=14820worker1 # [ 48.152479] nix-grpc-daemon[870]: time=2026-09-20T13:34:11.366Z level=INFO msg="Uploading 1 narinfos"4821lb # [ 48.155838] niks3-server[1058]: 2026/09/20 13:34:11 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4822lb # [ 48.161631] niks3-server[1058]: 2026/09/20 13:34:11 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4823lb # [ 48.170894] niks3-server[1058]: 2026/09/20 13:34:11 INFO Completed upload id=94824worker1 # [ 48.177518] nix-grpc-daemon[870]: time=2026-09-20T13:34:11.391Z level=INFO msg="Upload complete. (445ms)"4825lb # [ 48.172570] niks3-server[1058]: 2026/09/20 13:34:11 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4826worker1 # [ 48.181542] nix-grpc-daemon[780]: ts=2026-09-20T13:34:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37458 duration_s=0 paths=1 nar_bytes_in=4324827worker1 # [ 48.187802] nix-grpc-daemon[780]: ts=2026-09-20T13:34:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374584828worker1 # [ 48.196196] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4829worker1 # [ 48.203184] nix-daemon[845]: reaped child process 868, status = succeeded4830worker1 # [ 48.208111] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4831client # [ 48.225473] nix[887]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4832client # [ 48.275973] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4833client # Running as unit: dedup2.service; invocation ID: 2ac0fed3c1a84d4eb72380d8e81560664834client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 1.08 seconds)4835worker1 # [ 48.494830] nix-grpc-daemon[780]: ts=2026-09-20T13:34:11Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:374584836client # [ 48.523957] nix[903]: this derivation will be built:4837worker2 # [ 48.476513] nix-grpc-daemon[787]: ts=2026-09-20T13:34:11Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:450804838worker2 # [ 48.483277] nix-grpc-daemon[787]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' from 'http://lb:5751'...4839client # [ 48.546467] nix[903]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4840worker2 # [ 48.501743] nix-grpc-daemon[787]: ts=2026-09-20T13:34:11Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04841worker1 # [ 48.540511] nix-grpc-daemon[780]: ts=2026-09-20T13:34:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374584842client # [ 48.563260] nix[903]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4843client: must succeed: sleep 24844client: (finished: must succeed: sleep 2, in 2.05 seconds)4845client: waiting for success: ! systemctl is-active dedup1 dedup24846worker1 # [ 51.825287] nix-daemon[845]: reaped child process 894, status = succeeded4847worker1 # [ 51.991456] nix-daemon[845]: accepted connection from pid 923, user nix-grpc-daemon (trusted)4848worker1 # [ 52.006962] nix-daemon[845]: reaped child process 926, status = succeeded4849lb # [ 52.010555] niks3-server[1058]: 2026/09/20 13:34:15 INFO Received uploads request method=POST path=/api/pending_closures4850worker1 # [ 52.022327] nix-grpc-daemon[870]: time=2026-09-20T13:34:15.235Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4851worker1 # [ 52.023869] nix-grpc-daemon[870]: time=2026-09-20T13:34:15.237Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4852lb # [ 52.037771] niks3-server[1058]: 2026/09/20 13:34:15 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4853lb # [ 52.042231] niks3-server[1058]: 2026/09/20 13:34:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4854lb # [ 52.045039] niks3-server[1058]: 2026/09/20 13:34:15 INFO Signed narinfos id=10 count=14855worker1 # [ 52.052096] nix-grpc-daemon[870]: time=2026-09-20T13:34:15.265Z level=INFO msg="Uploading 1 narinfos"4856lb # [ 52.049800] niks3-server[1058]: 2026/09/20 13:34:15 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4857lb # [ 52.052301] niks3-server[1058]: 2026/09/20 13:34:15 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4858lb # [ 52.056974] niks3-server[1058]: 2026/09/20 13:34:15 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4859lb # [ 52.060324] niks3-server[1058]: 2026/09/20 13:34:15 INFO Completed upload id=104860worker1 # [ 52.068187] nix-grpc-daemon[870]: time=2026-09-20T13:34:15.281Z level=INFO msg="Upload complete. (248ms)"4861lb # [ 52.061893] niks3-server[1058]: 2026/09/20 13:34:15 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4862worker1 # [ 52.071743] nix-grpc-daemon[780]: ts=2026-09-20T13:34:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37458 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14863worker1 # [ 52.075172] nix-grpc-daemon[780]: ts=2026-09-20T13:34:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37458 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14864worker1 # [ 52.081737] nix-daemon[845]: reaped child process 892, status = succeeded4865client # [ 52.137452] systemd[1]: dedup1.service: Deactivated successfully.4866client # [ 52.141309] systemd[1]: dedup1.service: Consumed 104ms CPU time over 4.941s wall clock time, 37.3M memory peak, 8.5K incoming IP traffic, 8.9K outgoing IP traffic.4867client # [ 52.147837] systemd[1]: dedup2.service: Deactivated successfully.4868client # [ 52.152109] systemd[1]: dedup2.service: Consumed 124ms CPU time over 3.875s wall clock time, 31.1M memory peak, 8K incoming IP traffic, 6.7K outgoing IP traffic.4869client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.12 seconds)4870client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4871client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)4872worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4873worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4874worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4875worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.03 seconds)4876worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4877worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4878worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4879worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4880(finished: subtest: two clients wanting the same drv build it once, in 5.97 seconds)4881subtest: build hook: nix-daemon with builders = grpc://lb4882client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log4883client # [ 53.323031] systemd[1]: Started Nix Daemon.4884client # [ 53.499350] nix-daemon[945]: accepted connection from pid 942, user root (trusted)4885worker1 # [ 53.673475] nix-grpc-daemon[780]: ts=2026-09-20T13:34:16Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:374584886worker2 # [ 53.651631] nix-grpc-daemon[787]: ts=2026-09-20T13:34:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:450804887worker2 # [ 53.664767] nix-grpc-daemon[787]: ts=2026-09-20T13:34:16Z level=debug event=want client=6 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=04888worker2 # [ 53.669956] nix-grpc-daemon[787]: ts=2026-09-20T13:34:16Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.4:50051 clients=14889worker2 # [ 53.709148] nix-grpc-daemon[787]: ts=2026-09-20T13:34:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:450804890worker2 # [ 53.723221] nix-grpc-daemon[787]: ts=2026-09-20T13:34:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:450804891worker2 # [ 53.734605] nix-grpc-daemon[787]: ts=2026-09-20T13:34:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:450804892worker2 # [ 53.755520] nix-daemon[868]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4893worker2 # [ 53.953107] nix-daemon[868]: accepted connection from pid 1094, user nix-grpc-daemon (trusted)4894worker2 # [ 53.967381] nix-daemon[868]: reaped child process 1097, status = succeeded4895worker2 # [ 54.129273] nix-daemon[868]: accepted connection from pid 1099, user nix-grpc-daemon (trusted)4896worker2 # [ 54.140885] nix-daemon[868]: reaped child process 1102, status = succeeded4897lb # [ 54.183767] niks3-server[1058]: 2026/09/20 13:34:17 INFO Received uploads request method=POST path=/api/pending_closures4898worker2 # [ 54.169479] nix-grpc-daemon[880]: time=2026-09-20T13:34:17.503Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4899worker2 # [ 54.170956] nix-grpc-daemon[880]: time=2026-09-20T13:34:17.503Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"4900lb # [ 54.213684] niks3-server[1058]: 2026/09/20 13:34:17 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst4901lb # [ 54.225842] niks3-server[1058]: 2026/09/20 13:34:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4902worker2 # [ 54.206912] nix-grpc-daemon[880]: time=2026-09-20T13:34:17.541Z level=INFO msg="Uploading 1 narinfos"4903lb # [ 54.232056] niks3-server[1058]: 2026/09/20 13:34:17 INFO Signed narinfos id=11 count=14904lb # [ 54.241164] niks3-server[1058]: 2026/09/20 13:34:17 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls4905lb # [ 54.250096] niks3-server[1058]: 2026/09/20 13:34:17 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4906lb # [ 54.255883] niks3-server[1058]: 2026/09/20 13:34:17 INFO Completed upload id=114907lb # [ 54.259969] niks3-server[1058]: 2026/09/20 13:34:17 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo4908worker2 # [ 54.234203] nix-grpc-daemon[880]: time=2026-09-20T13:34:17.568Z level=INFO msg="Upload complete. (465ms)"4909worker2 # [ 54.238130] nix-grpc-daemon[787]: ts=2026-09-20T13:34:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45080 duration_s=0 paths=1 nar_bytes_in=5124910worker2 # [ 54.242780] nix-grpc-daemon[787]: ts=2026-09-20T13:34:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:450804911worker2 # [ 54.253262] nix-daemon[868]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4912worker2 # [ 54.260412] nix-daemon[868]: reaped child process 1092, status = succeeded4913worker2 # [ 54.264758] nix-daemon[868]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4914worker2 # [ 55.486885] nix-daemon[868]: reaped child process 1106, status = succeeded4915worker2 # [ 55.647467] nix-daemon[868]: accepted connection from pid 1121, user nix-grpc-daemon (trusted)4916worker2 # [ 55.658188] nix-daemon[868]: reaped child process 1124, status = succeeded4917lb # [ 55.700606] niks3-server[1058]: 2026/09/20 13:34:19 INFO Received uploads request method=POST path=/api/pending_closures4918worker2 # [ 55.683963] nix-grpc-daemon[880]: time=2026-09-20T13:34:19.018Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4919worker2 # [ 55.685488] nix-grpc-daemon[880]: time=2026-09-20T13:34:19.020Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"4920lb # [ 55.738308] niks3-server[1058]: 2026/09/20 13:34:19 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv4921lb # [ 55.749509] niks3-server[1058]: 2026/09/20 13:34:19 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4922lb # [ 55.758742] niks3-server[1058]: 2026/09/20 13:34:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4923lb # [ 55.763125] niks3-server[1058]: 2026/09/20 13:34:19 INFO Signed narinfos id=12 count=14924worker2 # [ 55.741400] nix-grpc-daemon[880]: time=2026-09-20T13:34:19.075Z level=INFO msg="Uploading 1 narinfos"4925lb # [ 55.768595] niks3-server[1058]: 2026/09/20 13:34:19 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls4926lb # [ 55.778097] niks3-server[1058]: 2026/09/20 13:34:19 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4927lb # [ 55.782630] niks3-server[1058]: 2026/09/20 13:34:19 INFO Completed upload id=124928lb # [ 55.785084] niks3-server[1058]: 2026/09/20 13:34:19 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo4929worker2 # [ 55.759066] nix-grpc-daemon[880]: time=2026-09-20T13:34:19.093Z level=INFO msg="Upload complete. (287ms)"4930worker2 # [ 55.766222] nix-grpc-daemon[787]: ts=2026-09-20T13:34:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45080 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=6 outputs=14931worker2 # [ 55.771633] nix-grpc-daemon[787]: ts=2026-09-20T13:34:19Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:450804932worker2 # [ 55.780217] nix-daemon[868]: reaped child process 1104, status = succeeded4933worker2 # [ 55.784594] nix-grpc-daemon[787]: ts=2026-09-20T13:34:19Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:450804934worker2 # [ 55.788767] nix-grpc-daemon[787]: ts=2026-09-20T13:34:19Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45080 duration_s=0 paths=1 nar_bytes_out=1284935lb # [ 55.895134] envoy[891]: conn peer=[2001:db8:1::1]:40132 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4139 tx=1712 ms=20294936lb # [ 55.898794] envoy[891]: conn peer=[2001:db8:1::1]:40144 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1576 tx=1487 ms=35974937lb # [ 55.902376] envoy[891]: conn peer=[2001:db8:1::1]:40134 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2955 tx=1353 ms=47094938lb # [ 55.905955] envoy[891]: conn peer=[2001:db8:1::1]:39544 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=554939lb # [ 55.909703] envoy[891]: conn peer=[2001:db8:1::1]:39540 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3388 tx=1660 ms=22014940worker2 # [ 56.042216] nix-grpc-daemon[787]: ts=2026-09-20T13:34:19Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:450804941worker1 # [ 56.080392] nix-grpc-daemon[780]: ts=2026-09-20T13:34:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374584942worker2 # [ 56.071221] nix-grpc-daemon[787]: ts=2026-09-20T13:34:19Z level=debug event=want client=7 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=04943worker2 # [ 56.076462] nix-grpc-daemon[787]: ts=2026-09-20T13:34:19Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.4:50051 clients=14944worker2 # [ 56.081857] nix-grpc-daemon[787]: ts=2026-09-20T13:34:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:450804945worker2 # [ 56.093738] nix-grpc-daemon[787]: ts=2026-09-20T13:34:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:450804946worker2 # [ 56.106407] nix-grpc-daemon[880]: time=2026-09-20T13:34:19.440Z level=INFO msg="All 1 paths already cached"4947worker2 # [ 56.113357] nix-grpc-daemon[787]: ts=2026-09-20T13:34:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:450804948worker2 # [ 56.124162] nix-daemon[868]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4949worker2 # [ 56.278068] nix-daemon[868]: accepted connection from pid 1130, user nix-grpc-daemon (trusted)4950worker2 # [ 56.291606] nix-daemon[868]: reaped child process 1133, status = succeeded4951worker2 # [ 56.434247] nix-daemon[868]: accepted connection from pid 1135, user nix-grpc-daemon (trusted)4952worker2 # [ 56.446087] nix-daemon[868]: reaped child process 1138, status = succeeded4953lb # [ 56.487892] niks3-server[1058]: 2026/09/20 13:34:19 INFO Received uploads request method=POST path=/api/pending_closures4954worker2 # [ 56.470823] nix-grpc-daemon[880]: time=2026-09-20T13:34:19.805Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4955worker2 # [ 56.472642] nix-grpc-daemon[880]: time=2026-09-20T13:34:19.806Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"4956lb # [ 56.513422] niks3-server[1058]: 2026/09/20 13:34:19 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst4957lb # [ 56.526833] niks3-server[1058]: 2026/09/20 13:34:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4958lb # [ 56.533424] niks3-server[1058]: 2026/09/20 13:34:19 INFO Signed narinfos id=13 count=14959worker2 # [ 56.511720] nix-grpc-daemon[880]: time=2026-09-20T13:34:19.846Z level=INFO msg="Uploading 1 narinfos"4960lb # [ 56.538403] niks3-server[1058]: 2026/09/20 13:34:19 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls4961lb # [ 56.550706] niks3-server[1058]: 2026/09/20 13:34:19 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4962worker2 # [ 56.531917] nix-grpc-daemon[880]: time=2026-09-20T13:34:19.866Z level=INFO msg="Upload complete. (396ms)"4963lb # [ 56.557048] niks3-server[1058]: 2026/09/20 13:34:19 INFO Completed upload id=134964worker2 # [ 56.533890] nix-grpc-daemon[787]: ts=2026-09-20T13:34:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45080 duration_s=0 paths=1 nar_bytes_in=6244965lb # [ 56.561122] niks3-server[1058]: 2026/09/20 13:34:19 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo4966worker2 # [ 56.540474] nix-grpc-daemon[787]: ts=2026-09-20T13:34:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:450804967worker2 # [ 56.542462] nix-daemon[868]: reaped child process 1128, status = succeeded4968worker2 # [ 56.546337] nix-daemon[868]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4969worker2 # [ 56.551901] nix-daemon[868]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4970worker2 # [ 57.977149] nix-daemon[868]: reaped child process 1142, status = succeeded4971worker2 # [ 58.132185] nix-daemon[868]: accepted connection from pid 1157, user nix-grpc-daemon (trusted)4972worker2 # [ 58.144668] nix-daemon[868]: reaped child process 1160, status = succeeded4973lb # [ 58.181596] niks3-server[1058]: 2026/09/20 13:34:21 INFO Received uploads request method=POST path=/api/pending_closures4974worker2 # [ 58.165058] nix-grpc-daemon[880]: time=2026-09-20T13:34:21.499Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4975worker2 # [ 58.166575] nix-grpc-daemon[880]: time=2026-09-20T13:34:21.501Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"4976lb # [ 58.217442] niks3-server[1058]: 2026/09/20 13:34:21 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv4977lb # [ 58.225799] niks3-server[1058]: 2026/09/20 13:34:21 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst4978lb # [ 58.237765] niks3-server[1058]: 2026/09/20 13:34:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4979lb # [ 58.244242] niks3-server[1058]: 2026/09/20 13:34:21 INFO Signed narinfos id=14 count=14980worker2 # [ 58.221957] nix-grpc-daemon[880]: time=2026-09-20T13:34:21.556Z level=INFO msg="Uploading 1 narinfos"4981lb # [ 58.246979] niks3-server[1058]: 2026/09/20 13:34:21 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls4982lb # [ 58.256818] niks3-server[1058]: 2026/09/20 13:34:21 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4983lb # [ 58.262110] niks3-server[1058]: 2026/09/20 13:34:21 INFO Completed upload id=144984worker2 # [ 58.237248] nix-grpc-daemon[880]: time=2026-09-20T13:34:21.571Z level=INFO msg="Upload complete. (274ms)"4985lb # [ 58.266645] niks3-server[1058]: 2026/09/20 13:34:21 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo4986worker2 # [ 58.243700] nix-grpc-daemon[787]: ts=2026-09-20T13:34:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45080 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=7 outputs=14987worker2 # [ 58.255885] nix-daemon[868]: reaped child process 1140, status = succeeded4988worker1 # [ 58.284845] nix-grpc-daemon[780]: ts=2026-09-20T13:34:21Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:374584989worker1 # [ 58.300618] nix-grpc-daemon[780]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...4990worker1 # [ 58.322348] nix-grpc-daemon[780]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...4991worker1 # [ 58.347883] nix-grpc-daemon[780]: ts=2026-09-20T13:34:21Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:374584992worker1 # [ 58.353199] nix-grpc-daemon[780]: ts=2026-09-20T13:34:21Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37458 duration_s=0 paths=1 nar_bytes_out=1764993worker2 # [ 58.567155] nix-grpc-daemon[787]: ts=2026-09-20T13:34:21Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:450804994worker2 # [ 58.572615] nix-grpc-daemon[787]: ts=2026-09-20T13:34:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:450804995worker2 # [ 58.581889] nix-grpc-daemon[880]: time=2026-09-20T13:34:21.916Z level=INFO msg="All 2 paths already cached"4996worker2 # [ 58.595413] nix-grpc-daemon[787]: ts=2026-09-20T13:34:21Z level=debug event=want client=8 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=04997worker2 # [ 58.600549] nix-grpc-daemon[787]: ts=2026-09-20T13:34:21Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.4:50051 clients=14998worker2 # [ 58.639372] nix-grpc-daemon[787]: ts=2026-09-20T13:34:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:450804999worker2 # [ 58.651271] nix-grpc-daemon[787]: ts=2026-09-20T13:34:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:450805000worker2 # [ 58.664363] nix-grpc-daemon[880]: time=2026-09-20T13:34:21.998Z level=INFO msg="All 2 paths already cached"5001worker2 # [ 58.673131] nix-grpc-daemon[787]: ts=2026-09-20T13:34:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:450805002worker2 # [ 58.683154] nix-daemon[868]: accepted connection from pid 787, user nix-grpc-daemon (trusted)5003worker2 # [ 58.835214] nix-daemon[868]: accepted connection from pid 1166, user nix-grpc-daemon (trusted)5004worker2 # [ 58.849087] nix-daemon[868]: reaped child process 1169, status = succeeded5005worker2 # [ 58.998046] nix-daemon[868]: accepted connection from pid 1171, user nix-grpc-daemon (trusted)5006worker2 # [ 59.009565] nix-daemon[868]: reaped child process 1174, status = succeeded5007lb # [ 59.050380] niks3-server[1058]: 2026/09/20 13:34:22 INFO Received uploads request method=POST path=/api/pending_closures5008worker2 # [ 59.033417] nix-grpc-daemon[880]: time=2026-09-20T13:34:22.368Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5009worker2 # [ 59.035265] nix-grpc-daemon[880]: time=2026-09-20T13:34:22.369Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"5010lb # [ 59.075993] niks3-server[1058]: 2026/09/20 13:34:22 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst5011lb # [ 59.087502] niks3-server[1058]: 2026/09/20 13:34:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign5012lb # [ 59.094046] niks3-server[1058]: 2026/09/20 13:34:22 INFO Signed narinfos id=15 count=15013worker2 # [ 59.072258] nix-grpc-daemon[880]: time=2026-09-20T13:34:22.406Z level=INFO msg="Uploading 1 narinfos"5014lb # [ 59.097292] niks3-server[1058]: 2026/09/20 13:34:22 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls5015lb # [ 59.110535] niks3-server[1058]: 2026/09/20 13:34:22 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete5016lb # [ 59.117749] niks3-server[1058]: 2026/09/20 13:34:22 INFO Completed upload id=155017worker2 # [ 59.092629] nix-grpc-daemon[880]: time=2026-09-20T13:34:22.427Z level=INFO msg="Upload complete. (398ms)"5018lb # [ 59.120407] niks3-server[1058]: 2026/09/20 13:34:22 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo5019worker2 # [ 59.095872] nix-grpc-daemon[787]: ts=2026-09-20T13:34:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45080 duration_s=0 paths=1 nar_bytes_in=7525020worker2 # [ 59.101679] nix-grpc-daemon[787]: ts=2026-09-20T13:34:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:450805021worker2 # [ 59.109413] nix-daemon[868]: accepted connection from pid 787, user nix-grpc-daemon (trusted)5022worker2 # [ 59.114130] nix-daemon[868]: reaped child process 1164, status = succeeded5023worker2 # [ 59.119606] nix-daemon[868]: accepted connection from pid 787, user nix-grpc-daemon (trusted)5024worker2 # [ 60.544821] nix-daemon[868]: reaped child process 1178, status = succeeded5025worker2 # [ 60.679087] nix-daemon[868]: accepted connection from pid 1193, user nix-grpc-daemon (trusted)5026worker2 # [ 60.690052] nix-daemon[868]: reaped child process 1196, status = succeeded5027lb # [ 60.731706] niks3-server[1058]: 2026/09/20 13:34:24 INFO Received uploads request method=POST path=/api/pending_closures5028worker2 # [ 60.715112] nix-grpc-daemon[880]: time=2026-09-20T13:34:24.049Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5029worker2 # [ 60.716565] nix-grpc-daemon[880]: time=2026-09-20T13:34:24.051Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"5030lb # [ 60.769392] niks3-server[1058]: 2026/09/20 13:34:24 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst5031lb # [ 60.779845] niks3-server[1058]: 2026/09/20 13:34:24 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv5032lb # [ 60.791070] niks3-server[1058]: 2026/09/20 13:34:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign5033lb # [ 60.796472] niks3-server[1058]: 2026/09/20 13:34:24 INFO Signed narinfos id=16 count=15034lb # [ 60.799186] niks3-server[1058]: 2026/09/20 13:34:24 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls5035worker2 # [ 60.774546] nix-grpc-daemon[880]: time=2026-09-20T13:34:24.108Z level=INFO msg="Uploading 1 narinfos"5036lb # [ 60.809364] niks3-server[1058]: 2026/09/20 13:34:24 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete5037lb # [ 60.811665] niks3-server[1058]: 2026/09/20 13:34:24 INFO Completed upload id=165038lb # [ 60.814453] niks3-server[1058]: 2026/09/20 13:34:24 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo5039worker2 # [ 60.788085] nix-grpc-daemon[880]: time=2026-09-20T13:34:24.122Z level=INFO msg="Upload complete. (261ms)"5040worker2 # [ 60.795655] nix-grpc-daemon[787]: ts=2026-09-20T13:34:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45080 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=8 outputs=15041worker1 # [ 60.830812] nix-grpc-daemon[780]: ts=2026-09-20T13:34:24Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:374585042worker2 # [ 60.806236] nix-daemon[868]: reaped child process 1176, status = succeeded5043worker1 # [ 60.839711] nix-grpc-daemon[780]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...5044worker2 # [ 60.835148] nix-grpc-daemon[787]: ts=2026-09-20T13:34:24Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:450805045worker2 # [ 60.840059] nix-grpc-daemon[787]: ts=2026-09-20T13:34:24Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45080 duration_s=0 paths=1 nar_bytes_out=2325046client # [ 60.962477] nix-daemon[945]: reaped child process 947, status = succeeded5047client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.86 seconds)5048client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5049client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.03 seconds)5050client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5051client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.03 seconds)5052client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5053client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.02 seconds)5054(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.94 seconds)5055subtest: an input only one worker has still reaches the builder5056client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only5057client # this derivation will be built:5058client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5059client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5060client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only, in 0.39 seconds)5061client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5062client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.17 seconds)5063worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5064worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.23 seconds)5065worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5066worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.03 seconds)5067client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&25068client # [ 62.023653] nix-daemon[945]: accepted connection from pid 1028, user root (trusted)5069client # this derivation will be built:5070client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5071worker2 # [ 62.227656] nix-grpc-daemon[787]: ts=2026-09-20T13:34:25Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:450805072client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...5073worker1 # [ 62.262903] nix-grpc-daemon[780]: ts=2026-09-20T13:34:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374585074worker1 # [ 62.433073] nix-daemon[845]: accepted connection from pid 946, user nix-grpc-daemon (trusted)5075worker1 # [ 62.444400] nix-daemon[845]: reaped child process 949, status = succeeded5076lb # [ 62.455567] niks3-server[1058]: 2026/09/20 13:34:25 INFO Received uploads request method=POST path=/api/pending_closures5077worker1 # [ 62.469116] nix-grpc-daemon[870]: time=2026-09-20T13:34:25.682Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5078worker1 # [ 62.471089] nix-grpc-daemon[870]: time=2026-09-20T13:34:25.684Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5079lb # [ 62.481114] niks3-server[1058]: 2026/09/20 13:34:25 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5080lb # [ 62.491988] niks3-server[1058]: 2026/09/20 13:34:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign5081lb # [ 62.498573] niks3-server[1058]: 2026/09/20 13:34:25 INFO Signed narinfos id=17 count=15082worker1 # [ 62.508525] nix-grpc-daemon[870]: time=2026-09-20T13:34:25.722Z level=INFO msg="Uploading 1 narinfos"5083lb # [ 62.502158] niks3-server[1058]: 2026/09/20 13:34:25 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5084lb # [ 62.515282] niks3-server[1058]: 2026/09/20 13:34:25 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete5085worker1 # [ 62.528143] nix-grpc-daemon[870]: time=2026-09-20T13:34:25.741Z level=INFO msg="Upload complete. (259ms)"5086lb # [ 62.521682] niks3-server[1058]: 2026/09/20 13:34:25 INFO Completed upload id=175087lb # [ 62.525094] niks3-server[1058]: 2026/09/20 13:34:25 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5088worker2 # [ 62.508510] nix-grpc-daemon[787]: ts=2026-09-20T13:34:25Z level=debug event=want client=9 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05089worker2 # [ 62.510898] nix-grpc-daemon[787]: ts=2026-09-20T13:34:25Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=15090worker1 # [ 62.583816] nix-grpc-daemon[780]: ts=2026-09-20T13:34:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374585091worker1 # [ 62.595783] nix-grpc-daemon[780]: ts=2026-09-20T13:34:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374585092client # copying 2 paths to 'lb:50051'...5093client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5094worker1 # [ 62.610792] nix-grpc-daemon[780]: ts=2026-09-20T13:34:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:374585095client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5096worker1 # [ 62.621155] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5097worker1 # [ 62.802951] nix-daemon[845]: accepted connection from pid 953, user nix-grpc-daemon (trusted)5098worker1 # [ 62.815814] nix-daemon[845]: reaped child process 956, status = succeeded5099worker1 # [ 62.986303] nix-daemon[845]: accepted connection from pid 958, user nix-grpc-daemon (trusted)5100worker1 # [ 62.997574] nix-daemon[845]: reaped child process 961, status = succeeded5101lb # [ 63.008641] niks3-server[1058]: 2026/09/20 13:34:26 INFO Received uploads request method=POST path=/api/pending_closures5102lb # [ 63.016811] niks3-server[1058]: 2026/09/20 13:34:26 INFO Received uploads request method=POST path=/api/pending_closures5103worker1 # [ 63.030423] nix-grpc-daemon[870]: time=2026-09-20T13:34:26.244Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5104worker1 # [ 63.031956] nix-grpc-daemon[870]: time=2026-09-20T13:34:26.245Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5105worker1 # [ 63.034485] nix-grpc-daemon[870]: time=2026-09-20T13:34:26.247Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5106lb # [ 63.049671] niks3-server[1058]: 2026/09/20 13:34:26 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5107lb # [ 63.067118] niks3-server[1058]: 2026/09/20 13:34:26 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5108lb # [ 63.074258] niks3-server[1058]: 2026/09/20 13:34:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5109lb # [ 63.077848] niks3-server[1058]: 2026/09/20 13:34:26 INFO Signed narinfos id=18 count=15110lb # [ 63.082660] niks3-server[1058]: 2026/09/20 13:34:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5111lb # [ 63.086920] niks3-server[1058]: 2026/09/20 13:34:26 INFO Signed narinfos id=19 count=25112lb # [ 63.088865] niks3-server[1058]: 2026/09/20 13:34:26 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5113worker1 # [ 63.096717] nix-grpc-daemon[870]: time=2026-09-20T13:34:26.309Z level=INFO msg="Uploading 3 narinfos"5114lb # [ 63.092160] niks3-server[1058]: 2026/09/20 13:34:26 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5115lb # [ 63.103527] niks3-server[1058]: 2026/09/20 13:34:26 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5116lb # [ 63.106833] niks3-server[1058]: 2026/09/20 13:34:26 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5117lb # [ 63.109401] niks3-server[1058]: 2026/09/20 13:34:26 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5118lb # [ 63.111669] niks3-server[1058]: 2026/09/20 13:34:26 INFO Completed upload id=185119lb # [ 63.113636] niks3-server[1058]: 2026/09/20 13:34:26 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5120lb # [ 63.116043] niks3-server[1058]: 2026/09/20 13:34:26 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5121lb # [ 63.118119] niks3-server[1058]: 2026/09/20 13:34:26 INFO Completed upload id=195122worker1 # [ 63.126227] nix-grpc-daemon[870]: time=2026-09-20T13:34:26.339Z level=INFO msg="Upload complete. (490ms)"5123worker1 # [ 63.131150] nix-grpc-daemon[780]: ts=2026-09-20T13:34:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37458 duration_s=0 paths=2 nar_bytes_in=9765124worker1 # [ 63.136748] nix-grpc-daemon[780]: ts=2026-09-20T13:34:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374585125worker1 # [ 63.147094] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5126worker1 # [ 63.155299] nix-daemon[845]: reaped child process 951, status = succeeded5127client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5128client # uses-input-w1onlya> node-a: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5129worker1 # [ 63.164520] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5130worker1 # [ 64.179372] nix-daemon[845]: reaped child process 965, status = succeeded5131worker1 # [ 64.321514] nix-daemon[845]: accepted connection from pid 980, user nix-grpc-daemon (trusted)5132worker1 # [ 64.333141] nix-daemon[845]: reaped child process 983, status = succeeded5133lb # [ 64.341216] niks3-server[1058]: 2026/09/20 13:34:27 INFO Received uploads request method=POST path=/api/pending_closures5134worker1 # [ 64.639167] nix-grpc-daemon[870]: time=2026-09-20T13:34:27.850Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5135lb # [ 64.653252] niks3-server[1058]: 2026/09/20 13:34:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5136lb # [ 64.660485] niks3-server[1058]: 2026/09/20 13:34:27 INFO Signed narinfos id=20 count=15137worker1 # [ 64.671713] nix-grpc-daemon[870]: time=2026-09-20T13:34:27.885Z level=INFO msg="Uploading 1 narinfos"5138lb # [ 64.667318] niks3-server[1058]: 2026/09/20 13:34:27 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5139lb # [ 64.679365] niks3-server[1058]: 2026/09/20 13:34:27 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5140lb # [ 64.686703] niks3-server[1058]: 2026/09/20 13:34:28 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5141lb # [ 64.699577] niks3-server[1058]: 2026/09/20 13:34:28 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5142lb # [ 64.705315] niks3-server[1058]: 2026/09/20 13:34:28 INFO Completed upload id=205143worker1 # [ 64.714318] nix-grpc-daemon[870]: time=2026-09-20T13:34:27.927Z level=INFO msg="Upload complete. (540ms)"5144worker1 # [ 64.719607] nix-grpc-daemon[780]: ts=2026-09-20T13:34:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37458 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=9 outputs=15145worker1 # [ 64.733344] nix-daemon[845]: reaped child process 963, status = succeeded5146client # copying 1 paths...5147worker2 # [ 64.733480] nix-grpc-daemon[787]: ts=2026-09-20T13:34:28Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:450805148worker2 # [ 64.748387] nix-grpc-daemon[787]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5149client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5150worker2 # [ 64.775152] nix-grpc-daemon[787]: ts=2026-09-20T13:34:28Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:450805151worker2 # [ 64.780060] nix-grpc-daemon[787]: ts=2026-09-20T13:34:28Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45080 duration_s=0 paths=1 nar_bytes_out=1205152client # [ 64.917970] nix-daemon[945]: reaped child process 1031, status = succeeded5153client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&2, in 3.05 seconds)5154client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&25155client # [ 65.083191] nix-daemon[945]: accepted connection from pid 1050, user root (trusted)5156client # this derivation will be built:5157client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5158worker2 # [ 65.231224] nix-grpc-daemon[787]: ts=2026-09-20T13:34:28Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:450805159client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...5160worker1 # [ 65.266767] nix-grpc-daemon[780]: ts=2026-09-20T13:34:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374585161worker1 # [ 65.275138] nix-grpc-daemon[870]: time=2026-09-20T13:34:28.488Z level=INFO msg="All 1 paths already cached"5162worker2 # [ 65.257464] nix-grpc-daemon[787]: ts=2026-09-20T13:34:28Z level=debug event=want client=10 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05163worker2 # [ 65.263308] nix-grpc-daemon[787]: ts=2026-09-20T13:34:28Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=15164worker2 # [ 65.269105] nix-grpc-daemon[787]: ts=2026-09-20T13:34:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:450805165worker2 # [ 65.280318] nix-grpc-daemon[787]: ts=2026-09-20T13:34:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:450805166client # copying 1 paths to 'lb:50051'...5167client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5168worker2 # [ 65.299456] nix-grpc-daemon[787]: ts=2026-09-20T13:34:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:450805169worker2 # [ 65.312407] nix-daemon[868]: accepted connection from pid 787, user nix-grpc-daemon (trusted)5170worker2 # [ 65.329864] nix-grpc-daemon[787]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 5 ms (attempt 1/5)5171worker2 # [ 65.335271] nix-grpc-daemon[787]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 61 ms (attempt 2/5)5172worker2 # [ 65.401734] nix-grpc-daemon[787]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 313 ms (attempt 3/5)5173worker2 # [ 65.721579] nix-grpc-daemon[787]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 297 ms (attempt 4/5)5174lb # [ 65.895517] envoy[891]: conn peer=[2001:db8:1::1]:39556 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3867 tx=1968 ms=23305175lb # [ 65.899299] envoy[891]: conn peer=[2001:db8:1::1]:39570 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=378 ms=545176lb # [ 65.902703] envoy[891]: conn peer=[2001:db8:1::1]:39594 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=411 tx=410 ms=175177lb # [ 65.906278] envoy[891]: conn peer=[2001:db8:1::1]:39584 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4315 tx=2221 ms=22885178lb # [ 65.909780] envoy[891]: conn peer=[2001:db8:1::1]:39598 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3839 tx=1876 ms=26045179lb # [ 65.913349] envoy[891]: conn peer=[2001:db8:1::1]:60740 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=545180worker2 # [ 66.024857] nix-grpc-daemon[787]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5181worker2 # [ 66.027241] nix-grpc-daemon[787]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5182worker2 # [ 66.175089] nix-daemon[868]: accepted connection from pid 1212, user nix-grpc-daemon (trusted)5183worker2 # [ 66.186762] nix-daemon[868]: reaped child process 1215, status = succeeded5184worker2 # [ 66.340812] nix-daemon[868]: accepted connection from pid 1217, user nix-grpc-daemon (trusted)5185worker2 # [ 66.351373] nix-daemon[868]: reaped child process 1220, status = succeeded5186lb # [ 66.390543] niks3-server[1058]: 2026/09/20 13:34:29 INFO Received uploads request method=POST path=/api/pending_closures5187worker2 # [ 66.374102] nix-grpc-daemon[880]: time=2026-09-20T13:34:29.708Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5188worker2 # [ 66.375908] nix-grpc-daemon[880]: time=2026-09-20T13:34:29.710Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5189lb # [ 66.420818] niks3-server[1058]: 2026/09/20 13:34:29 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5190lb # [ 66.429252] niks3-server[1058]: 2026/09/20 13:34:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5191lb # [ 66.433687] niks3-server[1058]: 2026/09/20 13:34:29 INFO Signed narinfos id=21 count=15192worker2 # [ 66.411951] nix-grpc-daemon[880]: time=2026-09-20T13:34:29.746Z level=INFO msg="Uploading 1 narinfos"5193lb # [ 66.440113] niks3-server[1058]: 2026/09/20 13:34:29 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5194lb # [ 66.450145] niks3-server[1058]: 2026/09/20 13:34:29 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5195lb # [ 66.456497] niks3-server[1058]: 2026/09/20 13:34:29 INFO Completed upload id=215196worker2 # [ 66.431613] nix-grpc-daemon[880]: time=2026-09-20T13:34:29.766Z level=INFO msg="Upload complete. (392ms)"5197lb # [ 66.458122] niks3-server[1058]: 2026/09/20 13:34:29 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5198worker2 # [ 66.435346] nix-grpc-daemon[787]: ts=2026-09-20T13:34:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45080 duration_s=1 paths=1 nar_bytes_in=5605199worker2 # [ 66.440800] nix-grpc-daemon[787]: ts=2026-09-20T13:34:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:450805200worker2 # [ 66.449400] nix-daemon[868]: accepted connection from pid 787, user nix-grpc-daemon (trusted)5201worker2 # [ 66.455945] nix-daemon[868]: reaped child process 1206, status = succeeded5202worker2 # [ 66.462825] nix-grpc-daemon[787]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...5203client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5204worker2 # [ 66.471292] nix-daemon[868]: accepted connection from pid 787, user nix-grpc-daemon (trusted)5205client # uses-input-w1onlyb> worker2: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5206worker2 # [ 67.282757] nix-daemon[868]: reaped child process 1225, status = succeeded5207worker2 # [ 67.414759] nix-daemon[868]: accepted connection from pid 1240, user nix-grpc-daemon (trusted)5208worker2 # [ 67.426838] nix-daemon[868]: reaped child process 1243, status = succeeded5209lb # [ 67.464431] niks3-server[1058]: 2026/09/20 13:34:30 INFO Received uploads request method=POST path=/api/pending_closures5210worker2 # [ 67.447062] nix-grpc-daemon[880]: time=2026-09-20T13:34:30.781Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5211lb # [ 67.493686] niks3-server[1058]: 2026/09/20 13:34:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5212lb # [ 67.499655] niks3-server[1058]: 2026/09/20 13:34:30 INFO Signed narinfos id=22 count=15213worker2 # [ 67.477694] nix-grpc-daemon[880]: time=2026-09-20T13:34:30.812Z level=INFO msg="Uploading 1 narinfos"5214lb # [ 67.506390] niks3-server[1058]: 2026/09/20 13:34:30 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5215lb # [ 67.516954] niks3-server[1058]: 2026/09/20 13:34:30 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5216lb # [ 67.524921] niks3-server[1058]: 2026/09/20 13:34:30 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5217lb # [ 67.529820] niks3-server[1058]: 2026/09/20 13:34:30 INFO Completed upload id=225218worker2 # [ 67.507262] nix-grpc-daemon[880]: time=2026-09-20T13:34:30.841Z level=INFO msg="Upload complete. (228ms)"5219lb # [ 67.534545] niks3-server[1058]: 2026/09/20 13:34:30 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5220worker2 # [ 67.512465] nix-grpc-daemon[787]: ts=2026-09-20T13:34:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45080 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=10 outputs=15221worker2 # [ 67.523433] nix-daemon[868]: reaped child process 1222, status = succeeded5222client # copying 1 paths...5223worker2 # [ 67.558773] nix-grpc-daemon[787]: ts=2026-09-20T13:34:30Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:450805224client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5225worker2 # [ 67.582134] nix-grpc-daemon[787]: ts=2026-09-20T13:34:30Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:450805226worker2 # [ 67.586408] nix-grpc-daemon[787]: ts=2026-09-20T13:34:30Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45080 duration_s=0 paths=1 nar_bytes_out=1205227client # [ 67.740741] nix-daemon[945]: reaped child process 1053, status = succeeded5228client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.83 seconds)5229client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&25230client # [ 67.911645] nix-daemon[945]: accepted connection from pid 1072, user root (trusted)5231client # this derivation will be built:5232client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5233worker2 # [ 68.046283] nix-grpc-daemon[787]: ts=2026-09-20T13:34:31Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:450805234client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...5235worker1 # [ 68.086833] nix-grpc-daemon[780]: ts=2026-09-20T13:34:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374585236worker1 # [ 68.089649] nix-grpc-daemon[870]: time=2026-09-20T13:34:31.303Z level=INFO msg="All 1 paths already cached"5237worker2 # [ 68.070093] nix-grpc-daemon[787]: ts=2026-09-20T13:34:31Z level=debug event=want client=11 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05238worker1 # [ 68.104656] nix-grpc-daemon[780]: ts=2026-09-20T13:34:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374585239worker2 # [ 68.075209] nix-grpc-daemon[787]: ts=2026-09-20T13:34:31Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=15240worker1 # [ 68.117642] nix-grpc-daemon[780]: ts=2026-09-20T13:34:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374585241worker1 # [ 68.129644] nix-grpc-daemon[870]: time=2026-09-20T13:34:31.343Z level=INFO msg="All 1 paths already cached"5242client # copying 1 paths to 'lb:50051'...5243client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5244worker1 # [ 68.137378] nix-grpc-daemon[780]: ts=2026-09-20T13:34:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:374585245worker1 # [ 68.150706] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5246worker1 # [ 68.329905] nix-daemon[845]: accepted connection from pid 987, user nix-grpc-daemon (trusted)5247worker1 # [ 68.341874] nix-daemon[845]: reaped child process 990, status = succeeded5248worker1 # [ 68.521814] nix-daemon[845]: accepted connection from pid 992, user nix-grpc-daemon (trusted)5249worker1 # [ 68.534666] nix-daemon[845]: reaped child process 995, status = succeeded5250lb # [ 68.542993] niks3-server[1058]: 2026/09/20 13:34:31 INFO Received uploads request method=POST path=/api/pending_closures5251worker1 # [ 68.556287] nix-grpc-daemon[870]: time=2026-09-20T13:34:31.770Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5252worker1 # [ 68.557990] nix-grpc-daemon[870]: time=2026-09-20T13:34:31.771Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5253lb # [ 68.568709] niks3-server[1058]: 2026/09/20 13:34:31 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5254lb # [ 68.582317] niks3-server[1058]: 2026/09/20 13:34:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5255worker1 # [ 68.593851] nix-grpc-daemon[870]: time=2026-09-20T13:34:31.807Z level=INFO msg="Uploading 1 narinfos"5256lb # [ 68.587154] niks3-server[1058]: 2026/09/20 13:34:31 INFO Signed narinfos id=23 count=15257lb # [ 68.594233] niks3-server[1058]: 2026/09/20 13:34:31 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5258lb # [ 68.604114] niks3-server[1058]: 2026/09/20 13:34:31 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5259worker1 # [ 68.616964] nix-grpc-daemon[870]: time=2026-09-20T13:34:31.830Z level=INFO msg="Upload complete. (452ms)"5260lb # [ 68.610653] niks3-server[1058]: 2026/09/20 13:34:31 INFO Completed upload id=235261lb # [ 68.614878] niks3-server[1058]: 2026/09/20 13:34:31 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5262worker1 # [ 68.620388] nix-grpc-daemon[780]: ts=2026-09-20T13:34:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37458 duration_s=0 paths=1 nar_bytes_in=5605263worker1 # [ 68.627340] nix-grpc-daemon[780]: ts=2026-09-20T13:34:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374585264worker1 # [ 68.637171] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5265worker1 # [ 68.643664] nix-daemon[845]: reaped child process 985, status = succeeded5266worker1 # [ 68.646941] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5267client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5268client # uses-input-w1onlyc> node-a: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5269worker1 # [ 69.660732] nix-daemon[845]: reaped child process 999, status = succeeded5270worker1 # [ 69.804300] nix-daemon[845]: accepted connection from pid 1014, user nix-grpc-daemon (trusted)5271worker1 # [ 69.815824] nix-daemon[845]: reaped child process 1017, status = succeeded5272lb # [ 69.824320] niks3-server[1058]: 2026/09/20 13:34:33 INFO Received uploads request method=POST path=/api/pending_closures5273worker1 # [ 69.837286] nix-grpc-daemon[870]: time=2026-09-20T13:34:33.050Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5274lb # [ 69.838958] niks3-server[1058]: 2026/09/20 13:34:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5275worker1 # [ 69.847634] nix-grpc-daemon[870]: time=2026-09-20T13:34:33.061Z level=INFO msg="Uploading 1 narinfos"5276lb # [ 69.841316] niks3-server[1058]: 2026/09/20 13:34:33 INFO Signed narinfos id=24 count=15277lb # [ 69.843468] niks3-server[1058]: 2026/09/20 13:34:33 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5278lb # [ 69.846365] niks3-server[1058]: 2026/09/20 13:34:33 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5279lb # [ 69.850662] niks3-server[1058]: 2026/09/20 13:34:33 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5280lb # [ 69.853406] niks3-server[1058]: 2026/09/20 13:34:33 INFO Completed upload id=245281worker1 # [ 69.859994] nix-grpc-daemon[870]: time=2026-09-20T13:34:33.073Z level=INFO msg="Upload complete. (206ms)"5282lb # [ 69.856826] niks3-server[1058]: 2026/09/20 13:34:33 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5283worker1 # [ 69.862318] nix-grpc-daemon[780]: ts=2026-09-20T13:34:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37458 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=11 outputs=15284worker1 # [ 69.868581] nix-daemon[845]: reaped child process 997, status = succeeded5285client # copying 1 paths...5286worker2 # [ 69.878885] nix-grpc-daemon[787]: ts=2026-09-20T13:34:33Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:450805287worker2 # [ 69.889897] nix-grpc-daemon[787]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5288client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5289worker2 # [ 69.921785] nix-grpc-daemon[787]: ts=2026-09-20T13:34:33Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:450805290worker2 # [ 69.926599] nix-grpc-daemon[787]: ts=2026-09-20T13:34:33Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45080 duration_s=0 paths=1 nar_bytes_out=1205291client # [ 70.080892] nix-daemon[945]: reaped child process 1075, status = succeeded5292client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.34 seconds)5293(finished: subtest: an input only one worker has still reaches the builder, in 9.03 seconds)5294subtest: upload whose reference the worker lacks is completed from the cache5295client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache5296client # this derivation will be built:5297client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5298client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5299client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache, in 0.42 seconds)5300client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache5301client # this derivation will be built:5302client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5303client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5304client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache, in 0.39 seconds)5305client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5306client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.18 seconds)5307worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5308worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.18 seconds)5309worker2 # [ 71.265478] systemd[1]: Stopping Nix worker-protocol over gRPC...5310worker2 # [ 71.849587] nix-grpc-daemon[787]: ts=2026-09-20T13:34:35Z level=info event=signal_exit5311worker2 # [ 72.104881] nix-grpc-daemon[787]: ts=2026-09-20T13:34:35Z level=info event=scheduler_yield5312worker1 # [ 72.138514] nix-grpc-daemon[780]: ts=2026-09-20T13:34:35Z level=info event=scheduler_restarting addr=lb:500515313lb # [ 72.133960] niks3-server[1058]: 2026/09/20 13:34:35 INFO lead: released remote=192.168.1.4:345005314worker2 # [ 72.112495] nix-grpc-daemon[787]: ts=2026-09-20T13:34:35Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:450705315worker2 # [ 72.116929] nix-grpc-daemon[787]: ts=2026-09-20T13:34:35Z level=info event=worker_gone addr=192.168.1.3:500515316worker2 # [ 72.121543] nix-grpc-daemon[787]: ts=2026-09-20T13:34:35Z level=info event=scheduler_restarting addr=lb:500515317worker2 # [ 72.124598] nix-grpc-daemon[787]: ts=2026-09-20T13:34:35Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:450705318worker2 # [ 72.127965] nix-grpc-daemon[787]: ts=2026-09-20T13:34:35Z level=info event=worker_gone addr=192.168.1.4:500515319worker1 # [ 72.340382] nix-grpc-daemon[780]: ts=2026-09-20T13:34:35Z level=info event=scheduler_connected addr=lb:500515320worker1 # [ 72.343692] nix-grpc-daemon[780]: ts=2026-09-20T13:34:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5321worker2 # [ 72.317608] nix-grpc-daemon[787]: ts=2026-09-20T13:34:35Z level=info event=scheduler_connected addr=lb:500515322worker2 # [ 72.319399] nix-grpc-daemon[787]: ts=2026-09-20T13:34:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5323worker1 # [ 72.544324] nix-grpc-daemon[780]: ts=2026-09-20T13:34:35Z level=info event=scheduler_connected addr=lb:500515324worker1 # [ 72.548269] nix-grpc-daemon[780]: ts=2026-09-20T13:34:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5325worker2 # [ 72.520298] nix-grpc-daemon[787]: ts=2026-09-20T13:34:35Z level=info event=scheduler_connected addr=lb:500515326worker2 # [ 72.522191] nix-grpc-daemon[787]: ts=2026-09-20T13:34:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5327worker1 # [ 72.748965] nix-grpc-daemon[780]: ts=2026-09-20T13:34:35Z level=info event=scheduler_connected addr=lb:500515328worker1 # [ 72.752880] nix-grpc-daemon[780]: ts=2026-09-20T13:34:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5329worker2 # [ 72.723414] nix-grpc-daemon[787]: ts=2026-09-20T13:34:36Z level=info event=scheduler_connected addr=lb:500515330worker2 # [ 72.727253] nix-grpc-daemon[787]: ts=2026-09-20T13:34:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5331worker1 # [ 72.953828] nix-grpc-daemon[780]: ts=2026-09-20T13:34:36Z level=info event=scheduler_connected addr=lb:500515332worker1 # [ 72.957745] nix-grpc-daemon[780]: ts=2026-09-20T13:34:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5333worker2 # [ 72.929486] nix-grpc-daemon[787]: ts=2026-09-20T13:34:36Z level=info event=scheduler_connected addr=lb:500515334worker2 # [ 72.932488] nix-grpc-daemon[787]: ts=2026-09-20T13:34:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5335worker2 # [ 73.105658] nix-grpc-daemon[787]: ts=2026-09-20T13:34:36Z level=info event=scheduler_restarting5336worker1 # [ 73.160294] nix-grpc-daemon[780]: ts=2026-09-20T13:34:36Z level=info event=scheduler_connected addr=lb:500515337worker1 # [ 73.163598] nix-grpc-daemon[780]: ts=2026-09-20T13:34:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5338worker2 # [ 73.143661] nix-daemon[868]: reaped child process 870, status = succeeded5339worker2 # [ 73.145952] nix-grpc-daemon[787]: ts=2026-09-20T13:34:36Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5340worker1 # [ 73.360743] nix-grpc-daemon[780]: ts=2026-09-20T13:34:36Z level=info event=scheduler_connected addr=lb:500515341worker1 # [ 73.364550] nix-grpc-daemon[780]: ts=2026-09-20T13:34:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5342worker1 # [ 73.565351] nix-grpc-daemon[780]: ts=2026-09-20T13:34:36Z level=info event=scheduler_connected addr=lb:500515343worker1 # [ 73.569205] nix-grpc-daemon[780]: ts=2026-09-20T13:34:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5344worker1 # [ 73.769626] nix-grpc-daemon[780]: ts=2026-09-20T13:34:36Z level=info event=scheduler_connected addr=lb:500515345worker1 # [ 73.771865] nix-grpc-daemon[780]: ts=2026-09-20T13:34:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5346worker1 # [ 73.972764] nix-grpc-daemon[780]: ts=2026-09-20T13:34:37Z level=info event=scheduler_connected addr=lb:500515347worker1 # [ 73.976693] nix-grpc-daemon[780]: ts=2026-09-20T13:34:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5348worker2 # [ 73.955846] nix-grpc-daemon[787]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5349worker2 # [ 73.959368] nix-grpc-daemon[787]: E0000 00:00:1789911277.289165 830 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5350worker2 # [ 73.989277] nix-grpc-daemon[787]: E0000 00:00:1789911277.323778 830 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5351worker1 # [ 74.177821] nix-grpc-daemon[780]: ts=2026-09-20T13:34:37Z level=info event=scheduler_connected addr=lb:500515352worker1 # [ 74.179680] nix-grpc-daemon[780]: ts=2026-09-20T13:34:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5353worker1 # [ 74.380143] nix-grpc-daemon[780]: ts=2026-09-20T13:34:37Z level=info event=scheduler_connected addr=lb:500515354worker1 # [ 74.381865] nix-grpc-daemon[780]: ts=2026-09-20T13:34:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5355worker1 # [ 74.582432] nix-grpc-daemon[780]: ts=2026-09-20T13:34:37Z level=info event=scheduler_connected addr=lb:500515356worker1 # [ 74.584112] nix-grpc-daemon[780]: ts=2026-09-20T13:34:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5357worker1 # [ 74.785060] nix-grpc-daemon[780]: ts=2026-09-20T13:34:37Z level=info event=scheduler_connected addr=lb:500515358worker1 # [ 74.788953] nix-grpc-daemon[780]: ts=2026-09-20T13:34:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5359worker1 # [ 74.989739] nix-grpc-daemon[780]: ts=2026-09-20T13:34:38Z level=info event=scheduler_connected addr=lb:500515360worker1 # [ 74.993696] nix-grpc-daemon[780]: ts=2026-09-20T13:34:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5361worker2 # [ 75.116067] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5362worker2 # [ 75.122625] systemd[1]: Stopped Nix worker-protocol over gRPC.5363worker2 # [ 75.124671] systemd[1]: nix-grpc-daemon.service: Consumed 2.570s CPU time over 57.557s wall clock time, 61.5M memory peak, 95.1K incoming IP traffic, 258K outgoing IP traffic.5364worker2 # [ 75.134927] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5365worker2 # [ 75.139345] systemd[1]: Closed Nix worker-protocol over gRPC.5366worker2 # [ 75.142958] systemd[1]: nix-grpc-daemon.socket: Consumed 307us CPU time over 1min 5.053s wall clock time, 256K memory peak, 91.5K incoming IP traffic, 71.5K outgoing IP traffic.5367lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5368lb: must succeed: curl -sf localhost:9901/clusters5369worker1 # [ 75.196130] nix-grpc-daemon[780]: ts=2026-09-20T13:34:38Z level=info event=scheduler_connected addr=lb:500515370worker1 # [ 75.199304] nix-grpc-daemon[780]: ts=2026-09-20T13:34:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5371lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5372lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.07 seconds)5373client: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25374worker1 # [ 75.396971] nix-grpc-daemon[780]: ts=2026-09-20T13:34:38Z level=info event=scheduler_connected addr=lb:500515375worker1 # [ 75.400892] nix-grpc-daemon[780]: ts=2026-09-20T13:34:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5376worker1 # [ 75.469797] nix-grpc-daemon[780]: ts=2026-09-20T13:34:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:374585377worker1 # [ 75.476799] nix-grpc-daemon[780]: ts=2026-09-20T13:34:38Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:374585378client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5379client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.25 seconds)5380worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5381worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds)5382worker2 # [ 75.554979] systemd[1]: Listening on Nix worker-protocol over gRPC.5383worker1 # [ 75.601377] nix-grpc-daemon[780]: ts=2026-09-20T13:34:38Z level=info event=scheduler_connected addr=lb:500515384worker1 # [ 75.604172] nix-grpc-daemon[780]: ts=2026-09-20T13:34:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5385lb # [ 75.611449] niks3-server[1058]: 2026/09/20 13:34:38 INFO lead: acquired remote=192.168.1.3:423805386worker1 # [ 75.620937] nix-grpc-daemon[780]: ts=2026-09-20T13:34:38Z level=info event=scheduler_take_over5387worker1 # [ 75.630451] systemd[1]: Stopping Nix worker-protocol over gRPC...5388worker1 # [ 75.804074] nix-grpc-daemon[780]: ts=2026-09-20T13:34:39Z level=info event=scheduler_connected addr=lb:500515389worker1 # [ 75.807944] nix-grpc-daemon[780]: ts=2026-09-20T13:34:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5390lb # [ 75.896265] envoy[891]: conn peer=[2001:db8:1::1]:60758 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=575391lb # [ 75.899901] envoy[891]: conn peer=[2001:db8:1::1]:60748 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3701 tx=1806 ms=24085392lb # [ 75.903399] envoy[891]: conn peer=[2001:db8:1::1]:60788 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=565393lb # [ 75.906800] envoy[891]: conn peer=[2001:db8:1::1]:60772 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3715 tx=1892 ms=19335394lb # [ 75.910333] envoy[891]: conn peer=[2001:db8:1::4]:45530 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=58665 tx=3137 ms=552755395lb # [ 75.913952] envoy[891]: conn peer=[2001:db8:1::1]:40846 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=225396worker1 # [ 76.008837] nix-grpc-daemon[780]: ts=2026-09-20T13:34:39Z level=info event=scheduler_connected addr=lb:500515397worker1 # [ 76.012749] nix-grpc-daemon[780]: ts=2026-09-20T13:34:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5398worker2 # [ 76.102923] systemd[1]: Starting scheduler-token.service...5399worker1 # [ 76.196732] nix-grpc-daemon[780]: ts=2026-09-20T13:34:39Z level=info event=signal_exit5400worker1 # [ 76.213508] nix-grpc-daemon[780]: ts=2026-09-20T13:34:39Z level=info event=scheduler_connected addr=lb:500515401worker1 # [ 76.217199] nix-grpc-daemon[780]: ts=2026-09-20T13:34:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5402worker2 # [ 76.200410] systemd[1]: scheduler-token.service: Deactivated successfully.5403worker2 # [ 76.203499] systemd[1]: Finished scheduler-token.service.5404worker2 # [ 76.205713] systemd[1]: scheduler-token.service: Consumed 36ms CPU time over 99ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5405worker2 # [ 76.213434] systemd[1]: Starting Nix worker-protocol over gRPC...5406worker2 # [ 76.375752] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:39Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5407worker2 # [ 76.377869] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:39Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5408worker2 # [ 76.381574] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:39Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15409worker1 # [ 76.417646] nix-grpc-daemon[780]: ts=2026-09-20T13:34:39Z level=info event=scheduler_connected addr=lb:500515410worker2 # [ 76.388062] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:39Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5411worker1 # [ 76.419732] nix-grpc-daemon[780]: ts=2026-09-20T13:34:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5412worker2 # [ 76.391378] systemd[1]: Started Nix worker-protocol over gRPC.5413worker2 # [ 76.410556] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:39Z level=info event=scheduler_connected addr=lb:500515414worker2 # [ 76.412160] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5415worker1 # [ 76.476549] nix-grpc-daemon[780]: ts=2026-09-20T13:34:39Z level=info event=scheduler_yield5416lb # [ 76.472228] niks3-server[1058]: 2026/09/20 13:34:39 INFO lead: released remote=192.168.1.3:423805417worker1 # [ 76.620776] nix-grpc-daemon[780]: ts=2026-09-20T13:34:39Z level=info event=scheduler_connected addr=lb:500515418worker1 # [ 76.624656] nix-grpc-daemon[780]: ts=2026-09-20T13:34:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5419worker2 # [ 76.613630] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:39Z level=info event=scheduler_connected addr=lb:500515420worker2 # [ 76.617353] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5421worker1 # [ 76.825568] nix-grpc-daemon[780]: ts=2026-09-20T13:34:40Z level=info event=scheduler_connected addr=lb:500515422worker1 # [ 76.829474] nix-grpc-daemon[780]: ts=2026-09-20T13:34:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5423worker2 # [ 76.817750] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:40Z level=info event=scheduler_connected addr=lb:500515424worker2 # [ 76.819433] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5425worker1 # [ 77.030168] nix-grpc-daemon[780]: ts=2026-09-20T13:34:40Z level=info event=scheduler_connected addr=lb:500515426worker1 # [ 77.034121] nix-grpc-daemon[780]: ts=2026-09-20T13:34:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5427worker2 # [ 77.020136] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:40Z level=info event=scheduler_connected addr=lb:500515428worker2 # [ 77.021806] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5429worker1 # [ 77.236434] nix-grpc-daemon[780]: ts=2026-09-20T13:34:40Z level=info event=scheduler_connected addr=lb:500515430worker1 # [ 77.239503] nix-grpc-daemon[780]: ts=2026-09-20T13:34:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5431worker2 # [ 77.223199] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:40Z level=info event=scheduler_connected addr=lb:500515432worker2 # [ 77.226994] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5433worker1 # [ 77.437525] nix-grpc-daemon[780]: ts=2026-09-20T13:34:40Z level=info event=scheduler_connected addr=lb:500515434worker1 # [ 77.441409] nix-grpc-daemon[780]: ts=2026-09-20T13:34:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5435worker2 # [ 77.427925] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:40Z level=info event=scheduler_connected addr=lb:500515436worker2 # [ 77.429201] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5437worker1 # [ 77.477157] nix-grpc-daemon[780]: ts=2026-09-20T13:34:40Z level=info event=scheduler_restarting5438worker1 # [ 77.502163] nix-daemon[845]: reaped child process 847, status = succeeded5439worker1 # [ 77.653229] nix-grpc-daemon[780]: ts=2026-09-20T13:34:40Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5440worker2 # [ 77.629149] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:40Z level=info event=scheduler_connected addr=lb:500515441worker2 # [ 77.632965] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5442worker2 # [ 77.833808] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:41Z level=info event=scheduler_connected addr=lb:500515443worker2 # [ 77.837781] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5444worker2 # [ 78.038894] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:41Z level=info event=scheduler_connected addr=lb:500515445worker2 # [ 78.042778] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5446worker2 # [ 78.243856] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:41Z level=info event=scheduler_connected addr=lb:500515447worker2 # [ 78.247702] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5448worker1 # [ 78.327539] nix-grpc-daemon[780]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5449worker1 # [ 78.331083] nix-grpc-daemon[780]: E0000 00:00:1789911281.539729 825 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true5450worker1 # [ 78.335804] nix-grpc-daemon[780]: E0000 00:00:1789911281.540397 825 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5451worker2 # [ 78.449730] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:41Z level=info event=scheduler_connected addr=lb:500515452worker2 # [ 78.452815] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5453worker2 # [ 78.650930] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:41Z level=info event=scheduler_connected addr=lb:500515454worker2 # [ 78.654899] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5455worker2 # [ 78.855959] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:42Z level=info event=scheduler_connected addr=lb:500515456worker2 # [ 78.859820] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5457worker2 # [ 79.060825] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:42Z level=info event=scheduler_connected addr=lb:500515458worker1 # [ 79.093839] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5459worker1 # [ 79.097602] systemd[1]: Stopped Nix worker-protocol over gRPC.5460worker2 # [ 79.064662] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5461worker1 # [ 79.099658] systemd[1]: nix-grpc-daemon.service: Consumed 1.407s CPU time over 1min 3.199s wall clock time, 47.5M memory peak, 59.9K incoming IP traffic, 113.4K outgoing IP traffic.5462worker1 # [ 79.111635] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5463worker1 # [ 79.115373] systemd[1]: Closed Nix worker-protocol over gRPC.5464worker1 # [ 79.118143] systemd[1]: nix-grpc-daemon.socket: Consumed 293us CPU time over 1min 9.852s wall clock time, 256K memory peak, 61.6K incoming IP traffic, 53.9K outgoing IP traffic.5465lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5466lb: must succeed: curl -sf localhost:9901/clusters5467lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5468lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.07 seconds)5469client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25470worker2 # [ 79.265912] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:42Z level=info event=scheduler_connected addr=lb:500515471worker2 # [ 79.269731] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5472worker2 # [ 79.342510] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:394585473worker2 # [ 79.353746] nix-daemon[868]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5474worker2 # [ 79.374163] nix-grpc-daemon[1270]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 29 ms (attempt 1/5)5475worker2 # [ 79.408884] nix-grpc-daemon[1270]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 25 ms (attempt 2/5)5476worker2 # [ 79.439638] nix-grpc-daemon[1270]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 259 ms (attempt 3/5)5477worker2 # [ 79.471784] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:42Z level=info event=scheduler_connected addr=lb:500515478worker2 # [ 79.474805] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5479worker2 # [ 79.672915] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:43Z level=info event=scheduler_connected addr=lb:500515480worker2 # [ 79.676826] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5481worker2 # [ 79.704478] nix-grpc-daemon[1270]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 692 ms (attempt 4/5)5482worker2 # [ 79.877761] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:43Z level=info event=scheduler_connected addr=lb:500515483worker2 # [ 79.881614] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5484worker2 # [ 80.082626] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:43Z level=info event=scheduler_connected addr=lb:500515485worker2 # [ 80.086491] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5486worker2 # [ 80.287738] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:43Z level=info event=scheduler_connected addr=lb:500515487worker2 # [ 80.291646] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5488worker2 # [ 80.402638] nix-grpc-daemon[1270]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5489client # copying 2 paths...5490client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5491worker2 # [ 80.425102] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:394585492client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5493worker2 # [ 80.435903] nix-daemon[868]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5494worker2 # [ 80.458103] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:43Z level=info event=niks3_push_started pid=13265495worker2 # [ 80.493140] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:43Z level=info event=scheduler_connected addr=lb:500515496worker2 # [ 80.494601] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5497worker2 # [ 80.619461] nix-grpc-daemon[1326]: time=2026-09-20T13:34:43.953Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""5498worker2 # [ 80.694059] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:44Z level=info event=scheduler_connected addr=lb:500515499worker2 # [ 80.695803] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5500worker2 # [ 80.754823] nix-daemon[868]: accepted connection from pid 1339, user nix-grpc-daemon (trusted)5501worker2 # [ 80.765605] nix-daemon[868]: reaped child process 1342, status = succeeded5502lb # [ 80.806263] niks3-server[1058]: 2026/09/20 13:34:44 INFO Received uploads request method=POST path=/api/pending_closures5503lb # [ 80.814771] niks3-server[1058]: 2026/09/20 13:34:44 INFO Received uploads request method=POST path=/api/pending_closures5504worker2 # [ 80.796440] nix-grpc-daemon[1326]: time=2026-09-20T13:34:44.131Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5505worker2 # [ 80.798034] nix-grpc-daemon[1326]: time=2026-09-20T13:34:44.132Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5506worker2 # [ 80.799879] nix-grpc-daemon[1326]: time=2026-09-20T13:34:44.134Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5507lb # [ 80.854137] niks3-server[1058]: 2026/09/20 13:34:44 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5508lb # [ 80.871965] niks3-server[1058]: 2026/09/20 13:34:44 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5509lb # [ 80.887065] niks3-server[1058]: 2026/09/20 13:34:44 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5510lb # [ 80.889295] niks3-server[1058]: 2026/09/20 13:34:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5511lb # [ 80.891193] niks3-server[1058]: 2026/09/20 13:34:44 INFO Signed narinfos id=25 count=15512lb # [ 80.893558] niks3-server[1058]: 2026/09/20 13:34:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5513lb # [ 80.895102] niks3-server[1058]: 2026/09/20 13:34:44 INFO Signed narinfos id=26 count=25514worker2 # [ 80.868732] nix-grpc-daemon[1326]: time=2026-09-20T13:34:44.203Z level=INFO msg="Uploading 3 narinfos"5515lb # [ 80.897712] niks3-server[1058]: 2026/09/20 13:34:44 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5516lb # [ 80.909993] niks3-server[1058]: 2026/09/20 13:34:44 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5517lb # [ 80.912719] niks3-server[1058]: 2026/09/20 13:34:44 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5518lb # [ 80.916213] niks3-server[1058]: 2026/09/20 13:34:44 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5519lb # [ 80.919344] niks3-server[1058]: 2026/09/20 13:34:44 INFO Completed upload id=265520lb # [ 80.920606] niks3-server[1058]: 2026/09/20 13:34:44 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5521worker2 # [ 80.896620] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:44Z level=info event=scheduler_connected addr=lb:500515522lb # [ 80.923541] niks3-server[1058]: 2026/09/20 13:34:44 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5523lb # [ 80.925647] niks3-server[1058]: 2026/09/20 13:34:44 INFO Completed upload id=255524worker2 # [ 80.900663] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5525worker2 # [ 80.907255] nix-grpc-daemon[1326]: time=2026-09-20T13:34:44.239Z level=INFO msg="Upload complete. (282ms)"5526worker2 # [ 80.909859] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39458 duration_s=0 paths=2 nar_bytes_in=3125527worker2 # [ 80.919118] nix-daemon[868]: reaped child process 1324, status = succeeded5528client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.77 seconds)5529worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5530worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds)5531worker1 # [ 81.058671] systemd[1]: Listening on Nix worker-protocol over gRPC.5532worker1 # [ 81.066784] systemd[1]: Starting Nix worker-protocol over gRPC...5533worker2 # [ 81.101748] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:44Z level=info event=scheduler_connected addr=lb:500515534worker2 # [ 81.105576] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5535worker1 # [ 81.239845] nix-grpc-daemon[1040]: ts=2026-09-20T13:34:44Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5536worker1 # [ 81.242134] nix-grpc-daemon[1040]: ts=2026-09-20T13:34:44Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5537worker1 # [ 81.245774] nix-grpc-daemon[1040]: ts=2026-09-20T13:34:44Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15538worker1 # [ 81.253057] nix-grpc-daemon[1040]: ts=2026-09-20T13:34:44Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5539worker1 # [ 81.257142] systemd[1]: Started Nix worker-protocol over gRPC.5540worker1 # [ 81.264199] nix-grpc-daemon[1040]: ts=2026-09-20T13:34:44Z level=info event=scheduler_take_over5541lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5542lb: must succeed: curl -sf localhost:9901/clusters5543lb # [ 81.258114] niks3-server[1058]: 2026/09/20 13:34:44 INFO lead: acquired remote=192.168.1.3:373025544worker1 # [ 81.281137] nix-grpc-daemon[1040]: ts=2026-09-20T13:34:44Z level=info event=scheduler_connected addr=lb:500515545worker1 # [ 81.286716] nix-grpc-daemon[1040]: ts=2026-09-20T13:34:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5546lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5547lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.07 seconds)5548(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.26 seconds)5549subtest: scheduler down: the other node takes the lock and keeps it5550lb: must succeed: curl -sf localhost:9901/clusters5551worker2 # [ 81.306665] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:44Z level=info event=scheduler_connected addr=lb:500515552worker1 # [ 81.347211] nix-grpc-daemon[1040]: ts=2026-09-20T13:34:44Z level=info event=oidc_jwks provider=mock keys=15553worker1 # [ 81.350455] nix-grpc-daemon[1040]: ts=2026-09-20T13:34:44Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:469965554worker1 # [ 81.353987] nix-grpc-daemon[1040]: ts=2026-09-20T13:34:44Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05555lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5556lb: must succeed: curl -sf localhost:9901/clusters5557lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5558lb: must succeed: curl -sf localhost:9901/clusters5559lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5560lb: must succeed: curl -sf localhost:9901/clusters5561worker1 # [ 81.487842] nix-grpc-daemon[1040]: ts=2026-09-20T13:34:44Z level=info event=scheduler_connected addr=lb:500515562worker1 # [ 81.492626] nix-grpc-daemon[1040]: ts=2026-09-20T13:34:44Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:469965563worker1 # [ 81.496221] nix-grpc-daemon[1040]: ts=2026-09-20T13:34:44Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05564lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5565worker1 # [ 81.570511] systemd[1]: Stopping Nix worker-protocol over gRPC...5566worker1 # [ 82.257170] nix-grpc-daemon[1040]: ts=2026-09-20T13:34:45Z level=info event=signal_exit5567worker1 # [ 82.463644] nix-grpc-daemon[1040]: ts=2026-09-20T13:34:45Z level=info event=scheduler_yield5568lb # [ 82.459858] niks3-server[1058]: 2026/09/20 13:34:45 INFO lead: released remote=192.168.1.3:373025569worker1 # [ 82.470284] nix-grpc-daemon[1040]: ts=2026-09-20T13:34:45Z level=info event=scheduler_restarting addr=lb:500515570worker1 # [ 82.474534] nix-grpc-daemon[1040]: ts=2026-09-20T13:34:45Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:469965571worker2 # [ 82.444208] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:45Z level=info event=scheduler_restarting addr=lb:500515572worker1 # [ 82.478972] nix-grpc-daemon[1040]: ts=2026-09-20T13:34:45Z level=info event=worker_gone addr=192.168.1.3:500515573worker1 # [ 82.482167] nix-grpc-daemon[1040]: ts=2026-09-20T13:34:45Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:469965574worker1 # [ 82.485444] nix-grpc-daemon[1040]: ts=2026-09-20T13:34:45Z level=info event=worker_gone addr=192.168.1.4:500515575worker1 # [ 82.671420] nix-grpc-daemon[1040]: ts=2026-09-20T13:34:45Z level=info event=scheduler_connected addr=lb:500515576worker2 # [ 82.644967] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:45Z level=info event=scheduler_connected addr=lb:500515577worker1 # [ 82.675881] nix-grpc-daemon[1040]: ts=2026-09-20T13:34:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5578worker2 # [ 82.648800] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5579worker1 # [ 82.876670] nix-grpc-daemon[1040]: ts=2026-09-20T13:34:46Z level=info event=scheduler_connected addr=lb:500515580worker1 # [ 82.880532] nix-grpc-daemon[1040]: ts=2026-09-20T13:34:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5581worker2 # [ 82.849796] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:46Z level=info event=scheduler_connected addr=lb:500515582worker2 # [ 82.853588] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5583worker1 # [ 83.081320] nix-grpc-daemon[1040]: ts=2026-09-20T13:34:46Z level=info event=scheduler_connected addr=lb:500515584worker1 # [ 83.085151] nix-grpc-daemon[1040]: ts=2026-09-20T13:34:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5585worker2 # [ 83.054729] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:46Z level=info event=scheduler_connected addr=lb:500515586worker2 # [ 83.058454] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5587worker1 # [ 83.287383] nix-grpc-daemon[1040]: ts=2026-09-20T13:34:46Z level=info event=scheduler_connected addr=lb:500515588worker2 # [ 83.259741] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:46Z level=info event=scheduler_connected addr=lb:500515589worker1 # [ 83.292751] nix-grpc-daemon[1040]: ts=2026-09-20T13:34:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5590worker2 # [ 83.263322] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5591worker1 # [ 83.469812] nix-grpc-daemon[1040]: ts=2026-09-20T13:34:46Z level=info event=scheduler_restarting5592worker2 # [ 83.465489] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:46Z level=info event=scheduler_connected addr=lb:500515593worker2 # [ 83.468749] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5594worker2 # [ 83.666631] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:47Z level=info event=scheduler_connected addr=lb:500515595worker2 # [ 83.670508] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5596worker2 # [ 83.871485] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:47Z level=info event=scheduler_connected addr=lb:500515597worker2 # [ 83.875323] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5598worker2 # [ 84.076343] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:47Z level=info event=scheduler_connected addr=lb:500515599worker2 # [ 84.080248] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5600worker1 # [ 84.294270] nix-grpc-daemon[1040]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5601worker1 # [ 84.297612] nix-grpc-daemon[1040]: E0000 00:00:1789911287.506911 1083 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true5602worker1 # [ 84.302307] nix-grpc-daemon[1040]: E0000 00:00:1789911287.507162 1083 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5603worker2 # [ 84.281285] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:47Z level=info event=scheduler_connected addr=lb:500515604worker2 # [ 84.285125] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5605worker2 # [ 84.487557] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:47Z level=info event=scheduler_connected addr=lb:500515606worker2 # [ 84.490671] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5607worker2 # [ 84.688444] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:48Z level=info event=scheduler_connected addr=lb:500515608worker2 # [ 84.692324] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5609worker2 # [ 84.893571] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:48Z level=info event=scheduler_connected addr=lb:500515610worker2 # [ 84.897325] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5611worker2 # [ 85.098359] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:48Z level=info event=scheduler_connected addr=lb:500515612worker2 # [ 85.102461] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5613worker1 # [ 85.284543] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5614worker1 # [ 85.287325] systemd[1]: Stopped Nix worker-protocol over gRPC.5615worker1 # [ 85.290652] systemd[1]: nix-grpc-daemon.service: Consumed 125ms CPU time over 4.217s wall clock time, 10.3M memory peak, 9.5K incoming IP traffic, 8.7K outgoing IP traffic.5616worker1 # [ 85.302154] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5617worker1 # [ 85.306414] systemd[1]: Closed Nix worker-protocol over gRPC.5618worker1 # [ 85.309485] systemd[1]: nix-grpc-daemon.socket: Consumed 537us CPU time over 4.254s wall clock time, 256K memory peak, 13.6K incoming IP traffic, 10.5K outgoing IP traffic.5619worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5620worker2 # [ 85.304131] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:48Z level=info event=scheduler_connected addr=lb:500515621worker2 # [ 85.307970] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5622worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.08 seconds)5623worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5624worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5625worker2 # [ 85.510346] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:48Z level=info event=scheduler_connected addr=lb:500515626worker2 # [ 85.513455] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5627worker2 # [ 85.711731] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:49Z level=info event=scheduler_connected addr=lb:500515628worker2 # [ 85.715783] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5629lb # [ 85.896341] envoy[891]: conn peer=[2001:db8:1::3]:35046 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=19488 tx=3524 ms=605985630lb # [ 85.900400] envoy[891]: conn peer=[2001:db8:1::1]:40862 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1009 tx=282 ms=15765631lb # [ 85.905521] envoy[891]: conn peer=[2001:db8:1::3]:38554 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1547 tx=400 ms=22185632worker2 # [ 85.916803] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:49Z level=info event=scheduler_connected addr=lb:500515633worker2 # [ 85.920524] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5634worker2 # [ 86.121595] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:49Z level=info event=scheduler_connected addr=lb:500515635worker2 # [ 86.125436] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5636worker2 # [ 86.326349] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:49Z level=info event=scheduler_connected addr=lb:500515637worker2 # [ 86.330276] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5638lb # [ 86.426506] niks3-server[1058]: 2026/09/20 13:34:49 INFO lead: acquired remote=192.168.1.4:501625639worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5640worker2 # [ 86.404845] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:49Z level=info event=scheduler_take_over5641worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5642worker2 # [ 86.534162] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:49Z level=info event=scheduler_connected addr=lb:500515643worker2 # [ 86.548151] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:49Z level=info event=oidc_jwks provider=mock keys=15644worker2 # [ 86.551887] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:49Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:383605645worker2 # [ 86.555759] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:49Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05646worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5647worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5648client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&25649worker2 # [ 87.757585] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:394585650client # don't know how to build these paths:5651client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5652worker2 # [ 87.766293] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:394585653worker2 # [ 87.799930] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:51Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05654worker2 # [ 87.804482] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:51Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.4:50051 clients=15655worker2 # [ 87.849146] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:394585656worker2 # [ 87.862096] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:394585657client # copying 1 paths to 'lb:50051'...5658client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5659worker2 # [ 87.874449] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:394585660worker2 # [ 87.886749] nix-daemon[868]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5661worker2 # [ 88.035419] nix-daemon[868]: accepted connection from pid 1366, user nix-grpc-daemon (trusted)5662worker2 # [ 88.047863] nix-daemon[868]: reaped child process 1369, status = succeeded5663worker2 # [ 88.214088] nix-daemon[868]: accepted connection from pid 1371, user nix-grpc-daemon (trusted)5664worker2 # [ 88.224679] nix-daemon[868]: reaped child process 1374, status = succeeded5665lb # [ 88.265895] niks3-server[1058]: 2026/09/20 13:34:51 INFO Received uploads request method=POST path=/api/pending_closures5666worker2 # [ 88.253601] nix-grpc-daemon[1326]: time=2026-09-20T13:34:51.587Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5667worker2 # [ 88.255653] nix-grpc-daemon[1326]: time=2026-09-20T13:34:51.589Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5668lb # [ 88.313150] niks3-server[1058]: 2026/09/20 13:34:51 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5669lb # [ 88.320757] niks3-server[1058]: 2026/09/20 13:34:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5670worker2 # [ 88.301365] nix-grpc-daemon[1326]: time=2026-09-20T13:34:51.635Z level=INFO msg="Uploading 1 narinfos"5671lb # [ 88.326269] niks3-server[1058]: 2026/09/20 13:34:51 INFO Signed narinfos id=27 count=15672lb # [ 88.332480] niks3-server[1058]: 2026/09/20 13:34:51 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5673lb # [ 88.342206] niks3-server[1058]: 2026/09/20 13:34:51 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5674lb # [ 88.351561] niks3-server[1058]: 2026/09/20 13:34:51 INFO Completed upload id=275675worker2 # [ 88.326863] nix-grpc-daemon[1326]: time=2026-09-20T13:34:51.661Z level=INFO msg="Upload complete. (427ms)"5676lb # [ 88.359539] niks3-server[1058]: 2026/09/20 13:34:51 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5677worker2 # [ 88.331922] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39458 duration_s=0 paths=1 nar_bytes_in=5205678worker2 # [ 88.337825] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:394585679worker2 # [ 88.349154] nix-daemon[868]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5680worker2 # [ 88.358763] nix-daemon[868]: reaped child process 1364, status = succeeded5681client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5682worker2 # [ 88.366201] nix-daemon[868]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5683client # farm-a> worker2: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5684worker2 # [ 88.979916] nix-daemon[868]: reaped child process 1378, status = succeeded5685client # farm-a> LOG-farm-a-on-standby5686worker2 # [ 89.119250] nix-daemon[868]: accepted connection from pid 1393, user nix-grpc-daemon (trusted)5687worker2 # [ 89.131092] nix-daemon[868]: reaped child process 1396, status = succeeded5688lb # [ 89.171527] niks3-server[1058]: 2026/09/20 13:34:52 INFO Received uploads request method=POST path=/api/pending_closures5689worker2 # [ 89.157458] nix-grpc-daemon[1326]: time=2026-09-20T13:34:52.492Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5690worker2 # [ 89.158935] nix-grpc-daemon[1326]: time=2026-09-20T13:34:52.493Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5691lb # [ 89.213087] niks3-server[1058]: 2026/09/20 13:34:52 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5692lb # [ 89.218491] niks3-server[1058]: 2026/09/20 13:34:52 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5693lb # [ 89.230514] niks3-server[1058]: 2026/09/20 13:34:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5694lb # [ 89.236933] niks3-server[1058]: 2026/09/20 13:34:52 INFO Signed narinfos id=28 count=15695worker2 # [ 89.212141] nix-grpc-daemon[1326]: time=2026-09-20T13:34:52.546Z level=INFO msg="Uploading 1 narinfos"5696lb # [ 89.239526] niks3-server[1058]: 2026/09/20 13:34:52 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5697lb # [ 89.249487] niks3-server[1058]: 2026/09/20 13:34:52 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5698lb # [ 89.257495] niks3-server[1058]: 2026/09/20 13:34:52 INFO Completed upload id=285699lb # [ 89.258968] niks3-server[1058]: 2026/09/20 13:34:52 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5700worker2 # [ 89.232606] nix-grpc-daemon[1326]: time=2026-09-20T13:34:52.567Z level=INFO msg="Upload complete. (258ms)"5701worker2 # [ 89.246479] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39458 duration_s=0 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15702worker2 # [ 89.251229] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:52Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05703worker2 # [ 89.255204] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:52Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.4:50051 clients=15704worker2 # [ 89.263660] nix-daemon[868]: reaped child process 1376, status = succeeded5705worker2 # [ 89.290238] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:394585706worker2 # [ 89.302561] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:394585707client # copying 1 paths to 'lb:50051'...5708worker2 # [ 89.316781] nix-grpc-daemon[1326]: time=2026-09-20T13:34:52.650Z level=INFO msg="All 1 paths already cached"5709client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5710worker2 # [ 89.321250] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:394585711worker2 # [ 89.332262] nix-daemon[868]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5712worker2 # [ 89.467197] nix-daemon[868]: accepted connection from pid 1402, user nix-grpc-daemon (trusted)5713worker2 # [ 89.479843] nix-daemon[868]: reaped child process 1405, status = succeeded5714worker2 # [ 89.634623] nix-daemon[868]: accepted connection from pid 1407, user nix-grpc-daemon (trusted)5715worker2 # [ 89.646794] nix-daemon[868]: reaped child process 1410, status = succeeded5716lb # [ 89.686324] niks3-server[1058]: 2026/09/20 13:34:53 INFO Received uploads request method=POST path=/api/pending_closures5717worker2 # [ 89.669206] nix-grpc-daemon[1326]: time=2026-09-20T13:34:53.003Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5718worker2 # [ 89.670905] nix-grpc-daemon[1326]: time=2026-09-20T13:34:53.005Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5719lb # [ 89.714514] niks3-server[1058]: 2026/09/20 13:34:53 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5720lb # [ 89.726599] niks3-server[1058]: 2026/09/20 13:34:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5721worker2 # [ 89.708214] nix-grpc-daemon[1326]: time=2026-09-20T13:34:53.042Z level=INFO msg="Uploading 1 narinfos"5722lb # [ 89.733266] niks3-server[1058]: 2026/09/20 13:34:53 INFO Signed narinfos id=29 count=15723lb # [ 89.737701] niks3-server[1058]: 2026/09/20 13:34:53 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5724lb # [ 89.751111] niks3-server[1058]: 2026/09/20 13:34:53 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5725lb # [ 89.755559] niks3-server[1058]: 2026/09/20 13:34:53 INFO Completed upload id=295726worker2 # [ 89.732909] nix-grpc-daemon[1326]: time=2026-09-20T13:34:53.067Z level=INFO msg="Upload complete. (390ms)"5727lb # [ 89.758167] niks3-server[1058]: 2026/09/20 13:34:53 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5728worker2 # [ 89.736773] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39458 duration_s=0 paths=1 nar_bytes_in=6405729worker2 # [ 89.741260] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:394585730worker2 # [ 89.749472] nix-daemon[868]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5731worker2 # [ 89.759124] nix-daemon[868]: reaped child process 1400, status = succeeded5732client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5733worker2 # [ 89.763376] nix-daemon[868]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5734client # farm-b> worker2: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5735client # farm-b> LOG-farm-b-on-standby5736worker2 # [ 90.777204] nix-daemon[868]: reaped child process 1414, status = succeeded5737worker2 # [ 90.895484] nix-daemon[868]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)5738worker2 # [ 90.907747] nix-daemon[868]: reaped child process 1432, status = succeeded5739lb # [ 90.942938] niks3-server[1058]: 2026/09/20 13:34:54 INFO Received uploads request method=POST path=/api/pending_closures5740worker2 # [ 90.928221] nix-grpc-daemon[1326]: time=2026-09-20T13:34:54.262Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5741worker2 # [ 90.930431] nix-grpc-daemon[1326]: time=2026-09-20T13:34:54.264Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"5742lb # [ 90.980254] niks3-server[1058]: 2026/09/20 13:34:54 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst5743lb # [ 90.990834] niks3-server[1058]: 2026/09/20 13:34:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5744worker2 # [ 90.971783] nix-grpc-daemon[1326]: time=2026-09-20T13:34:54.306Z level=INFO msg="Uploading 1 narinfos"5745lb # [ 90.998175] niks3-server[1058]: 2026/09/20 13:34:54 INFO Signed narinfos id=30 count=15746lb # [ 91.004562] niks3-server[1058]: 2026/09/20 13:34:54 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5747lb # [ 91.012853] niks3-server[1058]: 2026/09/20 13:34:54 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls5748lb # [ 91.020370] niks3-server[1058]: 2026/09/20 13:34:54 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5749lb # [ 91.025132] niks3-server[1058]: 2026/09/20 13:34:54 INFO Completed upload id=305750lb # [ 91.027135] niks3-server[1058]: 2026/09/20 13:34:54 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo5751worker2 # [ 91.001626] nix-grpc-daemon[1326]: time=2026-09-20T13:34:54.336Z level=INFO msg="Upload complete. (231ms)"5752worker2 # [ 91.009717] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39458 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=15753worker2 # [ 91.016596] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:54Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=05754worker2 # [ 91.020211] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:54Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.4:50051 clients=15755worker2 # [ 91.028028] nix-daemon[868]: reaped child process 1412, status = succeeded5756worker2 # [ 91.060323] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:394585757worker2 # [ 91.073353] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:394585758worker2 # [ 91.087270] nix-grpc-daemon[1326]: time=2026-09-20T13:34:54.421Z level=INFO msg="All 2 paths already cached"5759client # copying 1 paths to 'lb:50051'...5760client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...5761worker2 # [ 91.094443] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:394585762worker2 # [ 91.102813] nix-daemon[868]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5763worker2 # [ 91.242064] nix-daemon[868]: accepted connection from pid 1438, user nix-grpc-daemon (trusted)5764worker2 # [ 91.253808] nix-daemon[868]: reaped child process 1441, status = succeeded5765worker2 # [ 91.385745] nix-daemon[868]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)5766worker2 # [ 91.397228] nix-daemon[868]: reaped child process 1446, status = succeeded5767lb # [ 91.432697] niks3-server[1058]: 2026/09/20 13:34:54 INFO Received uploads request method=POST path=/api/pending_closures5768worker2 # [ 91.415612] nix-grpc-daemon[1326]: time=2026-09-20T13:34:54.750Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5769worker2 # [ 91.417239] nix-grpc-daemon[1326]: time=2026-09-20T13:34:54.751Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"5770lb # [ 91.460901] niks3-server[1058]: 2026/09/20 13:34:54 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst5771lb # [ 91.473505] niks3-server[1058]: 2026/09/20 13:34:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5772lb # [ 91.478667] niks3-server[1058]: 2026/09/20 13:34:54 INFO Signed narinfos id=31 count=15773worker2 # [ 91.456559] nix-grpc-daemon[1326]: time=2026-09-20T13:34:54.791Z level=INFO msg="Uploading 1 narinfos"5774lb # [ 91.484604] niks3-server[1058]: 2026/09/20 13:34:54 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls5775lb # [ 91.494434] niks3-server[1058]: 2026/09/20 13:34:54 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5776worker2 # [ 91.475707] nix-grpc-daemon[1326]: time=2026-09-20T13:34:54.810Z level=INFO msg="Upload complete. (361ms)"5777lb # [ 91.501083] niks3-server[1058]: 2026/09/20 13:34:54 INFO Completed upload id=315778worker2 # [ 91.477609] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39458 duration_s=0 paths=1 nar_bytes_in=7685779lb # [ 91.504678] niks3-server[1058]: 2026/09/20 13:34:54 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo5780worker2 # [ 91.482608] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:394585781worker2 # [ 91.490847] nix-daemon[868]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5782worker2 # [ 91.496696] nix-daemon[868]: reaped child process 1436, status = succeeded5783worker2 # [ 91.499771] nix-daemon[868]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5784client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...5785client # farm-top> worker2: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5786client # farm-top> LOG-farm-top-on-standby5787worker2 # [ 92.928278] nix-daemon[868]: reaped child process 1450, status = succeeded5788worker2 # [ 93.071211] nix-daemon[868]: accepted connection from pid 1465, user nix-grpc-daemon (trusted)5789worker2 # [ 93.083372] nix-daemon[868]: reaped child process 1468, status = succeeded5790lb # [ 93.123604] niks3-server[1058]: 2026/09/20 13:34:56 INFO Received uploads request method=POST path=/api/pending_closures5791worker2 # [ 93.107776] nix-grpc-daemon[1326]: time=2026-09-20T13:34:56.442Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5792worker2 # [ 93.109333] nix-grpc-daemon[1326]: time=2026-09-20T13:34:56.443Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"5793lb # [ 93.150915] niks3-server[1058]: 2026/09/20 13:34:56 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5794worker2 # [ 93.128932] nix-grpc-daemon[1326]: time=2026-09-20T13:34:56.463Z level=INFO msg="Uploading 1 narinfos"5795lb # [ 93.154296] niks3-server[1058]: 2026/09/20 13:34:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5796lb # [ 93.155778] niks3-server[1058]: 2026/09/20 13:34:56 INFO Signed narinfos id=32 count=15797lb # [ 93.157527] niks3-server[1058]: 2026/09/20 13:34:56 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst5798lb # [ 93.161555] niks3-server[1058]: 2026/09/20 13:34:56 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls5799lb # [ 93.165226] niks3-server[1058]: 2026/09/20 13:34:56 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5800lb # [ 93.169555] niks3-server[1058]: 2026/09/20 13:34:56 INFO Completed upload id=325801worker2 # [ 93.144538] nix-grpc-daemon[1326]: time=2026-09-20T13:34:56.479Z level=INFO msg="Upload complete. (237ms)"5802lb # [ 93.172353] niks3-server[1058]: 2026/09/20 13:34:56 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo5803worker2 # [ 93.146990] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39458 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=15804worker2 # [ 93.154655] nix-daemon[868]: reaped child process 1448, status = succeeded5805client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&2, in 5.65 seconds)5806client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath5807client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath, in 0.18 seconds)5808worker1 # [ 93.466682] systemd[1]: Listening on Nix worker-protocol over gRPC.5809worker1 # [ 93.472976] systemd[1]: Starting Nix worker-protocol over gRPC...5810worker1 # [ 93.627277] nix-grpc-daemon[1104]: ts=2026-09-20T13:34:56Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5811worker1 # [ 93.629166] nix-grpc-daemon[1104]: ts=2026-09-20T13:34:56Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5812worker1 # [ 93.632673] nix-grpc-daemon[1104]: ts=2026-09-20T13:34:56Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15813worker1 # [ 93.638985] nix-grpc-daemon[1104]: ts=2026-09-20T13:34:56Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5814worker1 # [ 93.643780] systemd[1]: Started Nix worker-protocol over gRPC.5815lb: must succeed: curl -sf localhost:9901/clusters5816worker1 # [ 93.656227] nix-grpc-daemon[1104]: ts=2026-09-20T13:34:56Z level=info event=scheduler_connected addr=lb:500515817worker2 # [ 93.626790] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:56Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:383605818worker2 # [ 93.630435] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:56Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05819lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5820worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5821worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5822lb: must succeed: curl -sf localhost:9901/clusters5823lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5824lb: must succeed: curl -sf localhost:9901/clusters5825lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5826worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5827worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5828(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.62 seconds)5829subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5830client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs5831client # [ 94.016172] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs.5832client # Running as unit: rs.service; invocation ID: 801ab6906fef4c43bd54183e8137bb585833client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs, in 0.07 seconds)5834client # [ 94.219864] nix[1175]: don't know how to build these paths:5835client # [ 94.220898] nix[1175]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5836worker2 # [ 94.169571] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:57Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:394585837worker2 # [ 94.180227] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:57Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=05838worker2 # [ 94.185398] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:57Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.4:50051 clients=15839worker2 # [ 94.224170] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:394585840worker2 # [ 94.236582] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:394585841client # [ 94.296262] nix[1175]: copying 1 paths to 'lb:50051'...5842worker2 # [ 94.247856] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:394585843client # [ 94.299248] nix[1175]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...5844worker2 # [ 94.261259] nix-daemon[868]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5845worker2 # [ 94.425663] nix-daemon[868]: accepted connection from pid 1481, user nix-grpc-daemon (trusted)5846worker2 # [ 94.437853] nix-daemon[868]: reaped child process 1484, status = succeeded5847worker2 # [ 94.591425] nix-daemon[868]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)5848worker2 # [ 94.602769] nix-daemon[868]: reaped child process 1489, status = succeeded5849lb # [ 94.643469] niks3-server[1058]: 2026/09/20 13:34:57 INFO Received uploads request method=POST path=/api/pending_closures5850worker2 # [ 94.630709] nix-grpc-daemon[1326]: time=2026-09-20T13:34:57.965Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5851worker2 # [ 94.632188] nix-grpc-daemon[1326]: time=2026-09-20T13:34:57.966Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"5852lb # [ 94.674774] niks3-server[1058]: 2026/09/20 13:34:57 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst5853lb # [ 94.683921] niks3-server[1058]: 2026/09/20 13:34:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5854worker2 # [ 94.664362] nix-grpc-daemon[1326]: time=2026-09-20T13:34:57.998Z level=INFO msg="Uploading 1 narinfos"5855lb # [ 94.689261] niks3-server[1058]: 2026/09/20 13:34:58 INFO Signed narinfos id=33 count=15856lb # [ 94.697893] niks3-server[1058]: 2026/09/20 13:34:58 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls5857lb # [ 94.702977] niks3-server[1058]: 2026/09/20 13:34:58 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5858lb # [ 94.705513] niks3-server[1058]: 2026/09/20 13:34:58 INFO Completed upload id=335859worker2 # [ 94.680620] nix-grpc-daemon[1326]: time=2026-09-20T13:34:58.015Z level=INFO msg="Upload complete. (407ms)"5860lb # [ 94.708144] niks3-server[1058]: 2026/09/20 13:34:58 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo5861worker2 # [ 94.684140] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39458 duration_s=0 paths=1 nar_bytes_in=4165862worker2 # [ 94.689261] nix-grpc-daemon[1270]: ts=2026-09-20T13:34:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:394585863worker2 # [ 94.696760] nix-daemon[868]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5864worker2 # [ 94.701295] nix-daemon[868]: reaped child process 1479, status = succeeded5865client # [ 94.754300] nix[1175]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...5866worker2 # [ 94.705624] nix-daemon[868]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5867lb # [ 95.895977] envoy[891]: conn peer=[2001:db8:1::1]:52060 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9607 tx=3839 ms=54085868lb: must succeed: curl -sf localhost:9901/clusters5869lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5870lb: must succeed: curl -sf localhost:9901/clusters5871lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5872lb: must succeed: curl -sf localhost:9901/clusters5873lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5874lb: must succeed: curl -sf localhost:9901/clusters5875lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5876worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5877worker1 # [ 96.865982] systemd-journald[287]: Received client request to rotate journal, rotating.5878worker1 # [ 96.884786] systemd-journald[287]: Vacuuming done, freed 0B of archived journals from /var/log/journal/049fe6204f3c41c1ac783e954b34d6cc.5879worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.06 seconds)5880worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service5881worker2 # [ 96.759513] systemd[1]: Stopping Nix worker-protocol over gRPC...5882worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)5883client: must succeed: sleep 5.05884worker2 # [ 97.397155] nix-grpc-daemon[1270]: ts=2026-09-20T13:35:00Z level=info event=draining5885client: (finished: must succeed: sleep 5.0, in 5.04 seconds)5886worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5887worker2 # [ 102.134152] nix-daemon[868]: reaped child process 1493, status = succeeded5888worker2 # [ 102.253666] nix-daemon[868]: accepted connection from pid 1533, user nix-grpc-daemon (trusted)5889worker2 # [ 102.265568] nix-daemon[868]: reaped child process 1536, status = succeeded5890lb # [ 102.304350] niks3-server[1058]: 2026/09/20 13:35:05 INFO Received uploads request method=POST path=/api/pending_closures5891worker2 # [ 102.289282] nix-grpc-daemon[1326]: time=2026-09-20T13:35:05.622Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5892worker2 # [ 102.290644] nix-grpc-daemon[1326]: time=2026-09-20T13:35:05.622Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"5893lb # [ 102.342279] niks3-server[1058]: 2026/09/20 13:35:05 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5894lb # [ 102.349609] niks3-server[1058]: 2026/09/20 13:35:05 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5895lb # [ 102.361702] niks3-server[1058]: 2026/09/20 13:35:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5896worker2 # [ 102.341083] nix-grpc-daemon[1326]: time=2026-09-20T13:35:05.675Z level=INFO msg="Uploading 1 narinfos"5897lb # [ 102.366107] niks3-server[1058]: 2026/09/20 13:35:05 INFO Signed narinfos id=34 count=15898lb # [ 102.370902] niks3-server[1058]: 2026/09/20 13:35:05 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls5899lb # [ 102.379082] niks3-server[1058]: 2026/09/20 13:35:05 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5900lb # [ 102.382261] niks3-server[1058]: 2026/09/20 13:35:05 INFO Completed upload id=345901worker2 # [ 102.357420] nix-grpc-daemon[1326]: time=2026-09-20T13:35:05.691Z level=INFO msg="Upload complete. (238ms)"5902lb # [ 102.384855] niks3-server[1058]: 2026/09/20 13:35:05 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo5903worker2 # [ 102.366484] nix-grpc-daemon[1270]: ts=2026-09-20T13:35:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39458 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=15904worker2 # [ 102.376835] nix-daemon[868]: reaped child process 1491, status = succeeded5905client # [ 102.445570] systemd[1]: rs.service: Deactivated successfully.5906worker2 # [ 102.399876] nix-grpc-daemon[1270]: ts=2026-09-20T13:35:05Z level=info event=signal_exit5907client # [ 102.449587] systemd[1]: rs.service: Consumed 111ms CPU time over 8.432s wall clock time, 37.1M memory peak, 8.4K incoming IP traffic, 8.9K outgoing IP traffic.5908worker2 # [ 102.624844] nix-grpc-daemon[1270]: ts=2026-09-20T13:35:05Z level=info event=scheduler_yield5909lb # [ 102.652096] niks3-server[1058]: 2026/09/20 13:35:05 INFO lead: released remote=192.168.1.4:501625910worker1 # [ 102.661679] nix-grpc-daemon[1104]: ts=2026-09-20T13:35:05Z level=info event=scheduler_restarting addr=lb:500515911worker2 # [ 102.629819] nix-grpc-daemon[1270]: ts=2026-09-20T13:35:05Z level=info event=scheduler_restarting addr=lb:500515912worker2 # [ 102.635110] nix-grpc-daemon[1270]: ts=2026-09-20T13:35:05Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:383605913worker2 # [ 102.638780] nix-grpc-daemon[1270]: ts=2026-09-20T13:35:05Z level=info event=worker_gone addr=192.168.1.3:500515914worker2 # [ 102.641987] nix-grpc-daemon[1270]: ts=2026-09-20T13:35:05Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:383605915worker2 # [ 102.645586] nix-grpc-daemon[1270]: ts=2026-09-20T13:35:05Z level=info event=worker_gone addr=192.168.1.4:500515916worker1 # [ 102.861710] nix-grpc-daemon[1104]: ts=2026-09-20T13:35:06Z level=info event=scheduler_connected addr=lb:500515917worker1 # [ 102.863705] nix-grpc-daemon[1104]: ts=2026-09-20T13:35:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5918worker2 # [ 102.831837] nix-grpc-daemon[1270]: ts=2026-09-20T13:35:06Z level=info event=scheduler_connected addr=lb:500515919worker2 # [ 102.834891] nix-grpc-daemon[1270]: ts=2026-09-20T13:35:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5920worker1 # [ 103.064617] nix-grpc-daemon[1104]: ts=2026-09-20T13:35:06Z level=info event=scheduler_connected addr=lb:500515921worker2 # [ 103.033272] nix-grpc-daemon[1270]: ts=2026-09-20T13:35:06Z level=info event=scheduler_connected addr=lb:500515922worker1 # [ 103.068509] nix-grpc-daemon[1104]: ts=2026-09-20T13:35:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5923worker2 # [ 103.037199] nix-grpc-daemon[1270]: ts=2026-09-20T13:35:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5924worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 1.11 seconds)5925client: waiting for success: ! systemctl is-active rs5926client: (finished: waiting for success: ! systemctl is-active rs, in 0.04 seconds)5927client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5928client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.03 seconds)5929client: must fail: journalctl -u rs | grep -q 'reconnecting'5930client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.02 seconds)5931lb: must succeed: curl -sf localhost:9901/clusters5932worker1 # [ 103.269374] nix-grpc-daemon[1104]: ts=2026-09-20T13:35:06Z level=info event=scheduler_connected addr=lb:500515933worker2 # [ 103.237990] nix-grpc-daemon[1270]: ts=2026-09-20T13:35:06Z level=info event=scheduler_connected addr=lb:500515934worker2 # [ 103.241066] nix-grpc-daemon[1270]: ts=2026-09-20T13:35:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5935worker1 # [ 103.273315] nix-grpc-daemon[1104]: ts=2026-09-20T13:35:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5936lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5937worker2 # [ 103.441226] nix-grpc-daemon[1270]: ts=2026-09-20T13:35:06Z level=info event=scheduler_connected addr=lb:500515938worker1 # [ 103.474742] nix-grpc-daemon[1104]: ts=2026-09-20T13:35:06Z level=info event=scheduler_connected addr=lb:500515939worker2 # [ 103.444442] nix-grpc-daemon[1270]: ts=2026-09-20T13:35:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5940worker1 # [ 103.477801] nix-grpc-daemon[1104]: ts=2026-09-20T13:35:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5941lb # [ 103.641768] niks3-server[1058]: 2026/09/20 13:35:06 INFO lead: acquired remote=192.168.1.3:551225942worker1 # [ 103.651523] nix-grpc-daemon[1104]: ts=2026-09-20T13:35:06Z level=info event=scheduler_take_over5943worker2 # [ 103.629208] nix-grpc-daemon[1270]: ts=2026-09-20T13:35:06Z level=info event=scheduler_restarting5944worker1 # [ 103.676402] nix-grpc-daemon[1104]: ts=2026-09-20T13:35:06Z level=info event=scheduler_connected addr=lb:500515945worker1 # [ 103.680180] nix-grpc-daemon[1104]: ts=2026-09-20T13:35:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5946worker2 # [ 103.652843] nix-grpc-daemon[1270]: ts=2026-09-20T13:35:06Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5947worker2 # [ 103.664558] nix-daemon[868]: reaped child process 1319, status = succeeded5948worker2 # [ 103.783559] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5949worker2 # [ 103.787592] systemd[1]: Stopped Nix worker-protocol over gRPC.5950worker2 # [ 103.789745] systemd[1]: nix-grpc-daemon.service: Consumed 1.386s CPU time over 27.568s wall clock time, 33.4M memory peak, 63.1K incoming IP traffic, 169.7K outgoing IP traffic.5951worker2 # [ 103.804898] systemd[1]: Starting scheduler-token.service...5952worker1 # [ 103.881432] nix-grpc-daemon[1104]: ts=2026-09-20T13:35:07Z level=info event=scheduler_connected addr=lb:500515953worker1 # [ 103.885417] nix-grpc-daemon[1104]: ts=2026-09-20T13:35:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5954worker2 # [ 103.867721] systemd[1]: scheduler-token.service: Deactivated successfully.5955worker2 # [ 103.869433] systemd[1]: Finished scheduler-token.service.5956worker2 # [ 103.870782] systemd[1]: scheduler-token.service: Consumed 22ms CPU time over 64ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5957worker2 # [ 103.875245] systemd[1]: Starting Nix worker-protocol over gRPC...5958worker2 # [ 103.999642] nix-grpc-daemon[1546]: ts=2026-09-20T13:35:07Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5959worker2 # [ 104.001301] nix-grpc-daemon[1546]: ts=2026-09-20T13:35:07Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5960worker2 # [ 104.004575] nix-grpc-daemon[1546]: ts=2026-09-20T13:35:07Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15961worker2 # [ 104.011451] nix-grpc-daemon[1546]: ts=2026-09-20T13:35:07Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5962worker2 # [ 104.013078] systemd[1]: Started Nix worker-protocol over gRPC.5963worker2 # [ 104.026293] nix-grpc-daemon[1546]: ts=2026-09-20T13:35:07Z level=info event=scheduler_connected addr=lb:500515964worker2 # [ 104.027935] nix-grpc-daemon[1546]: ts=2026-09-20T13:35:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5965worker1 # [ 104.085802] nix-grpc-daemon[1104]: ts=2026-09-20T13:35:07Z level=info event=scheduler_connected addr=lb:500515966worker1 # [ 104.087628] nix-grpc-daemon[1104]: ts=2026-09-20T13:35:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5967worker2 # [ 104.229255] nix-grpc-daemon[1546]: ts=2026-09-20T13:35:07Z level=info event=scheduler_connected addr=lb:500515968worker2 # [ 104.233172] nix-grpc-daemon[1546]: ts=2026-09-20T13:35:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5969worker1 # [ 104.288147] nix-grpc-daemon[1104]: ts=2026-09-20T13:35:07Z level=info event=scheduler_connected addr=lb:500515970worker1 # [ 104.289931] nix-grpc-daemon[1104]: ts=2026-09-20T13:35:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5971lb: must succeed: curl -sf localhost:9901/clusters5972lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5973worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5974worker2 # [ 104.434267] nix-grpc-daemon[1546]: ts=2026-09-20T13:35:07Z level=info event=scheduler_connected addr=lb:500515975worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5976worker1 # [ 104.475928] nix-grpc-daemon[1104]: ts=2026-09-20T13:35:07Z level=info event=oidc_jwks provider=mock keys=15977worker1 # [ 104.478737] nix-grpc-daemon[1104]: ts=2026-09-20T13:35:07Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:540685978worker1 # [ 104.481840] nix-grpc-daemon[1104]: ts=2026-09-20T13:35:07Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05979worker1 # [ 104.490252] nix-grpc-daemon[1104]: ts=2026-09-20T13:35:07Z level=info event=scheduler_connected addr=lb:500515980worker1 # [ 104.491958] nix-grpc-daemon[1104]: ts=2026-09-20T13:35:07Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:540685981worker1 # [ 104.493489] nix-grpc-daemon[1104]: ts=2026-09-20T13:35:07Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05982lb: must succeed: curl -sf localhost:9901/clusters5983lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5984worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5985worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5986(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 11.63 seconds)5987subtest: scheduler restart mid-build: build finishes, no double build5988client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid5989client # [ 105.654866] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid.5990client # Running as unit: mid.service; invocation ID: 4b9c555805394d00b4904bae102f11675991client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid, in 0.07 seconds)5992worker1 # [ 105.835645] nix-grpc-daemon[1104]: ts=2026-09-20T13:35:09Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:540745993worker1 # [ 105.846160] nix-daemon[845]: accepted connection from pid 1104, user nix-grpc-daemon (trusted)5994client # [ 105.876360] nix[1211]: don't know how to build these paths:5995client # [ 105.877208] nix[1211]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5996worker1 # [ 105.866876] nix-grpc-daemon[1104]: ts=2026-09-20T13:35:09Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05997worker1 # [ 105.871324] nix-grpc-daemon[1104]: ts=2026-09-20T13:35:09Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=15998lb # [ 105.895788] envoy[891]: conn peer=[2001:db8:1::1]:45476 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2930 tx=1309 ms=82085999lb # [ 105.897332] envoy[891]: conn peer=[2001:db8:1::4]:59280 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=41615 tx=2209 ms=272436000worker2 # [ 105.895897] nix-grpc-daemon[1546]: ts=2026-09-20T13:35:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372726001worker2 # [ 105.907148] nix-daemon[868]: accepted connection from pid 1546, user nix-grpc-daemon (trusted)6002worker2 # [ 105.923748] nix-grpc-daemon[1546]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 28 ms (attempt 1/5)6003worker2 # [ 105.957624] nix-grpc-daemon[1546]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 165 ms (attempt 2/5)6004worker2 # [ 106.123081] nix-grpc-daemon[1546]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 398 ms (attempt 3/5)6005worker2 # [ 106.526781] nix-grpc-daemon[1546]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 1 ms (attempt 4/5)6006worker2 # [ 106.533950] nix-grpc-daemon[1546]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6007worker2 # [ 106.550397] nix-grpc-daemon[1546]: ts=2026-09-20T13:35:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:372726008client # [ 106.610089] nix[1211]: copying 1 paths to 'lb:50051'...6009client # [ 106.614128] nix[1211]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...6010worker2 # [ 106.564755] nix-grpc-daemon[1546]: ts=2026-09-20T13:35:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:372726011worker2 # [ 106.573344] nix-daemon[868]: accepted connection from pid 1546, user nix-grpc-daemon (trusted)6012worker2 # [ 106.593684] nix-grpc-daemon[1546]: ts=2026-09-20T13:35:09Z level=info event=niks3_push_started pid=16066013worker2 # [ 106.776849] nix-grpc-daemon[1606]: time=2026-09-20T13:35:10.110Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""6014worker2 # [ 106.929664] nix-daemon[868]: accepted connection from pid 1623, user nix-grpc-daemon (trusted)6015worker2 # [ 106.940077] nix-daemon[868]: reaped child process 1626, status = succeeded6016worker2 # [ 107.093167] nix-daemon[868]: accepted connection from pid 1628, user nix-grpc-daemon (trusted)6017worker2 # [ 107.104975] nix-daemon[868]: reaped child process 1631, status = succeeded6018lb # [ 107.144812] niks3-server[1058]: 2026/09/20 13:35:10 INFO Received uploads request method=POST path=/api/pending_closures6019worker2 # [ 107.127983] nix-grpc-daemon[1606]: time=2026-09-20T13:35:10.462Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6020worker2 # [ 107.129616] nix-grpc-daemon[1606]: time=2026-09-20T13:35:10.464Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"6021lb # [ 107.175832] niks3-server[1058]: 2026/09/20 13:35:10 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst6022lb # [ 107.186264] niks3-server[1058]: 2026/09/20 13:35:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign6023lb # [ 107.196259] niks3-server[1058]: 2026/09/20 13:35:10 INFO Signed narinfos id=35 count=16024worker2 # [ 107.173964] nix-grpc-daemon[1606]: time=2026-09-20T13:35:10.508Z level=INFO msg="Uploading 1 narinfos"6025lb # [ 107.215267] niks3-server[1058]: 2026/09/20 13:35:10 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete6026lb # [ 107.221973] niks3-server[1058]: 2026/09/20 13:35:10 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls6027lb # [ 107.225749] niks3-server[1058]: 2026/09/20 13:35:10 INFO Completed upload id=356028worker2 # [ 107.200964] nix-grpc-daemon[1606]: time=2026-09-20T13:35:10.535Z level=INFO msg="Upload complete. (421ms)"6029worker2 # [ 107.205501] nix-grpc-daemon[1546]: ts=2026-09-20T13:35:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37272 duration_s=0 paths=1 nar_bytes_in=4246030worker2 # [ 107.211421] nix-grpc-daemon[1606]: time=2026-09-20T13:35:10.538Z level=WARN msg="Failed to register uploaded object" key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo error="server returned 404: object is not pending upload\n"6031worker2 # [ 107.216289] nix-grpc-daemon[1546]: ts=2026-09-20T13:35:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372726032worker2 # [ 107.222255] nix-daemon[868]: accepted connection from pid 1546, user nix-grpc-daemon (trusted)6033worker2 # [ 107.227224] nix-daemon[868]: reaped child process 1604, status = succeeded6034worker2 # [ 107.232423] nix-daemon[868]: accepted connection from pid 1546, user nix-grpc-daemon (trusted)6035client # [ 107.281873] nix[1211]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6036lb: must succeed: curl -sf localhost:9901/clusters6037lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6038lb: must succeed: curl -sf localhost:9901/clusters6039lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6040worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6041worker1 # [ 109.327655] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1104 (nix-grpc-daemon) on client request.6042lb # [ 109.330667] niks3-server[1058]: 2026/09/20 13:35:12 INFO lead: released remote=192.168.1.3:551226043worker1 # [ 109.339503] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6044worker2 # [ 109.310374] nix-grpc-daemon[1546]: ts=2026-09-20T13:35:12Z level=info event=scheduler_disconnected addr=lb:50051 error="Stream removed (RST_STREAM (Received RST_STREAM with error code 2))"6045client # [ 109.361658] nix[1211]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6046worker1 # [ 109.364298] nix-daemon[845]: reaped child process 1207, status = succeeded6047worker1 # [ 109.366407] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6048worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.08 seconds)6049worker1 # [ 109.367891] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6050client: must succeed: sleep 5.06051worker1 # [ 109.370280] systemd[1]: nix-grpc-daemon.service: Consumed 178ms CPU time over 15.891s wall clock time, 11.1M memory peak, 12.2K incoming IP traffic, 19.7K outgoing IP traffic.6052worker1 # [ 109.496617] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.6053worker1 # [ 109.506711] systemd[1]: Starting Nix worker-protocol over gRPC...6054worker2 # [ 109.510599] nix-grpc-daemon[1546]: ts=2026-09-20T13:35:12Z level=info event=scheduler_connected addr=lb:500516055worker1 # [ 109.682411] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:12Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6056worker1 # [ 109.684737] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:12Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6057worker1 # [ 109.688359] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:12Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16058worker1 # [ 109.695682] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:12Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6059worker1 # [ 109.699139] systemd[1]: Started Nix worker-protocol over gRPC.6060worker1 # [ 109.705627] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:12Z level=info event=scheduler_take_over6061lb # [ 109.698964] niks3-server[1058]: 2026/09/20 13:35:13 INFO lead: acquired remote=192.168.1.3:454066062worker1 # [ 109.714052] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:12Z level=info event=oidc_jwks provider=mock keys=16063worker1 # [ 109.716842] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:12Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:540846064worker1 # [ 109.718490] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:12Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=16065worker1 # [ 109.721956] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:12Z level=info event=scheduler_connected addr=lb:500516066worker1 # [ 109.723560] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:12Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:540846067worker1 # [ 109.724998] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:12Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06068client: (finished: must succeed: sleep 5.0, in 5.04 seconds)6069client: waiting for success: ! systemctl is-active mid6070worker2 # [ 114.652348] nix-daemon[868]: reaped child process 1635, status = succeeded6071worker2 # [ 114.789454] nix-daemon[868]: accepted connection from pid 1671, user nix-grpc-daemon (trusted)6072worker2 # [ 114.800090] nix-daemon[868]: reaped child process 1674, status = succeeded6073lb # [ 114.840951] niks3-server[1058]: 2026/09/20 13:35:18 INFO Received uploads request method=POST path=/api/pending_closures6074worker2 # [ 114.823846] nix-grpc-daemon[1606]: time=2026-09-20T13:35:18.158Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6075worker2 # [ 114.825350] nix-grpc-daemon[1606]: time=2026-09-20T13:35:18.159Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6076lb # [ 114.875734] niks3-server[1058]: 2026/09/20 13:35:18 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6077lb # [ 114.890145] niks3-server[1058]: 2026/09/20 13:35:18 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6078lb # [ 114.903281] niks3-server[1058]: 2026/09/20 13:35:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign6079lb # [ 114.906914] niks3-server[1058]: 2026/09/20 13:35:18 INFO Signed narinfos id=36 count=16080worker2 # [ 114.882865] nix-grpc-daemon[1606]: time=2026-09-20T13:35:18.216Z level=INFO msg="Uploading 1 narinfos"6081lb # [ 114.911830] niks3-server[1058]: 2026/09/20 13:35:18 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6082lb # [ 114.920654] niks3-server[1058]: 2026/09/20 13:35:18 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete6083lb # [ 114.923656] niks3-server[1058]: 2026/09/20 13:35:18 INFO Completed upload id=366084worker2 # [ 114.898774] nix-grpc-daemon[1606]: time=2026-09-20T13:35:18.233Z level=INFO msg="Upload complete. (253ms)"6085lb # [ 114.926048] niks3-server[1058]: 2026/09/20 13:35:18 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo6086worker2 # [ 114.903412] nix-grpc-daemon[1546]: ts=2026-09-20T13:35:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37272 duration_s=7 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16087worker2 # [ 114.914658] nix-daemon[868]: reaped child process 1633, status = succeeded6088client # [ 115.002850] systemd[1]: mid.service: Deactivated successfully.6089client # [ 115.007238] systemd[1]: mid.service: Consumed 120ms CPU time over 9.350s wall clock time, 39.3M memory peak, 8.8K incoming IP traffic, 9.2K outgoing IP traffic.6090client: (finished: waiting for success: ! systemctl is-active mid, in 1.09 seconds)6091client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6092client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.03 seconds)6093(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.16 seconds)6094subtest: interrupting the client stops the build on the worker6095client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr6096client # [ 115.786308] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr.6097client # Running as unit: intr.service; invocation ID: 8d24261c019549cfa333d8081f7be44c6098client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr, in 0.04 seconds)6099lb # [ 115.896506] envoy[891]: conn peer=[2001:db8:1::3]:40836 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2343 tx=602 ms=156916100lb # [ 115.900522] envoy[891]: conn peer=[2001:db8:1::1]:54884 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3018 tx=1425 ms=91296101worker2 # [ 115.924511] nix-grpc-daemon[1546]: ts=2026-09-20T13:35:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:372726102client # [ 115.980103] nix[1247]: don't know how to build these paths:6103client # [ 115.980978] nix[1247]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6104worker1 # [ 115.972858] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:19Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06105worker1 # [ 115.976803] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:19Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.3:50051 clients=16106worker1 # [ 116.019160] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453466107worker1 # [ 116.030918] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6108worker1 # [ 116.047482] nix-grpc-daemon[1235]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 37 ms (attempt 1/5)6109worker1 # [ 116.089969] nix-grpc-daemon[1235]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 79 ms (attempt 2/5)6110worker1 # [ 116.174857] nix-grpc-daemon[1235]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 272 ms (attempt 3/5)6111worker1 # [ 116.452803] nix-grpc-daemon[1235]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 572 ms (attempt 4/5)6112worker1 # [ 117.031611] nix-grpc-daemon[1235]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6113worker1 # [ 117.049671] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453466114client # [ 117.077586] nix[1247]: copying 1 paths to 'lb:50051'...6115client # [ 117.081128] nix[1247]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6116worker1 # [ 117.063623] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:453466117worker1 # [ 117.074558] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6118worker1 # [ 117.090489] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:20Z level=info event=niks3_push_started pid=13096119worker1 # [ 117.264980] nix-grpc-daemon[1309]: time=2026-09-20T13:35:20.478Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""6120worker1 # [ 117.416146] nix-daemon[845]: accepted connection from pid 1322, user nix-grpc-daemon (trusted)6121worker1 # [ 117.428536] nix-daemon[845]: reaped child process 1325, status = succeeded6122worker1 # [ 117.580261] nix-daemon[845]: accepted connection from pid 1327, user nix-grpc-daemon (trusted)6123worker1 # [ 117.592053] nix-daemon[845]: reaped child process 1330, status = succeeded6124lb # [ 117.600230] niks3-server[1058]: 2026/09/20 13:35:20 INFO Received uploads request method=POST path=/api/pending_closures6125worker1 # [ 117.614366] nix-grpc-daemon[1309]: time=2026-09-20T13:35:20.828Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6126worker1 # [ 117.617365] nix-grpc-daemon[1309]: time=2026-09-20T13:35:20.829Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6127lb # [ 117.630188] niks3-server[1058]: 2026/09/20 13:35:20 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6128lb # [ 117.639866] niks3-server[1058]: 2026/09/20 13:35:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign6129lb # [ 117.644384] niks3-server[1058]: 2026/09/20 13:35:20 INFO Signed narinfos id=37 count=16130worker1 # [ 117.654145] nix-grpc-daemon[1309]: time=2026-09-20T13:35:20.867Z level=INFO msg="Uploading 1 narinfos"6131lb # [ 117.655579] niks3-server[1058]: 2026/09/20 13:35:20 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6132lb # [ 117.662218] niks3-server[1058]: 2026/09/20 13:35:20 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete6133worker1 # [ 117.675514] nix-grpc-daemon[1309]: time=2026-09-20T13:35:20.889Z level=INFO msg="Upload complete. (407ms)"6134lb # [ 117.668847] niks3-server[1058]: 2026/09/20 13:35:20 INFO Completed upload id=376135worker1 # [ 117.677694] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45346 duration_s=0 paths=1 nar_bytes_in=4246136lb # [ 117.674223] niks3-server[1058]: 2026/09/20 13:35:20 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6137worker1 # [ 117.681457] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453466138worker1 # [ 117.690072] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6139worker1 # [ 117.691906] nix-daemon[845]: reaped child process 1307, status = succeeded6140worker1 # [ 117.696868] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6141client # [ 117.714112] nix[1247]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6142client: must succeed: systemctl kill -s INT intr6143worker1 # [ 119.243822] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:22Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6144client # [ 119.261088] systemd[1]: intr.service: Sent signal SIGINT to main process 1247 (nix) on client request.6145client # [ 119.265786] nix[1247]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6146client: (finished: must succeed: systemctl kill -s INT intr, in 0.05 seconds)6147client # [ 119.272694] nix[1247]: error: interrupted by the user6148client # [ 119.302819] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6149client # [ 119.304278] systemd[1]: intr.service: Failed with result 'exit-code'.6150client # [ 119.306235] systemd[1]: intr.service: Consumed 102ms CPU time over 3.519s wall clock time, 39.3M memory peak, 7.7K incoming IP traffic, 8.7K outgoing IP traffic.6151worker1 # [ 119.316278] nix-daemon[845]: reaped child process 1332, status = succeeded6152worker1 # [ 119.319654] nix-daemon[1334]: unexpected Nix daemon error: error: interrupted by the user6153worker1 # [ 119.324136] nix-daemon[845]: reaped child process 1334, status = failed with exit code 16154(finished: subtest: interrupting the client stops the build on the worker, in 4.75 seconds)6155subtest: a deploy mid-build drains: the build finishes, the new generation takes over6156client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw6157client # [ 120.535170] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw.6158client # Running as unit: sw.service; invocation ID: 5151fed5ac334ab5897c4a2b343507076159client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw, in 0.04 seconds)6160worker2 # [ 120.671083] nix-grpc-daemon[1546]: ts=2026-09-20T13:35:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:372726161client # [ 120.723513] nix[1268]: don't know how to build these paths:6162client # [ 120.724862] nix[1268]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6163worker1 # [ 120.714622] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:23Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06164worker1 # [ 120.719332] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:23Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.4:50051 clients=16165worker2 # [ 120.731447] nix-grpc-daemon[1546]: ts=2026-09-20T13:35:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372726166worker2 # [ 120.744090] nix-grpc-daemon[1546]: ts=2026-09-20T13:35:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:372726167client # [ 120.804402] nix[1268]: copying 1 paths to 'lb:50051'...6168worker2 # [ 120.755817] nix-grpc-daemon[1546]: ts=2026-09-20T13:35:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:372726169client # [ 120.807141] nix[1268]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6170worker2 # [ 120.768622] nix-daemon[868]: accepted connection from pid 1546, user nix-grpc-daemon (trusted)6171worker2 # [ 120.948659] nix-daemon[868]: accepted connection from pid 1706, user nix-grpc-daemon (trusted)6172worker2 # [ 120.961592] nix-daemon[868]: reaped child process 1709, status = succeeded6173worker2 # [ 121.123091] nix-daemon[868]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)6174worker2 # [ 121.133901] nix-daemon[868]: reaped child process 1714, status = succeeded6175lb # [ 121.177352] niks3-server[1058]: 2026/09/20 13:35:24 INFO Received uploads request method=POST path=/api/pending_closures6176worker2 # [ 121.161433] nix-grpc-daemon[1606]: time=2026-09-20T13:35:24.495Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6177worker2 # [ 121.163754] nix-grpc-daemon[1606]: time=2026-09-20T13:35:24.497Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6178lb # [ 121.206072] niks3-server[1058]: 2026/09/20 13:35:24 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6179lb # [ 121.217959] niks3-server[1058]: 2026/09/20 13:35:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6180lb # [ 121.222829] niks3-server[1058]: 2026/09/20 13:35:24 INFO Signed narinfos id=38 count=16181worker2 # [ 121.200921] nix-grpc-daemon[1606]: time=2026-09-20T13:35:24.535Z level=INFO msg="Uploading 1 narinfos"6182lb # [ 121.232506] niks3-server[1058]: 2026/09/20 13:35:24 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6183lb # [ 121.240225] niks3-server[1058]: 2026/09/20 13:35:24 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6184lb # [ 121.246751] niks3-server[1058]: 2026/09/20 13:35:24 INFO Completed upload id=386185worker2 # [ 121.221796] nix-grpc-daemon[1606]: time=2026-09-20T13:35:24.556Z level=INFO msg="Upload complete. (442ms)"6186worker2 # [ 121.223861] nix-grpc-daemon[1546]: ts=2026-09-20T13:35:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37272 duration_s=0 paths=1 nar_bytes_in=4166187lb # [ 121.249499] niks3-server[1058]: 2026/09/20 13:35:24 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6188worker2 # [ 121.227329] nix-grpc-daemon[1546]: ts=2026-09-20T13:35:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372726189worker2 # [ 121.234124] nix-daemon[868]: accepted connection from pid 1546, user nix-grpc-daemon (trusted)6190worker2 # [ 121.238255] nix-daemon[868]: reaped child process 1704, status = succeeded6191worker2 # [ 121.241985] nix-daemon[868]: accepted connection from pid 1546, user nix-grpc-daemon (trusted)6192client # [ 121.291174] nix[1268]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6193worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&26194worker2 # Checking switch inhibitors... done6195worker2 # [ 123.551829] nixos[1749]: switching to system configuration /nix/store/ffandj11jlfl27z7s4hy1jr045fm5clm-nixos-system-worker2-test6196worker2 # activating the configuration...6197worker2 # [ 123.555389] systemd[1]: Stopped target Remote File Systems.6198worker2 # [ 123.556272] systemd[1]: Stopped target Local File Systems.6199worker2 # setting up /etc...6200worker2 # [ 124.158442] systemd[1]: Reload requested from client PID 1749 ('.switch-to-conf') (unit backdoor.service)...6201worker2 # [ 124.160461] systemd[1]: Reloading...6202worker2 # [ 124.809138] systemd[1]: Reloading finished in 645 ms.6203worker2 # restarting sysinit-reactivation.target6204worker2 # [ 124.863976] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....6205worker2 # [ 124.993676] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..6206worker2 # [ 124.997691] systemd[1]: Reached target Reactivate sysinit units.6207worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service6208worker2 # [ 125.001958] systemd[1]: Reloading D-Bus System Message Bus...6209worker2 # [ 125.002969] dbus-broker-launch[534]: Caught SIGHUP, trigger reload.6210worker2 # [ 125.008190] systemd[1]: Reloading Nix worker-protocol over gRPC...6211worker2 # [ 125.038413] dbus-broker-launch[534]: Looking up NSS user entry for 'systemd-timesync'...6212worker2 # [ 125.040869] dbus-broker-launch[534]: NSS returned no entry for 'systemd-timesync'6213worker2 # [ 125.042079] dbus-broker-launch[534]: Invalid user-name in /nix/store/nl02z4vnl25dc1df60lqifjnw6c89zpg-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"6214worker2 # [ 125.052708] systemd[1]: Reloaded Nix worker-protocol over gRPC.6215worker2 # [ 125.071077] systemd[1]: Reloaded D-Bus System Message Bus.6216worker2 # [ 125.078692] systemd[1]: Reached target Remote File Systems.6217worker2 # [ 125.081311] systemd[1]: Starting linger-users.service...6218worker2 # [ 125.085056] systemd[1]: Reached target Local File Systems.6219worker2 # [ 125.092340] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore6220worker2 # [ 125.098854] systemd[1]: Starting Create SUID/SGID Wrappers...6221worker2 # [ 125.099904] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.6222worker2 # [ 125.101674] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc676223worker2 # [ 125.106305] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore6224worker2 # [ 125.107802] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6225worker2 # [ 125.110809] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6226worker2 # [ 125.114361] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os6227worker2 # [ 125.115621] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki6228worker2 # [ 125.152550] systemd[1]: linger-users.service: Deactivated successfully.6229worker2 # [ 125.153805] systemd[1]: Finished linger-users.service.6230worker2 # [ 125.521579] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.6231worker2 # [ 125.522882] systemd[1]: Finished Create SUID/SGID Wrappers.6232worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service6233worker2 # [ 125.781616] nixos[1749]: finished switching to system configuration /nix/store/ffandj11jlfl27z7s4hy1jr045fm5clm-nixos-system-worker2-test6234worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 2.86 seconds)6235worker2: must succeed: pgrep -f 'read -t [9]0 x'6236worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)6237client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&26238lb # [ 125.897838] envoy[891]: conn peer=[2001:db8:1::1]:42276 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2924 tx=754 ms=33126239client # don't know how to build these paths:6240client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6241worker1 # [ 126.039590] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:29Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:453466242worker2 # [ 126.012261] nix-grpc-daemon[1546]: ts=2026-09-20T13:35:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:372726243worker2 # [ 126.019308] nix-grpc-daemon[1546]: ts=2026-09-20T13:35:29Z level=info event=draining6244worker1 # [ 126.062164] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:29Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06245worker1 # [ 126.065992] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:29Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.3:50051 clients=16246worker1 # [ 126.115648] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453466247worker1 # [ 126.126572] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453466248client # copying 1 paths to 'lb:50051'...6249worker1 # [ 126.137288] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:453466250client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6251worker1 # [ 126.167731] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6252worker1 # [ 126.347860] nix-daemon[845]: accepted connection from pid 1388, user nix-grpc-daemon (trusted)6253worker1 # [ 126.361508] nix-daemon[845]: reaped child process 1391, status = succeeded6254worker1 # [ 126.506970] nix-daemon[845]: accepted connection from pid 1393, user nix-grpc-daemon (trusted)6255worker1 # [ 126.519245] nix-daemon[845]: reaped child process 1396, status = succeeded6256lb # [ 126.527396] niks3-server[1058]: 2026/09/20 13:35:29 INFO Received uploads request method=POST path=/api/pending_closures6257worker1 # [ 126.544540] nix-grpc-daemon[1309]: time=2026-09-20T13:35:29.758Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6258worker1 # [ 126.546295] nix-grpc-daemon[1309]: time=2026-09-20T13:35:29.759Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6259lb # [ 126.560115] niks3-server[1058]: 2026/09/20 13:35:29 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6260lb # [ 126.568139] niks3-server[1058]: 2026/09/20 13:35:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6261lb # [ 126.572962] niks3-server[1058]: 2026/09/20 13:35:29 INFO Signed narinfos id=39 count=16262worker1 # [ 126.582674] nix-grpc-daemon[1309]: time=2026-09-20T13:35:29.796Z level=INFO msg="Uploading 1 narinfos"6263lb # [ 126.587770] niks3-server[1058]: 2026/09/20 13:35:29 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6264lb # [ 126.592398] niks3-server[1058]: 2026/09/20 13:35:29 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6265lb # [ 126.601487] niks3-server[1058]: 2026/09/20 13:35:29 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6266lb # [ 126.606651] niks3-server[1058]: 2026/09/20 13:35:29 INFO Completed upload id=396267worker1 # [ 126.613641] nix-grpc-daemon[1309]: time=2026-09-20T13:35:29.827Z level=INFO msg="Upload complete. (430ms)"6268worker1 # [ 126.617531] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45346 duration_s=0 paths=1 nar_bytes_in=5286269worker1 # [ 126.622412] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453466270worker1 # [ 126.631195] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6271worker1 # [ 126.639253] nix-daemon[845]: reaped child process 1386, status = succeeded6272worker1 # [ 126.644757] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6273client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6274client # farm-a> node-a: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6275client # farm-a> LOG-farm-a-during-drain6276worker1 # [ 127.871261] nix-daemon[845]: reaped child process 1400, status = succeeded6277worker1 # [ 128.019937] nix-daemon[845]: accepted connection from pid 1415, user nix-grpc-daemon (trusted)6278worker1 # [ 128.031779] nix-daemon[845]: reaped child process 1418, status = succeeded6279lb # [ 128.038664] niks3-server[1058]: 2026/09/20 13:35:31 INFO Received uploads request method=POST path=/api/pending_closures6280worker1 # [ 128.055734] nix-grpc-daemon[1309]: time=2026-09-20T13:35:31.269Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6281worker1 # [ 128.057201] nix-grpc-daemon[1309]: time=2026-09-20T13:35:31.270Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6282lb # [ 128.069708] niks3-server[1058]: 2026/09/20 13:35:31 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6283lb # [ 128.097758] niks3-server[1058]: 2026/09/20 13:35:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6284lb # [ 128.104102] niks3-server[1058]: 2026/09/20 13:35:31 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6285lb # [ 128.114771] niks3-server[1058]: 2026/09/20 13:35:31 INFO Signed narinfos id=40 count=16286worker1 # [ 128.121228] nix-grpc-daemon[1309]: time=2026-09-20T13:35:31.334Z level=INFO msg="Uploading 1 narinfos"6287lb # [ 128.125991] niks3-server[1058]: 2026/09/20 13:35:31 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6288lb # [ 128.129643] niks3-server[1058]: 2026/09/20 13:35:31 INFO Completed upload id=406289lb # [ 128.130857] niks3-server[1058]: 2026/09/20 13:35:31 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6290worker1 # [ 128.136582] nix-grpc-daemon[1309]: time=2026-09-20T13:35:31.350Z level=INFO msg="Upload complete. (282ms)"6291lb # [ 128.133781] niks3-server[1058]: 2026/09/20 13:35:31 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6292worker1 # [ 128.145720] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45346 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=3 outputs=16293worker1 # [ 128.153475] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:31Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06294worker1 # [ 128.157295] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:31Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.3:50051 clients=16295worker1 # [ 128.165556] nix-daemon[845]: reaped child process 1398, status = succeeded6296worker1 # [ 128.197799] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453466297worker1 # [ 128.209382] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453466298worker1 # [ 128.223184] nix-grpc-daemon[1309]: time=2026-09-20T13:35:31.436Z level=INFO msg="All 1 paths already cached"6299client # copying 1 paths to 'lb:50051'...6300client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6301worker1 # [ 128.230828] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:453466302worker1 # [ 128.243276] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6303worker1 # [ 128.408368] nix-daemon[845]: accepted connection from pid 1424, user nix-grpc-daemon (trusted)6304worker1 # [ 128.421522] nix-daemon[845]: reaped child process 1427, status = succeeded6305worker1 # [ 128.569261] nix-daemon[845]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6306worker1 # [ 128.581109] nix-daemon[845]: reaped child process 1432, status = succeeded6307lb # [ 128.587902] niks3-server[1058]: 2026/09/20 13:35:31 INFO Received uploads request method=POST path=/api/pending_closures6308worker1 # [ 128.605089] nix-grpc-daemon[1309]: time=2026-09-20T13:35:31.818Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6309worker1 # [ 128.606631] nix-grpc-daemon[1309]: time=2026-09-20T13:35:31.820Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6310lb # [ 128.624156] niks3-server[1058]: 2026/09/20 13:35:31 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6311lb # [ 128.630350] niks3-server[1058]: 2026/09/20 13:35:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6312worker1 # [ 128.642636] nix-grpc-daemon[1309]: time=2026-09-20T13:35:31.856Z level=INFO msg="Uploading 1 narinfos"6313lb # [ 128.636262] niks3-server[1058]: 2026/09/20 13:35:31 INFO Signed narinfos id=41 count=16314lb # [ 128.642834] niks3-server[1058]: 2026/09/20 13:35:31 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6315lb # [ 128.653172] niks3-server[1058]: 2026/09/20 13:35:31 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6316worker1 # [ 128.669543] nix-grpc-daemon[1309]: time=2026-09-20T13:35:31.883Z level=INFO msg="Upload complete. (413ms)"6317lb # [ 128.662827] niks3-server[1058]: 2026/09/20 13:35:31 INFO Completed upload id=416318lb # [ 128.666053] niks3-server[1058]: 2026/09/20 13:35:31 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6319worker1 # [ 128.672625] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45346 duration_s=0 paths=1 nar_bytes_in=6406320worker1 # [ 128.678284] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453466321worker1 # [ 128.686247] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6322worker1 # [ 128.693378] nix-daemon[845]: reaped child process 1422, status = succeeded6323client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6324worker1 # [ 128.699290] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6325client # farm-b> node-a: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6326client # farm-b> LOG-farm-b-during-drain6327worker1 # [ 129.524460] nix-daemon[845]: reaped child process 1436, status = succeeded6328worker1 # [ 129.666203] nix-daemon[845]: accepted connection from pid 1451, user nix-grpc-daemon (trusted)6329worker1 # [ 129.678470] nix-daemon[845]: reaped child process 1454, status = succeeded6330lb # [ 129.686508] niks3-server[1058]: 2026/09/20 13:35:33 INFO Received uploads request method=POST path=/api/pending_closures6331worker1 # [ 129.700705] nix-grpc-daemon[1309]: time=2026-09-20T13:35:32.914Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6332worker1 # [ 129.702222] nix-grpc-daemon[1309]: time=2026-09-20T13:35:32.915Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6333lb # [ 129.725150] niks3-server[1058]: 2026/09/20 13:35:33 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6334lb # [ 129.731850] niks3-server[1058]: 2026/09/20 13:35:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6335lb # [ 129.737104] niks3-server[1058]: 2026/09/20 13:35:33 INFO Signed narinfos id=42 count=16336worker1 # [ 129.746781] nix-grpc-daemon[1309]: time=2026-09-20T13:35:32.960Z level=INFO msg="Uploading 1 narinfos"6337lb # [ 129.745903] niks3-server[1058]: 2026/09/20 13:35:33 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6338lb # [ 129.751454] niks3-server[1058]: 2026/09/20 13:35:33 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6339lb # [ 129.759881] niks3-server[1058]: 2026/09/20 13:35:33 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6340lb # [ 129.764917] niks3-server[1058]: 2026/09/20 13:35:33 INFO Completed upload id=426341worker1 # [ 129.771820] nix-grpc-daemon[1309]: time=2026-09-20T13:35:32.985Z level=INFO msg="Upload complete. (266ms)"6342lb # [ 129.767328] niks3-server[1058]: 2026/09/20 13:35:33 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6343worker1 # [ 129.778782] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45346 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=4 outputs=16344worker1 # [ 129.791382] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:33Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06345worker1 # [ 129.795211] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:33Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.3:50051 clients=16346worker1 # [ 129.803552] nix-daemon[845]: reaped child process 1434, status = succeeded6347worker1 # [ 129.834316] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453466348worker1 # [ 129.846668] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453466349worker1 # [ 129.860080] nix-grpc-daemon[1309]: time=2026-09-20T13:35:33.073Z level=INFO msg="All 2 paths already cached"6350client # copying 1 paths to 'lb:50051'...6351client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6352worker1 # [ 129.867986] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:453466353worker1 # [ 129.880316] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6354worker1 # [ 130.044384] nix-daemon[845]: accepted connection from pid 1460, user nix-grpc-daemon (trusted)6355worker1 # [ 130.056901] nix-daemon[845]: reaped child process 1463, status = succeeded6356worker1 # [ 130.205660] nix-daemon[845]: accepted connection from pid 1465, user nix-grpc-daemon (trusted)6357worker1 # [ 130.217778] nix-daemon[845]: reaped child process 1468, status = succeeded6358lb # [ 130.222275] niks3-server[1058]: 2026/09/20 13:35:33 INFO Received uploads request method=POST path=/api/pending_closures6359worker1 # [ 130.233550] nix-grpc-daemon[1309]: time=2026-09-20T13:35:33.447Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6360worker1 # [ 130.234878] nix-grpc-daemon[1309]: time=2026-09-20T13:35:33.447Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6361lb # [ 130.235609] niks3-server[1058]: 2026/09/20 13:35:33 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6362lb # [ 130.239676] niks3-server[1058]: 2026/09/20 13:35:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6363lb # [ 130.242465] niks3-server[1058]: 2026/09/20 13:35:33 INFO Signed narinfos id=43 count=16364worker1 # [ 130.249226] nix-grpc-daemon[1309]: time=2026-09-20T13:35:33.462Z level=INFO msg="Uploading 1 narinfos"6365lb # [ 130.246206] niks3-server[1058]: 2026/09/20 13:35:33 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6366lb # [ 130.250543] niks3-server[1058]: 2026/09/20 13:35:33 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6367lb # [ 130.254869] niks3-server[1058]: 2026/09/20 13:35:33 INFO Completed upload id=436368worker1 # [ 130.261537] nix-grpc-daemon[1309]: time=2026-09-20T13:35:33.475Z level=INFO msg="Upload complete. (367ms)"6369lb # [ 130.256550] niks3-server[1058]: 2026/09/20 13:35:33 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6370worker1 # [ 130.263458] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45346 duration_s=0 paths=1 nar_bytes_in=7686371worker1 # [ 130.267391] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453466372worker1 # [ 130.273065] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6373worker1 # [ 130.275363] nix-daemon[845]: reaped child process 1458, status = succeeded6374worker1 # [ 130.280555] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6375client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6376client # farm-top> node-a: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6377client # farm-top> LOG-farm-top-during-drain6378worker1 # [ 131.909224] nix-daemon[845]: reaped child process 1472, status = succeeded6379worker1 # [ 132.042592] nix-daemon[845]: accepted connection from pid 1487, user nix-grpc-daemon (trusted)6380worker1 # [ 132.054294] nix-daemon[845]: reaped child process 1490, status = succeeded6381lb # [ 132.062104] niks3-server[1058]: 2026/09/20 13:35:35 INFO Received uploads request method=POST path=/api/pending_closures6382worker1 # [ 132.072579] nix-grpc-daemon[1309]: time=2026-09-20T13:35:35.286Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6383worker1 # [ 132.075445] nix-grpc-daemon[1309]: time=2026-09-20T13:35:35.287Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6384lb # [ 132.078057] niks3-server[1058]: 2026/09/20 13:35:35 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6385lb # [ 132.082055] niks3-server[1058]: 2026/09/20 13:35:35 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6386lb # [ 132.085108] niks3-server[1058]: 2026/09/20 13:35:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6387lb # [ 132.087849] niks3-server[1058]: 2026/09/20 13:35:35 INFO Signed narinfos id=44 count=16388worker1 # [ 132.094745] nix-grpc-daemon[1309]: time=2026-09-20T13:35:35.308Z level=INFO msg="Uploading 1 narinfos"6389lb # [ 132.091249] niks3-server[1058]: 2026/09/20 13:35:35 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6390lb # [ 132.094796] niks3-server[1058]: 2026/09/20 13:35:35 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6391lb # [ 132.098673] niks3-server[1058]: 2026/09/20 13:35:35 INFO Completed upload id=446392worker1 # [ 132.105276] nix-grpc-daemon[1309]: time=2026-09-20T13:35:35.319Z level=INFO msg="Upload complete. (215ms)"6393lb # [ 132.100477] niks3-server[1058]: 2026/09/20 13:35:35 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6394worker1 # [ 132.107729] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45346 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=5 outputs=16395worker1 # [ 132.114781] nix-daemon[845]: reaped child process 1470, status = succeeded6396client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&2, in 6.29 seconds)6397client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath6398client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath, in 0.21 seconds)6399worker2: must succeed: pkill -f 'read -t [9]0 x'6400worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6401client: waiting for success: ! systemctl is-active sw6402worker2 # [ 132.468118] nix-daemon[868]: reaped child process 1718, status = succeeded6403worker2 # [ 132.608418] nix-daemon[868]: accepted connection from pid 1890, user nix-grpc-daemon (trusted)6404worker2 # [ 132.620287] nix-daemon[868]: reaped child process 1893, status = succeeded6405lb # [ 132.661867] niks3-server[1058]: 2026/09/20 13:35:35 INFO Received uploads request method=POST path=/api/pending_closures6406worker2 # [ 132.644389] nix-grpc-daemon[1606]: time=2026-09-20T13:35:35.978Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6407worker2 # [ 132.645978] nix-grpc-daemon[1606]: time=2026-09-20T13:35:35.980Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6408lb # [ 132.696955] niks3-server[1058]: 2026/09/20 13:35:36 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6409lb # [ 132.706883] niks3-server[1058]: 2026/09/20 13:35:36 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6410lb # [ 132.714492] niks3-server[1058]: 2026/09/20 13:35:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6411lb # [ 132.718621] niks3-server[1058]: 2026/09/20 13:35:36 INFO Signed narinfos id=45 count=16412worker2 # [ 132.696811] nix-grpc-daemon[1606]: time=2026-09-20T13:35:36.031Z level=INFO msg="Uploading 1 narinfos"6413lb # [ 132.726533] niks3-server[1058]: 2026/09/20 13:35:36 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6414lb # [ 132.734456] niks3-server[1058]: 2026/09/20 13:35:36 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6415lb # [ 132.740309] niks3-server[1058]: 2026/09/20 13:35:36 INFO Completed upload id=456416worker2 # [ 132.715323] nix-grpc-daemon[1606]: time=2026-09-20T13:35:36.049Z level=INFO msg="Upload complete. (255ms)"6417lb # [ 132.744361] niks3-server[1058]: 2026/09/20 13:35:36 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6418worker2 # [ 132.723401] nix-grpc-daemon[1546]: ts=2026-09-20T13:35:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37272 duration_s=11 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=2 outputs=16419worker2 # [ 132.735693] nix-daemon[868]: reaped child process 1716, status = succeeded6420client # [ 132.809875] systemd[1]: sw.service: Deactivated successfully.6421client # [ 132.813694] systemd[1]: sw.service: Consumed 110ms CPU time over 12.277s wall clock time, 37.1M memory peak, 8.5K incoming IP traffic, 8.9K outgoing IP traffic.6422worker2 # [ 133.022078] nix-grpc-daemon[1546]: ts=2026-09-20T13:35:36Z level=info event=signal_exit6423client: (finished: waiting for success: ! systemctl is-active sw, in 1.08 seconds)6424client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6425client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds)6426worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6427worker2 # [ 133.571122] systemd[1]: Starting scheduler-token.service...6428worker2 # [ 133.632422] systemd[1]: scheduler-token.service: Deactivated successfully.6429worker2 # [ 133.633965] systemd[1]: Finished scheduler-token.service.6430worker2 # [ 133.635215] systemd[1]: scheduler-token.service: Consumed 18ms CPU time over 64ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6431worker1 # [ 134.290507] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:37Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:540846432worker1 # [ 134.294200] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:37Z level=info event=worker_gone addr=192.168.1.4:500516433worker2 # [ 134.260093] nix-grpc-daemon[1546]: ts=2026-09-20T13:35:37Z level=info event=scheduler_restarting6434worker2 # [ 134.271396] nix-grpc-daemon[1546]: ts=2026-09-20T13:35:37Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6435worker2 # [ 134.291428] nix-daemon[868]: reaped child process 1599, status = succeeded6436worker2 # [ 134.517833] nix-grpc-daemon[1546]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6437worker2 # [ 134.521181] nix-grpc-daemon[1546]: E0000 00:00:1789911337.851740 1589 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true6438worker2 # [ 134.525730] nix-grpc-daemon[1546]: E0000 00:00:1789911337.852284 1589 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6439lb # [ 135.897964] envoy[891]: conn peer=[2001:db8:1::1]:50896 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9611 tx=3819 ms=60886440lb # [ 135.901879] envoy[891]: conn peer=[2001:db8:1::1]:42282 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2932 tx=1334 ms=120616441lb # [ 135.905479] envoy[891]: conn peer=[2001:db8:1::4]:38438 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=4067 tx=475 ms=302396442worker2 # [ 136.061046] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6443worker2 # [ 136.062069] systemd[1]: nix-grpc-daemon.service: Consumed 760ms CPU time over 32.184s wall clock time, 31M memory peak, 24.3K incoming IP traffic, 36.1K outgoing IP traffic.6444worker2 # [ 136.069107] systemd[1]: Starting Nix worker-protocol over gRPC...6445worker2 # [ 136.221468] nix-grpc-daemon[1910]: ts=2026-09-20T13:35:39Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6446worker2 # [ 136.223316] nix-grpc-daemon[1910]: ts=2026-09-20T13:35:39Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6447worker2 # [ 136.226739] nix-grpc-daemon[1910]: ts=2026-09-20T13:35:39Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16448worker2 # [ 136.233497] nix-grpc-daemon[1910]: ts=2026-09-20T13:35:39Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6449worker2 # [ 136.236273] systemd[1]: Started Nix worker-protocol over gRPC.6450worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.75 seconds)6451lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6452lb: must succeed: curl -sf localhost:9901/clusters6453worker2 # [ 136.249823] nix-grpc-daemon[1910]: ts=2026-09-20T13:35:39Z level=info event=scheduler_connected addr=lb:500516454worker1 # [ 136.285943] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:39Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:540846455worker1 # [ 136.290627] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:39Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06456lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6457lb: must succeed: curl -sf localhost:9901/clusters6458lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6459lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.14 seconds)6460lb: must succeed: curl -sf localhost:9901/clusters6461lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6462worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6463worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6464(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 17.04 seconds)6465subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6466client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16467worker1 # [ 137.712543] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:453466468worker2 # [ 137.688282] nix-grpc-daemon[1910]: ts=2026-09-20T13:35:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364426469worker2 # [ 137.699619] nix-daemon[868]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)6470worker2 # [ 137.717859] nix-grpc-daemon[1910]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 0 ms (attempt 1/5)6471worker2 # [ 137.723599] nix-grpc-daemon[1910]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 170 ms (attempt 2/5)6472worker2 # [ 137.899652] nix-grpc-daemon[1910]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 333 ms (attempt 3/5)6473worker2 # [ 138.238576] nix-grpc-daemon[1910]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 82 ms (attempt 4/5)6474worker2 # [ 138.326772] nix-grpc-daemon[1910]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6475worker1 # [ 138.402981] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:41Z level=debug event=want client=5 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06476worker1 # [ 138.405327] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:41Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16477worker1 # [ 138.407151] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:41Z level=debug event=want client=5 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06478worker1 # [ 138.408719] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:41Z level=debug event=want client=5 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06479worker1 # [ 138.446931] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453466480worker1 # [ 138.454815] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453466481worker1 # [ 138.461903] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:453466482worker1 # [ 138.465826] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6483worker1 # [ 138.613422] nix-daemon[845]: accepted connection from pid 1499, user nix-grpc-daemon (trusted)6484worker1 # [ 138.625808] nix-daemon[845]: reaped child process 1502, status = succeeded6485worker1 # [ 138.774265] nix-daemon[845]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6486worker1 # [ 138.786302] nix-daemon[845]: reaped child process 1507, status = succeeded6487lb # [ 138.793650] niks3-server[1058]: 2026/09/20 13:35:42 INFO Received uploads request method=POST path=/api/pending_closures6488worker1 # [ 138.809471] nix-grpc-daemon[1309]: time=2026-09-20T13:35:42.022Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6489worker1 # [ 138.811412] nix-grpc-daemon[1309]: time=2026-09-20T13:35:42.024Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6490lb # [ 138.821103] niks3-server[1058]: 2026/09/20 13:35:42 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6491lb # [ 138.832929] niks3-server[1058]: 2026/09/20 13:35:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6492worker1 # [ 138.844961] nix-grpc-daemon[1309]: time=2026-09-20T13:35:42.058Z level=INFO msg="Uploading 1 narinfos"6493lb # [ 138.837986] niks3-server[1058]: 2026/09/20 13:35:42 INFO Signed narinfos id=46 count=16494lb # [ 138.844876] niks3-server[1058]: 2026/09/20 13:35:42 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6495lb # [ 138.853630] niks3-server[1058]: 2026/09/20 13:35:42 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6496worker1 # [ 138.865228] nix-grpc-daemon[1309]: time=2026-09-20T13:35:42.078Z level=INFO msg="Upload complete. (393ms)"6497lb # [ 138.859086] niks3-server[1058]: 2026/09/20 13:35:42 INFO Completed upload id=466498worker1 # [ 138.867557] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45346 duration_s=0 paths=1 nar_bytes_in=5046499lb # [ 138.864139] niks3-server[1058]: 2026/09/20 13:35:42 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6500worker1 # [ 138.871778] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453466501worker1 # [ 138.877146] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6502worker1 # [ 138.886658] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6503worker1 # [ 138.888785] nix-daemon[845]: reaped child process 1497, status = succeeded6504worker1 # [ 139.915245] nix-daemon[845]: reaped child process 1511, status = succeeded6505worker1 # [ 140.038087] nix-daemon[845]: accepted connection from pid 1526, user nix-grpc-daemon (trusted)6506worker1 # [ 140.048909] nix-daemon[845]: reaped child process 1529, status = succeeded6507lb # [ 140.051686] niks3-server[1058]: 2026/09/20 13:35:43 INFO Received uploads request method=POST path=/api/pending_closures6508worker1 # [ 140.066244] nix-grpc-daemon[1309]: time=2026-09-20T13:35:43.279Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6509worker1 # [ 140.068799] nix-grpc-daemon[1309]: time=2026-09-20T13:35:43.281Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6510lb # [ 140.081643] niks3-server[1058]: 2026/09/20 13:35:43 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6511lb # [ 140.096632] niks3-server[1058]: 2026/09/20 13:35:43 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6512lb # [ 140.108975] niks3-server[1058]: 2026/09/20 13:35:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6513worker1 # [ 140.120334] nix-grpc-daemon[1309]: time=2026-09-20T13:35:43.333Z level=INFO msg="Uploading 1 narinfos"6514lb # [ 140.115107] niks3-server[1058]: 2026/09/20 13:35:43 INFO Signed narinfos id=47 count=16515lb # [ 140.118882] niks3-server[1058]: 2026/09/20 13:35:43 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6516lb # [ 140.126677] niks3-server[1058]: 2026/09/20 13:35:43 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6517lb # [ 140.132138] niks3-server[1058]: 2026/09/20 13:35:43 INFO Completed upload id=476518worker1 # [ 140.138809] nix-grpc-daemon[1309]: time=2026-09-20T13:35:43.352Z level=INFO msg="Upload complete. (244ms)"6519worker1 # [ 140.144594] nix-grpc-daemon[1309]: time=2026-09-20T13:35:43.355Z level=WARN msg="Failed to register uploaded object" key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo error="server returned 404: object is not pending upload\n"6520worker1 # [ 140.150626] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45346 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=6 outputs=16521worker1 # [ 140.155063] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:43Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16522worker1 # [ 140.163364] nix-daemon[845]: reaped child process 1509, status = succeeded6523worker1 # [ 140.188465] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453466524worker1 # [ 140.192625] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:43Z level=debug event=want client=5 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06525worker1 # [ 140.202703] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453466526worker1 # [ 140.211484] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:453466527worker1 # [ 140.219811] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6528worker1 # [ 140.360369] nix-daemon[845]: accepted connection from pid 1535, user nix-grpc-daemon (trusted)6529worker1 # [ 140.374089] nix-daemon[845]: reaped child process 1538, status = succeeded6530worker1 # [ 140.516393] nix-daemon[845]: accepted connection from pid 1540, user nix-grpc-daemon (trusted)6531worker1 # [ 140.527782] nix-daemon[845]: reaped child process 1543, status = succeeded6532lb # [ 140.536185] niks3-server[1058]: 2026/09/20 13:35:43 INFO Received uploads request method=POST path=/api/pending_closures6533worker1 # [ 140.546923] nix-grpc-daemon[1309]: time=2026-09-20T13:35:43.760Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6534worker1 # [ 140.548454] nix-grpc-daemon[1309]: time=2026-09-20T13:35:43.762Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6535lb # [ 140.551277] niks3-server[1058]: 2026/09/20 13:35:43 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6536lb # [ 140.555306] niks3-server[1058]: 2026/09/20 13:35:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6537lb # [ 140.557805] niks3-server[1058]: 2026/09/20 13:35:43 INFO Signed narinfos id=48 count=16538worker1 # [ 140.564477] nix-grpc-daemon[1309]: time=2026-09-20T13:35:43.778Z level=INFO msg="Uploading 1 narinfos"6539lb # [ 140.560604] niks3-server[1058]: 2026/09/20 13:35:43 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6540lb # [ 140.564994] niks3-server[1058]: 2026/09/20 13:35:43 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6541lb # [ 140.568205] niks3-server[1058]: 2026/09/20 13:35:43 INFO Completed upload id=486542worker1 # [ 140.576063] nix-grpc-daemon[1309]: time=2026-09-20T13:35:43.789Z level=INFO msg="Upload complete. (348ms)"6543lb # [ 140.569927] niks3-server[1058]: 2026/09/20 13:35:43 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6544worker1 # [ 140.577506] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45346 duration_s=0 paths=1 nar_bytes_in=5046545worker1 # [ 140.583389] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453466546worker1 # [ 140.587420] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6547worker1 # [ 140.589596] nix-daemon[845]: reaped child process 1533, status = succeeded6548worker1 # [ 140.594799] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6549worker1 # [ 141.407914] nix-daemon[845]: reaped child process 1547, status = succeeded6550worker1 # [ 141.558512] nix-daemon[845]: accepted connection from pid 1562, user nix-grpc-daemon (trusted)6551worker1 # [ 141.570208] nix-daemon[845]: reaped child process 1565, status = succeeded6552lb # [ 141.581349] niks3-server[1058]: 2026/09/20 13:35:44 INFO Received uploads request method=POST path=/api/pending_closures6553worker1 # [ 141.596124] nix-grpc-daemon[1309]: time=2026-09-20T13:35:44.809Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6554worker1 # [ 141.597694] nix-grpc-daemon[1309]: time=2026-09-20T13:35:44.811Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6555lb # [ 141.625168] niks3-server[1058]: 2026/09/20 13:35:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6556lb # [ 141.629533] niks3-server[1058]: 2026/09/20 13:35:44 INFO Signed narinfos id=49 count=16557worker1 # [ 141.638990] nix-grpc-daemon[1309]: time=2026-09-20T13:35:44.852Z level=INFO msg="Uploading 1 narinfos"6558lb # [ 141.639861] niks3-server[1058]: 2026/09/20 13:35:44 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6559lb # [ 141.646658] niks3-server[1058]: 2026/09/20 13:35:44 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6560lb # [ 141.650430] niks3-server[1058]: 2026/09/20 13:35:44 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6561lb # [ 141.660338] niks3-server[1058]: 2026/09/20 13:35:44 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6562lb # [ 141.666140] niks3-server[1058]: 2026/09/20 13:35:44 INFO Completed upload id=496563lb # [ 141.667788] niks3-server[1058]: 2026/09/20 13:35:44 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6564worker1 # [ 141.673064] nix-grpc-daemon[1309]: time=2026-09-20T13:35:44.886Z level=INFO msg="Upload complete. (272ms)"6565worker1 # [ 141.683377] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:44Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16566worker1 # [ 141.688267] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45346 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=7 outputs=16567worker1 # [ 141.693255] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453466568worker1 # [ 141.696327] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:44Z level=debug event=want client=5 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06569worker1 # [ 141.706138] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453466570worker1 # [ 141.708175] nix-daemon[845]: reaped child process 1545, status = succeeded6571worker1 # [ 141.713514] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:453466572worker1 # [ 141.717307] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6573worker1 # [ 141.884105] nix-daemon[845]: accepted connection from pid 1571, user nix-grpc-daemon (trusted)6574worker1 # [ 141.897179] nix-daemon[845]: reaped child process 1574, status = succeeded6575worker1 # [ 142.047435] nix-daemon[845]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)6576worker1 # [ 142.059184] nix-daemon[845]: reaped child process 1579, status = succeeded6577lb # [ 142.066998] niks3-server[1058]: 2026/09/20 13:35:45 INFO Received uploads request method=POST path=/api/pending_closures6578worker1 # [ 142.082502] nix-grpc-daemon[1309]: time=2026-09-20T13:35:45.296Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6579worker1 # [ 142.084073] nix-grpc-daemon[1309]: time=2026-09-20T13:35:45.297Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6580lb # [ 142.098069] niks3-server[1058]: 2026/09/20 13:35:45 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6581lb # [ 142.110070] niks3-server[1058]: 2026/09/20 13:35:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6582lb # [ 142.114501] niks3-server[1058]: 2026/09/20 13:35:45 INFO Signed narinfos id=50 count=16583worker1 # [ 142.124137] nix-grpc-daemon[1309]: time=2026-09-20T13:35:45.337Z level=INFO msg="Uploading 1 narinfos"6584lb # [ 142.121242] niks3-server[1058]: 2026/09/20 13:35:45 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6585lb # [ 142.131806] niks3-server[1058]: 2026/09/20 13:35:45 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6586lb # [ 142.137828] niks3-server[1058]: 2026/09/20 13:35:45 INFO Completed upload id=506587worker1 # [ 142.144602] nix-grpc-daemon[1309]: time=2026-09-20T13:35:45.358Z level=INFO msg="Upload complete. (421ms)"6588lb # [ 142.142325] niks3-server[1058]: 2026/09/20 13:35:45 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6589worker1 # [ 142.148140] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45346 duration_s=0 paths=1 nar_bytes_in=5046590worker1 # [ 142.153372] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453466591worker1 # [ 142.163108] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6592worker1 # [ 142.167283] nix-daemon[845]: reaped child process 1569, status = succeeded6593worker1 # [ 142.172060] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6594worker1 # [ 143.385589] nix-daemon[845]: reaped child process 1583, status = succeeded6595worker1 # [ 143.525849] nix-daemon[845]: accepted connection from pid 1598, user nix-grpc-daemon (trusted)6596worker1 # [ 143.537519] nix-daemon[845]: reaped child process 1601, status = succeeded6597lb # [ 143.546557] niks3-server[1058]: 2026/09/20 13:35:46 INFO Received uploads request method=POST path=/api/pending_closures6598worker1 # [ 143.557492] nix-grpc-daemon[1309]: time=2026-09-20T13:35:46.771Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6599worker1 # [ 143.559428] nix-grpc-daemon[1309]: time=2026-09-20T13:35:46.772Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6600lb # [ 143.565400] niks3-server[1058]: 2026/09/20 13:35:46 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6601lb # [ 143.568826] niks3-server[1058]: 2026/09/20 13:35:46 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv6602lb # [ 143.572496] niks3-server[1058]: 2026/09/20 13:35:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6603lb # [ 143.575097] niks3-server[1058]: 2026/09/20 13:35:46 INFO Signed narinfos id=51 count=16604worker1 # [ 143.581864] nix-grpc-daemon[1309]: time=2026-09-20T13:35:46.795Z level=INFO msg="Uploading 1 narinfos"6605lb # [ 143.576990] niks3-server[1058]: 2026/09/20 13:35:46 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls6606lb # [ 143.582075] niks3-server[1058]: 2026/09/20 13:35:46 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6607lb # [ 143.585432] niks3-server[1058]: 2026/09/20 13:35:46 INFO Completed upload id=516608worker1 # [ 143.593301] nix-grpc-daemon[1309]: time=2026-09-20T13:35:46.807Z level=INFO msg="Upload complete. (214ms)"6609lb # [ 143.587276] niks3-server[1058]: 2026/09/20 13:35:46 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo6610worker1 # [ 143.596738] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45346 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=8 outputs=16611worker1 # [ 143.600144] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:46Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=16612worker1 # [ 143.604512] nix-daemon[845]: reaped child process 1581, status = succeeded6613worker1 # [ 143.639805] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453466614worker1 # [ 143.645796] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:46Z level=debug event=want client=5 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=06615worker1 # [ 143.651966] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453466616worker1 # [ 143.660438] nix-grpc-daemon[1309]: time=2026-09-20T13:35:46.874Z level=INFO msg="All 1 paths already cached"6617worker1 # [ 143.665170] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:453466618worker1 # [ 143.668965] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6619worker1 # [ 143.815304] nix-daemon[845]: accepted connection from pid 1607, user nix-grpc-daemon (trusted)6620worker1 # [ 143.827298] nix-daemon[845]: reaped child process 1610, status = succeeded6621worker1 # [ 143.979701] nix-daemon[845]: accepted connection from pid 1612, user nix-grpc-daemon (trusted)6622worker1 # [ 143.992037] nix-daemon[845]: reaped child process 1615, status = succeeded6623lb # [ 143.995326] niks3-server[1058]: 2026/09/20 13:35:47 INFO Received uploads request method=POST path=/api/pending_closures6624worker1 # [ 144.009707] nix-grpc-daemon[1309]: time=2026-09-20T13:35:47.223Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6625worker1 # [ 144.011046] nix-grpc-daemon[1309]: time=2026-09-20T13:35:47.223Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"6626lb # [ 144.018428] niks3-server[1058]: 2026/09/20 13:35:47 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst6627lb # [ 144.030981] niks3-server[1058]: 2026/09/20 13:35:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6628worker1 # [ 144.042185] nix-grpc-daemon[1309]: time=2026-09-20T13:35:47.255Z level=INFO msg="Uploading 1 narinfos"6629lb # [ 144.035648] niks3-server[1058]: 2026/09/20 13:35:47 INFO Signed narinfos id=52 count=16630lb # [ 144.040611] niks3-server[1058]: 2026/09/20 13:35:47 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls6631lb # [ 144.051288] niks3-server[1058]: 2026/09/20 13:35:47 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6632worker1 # [ 144.064467] nix-grpc-daemon[1309]: time=2026-09-20T13:35:47.278Z level=INFO msg="Upload complete. (389ms)"6633lb # [ 144.057962] niks3-server[1058]: 2026/09/20 13:35:47 INFO Completed upload id=526634worker1 # [ 144.067076] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45346 duration_s=0 paths=1 nar_bytes_in=6246635lb # [ 144.061378] niks3-server[1058]: 2026/09/20 13:35:47 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo6636worker1 # [ 144.072398] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453466637worker1 # [ 144.076526] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6638worker1 # [ 144.078696] nix-daemon[845]: reaped child process 1605, status = succeeded6639worker1 # [ 144.083888] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6640worker1 # [ 145.497809] nix-daemon[845]: reaped child process 1619, status = succeeded6641worker1 # [ 145.637936] nix-daemon[845]: accepted connection from pid 1634, user nix-grpc-daemon (trusted)6642worker1 # [ 145.648905] nix-daemon[845]: reaped child process 1637, status = succeeded6643lb # [ 145.653996] niks3-server[1058]: 2026/09/20 13:35:48 INFO Received uploads request method=POST path=/api/pending_closures6644worker1 # [ 145.668391] nix-grpc-daemon[1309]: time=2026-09-20T13:35:48.881Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6645worker1 # [ 145.669827] nix-grpc-daemon[1309]: time=2026-09-20T13:35:48.883Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"6646lb # [ 145.686363] niks3-server[1058]: 2026/09/20 13:35:49 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv6647lb # [ 145.703369] niks3-server[1058]: 2026/09/20 13:35:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6648worker1 # [ 145.715319] nix-grpc-daemon[1309]: time=2026-09-20T13:35:48.928Z level=INFO msg="Uploading 1 narinfos"6649lb # [ 145.708756] niks3-server[1058]: 2026/09/20 13:35:49 INFO Signed narinfos id=53 count=16650lb # [ 145.715331] niks3-server[1058]: 2026/09/20 13:35:49 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst6651lb # [ 145.719798] niks3-server[1058]: 2026/09/20 13:35:49 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls6652lb # [ 145.728855] niks3-server[1058]: 2026/09/20 13:35:49 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6653lb # [ 145.734561] niks3-server[1058]: 2026/09/20 13:35:49 INFO Completed upload id=536654lb # [ 145.736487] niks3-server[1058]: 2026/09/20 13:35:49 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo6655worker1 # [ 145.742574] nix-grpc-daemon[1309]: time=2026-09-20T13:35:48.956Z level=INFO msg="Upload complete. (252ms)"6656worker1 # [ 145.749841] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45346 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=9 outputs=16657worker1 # [ 145.758207] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:48Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=16658worker1 # [ 145.761669] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453466659worker1 # [ 145.764669] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:48Z level=debug event=want client=5 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=06660worker1 # [ 145.772716] nix-daemon[845]: reaped child process 1617, status = succeeded6661worker1 # [ 145.774146] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453466662worker1 # [ 145.780674] nix-grpc-daemon[1309]: time=2026-09-20T13:35:48.994Z level=INFO msg="All 1 paths already cached"6663worker1 # [ 145.784896] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:453466664worker1 # [ 145.789455] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6665worker1 # [ 145.939992] nix-daemon[845]: accepted connection from pid 1643, user nix-grpc-daemon (trusted)6666worker1 # [ 145.953124] nix-daemon[845]: reaped child process 1646, status = succeeded6667worker1 # [ 146.109587] nix-daemon[845]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6668worker1 # [ 146.121549] nix-daemon[845]: reaped child process 1651, status = succeeded6669lb # [ 146.129319] niks3-server[1058]: 2026/09/20 13:35:49 INFO Received uploads request method=POST path=/api/pending_closures6670worker1 # [ 146.143921] nix-grpc-daemon[1309]: time=2026-09-20T13:35:49.357Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6671worker1 # [ 146.145384] nix-grpc-daemon[1309]: time=2026-09-20T13:35:49.359Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"6672lb # [ 146.155351] niks3-server[1058]: 2026/09/20 13:35:49 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst6673lb # [ 146.167910] niks3-server[1058]: 2026/09/20 13:35:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6674worker1 # [ 146.179257] nix-grpc-daemon[1309]: time=2026-09-20T13:35:49.392Z level=INFO msg="Uploading 1 narinfos"6675lb # [ 146.172583] niks3-server[1058]: 2026/09/20 13:35:49 INFO Signed narinfos id=54 count=16676lb # [ 146.179181] niks3-server[1058]: 2026/09/20 13:35:49 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls6677lb # [ 146.191605] niks3-server[1058]: 2026/09/20 13:35:49 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6678lb # [ 146.198221] niks3-server[1058]: 2026/09/20 13:35:49 INFO Completed upload id=546679worker1 # [ 146.207061] nix-grpc-daemon[1309]: time=2026-09-20T13:35:49.420Z level=INFO msg="Upload complete. (413ms)"6680lb # [ 146.203520] niks3-server[1058]: 2026/09/20 13:35:49 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo6681worker1 # [ 146.209600] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45346 duration_s=0 paths=1 nar_bytes_in=6246682worker1 # [ 146.214476] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453466683worker1 # [ 146.224870] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6684worker1 # [ 146.228598] nix-daemon[845]: reaped child process 1641, status = succeeded6685worker1 # [ 146.233584] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6686worker1 # [ 147.259937] nix-daemon[845]: reaped child process 1655, status = succeeded6687worker1 # [ 147.423683] nix-daemon[845]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)6688worker1 # [ 147.435719] nix-daemon[845]: reaped child process 1673, status = succeeded6689lb # [ 147.443566] niks3-server[1058]: 2026/09/20 13:35:50 INFO Received uploads request method=POST path=/api/pending_closures6690worker1 # [ 147.458250] nix-grpc-daemon[1309]: time=2026-09-20T13:35:50.671Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6691worker1 # [ 147.460467] nix-grpc-daemon[1309]: time=2026-09-20T13:35:50.673Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"6692lb # [ 147.473413] niks3-server[1058]: 2026/09/20 13:35:50 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv6693lb # [ 147.489102] niks3-server[1058]: 2026/09/20 13:35:50 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst6694lb # [ 147.500738] niks3-server[1058]: 2026/09/20 13:35:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6695worker1 # [ 147.512198] nix-grpc-daemon[1309]: time=2026-09-20T13:35:50.725Z level=INFO msg="Uploading 1 narinfos"6696lb # [ 147.505449] niks3-server[1058]: 2026/09/20 13:35:50 INFO Signed narinfos id=55 count=16697lb # [ 147.511490] niks3-server[1058]: 2026/09/20 13:35:50 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls6698lb # [ 147.518963] niks3-server[1058]: 2026/09/20 13:35:50 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6699lb # [ 147.523682] niks3-server[1058]: 2026/09/20 13:35:50 INFO Completed upload id=556700worker1 # [ 147.530696] nix-grpc-daemon[1309]: time=2026-09-20T13:35:50.744Z level=INFO msg="Upload complete. (289ms)"6701lb # [ 147.526181] niks3-server[1058]: 2026/09/20 13:35:50 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo6702worker1 # [ 147.542839] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45346 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=10 outputs=16703worker1 # [ 147.549456] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:50Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=16704worker1 # [ 147.554306] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453466705worker1 # [ 147.557890] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:50Z level=debug event=want client=5 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=06706worker1 # [ 147.567373] nix-daemon[845]: reaped child process 1653, status = succeeded6707worker1 # [ 147.569372] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453466708worker1 # [ 147.578570] nix-grpc-daemon[1309]: time=2026-09-20T13:35:50.792Z level=INFO msg="All 1 paths already cached"6709worker1 # [ 147.582356] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:453466710worker1 # [ 147.586339] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6711worker1 # [ 147.744564] nix-daemon[845]: accepted connection from pid 1679, user nix-grpc-daemon (trusted)6712worker1 # [ 147.756690] nix-daemon[845]: reaped child process 1682, status = succeeded6713worker1 # [ 147.913677] nix-daemon[845]: accepted connection from pid 1684, user nix-grpc-daemon (trusted)6714worker1 # [ 147.925399] nix-daemon[845]: reaped child process 1687, status = succeeded6715lb # [ 147.934117] niks3-server[1058]: 2026/09/20 13:35:51 INFO Received uploads request method=POST path=/api/pending_closures6716worker1 # [ 147.947745] nix-grpc-daemon[1309]: time=2026-09-20T13:35:51.161Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6717worker1 # [ 147.949476] nix-grpc-daemon[1309]: time=2026-09-20T13:35:51.163Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"6718lb # [ 147.960972] niks3-server[1058]: 2026/09/20 13:35:51 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst6719lb # [ 147.971955] niks3-server[1058]: 2026/09/20 13:35:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6720worker1 # [ 147.984304] nix-grpc-daemon[1309]: time=2026-09-20T13:35:51.198Z level=INFO msg="Uploading 1 narinfos"6721lb # [ 147.977885] niks3-server[1058]: 2026/09/20 13:35:51 INFO Signed narinfos id=56 count=16722lb # [ 147.982775] niks3-server[1058]: 2026/09/20 13:35:51 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls6723lb # [ 147.995115] niks3-server[1058]: 2026/09/20 13:35:51 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6724lb # [ 148.000562] niks3-server[1058]: 2026/09/20 13:35:51 INFO Completed upload id=566725worker1 # [ 148.007295] nix-grpc-daemon[1309]: time=2026-09-20T13:35:51.220Z level=INFO msg="Upload complete. (415ms)"6726lb # [ 148.003246] niks3-server[1058]: 2026/09/20 13:35:51 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo6727worker1 # [ 148.011117] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45346 duration_s=0 paths=1 nar_bytes_in=6246728worker1 # [ 148.016256] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453466729worker1 # [ 148.023775] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6730worker1 # [ 148.030835] nix-daemon[845]: reaped child process 1677, status = succeeded6731worker1 # [ 148.034732] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6732worker1 # [ 149.460475] nix-daemon[845]: reaped child process 1691, status = succeeded6733worker1 # [ 149.609495] nix-daemon[845]: accepted connection from pid 1706, user nix-grpc-daemon (trusted)6734worker1 # [ 149.621729] nix-daemon[845]: reaped child process 1709, status = succeeded6735lb # [ 149.627973] niks3-server[1058]: 2026/09/20 13:35:52 INFO Received uploads request method=POST path=/api/pending_closures6736worker1 # [ 149.643330] nix-grpc-daemon[1309]: time=2026-09-20T13:35:52.856Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6737worker1 # [ 149.644905] nix-grpc-daemon[1309]: time=2026-09-20T13:35:52.858Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"6738lb # [ 149.665507] niks3-server[1058]: 2026/09/20 13:35:52 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv6739lb # [ 149.670708] niks3-server[1058]: 2026/09/20 13:35:52 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst6740lb # [ 149.683395] niks3-server[1058]: 2026/09/20 13:35:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6741lb # [ 149.688979] niks3-server[1058]: 2026/09/20 13:35:53 INFO Signed narinfos id=57 count=16742worker1 # [ 149.698188] nix-grpc-daemon[1309]: time=2026-09-20T13:35:52.911Z level=INFO msg="Uploading 1 narinfos"6743lb # [ 149.693950] niks3-server[1058]: 2026/09/20 13:35:53 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls6744lb # [ 149.703113] niks3-server[1058]: 2026/09/20 13:35:53 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6745lb # [ 149.708764] niks3-server[1058]: 2026/09/20 13:35:53 INFO Completed upload id=576746worker1 # [ 149.715697] nix-grpc-daemon[1309]: time=2026-09-20T13:35:52.929Z level=INFO msg="Upload complete. (272ms)"6747lb # [ 149.711200] niks3-server[1058]: 2026/09/20 13:35:53 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo6748worker1 # [ 149.725644] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45346 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=11 outputs=16749worker1 # [ 149.731170] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:52Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=16750worker1 # [ 149.736710] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453466751worker1 # [ 149.741371] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:52Z level=debug event=want client=5 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=06752worker1 # [ 149.748523] nix-daemon[845]: reaped child process 1689, status = succeeded6753worker1 # [ 149.767573] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453466754worker1 # [ 149.778608] nix-grpc-daemon[1309]: time=2026-09-20T13:35:52.992Z level=INFO msg="All 2 paths already cached"6755worker1 # [ 149.784847] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:453466756worker1 # [ 149.791823] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6757worker1 # [ 149.922451] nix-daemon[845]: accepted connection from pid 1715, user nix-grpc-daemon (trusted)6758worker1 # [ 149.934472] nix-daemon[845]: reaped child process 1718, status = succeeded6759worker1 # [ 150.076349] nix-daemon[845]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6760worker1 # [ 150.088380] nix-daemon[845]: reaped child process 1723, status = succeeded6761lb # [ 150.094767] niks3-server[1058]: 2026/09/20 13:35:53 INFO Received uploads request method=POST path=/api/pending_closures6762worker1 # [ 150.109157] nix-grpc-daemon[1309]: time=2026-09-20T13:35:53.322Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6763worker1 # [ 150.111035] nix-grpc-daemon[1309]: time=2026-09-20T13:35:53.324Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"6764lb # [ 150.124175] niks3-server[1058]: 2026/09/20 13:35:53 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst6765lb # [ 150.133348] niks3-server[1058]: 2026/09/20 13:35:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6766lb # [ 150.138561] niks3-server[1058]: 2026/09/20 13:35:53 INFO Signed narinfos id=58 count=16767worker1 # [ 150.148247] nix-grpc-daemon[1309]: time=2026-09-20T13:35:53.361Z level=INFO msg="Uploading 1 narinfos"6768lb # [ 150.145103] niks3-server[1058]: 2026/09/20 13:35:53 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls6769lb # [ 150.158090] niks3-server[1058]: 2026/09/20 13:35:53 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6770worker1 # [ 150.169081] nix-grpc-daemon[1309]: time=2026-09-20T13:35:53.382Z level=INFO msg="Upload complete. (369ms)"6771lb # [ 150.162739] niks3-server[1058]: 2026/09/20 13:35:53 INFO Completed upload id=586772lb # [ 150.166384] niks3-server[1058]: 2026/09/20 13:35:53 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo6773worker1 # [ 150.172282] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45346 duration_s=0 paths=1 nar_bytes_in=7526774worker1 # [ 150.177464] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453466775worker1 # [ 150.186850] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6776worker1 # [ 150.190365] nix-daemon[845]: reaped child process 1713, status = succeeded6777worker1 # [ 150.195383] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6778worker1 # [ 151.410495] nix-daemon[845]: reaped child process 1727, status = succeeded6779worker1 # [ 151.555513] nix-daemon[845]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6780worker1 # [ 151.566097] nix-daemon[845]: reaped child process 1745, status = succeeded6781lb # [ 151.574586] niks3-server[1058]: 2026/09/20 13:35:54 INFO Received uploads request method=POST path=/api/pending_closures6782worker1 # [ 151.589887] nix-grpc-daemon[1309]: time=2026-09-20T13:35:54.803Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6783worker1 # [ 151.591399] nix-grpc-daemon[1309]: time=2026-09-20T13:35:54.805Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"6784lb # [ 151.606244] niks3-server[1058]: 2026/09/20 13:35:54 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst6785lb # [ 151.623150] niks3-server[1058]: 2026/09/20 13:35:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6786lb # [ 151.629625] niks3-server[1058]: 2026/09/20 13:35:54 INFO Signed narinfos id=59 count=16787worker1 # [ 151.640194] nix-grpc-daemon[1309]: time=2026-09-20T13:35:54.853Z level=INFO msg="Uploading 1 narinfos"6788lb # [ 151.635921] niks3-server[1058]: 2026/09/20 13:35:54 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv6789lb # [ 151.639653] niks3-server[1058]: 2026/09/20 13:35:54 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls6790lb # [ 151.649918] niks3-server[1058]: 2026/09/20 13:35:54 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6791lb # [ 151.655328] niks3-server[1058]: 2026/09/20 13:35:54 INFO Completed upload id=596792lb # [ 151.657303] niks3-server[1058]: 2026/09/20 13:35:54 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo6793worker1 # [ 151.663849] nix-grpc-daemon[1309]: time=2026-09-20T13:35:54.877Z level=INFO msg="Upload complete. (260ms)"6794worker1 # [ 151.672391] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45346 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=12 outputs=16795worker1 # [ 151.677933] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:54Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=16796worker1 # [ 151.688327] nix-daemon[845]: reaped child process 1725, status = succeeded6797worker1 # [ 151.712509] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453466798worker1 # [ 151.724429] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453466799worker1 # [ 151.735294] nix-grpc-daemon[1309]: time=2026-09-20T13:35:54.949Z level=INFO msg="All 2 paths already cached"6800worker1 # [ 151.741959] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:453466801worker1 # [ 151.752131] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6802worker1 # [ 151.915623] nix-daemon[845]: accepted connection from pid 1749, user nix-grpc-daemon (trusted)6803worker1 # [ 151.927489] nix-daemon[845]: reaped child process 1752, status = succeeded6804worker1 # [ 152.073299] nix-daemon[845]: accepted connection from pid 1754, user nix-grpc-daemon (trusted)6805worker1 # [ 152.085345] nix-daemon[845]: reaped child process 1757, status = succeeded6806lb # [ 152.090813] niks3-server[1058]: 2026/09/20 13:35:55 INFO Received uploads request method=POST path=/api/pending_closures6807worker1 # [ 152.101169] nix-grpc-daemon[1309]: time=2026-09-20T13:35:55.314Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6808worker1 # [ 152.103062] nix-grpc-daemon[1309]: time=2026-09-20T13:35:55.316Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"6809lb # [ 152.112302] niks3-server[1058]: 2026/09/20 13:35:55 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst6810lb # [ 152.120962] niks3-server[1058]: 2026/09/20 13:35:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6811worker1 # [ 152.133882] nix-grpc-daemon[1309]: time=2026-09-20T13:35:55.347Z level=INFO msg="Uploading 1 narinfos"6812lb # [ 152.127129] niks3-server[1058]: 2026/09/20 13:35:55 INFO Signed narinfos id=60 count=16813lb # [ 152.131678] niks3-server[1058]: 2026/09/20 13:35:55 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls6814lb # [ 152.142113] niks3-server[1058]: 2026/09/20 13:35:55 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6815worker1 # [ 152.155261] nix-grpc-daemon[1309]: time=2026-09-20T13:35:55.368Z level=INFO msg="Upload complete. (392ms)"6816lb # [ 152.148854] niks3-server[1058]: 2026/09/20 13:35:55 INFO Completed upload id=606817worker1 # [ 152.157194] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45346 duration_s=0 paths=1 nar_bytes_in=7526818lb # [ 152.152170] niks3-server[1058]: 2026/09/20 13:35:55 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo6819worker1 # [ 152.163457] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453466820worker1 # [ 152.167468] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6821worker1 # [ 152.169755] nix-daemon[845]: reaped child process 1747, status = succeeded6822worker1 # [ 152.175026] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6823worker1 # [ 153.187783] nix-daemon[845]: reaped child process 1761, status = succeeded6824worker1 # [ 153.335224] nix-daemon[845]: accepted connection from pid 1776, user nix-grpc-daemon (trusted)6825worker1 # [ 153.347248] nix-daemon[845]: reaped child process 1779, status = succeeded6826lb # [ 153.355643] niks3-server[1058]: 2026/09/20 13:35:56 INFO Received uploads request method=POST path=/api/pending_closures6827worker1 # [ 153.374292] nix-grpc-daemon[1309]: time=2026-09-20T13:35:56.587Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6828worker1 # [ 153.375672] nix-grpc-daemon[1309]: time=2026-09-20T13:35:56.587Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"6829lb # [ 153.398835] niks3-server[1058]: 2026/09/20 13:35:56 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv6830lb # [ 153.403653] niks3-server[1058]: 2026/09/20 13:35:56 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst6831lb # [ 153.413992] niks3-server[1058]: 2026/09/20 13:35:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6832lb # [ 153.419467] niks3-server[1058]: 2026/09/20 13:35:56 INFO Signed narinfos id=61 count=16833worker1 # [ 153.426345] nix-grpc-daemon[1309]: time=2026-09-20T13:35:56.639Z level=INFO msg="Uploading 1 narinfos"6834lb # [ 153.421634] niks3-server[1058]: 2026/09/20 13:35:56 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls6835lb # [ 153.427389] niks3-server[1058]: 2026/09/20 13:35:56 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6836lb # [ 153.430407] niks3-server[1058]: 2026/09/20 13:35:56 INFO Completed upload id=616837lb # [ 153.432055] niks3-server[1058]: 2026/09/20 13:35:56 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo6838worker1 # [ 153.438600] nix-grpc-daemon[1309]: time=2026-09-20T13:35:56.652Z level=INFO msg="Upload complete. (254ms)"6839worker1 # [ 153.446359] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45346 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=13 outputs=16840worker1 # [ 153.450766] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:56Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=16841worker1 # [ 153.454559] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453466842worker1 # [ 153.464348] nix-daemon[845]: reaped child process 1759, status = succeeded6843worker1 # [ 153.488595] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453466844worker1 # [ 153.500692] nix-grpc-daemon[1309]: time=2026-09-20T13:35:56.714Z level=INFO msg="All 2 paths already cached"6845worker1 # [ 153.508212] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:453466846worker1 # [ 153.516976] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6847worker1 # [ 153.673177] nix-daemon[845]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)6848worker1 # [ 153.685163] nix-daemon[845]: reaped child process 1786, status = succeeded6849worker1 # [ 153.835523] nix-daemon[845]: accepted connection from pid 1788, user nix-grpc-daemon (trusted)6850worker1 # [ 153.847364] nix-daemon[845]: reaped child process 1791, status = succeeded6851lb # [ 153.853050] niks3-server[1058]: 2026/09/20 13:35:57 INFO Received uploads request method=POST path=/api/pending_closures6852worker1 # [ 153.864517] nix-grpc-daemon[1309]: time=2026-09-20T13:35:57.078Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6853worker1 # [ 153.865944] nix-grpc-daemon[1309]: time=2026-09-20T13:35:57.078Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"6854lb # [ 153.865268] niks3-server[1058]: 2026/09/20 13:35:57 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst6855lb # [ 153.869057] niks3-server[1058]: 2026/09/20 13:35:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6856lb # [ 153.871108] niks3-server[1058]: 2026/09/20 13:35:57 INFO Signed narinfos id=62 count=16857worker1 # [ 153.879680] nix-grpc-daemon[1309]: time=2026-09-20T13:35:57.093Z level=INFO msg="Uploading 1 narinfos"6858lb # [ 153.875022] niks3-server[1058]: 2026/09/20 13:35:57 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls6859lb # [ 153.879053] niks3-server[1058]: 2026/09/20 13:35:57 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6860lb # [ 153.881573] niks3-server[1058]: 2026/09/20 13:35:57 INFO Completed upload id=626861worker1 # [ 153.888311] nix-grpc-daemon[1309]: time=2026-09-20T13:35:57.102Z level=INFO msg="Upload complete. (360ms)"6862lb # [ 153.884197] niks3-server[1058]: 2026/09/20 13:35:57 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo6863worker1 # [ 153.890288] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45346 duration_s=0 paths=1 nar_bytes_in=7526864worker1 # [ 153.892971] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453466865worker1 # [ 153.900716] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6866worker1 # [ 153.902460] nix-daemon[845]: reaped child process 1781, status = succeeded6867worker1 # [ 153.907291] nix-daemon[845]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)6868worker1 # [ 154.930782] nix-daemon[845]: reaped child process 1795, status = succeeded6869worker1 # [ 155.076398] nix-daemon[845]: accepted connection from pid 1810, user nix-grpc-daemon (trusted)6870worker1 # [ 155.087195] nix-daemon[845]: reaped child process 1813, status = succeeded6871lb # [ 155.095957] niks3-server[1058]: 2026/09/20 13:35:58 INFO Received uploads request method=POST path=/api/pending_closures6872worker1 # [ 155.111378] nix-grpc-daemon[1309]: time=2026-09-20T13:35:58.324Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6873worker1 # [ 155.113037] nix-grpc-daemon[1309]: time=2026-09-20T13:35:58.326Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"6874lb # [ 155.138229] niks3-server[1058]: 2026/09/20 13:35:58 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst6875lb # [ 155.147289] niks3-server[1058]: 2026/09/20 13:35:58 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv6876lb # [ 155.157283] niks3-server[1058]: 2026/09/20 13:35:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6877lb # [ 155.162575] niks3-server[1058]: 2026/09/20 13:35:58 INFO Signed narinfos id=63 count=16878worker1 # [ 155.168827] nix-grpc-daemon[1309]: time=2026-09-20T13:35:58.382Z level=INFO msg="Uploading 1 narinfos"6879lb # [ 155.167253] niks3-server[1058]: 2026/09/20 13:35:58 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls6880lb # [ 155.176318] niks3-server[1058]: 2026/09/20 13:35:58 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6881lb # [ 155.181881] niks3-server[1058]: 2026/09/20 13:35:58 INFO Completed upload id=636882worker1 # [ 155.188728] nix-grpc-daemon[1309]: time=2026-09-20T13:35:58.402Z level=INFO msg="Upload complete. (272ms)"6883lb # [ 155.184262] niks3-server[1058]: 2026/09/20 13:35:58 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo6884worker1 # [ 155.194774] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45346 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=14 outputs=16885worker1 # [ 155.208988] nix-daemon[845]: reaped child process 1793, status = succeeded6886client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 17.75 seconds)6887worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6888worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6889client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]'6890client # [ 155.382279] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6891client # Running as unit: gpu.service; invocation ID: dbbcd63ddfb54123b4021fd195291da26892client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.04 seconds)6893lb: must succeed: curl -sf localhost:9901/clusters6894lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6895lb: must succeed: curl -sf localhost:9901/clusters6896lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6897worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6898worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6899client # [ 155.569992] nix[1348]: don't know how to build these paths:6900client # [ 155.570919] nix[1348]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv6901worker2 # [ 155.519215] nix-grpc-daemon[1910]: ts=2026-09-20T13:35:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:364426902worker1 # [ 155.553572] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453466903client # [ 155.590367] nix[1348]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, waiting up to 30s for a worker6904worker1 # [ 155.571989] nix-grpc-daemon[1235]: ts=2026-09-20T13:35:58Z level=debug event=want client=6 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=06905lb # [ 155.898987] envoy[891]: conn peer=[2001:db8:1::1]:34744 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27068 tx=10696 ms=175406906lb: must succeed: curl -sf localhost:9901/clusters6907lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6908lb: must succeed: curl -sf localhost:9901/clusters6909lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6910worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6911worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6912client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6913client # [ 185.593431] nix[1348]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}6914client # [ 185.602076] nix[1348]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed6915client # [ 185.634310] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6916client # [ 185.637328] systemd[1]: gpu.service: Failed with result 'exit-code'.6917client # [ 185.641221] systemd[1]: gpu.service: Consumed 122ms CPU time over 30.251s wall clock time, 33.4M memory peak, 6.7K incoming IP traffic, 5.8K outgoing IP traffic.6918lb # [ 185.899176] envoy[891]: conn peer=[2001:db8:1::1]:45796 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=895 tx=498 ms=300486919client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.56 seconds)6920client: must succeed: journalctl -u gpu -o cat6921client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds)6922lb: must succeed: curl -sf localhost:9901/clusters6923lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6924lb: must succeed: curl -sf localhost:9901/clusters6925lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6926worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6927worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6928(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 48.98 seconds)6929subtest: low disk drains a worker and builds go to the other6930worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6931worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.96 seconds)6932worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6933worker1 # [ 187.723845] nix-grpc-daemon[1235]: ts=2026-09-20T13:36:30Z level=info event=unhealthy reason=min_free6934worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.11 seconds)6935client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&26936client # don't know how to build these paths:6937client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6938worker2 # [ 188.745346] nix-grpc-daemon[1910]: ts=2026-09-20T13:36:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:364426939worker1 # [ 188.785482] nix-grpc-daemon[1235]: ts=2026-09-20T13:36:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453466940worker1 # [ 188.815986] nix-grpc-daemon[1235]: ts=2026-09-20T13:36:32Z level=debug event=want client=7 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=06941worker1 # [ 188.820956] nix-grpc-daemon[1235]: ts=2026-09-20T13:36:32Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=16942worker2 # [ 188.830980] nix-grpc-daemon[1910]: ts=2026-09-20T13:36:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364426943worker2 # [ 188.846316] nix-grpc-daemon[1910]: ts=2026-09-20T13:36:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364426944client # copying 1 paths to 'lb:50051'...6945client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...6946worker2 # [ 188.860963] nix-grpc-daemon[1910]: ts=2026-09-20T13:36:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:364426947worker2 # [ 188.876583] nix-daemon[868]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)6948worker2 # [ 188.900826] nix-grpc-daemon[1910]: ts=2026-09-20T13:36:32Z level=info event=niks3_push_started pid=19666949worker2 # [ 189.111857] nix-grpc-daemon[1966]: time=2026-09-20T13:36:32.445Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""6950worker2 # [ 189.288103] nix-daemon[868]: accepted connection from pid 1979, user nix-grpc-daemon (trusted)6951worker2 # [ 189.300612] nix-daemon[868]: reaped child process 1982, status = succeeded6952worker2 # [ 189.443262] nix-daemon[868]: accepted connection from pid 1984, user nix-grpc-daemon (trusted)6953worker2 # [ 189.456301] nix-daemon[868]: reaped child process 1987, status = succeeded6954lb # [ 189.496987] niks3-server[1058]: 2026/09/20 13:36:32 INFO Received uploads request method=POST path=/api/pending_closures6955worker2 # [ 189.480766] nix-grpc-daemon[1966]: time=2026-09-20T13:36:32.815Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6956worker2 # [ 189.483520] nix-grpc-daemon[1966]: time=2026-09-20T13:36:32.816Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"6957lb # [ 189.534812] niks3-server[1058]: 2026/09/20 13:36:32 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst6958lb # [ 189.547428] niks3-server[1058]: 2026/09/20 13:36:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6959lb # [ 189.554656] niks3-server[1058]: 2026/09/20 13:36:32 INFO Signed narinfos id=64 count=16960worker2 # [ 189.532540] nix-grpc-daemon[1966]: time=2026-09-20T13:36:32.866Z level=INFO msg="Uploading 1 narinfos"6961lb # [ 189.563327] niks3-server[1058]: 2026/09/20 13:36:32 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls6962lb # [ 189.572199] niks3-server[1058]: 2026/09/20 13:36:32 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6963lb # [ 189.576971] niks3-server[1058]: 2026/09/20 13:36:32 INFO Completed upload id=646964worker2 # [ 189.552216] nix-grpc-daemon[1966]: time=2026-09-20T13:36:32.886Z level=INFO msg="Upload complete. (436ms)"6965lb # [ 189.578315] niks3-server[1058]: 2026/09/20 13:36:32 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo6966worker2 # [ 189.556364] nix-grpc-daemon[1910]: ts=2026-09-20T13:36:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36442 duration_s=0 paths=1 nar_bytes_in=5126967worker2 # [ 189.562132] nix-grpc-daemon[1910]: ts=2026-09-20T13:36:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364426968worker2 # [ 189.571485] nix-daemon[868]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)6969worker2 # [ 189.578934] nix-daemon[868]: reaped child process 1964, status = succeeded6970worker2 # [ 189.582092] nix-daemon[868]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)6971client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...6972client # farm-a> worker2-next: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6973client # farm-a> LOG-farm-a-drain6974worker2 # [ 190.411660] nix-daemon[868]: reaped child process 1991, status = succeeded6975worker2 # [ 190.529320] nix-daemon[868]: accepted connection from pid 2006, user nix-grpc-daemon (trusted)6976worker2 # [ 190.539894] nix-daemon[868]: reaped child process 2009, status = succeeded6977lb # [ 190.574537] niks3-server[1058]: 2026/09/20 13:36:33 INFO Received uploads request method=POST path=/api/pending_closures6978worker2 # [ 190.554113] nix-grpc-daemon[1966]: time=2026-09-20T13:36:33.888Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6979worker2 # [ 190.555566] nix-grpc-daemon[1966]: time=2026-09-20T13:36:33.890Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"6980lb # [ 190.594784] niks3-server[1058]: 2026/09/20 13:36:33 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6981lb # [ 190.597986] niks3-server[1058]: 2026/09/20 13:36:33 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6982lb # [ 190.602121] niks3-server[1058]: 2026/09/20 13:36:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6983lb # [ 190.603753] niks3-server[1058]: 2026/09/20 13:36:33 INFO Signed narinfos id=65 count=16984worker2 # [ 190.579336] nix-grpc-daemon[1966]: time=2026-09-20T13:36:33.913Z level=INFO msg="Uploading 1 narinfos"6985lb # [ 190.607781] niks3-server[1058]: 2026/09/20 13:36:33 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls6986lb # [ 190.612044] niks3-server[1058]: 2026/09/20 13:36:33 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6987lb # [ 190.615195] niks3-server[1058]: 2026/09/20 13:36:33 INFO Completed upload id=656988worker2 # [ 190.590178] nix-grpc-daemon[1966]: time=2026-09-20T13:36:33.924Z level=INFO msg="Upload complete. (200ms)"6989lb # [ 190.616798] niks3-server[1058]: 2026/09/20 13:36:33 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo6990worker2 # [ 190.592635] nix-grpc-daemon[1910]: ts=2026-09-20T13:36:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36442 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=15 outputs=16991worker2 # [ 190.599112] nix-daemon[868]: reaped child process 1989, status = succeeded6992worker1 # [ 190.634727] nix-grpc-daemon[1235]: ts=2026-09-20T13:36:33Z level=debug event=want client=7 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=06993worker1 # [ 190.638899] nix-grpc-daemon[1235]: ts=2026-09-20T13:36:33Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=16994worker2 # [ 190.645814] nix-grpc-daemon[1910]: ts=2026-09-20T13:36:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364426995worker2 # [ 190.655681] nix-grpc-daemon[1910]: ts=2026-09-20T13:36:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364426996worker2 # [ 190.665331] nix-grpc-daemon[1966]: time=2026-09-20T13:36:33.999Z level=INFO msg="All 1 paths already cached"6997client # copying 1 paths to 'lb:50051'...6998client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...6999worker2 # [ 190.670903] nix-grpc-daemon[1910]: ts=2026-09-20T13:36:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:364427000worker2 # [ 190.677663] nix-daemon[868]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)7001worker2 # [ 190.806048] nix-daemon[868]: accepted connection from pid 2013, user nix-grpc-daemon (trusted)7002worker2 # [ 190.818233] nix-daemon[868]: reaped child process 2016, status = succeeded7003worker2 # [ 190.939159] nix-daemon[868]: accepted connection from pid 2018, user nix-grpc-daemon (trusted)7004worker2 # [ 190.950843] nix-daemon[868]: reaped child process 2021, status = succeeded7005lb # [ 190.985079] niks3-server[1058]: 2026/09/20 13:36:34 INFO Received uploads request method=POST path=/api/pending_closures7006worker2 # [ 190.968134] nix-grpc-daemon[1966]: time=2026-09-20T13:36:34.302Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7007worker2 # [ 190.970106] nix-grpc-daemon[1966]: time=2026-09-20T13:36:34.304Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"7008lb # [ 191.014157] niks3-server[1058]: 2026/09/20 13:36:34 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst7009lb # [ 191.023072] niks3-server[1058]: 2026/09/20 13:36:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign7010worker2 # [ 191.002847] nix-grpc-daemon[1966]: time=2026-09-20T13:36:34.337Z level=INFO msg="Uploading 1 narinfos"7011lb # [ 191.027824] niks3-server[1058]: 2026/09/20 13:36:34 INFO Signed narinfos id=66 count=17012lb # [ 191.034322] niks3-server[1058]: 2026/09/20 13:36:34 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls7013lb # [ 191.045093] niks3-server[1058]: 2026/09/20 13:36:34 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete7014worker2 # [ 191.026204] nix-grpc-daemon[1966]: time=2026-09-20T13:36:34.360Z level=INFO msg="Upload complete. (337ms)"7015lb # [ 191.051376] niks3-server[1058]: 2026/09/20 13:36:34 INFO Completed upload id=667016worker2 # [ 191.030075] nix-grpc-daemon[1910]: ts=2026-09-20T13:36:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36442 duration_s=0 paths=1 nar_bytes_in=6247017lb # [ 191.054646] niks3-server[1058]: 2026/09/20 13:36:34 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo7018worker2 # [ 191.034405] nix-grpc-daemon[1910]: ts=2026-09-20T13:36:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364427019worker2 # [ 191.039154] nix-daemon[868]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)7020worker2 # [ 191.041457] nix-daemon[868]: reaped child process 2011, status = succeeded7021client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...7022client # farm-b> worker2-next: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7023worker2 # [ 191.046742] nix-daemon[868]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)7024client # farm-b> LOG-farm-b-drain7025worker2 # [ 191.872506] nix-daemon[868]: reaped child process 2025, status = succeeded7026worker2 # [ 192.016859] nix-daemon[868]: accepted connection from pid 2040, user nix-grpc-daemon (trusted)7027worker2 # [ 192.027635] nix-daemon[868]: reaped child process 2043, status = succeeded7028lb # [ 192.066342] niks3-server[1058]: 2026/09/20 13:36:35 INFO Received uploads request method=POST path=/api/pending_closures7029worker2 # [ 192.049116] nix-grpc-daemon[1966]: time=2026-09-20T13:36:35.383Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7030worker2 # [ 192.050562] nix-grpc-daemon[1966]: time=2026-09-20T13:36:35.385Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"7031lb # [ 192.099657] niks3-server[1058]: 2026/09/20 13:36:35 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst7032lb # [ 192.110179] niks3-server[1058]: 2026/09/20 13:36:35 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7033lb # [ 192.119845] niks3-server[1058]: 2026/09/20 13:36:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign7034lb # [ 192.124968] niks3-server[1058]: 2026/09/20 13:36:35 INFO Signed narinfos id=67 count=17035worker2 # [ 192.102860] nix-grpc-daemon[1966]: time=2026-09-20T13:36:35.437Z level=INFO msg="Uploading 1 narinfos"7036lb # [ 192.130545] niks3-server[1058]: 2026/09/20 13:36:35 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls7037lb # [ 192.138484] niks3-server[1058]: 2026/09/20 13:36:35 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete7038lb # [ 192.145216] niks3-server[1058]: 2026/09/20 13:36:35 INFO Completed upload id=677039worker2 # [ 192.120318] nix-grpc-daemon[1966]: time=2026-09-20T13:36:35.454Z level=INFO msg="Upload complete. (266ms)"7040lb # [ 192.148448] niks3-server[1058]: 2026/09/20 13:36:35 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo7041worker2 # [ 192.127098] nix-grpc-daemon[1910]: ts=2026-09-20T13:36:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36442 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=16 outputs=17042worker1 # [ 192.164416] nix-grpc-daemon[1235]: ts=2026-09-20T13:36:35Z level=debug event=want client=7 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=07043worker2 # [ 192.135282] nix-grpc-daemon[1910]: ts=2026-09-20T13:36:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364427044worker1 # [ 192.168844] nix-grpc-daemon[1235]: ts=2026-09-20T13:36:35Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=17045worker2 # [ 192.145536] nix-grpc-daemon[1910]: ts=2026-09-20T13:36:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364427046worker2 # [ 192.151641] nix-daemon[868]: reaped child process 2023, status = succeeded7047worker2 # [ 192.158236] nix-grpc-daemon[1966]: time=2026-09-20T13:36:35.492Z level=INFO msg="All 2 paths already cached"7048client # copying 1 paths to 'lb:50051'...7049worker2 # [ 192.164791] nix-grpc-daemon[1910]: ts=2026-09-20T13:36:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:364427050client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...7051worker2 # [ 192.171885] nix-daemon[868]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)7052worker2 # [ 192.313535] nix-daemon[868]: accepted connection from pid 2047, user nix-grpc-daemon (trusted)7053worker2 # [ 192.325515] nix-daemon[868]: reaped child process 2050, status = succeeded7054worker2 # [ 192.472432] nix-daemon[868]: accepted connection from pid 2052, user nix-grpc-daemon (trusted)7055worker2 # [ 192.484064] nix-daemon[868]: reaped child process 2055, status = succeeded7056lb # [ 192.522667] niks3-server[1058]: 2026/09/20 13:36:35 INFO Received uploads request method=POST path=/api/pending_closures7057worker2 # [ 192.505438] nix-grpc-daemon[1966]: time=2026-09-20T13:36:35.839Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7058worker2 # [ 192.506830] nix-grpc-daemon[1966]: time=2026-09-20T13:36:35.839Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"7059lb # [ 192.547081] niks3-server[1058]: 2026/09/20 13:36:35 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst7060lb # [ 192.558268] niks3-server[1058]: 2026/09/20 13:36:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign7061lb # [ 192.564994] niks3-server[1058]: 2026/09/20 13:36:35 INFO Signed narinfos id=68 count=17062worker2 # [ 192.543305] nix-grpc-daemon[1966]: time=2026-09-20T13:36:35.877Z level=INFO msg="Uploading 1 narinfos"7063lb # [ 192.568510] niks3-server[1058]: 2026/09/20 13:36:35 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls7064lb # [ 192.581982] niks3-server[1058]: 2026/09/20 13:36:35 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete7065lb # [ 192.586733] niks3-server[1058]: 2026/09/20 13:36:35 INFO Completed upload id=687066worker2 # [ 192.563902] nix-grpc-daemon[1966]: time=2026-09-20T13:36:35.898Z level=INFO msg="Upload complete. (385ms)"7067lb # [ 192.589596] niks3-server[1058]: 2026/09/20 13:36:35 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo7068worker2 # [ 192.566213] nix-grpc-daemon[1910]: ts=2026-09-20T13:36:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36442 duration_s=0 paths=1 nar_bytes_in=7527069worker2 # [ 192.570119] nix-grpc-daemon[1910]: ts=2026-09-20T13:36:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364427070worker2 # [ 192.577054] nix-daemon[868]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)7071worker2 # [ 192.580221] nix-daemon[868]: reaped child process 2045, status = succeeded7072client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...7073client # farm-top> worker2-next: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7074worker2 # [ 192.585198] nix-daemon[868]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)7075client # farm-top> LOG-farm-top-drain7076worker2 # [ 193.410642] nix-daemon[868]: reaped child process 2059, status = succeeded7077worker2 # [ 193.559799] nix-daemon[868]: accepted connection from pid 2074, user nix-grpc-daemon (trusted)7078worker2 # [ 193.570533] nix-daemon[868]: reaped child process 2077, status = succeeded7079lb # [ 193.609907] niks3-server[1058]: 2026/09/20 13:36:36 INFO Received uploads request method=POST path=/api/pending_closures7080worker2 # [ 193.591125] nix-grpc-daemon[1966]: time=2026-09-20T13:36:36.925Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7081worker2 # [ 193.592590] nix-grpc-daemon[1966]: time=2026-09-20T13:36:36.927Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"7082lb # [ 193.630340] niks3-server[1058]: 2026/09/20 13:36:36 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7083lb # [ 193.635277] niks3-server[1058]: 2026/09/20 13:36:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign7084lb # [ 193.637349] niks3-server[1058]: 2026/09/20 13:36:36 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7085lb # [ 193.641054] niks3-server[1058]: 2026/09/20 13:36:36 INFO Signed narinfos id=69 count=17086worker2 # [ 193.616084] nix-grpc-daemon[1966]: time=2026-09-20T13:36:36.950Z level=INFO msg="Uploading 1 narinfos"7087lb # [ 193.644049] niks3-server[1058]: 2026/09/20 13:36:36 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7088lb # [ 193.648307] niks3-server[1058]: 2026/09/20 13:36:36 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete7089lb # [ 193.651053] niks3-server[1058]: 2026/09/20 13:36:36 INFO Completed upload id=697090worker2 # [ 193.626056] nix-grpc-daemon[1966]: time=2026-09-20T13:36:36.960Z level=INFO msg="Upload complete. (234ms)"7091lb # [ 193.653593] niks3-server[1058]: 2026/09/20 13:36:36 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7092worker2 # [ 193.629143] nix-grpc-daemon[1910]: ts=2026-09-20T13:36:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36442 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=17 outputs=17093worker2 # [ 193.634791] nix-daemon[868]: reaped child process 2057, status = succeeded7094client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&2, in 5.13 seconds)7095client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath7096client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath, in 0.21 seconds)7097worker1: must succeed: rm /nix/.rw-store/fill7098worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.13 seconds)7099worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7100worker1 # [ 194.729570] nix-grpc-daemon[1235]: ts=2026-09-20T13:36:37Z level=info event=healthy reason=min_free7101worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.09 seconds)7102(finished: subtest: low disk drains a worker and builds go to the other, in 8.73 seconds)7103(finished: run the VM test script, in 196.11 seconds)7104test script finished in 196.31s7105cleanup7106kill QemuMachine (pid 51)7107client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7108kill QemuMachine (pid 50)7109lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7110kill QemuMachine (pid 49)7111worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7112kill QemuMachine (pid 52)7113worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7114(finished: cleanup, in 0.97 seconds)7115additionally exposed symbols:7116 client, lb, worker1, worker2,7117 vlan1,7118 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh