vm-test-run-nix-grpc-farm log

succeeded `vm-test-run-nix-grpc-farm` `checks.x86_64-linux.farm` · build #312 · raw

1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker2 # Disk image does not exist, creating the virtualisation disk image...13client # Disk image does not exist, creating the virtualisation disk image...14worker2: QEMU running (pid 52)15client: QEMU running (pid 49)16worker2 # Formatting '/build/vm-state-worker2/tmp.ZguDRj82tl', fmt=raw size=107374182417client # Formatting '/build/vm-state-client/tmp.zsaUwSD7UU', fmt=raw size=107374182418worker1: QEMU running (pid 51)19lb # Disk image does not exist, creating the virtualisation disk image...20client # mke2fs 1.47.4 (6-Mar-2025)21worker2 # mke2fs 1.47.4 (6-Mar-2025)22lb # Formatting '/build/vm-state-lb/tmp.mmVBTZLGcD', fmt=raw size=107374182423client # Discarding device blocks: 0/262144 done24worker2 # Discarding device blocks: 0/262144 done25worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes26lb # mke2fs 1.47.4 (6-Mar-2025)27lb: QEMU running (pid 50)28worker1 # Disk image does not exist, creating the virtualisation disk image...29lb # Discarding device blocks: 0/262144 done30client # Creating filesystem with 262144 4k blocks and 65536 inodes31lb # Creating filesystem with 262144 4k blocks and 65536 inodes32client # Filesystem UUID: 92f1f972-f9ea-41d1-9733-9e62f627ce0933worker2 # Filesystem UUID: ff24f9b4-b30c-4155-b4ba-22ea38f3ce3034worker2 # Superblock backups stored on blocks:35lb # Filesystem UUID: 7a9f0225-026b-4e34-8125-ea2b1f1a752036worker1 # Formatting '/build/vm-state-worker1/tmp.DmCVIbiqlw', fmt=raw size=107374182437(finished: start all VMs, in 0.25 seconds)38lb # Superblock backups stored on blocks:39worker1 # mke2fs 1.47.4 (6-Mar-2025)40client: waiting for unit network-addresses-eth1.service41lb # 32768, 98304, 163840, 22937642worker1 # Discarding device blocks: 0/262144 done43client: waiting for the VM to finish booting44lb # 45worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes46lb # Allocating group tables: 0/8 done47worker1 # Filesystem UUID: 47e6e4eb-c1dc-43e5-9534-729d8f39560048client # Superblock backups stored on blocks:49worker2 # 32768, 98304, 163840, 22937650lb # Writing inode tables: 0/8 done51worker1 # Superblock backups stored on blocks:52client # 32768, 98304, 163840, 22937653worker2 # 54lb # Creating journal (8192 blocks): done55worker1 # 32768, 98304, 163840, 22937656client # 57worker2 # Allocating group tables: 0/8 done58lb # Writing superblocks and filesystem accounting information: 0/8 done59worker1 # 60client # Allocating group tables: 0/8 done61worker2 # Writing inode tables: 0/8 done62lb # 63worker1 # Allocating group tables: 0/8 done64client # Writing inode tables: 0/8 done65worker2 # Creating journal (8192 blocks): done66lb # Virtualisation disk image created.67worker1 # Writing inode tables: 0/8 done68client # Creating journal (8192 blocks): done69worker1 # Creating journal (8192 blocks): done70client # Writing superblocks and filesystem accounting information: 0/8 done71worker1 # Writing superblocks and filesystem accounting information: 0/8 done72client # 73worker1 # 74client # Virtualisation disk image created.75worker1 # Virtualisation disk image created.76worker2 # Writing superblocks and filesystem accounting information: 0/8 done77worker2 # 78worker2 # Virtualisation disk image created.79worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)80client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)81lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)82worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)83worker2 # 84worker2 # 85worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0086client # 87client # 88client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0089worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...90worker2 # 91worker2 # 92worker2 # 93worker2 # 94worker2 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB0095client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...96client # 97client # 98client # 99client # 100client # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00101worker2 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...102worker2 # 103worker2 # 104client # Press Ctrl-B to configure iPXE (PCI 00:08.0)...105client # 106client # 107lb # 108lb # 109worker1 # 110worker1 # 111lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00112worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00113lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...114lb # 115lb # 116lb # 117lb # 118worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...119worker1 # 120worker1 # 121worker1 # 122worker1 # 123lb # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00124worker1 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00125lb # Press Ctrl-B to configure iPXE (PCI 00:08.0)...126lb # 127lb # 128worker1 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...129worker1 # 130worker1 # 131worker2 # Booting from ROM...132client # Booting from ROM...133lb # Booting from ROM...134worker1 # Booting from ROM...135worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026136worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/zsm1ghnr8qh8b1411vdyz80y3scphy1w-nixos-system-worker2-test/init regInfo=/nix/store/7y1fcrz08lgh7fjvd2r9plg4jvc3455l-closure-info/registration console=ttyS0,115200n8 console=tty0137worker2 # [ 0.000000] BIOS-provided physical RAM map:138worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable139worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved140worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved141worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable142worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved143worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved144worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved145worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved146worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved147worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved148worker2 # [ 0.000000] NX (Execute Disable) protection: active149worker2 # [ 0.000000] APIC: Static calls initialized150worker2 # [ 0.000000] SMBIOS 2.8 present.151worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014152worker2 # [ 0.000000] DMI: Memory slots populated: 1/1153worker2 # [ 0.000000] Hypervisor detected: KVM154worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000155worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00156worker2 # [ 0.000000] kvm-clock: using sched offset of 492282750 cycles157worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns158lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026159worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor160worker2 # [ 0.000821] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000161worker2 # [ 0.000848] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs162worker2 # [ 0.000851] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT163worker2 # [ 0.002759] found SMP MP-table at [mem 0x000f5450-0x000f545f]164lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0qln5mjy767qjp5wbmxjg8jay8k05dsm-nixos-system-lb-test/init regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration console=ttyS0,115200n8 console=tty0165worker2 # [ 0.002772] Using GB pages for direct mapping166lb # [ 0.000000] BIOS-provided physical RAM map:167worker2 # [ 0.002867] RAMDISK: [mem 0x5e367000-0x5ffcffff]168worker2 # [ 0.002875] ACPI: Early table checksum verification disabled169lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable170worker2 # [ 0.002879] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )171lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved172worker2 # [ 0.002882] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)173lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved174lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable175worker2 # [ 0.002887] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)176lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved177worker2 # [ 0.002895] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)178worker2 # [ 0.002897] ACPI: FACS 0x000000005FFE0000 000040179lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved180lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved181worker2 # [ 0.002899] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)182lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved183worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026184worker2 # [ 0.002900] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)185lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved186worker2 # [ 0.002902] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)187lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved188lb # [ 0.000000] NX (Execute Disable) protection: active189worker2 # [ 0.002903] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)190lb # [ 0.000000] APIC: Static calls initialized191lb # [ 0.000000] SMBIOS 2.8 present.192worker2 # [ 0.002905] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]193worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/r2avc8vpviyzpfayjcaxb3bd29b72jwf-nixos-system-worker1-test/init regInfo=/nix/store/ndb81zjw0jgimy116vc63p7fz1nz59zg-closure-info/registration console=ttyS0,115200n8 console=tty0194worker2 # [ 0.002906] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]195worker1 # [ 0.000000] BIOS-provided physical RAM map:196lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014197lb # [ 0.000000] DMI: Memory slots populated: 1/1198worker2 # [ 0.002906] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]199worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable200client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026201lb # [ 0.000000] Hypervisor detected: KVM202worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved203lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000204worker2 # [ 0.002907] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]205lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00206worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved207worker2 # [ 0.002907] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]208lb # [ 0.000001] kvm-clock: using sched offset of 503237254 cycles209worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable210worker2 # [ 0.002908] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]211worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved212worker2 # [ 0.002908] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]213lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns214client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/10858v71rm0q02ppdw768ccq9np319pi-nixos-system-client-test/init regInfo=/nix/store/gh9mxw0h96sjj5fi79aa4b3naclcg1a4-closure-info/registration console=ttyS0,115200n8 console=tty0215worker2 # [ 0.003134] No NUMA configuration found216lb # [ 0.000005] tsc: Detected 2400.012 MHz processor217worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved218client # [ 0.000000] BIOS-provided physical RAM map:219worker2 # [ 0.003135] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]220lb # [ 0.000816] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000221worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved222client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable223worker2 # [ 0.003138] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]224lb # [ 0.000843] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs225worker2 # [ 0.005464] Zone ranges:226client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved227worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved228worker2 # [ 0.005465] DMA [mem 0x0000000000001000-0x0000000000ffffff]229lb # [ 0.000846] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT230worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved231client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved232worker2 # [ 0.005466] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]233lb # [ 0.002751] found SMP MP-table at [mem 0x000f5450-0x000f545f]234worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved235worker2 # [ 0.005468] Normal empty236lb # [ 0.002763] Using GB pages for direct mapping237client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable238worker2 # [ 0.005469] Device empty239worker1 # [ 0.000000] NX (Execute Disable) protection: active240client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved241lb # [ 0.002845] RAMDISK: [mem 0x5e367000-0x5ffcffff]242worker1 # [ 0.000000] APIC: Static calls initialized243worker2 # [ 0.005469] Movable zone start for each node244worker1 # [ 0.000000] SMBIOS 2.8 present.245worker2 # [ 0.005470] Early memory node ranges246lb # [ 0.002852] ACPI: Early table checksum verification disabled247client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved248lb # [ 0.002855] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )249worker2 # [ 0.005470] node 0: [mem 0x0000000000001000-0x000000000009efff]250client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved251worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014252worker2 # [ 0.005471] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]253worker1 # [ 0.000000] DMI: Memory slots populated: 1/1254lb # [ 0.002860] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)255client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved256worker1 # [ 0.000000] Hypervisor detected: KVM257worker2 # [ 0.005472] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]258client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved259worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000260lb # [ 0.002864] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)261worker2 # [ 0.005492] On node 0, zone DMA: 1 pages in unavailable ranges262worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00263client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved264worker2 # [ 0.005761] On node 0, zone DMA: 97 pages in unavailable ranges265lb # [ 0.002871] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)266worker1 # [ 0.000000] kvm-clock: using sched offset of 517293764 cycles267client # [ 0.000000] NX (Execute Disable) protection: active268lb # [ 0.002873] ACPI: FACS 0x000000005FFE0000 000040269worker2 # [ 0.033157] On node 0, zone DMA32: 40 pages in unavailable ranges270client # [ 0.000000] APIC: Static calls initialized271worker2 # [ 0.033636] ACPI: PM-Timer IO Port: 0x608272client # [ 0.000000] SMBIOS 2.8 present.273worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns274lb # [ 0.002875] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)275worker2 # [ 0.033649] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])276worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor277client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014278lb # [ 0.002876] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)279worker1 # [ 0.000819] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000280worker2 # [ 0.033674] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23281client # [ 0.000000] DMI: Memory slots populated: 1/1282client # [ 0.000000] Hypervisor detected: KVM283worker2 # [ 0.033677] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)284lb # [ 0.002878] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)285worker1 # [ 0.000846] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs286client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000287worker2 # [ 0.033679] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)288worker1 # [ 0.000848] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT289lb # [ 0.002879] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)290client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00291worker2 # [ 0.033680] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)292client # [ 0.000000] kvm-clock: using sched offset of 513511222 cycles293lb # [ 0.002880] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]294worker1 # [ 0.002748] found SMP MP-table at [mem 0x000f5450-0x000f545f]295worker1 # [ 0.002759] Using GB pages for direct mapping296worker2 # [ 0.033681] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)297lb # [ 0.002882] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]298worker1 # [ 0.002865] RAMDISK: [mem 0x5e367000-0x5ffcffff]299client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns300worker2 # [ 0.033682] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)301worker1 # [ 0.002872] ACPI: Early table checksum verification disabled302lb # [ 0.002882] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]303client # [ 0.000005] tsc: Detected 2400.012 MHz processor304worker1 # [ 0.002875] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )305worker2 # [ 0.033684] ACPI: Using ACPI (MADT) for SMP configuration information306client # [ 0.000826] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000307lb # [ 0.002883] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]308worker2 # [ 0.033685] ACPI: HPET id: 0x8086a201 base: 0xfed00000309worker2 # [ 0.033689] TSC deadline timer available310lb # [ 0.002883] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]311client # [ 0.000855] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs312worker1 # [ 0.002879] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)313worker2 # [ 0.033693] CPU topo: Max. logical packages: 1314lb # [ 0.002883] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]315client # [ 0.000858] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT316worker1 # [ 0.002884] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)317worker2 # [ 0.033694] CPU topo: Max. logical dies: 1318worker2 # [ 0.033694] CPU topo: Max. dies per package: 1319client # [ 0.002793] found SMP MP-table at [mem 0x000f5450-0x000f545f]320lb # [ 0.002884] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]321worker1 # [ 0.002891] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)322worker2 # [ 0.033698] CPU topo: Max. threads per core: 1323client # [ 0.002808] Using GB pages for direct mapping324lb # [ 0.003119] No NUMA configuration found325worker1 # [ 0.002893] ACPI: FACS 0x000000005FFE0000 000040326worker2 # [ 0.033698] CPU topo: Num. cores per package: 1327client # [ 0.002962] RAMDISK: [mem 0x5e366000-0x5ffcffff]328lb # [ 0.003120] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]329worker2 # [ 0.033699] CPU topo: Num. threads per package: 1330client # [ 0.002971] ACPI: Early table checksum verification disabled331worker1 # [ 0.002895] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)332lb # [ 0.003122] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]333lb # [ 0.005440] Zone ranges:334worker2 # [ 0.033699] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs335client # [ 0.002975] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )336worker1 # [ 0.002896] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)337lb # [ 0.005441] DMA [mem 0x0000000000001000-0x0000000000ffffff]338worker2 # [ 0.033719] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()339client # [ 0.002980] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)340lb # [ 0.005443] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]341worker1 # [ 0.002898] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)342lb # [ 0.005444] Normal empty343worker2 # [ 0.033752] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]344lb # [ 0.005444] Device empty345client # [ 0.002985] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)346worker1 # [ 0.002899] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)347lb # [ 0.005445] Movable zone start for each node348worker2 # [ 0.033753] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]349lb # [ 0.005446] Early memory node ranges350client # [ 0.002994] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)351worker1 # [ 0.002900] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]352worker2 # [ 0.033755] [mem 0x60000000-0xafffffff] available for PCI devices353lb # [ 0.005446] node 0: [mem 0x0000000000001000-0x000000000009efff]354client # [ 0.002996] ACPI: FACS 0x000000005FFE0000 000040355worker2 # [ 0.033757] Booting paravirtualized kernel on KVM356worker1 # [ 0.002901] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]357lb # [ 0.005447] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]358client # [ 0.002997] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)359worker1 # [ 0.002902] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]360worker2 # [ 0.033760] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns361lb # [ 0.005448] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]362worker1 # [ 0.002902] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]363client # [ 0.002999] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)364lb # [ 0.005469] On node 0, zone DMA: 1 pages in unavailable ranges365worker2 # [ 0.038282] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1366worker1 # [ 0.002903] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]367lb # [ 0.005739] On node 0, zone DMA: 97 pages in unavailable ranges368client # [ 0.003001] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)369worker2 # [ 0.040632] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152370lb # [ 0.033459] On node 0, zone DMA32: 40 pages in unavailable ranges371worker1 # [ 0.002903] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]372worker2 # [ 0.040678] kvm-guest: PV spinlocks disabled, single CPU373client # [ 0.003002] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)374lb # [ 0.033938] ACPI: PM-Timer IO Port: 0x608375worker1 # [ 0.002904] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]376lb # [ 0.033952] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])377client # [ 0.003003] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]378worker1 # [ 0.003132] No NUMA configuration found379lb # [ 0.033982] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23380client # [ 0.003004] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]381worker1 # [ 0.003133] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]382lb # [ 0.033985] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)383worker1 # [ 0.003136] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]384client # [ 0.003005] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]385worker1 # [ 0.005813] Zone ranges:386lb # [ 0.033987] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)387client # [ 0.003005] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]388worker2 # [ 0.040679] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/zsm1ghnr8qh8b1411vdyz80y3scphy1w-nixos-system-worker2-test/init regInfo=/nix/store/7y1fcrz08lgh7fjvd2r9plg4jvc3455l-closure-info/registration console=ttyS0,115200n8 console=tty0389worker1 # [ 0.005814] DMA [mem 0x0000000000001000-0x0000000000ffffff]390lb # [ 0.033988] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)391worker1 # [ 0.005816] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]392client # [ 0.003006] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]393worker1 # [ 0.005817] Normal empty394lb # [ 0.033989] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)395worker1 # [ 0.005818] Device empty396client # [ 0.003006] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]397worker2 # [ 0.040776] Unknown kernel command line parameters "regInfo=/nix/store/7y1fcrz08lgh7fjvd2r9plg4jvc3455l-closure-info/registration", will be passed to user space.398worker1 # [ 0.005819] Movable zone start for each node399lb # [ 0.033990] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)400worker2 # [ 0.040789] random: crng init done401client # [ 0.003007] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]402worker1 # [ 0.005819] Early memory node ranges403lb # [ 0.033992] ACPI: Using ACPI (MADT) for SMP configuration information404client # [ 0.003254] No NUMA configuration found405worker2 # [ 0.040790] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes406worker1 # [ 0.005820] node 0: [mem 0x0000000000001000-0x000000000009efff]407lb # [ 0.033993] ACPI: HPET id: 0x8086a201 base: 0xfed00000408client # [ 0.003255] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]409lb # [ 0.033998] TSC deadline timer available410worker2 # [ 0.043061] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)411worker1 # [ 0.005821] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]412client # [ 0.003257] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]413lb # [ 0.034003] CPU topo: Max. logical packages: 1414client # [ 0.005947] Zone ranges:415worker1 # [ 0.005822] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]416worker2 # [ 0.044561] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)417lb # [ 0.034003] CPU topo: Max. logical dies: 1418client # [ 0.005949] DMA [mem 0x0000000000001000-0x0000000000ffffff]419worker2 # [ 0.044596] Fallback order for Node 0: 0420lb # [ 0.034004] CPU topo: Max. dies per package: 1421worker1 # [ 0.005845] On node 0, zone DMA: 1 pages in unavailable ranges422worker1 # [ 0.006113] On node 0, zone DMA: 97 pages in unavailable ranges423lb # [ 0.034007] CPU topo: Max. threads per core: 1424client # [ 0.005952] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]425worker2 # [ 0.044599] Built 1 zonelists, mobility grouping on. Total pages: 393078426client # [ 0.005953] Normal empty427worker2 # [ 0.044601] Policy zone: DMA32428lb # [ 0.034008] CPU topo: Num. cores per package: 1429worker1 # [ 0.034105] On node 0, zone DMA32: 40 pages in unavailable ranges430client # [ 0.005954] Device empty431lb # [ 0.034008] CPU topo: Num. threads per package: 1432worker1 # [ 0.034593] ACPI: PM-Timer IO Port: 0x608433client # [ 0.005955] Movable zone start for each node434worker2 # [ 0.047341] mem auto-init: stack:all(zero), heap alloc:on, heap free:off435client # [ 0.005956] Early memory node ranges436lb # [ 0.034009] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs437worker1 # [ 0.034607] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])438worker2 # [ 0.050708] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1439client # [ 0.005957] node 0: [mem 0x0000000000001000-0x000000000009efff]440lb # [ 0.034030] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()441worker1 # [ 0.034634] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23442worker2 # [ 0.054284] allocated 3145728 bytes of page_ext443client # [ 0.005958] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]444worker2 # [ 0.064930] ftrace: allocating 48731 entries in 192 pages445worker1 # [ 0.034636] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)446lb # [ 0.034074] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]447worker2 # [ 0.064932] ftrace: allocated 192 pages with 2 groups448client # [ 0.005959] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]449worker1 # [ 0.034638] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)450worker2 # [ 0.066079] Dynamic Preempt: lazy451lb # [ 0.034075] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]452client # [ 0.005984] On node 0, zone DMA: 1 pages in unavailable ranges453worker2 # [ 0.066207] rcu: Preemptible hierarchical RCU implementation.454worker1 # [ 0.034639] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)455lb # [ 0.034077] [mem 0x60000000-0xafffffff] available for PCI devices456client # [ 0.006313] On node 0, zone DMA: 97 pages in unavailable ranges457worker2 # [ 0.066208] rcu: RCU event tracing is enabled.458lb # [ 0.034078] Booting paravirtualized kernel on KVM459worker1 # [ 0.034640] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)460client # [ 0.036167] On node 0, zone DMA32: 40 pages in unavailable ranges461worker2 # [ 0.066209] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.462worker1 # [ 0.034641] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)463client # [ 0.036644] ACPI: PM-Timer IO Port: 0x608464lb # [ 0.034082] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns465worker2 # [ 0.066210] Trampoline variant of Tasks RCU enabled.466client # [ 0.036660] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])467worker1 # [ 0.034644] ACPI: Using ACPI (MADT) for SMP configuration information468worker2 # [ 0.066211] Rude variant of Tasks RCU enabled.469lb # [ 0.038651] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1470worker2 # [ 0.066211] Tracing variant of Tasks RCU enabled.471client # [ 0.036689] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23472worker1 # [ 0.034645] ACPI: HPET id: 0x8086a201 base: 0xfed00000473worker1 # [ 0.034650] TSC deadline timer available474lb # [ 0.041203] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152475client # [ 0.036692] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)476worker2 # [ 0.066216] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.477worker1 # [ 0.034655] CPU topo: Max. logical packages: 1478lb # [ 0.041246] kvm-guest: PV spinlocks disabled, single CPU479worker1 # [ 0.034656] CPU topo: Max. logical dies: 1480client # [ 0.036693] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)481worker2 # [ 0.066216] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1482worker1 # [ 0.034656] CPU topo: Max. dies per package: 1483client # [ 0.036695] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)484worker1 # [ 0.034662] CPU topo: Max. threads per core: 1485worker2 # [ 0.066276] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.486client # [ 0.036696] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)487worker1 # [ 0.034662] CPU topo: Num. cores per package: 1488worker1 # [ 0.034663] CPU topo: Num. threads per package: 1489worker2 # [ 0.066278] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.490client # [ 0.036697] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)491worker1 # [ 0.034663] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs492client # [ 0.036699] ACPI: Using ACPI (MADT) for SMP configuration information493lb # [ 0.041248] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0qln5mjy767qjp5wbmxjg8jay8k05dsm-nixos-system-lb-test/init regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration console=ttyS0,115200n8 console=tty0494worker2 # [ 0.066279] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.495worker1 # [ 0.034686] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()496client # [ 0.036700] ACPI: HPET id: 0x8086a201 base: 0xfed00000497worker2 # [ 0.070729] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16498client # [ 0.036707] TSC deadline timer available499worker1 # [ 0.034721] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]500client # [ 0.036712] CPU topo: Max. logical packages: 1501worker2 # [ 0.071013] rcu: srcu_init: Setting srcu_struct sizes based on contention.502lb # [ 0.041340] Unknown kernel command line parameters "regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration", will be passed to user space.503client # [ 0.036713] CPU topo: Max. logical dies: 1504lb # [ 0.041353] random: crng init done505worker1 # [ 0.034723] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]506client # [ 0.036713] CPU topo: Max. dies per package: 1507worker2 # [ 0.071019] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns508worker1 # [ 0.034725] [mem 0x60000000-0xafffffff] available for PCI devices509lb # [ 0.041354] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes510client # [ 0.036717] CPU topo: Max. threads per core: 1511worker1 # [ 0.034727] Booting paravirtualized kernel on KVM512client # [ 0.036717] CPU topo: Num. cores per package: 1513lb # [ 0.043536] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)514worker2 # [ 0.071128] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)515client # [ 0.036717] CPU topo: Num. threads per package: 1516worker2 # [ 0.074710] Console: colour VGA+ 80x25517worker1 # [ 0.034730] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns518lb # [ 0.044624] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)519worker2 # [ 0.074713] printk: legacy console [tty0] enabled520client # [ 0.036718] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs521lb # [ 0.044657] Fallback order for Node 0: 0522worker2 # [ 0.104799] printk: legacy console [ttyS0] enabled523worker1 # [ 0.039205] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1524client # [ 0.036744] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()525lb # [ 0.044660] Built 1 zonelists, mobility grouping on. Total pages: 393078526worker2 # [ 0.202798] ACPI: Core revision 20250807527worker1 # [ 0.041441] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152528lb # [ 0.044661] Policy zone: DMA32529client # [ 0.036784] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]530worker1 # [ 0.041483] kvm-guest: PV spinlocks disabled, single CPU531lb # [ 0.047213] mem auto-init: stack:all(zero), heap alloc:on, heap free:off532client # [ 0.036786] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]533worker2 # [ 0.203701] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns534lb # [ 0.050536] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1535client # [ 0.036788] [mem 0x60000000-0xafffffff] available for PCI devices536lb # [ 0.054215] allocated 3145728 bytes of page_ext537client # [ 0.036790] Booting paravirtualized kernel on KVM538worker2 # [ 0.205282] APIC: Switch to symmetric I/O mode setup539lb # [ 0.064414] ftrace: allocating 48731 entries in 192 pages540worker2 # [ 0.206272] x2apic enabled541lb # [ 0.064417] ftrace: allocated 192 pages with 2 groups542client # [ 0.036794] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns543lb # [ 0.065274] Dynamic Preempt: lazy544worker1 # [ 0.041484] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/r2avc8vpviyzpfayjcaxb3bd29b72jwf-nixos-system-worker1-test/init regInfo=/nix/store/ndb81zjw0jgimy116vc63p7fz1nz59zg-closure-info/registration console=ttyS0,115200n8 console=tty0545worker2 # [ 0.207022] APIC: Switched APIC routing to: physical x2apic546lb # [ 0.065407] rcu: Preemptible hierarchical RCU implementation.547client # [ 0.041346] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1548lb # [ 0.065408] rcu: RCU event tracing is enabled.549client # [ 0.043748] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152550lb # [ 0.065408] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.551worker1 # [ 0.041584] Unknown kernel command line parameters "regInfo=/nix/store/ndb81zjw0jgimy116vc63p7fz1nz59zg-closure-info/registration", will be passed to user space.552client # [ 0.043796] kvm-guest: PV spinlocks disabled, single CPU553worker1 # [ 0.041598] random: crng init done554lb # [ 0.065410] Trampoline variant of Tasks RCU enabled.555worker2 # [ 0.208881] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1556lb # [ 0.065410] Rude variant of Tasks RCU enabled.557worker1 # [ 0.041598] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes558lb # [ 0.065410] Tracing variant of Tasks RCU enabled.559worker2 # [ 0.209840] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns560worker1 # [ 0.043807] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)561lb # [ 0.065412] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.562worker1 # [ 0.044935] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)563worker2 # [ 0.211460] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)564lb # [ 0.065412] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1565worker1 # [ 0.044970] Fallback order for Node 0: 0566client # [ 0.043797] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/10858v71rm0q02ppdw768ccq9np319pi-nixos-system-client-test/init regInfo=/nix/store/gh9mxw0h96sjj5fi79aa4b3naclcg1a4-closure-info/registration console=ttyS0,115200n8 console=tty0567lb # [ 0.065461] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.568worker1 # [ 0.044973] Built 1 zonelists, mobility grouping on. Total pages: 393078569worker2 # [ 0.212551] x86/cpu: User Mode Instruction Prevention (UMIP) activated570worker1 # [ 0.044974] Policy zone: DMA32571lb # [ 0.065463] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.572worker2 # [ 0.213590] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127573client # [ 0.043901] Unknown kernel command line parameters "regInfo=/nix/store/gh9mxw0h96sjj5fi79aa4b3naclcg1a4-closure-info/registration", will be passed to user space.574worker1 # [ 0.047793] mem auto-init: stack:all(zero), heap alloc:on, heap free:off575client # [ 0.043914] random: crng init done576worker1 # [ 0.051122] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1577lb # [ 0.065464] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.578worker2 # [ 0.214458] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0579client # [ 0.043915] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes580worker1 # [ 0.054557] allocated 3145728 bytes of page_ext581lb # [ 0.069912] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16582worker1 # [ 0.064917] ftrace: allocating 48731 entries in 192 pages583client # [ 0.046273] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)584lb # [ 0.070181] rcu: srcu_init: Setting srcu_struct sizes based on contention.585worker1 # [ 0.064919] ftrace: allocated 192 pages with 2 groups586worker2 # [ 0.215461] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto587worker1 # [ 0.065787] Dynamic Preempt: lazy588client # [ 0.047795] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)589worker1 # [ 0.065905] rcu: Preemptible hierarchical RCU implementation.590lb # [ 0.070188] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns591client # [ 0.047832] Fallback order for Node 0: 0592worker2 # [ 0.217457] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl593worker1 # [ 0.065906] rcu: RCU event tracing is enabled.594client # [ 0.047836] Built 1 zonelists, mobility grouping on. Total pages: 393078595lb # [ 0.070295] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)596client # [ 0.047838] Policy zone: DMA32597worker1 # [ 0.065907] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.598worker2 # [ 0.218457] Transient Scheduler Attacks: Vulnerable: No microcode599lb # [ 0.073835] Console: colour VGA+ 80x25600worker1 # [ 0.065908] Trampoline variant of Tasks RCU enabled.601client # [ 0.050521] mem auto-init: stack:all(zero), heap alloc:on, heap free:off602lb # [ 0.073838] printk: legacy console [tty0] enabled603worker2 # [ 0.219457] Spectre V2 : Mitigation: Enhanced / Automatic IBRS604worker1 # [ 0.065909] Rude variant of Tasks RCU enabled.605lb # [ 0.103401] printk: legacy console [ttyS0] enabled606client # [ 0.054190] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1607worker1 # [ 0.065909] Tracing variant of Tasks RCU enabled.608client # [ 0.057843] allocated 3145728 bytes of page_ext609worker2 # [ 0.220457] Speculative Return Stack Overflow: Mitigation: Safe RET610lb # [ 0.199973] ACPI: Core revision 20250807611client # [ 0.068553] ftrace: allocating 48731 entries in 192 pages612worker1 # [ 0.065910] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.613client # [ 0.068556] ftrace: allocated 192 pages with 2 groups614worker1 # [ 0.065910] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1615client # [ 0.069515] Dynamic Preempt: lazy616lb # [ 0.200831] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns617client # [ 0.069682] rcu: Preemptible hierarchical RCU implementation.618worker1 # [ 0.065932] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.619lb # [ 0.202411] APIC: Switch to symmetric I/O mode setup620client # [ 0.069683] rcu: RCU event tracing is enabled.621lb # [ 0.203352] x2apic enabled622worker1 # [ 0.065934] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.623client # [ 0.069684] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.624worker2 # [ 0.221457] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization625client # [ 0.069686] Trampoline variant of Tasks RCU enabled.626lb # [ 0.204076] APIC: Switched APIC routing to: physical x2apic627worker1 # [ 0.065935] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.628worker2 # [ 0.223465] Spectre V2 : Enabling IBPB for BPF629client # [ 0.069686] Rude variant of Tasks RCU enabled.630worker1 # [ 0.070417] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16631client # [ 0.069687] Tracing variant of Tasks RCU enabled.632worker2 # [ 0.224458] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier633worker1 # [ 0.070697] rcu: srcu_init: Setting srcu_struct sizes based on contention.634client # [ 0.069687] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.635lb # [ 0.205835] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1636worker2 # [ 0.225458] active return thunk: srso_alias_return_thunk637client # [ 0.069688] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1638worker1 # [ 0.070703] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns639lb # [ 0.206771] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns640client # [ 0.069749] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.641worker2 # [ 0.226479] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'642worker1 # [ 0.070811] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)643worker1 # [ 0.074353] Console: colour VGA+ 80x25644worker2 # [ 0.227457] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'645client # [ 0.069751] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.646lb # [ 0.208335] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)647worker1 # [ 0.074357] printk: legacy console [tty0] enabled648worker2 # [ 0.228457] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'649worker1 # [ 0.103887] printk: legacy console [ttyS0] enabled650client # [ 0.069752] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.651lb # [ 0.210419] x86/cpu: User Mode Instruction Prevention (UMIP) activated652worker1 # [ 0.199903] ACPI: Core revision 20250807653client # [ 0.074654] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16654worker2 # [ 0.229457] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'655lb # [ 0.211508] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127656client # [ 0.074952] rcu: srcu_init: Setting srcu_struct sizes based on contention.657worker2 # [ 0.230457] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'658lb # [ 0.212333] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0659worker1 # [ 0.200794] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns660client # [ 0.074959] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns661worker2 # [ 0.231457] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'662worker1 # [ 0.202413] APIC: Switch to symmetric I/O mode setup663client # [ 0.075090] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)664lb # [ 0.213336] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto665worker1 # [ 0.203453] x2apic enabled666worker2 # [ 0.232457] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'667client # [ 0.078643] Console: colour VGA+ 80x25668client # [ 0.078647] printk: legacy console [tty0] enabled669worker2 # [ 0.233457] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256670worker1 # [ 0.204181] APIC: Switched APIC routing to: physical x2apic671lb # [ 0.214333] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl672client # [ 0.108377] printk: legacy console [ttyS0] enabled673worker2 # [ 0.235163] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64674lb # [ 0.215333] Transient Scheduler Attacks: Vulnerable: No microcode675client # [ 0.205110] ACPI: Core revision 20250807676worker2 # [ 0.235457] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512677worker1 # [ 0.206042] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1678lb # [ 0.216332] Spectre V2 : Mitigation: Enhanced / Automatic IBRS679client # [ 0.206011] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns680worker2 # [ 0.236457] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024681lb # [ 0.217332] Speculative Return Stack Overflow: Mitigation: Safe RET682worker1 # [ 0.207022] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns683client # [ 0.207580] APIC: Switch to symmetric I/O mode setup684worker2 # [ 0.237457] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8685client # [ 0.208531] x2apic enabled686worker1 # [ 0.208688] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)687worker2 # [ 0.238457] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.688client # [ 0.209322] APIC: Switched APIC routing to: physical x2apic689worker1 # [ 0.210705] x86/cpu: User Mode Instruction Prevention (UMIP) activated690lb # [ 0.218332] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization691worker1 # [ 0.211683] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127692client # [ 0.211147] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1693lb # [ 0.220341] Spectre V2 : Enabling IBPB for BPF694worker1 # [ 0.212685] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0695lb # [ 0.221107] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier696client # [ 0.212206] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns697lb # [ 0.222333] active return thunk: srso_alias_return_thunk698worker1 # [ 0.213689] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto699client # [ 0.213905] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)700lb # [ 0.223355] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'701worker1 # [ 0.214686] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl702client # [ 0.215992] x86/cpu: User Mode Instruction Prevention (UMIP) activated703lb # [ 0.224332] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'704worker1 # [ 0.216686] Transient Scheduler Attacks: Vulnerable: No microcode705client # [ 0.217037] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127706lb # [ 0.225332] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'707worker1 # [ 0.217685] Spectre V2 : Mitigation: Enhanced / Automatic IBRS708client # [ 0.217902] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0709lb # [ 0.226332] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'710worker1 # [ 0.218685] Speculative Return Stack Overflow: Mitigation: Safe RET711lb # [ 0.227332] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'712client # [ 0.218905] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto713lb # [ 0.228332] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'714client # [ 0.219901] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl715lb # [ 0.229332] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'716client # [ 0.221902] Transient Scheduler Attacks: Vulnerable: No microcode717worker1 # [ 0.219684] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization718lb # [ 0.231276] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256719client # [ 0.222901] Spectre V2 : Mitigation: Enhanced / Automatic IBRS720worker1 # [ 0.221695] Spectre V2 : Enabling IBPB for BPF721lb # [ 0.232333] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64722client # [ 0.223901] Speculative Return Stack Overflow: Mitigation: Safe RET723lb # [ 0.233332] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512724worker1 # [ 0.222686] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier725lb # [ 0.234332] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024726worker1 # [ 0.223686] active return thunk: srso_alias_return_thunk727lb # [ 0.235333] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8728worker1 # [ 0.224711] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'729client # [ 0.224901] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization730worker1 # [ 0.225685] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'731lb # [ 0.236333] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.732client # [ 0.226910] Spectre V2 : Enabling IBPB for BPF733worker1 # [ 0.226685] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'734worker1 # [ 0.227685] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'735client # [ 0.227652] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier736client # [ 0.228902] active return thunk: srso_alias_return_thunk737worker1 # [ 0.228685] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'738worker1 # [ 0.229685] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'739client # [ 0.229922] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'740worker1 # [ 0.230684] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'741client # [ 0.230901] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'742worker1 # [ 0.232594] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256743client # [ 0.231902] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'744worker1 # [ 0.233442] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64745client # [ 0.232901] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'746worker1 # [ 0.234400] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512747client # [ 0.233901] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'748worker1 # [ 0.235424] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024749client # [ 0.234902] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'750worker1 # [ 0.236422] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8751client # [ 0.235903] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'752worker1 # [ 0.237392] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.753client # [ 0.237903] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256754client # [ 0.238901] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64755client # [ 0.239902] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512756client # [ 0.240902] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024757client # [ 0.241901] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8758worker2 # [ 0.274274] Freeing SMP alternatives memory: 44K759worker2 # [ 0.275034] pid_max: default: 32768 minimum: 301760client # [ 0.242901] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.761worker2 # [ 0.275558] LSM: initializing lsm=capability,landlock,yama,bpf,ima762worker2 # [ 0.276576] landlock: Up and running.763worker2 # [ 0.277458] Yama: becoming mindful.764worker2 # [ 0.278563] LSM support for eBPF active765worker2 # [ 0.279312] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)766worker2 # [ 0.280495] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)767worker2 # [ 0.282969] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)768worker2 # [ 0.284013] Performance Events: Fam17h+ core perfctr, AMD PMU driver.769worker2 # [ 0.284462] ... version: 2770worker2 # [ 0.285214] ... bit width: 48771worker2 # [ 0.285461] ... generic counters: 6772worker2 # [ 0.286209] ... generic bitmap: 000000000000003f773worker2 # [ 0.286473] ... fixed-purpose counters: 0774worker2 # [ 0.287213] ... fixed-purpose bitmap: 0000000000000000775worker2 # [ 0.287459] ... value mask: 0000ffffffffffff776worker2 # [ 0.288400] ... max period: 00007fffffffffff777worker2 # [ 0.289194] ... global_ctrl mask: 000000000000003f778lb # [ 0.272612] Freeing SMP alternatives memory: 44K779worker2 # [ 0.289586] signal: max sigframe size: 3376780lb # [ 0.273336] pid_max: default: 32768 minimum: 301781worker2 # [ 0.290388] rcu: Hierarchical SRCU implementation.782lb # [ 0.274423] LSM: initializing lsm=capability,landlock,yama,bpf,ima783worker2 # [ 0.291098] rcu: Max phase no-delay instances is 400.784lb # [ 0.275446] landlock: Up and running.785lb # [ 0.276086] Yama: becoming mindful.786lb # [ 0.277018] LSM support for eBPF active787lb # [ 0.277478] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)788lb # [ 0.278370] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)789worker2 # [ 0.296184] smp: Bringing up secondary CPUs ...790worker2 # [ 0.296475] smp: Brought up 1 node, 1 CPU791lb # [ 0.281626] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)792worker2 # [ 0.297208] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)793lb # [ 0.282929] Performance Events: Fam17h+ core perfctr, AMD PMU driver.794lb # [ 0.283343] ... version: 2795worker2 # [ 0.297669] Memory: 1454564K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)796worker1 # [ 0.274743] Freeing SMP alternatives memory: 44K797lb # [ 0.284073] ... bit width: 48798worker2 # [ 0.298698] devtmpfs: initialized799worker1 # [ 0.275517] pid_max: default: 32768 minimum: 301800lb # [ 0.284337] ... generic counters: 6801worker2 # [ 0.299488] x86/mm: Memory block size: 128MB802worker1 # [ 0.276316] LSM: initializing lsm=capability,landlock,yama,bpf,ima803lb # [ 0.285058] ... generic bitmap: 000000000000003f804worker1 # [ 0.276793] landlock: Up and running.805lb # [ 0.285336] ... fixed-purpose counters: 0806worker1 # [ 0.277685] Yama: becoming mindful.807lb # [ 0.286044] ... fixed-purpose bitmap: 0000000000000000808worker2 # [ 0.301309] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)809worker1 # [ 0.278893] LSM support for eBPF active810lb # [ 0.286335] ... value mask: 0000ffffffffffff811worker2 # [ 0.302367] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).812worker1 # [ 0.279642] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)813lb # [ 0.287232] ... max period: 00007fffffffffff814worker2 # [ 0.303548] pinctrl core: initialized pinctrl subsystem815client # [ 0.279660] Freeing SMP alternatives memory: 44K816lb # [ 0.288010] ... global_ctrl mask: 000000000000003f817worker1 # [ 0.280739] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)818client # [ 0.280459] pid_max: default: 32768 minimum: 301819lb # [ 0.288434] signal: max sigframe size: 3376820worker2 # [ 0.304681] PM: RTC time: 12:21:08, date: 2026-09-20821client # [ 0.281005] LSM: initializing lsm=capability,landlock,yama,bpf,ima822lb # [ 0.289196] rcu: Hierarchical SRCU implementation.823client # [ 0.282033] landlock: Up and running.824lb # [ 0.289903] rcu: Max phase no-delay instances is 400.825client # [ 0.282902] Yama: becoming mindful.826worker1 # [ 0.283591] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)827client # [ 0.283910] LSM support for eBPF active828worker2 # [ 0.308302] NET: Registered PF_NETLINK/PF_ROUTE protocol family829worker1 # [ 0.284239] Performance Events: Fam17h+ core perfctr, AMD PMU driver.830client # [ 0.284679] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)831worker1 # [ 0.284690] ... version: 2832worker1 # [ 0.285375] ... bit width: 48833worker2 # [ 0.309699] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations834worker1 # [ 0.285688] ... generic counters: 6835client # [ 0.285907] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)836worker2 # [ 0.310484] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations837worker1 # [ 0.286392] ... generic bitmap: 000000000000003f838lb # [ 0.295035] smp: Bringing up secondary CPUs ...839worker1 # [ 0.286700] ... fixed-purpose counters: 0840lb # [ 0.295349] smp: Brought up 1 node, 1 CPU841client # [ 0.288291] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)842worker2 # [ 0.311754] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations843worker1 # [ 0.287399] ... fixed-purpose bitmap: 0000000000000000844lb # [ 0.295984] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)845worker2 # [ 0.312469] audit: initializing netlink subsys (disabled)846worker1 # [ 0.287687] ... value mask: 0000ffffffffffff847client # [ 0.289476] Performance Events: Fam17h+ core perfctr, AMD PMU driver.848worker1 # [ 0.288594] ... max period: 00007fffffffffff849client # [ 0.289912] ... version: 2850worker2 # [ 0.313652] thermal_sys: Registered thermal governor 'fair_share'851client # [ 0.290615] ... bit width: 48852worker1 # [ 0.289364] ... global_ctrl mask: 000000000000003f853lb # [ 0.296492] Memory: 1454564K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)854worker2 # [ 0.313654] thermal_sys: Registered thermal governor 'bang_bang'855worker1 # [ 0.289805] signal: max sigframe size: 3376856client # [ 0.290904] ... generic counters: 6857lb # [ 0.297579] devtmpfs: initialized858worker2 # [ 0.314463] thermal_sys: Registered thermal governor 'step_wise'859worker1 # [ 0.290591] rcu: Hierarchical SRCU implementation.860lb # [ 0.298340] x86/mm: Memory block size: 128MB861client # [ 0.291636] ... generic bitmap: 000000000000003f862worker2 # [ 0.315448] thermal_sys: Registered thermal governor 'user_space'863client # [ 0.291905] ... fixed-purpose counters: 0864worker1 # [ 0.291231] rcu: Max phase no-delay instances is 400.865client # [ 0.292573] ... fixed-purpose bitmap: 0000000000000000866worker2 # [ 0.316235] audit: type=2000 audit(1789906868.928:1): state=initialized audit_enabled=0 res=1867lb # [ 0.300035] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)868client # [ 0.292904] ... value mask: 0000ffffffffffff869worker2 # [ 0.317464] thermal_sys: Registered thermal governor 'power_allocator'870client # [ 0.293752] ... max period: 00007fffffffffff871lb # [ 0.301189] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).872worker2 # [ 0.317478] cpuidle: using governor menu873client # [ 0.294570] ... global_ctrl mask: 000000000000003f874lb # [ 0.302409] pinctrl core: initialized pinctrl subsystem875client # [ 0.295009] signal: max sigframe size: 3376876worker2 # [ 0.319711] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5877lb # [ 0.303561] PM: RTC time: 12:21:08, date: 2026-09-20878client # [ 0.295833] rcu: Hierarchical SRCU implementation.879worker1 # [ 0.296490] smp: Bringing up secondary CPUs ...880client # [ 0.296521] rcu: Max phase no-delay instances is 400.881worker2 # [ 0.320723] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]882worker1 # [ 0.297377] smp: Brought up 1 node, 1 CPU883worker2 # [ 0.321463] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry884worker1 # [ 0.297688] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)885worker2 # [ 0.322471] PCI: Using configuration type 1 for base access886lb # [ 0.307497] NET: Registered PF_NETLINK/PF_ROUTE protocol family887worker1 # [ 0.298904] Memory: 1454560K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)888worker2 # [ 0.323666] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.889worker1 # [ 0.300014] devtmpfs: initialized890lb # [ 0.308876] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations891worker1 # [ 0.300782] x86/mm: Memory block size: 128MB892client # [ 0.301778] smp: Bringing up secondary CPUs ...893client # [ 0.302453] smp: Brought up 1 node, 1 CPU894lb # [ 0.309359] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations895client # [ 0.302904] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)896worker1 # [ 0.302609] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)897lb # [ 0.310615] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations898lb # [ 0.311345] audit: initializing netlink subsys (disabled)899worker1 # [ 0.303577] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).900worker2 # [ 0.328735] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages901client # [ 0.304053] Memory: 1454548K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109348K reserved, 0K cma-reserved)902worker1 # [ 0.304787] pinctrl core: initialized pinctrl subsystem903lb # [ 0.312425] thermal_sys: Registered thermal governor 'fair_share'904worker2 # [ 0.329461] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page905client # [ 0.305211] devtmpfs: initialized906worker1 # [ 0.305912] PM: RTC time: 12:21:08, date: 2026-09-20907lb # [ 0.312428] thermal_sys: Registered thermal governor 'bang_bang'908client # [ 0.306060] x86/mm: Memory block size: 128MB909lb # [ 0.313336] thermal_sys: Registered thermal governor 'step_wise'910lb # [ 0.314309] thermal_sys: Registered thermal governor 'user_space'911client # [ 0.307808] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)912lb # [ 0.315128] audit: type=2000 audit(1789906868.950:1): state=initialized audit_enabled=0 res=1913worker1 # [ 0.309559] NET: Registered PF_NETLINK/PF_ROUTE protocol family914client # [ 0.308838] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).915worker2 # [ 0.334461] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages916lb # [ 0.317338] thermal_sys: Registered thermal governor 'power_allocator'917client # [ 0.310008] pinctrl core: initialized pinctrl subsystem918lb # [ 0.317354] cpuidle: using governor menu919worker2 # [ 0.335460] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page920worker1 # [ 0.310946] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations921client # [ 0.311179] PM: RTC time: 12:21:08, date: 2026-09-20922lb # [ 0.319356] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5923worker1 # [ 0.311710] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations924worker1 # [ 0.312978] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations925lb # [ 0.320560] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]926worker1 # [ 0.313701] audit: initializing netlink subsys (disabled)927lb # [ 0.321338] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry928client # [ 0.314835] NET: Registered PF_NETLINK/PF_ROUTE protocol family929lb # [ 0.322347] PCI: Using configuration type 1 for base access930worker1 # [ 0.314803] thermal_sys: Registered thermal governor 'fair_share'931worker1 # [ 0.314806] thermal_sys: Registered thermal governor 'bang_bang'932client # [ 0.316178] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations933lb # [ 0.323488] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.934worker1 # [ 0.315688] thermal_sys: Registered thermal governor 'step_wise'935client # [ 0.316932] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations936worker1 # [ 0.316599] thermal_sys: Registered thermal governor 'user_space'937client # [ 0.318206] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations938worker1 # [ 0.317468] audit: type=2000 audit(1789906868.964:1): state=initialized audit_enabled=0 res=1939client # [ 0.318917] audit: initializing netlink subsys (disabled)940worker2 # [ 0.345895] ACPI: Added _OSI(Module Device)941worker1 # [ 0.318690] thermal_sys: Registered thermal governor 'power_allocator'942worker1 # [ 0.318708] cpuidle: using governor menu943worker2 # [ 0.346461] ACPI: Added _OSI(Processor Device)944client # [ 0.320127] thermal_sys: Registered thermal governor 'fair_share'945worker2 # [ 0.347225] ACPI: Added _OSI(Processor Aggregator Device)946lb # [ 0.328579] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages947client # [ 0.320130] thermal_sys: Registered thermal governor 'bang_bang'948worker1 # [ 0.320904] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5949lb # [ 0.329337] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page950client # [ 0.320906] thermal_sys: Registered thermal governor 'step_wise'951worker1 # [ 0.321936] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]952client # [ 0.321894] thermal_sys: Registered thermal governor 'user_space'953worker1 # [ 0.322690] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry954client # [ 0.322626] audit: type=2000 audit(1789906868.969:1): state=initialized audit_enabled=0 res=1955worker1 # [ 0.323704] PCI: Using configuration type 1 for base access956client # [ 0.323908] thermal_sys: Registered thermal governor 'power_allocator'957client # [ 0.323929] cpuidle: using governor menu958worker1 # [ 0.324761] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.959lb # [ 0.334336] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages960client # [ 0.326168] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5961lb # [ 0.335335] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page962client # [ 0.327176] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]963worker2 # [ 0.355633] ACPI: 1 ACPI AML tables successfully acquired and loaded964client # [ 0.327908] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry965client # [ 0.328920] PCI: Using configuration type 1 for base access966client # [ 0.330081] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.967worker1 # [ 0.331981] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages968worker2 # [ 0.359875] ACPI: Interpreter enabled969worker2 # [ 0.360478] ACPI: PM: (supports S0 S3 S4 S5)970worker1 # [ 0.332690] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page971worker2 # [ 0.361229] ACPI: Using IOAPIC for interrupt routing972lb # [ 0.345739] ACPI: Added _OSI(Module Device)973client # [ 0.335209] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages974worker2 # [ 0.363513] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug975lb # [ 0.346337] ACPI: Added _OSI(Processor Device)976client # [ 0.335906] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page977lb # [ 0.347107] ACPI: Added _OSI(Processor Aggregator Device)978worker2 # [ 0.364462] PCI: Using E820 reservations for host bridge windows979worker1 # [ 0.337692] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages980worker1 # [ 0.338689] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page981worker2 # [ 0.367552] ACPI: Enabled 2 GPEs in block 00 to 3F982client # [ 0.340907] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages983client # [ 0.341903] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page984lb # [ 0.355543] ACPI: 1 ACPI AML tables successfully acquired and loaded985worker1 # [ 0.346841] ACPI: Added _OSI(Module Device)986worker1 # [ 0.347690] ACPI: Added _OSI(Processor Device)987worker1 # [ 0.348557] ACPI: Added _OSI(Processor Aggregator Device)988worker2 # [ 0.376172] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])989worker2 # [ 0.376467] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]990lb # [ 0.359740] ACPI: Interpreter enabled991lb # [ 0.360351] ACPI: PM: (supports S0 S3 S4 S5)992worker2 # [ 0.377539] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]993lb # [ 0.361050] ACPI: Using IOAPIC for interrupt routing994worker2 # [ 0.378582] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]995client # [ 0.352374] ACPI: Added _OSI(Module Device)996client # [ 0.352906] ACPI: Added _OSI(Processor Device)997worker2 # [ 0.379840] PCI host bridge to bus 0000:00998client # [ 0.353655] ACPI: Added _OSI(Processor Aggregator Device)999worker2 # [ 0.380464] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1000lb # [ 0.363387] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1001worker2 # [ 0.381460] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1002lb # [ 0.364336] PCI: Using E820 reservations for host bridge windows1003worker2 # [ 0.382460] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1004worker2 # [ 0.383461] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1005lb # [ 0.367462] ACPI: Enabled 2 GPEs in block 00 to 3F1006worker2 # [ 0.384460] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1007worker2 # [ 0.385460] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1008worker2 # [ 0.386460] pci_bus 0000:00: root bus resource [bus 00-ff]1009worker1 # [ 0.362633] ACPI: 1 ACPI AML tables successfully acquired and loaded1010worker2 # [ 0.387494] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1011client # [ 0.362516] ACPI: 1 ACPI AML tables successfully acquired and loaded1012worker2 # [ 0.388916] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1013worker1 # [ 0.366968] ACPI: Interpreter enabled1014lb # [ 0.375160] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1015worker1 # [ 0.367715] ACPI: PM: (supports S0 S3 S4 S5)1016client # [ 0.366323] ACPI: Interpreter enabled1017worker1 # [ 0.368454] ACPI: Using IOAPIC for interrupt routing1018lb # [ 0.376136] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1019client # [ 0.366921] ACPI: PM: (supports S0 S3 S4 S5)1020client # [ 0.367628] ACPI: Using IOAPIC for interrupt routing1021lb # [ 0.377413] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1022lb # [ 0.378467] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1023worker1 # [ 0.370763] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1024lb # [ 0.379674] PCI host bridge to bus 0000:001025worker1 # [ 0.371688] PCI: Using E820 reservations for host bridge windows1026client # [ 0.369955] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1027lb # [ 0.380340] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1028client # [ 0.370905] PCI: Using E820 reservations for host bridge windows1029worker2 # [ 0.391688] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1030lb # [ 0.381342] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1031worker2 # [ 0.392475] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1032worker1 # [ 0.374856] ACPI: Enabled 2 GPEs in block 00 to 3F1033lb # [ 0.382336] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1034worker2 # [ 0.393419] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1035client # [ 0.374006] ACPI: Enabled 2 GPEs in block 00 to 3F1036lb # [ 0.383336] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1037worker2 # [ 0.394843] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1038lb # [ 0.384335] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1039lb # [ 0.385335] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1040worker2 # [ 0.396222] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1041lb # [ 0.386336] pci_bus 0000:00: root bus resource [bus 00-ff]1042lb # [ 0.387483] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1043worker1 # [ 0.382622] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1044worker2 # [ 0.399476] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1045worker2 # [ 0.400325] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1046lb # [ 0.388857] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1047worker1 # [ 0.383419] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1048client # [ 0.381982] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1049worker2 # [ 0.401161] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1050worker1 # [ 0.384771] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1051client # [ 0.382910] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1052worker2 # [ 0.401466] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1053worker1 # [ 0.385816] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1054client # [ 0.383981] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1055worker1 # [ 0.387087] PCI host bridge to bus 0000:001056client # [ 0.385030] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1057worker2 # [ 0.403054] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1058worker1 # [ 0.387698] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1059client # [ 0.386325] PCI host bridge to bus 0000:001060worker1 # [ 0.388713] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1061client # [ 0.386918] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1062worker1 # [ 0.389687] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1063worker2 # [ 0.405318] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1064client # [ 0.387904] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1065worker1 # [ 0.390687] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1066worker2 # [ 0.406148] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1067client # [ 0.388906] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1068worker2 # [ 0.406489] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1069worker1 # [ 0.391688] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1070lb # [ 0.391400] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1071client # [ 0.389905] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1072worker1 # [ 0.392687] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1073lb # [ 0.392348] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1074worker2 # [ 0.408049] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1075worker1 # [ 0.393687] pci_bus 0000:00: root bus resource [bus 00-ff]1076client # [ 0.390906] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1077lb # [ 0.393357] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1078client # [ 0.391904] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1079worker1 # [ 0.394599] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1080client # [ 0.392913] pci_bus 0000:00: root bus resource [bus 00-ff]1081lb # [ 0.394657] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1082worker1 # [ 0.396128] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1083client # [ 0.393983] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1084worker2 # [ 0.411313] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1085lb # [ 0.396071] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1086worker2 # [ 0.412142] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1087client # [ 0.395434] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1088worker2 # [ 0.412482] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1089lb # [ 0.399366] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1090worker2 # [ 0.414016] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1091lb # [ 0.400245] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1092lb # [ 0.401128] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1093worker2 # [ 0.415469] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1094lb # [ 0.402341] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1095worker2 # [ 0.416328] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1096worker1 # [ 0.398757] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1097lb # [ 0.403944] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1098worker1 # [ 0.399699] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1099worker2 # [ 0.417190] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1100worker1 # [ 0.400629] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1101client # [ 0.397977] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1102worker2 # [ 0.418017] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1103lb # [ 0.405365] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1104worker1 # [ 0.401943] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1105client # [ 0.398916] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1106lb # [ 0.406182] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1107client # [ 0.399859] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1108lb # [ 0.407085] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1109worker1 # [ 0.403458] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1110worker2 # [ 0.420126] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1111client # [ 0.400882] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1112worker2 # [ 0.420467] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1113lb # [ 0.407921] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1114worker2 # [ 0.421436] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1115worker1 # [ 0.406709] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1116client # [ 0.402704] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1117worker2 # [ 0.422878] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1118worker1 # [ 0.407544] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1119lb # [ 0.410360] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1120worker1 # [ 0.408405] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1121lb # [ 0.411221] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1122worker1 # [ 0.408693] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1123client # [ 0.405922] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1124lb # [ 0.412079] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1125client # [ 0.406760] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1126worker1 # [ 0.410611] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1127client # [ 0.407677] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1128worker2 # [ 0.425950] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1129lb # [ 0.413732] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1130worker2 # [ 0.427152] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1131client # [ 0.408735] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1132worker2 # [ 0.427484] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1133lb # [ 0.415365] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1134worker1 # [ 0.413622] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1135lb # [ 0.416224] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1136client # [ 0.410309] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1137worker1 # [ 0.414287] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1138worker2 # [ 0.429071] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1139lb # [ 0.417062] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1140worker1 # [ 0.414709] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1141client # [ 0.411912] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1142worker1 # [ 0.416273] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1143worker2 # [ 0.431248] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1144lb # [ 0.418228] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1145client # [ 0.412749] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1146worker2 # [ 0.432070] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1147client # [ 0.413627] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1148worker2 # [ 0.432482] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1149lb # [ 0.420342] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1150lb # [ 0.421190] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1151client # [ 0.414510] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1152worker2 # [ 0.433465] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1153worker1 # [ 0.419605] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1154lb # [ 0.422096] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1155worker1 # [ 0.420286] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1156worker2 # [ 0.435044] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1157worker1 # [ 0.420709] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1158lb # [ 0.423720] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1159client # [ 0.416912] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1160worker1 # [ 0.422289] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1161worker2 # [ 0.436488] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1162client # [ 0.417783] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1163worker2 # [ 0.437447] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1164client # [ 0.418663] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1165lb # [ 0.426489] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1166worker1 # [ 0.424709] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1167lb # [ 0.427341] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1168worker2 # [ 0.438846] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1169client # [ 0.419478] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1170worker1 # [ 0.425592] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1171lb # [ 0.428334] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1172worker1 # [ 0.426434] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1173lb # [ 0.429930] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1174client # [ 0.421875] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1175worker2 # [ 0.441301] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1176client # [ 0.422547] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1177worker2 # [ 0.442468] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1178worker1 # [ 0.427287] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1179lb # [ 0.431342] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1180worker2 # [ 0.443418] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1181lb # [ 0.432173] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1182client # [ 0.422925] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1183lb # [ 0.433069] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1184worker1 # [ 0.429448] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1185worker2 # [ 0.444939] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1186client # [ 0.424501] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1187lb # [ 0.433351] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1188worker1 # [ 0.429694] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1189worker1 # [ 0.430633] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1190worker2 # [ 0.446499] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1191client # [ 0.425912] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1192lb # [ 0.434867] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1193worker2 # [ 0.447390] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1194client # [ 0.426725] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1195worker1 # [ 0.432170] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1196lb # [ 0.436150] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1197worker2 # [ 0.448200] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1198client # [ 0.427627] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1199lb # [ 0.437040] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1200client # [ 0.428534] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1201worker2 # [ 0.449032] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1202lb # [ 0.437886] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1203worker1 # [ 0.434719] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1204worker1 # [ 0.435567] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1205worker2 # [ 0.450255] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1206worker1 # [ 0.436428] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1207lb # [ 0.439367] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1208client # [ 0.430939] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1209worker2 # [ 0.451330] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1210lb # [ 0.440239] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1211client # [ 0.431785] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1212worker1 # [ 0.437369] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1213worker2 # [ 0.453165] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1214lb # [ 0.441321] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1215client # [ 0.432635] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1216lb # [ 0.442746] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1217client # [ 0.433567] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1218worker1 # [ 0.440722] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1219lb # [ 0.444342] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1220client # [ 0.434931] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1221lb # [ 0.445235] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1222worker1 # [ 0.441701] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1223client # [ 0.435809] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1224lb # [ 0.446111] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1225worker1 # [ 0.442719] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1226client # [ 0.436658] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1227worker1 # [ 0.443700] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1228lb # [ 0.447757] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1229client # [ 0.436910] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1230lb # [ 0.448903] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1231worker1 # [ 0.445433] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1232client # [ 0.438428] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1233lb # [ 0.449530] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1234client # [ 0.439916] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1235lb # [ 0.450868] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1236worker1 # [ 0.448498] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1237worker2 # [ 0.453701] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1238client # [ 0.440861] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1239worker1 # [ 0.449563] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1240worker2 # [ 0.456243] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1241client # [ 0.442276] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1242worker2 # [ 0.457089] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1243worker1 # [ 0.451938] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1244worker2 # [ 0.458201] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1245client # [ 0.443945] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1246client # [ 0.445558] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1247worker2 # [ 0.458725] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1248worker1 # [ 0.454716] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1249client # [ 0.445926] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1250worker2 # [ 0.459759] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1251worker1 # [ 0.455698] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1252client # [ 0.447502] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1253worker1 # [ 0.456717] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1254worker2 # [ 0.460720] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1255lb # [ 0.451539] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1256client # [ 0.448912] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1257worker2 # [ 0.462488] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1258worker1 # [ 0.458391] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1259client # [ 0.449716] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1260lb # [ 0.453880] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1261worker2 # [ 0.463479] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1262client # [ 0.450645] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1263lb # [ 0.454568] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1264worker2 # [ 0.464751] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1265worker1 # [ 0.461717] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1266client # [ 0.451488] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1267lb # [ 0.455971] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1268worker2 # [ 0.466262] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1269worker1 # [ 0.462696] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1270client # [ 0.452649] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1271worker1 # [ 0.463710] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1272lb # [ 0.456606] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1273client # [ 0.453109] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1274lb # [ 0.457637] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1275worker1 # [ 0.465294] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1276worker2 # [ 0.470183] ACPI: PCI: Interrupt link LNKA configured for IRQ 101277client # [ 0.454567] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1278lb # [ 0.458583] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1279worker2 # [ 0.470568] ACPI: PCI: Interrupt link LNKB configured for IRQ 101280worker1 # [ 0.467635] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1281lb # [ 0.460158] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1282worker2 # [ 0.471571] ACPI: PCI: Interrupt link LNKC configured for IRQ 111283lb # [ 0.461086] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1284worker2 # [ 0.472596] ACPI: PCI: Interrupt link LNKD configured for IRQ 111285worker1 # [ 0.469509] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1286worker2 # [ 0.473562] ACPI: PCI: Interrupt link LNKE configured for IRQ 101287lb # [ 0.461649] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1288worker2 # [ 0.474558] ACPI: PCI: Interrupt link LNKF configured for IRQ 101289worker1 # [ 0.471660] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1290lb # [ 0.463256] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1291worker2 # [ 0.475562] ACPI: PCI: Interrupt link LNKG configured for IRQ 111292worker2 # [ 0.476557] ACPI: PCI: Interrupt link LNKH configured for IRQ 111293worker2 # [ 0.477499] ACPI: PCI: Interrupt link GSIA configured for IRQ 161294worker2 # [ 0.478476] ACPI: PCI: Interrupt link GSIB configured for IRQ 171295lb # [ 0.467002] ACPI: PCI: Interrupt link LNKA configured for IRQ 101296worker2 # [ 0.479475] ACPI: PCI: Interrupt link GSIC configured for IRQ 181297lb # [ 0.467443] ACPI: PCI: Interrupt link LNKB configured for IRQ 101298worker2 # [ 0.480480] ACPI: PCI: Interrupt link GSID configured for IRQ 191299lb # [ 0.468450] ACPI: PCI: Interrupt link LNKC configured for IRQ 111300client # [ 0.455114] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1301worker2 # [ 0.481485] ACPI: PCI: Interrupt link GSIE configured for IRQ 201302lb # [ 0.469446] ACPI: PCI: Interrupt link LNKD configured for IRQ 111303worker2 # [ 0.482438] ACPI: PCI: Interrupt link GSIF configured for IRQ 211304client # [ 0.457585] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1305lb # [ 0.470432] ACPI: PCI: Interrupt link LNKE configured for IRQ 101306worker2 # [ 0.483209] ACPI: PCI: Interrupt link GSIG configured for IRQ 221307lb # [ 0.471432] ACPI: PCI: Interrupt link LNKF configured for IRQ 101308worker2 # [ 0.483473] ACPI: PCI: Interrupt link GSIH configured for IRQ 231309client # [ 0.458168] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1310lb # [ 0.472438] ACPI: PCI: Interrupt link LNKG configured for IRQ 111311worker2 # [ 0.485564] iommu: Default domain type: Translated1312client # [ 0.459555] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1313lb # [ 0.473433] ACPI: PCI: Interrupt link LNKH configured for IRQ 111314worker2 # [ 0.486354] iommu: DMA domain TLB invalidation policy: lazy mode1315lb # [ 0.474372] ACPI: PCI: Interrupt link GSIA configured for IRQ 161316worker1 # [ 0.472660] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1317client # [ 0.460172] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1318worker2 # [ 0.487491] ACPI: bus type USB registered1319lb # [ 0.475301] ACPI: PCI: Interrupt link GSIB configured for IRQ 171320worker2 # [ 0.488252] usbcore: registered new interface driver usbfs1321lb # [ 0.476092] ACPI: PCI: Interrupt link GSIC configured for IRQ 181322client # [ 0.461192] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1323worker1 # [ 0.474519] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1324worker2 # [ 0.489162] usbcore: registered new interface driver hub1325lb # [ 0.476350] ACPI: PCI: Interrupt link GSID configured for IRQ 191326client # [ 0.462154] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1327worker2 # [ 0.489475] usbcore: registered new device driver usb1328worker1 # [ 0.475644] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1329lb # [ 0.477310] ACPI: PCI: Interrupt link GSIE configured for IRQ 201330lb # [ 0.478054] ACPI: PCI: Interrupt link GSIF configured for IRQ 211331worker2 # [ 0.491421] NetLabel: Initializing1332client # [ 0.463932] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1333worker1 # [ 0.477565] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1334lb # [ 0.478347] ACPI: PCI: Interrupt link GSIG configured for IRQ 221335worker2 # [ 0.491921] NetLabel: domain hash size = 1281336lb # [ 0.479283] ACPI: PCI: Interrupt link GSIH configured for IRQ 231337worker2 # [ 0.492459] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1338client # [ 0.464773] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1339worker1 # [ 0.478657] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1340worker2 # [ 0.493404] NetLabel: unlabeled traffic allowed by default1341worker2 # [ 0.494155] PCI: Using ACPI for IRQ routing1342client # [ 0.466250] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1343lb # [ 0.481238] iommu: Default domain type: Translated1344worker1 # [ 0.480010] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1345lb # [ 0.481992] iommu: DMA domain TLB invalidation policy: lazy mode1346client # [ 0.467622] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1347worker1 # [ 0.480981] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1348lb # [ 0.482573] ACPI: bus type USB registered1349lb # [ 0.483327] usbcore: registered new interface driver usbfs1350lb # [ 0.484294] usbcore: registered new interface driver hub1351worker1 # [ 0.482717] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1352lb # [ 0.485043] usbcore: registered new device driver usb1353worker1 # [ 0.483666] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1354lb # [ 0.486249] NetLabel: Initializing1355lb # [ 0.486760] NetLabel: domain hash size = 1281356client # [ 0.470910] ACPI: PCI: Interrupt link LNKA configured for IRQ 101357worker1 # [ 0.485033] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1358client # [ 0.472054] ACPI: PCI: Interrupt link LNKB configured for IRQ 101359client # [ 0.473013] ACPI: PCI: Interrupt link LNKC configured for IRQ 111360lb # [ 0.487335] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1361worker1 # [ 0.486691] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1362client # [ 0.474002] ACPI: PCI: Interrupt link LNKD configured for IRQ 111363lb # [ 0.488380] NetLabel: unlabeled traffic allowed by default1364client # [ 0.475002] ACPI: PCI: Interrupt link LNKE configured for IRQ 101365lb # [ 0.489348] PCI: Using ACPI for IRQ routing1366client # [ 0.475998] ACPI: PCI: Interrupt link LNKF configured for IRQ 101367client # [ 0.477030] ACPI: PCI: Interrupt link LNKG configured for IRQ 111368client # [ 0.478005] ACPI: PCI: Interrupt link LNKH configured for IRQ 111369client # [ 0.478941] ACPI: PCI: Interrupt link GSIA configured for IRQ 161370worker1 # [ 0.491348] ACPI: PCI: Interrupt link LNKA configured for IRQ 101371client # [ 0.479917] ACPI: PCI: Interrupt link GSIB configured for IRQ 171372worker1 # [ 0.492648] ACPI: PCI: Interrupt link LNKB configured for IRQ 101373client # [ 0.480882] ACPI: PCI: Interrupt link GSIC configured for IRQ 181374worker1 # [ 0.493802] ACPI: PCI: Interrupt link LNKC configured for IRQ 111375client # [ 0.481918] ACPI: PCI: Interrupt link GSID configured for IRQ 191376client # [ 0.482911] ACPI: PCI: Interrupt link GSIE configured for IRQ 201377worker1 # [ 0.494864] ACPI: PCI: Interrupt link LNKD configured for IRQ 111378client # [ 0.483916] ACPI: PCI: Interrupt link GSIF configured for IRQ 211379worker1 # [ 0.495807] ACPI: PCI: Interrupt link LNKE configured for IRQ 101380client # [ 0.484919] ACPI: PCI: Interrupt link GSIG configured for IRQ 221381worker1 # [ 0.496818] ACPI: PCI: Interrupt link LNKF configured for IRQ 101382client # [ 0.485919] ACPI: PCI: Interrupt link GSIH configured for IRQ 231383worker1 # [ 0.497836] ACPI: PCI: Interrupt link LNKG configured for IRQ 111384worker1 # [ 0.498804] ACPI: PCI: Interrupt link LNKH configured for IRQ 111385client # [ 0.488170] iommu: Default domain type: Translated1386client # [ 0.488916] iommu: DMA domain TLB invalidation policy: lazy mode1387worker1 # [ 0.499728] ACPI: PCI: Interrupt link GSIA configured for IRQ 161388client # [ 0.490175] ACPI: bus type USB registered1389worker1 # [ 0.500718] ACPI: PCI: Interrupt link GSIB configured for IRQ 171390client # [ 0.490941] usbcore: registered new interface driver usbfs1391worker1 # [ 0.501666] ACPI: PCI: Interrupt link GSIC configured for IRQ 181392client # [ 0.491922] usbcore: registered new interface driver hub1393worker1 # [ 0.502411] ACPI: PCI: Interrupt link GSID configured for IRQ 191394client # [ 0.492872] usbcore: registered new device driver usb1395worker1 # [ 0.502712] ACPI: PCI: Interrupt link GSIE configured for IRQ 201396client # [ 0.494574] NetLabel: Initializing1397worker1 # [ 0.503711] ACPI: PCI: Interrupt link GSIF configured for IRQ 211398client # [ 0.494905] NetLabel: domain hash size = 1281399worker1 # [ 0.504665] ACPI: PCI: Interrupt link GSIG configured for IRQ 221400client # [ 0.495664] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1401worker1 # [ 0.505417] ACPI: PCI: Interrupt link GSIH configured for IRQ 231402client # [ 0.495943] NetLabel: unlabeled traffic allowed by default1403client # [ 0.496935] PCI: Using ACPI for IRQ routing1404worker1 # [ 0.506972] iommu: Default domain type: Translated1405worker1 # [ 0.507700] iommu: DMA domain TLB invalidation policy: lazy mode1406worker1 # [ 0.509015] ACPI: bus type USB registered1407worker1 # [ 0.509760] usbcore: registered new interface driver usbfs1408worker1 # [ 0.510715] usbcore: registered new interface driver hub1409worker1 # [ 0.511643] usbcore: registered new device driver usb1410worker1 # [ 0.513380] NetLabel: Initializing1411worker1 # [ 0.513689] NetLabel: domain hash size = 1281412worker1 # [ 0.514479] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1413worker1 # [ 0.515458] NetLabel: unlabeled traffic allowed by default1414worker1 # [ 0.516417] PCI: Using ACPI for IRQ routing1415worker2 # [ 0.540754] pci 0000:00:01.0: vgaarb: setting as boot VGA device1416worker2 # [ 0.541456] pci 0000:00:01.0: vgaarb: bridge control possible1417worker2 # [ 0.541456] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1418worker2 # [ 0.541465] vgaarb: loaded1419worker2 # [ 0.542241] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01420worker2 # [ 0.543083] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1421lb # [ 0.533395] pci 0000:00:01.0: vgaarb: setting as boot VGA device1422lb # [ 0.534331] pci 0000:00:01.0: vgaarb: bridge control possible1423worker2 # [ 0.547596] clocksource: Switched to clocksource kvm-clock1424lb # [ 0.534331] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1425lb # [ 0.534339] vgaarb: loaded1426worker2 # [ 0.549357] VFS: Disk quotas dquot_6.6.01427lb # [ 0.535085] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01428lb # [ 0.535337] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1429worker2 # [ 0.552018] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1430worker2 # [ 0.553503] pnp: PnP ACPI init1431worker2 # [ 0.554314] ACPI: IRQ 4 override to edge(!), high(!)1432lb # [ 0.539430] clocksource: Switched to clocksource kvm-clock1433worker2 # [ 0.555350] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1434lb # [ 0.541069] VFS: Disk quotas dquot_6.6.01435worker2 # [ 0.556955] pnp: PnP ACPI: found 6 devices1436lb # [ 0.543683] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1437lb # [ 0.545091] pnp: PnP ACPI init1438lb # [ 0.545951] ACPI: IRQ 4 override to edge(!), high(!)1439lb # [ 0.547017] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1440lb # [ 0.548701] pnp: PnP ACPI: found 6 devices1441worker2 # [ 0.564654] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1442worker2 # [ 0.566143] clocksource: Switched to clocksource acpi_pm1443worker2 # [ 0.567168] NET: Registered PF_INET protocol family1444worker2 # [ 0.568397] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1445client # [ 0.541144] pci 0000:00:01.0: vgaarb: setting as boot VGA device1446client # [ 0.541900] pci 0000:00:01.0: vgaarb: bridge control possible1447client # [ 0.541900] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1448lb # [ 0.556412] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1449client # [ 0.541908] vgaarb: loaded1450lb # [ 0.557863] clocksource: Switched to clocksource acpi_pm1451client # [ 0.542637] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01452lb # [ 0.558851] NET: Registered PF_INET protocol family1453client # [ 0.542905] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1454lb # [ 0.560067] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1455client # [ 0.547003] clocksource: Switched to clocksource kvm-clock1456client # [ 0.550621] VFS: Disk quotas dquot_6.6.01457client # [ 0.551415] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1458client # [ 0.552884] pnp: PnP ACPI init1459worker2 # [ 0.585401] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1460worker1 # [ 0.564274] pci 0000:00:01.0: vgaarb: setting as boot VGA device1461client # [ 0.553737] ACPI: IRQ 4 override to edge(!), high(!)1462worker1 # [ 0.564683] pci 0000:00:01.0: vgaarb: bridge control possible1463worker2 # [ 0.586986] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1464client # [ 0.554695] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1465client # [ 0.556264] pnp: PnP ACPI: found 6 devices1466worker2 # [ 0.588251] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1467worker1 # [ 0.564683] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1468worker1 # [ 0.564698] vgaarb: loaded1469worker2 # [ 0.589607] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1470worker1 # [ 0.565657] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01471worker2 # [ 0.590951] TCP: Hash tables configured (established 16384 bind 16384)1472worker1 # [ 0.566361] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1473worker2 # [ 0.592071] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1474lb # [ 0.577014] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1475worker2 # [ 0.593355] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1476worker2 # [ 0.594478] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1477lb # [ 0.578616] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1478worker2 # [ 0.595644] NET: Registered PF_UNIX/PF_LOCAL protocol family1479lb # [ 0.579934] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1480worker2 # [ 0.596625] NET: Registered PF_XDP protocol family1481client # [ 0.563865] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1482lb # [ 0.581313] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1483worker2 # [ 0.597430] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1484client # [ 0.565310] clocksource: Switched to clocksource acpi_pm1485worker1 # [ 0.571831] clocksource: Switched to clocksource kvm-clock1486worker2 # [ 0.598429] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1487client # [ 0.566261] NET: Registered PF_INET protocol family1488lb # [ 0.582545] TCP: Hash tables configured (established 16384 bind 16384)1489worker2 # [ 0.599412] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1490lb # [ 0.583618] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1491client # [ 0.567412] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1492worker2 # [ 0.600517] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1493lb # [ 0.584819] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1494worker2 # [ 0.601621] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1495lb # [ 0.585886] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1496worker1 # [ 0.576287] VFS: Disk quotas dquot_6.6.01497worker2 # [ 0.602743] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1498lb # [ 0.587096] NET: Registered PF_UNIX/PF_LOCAL protocol family1499lb # [ 0.588031] NET: Registered PF_XDP protocol family1500worker1 # [ 0.577175] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1501worker2 # [ 0.604489] ACPI: \_SB_.GSIA: Enabled at IRQ 161502lb # [ 0.588870] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1503worker1 # [ 0.578860] pnp: PnP ACPI init1504lb # [ 0.589895] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1505worker1 # [ 0.579827] ACPI: IRQ 4 override to edge(!), high(!)1506lb # [ 0.590940] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1507worker2 # [ 0.607163] ACPI: \_SB_.GSIB: Enabled at IRQ 171508lb # [ 0.592049] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1509worker1 # [ 0.581112] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1510worker2 # [ 0.609109] ACPI: \_SB_.GSIC: Enabled at IRQ 181511lb # [ 0.593204] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1512worker1 # [ 0.583067] pnp: PnP ACPI: found 6 devices1513lb # [ 0.594337] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1514worker2 # [ 0.611060] ACPI: \_SB_.GSID: Enabled at IRQ 191515lb # [ 0.596052] ACPI: \_SB_.GSIA: Enabled at IRQ 161516worker2 # [ 0.612876] PCI: CLS 0 bytes, default 641517lb # [ 0.598657] ACPI: \_SB_.GSIB: Enabled at IRQ 171518worker2 # [ 0.613751] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1519worker2 # [ 0.615557] Trying to unpack rootfs image as initramfs...1520lb # [ 0.600602] ACPI: \_SB_.GSIC: Enabled at IRQ 181521client # [ 0.584129] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1522lb # [ 0.602523] ACPI: \_SB_.GSID: Enabled at IRQ 191523client # [ 0.585749] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1524worker1 # [ 0.591512] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1525lb # [ 0.604213] PCI: CLS 0 bytes, default 641526client # [ 0.587075] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1527worker1 # [ 0.593195] clocksource: Switched to clocksource acpi_pm1528client # [ 0.588400] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1529worker1 # [ 0.594416] NET: Registered PF_INET protocol family1530lb # [ 0.605113] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1531client # [ 0.589659] TCP: Hash tables configured (established 16384 bind 16384)1532lb # [ 0.606845] Trying to unpack rootfs image as initramfs...1533worker1 # [ 0.595889] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1534client # [ 0.590813] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1535client # [ 0.592121] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1536client # [ 0.593261] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1537client # [ 0.594539] NET: Registered PF_UNIX/PF_LOCAL protocol family1538client # [ 0.595584] NET: Registered PF_XDP protocol family1539client # [ 0.596507] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1540client # [ 0.597568] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1541client # [ 0.598654] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1542client # [ 0.599800] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1543client # [ 0.601005] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1544client # [ 0.602211] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1545client # [ 0.604068] ACPI: \_SB_.GSIA: Enabled at IRQ 161546client # [ 0.606201] ACPI: \_SB_.GSIB: Enabled at IRQ 171547client # [ 0.608187] ACPI: \_SB_.GSIC: Enabled at IRQ 181548client # [ 0.610168] ACPI: \_SB_.GSID: Enabled at IRQ 191549client # [ 0.611857] PCI: CLS 0 bytes, default 641550worker1 # [ 0.617678] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1551client # [ 0.612778] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1552worker1 # [ 0.619655] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1553client # [ 0.614584] Trying to unpack rootfs image as initramfs...1554worker1 # [ 0.621258] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1555worker1 # [ 0.622888] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1556worker1 # [ 0.624476] TCP: Hash tables configured (established 16384 bind 16384)1557worker1 # [ 0.625628] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1558worker1 # [ 0.626957] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1559worker1 # [ 0.628026] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1560worker1 # [ 0.629245] NET: Registered PF_UNIX/PF_LOCAL protocol family1561worker1 # [ 0.630204] NET: Registered PF_XDP protocol family1562worker1 # [ 0.631020] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1563worker1 # [ 0.632002] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1564worker1 # [ 0.632973] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1565worker1 # [ 0.634062] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1566worker2 # [ 0.661589] Initialise system trusted keyrings1567worker1 # [ 0.635133] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1568worker1 # [ 0.636169] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1569worker1 # [ 0.637859] ACPI: \_SB_.GSIA: Enabled at IRQ 161570worker2 # [ 0.664506] workingset: timestamp_bits=40 max_order=19 bucket_order=01571lb # [ 0.649797] Initialise system trusted keyrings1572worker1 # [ 0.640235] ACPI: \_SB_.GSIB: Enabled at IRQ 171573worker1 # [ 0.642204] ACPI: \_SB_.GSIC: Enabled at IRQ 181574lb # [ 0.653212] workingset: timestamp_bits=40 max_order=19 bucket_order=01575worker1 # [ 0.644124] ACPI: \_SB_.GSID: Enabled at IRQ 191576worker1 # [ 0.645743] PCI: CLS 0 bytes, default 641577worker1 # [ 0.646620] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1578worker1 # [ 0.648294] Trying to unpack rootfs image as initramfs...1579worker2 # [ 0.685914] Key type asymmetric registered1580worker2 # [ 0.686696] Asymmetric key parser 'x509' registered1581worker2 # [ 0.689539] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1582lb # [ 0.676373] Key type asymmetric registered1583worker2 # [ 0.692534] io scheduler mq-deadline registered1584lb # [ 0.677115] Asymmetric key parser 'x509' registered1585worker2 # [ 0.693299] io scheduler kyber registered1586client # [ 0.661441] Initialise system trusted keyrings1587lb # [ 0.677993] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1588client # [ 0.662419] workingset: timestamp_bits=40 max_order=19 bucket_order=01589worker2 # [ 0.696011] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1590lb # [ 0.681294] io scheduler mq-deadline registered1591lb # [ 0.682074] io scheduler kyber registered1592worker2 # [ 0.697388] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1593lb # [ 0.686347] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1594worker2 # [ 0.703267] Linux agpgart interface v0.1031595lb # [ 0.687659] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1596worker2 # [ 0.704070] ACPI: bus type drm_connector registered1597worker2 # [ 0.706938] usbcore: registered new interface driver usbserial_generic1598worker2 # [ 0.708100] usbserial: USB Serial support registered for generic1599lb # [ 0.692949] Linux agpgart interface v0.1031600lb # [ 0.693791] ACPI: bus type drm_connector registered1601worker2 # [ 0.712467] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1602worker2 # [ 0.712467] Please enable it if your BIOS has the CPPC option.1603worker2 # [ 0.714810] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1604lb # [ 0.699278] usbcore: registered new interface driver usbserial_generic1605lb # [ 0.700344] usbserial: USB Serial support registered for generic1606lb # [ 0.701353] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1607lb # [ 0.701353] Please enable it if your BIOS has the CPPC option.1608client # [ 0.686242] Key type asymmetric registered1609worker1 # [ 0.692368] Initialise system trusted keyrings1610client # [ 0.687078] Asymmetric key parser 'x509' registered1611worker2 # [ 0.719587] drop_monitor: Initializing network drop monitor service1612lb # [ 0.703692] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1613client # [ 0.687885] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1614worker2 # [ 0.720778] NET: Registered PF_INET6 protocol family1615worker1 # [ 0.695167] workingset: timestamp_bits=40 max_order=19 bucket_order=01616lb # [ 0.707297] drop_monitor: Initializing network drop monitor service1617worker2 # [ 0.723903] Segment Routing with IPv61618lb # [ 0.708528] NET: Registered PF_INET6 protocol family1619client # [ 0.693106] io scheduler mq-deadline registered1620worker2 # [ 0.726486] In-situ OAM (IOAM) with IPv61621client # [ 0.694136] io scheduler kyber registered1622worker2 # [ 0.727567] IPI shorthand broadcast: enabled1623client # [ 0.696606] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1624lb # [ 0.713669] Segment Routing with IPv61625client # [ 0.697898] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1626lb # [ 0.717190] In-situ OAM (IOAM) with IPv61627lb # [ 0.718303] IPI shorthand broadcast: enabled1628client # [ 0.703831] Linux agpgart interface v0.1031629worker2 # [ 0.735903] sched_clock: Marking stable (599018528, 136437477)->(816882715, -81426710)1630client # [ 0.704660] ACPI: bus type drm_connector registered1631client # [ 0.707436] usbcore: registered new interface driver usbserial_generic1632worker2 # [ 0.740703] registered taskstats version 11633client # [ 0.708554] usbserial: USB Serial support registered for generic1634worker2 # [ 0.741639] Loading compiled-in X.509 certificates1635lb # [ 0.726603] sched_clock: Marking stable (592014677, 134143839)->(802143029, -75984513)1636worker1 # [ 0.717134] Key type asymmetric registered1637worker1 # [ 0.717838] Asymmetric key parser 'x509' registered1638client # [ 0.711981] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1639client # [ 0.711981] Please enable it if your BIOS has the CPPC option.1640worker1 # [ 0.718703] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1641client # [ 0.714158] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1642lb # [ 0.731355] registered taskstats version 11643lb # [ 0.732293] Loading compiled-in X.509 certificates1644client # [ 0.718118] drop_monitor: Initializing network drop monitor service1645worker1 # [ 0.724175] io scheduler mq-deadline registered1646worker1 # [ 0.724964] io scheduler kyber registered1647client # [ 0.719332] NET: Registered PF_INET6 protocol family1648worker1 # [ 0.727606] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1649client # [ 0.722476] Segment Routing with IPv61650worker1 # [ 0.728864] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1651client # [ 0.724993] In-situ OAM (IOAM) with IPv61652client # [ 0.726074] IPI shorthand broadcast: enabled1653worker2 # [ 0.759473] Demotion targets for Node 0: null1654worker1 # [ 0.734435] Linux agpgart interface v0.1031655worker2 # [ 0.761510] Key type .fscrypt registered1656worker1 # [ 0.735195] ACPI: bus type drm_connector registered1657worker2 # [ 0.762175] Key type fscrypt-provisioning registered1658worker2 # [ 0.763096] ima: No TPM chip found, activating TPM-bypass!1659worker1 # [ 0.738427] usbcore: registered new interface driver usbserial_generic1660lb # [ 0.750177] Demotion targets for Node 0: null1661worker1 # [ 0.739461] usbserial: USB Serial support registered for generic1662lb # [ 0.751038] Key type .fscrypt registered1663client # [ 0.734433] sched_clock: Marking stable (600014904, 133941805)->(815481752, -81525043)1664worker2 # [ 0.767466] ima: Allocated hash algorithm: sha11665lb # [ 0.751771] Key type fscrypt-provisioning registered1666worker2 # [ 0.768245] ima: No architecture policies found1667lb # [ 0.752716] ima: No TPM chip found, activating TPM-bypass!1668worker2 # [ 0.769212] PM: Magic number: 10:616:3801669worker1 # [ 0.743019] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1670worker1 # [ 0.743019] Please enable it if your BIOS has the CPPC option.1671worker1 # [ 0.745131] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1672client # [ 0.740188] registered taskstats version 11673lb # [ 0.757190] ima: Allocated hash algorithm: sha11674worker2 # [ 0.772905] RAS: Correctable Errors collector initialized.1675worker1 # [ 0.746466] drop_monitor: Initializing network drop monitor service1676client # [ 0.741144] Loading compiled-in X.509 certificates1677lb # [ 0.757953] ima: No architecture policies found1678worker1 # [ 0.747602] NET: Registered PF_INET6 protocol family1679lb # [ 0.758939] PM: Magic number: 10:616:3801680lb # [ 0.762686] RAS: Correctable Errors collector initialized.1681worker1 # [ 0.752519] Segment Routing with IPv61682worker2 # [ 0.782040] clk: Disabling unused clocks1683worker1 # [ 0.756032] In-situ OAM (IOAM) with IPv61684worker1 # [ 0.757124] IPI shorthand broadcast: enabled1685worker2 # [ 0.784477] PM: genpd: Disabling unused power domains1686lb # [ 0.771392] clk: Disabling unused clocks1687lb # [ 0.772083] PM: genpd: Disabling unused power domains1688client # [ 0.757977] Demotion targets for Node 0: null1689worker1 # [ 0.763654] sched_clock: Marking stable (630013287, 132980980)->(839092275, -76098008)1690client # [ 0.758978] Key type .fscrypt registered1691client # [ 0.759648] Key type fscrypt-provisioning registered1692client # [ 0.760632] ima: No TPM chip found, activating TPM-bypass!1693worker1 # [ 0.770198] registered taskstats version 11694client # [ 0.764973] ima: Allocated hash algorithm: sha11695worker1 # [ 0.771159] Loading compiled-in X.509 certificates1696client # [ 0.765734] ima: No architecture policies found1697client # [ 0.767972] PM: Magic number: 10:616:3801698client # [ 0.769724] RAS: Correctable Errors collector initialized.1699client # [ 0.778712] clk: Disabling unused clocks1700worker1 # [ 0.787614] Demotion targets for Node 0: null1701client # [ 0.781972] PM: genpd: Disabling unused power domains1702worker1 # [ 0.791057] Key type .fscrypt registered1703worker1 # [ 0.791724] Key type fscrypt-provisioning registered1704worker1 # [ 0.792644] ima: No TPM chip found, activating TPM-bypass!1705worker1 # [ 0.795008] ima: Allocated hash algorithm: sha11706worker1 # [ 0.795773] ima: No architecture policies found1707worker1 # [ 0.798009] PM: Magic number: 10:616:3801708worker1 # [ 0.799604] RAS: Correctable Errors collector initialized.1709worker1 # [ 0.808782] clk: Disabling unused clocks1710worker1 # [ 0.812010] PM: genpd: Disabling unused power domains1711worker2 # [ 0.949686] Freeing initrd memory: 29092K1712worker2 # [ 0.953017] Freeing unused decrypted memory: 2028K1713worker2 # [ 0.955654] Freeing unused kernel image (initmem) memory: 3644K1714worker2 # [ 0.956701] Write protecting the kernel read-only data: 32768k1715worker2 # [ 0.958570] Freeing unused kernel image (text/rodata gap) memory: 1208K1716lb # [ 0.943491] Freeing initrd memory: 29092K1717worker2 # [ 0.960072] Freeing unused kernel image (rodata/data gap) memory: 752K1718lb # [ 0.946746] Freeing unused decrypted memory: 2028K1719lb # [ 0.949542] Freeing unused kernel image (initmem) memory: 3644K1720lb # [ 0.950681] Write protecting the kernel read-only data: 32768k1721lb # [ 0.952644] Freeing unused kernel image (text/rodata gap) memory: 1208K1722lb # [ 0.954259] Freeing unused kernel image (rodata/data gap) memory: 752K1723client # [ 0.959741] Freeing initrd memory: 29096K1724client # [ 0.963055] Freeing unused decrypted memory: 2028K1725client # [ 0.965729] Freeing unused kernel image (initmem) memory: 3644K1726client # [ 0.966812] Write protecting the kernel read-only data: 32768k1727client # [ 0.968700] Freeing unused kernel image (text/rodata gap) memory: 1208K1728client # [ 0.970226] Freeing unused kernel image (rodata/data gap) memory: 752K1729worker1 # [ 0.981427] Freeing initrd memory: 29092K1730worker1 # [ 0.984576] Freeing unused decrypted memory: 2028K1731worker2 # [ 1.011200] x86/mm: Checked W+X mappings: passed, no W+X pages found.1732worker2 # [ 1.012241] Run /init as init process1733worker1 # [ 0.987241] Freeing unused kernel image (initmem) memory: 3644K1734worker1 # [ 0.988295] Write protecting the kernel read-only data: 32768k1735worker1 # [ 0.990124] Freeing unused kernel image (text/rodata gap) memory: 1208K1736worker1 # [ 0.991605] Freeing unused kernel image (rodata/data gap) memory: 752K1737lb # [ 1.005507] x86/mm: Checked W+X mappings: passed, no W+X pages found.1738lb # [ 1.006628] Run /init as init process1739worker2 # [ 1.022495] systemd[1]: Inserted module 'autofs4'1740lb # [ 1.016890] systemd[1]: Inserted module 'autofs4'1741worker2 # [ 1.038679] fuse: init (API version 7.45)1742worker2 # [ 1.044490] ACPI: \_SB_.GSIG: Enabled at IRQ 221743worker2 # [ 1.047181] ACPI: \_SB_.GSIH: Enabled at IRQ 231744lb # [ 1.033308] fuse: init (API version 7.45)1745worker2 # [ 1.051071] ACPI: \_SB_.GSIE: Enabled at IRQ 201746worker2 # [ 1.053650] ACPI: \_SB_.GSIF: Enabled at IRQ 211747client # [ 1.021611] x86/mm: Checked W+X mappings: passed, no W+X pages found.1748lb # [ 1.039057] ACPI: \_SB_.GSIG: Enabled at IRQ 221749client # [ 1.022689] Run /init as init process1750lb # [ 1.041775] ACPI: \_SB_.GSIH: Enabled at IRQ 231751lb # [ 1.046023] ACPI: \_SB_.GSIE: Enabled at IRQ 201752lb # [ 1.048770] ACPI: \_SB_.GSIF: Enabled at IRQ 211753client # [ 1.033232] systemd[1]: Inserted module 'autofs4'1754worker1 # [ 1.042843] x86/mm: Checked W+X mappings: passed, no W+X pages found.1755worker1 # [ 1.044214] Run /init as init process1756worker1 # [ 1.054465] systemd[1]: Inserted module 'autofs4'1757client # [ 1.049738] fuse: init (API version 7.45)1758worker2 # [ 1.082737] systemd[1]: Successfully made /usr/ read-only.1759client # [ 1.055764] ACPI: \_SB_.GSIG: Enabled at IRQ 221760client # [ 1.058572] ACPI: \_SB_.GSIH: Enabled at IRQ 231761client # [ 1.062333] ACPI: \_SB_.GSIE: Enabled at IRQ 201762lb # [ 1.079917] systemd[1]: Successfully made /usr/ read-only.1763worker1 # [ 1.070703] fuse: init (API version 7.45)1764client # [ 1.064915] ACPI: \_SB_.GSIF: Enabled at IRQ 211765worker1 # [ 1.076497] ACPI: \_SB_.GSIG: Enabled at IRQ 221766worker1 # [ 1.079194] ACPI: \_SB_.GSIH: Enabled at IRQ 231767worker1 # [ 1.083748] ACPI: \_SB_.GSIE: Enabled at IRQ 201768worker1 # [ 1.087303] ACPI: \_SB_.GSIF: Enabled at IRQ 211769client # [ 1.092741] systemd[1]: Successfully made /usr/ read-only.1770worker1 # [ 1.121851] systemd[1]: Successfully made /usr/ read-only.1771worker2 # [ 1.419662] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1772worker2 # [ 1.431719] systemd[1]: Detected virtualization kvm.1773worker2 # [ 1.433879] systemd[1]: Detected architecture x86-64.1774worker2 # [ 1.436034] systemd[1]: Running in initrd.1775worker2 # [ 1.438733] systemd[1]: Initializing machine ID from random generator.1776worker2 # [ 1.441690] systemd[1]: Hostname set to <worker2>.1777lb # [ 1.416560] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1778lb # [ 1.430516] systemd[1]: Detected virtualization kvm.1779lb # [ 1.433025] systemd[1]: Detected architecture x86-64.1780lb # [ 1.435494] systemd[1]: Running in initrd.1781lb # [ 1.438398] systemd[1]: Initializing machine ID from random generator.1782lb # [ 1.441806] systemd[1]: Hostname set to <lb>.1783client # [ 1.428307] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1784client # [ 1.433189] systemd[1]: Detected virtualization kvm.1785client # [ 1.434024] systemd[1]: Detected architecture x86-64.1786client # [ 1.434860] systemd[1]: Running in initrd.1787client # [ 1.436002] systemd[1]: Initializing machine ID from random generator.1788client # [ 1.437185] systemd[1]: Hostname set to <client>.1789worker1 # [ 1.458387] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1790worker1 # [ 1.470516] systemd[1]: Detected virtualization kvm.1791worker1 # [ 1.472734] systemd[1]: Detected architecture x86-64.1792worker1 # [ 1.474929] systemd[1]: Running in initrd.1793worker1 # [ 1.477496] systemd[1]: Initializing machine ID from random generator.1794worker1 # [ 1.480431] systemd[1]: Hostname set to <worker1>.1795worker2 # [ 1.654534] systemd[1]: bpf-restrict-fs: LSM BPF program attached1796lb # [ 1.647466] systemd[1]: bpf-restrict-fs: LSM BPF program attached1797client # [ 1.640801] systemd[1]: bpf-restrict-fs: LSM BPF program attached1798worker1 # [ 1.678150] systemd[1]: bpf-restrict-fs: LSM BPF program attached1799worker2 # [ 1.705853] systemd[1]: Queued start job for default target Initrd Default Target.1800lb # [ 1.694623] systemd[1]: Queued start job for default target Initrd Default Target.1801worker2 # [ 1.710863] systemd[1]: Created slice Slice /system/modprobe.1802worker2 # [ 1.712731] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1803lb # [ 1.698451] systemd[1]: Created slice Slice /system/modprobe.1804worker2 # [ 1.714276] systemd[1]: Expecting device /dev/disk/by-label/nixos...1805worker2 # [ 1.715421] systemd[1]: Reached target Path Units.1806lb # [ 1.699627] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1807worker2 # [ 1.716278] systemd[1]: Reached target Slice Units.1808lb # [ 1.700987] systemd[1]: Expecting device /dev/disk/by-label/nixos...1809worker2 # [ 1.717153] systemd[1]: Reached target Swaps.1810lb # [ 1.702062] systemd[1]: Reached target Path Units.1811worker2 # [ 1.717965] systemd[1]: Reached target Timer Units.1812lb # [ 1.702919] systemd[1]: Reached target Slice Units.1813worker2 # [ 1.718992] systemd[1]: Listening on D-Bus System Message Bus Socket.1814lb # [ 1.703773] systemd[1]: Reached target Swaps.1815lb # [ 1.704561] systemd[1]: Reached target Timer Units.1816worker2 # [ 1.720235] systemd[1]: Listening on Journal Socket (/dev/log).1817worker2 # [ 1.721396] systemd[1]: Listening on Journal Sockets.1818lb # [ 1.705513] systemd[1]: Listening on D-Bus System Message Bus Socket.1819worker2 # [ 1.722414] systemd[1]: Listening on udev Control Socket.1820lb # [ 1.706689] systemd[1]: Listening on Journal Socket (/dev/log).1821worker2 # [ 1.723472] systemd[1]: Listening on udev Kernel Socket.1822lb # [ 1.707830] systemd[1]: Listening on Journal Sockets.1823worker2 # [ 1.724388] systemd[1]: Reached target Socket Units.1824lb # [ 1.708829] systemd[1]: Listening on udev Control Socket.1825client # [ 1.692741] systemd[1]: Queued start job for default target Initrd Default Target.1826lb # [ 1.709849] systemd[1]: Listening on udev Kernel Socket.1827lb # [ 1.710830] systemd[1]: Reached target Socket Units.1828worker2 # [ 1.727543] systemd[1]: Starting Create List of Static Device Nodes...1829lb # [ 1.712707] systemd[1]: Starting Create List of Static Device Nodes...1830client # [ 1.697311] systemd[1]: Created slice Slice /system/modprobe.1831client # [ 1.698460] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1832client # [ 1.699902] systemd[1]: Expecting device /dev/disk/by-label/nixos...1833worker2 # [ 1.732533] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1834lb # [ 1.717238] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1835client # [ 1.701017] systemd[1]: Reached target Path Units.1836client # [ 1.701835] systemd[1]: Reached target Slice Units.1837client # [ 1.702750] systemd[1]: Reached target Swaps.1838client # [ 1.703618] systemd[1]: Reached target Timer Units.1839worker2 # [ 1.736811] systemd[1]: Starting Load Kernel Module configfs...1840client # [ 1.704642] systemd[1]: Listening on D-Bus System Message Bus Socket.1841client # [ 1.705848] systemd[1]: Listening on Journal Socket (/dev/log).1842client # [ 1.707046] systemd[1]: Listening on Journal Sockets.1843lb # [ 1.724253] systemd[1]: Starting Load Kernel Module configfs...1844client # [ 1.708048] systemd[1]: Listening on udev Control Socket.1845client # [ 1.709043] systemd[1]: Listening on udev Kernel Socket.1846client # [ 1.710009] systemd[1]: Reached target Socket Units.1847worker1 # [ 1.716745] systemd[1]: Queued start job for default target Initrd Default Target.1848client # [ 1.711770] systemd[1]: Starting Create List of Static Device Nodes...1849worker1 # [ 1.721295] systemd[1]: Created slice Slice /system/modprobe.1850lb # [ 1.733341] systemd[1]: Starting Journal Service...1851worker1 # [ 1.722454] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1852client # [ 1.717033] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1853worker1 # [ 1.723794] systemd[1]: Expecting device /dev/disk/by-label/nixos...1854worker1 # [ 1.724851] systemd[1]: Reached target Path Units.1855worker1 # [ 1.725690] systemd[1]: Reached target Slice Units.1856worker2 # [ 1.752972] systemd[1]: Starting Journal Service...1857worker1 # [ 1.726546] systemd[1]: Reached target Swaps.1858worker1 # [ 1.727307] systemd[1]: Reached target Timer Units.1859worker1 # [ 1.728244] systemd[1]: Listening on D-Bus System Message Bus Socket.1860worker1 # [ 1.729434] systemd[1]: Listening on Journal Socket (/dev/log).1861client # [ 1.724022] systemd[1]: Starting Load Kernel Module configfs...1862worker1 # [ 1.730542] systemd[1]: Listening on Journal Sockets.1863worker1 # [ 1.731513] systemd[1]: Listening on udev Control Socket.1864worker1 # [ 1.732497] systemd[1]: Listening on udev Kernel Socket.1865worker1 # [ 1.733426] systemd[1]: Reached target Socket Units.1866worker1 # [ 1.735177] systemd[1]: Starting Create List of Static Device Nodes...1867lb # [ 1.747240] systemd[1]: Starting Load Kernel Modules...1868worker1 # [ 1.738790] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1869worker2 # [ 1.767573] systemd[1]: Starting Load Kernel Modules...1870client # [ 1.738270] systemd[1]: Starting Journal Service...1871worker1 # [ 1.745562] systemd[1]: Starting Load Kernel Module configfs...1872client # [ 1.747271] systemd[1]: Starting Load Kernel Modules...1873lb # [ 1.754241] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1874worker1 # [ 1.760307] systemd[1]: Starting Journal Service...1875worker2 # [ 1.775064] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1876lb # [ 1.773644] netfs: FS-Cache loaded1877worker2 # [ 1.793321] netfs: FS-Cache loaded1878lb # [ 1.778361] 9pnet: Installing 9P2000 support1879client # [ 1.755030] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1880worker1 # [ 1.773865] systemd[1]: Starting Load Kernel Modules...1881lb # [ 1.786894] systemd[1]: Starting Coldplug All udev Devices...1882worker2 # [ 1.803754] 9pnet: Installing 9P2000 support1883client # [ 1.772795] netfs: FS-Cache loaded1884client # [ 1.777612] 9pnet: Installing 9P2000 support1885worker2 # [ 1.815188] systemd[1]: Starting Coldplug All udev Devices...1886client # [ 1.786646] systemd[1]: Starting Coldplug All udev Devices...1887worker1 # [ 1.781066] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1888worker2 # [ 1.822193] systemd-journald[67]: Collecting audit messages is disabled.1889worker1 # [ 1.796593] netfs: FS-Cache loaded1890lb # [ 1.807464] systemd[1]: Finished Create List of Static Device Nodes.1891worker1 # [ 1.801784] 9pnet: Installing 9P2000 support1892lb # [ 1.813026] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1893worker2 # [ 1.831608] systemd[1]: Finished Create List of Static Device Nodes.1894client # [ 1.801034] systemd[1]: Finished Create List of Static Device Nodes.1895lb # [ 1.820068] systemd-journald[67]: Collecting audit messages is disabled.1896worker2 # [ 1.836185] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1897lb # [ 1.822563] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1898client # [ 1.806527] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1899lb # [ 1.827573] systemd[1]: modprobe@configfs.service: Deactivated successfully.1900worker2 # [ 1.843869] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1901client # [ 1.814376] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1902worker1 # [ 1.821514] systemd[1]: Starting Coldplug All udev Devices...1903worker2 # [ 1.849839] systemd[1]: modprobe@configfs.service: Deactivated successfully.1904client # [ 1.817686] systemd-journald[67]: Collecting audit messages is disabled.1905lb # [ 1.835527] systemd[1]: Finished Load Kernel Module configfs.1906client # [ 1.820355] systemd[1]: modprobe@configfs.service: Deactivated successfully.1907lb # [ 1.839647] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1908worker2 # [ 1.856893] systemd[1]: Finished Load Kernel Module configfs.1909lb # [ 1.841819] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1910client # [ 1.827067] systemd[1]: Finished Load Kernel Module configfs.1911worker1 # [ 1.836057] systemd-journald[67]: Collecting audit messages is disabled.1912worker2 # [ 1.861996] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1913worker1 # [ 1.838150] systemd[1]: Finished Create List of Static Device Nodes.1914lb # [ 1.848963] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1915client # [ 1.833412] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1916worker2 # [ 1.866834] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1917lb # [ 1.853237] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1918worker1 # [ 1.844246] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1919client # [ 1.838429] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1920worker2 # [ 1.874484] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1921worker2 # [ 1.876346] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1922client # [ 1.845018] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1923worker1 # [ 1.851297] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1924client # [ 1.847974] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1925worker1 # [ 1.855348] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1926worker1 # [ 1.857837] systemd[1]: modprobe@configfs.service: Deactivated successfully.1927worker1 # [ 1.863475] systemd[1]: Finished Load Kernel Module configfs.1928worker1 # [ 1.864489] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1929lb # [ 1.879330] systemd[1]: Finished Load Kernel Modules.1930worker1 # [ 1.868529] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1931lb # [ 1.887515] systemd[1]: Starting Apply Kernel Variables...1932worker2 # [ 1.905121] systemd[1]: Finished Load Kernel Modules.1933worker1 # [ 1.880207] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1934client # [ 1.877483] systemd[1]: Finished Load Kernel Modules.1935worker2 # [ 1.913731] systemd[1]: Starting Apply Kernel Variables...1936client # [ 1.882547] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1937lb # [ 1.903472] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1938client # [ 1.891163] systemd[1]: Starting Apply Kernel Variables...1939worker1 # [ 1.899085] systemd[1]: Finished Load Kernel Modules.1940lb # [ 1.912350] systemd[1]: Starting Create Static Device Nodes in /dev...1941worker2 # [ 1.930632] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1942worker1 # [ 1.906406] systemd[1]: Starting Apply Kernel Variables...1943client # [ 1.903025] systemd[1]: Starting Create Static Device Nodes in /dev...1944worker2 # [ 1.940926] systemd[1]: Starting Create Static Device Nodes in /dev...1945lb # [ 1.926259] systemd[1]: Finished Apply Kernel Variables.1946worker2 # [ 1.811883] systemd-modules-load[68]: Inserted module 'dm_mod'1947worker2 # [ 1.815978] systemd-modules-load[68]: Inserted module 'virtio_balloon'1948worker2 # [ 1.817218] systemd-modules-load[68]: Inserted module 'virtio_gpu'1949lb # [ 1.939287] systemd[1]: Started Journal Service.1950worker2 # [ 1.955677] systemd[1]: Started Journal Service.1951lb # [ 1.807286] systemd-modules-load[68]: Inserted module 'dm_mod'1952lb # [ 1.809470] systemd-modules-load[68]: Inserted module 'virtio_balloon'1953worker1 # [ 1.933110] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1954lb # [ 1.811779] systemd-modules-load[68]: Inserted module 'virtio_gpu'1955worker2 # [ 1.827098] systemd[1]: Finished Apply Kernel Variables.1956client # [ 1.931518] systemd[1]: Finished Apply Kernel Variables.1957client # [ 1.935188] systemd[1]: Started Journal Service.1958client # [ 1.799872] systemd-modules-load[68]: Inserted module 'dm_mod'1959worker1 # [ 1.942343] systemd[1]: Starting Create Static Device Nodes in /dev...1960client # [ 1.803933] systemd-modules-load[68]: Inserted module 'virtio_balloon'1961client # [ 1.805051] systemd-modules-load[68]: Inserted module 'virtio_gpu'1962lb # [ 1.823151] systemd[1]: Finished Create Static Device Nodes in /dev.1963client # [ 1.809237] systemd[1]: Finished Create Static Device Nodes in /dev.1964lb # [ 1.826073] systemd[1]: Reached target Preparation for Local File Systems.1965lb # [ 1.827101] systemd[1]: Reached target Local File Systems.1966worker2 # [ 1.842665] systemd[1]: Finished Create Static Device Nodes in /dev.1967lb # [ 1.830068] systemd[1]: Starting Create System Files and Directories...1968worker2 # [ 1.845255] systemd[1]: Reached target Preparation for Local File Systems.1969client # [ 1.816119] systemd[1]: Reached target Preparation for Local File Systems.1970worker2 # [ 1.846304] systemd[1]: Reached target Local File Systems.1971client # [ 1.817093] systemd[1]: Reached target Local File Systems.1972lb # [ 1.835068] systemd[1]: Starting Rule-based Manager for Device Events and Files...1973worker1 # [ 1.959104] systemd[1]: Finished Apply Kernel Variables.1974client # [ 1.820066] systemd[1]: Starting Create System Files and Directories...1975worker2 # [ 1.851078] systemd[1]: Starting Create System Files and Directories...1976worker1 # [ 1.824862] systemd-modules-load[68]: Inserted module 'dm_mod'1977worker1 # [ 1.829987] systemd-modules-load[68]: Inserted module 'virtio_balloon'1978client # [ 1.824989] systemd[1]: Starting Rule-based Manager for Device Events and Files...1979worker1 # [ 1.832174] systemd-modules-load[68]: Inserted module 'virtio_gpu'1980worker1 # [ 1.966053] systemd[1]: Started Journal Service.1981worker2 # [ 1.856345] systemd[1]: Starting Rule-based Manager for Device Events and Files...1982lb # [ 1.860404] systemd[1]: Finished Create System Files and Directories.1983worker1 # [ 1.851162] systemd[1]: Finished Create Static Device Nodes in /dev.1984worker1 # [ 1.852220] systemd[1]: Reached target Preparation for Local File Systems.1985worker1 # [ 1.853263] systemd[1]: Reached target Local File Systems.1986client # [ 1.848237] systemd[1]: Finished Create System Files and Directories.1987worker1 # [ 1.856062] systemd[1]: Starting Create System Files and Directories...1988worker1 # [ 1.860649] systemd[1]: Starting Rule-based Manager for Device Events and Files...1989worker2 # [ 1.884116] systemd[1]: Finished Create System Files and Directories.1990lb # [ 1.886742] systemd-udevd[83]: Using default interface naming scheme 'v261'.1991client # [ 1.875284] systemd-udevd[83]: Using default interface naming scheme 'v261'.1992worker2 # [ 1.908300] systemd-udevd[83]: Using default interface naming scheme 'v261'.1993worker1 # [ 1.890134] systemd[1]: Finished Create System Files and Directories.1994lb # [ 1.912204] systemd[1]: Started Rule-based Manager for Device Events and Files.1995client # [ 1.900403] systemd[1]: Started Rule-based Manager for Device Events and Files.1996worker2 # [ 1.932757] systemd[1]: Started Rule-based Manager for Device Events and Files.1997worker1 # [ 1.914755] systemd-udevd[83]: Using default interface naming scheme 'v261'.1998worker1 # [ 1.941135] systemd[1]: Started Rule-based Manager for Device Events and Files.1999lb # [ 1.956181] systemd[1]: Finished Coldplug All udev Devices.2000lb # [ 1.957219] systemd[1]: Reached target System Initialization.

loading 2,034 hidden lines…

4035lb # [ 17.686155] envoy[979]: [2026-09-20 12:21:26.119][979][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation4036lb # [ 17.688142] envoy[979]: [2026-09-20 12:21:26.119][979][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options4037lb # [ 17.692169] envoy[979]: [2026-09-20 12:21:26.119][979][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input4038lb # [ 17.694024] envoy[979]: [2026-09-20 12:21:26.119][979][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed4039lb # [ 17.696720] envoy[979]: [2026-09-20 12:21:26.119][979][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4040lb # [ 17.699200] envoy[979]: [2026-09-20 12:21:26.119][979][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case4041lb # [ 17.701454] envoy[979]: [2026-09-20 12:21:26.119][979][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor4042lb # [ 17.704114] envoy[979]: [2026-09-20 12:21:26.119][979][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection4043lb # [ 17.706149] envoy[979]: [2026-09-20 12:21:26.119][979][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian24044lb # [ 17.707842] envoy[979]: [2026-09-20 12:21:26.125][979][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo4045lb # [ 17.709534] envoy[979]: [2026-09-20 12:21:26.125][979][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest4046lb # [ 17.714424] envoy[979]: [2026-09-20 12:21:26.125][979][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re24047lb # [ 17.716152] envoy[979]: [2026-09-20 12:21:26.125][979][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params4048lb # [ 17.725560] envoy[979]: [2026-09-20 12:21:26.125][979][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector4049lb # [ 17.731405] envoy[979]: [2026-09-20 12:21:26.126][979][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override4050lb # [ 17.734344] envoy[979]: [2026-09-20 12:21:26.126][979][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr4051lb # [ 17.736265] envoy[979]: [2026-09-20 12:21:26.126][979][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based4052lb # [ 17.740235] envoy[979]: [2026-09-20 12:21:26.126][979][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit4053lb # [ 17.742786] envoy[979]: [2026-09-20 12:21:26.126][979][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response4054lb # [ 17.744872] envoy[979]: [2026-09-20 12:21:26.126][979][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter4055lb # [ 17.747201] envoy[979]: [2026-09-20 12:21:26.126][979][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default4056lb # [ 17.749539] envoy[979]: [2026-09-20 12:21:26.126][979][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple4057lb # [ 17.751720] envoy[979]: [2026-09-20 12:21:26.126][979][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme4058lb # [ 17.755348] envoy[979]: [2026-09-20 12:21:26.126][979][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap4059lb # [ 17.759679] envoy[979]: [2026-09-20 12:21:26.126][979][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy4060lb # [ 17.762114] envoy[979]: [2026-09-20 12:21:26.126][979][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated4061lb # [ 17.764071] envoy[979]: [2026-09-20 12:21:26.126][979][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle4062lb # [ 17.766033] envoy[979]: [2026-09-20 12:21:26.126][979][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default4063lb # [ 17.767615] envoy[979]: [2026-09-20 12:21:26.126][979][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query4064lb # [ 17.770429] envoy[979]: [2026-09-20 12:21:26.127][979][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4065lb # [ 17.772384] envoy[979]: [2026-09-20 12:21:26.127][979][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory4066lb # [ 17.774694] envoy[979]: [2026-09-20 12:21:26.127][979][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid4067lb # [ 17.776814] envoy[979]: [2026-09-20 12:21:26.127][979][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain4068lb # [ 17.778825] envoy[979]: [2026-09-20 12:21:26.127][979][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default4069lb # [ 17.781064] envoy[979]: [2026-09-20 12:21:26.127][979][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple4070lb # [ 17.783814] envoy[979]: [2026-09-20 12:21:26.127][979][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule4071lb # [ 17.786363] envoy[979]: [2026-09-20 12:21:26.127][979][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4072lb # [ 17.788344] envoy[979]: [2026-09-20 12:21:26.127][979][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin4073lb # [ 17.791598] envoy[979]: [2026-09-20 12:21:26.127][979][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed4074lb # [ 17.793385] envoy[979]: [2026-09-20 12:21:26.127][979][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd4075lb # [ 17.798325] envoy[979]: [2026-09-20 12:21:26.127][979][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy4076lb # [ 17.809740] envoy[979]: [2026-09-20 12:21:26.127][979][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port4077lb # [ 17.811772] envoy[979]: [2026-09-20 12:21:26.127][979][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche4078lb # [ 17.813838] envoy[979]: [2026-09-20 12:21:26.127][979][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default4079lb # [ 17.816149] envoy[979]: [2026-09-20 12:21:26.127][979][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter4080lb # [ 17.818158] envoy[979]: [2026-09-20 12:21:26.127][979][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal4081lb # [ 17.820098] envoy[979]: [2026-09-20 12:21:26.127][979][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe4082lb # [ 17.823085] envoy[979]: [2026-09-20 12:21:26.127][979][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls4083lb # [ 17.828285] envoy[979]: [2026-09-20 12:21:26.127][979][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth24084lb # [ 17.830366] envoy[979]: [2026-09-20 12:21:26.127][979][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override4085lb # [ 17.832916] envoy[979]: [2026-09-20 12:21:26.127][979][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic4086lb # [ 17.834868] envoy[979]: [2026-09-20 12:21:26.127][979][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction4087lb # [ 17.839194] envoy[979]: [2026-09-20 12:21:26.127][979][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san4088lb # [ 17.845784] envoy[979]: [2026-09-20 12:21:26.127][979][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based4089lb # [ 17.847499] envoy[979]: [2026-09-20 12:21:26.127][979][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router4090lb # [ 17.872960] envoy[979]: [2026-09-20 12:21:26.264][979][info][main] [source/server/server.cc:531] HTTP header map info:4091lb # [ 17.874292] envoy[979]: [2026-09-20 12:21:26.310][979][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id4092lb # [ 17.888133] envoy[979]: [2026-09-20 12:21:26.310][979][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:4093lb # [ 17.889574] envoy[979]: [2026-09-20 12:21:26.310][979][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id4094lb # [ 17.896971] envoy[979]: [2026-09-20 12:21:26.310][979][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status4095lb # [ 17.946304] envoy[979]: [2026-09-20 12:21:26.514][979][info][main] [source/server/server.cc:968] runtime: {}4096lb # [ 17.952835] envoy[979]: [2026-09-20 12:21:26.520][979][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.4097lb # [ 17.955573] envoy[979]: [2026-09-20 12:21:26.523][979][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99014098lb # [ 17.959263] envoy[979]: [2026-09-20 12:21:26.527][979][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99014099lb # [ 17.962630] envoy[979]: [2026-09-20 12:21:26.530][979][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration4100lb # [ 17.965190] envoy[979]: [2026-09-20 12:21:26.533][979][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)4101lb # [ 17.966783] envoy[979]: [2026-09-20 12:21:26.533][979][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)4102lb # [ 18.018429] envoy[979]: [2026-09-20 12:21:26.586][979][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)4103worker2 # [ 18.034830] systemd[1]: Reached target Network is Online.4104worker1 # [ 18.013820] systemd[1]: Starting Nix worker-protocol over gRPC...4105worker2 # [ 18.043752] systemd[1]: Starting scheduler-token.service...4106lb # [ 18.040806] envoy[979]: [2026-09-20 12:21:26.608][979][info][config] [source/server/configuration_impl.cc:164] loading stats configuration4107lb # [ 18.043840] envoy[979]: [2026-09-20 12:21:26.611][979][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.4108lb # [ 18.047056] envoy[979]: [2026-09-20 12:21:26.611][979][info][main] [source/server/server.cc:1077] starting main dispatch loop4109worker2 # [ 18.168504] systemd[1]: scheduler-token.service: Deactivated successfully.4110worker2 # [ 18.170219] systemd[1]: Finished scheduler-token.service.4111worker2 # [ 18.172188] systemd[1]: scheduler-token.service: Consumed 38ms CPU time over 126ms wall clock time, 3.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4112worker2 # [ 18.176697] systemd[1]: Starting Nix worker-protocol over gRPC...4113worker1 # [ 18.221303] nix-grpc-daemon[788]: ts=2026-09-20T12:21:27Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4114worker1 # [ 18.232117] nix-grpc-daemon[788]: ts=2026-09-20T12:21:27Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4115worker1 # [ 18.236446] nix-grpc-daemon[788]: ts=2026-09-20T12:21:27Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14116worker1 # [ 18.273612] nix-grpc-daemon[788]: ts=2026-09-20T12:21:27Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4117worker1 # [ 18.277648] systemd[1]: Started Nix worker-protocol over gRPC.4118worker1 # [ 18.318512] nix-grpc-daemon[788]: ts=2026-09-20T12:21:27Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4119lb # [ 18.329081] envoy[979]: [2026-09-20 12:21:26.897][979][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization4120lb # [ 18.331122] envoy[979]: [2026-09-20 12:21:26.899][979][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized4121lb # [ 18.332792] envoy[979]: [2026-09-20 12:21:26.899][979][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager4122lb # [ 18.334398] envoy[979]: [2026-09-20 12:21:26.902][979][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers4123worker2 # [ 18.367765] nix-grpc-daemon[787]: ts=2026-09-20T12:21:27Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4124worker2 # [ 18.377685] nix-grpc-daemon[787]: ts=2026-09-20T12:21:27Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4125worker2 # [ 18.383483] nix-grpc-daemon[787]: ts=2026-09-20T12:21:27Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14126worker2 # [ 18.429969] nix-grpc-daemon[787]: ts=2026-09-20T12:21:27Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4127worker2 # [ 18.433903] systemd[1]: Started Nix worker-protocol over gRPC.4128worker2 # [ 18.485854] nix-grpc-daemon[787]: ts=2026-09-20T12:21:27Z level=info event=scheduler_connected addr=lb:500514129worker2 # [ 18.499171] nix-grpc-daemon[787]: ts=2026-09-20T12:21:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4130worker1 # [ 18.519937] nix-grpc-daemon[788]: ts=2026-09-20T12:21:27Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4131worker2 # [ 18.701078] nix-grpc-daemon[787]: ts=2026-09-20T12:21:27Z level=info event=scheduler_connected addr=lb:500514132worker2 # [ 18.702852] nix-grpc-daemon[787]: ts=2026-09-20T12:21:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4133worker2 # [ 18.903545] nix-grpc-daemon[787]: ts=2026-09-20T12:21:27Z level=info event=scheduler_connected addr=lb:500514134worker2 # [ 18.905334] nix-grpc-daemon[787]: ts=2026-09-20T12:21:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4135worker1 # [ 18.920725] nix-grpc-daemon[788]: ts=2026-09-20T12:21:27Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4136worker2 # [ 19.106253] nix-grpc-daemon[787]: ts=2026-09-20T12:21:27Z level=info event=scheduler_connected addr=lb:500514137worker2 # [ 19.108238] nix-grpc-daemon[787]: ts=2026-09-20T12:21:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4138worker2 # [ 19.308882] nix-grpc-daemon[787]: ts=2026-09-20T12:21:28Z level=info event=scheduler_connected addr=lb:500514139worker2 # [ 19.310638] nix-grpc-daemon[787]: ts=2026-09-20T12:21:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4140worker2 # [ 19.513203] nix-grpc-daemon[787]: ts=2026-09-20T12:21:28Z level=info event=scheduler_connected addr=lb:500514141worker2 # [ 19.516503] nix-grpc-daemon[787]: ts=2026-09-20T12:21:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4142worker2 # [ 19.714544] nix-grpc-daemon[787]: ts=2026-09-20T12:21:28Z level=info event=scheduler_connected addr=lb:500514143worker2 # [ 19.718373] nix-grpc-daemon[787]: ts=2026-09-20T12:21:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4144worker1 # [ 19.725404] nix-grpc-daemon[788]: ts=2026-09-20T12:21:28Z level=info event=scheduler_connected addr=lb:500514145worker1 # [ 19.729686] nix-grpc-daemon[788]: ts=2026-09-20T12:21:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4146worker2 # [ 19.919559] nix-grpc-daemon[787]: ts=2026-09-20T12:21:28Z level=info event=scheduler_connected addr=lb:500514147worker2 # [ 19.923330] nix-grpc-daemon[787]: ts=2026-09-20T12:21:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4148worker1 # [ 19.930918] nix-grpc-daemon[788]: ts=2026-09-20T12:21:28Z level=info event=scheduler_connected addr=lb:500514149worker1 # [ 19.934909] nix-grpc-daemon[788]: ts=2026-09-20T12:21:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4150worker2 # [ 20.124308] nix-grpc-daemon[787]: ts=2026-09-20T12:21:28Z level=info event=scheduler_connected addr=lb:500514151worker2 # [ 20.128126] nix-grpc-daemon[787]: ts=2026-09-20T12:21:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4152worker1 # [ 20.136089] nix-grpc-daemon[788]: ts=2026-09-20T12:21:29Z level=info event=scheduler_connected addr=lb:500514153worker1 # [ 20.140138] nix-grpc-daemon[788]: ts=2026-09-20T12:21:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4154worker2 # [ 20.328778] nix-grpc-daemon[787]: ts=2026-09-20T12:21:29Z level=info event=scheduler_connected addr=lb:500514155worker2 # [ 20.330685] nix-grpc-daemon[787]: ts=2026-09-20T12:21:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4156worker1 # [ 20.340422] nix-grpc-daemon[788]: ts=2026-09-20T12:21:29Z level=info event=scheduler_connected addr=lb:500514157worker1 # [ 20.342380] nix-grpc-daemon[788]: ts=2026-09-20T12:21:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4158worker2 # [ 20.531553] nix-grpc-daemon[787]: ts=2026-09-20T12:21:29Z level=info event=scheduler_connected addr=lb:500514159worker2 # [ 20.533349] nix-grpc-daemon[787]: ts=2026-09-20T12:21:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4160worker1 # [ 20.543443] nix-grpc-daemon[788]: ts=2026-09-20T12:21:29Z level=info event=scheduler_connected addr=lb:500514161worker1 # [ 20.547514] nix-grpc-daemon[788]: ts=2026-09-20T12:21:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4162worker2 # [ 20.734154] nix-grpc-daemon[787]: ts=2026-09-20T12:21:29Z level=info event=scheduler_connected addr=lb:500514163worker2 # [ 20.736316] nix-grpc-daemon[787]: ts=2026-09-20T12:21:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4164worker1 # [ 20.749962] nix-grpc-daemon[788]: ts=2026-09-20T12:21:29Z level=info event=scheduler_connected addr=lb:500514165worker1 # [ 20.753487] nix-grpc-daemon[788]: ts=2026-09-20T12:21:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4166worker2 # [ 20.937553] nix-grpc-daemon[787]: ts=2026-09-20T12:21:29Z level=info event=scheduler_connected addr=lb:500514167worker2 # [ 20.941482] nix-grpc-daemon[787]: ts=2026-09-20T12:21:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4168worker1 # [ 20.951680] nix-grpc-daemon[788]: ts=2026-09-20T12:21:29Z level=info event=scheduler_connected addr=lb:500514169worker1 # [ 20.955398] nix-grpc-daemon[788]: ts=2026-09-20T12:21:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4170worker2 # [ 21.142222] nix-grpc-daemon[787]: ts=2026-09-20T12:21:29Z level=info event=scheduler_connected addr=lb:500514171worker2 # [ 21.144173] nix-grpc-daemon[787]: ts=2026-09-20T12:21:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4172worker1 # [ 21.156398] nix-grpc-daemon[788]: ts=2026-09-20T12:21:30Z level=info event=scheduler_connected addr=lb:500514173worker1 # [ 21.160287] nix-grpc-daemon[788]: ts=2026-09-20T12:21:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4174worker2 # [ 21.344770] nix-grpc-daemon[787]: ts=2026-09-20T12:21:30Z level=info event=scheduler_connected addr=lb:500514175worker2 # [ 21.347313] nix-grpc-daemon[787]: ts=2026-09-20T12:21:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4176worker1 # [ 21.360836] nix-grpc-daemon[788]: ts=2026-09-20T12:21:30Z level=info event=scheduler_connected addr=lb:500514177worker1 # [ 21.364725] nix-grpc-daemon[788]: ts=2026-09-20T12:21:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4178worker2 # [ 21.548193] nix-grpc-daemon[787]: ts=2026-09-20T12:21:30Z level=info event=scheduler_connected addr=lb:500514179worker2 # [ 21.549821] nix-grpc-daemon[787]: ts=2026-09-20T12:21:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4180worker1 # [ 21.565571] nix-grpc-daemon[788]: ts=2026-09-20T12:21:30Z level=info event=scheduler_connected addr=lb:500514181worker1 # [ 21.569506] nix-grpc-daemon[788]: ts=2026-09-20T12:21:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4182worker2 # [ 21.751113] nix-grpc-daemon[787]: ts=2026-09-20T12:21:30Z level=info event=scheduler_connected addr=lb:500514183worker2 # [ 21.754925] nix-grpc-daemon[787]: ts=2026-09-20T12:21:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4184worker1 # [ 21.771835] nix-grpc-daemon[788]: ts=2026-09-20T12:21:30Z level=info event=scheduler_connected addr=lb:500514185worker1 # [ 21.775143] nix-grpc-daemon[788]: ts=2026-09-20T12:21:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4186worker2 # [ 21.955744] nix-grpc-daemon[787]: ts=2026-09-20T12:21:30Z level=info event=scheduler_connected addr=lb:500514187worker2 # [ 21.959546] nix-grpc-daemon[787]: ts=2026-09-20T12:21:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4188worker1 # [ 21.975994] nix-grpc-daemon[788]: ts=2026-09-20T12:21:30Z level=info event=scheduler_connected addr=lb:500514189worker1 # [ 21.979833] nix-grpc-daemon[788]: ts=2026-09-20T12:21:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4190worker2 # [ 22.160127] nix-grpc-daemon[787]: ts=2026-09-20T12:21:30Z level=info event=scheduler_connected addr=lb:500514191worker2 # [ 22.161817] nix-grpc-daemon[787]: ts=2026-09-20T12:21:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4192worker1 # [ 22.180924] nix-grpc-daemon[788]: ts=2026-09-20T12:21:31Z level=info event=scheduler_connected addr=lb:500514193worker1 # [ 22.184601] nix-grpc-daemon[788]: ts=2026-09-20T12:21:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4194worker2 # [ 22.362903] nix-grpc-daemon[787]: ts=2026-09-20T12:21:31Z level=info event=scheduler_connected addr=lb:500514195worker2 # [ 22.366862] nix-grpc-daemon[787]: ts=2026-09-20T12:21:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4196worker1 # [ 22.382651] nix-grpc-daemon[788]: ts=2026-09-20T12:21:31Z level=info event=scheduler_connected addr=lb:500514197worker1 # [ 22.386550] nix-grpc-daemon[788]: ts=2026-09-20T12:21:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4198worker2 # [ 22.569209] nix-grpc-daemon[787]: ts=2026-09-20T12:21:31Z level=info event=scheduler_connected addr=lb:500514199worker2 # [ 22.572375] nix-grpc-daemon[787]: ts=2026-09-20T12:21:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4200worker1 # [ 22.587134] nix-grpc-daemon[788]: ts=2026-09-20T12:21:31Z level=info event=scheduler_connected addr=lb:500514201worker1 # [ 22.589098] nix-grpc-daemon[788]: ts=2026-09-20T12:21:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4202worker2 # [ 22.770472] nix-grpc-daemon[787]: ts=2026-09-20T12:21:31Z level=info event=scheduler_connected addr=lb:500514203worker2 # [ 22.774215] nix-grpc-daemon[787]: ts=2026-09-20T12:21:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4204worker1 # [ 22.791475] nix-grpc-daemon[788]: ts=2026-09-20T12:21:31Z level=info event=scheduler_connected addr=lb:500514205worker1 # [ 22.794552] nix-grpc-daemon[788]: ts=2026-09-20T12:21:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4206worker2 # [ 22.975140] nix-grpc-daemon[787]: ts=2026-09-20T12:21:31Z level=info event=scheduler_connected addr=lb:500514207worker2 # [ 22.978165] nix-grpc-daemon[787]: ts=2026-09-20T12:21:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4208worker1 # [ 22.993134] nix-grpc-daemon[788]: ts=2026-09-20T12:21:31Z level=info event=scheduler_connected addr=lb:500514209worker1 # [ 22.996415] nix-grpc-daemon[788]: ts=2026-09-20T12:21:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4210worker2 # [ 23.179257] nix-grpc-daemon[787]: ts=2026-09-20T12:21:31Z level=info event=scheduler_connected addr=lb:500514211worker2 # [ 23.182966] nix-grpc-daemon[787]: ts=2026-09-20T12:21:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4212worker1 # [ 23.194600] nix-grpc-daemon[788]: ts=2026-09-20T12:21:32Z level=info event=scheduler_connected addr=lb:500514213worker1 # [ 23.196483] nix-grpc-daemon[788]: ts=2026-09-20T12:21:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4214worker2 # [ 23.383782] nix-grpc-daemon[787]: ts=2026-09-20T12:21:32Z level=info event=scheduler_connected addr=lb:500514215worker2 # [ 23.385610] nix-grpc-daemon[787]: ts=2026-09-20T12:21:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4216worker1 # [ 23.397533] nix-grpc-daemon[788]: ts=2026-09-20T12:21:32Z level=info event=scheduler_connected addr=lb:500514217worker1 # [ 23.401370] nix-grpc-daemon[788]: ts=2026-09-20T12:21:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4218worker2 # [ 23.588095] nix-grpc-daemon[787]: ts=2026-09-20T12:21:32Z level=info event=scheduler_connected addr=lb:500514219worker2 # [ 23.591306] nix-grpc-daemon[787]: ts=2026-09-20T12:21:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4220worker1 # [ 23.602196] nix-grpc-daemon[788]: ts=2026-09-20T12:21:32Z level=info event=scheduler_connected addr=lb:500514221worker1 # [ 23.605980] nix-grpc-daemon[788]: ts=2026-09-20T12:21:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4222worker2 # [ 23.788856] nix-grpc-daemon[787]: ts=2026-09-20T12:21:32Z level=info event=scheduler_connected addr=lb:500514223worker2 # [ 23.790811] nix-grpc-daemon[787]: ts=2026-09-20T12:21:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4224worker1 # [ 23.807257] nix-grpc-daemon[788]: ts=2026-09-20T12:21:32Z level=info event=scheduler_connected addr=lb:500514225worker1 # [ 23.808633] nix-grpc-daemon[788]: ts=2026-09-20T12:21:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4226worker2 # [ 23.991624] nix-grpc-daemon[787]: ts=2026-09-20T12:21:32Z level=info event=scheduler_connected addr=lb:500514227worker2 # [ 23.993244] nix-grpc-daemon[787]: ts=2026-09-20T12:21:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4228worker1 # [ 24.007956] nix-grpc-daemon[788]: ts=2026-09-20T12:21:32Z level=info event=scheduler_connected addr=lb:500514229worker1 # [ 24.009791] nix-grpc-daemon[788]: ts=2026-09-20T12:21:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4230worker2 # [ 24.194464] nix-grpc-daemon[787]: ts=2026-09-20T12:21:32Z level=info event=scheduler_connected addr=lb:500514231worker2 # [ 24.198202] nix-grpc-daemon[787]: ts=2026-09-20T12:21:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4232worker1 # [ 24.210529] nix-grpc-daemon[788]: ts=2026-09-20T12:21:33Z level=info event=scheduler_connected addr=lb:500514233worker1 # [ 24.212328] nix-grpc-daemon[788]: ts=2026-09-20T12:21:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4234worker2 # [ 24.398600] nix-grpc-daemon[787]: ts=2026-09-20T12:21:33Z level=info event=scheduler_connected addr=lb:500514235worker2 # [ 24.400329] nix-grpc-daemon[787]: ts=2026-09-20T12:21:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4236worker1 # [ 24.412915] nix-grpc-daemon[788]: ts=2026-09-20T12:21:33Z level=info event=scheduler_connected addr=lb:500514237worker1 # [ 24.414847] nix-grpc-daemon[788]: ts=2026-09-20T12:21:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4238worker2 # [ 24.602431] nix-grpc-daemon[787]: ts=2026-09-20T12:21:33Z level=info event=scheduler_connected addr=lb:500514239worker2 # [ 24.605924] nix-grpc-daemon[787]: ts=2026-09-20T12:21:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4240worker1 # [ 24.616128] nix-grpc-daemon[788]: ts=2026-09-20T12:21:33Z level=info event=scheduler_connected addr=lb:500514241worker1 # [ 24.617807] nix-grpc-daemon[788]: ts=2026-09-20T12:21:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4242worker2 # [ 24.807113] nix-grpc-daemon[787]: ts=2026-09-20T12:21:33Z level=info event=scheduler_connected addr=lb:500514243worker2 # [ 24.810915] nix-grpc-daemon[787]: ts=2026-09-20T12:21:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4244worker1 # [ 24.820734] nix-grpc-daemon[788]: ts=2026-09-20T12:21:33Z level=info event=scheduler_connected addr=lb:500514245worker1 # [ 24.823822] nix-grpc-daemon[788]: ts=2026-09-20T12:21:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4246worker2 # [ 25.011919] nix-grpc-daemon[787]: ts=2026-09-20T12:21:33Z level=info event=scheduler_connected addr=lb:500514247worker2 # [ 25.015708] nix-grpc-daemon[787]: ts=2026-09-20T12:21:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4248worker1 # [ 25.021625] nix-grpc-daemon[788]: ts=2026-09-20T12:21:33Z level=info event=scheduler_connected addr=lb:500514249worker1 # [ 25.025630] nix-grpc-daemon[788]: ts=2026-09-20T12:21:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4250worker2 # [ 25.216245] nix-grpc-daemon[787]: ts=2026-09-20T12:21:34Z level=info event=scheduler_connected addr=lb:500514251worker2 # [ 25.218392] nix-grpc-daemon[787]: ts=2026-09-20T12:21:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4252worker1 # [ 25.226644] nix-grpc-daemon[788]: ts=2026-09-20T12:21:34Z level=info event=scheduler_connected addr=lb:500514253worker1 # [ 25.230580] nix-grpc-daemon[788]: ts=2026-09-20T12:21:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4254worker2 # [ 25.419611] nix-grpc-daemon[787]: ts=2026-09-20T12:21:34Z level=info event=scheduler_connected addr=lb:500514255worker2 # [ 25.423519] nix-grpc-daemon[787]: ts=2026-09-20T12:21:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4256worker1 # [ 25.431643] nix-grpc-daemon[788]: ts=2026-09-20T12:21:34Z level=info event=scheduler_connected addr=lb:500514257worker1 # [ 25.435520] nix-grpc-daemon[788]: ts=2026-09-20T12:21:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4258worker2 # [ 25.625637] nix-grpc-daemon[787]: ts=2026-09-20T12:21:34Z level=info event=scheduler_connected addr=lb:500514259worker2 # [ 25.628846] nix-grpc-daemon[787]: ts=2026-09-20T12:21:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4260worker1 # [ 25.633543] nix-grpc-daemon[788]: ts=2026-09-20T12:21:34Z level=info event=scheduler_connected addr=lb:500514261worker1 # [ 25.637517] nix-grpc-daemon[788]: ts=2026-09-20T12:21:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4262worker2 # [ 25.826712] nix-grpc-daemon[787]: ts=2026-09-20T12:21:34Z level=info event=scheduler_connected addr=lb:500514263worker2 # [ 25.830600] nix-grpc-daemon[787]: ts=2026-09-20T12:21:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4264worker1 # [ 25.840421] nix-grpc-daemon[788]: ts=2026-09-20T12:21:34Z level=info event=scheduler_connected addr=lb:500514265worker1 # [ 25.843691] nix-grpc-daemon[788]: ts=2026-09-20T12:21:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4266worker2 # [ 26.031331] nix-grpc-daemon[787]: ts=2026-09-20T12:21:34Z level=info event=scheduler_connected addr=lb:500514267worker2 # [ 26.033132] nix-grpc-daemon[787]: ts=2026-09-20T12:21:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4268worker1 # [ 26.040456] nix-grpc-daemon[788]: ts=2026-09-20T12:21:34Z level=info event=scheduler_connected addr=lb:500514269worker1 # [ 26.042534] nix-grpc-daemon[788]: ts=2026-09-20T12:21:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4270worker2 # [ 26.234379] nix-grpc-daemon[787]: ts=2026-09-20T12:21:35Z level=info event=scheduler_connected addr=lb:500514271worker2 # [ 26.238225] nix-grpc-daemon[787]: ts=2026-09-20T12:21:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4272worker1 # [ 26.243286] nix-grpc-daemon[788]: ts=2026-09-20T12:21:35Z level=info event=scheduler_connected addr=lb:500514273worker1 # [ 26.247578] nix-grpc-daemon[788]: ts=2026-09-20T12:21:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4274worker2 # [ 26.438871] nix-grpc-daemon[787]: ts=2026-09-20T12:21:35Z level=info event=scheduler_connected addr=lb:500514275worker2 # [ 26.441315] nix-grpc-daemon[787]: ts=2026-09-20T12:21:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4276worker1 # [ 26.447866] nix-grpc-daemon[788]: ts=2026-09-20T12:21:35Z level=info event=scheduler_connected addr=lb:500514277worker1 # [ 26.450071] nix-grpc-daemon[788]: ts=2026-09-20T12:21:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4278worker2 # [ 26.643407] nix-grpc-daemon[787]: ts=2026-09-20T12:21:35Z level=info event=scheduler_connected addr=lb:500514279worker2 # [ 26.645881] nix-grpc-daemon[787]: ts=2026-09-20T12:21:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4280worker1 # [ 26.650764] nix-grpc-daemon[788]: ts=2026-09-20T12:21:35Z level=info event=scheduler_connected addr=lb:500514281worker1 # [ 26.652691] nix-grpc-daemon[788]: ts=2026-09-20T12:21:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4282worker2 # [ 26.844044] nix-grpc-daemon[787]: ts=2026-09-20T12:21:35Z level=info event=scheduler_connected addr=lb:500514283worker2 # [ 26.845825] nix-grpc-daemon[787]: ts=2026-09-20T12:21:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4284worker1 # [ 26.855352] nix-grpc-daemon[788]: ts=2026-09-20T12:21:35Z level=info event=scheduler_connected addr=lb:500514285worker1 # [ 26.858545] nix-grpc-daemon[788]: ts=2026-09-20T12:21:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4286worker2 # [ 27.047469] nix-grpc-daemon[787]: ts=2026-09-20T12:21:35Z level=info event=scheduler_connected addr=lb:500514287worker2 # [ 27.051641] nix-grpc-daemon[787]: ts=2026-09-20T12:21:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4288worker1 # [ 27.056369] nix-grpc-daemon[788]: ts=2026-09-20T12:21:35Z level=info event=scheduler_connected addr=lb:500514289worker1 # [ 27.061209] nix-grpc-daemon[788]: ts=2026-09-20T12:21:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4290worker2 # [ 27.252597] nix-grpc-daemon[787]: ts=2026-09-20T12:21:36Z level=info event=scheduler_connected addr=lb:500514291worker2 # [ 27.254676] nix-grpc-daemon[787]: ts=2026-09-20T12:21:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4292worker1 # [ 27.262682] nix-grpc-daemon[788]: ts=2026-09-20T12:21:36Z level=info event=scheduler_connected addr=lb:500514293worker1 # [ 27.265827] nix-grpc-daemon[788]: ts=2026-09-20T12:21:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4294worker2 # [ 27.455945] nix-grpc-daemon[787]: ts=2026-09-20T12:21:36Z level=info event=scheduler_connected addr=lb:500514295worker2 # [ 27.460210] nix-grpc-daemon[787]: ts=2026-09-20T12:21:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4296worker1 # [ 27.464081] nix-grpc-daemon[788]: ts=2026-09-20T12:21:36Z level=info event=scheduler_connected addr=lb:500514297worker1 # [ 27.467826] nix-grpc-daemon[788]: ts=2026-09-20T12:21:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4298worker2 # [ 27.662823] nix-grpc-daemon[787]: ts=2026-09-20T12:21:36Z level=info event=scheduler_connected addr=lb:500514299worker2 # [ 27.666183] nix-grpc-daemon[787]: ts=2026-09-20T12:21:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4300worker1 # [ 27.665655] nix-grpc-daemon[788]: ts=2026-09-20T12:21:36Z level=info event=scheduler_connected addr=lb:500514301worker1 # [ 27.668958] nix-grpc-daemon[788]: ts=2026-09-20T12:21:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4302worker2 # [ 27.867691] nix-grpc-daemon[787]: ts=2026-09-20T12:21:36Z level=info event=scheduler_connected addr=lb:500514303worker2 # [ 27.871582] nix-grpc-daemon[787]: ts=2026-09-20T12:21:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4304worker1 # [ 27.871618] nix-grpc-daemon[788]: ts=2026-09-20T12:21:36Z level=info event=scheduler_connected addr=lb:500514305worker1 # [ 27.874710] nix-grpc-daemon[788]: ts=2026-09-20T12:21:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4306worker2 # [ 28.072920] nix-grpc-daemon[787]: ts=2026-09-20T12:21:36Z level=info event=scheduler_connected addr=lb:500514307worker2 # [ 28.077222] nix-grpc-daemon[787]: ts=2026-09-20T12:21:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4308worker1 # [ 28.072462] nix-grpc-daemon[788]: ts=2026-09-20T12:21:36Z level=info event=scheduler_connected addr=lb:500514309worker1 # [ 28.076201] nix-grpc-daemon[788]: ts=2026-09-20T12:21:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4310worker2 # [ 28.278580] nix-grpc-daemon[787]: ts=2026-09-20T12:21:37Z level=info event=scheduler_connected addr=lb:500514311worker2 # [ 28.282701] nix-grpc-daemon[787]: ts=2026-09-20T12:21:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4312worker1 # [ 28.277157] nix-grpc-daemon[788]: ts=2026-09-20T12:21:37Z level=info event=scheduler_connected addr=lb:500514313worker1 # [ 28.281948] nix-grpc-daemon[788]: ts=2026-09-20T12:21:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4314worker2 # [ 28.484129] nix-grpc-daemon[787]: ts=2026-09-20T12:21:37Z level=info event=scheduler_connected addr=lb:500514315worker2 # [ 28.503335] nix-grpc-daemon[787]: ts=2026-09-20T12:21:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4316worker1 # [ 28.482576] nix-grpc-daemon[788]: ts=2026-09-20T12:21:37Z level=info event=scheduler_connected addr=lb:500514317worker1 # [ 28.486643] nix-grpc-daemon[788]: ts=2026-09-20T12:21:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4318worker2 # [ 28.706499] nix-grpc-daemon[787]: ts=2026-09-20T12:21:37Z level=info event=scheduler_connected addr=lb:500514319worker1 # [ 28.687873] nix-grpc-daemon[788]: ts=2026-09-20T12:21:37Z level=info event=scheduler_connected addr=lb:500514320worker2 # [ 28.710862] nix-grpc-daemon[787]: ts=2026-09-20T12:21:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4321worker1 # [ 28.692511] nix-grpc-daemon[788]: ts=2026-09-20T12:21:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4322worker2 # [ 28.912306] nix-grpc-daemon[787]: ts=2026-09-20T12:21:37Z level=info event=scheduler_connected addr=lb:500514323worker2 # [ 28.916433] nix-grpc-daemon[787]: ts=2026-09-20T12:21:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4324worker1 # [ 28.895274] nix-grpc-daemon[788]: ts=2026-09-20T12:21:37Z level=info event=scheduler_connected addr=lb:500514325worker1 # [ 28.898514] nix-grpc-daemon[788]: ts=2026-09-20T12:21:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4326worker2 # [ 29.117286] nix-grpc-daemon[787]: ts=2026-09-20T12:21:37Z level=info event=scheduler_connected addr=lb:500514327worker1 # [ 29.095619] nix-grpc-daemon[788]: ts=2026-09-20T12:21:37Z level=info event=scheduler_connected addr=lb:500514328worker2 # [ 29.119147] nix-grpc-daemon[787]: ts=2026-09-20T12:21:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4329worker1 # [ 29.097451] nix-grpc-daemon[788]: ts=2026-09-20T12:21:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4330worker1 # [ 29.296777] nix-grpc-daemon[788]: ts=2026-09-20T12:21:38Z level=info event=scheduler_connected addr=lb:500514331worker1 # [ 29.299311] nix-grpc-daemon[788]: ts=2026-09-20T12:21:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4332worker2 # [ 29.320799] nix-grpc-daemon[787]: ts=2026-09-20T12:21:38Z level=info event=scheduler_connected addr=lb:500514333worker2 # [ 29.325324] nix-grpc-daemon[787]: ts=2026-09-20T12:21:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4334worker1 # [ 29.500813] nix-grpc-daemon[788]: ts=2026-09-20T12:21:38Z level=info event=scheduler_connected addr=lb:500514335worker2 # [ 29.526216] nix-grpc-daemon[787]: ts=2026-09-20T12:21:38Z level=info event=scheduler_connected addr=lb:500514336worker1 # [ 29.505225] nix-grpc-daemon[788]: ts=2026-09-20T12:21:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4337worker2 # [ 29.530255] nix-grpc-daemon[787]: ts=2026-09-20T12:21:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4338worker1 # [ 29.706045] nix-grpc-daemon[788]: ts=2026-09-20T12:21:38Z level=info event=scheduler_connected addr=lb:500514339worker2 # [ 29.733302] nix-grpc-daemon[787]: ts=2026-09-20T12:21:38Z level=info event=scheduler_connected addr=lb:500514340worker1 # [ 29.709395] nix-grpc-daemon[788]: ts=2026-09-20T12:21:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4341worker2 # [ 29.735968] nix-grpc-daemon[787]: ts=2026-09-20T12:21:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4342worker2 # [ 29.933897] nix-grpc-daemon[787]: ts=2026-09-20T12:21:38Z level=info event=scheduler_connected addr=lb:500514343worker2 # [ 29.935919] nix-grpc-daemon[787]: ts=2026-09-20T12:21:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4344worker1 # [ 29.912285] nix-grpc-daemon[788]: ts=2026-09-20T12:21:38Z level=info event=scheduler_connected addr=lb:500514345worker1 # [ 29.915562] nix-grpc-daemon[788]: ts=2026-09-20T12:21:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4346worker2 # [ 30.137648] nix-grpc-daemon[787]: ts=2026-09-20T12:21:38Z level=info event=scheduler_connected addr=lb:500514347worker1 # [ 30.113083] nix-grpc-daemon[788]: ts=2026-09-20T12:21:39Z level=info event=scheduler_connected addr=lb:500514348worker2 # [ 30.139257] nix-grpc-daemon[787]: ts=2026-09-20T12:21:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4349worker1 # [ 30.117313] nix-grpc-daemon[788]: ts=2026-09-20T12:21:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4350lb # [ 30.283427] rustfs-bucket-start[1109]: mb s3://farm4351lb # [ 30.288310] systemd[1]: Finished rustfs-bucket.service.4352lb # [ 30.295527] systemd[1]: Starting niks3 server...4353worker2 # [ 30.339616] nix-grpc-daemon[787]: ts=2026-09-20T12:21:39Z level=info event=scheduler_connected addr=lb:500514354worker2 # [ 30.341934] nix-grpc-daemon[787]: ts=2026-09-20T12:21:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4355worker1 # [ 30.318238] nix-grpc-daemon[788]: ts=2026-09-20T12:21:39Z level=info event=scheduler_connected addr=lb:500514356worker1 # [ 30.322391] nix-grpc-daemon[788]: ts=2026-09-20T12:21:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4357worker2 # [ 30.543641] nix-grpc-daemon[787]: ts=2026-09-20T12:21:39Z level=info event=scheduler_connected addr=lb:500514358worker1 # [ 30.523180] nix-grpc-daemon[788]: ts=2026-09-20T12:21:39Z level=info event=scheduler_connected addr=lb:500514359worker2 # [ 30.547747] nix-grpc-daemon[787]: ts=2026-09-20T12:21:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4360lb # [ 30.536925] postgres[1124]: [1124] ERROR: relation "goose_db_version" does not exist at character 364361lb # [ 30.538479] postgres[1124]: [1124] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4362worker1 # [ 30.527235] nix-grpc-daemon[788]: ts=2026-09-20T12:21:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4363lb # [ 30.578367] niks3-server[1118]: 2026/09/20 12:21:39 OK 20241026095416_initial_model.sql (20.42ms)4364lb # [ 30.582373] niks3-server[1118]: 2026/09/20 12:21:39 OK 20251210153512_drop_unused_gin_index.sql (2.5ms)4365lb # [ 30.588245] niks3-server[1118]: 2026/09/20 12:21:39 OK 20251218171726_add_pins.sql (5.78ms)4366lb # [ 30.593314] niks3-server[1118]: 2026/09/20 12:21:39 OK 20260628120000_add_object_size_and_stats.sql (5.15ms)4367lb # [ 30.597888] niks3-server[1118]: 2026/09/20 12:21:39 OK 20260905000000_add_claims.sql (3.14ms)4368lb # [ 30.602945] niks3-server[1118]: 2026/09/20 12:21:39 OK 20260920000000_drop_claims.sql (4.96ms)4369lb # [ 30.604148] niks3-server[1118]: 2026/09/20 12:21:39 goose: successfully migrated database to version: 202609200000004370lb # [ 30.610288] niks3-server[1118]: 2026/09/20 12:21:39 OK 1_commit_pending_closure.sql (7.39ms)4371lb # [ 30.613083] niks3-server[1118]: 2026/09/20 12:21:39 OK 2_object_stats_trigger.sql (1.48ms)4372lb # [ 30.614604] niks3-server[1118]: 2026/09/20 12:21:39 goose: up to current file version: 24373lb # [ 30.622059] niks3-server[1118]: 2026/09/20 12:21:39 INFO Loaded signing key name=farm-test-1 path=/nix/store/c745bsy529madd2s6hz4z5is2436ni04-key4374lb # [ 30.694167] niks3-server[1118]: 2026/09/20 12:21:39 INFO Read proxy enabled — serving cache objects from S34375lb # [ 30.696513] niks3-server[1118]: 2026/09/20 12:21:39 INFO Using socket-activated listener address=0.0.0.0:57514376lb # [ 30.702639] niks3-server[1118]: 2026/09/20 12:21:39 INFO systemd watchdog enabled interval=15s4377lb # [ 30.705226] systemd[1]: Started niks3 server.4378lb # [ 30.706895] niks3-server[1118]: 2026/09/20 12:21:39 INFO Starting HTTP server address=0.0.0.0:57514379lb # [ 30.708889] systemd[1]: Reached target Multi-User System.4380lb # [ 30.710714] systemd[1]: Startup finished in 875ms (kernel) + 4.002s (initrd) + 25.832s (userspace) = 30.710s.4381lb # [ 30.717269] niks3-server[1118]: 2026/09/20 12:21:39 INFO lead: acquired remote=192.168.1.4:600584382worker2 # [ 30.736803] nix-grpc-daemon[787]: ts=2026-09-20T12:21:39Z level=info event=scheduler_take_over4383worker2 # [ 30.748858] nix-grpc-daemon[787]: ts=2026-09-20T12:21:39Z level=info event=scheduler_connected addr=lb:500514384worker1 # [ 30.728984] nix-grpc-daemon[788]: ts=2026-09-20T12:21:39Z level=info event=scheduler_connected addr=lb:500514385worker2 # [ 30.752340] nix-grpc-daemon[787]: ts=2026-09-20T12:21:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4386worker1 # [ 30.732277] nix-grpc-daemon[788]: ts=2026-09-20T12:21:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4387worker2 # [ 30.953887] nix-grpc-daemon[787]: ts=2026-09-20T12:21:39Z level=info event=scheduler_connected addr=lb:500514388worker1 # [ 30.931820] nix-grpc-daemon[788]: ts=2026-09-20T12:21:39Z level=info event=scheduler_connected addr=lb:500514389worker2 # [ 30.958210] nix-grpc-daemon[787]: ts=2026-09-20T12:21:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4390worker1 # [ 30.935454] nix-grpc-daemon[788]: ts=2026-09-20T12:21:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4391lb: (finished: waiting for unit niks3.service, in 19.86 seconds)4392lb: waiting for TCP port 5751 on localhost4393lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!4394lb: (finished: waiting for TCP port 5751 on localhost, in 0.07 seconds)4395lb: waiting for unit envoy.service4396lb: (finished: waiting for unit envoy.service, in 0.06 seconds)4397lb: waiting for TCP port 50051 on localhost4398worker1 # [ 31.132619] nix-grpc-daemon[788]: ts=2026-09-20T12:21:40Z level=info event=scheduler_connected addr=lb:500514399worker1 # [ 31.134553] nix-grpc-daemon[788]: ts=2026-09-20T12:21:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4400worker2 # [ 31.159364] nix-grpc-daemon[787]: ts=2026-09-20T12:21:39Z level=info event=scheduler_connected addr=lb:500514401lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!4402worker2 # [ 31.161562] nix-grpc-daemon[787]: ts=2026-09-20T12:21:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4403lb # [ 31.150756] envoy[979]: conn peer=[::1]:36526 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=44404lb: (finished: waiting for TCP port 50051 on localhost, in 0.04 seconds)4405worker2 # [ 31.246665] systemd[1]: Starting scheduler-token.service...4406worker2 # [ 31.310194] systemd[1]: scheduler-token.service: Deactivated successfully.4407worker2 # [ 31.311982] systemd[1]: Finished scheduler-token.service.4408worker2 # [ 31.313753] systemd[1]: scheduler-token.service: Consumed 21ms CPU time over 65ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4409lb: waiting until exactly no endpoints of x86_64-linux are unhealthy4410??? Warning (UserWarning): retry(): The 'timeout_seconds' argument is deprecated. Use 'timeout' instead.4411 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 1284412lb: must succeed: curl -sf localhost:9901/clusters4413worker1 # [ 31.335096] nix-grpc-daemon[788]: ts=2026-09-20T12:21:40Z level=info event=scheduler_connected addr=lb:500514414worker1 # [ 31.337066] nix-grpc-daemon[788]: ts=2026-09-20T12:21:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4415worker2 # [ 31.360945] nix-grpc-daemon[787]: ts=2026-09-20T12:21:40Z level=info event=scheduler_connected addr=lb:500514416worker2 # [ 31.363194] nix-grpc-daemon[787]: ts=2026-09-20T12:21:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4417lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4418lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.09 seconds)4419subtest: one node schedules and both builders hold a WorkerSession on it4420lb: must succeed: curl -sf localhost:9901/clusters4421lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4422worker1 # [ 31.538543] nix-grpc-daemon[788]: ts=2026-09-20T12:21:40Z level=info event=scheduler_connected addr=lb:500514423worker2 # [ 31.563706] nix-grpc-daemon[787]: ts=2026-09-20T12:21:40Z level=info event=scheduler_connected addr=lb:500514424worker2 # [ 31.569996] nix-grpc-daemon[787]: ts=2026-09-20T12:21:40Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:498504425worker2 # [ 31.572669] nix-grpc-daemon[787]: ts=2026-09-20T12:21:40Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04426worker2 # [ 31.587693] nix-grpc-daemon[787]: ts=2026-09-20T12:21:40Z level=info event=oidc_jwks provider=mock keys=14427worker2 # [ 31.590091] nix-grpc-daemon[787]: ts=2026-09-20T12:21:40Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:498504428worker2 # [ 31.592087] nix-grpc-daemon[787]: ts=2026-09-20T12:21:40Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04429lb: must succeed: curl -sf localhost:9901/clusters4430lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4431worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4432worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4433lb: must succeed: curl -sf localhost:9901/clusters4434lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4435lb: must succeed: curl -sf localhost:9901/clusters4436lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4437worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'4438worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.05 seconds)4439(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 1.32 seconds)4440subtest: balancer auth: client cert, bearer token, nothing, unknown CN4441worker2 # [ 33.050040] nix-grpc-daemon[787]: ts=2026-09-20T12:21:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:485224442worker2 # [ 33.056889] systemd[1]: Started Nix Daemon.4443worker2 # [ 33.269181] nix-daemon[857]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4444worker2 # [ 33.314660] nix-grpc-daemon[787]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 74 ms (attempt 1/5)4445worker2 # [ 33.391672] nix-grpc-daemon[787]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 1 ms (attempt 2/5)4446worker2 # [ 33.398581] nix-grpc-daemon[787]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 206 ms (attempt 3/5)4447worker2 # [ 33.610710] nix-grpc-daemon[787]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 36 ms (attempt 4/5)4448worker2 # [ 33.659474] nix-grpc-daemon[787]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4449worker2 # [ 33.687098] nix-grpc-daemon[787]: ts=2026-09-20T12:21:42Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:485224450worker2 # [ 33.706994] nix-grpc-daemon[787]: ts=2026-09-20T12:21:42Z level=debug event=path_info_miss path=10858v71rm0q02ppdw768ccq9np319pi-nixos-system-client-test error="error: path '/nix/store/10858v71rm0q02ppdw768ccq9np319pi-nixos-system-client-test' is required, but there is no substituter that can build it"4451worker2 # [ 33.716333] nix-grpc-daemon[787]: ts=2026-09-20T12:21:42Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:485224452worker2 # [ 33.724716] nix-grpc-daemon[787]: ts=2026-09-20T12:21:42Z level=debug event=path_info_miss path=10858v71rm0q02ppdw768ccq9np319pi-nixos-system-client-test error="error: path '/nix/store/10858v71rm0q02ppdw768ccq9np319pi-nixos-system-client-test' is required, but there is no substituter that can build it"4453client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4454client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.08 seconds)4455client: must succeed: install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4456client: (finished: must succeed: install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds)4457worker2 # [ 34.097791] nix-grpc-daemon[787]: ts=2026-09-20T12:21:42Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:485224458worker1 # [ 34.107942] nix-grpc-daemon[788]: ts=2026-09-20T12:21:43Z level=info event=oidc_jwks provider=mock keys=14459worker1 # [ 34.110196] nix-grpc-daemon[788]: ts=2026-09-20T12:21:43Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:514324460worker1 # [ 34.117585] systemd[1]: Started Nix Daemon.4461worker1 # [ 34.338758] nix-daemon[847]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4462worker1 # [ 34.376065] nix-grpc-daemon[788]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 100 ms (attempt 1/5)4463worker1 # [ 34.479165] nix-grpc-daemon[788]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 191 ms (attempt 2/5)4464worker1 # [ 34.676711] nix-grpc-daemon[788]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 339 ms (attempt 3/5)4465worker1 # [ 35.022474] nix-grpc-daemon[788]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 138 ms (attempt 4/5)4466worker1 # [ 35.168614] nix-grpc-daemon[788]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4467worker1 # [ 35.188817] nix-grpc-daemon[788]: ts=2026-09-20T12:21:44Z level=debug event=path_info_miss path=10858v71rm0q02ppdw768ccq9np319pi-nixos-system-client-test error="error: path '/nix/store/10858v71rm0q02ppdw768ccq9np319pi-nixos-system-client-test' is required, but there is no substituter that can build it"4468worker1 # [ 35.202513] nix-grpc-daemon[788]: ts=2026-09-20T12:21:44Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:514324469worker1 # [ 35.212943] nix-grpc-daemon[788]: ts=2026-09-20T12:21:44Z level=debug event=path_info_miss path=10858v71rm0q02ppdw768ccq9np319pi-nixos-system-client-test error="error: path '/nix/store/10858v71rm0q02ppdw768ccq9np319pi-nixos-system-client-test' is required, but there is no substituter that can build it"4470client: must succeed: rm -r /var/lib/nix-grpc-store4471client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.04 seconds)4472worker2 # [ 36.525699] nix-grpc-daemon[787]: ts=2026-09-20T12:21:45Z level=info event=denied method=QueryMissing cn=- role=none4473worker2 # [ 36.780593] nix-grpc-daemon[787]: ts=2026-09-20T12:21:45Z level=info event=denied method=QueryMissing cn=stranger role=none4474(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 4.08 seconds)4475subtest: DAG build is scheduled across workers and lands in the cache4476client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24477worker1 # [ 37.019953] nix-grpc-daemon[788]: ts=2026-09-20T12:21:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:514324478client # don't know how to build these paths:4479client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4480worker1 # [ 37.038413] nix-grpc-daemon[788]: ts=2026-09-20T12:21:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514324481worker2 # [ 37.100808] nix-grpc-daemon[787]: ts=2026-09-20T12:21:45Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04482worker2 # [ 37.102902] nix-grpc-daemon[787]: ts=2026-09-20T12:21:45Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.3:50051 clients=14483worker1 # [ 37.131800] nix-grpc-daemon[788]: ts=2026-09-20T12:21:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514324484worker1 # [ 37.147689] nix-grpc-daemon[788]: ts=2026-09-20T12:21:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514324485client # copying 1 paths to 'lb:50051'...4486worker1 # [ 37.156105] nix-grpc-daemon[788]: ts=2026-09-20T12:21:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:514324487client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4488worker1 # [ 37.183164] nix-daemon[847]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4489worker1 # [ 37.199442] nix-grpc-daemon[788]: ts=2026-09-20T12:21:46Z level=info event=niks3_push_started pid=8574490worker1 # [ 37.481877] nix-grpc-daemon[857]: time=2026-09-20T12:21:46.378Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""4491worker1 # [ 37.689114] nix-daemon[847]: accepted connection from pid 869, user nix-grpc-daemon (trusted)4492worker1 # [ 37.706143] nix-daemon[847]: reaped child process 872, status = succeeded4493worker1 # [ 37.883360] nix-daemon[847]: accepted connection from pid 875, user nix-grpc-daemon (trusted)4494worker1 # [ 37.898140] nix-daemon[847]: reaped child process 878, status = succeeded4495lb # [ 37.926904] niks3-server[1118]: 2026/09/20 12:21:46 INFO Received uploads request method=POST path=/api/pending_closures4496worker1 # [ 37.946966] nix-grpc-daemon[857]: time=2026-09-20T12:21:46.847Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4497worker1 # [ 37.949591] nix-grpc-daemon[857]: time=2026-09-20T12:21:46.850Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4498lb # [ 37.999559] niks3-server[1118]: 2026/09/20 12:21:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4499lb # [ 38.020676] niks3-server[1118]: 2026/09/20 12:21:46 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4500lb # [ 38.025425] niks3-server[1118]: 2026/09/20 12:21:46 INFO Signed narinfos id=1 count=14501lb # [ 38.027507] envoy[979]: conn peer=[2001:db8:1::1]:57016 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=715 tx=365 ms=7294502worker1 # [ 38.017891] nix-grpc-daemon[857]: time=2026-09-20T12:21:46.918Z level=INFO msg="Uploading 1 narinfos"4503lb # [ 38.030151] envoy[979]: conn peer=[2001:db8:1::1]:57028 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2613 tx=406 ms=11514504lb # [ 38.032102] envoy[979]: conn peer=[2001:db8:1::1]:57036 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=84505lb # [ 38.033790] envoy[979]: conn peer=[2001:db8:1::1]:57048 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=104506lb # [ 38.043363] niks3-server[1118]: 2026/09/20 12:21:46 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4507lb # [ 38.049187] niks3-server[1118]: 2026/09/20 12:21:46 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4508lb # [ 38.055355] niks3-server[1118]: 2026/09/20 12:21:46 INFO Completed upload id=14509lb # [ 38.056858] niks3-server[1118]: 2026/09/20 12:21:46 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo4510worker1 # [ 38.046270] nix-grpc-daemon[857]: time=2026-09-20T12:21:46.946Z level=INFO msg="Upload complete. (538ms)"4511worker1 # [ 38.053131] nix-grpc-daemon[788]: ts=2026-09-20T12:21:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51432 duration_s=0 paths=1 nar_bytes_in=5044512worker1 # [ 38.060360] nix-grpc-daemon[788]: ts=2026-09-20T12:21:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514324513worker1 # [ 38.070620] nix-daemon[847]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4514worker1 # [ 38.080887] nix-daemon[847]: reaped child process 855, status = succeeded4515client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4516client # farm-a> node-a: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4517worker1 # [ 38.090542] nix-daemon[847]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4518client # farm-a> LOG-farm-a-t14519worker1 # [ 39.119272] nix-daemon[847]: reaped child process 882, status = succeeded4520worker1 # [ 39.293931] nix-daemon[847]: accepted connection from pid 897, user nix-grpc-daemon (trusted)4521worker1 # [ 39.306716] nix-daemon[847]: reaped child process 901, status = succeeded4522lb # [ 39.326531] niks3-server[1118]: 2026/09/20 12:21:47 INFO Received uploads request method=POST path=/api/pending_closures4523worker1 # [ 39.325214] nix-grpc-daemon[857]: time=2026-09-20T12:21:48.225Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4524worker1 # [ 39.326908] nix-grpc-daemon[857]: time=2026-09-20T12:21:48.227Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4525lb # [ 39.372859] niks3-server[1118]: 2026/09/20 12:21:47 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4526lb # [ 39.378678] niks3-server[1118]: 2026/09/20 12:21:47 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4527lb # [ 39.386089] niks3-server[1118]: 2026/09/20 12:21:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4528lb # [ 39.392090] niks3-server[1118]: 2026/09/20 12:21:47 INFO Signed narinfos id=2 count=14529worker1 # [ 39.385555] nix-grpc-daemon[857]: time=2026-09-20T12:21:48.285Z level=INFO msg="Uploading 1 narinfos"4530lb # [ 39.396747] niks3-server[1118]: 2026/09/20 12:21:47 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4531lb # [ 39.405901] niks3-server[1118]: 2026/09/20 12:21:47 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4532lb # [ 39.409804] niks3-server[1118]: 2026/09/20 12:21:47 INFO Completed upload id=24533worker1 # [ 39.400402] nix-grpc-daemon[857]: time=2026-09-20T12:21:48.300Z level=INFO msg="Upload complete. (297ms)"4534lb # [ 39.411345] niks3-server[1118]: 2026/09/20 12:21:47 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4535worker1 # [ 39.406328] nix-grpc-daemon[788]: ts=2026-09-20T12:21:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51432 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14536worker1 # [ 39.416762] nix-daemon[847]: reaped child process 880, status = succeeded4537worker2 # [ 39.449669] nix-grpc-daemon[787]: ts=2026-09-20T12:21:48Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04538worker2 # [ 39.453979] nix-grpc-daemon[787]: ts=2026-09-20T12:21:48Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.3:50051 clients=14539worker1 # [ 39.476527] nix-grpc-daemon[788]: ts=2026-09-20T12:21:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514324540worker1 # [ 39.489265] nix-grpc-daemon[788]: ts=2026-09-20T12:21:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514324541worker1 # [ 39.503760] nix-grpc-daemon[857]: time=2026-09-20T12:21:48.404Z level=INFO msg="All 1 paths already cached"4542client # copying 1 paths to 'lb:50051'...4543worker1 # [ 39.511717] nix-grpc-daemon[788]: ts=2026-09-20T12:21:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:514324544client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4545worker1 # [ 39.535115] nix-daemon[847]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4546worker1 # [ 39.700862] nix-daemon[847]: accepted connection from pid 905, user nix-grpc-daemon (trusted)4547worker1 # [ 39.714807] nix-daemon[847]: reaped child process 908, status = succeeded4548worker1 # [ 39.865380] nix-daemon[847]: accepted connection from pid 910, user nix-grpc-daemon (trusted)4549worker1 # [ 39.877799] nix-daemon[847]: reaped child process 913, status = succeeded4550lb # [ 39.902739] niks3-server[1118]: 2026/09/20 12:21:48 INFO Received uploads request method=POST path=/api/pending_closures4551worker1 # [ 39.902677] nix-grpc-daemon[857]: time=2026-09-20T12:21:48.803Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4552worker1 # [ 39.904716] nix-grpc-daemon[857]: time=2026-09-20T12:21:48.804Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4553lb # [ 39.934722] niks3-server[1118]: 2026/09/20 12:21:48 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4554lb # [ 39.943231] niks3-server[1118]: 2026/09/20 12:21:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4555worker1 # [ 39.939084] nix-grpc-daemon[857]: time=2026-09-20T12:21:48.839Z level=INFO msg="Uploading 1 narinfos"4556lb # [ 39.948610] niks3-server[1118]: 2026/09/20 12:21:48 INFO Signed narinfos id=3 count=14557lb # [ 39.953590] niks3-server[1118]: 2026/09/20 12:21:48 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4558lb # [ 39.965758] niks3-server[1118]: 2026/09/20 12:21:48 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4559lb # [ 39.973148] niks3-server[1118]: 2026/09/20 12:21:48 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4560worker1 # [ 39.969145] nix-grpc-daemon[857]: time=2026-09-20T12:21:48.869Z level=INFO msg="Upload complete. (420ms)"4561lb # [ 39.978591] niks3-server[1118]: 2026/09/20 12:21:48 INFO Completed upload id=34562worker1 # [ 39.971739] nix-grpc-daemon[788]: ts=2026-09-20T12:21:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51432 duration_s=0 paths=1 nar_bytes_in=6244563worker1 # [ 39.977236] nix-grpc-daemon[788]: ts=2026-09-20T12:21:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514324564worker1 # [ 39.987106] nix-daemon[847]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4565worker1 # [ 39.994836] nix-daemon[847]: reaped child process 903, status = succeeded4566worker1 # [ 40.000306] nix-daemon[847]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4567client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4568client # farm-b> node-a: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4569client # farm-b> LOG-farm-b-t14570worker1 # [ 40.829504] nix-daemon[847]: reaped child process 917, status = succeeded4571worker1 # [ 40.969333] nix-daemon[847]: accepted connection from pid 932, user nix-grpc-daemon (trusted)4572worker1 # [ 40.983184] nix-daemon[847]: reaped child process 935, status = succeeded4573lb # [ 41.003290] niks3-server[1118]: 2026/09/20 12:21:49 INFO Received uploads request method=POST path=/api/pending_closures4574worker1 # [ 41.022236] nix-grpc-daemon[857]: time=2026-09-20T12:21:49.922Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4575worker1 # [ 41.024754] nix-grpc-daemon[857]: time=2026-09-20T12:21:49.924Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4576lb # [ 41.044824] niks3-server[1118]: 2026/09/20 12:21:49 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4577lb # [ 41.049853] niks3-server[1118]: 2026/09/20 12:21:49 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4578lb # [ 41.052058] niks3-server[1118]: 2026/09/20 12:21:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4579lb # [ 41.053853] niks3-server[1118]: 2026/09/20 12:21:49 INFO Signed narinfos id=4 count=14580worker1 # [ 41.045696] nix-grpc-daemon[857]: time=2026-09-20T12:21:49.946Z level=INFO msg="Uploading 1 narinfos"4581lb # [ 41.056786] niks3-server[1118]: 2026/09/20 12:21:49 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4582lb # [ 41.062337] niks3-server[1118]: 2026/09/20 12:21:49 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4583lb # [ 41.066272] niks3-server[1118]: 2026/09/20 12:21:49 INFO Completed upload id=44584worker1 # [ 41.057924] nix-grpc-daemon[857]: time=2026-09-20T12:21:49.958Z level=INFO msg="Upload complete. (249ms)"4585lb # [ 41.068521] niks3-server[1118]: 2026/09/20 12:21:49 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4586worker1 # [ 41.062373] nix-grpc-daemon[788]: ts=2026-09-20T12:21:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51432 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14587worker1 # [ 41.070497] nix-daemon[847]: reaped child process 915, status = succeeded4588worker2 # [ 41.092454] nix-grpc-daemon[787]: ts=2026-09-20T12:21:49Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04589worker2 # [ 41.096670] nix-grpc-daemon[787]: ts=2026-09-20T12:21:49Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.3:50051 clients=14590worker1 # [ 41.113908] nix-grpc-daemon[788]: ts=2026-09-20T12:21:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514324591worker1 # [ 41.124740] nix-grpc-daemon[788]: ts=2026-09-20T12:21:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514324592worker1 # [ 41.136424] nix-grpc-daemon[857]: time=2026-09-20T12:21:50.036Z level=INFO msg="All 2 paths already cached"4593client # copying 1 paths to 'lb:50051'...4594client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4595worker1 # [ 41.144256] nix-grpc-daemon[788]: ts=2026-09-20T12:21:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:514324596worker1 # [ 41.154158] nix-daemon[847]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4597worker1 # [ 41.325199] nix-daemon[847]: accepted connection from pid 939, user nix-grpc-daemon (trusted)4598worker1 # [ 41.341249] nix-daemon[847]: reaped child process 942, status = succeeded4599worker1 # [ 41.508488] nix-daemon[847]: accepted connection from pid 944, user nix-grpc-daemon (trusted)4600worker1 # [ 41.523550] nix-daemon[847]: reaped child process 947, status = succeeded4601lb # [ 41.547397] niks3-server[1118]: 2026/09/20 12:21:50 INFO Received uploads request method=POST path=/api/pending_closures4602worker1 # [ 41.542280] nix-grpc-daemon[857]: time=2026-09-20T12:21:50.442Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4603worker1 # [ 41.543911] nix-grpc-daemon[857]: time=2026-09-20T12:21:50.444Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4604lb # [ 41.565535] niks3-server[1118]: 2026/09/20 12:21:50 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4605lb # [ 41.567975] niks3-server[1118]: 2026/09/20 12:21:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4606lb # [ 41.569874] niks3-server[1118]: 2026/09/20 12:21:50 INFO Signed narinfos id=5 count=14607worker1 # [ 41.560512] nix-grpc-daemon[857]: time=2026-09-20T12:21:50.460Z level=INFO msg="Uploading 1 narinfos"4608lb # [ 41.573526] niks3-server[1118]: 2026/09/20 12:21:50 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4609lb # [ 41.578171] niks3-server[1118]: 2026/09/20 12:21:50 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4610lb # [ 41.582100] niks3-server[1118]: 2026/09/20 12:21:50 INFO Completed upload id=54611worker1 # [ 41.572452] nix-grpc-daemon[857]: time=2026-09-20T12:21:50.472Z level=INFO msg="Upload complete. (406ms)"4612lb # [ 41.583622] niks3-server[1118]: 2026/09/20 12:21:50 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4613worker1 # [ 41.574536] nix-grpc-daemon[788]: ts=2026-09-20T12:21:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51432 duration_s=0 paths=1 nar_bytes_in=7524614worker1 # [ 41.578750] nix-grpc-daemon[788]: ts=2026-09-20T12:21:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514324615worker1 # [ 41.585343] nix-daemon[847]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4616worker1 # [ 41.587754] nix-daemon[847]: reaped child process 937, status = succeeded4617worker1 # [ 41.593050] nix-daemon[847]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4618client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4619client # farm-top> node-a: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4620client # farm-top> LOG-farm-top-t14621worker1 # [ 43.215142] nix-daemon[847]: reaped child process 951, status = succeeded4622worker1 # [ 43.366964] nix-daemon[847]: accepted connection from pid 967, user nix-grpc-daemon (trusted)4623worker1 # [ 43.380374] nix-daemon[847]: reaped child process 970, status = succeeded4624lb # [ 43.404907] niks3-server[1118]: 2026/09/20 12:21:51 INFO Received uploads request method=POST path=/api/pending_closures4625worker1 # [ 43.405250] nix-grpc-daemon[857]: time=2026-09-20T12:21:52.305Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4626worker1 # [ 43.408333] nix-grpc-daemon[857]: time=2026-09-20T12:21:52.307Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4627lb # [ 43.446372] niks3-server[1118]: 2026/09/20 12:21:52 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4628lb # [ 43.454686] niks3-server[1118]: 2026/09/20 12:21:52 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4629lb # [ 43.467181] niks3-server[1118]: 2026/09/20 12:21:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4630lb # [ 43.471176] niks3-server[1118]: 2026/09/20 12:21:52 INFO Signed narinfos id=6 count=14631worker1 # [ 43.464609] nix-grpc-daemon[857]: time=2026-09-20T12:21:52.364Z level=INFO msg="Uploading 1 narinfos"4632lb # [ 43.478775] niks3-server[1118]: 2026/09/20 12:21:52 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4633lb # [ 43.482062] niks3-server[1118]: 2026/09/20 12:21:52 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4634lb # [ 43.486324] niks3-server[1118]: 2026/09/20 12:21:52 INFO Completed upload id=64635lb # [ 43.487727] niks3-server[1118]: 2026/09/20 12:21:52 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4636worker1 # [ 43.476980] nix-grpc-daemon[857]: time=2026-09-20T12:21:52.377Z level=INFO msg="Upload complete. (275ms)"4637worker1 # [ 43.482709] nix-grpc-daemon[788]: ts=2026-09-20T12:21:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51432 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14638worker1 # [ 43.495840] nix-daemon[847]: reaped child process 949, status = succeeded4639client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 6.74 seconds)4640client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4641client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.16 seconds)4642client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4643client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.01 seconds)4644client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4645client # copying 3 paths...4646client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4647client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4648client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4649client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.24 seconds)4650(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 7.23 seconds)4651subtest: repeat is answered Cached by the scheduler without building4652worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4653worker1 # finding garbage collector roots...4654worker1 # deleting specified paths...4655worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4656worker1 # deleting unused links...4657worker1 # note: hard linking is currently saving 0.0 KiB4658worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.25 seconds)4659worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4660worker2 # finding garbage collector roots...4661worker2 # deleting specified paths...4662worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4663worker2 # deleting unused links...4664worker2 # note: hard linking is currently saving 0.0 KiB4665worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.25 seconds)4666client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24667worker2 # [ 44.755825] nix-grpc-daemon[787]: ts=2026-09-20T12:21:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:485224668client # don't know how to build these paths:4669client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4670worker2 # [ 44.764623] nix-grpc-daemon[787]: ts=2026-09-20T12:21:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:485224671worker2 # [ 44.791639] nix-grpc-daemon[787]: ts=2026-09-20T12:21:53Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14672client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 0.28 seconds)4673client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4674client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.20 seconds)4675lb: must succeed: curl -sf localhost:9901/clusters4676lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4677lb: must succeed: curl -sf localhost:9901/clusters4678lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4679worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4680worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4681(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.25 seconds)4682subtest: inputs already in the cache are not sent to the scheduler4683lb: must succeed: curl -sf localhost:9901/clusters4684lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4685lb: must succeed: curl -sf localhost:9901/clusters4686lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4687worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4688worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.03 seconds)4689client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&24690worker2 # [ 45.583244] nix-grpc-daemon[787]: ts=2026-09-20T12:21:54Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:485224691client # don't know how to build these paths:4692client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4693worker2 # [ 45.591765] nix-grpc-daemon[787]: ts=2026-09-20T12:21:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:485224694worker2 # [ 45.604489] nix-grpc-daemon[787]: ts=2026-09-20T12:21:54Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04695worker2 # [ 45.608914] nix-grpc-daemon[787]: ts=2026-09-20T12:21:54Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.3:50051 clients=14696worker1 # [ 45.626479] nix-grpc-daemon[788]: ts=2026-09-20T12:21:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514324697worker1 # [ 45.639873] nix-grpc-daemon[788]: ts=2026-09-20T12:21:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514324698worker1 # [ 45.652765] nix-grpc-daemon[857]: time=2026-09-20T12:21:54.552Z level=INFO msg="All 2 paths already cached"4699client # copying 1 paths to 'lb:50051'...4700client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4701worker1 # [ 45.660814] nix-grpc-daemon[788]: ts=2026-09-20T12:21:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:514324702worker1 # [ 45.672791] nix-daemon[847]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4703worker1 # [ 45.859230] nix-daemon[847]: accepted connection from pid 987, user nix-grpc-daemon (trusted)4704worker1 # [ 45.874177] nix-daemon[847]: reaped child process 990, status = succeeded4705worker1 # [ 46.042874] nix-daemon[847]: accepted connection from pid 992, user nix-grpc-daemon (trusted)4706worker1 # [ 46.056598] nix-daemon[847]: reaped child process 995, status = succeeded4707lb # [ 46.081603] niks3-server[1118]: 2026/09/20 12:21:54 INFO Received uploads request method=POST path=/api/pending_closures4708worker1 # [ 46.086062] nix-grpc-daemon[857]: time=2026-09-20T12:21:54.986Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4709worker1 # [ 46.087463] nix-grpc-daemon[857]: time=2026-09-20T12:21:54.986Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4710lb # [ 46.117249] niks3-server[1118]: 2026/09/20 12:21:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4711lb # [ 46.122749] niks3-server[1118]: 2026/09/20 12:21:54 INFO Signed narinfos id=7 count=14712worker1 # [ 46.116373] nix-grpc-daemon[857]: time=2026-09-20T12:21:55.016Z level=INFO msg="Uploading 1 narinfos"4713lb # [ 46.127650] niks3-server[1118]: 2026/09/20 12:21:54 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4714lb # [ 46.141448] niks3-server[1118]: 2026/09/20 12:21:54 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4715lb # [ 46.145585] niks3-server[1118]: 2026/09/20 12:21:54 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4716lb # [ 46.152787] niks3-server[1118]: 2026/09/20 12:21:54 INFO Completed upload id=74717worker1 # [ 46.143286] nix-grpc-daemon[857]: time=2026-09-20T12:21:55.043Z level=INFO msg="Upload complete. (458ms)"4718worker1 # [ 46.147092] nix-grpc-daemon[788]: ts=2026-09-20T12:21:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51432 duration_s=0 paths=1 nar_bytes_in=7524719lb # [ 46.159583] niks3-server[1118]: 2026/09/20 12:21:54 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4720worker1 # [ 46.153343] nix-grpc-daemon[788]: ts=2026-09-20T12:21:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514324721worker1 # [ 46.161247] nix-daemon[847]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4722worker1 # [ 46.169266] nix-daemon[847]: reaped child process 985, status = succeeded4723worker1 # [ 46.174777] nix-daemon[847]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4724client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4725client # farm-top> node-a: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4726client # farm-top> LOG-farm-top-t1b4727worker1 # [ 47.203837] nix-daemon[847]: reaped child process 999, status = succeeded4728worker1 # [ 47.339627] nix-daemon[847]: accepted connection from pid 1014, user nix-grpc-daemon (trusted)4729worker1 # [ 47.353646] nix-daemon[847]: reaped child process 1017, status = succeeded4730lb # [ 47.380551] niks3-server[1118]: 2026/09/20 12:21:55 INFO Received uploads request method=POST path=/api/pending_closures4731worker1 # [ 47.381759] nix-grpc-daemon[857]: time=2026-09-20T12:21:56.281Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4732worker1 # [ 47.385149] nix-grpc-daemon[857]: time=2026-09-20T12:21:56.283Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4733lb # [ 47.426894] niks3-server[1118]: 2026/09/20 12:21:55 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4734lb # [ 47.433454] niks3-server[1118]: 2026/09/20 12:21:56 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4735lb # [ 47.440309] niks3-server[1118]: 2026/09/20 12:21:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4736worker1 # [ 47.436450] nix-grpc-daemon[857]: time=2026-09-20T12:21:56.336Z level=INFO msg="Uploading 1 narinfos"4737lb # [ 47.447234] niks3-server[1118]: 2026/09/20 12:21:56 INFO Signed narinfos id=8 count=14738lb # [ 47.452637] niks3-server[1118]: 2026/09/20 12:21:56 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4739lb # [ 47.461150] niks3-server[1118]: 2026/09/20 12:21:56 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4740lb # [ 47.468087] niks3-server[1118]: 2026/09/20 12:21:56 INFO Completed upload id=84741lb # [ 47.469591] niks3-server[1118]: 2026/09/20 12:21:56 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4742worker1 # [ 47.459142] nix-grpc-daemon[857]: time=2026-09-20T12:21:56.359Z level=INFO msg="Upload complete. (276ms)"4743worker1 # [ 47.464837] nix-grpc-daemon[788]: ts=2026-09-20T12:21:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51432 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14744worker1 # [ 47.479241] nix-daemon[847]: reaped child process 997, status = succeeded4745client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.11 seconds)4746client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath4747client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.20 seconds)4748lb: must succeed: curl -sf localhost:9901/clusters4749lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4750lb: must succeed: curl -sf localhost:9901/clusters4751lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4752worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4753worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4754(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.59 seconds)4755subtest: two clients wanting the same drv build it once4756client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4757client # [ 47.890894] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4758client # Running as unit: dedup1.service; invocation ID: 28445bbc44f24cd5a77a418ebf2405314759client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 0.04 seconds)4760client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4761lb # [ 48.027458] envoy[979]: conn peer=[2001:db8:1::1]:57060 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9511 tx=3855 ms=65124762lb # [ 48.031343] envoy[979]: conn peer=[2001:db8:1::1]:38268 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=484763lb # [ 48.034797] envoy[979]: conn peer=[2001:db8:1::1]:38280 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4139 tx=1682 ms=19144764worker2 # [ 48.094899] nix-grpc-daemon[787]: ts=2026-09-20T12:21:56Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:485224765client # [ 48.070601] nix[893]: don't know how to build these paths:4766client # [ 48.072126] nix[893]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4767worker2 # [ 48.109235] nix-grpc-daemon[787]: ts=2026-09-20T12:21:56Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04768worker2 # [ 48.113332] nix-grpc-daemon[787]: ts=2026-09-20T12:21:56Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.4:50051 clients=14769worker2 # [ 48.120926] nix-grpc-daemon[787]: ts=2026-09-20T12:21:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:485224770worker2 # [ 48.136927] nix-grpc-daemon[787]: ts=2026-09-20T12:21:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:485224771client # [ 48.116925] nix[893]: copying 1 paths to 'lb:50051'...4772client # [ 48.117757] nix[893]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4773worker2 # [ 48.147583] nix-grpc-daemon[787]: ts=2026-09-20T12:21:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:485224774worker2 # [ 48.156870] nix-daemon[857]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4775worker2 # [ 48.180718] nix-grpc-daemon[787]: ts=2026-09-20T12:21:56Z level=info event=niks3_push_started pid=9074776worker2 # [ 48.437894] nix-grpc-daemon[907]: time=2026-09-20T12:21:57.229Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""4777worker2 # [ 48.618822] nix-daemon[857]: accepted connection from pid 920, user nix-grpc-daemon (trusted)4778worker2 # [ 48.639796] nix-daemon[857]: reaped child process 923, status = succeeded4779worker2 # [ 48.788069] nix-daemon[857]: accepted connection from pid 925, user nix-grpc-daemon (trusted)4780worker2 # [ 48.800623] nix-daemon[857]: reaped child process 928, status = succeeded4781lb # [ 48.796903] niks3-server[1118]: 2026/09/20 12:21:57 INFO Received uploads request method=POST path=/api/pending_closures4782worker2 # [ 48.820428] nix-grpc-daemon[907]: time=2026-09-20T12:21:57.616Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4783worker2 # [ 48.822322] nix-grpc-daemon[907]: time=2026-09-20T12:21:57.617Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4784lb # [ 48.832574] niks3-server[1118]: 2026/09/20 12:21:57 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4785lb # [ 48.843803] niks3-server[1118]: 2026/09/20 12:21:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4786lb # [ 48.848095] niks3-server[1118]: 2026/09/20 12:21:57 INFO Signed narinfos id=9 count=14787worker2 # [ 48.864365] nix-grpc-daemon[907]: time=2026-09-20T12:21:57.659Z level=INFO msg="Uploading 1 narinfos"4788lb # [ 48.856581] niks3-server[1118]: 2026/09/20 12:21:57 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4789lb # [ 48.864390] niks3-server[1118]: 2026/09/20 12:21:57 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4790lb # [ 48.868360] niks3-server[1118]: 2026/09/20 12:21:57 INFO Completed upload id=94791worker2 # [ 48.882087] nix-grpc-daemon[907]: time=2026-09-20T12:21:57.677Z level=INFO msg="Upload complete. (426ms)"4792lb # [ 48.871560] niks3-server[1118]: 2026/09/20 12:21:57 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4793worker2 # [ 48.884856] nix-grpc-daemon[787]: ts=2026-09-20T12:21:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48522 duration_s=0 paths=1 nar_bytes_in=4324794worker2 # [ 48.888135] nix-grpc-daemon[787]: ts=2026-09-20T12:21:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:485224795worker2 # [ 48.894860] nix-daemon[857]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4796worker2 # [ 48.897177] nix-daemon[857]: reaped child process 905, status = succeeded4797client # [ 48.871549] nix[893]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4798worker2 # [ 48.903092] nix-daemon[857]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4799client # [ 48.958109] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4800client # Running as unit: dedup2.service; invocation ID: 8955257e4a6f47ed838cbb17beb891a24801client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 1.08 seconds)4802worker2 # [ 49.231240] nix-grpc-daemon[787]: ts=2026-09-20T12:21:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:485224803client # [ 49.214321] nix[908]: this derivation will be built:4804worker1 # [ 49.223569] nix-grpc-daemon[788]: ts=2026-09-20T12:21:58Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:514324805worker1 # [ 49.236113] nix-grpc-daemon[788]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' from 'http://lb:5751'...4806client # [ 49.249825] nix[908]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4807worker2 # [ 49.288818] nix-grpc-daemon[787]: ts=2026-09-20T12:21:58Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04808worker2 # [ 49.295946] nix-grpc-daemon[787]: ts=2026-09-20T12:21:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:485224809client # [ 49.271225] nix[908]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4810client: must succeed: sleep 24811client: (finished: must succeed: sleep 2, in 2.03 seconds)4812??? Warning (UserWarning): wait_until_succeeds(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4813 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394814client: waiting for success: ! systemctl is-active dedup1 dedup24815??? Warning (UserWarning): execute(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4816 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394817worker2 # [ 53.519130] nix-daemon[857]: reaped child process 932, status = succeeded4818worker2 # [ 53.668557] nix-daemon[857]: accepted connection from pid 967, user nix-grpc-daemon (trusted)4819worker2 # [ 53.680794] nix-daemon[857]: reaped child process 970, status = succeeded4820lb # [ 53.682846] niks3-server[1118]: 2026/09/20 12:22:02 INFO Received uploads request method=POST path=/api/pending_closures4821worker2 # [ 53.705083] nix-grpc-daemon[907]: time=2026-09-20T12:22:02.500Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4822worker2 # [ 53.706552] nix-grpc-daemon[907]: time=2026-09-20T12:22:02.502Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4823lb # [ 53.729940] niks3-server[1118]: 2026/09/20 12:22:02 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4824lb # [ 53.737532] niks3-server[1118]: 2026/09/20 12:22:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4825lb # [ 53.744610] niks3-server[1118]: 2026/09/20 12:22:02 INFO Signed narinfos id=10 count=14826worker2 # [ 53.761880] nix-grpc-daemon[907]: time=2026-09-20T12:22:02.556Z level=INFO msg="Uploading 1 narinfos"4827lb # [ 53.750135] niks3-server[1118]: 2026/09/20 12:22:02 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4828lb # [ 53.765715] niks3-server[1118]: 2026/09/20 12:22:02 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4829lb # [ 53.770823] niks3-server[1118]: 2026/09/20 12:22:02 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4830lb # [ 53.773046] niks3-server[1118]: 2026/09/20 12:22:02 INFO Completed upload id=104831worker2 # [ 53.786820] nix-grpc-daemon[907]: time=2026-09-20T12:22:02.582Z level=INFO msg="Upload complete. (271ms)"4832lb # [ 53.775056] niks3-server[1118]: 2026/09/20 12:22:02 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4833worker2 # [ 53.790926] nix-grpc-daemon[787]: ts=2026-09-20T12:22:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48522 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14834worker2 # [ 53.795798] nix-grpc-daemon[787]: ts=2026-09-20T12:22:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48522 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14835worker2 # [ 53.805350] nix-daemon[857]: reaped child process 930, status = succeeded4836client # [ 53.791694] systemd[1]: dedup1.service: Deactivated successfully.4837client # [ 53.793603] systemd[1]: dedup1.service: Consumed 99ms CPU time over 5.902s wall clock time, 37.3M memory peak, 8.7K incoming IP traffic, 9.1K outgoing IP traffic.4838client # [ 53.831794] systemd[1]: dedup2.service: Deactivated successfully.4839client # [ 53.832942] systemd[1]: dedup2.service: Consumed 129ms CPU time over 4.874s wall clock time, 33.2M memory peak, 7.8K incoming IP traffic, 6.6K outgoing IP traffic.4840client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.08 seconds)4841client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4842client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)4843worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4844worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.03 seconds)4845worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4846worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.03 seconds)4847worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4848worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4849worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4850worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4851(finished: subtest: two clients wanting the same drv build it once, in 6.87 seconds)4852subtest: build hook: nix-daemon with builders = grpc://lb4853client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log4854client # [ 54.895860] systemd[1]: Started Nix Daemon.4855client # [ 55.055184] nix-daemon[949]: accepted connection from pid 946, user root (trusted)4856worker1 # [ 55.238410] nix-grpc-daemon[788]: ts=2026-09-20T12:22:04Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:514324857worker1 # [ 55.243439] nix-grpc-daemon[788]: ts=2026-09-20T12:22:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514324858worker2 # [ 55.281308] nix-grpc-daemon[787]: ts=2026-09-20T12:22:04Z level=debug event=want client=6 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=04859worker2 # [ 55.284546] nix-grpc-daemon[787]: ts=2026-09-20T12:22:04Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.3:50051 clients=14860worker1 # [ 55.300124] nix-grpc-daemon[788]: ts=2026-09-20T12:22:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514324861worker1 # [ 55.307261] nix-grpc-daemon[788]: ts=2026-09-20T12:22:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514324862worker1 # [ 55.313487] nix-grpc-daemon[788]: ts=2026-09-20T12:22:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:514324863worker1 # [ 55.319895] nix-daemon[847]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4864worker1 # [ 55.486307] nix-daemon[847]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)4865worker1 # [ 55.500690] nix-daemon[847]: reaped child process 1050, status = succeeded4866worker1 # [ 55.639645] nix-daemon[847]: accepted connection from pid 1052, user nix-grpc-daemon (trusted)4867worker1 # [ 55.652145] nix-daemon[847]: reaped child process 1055, status = succeeded4868lb # [ 55.675669] niks3-server[1118]: 2026/09/20 12:22:04 INFO Received uploads request method=POST path=/api/pending_closures4869worker1 # [ 55.671278] nix-grpc-daemon[857]: time=2026-09-20T12:22:04.571Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4870worker1 # [ 55.672796] nix-grpc-daemon[857]: time=2026-09-20T12:22:04.573Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"4871lb # [ 55.693106] niks3-server[1118]: 2026/09/20 12:22:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4872lb # [ 55.694880] niks3-server[1118]: 2026/09/20 12:22:04 INFO Signed narinfos id=11 count=14873worker1 # [ 55.685748] nix-grpc-daemon[857]: time=2026-09-20T12:22:04.586Z level=INFO msg="Uploading 1 narinfos"4874lb # [ 55.698731] niks3-server[1118]: 2026/09/20 12:22:04 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst4875lb # [ 55.702761] niks3-server[1118]: 2026/09/20 12:22:04 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls4876lb # [ 55.705543] niks3-server[1118]: 2026/09/20 12:22:04 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4877lb # [ 55.709054] niks3-server[1118]: 2026/09/20 12:22:04 INFO Completed upload id=114878worker1 # [ 55.699570] nix-grpc-daemon[857]: time=2026-09-20T12:22:04.600Z level=INFO msg="Upload complete. (373ms)"4879worker1 # [ 55.701581] nix-grpc-daemon[788]: ts=2026-09-20T12:22:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51432 duration_s=0 paths=1 nar_bytes_in=5124880lb # [ 55.711723] niks3-server[1118]: 2026/09/20 12:22:04 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo4881worker1 # [ 55.704570] nix-grpc-daemon[788]: ts=2026-09-20T12:22:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514324882worker1 # [ 55.711200] nix-daemon[847]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4883worker1 # [ 55.714206] nix-daemon[847]: reaped child process 1045, status = succeeded4884worker1 # [ 55.718928] nix-daemon[847]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4885worker1 # [ 57.133189] nix-daemon[847]: reaped child process 1059, status = succeeded4886worker1 # [ 57.281515] nix-daemon[847]: accepted connection from pid 1074, user nix-grpc-daemon (trusted)4887worker1 # [ 57.297341] nix-daemon[847]: reaped child process 1077, status = succeeded4888lb # [ 57.321330] niks3-server[1118]: 2026/09/20 12:22:05 INFO Received uploads request method=POST path=/api/pending_closures4889worker1 # [ 57.317306] nix-grpc-daemon[857]: time=2026-09-20T12:22:06.217Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4890worker1 # [ 57.320058] nix-grpc-daemon[857]: time=2026-09-20T12:22:06.219Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"4891lb # [ 57.354470] niks3-server[1118]: 2026/09/20 12:22:05 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4892lb # [ 57.369497] niks3-server[1118]: 2026/09/20 12:22:05 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv4893lb # [ 57.376541] niks3-server[1118]: 2026/09/20 12:22:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4894lb # [ 57.380692] niks3-server[1118]: 2026/09/20 12:22:05 INFO Signed narinfos id=12 count=14895worker1 # [ 57.374080] nix-grpc-daemon[857]: time=2026-09-20T12:22:06.274Z level=INFO msg="Uploading 1 narinfos"4896lb # [ 57.391641] niks3-server[1118]: 2026/09/20 12:22:05 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls4897lb # [ 57.397653] niks3-server[1118]: 2026/09/20 12:22:05 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4898lb # [ 57.402932] niks3-server[1118]: 2026/09/20 12:22:05 INFO Completed upload id=124899lb # [ 57.405516] niks3-server[1118]: 2026/09/20 12:22:05 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo4900worker1 # [ 57.395072] nix-grpc-daemon[857]: time=2026-09-20T12:22:06.295Z level=INFO msg="Upload complete. (269ms)"4901worker2 # [ 57.427472] nix-grpc-daemon[787]: ts=2026-09-20T12:22:06Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:485224902worker1 # [ 57.401553] nix-grpc-daemon[788]: ts=2026-09-20T12:22:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51432 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=6 outputs=14903worker2 # [ 57.434305] nix-grpc-daemon[787]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...4904worker1 # [ 57.413212] nix-daemon[847]: reaped child process 1057, status = succeeded4905worker1 # [ 57.435282] nix-grpc-daemon[788]: ts=2026-09-20T12:22:06Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:514324906worker1 # [ 57.441520] nix-grpc-daemon[788]: ts=2026-09-20T12:22:06Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:51432 duration_s=0 paths=1 nar_bytes_out=1284907worker2 # [ 57.723151] nix-grpc-daemon[787]: ts=2026-09-20T12:22:06Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:485224908worker1 # [ 57.707940] nix-grpc-daemon[788]: ts=2026-09-20T12:22:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514324909worker1 # [ 57.718528] nix-grpc-daemon[857]: time=2026-09-20T12:22:06.618Z level=INFO msg="All 1 paths already cached"4910worker2 # [ 57.755735] nix-grpc-daemon[787]: ts=2026-09-20T12:22:06Z level=debug event=want client=7 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=04911worker2 # [ 57.760216] nix-grpc-daemon[787]: ts=2026-09-20T12:22:06Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.3:50051 clients=14912worker1 # [ 57.738924] nix-grpc-daemon[788]: ts=2026-09-20T12:22:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514324913worker1 # [ 57.751144] nix-grpc-daemon[788]: ts=2026-09-20T12:22:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514324914worker1 # [ 57.763725] nix-grpc-daemon[857]: time=2026-09-20T12:22:06.664Z level=INFO msg="All 1 paths already cached"4915worker1 # [ 57.770354] nix-grpc-daemon[788]: ts=2026-09-20T12:22:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:514324916worker1 # [ 57.780377] nix-daemon[847]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4917worker1 # [ 57.952374] nix-daemon[847]: accepted connection from pid 1081, user nix-grpc-daemon (trusted)4918worker1 # [ 57.968775] nix-daemon[847]: reaped child process 1084, status = succeeded4919lb # [ 58.028855] envoy[979]: conn peer=[2001:db8:1::1]:38290 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2972 tx=1372 ms=57094920lb # [ 58.030468] envoy[979]: conn peer=[2001:db8:1::1]:38294 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1576 tx=1487 ms=46214921lb # [ 58.031909] envoy[979]: conn peer=[2001:db8:1::1]:59670 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3402 tx=1770 ms=22594922lb # [ 58.035126] envoy[979]: conn peer=[2001:db8:1::1]:59678 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=594923worker1 # [ 58.135854] nix-daemon[847]: accepted connection from pid 1086, user nix-grpc-daemon (trusted)4924worker1 # [ 58.148381] nix-daemon[847]: reaped child process 1089, status = succeeded4925lb # [ 58.172795] niks3-server[1118]: 2026/09/20 12:22:06 INFO Received uploads request method=POST path=/api/pending_closures4926worker1 # [ 58.170467] nix-grpc-daemon[857]: time=2026-09-20T12:22:07.070Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4927worker1 # [ 58.172058] nix-grpc-daemon[857]: time=2026-09-20T12:22:07.072Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"4928lb # [ 58.199772] niks3-server[1118]: 2026/09/20 12:22:06 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst4929lb # [ 58.212868] niks3-server[1118]: 2026/09/20 12:22:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4930lb # [ 58.218743] niks3-server[1118]: 2026/09/20 12:22:06 INFO Signed narinfos id=13 count=14931worker1 # [ 58.212091] nix-grpc-daemon[857]: time=2026-09-20T12:22:07.112Z level=INFO msg="Uploading 1 narinfos"4932lb # [ 58.226517] niks3-server[1118]: 2026/09/20 12:22:06 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls4933lb # [ 58.235638] niks3-server[1118]: 2026/09/20 12:22:06 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4934worker1 # [ 58.232197] nix-grpc-daemon[857]: time=2026-09-20T12:22:07.132Z level=INFO msg="Upload complete. (437ms)"4935lb # [ 58.241633] niks3-server[1118]: 2026/09/20 12:22:06 INFO Completed upload id=134936lb # [ 58.243859] niks3-server[1118]: 2026/09/20 12:22:06 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo4937worker1 # [ 58.234675] nix-grpc-daemon[788]: ts=2026-09-20T12:22:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51432 duration_s=0 paths=1 nar_bytes_in=6244938worker1 # [ 58.240332] nix-grpc-daemon[788]: ts=2026-09-20T12:22:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514324939worker1 # [ 58.250069] nix-daemon[847]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4940worker1 # [ 58.256224] nix-daemon[847]: reaped child process 1079, status = succeeded4941worker1 # [ 58.260993] nix-daemon[847]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4942worker1 # [ 59.075454] nix-daemon[847]: reaped child process 1093, status = succeeded4943worker1 # [ 59.198666] nix-daemon[847]: accepted connection from pid 1108, user nix-grpc-daemon (trusted)4944worker1 # [ 59.212645] nix-daemon[847]: reaped child process 1111, status = succeeded4945lb # [ 59.234603] niks3-server[1118]: 2026/09/20 12:22:07 INFO Received uploads request method=POST path=/api/pending_closures4946worker1 # [ 59.235571] nix-grpc-daemon[857]: time=2026-09-20T12:22:08.135Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4947worker1 # [ 59.238278] nix-grpc-daemon[857]: time=2026-09-20T12:22:08.137Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"4948lb # [ 59.274396] niks3-server[1118]: 2026/09/20 12:22:07 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv4949lb # [ 59.279479] niks3-server[1118]: 2026/09/20 12:22:07 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst4950lb # [ 59.291252] niks3-server[1118]: 2026/09/20 12:22:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4951worker1 # [ 59.286806] nix-grpc-daemon[857]: time=2026-09-20T12:22:08.187Z level=INFO msg="Uploading 1 narinfos"4952lb # [ 59.296703] niks3-server[1118]: 2026/09/20 12:22:07 INFO Signed narinfos id=14 count=14953lb # [ 59.298907] niks3-server[1118]: 2026/09/20 12:22:07 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls4954lb # [ 59.306245] niks3-server[1118]: 2026/09/20 12:22:07 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4955lb # [ 59.309694] niks3-server[1118]: 2026/09/20 12:22:07 INFO Completed upload id=144956worker1 # [ 59.300330] nix-grpc-daemon[857]: time=2026-09-20T12:22:08.200Z level=INFO msg="Upload complete. (232ms)"4957lb # [ 59.312715] niks3-server[1118]: 2026/09/20 12:22:07 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo4958worker1 # [ 59.304829] nix-grpc-daemon[788]: ts=2026-09-20T12:22:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51432 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=7 outputs=14959worker2 # [ 59.332598] nix-grpc-daemon[787]: ts=2026-09-20T12:22:08Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:485224960worker2 # [ 59.340159] nix-grpc-daemon[787]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...4961worker1 # [ 59.317652] nix-daemon[847]: reaped child process 1091, status = succeeded4962worker2 # [ 59.357576] nix-grpc-daemon[787]: ts=2026-09-20T12:22:08Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:485224963worker2 # [ 59.362873] nix-grpc-daemon[787]: ts=2026-09-20T12:22:08Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48522 duration_s=0 paths=1 nar_bytes_out=1764964worker1 # [ 59.574179] nix-grpc-daemon[788]: ts=2026-09-20T12:22:08Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:514324965worker1 # [ 59.581219] nix-grpc-daemon[788]: ts=2026-09-20T12:22:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514324966worker1 # [ 59.607490] nix-grpc-daemon[857]: time=2026-09-20T12:22:08.507Z level=INFO msg="All 2 paths already cached"4967worker2 # [ 59.649629] nix-grpc-daemon[787]: ts=2026-09-20T12:22:08Z level=debug event=want client=8 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=04968worker1 # [ 59.628967] nix-grpc-daemon[788]: ts=2026-09-20T12:22:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514324969worker2 # [ 59.653926] nix-grpc-daemon[787]: ts=2026-09-20T12:22:08Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.3:50051 clients=14970worker1 # [ 59.641618] nix-grpc-daemon[788]: ts=2026-09-20T12:22:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514324971worker1 # [ 59.874337] nix-grpc-daemon[857]: time=2026-09-20T12:22:08.774Z level=INFO msg="All 2 paths already cached"4972worker1 # [ 59.883969] nix-grpc-daemon[788]: ts=2026-09-20T12:22:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:514324973worker1 # [ 59.893559] nix-daemon[847]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4974worker1 # [ 60.081371] nix-daemon[847]: accepted connection from pid 1115, user nix-grpc-daemon (trusted)4975worker1 # [ 60.095654] nix-daemon[847]: reaped child process 1118, status = succeeded4976worker1 # [ 60.257766] nix-daemon[847]: accepted connection from pid 1120, user nix-grpc-daemon (trusted)4977worker1 # [ 60.272785] nix-daemon[847]: reaped child process 1123, status = succeeded4978lb # [ 60.299203] niks3-server[1118]: 2026/09/20 12:22:08 INFO Received uploads request method=POST path=/api/pending_closures4979worker1 # [ 60.296345] nix-grpc-daemon[857]: time=2026-09-20T12:22:09.196Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4980worker1 # [ 60.298216] nix-grpc-daemon[857]: time=2026-09-20T12:22:09.198Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"4981lb # [ 60.329685] niks3-server[1118]: 2026/09/20 12:22:08 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst4982lb # [ 60.338224] niks3-server[1118]: 2026/09/20 12:22:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4983worker1 # [ 60.335785] nix-grpc-daemon[857]: time=2026-09-20T12:22:09.236Z level=INFO msg="Uploading 1 narinfos"4984lb # [ 60.345136] niks3-server[1118]: 2026/09/20 12:22:08 INFO Signed narinfos id=15 count=14985lb # [ 60.349740] niks3-server[1118]: 2026/09/20 12:22:08 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls4986lb # [ 60.361981] niks3-server[1118]: 2026/09/20 12:22:08 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4987lb # [ 60.368451] niks3-server[1118]: 2026/09/20 12:22:08 INFO Completed upload id=154988worker1 # [ 60.361256] nix-grpc-daemon[857]: time=2026-09-20T12:22:09.261Z level=INFO msg="Upload complete. (452ms)"4989lb # [ 60.371484] niks3-server[1118]: 2026/09/20 12:22:08 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo4990worker1 # [ 60.363981] nix-grpc-daemon[788]: ts=2026-09-20T12:22:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51432 duration_s=0 paths=1 nar_bytes_in=7524991worker1 # [ 60.370597] nix-grpc-daemon[788]: ts=2026-09-20T12:22:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514324992worker1 # [ 60.381088] nix-daemon[847]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4993worker1 # [ 60.385431] nix-daemon[847]: reaped child process 1113, status = succeeded4994worker1 # [ 60.390620] nix-daemon[847]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4995worker1 # [ 61.808114] nix-daemon[847]: reaped child process 1127, status = succeeded4996worker1 # [ 61.955754] nix-daemon[847]: accepted connection from pid 1142, user nix-grpc-daemon (trusted)4997worker1 # [ 61.970181] nix-daemon[847]: reaped child process 1145, status = succeeded4998lb # [ 61.996419] niks3-server[1118]: 2026/09/20 12:22:10 INFO Received uploads request method=POST path=/api/pending_closures4999worker1 # [ 61.996849] nix-grpc-daemon[857]: time=2026-09-20T12:22:10.896Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5000worker1 # [ 62.000055] nix-grpc-daemon[857]: time=2026-09-20T12:22:10.898Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"5001lb # [ 62.039786] niks3-server[1118]: 2026/09/20 12:22:10 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst5002lb # [ 62.048165] niks3-server[1118]: 2026/09/20 12:22:10 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv5003lb # [ 62.057639] niks3-server[1118]: 2026/09/20 12:22:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign5004lb # [ 62.063236] niks3-server[1118]: 2026/09/20 12:22:10 INFO Signed narinfos id=16 count=15005worker1 # [ 62.056199] nix-grpc-daemon[857]: time=2026-09-20T12:22:10.956Z level=INFO msg="Uploading 1 narinfos"5006lb # [ 62.066883] niks3-server[1118]: 2026/09/20 12:22:10 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls5007lb # [ 62.075536] niks3-server[1118]: 2026/09/20 12:22:10 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete5008lb # [ 62.079888] niks3-server[1118]: 2026/09/20 12:22:10 INFO Completed upload id=165009worker1 # [ 62.070774] nix-grpc-daemon[857]: time=2026-09-20T12:22:10.971Z level=INFO msg="Upload complete. (271ms)"5010lb # [ 62.081560] niks3-server[1118]: 2026/09/20 12:22:10 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo5011worker1 # [ 62.076567] nix-grpc-daemon[788]: ts=2026-09-20T12:22:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51432 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=8 outputs=15012worker2 # [ 62.104612] nix-grpc-daemon[787]: ts=2026-09-20T12:22:10Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:485225013worker1 # [ 62.088091] nix-daemon[847]: reaped child process 1125, status = succeeded5014worker2 # [ 62.113589] nix-grpc-daemon[787]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...5015worker2 # [ 62.141232] nix-grpc-daemon[787]: ts=2026-09-20T12:22:10Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:485225016worker2 # [ 62.145921] nix-grpc-daemon[787]: ts=2026-09-20T12:22:10Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48522 duration_s=0 paths=1 nar_bytes_out=2325017client # [ 62.227647] nix-daemon[949]: reaped child process 951, status = succeeded5018client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.52 seconds)5019client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5020client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.03 seconds)5021client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5022client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.04 seconds)5023client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5024client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)5025(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.61 seconds)5026subtest: an input only one worker has still reaches the builder5027client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only5028client # this derivation will be built:5029client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5030client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5031client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only, in 0.41 seconds)5032client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5033client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.20 seconds)5034worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5035worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.23 seconds)5036worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5037worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.03 seconds)5038client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&25039client # [ 63.393326] nix-daemon[949]: accepted connection from pid 1029, user root (trusted)5040client # this derivation will be built:5041client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5042client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...5043worker1 # [ 63.587949] nix-grpc-daemon[788]: ts=2026-09-20T12:22:12Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:514325044worker2 # [ 63.613659] nix-grpc-daemon[787]: ts=2026-09-20T12:22:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:485225045client # copying 1 paths...5046client # copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' to 'grpc://lb:50051'...5047worker1 # [ 63.602282] nix-grpc-daemon[788]: ts=2026-09-20T12:22:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:514325048worker1 # [ 63.615896] nix-daemon[847]: accepted connection from pid 788, user nix-grpc-daemon (trusted)5049worker1 # [ 63.799739] nix-daemon[847]: accepted connection from pid 1154, user nix-grpc-daemon (trusted)5050worker1 # [ 63.813710] nix-daemon[847]: reaped child process 1157, status = succeeded5051lb # [ 63.840636] niks3-server[1118]: 2026/09/20 12:22:12 INFO Received uploads request method=POST path=/api/pending_closures5052worker1 # [ 63.960802] nix-grpc-daemon[857]: time=2026-09-20T12:22:12.860Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5053worker1 # [ 63.965741] nix-grpc-daemon[857]: time=2026-09-20T12:22:12.864Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5054lb # [ 63.997160] niks3-server[1118]: 2026/09/20 12:22:12 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5055lb # [ 64.002600] niks3-server[1118]: 2026/09/20 12:22:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign5056lb # [ 64.007694] niks3-server[1118]: 2026/09/20 12:22:12 INFO Signed narinfos id=17 count=15057worker1 # [ 64.001716] nix-grpc-daemon[857]: time=2026-09-20T12:22:12.901Z level=INFO msg="Uploading 1 narinfos"5058lb # [ 64.016417] niks3-server[1118]: 2026/09/20 12:22:12 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5059lb # [ 64.027856] niks3-server[1118]: 2026/09/20 12:22:12 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete5060lb # [ 64.037618] niks3-server[1118]: 2026/09/20 12:22:12 INFO Completed upload id=175061worker1 # [ 64.028287] nix-grpc-daemon[857]: time=2026-09-20T12:22:12.928Z level=INFO msg="Upload complete. (401ms)"5062lb # [ 64.043054] niks3-server[1118]: 2026/09/20 12:22:12 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5063worker1 # [ 64.033193] nix-grpc-daemon[788]: ts=2026-09-20T12:22:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51432 duration_s=0 paths=1 nar_bytes_in=1205064worker1 # [ 64.047420] nix-daemon[847]: reaped child process 1152, status = succeeded5065worker2 # [ 64.071845] nix-grpc-daemon[787]: ts=2026-09-20T12:22:12Z level=debug event=want client=9 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05066worker2 # [ 64.076801] nix-grpc-daemon[787]: ts=2026-09-20T12:22:12Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=15067worker2 # [ 64.115782] nix-grpc-daemon[787]: ts=2026-09-20T12:22:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:485225068worker2 # [ 64.128552] nix-grpc-daemon[787]: ts=2026-09-20T12:22:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:485225069client # copying 2 paths to 'lb:50051'...5070client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5071client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5072worker2 # [ 64.145157] nix-grpc-daemon[787]: ts=2026-09-20T12:22:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:485225073worker2 # [ 64.157404] nix-daemon[857]: accepted connection from pid 787, user nix-grpc-daemon (trusted)5074worker2 # [ 64.342639] nix-daemon[857]: accepted connection from pid 997, user nix-grpc-daemon (trusted)5075worker2 # [ 64.359360] nix-daemon[857]: reaped child process 1000, status = succeeded5076worker2 # [ 64.526945] nix-daemon[857]: accepted connection from pid 1002, user nix-grpc-daemon (trusted)5077worker2 # [ 64.541263] nix-daemon[857]: reaped child process 1005, status = succeeded5078lb # [ 64.542505] niks3-server[1118]: 2026/09/20 12:22:13 INFO Received uploads request method=POST path=/api/pending_closures5079lb # [ 64.552902] niks3-server[1118]: 2026/09/20 12:22:13 INFO Received uploads request method=POST path=/api/pending_closures5080worker2 # [ 64.575817] nix-grpc-daemon[907]: time=2026-09-20T12:22:13.371Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5081worker2 # [ 64.577738] nix-grpc-daemon[907]: time=2026-09-20T12:22:13.372Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5082worker2 # [ 64.579996] nix-grpc-daemon[907]: time=2026-09-20T12:22:13.373Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5083lb # [ 64.590579] niks3-server[1118]: 2026/09/20 12:22:13 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5084lb # [ 64.600994] niks3-server[1118]: 2026/09/20 12:22:13 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5085lb # [ 64.611650] niks3-server[1118]: 2026/09/20 12:22:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5086lb # [ 64.614232] niks3-server[1118]: 2026/09/20 12:22:13 INFO Signed narinfos id=18 count=15087lb # [ 64.617312] niks3-server[1118]: 2026/09/20 12:22:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5088lb # [ 64.619348] niks3-server[1118]: 2026/09/20 12:22:13 INFO Signed narinfos id=19 count=25089lb # [ 64.621612] niks3-server[1118]: 2026/09/20 12:22:13 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5090worker2 # [ 64.634222] nix-grpc-daemon[907]: time=2026-09-20T12:22:13.428Z level=INFO msg="Uploading 3 narinfos"5091lb # [ 64.624185] niks3-server[1118]: 2026/09/20 12:22:13 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5092lb # [ 64.638433] niks3-server[1118]: 2026/09/20 12:22:13 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5093lb # [ 64.640637] niks3-server[1118]: 2026/09/20 12:22:13 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5094lb # [ 64.642576] niks3-server[1118]: 2026/09/20 12:22:13 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5095lb # [ 64.648378] niks3-server[1118]: 2026/09/20 12:22:13 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5096lb # [ 64.662102] niks3-server[1118]: 2026/09/20 12:22:13 INFO Completed upload id=185097lb # [ 64.664261] niks3-server[1118]: 2026/09/20 12:22:13 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5098lb # [ 64.667378] niks3-server[1118]: 2026/09/20 12:22:13 INFO Completed upload id=195099worker2 # [ 64.681353] nix-grpc-daemon[907]: time=2026-09-20T12:22:13.476Z level=INFO msg="Upload complete. (508ms)"5100worker2 # [ 64.687713] nix-grpc-daemon[787]: ts=2026-09-20T12:22:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48522 duration_s=0 paths=2 nar_bytes_in=9765101worker2 # [ 64.693478] nix-grpc-daemon[787]: ts=2026-09-20T12:22:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:485225102worker2 # [ 64.707112] nix-daemon[857]: accepted connection from pid 787, user nix-grpc-daemon (trusted)5103worker2 # [ 64.714420] nix-daemon[857]: reaped child process 995, status = succeeded5104worker2 # [ 64.727474] nix-grpc-daemon[787]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...5105client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5106client # uses-input-w1onlya> worker2: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5107worker2 # [ 64.742088] nix-daemon[857]: accepted connection from pid 787, user nix-grpc-daemon (trusted)5108worker2 # [ 65.560592] nix-daemon[857]: reaped child process 1010, status = succeeded5109worker2 # [ 65.720392] nix-daemon[857]: accepted connection from pid 1025, user nix-grpc-daemon (trusted)5110worker2 # [ 65.736262] nix-daemon[857]: reaped child process 1028, status = succeeded5111lb # [ 65.734603] niks3-server[1118]: 2026/09/20 12:22:14 INFO Received uploads request method=POST path=/api/pending_closures5112worker2 # [ 65.760306] nix-grpc-daemon[907]: time=2026-09-20T12:22:14.555Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5113lb # [ 65.771696] niks3-server[1118]: 2026/09/20 12:22:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5114lb # [ 65.778440] niks3-server[1118]: 2026/09/20 12:22:14 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5115lb # [ 65.783863] niks3-server[1118]: 2026/09/20 12:22:14 INFO Signed narinfos id=20 count=15116worker2 # [ 65.800473] nix-grpc-daemon[907]: time=2026-09-20T12:22:14.596Z level=INFO msg="Uploading 1 narinfos"5117lb # [ 65.790157] niks3-server[1118]: 2026/09/20 12:22:14 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5118lb # [ 65.802570] niks3-server[1118]: 2026/09/20 12:22:14 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5119lb # [ 65.807203] niks3-server[1118]: 2026/09/20 12:22:14 INFO Completed upload id=205120lb # [ 65.809984] niks3-server[1118]: 2026/09/20 12:22:14 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5121worker2 # [ 65.823365] nix-grpc-daemon[907]: time=2026-09-20T12:22:14.618Z level=INFO msg="Upload complete. (273ms)"5122worker2 # [ 65.830605] nix-grpc-daemon[787]: ts=2026-09-20T12:22:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48522 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=9 outputs=15123worker2 # [ 65.842180] nix-daemon[857]: reaped child process 1007, status = succeeded5124client # copying 1 paths...5125worker2 # [ 65.875130] nix-grpc-daemon[787]: ts=2026-09-20T12:22:14Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:485225126client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5127worker1 # [ 65.872114] nix-grpc-daemon[788]: ts=2026-09-20T12:22:14Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:514325128worker1 # [ 65.887666] nix-grpc-daemon[788]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5129worker1 # [ 65.898821] nix-grpc-daemon[788]: ts=2026-09-20T12:22:14Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:51432 duration_s=0 paths=1 nar_bytes_out=1205130client # [ 65.997406] nix-daemon[949]: reaped child process 1032, status = succeeded5131client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.82 seconds)5132client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&25133client # [ 66.165461] nix-daemon[949]: accepted connection from pid 1050, user root (trusted)5134client # this derivation will be built:5135client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5136worker1 # [ 66.346807] nix-grpc-daemon[788]: ts=2026-09-20T12:22:15Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:514325137worker1 # [ 66.349838] nix-grpc-daemon[788]: ts=2026-09-20T12:22:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514325138client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...5139worker1 # [ 66.357529] nix-grpc-daemon[857]: time=2026-09-20T12:22:15.257Z level=INFO msg="All 1 paths already cached"5140worker2 # [ 66.394187] nix-grpc-daemon[787]: ts=2026-09-20T12:22:15Z level=debug event=want client=10 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05141worker2 # [ 66.399364] nix-grpc-daemon[787]: ts=2026-09-20T12:22:15Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15142worker1 # [ 66.383761] nix-grpc-daemon[788]: ts=2026-09-20T12:22:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514325143worker1 # [ 66.392123] nix-grpc-daemon[788]: ts=2026-09-20T12:22:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514325144client # copying 1 paths to 'lb:50051'...5145client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5146worker1 # [ 66.405327] nix-grpc-daemon[788]: ts=2026-09-20T12:22:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:514325147worker1 # [ 66.413059] nix-daemon[847]: accepted connection from pid 788, user nix-grpc-daemon (trusted)5148worker1 # [ 66.422808] nix-grpc-daemon[788]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 29 ms (attempt 1/5)5149worker1 # [ 66.454641] nix-grpc-daemon[788]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 40 ms (attempt 2/5)5150worker1 # [ 66.497135] nix-grpc-daemon[788]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 77 ms (attempt 3/5)5151worker1 # [ 66.576814] nix-grpc-daemon[788]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 533 ms (attempt 4/5)5152worker1 # [ 67.111584] nix-grpc-daemon[788]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5153worker1 # [ 67.118452] nix-grpc-daemon[788]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5154worker1 # [ 67.316410] nix-daemon[847]: accepted connection from pid 1166, user nix-grpc-daemon (trusted)5155worker1 # [ 67.331534] nix-daemon[847]: reaped child process 1169, status = succeeded5156worker1 # [ 67.498784] nix-daemon[847]: accepted connection from pid 1171, user nix-grpc-daemon (trusted)5157worker1 # [ 67.511483] nix-daemon[847]: reaped child process 1174, status = succeeded5158lb # [ 67.537389] niks3-server[1118]: 2026/09/20 12:22:16 INFO Received uploads request method=POST path=/api/pending_closures5159worker1 # [ 67.537515] nix-grpc-daemon[857]: time=2026-09-20T12:22:16.437Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5160worker1 # [ 67.539390] nix-grpc-daemon[857]: time=2026-09-20T12:22:16.439Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5161lb # [ 67.568590] niks3-server[1118]: 2026/09/20 12:22:16 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5162lb # [ 67.580817] niks3-server[1118]: 2026/09/20 12:22:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5163lb # [ 67.586478] niks3-server[1118]: 2026/09/20 12:22:16 INFO Signed narinfos id=21 count=15164worker1 # [ 67.580075] nix-grpc-daemon[857]: time=2026-09-20T12:22:16.480Z level=INFO msg="Uploading 1 narinfos"5165lb # [ 67.594062] niks3-server[1118]: 2026/09/20 12:22:16 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5166lb # [ 67.605116] niks3-server[1118]: 2026/09/20 12:22:16 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5167lb # [ 67.611852] niks3-server[1118]: 2026/09/20 12:22:16 INFO Completed upload id=215168worker1 # [ 67.602405] nix-grpc-daemon[857]: time=2026-09-20T12:22:16.502Z level=INFO msg="Upload complete. (467ms)"5169lb # [ 67.613193] niks3-server[1118]: 2026/09/20 12:22:16 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5170worker1 # [ 67.604631] nix-grpc-daemon[788]: ts=2026-09-20T12:22:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51432 duration_s=1 paths=1 nar_bytes_in=5605171worker1 # [ 67.608322] nix-grpc-daemon[788]: ts=2026-09-20T12:22:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514325172worker1 # [ 67.615604] nix-daemon[847]: accepted connection from pid 788, user nix-grpc-daemon (trusted)5173worker1 # [ 67.619417] nix-daemon[847]: reaped child process 1160, status = succeeded5174worker1 # [ 67.624472] nix-daemon[847]: accepted connection from pid 788, user nix-grpc-daemon (trusted)5175client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5176client # uses-input-w1onlyb> node-a: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5177lb # [ 68.029299] envoy[979]: conn peer=[2001:db8:1::1]:59688 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3837 tx=1950 ms=16925178lb # [ 68.033143] envoy[979]: conn peer=[2001:db8:1::1]:59702 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=378 ms=555179lb # [ 68.036625] envoy[979]: conn peer=[2001:db8:1::1]:59716 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=585180lb # [ 68.040122] envoy[979]: conn peer=[2001:db8:1::1]:59708 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4315 tx=2196 ms=26035181lb # [ 68.043785] envoy[979]: conn peer=[2001:db8:1::1]:37492 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4253 tx=1769 ms=23655182lb # [ 68.047379] envoy[979]: conn peer=[2001:db8:1::1]:37496 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=329 ms=825183worker1 # [ 68.841527] nix-daemon[847]: reaped child process 1178, status = succeeded5184worker1 # [ 68.975158] nix-daemon[847]: accepted connection from pid 1193, user nix-grpc-daemon (trusted)5185worker1 # [ 68.990622] nix-daemon[847]: reaped child process 1196, status = succeeded5186lb # [ 69.016648] niks3-server[1118]: 2026/09/20 12:22:17 INFO Received uploads request method=POST path=/api/pending_closures5187worker1 # [ 69.011341] nix-grpc-daemon[857]: time=2026-09-20T12:22:17.911Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5188lb # [ 69.033184] niks3-server[1118]: 2026/09/20 12:22:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5189lb # [ 69.035444] niks3-server[1118]: 2026/09/20 12:22:17 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5190lb # [ 69.037577] niks3-server[1118]: 2026/09/20 12:22:17 INFO Signed narinfos id=22 count=15191worker1 # [ 69.027875] nix-grpc-daemon[857]: time=2026-09-20T12:22:17.928Z level=INFO msg="Uploading 1 narinfos"5192lb # [ 69.039708] niks3-server[1118]: 2026/09/20 12:22:17 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5193lb # [ 69.044425] niks3-server[1118]: 2026/09/20 12:22:17 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5194lb # [ 69.047566] niks3-server[1118]: 2026/09/20 12:22:17 INFO Completed upload id=225195worker1 # [ 69.038069] nix-grpc-daemon[857]: time=2026-09-20T12:22:17.938Z level=INFO msg="Upload complete. (206ms)"5196lb # [ 69.050328] niks3-server[1118]: 2026/09/20 12:22:17 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5197worker1 # [ 69.040578] nix-grpc-daemon[788]: ts=2026-09-20T12:22:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51432 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=10 outputs=15198client # copying 1 paths...5199worker1 # [ 69.046183] nix-grpc-daemon[788]: ts=2026-09-20T12:22:17Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:514325200client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5201worker1 # [ 69.050470] nix-daemon[847]: reaped child process 1176, status = succeeded5202worker2 # [ 69.085653] nix-grpc-daemon[787]: ts=2026-09-20T12:22:17Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:485225203worker2 # [ 69.091041] nix-grpc-daemon[787]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5204worker2 # [ 69.101693] nix-grpc-daemon[787]: ts=2026-09-20T12:22:17Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48522 duration_s=0 paths=1 nar_bytes_out=1205205client # [ 69.178611] nix-daemon[949]: reaped child process 1053, status = succeeded5206client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.18 seconds)5207client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&25208client # [ 69.358425] nix-daemon[949]: accepted connection from pid 1070, user root (trusted)5209client # this derivation will be built:5210client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5211worker1 # [ 69.545259] nix-grpc-daemon[788]: ts=2026-09-20T12:22:18Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:514325212worker2 # [ 69.571087] nix-grpc-daemon[787]: ts=2026-09-20T12:22:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:485225213client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...5214worker2 # [ 69.576974] nix-grpc-daemon[907]: time=2026-09-20T12:22:18.372Z level=INFO msg="All 1 paths already cached"5215worker2 # [ 69.586390] nix-grpc-daemon[787]: ts=2026-09-20T12:22:18Z level=debug event=want client=11 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05216worker2 # [ 69.588657] nix-grpc-daemon[787]: ts=2026-09-20T12:22:18Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15217worker2 # [ 69.591251] nix-grpc-daemon[787]: ts=2026-09-20T12:22:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:485225218worker2 # [ 69.598673] nix-grpc-daemon[787]: ts=2026-09-20T12:22:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:485225219worker2 # [ 69.606287] nix-grpc-daemon[907]: time=2026-09-20T12:22:18.401Z level=INFO msg="All 1 paths already cached"5220client # copying 1 paths to 'lb:50051'...5221client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5222worker2 # [ 69.610399] nix-grpc-daemon[787]: ts=2026-09-20T12:22:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:485225223worker2 # [ 69.615896] nix-daemon[857]: accepted connection from pid 787, user nix-grpc-daemon (trusted)5224worker2 # [ 69.770651] nix-daemon[857]: accepted connection from pid 1037, user nix-grpc-daemon (trusted)5225worker2 # [ 69.784344] nix-daemon[857]: reaped child process 1040, status = succeeded5226worker2 # [ 69.926455] nix-daemon[857]: accepted connection from pid 1042, user nix-grpc-daemon (trusted)5227worker2 # [ 69.940182] nix-daemon[857]: reaped child process 1045, status = succeeded5228lb # [ 69.936242] niks3-server[1118]: 2026/09/20 12:22:18 INFO Received uploads request method=POST path=/api/pending_closures5229worker2 # [ 69.958486] nix-grpc-daemon[907]: time=2026-09-20T12:22:18.753Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5230worker2 # [ 69.959840] nix-grpc-daemon[907]: time=2026-09-20T12:22:18.753Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5231lb # [ 69.961764] niks3-server[1118]: 2026/09/20 12:22:18 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5232lb # [ 69.973589] niks3-server[1118]: 2026/09/20 12:22:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5233lb # [ 69.979246] niks3-server[1118]: 2026/09/20 12:22:18 INFO Signed narinfos id=23 count=15234worker2 # [ 69.996369] nix-grpc-daemon[907]: time=2026-09-20T12:22:18.791Z level=INFO msg="Uploading 1 narinfos"5235lb # [ 69.986682] niks3-server[1118]: 2026/09/20 12:22:18 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5236lb # [ 69.998666] niks3-server[1118]: 2026/09/20 12:22:18 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5237worker2 # [ 70.017599] nix-grpc-daemon[907]: time=2026-09-20T12:22:18.813Z level=INFO msg="Upload complete. (396ms)"5238lb # [ 70.004111] niks3-server[1118]: 2026/09/20 12:22:18 INFO Completed upload id=235239worker2 # [ 70.019601] nix-grpc-daemon[787]: ts=2026-09-20T12:22:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48522 duration_s=0 paths=1 nar_bytes_in=5605240lb # [ 70.007863] niks3-server[1118]: 2026/09/20 12:22:18 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5241worker2 # [ 70.025935] nix-grpc-daemon[787]: ts=2026-09-20T12:22:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:485225242worker2 # [ 70.033490] nix-daemon[857]: accepted connection from pid 787, user nix-grpc-daemon (trusted)5243worker2 # [ 70.035833] nix-daemon[857]: reaped child process 1035, status = succeeded5244worker2 # [ 70.041081] nix-daemon[857]: accepted connection from pid 787, user nix-grpc-daemon (trusted)5245client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5246client # uses-input-w1onlyc> worker2: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5247worker2 # [ 71.064466] nix-daemon[857]: reaped child process 1049, status = succeeded5248worker2 # [ 71.225346] nix-daemon[857]: accepted connection from pid 1064, user nix-grpc-daemon (trusted)5249worker2 # [ 71.236255] nix-daemon[857]: reaped child process 1067, status = succeeded5250lb # [ 71.238656] niks3-server[1118]: 2026/09/20 12:22:19 INFO Received uploads request method=POST path=/api/pending_closures5251worker2 # [ 71.261136] nix-grpc-daemon[907]: time=2026-09-20T12:22:20.056Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5252lb # [ 71.271400] niks3-server[1118]: 2026/09/20 12:22:19 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5253lb # [ 71.278160] niks3-server[1118]: 2026/09/20 12:22:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5254worker2 # [ 71.296498] nix-grpc-daemon[907]: time=2026-09-20T12:22:20.091Z level=INFO msg="Uploading 1 narinfos"5255lb # [ 71.284356] niks3-server[1118]: 2026/09/20 12:22:19 INFO Signed narinfos id=24 count=15256lb # [ 71.289200] niks3-server[1118]: 2026/09/20 12:22:19 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5257lb # [ 71.301338] niks3-server[1118]: 2026/09/20 12:22:19 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5258worker2 # [ 71.321738] nix-grpc-daemon[907]: time=2026-09-20T12:22:20.117Z level=INFO msg="Upload complete. (271ms)"5259lb # [ 71.307991] niks3-server[1118]: 2026/09/20 12:22:19 INFO Completed upload id=245260lb # [ 71.311919] niks3-server[1118]: 2026/09/20 12:22:19 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5261worker2 # [ 71.326110] nix-grpc-daemon[787]: ts=2026-09-20T12:22:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48522 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=11 outputs=15262worker2 # [ 71.337337] nix-daemon[857]: reaped child process 1047, status = succeeded5263client # copying 1 paths...5264worker1 # [ 71.349628] nix-grpc-daemon[788]: ts=2026-09-20T12:22:20Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:514325265worker1 # [ 71.361246] nix-grpc-daemon[788]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5266client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5267worker2 # [ 71.416160] nix-grpc-daemon[787]: ts=2026-09-20T12:22:20Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:485225268worker2 # [ 71.421197] nix-grpc-daemon[787]: ts=2026-09-20T12:22:20Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48522 duration_s=0 paths=1 nar_bytes_out=1205269client # [ 71.456669] nix-daemon[949]: reaped child process 1073, status = succeeded5270client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.27 seconds)5271(finished: subtest: an input only one worker has still reaches the builder, in 9.13 seconds)5272subtest: upload whose reference the worker lacks is completed from the cache5273client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache5274client # this derivation will be built:5275client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5276client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5277client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache, in 0.40 seconds)5278client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache5279client # this derivation will be built:5280client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5281client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5282client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache, in 0.38 seconds)5283client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5284client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.18 seconds)5285worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5286worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.25 seconds)5287worker2 # [ 72.738365] systemd[1]: Stopping Nix worker-protocol over gRPC...5288worker2 # [ 73.452679] nix-grpc-daemon[787]: ts=2026-09-20T12:22:22Z level=info event=signal_exit5289worker2 # [ 73.717222] nix-grpc-daemon[787]: ts=2026-09-20T12:22:22Z level=info event=scheduler_yield5290lb # [ 73.705766] niks3-server[1118]: 2026/09/20 12:22:22 INFO lead: released remote=192.168.1.4:600585291worker2 # [ 73.721604] nix-grpc-daemon[787]: ts=2026-09-20T12:22:22Z level=info event=scheduler_restarting addr=lb:500515292worker2 # [ 73.727459] nix-grpc-daemon[787]: ts=2026-09-20T12:22:22Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:498505293worker1 # [ 73.704466] nix-grpc-daemon[788]: ts=2026-09-20T12:22:22Z level=info event=scheduler_restarting addr=lb:500515294worker2 # [ 73.730808] nix-grpc-daemon[787]: ts=2026-09-20T12:22:22Z level=info event=worker_gone addr=192.168.1.4:500515295worker2 # [ 73.733795] nix-grpc-daemon[787]: ts=2026-09-20T12:22:22Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:498505296worker2 # [ 73.737129] nix-grpc-daemon[787]: ts=2026-09-20T12:22:22Z level=info event=worker_gone addr=192.168.1.3:500515297worker2 # [ 73.922338] nix-grpc-daemon[787]: ts=2026-09-20T12:22:22Z level=info event=scheduler_connected addr=lb:500515298worker2 # [ 73.924432] nix-grpc-daemon[787]: ts=2026-09-20T12:22:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5299worker1 # [ 73.905171] nix-grpc-daemon[788]: ts=2026-09-20T12:22:22Z level=info event=scheduler_connected addr=lb:500515300worker1 # [ 73.908424] nix-grpc-daemon[788]: ts=2026-09-20T12:22:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5301worker2 # [ 74.124947] nix-grpc-daemon[787]: ts=2026-09-20T12:22:22Z level=info event=scheduler_connected addr=lb:500515302worker2 # [ 74.126661] nix-grpc-daemon[787]: ts=2026-09-20T12:22:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5303worker1 # [ 74.107166] nix-grpc-daemon[788]: ts=2026-09-20T12:22:23Z level=info event=scheduler_connected addr=lb:500515304worker1 # [ 74.110506] nix-grpc-daemon[788]: ts=2026-09-20T12:22:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5305worker2 # [ 74.327318] nix-grpc-daemon[787]: ts=2026-09-20T12:22:23Z level=info event=scheduler_connected addr=lb:500515306worker2 # [ 74.329157] nix-grpc-daemon[787]: ts=2026-09-20T12:22:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5307worker1 # [ 74.307901] nix-grpc-daemon[788]: ts=2026-09-20T12:22:23Z level=info event=scheduler_connected addr=lb:500515308worker1 # [ 74.311985] nix-grpc-daemon[788]: ts=2026-09-20T12:22:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5309worker2 # [ 74.531690] nix-grpc-daemon[787]: ts=2026-09-20T12:22:23Z level=info event=scheduler_connected addr=lb:500515310worker2 # [ 74.534793] nix-grpc-daemon[787]: ts=2026-09-20T12:22:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5311worker1 # [ 74.513208] nix-grpc-daemon[788]: ts=2026-09-20T12:22:23Z level=info event=scheduler_connected addr=lb:500515312worker1 # [ 74.517338] nix-grpc-daemon[788]: ts=2026-09-20T12:22:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5313worker2 # [ 74.721458] nix-grpc-daemon[787]: ts=2026-09-20T12:22:23Z level=info event=scheduler_restarting5314worker1 # [ 74.719241] nix-grpc-daemon[788]: ts=2026-09-20T12:22:23Z level=info event=scheduler_connected addr=lb:500515315worker2 # [ 74.743379] nix-grpc-daemon[787]: ts=2026-09-20T12:22:23Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5316worker1 # [ 74.723206] nix-grpc-daemon[788]: ts=2026-09-20T12:22:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5317worker2 # [ 74.757493] nix-daemon[857]: reaped child process 859, status = succeeded5318worker2 # [ 74.803794] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5319worker2 # [ 74.806672] systemd[1]: Stopped Nix worker-protocol over gRPC.5320worker2 # [ 74.809236] systemd[1]: nix-grpc-daemon.service: Consumed 1.441s CPU time over 56.624s wall clock time, 50.1M memory peak, 57.4K incoming IP traffic, 152.4K outgoing IP traffic.5321worker2 # [ 74.816973] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5322worker2 # [ 74.819202] systemd[1]: Closed Nix worker-protocol over gRPC.5323worker2 # [ 74.820500] systemd[1]: nix-grpc-daemon.socket: Consumed 306us CPU time over 1min 5.120s wall clock time, 256K memory peak, 72.2K incoming IP traffic, 61.4K outgoing IP traffic.5324lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5325lb: must succeed: curl -sf localhost:9901/clusters5326lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5327lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.08 seconds)5328client: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25329worker1 # [ 74.924266] nix-grpc-daemon[788]: ts=2026-09-20T12:22:23Z level=info event=scheduler_connected addr=lb:500515330worker1 # [ 74.928270] nix-grpc-daemon[788]: ts=2026-09-20T12:22:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5331worker1 # [ 75.087422] nix-grpc-daemon[788]: ts=2026-09-20T12:22:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:514325332worker1 # [ 75.094894] nix-grpc-daemon[788]: ts=2026-09-20T12:22:23Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:514325333client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5334client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.25 seconds)5335worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5336worker1 # [ 75.129134] nix-grpc-daemon[788]: ts=2026-09-20T12:22:24Z level=info event=scheduler_connected addr=lb:500515337worker1 # [ 75.132992] nix-grpc-daemon[788]: ts=2026-09-20T12:22:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5338worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.03 seconds)5339worker2 # [ 75.228073] systemd[1]: Listening on Nix worker-protocol over gRPC.5340worker1 # [ 75.267608] systemd[1]: Stopping Nix worker-protocol over gRPC...5341worker1 # [ 75.296959] nix-grpc-daemon[788]: ts=2026-09-20T12:22:24Z level=info event=signal_exit5342worker1 # [ 75.333587] nix-grpc-daemon[788]: ts=2026-09-20T12:22:24Z level=info event=scheduler_connected addr=lb:500515343worker1 # [ 75.337428] nix-grpc-daemon[788]: ts=2026-09-20T12:22:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5344worker1 # [ 75.538556] nix-grpc-daemon[788]: ts=2026-09-20T12:22:24Z level=info event=scheduler_connected addr=lb:500515345worker1 # [ 75.542639] nix-grpc-daemon[788]: ts=2026-09-20T12:22:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5346worker2 # [ 75.591359] systemd[1]: Starting scheduler-token.service...5347worker2 # [ 75.679592] systemd[1]: scheduler-token.service: Deactivated successfully.5348worker2 # [ 75.680830] systemd[1]: Finished scheduler-token.service.5349worker2 # [ 75.682291] systemd[1]: scheduler-token.service: Consumed 36ms CPU time over 88ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5350worker2 # [ 75.687185] systemd[1]: Starting Nix worker-protocol over gRPC...5351worker1 # [ 75.745213] nix-grpc-daemon[788]: ts=2026-09-20T12:22:24Z level=info event=scheduler_connected addr=lb:500515352worker1 # [ 75.748424] nix-grpc-daemon[788]: ts=2026-09-20T12:22:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5353worker2 # [ 75.837381] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:24Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5354worker2 # [ 75.839696] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:24Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5355worker2 # [ 75.843469] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:24Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15356worker2 # [ 75.850058] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:24Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5357worker2 # [ 75.854293] systemd[1]: Started Nix worker-protocol over gRPC.5358worker2 # [ 75.862733] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:24Z level=info event=scheduler_take_over5359lb # [ 75.851736] niks3-server[1118]: 2026/09/20 12:22:24 INFO lead: acquired remote=192.168.1.4:425985360worker2 # [ 75.873548] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:24Z level=info event=scheduler_connected addr=lb:500515361worker2 # [ 75.875222] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5362worker1 # [ 75.946090] nix-grpc-daemon[788]: ts=2026-09-20T12:22:24Z level=info event=scheduler_connected addr=lb:500515363worker2 # [ 75.974937] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:24Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:400365364worker2 # [ 75.979426] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:24Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05365worker2 # [ 76.076587] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:24Z level=info event=scheduler_connected addr=lb:500515366worker2 # [ 76.087138] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:24Z level=info event=oidc_jwks provider=mock keys=15367worker2 # [ 76.091287] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:24Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:400365368worker2 # [ 76.095212] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:24Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05369worker1 # [ 76.568405] nix-grpc-daemon[788]: ts=2026-09-20T12:22:25Z level=info event=scheduler_restarting5370worker2 # [ 76.600856] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:25Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:400365371worker2 # [ 76.604670] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:25Z level=info event=worker_gone addr=192.168.1.3:500515372worker1 # [ 76.592464] nix-grpc-daemon[788]: ts=2026-09-20T12:22:25Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5373worker1 # [ 76.608804] nix-daemon[847]: reaped child process 849, status = succeeded5374worker1 # [ 76.736870] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5375worker1 # [ 76.739284] systemd[1]: Stopped Nix worker-protocol over gRPC.5376worker1 # [ 76.740108] systemd[1]: nix-grpc-daemon.service: Consumed 2.760s CPU time over 58.723s wall clock time, 58.3M memory peak, 87.6K incoming IP traffic, 164.4K outgoing IP traffic.5377worker1 # [ 76.744299] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5378worker1 # [ 76.746190] systemd[1]: Closed Nix worker-protocol over gRPC.5379worker1 # [ 76.747535] systemd[1]: nix-grpc-daemon.socket: Consumed 356us CPU time over 1min 7.169s wall clock time, 256K memory peak, 74.7K incoming IP traffic, 62.1K outgoing IP traffic.5380lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5381lb: must succeed: curl -sf localhost:9901/clusters5382lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5383lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.05 seconds)5384client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25385worker2 # [ 77.039626] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:400465386worker2 # [ 77.050529] nix-daemon[857]: accepted connection from pid 1091, user nix-grpc-daemon (trusted)5387worker2 # [ 77.066983] nix-grpc-daemon[1091]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 47 ms (attempt 1/5)5388worker2 # [ 77.119754] nix-grpc-daemon[1091]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 194 ms (attempt 2/5)5389worker2 # [ 77.319718] nix-grpc-daemon[1091]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 385 ms (attempt 3/5)5390worker2 # [ 77.705422] nix-grpc-daemon[1091]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 105 ms (attempt 4/5)5391worker2 # [ 77.810925] nix-grpc-daemon[1091]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5392client # copying 2 paths...5393client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5394client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5395worker2 # [ 77.833729] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:400465396worker2 # [ 77.845094] nix-daemon[857]: accepted connection from pid 1091, user nix-grpc-daemon (trusted)5397worker2 # [ 77.865445] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:26Z level=info event=niks3_push_started pid=11475398lb # [ 78.029753] envoy[979]: conn peer=[2001:db8:1::1]:37504 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3702 tx=1800 ms=27845399lb # [ 78.033564] envoy[979]: conn peer=[2001:db8:1::1]:37518 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=329 ms=675400worker2 # [ 78.050387] nix-grpc-daemon[1147]: time=2026-09-20T12:22:26.845Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""5401lb # [ 78.037204] envoy[979]: conn peer=[2001:db8:1::1]:37532 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3717 tx=1916 ms=18675402lb # [ 78.040759] envoy[979]: conn peer=[2001:db8:1::1]:37534 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=421 tx=328 ms=185403lb # [ 78.044258] envoy[979]: conn peer=[2001:db8:1::4]:32768 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=57906 tx=2737 ms=562715404lb # [ 78.048655] envoy[979]: conn peer=[2001:db8:1::1]:60146 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=215405lb # [ 78.053333] envoy[979]: conn peer=192.168.1.3:34042 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=15860 tx=3137 ms=573625406worker2 # [ 78.212053] nix-daemon[857]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5407worker2 # [ 78.223657] nix-daemon[857]: reaped child process 1163, status = succeeded5408lb # [ 78.220631] niks3-server[1118]: 2026/09/20 12:22:26 INFO Received uploads request method=POST path=/api/pending_closures5409lb # [ 78.231102] niks3-server[1118]: 2026/09/20 12:22:26 INFO Received uploads request method=POST path=/api/pending_closures5410worker2 # [ 78.251630] nix-grpc-daemon[1147]: time=2026-09-20T12:22:27.047Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5411worker2 # [ 78.253156] nix-grpc-daemon[1147]: time=2026-09-20T12:22:27.048Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5412worker2 # [ 78.256056] nix-grpc-daemon[1147]: time=2026-09-20T12:22:27.050Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5413lb # [ 78.277689] niks3-server[1118]: 2026/09/20 12:22:26 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5414lb # [ 78.282470] niks3-server[1118]: 2026/09/20 12:22:26 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5415lb # [ 78.293938] niks3-server[1118]: 2026/09/20 12:22:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5416lb # [ 78.298272] niks3-server[1118]: 2026/09/20 12:22:26 INFO Signed narinfos id=25 count=15417lb # [ 78.304031] niks3-server[1118]: 2026/09/20 12:22:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5418lb # [ 78.306099] niks3-server[1118]: 2026/09/20 12:22:26 INFO Signed narinfos id=26 count=25419lb # [ 78.307474] niks3-server[1118]: 2026/09/20 12:22:26 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5420worker2 # [ 78.321861] nix-grpc-daemon[1147]: time=2026-09-20T12:22:27.116Z level=INFO msg="Uploading 3 narinfos"5421lb # [ 78.310901] niks3-server[1118]: 2026/09/20 12:22:26 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5422lb # [ 78.322984] niks3-server[1118]: 2026/09/20 12:22:26 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5423lb # [ 78.326071] niks3-server[1118]: 2026/09/20 12:22:26 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5424lb # [ 78.330312] niks3-server[1118]: 2026/09/20 12:22:26 INFO Completed upload id=255425lb # [ 78.332038] niks3-server[1118]: 2026/09/20 12:22:26 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5426lb # [ 78.334164] niks3-server[1118]: 2026/09/20 12:22:26 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5427lb # [ 78.335879] niks3-server[1118]: 2026/09/20 12:22:26 INFO Completed upload id=265428worker2 # [ 78.349890] nix-grpc-daemon[1147]: time=2026-09-20T12:22:27.145Z level=INFO msg="Upload complete. (295ms)"5429worker2 # [ 78.355699] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40046 duration_s=0 paths=2 nar_bytes_in=3125430lb # [ 78.345186] niks3-server[1118]: 2026/09/20 12:22:26 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5431worker2 # [ 78.365989] nix-daemon[857]: reaped child process 1145, status = succeeded5432client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.56 seconds)5433worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5434worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds)5435worker1 # [ 78.450851] systemd[1]: Listening on Nix worker-protocol over gRPC.5436worker1 # [ 78.459531] systemd[1]: Starting Nix worker-protocol over gRPC...5437worker1 # [ 78.618490] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:27Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5438worker1 # [ 78.621071] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:27Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5439worker1 # [ 78.625781] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:27Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15440worker1 # [ 78.633062] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:27Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5441worker1 # [ 78.639786] systemd[1]: Started Nix worker-protocol over gRPC.5442lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5443lb: must succeed: curl -sf localhost:9901/clusters5444worker1 # [ 78.651912] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:27Z level=info event=scheduler_connected addr=lb:500515445worker2 # [ 78.677388] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:27Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:400365446worker2 # [ 78.679153] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:27Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05447lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5448lb: must succeed: curl -sf localhost:9901/clusters5449lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5450lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.16 seconds)5451(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 8.33 seconds)5452subtest: scheduler down: the other node takes the lock and keeps it5453lb: must succeed: curl -sf localhost:9901/clusters5454lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5455lb: must succeed: curl -sf localhost:9901/clusters5456lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5457lb: must succeed: curl -sf localhost:9901/clusters5458lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5459lb: must succeed: curl -sf localhost:9901/clusters5460lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5461worker2 # [ 80.085643] systemd[1]: Stopping Nix worker-protocol over gRPC...5462worker2 # [ 80.853293] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:29Z level=info event=signal_exit5463worker2 # [ 80.862622] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:29Z level=info event=scheduler_yield5464lb # [ 80.852533] niks3-server[1118]: 2026/09/20 12:22:29 INFO lead: released remote=192.168.1.4:425985465worker1 # [ 80.844568] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:29Z level=info event=scheduler_restarting addr=lb:500515466worker2 # [ 80.870675] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:29Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:400365467worker2 # [ 80.876222] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:29Z level=info event=worker_gone addr=192.168.1.3:500515468worker2 # [ 80.879390] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:29Z level=info event=scheduler_restarting addr=lb:500515469worker2 # [ 80.882342] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:29Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:400365470worker2 # [ 80.888665] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:29Z level=info event=worker_gone addr=192.168.1.4:500515471worker1 # [ 81.044251] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:29Z level=info event=scheduler_connected addr=lb:500515472worker1 # [ 81.048288] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5473worker2 # [ 81.076062] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:29Z level=info event=scheduler_connected addr=lb:500515474worker2 # [ 81.077895] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5475worker1 # [ 81.248738] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:30Z level=info event=scheduler_connected addr=lb:500515476worker1 # [ 81.251089] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5477worker2 # [ 81.279357] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:30Z level=info event=scheduler_connected addr=lb:500515478worker2 # [ 81.283175] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5479worker1 # [ 81.451722] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:30Z level=info event=scheduler_connected addr=lb:500515480worker1 # [ 81.455827] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5481worker2 # [ 81.483644] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:30Z level=info event=scheduler_connected addr=lb:500515482worker2 # [ 81.485410] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5483worker1 # [ 81.656689] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:30Z level=info event=scheduler_connected addr=lb:500515484worker1 # [ 81.661323] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5485worker2 # [ 81.686530] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:30Z level=info event=scheduler_connected addr=lb:500515486worker2 # [ 81.690332] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5487worker2 # [ 81.867474] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:30Z level=info event=scheduler_restarting5488worker2 # [ 81.883620] nix-daemon[857]: reaped child process 1140, status = succeeded5489worker1 # [ 81.863239] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:30Z level=info event=scheduler_connected addr=lb:500515490worker1 # [ 81.866574] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5491worker2 # [ 81.889239] nix-grpc-daemon[1091]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5492worker2 # [ 81.894990] nix-grpc-daemon[1091]: E0000 00:00:1789906950.684308 1134 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5493worker2 # [ 81.904444] nix-grpc-daemon[1091]: E0000 00:00:1789906950.688748 1134 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5494worker2 # [ 81.911625] nix-grpc-daemon[1091]: ts=2026-09-20T12:22:30Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5495worker1 # [ 82.064150] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:30Z level=info event=scheduler_connected addr=lb:500515496worker1 # [ 82.068218] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5497worker1 # [ 82.268895] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:31Z level=info event=scheduler_connected addr=lb:500515498worker1 # [ 82.271537] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5499worker1 # [ 82.471655] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:31Z level=info event=scheduler_connected addr=lb:500515500worker1 # [ 82.473454] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5501worker1 # [ 82.673901] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:31Z level=info event=scheduler_connected addr=lb:500515502worker1 # [ 82.675765] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5503worker1 # [ 82.878934] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:31Z level=info event=scheduler_connected addr=lb:500515504worker1 # [ 82.882863] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5505worker1 # [ 83.079741] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:31Z level=info event=scheduler_connected addr=lb:500515506worker1 # [ 83.083830] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5507worker1 # [ 83.284799] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:32Z level=info event=scheduler_connected addr=lb:500515508worker1 # [ 83.288963] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5509worker1 # [ 83.489616] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:32Z level=info event=scheduler_connected addr=lb:500515510worker1 # [ 83.493824] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5511lb # [ 83.654570] niks3-server[1118]: 2026/09/20 12:22:32 INFO lead: acquired remote=192.168.1.3:367985512worker1 # [ 83.645600] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:32Z level=info event=scheduler_take_over5513worker1 # [ 83.694580] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:32Z level=info event=scheduler_connected addr=lb:500515514worker1 # [ 83.698697] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5515worker2 # [ 83.868219] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5516worker2 # [ 83.870223] systemd[1]: Stopped Nix worker-protocol over gRPC.5517worker2 # [ 83.871307] systemd[1]: nix-grpc-daemon.service: Consumed 316ms CPU time over 8.180s wall clock time, 30.6M memory peak, 19.5K incoming IP traffic, 27.3K outgoing IP traffic.5518worker2 # [ 83.875413] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5519worker2 # [ 83.877224] systemd[1]: Closed Nix worker-protocol over gRPC.5520worker2 # [ 83.878415] systemd[1]: nix-grpc-daemon.socket: Consumed 297us CPU time over 8.653s wall clock time, 256K memory peak, 19.3K incoming IP traffic, 13.9K outgoing IP traffic.5521worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5522worker1 # [ 83.902646] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:32Z level=info event=scheduler_connected addr=lb:500515523worker1 # [ 83.905892] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5524worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.09 seconds)5525worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5526worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5527worker1 # [ 84.102166] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:33Z level=info event=scheduler_connected addr=lb:500515528worker1 # [ 84.106315] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5529worker1 # [ 84.307168] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:33Z level=info event=scheduler_connected addr=lb:500515530worker1 # [ 84.311420] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5531worker1 # [ 84.512132] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:33Z level=info event=scheduler_connected addr=lb:500515532worker1 # [ 84.519690] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:33Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:382785533worker1 # [ 84.523744] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:33Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05534worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5535worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5536client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&25537worker1 # [ 85.276803] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:34Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:382805538worker1 # [ 85.288273] nix-daemon[847]: accepted connection from pid 1220, user nix-grpc-daemon (trusted)5539client # don't know how to build these paths:5540client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5541worker1 # [ 85.302378] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:382805542worker1 # [ 85.313392] nix-grpc-daemon[1220]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 35 ms (attempt 1/5)5543worker1 # [ 85.353940] nix-grpc-daemon[1220]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 53 ms (attempt 2/5)5544worker1 # [ 85.412807] nix-grpc-daemon[1220]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 359 ms (attempt 3/5)5545worker1 # [ 85.777558] nix-grpc-daemon[1220]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 124 ms (attempt 4/5)5546worker1 # [ 85.902224] nix-grpc-daemon[1220]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5547worker1 # [ 85.926684] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:34Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05548worker1 # [ 85.928807] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:34Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=15549worker1 # [ 85.973332] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:382805550worker1 # [ 85.986088] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:382805551client # copying 1 paths to 'lb:50051'...5552client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5553worker1 # [ 85.998593] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:382805554worker1 # [ 86.009164] nix-daemon[847]: accepted connection from pid 1220, user nix-grpc-daemon (trusted)5555worker1 # [ 86.357535] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:35Z level=info event=niks3_push_started pid=12905556worker1 # [ 86.567384] nix-grpc-daemon[1290]: time=2026-09-20T12:22:35.467Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""5557worker1 # [ 86.727939] nix-daemon[847]: accepted connection from pid 1303, user nix-grpc-daemon (trusted)5558worker1 # [ 86.740762] nix-daemon[847]: reaped child process 1306, status = succeeded5559worker1 # [ 86.902915] nix-daemon[847]: accepted connection from pid 1308, user nix-grpc-daemon (trusted)5560worker1 # [ 86.915644] nix-daemon[847]: reaped child process 1311, status = succeeded5561lb # [ 86.937273] niks3-server[1118]: 2026/09/20 12:22:35 INFO Received uploads request method=POST path=/api/pending_closures5562worker1 # [ 86.936222] nix-grpc-daemon[1290]: time=2026-09-20T12:22:35.836Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5563worker1 # [ 86.939479] nix-grpc-daemon[1290]: time=2026-09-20T12:22:35.838Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5564lb # [ 86.977428] niks3-server[1118]: 2026/09/20 12:22:35 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5565lb # [ 86.984609] niks3-server[1118]: 2026/09/20 12:22:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5566lb # [ 86.991469] niks3-server[1118]: 2026/09/20 12:22:35 INFO Signed narinfos id=27 count=15567worker1 # [ 86.986414] nix-grpc-daemon[1290]: time=2026-09-20T12:22:35.886Z level=INFO msg="Uploading 1 narinfos"5568lb # [ 87.004873] niks3-server[1118]: 2026/09/20 12:22:35 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5569lb # [ 87.014990] niks3-server[1118]: 2026/09/20 12:22:35 INFO Completed upload id=275570worker1 # [ 87.005545] nix-grpc-daemon[1290]: time=2026-09-20T12:22:35.905Z level=INFO msg="Upload complete. (434ms)"5571lb # [ 87.017575] niks3-server[1118]: 2026/09/20 12:22:35 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5572lb # [ 87.021207] niks3-server[1118]: 2026/09/20 12:22:35 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5573worker1 # [ 87.009894] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38280 duration_s=1 paths=1 nar_bytes_in=5205574worker1 # [ 87.016258] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:382805575worker1 # [ 87.023125] nix-daemon[847]: accepted connection from pid 1220, user nix-grpc-daemon (trusted)5576worker1 # [ 87.028313] nix-daemon[847]: reaped child process 1288, status = succeeded5577worker1 # [ 87.033347] nix-daemon[847]: accepted connection from pid 1220, user nix-grpc-daemon (trusted)5578client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5579client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5580client # farm-a> LOG-farm-a-on-standby5581lb # [ 88.029237] envoy[979]: conn peer=[2001:db8:1::1]:60154 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1010 tx=282 ms=13285582lb # [ 88.031493] envoy[979]: conn peer=[2001:db8:1::4]:55604 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=5303 tx=400 ms=60265583worker1 # [ 88.059797] nix-daemon[847]: reaped child process 1315, status = succeeded5584worker1 # [ 88.226497] nix-daemon[847]: accepted connection from pid 1330, user nix-grpc-daemon (trusted)5585worker1 # [ 88.238846] nix-daemon[847]: reaped child process 1333, status = succeeded5586lb # [ 88.265721] niks3-server[1118]: 2026/09/20 12:22:36 INFO Received uploads request method=POST path=/api/pending_closures5587worker1 # [ 88.264800] nix-grpc-daemon[1290]: time=2026-09-20T12:22:37.165Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5588worker1 # [ 88.268065] nix-grpc-daemon[1290]: time=2026-09-20T12:22:37.166Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5589lb # [ 88.301788] niks3-server[1118]: 2026/09/20 12:22:36 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5590lb # [ 88.324573] niks3-server[1118]: 2026/09/20 12:22:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5591lb # [ 88.329857] niks3-server[1118]: 2026/09/20 12:22:36 INFO Signed narinfos id=28 count=15592worker1 # [ 88.324024] nix-grpc-daemon[1290]: time=2026-09-20T12:22:37.223Z level=INFO msg="Uploading 1 narinfos"5593lb # [ 88.338654] niks3-server[1118]: 2026/09/20 12:22:36 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5594lb # [ 88.343718] niks3-server[1118]: 2026/09/20 12:22:36 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5595lb # [ 88.349209] niks3-server[1118]: 2026/09/20 12:22:36 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5596lb # [ 88.352520] niks3-server[1118]: 2026/09/20 12:22:36 INFO Completed upload id=285597worker1 # [ 88.342852] nix-grpc-daemon[1290]: time=2026-09-20T12:22:37.243Z level=INFO msg="Upload complete. (302ms)"5598lb # [ 88.356779] niks3-server[1118]: 2026/09/20 12:22:36 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5599worker1 # [ 88.351547] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38280 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15600worker1 # [ 88.369072] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:37Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05601worker1 # [ 88.371258] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:37Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=15602worker1 # [ 88.375441] nix-daemon[847]: reaped child process 1313, status = succeeded5603worker1 # [ 88.413348] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:382805604worker1 # [ 88.426990] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:382805605worker1 # [ 88.440689] nix-grpc-daemon[1290]: time=2026-09-20T12:22:37.341Z level=INFO msg="All 1 paths already cached"5606client # copying 1 paths to 'lb:50051'...5607client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5608worker1 # [ 88.448143] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:382805609worker1 # [ 88.456706] nix-daemon[847]: accepted connection from pid 1220, user nix-grpc-daemon (trusted)5610worker1 # [ 88.629626] nix-daemon[847]: accepted connection from pid 1339, user nix-grpc-daemon (trusted)5611worker1 # [ 88.644864] nix-daemon[847]: reaped child process 1342, status = succeeded5612worker1 # [ 88.816089] nix-daemon[847]: accepted connection from pid 1344, user nix-grpc-daemon (trusted)5613worker1 # [ 88.827432] nix-daemon[847]: reaped child process 1347, status = succeeded5614lb # [ 88.850778] niks3-server[1118]: 2026/09/20 12:22:37 INFO Received uploads request method=POST path=/api/pending_closures5615worker1 # [ 89.025523] nix-grpc-daemon[1290]: time=2026-09-20T12:22:37.925Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5616worker1 # [ 89.031333] nix-grpc-daemon[1290]: time=2026-09-20T12:22:37.929Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5617lb # [ 89.051677] niks3-server[1118]: 2026/09/20 12:22:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5618lb # [ 89.056525] niks3-server[1118]: 2026/09/20 12:22:37 INFO Signed narinfos id=29 count=15619worker1 # [ 89.046876] nix-grpc-daemon[1290]: time=2026-09-20T12:22:37.946Z level=INFO msg="Uploading 1 narinfos"5620lb # [ 89.059087] niks3-server[1118]: 2026/09/20 12:22:37 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5621lb # [ 89.067372] niks3-server[1118]: 2026/09/20 12:22:37 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5622lb # [ 89.079612] niks3-server[1118]: 2026/09/20 12:22:37 INFO Completed upload id=295623worker1 # [ 89.070391] nix-grpc-daemon[1290]: time=2026-09-20T12:22:37.970Z level=INFO msg="Upload complete. (599ms)"5624lb # [ 89.084604] niks3-server[1118]: 2026/09/20 12:22:37 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5625worker1 # [ 89.075460] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38280 duration_s=0 paths=1 nar_bytes_in=6405626lb # [ 89.091672] niks3-server[1118]: 2026/09/20 12:22:37 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5627worker1 # [ 89.081725] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:382805628worker1 # [ 89.099944] nix-daemon[847]: accepted connection from pid 1220, user nix-grpc-daemon (trusted)5629worker1 # [ 89.106283] nix-daemon[847]: reaped child process 1337, status = succeeded5630client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5631client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5632worker1 # [ 89.116693] nix-daemon[847]: accepted connection from pid 1220, user nix-grpc-daemon (trusted)5633client # farm-b> LOG-farm-b-on-standby5634worker1 # [ 89.744098] nix-daemon[847]: reaped child process 1351, status = succeeded5635worker1 # [ 89.917357] nix-daemon[847]: accepted connection from pid 1366, user nix-grpc-daemon (trusted)5636worker1 # [ 89.930701] nix-daemon[847]: reaped child process 1369, status = succeeded5637lb # [ 89.956644] niks3-server[1118]: 2026/09/20 12:22:38 INFO Received uploads request method=POST path=/api/pending_closures5638worker1 # [ 89.958694] nix-grpc-daemon[1290]: time=2026-09-20T12:22:38.858Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5639worker1 # [ 89.960428] nix-grpc-daemon[1290]: time=2026-09-20T12:22:38.860Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"5640lb # [ 89.994672] niks3-server[1118]: 2026/09/20 12:22:38 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst5641lb # [ 90.007901] niks3-server[1118]: 2026/09/20 12:22:38 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5642lb # [ 90.017102] niks3-server[1118]: 2026/09/20 12:22:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5643lb # [ 90.023621] niks3-server[1118]: 2026/09/20 12:22:38 INFO Signed narinfos id=30 count=15644lb # [ 90.026452] niks3-server[1118]: 2026/09/20 12:22:38 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls5645worker1 # [ 90.017777] nix-grpc-daemon[1290]: time=2026-09-20T12:22:38.917Z level=INFO msg="Uploading 1 narinfos"5646lb # [ 90.037380] niks3-server[1118]: 2026/09/20 12:22:38 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5647lb # [ 90.040111] niks3-server[1118]: 2026/09/20 12:22:38 INFO Completed upload id=305648worker1 # [ 90.030777] nix-grpc-daemon[1290]: time=2026-09-20T12:22:38.931Z level=INFO msg="Upload complete. (301ms)"5649lb # [ 90.042785] niks3-server[1118]: 2026/09/20 12:22:38 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo5650worker1 # [ 90.040158] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38280 duration_s=0 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=15651worker1 # [ 90.048803] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:38Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=05652worker1 # [ 90.052671] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:38Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=15653worker1 # [ 90.061803] nix-daemon[847]: reaped child process 1349, status = succeeded5654worker1 # [ 90.093561] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:382805655worker1 # [ 90.105894] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:382805656worker1 # [ 90.117700] nix-grpc-daemon[1290]: time=2026-09-20T12:22:39.018Z level=INFO msg="All 2 paths already cached"5657client # copying 1 paths to 'lb:50051'...5658client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...5659worker1 # [ 90.126088] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:382805660worker1 # [ 90.134908] nix-daemon[847]: accepted connection from pid 1220, user nix-grpc-daemon (trusted)5661worker1 # [ 90.294660] nix-daemon[847]: accepted connection from pid 1375, user nix-grpc-daemon (trusted)5662worker1 # [ 90.307486] nix-daemon[847]: reaped child process 1378, status = succeeded5663worker1 # [ 90.471037] nix-daemon[847]: accepted connection from pid 1380, user nix-grpc-daemon (trusted)5664worker1 # [ 90.483356] nix-daemon[847]: reaped child process 1383, status = succeeded5665lb # [ 90.508280] niks3-server[1118]: 2026/09/20 12:22:39 INFO Received uploads request method=POST path=/api/pending_closures5666worker1 # [ 90.507246] nix-grpc-daemon[1290]: time=2026-09-20T12:22:39.407Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5667worker1 # [ 90.508764] nix-grpc-daemon[1290]: time=2026-09-20T12:22:39.409Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"5668lb # [ 90.539884] niks3-server[1118]: 2026/09/20 12:22:39 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst5669lb # [ 90.550773] niks3-server[1118]: 2026/09/20 12:22:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5670lb # [ 90.556108] niks3-server[1118]: 2026/09/20 12:22:39 INFO Signed narinfos id=31 count=15671worker1 # [ 90.549443] nix-grpc-daemon[1290]: time=2026-09-20T12:22:39.449Z level=INFO msg="Uploading 1 narinfos"5672lb # [ 90.562959] niks3-server[1118]: 2026/09/20 12:22:39 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls5673lb # [ 90.573252] niks3-server[1118]: 2026/09/20 12:22:39 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5674lb # [ 90.580856] niks3-server[1118]: 2026/09/20 12:22:39 INFO Completed upload id=315675worker1 # [ 90.573801] nix-grpc-daemon[1290]: time=2026-09-20T12:22:39.474Z level=INFO msg="Upload complete. (426ms)"5676lb # [ 90.586253] niks3-server[1118]: 2026/09/20 12:22:39 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo5677worker1 # [ 90.577482] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38280 duration_s=0 paths=1 nar_bytes_in=7685678worker1 # [ 90.583584] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:382805679worker1 # [ 90.595111] nix-daemon[847]: accepted connection from pid 1220, user nix-grpc-daemon (trusted)5680worker1 # [ 90.598556] nix-daemon[847]: reaped child process 1373, status = succeeded5681worker1 # [ 90.604068] nix-daemon[847]: accepted connection from pid 1220, user nix-grpc-daemon (trusted)5682client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...5683client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5684client # farm-top> LOG-farm-top-on-standby5685worker1 # [ 91.828423] nix-daemon[847]: reaped child process 1387, status = succeeded5686worker1 # [ 91.993830] nix-daemon[847]: accepted connection from pid 1402, user nix-grpc-daemon (trusted)5687worker1 # [ 92.006667] nix-daemon[847]: reaped child process 1405, status = succeeded5688lb # [ 92.033686] niks3-server[1118]: 2026/09/20 12:22:40 INFO Received uploads request method=POST path=/api/pending_closures5689worker1 # [ 92.049604] nix-grpc-daemon[1290]: time=2026-09-20T12:22:40.949Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5690worker1 # [ 92.052066] nix-grpc-daemon[1290]: time=2026-09-20T12:22:40.951Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"5691lb # [ 92.090562] niks3-server[1118]: 2026/09/20 12:22:40 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5692lb # [ 92.098141] niks3-server[1118]: 2026/09/20 12:22:40 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst5693lb # [ 92.107791] niks3-server[1118]: 2026/09/20 12:22:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5694lb # [ 92.113115] niks3-server[1118]: 2026/09/20 12:22:40 INFO Signed narinfos id=32 count=15695worker1 # [ 92.106589] nix-grpc-daemon[1290]: time=2026-09-20T12:22:41.006Z level=INFO msg="Uploading 1 narinfos"5696lb # [ 92.120561] niks3-server[1118]: 2026/09/20 12:22:40 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls5697lb # [ 92.130214] niks3-server[1118]: 2026/09/20 12:22:40 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5698lb # [ 92.135685] niks3-server[1118]: 2026/09/20 12:22:40 INFO Completed upload id=325699worker1 # [ 92.126390] nix-grpc-daemon[1290]: time=2026-09-20T12:22:41.026Z level=INFO msg="Upload complete. (313ms)"5700lb # [ 92.138244] niks3-server[1118]: 2026/09/20 12:22:40 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo5701worker1 # [ 92.134373] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38280 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=15702worker1 # [ 92.147872] nix-daemon[847]: reaped child process 1385, status = succeeded5703client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&2, in 7.13 seconds)5704client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath5705client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath, in 0.21 seconds)5706worker2 # [ 92.477730] systemd[1]: Listening on Nix worker-protocol over gRPC.5707worker2 # [ 92.485214] systemd[1]: Starting scheduler-token.service...5708worker2 # [ 92.551981] systemd[1]: scheduler-token.service: Deactivated successfully.5709worker2 # [ 92.553213] systemd[1]: Finished scheduler-token.service.5710worker2 # [ 92.554658] systemd[1]: scheduler-token.service: Consumed 19ms CPU time over 67ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5711worker2 # [ 92.559923] systemd[1]: Starting Nix worker-protocol over gRPC...5712worker2 # [ 92.712274] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:41Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5713worker2 # [ 92.714278] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:41Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5714worker2 # [ 92.717834] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:41Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15715worker2 # [ 92.724759] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:41Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5716worker2 # [ 92.729847] systemd[1]: Started Nix worker-protocol over gRPC.5717lb: must succeed: curl -sf localhost:9901/clusters5718worker2 # [ 92.744592] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:41Z level=info event=scheduler_connected addr=lb:500515719worker1 # [ 92.735171] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:41Z level=info event=oidc_jwks provider=mock keys=15720worker1 # [ 92.738312] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:41Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:382785721worker1 # [ 92.742520] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:41Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05722lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5723worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5724worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5725lb: must succeed: curl -sf localhost:9901/clusters5726lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5727lb: must succeed: curl -sf localhost:9901/clusters5728lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5729worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5730worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5731(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 13.21 seconds)5732subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5733client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs5734client # [ 93.060424] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs.5735client # Running as unit: rs.service; invocation ID: bdd35d8b8ab1470eac7d4845593e22795736client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs, in 0.07 seconds)5737worker2 # [ 93.295606] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:42Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:361825738worker2 # [ 93.304880] nix-daemon[857]: accepted connection from pid 1187, user nix-grpc-daemon (trusted)5739client # [ 93.288986] nix[1172]: don't know how to build these paths:5740client # [ 93.289907] nix[1172]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5741worker1 # [ 93.307437] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:42Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=05742worker1 # [ 93.311442] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:42Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=15743worker1 # [ 93.351397] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:382805744worker1 # [ 93.370283] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:382805745client # [ 93.373275] nix[1172]: copying 1 paths to 'lb:50051'...5746client # [ 93.377142] nix[1172]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...5747worker1 # [ 93.384608] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:382805748worker1 # [ 93.395381] nix-daemon[847]: accepted connection from pid 1220, user nix-grpc-daemon (trusted)5749worker1 # [ 93.579591] nix-daemon[847]: accepted connection from pid 1418, user nix-grpc-daemon (trusted)5750worker1 # [ 93.592457] nix-daemon[847]: reaped child process 1421, status = succeeded5751worker1 # [ 93.765905] nix-daemon[847]: accepted connection from pid 1423, user nix-grpc-daemon (trusted)5752worker1 # [ 93.777543] nix-daemon[847]: reaped child process 1426, status = succeeded5753lb # [ 93.797772] niks3-server[1118]: 2026/09/20 12:22:42 INFO Received uploads request method=POST path=/api/pending_closures5754worker1 # [ 93.853579] nix-grpc-daemon[1290]: time=2026-09-20T12:22:42.753Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5755worker1 # [ 93.855745] nix-grpc-daemon[1290]: time=2026-09-20T12:22:42.755Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"5756lb # [ 93.890442] niks3-server[1118]: 2026/09/20 12:22:42 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst5757lb # [ 93.901172] niks3-server[1118]: 2026/09/20 12:22:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5758lb # [ 93.905481] niks3-server[1118]: 2026/09/20 12:22:42 INFO Signed narinfos id=33 count=15759worker1 # [ 93.899724] nix-grpc-daemon[1290]: time=2026-09-20T12:22:42.799Z level=INFO msg="Uploading 1 narinfos"5760lb # [ 93.917480] niks3-server[1118]: 2026/09/20 12:22:42 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls5761lb # [ 93.925128] niks3-server[1118]: 2026/09/20 12:22:42 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5762lb # [ 93.930323] niks3-server[1118]: 2026/09/20 12:22:42 INFO Completed upload id=335763worker1 # [ 93.920951] nix-grpc-daemon[1290]: time=2026-09-20T12:22:42.821Z level=WARN msg="Failed to register uploaded object" key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo error="server returned 404: object is not pending upload\n"5764worker1 # [ 93.928725] nix-grpc-daemon[1290]: time=2026-09-20T12:22:42.827Z level=INFO msg="Upload complete. (520ms)"5765worker1 # [ 93.933477] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38280 duration_s=0 paths=1 nar_bytes_in=4165766worker1 # [ 93.938118] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:382805767worker1 # [ 93.951130] nix-daemon[847]: accepted connection from pid 1220, user nix-grpc-daemon (trusted)5768worker1 # [ 93.959273] nix-daemon[847]: reaped child process 1416, status = succeeded5769client # [ 93.960311] nix[1172]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...5770worker1 # [ 93.968529] nix-daemon[847]: accepted connection from pid 1220, user nix-grpc-daemon (trusted)5771lb: must succeed: curl -sf localhost:9901/clusters5772lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5773lb: must succeed: curl -sf localhost:9901/clusters5774lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5775lb: must succeed: curl -sf localhost:9901/clusters5776lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5777lb: must succeed: curl -sf localhost:9901/clusters5778lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5779worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5780worker2 # [ 95.866734] systemd-journald[288]: Received client request to rotate journal, rotating.5781worker2 # [ 95.878548] systemd-journald[288]: Vacuuming done, freed 0B of archived journals from /var/log/journal/3cb733cf531544a5a7d4b6260e5d3f06.5782worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.05 seconds)5783worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5784worker1 # [ 95.769770] systemd[1]: Stopping Nix worker-protocol over gRPC...5785worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.05 seconds)5786??? Warning (UserWarning): sleep(): The 'secs' argument is deprecated. Use 'duration' instead.5787 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 5735788client: must succeed: sleep 5.05789worker1 # [ 96.644360] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:45Z level=info event=draining5790lb # [ 98.031595] envoy[979]: conn peer=[2001:db8:1::1]:56912 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9551 tx=3893 ms=68735791client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5792worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5793worker1 # [ 101.203410] nix-daemon[847]: reaped child process 1430, status = succeeded5794worker1 # [ 101.352802] nix-daemon[847]: accepted connection from pid 1470, user nix-grpc-daemon (trusted)5795worker1 # [ 101.365440] nix-daemon[847]: reaped child process 1473, status = succeeded5796lb # [ 101.386537] niks3-server[1118]: 2026/09/20 12:22:49 INFO Received uploads request method=POST path=/api/pending_closures5797worker1 # [ 101.387329] nix-grpc-daemon[1290]: time=2026-09-20T12:22:50.287Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5798worker1 # [ 101.389052] nix-grpc-daemon[1290]: time=2026-09-20T12:22:50.289Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"5799lb # [ 101.434477] niks3-server[1118]: 2026/09/20 12:22:50 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5800lb # [ 101.440576] niks3-server[1118]: 2026/09/20 12:22:50 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5801lb # [ 101.451987] niks3-server[1118]: 2026/09/20 12:22:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5802worker1 # [ 101.447210] nix-grpc-daemon[1290]: time=2026-09-20T12:22:50.347Z level=INFO msg="Uploading 1 narinfos"5803lb # [ 101.457054] niks3-server[1118]: 2026/09/20 12:22:50 INFO Signed narinfos id=34 count=15804lb # [ 101.462095] niks3-server[1118]: 2026/09/20 12:22:50 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls5805lb # [ 101.471952] niks3-server[1118]: 2026/09/20 12:22:50 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5806lb # [ 101.474525] niks3-server[1118]: 2026/09/20 12:22:50 INFO Completed upload id=345807worker1 # [ 101.465307] nix-grpc-daemon[1290]: time=2026-09-20T12:22:50.365Z level=INFO msg="Upload complete. (279ms)"5808lb # [ 101.476672] niks3-server[1118]: 2026/09/20 12:22:50 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo5809worker1 # [ 101.473496] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38280 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=15810worker1 # [ 101.487431] nix-daemon[847]: reaped child process 1428, status = succeeded5811client # [ 101.504907] systemd[1]: rs.service: Deactivated successfully.5812client # [ 101.508637] systemd[1]: rs.service: Consumed 117ms CPU time over 8.447s wall clock time, 37.1M memory peak, 8.5K incoming IP traffic, 9K outgoing IP traffic.5813worker1 # [ 101.644948] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:50Z level=info event=signal_exit5814worker1 # [ 101.865153] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:50Z level=info event=scheduler_yield5815worker2 # [ 101.891883] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:50Z level=info event=scheduler_restarting addr=lb:500515816lb # [ 101.877142] niks3-server[1118]: 2026/09/20 12:22:50 INFO lead: released remote=192.168.1.3:367985817worker1 # [ 101.872394] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:50Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:382785818worker1 # [ 101.877796] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:50Z level=info event=worker_gone addr=192.168.1.4:500515819worker1 # [ 101.880853] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:50Z level=info event=scheduler_restarting addr=lb:500515820worker1 # [ 101.883775] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:50Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:382785821worker1 # [ 101.887108] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:50Z level=info event=worker_gone addr=192.168.1.3:500515822worker2 # [ 102.092286] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:50Z level=info event=scheduler_connected addr=lb:500515823worker2 # [ 102.095527] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:50Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5824worker1 # [ 102.077769] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:50Z level=info event=scheduler_connected addr=lb:500515825worker1 # [ 102.082567] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:50Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5826worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 1.07 seconds)5827client: waiting for success: ! systemctl is-active rs5828client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds)5829client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5830client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.04 seconds)5831client: must fail: journalctl -u rs | grep -q 'reconnecting'5832client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5833lb: must succeed: curl -sf localhost:9901/clusters5834worker2 # [ 102.296562] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:51Z level=info event=scheduler_connected addr=lb:500515835worker2 # [ 102.299740] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:51Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5836worker1 # [ 102.283364] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:51Z level=info event=scheduler_connected addr=lb:500515837worker1 # [ 102.287897] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:51Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5838lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5839worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5840worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5841worker2 # [ 102.500593] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:51Z level=info event=scheduler_connected addr=lb:500515842worker2 # [ 102.503693] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:51Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5843worker1 # [ 102.491166] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:51Z level=info event=scheduler_connected addr=lb:500515844worker1 # [ 102.494380] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5845worker2 # [ 102.704550] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:51Z level=info event=scheduler_connected addr=lb:500515846worker2 # [ 102.706336] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5847worker1 # [ 102.691729] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:51Z level=info event=scheduler_connected addr=lb:500515848worker1 # [ 102.695710] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5849worker2 # [ 102.733842] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:51Z level=info event=scheduler_take_over5850lb # [ 102.721143] niks3-server[1118]: 2026/09/20 12:22:51 INFO lead: acquired remote=192.168.1.4:546565851worker1 # [ 102.869613] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:51Z level=info event=scheduler_restarting5852worker2 # [ 102.907500] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:51Z level=info event=scheduler_connected addr=lb:500515853worker2 # [ 102.909273] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5854worker1 # [ 102.893147] nix-daemon[847]: reaped child process 1281, status = succeeded5855worker1 # [ 102.905772] nix-grpc-daemon[1220]: ts=2026-09-20T12:22:51Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5856worker2 # [ 103.110575] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:51Z level=info event=scheduler_connected addr=lb:500515857worker2 # [ 103.114348] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5858worker2 # [ 103.315434] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:52Z level=info event=scheduler_connected addr=lb:500515859worker2 # [ 103.319392] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5860lb: must succeed: curl -sf localhost:9901/clusters5861lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5862worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5863worker1 # [ 103.491639] nix-grpc-daemon[1220]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5864worker2 # [ 103.520274] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:52Z level=info event=scheduler_connected addr=lb:500515865worker1 # [ 103.495088] nix-grpc-daemon[1220]: E0000 00:00:1789906972.390681 1263 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5866worker1 # [ 103.499908] nix-grpc-daemon[1220]: E0000 00:00:1789906972.391289 1263 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5867worker2 # [ 103.524851] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5868worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5869worker2 # [ 103.725567] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:52Z level=info event=scheduler_connected addr=lb:500515870worker2 # [ 103.727494] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5871worker2 # [ 103.928461] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:52Z level=info event=scheduler_connected addr=lb:500515872worker2 # [ 103.935499] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:52Z level=info event=oidc_jwks provider=mock keys=15873worker2 # [ 103.936796] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:52Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:454225874worker2 # [ 103.938421] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:52Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05875worker1 # [ 104.399401] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5876worker1 # [ 104.403383] systemd[1]: Stopped Nix worker-protocol over gRPC.5877worker1 # [ 104.405565] systemd[1]: nix-grpc-daemon.service: Consumed 1.416s CPU time over 25.940s wall clock time, 33.6M memory peak, 50K incoming IP traffic, 101.4K outgoing IP traffic.5878worker1 # [ 104.421559] systemd[1]: Starting Nix worker-protocol over gRPC...5879lb: must succeed: curl -sf localhost:9901/clusters5880lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5881worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5882worker1 # [ 104.590510] nix-grpc-daemon[1483]: ts=2026-09-20T12:22:53Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5883worker1 # [ 104.592316] nix-grpc-daemon[1483]: ts=2026-09-20T12:22:53Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5884worker1 # [ 104.595717] nix-grpc-daemon[1483]: ts=2026-09-20T12:22:53Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15885worker1 # [ 104.602995] nix-grpc-daemon[1483]: ts=2026-09-20T12:22:53Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5886worker1 # [ 104.606551] systemd[1]: Started Nix worker-protocol over gRPC.5887worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5888worker1 # [ 104.617661] nix-grpc-daemon[1483]: ts=2026-09-20T12:22:53Z level=info event=scheduler_connected addr=lb:500515889worker2 # [ 104.642120] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:53Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:454225890worker2 # [ 104.643917] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:53Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05891lb: must succeed: curl -sf localhost:9901/clusters5892lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5893worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5894worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5895(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.71 seconds)5896subtest: scheduler restart mid-build: build finishes, no double build5897client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid5898client # [ 105.774452] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid.5899client # Running as unit: mid.service; invocation ID: f2fcca1b166749c3bcbf04f5b924521c5900client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid, in 0.07 seconds)5901worker1 # [ 105.993230] nix-grpc-daemon[1483]: ts=2026-09-20T12:22:54Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:444745902worker1 # [ 106.002706] nix-daemon[847]: accepted connection from pid 1483, user nix-grpc-daemon (trusted)5903client # [ 106.007936] nix[1207]: don't know how to build these paths:5904client # [ 106.008995] nix[1207]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5905worker2 # [ 106.043709] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:54Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05906worker2 # [ 106.045529] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:54Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=15907worker2 # [ 106.086272] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:361825908worker2 # [ 106.096911] nix-grpc-daemon[1187]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 73 ms (attempt 1/5)5909worker2 # [ 106.176418] nix-grpc-daemon[1187]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 16 ms (attempt 2/5)5910worker2 # [ 106.199072] nix-grpc-daemon[1187]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 396 ms (attempt 3/5)5911worker2 # [ 106.602268] nix-grpc-daemon[1187]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 425 ms (attempt 4/5)5912worker2 # [ 107.028591] nix-grpc-daemon[1187]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5913worker2 # [ 107.049778] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:361825914client # [ 107.031420] nix[1207]: copying 1 paths to 'lb:50051'...5915client # [ 107.032862] nix[1207]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...5916worker2 # [ 107.062200] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:361825917worker2 # [ 107.074246] nix-daemon[857]: accepted connection from pid 1187, user nix-grpc-daemon (trusted)5918worker2 # [ 107.085561] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:55Z level=info event=niks3_push_started pid=13065919worker2 # [ 107.250321] nix-grpc-daemon[1306]: time=2026-09-20T12:22:56.045Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""5920worker2 # [ 107.414041] nix-daemon[857]: accepted connection from pid 1319, user nix-grpc-daemon (trusted)5921worker2 # [ 107.425161] nix-daemon[857]: reaped child process 1322, status = succeeded5922worker2 # [ 107.589541] nix-daemon[857]: accepted connection from pid 1324, user nix-grpc-daemon (trusted)5923worker2 # [ 107.601659] nix-daemon[857]: reaped child process 1327, status = succeeded5924lb # [ 107.605096] niks3-server[1118]: 2026/09/20 12:22:56 INFO Received uploads request method=POST path=/api/pending_closures5925worker2 # [ 107.627263] nix-grpc-daemon[1306]: time=2026-09-20T12:22:56.422Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5926worker2 # [ 107.629123] nix-grpc-daemon[1306]: time=2026-09-20T12:22:56.424Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"5927lb # [ 107.636741] niks3-server[1118]: 2026/09/20 12:22:56 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst5928worker2 # [ 107.666246] nix-grpc-daemon[1306]: time=2026-09-20T12:22:56.461Z level=INFO msg="Uploading 1 narinfos"5929lb # [ 107.653059] niks3-server[1118]: 2026/09/20 12:22:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5930lb # [ 107.656097] niks3-server[1118]: 2026/09/20 12:22:56 INFO Signed narinfos id=35 count=15931lb # [ 107.665577] niks3-server[1118]: 2026/09/20 12:22:56 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls5932lb # [ 107.670309] niks3-server[1118]: 2026/09/20 12:22:56 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5933lb # [ 107.675447] niks3-server[1118]: 2026/09/20 12:22:56 INFO Completed upload id=355934worker2 # [ 107.689219] nix-grpc-daemon[1306]: time=2026-09-20T12:22:56.484Z level=INFO msg="Upload complete. (434ms)"5935lb # [ 107.679761] niks3-server[1118]: 2026/09/20 12:22:56 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo5936worker2 # [ 107.691620] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36182 duration_s=0 paths=1 nar_bytes_in=4245937worker2 # [ 107.697489] nix-grpc-daemon[1187]: ts=2026-09-20T12:22:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:361825938worker2 # [ 107.705730] nix-daemon[857]: accepted connection from pid 1187, user nix-grpc-daemon (trusted)5939worker2 # [ 107.710351] nix-daemon[857]: reaped child process 1304, status = succeeded5940worker2 # [ 107.715315] nix-daemon[857]: accepted connection from pid 1187, user nix-grpc-daemon (trusted)5941client # [ 107.684858] nix[1207]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5942lb # [ 108.031724] envoy[979]: conn peer=[2001:db8:1::1]:50110 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2934 tx=1349 ms=82115943lb # [ 108.035403] envoy[979]: conn peer=[2001:db8:1::3]:34630 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=5767 tx=1349 ms=242525944lb: must succeed: curl -sf localhost:9901/clusters5945lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5946lb: must succeed: curl -sf localhost:9901/clusters5947lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5948worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5949worker2 # [ 109.601324] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1187 (nix-grpc-daemon) on client request.5950worker2 # [ 109.603597] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5951lb # [ 109.595482] niks3-server[1118]: 2026/09/20 12:22:58 INFO lead: released remote=192.168.1.4:546565952worker1 # [ 109.590988] nix-grpc-daemon[1483]: ts=2026-09-20T12:22:58Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5953client # [ 109.585431] nix[1207]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5954worker2 # [ 109.620473] systemd[1]: Starting scheduler-token.service...5955client # [ 109.590226] nix[1207]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5956worker2 # [ 109.630843] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5957worker2 # [ 109.632457] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5958worker2 # [ 109.633839] systemd[1]: nix-grpc-daemon.service: Consumed 399ms CPU time over 17.066s wall clock time, 31M memory peak, 17.1K incoming IP traffic, 32.6K outgoing IP traffic.5959lb # [ 109.621462] niks3-server[1118]: 2026/09/20 12:22:58 INFO lead: acquired remote=192.168.1.3:583045960worker1 # [ 109.612428] nix-grpc-daemon[1483]: ts=2026-09-20T12:22:58Z level=info event=scheduler_take_over5961worker2 # [ 109.646452] nix-daemon[857]: reaped child process 1329, status = succeeded5962worker2 # [ 109.648558] nix-daemon[857]: reaped child process 1243, status = succeeded5963worker2 # [ 109.658181] nix-daemon[1331]: unexpected Nix daemon error: error: interrupted by the user5964worker2 # [ 109.663047] nix-daemon[857]: reaped child process 1331, status = failed with exit code 15965worker2 # [ 109.699081] systemd[1]: scheduler-token.service: Deactivated successfully.5966worker2 # [ 109.700724] systemd[1]: Finished scheduler-token.service.5967worker2 # [ 109.702683] systemd[1]: scheduler-token.service: Consumed 17ms CPU time over 80ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5968worker2 # [ 109.704723] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.5969worker2 # [ 109.709129] systemd[1]: Starting scheduler-token.service...5970worker2 # [ 109.763672] systemd[1]: scheduler-token.service: Deactivated successfully.5971worker2 # [ 109.764786] systemd[1]: Finished scheduler-token.service.5972worker2 # [ 109.766390] systemd[1]: scheduler-token.service: Consumed 16ms CPU time over 54ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5973worker2 # [ 109.770952] systemd[1]: Starting Nix worker-protocol over gRPC...5974worker1 # [ 109.791985] nix-grpc-daemon[1483]: ts=2026-09-20T12:22:58Z level=info event=scheduler_connected addr=lb:500515975worker2 # [ 109.897399] nix-grpc-daemon[1373]: ts=2026-09-20T12:22:58Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5976worker2 # [ 109.899821] nix-grpc-daemon[1373]: ts=2026-09-20T12:22:58Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5977worker2 # [ 109.903391] nix-grpc-daemon[1373]: ts=2026-09-20T12:22:58Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15978worker2 # [ 109.909873] nix-grpc-daemon[1373]: ts=2026-09-20T12:22:58Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5979worker2 # [ 109.914135] systemd[1]: Started Nix worker-protocol over gRPC.5980worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.36 seconds)5981client: must succeed: sleep 5.05982worker2 # [ 109.930029] nix-grpc-daemon[1373]: ts=2026-09-20T12:22:58Z level=info event=scheduler_connected addr=lb:500515983worker2 # [ 109.932357] nix-grpc-daemon[1373]: ts=2026-09-20T12:22:58Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5984worker1 # [ 109.907373] nix-grpc-daemon[1483]: ts=2026-09-20T12:22:58Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5985client # [ 110.092382] nix[1207]: scheduler at lb:50051: no healthy upstream, reconnecting5986worker1 # [ 110.108558] nix-grpc-daemon[1483]: ts=2026-09-20T12:22:59Z level=info event=scheduler_connected addr=lb:500515987worker2 # [ 110.133853] nix-grpc-daemon[1373]: ts=2026-09-20T12:22:58Z level=info event=scheduler_connected addr=lb:500515988worker1 # [ 110.112713] nix-grpc-daemon[1483]: ts=2026-09-20T12:22:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5989worker2 # [ 110.137730] nix-grpc-daemon[1373]: ts=2026-09-20T12:22:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5990worker1 # [ 110.313770] nix-grpc-daemon[1483]: ts=2026-09-20T12:22:59Z level=info event=scheduler_connected addr=lb:500515991worker2 # [ 110.338718] nix-grpc-daemon[1373]: ts=2026-09-20T12:22:59Z level=info event=scheduler_connected addr=lb:500515992worker1 # [ 110.317768] nix-grpc-daemon[1483]: ts=2026-09-20T12:22:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5993worker2 # [ 110.342580] nix-grpc-daemon[1373]: ts=2026-09-20T12:22:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5994worker1 # [ 110.518789] nix-grpc-daemon[1483]: ts=2026-09-20T12:22:59Z level=info event=scheduler_connected addr=lb:500515995worker2 # [ 110.543672] nix-grpc-daemon[1373]: ts=2026-09-20T12:22:59Z level=info event=scheduler_connected addr=lb:500515996worker1 # [ 110.522827] nix-grpc-daemon[1483]: ts=2026-09-20T12:22:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5997worker2 # [ 110.547569] nix-grpc-daemon[1373]: ts=2026-09-20T12:22:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5998worker2 # [ 110.748724] nix-grpc-daemon[1373]: ts=2026-09-20T12:22:59Z level=info event=scheduler_connected addr=lb:500515999worker1 # [ 110.724583] nix-grpc-daemon[1483]: ts=2026-09-20T12:22:59Z level=info event=scheduler_connected addr=lb:500516000worker2 # [ 110.751954] nix-grpc-daemon[1373]: ts=2026-09-20T12:22:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6001worker1 # [ 110.728742] nix-grpc-daemon[1483]: ts=2026-09-20T12:22:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6002worker1 # [ 110.926052] nix-grpc-daemon[1483]: ts=2026-09-20T12:22:59Z level=info event=scheduler_connected addr=lb:500516003worker1 # [ 110.928154] nix-grpc-daemon[1483]: ts=2026-09-20T12:22:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6004worker2 # [ 110.953893] nix-grpc-daemon[1373]: ts=2026-09-20T12:22:59Z level=info event=scheduler_connected addr=lb:500516005worker2 # [ 110.957077] nix-grpc-daemon[1373]: ts=2026-09-20T12:22:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6006client # [ 111.095279] nix[1207]: scheduler at lb:50051: no healthy upstream, reconnecting6007worker1 # [ 111.128800] nix-grpc-daemon[1483]: ts=2026-09-20T12:23:00Z level=info event=scheduler_connected addr=lb:500516008worker1 # [ 111.130724] nix-grpc-daemon[1483]: ts=2026-09-20T12:23:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6009worker2 # [ 111.155314] nix-grpc-daemon[1373]: ts=2026-09-20T12:22:59Z level=info event=scheduler_connected addr=lb:500516010worker2 # [ 111.158466] nix-grpc-daemon[1373]: ts=2026-09-20T12:22:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6011worker1 # [ 111.331978] nix-grpc-daemon[1483]: ts=2026-09-20T12:23:00Z level=info event=scheduler_connected addr=lb:500516012worker2 # [ 111.356763] nix-grpc-daemon[1373]: ts=2026-09-20T12:23:00Z level=info event=scheduler_connected addr=lb:500516013worker1 # [ 111.336269] nix-grpc-daemon[1483]: ts=2026-09-20T12:23:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6014worker2 # [ 111.360563] nix-grpc-daemon[1373]: ts=2026-09-20T12:23:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6015worker1 # [ 111.537149] nix-grpc-daemon[1483]: ts=2026-09-20T12:23:00Z level=info event=scheduler_connected addr=lb:500516016worker2 # [ 111.561822] nix-grpc-daemon[1373]: ts=2026-09-20T12:23:00Z level=info event=scheduler_connected addr=lb:500516017worker1 # [ 111.548699] nix-grpc-daemon[1483]: ts=2026-09-20T12:23:00Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:444846018worker1 # [ 111.553727] nix-grpc-daemon[1483]: ts=2026-09-20T12:23:00Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06019worker1 # [ 111.564140] nix-grpc-daemon[1483]: ts=2026-09-20T12:23:00Z level=info event=oidc_jwks provider=mock keys=16020worker1 # [ 111.567721] nix-grpc-daemon[1483]: ts=2026-09-20T12:23:00Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:444846021worker1 # [ 111.571571] nix-grpc-daemon[1483]: ts=2026-09-20T12:23:00Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06022worker1 # [ 113.113081] nix-grpc-daemon[1483]: ts=2026-09-20T12:23:02Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06023worker1 # [ 113.119264] nix-grpc-daemon[1483]: ts=2026-09-20T12:23:02Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=16024worker1 # [ 113.158977] nix-grpc-daemon[1483]: ts=2026-09-20T12:23:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444746025worker1 # [ 113.171595] nix-grpc-daemon[1483]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 83 ms (attempt 1/5)6026worker1 # [ 113.260605] nix-grpc-daemon[1483]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 153 ms (attempt 2/5)6027worker1 # [ 113.419891] nix-grpc-daemon[1483]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 350 ms (attempt 3/5)6028worker1 # [ 113.776264] nix-grpc-daemon[1483]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 28 ms (attempt 4/5)6029worker1 # [ 113.804906] nix-grpc-daemon[1483]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6030worker1 # [ 113.820983] nix-grpc-daemon[1483]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' from 'http://lb:5751'...6031worker1 # [ 113.844620] nix-daemon[847]: accepted connection from pid 1483, user nix-grpc-daemon (trusted)6032client # [ 113.849773] nix[1207]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6033worker1 # [ 113.857649] nix-daemon[847]: accepted connection from pid 1483, user nix-grpc-daemon (trusted)6034client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6035client: waiting for success: ! systemctl is-active mid6036lb # [ 118.032598] envoy[979]: conn peer=[2001:db8:1::4]:46076 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=9471 tx=674 ms=168766037worker1 # [ 205.365733] nix-grpc-daemon[1483]: ts=2026-09-20T12:24:34Z level=info event=niks3_push_started pid=15866038worker1 # [ 205.398994] nix-daemon[847]: reaped child process 1562, status = succeeded6039worker1 # [ 205.560558] nix-grpc-daemon[1586]: time=2026-09-20T12:24:34.459Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""6040worker1 # [ 205.695184] nix-daemon[847]: accepted connection from pid 1599, user nix-grpc-daemon (trusted)6041worker1 # [ 205.707990] nix-daemon[847]: reaped child process 1602, status = succeeded6042lb # [ 205.729229] niks3-server[1118]: 2026/09/20 12:24:34 INFO Received uploads request method=POST path=/api/pending_closures6043worker1 # [ 205.733373] nix-grpc-daemon[1586]: time=2026-09-20T12:24:34.633Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6044worker1 # [ 205.734885] nix-grpc-daemon[1586]: time=2026-09-20T12:24:34.635Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6045lb # [ 205.792599] niks3-server[1118]: 2026/09/20 12:24:34 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6046lb # [ 205.794779] niks3-server[1118]: 2026/09/20 12:24:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign6047lb # [ 205.799176] niks3-server[1118]: 2026/09/20 12:24:34 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6048worker1 # [ 205.793219] nix-grpc-daemon[1586]: time=2026-09-20T12:24:34.693Z level=INFO msg="Uploading 1 narinfos"6049lb # [ 205.802710] niks3-server[1118]: 2026/09/20 12:24:34 INFO Signed narinfos id=36 count=16050lb # [ 205.813502] niks3-server[1118]: 2026/09/20 12:24:34 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete6051lb # [ 205.818992] niks3-server[1118]: 2026/09/20 12:24:34 INFO Completed upload id=366052worker1 # [ 205.809709] nix-grpc-daemon[1586]: time=2026-09-20T12:24:34.710Z level=INFO msg="Upload complete. (249ms)"6053lb # [ 205.821066] niks3-server[1118]: 2026/09/20 12:24:34 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo6054lb # [ 205.825424] niks3-server[1118]: 2026/09/20 12:24:34 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6055worker1 # [ 205.815275] nix-grpc-daemon[1483]: ts=2026-09-20T12:24:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44474 duration_s=92 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16056worker1 # [ 205.828919] nix-daemon[847]: reaped child process 1560, status = succeeded6057client # [ 205.846635] systemd[1]: mid.service: Deactivated successfully.6058client # [ 205.851325] systemd[1]: mid.service: Consumed 268ms CPU time over 1min 40.074s wall clock time, 39.3M memory peak, 4K written to disk, 10K incoming IP traffic, 11.3K outgoing IP traffic.6059client: (finished: waiting for success: ! systemctl is-active mid, in 91.53 seconds)6060client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6061client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.04 seconds)6062(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 101.01 seconds)6063subtest: interrupting the client stops the build on the worker6064client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr6065client # [ 206.758585] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr.6066client # Running as unit: intr.service; invocation ID: de303cd9983944deb1ea9ef4dae68f6b6067client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr, in 0.04 seconds)6068worker2 # [ 206.986747] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:35Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:351246069worker2 # [ 206.998541] nix-daemon[857]: accepted connection from pid 1373, user nix-grpc-daemon (trusted)6070client # [ 206.980778] nix[1672]: don't know how to build these paths:6071client # [ 206.982236] nix[1672]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6072worker1 # [ 206.999781] nix-grpc-daemon[1483]: ts=2026-09-20T12:24:35Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06073worker1 # [ 207.004466] nix-grpc-daemon[1483]: ts=2026-09-20T12:24:35Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=16074worker2 # [ 207.071672] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:351246075worker2 # [ 207.086969] nix-grpc-daemon[1373]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 99 ms (attempt 1/5)6076worker2 # [ 207.191718] nix-grpc-daemon[1373]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 5 ms (attempt 2/5)6077worker2 # [ 207.197440] nix-grpc-daemon[1373]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 237 ms (attempt 3/5)6078worker2 # [ 207.440126] nix-grpc-daemon[1373]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 323 ms (attempt 4/5)6079worker2 # [ 207.769442] nix-grpc-daemon[1373]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6080worker2 # [ 207.787928] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:351246081client # [ 207.767820] nix[1672]: copying 1 paths to 'lb:50051'...6082client # [ 207.771535] nix[1672]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6083worker2 # [ 207.801997] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:351246084worker2 # [ 207.810345] nix-daemon[857]: accepted connection from pid 1373, user nix-grpc-daemon (trusted)6085worker2 # [ 207.831211] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:36Z level=info event=niks3_push_started pid=14416086worker2 # [ 208.020232] nix-grpc-daemon[1441]: time=2026-09-20T12:24:36.814Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""6087lb # [ 208.039140] envoy[979]: conn peer=[2001:db8:1::1]:36312 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4192 tx=1782 ms=998426088worker2 # [ 208.175670] nix-daemon[857]: accepted connection from pid 1458, user nix-grpc-daemon (trusted)6089worker2 # [ 208.187351] nix-daemon[857]: reaped child process 1461, status = succeeded6090worker2 # [ 208.330078] nix-daemon[857]: accepted connection from pid 1463, user nix-grpc-daemon (trusted)6091worker2 # [ 208.341927] nix-daemon[857]: reaped child process 1466, status = succeeded6092lb # [ 208.339876] niks3-server[1118]: 2026/09/20 12:24:36 INFO Received uploads request method=POST path=/api/pending_closures6093worker2 # [ 208.365883] nix-grpc-daemon[1441]: time=2026-09-20T12:24:37.160Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6094worker2 # [ 208.367356] nix-grpc-daemon[1441]: time=2026-09-20T12:24:37.160Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6095lb # [ 208.384900] niks3-server[1118]: 2026/09/20 12:24:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign6096lb # [ 208.390817] niks3-server[1118]: 2026/09/20 12:24:36 INFO Signed narinfos id=37 count=16097worker2 # [ 208.408154] nix-grpc-daemon[1441]: time=2026-09-20T12:24:37.203Z level=INFO msg="Uploading 1 narinfos"6098lb # [ 208.405542] niks3-server[1118]: 2026/09/20 12:24:36 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete6099lb # [ 208.459720] niks3-server[1118]: 2026/09/20 12:24:37 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6100lb # [ 208.472666] niks3-server[1118]: 2026/09/20 12:24:37 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6101lb # [ 208.482803] niks3-server[1118]: 2026/09/20 12:24:37 INFO Completed upload id=376102worker2 # [ 208.496456] nix-grpc-daemon[1441]: time=2026-09-20T12:24:37.291Z level=INFO msg="Upload complete. (471ms)"6103worker2 # [ 208.502613] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35124 duration_s=0 paths=1 nar_bytes_in=4246104lb # [ 208.491635] niks3-server[1118]: 2026/09/20 12:24:37 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6105worker2 # [ 208.508867] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:351246106worker2 # [ 208.522129] nix-daemon[857]: accepted connection from pid 1373, user nix-grpc-daemon (trusted)6107worker2 # [ 208.530553] nix-daemon[857]: reaped child process 1439, status = succeeded6108worker2 # [ 208.540673] nix-daemon[857]: accepted connection from pid 1373, user nix-grpc-daemon (trusted)6109client # [ 208.509955] nix[1672]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6110client: must succeed: systemctl kill -s INT intr6111client # [ 210.295463] systemd[1]: intr.service: Sent signal SIGINT to main process 1672 (nix) on client request.6112client # [ 210.302871] nix[1672]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6113client: (finished: must succeed: systemctl kill -s INT intr, in 0.05 seconds)6114client # [ 210.310987] nix[1672]: error: interrupted by the user6115worker2 # [ 210.362208] nix-daemon[1470]: unexpected Nix daemon error: error: interrupted by the user6116worker2 # [ 210.365308] nix-daemon[857]: reaped child process 1468, status = succeeded6117worker2 # [ 210.368167] nix-daemon[857]: reaped child process 1470, status = failed with exit code 16118client # [ 210.339498] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6119client # [ 210.340954] systemd[1]: intr.service: Failed with result 'exit-code'.6120client # [ 210.342933] systemd[1]: intr.service: Consumed 119ms CPU time over 3.580s wall clock time, 37.1M memory peak, 7.6K incoming IP traffic, 8.7K outgoing IP traffic.6121(finished: subtest: interrupting the client stops the build on the worker, in 3.68 seconds)6122subtest: a deploy mid-build drains: the build finishes, the new generation takes over6123client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw6124client # [ 210.443475] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw.6125client # Running as unit: sw.service; invocation ID: bd8f0bd1f90f4f13973ec4d4372662eb6126client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw, in 0.04 seconds)6127worker2 # [ 210.675836] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:39Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:351246128client # [ 210.646913] nix[1691]: don't know how to build these paths:6129client # [ 210.647859] nix[1691]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6130worker1 # [ 210.666584] nix-grpc-daemon[1483]: ts=2026-09-20T12:24:39Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06131worker1 # [ 210.670810] nix-grpc-daemon[1483]: ts=2026-09-20T12:24:39Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=16132worker1 # [ 210.709907] nix-grpc-daemon[1483]: ts=2026-09-20T12:24:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444746133worker1 # [ 210.727401] nix-grpc-daemon[1483]: ts=2026-09-20T12:24:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:444746134client # [ 210.730184] nix[1691]: copying 1 paths to 'lb:50051'...6135client # [ 210.734188] nix[1691]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6136worker1 # [ 210.740919] nix-grpc-daemon[1483]: ts=2026-09-20T12:24:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:444746137worker1 # [ 210.749669] nix-daemon[847]: accepted connection from pid 1483, user nix-grpc-daemon (trusted)6138worker1 # [ 210.925733] nix-daemon[847]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)6139worker1 # [ 210.939361] nix-daemon[847]: reaped child process 1639, status = succeeded6140worker1 # [ 211.095713] nix-daemon[847]: accepted connection from pid 1641, user nix-grpc-daemon (trusted)6141worker1 # [ 211.108168] nix-daemon[847]: reaped child process 1644, status = succeeded6142lb # [ 211.132715] niks3-server[1118]: 2026/09/20 12:24:39 INFO Received uploads request method=POST path=/api/pending_closures6143worker1 # [ 211.128809] nix-grpc-daemon[1586]: time=2026-09-20T12:24:40.028Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6144worker1 # [ 211.130260] nix-grpc-daemon[1586]: time=2026-09-20T12:24:40.028Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6145lb # [ 211.147500] niks3-server[1118]: 2026/09/20 12:24:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6146lb # [ 211.149135] niks3-server[1118]: 2026/09/20 12:24:39 INFO Signed narinfos id=38 count=16147worker1 # [ 211.140938] nix-grpc-daemon[1586]: time=2026-09-20T12:24:40.041Z level=INFO msg="Uploading 1 narinfos"6148lb # [ 211.153334] niks3-server[1118]: 2026/09/20 12:24:39 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6149lb # [ 211.156156] niks3-server[1118]: 2026/09/20 12:24:39 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6150lb # [ 211.161786] niks3-server[1118]: 2026/09/20 12:24:39 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6151lb # [ 211.164238] niks3-server[1118]: 2026/09/20 12:24:39 INFO Completed upload id=386152worker1 # [ 211.154732] nix-grpc-daemon[1586]: time=2026-09-20T12:24:40.055Z level=INFO msg="Upload complete. (390ms)"6153lb # [ 211.166394] niks3-server[1118]: 2026/09/20 12:24:39 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6154worker1 # [ 211.156844] nix-grpc-daemon[1483]: ts=2026-09-20T12:24:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44474 duration_s=0 paths=1 nar_bytes_in=4166155worker1 # [ 211.161485] nix-grpc-daemon[1483]: ts=2026-09-20T12:24:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444746156worker1 # [ 211.167264] nix-daemon[847]: accepted connection from pid 1483, user nix-grpc-daemon (trusted)6157worker1 # [ 211.169534] nix-daemon[847]: reaped child process 1634, status = succeeded6158worker1 # [ 211.174930] nix-daemon[847]: accepted connection from pid 1483, user nix-grpc-daemon (trusted)6159client # [ 211.167449] nix[1691]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6160worker1: must succeed: systemctl reload nix-grpc-daemon.service6161worker1 # [ 212.943981] systemd[1]: Reloading Nix worker-protocol over gRPC...6162worker1 # [ 212.983507] systemd[1]: Reloaded Nix worker-protocol over gRPC.6163worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.08 seconds)6164worker1: must succeed: pgrep -f 'read -t [9]0 x'6165worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.04 seconds)6166client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&26167worker1 # [ 213.270060] nix-grpc-daemon[1483]: ts=2026-09-20T12:24:42Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:444746168client # don't know how to build these paths:6169client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6170worker2 # [ 213.302503] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:351246171worker1 # [ 213.296728] nix-grpc-daemon[1483]: ts=2026-09-20T12:24:42Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06172worker1 # [ 213.298456] nix-grpc-daemon[1483]: ts=2026-09-20T12:24:42Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=16173worker2 # [ 213.363680] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:351246174worker2 # [ 213.372898] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:351246175client # copying 1 paths to 'lb:50051'...6176worker2 # [ 213.382364] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:351246177client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6178worker2 # [ 213.400884] nix-daemon[857]: accepted connection from pid 1373, user nix-grpc-daemon (trusted)6179worker2 # [ 213.539334] nix-daemon[857]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)6180worker2 # [ 213.551137] nix-daemon[857]: reaped child process 1519, status = succeeded6181worker1 # [ 213.647163] nix-grpc-daemon[1483]: ts=2026-09-20T12:24:42Z level=info event=draining6182worker2 # [ 213.688306] nix-daemon[857]: accepted connection from pid 1521, user nix-grpc-daemon (trusted)6183worker2 # [ 213.700566] nix-daemon[857]: reaped child process 1524, status = succeeded6184lb # [ 213.699730] niks3-server[1118]: 2026/09/20 12:24:42 INFO Received uploads request method=POST path=/api/pending_closures6185worker2 # [ 213.721681] nix-grpc-daemon[1441]: time=2026-09-20T12:24:42.517Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6186worker2 # [ 213.723238] nix-grpc-daemon[1441]: time=2026-09-20T12:24:42.518Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6187lb # [ 213.733220] niks3-server[1118]: 2026/09/20 12:24:42 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6188lb # [ 213.742503] niks3-server[1118]: 2026/09/20 12:24:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6189lb # [ 213.748235] niks3-server[1118]: 2026/09/20 12:24:42 INFO Signed narinfos id=39 count=16190worker2 # [ 213.765228] nix-grpc-daemon[1441]: time=2026-09-20T12:24:42.560Z level=INFO msg="Uploading 1 narinfos"6191lb # [ 213.757319] niks3-server[1118]: 2026/09/20 12:24:42 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6192lb # [ 213.769108] niks3-server[1118]: 2026/09/20 12:24:42 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6193lb # [ 213.773978] niks3-server[1118]: 2026/09/20 12:24:42 INFO Completed upload id=396194worker2 # [ 213.789825] nix-grpc-daemon[1441]: time=2026-09-20T12:24:42.585Z level=INFO msg="Upload complete. (382ms)"6195lb # [ 213.777525] niks3-server[1118]: 2026/09/20 12:24:42 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6196worker2 # [ 213.793259] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35124 duration_s=0 paths=1 nar_bytes_in=5286197worker2 # [ 213.798975] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:351246198worker2 # [ 213.807865] nix-daemon[857]: accepted connection from pid 1373, user nix-grpc-daemon (trusted)6199worker2 # [ 213.815141] nix-daemon[857]: reaped child process 1514, status = succeeded6200worker2 # [ 213.820119] nix-daemon[857]: accepted connection from pid 1373, user nix-grpc-daemon (trusted)6201client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6202client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6203worker2 # [ 215.232806] nix-daemon[857]: reaped child process 1528, status = succeeded6204client # farm-a> LOG-farm-a-during-drain6205worker2 # [ 215.374359] nix-daemon[857]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)6206worker2 # [ 215.385631] nix-daemon[857]: reaped child process 1546, status = succeeded6207lb # [ 215.386586] niks3-server[1118]: 2026/09/20 12:24:43 INFO Received uploads request method=POST path=/api/pending_closures6208worker2 # [ 215.410618] nix-grpc-daemon[1441]: time=2026-09-20T12:24:44.205Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6209worker2 # [ 215.412188] nix-grpc-daemon[1441]: time=2026-09-20T12:24:44.207Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6210lb # [ 215.424100] niks3-server[1118]: 2026/09/20 12:24:43 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6211lb # [ 215.436791] niks3-server[1118]: 2026/09/20 12:24:44 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6212lb # [ 215.447729] niks3-server[1118]: 2026/09/20 12:24:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6213lb # [ 215.451241] niks3-server[1118]: 2026/09/20 12:24:44 INFO Signed narinfos id=40 count=16214worker2 # [ 215.467811] nix-grpc-daemon[1441]: time=2026-09-20T12:24:44.262Z level=INFO msg="Uploading 1 narinfos"6215lb # [ 215.455507] niks3-server[1118]: 2026/09/20 12:24:44 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6216lb # [ 215.464238] niks3-server[1118]: 2026/09/20 12:24:44 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6217lb # [ 215.467886] niks3-server[1118]: 2026/09/20 12:24:44 INFO Completed upload id=406218worker2 # [ 215.481640] nix-grpc-daemon[1441]: time=2026-09-20T12:24:44.277Z level=INFO msg="Upload complete. (254ms)"6219lb # [ 215.470664] niks3-server[1118]: 2026/09/20 12:24:44 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6220worker2 # [ 215.484077] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35124 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16221worker1 # [ 215.466538] nix-grpc-daemon[1483]: ts=2026-09-20T12:24:44Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06222worker2 # [ 215.490980] nix-daemon[857]: reaped child process 1526, status = succeeded6223worker1 # [ 215.468460] nix-grpc-daemon[1483]: ts=2026-09-20T12:24:44Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=16224worker2 # [ 215.532918] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:351246225worker2 # [ 215.541355] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:351246226worker2 # [ 215.551076] nix-grpc-daemon[1441]: time=2026-09-20T12:24:44.346Z level=INFO msg="All 1 paths already cached"6227client # copying 1 paths to 'lb:50051'...6228worker2 # [ 215.564776] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:351246229client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6230worker2 # [ 215.584162] nix-daemon[857]: accepted connection from pid 1373, user nix-grpc-daemon (trusted)6231worker2 # [ 215.750256] nix-daemon[857]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6232worker2 # [ 215.763108] nix-daemon[857]: reaped child process 1553, status = succeeded6233worker2 # [ 215.916173] nix-daemon[857]: accepted connection from pid 1555, user nix-grpc-daemon (trusted)6234worker2 # [ 215.927961] nix-daemon[857]: reaped child process 1558, status = succeeded6235lb # [ 215.928350] niks3-server[1118]: 2026/09/20 12:24:44 INFO Received uploads request method=POST path=/api/pending_closures6236worker2 # [ 215.950275] nix-grpc-daemon[1441]: time=2026-09-20T12:24:44.745Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6237worker2 # [ 215.951744] nix-grpc-daemon[1441]: time=2026-09-20T12:24:44.745Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6238lb # [ 215.952580] niks3-server[1118]: 2026/09/20 12:24:44 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6239lb # [ 215.963369] niks3-server[1118]: 2026/09/20 12:24:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6240lb # [ 215.967826] niks3-server[1118]: 2026/09/20 12:24:44 INFO Signed narinfos id=41 count=16241worker2 # [ 215.984578] nix-grpc-daemon[1441]: time=2026-09-20T12:24:44.779Z level=INFO msg="Uploading 1 narinfos"6242lb # [ 215.977109] niks3-server[1118]: 2026/09/20 12:24:44 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6243lb # [ 215.987649] niks3-server[1118]: 2026/09/20 12:24:44 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6244worker2 # [ 216.007739] nix-grpc-daemon[1441]: time=2026-09-20T12:24:44.803Z level=INFO msg="Upload complete. (411ms)"6245lb # [ 215.994079] niks3-server[1118]: 2026/09/20 12:24:44 INFO Completed upload id=416246worker2 # [ 216.009874] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35124 duration_s=0 paths=1 nar_bytes_in=6406247lb # [ 215.997805] niks3-server[1118]: 2026/09/20 12:24:44 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6248worker2 # [ 216.015522] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:351246249worker2 # [ 216.024614] nix-daemon[857]: accepted connection from pid 1373, user nix-grpc-daemon (trusted)6250worker2 # [ 216.027274] nix-daemon[857]: reaped child process 1548, status = succeeded6251client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6252worker2 # [ 216.032163] nix-daemon[857]: accepted connection from pid 1373, user nix-grpc-daemon (trusted)6253client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6254client # farm-b> LOG-farm-b-during-drain6255worker2 # [ 217.045400] nix-daemon[857]: reaped child process 1562, status = succeeded6256worker2 # [ 217.182678] nix-daemon[857]: accepted connection from pid 1577, user nix-grpc-daemon (trusted)6257worker2 # [ 217.194835] nix-daemon[857]: reaped child process 1580, status = succeeded6258lb # [ 217.196217] niks3-server[1118]: 2026/09/20 12:24:45 INFO Received uploads request method=POST path=/api/pending_closures6259worker2 # [ 217.214597] nix-grpc-daemon[1441]: time=2026-09-20T12:24:46.009Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6260worker2 # [ 217.216106] nix-grpc-daemon[1441]: time=2026-09-20T12:24:46.011Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6261lb # [ 217.217818] niks3-server[1118]: 2026/09/20 12:24:45 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6262lb # [ 217.219836] niks3-server[1118]: 2026/09/20 12:24:45 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6263lb # [ 217.222057] niks3-server[1118]: 2026/09/20 12:24:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6264lb # [ 217.223812] niks3-server[1118]: 2026/09/20 12:24:45 INFO Signed narinfos id=42 count=16265worker2 # [ 217.238859] nix-grpc-daemon[1441]: time=2026-09-20T12:24:46.034Z level=INFO msg="Uploading 1 narinfos"6266lb # [ 217.227451] niks3-server[1118]: 2026/09/20 12:24:45 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6267lb # [ 217.231714] niks3-server[1118]: 2026/09/20 12:24:45 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6268lb # [ 217.235285] niks3-server[1118]: 2026/09/20 12:24:45 INFO Completed upload id=426269worker2 # [ 217.248976] nix-grpc-daemon[1441]: time=2026-09-20T12:24:46.044Z level=INFO msg="Upload complete. (210ms)"6270lb # [ 217.236478] niks3-server[1118]: 2026/09/20 12:24:45 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6271worker2 # [ 217.251548] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35124 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16272worker1 # [ 217.234581] nix-grpc-daemon[1483]: ts=2026-09-20T12:24:46Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06273worker2 # [ 217.259851] nix-daemon[857]: reaped child process 1560, status = succeeded6274worker1 # [ 217.237331] nix-grpc-daemon[1483]: ts=2026-09-20T12:24:46Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=16275worker2 # [ 217.300913] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:351246276worker2 # [ 217.310282] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:351246277worker2 # [ 217.319210] nix-grpc-daemon[1441]: time=2026-09-20T12:24:46.114Z level=INFO msg="All 2 paths already cached"6278client # copying 1 paths to 'lb:50051'...6279client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6280worker2 # [ 217.325865] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:351246281worker2 # [ 217.334127] nix-daemon[857]: accepted connection from pid 1373, user nix-grpc-daemon (trusted)6282worker2 # [ 217.484055] nix-daemon[857]: accepted connection from pid 1584, user nix-grpc-daemon (trusted)6283worker2 # [ 217.495871] nix-daemon[857]: reaped child process 1587, status = succeeded6284worker2 # [ 217.645539] nix-daemon[857]: accepted connection from pid 1589, user nix-grpc-daemon (trusted)6285worker2 # [ 217.657091] nix-daemon[857]: reaped child process 1592, status = succeeded6286lb # [ 217.656813] niks3-server[1118]: 2026/09/20 12:24:46 INFO Received uploads request method=POST path=/api/pending_closures6287worker2 # [ 217.676716] nix-grpc-daemon[1441]: time=2026-09-20T12:24:46.472Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6288worker2 # [ 217.678594] nix-grpc-daemon[1441]: time=2026-09-20T12:24:46.473Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6289lb # [ 217.678049] niks3-server[1118]: 2026/09/20 12:24:46 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6290lb # [ 217.687782] niks3-server[1118]: 2026/09/20 12:24:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6291lb # [ 217.691620] niks3-server[1118]: 2026/09/20 12:24:46 INFO Signed narinfos id=43 count=16292worker2 # [ 217.707666] nix-grpc-daemon[1441]: time=2026-09-20T12:24:46.503Z level=INFO msg="Uploading 1 narinfos"6293lb # [ 217.695545] niks3-server[1118]: 2026/09/20 12:24:46 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6294lb # [ 217.706204] niks3-server[1118]: 2026/09/20 12:24:46 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6295worker2 # [ 217.727064] nix-grpc-daemon[1441]: time=2026-09-20T12:24:46.522Z level=INFO msg="Upload complete. (384ms)"6296lb # [ 217.713479] niks3-server[1118]: 2026/09/20 12:24:46 INFO Completed upload id=436297worker2 # [ 217.729046] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35124 duration_s=0 paths=1 nar_bytes_in=7686298lb # [ 217.715929] niks3-server[1118]: 2026/09/20 12:24:46 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6299worker2 # [ 217.732055] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:351246300worker2 # [ 217.737277] nix-daemon[857]: accepted connection from pid 1373, user nix-grpc-daemon (trusted)6301worker2 # [ 217.742084] nix-daemon[857]: reaped child process 1582, status = succeeded6302client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6303worker2 # [ 217.747163] nix-daemon[857]: accepted connection from pid 1373, user nix-grpc-daemon (trusted)6304client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6305lb # [ 218.039079] envoy[979]: conn peer=[2001:db8:1::1]:33300 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2937 tx=732 ms=33656306client # farm-top> LOG-farm-top-during-drain6307worker2 # [ 218.560927] nix-daemon[857]: reaped child process 1596, status = succeeded6308worker2 # [ 218.662171] nix-daemon[857]: accepted connection from pid 1611, user nix-grpc-daemon (trusted)6309worker2 # [ 218.674157] nix-daemon[857]: reaped child process 1614, status = succeeded6310lb # [ 218.670317] niks3-server[1118]: 2026/09/20 12:24:47 INFO Received uploads request method=POST path=/api/pending_closures6311worker2 # [ 218.693267] nix-grpc-daemon[1441]: time=2026-09-20T12:24:47.487Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6312worker2 # [ 218.694628] nix-grpc-daemon[1441]: time=2026-09-20T12:24:47.487Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6313lb # [ 218.704565] niks3-server[1118]: 2026/09/20 12:24:47 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6314lb # [ 218.710512] niks3-server[1118]: 2026/09/20 12:24:47 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6315lb # [ 218.723601] niks3-server[1118]: 2026/09/20 12:24:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6316worker2 # [ 218.742361] nix-grpc-daemon[1441]: time=2026-09-20T12:24:47.537Z level=INFO msg="Uploading 1 narinfos"6317lb # [ 218.729683] niks3-server[1118]: 2026/09/20 12:24:47 INFO Signed narinfos id=44 count=16318lb # [ 218.733943] niks3-server[1118]: 2026/09/20 12:24:47 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6319lb # [ 218.743614] niks3-server[1118]: 2026/09/20 12:24:47 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6320lb # [ 218.749565] niks3-server[1118]: 2026/09/20 12:24:47 INFO Completed upload id=446321worker2 # [ 218.763342] nix-grpc-daemon[1441]: time=2026-09-20T12:24:47.558Z level=INFO msg="Upload complete. (211ms)"6322lb # [ 218.751327] niks3-server[1118]: 2026/09/20 12:24:47 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6323worker2 # [ 218.766292] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35124 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16324worker2 # [ 218.773984] nix-daemon[857]: reaped child process 1594, status = succeeded6325client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&2, in 5.76 seconds)6326client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath6327client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath, in 0.19 seconds)6328worker1: must succeed: pkill -f 'read -t [9]0 x'6329worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6330client: waiting for success: ! systemctl is-active sw6331worker1 # [ 219.210852] nix-daemon[847]: reaped child process 1648, status = succeeded6332worker1 # [ 219.359647] nix-daemon[847]: accepted connection from pid 1695, user nix-grpc-daemon (trusted)6333worker1 # [ 219.373453] nix-daemon[847]: reaped child process 1698, status = succeeded6334lb # [ 219.396916] niks3-server[1118]: 2026/09/20 12:24:47 INFO Received uploads request method=POST path=/api/pending_closures6335worker1 # [ 219.396477] nix-grpc-daemon[1586]: time=2026-09-20T12:24:48.296Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6336worker1 # [ 219.397976] nix-grpc-daemon[1586]: time=2026-09-20T12:24:48.298Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6337lb # [ 219.436441] niks3-server[1118]: 2026/09/20 12:24:48 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6338lb # [ 219.441508] niks3-server[1118]: 2026/09/20 12:24:48 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6339lb # [ 219.451690] niks3-server[1118]: 2026/09/20 12:24:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6340lb # [ 219.454770] niks3-server[1118]: 2026/09/20 12:24:48 INFO Signed narinfos id=45 count=16341worker1 # [ 219.448203] nix-grpc-daemon[1586]: time=2026-09-20T12:24:48.348Z level=INFO msg="Uploading 1 narinfos"6342lb # [ 219.460532] niks3-server[1118]: 2026/09/20 12:24:48 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6343lb # [ 219.470447] niks3-server[1118]: 2026/09/20 12:24:48 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6344lb # [ 219.473031] niks3-server[1118]: 2026/09/20 12:24:48 INFO Completed upload id=456345lb # [ 219.474932] niks3-server[1118]: 2026/09/20 12:24:48 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6346worker1 # [ 219.464992] nix-grpc-daemon[1586]: time=2026-09-20T12:24:48.365Z level=INFO msg="Upload complete. (275ms)"6347worker1 # [ 219.472684] nix-grpc-daemon[1483]: ts=2026-09-20T12:24:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44474 duration_s=8 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16348worker1 # [ 219.486254] nix-daemon[847]: reaped child process 1646, status = succeeded6349client # [ 219.508170] systemd[1]: sw.service: Deactivated successfully.6350client # [ 219.510657] systemd[1]: sw.service: Consumed 123ms CPU time over 9.064s wall clock time, 39.1M memory peak, 8.6K incoming IP traffic, 9K outgoing IP traffic.6351worker1 # [ 219.649188] nix-grpc-daemon[1483]: ts=2026-09-20T12:24:48Z level=info event=signal_exit6352worker1 # [ 219.709942] nix-grpc-daemon[1483]: ts=2026-09-20T12:24:48Z level=info event=scheduler_yield6353worker2 # [ 219.735195] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:48Z level=info event=scheduler_restarting addr=lb:500516354lb # [ 219.724296] niks3-server[1118]: 2026/09/20 12:24:48 INFO lead: released remote=192.168.1.3:583046355worker1 # [ 219.715265] nix-grpc-daemon[1483]: ts=2026-09-20T12:24:48Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:444846356worker1 # [ 219.719656] nix-grpc-daemon[1483]: ts=2026-09-20T12:24:48Z level=info event=worker_gone addr=192.168.1.4:500516357worker1 # [ 219.722638] nix-grpc-daemon[1483]: ts=2026-09-20T12:24:48Z level=info event=scheduler_restarting addr=lb:500516358worker1 # [ 219.726488] nix-grpc-daemon[1483]: ts=2026-09-20T12:24:48Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:444846359worker1 # [ 219.729798] nix-grpc-daemon[1483]: ts=2026-09-20T12:24:48Z level=info event=worker_gone addr=192.168.1.3:500516360lb # [ 219.907259] niks3-server[1118]: 2026/09/20 12:24:48 INFO lead: acquired remote=192.168.1.4:369886361worker2 # [ 219.922376] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:48Z level=info event=scheduler_take_over6362worker2 # [ 219.936316] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:48Z level=info event=scheduler_connected addr=lb:500516363worker1 # [ 219.915705] nix-grpc-daemon[1483]: ts=2026-09-20T12:24:48Z level=info event=scheduler_connected addr=lb:500516364worker2 # [ 219.940169] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:48Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6365worker1 # [ 219.920489] nix-grpc-daemon[1483]: ts=2026-09-20T12:24:48Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6366client: (finished: waiting for success: ! systemctl is-active sw, in 1.09 seconds)6367client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6368worker2 # [ 220.140976] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:48Z level=info event=scheduler_connected addr=lb:500516369worker1 # [ 220.121827] nix-grpc-daemon[1483]: ts=2026-09-20T12:24:49Z level=info event=scheduler_connected addr=lb:500516370worker1 # [ 220.126797] nix-grpc-daemon[1483]: ts=2026-09-20T12:24:49Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6371worker2 # [ 220.159165] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:48Z level=info event=oidc_jwks provider=mock keys=16372worker2 # [ 220.163146] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:48Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:573846373worker2 # [ 220.166929] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:48Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06374client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.04 seconds)6375worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6376worker1 # [ 220.327658] nix-grpc-daemon[1483]: ts=2026-09-20T12:24:49Z level=info event=scheduler_connected addr=lb:500516377worker2 # [ 220.353664] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:49Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:573846378worker2 # [ 220.357356] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:49Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=0 running=06379worker2 # [ 220.742834] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:49Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:573846380worker1 # [ 220.720568] nix-grpc-daemon[1483]: ts=2026-09-20T12:24:49Z level=info event=scheduler_restarting6381worker2 # [ 220.746314] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:49Z level=info event=worker_gone addr=192.168.1.3:500516382worker1 # [ 220.732273] nix-grpc-daemon[1483]: ts=2026-09-20T12:24:49Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6383worker1 # [ 220.749278] nix-daemon[847]: reaped child process 1536, status = succeeded6384worker1 # [ 221.577153] nix-grpc-daemon[1483]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6385worker1 # [ 221.580647] nix-grpc-daemon[1483]: E0000 00:00:1789907090.477243 1526 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true6386worker1 # [ 221.585415] nix-grpc-daemon[1483]: E0000 00:00:1789907090.477445 1526 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6387worker1 # [ 222.744701] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6388worker1 # [ 222.747518] systemd[1]: nix-grpc-daemon.service: Consumed 969ms CPU time over 1min 58.321s wall clock time, 31.1M memory peak, 29K incoming IP traffic, 38.8K outgoing IP traffic.6389worker1 # [ 222.763680] systemd[1]: Starting Nix worker-protocol over gRPC...6390worker1 # [ 222.905471] nix-grpc-daemon[1712]: ts=2026-09-20T12:24:51Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6391worker1 # [ 222.907732] nix-grpc-daemon[1712]: ts=2026-09-20T12:24:51Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6392worker1 # [ 222.911601] nix-grpc-daemon[1712]: ts=2026-09-20T12:24:51Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16393worker1 # [ 222.918731] nix-grpc-daemon[1712]: ts=2026-09-20T12:24:51Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6394worker1 # [ 222.922355] systemd[1]: Started Nix worker-protocol over gRPC.6395worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.78 seconds)6396lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6397lb: must succeed: curl -sf localhost:9901/clusters6398worker1 # [ 222.935745] nix-grpc-daemon[1712]: ts=2026-09-20T12:24:51Z level=info event=scheduler_connected addr=lb:500516399worker2 # [ 222.961854] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:51Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:573846400worker2 # [ 222.965529] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:51Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06401lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6402lb: must succeed: curl -sf localhost:9901/clusters6403lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6404lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.11 seconds)6405lb: must succeed: curl -sf localhost:9901/clusters6406lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6407worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6408worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)6409(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 13.70 seconds)6410subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6411client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16412worker1 # [ 224.334898] nix-grpc-daemon[1712]: ts=2026-09-20T12:24:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:426366413worker1 # [ 224.347238] nix-daemon[847]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)6414worker2 # [ 224.385956] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:351246415worker2 # [ 224.428605] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:53Z level=debug event=want client=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06416worker2 # [ 224.432818] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:53Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16417worker2 # [ 224.437115] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:53Z level=debug event=want client=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06418worker2 # [ 224.441079] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:53Z level=debug event=want client=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06419worker1 # [ 224.450804] nix-grpc-daemon[1712]: ts=2026-09-20T12:24:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:426366420worker1 # [ 224.465516] nix-grpc-daemon[1712]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 97 ms (attempt 1/5)6421worker1 # [ 224.569145] nix-grpc-daemon[1712]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 56 ms (attempt 2/5)6422worker1 # [ 224.631962] nix-grpc-daemon[1712]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 381 ms (attempt 3/5)6423worker1 # [ 225.018852] nix-grpc-daemon[1712]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 454 ms (attempt 4/5)6424worker1 # [ 225.480436] nix-grpc-daemon[1712]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6425worker1 # [ 225.499475] nix-grpc-daemon[1712]: ts=2026-09-20T12:24:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:426366426worker1 # [ 225.511114] nix-grpc-daemon[1712]: ts=2026-09-20T12:24:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:426366427worker1 # [ 225.520980] nix-daemon[847]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)6428worker1 # [ 225.543426] nix-grpc-daemon[1712]: ts=2026-09-20T12:24:54Z level=info event=niks3_push_started pid=17686429worker1 # [ 225.733995] nix-grpc-daemon[1768]: time=2026-09-20T12:24:54.633Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""6430worker1 # [ 225.889619] nix-daemon[847]: accepted connection from pid 1781, user nix-grpc-daemon (trusted)6431worker1 # [ 225.903394] nix-daemon[847]: reaped child process 1784, status = succeeded6432worker1 # [ 226.068814] nix-daemon[847]: accepted connection from pid 1786, user nix-grpc-daemon (trusted)6433worker1 # [ 226.083187] nix-daemon[847]: reaped child process 1789, status = succeeded6434lb # [ 226.103767] niks3-server[1118]: 2026/09/20 12:24:54 INFO Received uploads request method=POST path=/api/pending_closures6435worker1 # [ 226.104077] nix-grpc-daemon[1768]: time=2026-09-20T12:24:55.003Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6436worker1 # [ 226.105492] nix-grpc-daemon[1768]: time=2026-09-20T12:24:55.003Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6437lb # [ 226.136362] niks3-server[1118]: 2026/09/20 12:24:54 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6438lb # [ 226.147750] niks3-server[1118]: 2026/09/20 12:24:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6439worker1 # [ 226.146676] nix-grpc-daemon[1768]: time=2026-09-20T12:24:55.046Z level=INFO msg="Uploading 1 narinfos"6440lb # [ 226.156604] niks3-server[1118]: 2026/09/20 12:24:54 INFO Signed narinfos id=46 count=16441lb # [ 226.162551] niks3-server[1118]: 2026/09/20 12:24:54 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6442lb # [ 226.172615] niks3-server[1118]: 2026/09/20 12:24:54 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6443lb # [ 226.177788] niks3-server[1118]: 2026/09/20 12:24:54 INFO Completed upload id=466444worker1 # [ 226.168464] nix-grpc-daemon[1768]: time=2026-09-20T12:24:55.068Z level=INFO msg="Upload complete. (431ms)"6445lb # [ 226.182273] niks3-server[1118]: 2026/09/20 12:24:54 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6446worker1 # [ 226.172330] nix-grpc-daemon[1712]: ts=2026-09-20T12:24:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42636 duration_s=0 paths=1 nar_bytes_in=5046447worker1 # [ 226.178286] nix-grpc-daemon[1712]: ts=2026-09-20T12:24:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:426366448worker1 # [ 226.189676] nix-daemon[847]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)6449worker1 # [ 226.192066] nix-daemon[847]: reaped child process 1766, status = succeeded6450worker1 # [ 226.197095] nix-daemon[847]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)6451worker1 # [ 227.413515] nix-daemon[847]: reaped child process 1793, status = succeeded6452worker1 # [ 227.534750] nix-daemon[847]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6453worker1 # [ 227.546782] nix-daemon[847]: reaped child process 1811, status = succeeded6454lb # [ 227.567437] niks3-server[1118]: 2026/09/20 12:24:56 INFO Received uploads request method=POST path=/api/pending_closures6455worker1 # [ 227.567035] nix-grpc-daemon[1768]: time=2026-09-20T12:24:56.467Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6456worker1 # [ 227.568519] nix-grpc-daemon[1768]: time=2026-09-20T12:24:56.468Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6457lb # [ 227.603243] niks3-server[1118]: 2026/09/20 12:24:56 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6458lb # [ 227.611608] niks3-server[1118]: 2026/09/20 12:24:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6459worker1 # [ 227.607025] nix-grpc-daemon[1768]: time=2026-09-20T12:24:56.506Z level=INFO msg="Uploading 1 narinfos"6460lb # [ 227.617141] niks3-server[1118]: 2026/09/20 12:24:56 INFO Signed narinfos id=47 count=16461lb # [ 227.624047] niks3-server[1118]: 2026/09/20 12:24:56 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6462lb # [ 227.629890] niks3-server[1118]: 2026/09/20 12:24:56 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6463lb # [ 227.636140] niks3-server[1118]: 2026/09/20 12:24:56 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6464lb # [ 227.638931] niks3-server[1118]: 2026/09/20 12:24:56 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6465lb # [ 227.641979] niks3-server[1118]: 2026/09/20 12:24:56 INFO Completed upload id=476466worker1 # [ 227.632501] nix-grpc-daemon[1768]: time=2026-09-20T12:24:56.532Z level=INFO msg="Upload complete. (228ms)"6467worker1 # [ 227.636472] nix-grpc-daemon[1712]: ts=2026-09-20T12:24:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42636 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=1 outputs=16468worker2 # [ 227.666289] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:56Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16469worker1 # [ 227.646925] nix-daemon[847]: reaped child process 1791, status = succeeded6470worker2 # [ 227.670714] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:56Z level=debug event=want client=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06471worker1 # [ 227.685515] nix-grpc-daemon[1712]: ts=2026-09-20T12:24:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:426366472worker1 # [ 227.698349] nix-grpc-daemon[1712]: ts=2026-09-20T12:24:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:426366473worker1 # [ 227.709737] nix-grpc-daemon[1712]: ts=2026-09-20T12:24:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:426366474worker1 # [ 227.719167] nix-daemon[847]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)6475worker1 # [ 227.854384] nix-daemon[847]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6476worker1 # [ 227.868161] nix-daemon[847]: reaped child process 1818, status = succeeded6477worker1 # [ 227.990403] nix-daemon[847]: accepted connection from pid 1820, user nix-grpc-daemon (trusted)6478worker1 # [ 228.004676] nix-daemon[847]: reaped child process 1823, status = succeeded6479lb # [ 228.022796] niks3-server[1118]: 2026/09/20 12:24:56 INFO Received uploads request method=POST path=/api/pending_closures6480worker1 # [ 228.020629] nix-grpc-daemon[1768]: time=2026-09-20T12:24:56.921Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6481worker1 # [ 228.022158] nix-grpc-daemon[1768]: time=2026-09-20T12:24:56.922Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6482lb # [ 228.039620] envoy[979]: conn peer=[2001:db8:1::1]:41262 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9568 tx=3854 ms=54996483lb # [ 228.044860] envoy[979]: conn peer=[2001:db8:1::1]:33312 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2936 tx=1349 ms=88366484lb # [ 228.048310] envoy[979]: conn peer=[2001:db8:1::3]:33474 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=3404 tx=928 ms=1161116485lb # [ 228.064443] niks3-server[1118]: 2026/09/20 12:24:56 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6486lb # [ 228.073886] niks3-server[1118]: 2026/09/20 12:24:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6487worker1 # [ 228.070068] nix-grpc-daemon[1768]: time=2026-09-20T12:24:56.970Z level=INFO msg="Uploading 1 narinfos"6488lb # [ 228.080422] niks3-server[1118]: 2026/09/20 12:24:56 INFO Signed narinfos id=48 count=16489lb # [ 228.084554] niks3-server[1118]: 2026/09/20 12:24:56 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6490lb # [ 228.095230] niks3-server[1118]: 2026/09/20 12:24:56 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6491lb # [ 228.104668] niks3-server[1118]: 2026/09/20 12:24:56 INFO Completed upload id=486492lb # [ 228.106276] niks3-server[1118]: 2026/09/20 12:24:56 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6493worker1 # [ 228.096546] nix-grpc-daemon[1768]: time=2026-09-20T12:24:56.996Z level=INFO msg="Upload complete. (366ms)"6494worker1 # [ 228.100567] nix-grpc-daemon[1712]: ts=2026-09-20T12:24:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42636 duration_s=0 paths=1 nar_bytes_in=5046495worker1 # [ 228.107398] nix-grpc-daemon[1712]: ts=2026-09-20T12:24:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:426366496worker1 # [ 228.116497] nix-daemon[847]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)6497worker1 # [ 228.126329] nix-daemon[847]: reaped child process 1813, status = succeeded6498worker1 # [ 228.132354] nix-daemon[847]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)6499worker1 # [ 229.547584] nix-daemon[847]: reaped child process 1827, status = succeeded6500worker1 # [ 229.692177] nix-daemon[847]: accepted connection from pid 1842, user nix-grpc-daemon (trusted)6501worker1 # [ 229.705863] nix-daemon[847]: reaped child process 1845, status = succeeded6502lb # [ 229.731582] niks3-server[1118]: 2026/09/20 12:24:58 INFO Received uploads request method=POST path=/api/pending_closures6503worker1 # [ 229.726906] nix-grpc-daemon[1768]: time=2026-09-20T12:24:58.627Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6504worker1 # [ 229.728456] nix-grpc-daemon[1768]: time=2026-09-20T12:24:58.628Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6505lb # [ 229.753104] niks3-server[1118]: 2026/09/20 12:24:58 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6506lb # [ 229.758623] niks3-server[1118]: 2026/09/20 12:24:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6507lb # [ 229.760846] niks3-server[1118]: 2026/09/20 12:24:58 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6508lb # [ 229.762902] niks3-server[1118]: 2026/09/20 12:24:58 INFO Signed narinfos id=49 count=16509worker1 # [ 229.753692] nix-grpc-daemon[1768]: time=2026-09-20T12:24:58.653Z level=INFO msg="Uploading 1 narinfos"6510lb # [ 229.765987] niks3-server[1118]: 2026/09/20 12:24:58 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6511lb # [ 229.770763] niks3-server[1118]: 2026/09/20 12:24:58 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6512lb # [ 229.773932] niks3-server[1118]: 2026/09/20 12:24:58 INFO Completed upload id=496513worker1 # [ 229.764495] nix-grpc-daemon[1768]: time=2026-09-20T12:24:58.664Z level=INFO msg="Upload complete. (224ms)"6514lb # [ 229.776628] niks3-server[1118]: 2026/09/20 12:24:58 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6515worker1 # [ 229.766870] nix-grpc-daemon[1712]: ts=2026-09-20T12:24:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42636 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=2 outputs=16516worker1 # [ 229.771790] nix-grpc-daemon[1712]: ts=2026-09-20T12:24:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:426366517worker2 # [ 229.791969] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:58Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16518worker1 # [ 229.778314] nix-grpc-daemon[1712]: ts=2026-09-20T12:24:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:426366519worker1 # [ 229.780744] nix-daemon[847]: reaped child process 1825, status = succeeded6520worker2 # [ 229.802113] nix-grpc-daemon[1373]: ts=2026-09-20T12:24:58Z level=debug event=want client=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06521worker1 # [ 229.783746] nix-grpc-daemon[1712]: ts=2026-09-20T12:24:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:426366522worker1 # [ 229.788137] nix-daemon[847]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)6523worker1 # [ 229.945497] nix-daemon[847]: accepted connection from pid 1849, user nix-grpc-daemon (trusted)6524worker1 # [ 229.960675] nix-daemon[847]: reaped child process 1852, status = succeeded6525worker1 # [ 230.125445] nix-daemon[847]: accepted connection from pid 1854, user nix-grpc-daemon (trusted)6526worker1 # [ 230.139749] nix-daemon[847]: reaped child process 1857, status = succeeded6527lb # [ 230.165127] niks3-server[1118]: 2026/09/20 12:24:58 INFO Received uploads request method=POST path=/api/pending_closures6528worker1 # [ 230.164098] nix-grpc-daemon[1768]: time=2026-09-20T12:24:59.064Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6529worker1 # [ 230.165669] nix-grpc-daemon[1768]: time=2026-09-20T12:24:59.066Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6530lb # [ 230.192788] niks3-server[1118]: 2026/09/20 12:24:58 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6531lb # [ 230.205936] niks3-server[1118]: 2026/09/20 12:24:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6532worker1 # [ 230.201055] nix-grpc-daemon[1768]: time=2026-09-20T12:24:59.101Z level=INFO msg="Uploading 1 narinfos"6533lb # [ 230.210937] niks3-server[1118]: 2026/09/20 12:24:58 INFO Signed narinfos id=50 count=16534lb # [ 230.217279] niks3-server[1118]: 2026/09/20 12:24:58 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6535lb # [ 230.227490] niks3-server[1118]: 2026/09/20 12:24:58 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6536worker1 # [ 230.224580] nix-grpc-daemon[1768]: time=2026-09-20T12:24:59.124Z level=INFO msg="Upload complete. (430ms)"6537lb # [ 230.234378] niks3-server[1118]: 2026/09/20 12:24:58 INFO Completed upload id=506538worker1 # [ 230.226911] nix-grpc-daemon[1712]: ts=2026-09-20T12:24:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42636 duration_s=0 paths=1 nar_bytes_in=5046539lb # [ 230.237597] niks3-server[1118]: 2026/09/20 12:24:58 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6540worker1 # [ 230.232514] nix-grpc-daemon[1712]: ts=2026-09-20T12:24:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:426366541worker1 # [ 230.241771] nix-daemon[847]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)6542worker1 # [ 230.247647] nix-daemon[847]: reaped child process 1847, status = succeeded6543worker1 # [ 230.252967] nix-daemon[847]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)6544worker1 # [ 231.282607] nix-daemon[847]: reaped child process 1861, status = succeeded6545worker1 # [ 231.441136] nix-daemon[847]: accepted connection from pid 1876, user nix-grpc-daemon (trusted)6546worker1 # [ 231.454427] nix-daemon[847]: reaped child process 1879, status = succeeded6547lb # [ 231.479811] niks3-server[1118]: 2026/09/20 12:25:00 INFO Received uploads request method=POST path=/api/pending_closures6548worker1 # [ 231.480351] nix-grpc-daemon[1768]: time=2026-09-20T12:25:00.379Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6549worker1 # [ 231.481766] nix-grpc-daemon[1768]: time=2026-09-20T12:25:00.379Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6550lb # [ 231.511549] niks3-server[1118]: 2026/09/20 12:25:00 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv6551lb # [ 231.527664] niks3-server[1118]: 2026/09/20 12:25:00 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6552lb # [ 231.533714] niks3-server[1118]: 2026/09/20 12:25:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6553lb # [ 231.537749] niks3-server[1118]: 2026/09/20 12:25:00 INFO Signed narinfos id=51 count=16554worker1 # [ 231.530977] nix-grpc-daemon[1768]: time=2026-09-20T12:25:00.431Z level=INFO msg="Uploading 1 narinfos"6555lb # [ 231.547086] niks3-server[1118]: 2026/09/20 12:25:00 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls6556lb # [ 231.549796] niks3-server[1118]: 2026/09/20 12:25:00 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6557lb # [ 231.553419] niks3-server[1118]: 2026/09/20 12:25:00 INFO Completed upload id=516558worker1 # [ 231.543957] nix-grpc-daemon[1768]: time=2026-09-20T12:25:00.444Z level=INFO msg="Upload complete. (283ms)"6559lb # [ 231.555843] niks3-server[1118]: 2026/09/20 12:25:00 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo6560worker2 # [ 231.572768] nix-grpc-daemon[1373]: ts=2026-09-20T12:25:00Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=16561worker2 # [ 231.577775] nix-grpc-daemon[1373]: ts=2026-09-20T12:25:00Z level=debug event=want client=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=06562worker1 # [ 231.552332] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42636 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=3 outputs=16563worker1 # [ 231.558197] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:426366564worker1 # [ 231.569319] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:426366565worker1 # [ 231.572217] nix-daemon[847]: reaped child process 1859, status = succeeded6566worker1 # [ 231.575865] nix-grpc-daemon[1768]: time=2026-09-20T12:25:00.476Z level=INFO msg="All 1 paths already cached"6567worker1 # [ 231.579661] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:426366568worker1 # [ 231.585443] nix-daemon[847]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)6569worker1 # [ 231.743939] nix-daemon[847]: accepted connection from pid 1883, user nix-grpc-daemon (trusted)6570worker1 # [ 231.760116] nix-daemon[847]: reaped child process 1886, status = succeeded6571worker1 # [ 231.925357] nix-daemon[847]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)6572worker1 # [ 231.940717] nix-daemon[847]: reaped child process 1891, status = succeeded6573lb # [ 231.964908] niks3-server[1118]: 2026/09/20 12:25:00 INFO Received uploads request method=POST path=/api/pending_closures6574worker1 # [ 231.963988] nix-grpc-daemon[1768]: time=2026-09-20T12:25:00.864Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6575worker1 # [ 231.965593] nix-grpc-daemon[1768]: time=2026-09-20T12:25:00.866Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"6576lb # [ 231.995141] niks3-server[1118]: 2026/09/20 12:25:00 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst6577lb # [ 232.004308] niks3-server[1118]: 2026/09/20 12:25:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6578lb # [ 232.010903] niks3-server[1118]: 2026/09/20 12:25:00 INFO Signed narinfos id=52 count=16579worker1 # [ 232.004732] nix-grpc-daemon[1768]: time=2026-09-20T12:25:00.905Z level=INFO msg="Uploading 1 narinfos"6580lb # [ 232.017832] niks3-server[1118]: 2026/09/20 12:25:00 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls6581lb # [ 232.026553] niks3-server[1118]: 2026/09/20 12:25:00 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6582worker1 # [ 232.023309] nix-grpc-daemon[1768]: time=2026-09-20T12:25:00.923Z level=INFO msg="Upload complete. (432ms)"6583lb # [ 232.033312] niks3-server[1118]: 2026/09/20 12:25:00 INFO Completed upload id=526584worker1 # [ 232.025405] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42636 duration_s=0 paths=1 nar_bytes_in=6246585lb # [ 232.037174] niks3-server[1118]: 2026/09/20 12:25:00 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo6586worker1 # [ 232.030406] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:426366587worker1 # [ 232.040038] nix-daemon[847]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)6588worker1 # [ 232.046151] nix-daemon[847]: reaped child process 1881, status = succeeded6589worker1 # [ 232.049473] nix-daemon[847]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)6590worker1 # [ 233.064724] nix-daemon[847]: reaped child process 1895, status = succeeded6591worker1 # [ 233.213786] nix-daemon[847]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)6592worker1 # [ 233.227541] nix-daemon[847]: reaped child process 1913, status = succeeded6593lb # [ 233.255210] niks3-server[1118]: 2026/09/20 12:25:01 INFO Received uploads request method=POST path=/api/pending_closures6594worker1 # [ 233.255381] nix-grpc-daemon[1768]: time=2026-09-20T12:25:02.153Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6595worker1 # [ 233.256792] nix-grpc-daemon[1768]: time=2026-09-20T12:25:02.154Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"6596lb # [ 233.300227] niks3-server[1118]: 2026/09/20 12:25:01 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst6597lb # [ 233.305680] niks3-server[1118]: 2026/09/20 12:25:01 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv6598lb # [ 233.313828] niks3-server[1118]: 2026/09/20 12:25:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6599lb # [ 233.318886] niks3-server[1118]: 2026/09/20 12:25:01 INFO Signed narinfos id=53 count=16600worker1 # [ 233.312473] nix-grpc-daemon[1768]: time=2026-09-20T12:25:02.212Z level=INFO msg="Uploading 1 narinfos"6601lb # [ 233.324380] niks3-server[1118]: 2026/09/20 12:25:01 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls6602lb # [ 233.334243] niks3-server[1118]: 2026/09/20 12:25:01 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6603lb # [ 233.340089] niks3-server[1118]: 2026/09/20 12:25:01 INFO Completed upload id=536604lb # [ 233.343357] niks3-server[1118]: 2026/09/20 12:25:01 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo6605worker1 # [ 233.332580] nix-grpc-daemon[1768]: time=2026-09-20T12:25:02.232Z level=INFO msg="Upload complete. (275ms)"6606worker2 # [ 233.362315] nix-grpc-daemon[1373]: ts=2026-09-20T12:25:02Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=16607worker1 # [ 233.339765] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42636 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=4 outputs=16608worker1 # [ 233.347563] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:426366609worker2 # [ 233.371635] nix-grpc-daemon[1373]: ts=2026-09-20T12:25:02Z level=debug event=want client=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=06610worker1 # [ 233.358561] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:426366611worker1 # [ 233.365237] nix-daemon[847]: reaped child process 1893, status = succeeded6612worker1 # [ 233.372140] nix-grpc-daemon[1768]: time=2026-09-20T12:25:02.272Z level=INFO msg="All 1 paths already cached"6613worker1 # [ 233.378425] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:426366614worker1 # [ 233.385345] nix-daemon[847]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)6615worker1 # [ 233.555385] nix-daemon[847]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)6616worker1 # [ 233.571176] nix-daemon[847]: reaped child process 1920, status = succeeded6617worker1 # [ 233.736620] nix-daemon[847]: accepted connection from pid 1922, user nix-grpc-daemon (trusted)6618worker1 # [ 233.751254] nix-daemon[847]: reaped child process 1925, status = succeeded6619lb # [ 233.770494] niks3-server[1118]: 2026/09/20 12:25:02 INFO Received uploads request method=POST path=/api/pending_closures6620worker1 # [ 233.768969] nix-grpc-daemon[1768]: time=2026-09-20T12:25:02.669Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6621worker1 # [ 233.770511] nix-grpc-daemon[1768]: time=2026-09-20T12:25:02.670Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"6622lb # [ 233.800140] niks3-server[1118]: 2026/09/20 12:25:02 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst6623lb # [ 233.810642] niks3-server[1118]: 2026/09/20 12:25:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6624worker1 # [ 233.807569] nix-grpc-daemon[1768]: time=2026-09-20T12:25:02.708Z level=INFO msg="Uploading 1 narinfos"6625lb # [ 233.817142] niks3-server[1118]: 2026/09/20 12:25:02 INFO Signed narinfos id=54 count=16626lb # [ 233.821628] niks3-server[1118]: 2026/09/20 12:25:02 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls6627lb # [ 233.833446] niks3-server[1118]: 2026/09/20 12:25:02 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6628lb # [ 233.839975] niks3-server[1118]: 2026/09/20 12:25:02 INFO Completed upload id=546629worker1 # [ 233.830307] nix-grpc-daemon[1768]: time=2026-09-20T12:25:02.730Z level=INFO msg="Upload complete. (436ms)"6630lb # [ 233.842605] niks3-server[1118]: 2026/09/20 12:25:02 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo6631worker1 # [ 233.833987] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42636 duration_s=0 paths=1 nar_bytes_in=6246632worker1 # [ 233.839971] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:426366633worker1 # [ 233.848498] nix-daemon[847]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)6634worker1 # [ 233.855202] nix-daemon[847]: reaped child process 1915, status = succeeded6635worker1 # [ 233.860367] nix-daemon[847]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)6636worker1 # [ 234.689280] nix-daemon[847]: reaped child process 1929, status = succeeded6637worker1 # [ 234.844482] nix-daemon[847]: accepted connection from pid 1944, user nix-grpc-daemon (trusted)6638worker1 # [ 234.859513] nix-daemon[847]: reaped child process 1947, status = succeeded6639lb # [ 234.880045] niks3-server[1118]: 2026/09/20 12:25:03 INFO Received uploads request method=POST path=/api/pending_closures6640worker1 # [ 234.876483] nix-grpc-daemon[1768]: time=2026-09-20T12:25:03.776Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6641worker1 # [ 234.879075] nix-grpc-daemon[1768]: time=2026-09-20T12:25:03.778Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"6642lb # [ 234.906524] niks3-server[1118]: 2026/09/20 12:25:03 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst6643lb # [ 234.909898] niks3-server[1118]: 2026/09/20 12:25:03 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv6644lb # [ 234.917366] niks3-server[1118]: 2026/09/20 12:25:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6645lb # [ 234.922398] niks3-server[1118]: 2026/09/20 12:25:03 INFO Signed narinfos id=55 count=16646worker1 # [ 234.915448] nix-grpc-daemon[1768]: time=2026-09-20T12:25:03.815Z level=INFO msg="Uploading 1 narinfos"6647lb # [ 234.925300] niks3-server[1118]: 2026/09/20 12:25:03 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls6648lb # [ 234.932728] niks3-server[1118]: 2026/09/20 12:25:03 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6649lb # [ 234.935603] niks3-server[1118]: 2026/09/20 12:25:03 INFO Completed upload id=556650worker1 # [ 234.926134] nix-grpc-daemon[1768]: time=2026-09-20T12:25:03.826Z level=INFO msg="Upload complete. (258ms)"6651lb # [ 234.938064] niks3-server[1118]: 2026/09/20 12:25:03 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo6652worker1 # [ 234.928785] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42636 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=5 outputs=16653worker2 # [ 234.954284] nix-grpc-daemon[1373]: ts=2026-09-20T12:25:03Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=16654worker1 # [ 234.936905] nix-daemon[847]: reaped child process 1927, status = succeeded6655worker1 # [ 234.938747] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:426366656worker2 # [ 234.963854] nix-grpc-daemon[1373]: ts=2026-09-20T12:25:03Z level=debug event=want client=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=06657worker1 # [ 234.944621] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:426366658worker1 # [ 234.950194] nix-grpc-daemon[1768]: time=2026-09-20T12:25:03.850Z level=INFO msg="All 1 paths already cached"6659worker1 # [ 234.954107] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:426366660worker1 # [ 234.959664] nix-daemon[847]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)6661worker1 # [ 235.110876] nix-daemon[847]: accepted connection from pid 1951, user nix-grpc-daemon (trusted)6662worker1 # [ 235.125113] nix-daemon[847]: reaped child process 1954, status = succeeded6663worker1 # [ 235.288261] nix-daemon[847]: accepted connection from pid 1956, user nix-grpc-daemon (trusted)6664worker1 # [ 235.301718] nix-daemon[847]: reaped child process 1959, status = succeeded6665lb # [ 235.327176] niks3-server[1118]: 2026/09/20 12:25:03 INFO Received uploads request method=POST path=/api/pending_closures6666worker1 # [ 235.325534] nix-grpc-daemon[1768]: time=2026-09-20T12:25:04.225Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6667worker1 # [ 235.326896] nix-grpc-daemon[1768]: time=2026-09-20T12:25:04.225Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"6668lb # [ 235.352487] niks3-server[1118]: 2026/09/20 12:25:03 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst6669lb # [ 235.363581] niks3-server[1118]: 2026/09/20 12:25:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6670worker1 # [ 235.358637] nix-grpc-daemon[1768]: time=2026-09-20T12:25:04.258Z level=INFO msg="Uploading 1 narinfos"6671lb # [ 235.368711] niks3-server[1118]: 2026/09/20 12:25:03 INFO Signed narinfos id=56 count=16672lb # [ 235.374928] niks3-server[1118]: 2026/09/20 12:25:03 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls6673lb # [ 235.387815] niks3-server[1118]: 2026/09/20 12:25:03 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6674worker1 # [ 235.383175] nix-grpc-daemon[1768]: time=2026-09-20T12:25:04.283Z level=INFO msg="Upload complete. (418ms)"6675lb # [ 235.392564] niks3-server[1118]: 2026/09/20 12:25:03 INFO Completed upload id=566676lb # [ 235.396212] niks3-server[1118]: 2026/09/20 12:25:03 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo6677worker1 # [ 235.385973] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42636 duration_s=0 paths=1 nar_bytes_in=6246678worker1 # [ 235.390914] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:426366679worker1 # [ 235.402062] nix-daemon[847]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)6680worker1 # [ 235.405406] nix-daemon[847]: reaped child process 1949, status = succeeded6681worker1 # [ 235.410822] nix-daemon[847]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)6682worker1 # [ 236.437600] nix-daemon[847]: reaped child process 1963, status = succeeded6683worker1 # [ 236.598854] nix-daemon[847]: accepted connection from pid 1978, user nix-grpc-daemon (trusted)6684worker1 # [ 236.612751] nix-daemon[847]: reaped child process 1981, status = succeeded6685lb # [ 236.639766] niks3-server[1118]: 2026/09/20 12:25:05 INFO Received uploads request method=POST path=/api/pending_closures6686worker1 # [ 236.640612] nix-grpc-daemon[1768]: time=2026-09-20T12:25:05.540Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6687worker1 # [ 236.643435] nix-grpc-daemon[1768]: time=2026-09-20T12:25:05.542Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"6688lb # [ 236.676832] niks3-server[1118]: 2026/09/20 12:25:05 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv6689lb # [ 236.681648] niks3-server[1118]: 2026/09/20 12:25:05 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst6690lb # [ 236.695719] niks3-server[1118]: 2026/09/20 12:25:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6691worker1 # [ 236.691318] nix-grpc-daemon[1768]: time=2026-09-20T12:25:05.591Z level=INFO msg="Uploading 1 narinfos"6692lb # [ 236.701100] niks3-server[1118]: 2026/09/20 12:25:05 INFO Signed narinfos id=57 count=16693lb # [ 236.708455] niks3-server[1118]: 2026/09/20 12:25:05 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls6694lb # [ 236.718746] niks3-server[1118]: 2026/09/20 12:25:05 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6695lb # [ 236.725165] niks3-server[1118]: 2026/09/20 12:25:05 INFO Completed upload id=576696worker1 # [ 236.715704] nix-grpc-daemon[1768]: time=2026-09-20T12:25:05.616Z level=INFO msg="Upload complete. (297ms)"6697lb # [ 236.727556] niks3-server[1118]: 2026/09/20 12:25:05 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo6698worker2 # [ 236.744192] nix-grpc-daemon[1373]: ts=2026-09-20T12:25:05Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=16699worker2 # [ 236.749358] nix-grpc-daemon[1373]: ts=2026-09-20T12:25:05Z level=debug event=want client=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=06700worker1 # [ 236.722881] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42636 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=6 outputs=16701worker1 # [ 236.730893] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:426366702worker1 # [ 236.743532] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:426366703worker1 # [ 236.749431] nix-daemon[847]: reaped child process 1961, status = succeeded6704worker1 # [ 236.756159] nix-grpc-daemon[1768]: time=2026-09-20T12:25:05.656Z level=INFO msg="All 2 paths already cached"6705worker1 # [ 236.762534] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:426366706worker1 # [ 236.769995] nix-daemon[847]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)6707worker1 # [ 236.897257] nix-daemon[847]: accepted connection from pid 1985, user nix-grpc-daemon (trusted)6708worker1 # [ 236.911422] nix-daemon[847]: reaped child process 1988, status = succeeded6709worker1 # [ 237.036848] nix-daemon[847]: accepted connection from pid 1990, user nix-grpc-daemon (trusted)6710worker1 # [ 237.050335] nix-daemon[847]: reaped child process 1993, status = succeeded6711lb # [ 237.071512] niks3-server[1118]: 2026/09/20 12:25:05 INFO Received uploads request method=POST path=/api/pending_closures6712worker1 # [ 237.069998] nix-grpc-daemon[1768]: time=2026-09-20T12:25:05.970Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6713worker1 # [ 237.071438] nix-grpc-daemon[1768]: time=2026-09-20T12:25:05.970Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"6714lb # [ 237.097478] niks3-server[1118]: 2026/09/20 12:25:05 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst6715lb # [ 237.107106] niks3-server[1118]: 2026/09/20 12:25:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6716lb # [ 237.112109] niks3-server[1118]: 2026/09/20 12:25:05 INFO Signed narinfos id=58 count=16717worker1 # [ 237.105552] nix-grpc-daemon[1768]: time=2026-09-20T12:25:06.005Z level=INFO msg="Uploading 1 narinfos"6718lb # [ 237.118930] niks3-server[1118]: 2026/09/20 12:25:05 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls6719lb # [ 237.129856] niks3-server[1118]: 2026/09/20 12:25:05 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6720lb # [ 237.134596] niks3-server[1118]: 2026/09/20 12:25:05 INFO Completed upload id=586721worker1 # [ 237.127245] nix-grpc-daemon[1768]: time=2026-09-20T12:25:06.027Z level=INFO msg="Upload complete. (352ms)"6722worker1 # [ 237.129545] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42636 duration_s=0 paths=1 nar_bytes_in=7526723lb # [ 237.140060] niks3-server[1118]: 2026/09/20 12:25:05 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo6724worker1 # [ 237.135609] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:426366725worker1 # [ 237.142051] nix-daemon[847]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)6726worker1 # [ 237.144461] nix-daemon[847]: reaped child process 1983, status = succeeded6727worker1 # [ 237.149561] nix-daemon[847]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)6728worker1 # [ 238.581322] nix-daemon[847]: reaped child process 1997, status = succeeded6729worker1 # [ 238.708106] nix-daemon[847]: accepted connection from pid 2012, user nix-grpc-daemon (trusted)6730worker1 # [ 238.721094] nix-daemon[847]: reaped child process 2015, status = succeeded6731lb # [ 238.744536] niks3-server[1118]: 2026/09/20 12:25:07 INFO Received uploads request method=POST path=/api/pending_closures6732worker1 # [ 238.743960] nix-grpc-daemon[1768]: time=2026-09-20T12:25:07.644Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6733worker1 # [ 238.748750] nix-grpc-daemon[1768]: time=2026-09-20T12:25:07.646Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"6734lb # [ 238.783148] niks3-server[1118]: 2026/09/20 12:25:07 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv6735lb # [ 238.793299] niks3-server[1118]: 2026/09/20 12:25:07 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst6736lb # [ 238.802350] niks3-server[1118]: 2026/09/20 12:25:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6737lb # [ 238.806341] niks3-server[1118]: 2026/09/20 12:25:07 INFO Signed narinfos id=59 count=16738worker1 # [ 238.799261] nix-grpc-daemon[1768]: time=2026-09-20T12:25:07.699Z level=INFO msg="Uploading 1 narinfos"6739lb # [ 238.814127] niks3-server[1118]: 2026/09/20 12:25:07 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls6740lb # [ 238.818176] niks3-server[1118]: 2026/09/20 12:25:07 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6741lb # [ 238.821238] niks3-server[1118]: 2026/09/20 12:25:07 INFO Completed upload id=596742lb # [ 238.823245] niks3-server[1118]: 2026/09/20 12:25:07 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo6743worker1 # [ 238.813181] nix-grpc-daemon[1768]: time=2026-09-20T12:25:07.713Z level=INFO msg="Upload complete. (254ms)"6744worker2 # [ 238.841725] nix-grpc-daemon[1373]: ts=2026-09-20T12:25:07Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=16745worker1 # [ 238.819701] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42636 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=7 outputs=16746worker1 # [ 238.831579] nix-daemon[847]: reaped child process 1995, status = succeeded6747worker1 # [ 238.861935] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:426366748worker1 # [ 238.872794] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:426366749worker1 # [ 238.882876] nix-grpc-daemon[1768]: time=2026-09-20T12:25:07.783Z level=INFO msg="All 2 paths already cached"6750worker1 # [ 238.888936] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:426366751worker1 # [ 238.899134] nix-daemon[847]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)6752worker1 # [ 239.066892] nix-daemon[847]: accepted connection from pid 2019, user nix-grpc-daemon (trusted)6753worker1 # [ 239.081550] nix-daemon[847]: reaped child process 2022, status = succeeded6754worker1 # [ 239.245650] nix-daemon[847]: accepted connection from pid 2024, user nix-grpc-daemon (trusted)6755worker1 # [ 239.259297] nix-daemon[847]: reaped child process 2027, status = succeeded6756lb # [ 239.285595] niks3-server[1118]: 2026/09/20 12:25:07 INFO Received uploads request method=POST path=/api/pending_closures6757worker1 # [ 239.285156] nix-grpc-daemon[1768]: time=2026-09-20T12:25:08.185Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6758worker1 # [ 239.286621] nix-grpc-daemon[1768]: time=2026-09-20T12:25:08.185Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"6759lb # [ 239.316473] niks3-server[1118]: 2026/09/20 12:25:07 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst6760lb # [ 239.328500] niks3-server[1118]: 2026/09/20 12:25:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6761lb # [ 239.335282] niks3-server[1118]: 2026/09/20 12:25:07 INFO Signed narinfos id=60 count=16762worker1 # [ 239.329098] nix-grpc-daemon[1768]: time=2026-09-20T12:25:08.229Z level=INFO msg="Uploading 1 narinfos"6763lb # [ 239.338605] niks3-server[1118]: 2026/09/20 12:25:07 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls6764lb # [ 239.351254] niks3-server[1118]: 2026/09/20 12:25:07 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6765lb # [ 239.356847] niks3-server[1118]: 2026/09/20 12:25:07 INFO Completed upload id=606766worker1 # [ 239.347308] nix-grpc-daemon[1768]: time=2026-09-20T12:25:08.247Z level=INFO msg="Upload complete. (437ms)"6767lb # [ 239.359525] niks3-server[1118]: 2026/09/20 12:25:07 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo6768worker1 # [ 239.350858] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42636 duration_s=0 paths=1 nar_bytes_in=7526769worker1 # [ 239.356175] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:426366770worker1 # [ 239.366155] nix-daemon[847]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)6771worker1 # [ 239.372265] nix-daemon[847]: reaped child process 2017, status = succeeded6772worker1 # [ 239.377406] nix-daemon[847]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)6773worker1 # [ 240.592238] nix-daemon[847]: reaped child process 2031, status = succeeded6774worker1 # [ 240.722516] nix-daemon[847]: accepted connection from pid 2046, user nix-grpc-daemon (trusted)6775worker1 # [ 240.736457] nix-daemon[847]: reaped child process 2049, status = succeeded6776lb # [ 240.761668] niks3-server[1118]: 2026/09/20 12:25:09 INFO Received uploads request method=POST path=/api/pending_closures6777worker1 # [ 240.756791] nix-grpc-daemon[1768]: time=2026-09-20T12:25:09.656Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6778worker1 # [ 240.758486] nix-grpc-daemon[1768]: time=2026-09-20T12:25:09.658Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"6779lb # [ 240.782072] niks3-server[1118]: 2026/09/20 12:25:09 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst6780lb # [ 240.785343] niks3-server[1118]: 2026/09/20 12:25:09 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv6781lb # [ 240.788471] niks3-server[1118]: 2026/09/20 12:25:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6782lb # [ 240.790828] niks3-server[1118]: 2026/09/20 12:25:09 INFO Signed narinfos id=61 count=16783worker1 # [ 240.781531] nix-grpc-daemon[1768]: time=2026-09-20T12:25:09.681Z level=INFO msg="Uploading 1 narinfos"6784lb # [ 240.794130] niks3-server[1118]: 2026/09/20 12:25:09 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls6785lb # [ 240.798395] niks3-server[1118]: 2026/09/20 12:25:09 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6786lb # [ 240.801146] niks3-server[1118]: 2026/09/20 12:25:09 INFO Completed upload id=616787worker1 # [ 240.791593] nix-grpc-daemon[1768]: time=2026-09-20T12:25:09.692Z level=INFO msg="Upload complete. (207ms)"6788lb # [ 240.803804] niks3-server[1118]: 2026/09/20 12:25:09 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo6789worker1 # [ 240.794458] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42636 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=8 outputs=16790worker2 # [ 240.818101] nix-grpc-daemon[1373]: ts=2026-09-20T12:25:09Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=16791worker1 # [ 240.798890] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:426366792worker1 # [ 240.805799] nix-daemon[847]: reaped child process 2029, status = succeeded6793worker1 # [ 240.807385] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:426366794worker1 # [ 240.813268] nix-grpc-daemon[1768]: time=2026-09-20T12:25:09.713Z level=INFO msg="All 2 paths already cached"6795worker1 # [ 240.817099] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:426366796worker1 # [ 240.826301] nix-daemon[847]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)6797worker1 # [ 240.974512] nix-daemon[847]: accepted connection from pid 2053, user nix-grpc-daemon (trusted)6798worker1 # [ 240.988638] nix-daemon[847]: reaped child process 2056, status = succeeded6799worker1 # [ 241.138173] nix-daemon[847]: accepted connection from pid 2058, user nix-grpc-daemon (trusted)6800worker1 # [ 241.152983] nix-daemon[847]: reaped child process 2061, status = succeeded6801lb # [ 241.178766] niks3-server[1118]: 2026/09/20 12:25:09 INFO Received uploads request method=POST path=/api/pending_closures6802worker1 # [ 241.178169] nix-grpc-daemon[1768]: time=2026-09-20T12:25:10.078Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6803worker1 # [ 241.179766] nix-grpc-daemon[1768]: time=2026-09-20T12:25:10.080Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"6804lb # [ 241.211702] niks3-server[1118]: 2026/09/20 12:25:09 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst6805lb # [ 241.215993] niks3-server[1118]: 2026/09/20 12:25:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6806lb # [ 241.220544] niks3-server[1118]: 2026/09/20 12:25:09 INFO Signed narinfos id=62 count=16807worker1 # [ 241.215397] nix-grpc-daemon[1768]: time=2026-09-20T12:25:10.115Z level=INFO msg="Uploading 1 narinfos"6808lb # [ 241.228600] niks3-server[1118]: 2026/09/20 12:25:09 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls6809lb # [ 241.241801] niks3-server[1118]: 2026/09/20 12:25:09 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6810lb # [ 241.246191] niks3-server[1118]: 2026/09/20 12:25:09 INFO Completed upload id=626811worker1 # [ 241.239043] nix-grpc-daemon[1768]: time=2026-09-20T12:25:10.139Z level=INFO msg="Upload complete. (406ms)"6812lb # [ 241.248567] niks3-server[1118]: 2026/09/20 12:25:09 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo6813worker1 # [ 241.241289] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42636 duration_s=0 paths=1 nar_bytes_in=7526814worker1 # [ 241.245600] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:426366815worker1 # [ 241.252643] nix-daemon[847]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)6816worker1 # [ 241.255188] nix-daemon[847]: reaped child process 2051, status = succeeded6817worker1 # [ 241.261096] nix-daemon[847]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)6818worker1 # [ 242.489708] nix-daemon[847]: reaped child process 2065, status = succeeded6819worker1 # [ 242.668747] nix-daemon[847]: accepted connection from pid 2080, user nix-grpc-daemon (trusted)6820worker1 # [ 242.682229] nix-daemon[847]: reaped child process 2083, status = succeeded6821lb # [ 242.703336] niks3-server[1118]: 2026/09/20 12:25:11 INFO Received uploads request method=POST path=/api/pending_closures6822worker1 # [ 242.703385] nix-grpc-daemon[1768]: time=2026-09-20T12:25:11.603Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6823worker1 # [ 242.704893] nix-grpc-daemon[1768]: time=2026-09-20T12:25:11.605Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"6824lb # [ 242.737989] niks3-server[1118]: 2026/09/20 12:25:11 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst6825lb # [ 242.754346] niks3-server[1118]: 2026/09/20 12:25:11 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv6826lb # [ 242.764306] niks3-server[1118]: 2026/09/20 12:25:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6827lb # [ 242.768505] niks3-server[1118]: 2026/09/20 12:25:11 INFO Signed narinfos id=63 count=16828worker1 # [ 242.761839] nix-grpc-daemon[1768]: time=2026-09-20T12:25:11.661Z level=INFO msg="Uploading 1 narinfos"6829lb # [ 242.773876] niks3-server[1118]: 2026/09/20 12:25:11 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls6830lb # [ 242.782636] niks3-server[1118]: 2026/09/20 12:25:11 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6831lb # [ 242.786173] niks3-server[1118]: 2026/09/20 12:25:11 INFO Completed upload id=636832worker1 # [ 242.776950] nix-grpc-daemon[1768]: time=2026-09-20T12:25:11.677Z level=INFO msg="Upload complete. (307ms)"6833lb # [ 242.788537] niks3-server[1118]: 2026/09/20 12:25:11 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo6834worker1 # [ 242.783671] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42636 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=9 outputs=16835worker1 # [ 242.796918] nix-daemon[847]: reaped child process 2063, status = succeeded6836client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 18.76 seconds)6837worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6838worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6839client: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&16840worker1 # [ 243.170510] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:12Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:426366841worker1 # [ 243.178146] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:426366842worker2 # [ 243.223293] nix-grpc-daemon[1373]: ts=2026-09-20T12:25:12Z level=debug event=want client=2 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=06843client: (finished: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&1, in 0.28 seconds)6844(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 19.11 seconds)6845subtest: low disk drains a worker and builds go to the other6846worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6847worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.03 seconds)6848worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6849worker1 # [ 244.929665] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:13Z level=info event=unhealthy reason=min_free6850worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.11 seconds)6851client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&26852client # don't know how to build these paths:6853client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6854worker1 # [ 245.586692] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:426366855worker2 # [ 245.614685] nix-grpc-daemon[1373]: ts=2026-09-20T12:25:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:351246856worker2 # [ 245.629637] nix-grpc-daemon[1373]: ts=2026-09-20T12:25:14Z level=debug event=want client=3 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=06857worker2 # [ 245.631695] nix-grpc-daemon[1373]: ts=2026-09-20T12:25:14Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=16858worker2 # [ 245.673669] nix-grpc-daemon[1373]: ts=2026-09-20T12:25:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:351246859worker2 # [ 245.681576] nix-grpc-daemon[1373]: ts=2026-09-20T12:25:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:351246860client # copying 1 paths to 'lb:50051'...6861worker2 # [ 245.690178] nix-grpc-daemon[1373]: ts=2026-09-20T12:25:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:351246862client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...6863worker2 # [ 245.696653] nix-daemon[857]: accepted connection from pid 1373, user nix-grpc-daemon (trusted)6864worker2 # [ 245.810924] nix-daemon[857]: accepted connection from pid 1639, user nix-grpc-daemon (trusted)6865worker2 # [ 245.823849] nix-daemon[857]: reaped child process 1642, status = succeeded6866worker2 # [ 245.934986] nix-daemon[857]: accepted connection from pid 1644, user nix-grpc-daemon (trusted)6867worker2 # [ 245.947789] nix-daemon[857]: reaped child process 1647, status = succeeded6868lb # [ 245.942557] niks3-server[1118]: 2026/09/20 12:25:14 INFO Received uploads request method=POST path=/api/pending_closures6869worker2 # [ 245.960838] nix-grpc-daemon[1441]: time=2026-09-20T12:25:14.755Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6870worker2 # [ 245.962160] nix-grpc-daemon[1441]: time=2026-09-20T12:25:14.756Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"6871lb # [ 245.954160] niks3-server[1118]: 2026/09/20 12:25:14 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst6872lb # [ 245.958239] niks3-server[1118]: 2026/09/20 12:25:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6873lb # [ 245.960462] niks3-server[1118]: 2026/09/20 12:25:14 INFO Signed narinfos id=64 count=16874worker2 # [ 245.975596] nix-grpc-daemon[1441]: time=2026-09-20T12:25:14.771Z level=INFO msg="Uploading 1 narinfos"6875lb # [ 245.964510] niks3-server[1118]: 2026/09/20 12:25:14 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls6876lb # [ 245.968791] niks3-server[1118]: 2026/09/20 12:25:14 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6877lb # [ 245.971570] niks3-server[1118]: 2026/09/20 12:25:14 INFO Completed upload id=646878worker2 # [ 245.986501] nix-grpc-daemon[1441]: time=2026-09-20T12:25:14.782Z level=INFO msg="Upload complete. (285ms)"6879lb # [ 245.973262] niks3-server[1118]: 2026/09/20 12:25:14 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo6880worker2 # [ 245.988621] nix-grpc-daemon[1373]: ts=2026-09-20T12:25:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35124 duration_s=0 paths=1 nar_bytes_in=5126881worker2 # [ 245.992439] nix-grpc-daemon[1373]: ts=2026-09-20T12:25:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:351246882worker2 # [ 245.998105] nix-daemon[857]: accepted connection from pid 1373, user nix-grpc-daemon (trusted)6883worker2 # [ 246.000249] nix-daemon[857]: reaped child process 1637, status = succeeded6884worker2 # [ 246.005104] nix-daemon[857]: accepted connection from pid 1373, user nix-grpc-daemon (trusted)6885client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...6886client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6887client # farm-a> LOG-farm-a-drain6888worker2 # [ 247.230955] nix-daemon[857]: reaped child process 1651, status = succeeded6889worker2 # [ 247.394537] nix-daemon[857]: accepted connection from pid 1666, user nix-grpc-daemon (trusted)6890worker2 # [ 247.406841] nix-daemon[857]: reaped child process 1669, status = succeeded6891lb # [ 247.408916] niks3-server[1118]: 2026/09/20 12:25:15 INFO Received uploads request method=POST path=/api/pending_closures6892worker2 # [ 247.427675] nix-grpc-daemon[1441]: time=2026-09-20T12:25:16.223Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6893worker2 # [ 247.429160] nix-grpc-daemon[1441]: time=2026-09-20T12:25:16.224Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"6894lb # [ 247.433934] niks3-server[1118]: 2026/09/20 12:25:16 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6895lb # [ 247.437469] niks3-server[1118]: 2026/09/20 12:25:16 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6896lb # [ 247.440447] niks3-server[1118]: 2026/09/20 12:25:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6897lb # [ 247.442295] niks3-server[1118]: 2026/09/20 12:25:16 INFO Signed narinfos id=65 count=16898worker2 # [ 247.457494] nix-grpc-daemon[1441]: time=2026-09-20T12:25:16.252Z level=INFO msg="Uploading 1 narinfos"6899lb # [ 247.446289] niks3-server[1118]: 2026/09/20 12:25:16 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls6900lb # [ 247.451156] niks3-server[1118]: 2026/09/20 12:25:16 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6901lb # [ 247.455220] niks3-server[1118]: 2026/09/20 12:25:16 INFO Completed upload id=656902worker2 # [ 247.468834] nix-grpc-daemon[1441]: time=2026-09-20T12:25:16.264Z level=INFO msg="Upload complete. (256ms)"6903lb # [ 247.456578] niks3-server[1118]: 2026/09/20 12:25:16 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo6904worker2 # [ 247.471236] nix-grpc-daemon[1373]: ts=2026-09-20T12:25:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35124 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=10 outputs=16905worker2 # [ 247.478213] nix-grpc-daemon[1373]: ts=2026-09-20T12:25:16Z level=debug event=want client=3 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=06906worker2 # [ 247.480143] nix-grpc-daemon[1373]: ts=2026-09-20T12:25:16Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=16907worker2 # [ 247.484175] nix-daemon[857]: reaped child process 1649, status = succeeded6908worker2 # [ 247.522312] nix-grpc-daemon[1373]: ts=2026-09-20T12:25:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:351246909worker2 # [ 247.533538] nix-grpc-daemon[1373]: ts=2026-09-20T12:25:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:351246910worker2 # [ 247.545181] nix-grpc-daemon[1441]: time=2026-09-20T12:25:16.340Z level=INFO msg="All 1 paths already cached"6911client # copying 1 paths to 'lb:50051'...6912client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...6913worker2 # [ 247.553179] nix-grpc-daemon[1373]: ts=2026-09-20T12:25:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:351246914worker2 # [ 247.559993] nix-daemon[857]: accepted connection from pid 1373, user nix-grpc-daemon (trusted)6915worker2 # [ 247.711373] nix-daemon[857]: accepted connection from pid 1675, user nix-grpc-daemon (trusted)6916worker2 # [ 247.723203] nix-daemon[857]: reaped child process 1678, status = succeeded6917worker2 # [ 247.868529] nix-daemon[857]: accepted connection from pid 1680, user nix-grpc-daemon (trusted)6918worker2 # [ 247.880680] nix-daemon[857]: reaped child process 1683, status = succeeded6919lb # [ 247.882081] niks3-server[1118]: 2026/09/20 12:25:16 INFO Received uploads request method=POST path=/api/pending_closures6920worker2 # [ 247.904226] nix-grpc-daemon[1441]: time=2026-09-20T12:25:16.699Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6921worker2 # [ 247.906086] nix-grpc-daemon[1441]: time=2026-09-20T12:25:16.701Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"6922lb # [ 247.912165] niks3-server[1118]: 2026/09/20 12:25:16 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst6923lb # [ 247.922954] niks3-server[1118]: 2026/09/20 12:25:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6924lb # [ 247.928552] niks3-server[1118]: 2026/09/20 12:25:16 INFO Signed narinfos id=66 count=16925worker2 # [ 247.945515] nix-grpc-daemon[1441]: time=2026-09-20T12:25:16.740Z level=INFO msg="Uploading 1 narinfos"6926lb # [ 247.936244] niks3-server[1118]: 2026/09/20 12:25:16 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls6927lb # [ 247.947625] niks3-server[1118]: 2026/09/20 12:25:16 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6928worker2 # [ 247.968109] nix-grpc-daemon[1441]: time=2026-09-20T12:25:16.763Z level=INFO msg="Upload complete. (397ms)"6929lb # [ 247.954463] niks3-server[1118]: 2026/09/20 12:25:16 INFO Completed upload id=666930lb # [ 247.957239] niks3-server[1118]: 2026/09/20 12:25:16 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo6931worker2 # [ 247.971216] nix-grpc-daemon[1373]: ts=2026-09-20T12:25:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35124 duration_s=0 paths=1 nar_bytes_in=6246932worker2 # [ 247.976984] nix-grpc-daemon[1373]: ts=2026-09-20T12:25:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:351246933worker2 # [ 247.985252] nix-daemon[857]: accepted connection from pid 1373, user nix-grpc-daemon (trusted)6934worker2 # [ 247.990115] nix-daemon[857]: reaped child process 1673, status = succeeded6935client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...6936worker2 # [ 247.995781] nix-daemon[857]: accepted connection from pid 1373, user nix-grpc-daemon (trusted)6937client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6938lb # [ 248.040276] envoy[979]: conn peer=[2001:db8:1::1]:35668 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27184 tx=10687 ms=185116939lb # [ 248.044137] envoy[979]: conn peer=[2001:db8:1::1]:54736 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=878 tx=481 ms=456940client # farm-b> LOG-farm-b-drain6941worker2 # [ 249.410304] nix-daemon[857]: reaped child process 1687, status = succeeded6942worker2 # [ 249.545577] nix-daemon[857]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6943worker2 # [ 249.557938] nix-daemon[857]: reaped child process 1705, status = succeeded6944lb # [ 249.559835] niks3-server[1118]: 2026/09/20 12:25:18 INFO Received uploads request method=POST path=/api/pending_closures6945worker2 # [ 249.577899] nix-grpc-daemon[1441]: time=2026-09-20T12:25:18.373Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6946worker2 # [ 249.579456] nix-grpc-daemon[1441]: time=2026-09-20T12:25:18.375Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"6947lb # [ 249.582989] niks3-server[1118]: 2026/09/20 12:25:18 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst6948lb # [ 249.586989] niks3-server[1118]: 2026/09/20 12:25:18 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6949lb # [ 249.588949] niks3-server[1118]: 2026/09/20 12:25:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6950lb # [ 249.591564] niks3-server[1118]: 2026/09/20 12:25:18 INFO Signed narinfos id=67 count=16951worker2 # [ 249.604983] nix-grpc-daemon[1441]: time=2026-09-20T12:25:18.400Z level=INFO msg="Uploading 1 narinfos"6952lb # [ 249.594513] niks3-server[1118]: 2026/09/20 12:25:18 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls6953lb # [ 249.598522] niks3-server[1118]: 2026/09/20 12:25:18 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6954lb # [ 249.602108] niks3-server[1118]: 2026/09/20 12:25:18 INFO Completed upload id=676955worker2 # [ 249.615911] nix-grpc-daemon[1441]: time=2026-09-20T12:25:18.411Z level=INFO msg="Upload complete. (213ms)"6956lb # [ 249.603742] niks3-server[1118]: 2026/09/20 12:25:18 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo6957worker2 # [ 249.618254] nix-grpc-daemon[1373]: ts=2026-09-20T12:25:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35124 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=11 outputs=16958worker2 # [ 249.625338] nix-grpc-daemon[1373]: ts=2026-09-20T12:25:18Z level=debug event=want client=3 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=06959worker2 # [ 249.627352] nix-grpc-daemon[1373]: ts=2026-09-20T12:25:18Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=16960worker2 # [ 249.631189] nix-daemon[857]: reaped child process 1685, status = succeeded6961worker2 # [ 249.668156] nix-grpc-daemon[1373]: ts=2026-09-20T12:25:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:351246962worker2 # [ 249.676693] nix-grpc-daemon[1373]: ts=2026-09-20T12:25:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:351246963worker2 # [ 249.685278] nix-grpc-daemon[1441]: time=2026-09-20T12:25:18.480Z level=INFO msg="All 2 paths already cached"6964client # copying 1 paths to 'lb:50051'...6965client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...6966worker2 # [ 249.691829] nix-grpc-daemon[1373]: ts=2026-09-20T12:25:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:351246967worker2 # [ 249.700125] nix-daemon[857]: accepted connection from pid 1373, user nix-grpc-daemon (trusted)6968worker2 # [ 249.856815] nix-daemon[857]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)6969worker2 # [ 249.869893] nix-daemon[857]: reaped child process 1714, status = succeeded6970worker2 # [ 250.017636] nix-daemon[857]: accepted connection from pid 1716, user nix-grpc-daemon (trusted)6971worker2 # [ 250.030560] nix-daemon[857]: reaped child process 1719, status = succeeded6972lb # [ 250.026077] niks3-server[1118]: 2026/09/20 12:25:18 INFO Received uploads request method=POST path=/api/pending_closures6973worker2 # [ 250.044175] nix-grpc-daemon[1441]: time=2026-09-20T12:25:18.839Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6974worker2 # [ 250.046053] nix-grpc-daemon[1441]: time=2026-09-20T12:25:18.841Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"6975lb # [ 250.041080] niks3-server[1118]: 2026/09/20 12:25:18 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst6976lb # [ 250.043235] niks3-server[1118]: 2026/09/20 12:25:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6977lb # [ 250.045275] niks3-server[1118]: 2026/09/20 12:25:18 INFO Signed narinfos id=68 count=16978worker2 # [ 250.059068] nix-grpc-daemon[1441]: time=2026-09-20T12:25:18.854Z level=INFO msg="Uploading 1 narinfos"6979lb # [ 250.050271] niks3-server[1118]: 2026/09/20 12:25:18 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls6980lb # [ 250.054478] niks3-server[1118]: 2026/09/20 12:25:18 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6981lb # [ 250.057817] niks3-server[1118]: 2026/09/20 12:25:18 INFO Completed upload id=686982worker2 # [ 250.071496] nix-grpc-daemon[1441]: time=2026-09-20T12:25:18.867Z level=INFO msg="Upload complete. (364ms)"6983lb # [ 250.060428] niks3-server[1118]: 2026/09/20 12:25:18 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo6984worker2 # [ 250.074136] nix-grpc-daemon[1373]: ts=2026-09-20T12:25:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35124 duration_s=0 paths=1 nar_bytes_in=7526985worker2 # [ 250.076494] nix-grpc-daemon[1373]: ts=2026-09-20T12:25:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:351246986worker2 # [ 250.082826] nix-daemon[857]: accepted connection from pid 1373, user nix-grpc-daemon (trusted)6987worker2 # [ 250.085326] nix-daemon[857]: reaped child process 1709, status = succeeded6988worker2 # [ 250.090339] nix-daemon[857]: accepted connection from pid 1373, user nix-grpc-daemon (trusted)6989client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...6990client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6991client # farm-top> LOG-farm-top-drain6992worker2 # [ 250.913818] nix-daemon[857]: reaped child process 1723, status = succeeded6993worker2 # [ 251.069727] nix-daemon[857]: accepted connection from pid 1738, user nix-grpc-daemon (trusted)6994worker2 # [ 251.081875] nix-daemon[857]: reaped child process 1741, status = succeeded6995lb # [ 251.087173] niks3-server[1118]: 2026/09/20 12:25:19 INFO Received uploads request method=POST path=/api/pending_closures6996worker2 # [ 251.107539] nix-grpc-daemon[1441]: time=2026-09-20T12:25:19.902Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6997worker2 # [ 251.108991] nix-grpc-daemon[1441]: time=2026-09-20T12:25:19.904Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"6998lb # [ 251.118190] niks3-server[1118]: 2026/09/20 12:25:19 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6999lb # [ 251.130603] niks3-server[1118]: 2026/09/20 12:25:19 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7000lb # [ 251.145375] niks3-server[1118]: 2026/09/20 12:25:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign7001lb # [ 251.149086] niks3-server[1118]: 2026/09/20 12:25:19 INFO Signed narinfos id=69 count=17002worker2 # [ 251.165690] nix-grpc-daemon[1441]: time=2026-09-20T12:25:19.961Z level=INFO msg="Uploading 1 narinfos"7003lb # [ 251.155634] niks3-server[1118]: 2026/09/20 12:25:19 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7004lb # [ 251.164510] niks3-server[1118]: 2026/09/20 12:25:19 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete7005lb # [ 251.166893] niks3-server[1118]: 2026/09/20 12:25:19 INFO Completed upload id=697006worker2 # [ 251.182145] nix-grpc-daemon[1441]: time=2026-09-20T12:25:19.977Z level=INFO msg="Upload complete. (283ms)"7007lb # [ 251.170513] niks3-server[1118]: 2026/09/20 12:25:19 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7008worker2 # [ 251.187782] nix-grpc-daemon[1373]: ts=2026-09-20T12:25:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35124 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=12 outputs=17009worker2 # [ 251.201434] nix-daemon[857]: reaped child process 1721, status = succeeded7010client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&2, in 5.84 seconds)7011client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath7012client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath, in 0.20 seconds)7013worker1: must succeed: rm /nix/.rw-store/fill7014worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.12 seconds)7015worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7016worker1 # [ 251.937981] nix-grpc-daemon[1712]: ts=2026-09-20T12:25:20Z level=info event=healthy reason=min_free7017worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.10 seconds)7018(finished: subtest: low disk drains a worker and builds go to the other, in 9.47 seconds)7019(finished: run the VM test script, in 253.65 seconds)7020test script finished in 253.90s7021cleanup7022kill QemuMachine (pid 49)7023client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7024kill QemuMachine (pid 50)7025lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7026kill QemuMachine (pid 51)7027worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7028kill QemuMachine (pid 52)7029worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7030(finished: cleanup, in 1.19 seconds)7031additionally exposed symbols:7032 client, lb, worker1, worker2,7033 vlan1,7034 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh

succeeded vm-test-run-nix-grpc-farm checks.x86_64-linux.farm · build #312 · raw

succeeded `vm-test-run-nix-grpc-farm` `checks.x86_64-linux.farm` · build #312 · raw