vm-test-run-nix-grpc-farm log

succeeded `vm-test-run-nix-grpc-farm` `checks.x86_64-linux.farm` · build #299 · raw

1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker2 # Disk image does not exist, creating the virtualisation disk image...13worker2: QEMU running (pid 52)14lb: QEMU running (pid 50)15worker2 # Formatting '/build/vm-state-worker2/tmp.Pv6vRMCBzU', fmt=raw size=107374182416lb # Disk image does not exist, creating the virtualisation disk image...17worker1: QEMU running (pid 51)18worker1 # Disk image does not exist, creating the virtualisation disk image...19worker2 # mke2fs 1.47.4 (6-Mar-2025)20worker1 # Formatting '/build/vm-state-worker1/tmp.KbDqrI5e8B', fmt=raw size=107374182421client: QEMU running (pid 49)22client # Disk image does not exist, creating the virtualisation disk image...23worker1 # mke2fs 1.47.4 (6-Mar-2025)24lb # Formatting '/build/vm-state-lb/tmp.9OiiwlmmFF', fmt=raw size=107374182425(finished: start all VMs, in 0.18 seconds)26client # Formatting '/build/vm-state-client/tmp.mn2lF2UfkP', fmt=raw size=107374182427lb # mke2fs 1.47.4 (6-Mar-2025)28client: waiting for unit network-addresses-eth1.service29worker2 # Discarding device blocks: 0/262144 done30lb # Discarding device blocks: 0/262144 done31client: waiting for the VM to finish booting32worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes33lb # Creating filesystem with 262144 4k blocks and 65536 inodes34client # mke2fs 1.47.4 (6-Mar-2025)35worker1 # Discarding device blocks: 0/262144 done36worker2 # Filesystem UUID: 7cd2a496-ae42-4af2-9593-4db24233d52f37lb # Filesystem UUID: 72841ba1-10d6-45a5-8711-8c7b788199ac38client # Discarding device blocks: 0/262144 done39worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes40worker2 # Superblock backups stored on blocks:41lb # Superblock backups stored on blocks:42client # Creating filesystem with 262144 4k blocks and 65536 inodes43worker1 # Filesystem UUID: 892b74b5-6ba7-4e8b-850e-77c934126f9e44worker2 # 32768, 98304, 163840, 22937645lb # 32768, 98304, 163840, 22937646client # Filesystem UUID: fd319dc1-c582-4f8c-b7ff-dee165f2df7447worker1 # Superblock backups stored on blocks:48worker2 # 49lb # 50client # Superblock backups stored on blocks:51worker1 # 32768, 98304, 163840, 22937652lb # Allocating group tables: 0/8 done53worker2 # Allocating group tables: 0/8 done54client # 32768, 98304, 163840, 22937655worker1 # 56lb # Writing inode tables: 0/8 done57worker2 # Writing inode tables: 0/8 done58client # 59worker1 # Allocating group tables: 0/8 done60lb # Creating journal (8192 blocks): done61worker2 # Creating journal (8192 blocks): done62client # Allocating group tables: 0/8 done63worker1 # Writing inode tables: 0/8 done64lb # Writing superblocks and filesystem accounting information: 0/8 done65worker2 # Writing superblocks and filesystem accounting information: 0/8 done66client # Writing inode tables: 0/8 done67worker1 # Creating journal (8192 blocks): done68lb # 69worker2 # 70client # Creating journal (8192 blocks): done71worker1 # Writing superblocks and filesystem accounting information: 0/8 done72lb # Virtualisation disk image created.73worker2 # Virtualisation disk image created.74client # Writing superblocks and filesystem accounting information: 0/8 done75worker1 # 76client # 77worker1 # Virtualisation disk image created.78client # Virtualisation disk image created.79worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)80lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)81client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)82worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)83worker1 # 84worker1 # 85worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0086worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...87worker1 # 88worker1 # 89worker1 # 90worker1 # 91lb # 92lb # 93worker1 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB0094lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0095lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...96lb # 97lb # 98lb # 99lb # 100worker1 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...101worker1 # 102worker1 # 103lb # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00104client # 105client # 106worker2 # 107worker2 # 108client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00109worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00110lb # Press Ctrl-B to configure iPXE (PCI 00:08.0)...111lb # 112lb # 113client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...114client # 115client # 116client # 117client # 118worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...119worker2 # 120worker2 # 121worker2 # 122worker2 # 123client # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00124worker2 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00125client # Press Ctrl-B to configure iPXE (PCI 00:08.0)...126client # 127client # 128worker2 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...129worker2 # 130worker2 # 131worker1 # Booting from ROM...132lb # Booting from ROM...133client # Booting from ROM...134worker2 # Booting from ROM...135worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026136worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/98yf21c3jz4dira9wimzcqdwlb0l32kr-nixos-system-worker1-test/init regInfo=/nix/store/rgb3swjz9fl8q4w8lswl5z8wlc1zgzv4-closure-info/registration console=ttyS0,115200n8 console=tty0137worker1 # [ 0.000000] BIOS-provided physical RAM map:138worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable139worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved140lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026141worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved142worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable143worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved144worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved145lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/xzrwmx591i25vlijx1m7lk76hjmbxra6-nixos-system-lb-test/init regInfo=/nix/store/5b1jdrkcd54zycf7j131lh8wai3gzg44-closure-info/registration console=ttyS0,115200n8 console=tty0146lb # [ 0.000000] BIOS-provided physical RAM map:147worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved148lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable149worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved150lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved151worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved152lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved153worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved154worker1 # [ 0.000000] NX (Execute Disable) protection: active155lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable156worker1 # [ 0.000000] APIC: Static calls initialized157worker1 # [ 0.000000] SMBIOS 2.8 present.158lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved159lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved160worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014161worker1 # [ 0.000000] DMI: Memory slots populated: 1/1162lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved163worker1 # [ 0.000000] Hypervisor detected: KVM164lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved165worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000166lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved167worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00168worker1 # [ 0.000001] kvm-clock: using sched offset of 452861606 cycles169client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026170lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved171lb # [ 0.000000] NX (Execute Disable) protection: active172lb # [ 0.000000] APIC: Static calls initialized173worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026174worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns175lb # [ 0.000000] SMBIOS 2.8 present.176worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor177worker1 # [ 0.000867] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000178lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014179worker1 # [ 0.000894] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs180client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/3yn33ipjwn92p38w4qwdnvyyq096zfzd-nixos-system-client-test/init regInfo=/nix/store/z9v5mszvmn20hi4xvr0fypzq6a1x1rya-closure-info/registration console=ttyS0,115200n8 console=tty0181lb # [ 0.000000] DMI: Memory slots populated: 1/1182lb # [ 0.000000] Hypervisor detected: KVM183client # [ 0.000000] BIOS-provided physical RAM map:184worker1 # [ 0.000896] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT185lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000186worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/2y6kwi1mdv3l7s63cygabx3gwjz592yw-nixos-system-worker2-test/init regInfo=/nix/store/rxnsaix8qkg1nyh96vsafrk0l5rbj4zn-closure-info/registration console=ttyS0,115200n8 console=tty0187worker1 # [ 0.002804] found SMP MP-table at [mem 0x000f5450-0x000f545f]188client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable189lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00190worker2 # [ 0.000000] BIOS-provided physical RAM map:191worker1 # [ 0.002815] Using GB pages for direct mapping192lb # [ 0.000001] kvm-clock: using sched offset of 458850865 cycles193client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved194worker1 # [ 0.002971] RAMDISK: [mem 0x5e367000-0x5ffcffff]195worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable196client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved197worker1 # [ 0.002982] ACPI: Early table checksum verification disabled198lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns199worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved200worker1 # [ 0.002985] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )201lb # [ 0.000005] tsc: Detected 2400.012 MHz processor202client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable203worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved204lb # [ 0.000814] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000205client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved206worker1 # [ 0.002989] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)207worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable208client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved209lb # [ 0.000844] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs210worker1 # [ 0.002993] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)211worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved212client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved213lb # [ 0.000847] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT214worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved215worker1 # [ 0.003000] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)216lb # [ 0.002754] found SMP MP-table at [mem 0x000f5450-0x000f545f]217client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved218worker1 # [ 0.003002] ACPI: FACS 0x000000005FFE0000 000040219lb # [ 0.002768] Using GB pages for direct mapping220worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved221client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved222lb # [ 0.002840] RAMDISK: [mem 0x5e367000-0x5ffcffff]223worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved224worker1 # [ 0.003003] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)225client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved226lb # [ 0.002850] ACPI: Early table checksum verification disabled227worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved228client # [ 0.000000] NX (Execute Disable) protection: active229worker1 # [ 0.003005] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)230lb # [ 0.002853] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )231client # [ 0.000000] APIC: Static calls initialized232worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved233lb # [ 0.002858] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)234worker1 # [ 0.003006] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)235worker2 # [ 0.000000] NX (Execute Disable) protection: active236client # [ 0.000000] SMBIOS 2.8 present.237worker2 # [ 0.000000] APIC: Static calls initialized238client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014239worker1 # [ 0.003008] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)240lb # [ 0.002863] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)241worker2 # [ 0.000000] SMBIOS 2.8 present.242client # [ 0.000000] DMI: Memory slots populated: 1/1243worker1 # [ 0.003009] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]244client # [ 0.000000] Hypervisor detected: KVM245lb # [ 0.002870] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)246worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014247lb # [ 0.002872] ACPI: FACS 0x000000005FFE0000 000040248client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000249worker1 # [ 0.003010] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]250worker2 # [ 0.000000] DMI: Memory slots populated: 1/1251client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00252worker2 # [ 0.000000] Hypervisor detected: KVM253lb # [ 0.002873] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)254worker1 # [ 0.003011] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]255client # [ 0.000000] kvm-clock: using sched offset of 451767050 cycles256worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000257worker1 # [ 0.003011] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]258worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00259lb # [ 0.002875] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)260client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns261worker1 # [ 0.003012] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]262worker2 # [ 0.000001] kvm-clock: using sched offset of 470997295 cycles263lb # [ 0.002877] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)264client # [ 0.000005] tsc: Detected 2400.012 MHz processor265worker1 # [ 0.003012] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]266client # [ 0.000818] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000267lb # [ 0.002878] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)268worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns269worker1 # [ 0.003013] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]270worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor271lb # [ 0.002879] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]272worker1 # [ 0.003246] No NUMA configuration found273client # [ 0.000845] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs274worker2 # [ 0.000827] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000275client # [ 0.000848] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT276lb # [ 0.002880] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]277worker1 # [ 0.003247] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]278client # [ 0.002744] found SMP MP-table at [mem 0x000f5450-0x000f545f]279worker2 # [ 0.000854] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs280worker1 # [ 0.003249] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]281lb # [ 0.002881] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]282worker1 # [ 0.005638] Zone ranges:283client # [ 0.002755] Using GB pages for direct mapping284worker2 # [ 0.000857] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT285lb # [ 0.002881] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]286worker1 # [ 0.005639] DMA [mem 0x0000000000001000-0x0000000000ffffff]287client # [ 0.002895] RAMDISK: [mem 0x5e366000-0x5ffcffff]288worker2 # [ 0.002764] found SMP MP-table at [mem 0x000f5450-0x000f545f]289client # [ 0.002903] ACPI: Early table checksum verification disabled290lb # [ 0.002882] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]291worker1 # [ 0.005640] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]292worker2 # [ 0.002775] Using GB pages for direct mapping293worker1 # [ 0.005641] Normal empty294worker1 # [ 0.005642] Device empty295client # [ 0.002907] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )296worker2 # [ 0.002902] RAMDISK: [mem 0x5e367000-0x5ffcffff]297lb # [ 0.002882] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]298worker1 # [ 0.005642] Movable zone start for each node299worker2 # [ 0.002911] ACPI: Early table checksum verification disabled300worker1 # [ 0.005643] Early memory node ranges301lb # [ 0.002883] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]302client # [ 0.002911] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)303worker2 # [ 0.002913] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )304lb # [ 0.003116] No NUMA configuration found305worker1 # [ 0.005643] node 0: [mem 0x0000000000001000-0x000000000009efff]306client # [ 0.002915] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)307lb # [ 0.003117] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]308worker1 # [ 0.005645] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]309worker2 # [ 0.002918] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)310lb # [ 0.003119] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]311client # [ 0.002922] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)312worker1 # [ 0.005646] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]313lb # [ 0.005492] Zone ranges:314worker2 # [ 0.002921] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)315client # [ 0.002924] ACPI: FACS 0x000000005FFE0000 000040316worker1 # [ 0.005665] On node 0, zone DMA: 1 pages in unavailable ranges317lb # [ 0.005493] DMA [mem 0x0000000000001000-0x0000000000ffffff]318worker2 # [ 0.002928] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)319client # [ 0.002926] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)320worker1 # [ 0.005921] On node 0, zone DMA: 97 pages in unavailable ranges321lb # [ 0.005494] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]322worker2 # [ 0.002930] ACPI: FACS 0x000000005FFE0000 000040323lb # [ 0.005495] Normal empty324worker1 # [ 0.031592] On node 0, zone DMA32: 40 pages in unavailable ranges325lb # [ 0.005496] Device empty326client # [ 0.002927] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)327worker1 # [ 0.032060] ACPI: PM-Timer IO Port: 0x608328lb # [ 0.005497] Movable zone start for each node329worker2 # [ 0.002932] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)330lb # [ 0.005498] Early memory node ranges331worker1 # [ 0.032071] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])332client # [ 0.002929] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)333worker2 # [ 0.002933] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)334lb # [ 0.005498] node 0: [mem 0x0000000000001000-0x000000000009efff]335worker1 # [ 0.032100] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23336client # [ 0.002930] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)337lb # [ 0.005499] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]338worker2 # [ 0.002935] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)339worker1 # [ 0.032102] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)340client # [ 0.002932] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]341lb # [ 0.005500] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]342worker1 # [ 0.032104] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)343worker2 # [ 0.002936] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)344lb # [ 0.005519] On node 0, zone DMA: 1 pages in unavailable ranges345client # [ 0.002933] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]346worker1 # [ 0.032105] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)347worker2 # [ 0.002938] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]348lb # [ 0.005787] On node 0, zone DMA: 97 pages in unavailable ranges349client # [ 0.002933] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]350worker1 # [ 0.032106] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)351worker2 # [ 0.002939] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]352lb # [ 0.031176] On node 0, zone DMA32: 40 pages in unavailable ranges353client # [ 0.002934] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]354lb # [ 0.031641] ACPI: PM-Timer IO Port: 0x608355worker1 # [ 0.032107] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)356worker2 # [ 0.002939] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]357client # [ 0.002934] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]358lb # [ 0.031654] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])359worker1 # [ 0.032109] ACPI: Using ACPI (MADT) for SMP configuration information360worker2 # [ 0.002940] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]361client # [ 0.002935] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]362worker1 # [ 0.032110] ACPI: HPET id: 0x8086a201 base: 0xfed00000363lb # [ 0.031682] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23364worker1 # [ 0.032113] TSC deadline timer available365worker2 # [ 0.002940] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]366client # [ 0.002935] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]367lb # [ 0.031685] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)368worker1 # [ 0.032117] CPU topo: Max. logical packages: 1369client # [ 0.003162] No NUMA configuration found370worker2 # [ 0.002941] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]371lb # [ 0.031686] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)372worker1 # [ 0.032118] CPU topo: Max. logical dies: 1373client # [ 0.003163] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]374worker1 # [ 0.032119] CPU topo: Max. dies per package: 1375worker2 # [ 0.002941] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]376lb # [ 0.031688] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)377worker1 # [ 0.032122] CPU topo: Max. threads per core: 1378worker2 # [ 0.003176] No NUMA configuration found379client # [ 0.003165] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]380client # [ 0.005461] Zone ranges:381lb # [ 0.031689] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)382worker1 # [ 0.032122] CPU topo: Num. cores per package: 1383worker2 # [ 0.003176] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]384client # [ 0.005462] DMA [mem 0x0000000000001000-0x0000000000ffffff]385worker1 # [ 0.032123] CPU topo: Num. threads per package: 1386lb # [ 0.031689] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)387worker2 # [ 0.003179] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]388worker2 # [ 0.005393] Zone ranges:389client # [ 0.005464] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]390worker1 # [ 0.032123] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs391lb # [ 0.031692] ACPI: Using ACPI (MADT) for SMP configuration information392client # [ 0.005466] Normal empty393client # [ 0.005466] Device empty394worker2 # [ 0.005394] DMA [mem 0x0000000000001000-0x0000000000ffffff]395lb # [ 0.031693] ACPI: HPET id: 0x8086a201 base: 0xfed00000396worker1 # [ 0.032141] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()397client # [ 0.005467] Movable zone start for each node398lb # [ 0.031697] TSC deadline timer available399worker2 # [ 0.005396] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]400client # [ 0.005468] Early memory node ranges401lb # [ 0.031701] CPU topo: Max. logical packages: 1402worker2 # [ 0.005397] Normal empty403worker1 # [ 0.032173] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]404worker2 # [ 0.005398] Device empty405lb # [ 0.031702] CPU topo: Max. logical dies: 1406client # [ 0.005468] node 0: [mem 0x0000000000001000-0x000000000009efff]407worker2 # [ 0.005399] Movable zone start for each node408worker1 # [ 0.032174] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]409lb # [ 0.031702] CPU topo: Max. dies per package: 1410worker2 # [ 0.005399] Early memory node ranges411client # [ 0.005469] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]412lb # [ 0.031706] CPU topo: Max. threads per core: 1413worker1 # [ 0.032176] [mem 0x60000000-0xafffffff] available for PCI devices414worker2 # [ 0.005400] node 0: [mem 0x0000000000001000-0x000000000009efff]415worker1 # [ 0.032177] Booting paravirtualized kernel on KVM416lb # [ 0.031706] CPU topo: Num. cores per package: 1417client # [ 0.005470] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]418lb # [ 0.031706] CPU topo: Num. threads per package: 1419worker2 # [ 0.005401] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]420client # [ 0.005490] On node 0, zone DMA: 1 pages in unavailable ranges421worker1 # [ 0.032180] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns422lb # [ 0.031707] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs423client # [ 0.005740] On node 0, zone DMA: 97 pages in unavailable ranges424worker2 # [ 0.005402] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]425worker2 # [ 0.005422] On node 0, zone DMA: 1 pages in unavailable ranges426lb # [ 0.031727] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()427client # [ 0.031301] On node 0, zone DMA32: 40 pages in unavailable ranges428worker1 # [ 0.036609] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1429client # [ 0.031757] ACPI: PM-Timer IO Port: 0x608430worker2 # [ 0.005685] On node 0, zone DMA: 97 pages in unavailable ranges431worker1 # [ 0.038691] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152432lb # [ 0.031760] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]433client # [ 0.031769] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])434worker2 # [ 0.031460] On node 0, zone DMA32: 40 pages in unavailable ranges435worker1 # [ 0.038745] kvm-guest: PV spinlocks disabled, single CPU436lb # [ 0.031761] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]437worker2 # [ 0.031918] ACPI: PM-Timer IO Port: 0x608438client # [ 0.031796] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23439lb # [ 0.031763] [mem 0x60000000-0xafffffff] available for PCI devices440worker2 # [ 0.031928] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])441client # [ 0.031799] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)442lb # [ 0.031764] Booting paravirtualized kernel on KVM443worker2 # [ 0.031956] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23444client # [ 0.031800] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)445lb # [ 0.031767] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns446client # [ 0.031802] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)447worker2 # [ 0.031958] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)448worker2 # [ 0.031960] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)449lb # [ 0.036273] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1450client # [ 0.031803] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)451worker2 # [ 0.031961] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)452worker1 # [ 0.038746] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/98yf21c3jz4dira9wimzcqdwlb0l32kr-nixos-system-worker1-test/init regInfo=/nix/store/rgb3swjz9fl8q4w8lswl5z8wlc1zgzv4-closure-info/registration console=ttyS0,115200n8 console=tty0453client # [ 0.031803] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)454lb # [ 0.038349] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152455lb # [ 0.038392] kvm-guest: PV spinlocks disabled, single CPU456worker2 # [ 0.031962] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)457client # [ 0.031806] ACPI: Using ACPI (MADT) for SMP configuration information458worker1 # [ 0.038839] Unknown kernel command line parameters "regInfo=/nix/store/rgb3swjz9fl8q4w8lswl5z8wlc1zgzv4-closure-info/registration", will be passed to user space.459client # [ 0.031807] ACPI: HPET id: 0x8086a201 base: 0xfed00000460worker1 # [ 0.038852] random: crng init done461worker2 # [ 0.031963] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)462client # [ 0.031811] TSC deadline timer available463worker1 # [ 0.038853] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes464client # [ 0.031815] CPU topo: Max. logical packages: 1465worker2 # [ 0.031965] ACPI: Using ACPI (MADT) for SMP configuration information466client # [ 0.031815] CPU topo: Max. logical dies: 1467worker2 # [ 0.031966] ACPI: HPET id: 0x8086a201 base: 0xfed00000468worker1 # [ 0.040917] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)469worker2 # [ 0.031970] TSC deadline timer available470client # [ 0.031816] CPU topo: Max. dies per package: 1471worker2 # [ 0.031974] CPU topo: Max. logical packages: 1472client # [ 0.031819] CPU topo: Max. threads per core: 1473worker1 # [ 0.041956] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)474lb # [ 0.038394] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/xzrwmx591i25vlijx1m7lk76hjmbxra6-nixos-system-lb-test/init regInfo=/nix/store/5b1jdrkcd54zycf7j131lh8wai3gzg44-closure-info/registration console=ttyS0,115200n8 console=tty0475worker1 # [ 0.041987] Fallback order for Node 0: 0476client # [ 0.031820] CPU topo: Num. cores per package: 1477worker2 # [ 0.031974] CPU topo: Max. logical dies: 1478worker2 # [ 0.031975] CPU topo: Max. dies per package: 1479client # [ 0.031820] CPU topo: Num. threads per package: 1480worker1 # [ 0.041990] Built 1 zonelists, mobility grouping on. Total pages: 393078481worker2 # [ 0.031978] CPU topo: Max. threads per core: 1482worker1 # [ 0.041991] Policy zone: DMA32483client # [ 0.031820] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs484lb # [ 0.038490] Unknown kernel command line parameters "regInfo=/nix/store/5b1jdrkcd54zycf7j131lh8wai3gzg44-closure-info/registration", will be passed to user space.485worker2 # [ 0.031979] CPU topo: Num. cores per package: 1486lb # [ 0.038503] random: crng init done487worker1 # [ 0.044570] mem auto-init: stack:all(zero), heap alloc:on, heap free:off488client # [ 0.031841] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()489worker2 # [ 0.031979] CPU topo: Num. threads per package: 1490lb # [ 0.038504] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes491worker1 # [ 0.047563] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1492worker2 # [ 0.031979] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs493client # [ 0.031874] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]494worker1 # [ 0.050781] allocated 3145728 bytes of page_ext495lb # [ 0.040537] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)496worker2 # [ 0.031998] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()497worker1 # [ 0.060600] ftrace: allocating 48731 entries in 192 pages498client # [ 0.031875] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]499worker1 # [ 0.060601] ftrace: allocated 192 pages with 2 groups500lb # [ 0.041561] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)501worker2 # [ 0.032028] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]502client # [ 0.031876] [mem 0x60000000-0xafffffff] available for PCI devices503worker1 # [ 0.061442] Dynamic Preempt: lazy504lb # [ 0.041592] Fallback order for Node 0: 0505client # [ 0.031878] Booting paravirtualized kernel on KVM506worker1 # [ 0.061571] rcu: Preemptible hierarchical RCU implementation.507worker2 # [ 0.032030] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]508lb # [ 0.041594] Built 1 zonelists, mobility grouping on. Total pages: 393078509worker1 # [ 0.061571] rcu: RCU event tracing is enabled.510lb # [ 0.041595] Policy zone: DMA32511worker2 # [ 0.032031] [mem 0x60000000-0xafffffff] available for PCI devices512client # [ 0.031881] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns513worker1 # [ 0.061572] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.514lb # [ 0.044119] mem auto-init: stack:all(zero), heap alloc:on, heap free:off515worker2 # [ 0.032032] Booting paravirtualized kernel on KVM516client # [ 0.036333] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1517lb # [ 0.047220] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1518worker1 # [ 0.061573] Trampoline variant of Tasks RCU enabled.519worker1 # [ 0.061574] Rude variant of Tasks RCU enabled.520worker2 # [ 0.032035] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns521lb # [ 0.050411] allocated 3145728 bytes of page_ext522client # [ 0.038422] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152523worker1 # [ 0.061574] Tracing variant of Tasks RCU enabled.524lb # [ 0.060277] ftrace: allocating 48731 entries in 192 pages525client # [ 0.038466] kvm-guest: PV spinlocks disabled, single CPU526worker2 # [ 0.036516] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1527lb # [ 0.060280] ftrace: allocated 192 pages with 2 groups528worker1 # [ 0.061575] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.529worker2 # [ 0.038619] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152530lb # [ 0.061157] Dynamic Preempt: lazy531worker1 # [ 0.061575] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1532worker2 # [ 0.038664] kvm-guest: PV spinlocks disabled, single CPU533lb # [ 0.061333] rcu: Preemptible hierarchical RCU implementation.534lb # [ 0.061334] rcu: RCU event tracing is enabled.535worker1 # [ 0.061585] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.536lb # [ 0.061334] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.537lb # [ 0.061336] Trampoline variant of Tasks RCU enabled.538worker1 # [ 0.061586] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.539client # [ 0.038468] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/3yn33ipjwn92p38w4qwdnvyyq096zfzd-nixos-system-client-test/init regInfo=/nix/store/z9v5mszvmn20hi4xvr0fypzq6a1x1rya-closure-info/registration console=ttyS0,115200n8 console=tty0540lb # [ 0.061336] Rude variant of Tasks RCU enabled.541worker1 # [ 0.061587] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.542lb # [ 0.061336] Tracing variant of Tasks RCU enabled.543worker2 # [ 0.038666] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/2y6kwi1mdv3l7s63cygabx3gwjz592yw-nixos-system-worker2-test/init regInfo=/nix/store/rxnsaix8qkg1nyh96vsafrk0l5rbj4zn-closure-info/registration console=ttyS0,115200n8 console=tty0544worker1 # [ 0.065983] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16545client # [ 0.038566] Unknown kernel command line parameters "regInfo=/nix/store/z9v5mszvmn20hi4xvr0fypzq6a1x1rya-closure-info/registration", will be passed to user space.546lb # [ 0.061337] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.547client # [ 0.038579] random: crng init done548worker1 # [ 0.066268] rcu: srcu_init: Setting srcu_struct sizes based on contention.549lb # [ 0.061338] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1550client # [ 0.038580] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes551worker2 # [ 0.038759] Unknown kernel command line parameters "regInfo=/nix/store/rxnsaix8qkg1nyh96vsafrk0l5rbj4zn-closure-info/registration", will be passed to user space.552worker1 # [ 0.066275] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns553worker2 # [ 0.038771] random: crng init done554lb # [ 0.061347] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.555client # [ 0.040676] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)556worker2 # [ 0.038772] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes557lb # [ 0.061349] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.558worker1 # [ 0.066377] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)559client # [ 0.041727] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)560worker1 # [ 0.069909] Console: colour VGA+ 80x25561client # [ 0.041760] Fallback order for Node 0: 0562worker2 # [ 0.040868] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)563worker1 # [ 0.069912] printk: legacy console [tty0] enabled564lb # [ 0.061350] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.565client # [ 0.041763] Built 1 zonelists, mobility grouping on. Total pages: 393078566worker1 # [ 0.099329] printk: legacy console [ttyS0] enabled567worker2 # [ 0.042085] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)568lb # [ 0.065631] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16569client # [ 0.041764] Policy zone: DMA32570worker2 # [ 0.042117] Fallback order for Node 0: 0571worker1 # [ 0.196233] ACPI: Core revision 20250807572client # [ 0.044343] mem auto-init: stack:all(zero), heap alloc:on, heap free:off573lb # [ 0.065902] rcu: srcu_init: Setting srcu_struct sizes based on contention.574worker2 # [ 0.042119] Built 1 zonelists, mobility grouping on. Total pages: 393078575client # [ 0.047553] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1576worker2 # [ 0.042120] Policy zone: DMA32577lb # [ 0.065911] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns578client # [ 0.050774] allocated 3145728 bytes of page_ext579worker1 # [ 0.197116] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns580worker2 # [ 0.044567] mem auto-init: stack:all(zero), heap alloc:on, heap free:off581client # [ 0.060590] ftrace: allocating 48731 entries in 192 pages582worker2 # [ 0.047601] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1583worker1 # [ 0.198617] APIC: Switch to symmetric I/O mode setup584lb # [ 0.066016] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)585client # [ 0.060592] ftrace: allocated 192 pages with 2 groups586lb # [ 0.069537] Console: colour VGA+ 80x25587worker2 # [ 0.050851] allocated 3145728 bytes of page_ext588client # [ 0.061421] Dynamic Preempt: lazy589worker1 # [ 0.199557] x2apic enabled590lb # [ 0.069540] printk: legacy console [tty0] enabled591worker2 # [ 0.060611] ftrace: allocating 48731 entries in 192 pages592client # [ 0.061545] rcu: Preemptible hierarchical RCU implementation.593lb # [ 0.098931] printk: legacy console [ttyS0] enabled594worker2 # [ 0.060613] ftrace: allocated 192 pages with 2 groups595client # [ 0.061546] rcu: RCU event tracing is enabled.596worker1 # [ 0.200302] APIC: Switched APIC routing to: physical x2apic597worker2 # [ 0.061461] Dynamic Preempt: lazy598lb # [ 0.194400] ACPI: Core revision 20250807599client # [ 0.061547] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.600worker2 # [ 0.061576] rcu: Preemptible hierarchical RCU implementation.601client # [ 0.061548] Trampoline variant of Tasks RCU enabled.602worker2 # [ 0.061577] rcu: RCU event tracing is enabled.603client # [ 0.061549] Rude variant of Tasks RCU enabled.604worker1 # [ 0.202176] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1605lb # [ 0.195276] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns606worker2 # [ 0.061577] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.607client # [ 0.061549] Tracing variant of Tasks RCU enabled.608worker2 # [ 0.061578] Trampoline variant of Tasks RCU enabled.609lb # [ 0.196815] APIC: Switch to symmetric I/O mode setup610client # [ 0.061550] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.611worker2 # [ 0.061579] Rude variant of Tasks RCU enabled.612worker1 # [ 0.203148] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns613lb # [ 0.197792] x2apic enabled614worker2 # [ 0.061579] Tracing variant of Tasks RCU enabled.615client # [ 0.061551] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1616lb # [ 0.198562] APIC: Switched APIC routing to: physical x2apic617worker1 # [ 0.204802] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)618worker2 # [ 0.061580] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.619client # [ 0.061608] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.620worker2 # [ 0.061581] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1621worker1 # [ 0.205887] x86/cpu: User Mode Instruction Prevention (UMIP) activated622client # [ 0.061610] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.623lb # [ 0.200392] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1624worker2 # [ 0.061638] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.625worker1 # [ 0.206928] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127626client # [ 0.061611] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.627worker2 # [ 0.061639] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.628client # [ 0.065897] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16629worker1 # [ 0.208486] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0630lb # [ 0.201346] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns631client # [ 0.066169] rcu: srcu_init: Setting srcu_struct sizes based on contention.632worker2 # [ 0.061640] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.633worker2 # [ 0.066008] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16634lb # [ 0.202965] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)635client # [ 0.066182] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns636worker1 # [ 0.208803] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto637worker2 # [ 0.066457] rcu: srcu_init: Setting srcu_struct sizes based on contention.638lb # [ 0.204049] x86/cpu: User Mode Instruction Prevention (UMIP) activated639client # [ 0.066284] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)640worker1 # [ 0.210799] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl641client # [ 0.069830] Console: colour VGA+ 80x25642worker2 # [ 0.066463] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns643lb # [ 0.205091] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127644client # [ 0.069833] printk: legacy console [tty0] enabled645worker1 # [ 0.211799] Transient Scheduler Attacks: Vulnerable: No microcode646client # [ 0.099561] printk: legacy console [ttyS0] enabled647lb # [ 0.205963] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0648worker2 # [ 0.066566] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)649worker1 # [ 0.213495] Spectre V2 : Mitigation: Enhanced / Automatic IBRS650worker2 # [ 0.070103] Console: colour VGA+ 80x25651client # [ 0.195372] ACPI: Core revision 20250807652worker2 # [ 0.070106] printk: legacy console [tty0] enabled653worker1 # [ 0.213799] Speculative Return Stack Overflow: Mitigation: Safe RET654worker2 # [ 0.099944] printk: legacy console [ttyS0] enabled655lb # [ 0.206966] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto656client # [ 0.196232] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns657worker2 # [ 0.196255] ACPI: Core revision 20250807658lb # [ 0.207962] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl659client # [ 0.197731] APIC: Switch to symmetric I/O mode setup660client # [ 0.198719] x2apic enabled661worker2 # [ 0.197155] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns662lb # [ 0.208962] Transient Scheduler Attacks: Vulnerable: No microcode663client # [ 0.199496] APIC: Switched APIC routing to: physical x2apic664lb # [ 0.209962] Spectre V2 : Mitigation: Enhanced / Automatic IBRS665worker2 # [ 0.198683] APIC: Switch to symmetric I/O mode setup666worker1 # [ 0.214799] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization667worker2 # [ 0.199641] x2apic enabled668lb # [ 0.210962] Speculative Return Stack Overflow: Mitigation: Safe RET669worker1 # [ 0.216807] Spectre V2 : Enabling IBPB for BPF670worker2 # [ 0.200396] APIC: Switched APIC routing to: physical x2apic671client # [ 0.201267] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1672worker1 # [ 0.217800] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier673worker1 # [ 0.218799] active return thunk: srso_alias_return_thunk674client # [ 0.202267] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns675worker2 # [ 0.202217] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1676worker1 # [ 0.219820] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'677lb # [ 0.211962] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization678client # [ 0.203988] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)679worker2 # [ 0.203213] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns680worker1 # [ 0.220799] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'681lb # [ 0.213969] Spectre V2 : Enabling IBPB for BPF682client # [ 0.205071] x86/cpu: User Mode Instruction Prevention (UMIP) activated683worker1 # [ 0.222532] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'684worker2 # [ 0.204850] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)685lb # [ 0.214963] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier686client # [ 0.206129] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127687worker1 # [ 0.223529] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'688worker2 # [ 0.205934] x86/cpu: User Mode Instruction Prevention (UMIP) activated689lb # [ 0.215963] active return thunk: srso_alias_return_thunk690client # [ 0.207664] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0691worker1 # [ 0.224538] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'692worker2 # [ 0.206973] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127693lb # [ 0.216983] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'694worker1 # [ 0.225540] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'695worker2 # [ 0.207847] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0696client # [ 0.207989] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto697lb # [ 0.217962] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'698worker1 # [ 0.226562] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'699lb # [ 0.218962] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'700client # [ 0.209985] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl701worker2 # [ 0.208850] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto702worker1 # [ 0.227799] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256703lb # [ 0.219962] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'704client # [ 0.210986] Transient Scheduler Attacks: Vulnerable: No microcode705worker1 # [ 0.228799] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64706worker2 # [ 0.210847] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl707lb # [ 0.220962] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'708client # [ 0.212692] Spectre V2 : Mitigation: Enhanced / Automatic IBRS709worker1 # [ 0.229715] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512710worker2 # [ 0.211847] Transient Scheduler Attacks: Vulnerable: No microcode711client # [ 0.212985] Speculative Return Stack Overflow: Mitigation: Safe RET712lb # [ 0.221962] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'713worker1 # [ 0.230499] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024714worker2 # [ 0.212846] Spectre V2 : Mitigation: Enhanced / Automatic IBRS715lb # [ 0.222962] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'716worker1 # [ 0.231518] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8717worker2 # [ 0.213846] Speculative Return Stack Overflow: Mitigation: Safe RET718lb # [ 0.223962] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256719worker1 # [ 0.232536] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.720lb # [ 0.225714] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64721client # [ 0.213985] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization722lb # [ 0.226690] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512723client # [ 0.215993] Spectre V2 : Enabling IBPB for BPF724lb # [ 0.227720] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024725worker2 # [ 0.214846] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization726client # [ 0.216986] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier727lb # [ 0.228708] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8728worker2 # [ 0.216855] Spectre V2 : Enabling IBPB for BPF729client # [ 0.217986] active return thunk: srso_alias_return_thunk730worker2 # [ 0.217848] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier731lb # [ 0.229674] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.732client # [ 0.219644] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'733worker2 # [ 0.218847] active return thunk: srso_alias_return_thunk734client # [ 0.220892] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'735worker2 # [ 0.219869] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'736client # [ 0.221985] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'737worker2 # [ 0.220847] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'738client # [ 0.222971] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'739worker2 # [ 0.222627] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'740client # [ 0.223985] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'741worker2 # [ 0.223642] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'742client # [ 0.224977] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'743worker2 # [ 0.224632] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'744client # [ 0.225985] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'745worker2 # [ 0.225627] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'746client # [ 0.226985] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256747worker2 # [ 0.226664] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'748client # [ 0.227985] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64749worker2 # [ 0.227847] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256750client # [ 0.228985] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512751worker2 # [ 0.228847] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64752client # [ 0.229985] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024753worker2 # [ 0.229846] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512754client # [ 0.230985] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8755worker2 # [ 0.230847] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024756client # [ 0.231985] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.757worker2 # [ 0.231847] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8758worker2 # [ 0.232846] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.759worker1 # [ 0.266960] Freeing SMP alternatives memory: 44K760worker1 # [ 0.267773] pid_max: default: 32768 minimum: 301761worker1 # [ 0.268479] LSM: initializing lsm=capability,landlock,yama,bpf,ima762worker1 # [ 0.269813] landlock: Up and running.763worker1 # [ 0.270459] Yama: becoming mindful.764lb # [ 0.263512] Freeing SMP alternatives memory: 44K765worker1 # [ 0.271012] LSM support for eBPF active766lb # [ 0.263965] pid_max: default: 32768 minimum: 301767worker1 # [ 0.271927] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)768lb # [ 0.265056] LSM: initializing lsm=capability,landlock,yama,bpf,ima769lb # [ 0.266067] landlock: Up and running.770worker1 # [ 0.272840] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)771lb # [ 0.266692] Yama: becoming mindful.772lb # [ 0.267626] LSM support for eBPF active773worker1 # [ 0.275712] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)774lb # [ 0.268573] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)775worker1 # [ 0.276298] Performance Events: Fam17h+ core perfctr, AMD PMU driver.776lb # [ 0.269967] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)777worker1 # [ 0.276804] ... version: 2778worker1 # [ 0.277509] ... bit width: 48779worker1 # [ 0.277801] ... generic counters: 6780lb # [ 0.272672] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)781worker1 # [ 0.278546] ... generic bitmap: 000000000000003f782worker1 # [ 0.278851] ... fixed-purpose counters: 0783lb # [ 0.273515] Performance Events: Fam17h+ core perfctr, AMD PMU driver.784worker1 # [ 0.279547] ... fixed-purpose bitmap: 0000000000000000785lb # [ 0.273967] ... version: 2786worker1 # [ 0.279801] ... value mask: 0000ffffffffffff787lb # [ 0.274698] ... bit width: 48788worker1 # [ 0.280691] ... max period: 00007fffffffffff789lb # [ 0.274965] ... generic counters: 6790client # [ 0.266702] Freeing SMP alternatives memory: 44K791worker1 # [ 0.281500] ... global_ctrl mask: 000000000000003f792lb # [ 0.275689] ... generic bitmap: 000000000000003f793client # [ 0.266989] pid_max: default: 32768 minimum: 301794worker1 # [ 0.281924] signal: max sigframe size: 3376795lb # [ 0.276013] ... fixed-purpose counters: 0796client # [ 0.268069] LSM: initializing lsm=capability,landlock,yama,bpf,ima797worker2 # [ 0.267622] Freeing SMP alternatives memory: 44K798worker1 # [ 0.282718] rcu: Hierarchical SRCU implementation.799lb # [ 0.276725] ... fixed-purpose bitmap: 0000000000000000800client # [ 0.269096] landlock: Up and running.801worker2 # [ 0.268378] pid_max: default: 32768 minimum: 301802worker1 # [ 0.283388] rcu: Max phase no-delay instances is 400.803lb # [ 0.276965] ... value mask: 0000ffffffffffff804client # [ 0.269986] Yama: becoming mindful.805worker2 # [ 0.268936] LSM: initializing lsm=capability,landlock,yama,bpf,ima806lb # [ 0.277826] ... max period: 00007fffffffffff807client # [ 0.270783] LSM support for eBPF active808worker2 # [ 0.269963] landlock: Up and running.809lb # [ 0.278694] ... global_ctrl mask: 000000000000003f810worker2 # [ 0.270848] Yama: becoming mindful.811client # [ 0.271556] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)812lb # [ 0.279082] signal: max sigframe size: 3376813worker2 # [ 0.271867] LSM support for eBPF active814lb # [ 0.279834] rcu: Hierarchical SRCU implementation.815client # [ 0.272021] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)816lb # [ 0.280522] rcu: Max phase no-delay instances is 400.817worker2 # [ 0.272656] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)818worker1 # [ 0.288319] smp: Bringing up secondary CPUs ...819worker2 # [ 0.273877] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)820worker1 # [ 0.288815] smp: Brought up 1 node, 1 CPU821client # [ 0.275139] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)822worker1 # [ 0.289475] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)823client # [ 0.276691] Performance Events: Fam17h+ core perfctr, AMD PMU driver.824worker2 # [ 0.276105] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)825client # [ 0.276994] ... version: 2826worker1 # [ 0.290023] Memory: 1454560K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)827client # [ 0.277734] ... bit width: 48828lb # [ 0.285496] smp: Bringing up secondary CPUs ...829worker2 # [ 0.277404] Performance Events: Fam17h+ core perfctr, AMD PMU driver.830client # [ 0.277987] ... generic counters: 6831worker1 # [ 0.291072] devtmpfs: initialized832lb # [ 0.285978] smp: Brought up 1 node, 1 CPU833worker2 # [ 0.277855] ... version: 2834worker2 # [ 0.278556] ... bit width: 48835client # [ 0.278686] ... generic bitmap: 000000000000003f836worker1 # [ 0.291912] x86/mm: Memory block size: 128MB837lb # [ 0.286649] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)838client # [ 0.278987] ... fixed-purpose counters: 0839worker2 # [ 0.278849] ... generic counters: 6840client # [ 0.279703] ... fixed-purpose bitmap: 0000000000000000841worker2 # [ 0.279576] ... generic bitmap: 000000000000003f842worker1 # [ 0.293676] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)843worker2 # [ 0.279849] ... fixed-purpose counters: 0844client # [ 0.279987] ... value mask: 0000ffffffffffff845lb # [ 0.287123] Memory: 1454576K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)846worker2 # [ 0.280588] ... fixed-purpose bitmap: 0000000000000000847lb # [ 0.288224] devtmpfs: initialized848client # [ 0.280829] ... max period: 00007fffffffffff849worker1 # [ 0.294693] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).850worker2 # [ 0.280849] ... value mask: 0000ffffffffffff851client # [ 0.281629] ... global_ctrl mask: 000000000000003f852lb # [ 0.289047] x86/mm: Memory block size: 128MB853worker1 # [ 0.295882] pinctrl core: initialized pinctrl subsystem854worker2 # [ 0.281712] ... max period: 00007fffffffffff855client # [ 0.282093] signal: max sigframe size: 3376856worker1 # [ 0.296922] PM: RTC time: 10:50:50, date: 2026-09-20857worker2 # [ 0.282488] ... global_ctrl mask: 000000000000003f858client # [ 0.282920] rcu: Hierarchical SRCU implementation.859lb # [ 0.290818] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)860worker2 # [ 0.282966] signal: max sigframe size: 3376861client # [ 0.283569] rcu: Max phase no-delay instances is 400.862worker2 # [ 0.283788] rcu: Hierarchical SRCU implementation.863lb # [ 0.291816] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).864worker2 # [ 0.284469] rcu: Max phase no-delay instances is 400.865lb # [ 0.293055] pinctrl core: initialized pinctrl subsystem866worker1 # [ 0.300369] NET: Registered PF_NETLINK/PF_ROUTE protocol family867lb # [ 0.294184] PM: RTC time: 10:50:50, date: 2026-09-20868worker1 # [ 0.301291] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations869client # [ 0.288507] smp: Bringing up secondary CPUs ...870worker1 # [ 0.301822] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations871client # [ 0.289002] smp: Brought up 1 node, 1 CPU872lb # [ 0.297832] NET: Registered PF_NETLINK/PF_ROUTE protocol family873client # [ 0.289698] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)874worker2 # [ 0.289327] smp: Bringing up secondary CPUs ...875worker1 # [ 0.303065] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations876worker2 # [ 0.289864] smp: Brought up 1 node, 1 CPU877worker1 # [ 0.303811] audit: initializing netlink subsys (disabled)878lb # [ 0.299154] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations879worker2 # [ 0.290563] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)880client # [ 0.290171] Memory: 1454552K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109348K reserved, 0K cma-reserved)881worker1 # [ 0.304803] audit: type=2000 audit(1789901450.844:1): state=initialized audit_enabled=0 res=1882lb # [ 0.299984] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations883client # [ 0.291236] devtmpfs: initialized884worker1 # [ 0.305944] thermal_sys: Registered thermal governor 'fair_share'885worker2 # [ 0.290986] Memory: 1454576K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)886client # [ 0.292049] x86/mm: Memory block size: 128MB887lb # [ 0.301237] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations888worker1 # [ 0.305946] thermal_sys: Registered thermal governor 'bang_bang'889worker2 # [ 0.292118] devtmpfs: initialized890lb # [ 0.301974] audit: initializing netlink subsys (disabled)891worker1 # [ 0.306803] thermal_sys: Registered thermal governor 'step_wise'892worker2 # [ 0.292900] x86/mm: Memory block size: 128MB893client # [ 0.293787] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)894worker1 # [ 0.307744] thermal_sys: Registered thermal governor 'user_space'895lb # [ 0.302966] audit: type=2000 audit(1789901450.850:1): state=initialized audit_enabled=0 res=1896worker1 # [ 0.308547] thermal_sys: Registered thermal governor 'power_allocator'897lb # [ 0.304124] thermal_sys: Registered thermal governor 'fair_share'898client # [ 0.294885] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).899worker2 # [ 0.294693] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)900worker1 # [ 0.308814] cpuidle: using governor menu901lb # [ 0.304127] thermal_sys: Registered thermal governor 'bang_bang'902client # [ 0.296079] pinctrl core: initialized pinctrl subsystem903worker2 # [ 0.295760] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).904lb # [ 0.304965] thermal_sys: Registered thermal governor 'step_wise'905client # [ 0.297161] PM: RTC time: 10:50:50, date: 2026-09-20906worker1 # [ 0.311027] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5907worker2 # [ 0.296943] pinctrl core: initialized pinctrl subsystem908lb # [ 0.305861] thermal_sys: Registered thermal governor 'user_space'909lb # [ 0.306674] thermal_sys: Registered thermal governor 'power_allocator'910worker1 # [ 0.312094] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]911worker2 # [ 0.298101] PM: RTC time: 10:50:50, date: 2026-09-20912lb # [ 0.306978] cpuidle: using governor menu913worker1 # [ 0.312804] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry914client # [ 0.300589] NET: Registered PF_NETLINK/PF_ROUTE protocol family915worker1 # [ 0.313813] PCI: Using configuration type 1 for base access916lb # [ 0.309188] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5917client # [ 0.301463] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations918worker1 # [ 0.314908] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.919worker2 # [ 0.301516] NET: Registered PF_NETLINK/PF_ROUTE protocol family920lb # [ 0.310191] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]921client # [ 0.302008] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations922lb # [ 0.310967] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry923worker2 # [ 0.302342] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations924lb # [ 0.311977] PCI: Using configuration type 1 for base access925client # [ 0.303250] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations926worker2 # [ 0.302870] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations927client # [ 0.303998] audit: initializing netlink subsys (disabled)928lb # [ 0.313096] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.929worker2 # [ 0.304128] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations930client # [ 0.304989] audit: type=2000 audit(1789901450.863:1): state=initialized audit_enabled=0 res=1931worker2 # [ 0.304859] audit: initializing netlink subsys (disabled)932client # [ 0.306083] thermal_sys: Registered thermal governor 'fair_share'933worker2 # [ 0.306013] thermal_sys: Registered thermal governor 'fair_share'934worker1 # [ 0.321818] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages935client # [ 0.306085] thermal_sys: Registered thermal governor 'bang_bang'936worker2 # [ 0.306016] thermal_sys: Registered thermal governor 'bang_bang'937client # [ 0.306988] thermal_sys: Registered thermal governor 'step_wise'938worker1 # [ 0.322802] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page939worker2 # [ 0.306850] thermal_sys: Registered thermal governor 'step_wise'940client # [ 0.307962] thermal_sys: Registered thermal governor 'user_space'941worker2 # [ 0.307795] thermal_sys: Registered thermal governor 'user_space'942client # [ 0.308713] thermal_sys: Registered thermal governor 'power_allocator'943client # [ 0.309001] cpuidle: using governor menu944lb # [ 0.319983] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages945worker2 # [ 0.308603] audit: type=2000 audit(1789901450.864:1): state=initialized audit_enabled=0 res=1946lb # [ 0.320965] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page947worker2 # [ 0.309853] thermal_sys: Registered thermal governor 'power_allocator'948client # [ 0.311208] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5949worker2 # [ 0.309877] cpuidle: using governor menu950worker1 # [ 0.327803] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages951client # [ 0.312222] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]952worker1 # [ 0.328802] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page953worker2 # [ 0.312108] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5954client # [ 0.312990] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry955worker2 # [ 0.313122] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]956client # [ 0.313999] PCI: Using configuration type 1 for base access957lb # [ 0.325966] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages958worker2 # [ 0.313852] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry959client # [ 0.315123] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.960worker2 # [ 0.314862] PCI: Using configuration type 1 for base access961lb # [ 0.326966] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page962worker2 # [ 0.315960] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.963client # [ 0.320002] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages964client # [ 0.320989] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page965worker2 # [ 0.321116] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages966worker1 # [ 0.340094] ACPI: Added _OSI(Module Device)967worker1 # [ 0.340803] ACPI: Added _OSI(Processor Device)968worker2 # [ 0.321851] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page969worker1 # [ 0.341542] ACPI: Added _OSI(Processor Aggregator Device)970lb # [ 0.338201] ACPI: Added _OSI(Module Device)971client # [ 0.325989] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages972lb # [ 0.338965] ACPI: Added _OSI(Processor Device)973client # [ 0.326987] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page974lb # [ 0.339750] ACPI: Added _OSI(Processor Aggregator Device)975worker2 # [ 0.326850] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages976worker2 # [ 0.327849] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page977worker1 # [ 0.347483] ACPI: 1 ACPI AML tables successfully acquired and loaded978client # [ 0.332981] ACPI: Added _OSI(Module Device)979lb # [ 0.345541] ACPI: 1 ACPI AML tables successfully acquired and loaded980worker1 # [ 0.351219] ACPI: Interpreter enabled981worker1 # [ 0.351816] ACPI: PM: (supports S0 S3 S4 S5)982worker1 # [ 0.352566] ACPI: Using IOAPIC for interrupt routing983client # [ 0.337989] ACPI: Added _OSI(Processor Device)984lb # [ 0.349366] ACPI: Interpreter enabled985client # [ 0.338800] ACPI: Added _OSI(Processor Aggregator Device)986lb # [ 0.349980] ACPI: PM: (supports S0 S3 S4 S5)987worker2 # [ 0.338232] ACPI: Added _OSI(Module Device)988worker1 # [ 0.354852] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug989lb # [ 0.350682] ACPI: Using IOAPIC for interrupt routing990worker2 # [ 0.338851] ACPI: Added _OSI(Processor Device)991worker1 # [ 0.355802] PCI: Using E820 reservations for host bridge windows992worker2 # [ 0.339555] ACPI: Added _OSI(Processor Aggregator Device)993lb # [ 0.353046] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug994worker1 # [ 0.358906] ACPI: Enabled 2 GPEs in block 00 to 3F995lb # [ 0.353966] PCI: Using E820 reservations for host bridge windows996client # [ 0.345956] ACPI: 1 ACPI AML tables successfully acquired and loaded997lb # [ 0.357082] ACPI: Enabled 2 GPEs in block 00 to 3F998worker2 # [ 0.348121] ACPI: 1 ACPI AML tables successfully acquired and loaded999worker1 # [ 0.366305] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1000client # [ 0.352436] ACPI: Interpreter enabled1001worker1 # [ 0.366808] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1002client # [ 0.353012] ACPI: PM: (supports S0 S3 S4 S5)1003worker1 # [ 0.367880] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1004client # [ 0.353733] ACPI: Using IOAPIC for interrupt routing1005worker2 # [ 0.352277] ACPI: Interpreter enabled1006lb # [ 0.364469] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1007worker1 # [ 0.368925] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1008worker2 # [ 0.352866] ACPI: PM: (supports S0 S3 S4 S5)1009client # [ 0.354051] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1010worker2 # [ 0.353671] ACPI: Using IOAPIC for interrupt routing1011worker1 # [ 0.370175] PCI host bridge to bus 0000:001012lb # [ 0.364972] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1013client # [ 0.354989] PCI: Using E820 reservations for host bridge windows1014worker1 # [ 0.370806] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1015lb # [ 0.366045] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1016worker1 # [ 0.371802] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1017lb # [ 0.367088] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1018client # [ 0.358104] ACPI: Enabled 2 GPEs in block 00 to 3F1019worker1 # [ 0.372802] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1020worker2 # [ 0.356889] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1021lb # [ 0.368339] PCI host bridge to bus 0000:001022worker2 # [ 0.357850] PCI: Using E820 reservations for host bridge windows1023worker1 # [ 0.373802] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1024lb # [ 0.368969] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1025worker1 # [ 0.374801] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1026lb # [ 0.369965] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1027worker1 # [ 0.375801] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1028lb # [ 0.370965] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1029worker2 # [ 0.360911] ACPI: Enabled 2 GPEs in block 00 to 3F1030worker1 # [ 0.376801] pci_bus 0000:00: root bus resource [bus 00-ff]1031lb # [ 0.371965] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1032worker1 # [ 0.377781] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1033lb # [ 0.372965] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1034lb # [ 0.373965] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1035worker1 # [ 0.379235] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1036lb # [ 0.374965] pci_bus 0000:00: root bus resource [bus 00-ff]1037client # [ 0.366742] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1038lb # [ 0.375934] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1039client # [ 0.366994] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1040worker2 # [ 0.367987] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1041lb # [ 0.377400] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1042client # [ 0.368898] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1043worker2 # [ 0.368855] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1044client # [ 0.370087] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1045worker2 # [ 0.369927] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1046client # [ 0.371335] PCI host bridge to bus 0000:001047worker2 # [ 0.370971] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1048worker1 # [ 0.381937] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1049client # [ 0.371993] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1050worker1 # [ 0.382814] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1051worker2 # [ 0.372212] PCI host bridge to bus 0000:001052client # [ 0.372988] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1053worker1 # [ 0.383824] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1054worker2 # [ 0.372853] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1055client # [ 0.373988] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1056worker2 # [ 0.373849] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1057lb # [ 0.380027] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1058worker1 # [ 0.385043] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1059client # [ 0.374988] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1060worker2 # [ 0.374849] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1061lb # [ 0.380978] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1062client # [ 0.375988] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1063worker2 # [ 0.375849] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1064lb # [ 0.381966] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1065worker1 # [ 0.386542] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1066client # [ 0.376987] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1067worker2 # [ 0.376849] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1068client # [ 0.377989] pci_bus 0000:00: root bus resource [bus 00-ff]1069worker2 # [ 0.377849] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1070lb # [ 0.383245] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1071worker2 # [ 0.378849] pci_bus 0000:00: root bus resource [bus 00-ff]1072client # [ 0.378981] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1073worker1 # [ 0.389838] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1074lb # [ 0.384669] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1075worker1 # [ 0.390678] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1076worker2 # [ 0.379871] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1077client # [ 0.380412] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1078worker1 # [ 0.391537] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1079worker2 # [ 0.381297] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1080worker1 # [ 0.391808] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1081lb # [ 0.388000] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1082lb # [ 0.388889] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1083worker1 # [ 0.393554] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1084lb # [ 0.389715] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1085lb # [ 0.389971] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1086client # [ 0.383033] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1087worker1 # [ 0.396789] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1088client # [ 0.384119] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1089lb # [ 0.391668] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1090worker1 # [ 0.397474] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1091worker2 # [ 0.383901] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1092client # [ 0.385052] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1093worker1 # [ 0.397823] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1094worker2 # [ 0.384986] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1095worker2 # [ 0.385871] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1096lb # [ 0.394891] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1097client # [ 0.386306] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1098worker1 # [ 0.399375] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1099lb # [ 0.395594] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1100worker2 # [ 0.387206] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1101lb # [ 0.395987] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1102client # [ 0.387685] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1103worker2 # [ 0.388554] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1104lb # [ 0.397519] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1105worker1 # [ 0.401837] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1106client # [ 0.390019] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1107worker1 # [ 0.403443] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1108client # [ 0.390941] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1109worker2 # [ 0.390857] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1110worker1 # [ 0.403824] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1111client # [ 0.391800] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1112worker2 # [ 0.391694] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1113client # [ 0.392813] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1114lb # [ 0.399999] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1115worker1 # [ 0.405381] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1116worker2 # [ 0.392605] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1117lb # [ 0.401749] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1118worker2 # [ 0.392855] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1119lb # [ 0.402754] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1120client # [ 0.394305] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1121worker1 # [ 0.407605] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1122worker2 # [ 0.394392] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1123worker1 # [ 0.408486] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1124lb # [ 0.404587] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1125client # [ 0.396024] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1126worker1 # [ 0.408823] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1127client # [ 0.396849] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1128worker2 # [ 0.395857] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1129client # [ 0.397770] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1130worker2 # [ 0.396727] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1131lb # [ 0.407006] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1132worker1 # [ 0.410360] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1133worker2 # [ 0.397631] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1134lb # [ 0.407917] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1135client # [ 0.399377] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1136worker2 # [ 0.399233] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1137lb # [ 0.408793] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1138worker1 # [ 0.413736] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1139worker1 # [ 0.414498] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1140client # [ 0.402021] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1141lb # [ 0.410523] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1142worker1 # [ 0.414824] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1143client # [ 0.402865] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1144worker2 # [ 0.401859] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1145worker1 # [ 0.416362] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1146client # [ 0.403699] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1147worker2 # [ 0.402856] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1148lb # [ 0.412999] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1149worker2 # [ 0.403803] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1150client # [ 0.404550] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1151lb # [ 0.413973] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1152lb # [ 0.414993] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1153worker2 # [ 0.405231] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1154worker1 # [ 0.418764] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1155client # [ 0.406020] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1156worker1 # [ 0.419477] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1157client # [ 0.406858] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1158worker2 # [ 0.406857] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1159lb # [ 0.416622] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1160worker1 # [ 0.419824] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1161client # [ 0.407712] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1162worker2 # [ 0.407673] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1163worker2 # [ 0.408598] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1164worker1 # [ 0.421398] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1165client # [ 0.408537] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1166lb # [ 0.419992] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1167worker2 # [ 0.409412] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1168lb # [ 0.420898] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1169worker1 # [ 0.423567] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1170lb # [ 0.421762] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1171worker1 # [ 0.423809] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1172client # [ 0.411661] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1173worker1 # [ 0.424751] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1174client # [ 0.411995] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1175worker2 # [ 0.411655] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1176lb # [ 0.423528] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1177worker2 # [ 0.412460] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1178client # [ 0.412919] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1179worker1 # [ 0.425618] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1180worker2 # [ 0.412872] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1181client # [ 0.414385] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1182worker1 # [ 0.427392] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1183worker2 # [ 0.414428] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1184lb # [ 0.425997] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1185client # [ 0.415996] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1186worker1 # [ 0.428814] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1187lb # [ 0.427678] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1188client # [ 0.416846] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1189worker2 # [ 0.416833] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1190worker1 # [ 0.429783] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1191lb # [ 0.427993] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1192client # [ 0.417693] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1193lb # [ 0.428974] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1194worker2 # [ 0.417463] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1195worker1 # [ 0.431157] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1196worker2 # [ 0.417873] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1197client # [ 0.419472] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1198lb # [ 0.430511] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1199worker1 # [ 0.433551] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1200worker2 # [ 0.419463] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1201client # [ 0.421007] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1202lb # [ 0.431981] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1203client # [ 0.421831] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1204worker1 # [ 0.434474] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1205worker2 # [ 0.420857] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1206client # [ 0.422697] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1207worker1 # [ 0.435603] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1208lb # [ 0.432994] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1209worker2 # [ 0.421711] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1210client # [ 0.422994] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1211worker2 # [ 0.422580] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1212worker1 # [ 0.437222] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1213lb # [ 0.434641] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1214worker2 # [ 0.422855] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1215client # [ 0.424524] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1216worker1 # [ 0.439525] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1217worker2 # [ 0.424363] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1218lb # [ 0.436799] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1219client # [ 0.426000] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1220worker1 # [ 0.439809] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1221lb # [ 0.437680] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1222client # [ 0.427011] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1223worker1 # [ 0.440775] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1224worker2 # [ 0.425872] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1225lb # [ 0.437993] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1226worker2 # [ 0.426827] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1227client # [ 0.428546] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1228worker1 # [ 0.442187] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1229lb # [ 0.439645] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1230worker2 # [ 0.428213] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1231client # [ 0.430007] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1232worker1 # [ 0.443628] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1233client # [ 0.430858] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1234worker2 # [ 0.429883] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1235worker1 # [ 0.444648] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1236client # [ 0.431736] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1237worker2 # [ 0.430728] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1238lb # [ 0.442995] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1239worker1 # [ 0.446492] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1240worker2 # [ 0.431602] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1241lb # [ 0.443943] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1242client # [ 0.432564] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1243lb # [ 0.444788] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1244worker2 # [ 0.432830] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1245lb # [ 0.446466] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1246client # [ 0.434995] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1247client # [ 0.435846] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1248worker2 # [ 0.434858] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1249lb # [ 0.447876] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1250client # [ 0.436721] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1251worker2 # [ 0.435722] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1252worker2 # [ 0.436606] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1253lb # [ 0.448956] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1254client # [ 0.437565] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1255worker2 # [ 0.437418] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1256client # [ 0.438602] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1257lb # [ 0.451003] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1258worker2 # [ 0.438531] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1259client # [ 0.439182] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1260client # [ 0.440605] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1261worker2 # [ 0.439049] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1262worker1 # [ 0.446999] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1263worker2 # [ 0.440534] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1264worker1 # [ 0.449558] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1265worker1 # [ 0.450029] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1266worker1 # [ 0.451483] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1267worker1 # [ 0.452070] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1268worker1 # [ 0.453128] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1269lb # [ 0.452213] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1270worker1 # [ 0.454049] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1271lb # [ 0.454781] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1272worker1 # [ 0.455833] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1273client # [ 0.441802] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1274lb # [ 0.456001] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1275worker2 # [ 0.441045] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1276worker1 # [ 0.456695] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1277lb # [ 0.457653] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1278worker2 # [ 0.443497] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1279client # [ 0.444600] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1280worker1 # [ 0.458142] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1281worker2 # [ 0.444106] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1282client # [ 0.445243] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1283lb # [ 0.458629] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1284worker1 # [ 0.459604] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1285worker2 # [ 0.445457] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1286lb # [ 0.459294] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1287client # [ 0.446969] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1288worker2 # [ 0.446139] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1289client # [ 0.447977] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1290lb # [ 0.460299] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1291worker1 # [ 0.463353] ACPI: PCI: Interrupt link LNKA configured for IRQ 101292client # [ 0.449253] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1293worker2 # [ 0.447172] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1294worker1 # [ 0.463909] ACPI: PCI: Interrupt link LNKB configured for IRQ 101295lb # [ 0.462025] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1296worker1 # [ 0.464900] ACPI: PCI: Interrupt link LNKC configured for IRQ 111297client # [ 0.450229] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1298worker2 # [ 0.448146] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1299lb # [ 0.462966] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1300worker1 # [ 0.465901] ACPI: PCI: Interrupt link LNKD configured for IRQ 111301worker1 # [ 0.466903] ACPI: PCI: Interrupt link LNKE configured for IRQ 101302worker2 # [ 0.449899] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1303lb # [ 0.464309] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1304client # [ 0.452655] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1305worker2 # [ 0.450732] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1306worker1 # [ 0.467898] ACPI: PCI: Interrupt link LNKF configured for IRQ 101307client # [ 0.452994] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1308worker1 # [ 0.468900] ACPI: PCI: Interrupt link LNKG configured for IRQ 111309lb # [ 0.465918] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1310worker2 # [ 0.451884] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1311worker1 # [ 0.469902] ACPI: PCI: Interrupt link LNKH configured for IRQ 111312client # [ 0.454244] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1313worker1 # [ 0.470838] ACPI: PCI: Interrupt link GSIA configured for IRQ 161314worker2 # [ 0.453542] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1315client # [ 0.456039] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1316worker1 # [ 0.471816] ACPI: PCI: Interrupt link GSIB configured for IRQ 171317worker1 # [ 0.472786] ACPI: PCI: Interrupt link GSIC configured for IRQ 181318lb # [ 0.469932] ACPI: PCI: Interrupt link LNKA configured for IRQ 101319worker1 # [ 0.473594] ACPI: PCI: Interrupt link GSID configured for IRQ 191320lb # [ 0.470851] ACPI: PCI: Interrupt link LNKB configured for IRQ 101321worker1 # [ 0.474568] ACPI: PCI: Interrupt link GSIE configured for IRQ 201322worker2 # [ 0.456814] ACPI: PCI: Interrupt link LNKA configured for IRQ 101323lb # [ 0.471857] ACPI: PCI: Interrupt link LNKC configured for IRQ 111324client # [ 0.459776] ACPI: PCI: Interrupt link LNKA configured for IRQ 101325worker1 # [ 0.474814] ACPI: PCI: Interrupt link GSIF configured for IRQ 211326worker2 # [ 0.457713] ACPI: PCI: Interrupt link LNKB configured for IRQ 101327lb # [ 0.472921] ACPI: PCI: Interrupt link LNKD configured for IRQ 111328worker1 # [ 0.475813] ACPI: PCI: Interrupt link GSIG configured for IRQ 221329client # [ 0.460825] ACPI: PCI: Interrupt link LNKB configured for IRQ 101330worker2 # [ 0.458702] ACPI: PCI: Interrupt link LNKC configured for IRQ 111331worker1 # [ 0.476759] ACPI: PCI: Interrupt link GSIH configured for IRQ 231332client # [ 0.461794] ACPI: PCI: Interrupt link LNKC configured for IRQ 111333lb # [ 0.473902] ACPI: PCI: Interrupt link LNKE configured for IRQ 101334worker2 # [ 0.459689] ACPI: PCI: Interrupt link LNKD configured for IRQ 111335client # [ 0.462796] ACPI: PCI: Interrupt link LNKD configured for IRQ 111336lb # [ 0.474936] ACPI: PCI: Interrupt link LNKF configured for IRQ 101337worker2 # [ 0.460710] ACPI: PCI: Interrupt link LNKE configured for IRQ 101338worker1 # [ 0.478573] iommu: Default domain type: Translated1339client # [ 0.463794] ACPI: PCI: Interrupt link LNKE configured for IRQ 101340worker2 # [ 0.461646] ACPI: PCI: Interrupt link LNKF configured for IRQ 101341worker1 # [ 0.478809] iommu: DMA domain TLB invalidation policy: lazy mode1342lb # [ 0.475953] ACPI: PCI: Interrupt link LNKG configured for IRQ 111343client # [ 0.464792] ACPI: PCI: Interrupt link LNKF configured for IRQ 101344worker1 # [ 0.479977] ACPI: bus type USB registered1345worker2 # [ 0.462641] ACPI: PCI: Interrupt link LNKG configured for IRQ 111346lb # [ 0.476907] ACPI: PCI: Interrupt link LNKH configured for IRQ 111347client # [ 0.465795] ACPI: PCI: Interrupt link LNKG configured for IRQ 111348worker2 # [ 0.463732] ACPI: PCI: Interrupt link LNKH configured for IRQ 111349worker1 # [ 0.480844] usbcore: registered new interface driver usbfs1350lb # [ 0.477825] ACPI: PCI: Interrupt link GSIA configured for IRQ 161351client # [ 0.466803] ACPI: PCI: Interrupt link LNKH configured for IRQ 111352worker1 # [ 0.481710] usbcore: registered new interface driver hub1353worker2 # [ 0.464682] ACPI: PCI: Interrupt link GSIA configured for IRQ 161354lb # [ 0.478754] ACPI: PCI: Interrupt link GSIB configured for IRQ 171355client # [ 0.467741] ACPI: PCI: Interrupt link GSIA configured for IRQ 161356worker1 # [ 0.482431] usbcore: registered new device driver usb1357worker2 # [ 0.465643] ACPI: PCI: Interrupt link GSIB configured for IRQ 171358client # [ 0.468001] ACPI: PCI: Interrupt link GSIB configured for IRQ 171359lb # [ 0.479805] ACPI: PCI: Interrupt link GSIC configured for IRQ 181360worker2 # [ 0.466614] ACPI: PCI: Interrupt link GSIC configured for IRQ 181361worker1 # [ 0.483782] NetLabel: Initializing1362client # [ 0.468938] ACPI: PCI: Interrupt link GSIC configured for IRQ 181363lb # [ 0.480821] ACPI: PCI: Interrupt link GSID configured for IRQ 191364worker2 # [ 0.466865] ACPI: PCI: Interrupt link GSID configured for IRQ 191365worker1 # [ 0.484281] NetLabel: domain hash size = 1281366client # [ 0.469720] ACPI: PCI: Interrupt link GSID configured for IRQ 191367lb # [ 0.481846] ACPI: PCI: Interrupt link GSIE configured for IRQ 201368worker2 # [ 0.467835] ACPI: PCI: Interrupt link GSIE configured for IRQ 201369worker1 # [ 0.484801] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1370client # [ 0.470771] ACPI: PCI: Interrupt link GSIE configured for IRQ 201371lb # [ 0.482933] ACPI: PCI: Interrupt link GSIF configured for IRQ 211372worker2 # [ 0.468608] ACPI: PCI: Interrupt link GSIF configured for IRQ 211373worker1 # [ 0.485792] NetLabel: unlabeled traffic allowed by default1374client # [ 0.471745] ACPI: PCI: Interrupt link GSIF configured for IRQ 211375worker1 # [ 0.486493] PCI: Using ACPI for IRQ routing1376worker2 # [ 0.468863] ACPI: PCI: Interrupt link GSIG configured for IRQ 221377lb # [ 0.483825] ACPI: PCI: Interrupt link GSIG configured for IRQ 221378client # [ 0.472000] ACPI: PCI: Interrupt link GSIG configured for IRQ 221379worker2 # [ 0.469802] ACPI: PCI: Interrupt link GSIH configured for IRQ 231380lb # [ 0.484813] ACPI: PCI: Interrupt link GSIH configured for IRQ 231381client # [ 0.472942] ACPI: PCI: Interrupt link GSIH configured for IRQ 231382worker2 # [ 0.471675] iommu: Default domain type: Translated1383lb # [ 0.486933] iommu: Default domain type: Translated1384client # [ 0.474758] iommu: Default domain type: Translated1385worker2 # [ 0.472468] iommu: DMA domain TLB invalidation policy: lazy mode1386client # [ 0.474996] iommu: DMA domain TLB invalidation policy: lazy mode1387lb # [ 0.487652] iommu: DMA domain TLB invalidation policy: lazy mode1388worker2 # [ 0.473119] ACPI: bus type USB registered1389client # [ 0.476183] ACPI: bus type USB registered1390lb # [ 0.488270] ACPI: bus type USB registered1391worker2 # [ 0.473876] usbcore: registered new interface driver usbfs1392client # [ 0.476901] usbcore: registered new interface driver usbfs1393lb # [ 0.489050] usbcore: registered new interface driver usbfs1394worker2 # [ 0.474784] usbcore: registered new interface driver hub1395client # [ 0.477633] usbcore: registered new interface driver hub1396worker2 # [ 0.475517] usbcore: registered new device driver usb1397client # [ 0.477997] usbcore: registered new device driver usb1398lb # [ 0.489986] usbcore: registered new interface driver hub1399lb # [ 0.490983] usbcore: registered new device driver usb1400worker2 # [ 0.476768] NetLabel: Initializing1401client # [ 0.479775] NetLabel: Initializing1402worker2 # [ 0.477305] NetLabel: domain hash size = 1281403client # [ 0.479989] NetLabel: domain hash size = 1281404lb # [ 0.493051] NetLabel: Initializing1405worker2 # [ 0.477849] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1406client # [ 0.480688] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1407lb # [ 0.493781] NetLabel: domain hash size = 1281408worker2 # [ 0.478808] NetLabel: unlabeled traffic allowed by default1409client # [ 0.481027] NetLabel: unlabeled traffic allowed by default1410worker2 # [ 0.479529] PCI: Using ACPI for IRQ routing1411client # [ 0.481918] PCI: Using ACPI for IRQ routing1412lb # [ 0.494587] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1413lb # [ 0.495024] NetLabel: unlabeled traffic allowed by default1414lb # [ 0.495982] PCI: Using ACPI for IRQ routing1415worker1 # [ 0.530100] pci 0000:00:01.0: vgaarb: setting as boot VGA device1416worker1 # [ 0.530798] pci 0000:00:01.0: vgaarb: bridge control possible1417worker1 # [ 0.530798] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1418worker1 # [ 0.530806] vgaarb: loaded1419worker1 # [ 0.531561] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01420worker1 # [ 0.531801] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1421worker1 # [ 0.534898] clocksource: Switched to clocksource kvm-clock1422client # [ 0.526182] pci 0000:00:01.0: vgaarb: setting as boot VGA device1423worker1 # [ 0.538418] VFS: Disk quotas dquot_6.6.01424worker2 # [ 0.524311] pci 0000:00:01.0: vgaarb: setting as boot VGA device1425client # [ 0.526984] pci 0000:00:01.0: vgaarb: bridge control possible1426worker1 # [ 0.539124] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1427worker2 # [ 0.524845] pci 0000:00:01.0: vgaarb: bridge control possible1428worker1 # [ 0.540429] pnp: PnP ACPI init1429client # [ 0.526984] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1430client # [ 0.526992] vgaarb: loaded1431worker2 # [ 0.524845] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1432lb # [ 0.540050] pci 0000:00:01.0: vgaarb: setting as boot VGA device1433worker1 # [ 0.541227] ACPI: IRQ 4 override to edge(!), high(!)1434worker2 # [ 0.524854] vgaarb: loaded1435client # [ 0.527690] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01436lb # [ 0.540961] pci 0000:00:01.0: vgaarb: bridge control possible1437worker1 # [ 0.542184] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1438worker2 # [ 0.525595] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01439client # [ 0.527990] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1440worker2 # [ 0.525851] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1441worker1 # [ 0.543692] pnp: PnP ACPI: found 6 devices1442lb # [ 0.540961] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1443lb # [ 0.540970] vgaarb: loaded1444lb # [ 0.541721] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01445lb # [ 0.541965] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1446worker2 # [ 0.528936] clocksource: Switched to clocksource kvm-clock1447client # [ 0.533078] clocksource: Switched to clocksource kvm-clock1448lb # [ 0.546058] clocksource: Switched to clocksource kvm-clock1449client # [ 0.534762] VFS: Disk quotas dquot_6.6.01450worker2 # [ 0.532462] VFS: Disk quotas dquot_6.6.01451worker1 # [ 0.551324] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1452worker2 # [ 0.533171] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1453worker1 # [ 0.552802] clocksource: Switched to clocksource acpi_pm1454worker2 # [ 0.534558] pnp: PnP ACPI init1455client # [ 0.537287] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1456worker1 # [ 0.553780] NET: Registered PF_INET protocol family1457lb # [ 0.549766] VFS: Disk quotas dquot_6.6.01458worker2 # [ 0.535321] ACPI: IRQ 4 override to edge(!), high(!)1459client # [ 0.538573] pnp: PnP ACPI init1460lb # [ 0.550578] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1461worker2 # [ 0.536278] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1462worker1 # [ 0.554943] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1463client # [ 0.539346] ACPI: IRQ 4 override to edge(!), high(!)1464lb # [ 0.552102] pnp: PnP ACPI init1465worker2 # [ 0.537766] pnp: PnP ACPI: found 6 devices1466client # [ 0.540273] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1467lb # [ 0.553027] ACPI: IRQ 4 override to edge(!), high(!)1468client # [ 0.541731] pnp: PnP ACPI: found 6 devices1469lb # [ 0.554164] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1470lb # [ 0.555766] pnp: PnP ACPI: found 6 devices1471worker2 # [ 0.545339] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1472worker2 # [ 0.546877] clocksource: Switched to clocksource acpi_pm1473client # [ 0.549530] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1474worker2 # [ 0.547865] NET: Registered PF_INET protocol family1475client # [ 0.550939] clocksource: Switched to clocksource acpi_pm1476client # [ 0.551904] NET: Registered PF_INET protocol family1477worker2 # [ 0.549072] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1478lb # [ 0.563495] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1479client # [ 0.553090] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1480lb # [ 0.565107] clocksource: Switched to clocksource acpi_pm1481lb # [ 0.566211] NET: Registered PF_INET protocol family1482worker1 # [ 0.571241] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1483lb # [ 0.567420] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1484worker1 # [ 0.572863] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1485worker1 # [ 0.574223] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1486worker1 # [ 0.575629] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1487worker1 # [ 0.576967] TCP: Hash tables configured (established 16384 bind 16384)1488worker1 # [ 0.578144] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1489worker1 # [ 0.579461] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1490worker1 # [ 0.580603] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1491worker1 # [ 0.581913] NET: Registered PF_UNIX/PF_LOCAL protocol family1492worker1 # [ 0.582933] NET: Registered PF_XDP protocol family1493worker1 # [ 0.583828] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1494worker2 # [ 0.565577] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1495worker1 # [ 0.584874] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1496client # [ 0.569145] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1497worker2 # [ 0.567002] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1498worker1 # [ 0.585887] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1499client # [ 0.570639] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1500worker1 # [ 0.586982] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1501worker2 # [ 0.568332] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1502client # [ 0.571922] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1503worker1 # [ 0.588079] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1504worker2 # [ 0.569699] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1505lb # [ 0.584235] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1506worker1 # [ 0.589161] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1507client # [ 0.573218] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1508worker2 # [ 0.570933] TCP: Hash tables configured (established 16384 bind 16384)1509client # [ 0.574399] TCP: Hash tables configured (established 16384 bind 16384)1510lb # [ 0.585871] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1511worker1 # [ 0.590834] ACPI: \_SB_.GSIA: Enabled at IRQ 161512worker2 # [ 0.572039] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1513client # [ 0.575555] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1514lb # [ 0.587402] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1515worker2 # [ 0.573328] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1516client # [ 0.576887] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1517worker1 # [ 0.593457] ACPI: \_SB_.GSIB: Enabled at IRQ 171518lb # [ 0.588938] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1519worker2 # [ 0.574998] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1520client # [ 0.578079] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1521lb # [ 0.590345] TCP: Hash tables configured (established 16384 bind 16384)1522worker2 # [ 0.576326] NET: Registered PF_UNIX/PF_LOCAL protocol family1523client # [ 0.579242] NET: Registered PF_UNIX/PF_LOCAL protocol family1524worker1 # [ 0.595429] ACPI: \_SB_.GSIC: Enabled at IRQ 181525worker2 # [ 0.577302] NET: Registered PF_XDP protocol family1526client # [ 0.580161] NET: Registered PF_XDP protocol family1527lb # [ 0.591652] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1528worker2 # [ 0.578124] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1529client # [ 0.580979] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1530worker1 # [ 0.597334] ACPI: \_SB_.GSID: Enabled at IRQ 191531lb # [ 0.593155] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1532worker2 # [ 0.579128] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1533client # [ 0.581951] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1534worker1 # [ 0.598974] PCI: CLS 0 bytes, default 641535worker2 # [ 0.580127] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1536lb # [ 0.594385] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1537client # [ 0.582925] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1538worker2 # [ 0.581188] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1539client # [ 0.583970] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1540lb # [ 0.595741] NET: Registered PF_UNIX/PF_LOCAL protocol family1541worker1 # [ 0.599801] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1542worker2 # [ 0.582251] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1543client # [ 0.585019] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1544lb # [ 0.596839] NET: Registered PF_XDP protocol family1545worker1 # [ 0.601434] Trying to unpack rootfs image as initramfs...1546worker2 # [ 0.583315] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1547client # [ 0.586099] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1548lb # [ 0.597818] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1549lb # [ 0.598954] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1550client # [ 0.587718] ACPI: \_SB_.GSIA: Enabled at IRQ 161551worker2 # [ 0.585069] ACPI: \_SB_.GSIA: Enabled at IRQ 161552lb # [ 0.600091] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1553lb # [ 0.601256] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1554client # [ 0.589976] ACPI: \_SB_.GSIB: Enabled at IRQ 171555worker2 # [ 0.587945] ACPI: \_SB_.GSIB: Enabled at IRQ 171556lb # [ 0.602456] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1557client # [ 0.591964] ACPI: \_SB_.GSIC: Enabled at IRQ 181558lb # [ 0.603623] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1559worker2 # [ 0.589881] ACPI: \_SB_.GSIC: Enabled at IRQ 181560lb # [ 0.605421] ACPI: \_SB_.GSIA: Enabled at IRQ 161561client # [ 0.594175] ACPI: \_SB_.GSID: Enabled at IRQ 191562worker2 # [ 0.591792] ACPI: \_SB_.GSID: Enabled at IRQ 191563client # [ 0.595908] PCI: CLS 0 bytes, default 641564worker2 # [ 0.593554] PCI: CLS 0 bytes, default 641565lb # [ 0.608241] ACPI: \_SB_.GSIB: Enabled at IRQ 171566client # [ 0.596934] Trying to unpack rootfs image as initramfs...1567worker2 # [ 0.594547] Trying to unpack rootfs image as initramfs...1568lb # [ 0.610309] ACPI: \_SB_.GSIC: Enabled at IRQ 181569lb # [ 0.612237] ACPI: \_SB_.GSID: Enabled at IRQ 191570lb # [ 0.613865] PCI: CLS 0 bytes, default 641571worker2 # [ 0.598903] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1572client # [ 0.602964] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1573lb # [ 0.614771] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1574lb # [ 0.616743] Trying to unpack rootfs image as initramfs...1575worker1 # [ 0.647946] Initialise system trusted keyrings1576worker1 # [ 0.648913] workingset: timestamp_bits=40 max_order=19 bucket_order=01577client # [ 0.643575] Initialise system trusted keyrings1578worker2 # [ 0.640890] Initialise system trusted keyrings1579worker2 # [ 0.643462] workingset: timestamp_bits=40 max_order=19 bucket_order=01580client # [ 0.646886] workingset: timestamp_bits=40 max_order=19 bucket_order=01581lb # [ 0.662157] Initialise system trusted keyrings1582lb # [ 0.665119] workingset: timestamp_bits=40 max_order=19 bucket_order=01583worker1 # [ 0.671598] Key type asymmetric registered1584worker1 # [ 0.672376] Asymmetric key parser 'x509' registered1585worker1 # [ 0.673301] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1586worker1 # [ 0.678681] io scheduler mq-deadline registered1587worker1 # [ 0.679419] io scheduler kyber registered1588worker1 # [ 0.683107] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1589client # [ 0.668127] Key type asymmetric registered1590worker2 # [ 0.665505] Key type asymmetric registered1591worker2 # [ 0.666195] Asymmetric key parser 'x509' registered1592worker1 # [ 0.684440] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1593worker2 # [ 0.667083] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1594client # [ 0.670861] Asymmetric key parser 'x509' registered1595client # [ 0.671701] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1596worker1 # [ 0.688317] Linux agpgart interface v0.1031597worker1 # [ 0.689152] ACPI: bus type drm_connector registered1598client # [ 0.674935] io scheduler mq-deadline registered1599lb # [ 0.686667] Key type asymmetric registered1600worker2 # [ 0.672549] io scheduler mq-deadline registered1601client # [ 0.675683] io scheduler kyber registered1602lb # [ 0.687515] Asymmetric key parser 'x509' registered1603worker2 # [ 0.673280] io scheduler kyber registered1604worker1 # [ 0.691985] usbcore: registered new interface driver usbserial_generic1605worker1 # [ 0.693052] usbserial: USB Serial support registered for generic1606lb # [ 0.690154] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1607worker2 # [ 0.676031] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1608client # [ 0.679408] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1609worker2 # [ 0.677360] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1610client # [ 0.680646] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1611worker1 # [ 0.696560] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1612worker1 # [ 0.696560] Please enable it if your BIOS has the CPPC option.1613lb # [ 0.694151] io scheduler mq-deadline registered1614lb # [ 0.694924] io scheduler kyber registered1615worker1 # [ 0.698913] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1616client # [ 0.685906] Linux agpgart interface v0.1031617worker2 # [ 0.683262] Linux agpgart interface v0.1031618lb # [ 0.697621] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1619client # [ 0.686647] ACPI: bus type drm_connector registered1620worker2 # [ 0.684044] ACPI: bus type drm_connector registered1621worker1 # [ 0.702708] drop_monitor: Initializing network drop monitor service1622worker1 # [ 0.703892] NET: Registered PF_INET6 protocol family1623client # [ 0.688972] usbcore: registered new interface driver usbserial_generic1624worker2 # [ 0.686911] usbcore: registered new interface driver usbserial_generic1625lb # [ 0.701088] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1626client # [ 0.690021] usbserial: USB Serial support registered for generic1627worker2 # [ 0.687934] usbserial: USB Serial support registered for generic1628worker1 # [ 0.707006] Segment Routing with IPv61629lb # [ 0.704755] Linux agpgart interface v0.1031630client # [ 0.692842] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1631worker1 # [ 0.709557] In-situ OAM (IOAM) with IPv61632lb # [ 0.705640] ACPI: bus type drm_connector registered1633client # [ 0.692842] Please enable it if your BIOS has the CPPC option.1634worker1 # [ 0.710585] IPI shorthand broadcast: enabled1635worker2 # [ 0.691429] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1636client # [ 0.695017] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1637worker2 # [ 0.691429] Please enable it if your BIOS has the CPPC option.1638worker2 # [ 0.693633] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1639lb # [ 0.710229] usbcore: registered new interface driver usbserial_generic1640client # [ 0.699010] drop_monitor: Initializing network drop monitor service1641lb # [ 0.711420] usbserial: USB Serial support registered for generic1642client # [ 0.700171] NET: Registered PF_INET6 protocol family1643worker2 # [ 0.697606] drop_monitor: Initializing network drop monitor service1644worker2 # [ 0.698808] NET: Registered PF_INET6 protocol family1645worker1 # [ 0.718835] sched_clock: Marking stable (584014755, 134522768)->(794490537, -75953014)1646lb # [ 0.715086] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1647client # [ 0.704930] Segment Routing with IPv61648lb # [ 0.715086] Please enable it if your BIOS has the CPPC option.1649client # [ 0.705586] In-situ OAM (IOAM) with IPv61650worker2 # [ 0.702926] Segment Routing with IPv61651lb # [ 0.717416] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1652worker2 # [ 0.703631] In-situ OAM (IOAM) with IPv61653client # [ 0.706602] IPI shorthand broadcast: enabled1654worker1 # [ 0.723629] registered taskstats version 11655worker1 # [ 0.724615] Loading compiled-in X.509 certificates1656worker2 # [ 0.706699] IPI shorthand broadcast: enabled1657lb # [ 0.721219] drop_monitor: Initializing network drop monitor service1658lb # [ 0.722441] NET: Registered PF_INET6 protocol family1659client # [ 0.712760] sched_clock: Marking stable (578013179, 133802771)->(785120254, -73304304)1660lb # [ 0.725602] Segment Routing with IPv61661lb # [ 0.728099] In-situ OAM (IOAM) with IPv61662lb # [ 0.729182] IPI shorthand broadcast: enabled1663worker2 # [ 0.715040] sched_clock: Marking stable (580017860, 134382298)->(789619621, -75219463)1664client # [ 0.719003] registered taskstats version 11665client # [ 0.720017] Loading compiled-in X.509 certificates1666worker2 # [ 0.718497] registered taskstats version 11667worker2 # [ 0.719507] Loading compiled-in X.509 certificates1668lb # [ 0.737472] sched_clock: Marking stable (604013482, 133050864)->(810613127, -73548781)1669worker1 # [ 0.743558] Demotion targets for Node 0: null1670worker1 # [ 0.744449] Key type .fscrypt registered1671worker1 # [ 0.745168] Key type fscrypt-provisioning registered1672lb # [ 0.742340] registered taskstats version 11673lb # [ 0.743296] Loading compiled-in X.509 certificates1674worker1 # [ 0.748663] ima: No TPM chip found, activating TPM-bypass!1675worker1 # [ 0.749603] ima: Allocated hash algorithm: sha11676worker1 # [ 0.750436] ima: No architecture policies found1677client # [ 0.737832] Demotion targets for Node 0: null1678worker1 # [ 0.754560] PM: Magic number: 10:369:8311679client # [ 0.738735] Key type .fscrypt registered1680client # [ 0.739407] Key type fscrypt-provisioning registered1681worker1 # [ 0.756171] RAS: Correctable Errors collector initialized.1682client # [ 0.740337] ima: No TPM chip found, activating TPM-bypass!1683worker2 # [ 0.738420] Demotion targets for Node 0: null1684worker2 # [ 0.739357] Key type .fscrypt registered1685client # [ 0.742835] ima: Allocated hash algorithm: sha11686client # [ 0.743637] ima: No architecture policies found1687worker2 # [ 0.741412] Key type fscrypt-provisioning registered1688worker2 # [ 0.742429] ima: No TPM chip found, activating TPM-bypass!1689worker2 # [ 0.743372] ima: Allocated hash algorithm: sha11690client # [ 0.747838] PM: Magic number: 10:369:8311691worker1 # [ 0.765257] clk: Disabling unused clocks1692client # [ 0.749430] RAS: Correctable Errors collector initialized.1693worker2 # [ 0.747453] ima: No architecture policies found1694worker2 # [ 0.748475] PM: Magic number: 10:369:8311695lb # [ 0.763246] Demotion targets for Node 0: null1696worker1 # [ 0.767553] PM: genpd: Disabling unused power domains1697lb # [ 0.764125] Key type .fscrypt registered1698worker2 # [ 0.750195] RAS: Correctable Errors collector initialized.1699lb # [ 0.764792] Key type fscrypt-provisioning registered1700lb # [ 0.768185] ima: No TPM chip found, activating TPM-bypass!1701lb # [ 0.769107] ima: Allocated hash algorithm: sha11702lb # [ 0.769904] ima: No architecture policies found1703client # [ 0.758524] clk: Disabling unused clocks1704lb # [ 0.772258] PM: Magic number: 10:369:8311705client # [ 0.760840] PM: genpd: Disabling unused power domains1706worker2 # [ 0.759137] clk: Disabling unused clocks1707lb # [ 0.773980] RAS: Correctable Errors collector initialized.1708worker2 # [ 0.762416] PM: genpd: Disabling unused power domains1709lb # [ 0.784767] clk: Disabling unused clocks1710lb # [ 0.787079] PM: genpd: Disabling unused power domains1711client # [ 0.918613] Freeing initrd memory: 29096K1712client # [ 0.921744] Freeing unused decrypted memory: 2028K1713worker1 # [ 0.938485] Freeing initrd memory: 29092K1714client # [ 0.924289] Freeing unused kernel image (initmem) memory: 3644K1715client # [ 0.925319] Write protecting the kernel read-only data: 32768k1716worker1 # [ 0.941831] Freeing unused decrypted memory: 2028K1717client # [ 0.927134] Freeing unused kernel image (text/rodata gap) memory: 1208K1718client # [ 0.928598] Freeing unused kernel image (rodata/data gap) memory: 752K1719worker1 # [ 0.944686] Freeing unused kernel image (initmem) memory: 3644K1720worker1 # [ 0.945829] Write protecting the kernel read-only data: 32768k1721worker1 # [ 0.948070] Freeing unused kernel image (text/rodata gap) memory: 1208K1722worker2 # [ 0.929978] Freeing initrd memory: 29092K1723lb # [ 0.945460] Freeing initrd memory: 29092K1724worker1 # [ 0.949706] Freeing unused kernel image (rodata/data gap) memory: 752K1725worker2 # [ 0.933308] Freeing unused decrypted memory: 2028K1726lb # [ 0.948545] Freeing unused decrypted memory: 2028K1727worker2 # [ 0.936124] Freeing unused kernel image (initmem) memory: 3644K1728lb # [ 0.951122] Freeing unused kernel image (initmem) memory: 3644K1729worker2 # [ 0.937244] Write protecting the kernel read-only data: 32768k1730lb # [ 0.952172] Write protecting the kernel read-only data: 32768k1731worker2 # [ 0.939207] Freeing unused kernel image (text/rodata gap) memory: 1208K1732lb # [ 0.953978] Freeing unused kernel image (text/rodata gap) memory: 1208K1733worker2 # [ 0.940780] Freeing unused kernel image (rodata/data gap) memory: 752K1734lb # [ 0.955482] Freeing unused kernel image (rodata/data gap) memory: 752K1735client # [ 0.979518] x86/mm: Checked W+X mappings: passed, no W+X pages found.1736client # [ 0.980575] Run /init as init process1737worker1 # [ 1.001098] x86/mm: Checked W+X mappings: passed, no W+X pages found.1738worker1 # [ 1.002239] Run /init as init process1739client # [ 0.990644] systemd[1]: Inserted module 'autofs4'1740worker2 # [ 0.991968] x86/mm: Checked W+X mappings: passed, no W+X pages found.1741lb # [ 1.006863] x86/mm: Checked W+X mappings: passed, no W+X pages found.1742worker2 # [ 0.993020] Run /init as init process1743lb # [ 1.008043] Run /init as init process1744worker1 # [ 1.013026] systemd[1]: Inserted module 'autofs4'1745lb # [ 1.018117] systemd[1]: Inserted module 'autofs4'1746worker2 # [ 1.003827] systemd[1]: Inserted module 'autofs4'1747client # [ 1.006901] fuse: init (API version 7.45)1748client # [ 1.012673] ACPI: \_SB_.GSIG: Enabled at IRQ 221749worker1 # [ 1.030029] fuse: init (API version 7.45)1750client # [ 1.015021] ACPI: \_SB_.GSIH: Enabled at IRQ 231751client # [ 1.018036] ACPI: \_SB_.GSIE: Enabled at IRQ 201752worker1 # [ 1.036020] ACPI: \_SB_.GSIG: Enabled at IRQ 221753client # [ 1.020245] ACPI: \_SB_.GSIF: Enabled at IRQ 211754worker1 # [ 1.038479] ACPI: \_SB_.GSIH: Enabled at IRQ 231755lb # [ 1.034806] fuse: init (API version 7.45)1756worker2 # [ 1.020899] fuse: init (API version 7.45)1757worker1 # [ 1.041742] ACPI: \_SB_.GSIE: Enabled at IRQ 201758worker1 # [ 1.043930] ACPI: \_SB_.GSIF: Enabled at IRQ 211759lb # [ 1.040536] ACPI: \_SB_.GSIG: Enabled at IRQ 221760worker2 # [ 1.026947] ACPI: \_SB_.GSIG: Enabled at IRQ 221761lb # [ 1.043212] ACPI: \_SB_.GSIH: Enabled at IRQ 231762worker2 # [ 1.029665] ACPI: \_SB_.GSIH: Enabled at IRQ 231763lb # [ 1.046638] ACPI: \_SB_.GSIE: Enabled at IRQ 201764worker2 # [ 1.033288] ACPI: \_SB_.GSIE: Enabled at IRQ 201765lb # [ 1.049134] ACPI: \_SB_.GSIF: Enabled at IRQ 211766worker2 # [ 1.035783] ACPI: \_SB_.GSIF: Enabled at IRQ 211767client # [ 1.047913] systemd[1]: Successfully made /usr/ read-only.1768worker1 # [ 1.072484] systemd[1]: Successfully made /usr/ read-only.1769lb # [ 1.078411] systemd[1]: Successfully made /usr/ read-only.1770worker2 # [ 1.064414] systemd[1]: Successfully made /usr/ read-only.1771client # [ 1.384610] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1772client # [ 1.396790] systemd[1]: Detected virtualization kvm.1773client # [ 1.398997] systemd[1]: Detected architecture x86-64.1774client # [ 1.401115] systemd[1]: Running in initrd.1775worker1 # [ 1.409508] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1776client # [ 1.403720] systemd[1]: Initializing machine ID from random generator.1777worker1 # [ 1.421594] systemd[1]: Detected virtualization kvm.1778client # [ 1.406640] systemd[1]: Hostname set to <client>.1779worker1 # [ 1.423728] systemd[1]: Detected architecture x86-64.1780worker1 # [ 1.425843] systemd[1]: Running in initrd.1781worker1 # [ 1.428294] systemd[1]: Initializing machine ID from random generator.1782lb # [ 1.415228] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1783worker2 # [ 1.400975] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1784worker1 # [ 1.431177] systemd[1]: Hostname set to <worker1>.1785lb # [ 1.427268] systemd[1]: Detected virtualization kvm.1786worker2 # [ 1.412936] systemd[1]: Detected virtualization kvm.1787worker2 # [ 1.415075] systemd[1]: Detected architecture x86-64.1788lb # [ 1.429546] systemd[1]: Detected architecture x86-64.1789worker2 # [ 1.417197] systemd[1]: Running in initrd.1790lb # [ 1.431743] systemd[1]: Running in initrd.1791worker2 # [ 1.419734] systemd[1]: Initializing machine ID from random generator.1792lb # [ 1.434267] systemd[1]: Initializing machine ID from random generator.1793lb # [ 1.437186] systemd[1]: Hostname set to <lb>.1794worker2 # [ 1.422612] systemd[1]: Hostname set to <worker2>.1795worker1 # [ 1.534194] systemd[1]: bpf-restrict-fs: LSM BPF program attached1796worker1 # [ 1.572752] systemd[1]: Queued start job for default target Initrd Default Target.1797worker1 # [ 1.576835] systemd[1]: Created slice Slice /system/modprobe.1798worker1 # [ 1.578022] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1799worker1 # [ 1.579433] systemd[1]: Expecting device /dev/disk/by-label/nixos...1800worker1 # [ 1.580620] systemd[1]: Reached target Path Units.1801worker1 # [ 1.581507] systemd[1]: Reached target Slice Units.1802worker1 # [ 1.582412] systemd[1]: Reached target Swaps.1803worker1 # [ 1.583235] systemd[1]: Reached target Timer Units.1804worker1 # [ 1.584237] systemd[1]: Listening on D-Bus System Message Bus Socket.1805worker1 # [ 1.585470] systemd[1]: Listening on Journal Socket (/dev/log).1806worker1 # [ 1.586649] systemd[1]: Listening on Journal Sockets.1807worker1 # [ 1.587642] systemd[1]: Listening on udev Control Socket.1808worker1 # [ 1.588642] systemd[1]: Listening on udev Kernel Socket.1809worker1 # [ 1.589589] systemd[1]: Reached target Socket Units.1810worker1 # [ 1.591358] systemd[1]: Starting Create List of Static Device Nodes...1811worker1 # [ 1.596614] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1812worker1 # [ 1.604715] systemd[1]: Starting Load Kernel Module configfs...1813worker1 # [ 1.617617] systemd[1]: Starting Journal Service...1814worker1 # [ 1.629640] systemd[1]: Starting Load Kernel Modules...1815worker1 # [ 1.636668] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1816client # [ 1.633175] systemd[1]: bpf-restrict-fs: LSM BPF program attached1817worker1 # [ 1.652407] netfs: FS-Cache loaded1818worker2 # [ 1.635834] systemd[1]: bpf-restrict-fs: LSM BPF program attached1819worker1 # [ 1.657161] 9pnet: Installing 9P2000 support1820lb # [ 1.655350] systemd[1]: bpf-restrict-fs: LSM BPF program attached1821worker1 # [ 1.669665] systemd[1]: Starting Coldplug All udev Devices...1822worker1 # [ 1.680639] systemd[1]: Finished Create List of Static Device Nodes.1823worker1 # [ 1.687184] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1824worker2 # [ 1.675072] systemd[1]: Queued start job for default target Initrd Default Target.1825worker1 # [ 1.694936] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1826worker2 # [ 1.678715] systemd[1]: Created slice Slice /system/modprobe.1827worker2 # [ 1.679854] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1828worker2 # [ 1.681195] systemd[1]: Expecting device /dev/disk/by-label/nixos...1829worker1 # [ 1.699971] systemd[1]: modprobe@configfs.service: Deactivated successfully.1830worker2 # [ 1.682242] systemd[1]: Reached target Path Units.1831worker2 # [ 1.683075] systemd[1]: Reached target Slice Units.1832worker1 # [ 1.701467] systemd-journald[67]: Collecting audit messages is disabled.1833worker2 # [ 1.683971] systemd[1]: Reached target Swaps.1834worker2 # [ 1.684751] systemd[1]: Reached target Timer Units.1835worker2 # [ 1.685696] systemd[1]: Listening on D-Bus System Message Bus Socket.1836client # [ 1.688871] systemd[1]: Queued start job for default target Initrd Default Target.1837worker2 # [ 1.686866] systemd[1]: Listening on Journal Socket (/dev/log).1838worker2 # [ 1.687991] systemd[1]: Listening on Journal Sockets.1839worker1 # [ 1.706931] systemd[1]: Finished Load Kernel Module configfs.1840worker2 # [ 1.688967] systemd[1]: Listening on udev Control Socket.1841client # [ 1.692229] systemd[1]: Created slice Slice /system/modprobe.1842worker2 # [ 1.689953] systemd[1]: Listening on udev Kernel Socket.1843worker2 # [ 1.690886] systemd[1]: Reached target Socket Units.1844client # [ 1.693476] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1845client # [ 1.694921] systemd[1]: Expecting device /dev/disk/by-label/nixos...1846worker2 # [ 1.692665] systemd[1]: Starting Create List of Static Device Nodes...1847client # [ 1.696062] systemd[1]: Reached target Path Units.1848client # [ 1.696961] systemd[1]: Reached target Slice Units.1849client # [ 1.697873] systemd[1]: Reached target Swaps.1850worker1 # [ 1.713005] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1851client # [ 1.698675] systemd[1]: Reached target Timer Units.1852worker2 # [ 1.696225] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1853client # [ 1.699714] systemd[1]: Listening on D-Bus System Message Bus Socket.1854lb # [ 1.711195] systemd[1]: Queued start job for default target Initrd Default Target.1855client # [ 1.700970] systemd[1]: Listening on Journal Socket (/dev/log).1856worker1 # [ 1.716781] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1857client # [ 1.702123] systemd[1]: Listening on Journal Sockets.1858client # [ 1.703142] systemd[1]: Listening on udev Control Socket.1859lb # [ 1.715405] systemd[1]: Created slice Slice /system/modprobe.1860client # [ 1.704250] systemd[1]: Listening on udev Kernel Socket.1861client # [ 1.705257] systemd[1]: Reached target Socket Units.1862lb # [ 1.716571] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1863lb # [ 1.718123] systemd[1]: Expecting device /dev/disk/by-label/nixos...1864client # [ 1.707140] systemd[1]: Starting Create List of Static Device Nodes...1865lb # [ 1.719240] systemd[1]: Reached target Path Units.1866lb # [ 1.720111] systemd[1]: Reached target Slice Units.1867lb # [ 1.720983] systemd[1]: Reached target Swaps.1868worker1 # [ 1.724615] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1869lb # [ 1.721804] systemd[1]: Reached target Timer Units.1870worker2 # [ 1.707485] systemd[1]: Starting Load Kernel Module configfs...1871worker1 # [ 1.726010] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1872client # [ 1.710902] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1873lb # [ 1.722848] systemd[1]: Listening on D-Bus System Message Bus Socket.1874lb # [ 1.724178] systemd[1]: Listening on Journal Socket (/dev/log).1875lb # [ 1.725340] systemd[1]: Listening on Journal Sockets.1876lb # [ 1.726345] systemd[1]: Listening on udev Control Socket.1877lb # [ 1.727386] systemd[1]: Listening on udev Kernel Socket.1878lb # [ 1.729659] systemd[1]: Reached target Socket Units.1879client # [ 1.719901] systemd[1]: Starting Load Kernel Module configfs...1880worker2 # [ 1.717470] systemd[1]: Starting Journal Service...1881lb # [ 1.732130] systemd[1]: Starting Create List of Static Device Nodes...1882lb # [ 1.735899] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1883worker2 # [ 1.726580] systemd[1]: Starting Load Kernel Modules...1884lb # [ 1.742751] systemd[1]: Starting Load Kernel Module configfs...1885client # [ 1.736970] systemd[1]: Starting Journal Service...1886worker1 # [ 1.754139] systemd[1]: Finished Load Kernel Modules.1887worker1 # [ 1.761745] systemd[1]: Starting Apply Kernel Variables...1888lb # [ 1.758686] systemd[1]: Starting Journal Service...1889client # [ 1.748915] systemd[1]: Starting Load Kernel Modules...1890worker2 # [ 1.735547] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1891worker2 # [ 1.753237] netfs: FS-Cache loaded1892worker1 # [ 1.773632] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1893lb # [ 1.771135] systemd[1]: Starting Load Kernel Modules...1894worker2 # [ 1.758461] 9pnet: Installing 9P2000 support1895worker1 # [ 1.781984] systemd[1]: Starting Create Static Device Nodes in /dev...1896client # [ 1.755012] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1897client # [ 1.772251] netfs: FS-Cache loaded1898worker2 # [ 1.772162] systemd[1]: Starting Coldplug All udev Devices...1899client # [ 1.777902] 9pnet: Installing 9P2000 support1900lb # [ 1.778136] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1901lb # [ 1.793259] netfs: FS-Cache loaded1902worker1 # [ 1.800860] systemd[1]: Finished Apply Kernel Variables.1903lb # [ 1.797843] 9pnet: Installing 9P2000 support1904worker2 # [ 1.783997] systemd-journald[67]: Collecting audit messages is disabled.1905worker2 # [ 1.786664] systemd[1]: Finished Create List of Static Device Nodes.1906client # [ 1.791070] systemd[1]: Starting Coldplug All udev Devices...1907worker2 # [ 1.790066] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1908lb # [ 1.806825] systemd[1]: Starting Coldplug All udev Devices...1909worker2 # [ 1.798474] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1910worker1 # [ 1.817093] systemd[1]: Finished Create Static Device Nodes in /dev.1911worker1 # [ 1.684785] systemd-modules-load[68]: Inserted module 'dm_mod'1912worker1 # [ 1.686097] systemd-modules-load[68]: Inserted module 'virtio_balloon'1913client # [ 1.806193] systemd-journald[67]: Collecting audit messages is disabled.1914worker1 # [ 1.688286] systemd-modules-load[68]: Inserted module 'virtio_gpu'1915client # [ 1.807463] systemd[1]: Finished Create List of Static Device Nodes.1916worker1 # [ 1.823791] systemd[1]: Started Journal Service.1917worker2 # [ 1.805822] systemd[1]: modprobe@configfs.service: Deactivated successfully.1918lb # [ 1.820583] systemd[1]: Finished Create List of Static Device Nodes.1919client # [ 1.812454] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1920worker2 # [ 1.810507] systemd[1]: Finished Load Kernel Module configfs.1921lb # [ 1.825632] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1922worker1 # [ 1.699133] systemd[1]: Reached target Preparation for Local File Systems.1923worker2 # [ 1.814212] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1924worker1 # [ 1.701279] systemd[1]: Reached target Local File Systems.1925client # [ 1.821240] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1926worker2 # [ 1.817820] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1927worker1 # [ 1.703151] systemd[1]: Starting Create System Files and Directories...1928lb # [ 1.834455] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1929worker1 # [ 1.707791] systemd[1]: Starting Rule-based Manager for Device Events and Files...1930client # [ 1.827270] systemd[1]: modprobe@configfs.service: Deactivated successfully.1931lb # [ 1.840476] systemd[1]: modprobe@configfs.service: Deactivated successfully.1932worker2 # [ 1.826542] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1933lb # [ 1.842384] systemd-journald[67]: Collecting audit messages is disabled.1934worker2 # [ 1.828194] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1935client # [ 1.834197] systemd[1]: Finished Load Kernel Module configfs.1936lb # [ 1.847149] systemd[1]: Finished Load Kernel Module configfs.1937client # [ 1.838278] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1938client # [ 1.840553] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1939lb # [ 1.852512] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1940lb # [ 1.855802] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1941client # [ 1.850001] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1942lb # [ 1.862863] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1943worker1 # [ 1.733349] systemd[1]: Finished Create System Files and Directories.1944client # [ 1.851902] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1945lb # [ 1.864319] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1946worker2 # [ 1.855897] systemd[1]: Finished Load Kernel Modules.1947worker2 # [ 1.862580] systemd[1]: Starting Apply Kernel Variables...1948worker1 # [ 1.756864] systemd-udevd[83]: Using default interface naming scheme 'v261'.1949worker2 # [ 1.873512] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1950client # [ 1.879957] systemd[1]: Finished Load Kernel Modules.1951lb # [ 1.895161] systemd[1]: Finished Load Kernel Modules.1952worker2 # [ 1.884605] systemd[1]: Starting Create Static Device Nodes in /dev...1953lb # [ 1.901838] systemd[1]: Starting Apply Kernel Variables...1954client # [ 1.890105] systemd[1]: Starting Apply Kernel Variables...1955worker1 # [ 1.781725] systemd[1]: Started Rule-based Manager for Device Events and Files.1956lb # [ 1.913381] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1957worker2 # [ 1.900501] systemd[1]: Finished Apply Kernel Variables.1958client # [ 1.903953] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1959worker2 # [ 1.770443] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.906034] systemd[1]: Started Journal Service.1960worker2 # 1961lb # [ 1.923644] systemd[1]: Starting Create Static Device Nodes in /dev...1962worker2 # [ 1.775225] systemd-modules-load[68]: Inserted module 'virtio_balloon'1963worker2 # [ 1.776263] systemd-modules-load[68]: Inserted module 'virtio_gpu'1964client # [ 1.915020] systemd[1]: Starting Create Static Device Nodes in /dev...1965lb # [ 1.937173] systemd[1]: Finished Apply Kernel Variables.1966worker2 # [ 1.790143] systemd[1]: Finished Create Static Device Nodes in /dev.1967client # [ 1.927432] systemd[1]: Finished Apply Kernel Variables.1968worker2 # [ 1.792221] systemd[1]: Reached target Preparation for Local File Systems.1969worker2 # [ 1.793244] systemd[1]: Reached target Local File Systems.1970worker2 # [ 1.795077] systemd[1]: Starting Create System Files and Directories...1971client # [ 1.932535] systemd[1]: Started Journal Service.1972client # [ 1.796792] systemd-modules-load[68]: Inserted module 'dm_mod'1973client # [ 1.800366] systemd-modules-load[68]: Inserted module 'virtio_balloon'1974client # [ 1.801444] systemd-modules-load[68]: Inserted module 'virtio_gpu'1975worker2 # [ 1.800089] systemd[1]: Starting Rule-based Manager for Device Events and Files...1976lb # [ 1.954500] systemd[1]: Started Journal Service.1977worker1 # [ 1.826142] systemd[1]: Finished Coldplug All udev Devices.1978worker1 # [ 1.826950] systemd[1]: Reached target System Initialization.1979lb # [ 1.824207] systemd-modules-load[68]: Inserted module 'dm_mod'1980worker1 # [ 1.827786] systemd[1]: Reached target Basic System.1981lb # [ 1.825575] systemd-modules-load[68]: Inserted module 'virtio_balloon'1982lb # [ 1.828290] systemd-modules-load[68]: Inserted module 'virtio_gpu'1983lb # [ 1.829356] systemd[1]: Finished Create Static Device Nodes in /dev.1984lb # [ 1.830427] systemd[1]: Reached target Preparation for Local File Systems.1985lb # [ 1.832312] systemd[1]: Reached target Local File Systems.1986client # [ 1.821211] systemd[1]: Finished Create Static Device Nodes in /dev.1987lb # [ 1.836074] systemd[1]: Starting Create System Files and Directories...1988client # [ 1.824088] systemd[1]: Reached target Preparation for Local File Systems.1989client # [ 1.825099] systemd[1]: Reached target Local File Systems.1990client # [ 1.828077] systemd[1]: Starting Create System Files and Directories...1991worker2 # [ 1.825454] systemd[1]: Finished Create System Files and Directories.1992lb # [ 1.841778] systemd[1]: Starting Rule-based Manager for Device Events and Files...1993client # [ 1.833082] systemd[1]: Starting Rule-based Manager for Device Events and Files...1994worker2 # [ 1.850315] systemd-udevd[83]: Using default interface naming scheme 'v261'.1995lb # [ 1.867239] systemd[1]: Finished Create System Files and Directories.1996client # [ 1.858228] systemd[1]: Finished Create System Files and Directories.1997worker2 # [ 1.873513] systemd[1]: Started Rule-based Manager for Device Events and Files.1998lb # [ 1.893843] systemd-udevd[77]: Using default interface naming scheme 'v261'.1999client # [ 1.882382] systemd-udevd[83]: Using default interface naming scheme 'v261'.2000lb # [ 1.919105] systemd[1]: Started Rule-based Manager for Device Events and Files.

loading 1,821 hidden lines…

3822worker2 # [ 10.581745] systemd[1]: Finished resolvconf update.3823client # [ 10.585051] systemd[1]: Finished Address configuration of eth1.3824worker2 # [ 10.583321] systemd[1]: Reached target Preparation for Network.3825worker2 # [ 10.588125] systemd[1]: Starting DHCP Client...3826worker2 # [ 10.592169] systemd[1]: Starting Address configuration of eth1...3827worker2 # [ 10.595339] systemd[1]: Starting Extra networking commands....3828worker2 # [ 10.693260] network-addresses-eth1-start[629]: adding address 192.168.1.4/24... done3829worker2 # [ 10.708621] network-addresses-eth1-start[629]: adding address 2001:db8:1::4/64... done3830worker2 # [ 10.728296] systemd[1]: Finished Address configuration of eth1.3831worker2 # [ 10.759489] dhcpcd[637]: dhcpcd-10.3.2 starting3832worker2 # [ 10.768791] systemd[1]: Finished Extra networking commands..3833worker2 # [ 10.771620] dhcpcd[692]: dev: loaded udev3834worker2 # [ 10.773475] systemd[1]: Reached target Network.3835worker2 # [ 10.776252] systemd[1]: Starting Permit User Sessions...3836worker2 # [ 10.934047] 8021q: 802.1Q VLAN Support v1.83837worker2 # [ 10.934697] 8021q: adding VLAN 0 to HW filter on device eth13838worker2 # [ 10.808448] systemd[1]: Finished Permit User Sessions.3839worker2 # [ 10.812413] systemd[1]: Started Getty on tty1.3840worker2 # [ 10.814290] systemd[1]: Reached target Login Prompts.3841worker2 # [ 10.835307] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.3842worker2 # [ 11.024933] cfg80211: Loading compiled-in X.509 certificates for regulatory database3843worker2 # [ 11.053736] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'3844worker2 # [ 11.054599] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'3845worker2 # [ 11.057623] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -23846worker2 # [ 11.058507] cfg80211: failed to load regulatory.db3847worker2 # [ 11.112413] 8021q: adding VLAN 0 to HW filter on device eth03848worker2 # [ 10.979331] dhcpcd[692]: eth0: waiting for carrier3849worker2 # [ 10.980258] dhcpcd[692]: eth0: carrier acquired3850worker2 # [ 10.989974] dhcpcd[692]: DUID 00:01:00:01:32:42:77:15:52:54:00:12:34:563851worker2 # [ 10.990899] dhcpcd[692]: eth0: IAID 00:12:34:563852worker2 # [ 10.991603] dhcpcd[692]: eth0: adding address fe80::5054:ff:fe12:34563853client: (finished: waiting for unit network-addresses-eth1.service, in 11.86 seconds)3854client: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; }3855client: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; }, in 0.03 seconds)3856lb: waiting for unit network-addresses-eth1.service3857lb: waiting for the VM to finish booting3858lb: Guest shell says: b'Spawning backdoor root shell...\n'3859lb: connected to guest root shell3860lb: (connecting took 0.00 seconds)3861lb: (finished: waiting for the VM to finish booting, in 0.00 seconds)3862lb: (finished: waiting for unit network-addresses-eth1.service, in 0.05 seconds)3863lb: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; }3864lb: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; }, in 0.02 seconds)3865worker1: waiting for unit network-addresses-eth1.service3866worker1: waiting for the VM to finish booting3867worker1: Guest shell says: b'Spawning backdoor root shell...\n'3868worker1: connected to guest root shell3869worker1: (connecting took 0.00 seconds)3870worker1: (finished: waiting for the VM to finish booting, in 0.00 seconds)3871worker1: (finished: waiting for unit network-addresses-eth1.service, in 0.08 seconds)3872worker1: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; }3873worker1: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; }, in 0.03 seconds)3874worker2: waiting for unit network-addresses-eth1.service3875worker2: waiting for the VM to finish booting3876worker2: Guest shell says: b'Spawning backdoor root shell...\n'3877worker2: connected to guest root shell3878worker2: (connecting took 0.00 seconds)3879worker2: (finished: waiting for the VM to finish booting, in 0.00 seconds)3880worker2: (finished: waiting for unit network-addresses-eth1.service, in 0.10 seconds)3881worker2: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; }3882worker2: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; }, in 0.03 seconds)3883lb: waiting for unit niks3.service3884worker2 # [ 11.589093] dhcpcd[692]: eth0: soliciting a DHCP lease3885worker2 # [ 11.746170] NET: Registered PF_PACKET protocol family3886worker2 # [ 11.616920] dhcpcd[692]: eth0: offered 10.0.2.15 from 10.0.2.23887worker2 # [ 11.620326] dhcpcd[692]: eth0: probing address 10.0.2.15/243888client # [ 11.646231] systemd[1]: Finished Extra networking commands..3889client # [ 11.647828] systemd[1]: Reached target Network.3890client # [ 11.650922] systemd[1]: Starting Permit User Sessions...3891worker1 # [ 11.667286] dhcpcd[693]: eth0: soliciting a DHCP lease3892client # [ 11.673674] systemd[1]: Finished Permit User Sessions.3893client # [ 11.677340] systemd[1]: Started Getty on tty1.3894client # [ 11.678610] systemd[1]: Reached target Login Prompts.3895worker1 # [ 11.828250] NET: Registered PF_PACKET protocol family3896worker1 # [ 11.700426] dhcpcd[693]: eth0: offered 10.0.2.15 from 10.0.2.23897worker1 # [ 11.704361] dhcpcd[693]: eth0: probing address 10.0.2.15/243898lb # [ 11.866922] dhcpcd[710]: eth0: soliciting a DHCP lease3899lb # [ 12.015703] NET: Registered PF_PACKET protocol family3900lb # [ 11.889563] dhcpcd[710]: eth0: offered 10.0.2.15 from 10.0.2.23901lb # [ 11.892205] dhcpcd[710]: eth0: probing address 10.0.2.15/243902client # [ 11.921628] dhcpcd[563]: eth0: soliciting a DHCP lease3903client # [ 12.079430] NET: Registered PF_PACKET protocol family3904client # [ 11.951672] dhcpcd[563]: eth0: offered 10.0.2.15 from 10.0.2.23905client # [ 11.956384] dhcpcd[563]: eth0: probing address 10.0.2.15/243906lb # [ 12.093088] postgresql-pre-start[734]: selecting default time zone ... UTC3907lb # [ 12.098087] postgresql-pre-start[734]: creating configuration files ... ok3908worker2 # [ 12.261938] dhcpcd[692]: eth0: soliciting an IPv6 router3909worker2 # [ 12.263976] dhcpcd[692]: eth0: Router Advertisement from fe80::23910worker2 # [ 12.266128] dhcpcd[692]: eth0: adding address fec0::5054:ff:fe12:3456/643911worker2 # [ 12.268267] dhcpcd[692]: eth0: adding route to fec0::/643912worker2 # [ 12.270183] dhcpcd[692]: eth0: adding default route via fe80::23913lb # [ 12.333564] postgresql-pre-start[734]: running bootstrap script ... ok3914client # [ 12.430455] dhcpcd[563]: eth0: soliciting an IPv6 router3915client # [ 12.433150] dhcpcd[563]: eth0: Router Advertisement from fe80::23916client # [ 12.435353] dhcpcd[563]: eth0: adding address fec0::5054:ff:fe12:3456/643917client # [ 12.437592] dhcpcd[563]: eth0: adding route to fec0::/643918client # [ 12.439533] dhcpcd[563]: eth0: adding default route via fe80::23919lb # [ 12.471156] dhcpcd[710]: eth0: soliciting an IPv6 router3920lb # [ 12.472637] dhcpcd[710]: eth0: Router Advertisement from fe80::23921lb # [ 12.473904] dhcpcd[710]: eth0: adding address fec0::5054:ff:fe12:3456/643922lb # [ 12.474889] dhcpcd[710]: eth0: adding route to fec0::/643923lb # [ 12.476140] dhcpcd[710]: eth0: adding default route via fe80::23924lb # [ 12.929808] postgresql-pre-start[734]: performing post-bootstrap initialization ... ok3925worker1 # [ 12.977867] dhcpcd[693]: eth0: soliciting an IPv6 router3926worker1 # [ 12.980886] dhcpcd[693]: eth0: Router Advertisement from fe80::23927worker1 # [ 12.982945] dhcpcd[693]: eth0: adding address fec0::5054:ff:fe12:3456/643928worker1 # [ 12.985262] dhcpcd[693]: eth0: adding route to fec0::/643929worker1 # [ 12.987193] dhcpcd[693]: eth0: adding default route via fe80::23930lb # [ 13.056044] postgresql-pre-start[734]: syncing data to disk ... ok3931lb # [ 13.057748] postgresql-pre-start[734]: initdb: warning: enabling "trust" authentication for local connections3932lb # [ 13.058993] postgresql-pre-start[734]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.3933lb # [ 13.061030] postgresql-pre-start[734]: Success. You can now start the database server using:3934lb # [ 13.062075] postgresql-pre-start[734]: pg_ctl -D /var/lib/postgresql/18 -l logfile start3935lb # [ 13.167050] postgres[792]: [792] LOG: starting PostgreSQL 18.6 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit3936lb # [ 13.170991] postgres[792]: [792] LOG: listening on IPv6 address "::1", port 54323937lb # [ 13.172118] postgres[792]: [792] LOG: listening on IPv4 address "127.0.0.1", port 54323938lb # [ 13.174117] postgres[792]: [792] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"3939lb # [ 13.187188] postgres[801]: [801] LOG: database system was shut down at 2026-09-20 10:51:04 GMT3940lb # [ 13.190735] postgres[792]: [792] LOG: database system is ready to accept connections3941lb # [ 13.195906] systemd[1]: Started PostgreSQL Server.3942lb # [ 13.201948] systemd[1]: Starting PostgreSQL Setup Scripts...3943lb # [ 13.375840] postgresql-setup-start[812]: CREATE DATABASE3944lb # [ 13.416579] postgresql-setup-start[817]: CREATE ROLE3945lb # [ 13.435302] postgresql-setup-start[819]: ALTER DATABASE3946lb # [ 13.439701] systemd[1]: Finished PostgreSQL Setup Scripts.3947lb # [ 13.441524] systemd[1]: Reached target PostgreSQL.3948worker2 # [ 16.292841] dhcpcd[692]: eth0: leased 10.0.2.15 for 86400 seconds3949worker2 # [ 16.295904] dhcpcd[692]: eth0: adding route to 10.0.2.0/243950worker2 # [ 16.298499] dhcpcd[692]: eth0: adding default route via 10.0.2.23951worker2 # [ 16.423468] systemd[1]: Started DHCP Client.3952worker2 # [ 16.424870] systemd[1]: Reached target Multi-User System.3953worker2 # [ 16.426318] systemd[1]: Startup finished in 862ms (kernel) + 5.024s (initrd) + 10.539s (userspace) = 16.425s.3954worker1 # [ 17.140444] dhcpcd[693]: eth0: leased 10.0.2.15 for 86400 seconds3955worker1 # [ 17.143290] dhcpcd[693]: eth0: adding route to 10.0.2.0/243956worker1 # [ 17.145667] dhcpcd[693]: eth0: adding default route via 10.0.2.23957worker1 # [ 17.267851] systemd[1]: Started DHCP Client.3958worker1 # [ 17.269878] systemd[1]: Reached target Multi-User System.3959worker1 # [ 17.270791] systemd[1]: Startup finished in 871ms (kernel) + 4.049s (initrd) + 12.349s (userspace) = 17.269s.3960lb # [ 17.404386] dhcpcd[710]: eth0: leased 10.0.2.15 for 86400 seconds3961lb # [ 17.407163] dhcpcd[710]: eth0: adding route to 10.0.2.0/243962lb # [ 17.409522] dhcpcd[710]: eth0: adding default route via 10.0.2.23963client # [ 17.466967] dhcpcd[563]: eth0: leased 10.0.2.15 for 86400 seconds3964client # [ 17.467902] dhcpcd[563]: eth0: adding route to 10.0.2.0/243965client # [ 17.468623] dhcpcd[563]: eth0: adding default route via 10.0.2.23966lb # [ 17.535611] systemd[1]: Started DHCP Client.3967lb # [ 17.538116] systemd[1]: Reached target Network is Online.3968lb # [ 17.542396] systemd[1]: Started Envoy reverse proxy.3969lb # [ 17.545784] systemd[1]: Started mock-oidc.service.3970client # [ 17.577656] systemd[1]: Started DHCP Client.3971client # [ 17.579410] systemd[1]: Reached target Multi-User System.3972client # [ 17.580537] systemd[1]: Startup finished in 850ms (kernel) + 4.938s (initrd) + 11.791s (userspace) = 17.580s.3973lb # [ 17.640585] mock-oidc-server[920]: Mock OIDC Server running3974lb # [ 17.641596] mock-oidc-server[920]: OIDC Address: 192.168.1.2:80803975lb # [ 17.642783] mock-oidc-server[920]: Issue Address: 0.0.0.0:80813976lb # [ 17.644183] mock-oidc-server[920]: Issuer: http://192.168.1.2:8080/oidc3977lb # [ 17.645195] mock-oidc-server[920]: JWKS: http://192.168.1.2:8080/oidc/.well-known/jwks.json3978lb # [ 17.646555] mock-oidc-server[920]: Discovery: http://192.168.1.2:8080/oidc/.well-known/openid-configuration3979lb # [ 17.648112] mock-oidc-server[920]: Issue tokens: http://0.0.0.0:8081/issue?sub=...3980lb # [ 18.069355] envoy[919]: [2026-09-20 10:51:09.159][919][info][main] [source/server/server.cc:467] initializing epoch 0 (base id=0, hot restart version=11.104)3981lb # [ 18.071161] envoy[919]: [2026-09-20 10:51:09.159][919][info][main] [source/server/server.cc:469] statically linked extensions:3982lb # [ 18.072584] envoy[919]: [2026-09-20 10:51:09.159][919][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input3983lb # [ 18.074528] envoy[919]: [2026-09-20 10:51:09.160][919][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory3984lb # [ 18.077348] envoy[919]: [2026-09-20 10:51:09.160][919][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality3985lb # [ 18.093371] envoy[919]: [2026-09-20 10:51:09.160][919][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log3986lb # [ 18.099962] envoy[919]: [2026-09-20 10:51:09.160][919][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name3987lb # [ 18.103043] envoy[919]: [2026-09-20 10:51:09.160][919][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff3988lb # [ 18.105173] envoy[919]: [2026-09-20 10:51:09.160][919][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation3989lb # [ 18.107209] envoy[919]: [2026-09-20 10:51:09.160][919][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default3990lb # [ 18.109223] envoy[919]: [2026-09-20 10:51:09.160][919][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource3991lb # [ 18.112077] envoy[919]: [2026-09-20 10:51:09.160][919][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm3992lb # [ 18.118133] envoy[919]: [2026-09-20 10:51:09.160][919][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v83993lb # [ 18.119897] envoy[919]: [2026-09-20 10:51:09.160][919][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router3994lb # [ 18.122570] envoy[919]: [2026-09-20 10:51:09.160][919][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router3995lb # [ 18.124156] envoy[919]: [2026-09-20 10:51:09.160][919][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts3996lb # [ 18.127083] envoy[919]: [2026-09-20 10:51:09.160][919][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher3997lb # [ 18.129687] envoy[919]: [2026-09-20 10:51:09.160][919][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header3998lb # [ 18.133095] envoy[919]: [2026-09-20 10:51:09.160][919][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name3999lb # [ 18.135328] envoy[919]: [2026-09-20 10:51:09.160][919][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua4000lb # [ 18.136910] envoy[919]: [2026-09-20 10:51:09.160][919][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port4001lb # [ 18.143738] envoy[919]: [2026-09-20 10:51:09.160][919][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert4002lb # [ 18.145971] envoy[919]: [2026-09-20 10:51:09.160][919][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service4003lb # [ 18.150242] envoy[919]: [2026-09-20 10:51:09.160][919][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata4004lb # [ 18.152292] envoy[919]: [2026-09-20 10:51:09.160][919][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config4005lb # [ 18.155388] envoy[919]: [2026-09-20 10:51:09.160][919][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator4006lb # [ 18.157619] envoy[919]: [2026-09-20 10:51:09.160][919][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities4007lb # [ 18.159300] envoy[919]: [2026-09-20 10:51:09.165][919][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats4008lb # [ 18.161538] envoy[919]: [2026-09-20 10:51:09.165][919][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http14009lb # [ 18.164923] envoy[919]: [2026-09-20 10:51:09.165][919][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state4010lb # [ 18.168098] envoy[919]: [2026-09-20 10:51:09.165][919][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable4011lb # [ 18.170115] envoy[919]: [2026-09-20 10:51:09.165][919][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json4012lb # [ 18.171925] envoy[919]: [2026-09-20 10:51:09.165][919][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher4013lb # [ 18.174606] envoy[919]: [2026-09-20 10:51:09.165][919][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy4014lb # [ 18.176905] envoy[919]: [2026-09-20 10:51:09.165][919][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router4015lb # [ 18.179088] envoy[919]: [2026-09-20 10:51:09.165][919][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection4016lb # [ 18.183674] envoy[919]: [2026-09-20 10:51:09.165][919][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso4017lb # [ 18.185581] envoy[919]: [2026-09-20 10:51:09.165][919][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default4018lb # [ 18.187225] envoy[919]: [2026-09-20 10:51:09.165][919][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file4019lb # [ 18.188942] envoy[919]: [2026-09-20 10:51:09.165][919][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder4020lb # [ 18.191102] envoy[919]: [2026-09-20 10:51:09.165][919][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action4021lb # [ 18.194711] envoy[919]: [2026-09-20 10:51:09.165][919][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator4022lb # [ 18.197064] envoy[919]: [2026-09-20 10:51:09.165][919][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor4023lb # [ 18.199969] envoy[919]: [2026-09-20 10:51:09.168][919][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default4024lb # [ 18.201645] envoy[919]: [2026-09-20 10:51:09.168][919][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default4025lb # [ 18.203160] envoy[919]: [2026-09-20 10:51:09.168][919][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind4026lb # [ 18.204706] envoy[919]: [2026-09-20 10:51:09.168][919][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift4027lb # [ 18.207674] envoy[919]: [2026-09-20 10:51:09.168][919][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default4028lb # [ 18.210064] envoy[919]: [2026-09-20 10:51:09.168][919][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher4029lb # [ 18.212241] envoy[919]: [2026-09-20 10:51:09.168][919][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls4030lb # [ 18.217080] envoy[919]: [2026-09-20 10:51:09.175][919][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value4031lb # [ 18.220197] envoy[919]: [2026-09-20 10:51:09.175][919][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation4032lb # [ 18.222027] envoy[919]: [2026-09-20 10:51:09.175][919][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options4033lb # [ 18.225803] envoy[919]: [2026-09-20 10:51:09.175][919][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input4034lb # [ 18.227628] envoy[919]: [2026-09-20 10:51:09.175][919][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed4035lb # [ 18.229887] envoy[919]: [2026-09-20 10:51:09.175][919][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4036lb # [ 18.232534] envoy[919]: [2026-09-20 10:51:09.175][919][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case4037lb # [ 18.234949] envoy[919]: [2026-09-20 10:51:09.175][919][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor4038lb # [ 18.237430] envoy[919]: [2026-09-20 10:51:09.175][919][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection4039lb # [ 18.239243] envoy[919]: [2026-09-20 10:51:09.175][919][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian24040lb # [ 18.241104] envoy[919]: [2026-09-20 10:51:09.175][919][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo4041lb # [ 18.243187] envoy[919]: [2026-09-20 10:51:09.175][919][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest4042lb # [ 18.248254] envoy[919]: [2026-09-20 10:51:09.175][919][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re24043lb # [ 18.249903] envoy[919]: [2026-09-20 10:51:09.175][919][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params4044lb # [ 18.259262] envoy[919]: [2026-09-20 10:51:09.175][919][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector4045lb # [ 18.265105] envoy[919]: [2026-09-20 10:51:09.175][919][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override4046lb # [ 18.267980] envoy[919]: [2026-09-20 10:51:09.175][919][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr4047lb # [ 18.269798] envoy[919]: [2026-09-20 10:51:09.175][919][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based4048lb # [ 18.273746] envoy[919]: [2026-09-20 10:51:09.175][919][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit4049lb # [ 18.276496] envoy[919]: [2026-09-20 10:51:09.175][919][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response4050lb # [ 18.278727] envoy[919]: [2026-09-20 10:51:09.175][919][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter4051lb # [ 18.281084] envoy[919]: [2026-09-20 10:51:09.175][919][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default4052lb # [ 18.282897] envoy[919]: [2026-09-20 10:51:09.175][919][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple4053lb # [ 18.285000] envoy[919]: [2026-09-20 10:51:09.175][919][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme4054lb # [ 18.287900] envoy[919]: [2026-09-20 10:51:09.175][919][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap4055lb # [ 18.292085] envoy[919]: [2026-09-20 10:51:09.175][919][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy4056lb # [ 18.294906] envoy[919]: [2026-09-20 10:51:09.175][919][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated4057lb # [ 18.296908] envoy[919]: [2026-09-20 10:51:09.175][919][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle4058lb # [ 18.299050] envoy[919]: [2026-09-20 10:51:09.175][919][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default4059lb # [ 18.301080] envoy[919]: [2026-09-20 10:51:09.176][919][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query4060lb # [ 18.303983] envoy[919]: [2026-09-20 10:51:09.176][919][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4061lb # [ 18.305996] envoy[919]: [2026-09-20 10:51:09.176][919][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory4062lb # [ 18.308318] envoy[919]: [2026-09-20 10:51:09.176][919][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid4063lb # [ 18.309785] envoy[919]: [2026-09-20 10:51:09.176][919][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain4064lb # [ 18.311458] envoy[919]: [2026-09-20 10:51:09.176][919][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default4065lb # [ 18.313724] envoy[919]: [2026-09-20 10:51:09.176][919][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple4066lb # [ 18.316077] envoy[919]: [2026-09-20 10:51:09.176][919][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule4067lb # [ 18.318701] envoy[919]: [2026-09-20 10:51:09.176][919][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4068lb # [ 18.320771] envoy[919]: [2026-09-20 10:51:09.176][919][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin4069lb # [ 18.323782] envoy[919]: [2026-09-20 10:51:09.176][919][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed4070lb # [ 18.325856] envoy[919]: [2026-09-20 10:51:09.176][919][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd4071lb # [ 18.331154] envoy[919]: [2026-09-20 10:51:09.176][919][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy4072lb # [ 18.342853] envoy[919]: [2026-09-20 10:51:09.176][919][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port4073lb # [ 18.344771] envoy[919]: [2026-09-20 10:51:09.176][919][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche4074lb # [ 18.346736] envoy[919]: [2026-09-20 10:51:09.176][919][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default4075lb # [ 18.348646] envoy[919]: [2026-09-20 10:51:09.176][919][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter4076lb # [ 18.351087] envoy[919]: [2026-09-20 10:51:09.176][919][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal4077lb # [ 18.352722] envoy[919]: [2026-09-20 10:51:09.176][919][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe4078lb # [ 18.355247] envoy[919]: [2026-09-20 10:51:09.176][919][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls4079lb # [ 18.361040] envoy[919]: [2026-09-20 10:51:09.176][919][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth24080lb # [ 18.363294] envoy[919]: [2026-09-20 10:51:09.176][919][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override4081lb # [ 18.365725] envoy[919]: [2026-09-20 10:51:09.176][919][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic4082lb # [ 18.367376] envoy[919]: [2026-09-20 10:51:09.176][919][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction4083lb # [ 18.372634] envoy[919]: [2026-09-20 10:51:09.176][919][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san4084lb # [ 18.378714] envoy[919]: [2026-09-20 10:51:09.176][919][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based4085lb # [ 18.380432] envoy[919]: [2026-09-20 10:51:09.176][919][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router4086lb # [ 18.406067] envoy[919]: [2026-09-20 10:51:09.303][919][info][main] [source/server/server.cc:531] HTTP header map info:4087lb # [ 18.407558] envoy[919]: [2026-09-20 10:51:09.364][919][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id4088lb # [ 18.420832] envoy[919]: [2026-09-20 10:51:09.364][919][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:4089lb # [ 18.422777] envoy[919]: [2026-09-20 10:51:09.364][919][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id4090lb # [ 18.430321] envoy[919]: [2026-09-20 10:51:09.364][919][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status4091lb # [ 18.470264] envoy[919]: [2026-09-20 10:51:09.561][919][info][main] [source/server/server.cc:968] runtime: {}4092lb # [ 18.477347] envoy[919]: [2026-09-20 10:51:09.568][919][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.4093lb # [ 18.479673] envoy[919]: [2026-09-20 10:51:09.570][919][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99014094lb # [ 18.482204] envoy[919]: [2026-09-20 10:51:09.573][919][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99014095lb # [ 18.485319] envoy[919]: [2026-09-20 10:51:09.576][919][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration4096lb # [ 18.487057] envoy[919]: [2026-09-20 10:51:09.577][919][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)4097lb # [ 18.488616] envoy[919]: [2026-09-20 10:51:09.577][919][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)4098lb # [ 18.533053] envoy[919]: [2026-09-20 10:51:09.624][919][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)4099worker1 # [ 18.542202] systemd[1]: Starting Nix worker-protocol over gRPC...4100worker2 # [ 18.526662] systemd[1]: Starting Nix worker-protocol over gRPC...4101lb # [ 18.553977] envoy[919]: [2026-09-20 10:51:09.645][919][info][config] [source/server/configuration_impl.cc:164] loading stats configuration4102lb # [ 18.556567] envoy[919]: [2026-09-20 10:51:09.647][919][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.4103lb # [ 18.559474] envoy[919]: [2026-09-20 10:51:09.647][919][info][main] [source/server/server.cc:1077] starting main dispatch loop4104worker2 # [ 18.740314] nix-grpc-daemon[783]: ts=2026-09-20T10:51:08Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4105worker1 # [ 18.768546] nix-grpc-daemon[784]: ts=2026-09-20T10:51:09Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4106worker2 # [ 18.751067] nix-grpc-daemon[783]: ts=2026-09-20T10:51:08Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14107worker1 # [ 18.780178] nix-grpc-daemon[784]: ts=2026-09-20T10:51:09Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14108worker2 # [ 18.780793] nix-grpc-daemon[783]: ts=2026-09-20T10:51:09Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4109worker2 # [ 18.783885] systemd[1]: Started Nix worker-protocol over gRPC.4110worker1 # [ 18.821577] nix-grpc-daemon[784]: ts=2026-09-20T10:51:09Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4111worker1 # [ 18.826278] systemd[1]: Started Nix worker-protocol over gRPC.4112lb # [ 18.844047] envoy[919]: [2026-09-20 10:51:09.935][919][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization4113lb # [ 18.845814] envoy[919]: [2026-09-20 10:51:09.936][919][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized4114lb # [ 18.847567] envoy[919]: [2026-09-20 10:51:09.936][919][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager4115lb # [ 18.849223] envoy[919]: [2026-09-20 10:51:09.938][919][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers4116worker2 # [ 18.850420] nix-grpc-daemon[783]: ts=2026-09-20T10:51:09Z level=info event=scheduler_connected addr=lb:500514117worker2 # [ 18.858515] nix-grpc-daemon[783]: ts=2026-09-20T10:51:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4118worker1 # [ 18.900427] nix-grpc-daemon[784]: ts=2026-09-20T10:51:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4119worker2 # [ 19.059816] nix-grpc-daemon[783]: ts=2026-09-20T10:51:09Z level=info event=scheduler_connected addr=lb:500514120worker2 # [ 19.064155] nix-grpc-daemon[783]: ts=2026-09-20T10:51:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4121worker1 # [ 19.101747] nix-grpc-daemon[784]: ts=2026-09-20T10:51:10Z level=info event=scheduler_connected addr=lb:500514122worker1 # [ 19.105781] nix-grpc-daemon[784]: ts=2026-09-20T10:51:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4123worker2 # [ 19.264757] nix-grpc-daemon[783]: ts=2026-09-20T10:51:09Z level=info event=scheduler_connected addr=lb:500514124worker2 # [ 19.268643] nix-grpc-daemon[783]: ts=2026-09-20T10:51:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4125worker1 # [ 19.306927] nix-grpc-daemon[784]: ts=2026-09-20T10:51:10Z level=info event=scheduler_connected addr=lb:500514126worker1 # [ 19.311987] nix-grpc-daemon[784]: ts=2026-09-20T10:51:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4127worker2 # [ 19.469311] nix-grpc-daemon[783]: ts=2026-09-20T10:51:09Z level=info event=scheduler_connected addr=lb:500514128worker2 # [ 19.471129] nix-grpc-daemon[783]: ts=2026-09-20T10:51:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4129worker1 # [ 19.513062] nix-grpc-daemon[784]: ts=2026-09-20T10:51:10Z level=info event=scheduler_connected addr=lb:500514130worker1 # [ 19.517288] nix-grpc-daemon[784]: ts=2026-09-20T10:51:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4131worker2 # [ 19.671716] nix-grpc-daemon[783]: ts=2026-09-20T10:51:09Z level=info event=scheduler_connected addr=lb:500514132worker2 # [ 19.673554] nix-grpc-daemon[783]: ts=2026-09-20T10:51:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4133worker1 # [ 19.717535] nix-grpc-daemon[784]: ts=2026-09-20T10:51:10Z level=info event=scheduler_connected addr=lb:500514134worker1 # [ 19.719295] nix-grpc-daemon[784]: ts=2026-09-20T10:51:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4135worker2 # [ 19.875902] nix-grpc-daemon[783]: ts=2026-09-20T10:51:10Z level=info event=scheduler_connected addr=lb:500514136worker2 # [ 19.879104] nix-grpc-daemon[783]: ts=2026-09-20T10:51:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4137worker1 # [ 19.921752] nix-grpc-daemon[784]: ts=2026-09-20T10:51:11Z level=info event=scheduler_connected addr=lb:500514138worker1 # [ 19.924908] nix-grpc-daemon[784]: ts=2026-09-20T10:51:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4139worker2 # [ 20.076522] nix-grpc-daemon[783]: ts=2026-09-20T10:51:10Z level=info event=scheduler_connected addr=lb:500514140worker2 # [ 20.080568] nix-grpc-daemon[783]: ts=2026-09-20T10:51:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4141worker1 # [ 20.122126] nix-grpc-daemon[784]: ts=2026-09-20T10:51:11Z level=info event=scheduler_connected addr=lb:500514142worker1 # [ 20.126219] nix-grpc-daemon[784]: ts=2026-09-20T10:51:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4143worker2 # [ 20.281582] nix-grpc-daemon[783]: ts=2026-09-20T10:51:10Z level=info event=scheduler_connected addr=lb:500514144worker2 # [ 20.285644] nix-grpc-daemon[783]: ts=2026-09-20T10:51:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4145worker1 # [ 20.326927] nix-grpc-daemon[784]: ts=2026-09-20T10:51:11Z level=info event=scheduler_connected addr=lb:500514146worker1 # [ 20.329148] nix-grpc-daemon[784]: ts=2026-09-20T10:51:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4147worker2 # [ 20.486652] nix-grpc-daemon[783]: ts=2026-09-20T10:51:10Z level=info event=scheduler_connected addr=lb:500514148worker2 # [ 20.490547] nix-grpc-daemon[783]: ts=2026-09-20T10:51:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4149worker1 # [ 20.530149] nix-grpc-daemon[784]: ts=2026-09-20T10:51:11Z level=info event=scheduler_connected addr=lb:500514150worker1 # [ 20.534474] nix-grpc-daemon[784]: ts=2026-09-20T10:51:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4151worker2 # [ 20.691771] nix-grpc-daemon[783]: ts=2026-09-20T10:51:10Z level=info event=scheduler_connected addr=lb:500514152worker2 # [ 20.695509] nix-grpc-daemon[783]: ts=2026-09-20T10:51:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4153worker1 # [ 20.735088] nix-grpc-daemon[784]: ts=2026-09-20T10:51:11Z level=info event=scheduler_connected addr=lb:500514154worker1 # [ 20.737287] nix-grpc-daemon[784]: ts=2026-09-20T10:51:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4155worker2 # [ 20.897667] nix-grpc-daemon[783]: ts=2026-09-20T10:51:11Z level=info event=scheduler_connected addr=lb:500514156worker2 # [ 20.900751] nix-grpc-daemon[783]: ts=2026-09-20T10:51:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4157worker1 # [ 20.939701] nix-grpc-daemon[784]: ts=2026-09-20T10:51:12Z level=info event=scheduler_connected addr=lb:500514158worker1 # [ 20.942831] nix-grpc-daemon[784]: ts=2026-09-20T10:51:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4159worker2 # [ 21.098438] nix-grpc-daemon[783]: ts=2026-09-20T10:51:11Z level=info event=scheduler_connected addr=lb:500514160worker2 # [ 21.100605] nix-grpc-daemon[783]: ts=2026-09-20T10:51:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4161worker1 # [ 21.140574] nix-grpc-daemon[784]: ts=2026-09-20T10:51:12Z level=info event=scheduler_connected addr=lb:500514162worker1 # [ 21.144920] nix-grpc-daemon[784]: ts=2026-09-20T10:51:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4163worker2 # [ 21.301190] nix-grpc-daemon[783]: ts=2026-09-20T10:51:11Z level=info event=scheduler_connected addr=lb:500514164worker2 # [ 21.302834] nix-grpc-daemon[783]: ts=2026-09-20T10:51:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4165worker1 # [ 21.345658] nix-grpc-daemon[784]: ts=2026-09-20T10:51:12Z level=info event=scheduler_connected addr=lb:500514166worker1 # [ 21.349716] nix-grpc-daemon[784]: ts=2026-09-20T10:51:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4167worker2 # [ 21.504116] nix-grpc-daemon[783]: ts=2026-09-20T10:51:11Z level=info event=scheduler_connected addr=lb:500514168worker2 # [ 21.508526] nix-grpc-daemon[783]: ts=2026-09-20T10:51:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4169worker1 # [ 21.550870] nix-grpc-daemon[784]: ts=2026-09-20T10:51:12Z level=info event=scheduler_connected addr=lb:500514170worker1 # [ 21.555477] nix-grpc-daemon[784]: ts=2026-09-20T10:51:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4171worker2 # [ 21.709165] nix-grpc-daemon[783]: ts=2026-09-20T10:51:11Z level=info event=scheduler_connected addr=lb:500514172worker2 # [ 21.711183] nix-grpc-daemon[783]: ts=2026-09-20T10:51:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4173worker1 # [ 21.756350] nix-grpc-daemon[784]: ts=2026-09-20T10:51:12Z level=info event=scheduler_connected addr=lb:500514174worker1 # [ 21.760371] nix-grpc-daemon[784]: ts=2026-09-20T10:51:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4175worker2 # [ 21.913278] nix-grpc-daemon[783]: ts=2026-09-20T10:51:12Z level=info event=scheduler_connected addr=lb:500514176worker2 # [ 21.916341] nix-grpc-daemon[783]: ts=2026-09-20T10:51:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4177worker1 # [ 21.962622] nix-grpc-daemon[784]: ts=2026-09-20T10:51:13Z level=info event=scheduler_connected addr=lb:500514178worker1 # [ 21.965769] nix-grpc-daemon[784]: ts=2026-09-20T10:51:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4179worker2 # [ 22.114337] nix-grpc-daemon[783]: ts=2026-09-20T10:51:12Z level=info event=scheduler_connected addr=lb:500514180worker2 # [ 22.118239] nix-grpc-daemon[783]: ts=2026-09-20T10:51:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4181worker1 # [ 22.163264] nix-grpc-daemon[784]: ts=2026-09-20T10:51:13Z level=info event=scheduler_connected addr=lb:500514182worker1 # [ 22.165181] nix-grpc-daemon[784]: ts=2026-09-20T10:51:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4183worker2 # [ 22.319122] nix-grpc-daemon[783]: ts=2026-09-20T10:51:12Z level=info event=scheduler_connected addr=lb:500514184worker2 # [ 22.322932] nix-grpc-daemon[783]: ts=2026-09-20T10:51:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4185worker1 # [ 22.366374] nix-grpc-daemon[784]: ts=2026-09-20T10:51:13Z level=info event=scheduler_connected addr=lb:500514186worker1 # [ 22.370373] nix-grpc-daemon[784]: ts=2026-09-20T10:51:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4187worker2 # [ 22.524114] nix-grpc-daemon[783]: ts=2026-09-20T10:51:12Z level=info event=scheduler_connected addr=lb:500514188worker2 # [ 22.528354] nix-grpc-daemon[783]: ts=2026-09-20T10:51:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4189worker1 # [ 22.571297] nix-grpc-daemon[784]: ts=2026-09-20T10:51:13Z level=info event=scheduler_connected addr=lb:500514190lb # [ 22.571027] rustfs-bucket-start[972]: mb s3://farm4191worker1 # [ 22.575201] nix-grpc-daemon[784]: ts=2026-09-20T10:51:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4192lb # [ 22.576366] systemd[1]: Finished rustfs-bucket.service.4193lb # [ 22.582491] systemd[1]: Starting niks3 server...4194worker2 # [ 22.728911] nix-grpc-daemon[783]: ts=2026-09-20T10:51:12Z level=info event=scheduler_connected addr=lb:500514195worker2 # [ 22.733328] nix-grpc-daemon[783]: ts=2026-09-20T10:51:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4196worker1 # [ 22.775875] nix-grpc-daemon[784]: ts=2026-09-20T10:51:13Z level=info event=scheduler_connected addr=lb:500514197worker1 # [ 22.777982] nix-grpc-daemon[784]: ts=2026-09-20T10:51:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4198lb # [ 22.811837] postgres[992]: [992] ERROR: relation "goose_db_version" does not exist at character 364199lb # [ 22.813203] postgres[992]: [992] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4200lb # [ 22.859700] niks3-server[983]: 2026/09/20 10:51:13 OK 20241026095416_initial_model.sql (20.03ms)4201lb # [ 22.862970] niks3-server[983]: 2026/09/20 10:51:13 OK 20251210153512_drop_unused_gin_index.sql (3.31ms)4202lb # [ 22.867624] niks3-server[983]: 2026/09/20 10:51:13 OK 20251218171726_add_pins.sql (4.59ms)4203lb # [ 22.872164] niks3-server[983]: 2026/09/20 10:51:13 OK 20260628120000_add_object_size_and_stats.sql (4.6ms)4204lb # [ 22.876301] niks3-server[983]: 2026/09/20 10:51:13 OK 20260905000000_add_claims.sql (2.75ms)4205lb # [ 22.880867] niks3-server[983]: 2026/09/20 10:51:13 OK 20260920000000_drop_claims.sql (4.28ms)4206lb # [ 22.882127] niks3-server[983]: 2026/09/20 10:51:13 goose: successfully migrated database to version: 202609200000004207lb # [ 22.887289] niks3-server[983]: 2026/09/20 10:51:13 OK 1_commit_pending_closure.sql (6.58ms)4208lb # [ 22.889623] niks3-server[983]: 2026/09/20 10:51:13 OK 2_object_stats_trigger.sql (2.31ms)4209lb # [ 22.890796] niks3-server[983]: 2026/09/20 10:51:13 goose: up to current file version: 24210lb # [ 22.896436] niks3-server[983]: 2026/09/20 10:51:13 INFO Loaded signing key name=farm-test-1 path=/nix/store/c745bsy529madd2s6hz4z5is2436ni04-key4211worker2 # [ 22.935715] nix-grpc-daemon[783]: ts=2026-09-20T10:51:13Z level=info event=scheduler_connected addr=lb:500514212worker2 # [ 22.938817] nix-grpc-daemon[783]: ts=2026-09-20T10:51:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4213lb # [ 22.961710] niks3-server[983]: 2026/09/20 10:51:14 INFO Read proxy enabled — serving cache objects from S34214lb # [ 22.963872] niks3-server[983]: 2026/09/20 10:51:14 INFO Using socket-activated listener address=0.0.0.0:57514215lb # [ 22.970095] niks3-server[983]: 2026/09/20 10:51:14 INFO systemd watchdog enabled interval=15s4216lb # [ 22.971236] systemd[1]: Started niks3 server.4217lb # [ 22.971887] systemd[1]: Reached target Multi-User System.4218lb # [ 22.973812] systemd[1]: Startup finished in 878ms (kernel) + 4.059s (initrd) + 18.035s (userspace) = 22.973s.4219lb # [ 22.975460] niks3-server[983]: 2026/09/20 10:51:14 INFO Starting HTTP server address=0.0.0.0:57514220worker1 # [ 22.980662] nix-grpc-daemon[784]: ts=2026-09-20T10:51:14Z level=info event=scheduler_connected addr=lb:500514221lb # [ 22.983473] niks3-server[983]: 2026/09/20 10:51:14 INFO lead: acquired remote=192.168.1.3:361664222worker1 # [ 22.984119] nix-grpc-daemon[784]: ts=2026-09-20T10:51:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4223worker1 # [ 22.992382] nix-grpc-daemon[784]: ts=2026-09-20T10:51:14Z level=info event=scheduler_take_over4224worker2 # [ 23.136571] nix-grpc-daemon[783]: ts=2026-09-20T10:51:13Z level=info event=scheduler_connected addr=lb:500514225worker2 # [ 23.140457] nix-grpc-daemon[783]: ts=2026-09-20T10:51:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4226worker1 # [ 23.181896] nix-grpc-daemon[784]: ts=2026-09-20T10:51:14Z level=info event=scheduler_connected addr=lb:500514227worker1 # [ 23.185984] nix-grpc-daemon[784]: ts=2026-09-20T10:51:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4228worker2 # [ 23.341165] nix-grpc-daemon[783]: ts=2026-09-20T10:51:13Z level=info event=scheduler_connected addr=lb:500514229worker2 # [ 23.344953] nix-grpc-daemon[783]: ts=2026-09-20T10:51:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4230worker1 # [ 23.386816] nix-grpc-daemon[784]: ts=2026-09-20T10:51:14Z level=info event=scheduler_connected addr=lb:500514231worker1 # [ 23.390861] nix-grpc-daemon[784]: ts=2026-09-20T10:51:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4232worker2 # [ 23.545693] nix-grpc-daemon[783]: ts=2026-09-20T10:51:13Z level=info event=scheduler_connected addr=lb:500514233worker2 # [ 23.549715] nix-grpc-daemon[783]: ts=2026-09-20T10:51:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4234worker1 # [ 23.591670] nix-grpc-daemon[784]: ts=2026-09-20T10:51:14Z level=info event=scheduler_connected addr=lb:500514235worker1 # [ 23.595582] nix-grpc-daemon[784]: ts=2026-09-20T10:51:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4236worker2 # [ 23.750500] nix-grpc-daemon[783]: ts=2026-09-20T10:51:13Z level=info event=scheduler_connected addr=lb:500514237worker2 # [ 23.754561] nix-grpc-daemon[783]: ts=2026-09-20T10:51:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4238worker1 # [ 23.796530] nix-grpc-daemon[784]: ts=2026-09-20T10:51:14Z level=info event=scheduler_connected addr=lb:500514239worker1 # [ 23.800464] nix-grpc-daemon[784]: ts=2026-09-20T10:51:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4240lb: (finished: waiting for unit niks3.service, in 12.36 seconds)4241lb: waiting for TCP port 5751 on localhost4242worker2 # [ 23.955885] nix-grpc-daemon[783]: ts=2026-09-20T10:51:14Z level=info event=scheduler_connected addr=lb:500514243lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!4244worker1 # [ 23.985711] nix-grpc-daemon[784]: ts=2026-09-20T10:51:15Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04245lb: (finished: waiting for TCP port 5751 on localhost, in 0.07 seconds)4246lb: waiting for unit envoy.service4247worker1 # [ 24.000908] nix-grpc-daemon[784]: ts=2026-09-20T10:51:15Z level=info event=scheduler_connected addr=lb:500514248worker1 # [ 24.004856] nix-grpc-daemon[784]: ts=2026-09-20T10:51:15Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04249lb: (finished: waiting for unit envoy.service, in 0.06 seconds)4250lb: waiting for TCP port 50051 on localhost4251lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!4252lb # [ 24.085499] envoy[919]: conn peer=[::1]:51742 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=34253lb: (finished: waiting for TCP port 50051 on localhost, in 0.04 seconds)4254lb: waiting until exactly no endpoints of x86_64-linux are unhealthy4255??? Warning (UserWarning): retry(): The 'timeout_seconds' argument is deprecated. Use 'timeout' instead.4256 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 1284257lb: must succeed: curl -sf localhost:9901/clusters4258lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4259lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.09 seconds)4260subtest: one node schedules and both builders hold a WorkerSession on it4261lb: must succeed: curl -sf localhost:9901/clusters4262lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4263worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4264worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4265(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.12 seconds)4266subtest: balancer auth: client cert, bearer token, nothing, unknown CN4267worker1 # [ 24.720362] nix-grpc-daemon[784]: ts=2026-09-20T10:51:15Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:389064268worker1 # [ 24.733851] systemd[1]: Started Nix Daemon.4269worker1 # [ 24.945417] nix-daemon[843]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4270worker1 # [ 24.992821] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 85 ms (attempt 1/5)4271worker1 # [ 25.080850] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 115 ms (attempt 2/5)4272worker1 # [ 25.201981] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 205 ms (attempt 3/5)4273worker1 # [ 25.413105] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 335 ms (attempt 4/5)4274worker1 # [ 25.759727] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4275worker1 # [ 25.792306] nix-grpc-daemon[784]: ts=2026-09-20T10:51:16Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:389064276worker1 # [ 25.810951] nix-grpc-daemon[784]: ts=2026-09-20T10:51:16Z level=debug event=path_info_miss path=3yn33ipjwn92p38w4qwdnvyyq096zfzd-nixos-system-client-test error="error: path '/nix/store/3yn33ipjwn92p38w4qwdnvyyq096zfzd-nixos-system-client-test' is required, but there is no substituter that can build it"4277worker2 # [ 25.818225] nix-grpc-daemon[783]: ts=2026-09-20T10:51:16Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:418164278worker2 # [ 25.832542] systemd[1]: Started Nix Daemon.4279worker2 # [ 26.066094] nix-daemon[839]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4280worker2 # [ 26.098184] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 48 ms (attempt 1/5)4281worker2 # [ 26.146440] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 123 ms (attempt 2/5)4282worker2 # [ 26.272613] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 10 ms (attempt 3/5)4283worker2 # [ 26.285552] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 195 ms (attempt 4/5)4284worker2 # [ 26.488141] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4285worker2 # [ 26.502939] nix-grpc-daemon[783]: ts=2026-09-20T10:51:16Z level=debug event=path_info_miss path=3yn33ipjwn92p38w4qwdnvyyq096zfzd-nixos-system-client-test error="error: path '/nix/store/3yn33ipjwn92p38w4qwdnvyyq096zfzd-nixos-system-client-test' is required, but there is no substituter that can build it"4286client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4287client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.08 seconds)4288client: must succeed: install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4289client: (finished: must succeed: install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds)4290worker1 # [ 26.897655] nix-grpc-daemon[784]: ts=2026-09-20T10:51:17Z level=info event=oidc_jwks provider=mock keys=14291worker1 # [ 26.902720] nix-grpc-daemon[784]: ts=2026-09-20T10:51:17Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:389064292worker2 # [ 26.906613] nix-grpc-daemon[783]: ts=2026-09-20T10:51:17Z level=info event=oidc_jwks provider=mock keys=14293worker2 # [ 26.908557] nix-grpc-daemon[783]: ts=2026-09-20T10:51:17Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:418164294worker2 # [ 26.914825] nix-grpc-daemon[783]: ts=2026-09-20T10:51:17Z level=debug event=path_info_miss path=3yn33ipjwn92p38w4qwdnvyyq096zfzd-nixos-system-client-test error="error: path '/nix/store/3yn33ipjwn92p38w4qwdnvyyq096zfzd-nixos-system-client-test' is required, but there is no substituter that can build it"4295worker1 # [ 26.937711] nix-grpc-daemon[784]: ts=2026-09-20T10:51:18Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:389064296worker1 # [ 26.947880] nix-grpc-daemon[784]: ts=2026-09-20T10:51:18Z level=debug event=path_info_miss path=3yn33ipjwn92p38w4qwdnvyyq096zfzd-nixos-system-client-test error="error: path '/nix/store/3yn33ipjwn92p38w4qwdnvyyq096zfzd-nixos-system-client-test' is required, but there is no substituter that can build it"4297client: must succeed: rm -r /var/lib/nix-grpc-store4298client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.03 seconds)4299worker1 # [ 27.187963] nix-grpc-daemon[784]: ts=2026-09-20T10:51:18Z level=info event=denied method=QueryMissing cn=- role=none4300worker1 # [ 27.372701] nix-grpc-daemon[784]: ts=2026-09-20T10:51:18Z level=info event=denied method=QueryMissing cn=stranger role=none4301(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 2.95 seconds)4302subtest: DAG build is scheduled across workers and lands in the cache4303client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24304client # don't know how to build these paths:4305client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4306worker1 # [ 27.576850] nix-grpc-daemon[784]: ts=2026-09-20T10:51:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:389064307worker1 # [ 27.585823] nix-grpc-daemon[784]: ts=2026-09-20T10:51:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:389064308worker1 # [ 27.623524] nix-grpc-daemon[784]: ts=2026-09-20T10:51:18Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04309worker1 # [ 27.627845] nix-grpc-daemon[784]: ts=2026-09-20T10:51:18Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.4:50051 clients=14310worker2 # [ 27.661501] nix-grpc-daemon[783]: ts=2026-09-20T10:51:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418164311worker2 # [ 27.682368] nix-grpc-daemon[783]: ts=2026-09-20T10:51:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418164312client # copying 1 paths to 'lb:50051'...4313worker2 # [ 27.695387] nix-grpc-daemon[783]: ts=2026-09-20T10:51:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418164314client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4315worker2 # [ 27.717312] nix-daemon[839]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4316worker2 # [ 27.742271] nix-grpc-daemon[783]: ts=2026-09-20T10:51:17Z level=info event=niks3_push_started pid=8494317worker2 # [ 28.189138] nix-daemon[839]: accepted connection from pid 860, user nix-grpc-daemon (trusted)4318worker2 # [ 28.207052] nix-daemon[839]: reaped child process 864, status = succeeded4319worker2 # [ 28.352930] nix-daemon[839]: accepted connection from pid 866, user nix-grpc-daemon (trusted)4320worker2 # [ 28.366561] nix-daemon[839]: reaped child process 869, status = succeeded4321lb # [ 28.397659] niks3-server[983]: 2026/09/20 10:51:19 INFO Received uploads request method=POST path=/api/pending_closures4322worker2 # [ 28.399676] nix-grpc-daemon[849]: time=2026-09-20T10:51:18.630Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4323worker2 # [ 28.402449] nix-grpc-daemon[849]: time=2026-09-20T10:51:18.633Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4324lb # [ 28.444875] niks3-server[983]: 2026/09/20 10:51:19 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4325lb # [ 28.447671] niks3-server[983]: 2026/09/20 10:51:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4326lb # [ 28.453610] niks3-server[983]: 2026/09/20 10:51:19 INFO Signed narinfos id=1 count=14327worker2 # [ 28.440768] nix-grpc-daemon[849]: time=2026-09-20T10:51:18.672Z level=INFO msg="Uploading 1 narinfos"4328lb # [ 28.468290] niks3-server[983]: 2026/09/20 10:51:19 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4329lb # [ 28.476079] niks3-server[983]: 2026/09/20 10:51:19 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4330lb # [ 28.481577] niks3-server[983]: 2026/09/20 10:51:19 INFO Completed upload id=14331worker2 # [ 28.466372] nix-grpc-daemon[849]: time=2026-09-20T10:51:18.698Z level=INFO msg="Upload complete. (435ms)"4332worker2 # [ 28.471418] nix-grpc-daemon[783]: ts=2026-09-20T10:51:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41816 duration_s=0 paths=1 nar_bytes_in=5044333worker2 # [ 28.477600] nix-grpc-daemon[849]: time=2026-09-20T10:51:18.701Z level=WARN msg="Failed to register uploaded object" key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo error="server returned 404: object is not pending upload\n"4334worker2 # [ 28.482151] nix-grpc-daemon[783]: ts=2026-09-20T10:51:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418164335worker2 # [ 28.490495] nix-daemon[839]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4336worker2 # [ 28.493402] nix-daemon[839]: reaped child process 847, status = succeeded4337client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4338worker2 # [ 28.500204] nix-daemon[839]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4339client # farm-a> worker2: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4340lb # [ 28.539175] envoy[919]: conn peer=[2001:db8:1::1]:39808 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=749 tx=422 ms=18484341lb # [ 28.540652] envoy[919]: conn peer=[2001:db8:1::1]:39810 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=844342lb # [ 28.541986] envoy[919]: conn peer=[2001:db8:1::1]:39822 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=64343lb # [ 28.543303] envoy[919]: conn peer=[2001:db8:1::1]:39830 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=84344client # farm-a> LOG-farm-a-t14345worker2 # [ 29.928504] nix-daemon[839]: reaped child process 873, status = succeeded4346worker2 # [ 30.093499] nix-daemon[839]: accepted connection from pid 888, user nix-grpc-daemon (trusted)4347worker2 # [ 30.107645] nix-daemon[839]: reaped child process 891, status = succeeded4348lb # [ 30.138616] niks3-server[983]: 2026/09/20 10:51:21 INFO Received uploads request method=POST path=/api/pending_closures4349worker2 # [ 30.130220] nix-grpc-daemon[849]: time=2026-09-20T10:51:20.361Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4350worker2 # [ 30.132083] nix-grpc-daemon[849]: time=2026-09-20T10:51:20.363Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4351lb # [ 30.173448] niks3-server[983]: 2026/09/20 10:51:21 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4352lb # [ 30.175904] niks3-server[983]: 2026/09/20 10:51:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4353lb # [ 30.177904] niks3-server[983]: 2026/09/20 10:51:21 INFO Signed narinfos id=2 count=14354worker2 # [ 30.162627] nix-grpc-daemon[849]: time=2026-09-20T10:51:20.394Z level=INFO msg="Uploading 1 narinfos"4355lb # [ 30.181609] niks3-server[983]: 2026/09/20 10:51:21 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4356lb # [ 30.183710] niks3-server[983]: 2026/09/20 10:51:21 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4357lb # [ 30.189327] niks3-server[983]: 2026/09/20 10:51:21 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4358lb # [ 30.191928] niks3-server[983]: 2026/09/20 10:51:21 INFO Completed upload id=24359worker2 # [ 30.176309] nix-grpc-daemon[849]: time=2026-09-20T10:51:20.408Z level=INFO msg="Upload complete. (265ms)"4360lb # [ 30.194204] niks3-server[983]: 2026/09/20 10:51:21 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4361worker2 # [ 30.179659] nix-grpc-daemon[783]: ts=2026-09-20T10:51:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41816 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14362worker2 # [ 30.187860] nix-daemon[839]: reaped child process 871, status = succeeded4363worker1 # [ 30.219591] nix-grpc-daemon[784]: ts=2026-09-20T10:51:21Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04364worker1 # [ 30.222704] nix-grpc-daemon[784]: ts=2026-09-20T10:51:21Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.4:50051 clients=14365worker2 # [ 30.247117] nix-grpc-daemon[783]: ts=2026-09-20T10:51:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418164366worker2 # [ 30.259257] nix-grpc-daemon[783]: ts=2026-09-20T10:51:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418164367worker2 # [ 30.271375] nix-grpc-daemon[849]: time=2026-09-20T10:51:20.503Z level=INFO msg="All 1 paths already cached"4368client # copying 1 paths to 'lb:50051'...4369worker2 # [ 30.278990] nix-grpc-daemon[783]: ts=2026-09-20T10:51:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418164370client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4371worker2 # [ 30.297142] nix-daemon[839]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4372worker2 # [ 30.486298] nix-daemon[839]: accepted connection from pid 896, user nix-grpc-daemon (trusted)4373worker2 # [ 30.500365] nix-daemon[839]: reaped child process 899, status = succeeded4374worker2 # [ 30.669286] nix-daemon[839]: accepted connection from pid 902, user nix-grpc-daemon (trusted)4375worker2 # [ 30.683346] nix-daemon[839]: reaped child process 905, status = succeeded4376lb # [ 30.712115] niks3-server[983]: 2026/09/20 10:51:21 INFO Received uploads request method=POST path=/api/pending_closures4377worker2 # [ 30.717774] nix-grpc-daemon[849]: time=2026-09-20T10:51:20.949Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4378worker2 # [ 30.719285] nix-grpc-daemon[849]: time=2026-09-20T10:51:20.949Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4379lb # [ 30.758427] niks3-server[983]: 2026/09/20 10:51:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4380lb # [ 30.766861] niks3-server[983]: 2026/09/20 10:51:21 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4381lb # [ 30.771775] niks3-server[983]: 2026/09/20 10:51:21 INFO Signed narinfos id=3 count=14382worker2 # [ 30.760174] nix-grpc-daemon[849]: time=2026-09-20T10:51:20.991Z level=INFO msg="Uploading 1 narinfos"4383lb # [ 30.780276] niks3-server[983]: 2026/09/20 10:51:21 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4384lb # [ 30.787451] niks3-server[983]: 2026/09/20 10:51:21 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4385lb # [ 30.920956] niks3-server[983]: 2026/09/20 10:51:22 INFO Completed upload id=34386worker2 # [ 30.905757] nix-grpc-daemon[849]: time=2026-09-20T10:51:21.137Z level=INFO msg="Upload complete. (597ms)"4387lb # [ 30.927160] niks3-server[983]: 2026/09/20 10:51:22 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4388worker2 # [ 30.911290] nix-grpc-daemon[783]: ts=2026-09-20T10:51:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41816 duration_s=0 paths=1 nar_bytes_in=6244389worker2 # [ 30.919139] nix-grpc-daemon[783]: ts=2026-09-20T10:51:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418164390worker2 # [ 30.932349] nix-daemon[839]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4391worker2 # [ 30.941915] nix-daemon[839]: reaped child process 894, status = succeeded4392client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4393worker2 # [ 30.953966] nix-daemon[839]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4394client # farm-b> worker2: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4395client # farm-b> LOG-farm-b-t14396worker2 # [ 32.185333] nix-daemon[839]: reaped child process 909, status = succeeded4397worker2 # [ 32.340558] nix-daemon[839]: accepted connection from pid 924, user nix-grpc-daemon (trusted)4398worker2 # [ 32.353472] nix-daemon[839]: reaped child process 927, status = succeeded4399lb # [ 32.386327] niks3-server[983]: 2026/09/20 10:51:23 INFO Received uploads request method=POST path=/api/pending_closures4400worker2 # [ 32.380866] nix-grpc-daemon[849]: time=2026-09-20T10:51:22.612Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4401worker2 # [ 32.382303] nix-grpc-daemon[849]: time=2026-09-20T10:51:22.614Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4402lb # [ 32.431743] niks3-server[983]: 2026/09/20 10:51:23 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4403lb # [ 32.442632] niks3-server[983]: 2026/09/20 10:51:23 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4404lb # [ 32.444354] niks3-server[983]: 2026/09/20 10:51:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4405lb # [ 32.446567] niks3-server[983]: 2026/09/20 10:51:23 INFO Signed narinfos id=4 count=14406worker2 # [ 32.432420] nix-grpc-daemon[849]: time=2026-09-20T10:51:22.664Z level=INFO msg="Uploading 1 narinfos"4407lb # [ 32.450264] niks3-server[983]: 2026/09/20 10:51:23 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4408lb # [ 32.457063] niks3-server[983]: 2026/09/20 10:51:23 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4409lb # [ 32.461098] niks3-server[983]: 2026/09/20 10:51:23 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4410lb # [ 32.464088] niks3-server[983]: 2026/09/20 10:51:23 INFO Completed upload id=44411worker2 # [ 32.448747] nix-grpc-daemon[849]: time=2026-09-20T10:51:22.680Z level=INFO msg="Upload complete. (282ms)"4412worker2 # [ 32.454778] nix-grpc-daemon[783]: ts=2026-09-20T10:51:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41816 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14413worker1 # [ 32.484061] nix-grpc-daemon[784]: ts=2026-09-20T10:51:23Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04414worker2 # [ 32.468115] nix-daemon[839]: reaped child process 907, status = succeeded4415worker1 # [ 32.488869] nix-grpc-daemon[784]: ts=2026-09-20T10:51:23Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.4:50051 clients=14416worker2 # [ 32.509508] nix-grpc-daemon[783]: ts=2026-09-20T10:51:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418164417worker2 # [ 32.519689] nix-grpc-daemon[783]: ts=2026-09-20T10:51:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418164418worker2 # [ 32.532359] nix-grpc-daemon[849]: time=2026-09-20T10:51:22.764Z level=INFO msg="All 2 paths already cached"4419client # copying 1 paths to 'lb:50051'...4420worker2 # [ 32.540720] nix-grpc-daemon[783]: ts=2026-09-20T10:51:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418164421client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4422worker2 # [ 32.560131] nix-daemon[839]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4423worker2 # [ 32.731844] nix-daemon[839]: accepted connection from pid 931, user nix-grpc-daemon (trusted)4424worker2 # [ 32.745110] nix-daemon[839]: reaped child process 934, status = succeeded4425worker2 # [ 32.901643] nix-daemon[839]: accepted connection from pid 936, user nix-grpc-daemon (trusted)4426worker2 # [ 32.914338] nix-daemon[839]: reaped child process 939, status = succeeded4427lb # [ 32.945393] niks3-server[983]: 2026/09/20 10:51:24 INFO Received uploads request method=POST path=/api/pending_closures4428worker2 # [ 32.938533] nix-grpc-daemon[849]: time=2026-09-20T10:51:23.170Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4429worker2 # [ 32.940139] nix-grpc-daemon[849]: time=2026-09-20T10:51:23.172Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4430lb # [ 32.976989] niks3-server[983]: 2026/09/20 10:51:24 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4431lb # [ 32.988365] niks3-server[983]: 2026/09/20 10:51:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4432lb # [ 32.993497] niks3-server[983]: 2026/09/20 10:51:24 INFO Signed narinfos id=5 count=14433worker2 # [ 32.981047] nix-grpc-daemon[849]: time=2026-09-20T10:51:23.212Z level=INFO msg="Uploading 1 narinfos"4434lb # [ 33.002444] niks3-server[983]: 2026/09/20 10:51:24 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4435lb # [ 33.013415] niks3-server[983]: 2026/09/20 10:51:24 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4436lb # [ 33.019481] niks3-server[983]: 2026/09/20 10:51:24 INFO Completed upload id=54437lb # [ 33.023362] niks3-server[983]: 2026/09/20 10:51:24 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4438worker2 # [ 33.007154] nix-grpc-daemon[849]: time=2026-09-20T10:51:23.238Z level=INFO msg="Upload complete. (433ms)"4439worker2 # [ 33.012297] nix-grpc-daemon[783]: ts=2026-09-20T10:51:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41816 duration_s=0 paths=1 nar_bytes_in=7524440worker2 # [ 33.016994] nix-grpc-daemon[783]: ts=2026-09-20T10:51:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418164441worker2 # [ 33.026207] nix-daemon[839]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4442worker2 # [ 33.034658] nix-daemon[839]: reaped child process 929, status = succeeded4443client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4444client # farm-top> worker2: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4445worker2 # [ 33.040339] nix-daemon[839]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4446client # farm-top> LOG-farm-top-t14447worker2 # [ 33.868765] nix-daemon[839]: reaped child process 943, status = succeeded4448worker2 # [ 34.017182] nix-daemon[839]: accepted connection from pid 959, user nix-grpc-daemon (trusted)4449worker2 # [ 34.029766] nix-daemon[839]: reaped child process 962, status = succeeded4450lb # [ 34.061649] niks3-server[983]: 2026/09/20 10:51:25 INFO Received uploads request method=POST path=/api/pending_closures4451worker2 # [ 34.057234] nix-grpc-daemon[849]: time=2026-09-20T10:51:24.289Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4452worker2 # [ 34.060514] nix-grpc-daemon[849]: time=2026-09-20T10:51:24.291Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4453lb # [ 34.102650] niks3-server[983]: 2026/09/20 10:51:25 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4454lb # [ 34.109780] niks3-server[983]: 2026/09/20 10:51:25 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4455lb # [ 34.122857] niks3-server[983]: 2026/09/20 10:51:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4456worker2 # [ 34.112498] nix-grpc-daemon[849]: time=2026-09-20T10:51:24.344Z level=INFO msg="Uploading 1 narinfos"4457lb # [ 34.128098] niks3-server[983]: 2026/09/20 10:51:25 INFO Signed narinfos id=6 count=14458lb # [ 34.132921] niks3-server[983]: 2026/09/20 10:51:25 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4459lb # [ 34.142558] niks3-server[983]: 2026/09/20 10:51:25 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4460lb # [ 34.145644] niks3-server[983]: 2026/09/20 10:51:25 INFO Completed upload id=64461lb # [ 34.147817] niks3-server[983]: 2026/09/20 10:51:25 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4462worker2 # [ 34.131410] nix-grpc-daemon[849]: time=2026-09-20T10:51:24.363Z level=INFO msg="Upload complete. (283ms)"4463worker2 # [ 34.137821] nix-grpc-daemon[783]: ts=2026-09-20T10:51:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41816 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14464worker2 # [ 34.152258] nix-daemon[839]: reaped child process 941, status = succeeded4465client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 6.81 seconds)4466client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4467client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.17 seconds)4468client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4469client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.01 seconds)4470client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4471client # copying 3 paths...4472client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4473client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4474client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4475client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.26 seconds)4476(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 7.34 seconds)4477subtest: repeat is answered Cached by the scheduler without building4478worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4479worker1 # finding garbage collector roots...4480worker1 # deleting specified paths...4481worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4482worker1 # deleting unused links...4483worker1 # note: hard linking is currently saving 0.0 KiB4484worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.27 seconds)4485worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4486worker2 # finding garbage collector roots...4487worker2 # deleting specified paths...4488worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4489worker2 # deleting unused links...4490worker2 # note: hard linking is currently saving 0.0 KiB4491worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.24 seconds)4492client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24493worker2 # [ 35.426929] nix-grpc-daemon[783]: ts=2026-09-20T10:51:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:418164494client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4495client # /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4496worker1 # [ 35.466452] nix-grpc-daemon[784]: ts=2026-09-20T10:51:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:389064497worker1 # [ 35.494203] nix-grpc-daemon[784]: ts=2026-09-20T10:51:26Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14498client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 0.28 seconds)4499client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4500client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.15 seconds)4501lb: must succeed: curl -sf localhost:9901/clusters4502lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4503lb: must succeed: curl -sf localhost:9901/clusters4504lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4505worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4506worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4507(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.15 seconds)4508subtest: inputs already in the cache are not sent to the scheduler4509lb: must succeed: curl -sf localhost:9901/clusters4510lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4511lb: must succeed: curl -sf localhost:9901/clusters4512lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4513worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4514worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.03 seconds)4515client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&24516client # don't know how to build these paths:4517worker1 # [ 36.227555] nix-grpc-daemon[784]: ts=2026-09-20T10:51:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:389064518client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4519worker1 # [ 36.230229] nix-grpc-daemon[784]: ts=2026-09-20T10:51:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:389064520worker1 # [ 36.236955] nix-grpc-daemon[784]: ts=2026-09-20T10:51:27Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04521worker1 # [ 36.238832] nix-grpc-daemon[784]: ts=2026-09-20T10:51:27Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.4:50051 clients=14522worker2 # [ 36.262166] nix-grpc-daemon[783]: ts=2026-09-20T10:51:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418164523worker2 # [ 36.272398] nix-grpc-daemon[783]: ts=2026-09-20T10:51:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418164524worker2 # [ 36.282716] nix-grpc-daemon[849]: time=2026-09-20T10:51:26.514Z level=INFO msg="All 2 paths already cached"4525client # copying 1 paths to 'lb:50051'...4526worker2 # [ 36.289994] nix-grpc-daemon[783]: ts=2026-09-20T10:51:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418164527client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4528worker2 # [ 36.305573] nix-daemon[839]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4529worker2 # [ 36.485421] nix-daemon[839]: accepted connection from pid 979, user nix-grpc-daemon (trusted)4530worker2 # [ 36.498686] nix-daemon[839]: reaped child process 982, status = succeeded4531worker2 # [ 36.675145] nix-daemon[839]: accepted connection from pid 984, user nix-grpc-daemon (trusted)4532worker2 # [ 36.688079] nix-daemon[839]: reaped child process 988, status = succeeded4533lb # [ 36.720932] niks3-server[983]: 2026/09/20 10:51:27 INFO Received uploads request method=POST path=/api/pending_closures4534worker2 # [ 36.710340] nix-grpc-daemon[849]: time=2026-09-20T10:51:26.942Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4535worker2 # [ 36.711768] nix-grpc-daemon[849]: time=2026-09-20T10:51:26.942Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4536lb # [ 36.734365] niks3-server[983]: 2026/09/20 10:51:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4537lb # [ 36.736071] niks3-server[983]: 2026/09/20 10:51:27 INFO Signed narinfos id=7 count=14538worker2 # [ 36.721769] nix-grpc-daemon[849]: time=2026-09-20T10:51:26.953Z level=INFO msg="Uploading 1 narinfos"4539lb # [ 36.739715] niks3-server[983]: 2026/09/20 10:51:27 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4540lb # [ 36.742434] niks3-server[983]: 2026/09/20 10:51:27 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4541lb # [ 36.747440] niks3-server[983]: 2026/09/20 10:51:27 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4542lb # [ 36.750650] niks3-server[983]: 2026/09/20 10:51:27 INFO Completed upload id=74543worker2 # [ 36.735098] nix-grpc-daemon[849]: time=2026-09-20T10:51:26.967Z level=INFO msg="Upload complete. (417ms)"4544lb # [ 36.752089] niks3-server[983]: 2026/09/20 10:51:27 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4545worker2 # [ 36.737400] nix-grpc-daemon[783]: ts=2026-09-20T10:51:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41816 duration_s=0 paths=1 nar_bytes_in=7524546worker2 # [ 36.740784] nix-grpc-daemon[783]: ts=2026-09-20T10:51:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418164547worker2 # [ 36.747882] nix-daemon[839]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4548worker2 # [ 36.750219] nix-daemon[839]: reaped child process 977, status = succeeded4549worker2 # [ 36.756129] nix-daemon[839]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4550client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4551client # farm-top> worker2: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4552client # farm-top> LOG-farm-top-t1b4553worker2 # [ 38.371708] nix-daemon[839]: reaped child process 992, status = succeeded4554worker2 # [ 38.491763] nix-daemon[839]: accepted connection from pid 1007, user nix-grpc-daemon (trusted)4555worker2 # [ 38.503923] nix-daemon[839]: reaped child process 1010, status = succeeded4556lb # [ 38.531774] niks3-server[983]: 2026/09/20 10:51:29 INFO Received uploads request method=POST path=/api/pending_closures4557lb # [ 38.540562] envoy[919]: conn peer=[2001:db8:1::1]:39844 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9511 tx=3816 ms=66194558worker2 # [ 38.529574] nix-grpc-daemon[849]: time=2026-09-20T10:51:28.761Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4559worker2 # [ 38.531045] nix-grpc-daemon[849]: time=2026-09-20T10:51:28.763Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4560lb # [ 38.545507] envoy[919]: conn peer=[2001:db8:1::1]:47564 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=634561lb # [ 38.577173] niks3-server[983]: 2026/09/20 10:51:29 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4562lb # [ 38.585664] niks3-server[983]: 2026/09/20 10:51:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4563lb # [ 38.590213] niks3-server[983]: 2026/09/20 10:51:29 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4564lb # [ 38.592389] niks3-server[983]: 2026/09/20 10:51:29 INFO Signed narinfos id=8 count=14565worker2 # [ 38.578886] nix-grpc-daemon[849]: time=2026-09-20T10:51:28.810Z level=INFO msg="Uploading 1 narinfos"4566lb # [ 38.601458] niks3-server[983]: 2026/09/20 10:51:29 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4567lb # [ 38.603388] niks3-server[983]: 2026/09/20 10:51:29 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4568lb # [ 38.606693] niks3-server[983]: 2026/09/20 10:51:29 INFO Completed upload id=84569lb # [ 38.608112] niks3-server[983]: 2026/09/20 10:51:29 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4570worker2 # [ 38.592162] nix-grpc-daemon[849]: time=2026-09-20T10:51:28.824Z level=INFO msg="Upload complete. (228ms)"4571worker2 # [ 38.594831] nix-grpc-daemon[783]: ts=2026-09-20T10:51:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41816 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14572worker2 # [ 38.601750] nix-daemon[839]: reaped child process 990, status = succeeded4573client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.65 seconds)4574client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath4575client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.20 seconds)4576lb: must succeed: curl -sf localhost:9901/clusters4577lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4578lb: must succeed: curl -sf localhost:9901/clusters4579lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4580worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4581worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.03 seconds)4582(finished: subtest: inputs already in the cache are not sent to the scheduler, in 3.08 seconds)4583subtest: two clients wanting the same drv build it once4584client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4585client # [ 38.999788] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4586client # Running as unit: dedup1.service; invocation ID: 77bc62a67e3e41fc8b78c134f010f1944587client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 0.06 seconds)4588client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4589worker2 # [ 39.174744] nix-grpc-daemon[783]: ts=2026-09-20T10:51:29Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:418164590client # [ 39.180393] nix[881]: don't know how to build these paths:4591client # [ 39.181836] nix[881]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4592worker1 # [ 39.207637] nix-grpc-daemon[784]: ts=2026-09-20T10:51:30Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04593worker1 # [ 39.213271] nix-grpc-daemon[784]: ts=2026-09-20T10:51:30Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.3:50051 clients=14594worker1 # [ 39.217953] nix-grpc-daemon[784]: ts=2026-09-20T10:51:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:389064595worker1 # [ 39.232066] nix-grpc-daemon[784]: ts=2026-09-20T10:51:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:389064596client # [ 39.226777] nix[881]: copying 1 paths to 'lb:50051'...4597client # [ 39.228187] nix[881]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4598worker1 # [ 39.243074] nix-grpc-daemon[784]: ts=2026-09-20T10:51:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:389064599worker1 # [ 39.256189] nix-daemon[843]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4600worker1 # [ 39.277486] nix-grpc-daemon[784]: ts=2026-09-20T10:51:30Z level=info event=niks3_push_started pid=8954601worker1 # [ 39.710083] nix-daemon[843]: accepted connection from pid 908, user nix-grpc-daemon (trusted)4602worker1 # [ 39.726131] nix-daemon[843]: reaped child process 911, status = succeeded4603worker1 # [ 39.899495] nix-daemon[843]: accepted connection from pid 913, user nix-grpc-daemon (trusted)4604worker1 # [ 39.913501] nix-daemon[843]: reaped child process 916, status = succeeded4605lb # [ 39.927770] niks3-server[983]: 2026/09/20 10:51:31 INFO Received uploads request method=POST path=/api/pending_closures4606worker1 # [ 39.943633] nix-grpc-daemon[895]: time=2026-09-20T10:51:31.027Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4607worker1 # [ 39.946519] nix-grpc-daemon[895]: time=2026-09-20T10:51:31.029Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4608lb # [ 39.975581] niks3-server[983]: 2026/09/20 10:51:31 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4609lb # [ 39.983761] niks3-server[983]: 2026/09/20 10:51:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4610worker1 # [ 39.995982] nix-grpc-daemon[895]: time=2026-09-20T10:51:31.081Z level=INFO msg="Uploading 1 narinfos"4611lb # [ 39.992585] niks3-server[983]: 2026/09/20 10:51:31 INFO Signed narinfos id=9 count=14612lb # [ 40.004716] niks3-server[983]: 2026/09/20 10:51:31 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4613lb # [ 40.012978] niks3-server[983]: 2026/09/20 10:51:31 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4614worker1 # [ 40.055807] nix-grpc-daemon[895]: time=2026-09-20T10:51:31.141Z level=INFO msg="Upload complete. (515ms)"4615lb # [ 40.052828] niks3-server[983]: 2026/09/20 10:51:31 INFO Completed upload id=94616worker1 # [ 40.059074] nix-grpc-daemon[784]: ts=2026-09-20T10:51:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38906 duration_s=0 paths=1 nar_bytes_in=4324617lb # [ 40.057074] niks3-server[983]: 2026/09/20 10:51:31 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4618worker1 # [ 40.066470] nix-grpc-daemon[784]: ts=2026-09-20T10:51:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:389064619worker1 # [ 40.071240] nix-daemon[843]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4620worker1 # [ 40.074042] nix-daemon[843]: reaped child process 893, status = succeeded4621worker1 # [ 40.079975] nix-daemon[843]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4622client # [ 40.065156] nix[881]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4623client # [ 40.079644] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4624client # Running as unit: dedup2.service; invocation ID: 1dafbbf6023a4ec5b00e38c8d457c5164625client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 1.09 seconds)4626worker2 # [ 40.323493] nix-grpc-daemon[783]: ts=2026-09-20T10:51:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:418164627client # [ 40.328495] nix[897]: don't know how to build these paths:4628client # [ 40.331221] nix[897]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4629worker1 # [ 40.354490] nix-grpc-daemon[784]: ts=2026-09-20T10:51:31Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04630worker1 # [ 40.358108] nix-grpc-daemon[784]: ts=2026-09-20T10:51:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:389064631client # [ 40.345333] nix[897]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4632client: must succeed: sleep 24633client: (finished: must succeed: sleep 2, in 2.06 seconds)4634worker1 # [ 43.495961] nix-daemon[843]: reaped child process 920, status = succeeded4635??? Warning (UserWarning): wait_until_succeeds(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4636 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394637client: waiting for success: ! systemctl is-active dedup1 dedup24638??? Warning (UserWarning): execute(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4639 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394640worker1 # [ 43.629820] nix-daemon[843]: accepted connection from pid 952, user nix-grpc-daemon (trusted)4641worker1 # [ 43.643392] nix-daemon[843]: reaped child process 955, status = succeeded4642lb # [ 43.650992] niks3-server[983]: 2026/09/20 10:51:34 INFO Received uploads request method=POST path=/api/pending_closures4643worker1 # [ 43.658731] nix-grpc-daemon[895]: time=2026-09-20T10:51:34.744Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4644worker1 # [ 43.660364] nix-grpc-daemon[895]: time=2026-09-20T10:51:34.746Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4645lb # [ 43.680079] niks3-server[983]: 2026/09/20 10:51:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4646lb # [ 43.682686] niks3-server[983]: 2026/09/20 10:51:34 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4647lb # [ 43.684786] niks3-server[983]: 2026/09/20 10:51:34 INFO Signed narinfos id=10 count=14648worker1 # [ 43.688308] nix-grpc-daemon[895]: time=2026-09-20T10:51:34.773Z level=INFO msg="Uploading 1 narinfos"4649lb # [ 43.686687] niks3-server[983]: 2026/09/20 10:51:34 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4650lb # [ 43.689813] niks3-server[983]: 2026/09/20 10:51:34 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4651lb # [ 43.694885] niks3-server[983]: 2026/09/20 10:51:34 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4652lb # [ 43.698773] niks3-server[983]: 2026/09/20 10:51:34 INFO Completed upload id=104653worker1 # [ 43.701801] nix-grpc-daemon[895]: time=2026-09-20T10:51:34.787Z level=INFO msg="Upload complete. (211ms)"4654lb # [ 43.700513] niks3-server[983]: 2026/09/20 10:51:34 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4655worker1 # [ 43.704934] nix-grpc-daemon[784]: ts=2026-09-20T10:51:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38906 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14656worker1 # [ 43.708785] nix-grpc-daemon[784]: ts=2026-09-20T10:51:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38906 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14657worker1 # [ 43.715171] nix-daemon[843]: reaped child process 918, status = succeeded4658client # [ 43.738309] systemd[1]: dedup1.service: Deactivated successfully.4659client # [ 43.741390] systemd[1]: dedup1.service: Consumed 103ms CPU time over 4.741s wall clock time, 37.3M memory peak, 8.4K incoming IP traffic, 8.9K outgoing IP traffic.4660client # [ 43.747445] systemd[1]: dedup2.service: Deactivated successfully.4661client # [ 43.749549] systemd[1]: dedup2.service: Consumed 125ms CPU time over 3.668s wall clock time, 31.2M memory peak, 7.4K incoming IP traffic, 6.4K outgoing IP traffic.4662client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.10 seconds)4663client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4664client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)4665worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4666worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.06 seconds)4667worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4668worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.06 seconds)4669worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4670worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4671worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4672worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4673(finished: subtest: two clients wanting the same drv build it once, in 5.99 seconds)4674subtest: build hook: nix-daemon with builders = grpc://lb4675client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log4676client # [ 45.120572] systemd[1]: Started Nix Daemon.4677client # [ 45.277040] nix-daemon[939]: accepted connection from pid 936, user root (trusted)4678worker1 # [ 45.465467] nix-grpc-daemon[784]: ts=2026-09-20T10:51:36Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:389064679worker2 # [ 45.454362] nix-grpc-daemon[783]: ts=2026-09-20T10:51:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418164680worker1 # [ 45.482600] nix-grpc-daemon[784]: ts=2026-09-20T10:51:36Z level=debug event=want client=6 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=04681worker1 # [ 45.486797] nix-grpc-daemon[784]: ts=2026-09-20T10:51:36Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.4:50051 clients=14682worker2 # [ 45.508134] nix-grpc-daemon[783]: ts=2026-09-20T10:51:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418164683worker2 # [ 45.520919] nix-grpc-daemon[783]: ts=2026-09-20T10:51:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418164684worker2 # [ 45.533141] nix-grpc-daemon[783]: ts=2026-09-20T10:51:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418164685worker2 # [ 45.553522] nix-daemon[839]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4686worker2 # [ 45.737573] nix-daemon[839]: accepted connection from pid 1037, user nix-grpc-daemon (trusted)4687worker2 # [ 45.751391] nix-daemon[839]: reaped child process 1040, status = succeeded4688worker2 # [ 45.918397] nix-daemon[839]: accepted connection from pid 1042, user nix-grpc-daemon (trusted)4689worker2 # [ 45.932191] nix-daemon[839]: reaped child process 1045, status = succeeded4690lb # [ 45.962993] niks3-server[983]: 2026/09/20 10:51:37 INFO Received uploads request method=POST path=/api/pending_closures4691worker2 # [ 45.957729] nix-grpc-daemon[849]: time=2026-09-20T10:51:36.189Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4692worker2 # [ 45.959252] nix-grpc-daemon[849]: time=2026-09-20T10:51:36.189Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"4693lb # [ 45.993495] niks3-server[983]: 2026/09/20 10:51:37 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst4694lb # [ 46.002935] niks3-server[983]: 2026/09/20 10:51:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4695lb # [ 46.008347] niks3-server[983]: 2026/09/20 10:51:37 INFO Signed narinfos id=11 count=14696worker2 # [ 45.997669] nix-grpc-daemon[849]: time=2026-09-20T10:51:36.229Z level=INFO msg="Uploading 1 narinfos"4697lb # [ 46.016558] niks3-server[983]: 2026/09/20 10:51:37 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls4698lb # [ 46.027852] niks3-server[983]: 2026/09/20 10:51:37 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4699lb # [ 46.034400] niks3-server[983]: 2026/09/20 10:51:37 INFO Completed upload id=114700worker2 # [ 46.018858] nix-grpc-daemon[849]: time=2026-09-20T10:51:36.250Z level=INFO msg="Upload complete. (453ms)"4701lb # [ 46.037700] niks3-server[983]: 2026/09/20 10:51:37 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo4702worker2 # [ 46.024187] nix-grpc-daemon[783]: ts=2026-09-20T10:51:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41816 duration_s=0 paths=1 nar_bytes_in=5124703worker2 # [ 46.028821] nix-grpc-daemon[783]: ts=2026-09-20T10:51:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418164704worker2 # [ 46.037367] nix-daemon[839]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4705worker2 # [ 46.045195] nix-daemon[839]: reaped child process 1035, status = succeeded4706worker2 # [ 46.050092] nix-daemon[839]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4707worker2 # [ 47.077713] nix-daemon[839]: reaped child process 1049, status = succeeded4708worker2 # [ 47.232455] nix-daemon[839]: accepted connection from pid 1064, user nix-grpc-daemon (trusted)4709worker2 # [ 47.245806] nix-daemon[839]: reaped child process 1067, status = succeeded4710lb # [ 47.277494] niks3-server[983]: 2026/09/20 10:51:38 INFO Received uploads request method=POST path=/api/pending_closures4711worker2 # [ 47.269656] nix-grpc-daemon[849]: time=2026-09-20T10:51:37.501Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4712worker2 # [ 47.271173] nix-grpc-daemon[849]: time=2026-09-20T10:51:37.503Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"4713lb # [ 47.312730] niks3-server[983]: 2026/09/20 10:51:38 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv4714lb # [ 47.322793] niks3-server[983]: 2026/09/20 10:51:38 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4715lb # [ 47.332715] niks3-server[983]: 2026/09/20 10:51:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4716lb # [ 47.338072] niks3-server[983]: 2026/09/20 10:51:38 INFO Signed narinfos id=12 count=14717worker2 # [ 47.326368] nix-grpc-daemon[849]: time=2026-09-20T10:51:37.558Z level=INFO msg="Uploading 1 narinfos"4718lb # [ 47.343383] niks3-server[983]: 2026/09/20 10:51:38 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls4719lb # [ 47.352824] niks3-server[983]: 2026/09/20 10:51:38 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4720lb # [ 47.359465] niks3-server[983]: 2026/09/20 10:51:38 INFO Completed upload id=124721worker2 # [ 47.343972] nix-grpc-daemon[849]: time=2026-09-20T10:51:37.575Z level=INFO msg="Upload complete. (286ms)"4722lb # [ 47.361590] niks3-server[983]: 2026/09/20 10:51:38 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo4723worker2 # [ 47.349622] nix-grpc-daemon[783]: ts=2026-09-20T10:51:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41816 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=6 outputs=14724worker1 # [ 47.371673] nix-grpc-daemon[784]: ts=2026-09-20T10:51:38Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:389064725worker2 # [ 47.363187] nix-daemon[839]: reaped child process 1047, status = succeeded4726worker1 # [ 47.383688] nix-grpc-daemon[784]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...4727worker1 # [ 47.418307] nix-grpc-daemon[784]: ts=2026-09-20T10:51:38Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:389064728worker1 # [ 47.424151] nix-grpc-daemon[784]: ts=2026-09-20T10:51:38Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:38906 duration_s=0 paths=1 nar_bytes_out=1284729worker1 # [ 47.672225] nix-grpc-daemon[784]: ts=2026-09-20T10:51:38Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:389064730worker2 # [ 47.655756] nix-grpc-daemon[783]: ts=2026-09-20T10:51:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418164731worker2 # [ 47.670025] nix-grpc-daemon[849]: time=2026-09-20T10:51:37.902Z level=INFO msg="All 1 paths already cached"4732worker1 # [ 47.700258] nix-grpc-daemon[784]: ts=2026-09-20T10:51:38Z level=debug event=want client=7 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=04733worker1 # [ 47.704820] nix-grpc-daemon[784]: ts=2026-09-20T10:51:38Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.4:50051 clients=14734worker2 # [ 47.688687] nix-grpc-daemon[783]: ts=2026-09-20T10:51:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418164735worker2 # [ 47.700581] nix-grpc-daemon[783]: ts=2026-09-20T10:51:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418164736worker2 # [ 47.712470] nix-grpc-daemon[849]: time=2026-09-20T10:51:37.944Z level=INFO msg="All 1 paths already cached"4737worker2 # [ 47.719316] nix-grpc-daemon[783]: ts=2026-09-20T10:51:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418164738worker2 # [ 47.727563] nix-daemon[839]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4739worker2 # [ 47.900147] nix-daemon[839]: accepted connection from pid 1071, user nix-grpc-daemon (trusted)4740worker2 # [ 47.913978] nix-daemon[839]: reaped child process 1074, status = succeeded4741worker2 # [ 48.057255] nix-daemon[839]: accepted connection from pid 1076, user nix-grpc-daemon (trusted)4742worker2 # [ 48.070844] nix-daemon[839]: reaped child process 1079, status = succeeded4743lb # [ 48.100802] niks3-server[983]: 2026/09/20 10:51:39 INFO Received uploads request method=POST path=/api/pending_closures4744worker2 # [ 48.089409] nix-grpc-daemon[849]: time=2026-09-20T10:51:38.321Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4745worker2 # [ 48.091401] nix-grpc-daemon[849]: time=2026-09-20T10:51:38.322Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"4746lb # [ 48.116241] niks3-server[983]: 2026/09/20 10:51:39 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst4747lb # [ 48.120315] niks3-server[983]: 2026/09/20 10:51:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4748lb # [ 48.122364] niks3-server[983]: 2026/09/20 10:51:39 INFO Signed narinfos id=13 count=14749worker2 # [ 48.107072] nix-grpc-daemon[849]: time=2026-09-20T10:51:38.338Z level=INFO msg="Uploading 1 narinfos"4750lb # [ 48.127830] niks3-server[983]: 2026/09/20 10:51:39 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls4751lb # [ 48.130830] niks3-server[983]: 2026/09/20 10:51:39 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4752lb # [ 48.134153] niks3-server[983]: 2026/09/20 10:51:39 INFO Completed upload id=134753worker2 # [ 48.118623] nix-grpc-daemon[849]: time=2026-09-20T10:51:38.350Z level=INFO msg="Upload complete. (377ms)"4754lb # [ 48.136067] niks3-server[983]: 2026/09/20 10:51:39 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo4755worker2 # [ 48.121091] nix-grpc-daemon[783]: ts=2026-09-20T10:51:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41816 duration_s=0 paths=1 nar_bytes_in=6244756worker2 # [ 48.125182] nix-grpc-daemon[783]: ts=2026-09-20T10:51:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418164757worker2 # [ 48.131389] nix-daemon[839]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4758worker2 # [ 48.134474] nix-daemon[839]: reaped child process 1069, status = succeeded4759worker2 # [ 48.139942] nix-daemon[839]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4760lb # [ 48.540557] envoy[919]: conn peer=[2001:db8:1::1]:47580 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4139 tx=1684 ms=23964761lb # [ 48.544296] envoy[919]: conn peer=[2001:db8:1::1]:47588 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2955 tx=1352 ms=45254762lb # [ 48.547861] envoy[919]: conn peer=[2001:db8:1::1]:47590 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1440 tx=1082 ms=33864763lb # [ 48.551367] envoy[919]: conn peer=[2001:db8:1::1]:38092 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=644764lb # [ 48.554756] envoy[919]: conn peer=[2001:db8:1::1]:38082 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3423 tx=1771 ms=20154765worker2 # [ 49.152968] nix-daemon[839]: reaped child process 1083, status = succeeded4766worker2 # [ 49.298071] nix-daemon[839]: accepted connection from pid 1098, user nix-grpc-daemon (trusted)4767worker2 # [ 49.311835] nix-daemon[839]: reaped child process 1101, status = succeeded4768lb # [ 49.342451] niks3-server[983]: 2026/09/20 10:51:40 INFO Received uploads request method=POST path=/api/pending_closures4769worker2 # [ 49.336533] nix-grpc-daemon[849]: time=2026-09-20T10:51:39.567Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4770worker2 # [ 49.337870] nix-grpc-daemon[849]: time=2026-09-20T10:51:39.567Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"4771lb # [ 49.382578] niks3-server[983]: 2026/09/20 10:51:40 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst4772lb # [ 49.390228] niks3-server[983]: 2026/09/20 10:51:40 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv4773lb # [ 49.397209] niks3-server[983]: 2026/09/20 10:51:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4774lb # [ 49.401857] niks3-server[983]: 2026/09/20 10:51:40 INFO Signed narinfos id=14 count=14775worker2 # [ 49.389076] nix-grpc-daemon[849]: time=2026-09-20T10:51:39.620Z level=INFO msg="Uploading 1 narinfos"4776lb # [ 49.409163] niks3-server[983]: 2026/09/20 10:51:40 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls4777lb # [ 49.417454] niks3-server[983]: 2026/09/20 10:51:40 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4778lb # [ 49.424373] niks3-server[983]: 2026/09/20 10:51:40 INFO Completed upload id=144779lb # [ 49.426129] niks3-server[983]: 2026/09/20 10:51:40 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo4780worker2 # [ 49.410131] nix-grpc-daemon[849]: time=2026-09-20T10:51:39.642Z level=INFO msg="Upload complete. (263ms)"4781worker2 # [ 49.415724] nix-grpc-daemon[783]: ts=2026-09-20T10:51:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41816 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=7 outputs=14782worker1 # [ 49.440271] nix-grpc-daemon[784]: ts=2026-09-20T10:51:40Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:389064783worker2 # [ 49.429742] nix-daemon[839]: reaped child process 1081, status = succeeded4784worker1 # [ 49.448399] nix-grpc-daemon[784]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...4785worker2 # [ 49.455963] nix-grpc-daemon[783]: ts=2026-09-20T10:51:39Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:418164786worker2 # [ 49.461472] nix-grpc-daemon[783]: ts=2026-09-20T10:51:39Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41816 duration_s=0 paths=1 nar_bytes_out=1764787worker1 # [ 49.672784] nix-grpc-daemon[784]: ts=2026-09-20T10:51:40Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:389064788worker1 # [ 49.681413] nix-grpc-daemon[784]: ts=2026-09-20T10:51:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:389064789worker1 # [ 49.695749] nix-grpc-daemon[895]: time=2026-09-20T10:51:40.781Z level=INFO msg="All 2 paths already cached"4790worker1 # [ 49.706213] nix-grpc-daemon[784]: ts=2026-09-20T10:51:40Z level=debug event=want client=8 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=04791worker1 # [ 49.710697] nix-grpc-daemon[784]: ts=2026-09-20T10:51:40Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.4:50051 clients=14792worker2 # [ 49.694391] nix-grpc-daemon[783]: ts=2026-09-20T10:51:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418164793worker2 # [ 49.703837] nix-grpc-daemon[783]: ts=2026-09-20T10:51:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418164794worker2 # [ 49.709895] nix-grpc-daemon[849]: time=2026-09-20T10:51:39.941Z level=INFO msg="All 2 paths already cached"4795worker2 # [ 49.713160] nix-grpc-daemon[783]: ts=2026-09-20T10:51:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418164796worker2 # [ 49.718068] nix-daemon[839]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4797worker2 # [ 49.867319] nix-daemon[839]: accepted connection from pid 1105, user nix-grpc-daemon (trusted)4798worker2 # [ 49.882301] nix-daemon[839]: reaped child process 1108, status = succeeded4799worker2 # [ 50.027703] nix-daemon[839]: accepted connection from pid 1110, user nix-grpc-daemon (trusted)4800worker2 # [ 50.040825] nix-daemon[839]: reaped child process 1113, status = succeeded4801lb # [ 50.070929] niks3-server[983]: 2026/09/20 10:51:41 INFO Received uploads request method=POST path=/api/pending_closures4802worker2 # [ 50.064796] nix-grpc-daemon[849]: time=2026-09-20T10:51:40.296Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4803worker2 # [ 50.066284] nix-grpc-daemon[849]: time=2026-09-20T10:51:40.298Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"4804lb # [ 50.102089] niks3-server[983]: 2026/09/20 10:51:41 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst4805lb # [ 50.112063] niks3-server[983]: 2026/09/20 10:51:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4806worker2 # [ 50.100714] nix-grpc-daemon[849]: time=2026-09-20T10:51:40.332Z level=INFO msg="Uploading 1 narinfos"4807lb # [ 50.116611] niks3-server[983]: 2026/09/20 10:51:41 INFO Signed narinfos id=15 count=14808lb # [ 50.122627] niks3-server[983]: 2026/09/20 10:51:41 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls4809lb # [ 50.132944] niks3-server[983]: 2026/09/20 10:51:41 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4810lb # [ 50.139675] niks3-server[983]: 2026/09/20 10:51:41 INFO Completed upload id=154811worker2 # [ 50.126133] nix-grpc-daemon[849]: time=2026-09-20T10:51:40.358Z level=INFO msg="Upload complete. (402ms)"4812lb # [ 50.142794] niks3-server[983]: 2026/09/20 10:51:41 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo4813worker2 # [ 50.129094] nix-grpc-daemon[783]: ts=2026-09-20T10:51:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41816 duration_s=0 paths=1 nar_bytes_in=7524814worker2 # [ 50.134242] nix-grpc-daemon[783]: ts=2026-09-20T10:51:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418164815worker2 # [ 50.143140] nix-daemon[839]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4816worker2 # [ 50.149969] nix-daemon[839]: reaped child process 1103, status = succeeded4817worker2 # [ 50.153242] nix-daemon[839]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4818worker2 # [ 51.381925] nix-daemon[839]: reaped child process 1117, status = succeeded4819worker2 # [ 51.532321] nix-daemon[839]: accepted connection from pid 1132, user nix-grpc-daemon (trusted)4820worker2 # [ 51.545316] nix-daemon[839]: reaped child process 1135, status = succeeded4821lb # [ 51.576449] niks3-server[983]: 2026/09/20 10:51:42 INFO Received uploads request method=POST path=/api/pending_closures4822worker2 # [ 51.569928] nix-grpc-daemon[849]: time=2026-09-20T10:51:41.801Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4823worker2 # [ 51.571429] nix-grpc-daemon[849]: time=2026-09-20T10:51:41.803Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"4824lb # [ 51.617447] niks3-server[983]: 2026/09/20 10:51:42 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst4825lb # [ 51.622161] niks3-server[983]: 2026/09/20 10:51:42 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv4826lb # [ 51.630345] niks3-server[983]: 2026/09/20 10:51:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4827lb # [ 51.634246] niks3-server[983]: 2026/09/20 10:51:42 INFO Signed narinfos id=16 count=14828worker2 # [ 51.622156] nix-grpc-daemon[849]: time=2026-09-20T10:51:41.854Z level=INFO msg="Uploading 1 narinfos"4829lb # [ 51.640689] niks3-server[983]: 2026/09/20 10:51:42 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls4830lb # [ 51.648256] niks3-server[983]: 2026/09/20 10:51:42 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4831lb # [ 51.658810] niks3-server[983]: 2026/09/20 10:51:42 INFO Completed upload id=164832lb # [ 51.660616] niks3-server[983]: 2026/09/20 10:51:42 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo4833worker2 # [ 51.644703] nix-grpc-daemon[849]: time=2026-09-20T10:51:41.876Z level=INFO msg="Upload complete. (283ms)"4834worker1 # [ 51.670122] nix-grpc-daemon[784]: ts=2026-09-20T10:51:42Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:389064835worker2 # [ 51.650918] nix-grpc-daemon[783]: ts=2026-09-20T10:51:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41816 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=8 outputs=14836worker1 # [ 51.675371] nix-grpc-daemon[784]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...4837worker2 # [ 51.664428] nix-daemon[839]: reaped child process 1115, status = succeeded4838worker2 # [ 51.674703] nix-grpc-daemon[783]: ts=2026-09-20T10:51:41Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:418164839worker2 # [ 51.678565] nix-grpc-daemon[783]: ts=2026-09-20T10:51:41Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41816 duration_s=0 paths=1 nar_bytes_out=2324840client # [ 51.793726] nix-daemon[939]: reaped child process 941, status = succeeded4841client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.87 seconds)4842client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top4843client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.03 seconds)4844client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4845client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.03 seconds)4846client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4847client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.02 seconds)4848(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 6.96 seconds)4849subtest: an input only one worker has still reaches the builder4850client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only4851client # this derivation will be built:4852client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv4853client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...4854client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only, in 0.37 seconds)4855client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure4856client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.18 seconds)4857worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4858worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.21 seconds)4859worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only4860worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)4861client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&24862client # [ 52.874091] nix-daemon[939]: accepted connection from pid 1018, user root (trusted)4863client # this derivation will be built:4864client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv4865worker1 # [ 53.059781] nix-grpc-daemon[784]: ts=2026-09-20T10:51:44Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:389064866worker2 # [ 53.043427] nix-grpc-daemon[783]: ts=2026-09-20T10:51:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418164867client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux'...4868client # copying 1 paths...4869client # copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' to 'grpc://lb:50051'...4870worker1 # [ 53.070138] nix-grpc-daemon[784]: ts=2026-09-20T10:51:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:389064871worker1 # [ 53.082345] nix-daemon[843]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4872worker1 # [ 53.260209] nix-daemon[843]: accepted connection from pid 982, user nix-grpc-daemon (trusted)4873worker1 # [ 53.273547] nix-daemon[843]: reaped child process 985, status = succeeded4874lb # [ 53.286669] niks3-server[983]: 2026/09/20 10:51:44 INFO Received uploads request method=POST path=/api/pending_closures4875worker1 # [ 53.351705] nix-grpc-daemon[895]: time=2026-09-20T10:51:44.436Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4876worker1 # [ 53.353284] nix-grpc-daemon[895]: time=2026-09-20T10:51:44.436Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"4877lb # [ 53.369568] niks3-server[983]: 2026/09/20 10:51:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4878lb # [ 53.376803] niks3-server[983]: 2026/09/20 10:51:44 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4879lb # [ 53.381715] niks3-server[983]: 2026/09/20 10:51:44 INFO Signed narinfos id=17 count=14880worker1 # [ 53.388713] nix-grpc-daemon[895]: time=2026-09-20T10:51:44.473Z level=INFO msg="Uploading 1 narinfos"4881lb # [ 53.389265] niks3-server[983]: 2026/09/20 10:51:44 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls4882lb # [ 53.399970] niks3-server[983]: 2026/09/20 10:51:44 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4883lb # [ 53.406112] niks3-server[983]: 2026/09/20 10:51:44 INFO Completed upload id=174884worker1 # [ 53.408530] nix-grpc-daemon[895]: time=2026-09-20T10:51:44.494Z level=INFO msg="Upload complete. (316ms)"4885lb # [ 53.411304] niks3-server[983]: 2026/09/20 10:51:44 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo4886worker1 # [ 53.414181] nix-grpc-daemon[784]: ts=2026-09-20T10:51:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38906 duration_s=0 paths=1 nar_bytes_in=1204887worker1 # [ 53.427846] nix-grpc-daemon[784]: ts=2026-09-20T10:51:44Z level=debug event=want client=9 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=04888worker1 # [ 53.435329] nix-grpc-daemon[784]: ts=2026-09-20T10:51:44Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=14889worker1 # [ 53.442519] nix-daemon[843]: reaped child process 980, status = succeeded4890worker1 # [ 53.472526] nix-grpc-daemon[784]: ts=2026-09-20T10:51:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:389064891worker1 # [ 53.485268] nix-grpc-daemon[784]: ts=2026-09-20T10:51:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:389064892client # copying 2 paths to 'lb:50051'...4893worker1 # [ 53.501160] nix-grpc-daemon[784]: ts=2026-09-20T10:51:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:389064894client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...4895client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...4896worker1 # [ 53.518874] nix-daemon[843]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4897worker1 # [ 53.690354] nix-daemon[843]: accepted connection from pid 991, user nix-grpc-daemon (trusted)4898worker1 # [ 53.704853] nix-daemon[843]: reaped child process 994, status = succeeded4899worker1 # [ 53.865504] nix-daemon[843]: accepted connection from pid 996, user nix-grpc-daemon (trusted)4900worker1 # [ 53.879235] nix-daemon[843]: reaped child process 999, status = succeeded4901lb # [ 53.891233] niks3-server[983]: 2026/09/20 10:51:44 INFO Received uploads request method=POST path=/api/pending_closures4902lb # [ 53.899811] niks3-server[983]: 2026/09/20 10:51:44 INFO Received uploads request method=POST path=/api/pending_closures4903worker1 # [ 53.911219] nix-grpc-daemon[895]: time=2026-09-20T10:51:44.996Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4904worker1 # [ 53.912745] nix-grpc-daemon[895]: time=2026-09-20T10:51:44.996Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"4905worker1 # [ 53.914374] nix-grpc-daemon[895]: time=2026-09-20T10:51:44.996Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"4906lb # [ 53.938959] niks3-server[983]: 2026/09/20 10:51:45 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst4907lb # [ 53.949093] niks3-server[983]: 2026/09/20 10:51:45 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls4908lb # [ 53.953467] niks3-server[983]: 2026/09/20 10:51:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4909lb # [ 53.958821] niks3-server[983]: 2026/09/20 10:51:45 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst4910lb # [ 53.962277] niks3-server[983]: 2026/09/20 10:51:45 INFO Signed narinfos id=19 count=24911lb # [ 53.965796] niks3-server[983]: 2026/09/20 10:51:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4912lb # [ 53.967622] niks3-server[983]: 2026/09/20 10:51:45 INFO Signed narinfos id=18 count=14913lb # [ 53.969508] niks3-server[983]: 2026/09/20 10:51:45 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls4914worker1 # [ 53.972617] nix-grpc-daemon[895]: time=2026-09-20T10:51:45.056Z level=INFO msg="Uploading 3 narinfos"4915lb # [ 53.980126] niks3-server[983]: 2026/09/20 10:51:45 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo4916lb # [ 53.983757] niks3-server[983]: 2026/09/20 10:51:45 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo4917lb # [ 53.986710] niks3-server[983]: 2026/09/20 10:51:45 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4918lb # [ 53.990539] niks3-server[983]: 2026/09/20 10:51:45 INFO Completed upload id=184919lb # [ 53.991849] niks3-server[983]: 2026/09/20 10:51:45 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4920lb # [ 53.993977] niks3-server[983]: 2026/09/20 10:51:45 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo4921lb # [ 53.996431] niks3-server[983]: 2026/09/20 10:51:45 INFO Completed upload id=194922worker1 # [ 53.999470] nix-grpc-daemon[895]: time=2026-09-20T10:51:45.085Z level=INFO msg="Upload complete. (463ms)"4923worker1 # [ 54.004469] nix-grpc-daemon[784]: ts=2026-09-20T10:51:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38906 duration_s=0 paths=2 nar_bytes_in=9764924worker1 # [ 54.011351] nix-grpc-daemon[784]: ts=2026-09-20T10:51:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:389064925worker1 # [ 54.020150] nix-daemon[843]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4926worker1 # [ 54.032227] nix-daemon[843]: reaped child process 989, status = succeeded4927worker1 # [ 54.036395] nix-daemon[843]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4928client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...4929client # uses-input-w1onlya> node-a: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv4930worker1 # [ 55.050899] nix-daemon[843]: reaped child process 1003, status = succeeded4931worker1 # [ 55.185098] nix-daemon[843]: accepted connection from pid 1018, user nix-grpc-daemon (trusted)4932worker1 # [ 55.198051] nix-daemon[843]: reaped child process 1021, status = succeeded4933lb # [ 55.211369] niks3-server[983]: 2026/09/20 10:51:46 INFO Received uploads request method=POST path=/api/pending_closures4934worker1 # [ 55.222771] nix-grpc-daemon[895]: time=2026-09-20T10:51:46.308Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4935lb # [ 55.240277] niks3-server[983]: 2026/09/20 10:51:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4936lb # [ 55.245292] niks3-server[983]: 2026/09/20 10:51:46 INFO Signed narinfos id=20 count=14937worker1 # [ 55.251135] nix-grpc-daemon[895]: time=2026-09-20T10:51:46.336Z level=INFO msg="Uploading 1 narinfos"4938lb # [ 55.251882] niks3-server[983]: 2026/09/20 10:51:46 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv4939lb # [ 55.262651] niks3-server[983]: 2026/09/20 10:51:46 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls4940lb # [ 55.269591] niks3-server[983]: 2026/09/20 10:51:46 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4941worker1 # [ 55.277256] nix-grpc-daemon[895]: time=2026-09-20T10:51:46.363Z level=INFO msg="Upload complete. (234ms)"4942lb # [ 55.274846] niks3-server[983]: 2026/09/20 10:51:46 INFO Completed upload id=204943lb # [ 55.280483] niks3-server[983]: 2026/09/20 10:51:46 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo4944worker1 # [ 55.282153] nix-grpc-daemon[784]: ts=2026-09-20T10:51:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38906 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=9 outputs=14945worker1 # [ 55.292630] nix-daemon[843]: reaped child process 1001, status = succeeded4946client # copying 1 paths...4947worker1 # [ 55.327302] nix-grpc-daemon[784]: ts=2026-09-20T10:51:46Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:389064948client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...4949worker1 # [ 55.345141] nix-grpc-daemon[784]: ts=2026-09-20T10:51:46Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:389064950worker1 # [ 55.349213] nix-grpc-daemon[784]: ts=2026-09-20T10:51:46Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:38906 duration_s=0 paths=1 nar_bytes_out=1204951client # [ 55.437328] nix-daemon[939]: reaped child process 1021, status = succeeded4952client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.77 seconds)4953client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&24954client # [ 55.614762] nix-daemon[939]: accepted connection from pid 1038, user root (trusted)4955client # this derivation will be built:4956client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv4957worker2 # [ 55.787474] nix-grpc-daemon[783]: ts=2026-09-20T10:51:46Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:418164958client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux'...4959worker1 # [ 55.813318] nix-grpc-daemon[784]: ts=2026-09-20T10:51:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:389064960worker1 # [ 55.822399] nix-grpc-daemon[895]: time=2026-09-20T10:51:46.908Z level=INFO msg="All 1 paths already cached"4961worker1 # [ 55.836685] nix-grpc-daemon[784]: ts=2026-09-20T10:51:46Z level=debug event=want client=10 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=04962worker1 # [ 55.841901] nix-grpc-daemon[784]: ts=2026-09-20T10:51:46Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=14963worker2 # [ 55.825796] nix-grpc-daemon[783]: ts=2026-09-20T10:51:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418164964worker2 # [ 55.838287] nix-grpc-daemon[783]: ts=2026-09-20T10:51:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418164965client # copying 1 paths to 'lb:50051'...4966client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...4967worker2 # [ 55.857103] nix-grpc-daemon[783]: ts=2026-09-20T10:51:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418164968worker2 # [ 55.869072] nix-daemon[839]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4969worker2 # [ 55.886338] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 24 ms (attempt 1/5)4970worker2 # [ 55.916916] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 42 ms (attempt 2/5)4971worker2 # [ 55.965357] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 222 ms (attempt 3/5)4972worker2 # [ 56.192169] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 579 ms (attempt 4/5)4973worker2 # [ 56.778070] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4974worker2 # [ 56.785717] nix-grpc-daemon[783]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...4975worker2 # [ 56.965693] nix-daemon[839]: accepted connection from pid 1146, user nix-grpc-daemon (trusted)4976worker2 # [ 56.980669] nix-daemon[839]: reaped child process 1149, status = succeeded4977worker2 # [ 57.143421] nix-daemon[839]: accepted connection from pid 1151, user nix-grpc-daemon (trusted)4978worker2 # [ 57.156491] nix-daemon[839]: reaped child process 1154, status = succeeded4979lb # [ 57.188274] niks3-server[983]: 2026/09/20 10:51:48 INFO Received uploads request method=POST path=/api/pending_closures4980worker2 # [ 57.180406] nix-grpc-daemon[849]: time=2026-09-20T10:51:47.412Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4981worker2 # [ 57.181907] nix-grpc-daemon[849]: time=2026-09-20T10:51:47.413Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"4982lb # [ 57.208252] niks3-server[983]: 2026/09/20 10:51:48 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst4983lb # [ 57.211128] niks3-server[983]: 2026/09/20 10:51:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4984lb # [ 57.212874] niks3-server[983]: 2026/09/20 10:51:48 INFO Signed narinfos id=21 count=14985worker2 # [ 57.197314] nix-grpc-daemon[849]: time=2026-09-20T10:51:47.429Z level=INFO msg="Uploading 1 narinfos"4986lb # [ 57.216768] niks3-server[983]: 2026/09/20 10:51:48 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls4987lb # [ 57.221204] niks3-server[983]: 2026/09/20 10:51:48 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4988lb # [ 57.223852] niks3-server[983]: 2026/09/20 10:51:48 INFO Completed upload id=214989worker2 # [ 57.208240] nix-grpc-daemon[849]: time=2026-09-20T10:51:47.440Z level=INFO msg="Upload complete. (400ms)"4990lb # [ 57.226570] niks3-server[983]: 2026/09/20 10:51:48 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo4991worker2 # [ 57.210551] nix-grpc-daemon[783]: ts=2026-09-20T10:51:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41816 duration_s=1 paths=1 nar_bytes_in=5604992worker2 # [ 57.214172] nix-grpc-daemon[783]: ts=2026-09-20T10:51:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418164993worker2 # [ 57.219445] nix-daemon[839]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4994worker2 # [ 57.223835] nix-daemon[839]: reaped child process 1140, status = succeeded4995worker2 # [ 57.232934] nix-grpc-daemon[783]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...4996client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...4997worker2 # [ 57.241278] nix-daemon[839]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4998client # uses-input-w1onlyb> worker2: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv4999worker2 # [ 57.864563] nix-daemon[839]: reaped child process 1159, status = succeeded5000worker2 # [ 58.008327] nix-daemon[839]: accepted connection from pid 1174, user nix-grpc-daemon (trusted)5001worker2 # [ 58.020344] nix-daemon[839]: reaped child process 1177, status = succeeded5002lb # [ 58.046819] niks3-server[983]: 2026/09/20 10:51:49 INFO Received uploads request method=POST path=/api/pending_closures5003worker2 # [ 58.039155] nix-grpc-daemon[849]: time=2026-09-20T10:51:48.271Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5004lb # [ 58.073065] niks3-server[983]: 2026/09/20 10:51:49 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5005lb # [ 58.082128] niks3-server[983]: 2026/09/20 10:51:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5006lb # [ 58.088662] niks3-server[983]: 2026/09/20 10:51:49 INFO Signed narinfos id=22 count=15007worker2 # [ 58.075921] nix-grpc-daemon[849]: time=2026-09-20T10:51:48.307Z level=INFO msg="Uploading 1 narinfos"5008lb # [ 58.095364] niks3-server[983]: 2026/09/20 10:51:49 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5009lb # [ 58.107651] niks3-server[983]: 2026/09/20 10:51:49 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5010worker2 # [ 58.097837] nix-grpc-daemon[849]: time=2026-09-20T10:51:48.329Z level=INFO msg="Upload complete. (249ms)"5011lb # [ 58.113391] niks3-server[983]: 2026/09/20 10:51:49 INFO Completed upload id=225012lb # [ 58.115624] niks3-server[983]: 2026/09/20 10:51:49 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5013worker2 # [ 58.100942] nix-grpc-daemon[783]: ts=2026-09-20T10:51:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41816 duration_s=0 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=10 outputs=15014worker2 # [ 58.113357] nix-daemon[839]: reaped child process 1156, status = succeeded5015client # copying 1 paths...5016worker1 # [ 58.169885] nix-grpc-daemon[784]: ts=2026-09-20T10:51:49Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:389065017worker1 # [ 58.181721] nix-grpc-daemon[784]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5018client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5019worker2 # [ 58.195180] nix-grpc-daemon[783]: ts=2026-09-20T10:51:48Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:418165020worker2 # [ 58.200532] nix-grpc-daemon[783]: ts=2026-09-20T10:51:48Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41816 duration_s=0 paths=1 nar_bytes_out=1205021client # [ 58.266689] nix-daemon[939]: reaped child process 1041, status = succeeded5022client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.82 seconds)5023client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&25024client # [ 58.414922] nix-daemon[939]: accepted connection from pid 1058, user root (trusted)5025client # this derivation will be built:5026client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5027lb # [ 58.541461] envoy[919]: conn peer=[2001:db8:1::1]:38100 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3837 tx=1972 ms=18175028lb # [ 58.545240] envoy[919]: conn peer=[2001:db8:1::1]:38114 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=378 ms=175029lb # [ 58.548632] envoy[919]: conn peer=[2001:db8:1::1]:38118 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4341 tx=2222 ms=20775030lb # [ 58.552089] envoy[919]: conn peer=[2001:db8:1::1]:38124 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=545031lb # [ 58.555447] envoy[919]: conn peer=[2001:db8:1::1]:45964 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4220 tx=1766 ms=23385032lb # [ 58.558875] envoy[919]: conn peer=[2001:db8:1::1]:45972 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=535033lb # [ 58.562252] envoy[919]: conn peer=[2001:db8:1::1]:45986 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3711 tx=1919 ms=24245034lb # [ 58.565688] envoy[919]: conn peer=[2001:db8:1::1]:46000 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=175035worker2 # [ 58.577811] nix-grpc-daemon[783]: ts=2026-09-20T10:51:48Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:418165036client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux'...5037worker2 # [ 58.583804] nix-grpc-daemon[783]: ts=2026-09-20T10:51:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418165038worker2 # [ 58.593309] nix-grpc-daemon[849]: time=2026-09-20T10:51:48.824Z level=INFO msg="All 1 paths already cached"5039worker1 # [ 58.624410] nix-grpc-daemon[784]: ts=2026-09-20T10:51:49Z level=debug event=want client=11 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05040worker1 # [ 58.626712] nix-grpc-daemon[784]: ts=2026-09-20T10:51:49Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=15041worker1 # [ 58.630644] nix-grpc-daemon[784]: ts=2026-09-20T10:51:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:389065042worker1 # [ 58.638157] nix-grpc-daemon[784]: ts=2026-09-20T10:51:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:389065043worker1 # [ 58.647165] nix-grpc-daemon[895]: time=2026-09-20T10:51:49.732Z level=INFO msg="All 1 paths already cached"5044client # copying 1 paths to 'lb:50051'...5045client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5046worker1 # [ 58.651443] nix-grpc-daemon[784]: ts=2026-09-20T10:51:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:389065047worker1 # [ 58.655601] nix-daemon[843]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5048worker1 # [ 58.821893] nix-daemon[843]: accepted connection from pid 1030, user nix-grpc-daemon (trusted)5049worker1 # [ 58.835291] nix-daemon[843]: reaped child process 1033, status = succeeded5050worker1 # [ 59.001712] nix-daemon[843]: accepted connection from pid 1035, user nix-grpc-daemon (trusted)5051worker1 # [ 59.015414] nix-daemon[843]: reaped child process 1038, status = succeeded5052lb # [ 59.029490] niks3-server[983]: 2026/09/20 10:51:50 INFO Received uploads request method=POST path=/api/pending_closures5053worker1 # [ 59.040671] nix-grpc-daemon[895]: time=2026-09-20T10:51:50.126Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5054worker1 # [ 59.042193] nix-grpc-daemon[895]: time=2026-09-20T10:51:50.126Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5055lb # [ 59.055952] niks3-server[983]: 2026/09/20 10:51:50 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5056lb # [ 59.064775] niks3-server[983]: 2026/09/20 10:51:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5057lb # [ 59.069241] niks3-server[983]: 2026/09/20 10:51:50 INFO Signed narinfos id=23 count=15058worker1 # [ 59.075086] nix-grpc-daemon[895]: time=2026-09-20T10:51:50.160Z level=INFO msg="Uploading 1 narinfos"5059lb # [ 59.077762] niks3-server[983]: 2026/09/20 10:51:50 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5060lb # [ 59.088836] niks3-server[983]: 2026/09/20 10:51:50 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5061lb # [ 59.093386] niks3-server[983]: 2026/09/20 10:51:50 INFO Completed upload id=235062worker1 # [ 59.098353] nix-grpc-daemon[895]: time=2026-09-20T10:51:50.184Z level=INFO msg="Upload complete. (436ms)"5063lb # [ 59.098514] niks3-server[983]: 2026/09/20 10:51:50 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5064worker1 # [ 59.100609] nix-grpc-daemon[784]: ts=2026-09-20T10:51:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38906 duration_s=0 paths=1 nar_bytes_in=5605065worker1 # [ 59.105688] nix-grpc-daemon[784]: ts=2026-09-20T10:51:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:389065066worker1 # [ 59.114691] nix-daemon[843]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5067worker1 # [ 59.120691] nix-daemon[843]: reaped child process 1028, status = succeeded5068worker1 # [ 59.123874] nix-daemon[843]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5069client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5070client # uses-input-w1onlyc> node-a: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5071worker1 # [ 59.948433] nix-daemon[843]: reaped child process 1042, status = succeeded5072worker1 # [ 60.109285] nix-daemon[843]: accepted connection from pid 1057, user nix-grpc-daemon (trusted)5073worker1 # [ 60.121956] nix-daemon[843]: reaped child process 1060, status = succeeded5074lb # [ 60.137175] niks3-server[983]: 2026/09/20 10:51:51 INFO Received uploads request method=POST path=/api/pending_closures5075worker1 # [ 60.147604] nix-grpc-daemon[895]: time=2026-09-20T10:51:51.233Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5076lb # [ 60.165235] niks3-server[983]: 2026/09/20 10:51:51 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5077lb # [ 60.174211] niks3-server[983]: 2026/09/20 10:51:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5078worker1 # [ 60.182872] nix-grpc-daemon[895]: time=2026-09-20T10:51:51.268Z level=INFO msg="Uploading 1 narinfos"5079lb # [ 60.180162] niks3-server[983]: 2026/09/20 10:51:51 INFO Signed narinfos id=24 count=15080lb # [ 60.184366] niks3-server[983]: 2026/09/20 10:51:51 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5081lb # [ 60.197327] niks3-server[983]: 2026/09/20 10:51:51 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5082lb # [ 60.203152] niks3-server[983]: 2026/09/20 10:51:51 INFO Completed upload id=245083worker1 # [ 60.206146] nix-grpc-daemon[895]: time=2026-09-20T10:51:51.291Z level=INFO msg="Upload complete. (274ms)"5084lb # [ 60.205421] niks3-server[983]: 2026/09/20 10:51:51 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5085worker1 # [ 60.210814] nix-grpc-daemon[784]: ts=2026-09-20T10:51:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38906 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=11 outputs=15086worker1 # [ 60.223274] nix-daemon[843]: reaped child process 1040, status = succeeded5087client # copying 1 paths...5088worker1 # [ 60.257838] nix-grpc-daemon[784]: ts=2026-09-20T10:51:51Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:389065089client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5090worker2 # [ 60.257657] nix-grpc-daemon[783]: ts=2026-09-20T10:51:50Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:418165091worker2 # [ 60.269516] nix-grpc-daemon[783]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5092worker2 # [ 60.283691] nix-grpc-daemon[783]: ts=2026-09-20T10:51:50Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41816 duration_s=0 paths=1 nar_bytes_out=1205093client # [ 60.390463] nix-daemon[939]: reaped child process 1061, status = succeeded5094client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.13 seconds)5095(finished: subtest: an input only one worker has still reaches the builder, in 8.51 seconds)5096subtest: upload whose reference the worker lacks is completed from the cache5097client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache5098client # this derivation will be built:5099client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5100client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5101client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache, in 0.38 seconds)5102client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache5103client # this derivation will be built:5104client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5105client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5106client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache, in 0.38 seconds)5107client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5108client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.18 seconds)5109worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5110worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.20 seconds)5111worker2 # [ 61.610179] systemd[1]: Stopping Nix worker-protocol over gRPC...5112worker2 # [ 61.801200] nix-grpc-daemon[783]: ts=2026-09-20T10:51:52Z level=info event=signal_exit5113worker2 # [ 63.054539] nix-grpc-daemon[783]: ts=2026-09-20T10:51:53Z level=info event=scheduler_restarting5114worker1 # [ 63.081893] nix-grpc-daemon[784]: ts=2026-09-20T10:51:54Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:388905115worker1 # [ 63.085537] nix-grpc-daemon[784]: ts=2026-09-20T10:51:54Z level=info event=worker_gone addr=192.168.1.4:500515116worker2 # [ 63.075703] nix-grpc-daemon[783]: ts=2026-09-20T10:51:53Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5117worker2 # [ 63.111620] nix-daemon[839]: reaped child process 841, status = succeeded5118worker2 # [ 63.898541] nix-grpc-daemon[783]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5119worker2 # [ 63.902440] nix-grpc-daemon[783]: E0000 00:00:1789901514.129952 826 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true5120worker2 # [ 63.907866] nix-grpc-daemon[783]: E0000 00:00:1789901514.139556 826 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5121worker2 # [ 64.987392] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5122worker2 # [ 64.990542] systemd[1]: Stopped Nix worker-protocol over gRPC.5123worker2 # [ 64.994671] systemd[1]: nix-grpc-daemon.service: Consumed 2.507s CPU time over 46.461s wall clock time, 57.4M memory peak, 78.8K incoming IP traffic, 143.1K outgoing IP traffic.5124worker2 # [ 65.007552] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5125worker2 # [ 65.011147] systemd[1]: Closed Nix worker-protocol over gRPC.5126worker2 # [ 65.014162] systemd[1]: nix-grpc-daemon.socket: Consumed 307us CPU time over 54.809s wall clock time, 256K memory peak, 68.6K incoming IP traffic, 55.2K outgoing IP traffic.5127lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5128lb: must succeed: curl -sf localhost:9901/clusters5129lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5130lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.07 seconds)5131client: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25132worker1 # [ 65.312777] nix-grpc-daemon[784]: ts=2026-09-20T10:51:56Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:389065133client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5134worker1 # [ 65.319349] nix-grpc-daemon[784]: ts=2026-09-20T10:51:56Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:389065135client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.23 seconds)5136worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5137worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.03 seconds)5138worker2 # [ 65.406910] systemd[1]: Listening on Nix worker-protocol over gRPC.5139worker1 # [ 65.469562] systemd[1]: Stopping Nix worker-protocol over gRPC...5140worker1 # [ 65.845931] nix-grpc-daemon[784]: ts=2026-09-20T10:51:56Z level=info event=signal_exit5141worker2 # [ 65.973596] systemd[1]: Starting Nix worker-protocol over gRPC...5142worker1 # [ 66.093977] nix-grpc-daemon[784]: ts=2026-09-20T10:51:57Z level=info event=scheduler_yield5143lb # [ 66.094909] niks3-server[983]: 2026/09/20 10:51:57 INFO lead: released remote=192.168.1.3:361665144worker1 # [ 66.099990] nix-grpc-daemon[784]: ts=2026-09-20T10:51:57Z level=info event=scheduler_restarting addr=lb:500515145worker1 # [ 66.103091] nix-grpc-daemon[784]: ts=2026-09-20T10:51:57Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:388905146worker1 # [ 66.106404] nix-grpc-daemon[784]: ts=2026-09-20T10:51:57Z level=info event=worker_gone addr=192.168.1.3:500515147worker2 # [ 66.167067] nix-grpc-daemon[1199]: ts=2026-09-20T10:51:56Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5148worker2 # [ 66.170341] nix-grpc-daemon[1199]: ts=2026-09-20T10:51:56Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15149worker2 # [ 66.177054] nix-grpc-daemon[1199]: ts=2026-09-20T10:51:56Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5150worker2 # [ 66.181317] systemd[1]: Started Nix worker-protocol over gRPC.5151worker2 # [ 66.191055] nix-grpc-daemon[1199]: ts=2026-09-20T10:51:56Z level=info event=scheduler_take_over5152lb # [ 66.206654] niks3-server[983]: 2026/09/20 10:51:57 INFO lead: acquired remote=192.168.1.4:539625153worker2 # [ 66.209440] nix-grpc-daemon[1199]: ts=2026-09-20T10:51:56Z level=info event=scheduler_connected addr=lb:500515154worker2 # [ 66.211185] nix-grpc-daemon[1199]: ts=2026-09-20T10:51:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5155worker1 # [ 66.299083] nix-grpc-daemon[784]: ts=2026-09-20T10:51:57Z level=info event=scheduler_connected addr=lb:500515156worker1 # [ 66.301174] nix-grpc-daemon[784]: ts=2026-09-20T10:51:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5157worker2 # [ 66.412109] nix-grpc-daemon[1199]: ts=2026-09-20T10:51:56Z level=info event=scheduler_connected addr=lb:500515158worker2 # [ 66.416292] nix-grpc-daemon[1199]: ts=2026-09-20T10:51:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5159worker1 # [ 66.502150] nix-grpc-daemon[784]: ts=2026-09-20T10:51:57Z level=info event=scheduler_connected addr=lb:500515160worker1 # [ 66.504202] nix-grpc-daemon[784]: ts=2026-09-20T10:51:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5161worker2 # [ 66.617131] nix-grpc-daemon[1199]: ts=2026-09-20T10:51:56Z level=info event=scheduler_connected addr=lb:500515162worker2 # [ 66.621162] nix-grpc-daemon[1199]: ts=2026-09-20T10:51:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5163worker1 # [ 66.705255] nix-grpc-daemon[784]: ts=2026-09-20T10:51:57Z level=info event=scheduler_connected addr=lb:500515164worker1 # [ 66.709758] nix-grpc-daemon[784]: ts=2026-09-20T10:51:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5165worker2 # [ 66.822180] nix-grpc-daemon[1199]: ts=2026-09-20T10:51:57Z level=info event=scheduler_connected addr=lb:500515166worker2 # [ 66.826313] nix-grpc-daemon[1199]: ts=2026-09-20T10:51:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5167worker1 # [ 66.910664] nix-grpc-daemon[784]: ts=2026-09-20T10:51:57Z level=info event=scheduler_connected addr=lb:500515168worker1 # [ 66.915201] nix-grpc-daemon[784]: ts=2026-09-20T10:51:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5169worker2 # [ 67.027147] nix-grpc-daemon[1199]: ts=2026-09-20T10:51:57Z level=info event=scheduler_connected addr=lb:500515170worker2 # [ 67.031248] nix-grpc-daemon[1199]: ts=2026-09-20T10:51:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5171worker1 # [ 67.098210] nix-grpc-daemon[784]: ts=2026-09-20T10:51:58Z level=info event=scheduler_restarting5172worker1 # [ 67.129592] nix-grpc-daemon[784]: ts=2026-09-20T10:51:58Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5173worker1 # [ 67.140164] nix-daemon[843]: reaped child process 845, status = succeeded5174worker2 # [ 67.232461] nix-grpc-daemon[1199]: ts=2026-09-20T10:51:57Z level=info event=scheduler_connected addr=lb:500515175worker2 # [ 67.236192] nix-grpc-daemon[1199]: ts=2026-09-20T10:51:57Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05176worker1 # [ 67.970464] nix-grpc-daemon[784]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5177worker1 # [ 67.974142] nix-grpc-daemon[784]: E0000 00:00:1789901519.054669 827 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5178worker1 # [ 67.978921] nix-grpc-daemon[784]: E0000 00:00:1789901519.055320 827 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5179lb # [ 68.541294] envoy[919]: conn peer=[2001:db8:1::1]:46002 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3713 tx=1801 ms=17595180lb # [ 68.545096] envoy[919]: conn peer=[2001:db8:1::1]:46014 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=329 ms=805181lb # [ 68.548484] envoy[919]: conn peer=[2001:db8:1::4]:59642 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=6918 tx=1691 ms=442225182lb # [ 68.552092] envoy[919]: conn peer=[2001:db8:1::1]:35640 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=165183lb # [ 68.555595] envoy[919]: conn peer=[2001:db8:1::3]:34084 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=6769 tx=1461 ms=482325184worker1 # [ 68.921775] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5185worker1 # [ 68.927351] systemd[1]: Stopped Nix worker-protocol over gRPC.5186worker1 # [ 68.929406] systemd[1]: nix-grpc-daemon.service: Consumed 1.467s CPU time over 50.380s wall clock time, 52.7M memory peak, 53.8K incoming IP traffic, 98.4K outgoing IP traffic.5187worker1 # [ 68.940283] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5188worker1 # [ 68.944539] systemd[1]: Closed Nix worker-protocol over gRPC.5189worker1 # [ 68.948427] systemd[1]: nix-grpc-daemon.socket: Consumed 325us CPU time over 59.596s wall clock time, 256K memory peak, 72.5K incoming IP traffic, 59.7K outgoing IP traffic.5190lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5191lb: must succeed: curl -sf localhost:9901/clusters5192lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5193lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.05 seconds)5194client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25195worker2 # [ 69.164663] nix-grpc-daemon[1199]: ts=2026-09-20T10:51:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:339445196worker2 # [ 69.173553] nix-daemon[839]: accepted connection from pid 1199, user nix-grpc-daemon (trusted)5197worker2 # [ 69.191304] nix-grpc-daemon[1199]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 62 ms (attempt 1/5)5198worker2 # [ 69.253445] nix-grpc-daemon[1199]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 96 ms (attempt 2/5)5199worker2 # [ 69.353510] nix-grpc-daemon[1199]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 300 ms (attempt 3/5)5200worker2 # [ 69.659680] nix-grpc-daemon[1199]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 517 ms (attempt 4/5)5201worker2 # [ 70.182388] nix-grpc-daemon[1199]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5202client # copying 2 paths...5203client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5204worker2 # [ 70.204134] nix-grpc-daemon[1199]: ts=2026-09-20T10:52:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:339445205client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5206worker2 # [ 70.216385] nix-daemon[839]: accepted connection from pid 1199, user nix-grpc-daemon (trusted)5207worker2 # [ 70.236316] nix-grpc-daemon[1199]: ts=2026-09-20T10:52:00Z level=info event=niks3_push_started pid=12555208worker2 # [ 70.601517] nix-daemon[839]: accepted connection from pid 1268, user nix-grpc-daemon (trusted)5209worker2 # [ 70.614927] nix-daemon[839]: reaped child process 1271, status = succeeded5210lb # [ 70.648692] niks3-server[983]: 2026/09/20 10:52:01 INFO Received uploads request method=POST path=/api/pending_closures5211lb # [ 70.659443] niks3-server[983]: 2026/09/20 10:52:01 INFO Received uploads request method=POST path=/api/pending_closures5212worker2 # [ 70.651543] nix-grpc-daemon[1255]: time=2026-09-20T10:52:00.883Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5213worker2 # [ 70.654147] nix-grpc-daemon[1255]: time=2026-09-20T10:52:00.885Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5214worker2 # [ 70.655746] nix-grpc-daemon[1255]: time=2026-09-20T10:52:00.885Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5215lb # [ 70.713959] niks3-server[983]: 2026/09/20 10:52:01 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5216lb # [ 70.723705] niks3-server[983]: 2026/09/20 10:52:01 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5217lb # [ 70.729078] niks3-server[983]: 2026/09/20 10:52:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5218lb # [ 70.732832] niks3-server[983]: 2026/09/20 10:52:01 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5219lb # [ 70.737335] niks3-server[983]: 2026/09/20 10:52:01 INFO Signed narinfos id=26 count=25220lb # [ 70.739235] niks3-server[983]: 2026/09/20 10:52:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5221lb # [ 70.741436] niks3-server[983]: 2026/09/20 10:52:01 INFO Signed narinfos id=25 count=15222worker2 # [ 70.727148] nix-grpc-daemon[1255]: time=2026-09-20T10:52:00.957Z level=INFO msg="Uploading 3 narinfos"5223lb # [ 70.744301] niks3-server[983]: 2026/09/20 10:52:01 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5224lb # [ 70.753871] niks3-server[983]: 2026/09/20 10:52:01 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5225lb # [ 70.758823] niks3-server[983]: 2026/09/20 10:52:01 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5226lb # [ 70.762028] niks3-server[983]: 2026/09/20 10:52:01 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5227lb # [ 70.764797] niks3-server[983]: 2026/09/20 10:52:01 INFO Completed upload id=255228lb # [ 70.766062] niks3-server[983]: 2026/09/20 10:52:01 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5229lb # [ 70.769272] niks3-server[983]: 2026/09/20 10:52:01 INFO Completed upload id=265230worker2 # [ 70.753994] nix-grpc-daemon[1255]: time=2026-09-20T10:52:00.985Z level=INFO msg="Upload complete. (312ms)"5231worker2 # [ 70.760157] nix-grpc-daemon[1255]: time=2026-09-20T10:52:00.989Z level=WARN msg="Failed to register uploaded object" key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo error="server returned 404: object is not pending upload\n"5232worker2 # [ 70.765975] nix-grpc-daemon[1199]: ts=2026-09-20T10:52:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33944 duration_s=0 paths=2 nar_bytes_in=3125233worker2 # [ 70.777090] nix-daemon[839]: reaped child process 1253, status = succeeded5234client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.80 seconds)5235worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5236worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds)5237worker1 # [ 70.871507] systemd[1]: Listening on Nix worker-protocol over gRPC.5238worker1 # [ 70.875209] systemd[1]: Starting Nix worker-protocol over gRPC...5239worker1 # [ 71.019458] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:02Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5240worker1 # [ 71.023240] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:02Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15241worker1 # [ 71.031054] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:02Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5242worker1 # [ 71.035168] systemd[1]: Started Nix worker-protocol over gRPC.5243lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5244lb: must succeed: curl -sf localhost:9901/clusters5245worker1 # [ 71.069041] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:02Z level=info event=scheduler_connected addr=lb:500515246worker2 # [ 71.053721] nix-grpc-daemon[1199]: ts=2026-09-20T10:52:01Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05247lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5248lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.07 seconds)5249(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.71 seconds)5250subtest: scheduler down: the other node takes the lock and keeps it5251lb: must succeed: curl -sf localhost:9901/clusters5252lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5253lb: must succeed: curl -sf localhost:9901/clusters5254lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5255lb: must succeed: curl -sf localhost:9901/clusters5256lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5257lb: must succeed: curl -sf localhost:9901/clusters5258lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5259worker2 # [ 71.346636] systemd[1]: Stopping Nix worker-protocol over gRPC...5260worker2 # [ 72.184160] nix-grpc-daemon[1199]: ts=2026-09-20T10:52:02Z level=info event=signal_exit5261worker2 # [ 72.394457] nix-grpc-daemon[1199]: ts=2026-09-20T10:52:02Z level=info event=scheduler_yield5262lb # [ 72.415204] niks3-server[983]: 2026/09/20 10:52:03 INFO lead: released remote=192.168.1.4:539625263worker2 # [ 72.399379] nix-grpc-daemon[1199]: ts=2026-09-20T10:52:02Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:339365264worker1 # [ 72.417905] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:03Z level=info event=scheduler_restarting addr=lb:500515265worker2 # [ 72.404201] nix-grpc-daemon[1199]: ts=2026-09-20T10:52:02Z level=info event=worker_gone addr=192.168.1.3:500515266worker2 # [ 72.408979] nix-grpc-daemon[1199]: ts=2026-09-20T10:52:02Z level=info event=scheduler_restarting addr=lb:500515267worker2 # [ 72.412166] nix-grpc-daemon[1199]: ts=2026-09-20T10:52:02Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:339365268worker2 # [ 72.415509] nix-grpc-daemon[1199]: ts=2026-09-20T10:52:02Z level=info event=worker_gone addr=192.168.1.4:500515269worker1 # [ 72.617204] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:03Z level=info event=scheduler_connected addr=lb:500515270worker1 # [ 72.619352] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5271worker2 # [ 72.605341] nix-grpc-daemon[1199]: ts=2026-09-20T10:52:02Z level=info event=scheduler_connected addr=lb:500515272worker2 # [ 72.609342] nix-grpc-daemon[1199]: ts=2026-09-20T10:52:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5273worker1 # [ 72.820602] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:03Z level=info event=scheduler_connected addr=lb:500515274worker1 # [ 72.824641] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5275worker2 # [ 72.809692] nix-grpc-daemon[1199]: ts=2026-09-20T10:52:03Z level=info event=scheduler_connected addr=lb:500515276worker2 # [ 72.811913] nix-grpc-daemon[1199]: ts=2026-09-20T10:52:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5277worker1 # [ 73.025547] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:04Z level=info event=scheduler_connected addr=lb:500515278worker2 # [ 73.012486] nix-grpc-daemon[1199]: ts=2026-09-20T10:52:03Z level=info event=scheduler_connected addr=lb:500515279worker1 # [ 73.029718] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5280worker2 # [ 73.014344] nix-grpc-daemon[1199]: ts=2026-09-20T10:52:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5281worker1 # [ 73.230626] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:04Z level=info event=scheduler_connected addr=lb:500515282worker2 # [ 73.216347] nix-grpc-daemon[1199]: ts=2026-09-20T10:52:03Z level=info event=scheduler_connected addr=lb:500515283worker1 # [ 73.234755] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5284worker2 # [ 73.219511] nix-grpc-daemon[1199]: ts=2026-09-20T10:52:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5285worker2 # [ 73.398630] nix-grpc-daemon[1199]: ts=2026-09-20T10:52:03Z level=info event=scheduler_restarting5286worker2 # [ 73.411940] nix-daemon[839]: reaped child process 1248, status = succeeded5287worker2 # [ 73.420818] nix-grpc-daemon[1199]: ts=2026-09-20T10:52:03Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5288worker1 # [ 73.437572] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:04Z level=info event=scheduler_connected addr=lb:500515289worker1 # [ 73.441569] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5290worker1 # [ 73.638059] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:04Z level=info event=scheduler_connected addr=lb:500515291worker1 # [ 73.642311] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5292worker1 # [ 73.843173] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:04Z level=info event=scheduler_connected addr=lb:500515293worker1 # [ 73.847325] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5294worker1 # [ 74.047449] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:05Z level=info event=scheduler_connected addr=lb:500515295worker1 # [ 74.049273] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5296worker2 # [ 74.205659] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5297worker2 # [ 74.207323] systemd[1]: Stopped Nix worker-protocol over gRPC.5298worker2 # [ 74.208106] systemd[1]: nix-grpc-daemon.service: Consumed 375ms CPU time over 8.231s wall clock time, 32.6M memory peak, 18.2K incoming IP traffic, 24.2K outgoing IP traffic.5299worker2 # [ 74.212559] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5300worker2 # [ 74.214289] systemd[1]: Closed Nix worker-protocol over gRPC.5301worker2 # [ 74.215548] systemd[1]: nix-grpc-daemon.socket: Consumed 307us CPU time over 8.811s wall clock time, 256K memory peak, 19.5K incoming IP traffic, 14.9K outgoing IP traffic.5302worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5303worker1 # [ 74.250179] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:05Z level=info event=scheduler_connected addr=lb:500515304worker1 # [ 74.254693] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5305worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.08 seconds)5306worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5307worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5308worker1 # [ 74.455779] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:05Z level=info event=scheduler_connected addr=lb:500515309worker1 # [ 74.459678] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5310worker1 # [ 74.660540] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:05Z level=info event=scheduler_connected addr=lb:500515311worker1 # [ 74.664573] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5312worker1 # [ 74.865630] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:05Z level=info event=scheduler_connected addr=lb:500515313worker1 # [ 74.869594] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5314worker1 # [ 75.070197] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:06Z level=info event=scheduler_connected addr=lb:500515315worker1 # [ 75.074090] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5316worker1 # [ 75.274941] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:06Z level=info event=scheduler_connected addr=lb:500515317worker1 # [ 75.279157] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5318worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5319worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5320worker1 # [ 75.480848] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:06Z level=info event=scheduler_connected addr=lb:500515321worker1 # [ 75.484622] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5322worker1 # [ 75.685480] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:06Z level=info event=scheduler_connected addr=lb:500515323worker1 # [ 75.689441] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5324worker1 # [ 75.890400] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:06Z level=info event=scheduler_connected addr=lb:500515325worker1 # [ 75.894447] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5326worker1 # [ 76.047740] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:07Z level=info event=scheduler_take_over5327lb # [ 76.045236] niks3-server[983]: 2026/09/20 10:52:07 INFO lead: acquired remote=192.168.1.3:446685328worker1 # [ 76.095182] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:07Z level=info event=scheduler_connected addr=lb:500515329worker1 # [ 76.101867] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:07Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05330worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5331worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5332client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&25333worker1 # [ 76.704881] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:376305334worker1 # [ 76.716692] nix-daemon[843]: accepted connection from pid 1083, user nix-grpc-daemon (trusted)5335client # don't know how to build these paths:5336client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5337worker1 # [ 76.731200] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376305338worker1 # [ 76.741997] nix-grpc-daemon[1083]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 30 ms (attempt 1/5)5339worker1 # [ 76.772555] nix-grpc-daemon[1083]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 92 ms (attempt 2/5)5340worker1 # [ 76.870601] nix-grpc-daemon[1083]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 230 ms (attempt 3/5)5341worker1 # [ 77.101227] nix-grpc-daemon[1083]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 28 ms (attempt 4/5)5342worker1 # [ 77.135265] nix-grpc-daemon[1083]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5343worker1 # [ 77.158432] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:08Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05344worker1 # [ 77.163561] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:08Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=15345worker1 # [ 77.202463] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376305346worker1 # [ 77.213588] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376305347client # copying 1 paths to 'lb:50051'...5348client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5349worker1 # [ 77.224321] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376305350worker1 # [ 77.232729] nix-daemon[843]: accepted connection from pid 1083, user nix-grpc-daemon (trusted)5351worker1 # [ 77.254417] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:08Z level=info event=niks3_push_started pid=11565352worker1 # [ 77.597156] nix-daemon[843]: accepted connection from pid 1169, user nix-grpc-daemon (trusted)5353worker1 # [ 77.610238] nix-daemon[843]: reaped child process 1172, status = succeeded5354worker1 # [ 77.772820] nix-daemon[843]: accepted connection from pid 1174, user nix-grpc-daemon (trusted)5355worker1 # [ 77.785344] nix-daemon[843]: reaped child process 1177, status = succeeded5356lb # [ 77.800363] niks3-server[983]: 2026/09/20 10:52:08 INFO Received uploads request method=POST path=/api/pending_closures5357worker1 # [ 77.810061] nix-grpc-daemon[1156]: time=2026-09-20T10:52:08.895Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5358worker1 # [ 77.813365] nix-grpc-daemon[1156]: time=2026-09-20T10:52:08.897Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5359lb # [ 77.831658] niks3-server[983]: 2026/09/20 10:52:08 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5360lb # [ 77.843836] niks3-server[983]: 2026/09/20 10:52:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5361lb # [ 77.849125] niks3-server[983]: 2026/09/20 10:52:08 INFO Signed narinfos id=27 count=15362worker1 # [ 77.855261] nix-grpc-daemon[1156]: time=2026-09-20T10:52:08.940Z level=INFO msg="Uploading 1 narinfos"5363lb # [ 77.863645] niks3-server[983]: 2026/09/20 10:52:08 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5364lb # [ 77.868280] niks3-server[983]: 2026/09/20 10:52:08 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5365lb # [ 77.875640] niks3-server[983]: 2026/09/20 10:52:08 INFO Completed upload id=275366worker1 # [ 77.878648] nix-grpc-daemon[1156]: time=2026-09-20T10:52:08.964Z level=INFO msg="Upload complete. (437ms)"5367lb # [ 77.878358] niks3-server[983]: 2026/09/20 10:52:08 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5368worker1 # [ 77.882956] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37630 duration_s=0 paths=1 nar_bytes_in=5205369worker1 # [ 77.889455] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376305370worker1 # [ 77.898808] nix-daemon[843]: accepted connection from pid 1083, user nix-grpc-daemon (trusted)5371worker1 # [ 77.902524] nix-daemon[843]: reaped child process 1154, status = succeeded5372worker1 # [ 77.907420] nix-daemon[843]: accepted connection from pid 1083, user nix-grpc-daemon (trusted)5373client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5374client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5375lb # [ 78.540724] envoy[919]: conn peer=[2001:db8:1::1]:35648 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1006 tx=282 ms=16085376lb # [ 78.543118] envoy[919]: conn peer=[2001:db8:1::4]:51414 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2293 tx=512 ms=72125377client # farm-a> LOG-farm-a-on-standby5378worker1 # [ 79.135095] nix-daemon[843]: reaped child process 1181, status = succeeded5379worker1 # [ 79.300786] nix-daemon[843]: accepted connection from pid 1196, user nix-grpc-daemon (trusted)5380worker1 # [ 79.313309] nix-daemon[843]: reaped child process 1199, status = succeeded5381lb # [ 79.325601] niks3-server[983]: 2026/09/20 10:52:10 INFO Received uploads request method=POST path=/api/pending_closures5382worker1 # [ 79.336991] nix-grpc-daemon[1156]: time=2026-09-20T10:52:10.422Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5383worker1 # [ 79.340058] nix-grpc-daemon[1156]: time=2026-09-20T10:52:10.424Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5384lb # [ 79.359153] niks3-server[983]: 2026/09/20 10:52:10 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5385lb # [ 79.370773] niks3-server[983]: 2026/09/20 10:52:10 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5386lb # [ 79.382774] niks3-server[983]: 2026/09/20 10:52:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5387lb # [ 79.388228] niks3-server[983]: 2026/09/20 10:52:10 INFO Signed narinfos id=28 count=15388worker1 # [ 79.394487] nix-grpc-daemon[1156]: time=2026-09-20T10:52:10.479Z level=INFO msg="Uploading 1 narinfos"5389lb # [ 79.392313] niks3-server[983]: 2026/09/20 10:52:10 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5390lb # [ 79.401955] niks3-server[983]: 2026/09/20 10:52:10 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5391lb # [ 79.408126] niks3-server[983]: 2026/09/20 10:52:10 INFO Completed upload id=285392worker1 # [ 79.411131] nix-grpc-daemon[1156]: time=2026-09-20T10:52:10.496Z level=INFO msg="Upload complete. (295ms)"5393lb # [ 79.410479] niks3-server[983]: 2026/09/20 10:52:10 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5394worker1 # [ 79.420653] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37630 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15395worker1 # [ 79.429561] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:10Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05396worker1 # [ 79.434195] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:10Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=15397worker1 # [ 79.441767] nix-daemon[843]: reaped child process 1179, status = succeeded5398worker1 # [ 79.473822] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376305399worker1 # [ 79.486792] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376305400worker1 # [ 79.500202] nix-grpc-daemon[1156]: time=2026-09-20T10:52:10.585Z level=INFO msg="All 1 paths already cached"5401client # copying 1 paths to 'lb:50051'...5402client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5403worker1 # [ 79.508281] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376305404worker1 # [ 79.518161] nix-daemon[843]: accepted connection from pid 1083, user nix-grpc-daemon (trusted)5405worker1 # [ 79.688568] nix-daemon[843]: accepted connection from pid 1205, user nix-grpc-daemon (trusted)5406worker1 # [ 79.701728] nix-daemon[843]: reaped child process 1208, status = succeeded5407worker1 # [ 79.863114] nix-daemon[843]: accepted connection from pid 1210, user nix-grpc-daemon (trusted)5408worker1 # [ 79.875619] nix-daemon[843]: reaped child process 1213, status = succeeded5409lb # [ 79.886975] niks3-server[983]: 2026/09/20 10:52:10 INFO Received uploads request method=POST path=/api/pending_closures5410worker1 # [ 79.897597] nix-grpc-daemon[1156]: time=2026-09-20T10:52:10.983Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5411worker1 # [ 79.898984] nix-grpc-daemon[1156]: time=2026-09-20T10:52:10.983Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5412lb # [ 79.912350] niks3-server[983]: 2026/09/20 10:52:11 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5413lb # [ 79.923124] niks3-server[983]: 2026/09/20 10:52:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5414lb # [ 79.928418] niks3-server[983]: 2026/09/20 10:52:11 INFO Signed narinfos id=29 count=15415worker1 # [ 79.934460] nix-grpc-daemon[1156]: time=2026-09-20T10:52:11.020Z level=INFO msg="Uploading 1 narinfos"5416lb # [ 79.931930] niks3-server[983]: 2026/09/20 10:52:11 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5417lb # [ 79.945686] niks3-server[983]: 2026/09/20 10:52:11 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5418lb # [ 79.949928] niks3-server[983]: 2026/09/20 10:52:11 INFO Completed upload id=295419worker1 # [ 79.952843] nix-grpc-daemon[1156]: time=2026-09-20T10:52:11.038Z level=INFO msg="Upload complete. (424ms)"5420lb # [ 79.953586] niks3-server[983]: 2026/09/20 10:52:11 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5421worker1 # [ 79.955787] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37630 duration_s=0 paths=1 nar_bytes_in=6405422worker1 # [ 79.960876] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376305423worker1 # [ 79.970060] nix-daemon[843]: accepted connection from pid 1083, user nix-grpc-daemon (trusted)5424worker1 # [ 79.975750] nix-daemon[843]: reaped child process 1203, status = succeeded5425worker1 # [ 79.979067] nix-daemon[843]: accepted connection from pid 1083, user nix-grpc-daemon (trusted)5426client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5427client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5428client # farm-b> LOG-farm-b-on-standby5429worker1 # [ 80.993704] nix-daemon[843]: reaped child process 1217, status = succeeded5430worker1 # [ 81.134457] nix-daemon[843]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)5431worker1 # [ 81.147400] nix-daemon[843]: reaped child process 1235, status = succeeded5432lb # [ 81.161243] niks3-server[983]: 2026/09/20 10:52:12 INFO Received uploads request method=POST path=/api/pending_closures5433worker1 # [ 81.169558] nix-grpc-daemon[1156]: time=2026-09-20T10:52:12.255Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5434worker1 # [ 81.171272] nix-grpc-daemon[1156]: time=2026-09-20T10:52:12.256Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"5435lb # [ 81.180410] niks3-server[983]: 2026/09/20 10:52:12 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5436lb # [ 81.182921] niks3-server[983]: 2026/09/20 10:52:12 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst5437lb # [ 81.187145] niks3-server[983]: 2026/09/20 10:52:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5438lb # [ 81.189712] niks3-server[983]: 2026/09/20 10:52:12 INFO Signed narinfos id=30 count=15439worker1 # [ 81.192558] nix-grpc-daemon[1156]: time=2026-09-20T10:52:12.278Z level=INFO msg="Uploading 1 narinfos"5440lb # [ 81.192729] niks3-server[983]: 2026/09/20 10:52:12 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls5441lb # [ 81.197154] niks3-server[983]: 2026/09/20 10:52:12 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5442lb # [ 81.200034] niks3-server[983]: 2026/09/20 10:52:12 INFO Completed upload id=305443worker1 # [ 81.202923] nix-grpc-daemon[1156]: time=2026-09-20T10:52:12.288Z level=INFO msg="Upload complete. (215ms)"5444lb # [ 81.202802] niks3-server[983]: 2026/09/20 10:52:12 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo5445worker1 # [ 81.205890] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37630 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=15446worker1 # [ 81.212335] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:12Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=05447worker1 # [ 81.214315] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:12Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=15448worker1 # [ 81.218437] nix-daemon[843]: reaped child process 1215, status = succeeded5449worker1 # [ 81.256899] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376305450worker1 # [ 81.267947] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376305451worker1 # [ 81.279576] nix-grpc-daemon[1156]: time=2026-09-20T10:52:12.365Z level=INFO msg="All 2 paths already cached"5452client # copying 1 paths to 'lb:50051'...5453client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...5454worker1 # [ 81.285892] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376305455worker1 # [ 81.295278] nix-daemon[843]: accepted connection from pid 1083, user nix-grpc-daemon (trusted)5456worker1 # [ 81.461441] nix-daemon[843]: accepted connection from pid 1241, user nix-grpc-daemon (trusted)5457worker1 # [ 81.474320] nix-daemon[843]: reaped child process 1244, status = succeeded5458worker1 # [ 81.624348] nix-daemon[843]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5459worker1 # [ 81.637166] nix-daemon[843]: reaped child process 1249, status = succeeded5460lb # [ 81.649188] niks3-server[983]: 2026/09/20 10:52:12 INFO Received uploads request method=POST path=/api/pending_closures5461worker1 # [ 81.658541] nix-grpc-daemon[1156]: time=2026-09-20T10:52:12.744Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5462worker1 # [ 81.659901] nix-grpc-daemon[1156]: time=2026-09-20T10:52:12.744Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"5463lb # [ 81.665477] niks3-server[983]: 2026/09/20 10:52:12 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst5464lb # [ 81.670819] niks3-server[983]: 2026/09/20 10:52:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5465lb # [ 81.673671] niks3-server[983]: 2026/09/20 10:52:12 INFO Signed narinfos id=31 count=15466worker1 # [ 81.676630] nix-grpc-daemon[1156]: time=2026-09-20T10:52:12.762Z level=INFO msg="Uploading 1 narinfos"5467lb # [ 81.675241] niks3-server[983]: 2026/09/20 10:52:12 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls5468lb # [ 81.680857] niks3-server[983]: 2026/09/20 10:52:12 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5469lb # [ 81.683297] niks3-server[983]: 2026/09/20 10:52:12 INFO Completed upload id=315470worker1 # [ 81.686171] nix-grpc-daemon[1156]: time=2026-09-20T10:52:12.771Z level=INFO msg="Upload complete. (381ms)"5471lb # [ 81.685937] niks3-server[983]: 2026/09/20 10:52:12 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo5472worker1 # [ 81.688182] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37630 duration_s=0 paths=1 nar_bytes_in=7685473worker1 # [ 81.691042] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376305474worker1 # [ 81.697066] nix-daemon[843]: accepted connection from pid 1083, user nix-grpc-daemon (trusted)5475worker1 # [ 81.701112] nix-daemon[843]: reaped child process 1239, status = succeeded5476worker1 # [ 81.706214] nix-daemon[843]: accepted connection from pid 1083, user nix-grpc-daemon (trusted)5477client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...5478client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5479worker1 # [ 83.119583] nix-daemon[843]: reaped child process 1253, status = succeeded5480client # farm-top> LOG-farm-top-on-standby5481worker1 # [ 83.263521] nix-daemon[843]: accepted connection from pid 1268, user nix-grpc-daemon (trusted)5482worker1 # [ 83.277188] nix-daemon[843]: reaped child process 1271, status = succeeded5483lb # [ 83.288674] niks3-server[983]: 2026/09/20 10:52:14 INFO Received uploads request method=POST path=/api/pending_closures5484worker1 # [ 83.299446] nix-grpc-daemon[1156]: time=2026-09-20T10:52:14.384Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5485worker1 # [ 83.302057] nix-grpc-daemon[1156]: time=2026-09-20T10:52:14.386Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"5486lb # [ 83.323598] niks3-server[983]: 2026/09/20 10:52:14 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5487lb # [ 83.334743] niks3-server[983]: 2026/09/20 10:52:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5488worker1 # [ 83.347033] nix-grpc-daemon[1156]: time=2026-09-20T10:52:14.432Z level=INFO msg="Uploading 1 narinfos"5489lb # [ 83.344125] niks3-server[983]: 2026/09/20 10:52:14 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst5490lb # [ 83.348464] niks3-server[983]: 2026/09/20 10:52:14 INFO Signed narinfos id=32 count=15491lb # [ 83.364589] niks3-server[983]: 2026/09/20 10:52:14 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5492lb # [ 83.374989] niks3-server[983]: 2026/09/20 10:52:14 INFO Completed upload id=325493worker1 # [ 83.377881] nix-grpc-daemon[1156]: time=2026-09-20T10:52:14.463Z level=INFO msg="Upload complete. (262ms)"5494lb # [ 83.379311] niks3-server[983]: 2026/09/20 10:52:14 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls5495lb # [ 83.382268] niks3-server[983]: 2026/09/20 10:52:14 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo5496worker1 # [ 83.386360] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37630 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=15497worker1 # [ 83.398846] nix-daemon[843]: reaped child process 1251, status = succeeded5498client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&2, in 6.94 seconds)5499client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath5500client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath, in 0.15 seconds)5501worker2 # [ 83.634543] systemd[1]: Listening on Nix worker-protocol over gRPC.5502worker2 # [ 83.641895] systemd[1]: Starting Nix worker-protocol over gRPC...5503worker2 # [ 83.802249] nix-grpc-daemon[1292]: ts=2026-09-20T10:52:14Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5504worker2 # [ 83.806201] nix-grpc-daemon[1292]: ts=2026-09-20T10:52:14Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15505worker2 # [ 83.812528] nix-grpc-daemon[1292]: ts=2026-09-20T10:52:14Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5506worker2 # [ 83.816135] systemd[1]: Started Nix worker-protocol over gRPC.5507lb: must succeed: curl -sf localhost:9901/clusters5508worker2 # [ 83.843387] nix-grpc-daemon[1292]: ts=2026-09-20T10:52:14Z level=info event=scheduler_connected addr=lb:500515509worker1 # [ 83.863597] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:14Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05510lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5511worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5512worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5513lb: must succeed: curl -sf localhost:9901/clusters5514lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5515lb: must succeed: curl -sf localhost:9901/clusters5516lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5517worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5518worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5519(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 13.01 seconds)5520subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5521client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs5522client # [ 84.172746] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs.5523client # Running as unit: rs.service; invocation ID: fda86ce8513c4590b85805c33f0532cd5524client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs, in 0.07 seconds)5525worker1 # [ 84.389885] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:15Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:376305526client # [ 84.380800] nix[1159]: don't know how to build these paths:5527client # [ 84.382236] nix[1159]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5528worker1 # [ 84.406199] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:15Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=05529worker1 # [ 84.411471] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:15Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=15530worker1 # [ 84.450561] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376305531worker1 # [ 84.463676] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376305532client # [ 84.459187] nix[1159]: copying 1 paths to 'lb:50051'...5533worker1 # [ 84.475322] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376305534client # [ 84.461361] nix[1159]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...5535worker1 # [ 84.485403] nix-daemon[843]: accepted connection from pid 1083, user nix-grpc-daemon (trusted)5536worker1 # [ 84.659329] nix-daemon[843]: accepted connection from pid 1284, user nix-grpc-daemon (trusted)5537worker1 # [ 84.673637] nix-daemon[843]: reaped child process 1287, status = succeeded5538worker1 # [ 84.833739] nix-daemon[843]: accepted connection from pid 1289, user nix-grpc-daemon (trusted)5539worker1 # [ 84.846609] nix-daemon[843]: reaped child process 1292, status = succeeded5540lb # [ 84.860214] niks3-server[983]: 2026/09/20 10:52:15 INFO Received uploads request method=POST path=/api/pending_closures5541worker1 # [ 84.869399] nix-grpc-daemon[1156]: time=2026-09-20T10:52:15.954Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5542worker1 # [ 84.870832] nix-grpc-daemon[1156]: time=2026-09-20T10:52:15.954Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"5543lb # [ 84.890190] niks3-server[983]: 2026/09/20 10:52:15 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst5544lb # [ 84.904948] niks3-server[983]: 2026/09/20 10:52:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5545lb # [ 84.911330] niks3-server[983]: 2026/09/20 10:52:16 INFO Signed narinfos id=33 count=15546worker1 # [ 84.917508] nix-grpc-daemon[1156]: time=2026-09-20T10:52:16.002Z level=INFO msg="Uploading 1 narinfos"5547lb # [ 84.915929] niks3-server[983]: 2026/09/20 10:52:16 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls5548lb # [ 84.927219] niks3-server[983]: 2026/09/20 10:52:16 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5549lb # [ 84.930921] niks3-server[983]: 2026/09/20 10:52:16 INFO Completed upload id=335550lb # [ 84.932868] niks3-server[983]: 2026/09/20 10:52:16 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo5551worker1 # [ 84.935164] nix-grpc-daemon[1156]: time=2026-09-20T10:52:16.020Z level=INFO msg="Upload complete. (443ms)"5552worker1 # [ 84.938810] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37630 duration_s=0 paths=1 nar_bytes_in=4165553worker1 # [ 84.944476] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376305554worker1 # [ 84.952199] nix-daemon[843]: accepted connection from pid 1083, user nix-grpc-daemon (trusted)5555worker1 # [ 84.958474] nix-daemon[843]: reaped child process 1282, status = succeeded5556worker1 # [ 84.963474] nix-daemon[843]: accepted connection from pid 1083, user nix-grpc-daemon (trusted)5557client # [ 84.947972] nix[1159]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...5558lb: must succeed: curl -sf localhost:9901/clusters5559lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5560lb: must succeed: curl -sf localhost:9901/clusters5561lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5562lb: must succeed: curl -sf localhost:9901/clusters5563lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5564lb: must succeed: curl -sf localhost:9901/clusters5565lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5566worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5567worker2 # [ 87.020577] systemd-journald[286]: Received client request to rotate journal, rotating.5568worker2 # [ 87.038795] systemd-journald[286]: Vacuuming done, freed 0B of archived journals from /var/log/journal/463853bb793c45feae5837788dec5dd0.5569worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.07 seconds)5570worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5571worker1 # [ 86.976683] systemd[1]: Stopping Nix worker-protocol over gRPC...5572worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)5573??? Warning (UserWarning): sleep(): The 'secs' argument is deprecated. Use 'duration' instead.5574 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 5735575client: must succeed: sleep 5.05576worker1 # [ 87.044215] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:18Z level=info event=draining5577lb # [ 88.541792] envoy[919]: conn peer=[2001:db8:1::1]:44808 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9607 tx=3834 ms=67045578client: (finished: must succeed: sleep 5.0, in 5.04 seconds)5579worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5580worker1 # [ 92.400664] nix-daemon[843]: reaped child process 1296, status = succeeded5581worker1 # [ 92.565666] nix-daemon[843]: accepted connection from pid 1336, user nix-grpc-daemon (trusted)5582worker1 # [ 92.578451] nix-daemon[843]: reaped child process 1339, status = succeeded5583lb # [ 92.592456] niks3-server[983]: 2026/09/20 10:52:23 INFO Received uploads request method=POST path=/api/pending_closures5584worker1 # [ 92.603428] nix-grpc-daemon[1156]: time=2026-09-20T10:52:23.688Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5585worker1 # [ 92.605521] nix-grpc-daemon[1156]: time=2026-09-20T10:52:23.690Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"5586lb # [ 92.634753] niks3-server[983]: 2026/09/20 10:52:23 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5587lb # [ 92.645596] niks3-server[983]: 2026/09/20 10:52:23 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5588lb # [ 92.652204] niks3-server[983]: 2026/09/20 10:52:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5589lb # [ 92.658168] niks3-server[983]: 2026/09/20 10:52:23 INFO Signed narinfos id=34 count=15590worker1 # [ 92.663594] nix-grpc-daemon[1156]: time=2026-09-20T10:52:23.749Z level=INFO msg="Uploading 1 narinfos"5591lb # [ 92.663994] niks3-server[983]: 2026/09/20 10:52:23 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls5592lb # [ 92.675109] niks3-server[983]: 2026/09/20 10:52:23 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5593lb # [ 92.678909] niks3-server[983]: 2026/09/20 10:52:23 INFO Completed upload id=345594lb # [ 92.680443] niks3-server[983]: 2026/09/20 10:52:23 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo5595worker1 # [ 92.683160] nix-grpc-daemon[1156]: time=2026-09-20T10:52:23.768Z level=INFO msg="Upload complete. (301ms)"5596worker1 # [ 92.691242] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37630 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=15597worker1 # [ 92.703728] nix-daemon[843]: reaped child process 1294, status = succeeded5598client # [ 92.710951] systemd[1]: rs.service: Deactivated successfully.5599client # [ 92.714817] systemd[1]: rs.service: Consumed 111ms CPU time over 8.541s wall clock time, 37.1M memory peak, 8.5K incoming IP traffic, 9K outgoing IP traffic.5600worker1 # [ 93.047867] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:24Z level=info event=signal_exit5601worker1 # [ 93.275672] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:24Z level=info event=scheduler_yield5602lb # [ 93.275090] niks3-server[983]: 2026/09/20 10:52:24 INFO lead: released remote=192.168.1.3:446685603worker1 # [ 93.279610] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:24Z level=info event=scheduler_restarting addr=lb:500515604worker2 # [ 93.261154] nix-grpc-daemon[1292]: ts=2026-09-20T10:52:23Z level=info event=scheduler_restarting addr=lb:500515605worker1 # [ 93.284828] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:24Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:376245606worker1 # [ 93.288726] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:24Z level=info event=worker_gone addr=192.168.1.3:500515607worker1 # [ 93.291795] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:24Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:376245608worker1 # [ 93.295197] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:24Z level=info event=worker_gone addr=192.168.1.4:500515609worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 1.10 seconds)5610client: waiting for success: ! systemctl is-active rs5611client: (finished: waiting for success: ! systemctl is-active rs, in 0.03 seconds)5612client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5613client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.03 seconds)5614client: must fail: journalctl -u rs | grep -q 'reconnecting'5615client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.02 seconds)5616lb: must succeed: curl -sf localhost:9901/clusters5617worker2 # [ 93.461539] nix-grpc-daemon[1292]: ts=2026-09-20T10:52:23Z level=info event=scheduler_connected addr=lb:500515618worker1 # [ 93.480487] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:24Z level=info event=scheduler_connected addr=lb:500515619worker2 # [ 93.465750] nix-grpc-daemon[1292]: ts=2026-09-20T10:52:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5620worker1 # [ 93.484583] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5621lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5622worker2 # [ 93.666372] nix-grpc-daemon[1292]: ts=2026-09-20T10:52:23Z level=info event=scheduler_connected addr=lb:500515623worker1 # [ 93.685755] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:24Z level=info event=scheduler_connected addr=lb:500515624worker2 # [ 93.670241] nix-grpc-daemon[1292]: ts=2026-09-20T10:52:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5625worker1 # [ 93.690495] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5626worker2 # [ 93.824127] nix-grpc-daemon[1292]: ts=2026-09-20T10:52:24Z level=info event=scheduler_take_over5627lb # [ 93.840361] niks3-server[983]: 2026/09/20 10:52:24 INFO lead: acquired remote=192.168.1.4:365005628worker2 # [ 93.870910] nix-grpc-daemon[1292]: ts=2026-09-20T10:52:24Z level=info event=scheduler_connected addr=lb:500515629worker1 # [ 93.890574] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:24Z level=info event=scheduler_connected addr=lb:500515630worker2 # [ 93.874667] nix-grpc-daemon[1292]: ts=2026-09-20T10:52:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5631worker1 # [ 93.894461] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5632worker2 # [ 94.075724] nix-grpc-daemon[1292]: ts=2026-09-20T10:52:24Z level=info event=scheduler_connected addr=lb:500515633worker1 # [ 94.094997] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:25Z level=info event=scheduler_connected addr=lb:500515634worker2 # [ 94.079798] nix-grpc-daemon[1292]: ts=2026-09-20T10:52:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5635worker1 # [ 94.099132] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5636worker1 # [ 94.276898] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:25Z level=info event=scheduler_restarting5637worker2 # [ 94.281229] nix-grpc-daemon[1292]: ts=2026-09-20T10:52:24Z level=info event=scheduler_connected addr=lb:500515638worker1 # [ 94.298256] nix-daemon[843]: reaped child process 1147, status = succeeded5639worker2 # [ 94.287755] nix-grpc-daemon[1292]: ts=2026-09-20T10:52:24Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05640worker1 # [ 94.313392] nix-grpc-daemon[1083]: ts=2026-09-20T10:52:25Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5641lb: must succeed: curl -sf localhost:9901/clusters5642lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5643worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5644worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5645worker1 # [ 95.089876] nix-grpc-daemon[1083]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5646worker1 # [ 95.093608] nix-grpc-daemon[1083]: E0000 00:00:1789901546.174404 1126 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5647worker1 # [ 95.098373] nix-grpc-daemon[1083]: E0000 00:00:1789901546.174958 1126 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5648lb: must succeed: curl -sf localhost:9901/clusters5649lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5650worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5651worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5652worker1 # [ 96.008065] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5653worker1 # [ 96.012090] systemd[1]: Stopped Nix worker-protocol over gRPC.5654worker1 # [ 96.014219] systemd[1]: nix-grpc-daemon.service: Consumed 1.308s CPU time over 25.132s wall clock time, 35.4M memory peak, 47.3K incoming IP traffic, 91.2K outgoing IP traffic.5655worker1 # [ 96.030462] systemd[1]: Starting Nix worker-protocol over gRPC...5656worker1 # [ 96.206265] nix-grpc-daemon[1346]: ts=2026-09-20T10:52:27Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5657worker1 # [ 96.209317] nix-grpc-daemon[1346]: ts=2026-09-20T10:52:27Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15658worker1 # [ 96.215121] nix-grpc-daemon[1346]: ts=2026-09-20T10:52:27Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5659worker1 # [ 96.217985] systemd[1]: Started Nix worker-protocol over gRPC.5660worker1 # [ 96.241488] nix-grpc-daemon[1346]: ts=2026-09-20T10:52:27Z level=info event=scheduler_connected addr=lb:500515661worker2 # [ 96.225943] nix-grpc-daemon[1292]: ts=2026-09-20T10:52:26Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05662lb: must succeed: curl -sf localhost:9901/clusters5663lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5664worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5665worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5666(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.79 seconds)5667subtest: scheduler restart mid-build: build finishes, no double build5668client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid5669client # [ 96.963335] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid.5670client # Running as unit: mid.service; invocation ID: 0aa511809f0947b58d467838c8a2eb6b5671client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid, in 0.07 seconds)5672worker1 # [ 97.175584] nix-grpc-daemon[1346]: ts=2026-09-20T10:52:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:364285673worker1 # [ 97.181420] nix-daemon[843]: accepted connection from pid 1346, user nix-grpc-daemon (trusted)5674client # [ 97.172536] nix[1194]: don't know how to build these paths:5675client # [ 97.173481] nix[1194]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5676worker2 # [ 97.178210] nix-grpc-daemon[1292]: ts=2026-09-20T10:52:27Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05677worker2 # [ 97.184129] nix-grpc-daemon[1292]: ts=2026-09-20T10:52:27Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=15678worker2 # [ 97.233767] nix-grpc-daemon[1292]: ts=2026-09-20T10:52:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409905679worker2 # [ 97.244641] nix-daemon[839]: accepted connection from pid 1292, user nix-grpc-daemon (trusted)5680worker2 # [ 97.261780] nix-grpc-daemon[1292]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 28 ms (attempt 1/5)5681worker2 # [ 97.295670] nix-grpc-daemon[1292]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 53 ms (attempt 2/5)5682worker2 # [ 97.354327] nix-grpc-daemon[1292]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 382 ms (attempt 3/5)5683worker2 # [ 97.742300] nix-grpc-daemon[1292]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 415 ms (attempt 4/5)5684worker2 # [ 98.164601] nix-grpc-daemon[1292]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5685worker2 # [ 98.184661] nix-grpc-daemon[1292]: ts=2026-09-20T10:52:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409905686client # [ 98.198932] nix[1194]: copying 1 paths to 'lb:50051'...5687worker2 # [ 98.196395] nix-grpc-daemon[1292]: ts=2026-09-20T10:52:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409905688client # [ 98.202523] nix[1194]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...5689worker2 # [ 98.208116] nix-daemon[839]: accepted connection from pid 1292, user nix-grpc-daemon (trusted)5690worker2 # [ 98.228249] nix-grpc-daemon[1292]: ts=2026-09-20T10:52:28Z level=info event=niks3_push_started pid=14115691lb # [ 98.543610] envoy[919]: conn peer=[2001:db8:1::1]:41048 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2934 tx=1324 ms=83095692lb # [ 98.548631] envoy[919]: conn peer=[2001:db8:1::3]:41118 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=5767 tx=1297 ms=232485693worker2 # [ 98.556681] nix-daemon[839]: accepted connection from pid 1424, user nix-grpc-daemon (trusted)5694worker2 # [ 98.570304] nix-daemon[839]: reaped child process 1427, status = succeeded5695worker2 # [ 98.736668] nix-daemon[839]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)5696worker2 # [ 98.749086] nix-daemon[839]: reaped child process 1432, status = succeeded5697lb # [ 98.783143] niks3-server[983]: 2026/09/20 10:52:29 INFO Received uploads request method=POST path=/api/pending_closures5698worker2 # [ 98.781218] nix-grpc-daemon[1411]: time=2026-09-20T10:52:29.011Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5699worker2 # [ 98.782615] nix-grpc-daemon[1411]: time=2026-09-20T10:52:29.012Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"5700lb # [ 98.826665] niks3-server[983]: 2026/09/20 10:52:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5701lb # [ 98.845473] niks3-server[983]: 2026/09/20 10:52:29 INFO Signed narinfos id=35 count=15702worker2 # [ 98.832575] nix-grpc-daemon[1411]: time=2026-09-20T10:52:29.064Z level=INFO msg="Uploading 1 narinfos"5703lb # [ 98.854280] niks3-server[983]: 2026/09/20 10:52:29 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5704lb # [ 98.901491] niks3-server[983]: 2026/09/20 10:52:29 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst5705lb # [ 98.909817] niks3-server[983]: 2026/09/20 10:52:30 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls5706lb # [ 98.916490] niks3-server[983]: 2026/09/20 10:52:30 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo5707lb # [ 98.922463] niks3-server[983]: 2026/09/20 10:52:30 INFO Completed upload id=355708worker2 # [ 98.907174] nix-grpc-daemon[1411]: time=2026-09-20T10:52:29.138Z level=INFO msg="Upload complete. (498ms)"5709worker2 # [ 98.912702] nix-grpc-daemon[1292]: ts=2026-09-20T10:52:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40990 duration_s=0 paths=1 nar_bytes_in=4245710worker2 # [ 98.920501] nix-grpc-daemon[1292]: ts=2026-09-20T10:52:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409905711worker2 # [ 98.930601] nix-daemon[839]: accepted connection from pid 1292, user nix-grpc-daemon (trusted)5712worker2 # [ 98.942096] nix-daemon[839]: reaped child process 1409, status = succeeded5713client # [ 98.953425] nix[1194]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5714worker2 # [ 98.951459] nix-daemon[839]: accepted connection from pid 1292, user nix-grpc-daemon (trusted)5715lb: must succeed: curl -sf localhost:9901/clusters5716lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5717lb: must succeed: curl -sf localhost:9901/clusters5718lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5719worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5720worker2 # [ 100.775396] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1292 (nix-grpc-daemon) on client request.5721worker2 # [ 100.778384] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5722lb # [ 100.799938] niks3-server[983]: 2026/09/20 10:52:31 INFO lead: released remote=192.168.1.4:365005723client # [ 100.797772] nix[1194]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5724worker1 # [ 100.812483] nix-grpc-daemon[1346]: ts=2026-09-20T10:52:31Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5725worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.06 seconds)5726client: must succeed: sleep 5.05727client # [ 100.802522] nix[1194]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5728worker2 # [ 100.809392] nix-daemon[839]: reaped child process 1434, status = succeeded5729worker2 # [ 100.810690] nix-daemon[839]: reaped child process 1400, status = succeeded5730worker2 # [ 100.815322] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5731worker2 # [ 100.816769] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5732worker2 # [ 100.818572] systemd[1]: nix-grpc-daemon.service: Consumed 425ms CPU time over 17.173s wall clock time, 33.1M memory peak, 14.7K incoming IP traffic, 23K outgoing IP traffic.5733worker2 # [ 100.825283] nix-daemon[1436]: unexpected Nix daemon error: error: interrupted by the user5734worker2 # [ 100.829920] nix-daemon[839]: reaped child process 1436, status = failed with exit code 15735worker1 # [ 101.012496] nix-grpc-daemon[1346]: ts=2026-09-20T10:52:32Z level=info event=scheduler_connected addr=lb:500515736worker2 # [ 100.996902] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.5737worker2 # [ 101.006936] systemd[1]: Starting Nix worker-protocol over gRPC...5738worker2 # [ 101.204307] nix-grpc-daemon[1472]: ts=2026-09-20T10:52:31Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5739worker2 # [ 101.207671] nix-grpc-daemon[1472]: ts=2026-09-20T10:52:31Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15740lb # [ 101.223131] niks3-server[983]: 2026/09/20 10:52:32 INFO lead: acquired remote=192.168.1.3:453525741worker1 # [ 101.226469] nix-grpc-daemon[1346]: ts=2026-09-20T10:52:32Z level=info event=scheduler_take_over5742worker2 # [ 101.214972] nix-grpc-daemon[1472]: ts=2026-09-20T10:52:31Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5743worker2 # [ 101.219309] systemd[1]: Started Nix worker-protocol over gRPC.5744worker1 # [ 101.244945] nix-grpc-daemon[1346]: ts=2026-09-20T10:52:32Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5745worker2 # [ 101.245232] nix-grpc-daemon[1472]: ts=2026-09-20T10:52:31Z level=info event=scheduler_connected addr=lb:500515746worker2 # [ 101.247695] nix-grpc-daemon[1472]: ts=2026-09-20T10:52:31Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5747client # [ 101.304146] nix[1194]: scheduler at lb:50051: no healthy upstream, reconnecting5748worker1 # [ 101.446158] nix-grpc-daemon[1346]: ts=2026-09-20T10:52:32Z level=info event=scheduler_connected addr=lb:500515749worker1 # [ 101.450301] nix-grpc-daemon[1346]: ts=2026-09-20T10:52:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5750worker2 # [ 101.449112] nix-grpc-daemon[1472]: ts=2026-09-20T10:52:31Z level=info event=scheduler_connected addr=lb:500515751worker2 # [ 101.453242] nix-grpc-daemon[1472]: ts=2026-09-20T10:52:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5752worker1 # [ 101.650558] nix-grpc-daemon[1346]: ts=2026-09-20T10:52:32Z level=info event=scheduler_connected addr=lb:500515753worker1 # [ 101.652418] nix-grpc-daemon[1346]: ts=2026-09-20T10:52:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5754worker2 # [ 101.654421] nix-grpc-daemon[1472]: ts=2026-09-20T10:52:31Z level=info event=scheduler_connected addr=lb:500515755worker2 # [ 101.658337] nix-grpc-daemon[1472]: ts=2026-09-20T10:52:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5756worker1 # [ 101.855239] nix-grpc-daemon[1346]: ts=2026-09-20T10:52:32Z level=info event=scheduler_connected addr=lb:500515757worker1 # [ 101.858479] nix-grpc-daemon[1346]: ts=2026-09-20T10:52:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5758worker2 # [ 101.859411] nix-grpc-daemon[1472]: ts=2026-09-20T10:52:32Z level=info event=scheduler_connected addr=lb:500515759worker2 # [ 101.863503] nix-grpc-daemon[1472]: ts=2026-09-20T10:52:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5760worker1 # [ 102.056164] nix-grpc-daemon[1346]: ts=2026-09-20T10:52:33Z level=info event=scheduler_connected addr=lb:500515761worker1 # [ 102.060281] nix-grpc-daemon[1346]: ts=2026-09-20T10:52:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5762worker2 # [ 102.064376] nix-grpc-daemon[1472]: ts=2026-09-20T10:52:32Z level=info event=scheduler_connected addr=lb:500515763worker2 # [ 102.068368] nix-grpc-daemon[1472]: ts=2026-09-20T10:52:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5764worker1 # [ 102.260778] nix-grpc-daemon[1346]: ts=2026-09-20T10:52:33Z level=info event=scheduler_connected addr=lb:500515765worker1 # [ 102.264741] nix-grpc-daemon[1346]: ts=2026-09-20T10:52:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5766worker2 # [ 102.267814] nix-grpc-daemon[1472]: ts=2026-09-20T10:52:32Z level=info event=scheduler_connected addr=lb:500515767worker2 # [ 102.271194] nix-grpc-daemon[1472]: ts=2026-09-20T10:52:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5768client # [ 102.306849] nix[1194]: scheduler at lb:50051: no healthy upstream, reconnecting5769worker1 # [ 102.465634] nix-grpc-daemon[1346]: ts=2026-09-20T10:52:33Z level=info event=scheduler_connected addr=lb:500515770worker1 # [ 102.469737] nix-grpc-daemon[1346]: ts=2026-09-20T10:52:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5771worker2 # [ 102.468646] nix-grpc-daemon[1472]: ts=2026-09-20T10:52:32Z level=info event=scheduler_connected addr=lb:500515772worker2 # [ 102.472803] nix-grpc-daemon[1472]: ts=2026-09-20T10:52:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5773worker1 # [ 102.670650] nix-grpc-daemon[1346]: ts=2026-09-20T10:52:33Z level=info event=scheduler_connected addr=lb:500515774worker1 # [ 102.674629] nix-grpc-daemon[1346]: ts=2026-09-20T10:52:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5775worker2 # [ 102.673707] nix-grpc-daemon[1472]: ts=2026-09-20T10:52:32Z level=info event=scheduler_connected addr=lb:500515776worker2 # [ 102.677757] nix-grpc-daemon[1472]: ts=2026-09-20T10:52:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5777worker1 # [ 102.876718] nix-grpc-daemon[1346]: ts=2026-09-20T10:52:33Z level=info event=scheduler_connected addr=lb:500515778worker1 # [ 102.880473] nix-grpc-daemon[1346]: ts=2026-09-20T10:52:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5779worker2 # [ 102.878914] nix-grpc-daemon[1472]: ts=2026-09-20T10:52:33Z level=info event=scheduler_connected addr=lb:500515780worker2 # [ 102.883079] nix-grpc-daemon[1472]: ts=2026-09-20T10:52:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5781worker1 # [ 103.081378] nix-grpc-daemon[1346]: ts=2026-09-20T10:52:34Z level=info event=scheduler_connected addr=lb:500515782worker1 # [ 103.092072] nix-grpc-daemon[1346]: ts=2026-09-20T10:52:34Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05783worker2 # [ 103.084160] nix-grpc-daemon[1472]: ts=2026-09-20T10:52:33Z level=info event=scheduler_connected addr=lb:500515784worker1 # [ 103.103376] nix-grpc-daemon[1346]: ts=2026-09-20T10:52:34Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05785worker1 # [ 104.331911] nix-grpc-daemon[1346]: ts=2026-09-20T10:52:35Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05786worker1 # [ 104.338329] nix-grpc-daemon[1346]: ts=2026-09-20T10:52:35Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=15787worker1 # [ 104.380781] nix-grpc-daemon[1346]: ts=2026-09-20T10:52:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364285788worker1 # [ 104.392738] nix-grpc-daemon[1346]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 10 ms (attempt 1/5)5789worker1 # [ 104.408448] nix-grpc-daemon[1346]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 23 ms (attempt 2/5)5790worker1 # [ 104.437153] nix-grpc-daemon[1346]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 302 ms (attempt 3/5)5791worker1 # [ 104.745158] nix-grpc-daemon[1346]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 342 ms (attempt 4/5)5792worker1 # [ 105.093431] nix-grpc-daemon[1346]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5793worker1 # [ 105.104380] nix-grpc-daemon[1346]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' from 'http://lb:5751'...5794worker1 # [ 105.125298] nix-daemon[843]: accepted connection from pid 1346, user nix-grpc-daemon (trusted)5795client # [ 105.121895] nix[1194]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5796worker1 # [ 105.138484] nix-daemon[843]: accepted connection from pid 1346, user nix-grpc-daemon (trusted)5797client: (finished: must succeed: sleep 5.0, in 5.04 seconds)5798client: waiting for success: ! systemctl is-active mid5799lb # [ 108.543627] envoy[919]: conn peer=[2001:db8:1::4]:58780 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1560 tx=460 ms=169545800worker1 # [ 196.442420] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:07Z level=info event=niks3_push_started pid=14495801worker1 # [ 196.455783] nix-daemon[843]: reaped child process 1425, status = succeeded5802worker1 # [ 196.720515] nix-daemon[843]: accepted connection from pid 1462, user nix-grpc-daemon (trusted)5803worker1 # [ 196.731896] nix-daemon[843]: reaped child process 1465, status = succeeded5804lb # [ 196.738664] niks3-server[983]: 2026/09/20 10:54:07 INFO Received uploads request method=POST path=/api/pending_closures5805worker1 # [ 196.747293] nix-grpc-daemon[1449]: time=2026-09-20T10:54:07.832Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5806worker1 # [ 196.749142] nix-grpc-daemon[1449]: time=2026-09-20T10:54:07.834Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"5807lb # [ 196.769385] niks3-server[983]: 2026/09/20 10:54:07 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5808lb # [ 196.773855] niks3-server[983]: 2026/09/20 10:54:07 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5809lb # [ 196.777031] niks3-server[983]: 2026/09/20 10:54:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5810lb # [ 196.779130] niks3-server[983]: 2026/09/20 10:54:07 INFO Signed narinfos id=36 count=15811worker1 # [ 196.784081] nix-grpc-daemon[1449]: time=2026-09-20T10:54:07.869Z level=INFO msg="Uploading 1 narinfos"5812lb # [ 196.783431] niks3-server[983]: 2026/09/20 10:54:07 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls5813lb # [ 196.788489] niks3-server[983]: 2026/09/20 10:54:07 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5814lb # [ 196.791791] niks3-server[983]: 2026/09/20 10:54:07 INFO Completed upload id=365815worker1 # [ 196.796170] nix-grpc-daemon[1449]: time=2026-09-20T10:54:07.881Z level=INFO msg="Upload complete. (212ms)"5816lb # [ 196.793385] niks3-server[983]: 2026/09/20 10:54:07 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo5817worker1 # [ 196.799675] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36428 duration_s=92 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=15818worker1 # [ 196.806870] nix-daemon[843]: reaped child process 1423, status = succeeded5819client # [ 196.826479] systemd[1]: mid.service: Deactivated successfully.5820client # [ 196.829276] systemd[1]: mid.service: Consumed 210ms CPU time over 1min 39.862s wall clock time, 39.3M memory peak, 4K written to disk, 10.1K incoming IP traffic, 11.4K outgoing IP traffic.5821client: (finished: waiting for success: ! systemctl is-active mid, in 91.19 seconds)5822client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5823client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.03 seconds)5824(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 100.44 seconds)5825subtest: interrupting the client stops the build on the worker5826client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr5827client # [ 197.375197] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr.5828client # Running as unit: intr.service; invocation ID: b308bb3f7a924dc69886a0f13a6ff3545829client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr, in 0.03 seconds)5830client # [ 197.550889] nix[1659]: don't know how to build these paths:5831client # [ 197.552335] nix[1659]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv5832worker1 # [ 197.565737] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:08Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:364285833worker1 # [ 197.576739] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:08Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=05834worker1 # [ 197.580779] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:08Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=15835worker2 # [ 197.607815] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446945836worker2 # [ 197.619767] nix-daemon[839]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)5837worker2 # [ 197.636906] nix-grpc-daemon[1472]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 36 ms (attempt 1/5)5838worker2 # [ 197.678871] nix-grpc-daemon[1472]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 82 ms (attempt 2/5)5839worker2 # [ 197.767059] nix-grpc-daemon[1472]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 63 ms (attempt 3/5)5840worker2 # [ 197.835700] nix-grpc-daemon[1472]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 538 ms (attempt 4/5)5841worker2 # [ 198.379980] nix-grpc-daemon[1472]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5842worker2 # [ 198.394497] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446945843client # [ 198.406396] nix[1659]: copying 1 paths to 'lb:50051'...5844client # [ 198.408530] nix[1659]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...5845worker2 # [ 198.405521] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:446945846worker2 # [ 198.415161] nix-daemon[839]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)5847worker2 # [ 198.434121] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:08Z level=info event=niks3_push_started pid=15405848lb # [ 198.547245] envoy[919]: conn peer=[2001:db8:1::1]:37260 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4192 tx=1758 ms=996415849worker2 # [ 198.791387] nix-daemon[839]: accepted connection from pid 1557, user nix-grpc-daemon (trusted)5850worker2 # [ 198.803278] nix-daemon[839]: reaped child process 1560, status = succeeded5851worker2 # [ 198.959259] nix-daemon[839]: accepted connection from pid 1562, user nix-grpc-daemon (trusted)5852worker2 # [ 198.972616] nix-daemon[839]: reaped child process 1565, status = succeeded5853lb # [ 199.002695] niks3-server[983]: 2026/09/20 10:54:10 INFO Received uploads request method=POST path=/api/pending_closures5854worker2 # [ 198.996307] nix-grpc-daemon[1540]: time=2026-09-20T10:54:09.226Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5855worker2 # [ 198.997701] nix-grpc-daemon[1540]: time=2026-09-20T10:54:09.227Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"5856lb # [ 199.032649] niks3-server[983]: 2026/09/20 10:54:10 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst5857lb # [ 199.042281] niks3-server[983]: 2026/09/20 10:54:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5858lb # [ 199.046908] niks3-server[983]: 2026/09/20 10:54:10 INFO Signed narinfos id=37 count=15859worker2 # [ 199.034137] nix-grpc-daemon[1540]: time=2026-09-20T10:54:09.265Z level=INFO msg="Uploading 1 narinfos"5860lb # [ 199.060786] niks3-server[983]: 2026/09/20 10:54:10 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls5861lb # [ 199.068123] niks3-server[983]: 2026/09/20 10:54:10 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5862lb # [ 199.073595] niks3-server[983]: 2026/09/20 10:54:10 INFO Completed upload id=375863worker2 # [ 199.058028] nix-grpc-daemon[1540]: time=2026-09-20T10:54:09.289Z level=INFO msg="Upload complete. (427ms)"5864lb # [ 199.076667] niks3-server[983]: 2026/09/20 10:54:10 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo5865worker2 # [ 199.061438] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44694 duration_s=0 paths=1 nar_bytes_in=4245866worker2 # [ 199.067307] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446945867worker2 # [ 199.073058] nix-daemon[839]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)5868worker2 # [ 199.077229] nix-daemon[839]: reaped child process 1538, status = succeeded5869worker2 # [ 199.082059] nix-daemon[839]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)5870client # [ 199.085635] nix[1659]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...5871client: must succeed: systemctl kill -s INT intr5872client # [ 200.943439] systemd[1]: intr.service: Sent signal SIGINT to main process 1659 (nix) on client request.5873client # [ 200.950584] nix[1659]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5874client: (finished: must succeed: systemctl kill -s INT intr, in 0.06 seconds)5875client # [ 200.957785] nix[1659]: error: interrupted by the user5876client # [ 200.984117] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5877client # [ 200.986040] systemd[1]: intr.service: Failed with result 'exit-code'.5878client # [ 200.987866] systemd[1]: intr.service: Consumed 99ms CPU time over 3.610s wall clock time, 39.1M memory peak, 7.8K incoming IP traffic, 8.7K outgoing IP traffic.5879worker2 # [ 201.099970] nix-daemon[839]: reaped child process 1567, status = succeeded5880worker2 # [ 201.102435] nix-daemon[1569]: unexpected Nix daemon error: error: interrupted by the user5881worker2 # [ 201.106397] nix-daemon[839]: reaped child process 1569, status = failed with exit code 15882(finished: subtest: interrupting the client stops the build on the worker, in 4.84 seconds)5883subtest: a deploy mid-build drains: the build finishes, the new generation takes over5884client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw5885client # [ 202.244435] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw.5886client # Running as unit: sw.service; invocation ID: eaf8c5d70b6b4b76b9a83dc0847e58a95887client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw, in 0.07 seconds)5888worker2 # [ 202.429452] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:12Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:446945889client # [ 202.439089] nix[1679]: don't know how to build these paths:5890client # [ 202.439957] nix[1679]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv5891worker1 # [ 202.464615] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:13Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=05892worker1 # [ 202.468689] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:13Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=15893worker1 # [ 202.506857] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364285894worker1 # [ 202.522807] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364285895client # [ 202.518329] nix[1679]: copying 1 paths to 'lb:50051'...5896worker1 # [ 202.534773] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:364285897client # [ 202.521884] nix[1679]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...5898worker1 # [ 202.543939] nix-daemon[843]: accepted connection from pid 1346, user nix-grpc-daemon (trusted)5899worker1 # [ 202.710374] nix-daemon[843]: accepted connection from pid 1503, user nix-grpc-daemon (trusted)5900worker1 # [ 202.723132] nix-daemon[843]: reaped child process 1506, status = succeeded5901worker1 # [ 202.877447] nix-daemon[843]: accepted connection from pid 1508, user nix-grpc-daemon (trusted)5902worker1 # [ 202.889811] nix-daemon[843]: reaped child process 1511, status = succeeded5903lb # [ 202.901598] niks3-server[983]: 2026/09/20 10:54:13 INFO Received uploads request method=POST path=/api/pending_closures5904worker1 # [ 202.913471] nix-grpc-daemon[1449]: time=2026-09-20T10:54:13.998Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5905worker1 # [ 202.914992] nix-grpc-daemon[1449]: time=2026-09-20T10:54:14.000Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"5906lb # [ 202.933880] niks3-server[983]: 2026/09/20 10:54:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5907lb # [ 202.939323] niks3-server[983]: 2026/09/20 10:54:14 INFO Signed narinfos id=38 count=15908worker1 # [ 202.945537] nix-grpc-daemon[1449]: time=2026-09-20T10:54:14.031Z level=INFO msg="Uploading 1 narinfos"5909lb # [ 202.949753] niks3-server[983]: 2026/09/20 10:54:14 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst5910lb # [ 202.958730] niks3-server[983]: 2026/09/20 10:54:14 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls5911lb # [ 202.964121] niks3-server[983]: 2026/09/20 10:54:14 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5912lb # [ 202.971507] niks3-server[983]: 2026/09/20 10:54:14 INFO Completed upload id=385913worker1 # [ 202.974288] nix-grpc-daemon[1449]: time=2026-09-20T10:54:14.059Z level=INFO msg="Upload complete. (416ms)"5914lb # [ 202.973260] niks3-server[983]: 2026/09/20 10:54:14 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo5915worker1 # [ 202.977121] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36428 duration_s=0 paths=1 nar_bytes_in=4165916worker1 # [ 202.981837] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364285917worker1 # [ 202.991073] nix-daemon[843]: accepted connection from pid 1346, user nix-grpc-daemon (trusted)5918worker1 # [ 202.996117] nix-daemon[843]: reaped child process 1501, status = succeeded5919worker1 # [ 202.999797] nix-daemon[843]: accepted connection from pid 1346, user nix-grpc-daemon (trusted)5920client # [ 202.984270] nix[1679]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...5921worker1: must succeed: systemctl reload nix-grpc-daemon.service5922worker1 # [ 204.741165] systemd[1]: Reloading Nix worker-protocol over gRPC...5923worker1 # [ 204.776577] systemd[1]: Reloaded Nix worker-protocol over gRPC.5924worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.08 seconds)5925worker1: must succeed: pgrep -f 'read -t [9]0 x'5926worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.04 seconds)5927client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&25928client # don't know how to build these paths:5929worker2 # [ 205.010137] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:15Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:446945930client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv5931worker2 # [ 205.013279] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446945932worker1 # [ 205.053072] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:16Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=05933worker1 # [ 205.055162] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:16Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=15934worker2 # [ 205.080124] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446945935worker2 # [ 205.091673] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446945936client # copying 1 paths to 'lb:50051'...5937worker2 # [ 205.104459] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:446945938client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...5939worker2 # [ 205.128356] nix-daemon[839]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)5940worker1 # [ 205.256681] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:16Z level=info event=draining5941worker2 # [ 205.294941] nix-daemon[839]: accepted connection from pid 1619, user nix-grpc-daemon (trusted)5942worker2 # [ 205.308465] nix-daemon[839]: reaped child process 1622, status = succeeded5943worker2 # [ 205.466320] nix-daemon[839]: accepted connection from pid 1624, user nix-grpc-daemon (trusted)5944worker2 # [ 205.478943] nix-daemon[839]: reaped child process 1627, status = succeeded5945lb # [ 205.508492] niks3-server[983]: 2026/09/20 10:54:16 INFO Received uploads request method=POST path=/api/pending_closures5946worker2 # [ 205.504537] nix-grpc-daemon[1540]: time=2026-09-20T10:54:15.736Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5947worker2 # [ 205.506548] nix-grpc-daemon[1540]: time=2026-09-20T10:54:15.738Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"5948lb # [ 205.546875] niks3-server[983]: 2026/09/20 10:54:16 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst5949lb # [ 205.551571] niks3-server[983]: 2026/09/20 10:54:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5950lb # [ 205.558526] niks3-server[983]: 2026/09/20 10:54:16 INFO Signed narinfos id=39 count=15951worker2 # [ 205.546216] nix-grpc-daemon[1540]: time=2026-09-20T10:54:15.777Z level=INFO msg="Uploading 1 narinfos"5952lb # [ 205.566242] niks3-server[983]: 2026/09/20 10:54:16 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls5953lb # [ 205.574451] niks3-server[983]: 2026/09/20 10:54:16 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5954lb # [ 205.584258] niks3-server[983]: 2026/09/20 10:54:16 INFO Completed upload id=395955worker2 # [ 205.568682] nix-grpc-daemon[1540]: time=2026-09-20T10:54:15.800Z level=INFO msg="Upload complete. (428ms)"5956lb # [ 205.585476] niks3-server[983]: 2026/09/20 10:54:16 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo5957worker2 # [ 205.571370] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44694 duration_s=0 paths=1 nar_bytes_in=5285958worker2 # [ 205.576919] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446945959worker2 # [ 205.585835] nix-daemon[839]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)5960worker2 # [ 205.594149] nix-daemon[839]: reaped child process 1617, status = succeeded5961worker2 # [ 205.598982] nix-daemon[839]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)5962client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...5963client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv5964client # farm-a> LOG-farm-a-during-drain5965worker2 # [ 206.213603] nix-daemon[839]: reaped child process 1631, status = succeeded5966worker2 # [ 206.342413] nix-daemon[839]: accepted connection from pid 1646, user nix-grpc-daemon (trusted)5967worker2 # [ 206.356353] nix-daemon[839]: reaped child process 1649, status = succeeded5968lb # [ 206.385806] niks3-server[983]: 2026/09/20 10:54:17 INFO Received uploads request method=POST path=/api/pending_closures5969worker2 # [ 206.380211] nix-grpc-daemon[1540]: time=2026-09-20T10:54:16.610Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5970worker2 # [ 206.381776] nix-grpc-daemon[1540]: time=2026-09-20T10:54:16.611Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"5971lb # [ 206.422281] niks3-server[983]: 2026/09/20 10:54:17 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv5972lb # [ 206.436272] niks3-server[983]: 2026/09/20 10:54:17 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst5973lb # [ 206.446559] niks3-server[983]: 2026/09/20 10:54:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5974lb # [ 206.452973] niks3-server[983]: 2026/09/20 10:54:17 INFO Signed narinfos id=40 count=15975worker2 # [ 206.440516] nix-grpc-daemon[1540]: time=2026-09-20T10:54:16.671Z level=INFO msg="Uploading 1 narinfos"5976lb # [ 206.456333] niks3-server[983]: 2026/09/20 10:54:17 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls5977lb # [ 206.466115] niks3-server[983]: 2026/09/20 10:54:17 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5978lb # [ 206.469866] niks3-server[983]: 2026/09/20 10:54:17 INFO Completed upload id=405979lb # [ 206.471060] niks3-server[983]: 2026/09/20 10:54:17 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo5980worker2 # [ 206.454326] nix-grpc-daemon[1540]: time=2026-09-20T10:54:16.686Z level=INFO msg="Upload complete. (249ms)"5981worker2 # [ 206.459491] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44694 duration_s=0 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=15982worker1 # [ 206.481727] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:17Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=05983worker1 # [ 206.483605] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:17Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=15984worker2 # [ 206.472375] nix-daemon[839]: reaped child process 1629, status = succeeded5985worker2 # [ 206.512831] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446945986worker2 # [ 206.523763] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446945987worker2 # [ 206.533408] nix-grpc-daemon[1540]: time=2026-09-20T10:54:16.765Z level=INFO msg="All 1 paths already cached"5988client # copying 1 paths to 'lb:50051'...5989worker2 # [ 206.538956] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:446945990client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...5991worker2 # [ 206.555063] nix-daemon[839]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)5992worker2 # [ 206.718046] nix-daemon[839]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)5993worker2 # [ 206.730931] nix-daemon[839]: reaped child process 1656, status = succeeded5994worker2 # [ 206.879333] nix-daemon[839]: accepted connection from pid 1658, user nix-grpc-daemon (trusted)5995worker2 # [ 206.891556] nix-daemon[839]: reaped child process 1661, status = succeeded5996lb # [ 206.920928] niks3-server[983]: 2026/09/20 10:54:18 INFO Received uploads request method=POST path=/api/pending_closures5997worker2 # [ 206.913403] nix-grpc-daemon[1540]: time=2026-09-20T10:54:17.145Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5998worker2 # [ 206.915143] nix-grpc-daemon[1540]: time=2026-09-20T10:54:17.147Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"5999lb # [ 206.950786] niks3-server[983]: 2026/09/20 10:54:18 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6000lb # [ 206.959863] niks3-server[983]: 2026/09/20 10:54:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6001lb # [ 206.964556] niks3-server[983]: 2026/09/20 10:54:18 INFO Signed narinfos id=41 count=16002worker2 # [ 206.952086] nix-grpc-daemon[1540]: time=2026-09-20T10:54:17.183Z level=INFO msg="Uploading 1 narinfos"6003lb # [ 206.977581] niks3-server[983]: 2026/09/20 10:54:18 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6004lb # [ 206.983093] niks3-server[983]: 2026/09/20 10:54:18 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6005worker2 # [ 206.973881] nix-grpc-daemon[1540]: time=2026-09-20T10:54:17.205Z level=INFO msg="Upload complete. (410ms)"6006lb # [ 206.989823] niks3-server[983]: 2026/09/20 10:54:18 INFO Completed upload id=416007worker2 # [ 206.975893] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44694 duration_s=0 paths=1 nar_bytes_in=6406008lb # [ 206.993143] niks3-server[983]: 2026/09/20 10:54:18 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6009worker2 # [ 206.981414] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446946010worker2 # [ 206.986586] nix-daemon[839]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6011worker2 # [ 206.988990] nix-daemon[839]: reaped child process 1651, status = succeeded6012worker2 # [ 206.993983] nix-daemon[839]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6013client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6014client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6015worker2 # [ 208.208269] nix-daemon[839]: reaped child process 1665, status = succeeded6016client # farm-b> LOG-farm-b-during-drain6017worker2 # [ 208.310468] nix-daemon[839]: accepted connection from pid 1680, user nix-grpc-daemon (trusted)6018worker2 # [ 208.323343] nix-daemon[839]: reaped child process 1683, status = succeeded6019lb # [ 208.348198] niks3-server[983]: 2026/09/20 10:54:19 INFO Received uploads request method=POST path=/api/pending_closures6020worker2 # [ 208.341030] nix-grpc-daemon[1540]: time=2026-09-20T10:54:18.572Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6021worker2 # [ 208.342541] nix-grpc-daemon[1540]: time=2026-09-20T10:54:18.574Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6022lb # [ 208.377346] niks3-server[983]: 2026/09/20 10:54:19 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6023lb # [ 208.391232] niks3-server[983]: 2026/09/20 10:54:19 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6024lb # [ 208.403124] niks3-server[983]: 2026/09/20 10:54:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6025lb # [ 208.409374] niks3-server[983]: 2026/09/20 10:54:19 INFO Signed narinfos id=42 count=16026worker2 # [ 208.396312] nix-grpc-daemon[1540]: time=2026-09-20T10:54:18.628Z level=INFO msg="Uploading 1 narinfos"6027lb # [ 208.411911] niks3-server[983]: 2026/09/20 10:54:19 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6028lb # [ 208.422946] niks3-server[983]: 2026/09/20 10:54:19 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6029lb # [ 208.428088] niks3-server[983]: 2026/09/20 10:54:19 INFO Completed upload id=426030lb # [ 208.430977] niks3-server[983]: 2026/09/20 10:54:19 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6031worker2 # [ 208.414321] nix-grpc-daemon[1540]: time=2026-09-20T10:54:18.646Z level=INFO msg="Upload complete. (213ms)"6032worker2 # [ 208.420333] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44694 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16033worker1 # [ 208.441809] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:19Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06034worker1 # [ 208.444314] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:19Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=16035worker2 # [ 208.432346] nix-daemon[839]: reaped child process 1663, status = succeeded6036worker2 # [ 208.469199] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446946037worker2 # [ 208.487229] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446946038worker2 # [ 208.501125] nix-grpc-daemon[1540]: time=2026-09-20T10:54:18.733Z level=INFO msg="All 2 paths already cached"6039client # copying 1 paths to 'lb:50051'...6040worker2 # [ 208.509476] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:446946041client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6042worker2 # [ 208.527465] nix-daemon[839]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6043lb # [ 208.547382] envoy[919]: conn peer=[2001:db8:1::1]:50540 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2924 tx=732 ms=34176044worker2 # [ 208.690683] nix-daemon[839]: accepted connection from pid 1687, user nix-grpc-daemon (trusted)6045worker2 # [ 208.703559] nix-daemon[839]: reaped child process 1690, status = succeeded6046worker2 # [ 208.853661] nix-daemon[839]: accepted connection from pid 1692, user nix-grpc-daemon (trusted)6047worker2 # [ 208.866244] nix-daemon[839]: reaped child process 1695, status = succeeded6048lb # [ 208.895513] niks3-server[983]: 2026/09/20 10:54:19 INFO Received uploads request method=POST path=/api/pending_closures6049worker2 # [ 208.886981] nix-grpc-daemon[1540]: time=2026-09-20T10:54:19.119Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6050worker2 # [ 208.889068] nix-grpc-daemon[1540]: time=2026-09-20T10:54:19.120Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6051lb # [ 208.923368] niks3-server[983]: 2026/09/20 10:54:20 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6052lb # [ 208.931112] niks3-server[983]: 2026/09/20 10:54:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6053worker2 # [ 208.921545] nix-grpc-daemon[1540]: time=2026-09-20T10:54:19.153Z level=INFO msg="Uploading 1 narinfos"6054lb # [ 208.937125] niks3-server[983]: 2026/09/20 10:54:20 INFO Signed narinfos id=43 count=16055lb # [ 208.943242] niks3-server[983]: 2026/09/20 10:54:20 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6056lb # [ 208.954244] niks3-server[983]: 2026/09/20 10:54:20 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6057lb # [ 208.958728] niks3-server[983]: 2026/09/20 10:54:20 INFO Completed upload id=436058worker2 # [ 208.945705] nix-grpc-daemon[1540]: time=2026-09-20T10:54:19.177Z level=INFO msg="Upload complete. (403ms)"6059lb # [ 208.963723] niks3-server[983]: 2026/09/20 10:54:20 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6060worker2 # [ 208.949065] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44694 duration_s=0 paths=1 nar_bytes_in=7686061worker2 # [ 208.954204] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446946062worker2 # [ 208.962669] nix-daemon[839]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6063worker2 # [ 208.970150] nix-daemon[839]: reaped child process 1685, status = succeeded6064client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6065worker2 # [ 208.973434] nix-daemon[839]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6066client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6067client # farm-top> LOG-farm-top-during-drain6068worker2 # [ 210.400688] nix-daemon[839]: reaped child process 1699, status = succeeded6069worker2 # [ 210.555423] nix-daemon[839]: accepted connection from pid 1714, user nix-grpc-daemon (trusted)6070worker2 # [ 210.568731] nix-daemon[839]: reaped child process 1717, status = succeeded6071lb # [ 210.597802] niks3-server[983]: 2026/09/20 10:54:21 INFO Received uploads request method=POST path=/api/pending_closures6072worker2 # [ 210.590724] nix-grpc-daemon[1540]: time=2026-09-20T10:54:20.822Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6073worker2 # [ 210.592232] nix-grpc-daemon[1540]: time=2026-09-20T10:54:20.824Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6074lb # [ 210.635940] niks3-server[983]: 2026/09/20 10:54:21 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6075lb # [ 210.643587] niks3-server[983]: 2026/09/20 10:54:21 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6076lb # [ 210.654690] niks3-server[983]: 2026/09/20 10:54:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6077lb # [ 210.660656] niks3-server[983]: 2026/09/20 10:54:21 INFO Signed narinfos id=44 count=16078worker2 # [ 210.647323] nix-grpc-daemon[1540]: time=2026-09-20T10:54:20.879Z level=INFO msg="Uploading 1 narinfos"6079lb # [ 210.664937] niks3-server[983]: 2026/09/20 10:54:21 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6080lb # [ 210.675976] niks3-server[983]: 2026/09/20 10:54:21 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6081lb # [ 210.682364] niks3-server[983]: 2026/09/20 10:54:21 INFO Completed upload id=446082worker2 # [ 210.666538] nix-grpc-daemon[1540]: time=2026-09-20T10:54:20.898Z level=INFO msg="Upload complete. (285ms)"6083lb # [ 210.684995] niks3-server[983]: 2026/09/20 10:54:21 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6084worker2 # [ 210.672477] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44694 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16085worker2 # [ 210.686582] nix-daemon[839]: reaped child process 1697, status = succeeded6086client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&2, in 5.90 seconds)6087client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath6088client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath, in 0.16 seconds)6089worker1: must succeed: pkill -f 'read -t [9]0 x'6090worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6091client: waiting for success: ! systemctl is-active sw6092worker1 # [ 211.022108] nix-daemon[843]: reaped child process 1515, status = succeeded6093worker1 # [ 211.152370] nix-daemon[843]: accepted connection from pid 1561, user nix-grpc-daemon (trusted)6094worker1 # [ 211.166127] nix-daemon[843]: reaped child process 1564, status = succeeded6095lb # [ 211.179317] niks3-server[983]: 2026/09/20 10:54:22 INFO Received uploads request method=POST path=/api/pending_closures6096worker1 # [ 211.189655] nix-grpc-daemon[1449]: time=2026-09-20T10:54:22.275Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6097worker1 # [ 211.191175] nix-grpc-daemon[1449]: time=2026-09-20T10:54:22.276Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6098lb # [ 211.208874] niks3-server[983]: 2026/09/20 10:54:22 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6099lb # [ 211.220566] niks3-server[983]: 2026/09/20 10:54:22 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6100lb # [ 211.232755] niks3-server[983]: 2026/09/20 10:54:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6101lb # [ 211.236771] niks3-server[983]: 2026/09/20 10:54:22 INFO Signed narinfos id=45 count=16102worker1 # [ 211.242238] nix-grpc-daemon[1449]: time=2026-09-20T10:54:22.327Z level=INFO msg="Uploading 1 narinfos"6103lb # [ 211.240715] niks3-server[983]: 2026/09/20 10:54:22 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6104lb # [ 211.250158] niks3-server[983]: 2026/09/20 10:54:22 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6105lb # [ 211.252968] niks3-server[983]: 2026/09/20 10:54:22 INFO Completed upload id=456106lb # [ 211.255880] niks3-server[983]: 2026/09/20 10:54:22 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6107worker1 # [ 211.255942] nix-grpc-daemon[1449]: time=2026-09-20T10:54:22.341Z level=INFO msg="Upload complete. (240ms)"6108worker1 # [ 211.263364] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36428 duration_s=8 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16109worker1 # [ 211.274445] nix-daemon[843]: reaped child process 1513, status = succeeded6110client # [ 211.277699] systemd[1]: sw.service: Deactivated successfully.6111client # [ 211.280871] systemd[1]: sw.service: Consumed 109ms CPU time over 9.032s wall clock time, 37.1M memory peak, 8.3K incoming IP traffic, 9K outgoing IP traffic.6112client: (finished: waiting for success: ! systemctl is-active sw, in 1.07 seconds)6113client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6114client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds)6115worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6116worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 0.05 seconds)6117lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6118lb: must succeed: curl -sf localhost:9901/clusters6119lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6120worker1 # [ 212.259207] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:23Z level=info event=signal_exit6121lb # [ 212.536118] niks3-server[983]: 2026/09/20 10:54:23 INFO lead: released remote=192.168.1.3:453526122worker1 # [ 212.537938] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:23Z level=info event=scheduler_yield6123worker2 # [ 212.524153] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:22Z level=info event=scheduler_restarting addr=lb:500516124worker1 # [ 212.542359] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:23Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:360926125worker1 # [ 212.547510] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:23Z level=info event=worker_gone addr=192.168.1.4:500516126worker1 # [ 212.550647] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:23Z level=info event=scheduler_restarting addr=lb:500516127worker1 # [ 212.553655] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:23Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:360926128worker1 # [ 212.557052] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:23Z level=info event=worker_gone addr=192.168.1.3:500516129worker2 # [ 212.724178] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:22Z level=info event=scheduler_connected addr=lb:500516130worker1 # [ 212.747670] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:23Z level=info event=scheduler_connected addr=lb:500516131worker2 # [ 212.728184] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:22Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6132worker1 # [ 212.752436] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:23Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6133worker2 # [ 212.928974] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:23Z level=info event=scheduler_connected addr=lb:500516134worker1 # [ 212.952852] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:24Z level=info event=scheduler_connected addr=lb:500516135worker2 # [ 212.932998] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:23Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6136worker1 # [ 212.955273] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:24Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6137lb: must succeed: curl -sf localhost:9901/clusters6138worker2 # [ 213.133753] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:23Z level=info event=scheduler_connected addr=lb:500516139worker1 # [ 213.155723] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:24Z level=info event=scheduler_connected addr=lb:500516140worker1 # [ 213.158422] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:24Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6141worker2 # [ 213.137707] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:23Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6142lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6143worker2 # [ 213.338760] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:23Z level=info event=scheduler_connected addr=lb:500516144worker1 # [ 213.359830] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:24Z level=info event=scheduler_connected addr=lb:500516145worker2 # [ 213.342867] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6146worker1 # [ 213.364435] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6147worker1 # [ 213.541771] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:24Z level=info event=scheduler_restarting6148worker1 # [ 213.560799] nix-daemon[843]: reaped child process 1399, status = succeeded6149worker2 # [ 213.545231] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:23Z level=info event=scheduler_connected addr=lb:500516150worker2 # [ 213.548469] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6151worker1 # [ 213.575896] nix-grpc-daemon[1346]: ts=2026-09-20T10:54:24Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6152worker2 # [ 213.746193] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:23Z level=info event=scheduler_connected addr=lb:500516153worker2 # [ 213.750455] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6154worker2 # [ 213.951329] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:24Z level=info event=scheduler_connected addr=lb:500516155worker2 # [ 213.955707] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6156worker1 # [ 214.171084] nix-grpc-daemon[1346]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6157worker2 # [ 214.156885] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:24Z level=info event=scheduler_connected addr=lb:500516158worker1 # [ 214.174602] nix-grpc-daemon[1346]: E0000 00:00:1789901665.256525 1389 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true6159worker2 # [ 214.161151] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6160worker1 # [ 214.179437] nix-grpc-daemon[1346]: E0000 00:00:1789901665.256693 1389 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6161lb: must succeed: curl -sf localhost:9901/clusters6162lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6163worker1 # [ 214.358525] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6164worker1 # [ 214.360998] systemd[1]: nix-grpc-daemon.service: Consumed 884ms CPU time over 1min 58.326s wall clock time, 32.9M memory peak, 27.5K incoming IP traffic, 38.6K outgoing IP traffic.6165worker2 # [ 214.362333] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:24Z level=info event=scheduler_connected addr=lb:500516166worker2 # [ 214.366516] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6167worker2 # [ 214.568421] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:24Z level=info event=scheduler_connected addr=lb:500516168worker2 # [ 214.572090] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6169worker2 # [ 214.772991] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:25Z level=info event=scheduler_connected addr=lb:500516170worker2 # [ 214.777351] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6171worker2 # [ 214.978136] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:25Z level=info event=scheduler_connected addr=lb:500516172worker2 # [ 214.982290] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6173worker2 # [ 215.183183] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:25Z level=info event=scheduler_connected addr=lb:500516174worker2 # [ 215.187167] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6175lb: must succeed: curl -sf localhost:9901/clusters6176lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6177worker1 # [ 215.346511] systemd[1]: Starting Nix worker-protocol over gRPC...6178worker2 # [ 215.388304] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:25Z level=info event=scheduler_connected addr=lb:500516179worker2 # [ 215.392697] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6180worker1 # [ 215.527955] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:26Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6181worker1 # [ 215.531083] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:26Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16182worker1 # [ 215.536899] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:26Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6183worker1 # [ 215.538646] systemd[1]: Started Nix worker-protocol over gRPC.6184worker1 # [ 215.549946] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:26Z level=info event=scheduler_take_over6185lb # [ 215.547964] niks3-server[983]: 2026/09/20 10:54:26 INFO lead: acquired remote=192.168.1.3:490546186worker1 # [ 215.567636] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:26Z level=info event=scheduler_connected addr=lb:500516187worker1 # [ 215.570573] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:26Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06188worker1 # [ 215.612480] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:26Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06189worker2 # [ 215.594421] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:25Z level=info event=scheduler_connected addr=lb:500516190lb: must succeed: curl -sf localhost:9901/clusters6191lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6192lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 4.33 seconds)6193lb: must succeed: curl -sf localhost:9901/clusters6194lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6195worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6196worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6197(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 14.30 seconds)6198subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6199client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16200worker2 # [ 216.646525] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:446946201worker1 # [ 216.675062] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:551166202worker1 # [ 216.680568] nix-daemon[843]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)6203worker1 # [ 216.688286] nix-grpc-daemon[1576]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 93 ms (attempt 1/5)6204worker1 # [ 216.783968] nix-grpc-daemon[1576]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 140 ms (attempt 2/5)6205worker1 # [ 216.927484] nix-grpc-daemon[1576]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 370 ms (attempt 3/5)6206worker1 # [ 217.304447] nix-grpc-daemon[1576]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 616 ms (attempt 4/5)6207worker1 # [ 217.926919] nix-grpc-daemon[1576]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6208worker1 # [ 217.958152] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:29Z level=debug event=want client=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06209worker1 # [ 217.963388] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:29Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16210worker1 # [ 217.967640] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:29Z level=debug event=want client=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06211worker1 # [ 217.971424] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:29Z level=debug event=want client=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06212worker1 # [ 218.002508] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:551166213worker1 # [ 218.012908] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:551166214worker1 # [ 218.022468] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:551166215worker1 # [ 218.031950] nix-daemon[843]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)6216worker1 # [ 218.052831] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:29Z level=info event=niks3_push_started pid=16376217worker1 # [ 218.374354] nix-daemon[843]: accepted connection from pid 1650, user nix-grpc-daemon (trusted)6218worker1 # [ 218.387271] nix-daemon[843]: reaped child process 1653, status = succeeded6219worker1 # [ 218.547788] nix-daemon[843]: accepted connection from pid 1655, user nix-grpc-daemon (trusted)6220lb # [ 218.548879] envoy[919]: conn peer=[2001:db8:1::1]:56664 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9572 tx=3826 ms=56746221lb # [ 218.553174] envoy[919]: conn peer=[2001:db8:1::1]:50542 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2919 tx=1308 ms=88276222worker1 # [ 218.561935] nix-daemon[843]: reaped child process 1658, status = succeeded6223lb # [ 218.557536] envoy[919]: conn peer=[2001:db8:1::3]:34824 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=3787 tx=1035 ms=1173296224lb # [ 218.575281] niks3-server[983]: 2026/09/20 10:54:29 INFO Received uploads request method=POST path=/api/pending_closures6225worker1 # [ 218.587740] nix-grpc-daemon[1637]: time=2026-09-20T10:54:29.673Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6226worker1 # [ 218.590942] nix-grpc-daemon[1637]: time=2026-09-20T10:54:29.674Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6227lb # [ 218.615153] niks3-server[983]: 2026/09/20 10:54:29 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6228lb # [ 218.621445] niks3-server[983]: 2026/09/20 10:54:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6229lb # [ 218.625882] niks3-server[983]: 2026/09/20 10:54:29 INFO Signed narinfos id=46 count=16230worker1 # [ 218.632567] nix-grpc-daemon[1637]: time=2026-09-20T10:54:29.717Z level=INFO msg="Uploading 1 narinfos"6231lb # [ 218.639144] niks3-server[983]: 2026/09/20 10:54:29 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6232lb # [ 218.647291] niks3-server[983]: 2026/09/20 10:54:29 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6233lb # [ 218.651440] niks3-server[983]: 2026/09/20 10:54:29 INFO Completed upload id=466234worker1 # [ 218.654428] nix-grpc-daemon[1637]: time=2026-09-20T10:54:29.740Z level=INFO msg="Upload complete. (430ms)"6235lb # [ 218.653385] niks3-server[983]: 2026/09/20 10:54:29 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6236worker1 # [ 218.657675] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55116 duration_s=0 paths=1 nar_bytes_in=5046237worker1 # [ 218.664273] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:551166238worker1 # [ 218.672773] nix-daemon[843]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)6239worker1 # [ 218.677808] nix-daemon[843]: reaped child process 1635, status = succeeded6240worker1 # [ 218.683711] nix-daemon[843]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)6241worker1 # [ 219.709936] nix-daemon[843]: reaped child process 1662, status = succeeded6242worker1 # [ 219.833329] nix-daemon[843]: accepted connection from pid 1677, user nix-grpc-daemon (trusted)6243worker1 # [ 219.845885] nix-daemon[843]: reaped child process 1680, status = succeeded6244lb # [ 219.854814] niks3-server[983]: 2026/09/20 10:54:30 INFO Received uploads request method=POST path=/api/pending_closures6245worker1 # [ 219.866108] nix-grpc-daemon[1637]: time=2026-09-20T10:54:30.951Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6246worker1 # [ 219.867716] nix-grpc-daemon[1637]: time=2026-09-20T10:54:30.953Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6247lb # [ 219.887499] niks3-server[983]: 2026/09/20 10:54:30 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6248lb # [ 219.899561] niks3-server[983]: 2026/09/20 10:54:30 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6249lb # [ 219.910752] niks3-server[983]: 2026/09/20 10:54:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6250lb # [ 219.915580] niks3-server[983]: 2026/09/20 10:54:31 INFO Signed narinfos id=47 count=16251worker1 # [ 219.921774] nix-grpc-daemon[1637]: time=2026-09-20T10:54:31.006Z level=INFO msg="Uploading 1 narinfos"6252lb # [ 219.920304] niks3-server[983]: 2026/09/20 10:54:31 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6253lb # [ 219.930190] niks3-server[983]: 2026/09/20 10:54:31 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6254lb # [ 219.936341] niks3-server[983]: 2026/09/20 10:54:31 INFO Completed upload id=476255worker1 # [ 219.939602] nix-grpc-daemon[1637]: time=2026-09-20T10:54:31.025Z level=INFO msg="Upload complete. (249ms)"6256lb # [ 219.937904] niks3-server[983]: 2026/09/20 10:54:31 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6257worker1 # [ 219.951189] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55116 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=1 outputs=16258worker1 # [ 219.956390] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:31Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16259worker1 # [ 219.960171] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:31Z level=debug event=want client=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06260worker1 # [ 219.967451] nix-daemon[843]: reaped child process 1660, status = succeeded6261worker1 # [ 219.992587] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:551166262worker1 # [ 220.003658] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:551166263worker1 # [ 220.013921] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:551166264worker1 # [ 220.020941] nix-daemon[843]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)6265worker1 # [ 220.173879] nix-daemon[843]: accepted connection from pid 1686, user nix-grpc-daemon (trusted)6266worker1 # [ 220.186972] nix-daemon[843]: reaped child process 1689, status = succeeded6267worker1 # [ 220.339982] nix-daemon[843]: accepted connection from pid 1691, user nix-grpc-daemon (trusted)6268worker1 # [ 220.352827] nix-daemon[843]: reaped child process 1694, status = succeeded6269lb # [ 220.363808] niks3-server[983]: 2026/09/20 10:54:31 INFO Received uploads request method=POST path=/api/pending_closures6270worker1 # [ 220.374132] nix-grpc-daemon[1637]: time=2026-09-20T10:54:31.459Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6271worker1 # [ 220.375639] nix-grpc-daemon[1637]: time=2026-09-20T10:54:31.461Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6272lb # [ 220.390245] niks3-server[983]: 2026/09/20 10:54:31 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6273lb # [ 220.401211] niks3-server[983]: 2026/09/20 10:54:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6274lb # [ 220.405490] niks3-server[983]: 2026/09/20 10:54:31 INFO Signed narinfos id=48 count=16275worker1 # [ 220.411291] nix-grpc-daemon[1637]: time=2026-09-20T10:54:31.497Z level=INFO msg="Uploading 1 narinfos"6276lb # [ 220.413860] niks3-server[983]: 2026/09/20 10:54:31 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6277lb # [ 220.424765] niks3-server[983]: 2026/09/20 10:54:31 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6278lb # [ 220.429225] niks3-server[983]: 2026/09/20 10:54:31 INFO Completed upload id=486279worker1 # [ 220.434688] nix-grpc-daemon[1637]: time=2026-09-20T10:54:31.520Z level=INFO msg="Upload complete. (403ms)"6280lb # [ 220.434058] niks3-server[983]: 2026/09/20 10:54:31 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6281worker1 # [ 220.437622] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55116 duration_s=0 paths=1 nar_bytes_in=5046282worker1 # [ 220.445770] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:551166283worker1 # [ 220.455551] nix-daemon[843]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)6284worker1 # [ 220.457886] nix-daemon[843]: reaped child process 1684, status = succeeded6285worker1 # [ 220.463497] nix-daemon[843]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)6286worker1 # [ 221.692952] nix-daemon[843]: reaped child process 1698, status = succeeded6287worker1 # [ 221.831676] nix-daemon[843]: accepted connection from pid 1713, user nix-grpc-daemon (trusted)6288worker1 # [ 221.844835] nix-daemon[843]: reaped child process 1716, status = succeeded6289lb # [ 221.855414] niks3-server[983]: 2026/09/20 10:54:32 INFO Received uploads request method=POST path=/api/pending_closures6290worker1 # [ 221.866733] nix-grpc-daemon[1637]: time=2026-09-20T10:54:32.952Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6291worker1 # [ 221.868211] nix-grpc-daemon[1637]: time=2026-09-20T10:54:32.953Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6292lb # [ 221.884487] niks3-server[983]: 2026/09/20 10:54:32 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6293lb # [ 221.898399] niks3-server[983]: 2026/09/20 10:54:32 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6294lb # [ 221.912896] niks3-server[983]: 2026/09/20 10:54:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6295lb # [ 221.920103] niks3-server[983]: 2026/09/20 10:54:33 INFO Signed narinfos id=49 count=16296worker1 # [ 221.925393] nix-grpc-daemon[1637]: time=2026-09-20T10:54:33.011Z level=INFO msg="Uploading 1 narinfos"6297lb # [ 221.924110] niks3-server[983]: 2026/09/20 10:54:33 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6298lb # [ 221.934023] niks3-server[983]: 2026/09/20 10:54:33 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6299lb # [ 221.936906] niks3-server[983]: 2026/09/20 10:54:33 INFO Completed upload id=496300worker1 # [ 221.939933] nix-grpc-daemon[1637]: time=2026-09-20T10:54:33.025Z level=INFO msg="Upload complete. (268ms)"6301lb # [ 221.939288] niks3-server[983]: 2026/09/20 10:54:33 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6302worker1 # [ 221.953254] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55116 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=2 outputs=16303worker1 # [ 221.959224] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:33Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16304worker1 # [ 221.962730] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:551166305worker1 # [ 221.966220] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:33Z level=debug event=want client=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06306worker1 # [ 221.976957] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:551166307worker1 # [ 221.978852] nix-daemon[843]: reaped child process 1696, status = succeeded6308worker1 # [ 221.984862] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:551166309worker1 # [ 221.989353] nix-daemon[843]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)6310worker1 # [ 222.148098] nix-daemon[843]: accepted connection from pid 1722, user nix-grpc-daemon (trusted)6311worker1 # [ 222.159688] nix-daemon[843]: reaped child process 1725, status = succeeded6312worker1 # [ 222.314657] nix-daemon[843]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6313worker1 # [ 222.327635] nix-daemon[843]: reaped child process 1730, status = succeeded6314lb # [ 222.338047] niks3-server[983]: 2026/09/20 10:54:33 INFO Received uploads request method=POST path=/api/pending_closures6315worker1 # [ 222.345150] nix-grpc-daemon[1637]: time=2026-09-20T10:54:33.430Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6316worker1 # [ 222.347060] nix-grpc-daemon[1637]: time=2026-09-20T10:54:33.432Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6317lb # [ 222.354071] niks3-server[983]: 2026/09/20 10:54:33 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6318lb # [ 222.356573] niks3-server[983]: 2026/09/20 10:54:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6319lb # [ 222.359250] niks3-server[983]: 2026/09/20 10:54:33 INFO Signed narinfos id=50 count=16320worker1 # [ 222.363529] nix-grpc-daemon[1637]: time=2026-09-20T10:54:33.449Z level=INFO msg="Uploading 1 narinfos"6321lb # [ 222.362699] niks3-server[983]: 2026/09/20 10:54:33 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6322lb # [ 222.368153] niks3-server[983]: 2026/09/20 10:54:33 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6323lb # [ 222.371245] niks3-server[983]: 2026/09/20 10:54:33 INFO Completed upload id=506324worker1 # [ 222.374149] nix-grpc-daemon[1637]: time=2026-09-20T10:54:33.459Z level=INFO msg="Upload complete. (378ms)"6325worker1 # [ 222.376329] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55116 duration_s=0 paths=1 nar_bytes_in=5046326lb # [ 222.374073] niks3-server[983]: 2026/09/20 10:54:33 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6327worker1 # [ 222.379381] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:551166328worker1 # [ 222.386058] nix-daemon[843]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)6329worker1 # [ 222.389619] nix-daemon[843]: reaped child process 1720, status = succeeded6330worker1 # [ 222.394642] nix-daemon[843]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)6331worker1 # [ 223.209662] nix-daemon[843]: reaped child process 1734, status = succeeded6332worker1 # [ 223.345067] nix-daemon[843]: accepted connection from pid 1749, user nix-grpc-daemon (trusted)6333worker1 # [ 223.358227] nix-daemon[843]: reaped child process 1752, status = succeeded6334lb # [ 223.365750] niks3-server[983]: 2026/09/20 10:54:34 INFO Received uploads request method=POST path=/api/pending_closures6335worker1 # [ 223.377292] nix-grpc-daemon[1637]: time=2026-09-20T10:54:34.461Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6336worker1 # [ 223.378667] nix-grpc-daemon[1637]: time=2026-09-20T10:54:34.461Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6337lb # [ 223.404144] niks3-server[983]: 2026/09/20 10:54:34 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6338lb # [ 223.409920] niks3-server[983]: 2026/09/20 10:54:34 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv6339lb # [ 223.417663] niks3-server[983]: 2026/09/20 10:54:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6340worker1 # [ 223.426166] nix-grpc-daemon[1637]: time=2026-09-20T10:54:34.511Z level=INFO msg="Uploading 1 narinfos"6341lb # [ 223.423143] niks3-server[983]: 2026/09/20 10:54:34 INFO Signed narinfos id=51 count=16342lb # [ 223.426910] niks3-server[983]: 2026/09/20 10:54:34 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls6343lb # [ 223.435236] niks3-server[983]: 2026/09/20 10:54:34 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6344worker1 # [ 223.443228] nix-grpc-daemon[1637]: time=2026-09-20T10:54:34.528Z level=INFO msg="Upload complete. (242ms)"6345lb # [ 223.440435] niks3-server[983]: 2026/09/20 10:54:34 INFO Completed upload id=516346lb # [ 223.445411] niks3-server[983]: 2026/09/20 10:54:34 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo6347worker1 # [ 223.452688] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55116 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=3 outputs=16348worker1 # [ 223.459429] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:34Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=16349worker1 # [ 223.463345] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:551166350worker1 # [ 223.467233] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:34Z level=debug event=want client=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=06351worker1 # [ 223.474808] nix-daemon[843]: reaped child process 1732, status = succeeded6352worker1 # [ 223.476209] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:551166353worker1 # [ 223.484211] nix-grpc-daemon[1637]: time=2026-09-20T10:54:34.569Z level=INFO msg="All 1 paths already cached"6354worker1 # [ 223.488464] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:551166355worker1 # [ 223.492638] nix-daemon[843]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)6356worker1 # [ 223.642160] nix-daemon[843]: accepted connection from pid 1758, user nix-grpc-daemon (trusted)6357worker1 # [ 223.655785] nix-daemon[843]: reaped child process 1761, status = succeeded6358worker1 # [ 223.804700] nix-daemon[843]: accepted connection from pid 1763, user nix-grpc-daemon (trusted)6359worker1 # [ 223.817926] nix-daemon[843]: reaped child process 1766, status = succeeded6360lb # [ 223.826503] niks3-server[983]: 2026/09/20 10:54:34 INFO Received uploads request method=POST path=/api/pending_closures6361worker1 # [ 223.836521] nix-grpc-daemon[1637]: time=2026-09-20T10:54:34.921Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6362worker1 # [ 223.837941] nix-grpc-daemon[1637]: time=2026-09-20T10:54:34.922Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"6363lb # [ 223.851479] niks3-server[983]: 2026/09/20 10:54:34 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst6364lb # [ 223.859902] niks3-server[983]: 2026/09/20 10:54:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6365lb # [ 223.863936] niks3-server[983]: 2026/09/20 10:54:34 INFO Signed narinfos id=52 count=16366worker1 # [ 223.869808] nix-grpc-daemon[1637]: time=2026-09-20T10:54:34.955Z level=INFO msg="Uploading 1 narinfos"6367lb # [ 223.874811] niks3-server[983]: 2026/09/20 10:54:34 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls6368lb # [ 223.882779] niks3-server[983]: 2026/09/20 10:54:34 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6369worker1 # [ 223.891475] nix-grpc-daemon[1637]: time=2026-09-20T10:54:34.977Z level=INFO msg="Upload complete. (392ms)"6370lb # [ 223.888686] niks3-server[983]: 2026/09/20 10:54:34 INFO Completed upload id=526371worker1 # [ 223.893669] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55116 duration_s=0 paths=1 nar_bytes_in=6246372lb # [ 223.892239] niks3-server[983]: 2026/09/20 10:54:34 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo6373worker1 # [ 223.899201] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:551166374worker1 # [ 223.905835] nix-daemon[843]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)6375worker1 # [ 223.907293] nix-daemon[843]: reaped child process 1756, status = succeeded6376worker1 # [ 223.912741] nix-daemon[843]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)6377worker1 # [ 224.539141] nix-daemon[843]: reaped child process 1770, status = succeeded6378worker1 # [ 224.698596] nix-daemon[843]: accepted connection from pid 1785, user nix-grpc-daemon (trusted)6379worker1 # [ 224.711615] nix-daemon[843]: reaped child process 1788, status = succeeded6380lb # [ 224.723523] niks3-server[983]: 2026/09/20 10:54:35 INFO Received uploads request method=POST path=/api/pending_closures6381worker1 # [ 224.735908] nix-grpc-daemon[1637]: time=2026-09-20T10:54:35.821Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6382worker1 # [ 224.737540] nix-grpc-daemon[1637]: time=2026-09-20T10:54:35.823Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"6383lb # [ 224.763287] niks3-server[983]: 2026/09/20 10:54:35 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv6384lb # [ 224.770849] niks3-server[983]: 2026/09/20 10:54:35 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst6385lb # [ 224.783093] niks3-server[983]: 2026/09/20 10:54:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6386worker1 # [ 224.790543] nix-grpc-daemon[1637]: time=2026-09-20T10:54:35.876Z level=INFO msg="Uploading 1 narinfos"6387lb # [ 224.788087] niks3-server[983]: 2026/09/20 10:54:35 INFO Signed narinfos id=53 count=16388lb # [ 224.792465] niks3-server[983]: 2026/09/20 10:54:35 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls6389lb # [ 224.800807] niks3-server[983]: 2026/09/20 10:54:35 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6390lb # [ 224.804049] niks3-server[983]: 2026/09/20 10:54:35 INFO Completed upload id=536391lb # [ 224.806522] niks3-server[983]: 2026/09/20 10:54:35 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo6392worker1 # [ 224.807153] nix-grpc-daemon[1637]: time=2026-09-20T10:54:35.892Z level=INFO msg="Upload complete. (285ms)"6393worker1 # [ 224.819305] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55116 duration_s=0 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=4 outputs=16394worker1 # [ 224.823848] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:35Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=16395worker1 # [ 224.827259] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:551166396worker1 # [ 224.831225] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:35Z level=debug event=want client=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=06397worker1 # [ 224.840712] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:551166398worker1 # [ 224.842866] nix-daemon[843]: reaped child process 1768, status = succeeded6399worker1 # [ 224.846829] nix-grpc-daemon[1637]: time=2026-09-20T10:54:35.932Z level=INFO msg="All 1 paths already cached"6400worker1 # [ 224.849939] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:551166401worker1 # [ 224.854638] nix-daemon[843]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)6402worker1 # [ 225.014749] nix-daemon[843]: accepted connection from pid 1794, user nix-grpc-daemon (trusted)6403worker1 # [ 225.028542] nix-daemon[843]: reaped child process 1797, status = succeeded6404worker1 # [ 225.194867] nix-daemon[843]: accepted connection from pid 1799, user nix-grpc-daemon (trusted)6405worker1 # [ 225.207749] nix-daemon[843]: reaped child process 1802, status = succeeded6406lb # [ 225.217799] niks3-server[983]: 2026/09/20 10:54:36 INFO Received uploads request method=POST path=/api/pending_closures6407worker1 # [ 225.229232] nix-grpc-daemon[1637]: time=2026-09-20T10:54:36.315Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6408worker1 # [ 225.230704] nix-grpc-daemon[1637]: time=2026-09-20T10:54:36.316Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"6409lb # [ 225.244135] niks3-server[983]: 2026/09/20 10:54:36 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst6410lb # [ 225.255685] niks3-server[983]: 2026/09/20 10:54:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6411lb # [ 225.262643] niks3-server[983]: 2026/09/20 10:54:36 INFO Signed narinfos id=54 count=16412worker1 # [ 225.268928] nix-grpc-daemon[1637]: time=2026-09-20T10:54:36.354Z level=INFO msg="Uploading 1 narinfos"6413lb # [ 225.269782] niks3-server[983]: 2026/09/20 10:54:36 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls6414lb # [ 225.279739] niks3-server[983]: 2026/09/20 10:54:36 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6415lb # [ 225.286109] niks3-server[983]: 2026/09/20 10:54:36 INFO Completed upload id=546416worker1 # [ 225.288594] nix-grpc-daemon[1637]: time=2026-09-20T10:54:36.374Z level=INFO msg="Upload complete. (429ms)"6417lb # [ 225.288369] niks3-server[983]: 2026/09/20 10:54:36 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo6418worker1 # [ 225.292505] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55116 duration_s=0 paths=1 nar_bytes_in=6246419worker1 # [ 225.298539] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:551166420worker1 # [ 225.305143] nix-daemon[843]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)6421worker1 # [ 225.316942] nix-daemon[843]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)6422worker1 # [ 225.319184] nix-daemon[843]: reaped child process 1792, status = succeeded6423worker1 # [ 226.333198] nix-daemon[843]: reaped child process 1806, status = succeeded6424worker1 # [ 226.478493] nix-daemon[843]: accepted connection from pid 1821, user nix-grpc-daemon (trusted)6425worker1 # [ 226.491696] nix-daemon[843]: reaped child process 1824, status = succeeded6426lb # [ 226.503649] niks3-server[983]: 2026/09/20 10:54:37 INFO Received uploads request method=POST path=/api/pending_closures6427worker1 # [ 226.514743] nix-grpc-daemon[1637]: time=2026-09-20T10:54:37.600Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6428worker1 # [ 226.516315] nix-grpc-daemon[1637]: time=2026-09-20T10:54:37.602Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"6429lb # [ 226.543877] niks3-server[983]: 2026/09/20 10:54:37 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst6430lb # [ 226.550599] niks3-server[983]: 2026/09/20 10:54:37 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv6431lb # [ 226.561582] niks3-server[983]: 2026/09/20 10:54:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6432lb # [ 226.566950] niks3-server[983]: 2026/09/20 10:54:37 INFO Signed narinfos id=55 count=16433worker1 # [ 226.572858] nix-grpc-daemon[1637]: time=2026-09-20T10:54:37.658Z level=INFO msg="Uploading 1 narinfos"6434lb # [ 226.573720] niks3-server[983]: 2026/09/20 10:54:37 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls6435lb # [ 226.583107] niks3-server[983]: 2026/09/20 10:54:37 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6436lb # [ 226.588099] niks3-server[983]: 2026/09/20 10:54:37 INFO Completed upload id=556437lb # [ 226.592480] niks3-server[983]: 2026/09/20 10:54:37 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo6438worker1 # [ 226.591275] nix-grpc-daemon[1637]: time=2026-09-20T10:54:37.676Z level=INFO msg="Upload complete. (266ms)"6439worker1 # [ 226.601558] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55116 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=5 outputs=16440worker1 # [ 226.606698] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:37Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=16441worker1 # [ 226.612126] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:551166442worker1 # [ 226.616321] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:37Z level=debug event=want client=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=06443worker1 # [ 226.626128] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:551166444worker1 # [ 226.627857] nix-daemon[843]: reaped child process 1804, status = succeeded6445worker1 # [ 226.634456] nix-grpc-daemon[1637]: time=2026-09-20T10:54:37.720Z level=INFO msg="All 1 paths already cached"6446worker1 # [ 226.638559] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:551166447worker1 # [ 226.642662] nix-daemon[843]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)6448worker1 # [ 226.804306] nix-daemon[843]: accepted connection from pid 1830, user nix-grpc-daemon (trusted)6449worker1 # [ 226.818763] nix-daemon[843]: reaped child process 1833, status = succeeded6450worker1 # [ 226.976428] nix-daemon[843]: accepted connection from pid 1835, user nix-grpc-daemon (trusted)6451worker1 # [ 226.989222] nix-daemon[843]: reaped child process 1838, status = succeeded6452lb # [ 227.000096] niks3-server[983]: 2026/09/20 10:54:38 INFO Received uploads request method=POST path=/api/pending_closures6453worker1 # [ 227.010267] nix-grpc-daemon[1637]: time=2026-09-20T10:54:38.096Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6454worker1 # [ 227.011892] nix-grpc-daemon[1637]: time=2026-09-20T10:54:38.097Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"6455lb # [ 227.028597] niks3-server[983]: 2026/09/20 10:54:38 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst6456lb # [ 227.037114] niks3-server[983]: 2026/09/20 10:54:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6457lb # [ 227.041104] niks3-server[983]: 2026/09/20 10:54:38 INFO Signed narinfos id=56 count=16458worker1 # [ 227.046775] nix-grpc-daemon[1637]: time=2026-09-20T10:54:38.132Z level=INFO msg="Uploading 1 narinfos"6459lb # [ 227.050065] niks3-server[983]: 2026/09/20 10:54:38 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls6460lb # [ 227.062626] niks3-server[983]: 2026/09/20 10:54:38 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6461lb # [ 227.067110] niks3-server[983]: 2026/09/20 10:54:38 INFO Completed upload id=566462worker1 # [ 227.070060] nix-grpc-daemon[1637]: time=2026-09-20T10:54:38.155Z level=INFO msg="Upload complete. (421ms)"6463lb # [ 227.069572] niks3-server[983]: 2026/09/20 10:54:38 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo6464worker1 # [ 227.075164] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55116 duration_s=0 paths=1 nar_bytes_in=6246465worker1 # [ 227.080143] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:551166466worker1 # [ 227.088070] nix-daemon[843]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)6467worker1 # [ 227.092988] nix-daemon[843]: reaped child process 1828, status = succeeded6468worker1 # [ 227.098294] nix-daemon[843]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)6469worker1 # [ 228.515254] nix-daemon[843]: reaped child process 1842, status = succeeded6470worker1 # [ 228.647500] nix-daemon[843]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6471worker1 # [ 228.659253] nix-daemon[843]: reaped child process 1860, status = succeeded6472lb # [ 228.667906] niks3-server[983]: 2026/09/20 10:54:39 INFO Received uploads request method=POST path=/api/pending_closures6473worker1 # [ 228.680893] nix-grpc-daemon[1637]: time=2026-09-20T10:54:39.766Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6474worker1 # [ 228.682526] nix-grpc-daemon[1637]: time=2026-09-20T10:54:39.768Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"6475lb # [ 228.706266] niks3-server[983]: 2026/09/20 10:54:39 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv6476lb # [ 228.712607] niks3-server[983]: 2026/09/20 10:54:39 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst6477lb # [ 228.725901] niks3-server[983]: 2026/09/20 10:54:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6478worker1 # [ 228.733607] nix-grpc-daemon[1637]: time=2026-09-20T10:54:39.819Z level=INFO msg="Uploading 1 narinfos"6479lb # [ 228.730585] niks3-server[983]: 2026/09/20 10:54:39 INFO Signed narinfos id=57 count=16480lb # [ 228.736947] niks3-server[983]: 2026/09/20 10:54:39 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls6481lb # [ 228.746101] niks3-server[983]: 2026/09/20 10:54:39 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6482lb # [ 228.751764] niks3-server[983]: 2026/09/20 10:54:39 INFO Completed upload id=576483worker1 # [ 228.754970] nix-grpc-daemon[1637]: time=2026-09-20T10:54:39.840Z level=INFO msg="Upload complete. (248ms)"6484lb # [ 228.753506] niks3-server[983]: 2026/09/20 10:54:39 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo6485worker1 # [ 228.766184] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55116 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=6 outputs=16486worker1 # [ 228.772271] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:39Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=16487worker1 # [ 228.775831] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:551166488worker1 # [ 228.779202] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:39Z level=debug event=want client=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=06489worker1 # [ 228.789348] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:551166490worker1 # [ 228.791838] nix-daemon[843]: reaped child process 1840, status = succeeded6491worker1 # [ 228.797100] nix-grpc-daemon[1637]: time=2026-09-20T10:54:39.882Z level=INFO msg="All 2 paths already cached"6492worker1 # [ 228.801465] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:551166493worker1 # [ 228.805770] nix-daemon[843]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)6494worker1 # [ 228.938976] nix-daemon[843]: accepted connection from pid 1866, user nix-grpc-daemon (trusted)6495worker1 # [ 228.952056] nix-daemon[843]: reaped child process 1869, status = succeeded6496worker1 # [ 229.098142] nix-daemon[843]: accepted connection from pid 1871, user nix-grpc-daemon (trusted)6497worker1 # [ 229.110793] nix-daemon[843]: reaped child process 1874, status = succeeded6498lb # [ 229.121678] niks3-server[983]: 2026/09/20 10:54:40 INFO Received uploads request method=POST path=/api/pending_closures6499worker1 # [ 229.131792] nix-grpc-daemon[1637]: time=2026-09-20T10:54:40.217Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6500worker1 # [ 229.133357] nix-grpc-daemon[1637]: time=2026-09-20T10:54:40.219Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"6501lb # [ 229.149141] niks3-server[983]: 2026/09/20 10:54:40 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst6502lb # [ 229.159735] niks3-server[983]: 2026/09/20 10:54:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6503lb # [ 229.166346] niks3-server[983]: 2026/09/20 10:54:40 INFO Signed narinfos id=58 count=16504worker1 # [ 229.172032] nix-grpc-daemon[1637]: time=2026-09-20T10:54:40.257Z level=INFO msg="Uploading 1 narinfos"6505lb # [ 229.169839] niks3-server[983]: 2026/09/20 10:54:40 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls6506lb # [ 229.183483] niks3-server[983]: 2026/09/20 10:54:40 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6507lb # [ 229.189407] niks3-server[983]: 2026/09/20 10:54:40 INFO Completed upload id=586508worker1 # [ 229.192346] nix-grpc-daemon[1637]: time=2026-09-20T10:54:40.278Z level=INFO msg="Upload complete. (380ms)"6509lb # [ 229.193652] niks3-server[983]: 2026/09/20 10:54:40 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo6510worker1 # [ 229.196400] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55116 duration_s=0 paths=1 nar_bytes_in=7526511worker1 # [ 229.201517] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:551166512worker1 # [ 229.209490] nix-daemon[843]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)6513worker1 # [ 229.216170] nix-daemon[843]: reaped child process 1864, status = succeeded6514worker1 # [ 229.221230] nix-daemon[843]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)6515worker1 # [ 230.247381] nix-daemon[843]: reaped child process 1878, status = succeeded6516worker1 # [ 230.412286] nix-daemon[843]: accepted connection from pid 1893, user nix-grpc-daemon (trusted)6517worker1 # [ 230.425462] nix-daemon[843]: reaped child process 1896, status = succeeded6518lb # [ 230.439573] niks3-server[983]: 2026/09/20 10:54:41 INFO Received uploads request method=POST path=/api/pending_closures6519worker1 # [ 230.451332] nix-grpc-daemon[1637]: time=2026-09-20T10:54:41.535Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6520worker1 # [ 230.452732] nix-grpc-daemon[1637]: time=2026-09-20T10:54:41.535Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"6521lb # [ 230.479539] niks3-server[983]: 2026/09/20 10:54:41 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst6522lb # [ 230.486275] niks3-server[983]: 2026/09/20 10:54:41 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv6523lb # [ 230.495152] niks3-server[983]: 2026/09/20 10:54:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6524lb # [ 230.501122] niks3-server[983]: 2026/09/20 10:54:41 INFO Signed narinfos id=59 count=16525worker1 # [ 230.504385] nix-grpc-daemon[1637]: time=2026-09-20T10:54:41.589Z level=INFO msg="Uploading 1 narinfos"6526lb # [ 230.504782] niks3-server[983]: 2026/09/20 10:54:41 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls6527lb # [ 230.514097] niks3-server[983]: 2026/09/20 10:54:41 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6528lb # [ 230.519392] niks3-server[983]: 2026/09/20 10:54:41 INFO Completed upload id=596529worker1 # [ 230.521847] nix-grpc-daemon[1637]: time=2026-09-20T10:54:41.607Z level=INFO msg="Upload complete. (291ms)"6530lb # [ 230.524214] niks3-server[983]: 2026/09/20 10:54:41 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo6531worker1 # [ 230.527946] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55116 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=7 outputs=16532worker1 # [ 230.536376] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:41Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=16533worker1 # [ 230.539936] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:551166534worker1 # [ 230.550204] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:551166535worker1 # [ 230.554701] nix-daemon[843]: reaped child process 1876, status = succeeded6536worker1 # [ 230.559305] nix-grpc-daemon[1637]: time=2026-09-20T10:54:41.645Z level=INFO msg="All 2 paths already cached"6537worker1 # [ 230.563065] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:551166538worker1 # [ 230.567146] nix-daemon[843]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)6539worker1 # [ 230.732695] nix-daemon[843]: accepted connection from pid 1900, user nix-grpc-daemon (trusted)6540worker1 # [ 230.750585] nix-daemon[843]: reaped child process 1903, status = succeeded6541worker1 # [ 230.904740] nix-daemon[843]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6542worker1 # [ 230.917419] nix-daemon[843]: reaped child process 1908, status = succeeded6543lb # [ 230.930891] niks3-server[983]: 2026/09/20 10:54:42 INFO Received uploads request method=POST path=/api/pending_closures6544worker1 # [ 230.941919] nix-grpc-daemon[1637]: time=2026-09-20T10:54:42.027Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6545worker1 # [ 230.943575] nix-grpc-daemon[1637]: time=2026-09-20T10:54:42.029Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"6546lb # [ 230.961248] niks3-server[983]: 2026/09/20 10:54:42 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst6547lb # [ 230.972424] niks3-server[983]: 2026/09/20 10:54:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6548worker1 # [ 230.979792] nix-grpc-daemon[1637]: time=2026-09-20T10:54:42.065Z level=INFO msg="Uploading 1 narinfos"6549lb # [ 230.978154] niks3-server[983]: 2026/09/20 10:54:42 INFO Signed narinfos id=60 count=16550lb # [ 230.983206] niks3-server[983]: 2026/09/20 10:54:42 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls6551lb # [ 230.995346] niks3-server[983]: 2026/09/20 10:54:42 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6552worker1 # [ 231.003590] nix-grpc-daemon[1637]: time=2026-09-20T10:54:42.089Z level=INFO msg="Upload complete. (430ms)"6553lb # [ 231.000878] niks3-server[983]: 2026/09/20 10:54:42 INFO Completed upload id=606554lb # [ 231.003898] niks3-server[983]: 2026/09/20 10:54:42 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo6555worker1 # [ 231.006167] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55116 duration_s=0 paths=1 nar_bytes_in=7526556worker1 # [ 231.011216] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:551166557worker1 # [ 231.020925] nix-daemon[843]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)6558worker1 # [ 231.025128] nix-daemon[843]: reaped child process 1898, status = succeeded6559worker1 # [ 231.030056] nix-daemon[843]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)6560worker1 # [ 232.645965] nix-daemon[843]: reaped child process 1912, status = succeeded6561worker1 # [ 232.781297] nix-daemon[843]: accepted connection from pid 1927, user nix-grpc-daemon (trusted)6562worker1 # [ 232.794194] nix-daemon[843]: reaped child process 1930, status = succeeded6563lb # [ 232.806567] niks3-server[983]: 2026/09/20 10:54:43 INFO Received uploads request method=POST path=/api/pending_closures6564worker1 # [ 232.813586] nix-grpc-daemon[1637]: time=2026-09-20T10:54:43.899Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6565worker1 # [ 232.815096] nix-grpc-daemon[1637]: time=2026-09-20T10:54:43.900Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"6566lb # [ 232.824551] niks3-server[983]: 2026/09/20 10:54:43 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst6567lb # [ 232.827225] niks3-server[983]: 2026/09/20 10:54:43 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv6568lb # [ 232.831114] niks3-server[983]: 2026/09/20 10:54:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6569lb # [ 232.832721] niks3-server[983]: 2026/09/20 10:54:43 INFO Signed narinfos id=61 count=16570worker1 # [ 232.835907] nix-grpc-daemon[1637]: time=2026-09-20T10:54:43.921Z level=INFO msg="Uploading 1 narinfos"6571lb # [ 232.837887] niks3-server[983]: 2026/09/20 10:54:43 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls6572lb # [ 232.841081] niks3-server[983]: 2026/09/20 10:54:43 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6573lb # [ 232.844439] niks3-server[983]: 2026/09/20 10:54:43 INFO Completed upload id=616574worker1 # [ 232.847291] nix-grpc-daemon[1637]: time=2026-09-20T10:54:43.933Z level=INFO msg="Upload complete. (209ms)"6575lb # [ 232.845537] niks3-server[983]: 2026/09/20 10:54:43 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo6576worker1 # [ 232.849617] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55116 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=8 outputs=16577worker1 # [ 232.853659] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:43Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=16578worker1 # [ 232.855904] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:551166579worker1 # [ 232.862313] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:551166580worker1 # [ 232.864891] nix-daemon[843]: reaped child process 1910, status = succeeded6581worker1 # [ 232.868479] nix-grpc-daemon[1637]: time=2026-09-20T10:54:43.954Z level=INFO msg="All 2 paths already cached"6582worker1 # [ 232.871949] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:551166583worker1 # [ 232.875998] nix-daemon[843]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)6584worker1 # [ 233.031208] nix-daemon[843]: accepted connection from pid 1934, user nix-grpc-daemon (trusted)6585worker1 # [ 233.044419] nix-daemon[843]: reaped child process 1937, status = succeeded6586worker1 # [ 233.194806] nix-daemon[843]: accepted connection from pid 1939, user nix-grpc-daemon (trusted)6587worker1 # [ 233.207520] nix-daemon[843]: reaped child process 1942, status = succeeded6588lb # [ 233.220220] niks3-server[983]: 2026/09/20 10:54:44 INFO Received uploads request method=POST path=/api/pending_closures6589worker1 # [ 233.230822] nix-grpc-daemon[1637]: time=2026-09-20T10:54:44.316Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6590worker1 # [ 233.232384] nix-grpc-daemon[1637]: time=2026-09-20T10:54:44.318Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"6591lb # [ 233.247223] niks3-server[983]: 2026/09/20 10:54:44 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst6592lb # [ 233.258379] niks3-server[983]: 2026/09/20 10:54:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6593worker1 # [ 233.266027] nix-grpc-daemon[1637]: time=2026-09-20T10:54:44.351Z level=INFO msg="Uploading 1 narinfos"6594lb # [ 233.263605] niks3-server[983]: 2026/09/20 10:54:44 INFO Signed narinfos id=62 count=16595lb # [ 233.269806] niks3-server[983]: 2026/09/20 10:54:44 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls6596lb # [ 233.278788] niks3-server[983]: 2026/09/20 10:54:44 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6597worker1 # [ 233.287957] nix-grpc-daemon[1637]: time=2026-09-20T10:54:44.373Z level=INFO msg="Upload complete. (406ms)"6598lb # [ 233.285134] niks3-server[983]: 2026/09/20 10:54:44 INFO Completed upload id=626599worker1 # [ 233.290567] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55116 duration_s=0 paths=1 nar_bytes_in=7526600lb # [ 233.289575] niks3-server[983]: 2026/09/20 10:54:44 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo6601worker1 # [ 233.296474] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:551166602worker1 # [ 233.303726] nix-daemon[843]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)6603worker1 # [ 233.307527] nix-daemon[843]: reaped child process 1932, status = succeeded6604worker1 # [ 233.314121] nix-daemon[843]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)6605worker1 # [ 234.341959] nix-daemon[843]: reaped child process 1946, status = succeeded6606worker1 # [ 234.511443] nix-daemon[843]: accepted connection from pid 1961, user nix-grpc-daemon (trusted)6607worker1 # [ 234.524399] nix-daemon[843]: reaped child process 1964, status = succeeded6608lb # [ 234.539396] niks3-server[983]: 2026/09/20 10:54:45 INFO Received uploads request method=POST path=/api/pending_closures6609worker1 # [ 234.552077] nix-grpc-daemon[1637]: time=2026-09-20T10:54:45.637Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6610worker1 # [ 234.553660] nix-grpc-daemon[1637]: time=2026-09-20T10:54:45.639Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"6611lb # [ 234.575295] niks3-server[983]: 2026/09/20 10:54:45 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv6612lb # [ 234.592120] niks3-server[983]: 2026/09/20 10:54:45 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst6613lb # [ 234.600422] niks3-server[983]: 2026/09/20 10:54:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6614lb # [ 234.605246] niks3-server[983]: 2026/09/20 10:54:45 INFO Signed narinfos id=63 count=16615worker1 # [ 234.611093] nix-grpc-daemon[1637]: time=2026-09-20T10:54:45.696Z level=INFO msg="Uploading 1 narinfos"6616lb # [ 234.610350] niks3-server[983]: 2026/09/20 10:54:45 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls6617lb # [ 234.619348] niks3-server[983]: 2026/09/20 10:54:45 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6618lb # [ 234.623838] niks3-server[983]: 2026/09/20 10:54:45 INFO Completed upload id=636619worker1 # [ 234.628983] nix-grpc-daemon[1637]: time=2026-09-20T10:54:45.714Z level=INFO msg="Upload complete. (305ms)"6620worker1 # [ 234.634959] nix-grpc-daemon[1637]: time=2026-09-20T10:54:45.715Z level=WARN msg="Failed to register uploaded object" key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo error="server returned 404: object is not pending upload\n"6621worker1 # [ 234.640961] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55116 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=9 outputs=16622worker1 # [ 234.654140] nix-daemon[843]: reaped child process 1944, status = succeeded6623client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 18.18 seconds)6624worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6625worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6626client: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&16627worker1 # [ 234.915264] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:46Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:551166628worker1 # [ 234.922167] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:551166629worker1 # [ 234.938149] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:46Z level=debug event=want client=2 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=06630client: (finished: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&1, in 0.22 seconds)6631(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 18.46 seconds)6632subtest: low disk drains a worker and builds go to the other6633worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6634worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.99 seconds)6635worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6636worker1 # [ 236.549565] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:47Z level=info event=unhealthy reason=min_free6637worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.11 seconds)6638client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&26639worker1 # [ 237.286970] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:48Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:551166640client # don't know how to build these paths:6641client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6642worker1 # [ 237.295818] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:551166643worker1 # [ 237.322219] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:48Z level=debug event=want client=3 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=06644worker1 # [ 237.326504] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:48Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=16645worker2 # [ 237.349918] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446946646worker2 # [ 237.362881] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446946647client # copying 1 paths to 'lb:50051'...6648client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...6649worker2 # [ 237.375846] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:446946650worker2 # [ 237.387867] nix-daemon[839]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6651worker2 # [ 237.553982] nix-daemon[839]: accepted connection from pid 1726, user nix-grpc-daemon (trusted)6652worker2 # [ 237.566752] nix-daemon[839]: reaped child process 1729, status = succeeded6653worker2 # [ 237.699588] nix-daemon[839]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)6654worker2 # [ 237.712227] nix-daemon[839]: reaped child process 1734, status = succeeded6655lb # [ 237.738597] niks3-server[983]: 2026/09/20 10:54:48 INFO Received uploads request method=POST path=/api/pending_closures6656worker2 # [ 237.727272] nix-grpc-daemon[1540]: time=2026-09-20T10:54:47.959Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6657worker2 # [ 237.728902] nix-grpc-daemon[1540]: time=2026-09-20T10:54:47.960Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"6658lb # [ 237.763501] niks3-server[983]: 2026/09/20 10:54:48 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst6659lb # [ 237.777252] niks3-server[983]: 2026/09/20 10:54:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6660lb # [ 237.788070] niks3-server[983]: 2026/09/20 10:54:48 INFO Signed narinfos id=64 count=16661worker2 # [ 237.775388] nix-grpc-daemon[1540]: time=2026-09-20T10:54:48.007Z level=INFO msg="Uploading 1 narinfos"6662lb # [ 237.796886] niks3-server[983]: 2026/09/20 10:54:48 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls6663lb # [ 237.806962] niks3-server[983]: 2026/09/20 10:54:48 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6664lb # [ 237.812558] niks3-server[983]: 2026/09/20 10:54:48 INFO Completed upload id=646665worker2 # [ 237.797066] nix-grpc-daemon[1540]: time=2026-09-20T10:54:48.028Z level=INFO msg="Upload complete. (398ms)"6666lb # [ 237.816931] niks3-server[983]: 2026/09/20 10:54:48 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo6667worker2 # [ 237.801223] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44694 duration_s=0 paths=1 nar_bytes_in=5126668worker2 # [ 237.806456] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446946669worker2 # [ 237.815869] nix-daemon[839]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6670worker2 # [ 237.823317] nix-daemon[839]: reaped child process 1724, status = succeeded6671worker2 # [ 237.828310] nix-daemon[839]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6672client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...6673client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6674client # farm-a> LOG-farm-a-drain6675lb # [ 238.550825] envoy[919]: conn peer=[2001:db8:1::1]:46304 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27089 tx=10709 ms=179936676lb # [ 238.555290] envoy[919]: conn peer=[2001:db8:1::1]:46494 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=878 tx=480 ms=366677worker2 # [ 238.644529] nix-daemon[839]: reaped child process 1738, status = succeeded6678worker2 # [ 238.785634] nix-daemon[839]: accepted connection from pid 1753, user nix-grpc-daemon (trusted)6679worker2 # [ 238.798495] nix-daemon[839]: reaped child process 1756, status = succeeded6680lb # [ 238.831461] niks3-server[983]: 2026/09/20 10:54:49 INFO Received uploads request method=POST path=/api/pending_closures6681worker2 # [ 238.823295] nix-grpc-daemon[1540]: time=2026-09-20T10:54:49.054Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6682worker2 # [ 238.824693] nix-grpc-daemon[1540]: time=2026-09-20T10:54:49.054Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"6683lb # [ 238.869329] niks3-server[983]: 2026/09/20 10:54:49 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6684lb # [ 238.873934] niks3-server[983]: 2026/09/20 10:54:49 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6685lb # [ 238.882464] niks3-server[983]: 2026/09/20 10:54:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6686worker2 # [ 238.871200] nix-grpc-daemon[1540]: time=2026-09-20T10:54:49.103Z level=INFO msg="Uploading 1 narinfos"6687lb # [ 238.887114] niks3-server[983]: 2026/09/20 10:54:49 INFO Signed narinfos id=65 count=16688lb # [ 238.890440] niks3-server[983]: 2026/09/20 10:54:49 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls6689lb # [ 238.898153] niks3-server[983]: 2026/09/20 10:54:49 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6690lb # [ 238.901932] niks3-server[983]: 2026/09/20 10:54:49 INFO Completed upload id=656691worker2 # [ 238.886458] nix-grpc-daemon[1540]: time=2026-09-20T10:54:49.118Z level=INFO msg="Upload complete. (251ms)"6692lb # [ 238.903501] niks3-server[983]: 2026/09/20 10:54:49 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo6693worker2 # [ 238.891498] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44694 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=10 outputs=16694worker2 # [ 238.902507] nix-daemon[839]: reaped child process 1736, status = succeeded6695worker1 # [ 238.958891] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:50Z level=debug event=want client=3 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=06696worker1 # [ 238.963530] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:50Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=16697worker2 # [ 238.984440] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446946698worker2 # [ 238.996587] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446946699worker2 # [ 239.009084] nix-grpc-daemon[1540]: time=2026-09-20T10:54:49.240Z level=INFO msg="All 1 paths already cached"6700client # copying 1 paths to 'lb:50051'...6701client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...6702worker2 # [ 239.016837] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:446946703worker2 # [ 239.025310] nix-daemon[839]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6704worker2 # [ 239.177043] nix-daemon[839]: accepted connection from pid 1760, user nix-grpc-daemon (trusted)6705worker2 # [ 239.189341] nix-daemon[839]: reaped child process 1763, status = succeeded6706worker2 # [ 239.338280] nix-daemon[839]: accepted connection from pid 1765, user nix-grpc-daemon (trusted)6707worker2 # [ 239.350566] nix-daemon[839]: reaped child process 1768, status = succeeded6708lb # [ 239.381560] niks3-server[983]: 2026/09/20 10:54:50 INFO Received uploads request method=POST path=/api/pending_closures6709worker2 # [ 239.373867] nix-grpc-daemon[1540]: time=2026-09-20T10:54:49.605Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6710worker2 # [ 239.375354] nix-grpc-daemon[1540]: time=2026-09-20T10:54:49.607Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"6711lb # [ 239.409785] niks3-server[983]: 2026/09/20 10:54:50 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst6712lb # [ 239.421465] niks3-server[983]: 2026/09/20 10:54:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6713worker2 # [ 239.410784] nix-grpc-daemon[1540]: time=2026-09-20T10:54:49.642Z level=INFO msg="Uploading 1 narinfos"6714lb # [ 239.426932] niks3-server[983]: 2026/09/20 10:54:50 INFO Signed narinfos id=66 count=16715lb # [ 239.433528] niks3-server[983]: 2026/09/20 10:54:50 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls6716lb # [ 239.444142] niks3-server[983]: 2026/09/20 10:54:50 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6717worker2 # [ 239.435089] nix-grpc-daemon[1540]: time=2026-09-20T10:54:49.667Z level=INFO msg="Upload complete. (395ms)"6718lb # [ 239.450689] niks3-server[983]: 2026/09/20 10:54:50 INFO Completed upload id=666719worker2 # [ 239.437162] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44694 duration_s=0 paths=1 nar_bytes_in=6246720lb # [ 239.454163] niks3-server[983]: 2026/09/20 10:54:50 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo6721worker2 # [ 239.442888] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446946722worker2 # [ 239.453415] nix-daemon[839]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6723worker2 # [ 239.456222] nix-daemon[839]: reaped child process 1758, status = succeeded6724worker2 # [ 239.461051] nix-daemon[839]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6725client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...6726client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6727worker2 # [ 240.475255] nix-daemon[839]: reaped child process 1772, status = succeeded6728client # farm-b> LOG-farm-b-drain6729worker2 # [ 240.603706] nix-daemon[839]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)6730worker2 # [ 240.615530] nix-daemon[839]: reaped child process 1790, status = succeeded6731lb # [ 240.643673] niks3-server[983]: 2026/09/20 10:54:51 INFO Received uploads request method=POST path=/api/pending_closures6732worker2 # [ 240.636471] nix-grpc-daemon[1540]: time=2026-09-20T10:54:50.868Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6733worker2 # [ 240.638031] nix-grpc-daemon[1540]: time=2026-09-20T10:54:50.870Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"6734lb # [ 240.681993] niks3-server[983]: 2026/09/20 10:54:51 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst6735lb # [ 240.692457] niks3-server[983]: 2026/09/20 10:54:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6736lb # [ 240.700284] niks3-server[983]: 2026/09/20 10:54:51 INFO Signed narinfos id=67 count=16737worker2 # [ 240.687541] nix-grpc-daemon[1540]: time=2026-09-20T10:54:50.919Z level=INFO msg="Uploading 1 narinfos"6738lb # [ 240.707712] niks3-server[983]: 2026/09/20 10:54:51 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6739lb # [ 240.724047] niks3-server[983]: 2026/09/20 10:54:51 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6740lb # [ 240.727613] niks3-server[983]: 2026/09/20 10:54:51 INFO Completed upload id=676741worker2 # [ 240.712080] nix-grpc-daemon[1540]: time=2026-09-20T10:54:50.943Z level=INFO msg="Upload complete. (244ms)"6742lb # [ 240.729453] niks3-server[983]: 2026/09/20 10:54:51 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo6743lb # [ 240.733589] niks3-server[983]: 2026/09/20 10:54:51 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls6744worker2 # [ 240.716924] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44694 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=11 outputs=16745worker1 # [ 240.742816] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:51Z level=debug event=want client=3 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=06746worker2 # [ 240.729605] nix-daemon[839]: reaped child process 1770, status = succeeded6747worker1 # [ 240.747409] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:51Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=16748worker2 # [ 240.773061] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446946749worker2 # [ 240.785936] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446946750worker2 # [ 240.800075] nix-grpc-daemon[1540]: time=2026-09-20T10:54:51.031Z level=INFO msg="All 2 paths already cached"6751client # copying 1 paths to 'lb:50051'...6752client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...6753worker2 # [ 240.808132] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:446946754worker2 # [ 240.819129] nix-daemon[839]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6755worker2 # [ 240.958526] nix-daemon[839]: accepted connection from pid 1794, user nix-grpc-daemon (trusted)6756worker2 # [ 240.971728] nix-daemon[839]: reaped child process 1797, status = succeeded6757worker2 # [ 241.099666] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted)6758worker2 # [ 241.112522] nix-daemon[839]: reaped child process 1802, status = succeeded6759lb # [ 241.139697] niks3-server[983]: 2026/09/20 10:54:52 INFO Received uploads request method=POST path=/api/pending_closures6760worker2 # [ 241.134553] nix-grpc-daemon[1540]: time=2026-09-20T10:54:51.366Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6761worker2 # [ 241.136286] nix-grpc-daemon[1540]: time=2026-09-20T10:54:51.368Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"6762lb # [ 241.172109] niks3-server[983]: 2026/09/20 10:54:52 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst6763lb # [ 241.183109] niks3-server[983]: 2026/09/20 10:54:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6764worker2 # [ 241.173235] nix-grpc-daemon[1540]: time=2026-09-20T10:54:51.405Z level=INFO msg="Uploading 1 narinfos"6765lb # [ 241.188912] niks3-server[983]: 2026/09/20 10:54:52 INFO Signed narinfos id=68 count=16766lb # [ 241.194640] niks3-server[983]: 2026/09/20 10:54:52 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls6767lb # [ 241.206042] niks3-server[983]: 2026/09/20 10:54:52 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6768lb # [ 241.213422] niks3-server[983]: 2026/09/20 10:54:52 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo6769lb # [ 241.218642] niks3-server[983]: 2026/09/20 10:54:52 INFO Completed upload id=686770worker2 # [ 241.203230] nix-grpc-daemon[1540]: time=2026-09-20T10:54:51.435Z level=INFO msg="Upload complete. (371ms)"6771worker2 # [ 241.208128] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44694 duration_s=0 paths=1 nar_bytes_in=7526772worker2 # [ 241.213117] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446946773worker2 # [ 241.222113] nix-daemon[839]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6774worker2 # [ 241.230740] nix-daemon[839]: reaped child process 1792, status = succeeded6775worker2 # [ 241.236075] nix-daemon[839]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6776client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...6777client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6778client # farm-top> LOG-farm-top-drain6779worker2 # [ 242.050942] nix-daemon[839]: reaped child process 1806, status = succeeded6780worker2 # [ 242.175534] nix-daemon[839]: accepted connection from pid 1821, user nix-grpc-daemon (trusted)6781worker2 # [ 242.187417] nix-daemon[839]: reaped child process 1824, status = succeeded6782lb # [ 242.215318] niks3-server[983]: 2026/09/20 10:54:53 INFO Received uploads request method=POST path=/api/pending_closures6783worker2 # [ 242.208722] nix-grpc-daemon[1540]: time=2026-09-20T10:54:52.440Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6784worker2 # [ 242.210205] nix-grpc-daemon[1540]: time=2026-09-20T10:54:52.442Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"6785lb # [ 242.249532] niks3-server[983]: 2026/09/20 10:54:53 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6786lb # [ 242.260922] niks3-server[983]: 2026/09/20 10:54:53 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst6787lb # [ 242.274139] niks3-server[983]: 2026/09/20 10:54:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6788worker2 # [ 242.262997] nix-grpc-daemon[1540]: time=2026-09-20T10:54:52.494Z level=INFO msg="Uploading 1 narinfos"6789lb # [ 242.278576] niks3-server[983]: 2026/09/20 10:54:53 INFO Signed narinfos id=69 count=16790lb # [ 242.284889] niks3-server[983]: 2026/09/20 10:54:53 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls6791lb # [ 242.294214] niks3-server[983]: 2026/09/20 10:54:53 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6792lb # [ 242.299184] niks3-server[983]: 2026/09/20 10:54:53 INFO Completed upload id=696793lb # [ 242.301922] niks3-server[983]: 2026/09/20 10:54:53 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo6794worker2 # [ 242.285069] nix-grpc-daemon[1540]: time=2026-09-20T10:54:52.517Z level=INFO msg="Upload complete. (242ms)"6795worker2 # [ 242.292213] nix-grpc-daemon[1472]: ts=2026-09-20T10:54:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44694 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=12 outputs=16796worker2 # [ 242.304752] nix-daemon[839]: reaped child process 1804, status = succeeded6797client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&2, in 5.28 seconds)6798client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath6799client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath, in 0.19 seconds)6800worker1: must succeed: rm /nix/.rw-store/fill6801worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.12 seconds)6802worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6803worker1 # [ 243.557777] nix-grpc-daemon[1576]: ts=2026-09-20T10:54:54Z level=info event=healthy reason=min_free6804worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.11 seconds)6805(finished: subtest: low disk drains a worker and builds go to the other, in 8.87 seconds)6806(finished: run the VM test script, in 244.66 seconds)6807test script finished in 244.92s6808cleanup6809kill QemuMachine (pid 49)6810client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6811kill QemuMachine (pid 50)6812lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6813kill QemuMachine (pid 51)6814worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6815kill QemuMachine (pid 52)6816worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6817(finished: cleanup, in 0.92 seconds)6818additionally exposed symbols:6819 client, lb, worker1, worker2,6820 vlan1,6821 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh

succeeded vm-test-run-nix-grpc-farm checks.x86_64-linux.farm · build #299 · raw

succeeded `vm-test-run-nix-grpc-farm` `checks.x86_64-linux.farm` · build #299 · raw