vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #421
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12lb # Disk image does not exist, creating the virtualisation disk image...13lb # Formatting '/build/vm-state-lb/tmp.kMgzvRbfMf', fmt=raw size=107374182414lb # mke2fs 1.47.4 (6-Mar-2025)15worker1 # Disk image does not exist, creating the virtualisation disk image...16lb # Discarding device blocks: 0/262144 done17worker1: QEMU running (pid 51)18worker1 # Formatting '/build/vm-state-worker1/tmp.fmphpR5zm6', fmt=raw size=107374182419lb # Creating filesystem with 262144 4k blocks and 65536 inodes20client # Disk image does not exist, creating the virtualisation disk image...21worker1 # mke2fs 1.47.4 (6-Mar-2025)22lb # Filesystem UUID: 6ea26525-96f5-4c70-9540-848bdd2b4ba923client # Formatting '/build/vm-state-client/tmp.IHLWAce3hT', fmt=raw size=107374182424worker1 # Discarding device blocks: 0/262144 done25lb # Superblock backups stored on blocks:26lb # 32768, 98304, 163840, 22937627lb # 28worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes29client: QEMU running (pid 50)30lb: QEMU running (pid 49)31lb # Allocating group tables: 0/8 done32client # mke2fs 1.47.4 (6-Mar-2025)33lb # Writing inode tables: 0/8 done34client # Discarding device blocks: 0/262144 done35lb # Creating journal (8192 blocks): done36client # Creating filesystem with 262144 4k blocks and 65536 inodes37lb # Writing superblocks and filesystem accounting information: 0/8 done38client # Filesystem UUID: 2c4627c9-ceca-4138-98e7-ebe4e979e6f139lb # 40client # Superblock backups stored on blocks:41lb # Virtualisation disk image created.42client # 32768, 98304, 163840, 22937643worker1 # Filesystem UUID: 88d53f30-4e1b-4b3d-ab04-1566fa41b8ac44client # 45worker1 # Superblock backups stored on blocks:46client # Allocating group tables: 0/8 done47worker1 # 32768, 98304, 163840, 22937648client # Writing inode tables: 0/8 done49worker1 # 50worker1 # Allocating group tables: 0/8 done51worker1 # Writing inode tables: 0/8 done52worker1 # Creating journal (8192 blocks): done53worker1 # Writing superblocks and filesystem accounting information: 0/8 done54worker1 # 55worker1 # Virtualisation disk image created.56client # Creating journal (8192 blocks): done57client # Writing superblocks and filesystem accounting information: 0/8 done58client # 59client # Virtualisation disk image created.60worker2 # Disk image does not exist, creating the virtualisation disk image...61worker2: QEMU running (pid 52)62worker2 # Formatting '/build/vm-state-worker2/tmp.EYMoUS71TN', fmt=raw size=107374182463worker2 # mke2fs 1.47.4 (6-Mar-2025)64worker2 # Discarding device blocks: 0/262144 done65worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes66worker2 # Filesystem UUID: 2d11e159-5af1-4dcc-9925-7ecad8b3bb7667(finished: start all VMs, in 0.27 seconds)68worker2 # Superblock backups stored on blocks:69worker2 # 32768, 98304, 163840, 22937670worker2 # 71client: waiting for unit network-addresses-eth1.service72worker2 # Allocating group tables: 0/8 done73client: waiting for the VM to finish booting74worker2 # Writing inode tables: 0/8 done75worker2 # Creating journal (8192 blocks): done76worker2 # Writing superblocks and filesystem accounting information: 0/8 done77worker2 # 78worker2 # Virtualisation disk image created.79lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)80worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)81client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)82worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)83lb # 84lb # 85worker1 # 86worker1 # 87lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0088worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0089lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...90lb # 91lb # 92lb # 93lb # 94worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...95worker1 # 96worker1 # 97lb # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB0098worker1 # 99worker1 # 100worker1 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00101client # 102client # 103client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00104lb # Press Ctrl-B to configure iPXE (PCI 00:08.0)...105lb # 106lb # 107worker1 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...108worker1 # 109worker1 # 110client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...111client # 112client # 113client # 114client # 115client # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00116client # Press Ctrl-B to configure iPXE (PCI 00:08.0)...117client # 118client # 119worker2 # 120worker2 # 121worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00122worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...123worker2 # 124worker2 # 125worker2 # 126worker2 # 127worker2 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00128worker2 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...129worker2 # 130worker2 # 131worker1 # Booting from ROM...132lb # Booting from ROM...133client # Booting from ROM...134worker2 # Booting from ROM...135worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026136lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026137worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/qmcgfpcw2i7m6pmgb5saqy8wfr82vfq3-nixos-system-worker1-test/init regInfo=/nix/store/q5s318i3gaswdpaw2h8ilxrpz9w65pb1-closure-info/registration console=ttyS0,115200n8 console=tty0138worker1 # [ 0.000000] BIOS-provided physical RAM map:139lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0qln5mjy767qjp5wbmxjg8jay8k05dsm-nixos-system-lb-test/init regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration console=ttyS0,115200n8 console=tty0140worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable141lb # [ 0.000000] BIOS-provided physical RAM map:142worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved143lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable144worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved145lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved146worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable147lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved148worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved149lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable150worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved151lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved152worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved153lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved154worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved155lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved156worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved157lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved158worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved159lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved160worker1 # [ 0.000000] NX (Execute Disable) protection: active161worker1 # [ 0.000000] APIC: Static calls initialized162lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved163worker1 # [ 0.000000] SMBIOS 2.8 present.164lb # [ 0.000000] NX (Execute Disable) protection: active165lb # [ 0.000000] APIC: Static calls initialized166lb # [ 0.000000] SMBIOS 2.8 present.167worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014168worker1 # [ 0.000000] DMI: Memory slots populated: 1/1169worker1 # [ 0.000000] Hypervisor detected: KVM170lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014171worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000172lb # [ 0.000000] DMI: Memory slots populated: 1/1173lb # [ 0.000000] Hypervisor detected: KVM174worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00175lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000176worker1 # [ 0.000000] kvm-clock: using sched offset of 455318733 cycles177lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00178worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns179lb # [ 0.000000] kvm-clock: using sched offset of 458377788 cycles180worker1 # [ 0.000006] tsc: Detected 2400.012 MHz processor181worker1 # [ 0.000812] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000182lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns183lb # [ 0.000005] tsc: Detected 2400.012 MHz processor184client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026185worker1 # [ 0.000839] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs186lb # [ 0.000805] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000187worker1 # [ 0.000842] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT188lb # [ 0.000832] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs189worker1 # [ 0.002742] found SMP MP-table at [mem 0x000f5450-0x000f545f]190worker1 # [ 0.002754] Using GB pages for direct mapping191lb # [ 0.000835] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT192worker1 # [ 0.002867] RAMDISK: [mem 0x5e367000-0x5ffcffff]193lb # [ 0.002737] found SMP MP-table at [mem 0x000f5450-0x000f545f]194worker1 # [ 0.002877] ACPI: Early table checksum verification disabled195lb # [ 0.002748] Using GB pages for direct mapping196client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/jwg6hsvasqr1bfb4p1nn6ayia3202nfr-nixos-system-client-test/init regInfo=/nix/store/ycwmfabcj5bpk7yq23lg5g1pj9l5i2yl-closure-info/registration console=ttyS0,115200n8 console=tty0197lb # [ 0.002846] RAMDISK: [mem 0x5e367000-0x5ffcffff]198worker1 # [ 0.002880] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )199client # [ 0.000000] BIOS-provided physical RAM map:200lb # [ 0.002854] ACPI: Early table checksum verification disabled201client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable202worker1 # [ 0.002884] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)203lb # [ 0.002857] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )204client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved205worker1 # [ 0.002889] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)206lb # [ 0.002862] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)207client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved208worker1 # [ 0.002897] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)209lb # [ 0.002866] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)210client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable211worker1 # [ 0.002899] ACPI: FACS 0x000000005FFE0000 000040212lb # [ 0.002873] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)213client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved214worker1 # [ 0.002900] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)215lb # [ 0.002875] ACPI: FACS 0x000000005FFE0000 000040216client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved217worker1 # [ 0.002902] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)218lb # [ 0.002877] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)219client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved220worker1 # [ 0.002903] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)221client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved222lb # [ 0.002878] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)223client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved224worker1 # [ 0.002905] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)225lb # [ 0.002880] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)226client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved227worker1 # [ 0.002906] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]228lb # [ 0.002881] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)229client # [ 0.000000] NX (Execute Disable) protection: active230worker1 # [ 0.002907] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]231client # [ 0.000000] APIC: Static calls initialized232lb # [ 0.002883] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]233client # [ 0.000000] SMBIOS 2.8 present.234worker1 # [ 0.002908] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]235lb # [ 0.002884] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]236worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026237worker1 # [ 0.002908] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]238client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014239lb # [ 0.002884] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]240client # [ 0.000000] DMI: Memory slots populated: 1/1241worker1 # [ 0.002909] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]242lb # [ 0.002885] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]243client # [ 0.000000] Hypervisor detected: KVM244worker1 # [ 0.002909] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]245client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000246lb # [ 0.002885] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]247client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00248worker1 # [ 0.002910] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]249lb # [ 0.002886] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]250worker1 # [ 0.003156] No NUMA configuration found251client # [ 0.000000] kvm-clock: using sched offset of 477024815 cycles252worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/lapab15gsml370yzals8ffgdzxiibh3q-nixos-system-worker2-test/init regInfo=/nix/store/x39icyk4l2nfvms09gpv9vdscv458nsv-closure-info/registration console=ttyS0,115200n8 console=tty0253lb # [ 0.002886] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]254worker2 # [ 0.000000] BIOS-provided physical RAM map:255worker1 # [ 0.003157] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]256lb # [ 0.003127] No NUMA configuration found257client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns258worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable259worker1 # [ 0.003160] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]260worker1 # [ 0.005380] Zone ranges:261lb # [ 0.003128] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]262client # [ 0.000005] tsc: Detected 2400.012 MHz processor263worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved264worker1 # [ 0.005381] DMA [mem 0x0000000000001000-0x0000000000ffffff]265lb # [ 0.003130] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]266client # [ 0.000816] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000267lb # [ 0.005372] Zone ranges:268worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved269worker1 # [ 0.005383] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]270worker1 # [ 0.005384] Normal empty271client # [ 0.000843] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs272lb # [ 0.005374] DMA [mem 0x0000000000001000-0x0000000000ffffff]273worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable274worker1 # [ 0.005385] Device empty275lb # [ 0.005375] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]276worker1 # [ 0.005385] Movable zone start for each node277client # [ 0.000846] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT278lb # [ 0.005376] Normal empty279worker1 # [ 0.005386] Early memory node ranges280worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved281lb # [ 0.005377] Device empty282client # [ 0.002750] found SMP MP-table at [mem 0x000f5450-0x000f545f]283lb # [ 0.005378] Movable zone start for each node284worker1 # [ 0.005386] node 0: [mem 0x0000000000001000-0x000000000009efff]285worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved286client # [ 0.002761] Using GB pages for direct mapping287lb # [ 0.005379] Early memory node ranges288client # [ 0.002845] RAMDISK: [mem 0x5e366000-0x5ffcffff]289worker1 # [ 0.005387] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]290worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved291lb # [ 0.005379] node 0: [mem 0x0000000000001000-0x000000000009efff]292client # [ 0.002855] ACPI: Early table checksum verification disabled293worker1 # [ 0.005389] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]294worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved295lb # [ 0.005380] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]296client # [ 0.002858] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )297worker1 # [ 0.005409] On node 0, zone DMA: 1 pages in unavailable ranges298worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved299lb # [ 0.005381] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]300worker1 # [ 0.005659] On node 0, zone DMA: 97 pages in unavailable ranges301client # [ 0.002862] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)302worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved303lb # [ 0.005401] On node 0, zone DMA: 1 pages in unavailable ranges304worker1 # [ 0.031896] On node 0, zone DMA32: 40 pages in unavailable ranges305worker2 # [ 0.000000] NX (Execute Disable) protection: active306lb # [ 0.005668] On node 0, zone DMA: 97 pages in unavailable ranges307client # [ 0.002867] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)308worker1 # [ 0.032385] ACPI: PM-Timer IO Port: 0x608309worker2 # [ 0.000000] APIC: Static calls initialized310lb # [ 0.032342] On node 0, zone DMA32: 40 pages in unavailable ranges311worker1 # [ 0.032399] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])312worker2 # [ 0.000000] SMBIOS 2.8 present.313client # [ 0.002875] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)314lb # [ 0.032802] ACPI: PM-Timer IO Port: 0x608315worker1 # [ 0.032429] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23316client # [ 0.002877] ACPI: FACS 0x000000005FFE0000 000040317lb # [ 0.032815] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])318worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014319worker2 # [ 0.000000] DMI: Memory slots populated: 1/1320worker1 # [ 0.032432] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)321lb # [ 0.032845] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23322client # [ 0.002878] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)323worker2 # [ 0.000000] Hypervisor detected: KVM324worker1 # [ 0.032434] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)325worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000326lb # [ 0.032848] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)327client # [ 0.002880] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)328worker1 # [ 0.032435] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)329worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00330lb # [ 0.032850] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)331client # [ 0.002881] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)332worker2 # [ 0.000001] kvm-clock: using sched offset of 466571556 cycles333worker1 # [ 0.032436] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)334lb # [ 0.032851] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)335worker1 # [ 0.032437] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)336lb # [ 0.032852] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)337client # [ 0.002883] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)338worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns339worker1 # [ 0.032440] ACPI: Using ACPI (MADT) for SMP configuration information340worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor341client # [ 0.002884] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]342lb # [ 0.032853] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)343worker1 # [ 0.032441] ACPI: HPET id: 0x8086a201 base: 0xfed00000344worker2 # [ 0.000819] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000345lb # [ 0.032855] ACPI: Using ACPI (MADT) for SMP configuration information346worker1 # [ 0.032446] TSC deadline timer available347client # [ 0.002885] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]348lb # [ 0.032856] ACPI: HPET id: 0x8086a201 base: 0xfed00000349worker1 # [ 0.032450] CPU topo: Max. logical packages: 1350worker2 # [ 0.000846] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs351client # [ 0.002886] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]352lb # [ 0.032860] TSC deadline timer available353worker1 # [ 0.032451] CPU topo: Max. logical dies: 1354worker2 # [ 0.000849] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT355lb # [ 0.032864] CPU topo: Max. logical packages: 1356worker1 # [ 0.032451] CPU topo: Max. dies per package: 1357client # [ 0.002886] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]358lb # [ 0.032865] CPU topo: Max. logical dies: 1359worker1 # [ 0.032455] CPU topo: Max. threads per core: 1360worker2 # [ 0.002748] found SMP MP-table at [mem 0x000f5450-0x000f545f]361client # [ 0.002887] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]362lb # [ 0.032865] CPU topo: Max. dies per package: 1363worker1 # [ 0.032456] CPU topo: Num. cores per package: 1364worker2 # [ 0.002760] Using GB pages for direct mapping365lb # [ 0.032869] CPU topo: Max. threads per core: 1366worker2 # [ 0.002854] RAMDISK: [mem 0x5e367000-0x5ffcffff]367worker1 # [ 0.032456] CPU topo: Num. threads per package: 1368client # [ 0.002887] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]369lb # [ 0.032870] CPU topo: Num. cores per package: 1370worker2 # [ 0.002862] ACPI: Early table checksum verification disabled371worker1 # [ 0.032456] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs372client # [ 0.002888] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]373lb # [ 0.032870] CPU topo: Num. threads per package: 1374worker2 # [ 0.002865] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )375client # [ 0.003128] No NUMA configuration found376worker1 # [ 0.032479] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()377lb # [ 0.032870] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs378client # [ 0.003129] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]379worker2 # [ 0.002870] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)380worker1 # [ 0.032523] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]381lb # [ 0.032891] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()382client # [ 0.003132] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]383client # [ 0.005744] Zone ranges:384worker2 # [ 0.002874] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)385worker1 # [ 0.032524] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]386lb # [ 0.032924] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]387client # [ 0.005746] DMA [mem 0x0000000000001000-0x0000000000ffffff]388worker1 # [ 0.032526] [mem 0x60000000-0xafffffff] available for PCI devices389worker2 # [ 0.002881] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)390lb # [ 0.032926] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]391worker1 # [ 0.032527] Booting paravirtualized kernel on KVM392client # [ 0.005747] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]393worker2 # [ 0.002883] ACPI: FACS 0x000000005FFE0000 000040394lb # [ 0.032927] [mem 0x60000000-0xafffffff] available for PCI devices395client # [ 0.005748] Normal empty396client # [ 0.005749] Device empty397lb # [ 0.032929] Booting paravirtualized kernel on KVM398worker2 # [ 0.002885] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)399worker1 # [ 0.032531] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns400client # [ 0.005750] Movable zone start for each node401client # [ 0.005751] Early memory node ranges402worker1 # [ 0.037003] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1403lb # [ 0.032932] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns404worker2 # [ 0.002886] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)405client # [ 0.005751] node 0: [mem 0x0000000000001000-0x000000000009efff]406worker1 # [ 0.039316] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152407lb # [ 0.037404] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1408worker2 # [ 0.002888] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)409client # [ 0.005753] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]410worker1 # [ 0.039361] kvm-guest: PV spinlocks disabled, single CPU411lb # [ 0.039758] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152412client # [ 0.005753] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]413worker2 # [ 0.002890] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)414lb # [ 0.039808] kvm-guest: PV spinlocks disabled, single CPU415client # [ 0.005773] On node 0, zone DMA: 1 pages in unavailable ranges416worker2 # [ 0.002891] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]417client # [ 0.006035] On node 0, zone DMA: 97 pages in unavailable ranges418worker2 # [ 0.002892] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]419client # [ 0.032369] On node 0, zone DMA32: 40 pages in unavailable ranges420worker2 # [ 0.002893] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]421client # [ 0.032876] ACPI: PM-Timer IO Port: 0x608422worker1 # [ 0.039362] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/qmcgfpcw2i7m6pmgb5saqy8wfr82vfq3-nixos-system-worker1-test/init regInfo=/nix/store/q5s318i3gaswdpaw2h8ilxrpz9w65pb1-closure-info/registration console=ttyS0,115200n8 console=tty0423client # [ 0.032889] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])424worker2 # [ 0.002893] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]425lb # [ 0.039809] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0qln5mjy767qjp5wbmxjg8jay8k05dsm-nixos-system-lb-test/init regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration console=ttyS0,115200n8 console=tty0426client # [ 0.032922] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23427worker2 # [ 0.002893] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]428worker1 # [ 0.039458] Unknown kernel command line parameters "regInfo=/nix/store/q5s318i3gaswdpaw2h8ilxrpz9w65pb1-closure-info/registration", will be passed to user space.429client # [ 0.032925] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)430worker2 # [ 0.002894] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]431worker1 # [ 0.039472] random: crng init done432lb # [ 0.039907] Unknown kernel command line parameters "regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration", will be passed to user space.433client # [ 0.032926] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)434worker2 # [ 0.002894] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]435lb # [ 0.039920] random: crng init done436worker1 # [ 0.039472] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes437worker2 # [ 0.003130] No NUMA configuration found438client # [ 0.032927] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)439lb # [ 0.039921] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes440worker2 # [ 0.003131] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]441worker1 # [ 0.041538] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)442client # [ 0.032928] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)443worker2 # [ 0.003133] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]444worker1 # [ 0.042567] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)445worker2 # [ 0.005428] Zone ranges:446client # [ 0.032929] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)447worker1 # [ 0.042604] Fallback order for Node 0: 0448lb # [ 0.042069] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)449lb # [ 0.043141] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)450worker2 # [ 0.005430] DMA [mem 0x0000000000001000-0x0000000000ffffff]451client # [ 0.032932] ACPI: Using ACPI (MADT) for SMP configuration information452worker1 # [ 0.042611] Built 1 zonelists, mobility grouping on. Total pages: 393078453lb # [ 0.043173] Fallback order for Node 0: 0454client # [ 0.032933] ACPI: HPET id: 0x8086a201 base: 0xfed00000455worker1 # [ 0.042612] Policy zone: DMA32456worker2 # [ 0.005431] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]457worker2 # [ 0.005433] Normal empty458lb # [ 0.043176] Built 1 zonelists, mobility grouping on. Total pages: 393078459client # [ 0.032937] TSC deadline timer available460worker2 # [ 0.005433] Device empty461worker1 # [ 0.045067] mem auto-init: stack:all(zero), heap alloc:on, heap free:off462lb # [ 0.043177] Policy zone: DMA32463client # [ 0.032941] CPU topo: Max. logical packages: 1464worker2 # [ 0.005434] Movable zone start for each node465worker1 # [ 0.048318] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1466worker2 # [ 0.005435] Early memory node ranges467lb # [ 0.046015] mem auto-init: stack:all(zero), heap alloc:on, heap free:off468client # [ 0.032942] CPU topo: Max. logical dies: 1469worker1 # [ 0.051816] allocated 3145728 bytes of page_ext470client # [ 0.032942] CPU topo: Max. dies per package: 1471worker2 # [ 0.005435] node 0: [mem 0x0000000000001000-0x000000000009efff]472lb # [ 0.049246] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1473client # [ 0.032945] CPU topo: Max. threads per core: 1474worker1 # [ 0.062024] ftrace: allocating 48731 entries in 192 pages475lb # [ 0.052652] allocated 3145728 bytes of page_ext476worker2 # [ 0.005436] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]477client # [ 0.032946] CPU topo: Num. cores per package: 1478worker1 # [ 0.062026] ftrace: allocated 192 pages with 2 groups479lb # [ 0.063028] ftrace: allocating 48731 entries in 192 pages480worker1 # [ 0.062896] Dynamic Preempt: lazy481client # [ 0.032946] CPU topo: Num. threads per package: 1482worker2 # [ 0.005437] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]483lb # [ 0.063030] ftrace: allocated 192 pages with 2 groups484worker1 # [ 0.063053] rcu: Preemptible hierarchical RCU implementation.485lb # [ 0.063865] Dynamic Preempt: lazy486client # [ 0.032947] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs487worker2 # [ 0.005457] On node 0, zone DMA: 1 pages in unavailable ranges488worker1 # [ 0.063053] rcu: RCU event tracing is enabled.489lb # [ 0.064003] rcu: Preemptible hierarchical RCU implementation.490worker2 # [ 0.005730] On node 0, zone DMA: 97 pages in unavailable ranges491client # [ 0.032967] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()492worker1 # [ 0.063054] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.493lb # [ 0.064004] rcu: RCU event tracing is enabled.494worker2 # [ 0.031578] On node 0, zone DMA32: 40 pages in unavailable ranges495worker1 # [ 0.063055] Trampoline variant of Tasks RCU enabled.496client # [ 0.033000] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]497worker2 # [ 0.032048] ACPI: PM-Timer IO Port: 0x608498lb # [ 0.064004] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.499worker1 # [ 0.063056] Rude variant of Tasks RCU enabled.500worker2 # [ 0.032060] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])501lb # [ 0.064006] Trampoline variant of Tasks RCU enabled.502client # [ 0.033002] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]503worker1 # [ 0.063056] Tracing variant of Tasks RCU enabled.504lb # [ 0.064006] Rude variant of Tasks RCU enabled.505worker2 # [ 0.032091] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23506client # [ 0.033003] [mem 0x60000000-0xafffffff] available for PCI devices507lb # [ 0.064007] Tracing variant of Tasks RCU enabled.508worker1 # [ 0.063057] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.509client # [ 0.033005] Booting paravirtualized kernel on KVM510worker2 # [ 0.032094] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)511worker1 # [ 0.063057] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1512lb # [ 0.064007] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.513worker2 # [ 0.032096] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)514client # [ 0.033008] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns515lb # [ 0.064008] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1516worker2 # [ 0.032097] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)517worker1 # [ 0.063114] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.518client # [ 0.037493] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1519lb # [ 0.064023] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.520worker2 # [ 0.032098] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)521worker1 # [ 0.063115] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.522client # [ 0.039599] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152523worker2 # [ 0.032099] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)524client # [ 0.039649] kvm-guest: PV spinlocks disabled, single CPU525lb # [ 0.064025] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.526worker1 # [ 0.063116] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.527worker2 # [ 0.032101] ACPI: Using ACPI (MADT) for SMP configuration information528worker1 # [ 0.067462] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16529lb # [ 0.064027] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.530worker2 # [ 0.032102] ACPI: HPET id: 0x8086a201 base: 0xfed00000531worker2 # [ 0.032106] TSC deadline timer available532lb # [ 0.068407] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16533worker1 # [ 0.067746] rcu: srcu_init: Setting srcu_struct sizes based on contention.534worker2 # [ 0.032110] CPU topo: Max. logical packages: 1535lb # [ 0.068669] rcu: srcu_init: Setting srcu_struct sizes based on contention.536worker2 # [ 0.032110] CPU topo: Max. logical dies: 1537worker1 # [ 0.067754] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns538worker2 # [ 0.032111] CPU topo: Max. dies per package: 1539lb # [ 0.068677] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns540worker2 # [ 0.032114] CPU topo: Max. threads per core: 1541client # [ 0.039651] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/jwg6hsvasqr1bfb4p1nn6ayia3202nfr-nixos-system-client-test/init regInfo=/nix/store/ycwmfabcj5bpk7yq23lg5g1pj9l5i2yl-closure-info/registration console=ttyS0,115200n8 console=tty0542worker1 # [ 0.067863] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)543worker2 # [ 0.032115] CPU topo: Num. cores per package: 1544worker1 # [ 0.071427] Console: colour VGA+ 80x25545lb # [ 0.068786] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)546worker2 # [ 0.032115] CPU topo: Num. threads per package: 1547worker1 # [ 0.071430] printk: legacy console [tty0] enabled548lb # [ 0.072320] Console: colour VGA+ 80x25549client # [ 0.039750] Unknown kernel command line parameters "regInfo=/nix/store/ycwmfabcj5bpk7yq23lg5g1pj9l5i2yl-closure-info/registration", will be passed to user space.550worker2 # [ 0.032116] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs551worker1 # [ 0.100972] printk: legacy console [ttyS0] enabled552lb # [ 0.072323] printk: legacy console [tty0] enabled553client # [ 0.039763] random: crng init done554worker2 # [ 0.032137] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()555lb # [ 0.101841] printk: legacy console [ttyS0] enabled556worker1 # [ 0.197246] ACPI: Core revision 20250807557client # [ 0.039764] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes558lb # [ 0.197935] ACPI: Core revision 20250807559worker2 # [ 0.032169] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]560client # [ 0.041835] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)561worker1 # [ 0.198145] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns562worker2 # [ 0.032171] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]563lb # [ 0.198821] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns564client # [ 0.042889] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)565worker2 # [ 0.032172] [mem 0x60000000-0xafffffff] available for PCI devices566worker1 # [ 0.199713] APIC: Switch to symmetric I/O mode setup567client # [ 0.042923] Fallback order for Node 0: 0568worker2 # [ 0.032174] Booting paravirtualized kernel on KVM569worker1 # [ 0.200647] x2apic enabled570lb # [ 0.200323] APIC: Switch to symmetric I/O mode setup571client # [ 0.042926] Built 1 zonelists, mobility grouping on. Total pages: 393078572lb # [ 0.201310] x2apic enabled573client # [ 0.042928] Policy zone: DMA32574worker2 # [ 0.032177] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns575worker1 # [ 0.201397] APIC: Switched APIC routing to: physical x2apic576client # [ 0.045514] mem auto-init: stack:all(zero), heap alloc:on, heap free:off577lb # [ 0.202066] APIC: Switched APIC routing to: physical x2apic578worker2 # [ 0.036640] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1579client # [ 0.048754] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1580worker2 # [ 0.039190] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152581client # [ 0.052000] allocated 3145728 bytes of page_ext582worker1 # [ 0.203200] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1583worker2 # [ 0.039237] kvm-guest: PV spinlocks disabled, single CPU584client # [ 0.062433] ftrace: allocating 48731 entries in 192 pages585lb # [ 0.203852] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1586client # [ 0.062435] ftrace: allocated 192 pages with 2 groups587client # [ 0.063286] Dynamic Preempt: lazy588worker1 # [ 0.204148] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns589client # [ 0.063463] rcu: Preemptible hierarchical RCU implementation.590lb # [ 0.204809] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns591client # [ 0.063464] rcu: RCU event tracing is enabled.592worker1 # [ 0.205734] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)593client # [ 0.063464] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.594lb # [ 0.206423] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)595worker2 # [ 0.039238] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/lapab15gsml370yzals8ffgdzxiibh3q-nixos-system-worker2-test/init regInfo=/nix/store/x39icyk4l2nfvms09gpv9vdscv458nsv-closure-info/registration console=ttyS0,115200n8 console=tty0596worker1 # [ 0.206820] x86/cpu: User Mode Instruction Prevention (UMIP) activated597client # [ 0.063466] Trampoline variant of Tasks RCU enabled.598client # [ 0.063467] Rude variant of Tasks RCU enabled.599lb # [ 0.207508] x86/cpu: User Mode Instruction Prevention (UMIP) activated600worker1 # [ 0.207855] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127601client # [ 0.063467] Tracing variant of Tasks RCU enabled.602worker2 # [ 0.039336] Unknown kernel command line parameters "regInfo=/nix/store/x39icyk4l2nfvms09gpv9vdscv458nsv-closure-info/registration", will be passed to user space.603lb # [ 0.208559] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127604worker1 # [ 0.208731] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0605worker2 # [ 0.039349] random: crng init done606client # [ 0.063468] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.607lb # [ 0.209420] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0608worker2 # [ 0.039350] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes609client # [ 0.063469] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1610worker1 # [ 0.209735] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto611worker2 # [ 0.041478] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)612client # [ 0.063496] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.613lb # [ 0.210423] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto614worker2 # [ 0.042510] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)615worker1 # [ 0.210731] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl616client # [ 0.063499] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.617worker2 # [ 0.042545] Fallback order for Node 0: 0618lb # [ 0.211420] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl619worker1 # [ 0.211731] Transient Scheduler Attacks: Vulnerable: No microcode620worker2 # [ 0.042548] Built 1 zonelists, mobility grouping on. Total pages: 393078621client # [ 0.063499] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.622worker2 # [ 0.042549] Policy zone: DMA32623lb # [ 0.212420] Transient Scheduler Attacks: Vulnerable: No microcode624worker1 # [ 0.212731] Spectre V2 : Mitigation: Enhanced / Automatic IBRS625client # [ 0.067934] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16626worker2 # [ 0.044978] mem auto-init: stack:all(zero), heap alloc:on, heap free:off627lb # [ 0.213419] Spectre V2 : Mitigation: Enhanced / Automatic IBRS628client # [ 0.068215] rcu: srcu_init: Setting srcu_struct sizes based on contention.629worker1 # [ 0.213731] Speculative Return Stack Overflow: Mitigation: Safe RET630worker2 # [ 0.048184] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1631worker2 # [ 0.051552] allocated 3145728 bytes of page_ext632lb # [ 0.214420] Speculative Return Stack Overflow: Mitigation: Safe RET633client # [ 0.068224] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns634worker2 # [ 0.062073] ftrace: allocating 48731 entries in 192 pages635worker2 # [ 0.062075] ftrace: allocated 192 pages with 2 groups636client # [ 0.068333] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)637worker2 # [ 0.062953] Dynamic Preempt: lazy638client # [ 0.072040] Console: colour VGA+ 80x25639worker2 # [ 0.063127] rcu: Preemptible hierarchical RCU implementation.640client # [ 0.072043] printk: legacy console [tty0] enabled641worker2 # [ 0.063128] rcu: RCU event tracing is enabled.642worker1 # [ 0.214731] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization643client # [ 0.102428] printk: legacy console [ttyS0] enabled644worker2 # [ 0.063128] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.645lb # [ 0.215419] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization646worker1 # [ 0.216739] Spectre V2 : Enabling IBPB for BPF647client # [ 0.200337] ACPI: Core revision 20250807648worker2 # [ 0.063130] Trampoline variant of Tasks RCU enabled.649lb # [ 0.217428] Spectre V2 : Enabling IBPB for BPF650worker2 # [ 0.063130] Rude variant of Tasks RCU enabled.651worker1 # [ 0.217450] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier652worker2 # [ 0.063131] Tracing variant of Tasks RCU enabled.653client # [ 0.201259] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns654lb # [ 0.218140] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier655worker1 # [ 0.218731] active return thunk: srso_alias_return_thunk656worker2 # [ 0.063131] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.657client # [ 0.202892] APIC: Switch to symmetric I/O mode setup658lb # [ 0.219420] active return thunk: srso_alias_return_thunk659worker2 # [ 0.063132] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1660client # [ 0.203856] x2apic enabled661worker1 # [ 0.219574] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'662lb # [ 0.220261] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'663worker2 # [ 0.063147] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.664worker1 # [ 0.220731] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'665client # [ 0.204634] APIC: Switched APIC routing to: physical x2apic666lb # [ 0.221420] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'667worker2 # [ 0.063149] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.668worker1 # [ 0.221731] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'669lb # [ 0.222419] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'670worker1 # [ 0.222731] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'671worker2 # [ 0.063150] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.672client # [ 0.206451] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1673worker2 # [ 0.067542] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16674lb # [ 0.223419] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'675worker1 # [ 0.223671] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'676worker2 # [ 0.067880] rcu: srcu_init: Setting srcu_struct sizes based on contention.677lb # [ 0.224419] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'678client # [ 0.207415] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns679worker1 # [ 0.224461] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'680worker2 # [ 0.067891] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns681lb # [ 0.225419] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'682worker1 # [ 0.225486] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'683client # [ 0.209054] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)684worker2 # [ 0.067998] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)685lb # [ 0.226419] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'686worker1 # [ 0.226731] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256687worker2 # [ 0.071556] Console: colour VGA+ 80x25688client # [ 0.210143] x86/cpu: User Mode Instruction Prevention (UMIP) activated689worker2 # [ 0.071559] printk: legacy console [tty0] enabled690lb # [ 0.227420] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256691worker1 # [ 0.227731] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64692worker2 # [ 0.101122] printk: legacy console [ttyS0] enabled693client # [ 0.211174] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127694lb # [ 0.228420] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64695worker1 # [ 0.228731] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512696worker2 # [ 0.196677] ACPI: Core revision 20250807697client # [ 0.212052] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0698lb # [ 0.229419] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512699worker1 # [ 0.229639] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024700lb # [ 0.230419] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024701worker2 # [ 0.197554] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns702worker1 # [ 0.230434] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8703client # [ 0.213056] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto704worker2 # [ 0.199109] APIC: Switch to symmetric I/O mode setup705lb # [ 0.231328] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8706worker1 # [ 0.231422] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.707worker2 # [ 0.200045] x2apic enabled708client # [ 0.215052] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl709lb # [ 0.232159] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.710client # [ 0.216052] Transient Scheduler Attacks: Vulnerable: No microcode711worker2 # [ 0.200784] APIC: Switched APIC routing to: physical x2apic712client # [ 0.217052] Spectre V2 : Mitigation: Enhanced / Automatic IBRS713worker2 # [ 0.202573] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1714client # [ 0.218052] Speculative Return Stack Overflow: Mitigation: Safe RET715worker2 # [ 0.203525] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns716worker2 # [ 0.205159] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)717worker2 # [ 0.206245] x86/cpu: User Mode Instruction Prevention (UMIP) activated718client # [ 0.219052] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization719client # [ 0.221060] Spectre V2 : Enabling IBPB for BPF720worker2 # [ 0.207334] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127721worker2 # [ 0.208156] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0722client # [ 0.222053] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier723client # [ 0.223053] active return thunk: srso_alias_return_thunk724worker2 # [ 0.209159] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto725client # [ 0.224074] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'726worker2 # [ 0.210156] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl727client # [ 0.225052] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'728worker2 # [ 0.211156] Transient Scheduler Attacks: Vulnerable: No microcode729client # [ 0.226817] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'730worker2 # [ 0.212156] Spectre V2 : Mitigation: Enhanced / Automatic IBRS731client # [ 0.227805] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'732worker2 # [ 0.213156] Speculative Return Stack Overflow: Mitigation: Safe RET733client # [ 0.228822] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'734client # [ 0.229797] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'735client # [ 0.230846] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'736client # [ 0.232052] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256737worker2 # [ 0.214164] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization738client # [ 0.233052] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64739worker2 # [ 0.216163] Spectre V2 : Enabling IBPB for BPF740client # [ 0.234052] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512741worker2 # [ 0.217157] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier742client # [ 0.235052] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024743worker2 # [ 0.218156] active return thunk: srso_alias_return_thunk744client # [ 0.236052] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8745worker2 # [ 0.219177] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'746client # [ 0.237052] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.747worker2 # [ 0.220156] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'748worker2 # [ 0.221155] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'749worker2 # [ 0.222156] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'750worker2 # [ 0.223155] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'751worker2 # [ 0.224155] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'752worker2 # [ 0.225155] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'753worker2 # [ 0.226156] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256754worker2 # [ 0.227155] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64755worker2 # [ 0.228155] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512756worker2 # [ 0.229155] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024757worker2 # [ 0.230156] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8758worker1 # [ 0.267063] Freeing SMP alternatives memory: 44K759lb # [ 0.267407] Freeing SMP alternatives memory: 44K760worker2 # [ 0.231155] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.761worker1 # [ 0.267735] pid_max: default: 32768 minimum: 301762lb # [ 0.268201] pid_max: default: 32768 minimum: 301763worker1 # [ 0.268546] LSM: initializing lsm=capability,landlock,yama,bpf,ima764lb # [ 0.268517] LSM: initializing lsm=capability,landlock,yama,bpf,ima765worker1 # [ 0.269772] landlock: Up and running.766lb # [ 0.269540] landlock: Up and running.767worker1 # [ 0.270381] Yama: becoming mindful.768lb # [ 0.270420] Yama: becoming mindful.769worker1 # [ 0.270962] LSM support for eBPF active770lb # [ 0.271549] LSM support for eBPF active771worker1 # [ 0.271870] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)772lb # [ 0.272294] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)773worker1 # [ 0.272767] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)774lb # [ 0.273461] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)775worker1 # [ 0.275459] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)776lb # [ 0.275637] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)777worker1 # [ 0.276288] Performance Events: Fam17h+ core perfctr, AMD PMU driver.778lb # [ 0.276975] Performance Events: Fam17h+ core perfctr, AMD PMU driver.779worker1 # [ 0.276736] ... version: 2780lb # [ 0.277429] ... version: 2781worker1 # [ 0.277444] ... bit width: 48782lb # [ 0.278104] ... bit width: 48783worker1 # [ 0.277734] ... generic counters: 6784lb # [ 0.278422] ... generic counters: 6785worker1 # [ 0.278422] ... generic bitmap: 000000000000003f786lb # [ 0.279119] ... generic bitmap: 000000000000003f787worker1 # [ 0.278780] ... fixed-purpose counters: 0788lb # [ 0.279422] ... fixed-purpose counters: 0789worker1 # [ 0.279480] ... fixed-purpose bitmap: 0000000000000000790lb # [ 0.280101] ... fixed-purpose bitmap: 0000000000000000791worker1 # [ 0.279734] ... value mask: 0000ffffffffffff792lb # [ 0.280422] ... value mask: 0000ffffffffffff793worker1 # [ 0.280589] ... max period: 00007fffffffffff794lb # [ 0.281276] ... max period: 00007fffffffffff795worker1 # [ 0.281396] ... global_ctrl mask: 000000000000003f796lb # [ 0.282082] ... global_ctrl mask: 000000000000003f797lb # [ 0.282533] signal: max sigframe size: 3376798worker1 # [ 0.281857] signal: max sigframe size: 3376799worker1 # [ 0.282667] rcu: Hierarchical SRCU implementation.800lb # [ 0.283302] rcu: Hierarchical SRCU implementation.801worker1 # [ 0.283303] rcu: Max phase no-delay instances is 400.802lb # [ 0.284067] rcu: Max phase no-delay instances is 400.803client # [ 0.272293] Freeing SMP alternatives memory: 44K804client # [ 0.273055] pid_max: default: 32768 minimum: 301805client # [ 0.273899] LSM: initializing lsm=capability,landlock,yama,bpf,ima806worker1 # [ 0.288293] smp: Bringing up secondary CPUs ...807lb # [ 0.288967] smp: Bringing up secondary CPUs ...808client # [ 0.275107] landlock: Up and running.809worker1 # [ 0.288747] smp: Brought up 1 node, 1 CPU810lb # [ 0.289437] smp: Brought up 1 node, 1 CPU811client # [ 0.275738] Yama: becoming mindful.812worker1 # [ 0.289408] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)813lb # [ 0.290095] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)814client # [ 0.276268] LSM support for eBPF active815client # [ 0.277174] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)816worker1 # [ 0.289890] Memory: 1454560K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)817lb # [ 0.290630] Memory: 1454576K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)818client # [ 0.278090] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)819worker1 # [ 0.291011] devtmpfs: initialized820lb # [ 0.291686] devtmpfs: initialized821worker1 # [ 0.291806] x86/mm: Memory block size: 128MB822lb # [ 0.292511] x86/mm: Memory block size: 128MB823client # [ 0.280852] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)824lb # [ 0.294225] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)825worker1 # [ 0.293604] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)826client # [ 0.281586] Performance Events: Fam17h+ core perfctr, AMD PMU driver.827lb # [ 0.295296] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).828worker1 # [ 0.294634] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).829client # [ 0.282057] ... version: 2830client # [ 0.282755] ... bit width: 48831worker2 # [ 0.267522] Freeing SMP alternatives memory: 44K832lb # [ 0.296519] pinctrl core: initialized pinctrl subsystem833worker1 # [ 0.295816] pinctrl core: initialized pinctrl subsystem834client # [ 0.283054] ... generic counters: 6835worker2 # [ 0.268159] pid_max: default: 32768 minimum: 301836lb # [ 0.297613] PM: RTC time: 13:33:03, date: 2026-09-20837worker1 # [ 0.296928] PM: RTC time: 13:33:03, date: 2026-09-20838client # [ 0.283761] ... generic bitmap: 000000000000003f839worker2 # [ 0.269179] LSM: initializing lsm=capability,landlock,yama,bpf,ima840client # [ 0.284105] ... fixed-purpose counters: 0841worker2 # [ 0.270268] landlock: Up and running.842client # [ 0.284798] ... fixed-purpose bitmap: 0000000000000000843worker2 # [ 0.270895] Yama: becoming mindful.844client # [ 0.285055] ... value mask: 0000ffffffffffff845worker2 # [ 0.271393] LSM support for eBPF active846client # [ 0.285953] ... max period: 00007fffffffffff847worker1 # [ 0.300519] NET: Registered PF_NETLINK/PF_ROUTE protocol family848worker2 # [ 0.272283] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)849lb # [ 0.302009] NET: Registered PF_NETLINK/PF_ROUTE protocol family850client # [ 0.286725] ... global_ctrl mask: 000000000000003f851client # [ 0.287182] signal: max sigframe size: 3376852worker2 # [ 0.273192] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)853worker1 # [ 0.301963] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations854lb # [ 0.302975] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations855client # [ 0.287994] rcu: Hierarchical SRCU implementation.856worker1 # [ 0.302754] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations857client # [ 0.288623] rcu: Max phase no-delay instances is 400.858lb # [ 0.303462] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations859worker2 # [ 0.276087] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)860worker1 # [ 0.304015] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations861lb # [ 0.304741] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations862worker1 # [ 0.304743] audit: initializing netlink subsys (disabled)863worker2 # [ 0.276698] Performance Events: Fam17h+ core perfctr, AMD PMU driver.864lb # [ 0.305433] audit: initializing netlink subsys (disabled)865worker2 # [ 0.277163] ... version: 2866worker1 # [ 0.305736] audit: type=2000 audit(1789911184.577:1): state=initialized audit_enabled=0 res=1867lb # [ 0.306561] thermal_sys: Registered thermal governor 'fair_share'868worker2 # [ 0.277874] ... bit width: 48869worker1 # [ 0.306921] thermal_sys: Registered thermal governor 'fair_share'870worker2 # [ 0.278158] ... generic counters: 6871lb # [ 0.306563] thermal_sys: Registered thermal governor 'bang_bang'872client # [ 0.293633] smp: Bringing up secondary CPUs ...873worker1 # [ 0.306923] thermal_sys: Registered thermal governor 'bang_bang'874worker2 # [ 0.278856] ... generic bitmap: 000000000000003f875lb # [ 0.307423] thermal_sys: Registered thermal governor 'step_wise'876client # [ 0.294069] smp: Brought up 1 node, 1 CPU877worker2 # [ 0.279209] ... fixed-purpose counters: 0878worker1 # [ 0.307734] thermal_sys: Registered thermal governor 'step_wise'879lb # [ 0.308331] thermal_sys: Registered thermal governor 'user_space'880client # [ 0.294731] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)881worker2 # [ 0.279882] ... fixed-purpose bitmap: 0000000000000000882worker1 # [ 0.308646] thermal_sys: Registered thermal governor 'user_space'883lb # [ 0.309107] audit: type=2000 audit(1789911184.578:1): state=initialized audit_enabled=0 res=1884worker2 # [ 0.280159] ... value mask: 0000ffffffffffff885worker1 # [ 0.309420] thermal_sys: Registered thermal governor 'power_allocator'886lb # [ 0.310425] thermal_sys: Registered thermal governor 'power_allocator'887worker1 # [ 0.309749] cpuidle: using governor menu888worker2 # [ 0.281037] ... max period: 00007fffffffffff889client # [ 0.295260] Memory: 1454540K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109348K reserved, 0K cma-reserved)890lb # [ 0.310438] cpuidle: using governor menu891worker2 # [ 0.281832] ... global_ctrl mask: 000000000000003f892client # [ 0.296314] devtmpfs: initialized893worker1 # [ 0.311923] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5894worker2 # [ 0.282293] signal: max sigframe size: 3376895lb # [ 0.312436] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5896client # [ 0.297112] x86/mm: Memory block size: 128MB897worker2 # [ 0.283118] rcu: Hierarchical SRCU implementation.898worker1 # [ 0.312963] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]899worker2 # [ 0.283746] rcu: Max phase no-delay instances is 400.900lb # [ 0.313660] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]901worker1 # [ 0.313738] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry902client # [ 0.298909] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)903lb # [ 0.314425] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry904worker1 # [ 0.314747] PCI: Using configuration type 1 for base access905lb # [ 0.315433] PCI: Using configuration type 1 for base access906client # [ 0.299955] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).907worker1 # [ 0.315834] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.908client # [ 0.301145] pinctrl core: initialized pinctrl subsystem909lb # [ 0.316534] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.910client # [ 0.302255] PM: RTC time: 13:33:03, date: 2026-09-20911worker2 # [ 0.288889] smp: Bringing up secondary CPUs ...912worker2 # [ 0.289173] smp: Brought up 1 node, 1 CPU913worker2 # [ 0.289850] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)914client # [ 0.305928] NET: Registered PF_NETLINK/PF_ROUTE protocol family915lb # [ 0.321711] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages916worker2 # [ 0.290320] Memory: 1454560K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)917client # [ 0.307297] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations918lb # [ 0.322423] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page919worker1 # [ 0.322751] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages920worker2 # [ 0.291404] devtmpfs: initialized921client # [ 0.308081] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations922worker2 # [ 0.292227] x86/mm: Memory block size: 128MB923worker1 # [ 0.323734] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page924client # [ 0.309340] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations925worker2 # [ 0.293923] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)926client # [ 0.310065] audit: initializing netlink subsys (disabled)927worker2 # [ 0.295123] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).928lb # [ 0.327423] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages929client # [ 0.311057] audit: type=2000 audit(1789911184.599:1): state=initialized audit_enabled=0 res=1930worker2 # [ 0.296251] pinctrl core: initialized pinctrl subsystem931lb # [ 0.328422] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page932client # [ 0.312193] thermal_sys: Registered thermal governor 'fair_share'933worker1 # [ 0.328735] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages934worker2 # [ 0.297336] PM: RTC time: 13:33:03, date: 2026-09-20935client # [ 0.312195] thermal_sys: Registered thermal governor 'bang_bang'936worker1 # [ 0.329741] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page937client # [ 0.313056] thermal_sys: Registered thermal governor 'step_wise'938client # [ 0.314000] thermal_sys: Registered thermal governor 'user_space'939client # [ 0.314852] thermal_sys: Registered thermal governor 'power_allocator'940worker2 # [ 0.301139] NET: Registered PF_NETLINK/PF_ROUTE protocol family941client # [ 0.315975] cpuidle: using governor menu942worker2 # [ 0.302500] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations943client # [ 0.318141] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5944worker2 # [ 0.303186] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations945client # [ 0.319320] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]946client # [ 0.320058] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry947worker2 # [ 0.304424] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations948lb # [ 0.338821] ACPI: Added _OSI(Module Device)949client # [ 0.321070] PCI: Using configuration type 1 for base access950worker2 # [ 0.305169] audit: initializing netlink subsys (disabled)951lb # [ 0.339424] ACPI: Added _OSI(Processor Device)952lb # [ 0.340148] ACPI: Added _OSI(Processor Aggregator Device)953worker2 # [ 0.306161] audit: type=2000 audit(1789911184.613:1): state=initialized audit_enabled=0 res=1954client # [ 0.322168] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.955worker1 # [ 0.341025] ACPI: Added _OSI(Module Device)956worker2 # [ 0.307325] thermal_sys: Registered thermal governor 'fair_share'957worker1 # [ 0.341735] ACPI: Added _OSI(Processor Device)958worker1 # [ 0.342493] ACPI: Added _OSI(Processor Aggregator Device)959worker2 # [ 0.307327] thermal_sys: Registered thermal governor 'bang_bang'960worker2 # [ 0.308160] thermal_sys: Registered thermal governor 'step_wise'961worker2 # [ 0.309159] thermal_sys: Registered thermal governor 'user_space'962worker2 # [ 0.310158] thermal_sys: Registered thermal governor 'power_allocator'963worker2 # [ 0.311173] cpuidle: using governor menu964client # [ 0.329084] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages965worker2 # [ 0.313455] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5966client # [ 0.330056] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page967worker1 # [ 0.348571] ACPI: 1 ACPI AML tables successfully acquired and loaded968lb # [ 0.348718] ACPI: 1 ACPI AML tables successfully acquired and loaded969worker2 # [ 0.314434] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]970worker2 # [ 0.315162] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry971worker2 # [ 0.316171] PCI: Using configuration type 1 for base access972worker1 # [ 0.352380] ACPI: Interpreter enabled973lb # [ 0.352855] ACPI: Interpreter enabled974worker2 # [ 0.317353] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.975worker1 # [ 0.352750] ACPI: PM: (supports S0 S3 S4 S5)976client # [ 0.335056] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages977lb # [ 0.353438] ACPI: PM: (supports S0 S3 S4 S5)978worker1 # [ 0.353466] ACPI: Using IOAPIC for interrupt routing979client # [ 0.336055] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page980lb # [ 0.354135] ACPI: Using IOAPIC for interrupt routing981lb # [ 0.354479] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug982lb # [ 0.355423] PCI: Using E820 reservations for host bridge windows983worker1 # [ 0.355796] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug984worker1 # [ 0.356735] PCI: Using E820 reservations for host bridge windows985lb # [ 0.358487] ACPI: Enabled 2 GPEs in block 00 to 3F986worker2 # [ 0.324185] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages987worker2 # [ 0.325160] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page988worker1 # [ 0.359877] ACPI: Enabled 2 GPEs in block 00 to 3F989client # [ 0.347301] ACPI: Added _OSI(Module Device)990client # [ 0.348073] ACPI: Added _OSI(Processor Device)991worker2 # [ 0.330160] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages992client # [ 0.348808] ACPI: Added _OSI(Processor Aggregator Device)993worker2 # [ 0.331159] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page994lb # [ 0.367101] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])995worker1 # [ 0.368010] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])996lb # [ 0.367429] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]997worker1 # [ 0.368741] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]998lb # [ 0.369378] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]999worker1 # [ 0.369815] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1000lb # [ 0.370391] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1001client # [ 0.354693] ACPI: 1 ACPI AML tables successfully acquired and loaded1002worker1 # [ 0.370858] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1003lb # [ 0.371651] PCI host bridge to bus 0000:001004worker1 # [ 0.372129] PCI host bridge to bus 0000:001005lb # [ 0.372378] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1006worker1 # [ 0.372739] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1007lb # [ 0.373207] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1008worker1 # [ 0.373734] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1009client # [ 0.358460] ACPI: Interpreter enabled1010lb # [ 0.374208] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1011client # [ 0.359070] ACPI: PM: (supports S0 S3 S4 S5)1012worker2 # [ 0.342395] ACPI: Added _OSI(Module Device)1013worker1 # [ 0.374734] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1014lb # [ 0.375293] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1015client # [ 0.359833] ACPI: Using IOAPIC for interrupt routing1016worker2 # [ 0.343076] ACPI: Added _OSI(Processor Device)1017worker1 # [ 0.375734] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1018lb # [ 0.376295] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1019worker2 # [ 0.343727] ACPI: Added _OSI(Processor Aggregator Device)1020worker1 # [ 0.376734] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1021lb # [ 0.377291] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1022lb # [ 0.378326] pci_bus 0000:00: root bus resource [bus 00-ff]1023worker1 # [ 0.377735] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1024client # [ 0.362092] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1025worker1 # [ 0.378734] pci_bus 0000:00: root bus resource [bus 00-ff]1026lb # [ 0.379153] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1027client # [ 0.363056] PCI: Using E820 reservations for host bridge windows1028worker1 # [ 0.379721] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1029lb # [ 0.380908] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1030client # [ 0.366114] ACPI: Enabled 2 GPEs in block 00 to 3F1031worker1 # [ 0.381214] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1032worker2 # [ 0.349962] ACPI: 1 ACPI AML tables successfully acquired and loaded1033worker2 # [ 0.353924] ACPI: Interpreter enabled1034worker2 # [ 0.354175] ACPI: PM: (supports S0 S3 S4 S5)1035worker2 # [ 0.354944] ACPI: Using IOAPIC for interrupt routing1036lb # [ 0.383555] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1037worker2 # [ 0.355874] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1038client # [ 0.373694] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1039worker1 # [ 0.383825] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1040lb # [ 0.384435] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1041client # [ 0.374061] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1042worker1 # [ 0.384746] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1043lb # [ 0.385373] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1044client # [ 0.375135] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1045worker1 # [ 0.385678] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1046lb # [ 0.386436] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1047client # [ 0.376178] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1048worker1 # [ 0.386993] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1049worker2 # [ 0.361159] PCI: Using E820 reservations for host bridge windows1050client # [ 0.377421] PCI host bridge to bus 0000:001051lb # [ 0.388150] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1052worker2 # [ 0.362263] ACPI: Enabled 2 GPEs in block 00 to 3F1053client # [ 0.378059] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1054worker1 # [ 0.388743] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1055client # [ 0.379055] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1056lb # [ 0.390430] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1057client # [ 0.380055] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1058lb # [ 0.391255] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1059client # [ 0.381055] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1060worker1 # [ 0.390769] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1061lb # [ 0.392132] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1062client # [ 0.382055] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1063worker1 # [ 0.391631] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1064lb # [ 0.392428] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1065worker1 # [ 0.392480] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1066client # [ 0.383055] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1067client # [ 0.384055] pci_bus 0000:00: root bus resource [bus 00-ff]1068worker1 # [ 0.392749] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1069lb # [ 0.393911] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1070worker2 # [ 0.370320] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1071client # [ 0.385044] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1072worker1 # [ 0.394419] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1073worker2 # [ 0.371167] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1074lb # [ 0.395430] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1075client # [ 0.386507] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1076lb # [ 0.396266] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1077worker2 # [ 0.372236] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1078lb # [ 0.397136] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1079worker1 # [ 0.396765] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1080worker2 # [ 0.373281] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1081worker1 # [ 0.397743] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1082worker2 # [ 0.374527] PCI host bridge to bus 0000:001083lb # [ 0.397994] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1084worker1 # [ 0.398739] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1085worker2 # [ 0.375164] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1086worker2 # [ 0.376159] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1087worker1 # [ 0.400312] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1088worker2 # [ 0.377158] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1089worker2 # [ 0.378158] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1090lb # [ 0.400521] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1091client # [ 0.389246] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1092lb # [ 0.401428] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1093worker2 # [ 0.379158] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1094client # [ 0.390068] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1095worker1 # [ 0.402913] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1096lb # [ 0.402355] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1097worker2 # [ 0.380158] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1098client # [ 0.391016] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1099worker1 # [ 0.403740] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1100worker2 # [ 0.381159] pci_bus 0000:00: root bus resource [bus 00-ff]1101lb # [ 0.403761] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1102worker1 # [ 0.404693] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1103client # [ 0.392231] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1104worker2 # [ 0.382160] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1105worker1 # [ 0.406106] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1106lb # [ 0.405430] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1107worker2 # [ 0.383619] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1108client # [ 0.393839] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1109lb # [ 0.406247] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1110lb # [ 0.407129] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1111worker1 # [ 0.407772] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1112worker1 # [ 0.408647] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1113client # [ 0.396075] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1114lb # [ 0.407980] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1115client # [ 0.396920] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1116worker1 # [ 0.409472] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1117client # [ 0.397783] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1118lb # [ 0.409440] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1119client # [ 0.398061] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1120worker1 # [ 0.410274] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1121lb # [ 0.410270] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1122lb # [ 0.411129] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1123client # [ 0.399602] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1124worker1 # [ 0.412741] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1125worker2 # [ 0.386291] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1126worker1 # [ 0.413549] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1127worker2 # [ 0.387217] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1128lb # [ 0.411995] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1129worker1 # [ 0.414477] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1130worker2 # [ 0.388174] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1131client # [ 0.401064] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1132client # [ 0.401984] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1133worker1 # [ 0.415422] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1134worker2 # [ 0.389622] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1135client # [ 0.402810] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1136lb # [ 0.414447] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1137lb # [ 0.415278] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1138client # [ 0.403676] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1139worker2 # [ 0.390927] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1140lb # [ 0.416427] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1141worker1 # [ 0.418492] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1142worker1 # [ 0.418741] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1143lb # [ 0.418031] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1144worker2 # [ 0.393192] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1145worker1 # [ 0.419787] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1146client # [ 0.406075] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1147worker2 # [ 0.394040] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1148lb # [ 0.419429] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1149client # [ 0.407062] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1150worker2 # [ 0.394883] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1151worker1 # [ 0.421301] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1152lb # [ 0.420250] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1153client # [ 0.407992] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1154worker2 # [ 0.395164] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1155lb # [ 0.421128] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1156worker1 # [ 0.422767] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1157lb # [ 0.421429] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1158client # [ 0.409440] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1159worker2 # [ 0.396737] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1160worker1 # [ 0.423625] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1161lb # [ 0.422954] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1162worker1 # [ 0.424508] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1163client # [ 0.411064] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1164worker1 # [ 0.425613] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1165worker2 # [ 0.399193] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1166client # [ 0.411902] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1167lb # [ 0.424437] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1168worker2 # [ 0.400091] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1169client # [ 0.412770] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1170lb # [ 0.425357] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1171worker2 # [ 0.400908] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1172worker1 # [ 0.427054] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1173client # [ 0.413611] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1174lb # [ 0.426824] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1175worker2 # [ 0.401709] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1176worker1 # [ 0.428624] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1177worker1 # [ 0.429506] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1178client # [ 0.415064] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1179lb # [ 0.428430] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1180client # [ 0.415910] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1181lb # [ 0.429248] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1182worker1 # [ 0.431232] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1183client # [ 0.416798] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1184lb # [ 0.430124] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1185worker2 # [ 0.404351] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1186worker2 # [ 0.405165] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1187client # [ 0.417695] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1188lb # [ 0.430979] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1189worker2 # [ 0.406125] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1190worker1 # [ 0.433497] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1191worker1 # [ 0.434371] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1192lb # [ 0.432429] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1193worker2 # [ 0.407581] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1194worker1 # [ 0.434756] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1195client # [ 0.420078] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1196lb # [ 0.433250] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1197lb # [ 0.434126] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1198client # [ 0.420922] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1199worker2 # [ 0.409191] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1200worker1 # [ 0.436315] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1201client # [ 0.422077] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1202worker2 # [ 0.410015] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1203lb # [ 0.435011] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1204worker2 # [ 0.410877] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1205worker1 # [ 0.437742] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1206client # [ 0.423678] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1207lb # [ 0.436030] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1208worker1 # [ 0.438579] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1209worker1 # [ 0.439472] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1210lb # [ 0.436626] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1211worker2 # [ 0.411744] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1212client # [ 0.425063] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1213client # [ 0.425908] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1214lb # [ 0.437981] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1215worker1 # [ 0.441146] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1216worker2 # [ 0.414195] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1217client # [ 0.426771] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1218worker2 # [ 0.415042] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1219client # [ 0.427061] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1220worker1 # [ 0.442441] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1221worker2 # [ 0.415899] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1222worker1 # [ 0.442941] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1223client # [ 0.428602] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1224worker2 # [ 0.416756] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1225worker1 # [ 0.444324] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1226client # [ 0.429886] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1227client # [ 0.430785] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1228worker2 # [ 0.419195] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1229client # [ 0.431653] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1230worker2 # [ 0.420165] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1231worker2 # [ 0.421100] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1232client # [ 0.433080] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1233worker2 # [ 0.422594] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1234client # [ 0.433950] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1235lb # [ 0.438627] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1236client # [ 0.434785] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1237worker2 # [ 0.424179] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1238lb # [ 0.441405] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1239worker2 # [ 0.425032] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1240client # [ 0.436017] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1241lb # [ 0.442278] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1242worker2 # [ 0.425924] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1243worker2 # [ 0.426164] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1244client # [ 0.438062] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1245worker1 # [ 0.444957] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1246lb # [ 0.444429] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1247client # [ 0.438918] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1248worker2 # [ 0.428181] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1249worker1 # [ 0.447378] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1250client # [ 0.439783] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1251lb # [ 0.445600] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1252worker1 # [ 0.448026] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1253lb # [ 0.446719] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1254worker2 # [ 0.430054] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1255client # [ 0.440640] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1256worker2 # [ 0.430883] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1257lb # [ 0.447682] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1258client # [ 0.441694] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1259worker1 # [ 0.449312] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1260worker2 # [ 0.431701] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1261worker1 # [ 0.449990] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1262client # [ 0.442273] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1263lb # [ 0.449988] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1264worker1 # [ 0.451026] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1265lb # [ 0.450430] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1266client # [ 0.443592] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1267worker2 # [ 0.433168] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1268worker1 # [ 0.451990] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1269lb # [ 0.451648] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1270worker2 # [ 0.434106] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1271worker2 # [ 0.434918] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1272lb # [ 0.453300] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1273worker1 # [ 0.453627] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1274worker1 # [ 0.454564] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1275worker2 # [ 0.435714] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1276worker1 # [ 0.455838] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1277lb # [ 0.456980] ACPI: PCI: Interrupt link LNKA configured for IRQ 101278worker2 # [ 0.437195] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1279worker1 # [ 0.457514] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1280worker2 # [ 0.438045] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1281lb # [ 0.457537] ACPI: PCI: Interrupt link LNKB configured for IRQ 101282worker2 # [ 0.438920] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1283lb # [ 0.458524] ACPI: PCI: Interrupt link LNKC configured for IRQ 111284lb # [ 0.459533] ACPI: PCI: Interrupt link LNKD configured for IRQ 111285worker2 # [ 0.440643] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1286lb # [ 0.460522] ACPI: PCI: Interrupt link LNKE configured for IRQ 101287worker1 # [ 0.460793] ACPI: PCI: Interrupt link LNKA configured for IRQ 101288lb # [ 0.461524] ACPI: PCI: Interrupt link LNKF configured for IRQ 101289worker1 # [ 0.461837] ACPI: PCI: Interrupt link LNKB configured for IRQ 101290client # [ 0.444252] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1291worker2 # [ 0.441843] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1292lb # [ 0.462523] ACPI: PCI: Interrupt link LNKG configured for IRQ 111293worker1 # [ 0.462834] ACPI: PCI: Interrupt link LNKC configured for IRQ 111294lb # [ 0.463531] ACPI: PCI: Interrupt link LNKH configured for IRQ 111295client # [ 0.446627] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1296worker1 # [ 0.463833] ACPI: PCI: Interrupt link LNKD configured for IRQ 111297worker2 # [ 0.442368] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1298lb # [ 0.464456] ACPI: PCI: Interrupt link GSIA configured for IRQ 161299worker1 # [ 0.464831] ACPI: PCI: Interrupt link LNKE configured for IRQ 101300worker2 # [ 0.443712] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1301client # [ 0.447302] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1302lb # [ 0.465379] ACPI: PCI: Interrupt link GSIB configured for IRQ 171303worker1 # [ 0.465866] ACPI: PCI: Interrupt link LNKF configured for IRQ 101304lb # [ 0.466114] ACPI: PCI: Interrupt link GSIC configured for IRQ 181305client # [ 0.448597] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1306worker1 # [ 0.466846] ACPI: PCI: Interrupt link LNKG configured for IRQ 111307lb # [ 0.466435] ACPI: PCI: Interrupt link GSID configured for IRQ 191308lb # [ 0.467439] ACPI: PCI: Interrupt link GSIE configured for IRQ 201309worker1 # [ 0.467834] ACPI: PCI: Interrupt link LNKH configured for IRQ 111310client # [ 0.449329] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1311lb # [ 0.468407] ACPI: PCI: Interrupt link GSIF configured for IRQ 211312worker1 # [ 0.468769] ACPI: PCI: Interrupt link GSIA configured for IRQ 161313lb # [ 0.469142] ACPI: PCI: Interrupt link GSIG configured for IRQ 221314worker1 # [ 0.469747] ACPI: PCI: Interrupt link GSIB configured for IRQ 171315client # [ 0.450349] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1316lb # [ 0.469438] ACPI: PCI: Interrupt link GSIH configured for IRQ 231317worker1 # [ 0.470740] ACPI: PCI: Interrupt link GSIC configured for IRQ 181318client # [ 0.451319] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1319worker1 # [ 0.471753] ACPI: PCI: Interrupt link GSID configured for IRQ 191320lb # [ 0.471509] iommu: Default domain type: Translated1321worker1 # [ 0.472716] ACPI: PCI: Interrupt link GSIE configured for IRQ 201322client # [ 0.452868] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1323lb # [ 0.472346] iommu: DMA domain TLB invalidation policy: lazy mode1324worker1 # [ 0.473510] ACPI: PCI: Interrupt link GSIF configured for IRQ 211325client # [ 0.453727] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1326lb # [ 0.473411] ACPI: bus type USB registered1327worker1 # [ 0.473750] ACPI: PCI: Interrupt link GSIG configured for IRQ 221328lb # [ 0.473962] usbcore: registered new interface driver usbfs1329worker1 # [ 0.474746] ACPI: PCI: Interrupt link GSIH configured for IRQ 231330client # [ 0.454383] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1331lb # [ 0.474438] usbcore: registered new interface driver hub1332lb # [ 0.475284] usbcore: registered new device driver usb1333worker2 # [ 0.444370] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1334client # [ 0.455858] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1335worker1 # [ 0.476865] iommu: Default domain type: Translated1336lb # [ 0.476974] NetLabel: Initializing1337worker1 # [ 0.477706] iommu: DMA domain TLB invalidation policy: lazy mode1338worker2 # [ 0.446714] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1339lb # [ 0.477424] NetLabel: domain hash size = 1281340worker1 # [ 0.478799] ACPI: bus type USB registered1341lb # [ 0.478143] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1342worker2 # [ 0.447391] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1343worker1 # [ 0.479569] usbcore: registered new interface driver usbfs1344lb # [ 0.478463] NetLabel: unlabeled traffic allowed by default1345worker1 # [ 0.480493] usbcore: registered new interface driver hub1346lb # [ 0.479343] PCI: Using ACPI for IRQ routing1347client # [ 0.460073] ACPI: PCI: Interrupt link LNKA configured for IRQ 101348worker2 # [ 0.448733] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1349worker1 # [ 0.480745] usbcore: registered new device driver usb1350client # [ 0.461159] ACPI: PCI: Interrupt link LNKB configured for IRQ 101351worker2 # [ 0.449478] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1352worker1 # [ 0.482493] NetLabel: Initializing1353client # [ 0.462157] ACPI: PCI: Interrupt link LNKC configured for IRQ 111354worker1 # [ 0.482741] NetLabel: domain hash size = 1281355client # [ 0.463157] ACPI: PCI: Interrupt link LNKD configured for IRQ 111356worker2 # [ 0.450448] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1357worker1 # [ 0.483494] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1358client # [ 0.464152] ACPI: PCI: Interrupt link LNKE configured for IRQ 101359worker1 # [ 0.483771] NetLabel: unlabeled traffic allowed by default1360worker2 # [ 0.451414] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1361client # [ 0.465152] ACPI: PCI: Interrupt link LNKF configured for IRQ 101362worker1 # [ 0.484691] PCI: Using ACPI for IRQ routing1363worker2 # [ 0.453030] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1364client # [ 0.466156] ACPI: PCI: Interrupt link LNKG configured for IRQ 111365worker2 # [ 0.453802] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1366client # [ 0.467157] ACPI: PCI: Interrupt link LNKH configured for IRQ 111367client # [ 0.468097] ACPI: PCI: Interrupt link GSIA configured for IRQ 161368worker2 # [ 0.454489] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1369client # [ 0.469068] ACPI: PCI: Interrupt link GSIB configured for IRQ 171370worker2 # [ 0.455751] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1371client # [ 0.470062] ACPI: PCI: Interrupt link GSIC configured for IRQ 181372client # [ 0.471040] ACPI: PCI: Interrupt link GSID configured for IRQ 191373client # [ 0.471863] ACPI: PCI: Interrupt link GSIE configured for IRQ 201374client # [ 0.472862] ACPI: PCI: Interrupt link GSIF configured for IRQ 211375worker2 # [ 0.459778] ACPI: PCI: Interrupt link LNKA configured for IRQ 101376client # [ 0.473857] ACPI: PCI: Interrupt link GSIG configured for IRQ 221377worker2 # [ 0.460266] ACPI: PCI: Interrupt link LNKB configured for IRQ 101378client # [ 0.474847] ACPI: PCI: Interrupt link GSIH configured for IRQ 231379worker2 # [ 0.461262] ACPI: PCI: Interrupt link LNKC configured for IRQ 111380client # [ 0.476958] iommu: Default domain type: Translated1381worker2 # [ 0.462255] ACPI: PCI: Interrupt link LNKD configured for IRQ 111382client # [ 0.477656] iommu: DMA domain TLB invalidation policy: lazy mode1383worker2 # [ 0.463254] ACPI: PCI: Interrupt link LNKE configured for IRQ 101384client # [ 0.478315] ACPI: bus type USB registered1385worker2 # [ 0.464258] ACPI: PCI: Interrupt link LNKF configured for IRQ 101386client # [ 0.479110] usbcore: registered new interface driver usbfs1387worker2 # [ 0.465256] ACPI: PCI: Interrupt link LNKG configured for IRQ 111388client # [ 0.480034] usbcore: registered new interface driver hub1389worker2 # [ 0.466288] ACPI: PCI: Interrupt link LNKH configured for IRQ 111390client # [ 0.480751] usbcore: registered new device driver usb1391worker2 # [ 0.467198] ACPI: PCI: Interrupt link GSIA configured for IRQ 161392worker2 # [ 0.468170] ACPI: PCI: Interrupt link GSIB configured for IRQ 171393client # [ 0.481936] NetLabel: Initializing1394worker2 # [ 0.469108] ACPI: PCI: Interrupt link GSIC configured for IRQ 181395client # [ 0.482578] NetLabel: domain hash size = 1281396worker2 # [ 0.469873] ACPI: PCI: Interrupt link GSID configured for IRQ 191397client # [ 0.483054] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1398worker2 # [ 0.470170] ACPI: PCI: Interrupt link GSIE configured for IRQ 201399client # [ 0.484066] NetLabel: unlabeled traffic allowed by default1400client # [ 0.485024] PCI: Using ACPI for IRQ routing1401worker2 # [ 0.471089] ACPI: PCI: Interrupt link GSIF configured for IRQ 211402worker2 # [ 0.471868] ACPI: PCI: Interrupt link GSIG configured for IRQ 221403worker2 # [ 0.472173] ACPI: PCI: Interrupt link GSIH configured for IRQ 231404worker2 # [ 0.474215] iommu: Default domain type: Translated1405worker2 # [ 0.475003] iommu: DMA domain TLB invalidation policy: lazy mode1406worker2 # [ 0.476142] ACPI: bus type USB registered1407worker2 # [ 0.476707] usbcore: registered new interface driver usbfs1408worker2 # [ 0.477175] usbcore: registered new interface driver hub1409worker2 # [ 0.478031] usbcore: registered new device driver usb1410worker2 # [ 0.479614] NetLabel: Initializing1411worker2 # [ 0.480159] NetLabel: domain hash size = 1281412worker2 # [ 0.480871] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1413worker2 # [ 0.481201] NetLabel: unlabeled traffic allowed by default1414worker2 # [ 0.482096] PCI: Using ACPI for IRQ routing1415lb # [ 0.523604] pci 0000:00:01.0: vgaarb: setting as boot VGA device1416lb # [ 0.524418] pci 0000:00:01.0: vgaarb: bridge control possible1417lb # [ 0.524418] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1418lb # [ 0.524428] vgaarb: loaded1419lb # [ 0.525153] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01420lb # [ 0.525424] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1421worker1 # [ 0.529397] pci 0000:00:01.0: vgaarb: setting as boot VGA device1422worker1 # [ 0.529730] pci 0000:00:01.0: vgaarb: bridge control possible1423worker1 # [ 0.529730] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1424lb # [ 0.529509] clocksource: Switched to clocksource kvm-clock1425worker1 # [ 0.529738] vgaarb: loaded1426worker1 # [ 0.530460] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01427worker1 # [ 0.530734] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1428lb # [ 0.533023] VFS: Disk quotas dquot_6.6.01429lb # [ 0.533716] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1430lb # [ 0.535049] pnp: PnP ACPI init1431lb # [ 0.535840] ACPI: IRQ 4 override to edge(!), high(!)1432lb # [ 0.536834] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1433lb # [ 0.538306] pnp: PnP ACPI: found 6 devices1434worker1 # [ 0.537870] clocksource: Switched to clocksource kvm-clock1435worker1 # [ 0.541368] VFS: Disk quotas dquot_6.6.01436worker1 # [ 0.542089] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1437worker1 # [ 0.543417] pnp: PnP ACPI init1438worker1 # [ 0.544353] ACPI: IRQ 4 override to edge(!), high(!)1439lb # [ 0.546009] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1440worker1 # [ 0.545500] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1441lb # [ 0.547430] clocksource: Switched to clocksource acpi_pm1442worker1 # [ 0.547098] pnp: PnP ACPI: found 6 devices1443lb # [ 0.548368] NET: Registered PF_INET protocol family1444lb # [ 0.549556] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1445client # [ 0.533051] pci 0000:00:01.0: vgaarb: setting as boot VGA device1446client # [ 0.533051] pci 0000:00:01.0: vgaarb: bridge control possible1447client # [ 0.533051] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1448client # [ 0.533059] vgaarb: loaded1449worker1 # [ 0.554741] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1450client # [ 0.533832] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01451worker1 # [ 0.556213] clocksource: Switched to clocksource acpi_pm1452client # [ 0.534056] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1453worker1 # [ 0.557184] NET: Registered PF_INET protocol family1454worker1 # [ 0.558340] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1455worker2 # [ 0.527480] pci 0000:00:01.0: vgaarb: setting as boot VGA device1456client # [ 0.538156] clocksource: Switched to clocksource kvm-clock1457worker2 # [ 0.528155] pci 0000:00:01.0: vgaarb: bridge control possible1458worker2 # [ 0.528155] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1459worker2 # [ 0.528163] vgaarb: loaded1460client # [ 0.541774] VFS: Disk quotas dquot_6.6.01461worker2 # [ 0.528896] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01462lb # [ 0.566393] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1463worker2 # [ 0.529160] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1464client # [ 0.542584] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1465client # [ 0.543976] pnp: PnP ACPI init1466lb # [ 0.567896] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1467client # [ 0.544796] ACPI: IRQ 4 override to edge(!), high(!)1468lb # [ 0.569211] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1469client # [ 0.545847] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1470lb # [ 0.570498] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1471worker2 # [ 0.532254] clocksource: Switched to clocksource kvm-clock1472client # [ 0.547418] pnp: PnP ACPI: found 6 devices1473lb # [ 0.571691] TCP: Hash tables configured (established 16384 bind 16384)1474lb # [ 0.572806] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1475lb # [ 0.574030] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1476worker2 # [ 0.535853] VFS: Disk quotas dquot_6.6.01477lb # [ 0.575094] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1478worker2 # [ 0.536566] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1479lb # [ 0.576278] NET: Registered PF_UNIX/PF_LOCAL protocol family1480worker2 # [ 0.537894] pnp: PnP ACPI init1481worker1 # [ 0.575135] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1482lb # [ 0.577185] NET: Registered PF_XDP protocol family1483worker2 # [ 0.538677] ACPI: IRQ 4 override to edge(!), high(!)1484worker1 # [ 0.576534] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1485lb # [ 0.577989] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1486worker2 # [ 0.539598] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1487lb # [ 0.578955] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1488worker1 # [ 0.577830] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1489worker2 # [ 0.541105] pnp: PnP ACPI: found 6 devices1490client # [ 0.555084] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1491lb # [ 0.579919] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1492worker1 # [ 0.579170] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1493client # [ 0.556570] clocksource: Switched to clocksource acpi_pm1494lb # [ 0.580960] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1495worker1 # [ 0.580360] TCP: Hash tables configured (established 16384 bind 16384)1496client # [ 0.557595] NET: Registered PF_INET protocol family1497lb # [ 0.582020] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1498worker1 # [ 0.581547] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1499lb # [ 0.583085] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1500client # [ 0.558806] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1501worker1 # [ 0.583003] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1502lb # [ 0.584731] ACPI: \_SB_.GSIA: Enabled at IRQ 161503worker1 # [ 0.584214] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1504worker1 # [ 0.585544] NET: Registered PF_UNIX/PF_LOCAL protocol family1505lb # [ 0.587150] ACPI: \_SB_.GSIB: Enabled at IRQ 171506worker1 # [ 0.586628] NET: Registered PF_XDP protocol family1507worker2 # [ 0.548786] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1508worker1 # [ 0.587567] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1509lb # [ 0.589131] ACPI: \_SB_.GSIC: Enabled at IRQ 181510worker2 # [ 0.550193] clocksource: Switched to clocksource acpi_pm1511worker1 # [ 0.588660] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1512worker2 # [ 0.551232] NET: Registered PF_INET protocol family1513lb # [ 0.591041] ACPI: \_SB_.GSID: Enabled at IRQ 191514worker1 # [ 0.589749] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1515worker2 # [ 0.552407] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1516worker1 # [ 0.591003] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1517lb # [ 0.592661] PCI: CLS 0 bytes, default 641518worker1 # [ 0.592229] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1519lb # [ 0.593612] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1520worker1 # [ 0.593444] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1521lb # [ 0.595327] Trying to unpack rootfs image as initramfs...1522worker1 # [ 0.595429] ACPI: \_SB_.GSIA: Enabled at IRQ 161523worker1 # [ 0.598221] ACPI: \_SB_.GSIB: Enabled at IRQ 171524client # [ 0.575172] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1525worker1 # [ 0.600275] ACPI: \_SB_.GSIC: Enabled at IRQ 181526client # [ 0.576703] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1527client # [ 0.578016] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1528worker1 # [ 0.602340] ACPI: \_SB_.GSID: Enabled at IRQ 191529client # [ 0.579348] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1530worker1 # [ 0.604192] PCI: CLS 0 bytes, default 641531client # [ 0.580592] TCP: Hash tables configured (established 16384 bind 16384)1532client # [ 0.581753] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1533worker1 # [ 0.605198] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1534client # [ 0.583041] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1535worker1 # [ 0.607161] Trying to unpack rootfs image as initramfs...1536client # [ 0.584190] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1537worker2 # [ 0.569108] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1538client # [ 0.585414] NET: Registered PF_UNIX/PF_LOCAL protocol family1539worker2 # [ 0.570871] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1540client # [ 0.586388] NET: Registered PF_XDP protocol family1541worker2 # [ 0.572217] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1542client # [ 0.587250] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1543client # [ 0.588289] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1544worker2 # [ 0.573554] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1545client # [ 0.589365] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1546worker2 # [ 0.574791] TCP: Hash tables configured (established 16384 bind 16384)1547client # [ 0.590565] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1548worker2 # [ 0.575973] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1549client # [ 0.591759] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1550worker2 # [ 0.577287] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1551client # [ 0.592947] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1552worker2 # [ 0.578366] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1553worker2 # [ 0.579565] NET: Registered PF_UNIX/PF_LOCAL protocol family1554client # [ 0.594635] ACPI: \_SB_.GSIA: Enabled at IRQ 161555worker2 # [ 0.580507] NET: Registered PF_XDP protocol family1556worker2 # [ 0.581317] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1557worker2 # [ 0.582299] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1558client # [ 0.597236] ACPI: \_SB_.GSIB: Enabled at IRQ 171559worker2 # [ 0.583249] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1560client # [ 0.599232] ACPI: \_SB_.GSIC: Enabled at IRQ 181561worker2 # [ 0.584345] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1562worker2 # [ 0.585404] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1563client # [ 0.601186] ACPI: \_SB_.GSID: Enabled at IRQ 191564worker2 # [ 0.586461] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1565client # [ 0.602815] PCI: CLS 0 bytes, default 641566worker2 # [ 0.588152] ACPI: \_SB_.GSIA: Enabled at IRQ 161567client # [ 0.603716] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1568worker2 # [ 0.590732] ACPI: \_SB_.GSIB: Enabled at IRQ 171569client # [ 0.605441] Trying to unpack rootfs image as initramfs...1570worker2 # [ 0.592654] ACPI: \_SB_.GSIC: Enabled at IRQ 181571worker2 # [ 0.594669] ACPI: \_SB_.GSID: Enabled at IRQ 191572worker2 # [ 0.596441] PCI: CLS 0 bytes, default 641573worker2 # [ 0.597422] Trying to unpack rootfs image as initramfs...1574lb # [ 0.640636] Initialise system trusted keyrings1575worker2 # [ 0.601542] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1576lb # [ 0.643816] workingset: timestamp_bits=40 max_order=19 bucket_order=01577worker1 # [ 0.652377] Initialise system trusted keyrings1578worker1 # [ 0.656047] workingset: timestamp_bits=40 max_order=19 bucket_order=01579lb # [ 0.666926] Key type asymmetric registered1580lb # [ 0.667634] Asymmetric key parser 'x509' registered1581lb # [ 0.668475] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1582lb # [ 0.673933] io scheduler mq-deadline registered1583lb # [ 0.674691] io scheduler kyber registered1584client # [ 0.651282] Initialise system trusted keyrings1585lb # [ 0.677396] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1586client # [ 0.652302] workingset: timestamp_bits=40 max_order=19 bucket_order=01587worker1 # [ 0.677255] Key type asymmetric registered1588lb # [ 0.678687] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1589worker1 # [ 0.678094] Asymmetric key parser 'x509' registered1590worker1 # [ 0.681089] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1591worker2 # [ 0.644784] Initialise system trusted keyrings1592lb # [ 0.683920] Linux agpgart interface v0.1031593lb # [ 0.684655] ACPI: bus type drm_connector registered1594worker1 # [ 0.684077] io scheduler mq-deadline registered1595worker1 # [ 0.684902] io scheduler kyber registered1596worker2 # [ 0.648342] workingset: timestamp_bits=40 max_order=19 bucket_order=01597lb # [ 0.687915] usbcore: registered new interface driver usbserial_generic1598lb # [ 0.688944] usbserial: USB Serial support registered for generic1599worker1 # [ 0.689014] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1600worker1 # [ 0.690449] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1601lb # [ 0.691779] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1602lb # [ 0.691779] Please enable it if your BIOS has the CPPC option.1603lb # [ 0.693967] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1604worker1 # [ 0.695838] Linux agpgart interface v0.1031605worker1 # [ 0.696694] ACPI: bus type drm_connector registered1606lb # [ 0.697893] drop_monitor: Initializing network drop monitor service1607lb # [ 0.699044] NET: Registered PF_INET6 protocol family1608client # [ 0.676012] Key type asymmetric registered1609client # [ 0.676727] Asymmetric key parser 'x509' registered1610worker1 # [ 0.700534] usbcore: registered new interface driver usbserial_generic1611lb # [ 0.702227] Segment Routing with IPv61612client # [ 0.677600] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1613worker1 # [ 0.701756] usbserial: USB Serial support registered for generic1614lb # [ 0.704794] In-situ OAM (IOAM) with IPv61615lb # [ 0.705845] IPI shorthand broadcast: enabled1616worker2 # [ 0.667674] Key type asymmetric registered1617worker1 # [ 0.705003] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1618client # [ 0.683016] io scheduler mq-deadline registered1619worker1 # [ 0.705003] Please enable it if your BIOS has the CPPC option.1620client # [ 0.683772] io scheduler kyber registered1621worker1 # [ 0.707437] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1622worker2 # [ 0.672300] Asymmetric key parser 'x509' registered1623client # [ 0.687056] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1624worker2 # [ 0.673207] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1625client # [ 0.688319] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1626lb # [ 0.713126] sched_clock: Marking stable (579013386, 133741861)->(782730797, -69975550)1627worker1 # [ 0.713155] drop_monitor: Initializing network drop monitor service1628worker2 # [ 0.676464] io scheduler mq-deadline registered1629worker1 # [ 0.714463] NET: Registered PF_INET6 protocol family1630worker2 # [ 0.677220] io scheduler kyber registered1631client # [ 0.693689] Linux agpgart interface v0.1031632lb # [ 0.718150] registered taskstats version 11633worker1 # [ 0.717479] Segment Routing with IPv61634client # [ 0.694465] ACPI: bus type drm_connector registered1635lb # [ 0.719095] Loading compiled-in X.509 certificates1636worker2 # [ 0.680045] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1637worker1 # [ 0.720022] In-situ OAM (IOAM) with IPv61638worker2 # [ 0.682296] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1639client # [ 0.697334] usbcore: registered new interface driver usbserial_generic1640worker1 # [ 0.721215] IPI shorthand broadcast: enabled1641client # [ 0.698400] usbserial: USB Serial support registered for generic1642worker2 # [ 0.687614] Linux agpgart interface v0.1031643client # [ 0.701871] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1644worker2 # [ 0.688446] ACPI: bus type drm_connector registered1645client # [ 0.701871] Please enable it if your BIOS has the CPPC option.1646client # [ 0.704046] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1647worker2 # [ 0.691786] usbcore: registered new interface driver usbserial_generic1648worker1 # [ 0.729548] sched_clock: Marking stable (595014936, 133976480)->(799919910, -70928494)1649worker2 # [ 0.692897] usbserial: USB Serial support registered for generic1650client # [ 0.708011] drop_monitor: Initializing network drop monitor service1651client # [ 0.709153] NET: Registered PF_INET6 protocol family1652worker2 # [ 0.696304] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1653worker1 # [ 0.735142] registered taskstats version 11654client # [ 0.712348] Segment Routing with IPv61655worker2 # [ 0.696304] Please enable it if your BIOS has the CPPC option.1656worker1 # [ 0.736223] Loading compiled-in X.509 certificates1657worker2 # [ 0.698514] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1658lb # [ 0.738773] Demotion targets for Node 0: null1659client # [ 0.714887] In-situ OAM (IOAM) with IPv61660lb # [ 0.739638] Key type .fscrypt registered1661client # [ 0.715976] IPI shorthand broadcast: enabled1662lb # [ 0.740331] Key type fscrypt-provisioning registered1663worker2 # [ 0.702490] drop_monitor: Initializing network drop monitor service1664worker2 # [ 0.703663] NET: Registered PF_INET6 protocol family1665lb # [ 0.743907] ima: No TPM chip found, activating TPM-bypass!1666lb # [ 0.744813] ima: Allocated hash algorithm: sha11667lb # [ 0.745580] ima: No architecture policies found1668worker2 # [ 0.706857] Segment Routing with IPv61669worker2 # [ 0.707537] In-situ OAM (IOAM) with IPv61670lb # [ 0.747956] PM: Magic number: 10:928:5841671client # [ 0.724243] sched_clock: Marking stable (587014931, 136827026)->(798614907, -74772950)1672worker2 # [ 0.710596] IPI shorthand broadcast: enabled1673lb # [ 0.749570] RAS: Correctable Errors collector initialized.1674client # [ 0.729035] registered taskstats version 11675client # [ 0.729990] Loading compiled-in X.509 certificates1676worker1 # [ 0.755009] Demotion targets for Node 0: null1677worker1 # [ 0.756116] Key type .fscrypt registered1678worker1 # [ 0.756865] Key type fscrypt-provisioning registered1679worker2 # [ 0.718935] sched_clock: Marking stable (585015182, 133264523)->(791819811, -73540106)1680lb # [ 0.759443] clk: Disabling unused clocks1681worker1 # [ 0.760133] ima: No TPM chip found, activating TPM-bypass!1682worker2 # [ 0.722750] registered taskstats version 11683lb # [ 0.761780] PM: genpd: Disabling unused power domains1684worker1 # [ 0.761166] ima: Allocated hash algorithm: sha11685worker2 # [ 0.723703] Loading compiled-in X.509 certificates1686worker1 # [ 0.764074] ima: No architecture policies found1687worker1 # [ 0.765151] PM: Magic number: 10:928:5841688worker1 # [ 0.766869] RAS: Correctable Errors collector initialized.1689client # [ 0.748864] Demotion targets for Node 0: null1690client # [ 0.749755] Key type .fscrypt registered1691client # [ 0.750436] Key type fscrypt-provisioning registered1692client # [ 0.753974] ima: No TPM chip found, activating TPM-bypass!1693worker1 # [ 0.777731] clk: Disabling unused clocks1694client # [ 0.754882] ima: Allocated hash algorithm: sha11695client # [ 0.755647] ima: No architecture policies found1696worker1 # [ 0.780006] PM: genpd: Disabling unused power domains1697worker2 # [ 0.742322] Demotion targets for Node 0: null1698worker2 # [ 0.743348] Key type .fscrypt registered1699client # [ 0.758061] PM: Magic number: 10:928:5841700worker2 # [ 0.744020] Key type fscrypt-provisioning registered1701client # [ 0.759685] RAS: Correctable Errors collector initialized.1702worker2 # [ 0.747412] ima: No TPM chip found, activating TPM-bypass!1703worker2 # [ 0.748323] ima: Allocated hash algorithm: sha11704worker2 # [ 0.749153] ima: No architecture policies found1705worker2 # [ 0.753511] PM: Magic number: 10:928:5841706client # [ 0.768514] clk: Disabling unused clocks1707worker2 # [ 0.755175] RAS: Correctable Errors collector initialized.1708client # [ 0.772862] PM: genpd: Disabling unused power domains1709worker2 # [ 0.763008] clk: Disabling unused clocks1710worker2 # [ 0.767311] PM: genpd: Disabling unused power domains1711lb # [ 0.926841] Freeing initrd memory: 29092K1712lb # [ 0.929995] Freeing unused decrypted memory: 2028K1713lb # [ 0.932557] Freeing unused kernel image (initmem) memory: 3644K1714lb # [ 0.933626] Write protecting the kernel read-only data: 32768k1715lb # [ 0.935478] Freeing unused kernel image (text/rodata gap) memory: 1208K1716lb # [ 0.936963] Freeing unused kernel image (rodata/data gap) memory: 752K1717worker1 # [ 0.945819] Freeing initrd memory: 29092K1718worker1 # [ 0.949223] Freeing unused decrypted memory: 2028K1719worker1 # [ 0.951896] Freeing unused kernel image (initmem) memory: 3644K1720worker1 # [ 0.953107] Write protecting the kernel read-only data: 32768k1721worker1 # [ 0.955066] Freeing unused kernel image (text/rodata gap) memory: 1208K1722worker1 # [ 0.956690] Freeing unused kernel image (rodata/data gap) memory: 752K1723client # [ 0.939193] Freeing initrd memory: 29096K1724client # [ 0.942435] Freeing unused decrypted memory: 2028K1725client # [ 0.945117] Freeing unused kernel image (initmem) memory: 3644K1726client # [ 0.946199] Write protecting the kernel read-only data: 32768k1727client # [ 0.948142] Freeing unused kernel image (text/rodata gap) memory: 1208K1728client # [ 0.949625] Freeing unused kernel image (rodata/data gap) memory: 752K1729worker2 # [ 0.940488] Freeing initrd memory: 29092K1730worker2 # [ 0.943839] Freeing unused decrypted memory: 2028K1731worker2 # [ 0.946531] Freeing unused kernel image (initmem) memory: 3644K1732worker2 # [ 0.947630] Write protecting the kernel read-only data: 32768k1733lb # [ 0.988456] x86/mm: Checked W+X mappings: passed, no W+X pages found.1734worker2 # [ 0.949601] Freeing unused kernel image (text/rodata gap) memory: 1208K1735lb # [ 0.989494] Run /init as init process1736worker2 # [ 0.951081] Freeing unused kernel image (rodata/data gap) memory: 752K1737lb # [ 0.999710] systemd[1]: Inserted module 'autofs4'1738worker1 # [ 1.007720] x86/mm: Checked W+X mappings: passed, no W+X pages found.1739worker1 # [ 1.008994] Run /init as init process1740lb # [ 1.016213] fuse: init (API version 7.45)1741worker1 # [ 1.019801] systemd[1]: Inserted module 'autofs4'1742lb # [ 1.022225] ACPI: \_SB_.GSIG: Enabled at IRQ 221743lb # [ 1.024637] ACPI: \_SB_.GSIH: Enabled at IRQ 231744client # [ 1.001027] x86/mm: Checked W+X mappings: passed, no W+X pages found.1745client # [ 1.002079] Run /init as init process1746lb # [ 1.027636] ACPI: \_SB_.GSIE: Enabled at IRQ 201747lb # [ 1.029838] ACPI: \_SB_.GSIF: Enabled at IRQ 211748client # [ 1.013256] systemd[1]: Inserted module 'autofs4'1749worker1 # [ 1.036707] fuse: init (API version 7.45)1750worker2 # [ 1.002814] x86/mm: Checked W+X mappings: passed, no W+X pages found.1751worker2 # [ 1.003949] Run /init as init process1752worker1 # [ 1.042743] ACPI: \_SB_.GSIG: Enabled at IRQ 221753worker1 # [ 1.045562] ACPI: \_SB_.GSIH: Enabled at IRQ 231754worker1 # [ 1.049156] ACPI: \_SB_.GSIE: Enabled at IRQ 201755worker1 # [ 1.051610] ACPI: \_SB_.GSIF: Enabled at IRQ 211756worker2 # [ 1.014183] systemd[1]: Inserted module 'autofs4'1757client # [ 1.029685] fuse: init (API version 7.45)1758lb # [ 1.055583] systemd[1]: Successfully made /usr/ read-only.1759client # [ 1.035588] ACPI: \_SB_.GSIG: Enabled at IRQ 221760client # [ 1.038234] ACPI: \_SB_.GSIH: Enabled at IRQ 231761client # [ 1.041478] ACPI: \_SB_.GSIE: Enabled at IRQ 201762client # [ 1.044029] ACPI: \_SB_.GSIF: Enabled at IRQ 211763worker2 # [ 1.030828] fuse: init (API version 7.45)1764worker2 # [ 1.036659] ACPI: \_SB_.GSIG: Enabled at IRQ 221765worker2 # [ 1.039313] ACPI: \_SB_.GSIH: Enabled at IRQ 231766worker1 # [ 1.079322] systemd[1]: Successfully made /usr/ read-only.1767worker2 # [ 1.043255] ACPI: \_SB_.GSIE: Enabled at IRQ 201768worker2 # [ 1.046145] ACPI: \_SB_.GSIF: Enabled at IRQ 211769client # [ 1.071739] systemd[1]: Successfully made /usr/ read-only.1770worker2 # [ 1.074147] systemd[1]: Successfully made /usr/ read-only.1771lb # [ 1.392057] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1772lb # [ 1.404026] systemd[1]: Detected virtualization kvm.1773lb # [ 1.406070] systemd[1]: Detected architecture x86-64.1774lb # [ 1.408146] systemd[1]: Running in initrd.1775lb # [ 1.410537] systemd[1]: Initializing machine ID from random generator.1776lb # [ 1.413340] systemd[1]: Hostname set to <lb>.1777worker1 # [ 1.416210] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1778worker1 # [ 1.428812] systemd[1]: Detected virtualization kvm.1779worker1 # [ 1.431284] systemd[1]: Detected architecture x86-64.1780worker1 # [ 1.433414] systemd[1]: Running in initrd.1781client # [ 1.407386] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1782client # [ 1.412549] systemd[1]: Detected virtualization kvm.1783client # [ 1.413461] systemd[1]: Detected architecture x86-64.1784client # [ 1.414381] systemd[1]: Running in initrd.1785worker1 # [ 1.436156] systemd[1]: Initializing machine ID from random generator.1786client # [ 1.415580] systemd[1]: Initializing machine ID from random generator.1787worker1 # [ 1.439246] systemd[1]: Hostname set to <worker1>.1788client # [ 1.417521] systemd[1]: Hostname set to <client>.1789worker2 # [ 1.410939] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1790worker2 # [ 1.422938] systemd[1]: Detected virtualization kvm.1791worker2 # [ 1.425106] systemd[1]: Detected architecture x86-64.1792worker2 # [ 1.427320] systemd[1]: Running in initrd.1793worker2 # [ 1.429814] systemd[1]: Initializing machine ID from random generator.1794worker2 # [ 1.432701] systemd[1]: Hostname set to <worker2>.1795lb # [ 1.605010] systemd[1]: bpf-restrict-fs: LSM BPF program attached1796lb # [ 1.643707] systemd[1]: Queued start job for default target Initrd Default Target.1797lb # [ 1.648072] systemd[1]: Created slice Slice /system/modprobe.1798lb # [ 1.649216] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1799lb # [ 1.650596] systemd[1]: Expecting device /dev/disk/by-label/nixos...1800lb # [ 1.651766] systemd[1]: Reached target Path Units.1801lb # [ 1.652589] systemd[1]: Reached target Slice Units.1802lb # [ 1.653450] systemd[1]: Reached target Swaps.1803worker1 # [ 1.652807] systemd[1]: bpf-restrict-fs: LSM BPF program attached1804lb # [ 1.654220] systemd[1]: Reached target Timer Units.1805lb # [ 1.655148] systemd[1]: Listening on D-Bus System Message Bus Socket.1806lb # [ 1.656293] systemd[1]: Listening on Journal Socket (/dev/log).1807lb # [ 1.657498] systemd[1]: Listening on Journal Sockets.1808lb # [ 1.658492] systemd[1]: Listening on udev Control Socket.1809client # [ 1.633166] systemd[1]: bpf-restrict-fs: LSM BPF program attached1810lb # [ 1.659476] systemd[1]: Listening on udev Kernel Socket.1811lb # [ 1.660388] systemd[1]: Reached target Socket Units.1812lb # [ 1.662263] systemd[1]: Starting Create List of Static Device Nodes...1813lb # [ 1.665829] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1814lb # [ 1.671835] systemd[1]: Starting Load Kernel Module configfs...1815lb # [ 1.684479] systemd[1]: Starting Journal Service...1816lb # [ 1.696823] systemd[1]: Starting Load Kernel Modules...1817worker2 # [ 1.657534] systemd[1]: bpf-restrict-fs: LSM BPF program attached1818worker1 # [ 1.702572] systemd[1]: Queued start job for default target Initrd Default Target.1819client # [ 1.682023] systemd[1]: Queued start job for default target Initrd Default Target.1820worker1 # [ 1.707899] systemd[1]: Created slice Slice /system/modprobe.1821client # [ 1.686161] systemd[1]: Created slice Slice /system/modprobe.1822worker1 # [ 1.709515] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1823client # [ 1.687454] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1824worker1 # [ 1.711194] systemd[1]: Expecting device /dev/disk/by-label/nixos...1825client # [ 1.689027] systemd[1]: Expecting device /dev/disk/by-label/nixos...1826worker1 # [ 1.712573] systemd[1]: Reached target Path Units.1827client # [ 1.690198] systemd[1]: Reached target Path Units.1828worker1 # [ 1.713676] systemd[1]: Reached target Slice Units.1829lb # [ 1.702808] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1830client # [ 1.691080] systemd[1]: Reached target Slice Units.1831worker1 # [ 1.714787] systemd[1]: Reached target Swaps.1832client # [ 1.692057] systemd[1]: Reached target Swaps.1833worker1 # [ 1.715812] systemd[1]: Reached target Timer Units.1834client # [ 1.692981] systemd[1]: Reached target Timer Units.1835worker1 # [ 1.717118] systemd[1]: Listening on D-Bus System Message Bus Socket.1836client # [ 1.693998] systemd[1]: Listening on D-Bus System Message Bus Socket.1837client # [ 1.695290] systemd[1]: Listening on Journal Socket (/dev/log).1838worker1 # [ 1.718627] systemd[1]: Listening on Journal Socket (/dev/log).1839client # [ 1.696562] systemd[1]: Listening on Journal Sockets.1840worker1 # [ 1.720283] systemd[1]: Listening on Journal Sockets.1841client # [ 1.697665] systemd[1]: Listening on udev Control Socket.1842worker1 # [ 1.721538] systemd[1]: Listening on udev Control Socket.1843lb # [ 1.723004] netfs: FS-Cache loaded1844client # [ 1.698797] systemd[1]: Listening on udev Kernel Socket.1845worker1 # [ 1.722870] systemd[1]: Listening on udev Kernel Socket.1846client # [ 1.699894] systemd[1]: Reached target Socket Units.1847worker1 # [ 1.724132] systemd[1]: Reached target Socket Units.1848client # [ 1.701747] systemd[1]: Starting Create List of Static Device Nodes...1849worker1 # [ 1.726079] systemd[1]: Starting Create List of Static Device Nodes...1850lb # [ 1.727659] 9pnet: Installing 9P2000 support1851client # [ 1.705668] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1852worker1 # [ 1.729839] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1853client # [ 1.714111] systemd[1]: Starting Load Kernel Module configfs...1854worker1 # [ 1.738279] systemd[1]: Starting Load Kernel Module configfs...1855lb # [ 1.739668] systemd[1]: Starting Coldplug All udev Devices...1856worker2 # [ 1.712115] systemd[1]: Queued start job for default target Initrd Default Target.1857worker1 # [ 1.753684] systemd[1]: Starting Journal Service...1858worker2 # [ 1.716628] systemd[1]: Created slice Slice /system/modprobe.1859lb # [ 1.756360] systemd[1]: Finished Create List of Static Device Nodes.1860worker2 # [ 1.717761] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1861client # [ 1.733061] systemd[1]: Starting Journal Service...1862worker2 # [ 1.719109] systemd[1]: Expecting device /dev/disk/by-label/nixos...1863worker2 # [ 1.720169] systemd[1]: Reached target Path Units.1864worker2 # [ 1.721007] systemd[1]: Reached target Slice Units.1865worker2 # [ 1.721864] systemd[1]: Reached target Swaps.1866worker2 # [ 1.722654] systemd[1]: Reached target Timer Units.1867worker2 # [ 1.723590] systemd[1]: Listening on D-Bus System Message Bus Socket.1868lb # [ 1.762533] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1869worker2 # [ 1.724761] systemd[1]: Listening on Journal Socket (/dev/log).1870worker2 # [ 1.725952] systemd[1]: Listening on Journal Sockets.1871worker2 # [ 1.726938] systemd[1]: Listening on udev Control Socket.1872worker2 # [ 1.727921] systemd[1]: Listening on udev Kernel Socket.1873worker2 # [ 1.728840] systemd[1]: Reached target Socket Units.1874worker1 # [ 1.767108] systemd[1]: Starting Load Kernel Modules...1875worker2 # [ 1.730614] systemd[1]: Starting Create List of Static Device Nodes...1876lb # [ 1.772213] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1877worker2 # [ 1.734349] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1878client # [ 1.748981] systemd[1]: Starting Load Kernel Modules...1879lb # [ 1.774796] systemd-journald[67]: Collecting audit messages is disabled.1880lb # [ 1.777372] systemd[1]: modprobe@configfs.service: Deactivated successfully.1881worker2 # [ 1.743464] systemd[1]: Starting Load Kernel Module configfs...1882lb # [ 1.785290] systemd[1]: Finished Load Kernel Module configfs.1883worker1 # [ 1.774098] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1884lb # [ 1.788714] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1885worker1 # [ 1.790385] netfs: FS-Cache loaded1886lb # [ 1.792292] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1887worker2 # [ 1.756457] systemd[1]: Starting Journal Service...1888worker1 # [ 1.795405] 9pnet: Installing 9P2000 support1889client # [ 1.758969] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1890lb # [ 1.798982] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1891client # [ 1.777419] netfs: FS-Cache loaded1892lb # [ 1.804008] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1893worker1 # [ 1.805239] systemd[1]: Starting Coldplug All udev Devices...1894client # [ 1.782359] 9pnet: Installing 9P2000 support1895worker2 # [ 1.770380] systemd[1]: Starting Load Kernel Modules...1896client # [ 1.795168] systemd[1]: Starting Coldplug All udev Devices...1897worker1 # [ 1.821149] systemd[1]: Finished Create List of Static Device Nodes.1898worker1 # [ 1.825849] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1899worker2 # [ 1.777341] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1900worker2 # [ 1.793492] netfs: FS-Cache loaded1901lb # [ 1.832359] systemd[1]: Finished Load Kernel Modules.1902worker1 # [ 1.834474] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1903worker2 # [ 1.798371] 9pnet: Installing 9P2000 support1904client # [ 1.813945] systemd[1]: Finished Create List of Static Device Nodes.1905worker1 # [ 1.837447] systemd-journald[67]: Collecting audit messages is disabled.1906lb # [ 1.841013] systemd[1]: Starting Apply Kernel Variables...1907worker1 # [ 1.841428] systemd[1]: modprobe@configfs.service: Deactivated successfully.1908client # [ 1.820637] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1909worker1 # [ 1.846398] systemd[1]: Finished Load Kernel Module configfs.1910worker2 # [ 1.811432] systemd[1]: Starting Coldplug All udev Devices...1911client # [ 1.827955] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1912worker1 # [ 1.852486] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1913worker1 # [ 1.856765] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1914client # [ 1.834266] systemd-journald[67]: Collecting audit messages is disabled.1915lb # [ 1.859878] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1916client # [ 1.835990] systemd[1]: modprobe@configfs.service: Deactivated successfully.1917worker2 # [ 1.823413] systemd[1]: Finished Create List of Static Device Nodes.1918worker1 # [ 1.864265] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1919client # [ 1.843400] systemd[1]: Finished Load Kernel Module configfs.1920worker2 # [ 1.828895] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1921worker1 # [ 1.868201] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1922lb # [ 1.869864] systemd[1]: Starting Create Static Device Nodes in /dev...1923client # [ 1.847523] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1924worker2 # [ 1.836685] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1925client # [ 1.851743] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1926worker2 # [ 1.841723] systemd[1]: modprobe@configfs.service: Deactivated successfully.1927worker2 # [ 1.843252] systemd-journald[67]: Collecting audit messages is disabled.1928lb # [ 1.883436] systemd[1]: Finished Apply Kernel Variables.1929client # [ 1.860067] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1930client # [ 1.862051] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1931worker2 # [ 1.850560] systemd[1]: Finished Load Kernel Module configfs.1932lb # [ 1.759371] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.894249] systemd[1]: Started Journal Service.1933worker2 # [ 1.854789] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1934lb # 1935lb # [ 1.763242] systemd-modules-load[68]: Inserted module 'virtio_balloon'1936lb # [ 1.764328] systemd-modules-load[68]: Inserted module 'virtio_gpu'1937worker2 # [ 1.859694] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1938worker1 # [ 1.900632] systemd[1]: Finished Load Kernel Modules.1939worker2 # [ 1.867526] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1940worker2 # [ 1.868828] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1941worker1 # [ 1.909208] systemd[1]: Starting Apply Kernel Variables...1942lb # [ 1.780781] systemd[1]: Finished Create Static Device Nodes in /dev.1943lb # [ 1.782300] systemd[1]: Reached target Preparation for Local File Systems.1944lb # [ 1.784811] systemd[1]: Reached target Local File Systems.1945lb # [ 1.786361] systemd[1]: Starting Create System Files and Directories...1946client # [ 1.896934] systemd[1]: Finished Load Kernel Modules.1947worker1 # [ 1.922120] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1948lb # [ 1.791372] systemd[1]: Starting Rule-based Manager for Device Events and Files...1949client # [ 1.906271] systemd[1]: Starting Apply Kernel Variables...1950worker1 # [ 1.931418] systemd[1]: Starting Create Static Device Nodes in /dev...1951worker2 # [ 1.896812] systemd[1]: Finished Load Kernel Modules.1952worker2 # [ 1.905474] systemd[1]: Starting Apply Kernel Variables...1953client # [ 1.920988] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1954lb # [ 1.817474] systemd[1]: Finished Create System Files and Directories.1955worker1 # [ 1.954585] systemd[1]: Finished Apply Kernel Variables.1956worker2 # [ 1.919447] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1957worker1 # [ 1.959415] systemd[1]: Started Journal Service.1958client # [ 1.936959] systemd[1]: Starting Create Static Device Nodes in /dev...1959worker1 # [ 1.824890] systemd-modules-load[68]: Inserted module 'dm_mod'1960worker1 # [ 1.828955] systemd-modules-load[68]: Inserted module 'virtio_balloon'1961worker1 # [ 1.832206] systemd-modules-load[68]: Inserted module 'virtio_gpu'1962worker2 # [ 1.928481] systemd[1]: Starting Create Static Device Nodes in /dev...1963worker1 # [ 1.841948] systemd[1]: Finished Create Static Device Nodes in /dev.1964lb # [ 1.845483] systemd-udevd[83]: Using default interface naming scheme 'v261'.1965worker1 # [ 1.845071] systemd[1]: Reached target Preparation for Local File Systems.1966worker1 # [ 1.846059] systemd[1]: Reached target Local File Systems.1967worker2 # [ 1.943358] systemd[1]: Finished Apply Kernel Variables.1968worker1 # [ 1.849065] systemd[1]: Starting Create System Files and Directories...1969client # [ 1.963350] systemd[1]: Finished Apply Kernel Variables.1970client # [ 1.827857] systemd-modules-load[68]: Inserted module 'dm_mod'1971worker1 # [ 1.856073] systemd[1]: Starting Rule-based Manager for Device Events and Files...1972client # [ 1.831428] systemd-modules-load[68]: Inserted module 'virtio_balloon'1973client # [ 1.833316] systemd-modules-load[68]: Inserted module 'virtio_gpu'1974client # [ 1.971239] systemd[1]: Started Journal Service.1975worker2 # [ 1.957170] systemd[1]: Started Journal Service.1976worker2 # [ 1.827224] systemd-modules-load[68]: Inserted module 'dm_mod'1977worker2 # [ 1.828680] systemd-modules-load[68]: Inserted module 'virtio_balloon'1978lb # [ 1.869754] systemd[1]: Started Rule-based Manager for Device Events and Files.1979worker2 # [ 1.833294] systemd-modules-load[68]: Inserted module 'virtio_gpu'1980worker2 # [ 1.843159] systemd[1]: Finished Create Static Device Nodes in /dev.1981client # [ 1.855176] systemd[1]: Finished Create Static Device Nodes in /dev.1982worker2 # [ 1.844166] systemd[1]: Reached target Preparation for Local File Systems.1983worker2 # [ 1.845149] systemd[1]: Reached target Local File Systems.1984client # [ 1.856257] systemd[1]: Reached target Preparation for Local File Systems.1985client # [ 1.857243] systemd[1]: Reached target Local File Systems.1986worker1 # [ 1.883470] systemd[1]: Finished Create System Files and Directories.1987worker2 # [ 1.848029] systemd[1]: Starting Create System Files and Directories...1988client # [ 1.861079] systemd[1]: Starting Create System Files and Directories...1989worker2 # [ 1.855073] systemd[1]: Starting Rule-based Manager for Device Events and Files...1990client # [ 1.869072] systemd[1]: Starting Rule-based Manager for Device Events and Files...1991worker1 # [ 1.906287] systemd-udevd[83]: Using default interface naming scheme 'v261'.1992worker2 # [ 1.878963] systemd[1]: Finished Create System Files and Directories.1993lb # [ 1.918766] systemd[1]: Finished Coldplug All udev Devices.1994lb # [ 1.919776] systemd[1]: Reached target System Initialization.1995lb # [ 1.920564] systemd[1]: Reached target Basic System.1996client # [ 1.903528] systemd[1]: Finished Create System Files and Directories.1997worker1 # [ 1.929956] systemd[1]: Started Rule-based Manager for Device Events and Files.1998worker2 # [ 1.901357] systemd-udevd[83]: Using default interface naming scheme 'v261'.1999client # [ 1.925421] systemd-udevd[84]: Using default interface naming scheme 'v261'.2000worker2 # [ 1.926147] systemd[1]: Started Rule-based Manager for Device Events and Files.
loading 2,044 hidden lines…
4045lb # [ 16.601069] envoy[911]: [2026-09-20 13:33:20.441][911][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm4046lb # [ 16.606980] envoy[911]: [2026-09-20 13:33:20.441][911][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert4047lb # [ 16.608695] envoy[911]: [2026-09-20 13:33:20.441][911][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options4048lb # [ 16.612445] envoy[911]: [2026-09-20 13:33:20.441][911][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default4049lb # [ 16.614274] envoy[911]: [2026-09-20 13:33:20.441][911][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory4050lb # [ 16.617453] envoy[911]: [2026-09-20 13:33:20.444][911][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san4051lb # [ 16.624469] envoy[911]: [2026-09-20 13:33:20.444][911][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator4052lb # [ 16.627260] envoy[911]: [2026-09-20 13:33:20.444][911][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed4053lb # [ 16.629539] envoy[911]: [2026-09-20 13:33:20.444][911][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls4054lb # [ 16.634659] envoy[911]: [2026-09-20 13:33:20.444][911][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua4055lb # [ 16.636536] envoy[911]: [2026-09-20 13:33:20.444][911][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation4056lb # [ 16.639160] envoy[911]: [2026-09-20 13:33:20.444][911][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic4057lb # [ 16.641481] envoy[911]: [2026-09-20 13:33:20.444][911][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default4058lb # [ 16.643992] envoy[911]: [2026-09-20 13:33:20.444][911][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher4059lb # [ 16.646109] envoy[911]: [2026-09-20 13:33:20.444][911][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4060lb # [ 16.648775] envoy[911]: [2026-09-20 13:33:20.444][911][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls4061lb # [ 16.656102] envoy[911]: [2026-09-20 13:33:20.444][911][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v84062lb # [ 16.658132] envoy[911]: [2026-09-20 13:33:20.444][911][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value4063lb # [ 16.661345] envoy[911]: [2026-09-20 13:33:20.444][911][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router4064lb # [ 16.690151] envoy[911]: [2026-09-20 13:33:20.444][911][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory4065lb # [ 16.693216] envoy[911]: [2026-09-20 13:33:20.444][911][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy4066lb # [ 16.696138] envoy[911]: [2026-09-20 13:33:20.444][911][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy4067lb # [ 16.709193] envoy[911]: [2026-09-20 13:33:20.444][911][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re24068lb # [ 16.711217] envoy[911]: [2026-09-20 13:33:20.444][911][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default4069lb # [ 16.713308] envoy[911]: [2026-09-20 13:33:20.444][911][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle4070lb # [ 16.715463] envoy[911]: [2026-09-20 13:33:20.444][911][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name4071lb # [ 16.717930] envoy[911]: [2026-09-20 13:33:20.444][911][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed4072lb # [ 16.720340] envoy[911]: [2026-09-20 13:33:20.444][911][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http14073lb # [ 16.722407] envoy[911]: [2026-09-20 13:33:20.445][911][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind4074lb # [ 16.724125] envoy[911]: [2026-09-20 13:33:20.445][911][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action4075lb # [ 16.727064] envoy[911]: [2026-09-20 13:33:20.445][911][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override4076lb # [ 16.729414] envoy[911]: [2026-09-20 13:33:20.445][911][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift4077lb # [ 16.732354] envoy[911]: [2026-09-20 13:33:20.445][911][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd4078lb # [ 16.736702] envoy[911]: [2026-09-20 13:33:20.445][911][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth24079lb # [ 16.738847] envoy[911]: [2026-09-20 13:33:20.445][911][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name4080lb # [ 16.740943] envoy[911]: [2026-09-20 13:33:20.445][911][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter4081lb # [ 16.742826] envoy[911]: [2026-09-20 13:33:20.445][911][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities4082lb # [ 16.745070] envoy[911]: [2026-09-20 13:33:20.445][911][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff4083lb # [ 16.747264] envoy[911]: [2026-09-20 13:33:20.456][911][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override4084lb # [ 16.750423] envoy[911]: [2026-09-20 13:33:20.456][911][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap4085lb # [ 16.755413] envoy[911]: [2026-09-20 13:33:20.456][911][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router4086lb # [ 16.758322] envoy[911]: [2026-09-20 13:33:20.456][911][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector4087lb # [ 16.764276] envoy[911]: [2026-09-20 13:33:20.456][911][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router4088lb # [ 16.766170] envoy[911]: [2026-09-20 13:33:20.456][911][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case4089lb # [ 16.768965] envoy[911]: [2026-09-20 13:33:20.465][911][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo4090lb # [ 16.770598] envoy[911]: [2026-09-20 13:33:20.583][911][info][main] [source/server/server.cc:531] HTTP header map info:4091lb # [ 16.772085] envoy[911]: [2026-09-20 13:33:20.638][911][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id4092lb # [ 16.786171] envoy[911]: [2026-09-20 13:33:20.638][911][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:4093lb # [ 16.787670] envoy[911]: [2026-09-20 13:33:20.638][911][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id4094lb # [ 16.795574] envoy[911]: [2026-09-20 13:33:20.638][911][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status4095lb # [ 16.836687] envoy[911]: [2026-09-20 13:33:20.833][911][info][main] [source/server/server.cc:968] runtime: {}4096lb # [ 16.843141] envoy[911]: [2026-09-20 13:33:20.840][911][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.4097lb # [ 16.845438] envoy[911]: [2026-09-20 13:33:20.842][911][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99014098lb # [ 16.848092] envoy[911]: [2026-09-20 13:33:20.845][911][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99014099lb # [ 16.850797] envoy[911]: [2026-09-20 13:33:20.848][911][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration4100lb # [ 16.852499] envoy[911]: [2026-09-20 13:33:20.849][911][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)4101lb # [ 16.853972] envoy[911]: [2026-09-20 13:33:20.849][911][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)4102worker1 # [ 16.887884] systemd[1]: Starting Nix worker-protocol over gRPC...4103lb # [ 16.889360] envoy[911]: [2026-09-20 13:33:20.886][911][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)4104worker2 # [ 16.852367] systemd[1]: Reached target Network is Online.4105worker2 # [ 16.855737] systemd[1]: Starting scheduler-token.service...4106lb # [ 16.905974] envoy[911]: [2026-09-20 13:33:20.903][911][info][config] [source/server/configuration_impl.cc:164] loading stats configuration4107lb # [ 16.908143] envoy[911]: [2026-09-20 13:33:20.905][911][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.4108lb # [ 16.910988] envoy[911]: [2026-09-20 13:33:20.905][911][info][main] [source/server/server.cc:1077] starting main dispatch loop4109worker2 # [ 16.934722] systemd[1]: scheduler-token.service: Deactivated successfully.4110worker2 # [ 16.936979] systemd[1]: Finished scheduler-token.service.4111worker2 # [ 16.938241] systemd[1]: scheduler-token.service: Consumed 21ms CPU time over 80ms wall clock time, 3.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4112worker2 # [ 16.942643] systemd[1]: Starting Nix worker-protocol over gRPC...4113worker1 # [ 17.040068] nix-grpc-daemon[783]: ts=2026-09-20T13:33:20Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4114worker1 # [ 17.047433] nix-grpc-daemon[783]: ts=2026-09-20T13:33:20Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4115worker1 # [ 17.052875] nix-grpc-daemon[783]: ts=2026-09-20T13:33:20Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14116worker1 # [ 17.084962] nix-grpc-daemon[783]: ts=2026-09-20T13:33:20Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4117worker1 # [ 17.088683] systemd[1]: Started Nix worker-protocol over gRPC.4118worker1 # [ 17.116604] nix-grpc-daemon[783]: ts=2026-09-20T13:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4119worker2 # [ 17.094511] nix-grpc-daemon[786]: ts=2026-09-20T13:33:21Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4120lb # [ 17.135323] envoy[911]: [2026-09-20 13:33:21.132][911][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization4121lb # [ 17.137085] envoy[911]: [2026-09-20 13:33:21.134][911][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized4122lb # [ 17.138769] envoy[911]: [2026-09-20 13:33:21.134][911][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager4123lb # [ 17.140419] envoy[911]: [2026-09-20 13:33:21.136][911][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers4124worker2 # [ 17.103881] nix-grpc-daemon[786]: ts=2026-09-20T13:33:21Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4125worker2 # [ 17.107856] nix-grpc-daemon[786]: ts=2026-09-20T13:33:21Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14126worker2 # [ 17.143549] nix-grpc-daemon[786]: ts=2026-09-20T13:33:21Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4127worker2 # [ 17.149410] systemd[1]: Started Nix worker-protocol over gRPC.4128worker2 # [ 17.188108] nix-grpc-daemon[786]: ts=2026-09-20T13:33:21Z level=info event=scheduler_connected addr=lb:500514129worker2 # [ 17.197909] nix-grpc-daemon[786]: ts=2026-09-20T13:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4130worker1 # [ 17.318102] nix-grpc-daemon[783]: ts=2026-09-20T13:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4131worker2 # [ 17.399662] nix-grpc-daemon[786]: ts=2026-09-20T13:33:21Z level=info event=scheduler_connected addr=lb:500514132worker2 # [ 17.401363] nix-grpc-daemon[786]: ts=2026-09-20T13:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4133worker2 # [ 17.602613] nix-grpc-daemon[786]: ts=2026-09-20T13:33:21Z level=info event=scheduler_connected addr=lb:500514134worker2 # [ 17.606405] nix-grpc-daemon[786]: ts=2026-09-20T13:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4135worker1 # [ 17.718644] nix-grpc-daemon[783]: ts=2026-09-20T13:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4136worker2 # [ 17.807065] nix-grpc-daemon[786]: ts=2026-09-20T13:33:21Z level=info event=scheduler_connected addr=lb:500514137worker2 # [ 17.808805] nix-grpc-daemon[786]: ts=2026-09-20T13:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4138worker2 # [ 18.009975] nix-grpc-daemon[786]: ts=2026-09-20T13:33:22Z level=info event=scheduler_connected addr=lb:500514139worker2 # [ 18.013756] nix-grpc-daemon[786]: ts=2026-09-20T13:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4140worker2 # [ 18.215548] nix-grpc-daemon[786]: ts=2026-09-20T13:33:22Z level=info event=scheduler_connected addr=lb:500514141worker2 # [ 18.219400] nix-grpc-daemon[786]: ts=2026-09-20T13:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4142worker2 # [ 18.416934] nix-grpc-daemon[786]: ts=2026-09-20T13:33:22Z level=info event=scheduler_connected addr=lb:500514143worker2 # [ 18.418869] nix-grpc-daemon[786]: ts=2026-09-20T13:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4144worker1 # [ 18.522851] nix-grpc-daemon[783]: ts=2026-09-20T13:33:21Z level=info event=scheduler_connected addr=lb:500514145worker1 # [ 18.527273] nix-grpc-daemon[783]: ts=2026-09-20T13:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4146worker2 # [ 18.620267] nix-grpc-daemon[786]: ts=2026-09-20T13:33:22Z level=info event=scheduler_connected addr=lb:500514147worker2 # [ 18.624142] nix-grpc-daemon[786]: ts=2026-09-20T13:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4148worker1 # [ 18.728481] nix-grpc-daemon[783]: ts=2026-09-20T13:33:21Z level=info event=scheduler_connected addr=lb:500514149worker1 # [ 18.732324] nix-grpc-daemon[783]: ts=2026-09-20T13:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4150worker2 # [ 18.825198] nix-grpc-daemon[786]: ts=2026-09-20T13:33:22Z level=info event=scheduler_connected addr=lb:500514151worker2 # [ 18.829188] nix-grpc-daemon[786]: ts=2026-09-20T13:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4152worker1 # [ 18.933203] nix-grpc-daemon[783]: ts=2026-09-20T13:33:22Z level=info event=scheduler_connected addr=lb:500514153worker1 # [ 18.937155] nix-grpc-daemon[783]: ts=2026-09-20T13:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4154worker2 # [ 19.030650] nix-grpc-daemon[786]: ts=2026-09-20T13:33:23Z level=info event=scheduler_connected addr=lb:500514155worker2 # [ 19.034395] nix-grpc-daemon[786]: ts=2026-09-20T13:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4156worker1 # [ 19.137458] nix-grpc-daemon[783]: ts=2026-09-20T13:33:22Z level=info event=scheduler_connected addr=lb:500514157worker1 # [ 19.139769] nix-grpc-daemon[783]: ts=2026-09-20T13:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4158worker2 # [ 19.236125] nix-grpc-daemon[786]: ts=2026-09-20T13:33:23Z level=info event=scheduler_connected addr=lb:500514159worker2 # [ 19.239865] nix-grpc-daemon[786]: ts=2026-09-20T13:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4160worker1 # [ 19.340552] nix-grpc-daemon[783]: ts=2026-09-20T13:33:22Z level=info event=scheduler_connected addr=lb:500514161worker1 # [ 19.342292] nix-grpc-daemon[783]: ts=2026-09-20T13:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4162worker2 # [ 19.441333] nix-grpc-daemon[786]: ts=2026-09-20T13:33:23Z level=info event=scheduler_connected addr=lb:500514163worker2 # [ 19.445197] nix-grpc-daemon[786]: ts=2026-09-20T13:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4164worker1 # [ 19.543529] nix-grpc-daemon[783]: ts=2026-09-20T13:33:22Z level=info event=scheduler_connected addr=lb:500514165worker1 # [ 19.546735] nix-grpc-daemon[783]: ts=2026-09-20T13:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4166worker2 # [ 19.645912] nix-grpc-daemon[786]: ts=2026-09-20T13:33:23Z level=info event=scheduler_connected addr=lb:500514167worker2 # [ 19.647862] nix-grpc-daemon[786]: ts=2026-09-20T13:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4168worker1 # [ 19.747827] nix-grpc-daemon[783]: ts=2026-09-20T13:33:22Z level=info event=scheduler_connected addr=lb:500514169worker1 # [ 19.751515] nix-grpc-daemon[783]: ts=2026-09-20T13:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4170worker2 # [ 19.849194] nix-grpc-daemon[786]: ts=2026-09-20T13:33:23Z level=info event=scheduler_connected addr=lb:500514171worker2 # [ 19.850997] nix-grpc-daemon[786]: ts=2026-09-20T13:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4172worker1 # [ 19.952111] nix-grpc-daemon[783]: ts=2026-09-20T13:33:23Z level=info event=scheduler_connected addr=lb:500514173worker1 # [ 19.953825] nix-grpc-daemon[783]: ts=2026-09-20T13:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4174worker2 # [ 20.052253] nix-grpc-daemon[786]: ts=2026-09-20T13:33:24Z level=info event=scheduler_connected addr=lb:500514175worker2 # [ 20.055458] nix-grpc-daemon[786]: ts=2026-09-20T13:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4176worker1 # [ 20.155204] nix-grpc-daemon[783]: ts=2026-09-20T13:33:23Z level=info event=scheduler_connected addr=lb:500514177worker1 # [ 20.159143] nix-grpc-daemon[783]: ts=2026-09-20T13:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4178worker2 # [ 20.258803] nix-grpc-daemon[786]: ts=2026-09-20T13:33:24Z level=info event=scheduler_connected addr=lb:500514179worker2 # [ 20.262361] nix-grpc-daemon[786]: ts=2026-09-20T13:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4180worker1 # [ 20.359352] nix-grpc-daemon[783]: ts=2026-09-20T13:33:23Z level=info event=scheduler_connected addr=lb:500514181worker1 # [ 20.361038] nix-grpc-daemon[783]: ts=2026-09-20T13:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4182worker2 # [ 20.460244] nix-grpc-daemon[786]: ts=2026-09-20T13:33:24Z level=info event=scheduler_connected addr=lb:500514183worker2 # [ 20.464110] nix-grpc-daemon[786]: ts=2026-09-20T13:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4184worker1 # [ 20.561966] nix-grpc-daemon[783]: ts=2026-09-20T13:33:23Z level=info event=scheduler_connected addr=lb:500514185worker1 # [ 20.563721] nix-grpc-daemon[783]: ts=2026-09-20T13:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4186worker2 # [ 20.665737] nix-grpc-daemon[786]: ts=2026-09-20T13:33:24Z level=info event=scheduler_connected addr=lb:500514187worker2 # [ 20.669541] nix-grpc-daemon[786]: ts=2026-09-20T13:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4188worker1 # [ 20.764262] nix-grpc-daemon[783]: ts=2026-09-20T13:33:24Z level=info event=scheduler_connected addr=lb:500514189worker1 # [ 20.765886] nix-grpc-daemon[783]: ts=2026-09-20T13:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4190worker2 # [ 20.870940] nix-grpc-daemon[786]: ts=2026-09-20T13:33:25Z level=info event=scheduler_connected addr=lb:500514191worker2 # [ 20.874872] nix-grpc-daemon[786]: ts=2026-09-20T13:33:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4192worker1 # [ 20.966409] nix-grpc-daemon[783]: ts=2026-09-20T13:33:24Z level=info event=scheduler_connected addr=lb:500514193worker1 # [ 20.968239] nix-grpc-daemon[783]: ts=2026-09-20T13:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4194worker2 # [ 21.076473] nix-grpc-daemon[786]: ts=2026-09-20T13:33:25Z level=info event=scheduler_connected addr=lb:500514195worker2 # [ 21.080424] nix-grpc-daemon[786]: ts=2026-09-20T13:33:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4196worker1 # [ 21.168873] nix-grpc-daemon[783]: ts=2026-09-20T13:33:24Z level=info event=scheduler_connected addr=lb:500514197worker1 # [ 21.172709] nix-grpc-daemon[783]: ts=2026-09-20T13:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4198worker2 # [ 21.282175] nix-grpc-daemon[786]: ts=2026-09-20T13:33:25Z level=info event=scheduler_connected addr=lb:500514199worker2 # [ 21.285200] nix-grpc-daemon[786]: ts=2026-09-20T13:33:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4200worker1 # [ 21.373712] nix-grpc-daemon[783]: ts=2026-09-20T13:33:24Z level=info event=scheduler_connected addr=lb:500514201worker1 # [ 21.377663] nix-grpc-daemon[783]: ts=2026-09-20T13:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4202worker2 # [ 21.486754] nix-grpc-daemon[786]: ts=2026-09-20T13:33:25Z level=info event=scheduler_connected addr=lb:500514203worker2 # [ 21.490784] nix-grpc-daemon[786]: ts=2026-09-20T13:33:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4204worker1 # [ 21.579706] nix-grpc-daemon[783]: ts=2026-09-20T13:33:24Z level=info event=scheduler_connected addr=lb:500514205worker1 # [ 21.582749] nix-grpc-daemon[783]: ts=2026-09-20T13:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4206worker2 # [ 21.691978] nix-grpc-daemon[786]: ts=2026-09-20T13:33:25Z level=info event=scheduler_connected addr=lb:500514207worker2 # [ 21.696206] nix-grpc-daemon[786]: ts=2026-09-20T13:33:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4208worker1 # [ 21.780841] nix-grpc-daemon[783]: ts=2026-09-20T13:33:25Z level=info event=scheduler_connected addr=lb:500514209worker1 # [ 21.784975] nix-grpc-daemon[783]: ts=2026-09-20T13:33:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4210worker2 # [ 21.897097] nix-grpc-daemon[786]: ts=2026-09-20T13:33:26Z level=info event=scheduler_connected addr=lb:500514211worker2 # [ 21.898939] nix-grpc-daemon[786]: ts=2026-09-20T13:33:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4212worker1 # [ 21.985624] nix-grpc-daemon[783]: ts=2026-09-20T13:33:25Z level=info event=scheduler_connected addr=lb:500514213worker1 # [ 21.987415] nix-grpc-daemon[783]: ts=2026-09-20T13:33:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4214worker2 # [ 22.100774] nix-grpc-daemon[786]: ts=2026-09-20T13:33:26Z level=info event=scheduler_connected addr=lb:500514215worker2 # [ 22.105194] nix-grpc-daemon[786]: ts=2026-09-20T13:33:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4216worker1 # [ 22.187890] nix-grpc-daemon[783]: ts=2026-09-20T13:33:25Z level=info event=scheduler_connected addr=lb:500514217worker1 # [ 22.189701] nix-grpc-daemon[783]: ts=2026-09-20T13:33:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4218worker2 # [ 22.306525] nix-grpc-daemon[786]: ts=2026-09-20T13:33:26Z level=info event=scheduler_connected addr=lb:500514219worker2 # [ 22.308453] nix-grpc-daemon[786]: ts=2026-09-20T13:33:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4220worker1 # [ 22.390660] nix-grpc-daemon[783]: ts=2026-09-20T13:33:25Z level=info event=scheduler_connected addr=lb:500514221worker1 # [ 22.394403] nix-grpc-daemon[783]: ts=2026-09-20T13:33:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4222worker2 # [ 22.509615] nix-grpc-daemon[786]: ts=2026-09-20T13:33:26Z level=info event=scheduler_connected addr=lb:500514223worker2 # [ 22.511610] nix-grpc-daemon[786]: ts=2026-09-20T13:33:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4224worker1 # [ 22.596271] nix-grpc-daemon[783]: ts=2026-09-20T13:33:25Z level=info event=scheduler_connected addr=lb:500514225worker1 # [ 22.599857] nix-grpc-daemon[783]: ts=2026-09-20T13:33:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4226worker2 # [ 22.712755] nix-grpc-daemon[786]: ts=2026-09-20T13:33:26Z level=info event=scheduler_connected addr=lb:500514227worker2 # [ 22.714643] nix-grpc-daemon[786]: ts=2026-09-20T13:33:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4228worker1 # [ 22.800234] nix-grpc-daemon[783]: ts=2026-09-20T13:33:26Z level=info event=scheduler_connected addr=lb:500514229worker1 # [ 22.801866] nix-grpc-daemon[783]: ts=2026-09-20T13:33:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4230worker2 # [ 22.915395] nix-grpc-daemon[786]: ts=2026-09-20T13:33:27Z level=info event=scheduler_connected addr=lb:500514231worker2 # [ 22.917304] nix-grpc-daemon[786]: ts=2026-09-20T13:33:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4232worker1 # [ 23.002882] nix-grpc-daemon[783]: ts=2026-09-20T13:33:26Z level=info event=scheduler_connected addr=lb:500514233worker1 # [ 23.006756] nix-grpc-daemon[783]: ts=2026-09-20T13:33:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4234worker2 # [ 23.118805] nix-grpc-daemon[786]: ts=2026-09-20T13:33:27Z level=info event=scheduler_connected addr=lb:500514235worker2 # [ 23.122910] nix-grpc-daemon[786]: ts=2026-09-20T13:33:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4236worker1 # [ 23.207068] nix-grpc-daemon[783]: ts=2026-09-20T13:33:26Z level=info event=scheduler_connected addr=lb:500514237worker1 # [ 23.208802] nix-grpc-daemon[783]: ts=2026-09-20T13:33:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4238worker2 # [ 23.325258] nix-grpc-daemon[786]: ts=2026-09-20T13:33:27Z level=info event=scheduler_connected addr=lb:500514239worker2 # [ 23.328791] nix-grpc-daemon[786]: ts=2026-09-20T13:33:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4240worker1 # [ 23.409433] nix-grpc-daemon[783]: ts=2026-09-20T13:33:26Z level=info event=scheduler_connected addr=lb:500514241worker1 # [ 23.411130] nix-grpc-daemon[783]: ts=2026-09-20T13:33:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4242worker2 # [ 23.530685] nix-grpc-daemon[786]: ts=2026-09-20T13:33:27Z level=info event=scheduler_connected addr=lb:500514243worker2 # [ 23.534556] nix-grpc-daemon[786]: ts=2026-09-20T13:33:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4244worker1 # [ 23.612094] nix-grpc-daemon[783]: ts=2026-09-20T13:33:26Z level=info event=scheduler_connected addr=lb:500514245worker1 # [ 23.613818] nix-grpc-daemon[783]: ts=2026-09-20T13:33:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4246worker2 # [ 23.735471] nix-grpc-daemon[786]: ts=2026-09-20T13:33:27Z level=info event=scheduler_connected addr=lb:500514247worker2 # [ 23.737359] nix-grpc-daemon[786]: ts=2026-09-20T13:33:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4248worker1 # [ 23.814933] nix-grpc-daemon[783]: ts=2026-09-20T13:33:27Z level=info event=scheduler_connected addr=lb:500514249worker1 # [ 23.819130] nix-grpc-daemon[783]: ts=2026-09-20T13:33:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4250worker2 # [ 23.938781] nix-grpc-daemon[786]: ts=2026-09-20T13:33:28Z level=info event=scheduler_connected addr=lb:500514251worker2 # [ 23.942572] nix-grpc-daemon[786]: ts=2026-09-20T13:33:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4252worker1 # [ 24.019622] nix-grpc-daemon[783]: ts=2026-09-20T13:33:27Z level=info event=scheduler_connected addr=lb:500514253worker1 # [ 24.021374] nix-grpc-daemon[783]: ts=2026-09-20T13:33:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4254worker2 # [ 24.143134] nix-grpc-daemon[786]: ts=2026-09-20T13:33:28Z level=info event=scheduler_connected addr=lb:500514255worker2 # [ 24.144977] nix-grpc-daemon[786]: ts=2026-09-20T13:33:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4256worker1 # [ 24.222385] nix-grpc-daemon[783]: ts=2026-09-20T13:33:27Z level=info event=scheduler_connected addr=lb:500514257worker1 # [ 24.226180] nix-grpc-daemon[783]: ts=2026-09-20T13:33:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4258worker2 # [ 24.346170] nix-grpc-daemon[786]: ts=2026-09-20T13:33:28Z level=info event=scheduler_connected addr=lb:500514259worker2 # [ 24.347861] nix-grpc-daemon[786]: ts=2026-09-20T13:33:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4260worker1 # [ 24.426466] nix-grpc-daemon[783]: ts=2026-09-20T13:33:27Z level=info event=scheduler_connected addr=lb:500514261worker1 # [ 24.428119] nix-grpc-daemon[783]: ts=2026-09-20T13:33:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4262worker2 # [ 24.548596] nix-grpc-daemon[786]: ts=2026-09-20T13:33:28Z level=info event=scheduler_connected addr=lb:500514263worker2 # [ 24.550309] nix-grpc-daemon[786]: ts=2026-09-20T13:33:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4264worker1 # [ 24.629230] nix-grpc-daemon[783]: ts=2026-09-20T13:33:27Z level=info event=scheduler_connected addr=lb:500514265worker1 # [ 24.630975] nix-grpc-daemon[783]: ts=2026-09-20T13:33:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4266worker2 # [ 24.751789] nix-grpc-daemon[786]: ts=2026-09-20T13:33:28Z level=info event=scheduler_connected addr=lb:500514267worker2 # [ 24.755606] nix-grpc-daemon[786]: ts=2026-09-20T13:33:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4268worker1 # [ 24.831519] nix-grpc-daemon[783]: ts=2026-09-20T13:33:28Z level=info event=scheduler_connected addr=lb:500514269worker1 # [ 24.833265] nix-grpc-daemon[783]: ts=2026-09-20T13:33:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4270worker2 # [ 24.956812] nix-grpc-daemon[786]: ts=2026-09-20T13:33:29Z level=info event=scheduler_connected addr=lb:500514271worker2 # [ 24.960653] nix-grpc-daemon[786]: ts=2026-09-20T13:33:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4272worker1 # [ 25.034384] nix-grpc-daemon[783]: ts=2026-09-20T13:33:28Z level=info event=scheduler_connected addr=lb:500514273worker1 # [ 25.038258] nix-grpc-daemon[783]: ts=2026-09-20T13:33:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4274worker2 # [ 25.161247] nix-grpc-daemon[786]: ts=2026-09-20T13:33:29Z level=info event=scheduler_connected addr=lb:500514275worker2 # [ 25.163217] nix-grpc-daemon[786]: ts=2026-09-20T13:33:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4276worker1 # [ 25.238578] nix-grpc-daemon[783]: ts=2026-09-20T13:33:28Z level=info event=scheduler_connected addr=lb:500514277worker1 # [ 25.240258] nix-grpc-daemon[783]: ts=2026-09-20T13:33:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4278worker2 # [ 25.364355] nix-grpc-daemon[786]: ts=2026-09-20T13:33:29Z level=info event=scheduler_connected addr=lb:500514279worker2 # [ 25.366679] nix-grpc-daemon[786]: ts=2026-09-20T13:33:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4280worker1 # [ 25.441326] nix-grpc-daemon[783]: ts=2026-09-20T13:33:28Z level=info event=scheduler_connected addr=lb:500514281worker1 # [ 25.445200] nix-grpc-daemon[783]: ts=2026-09-20T13:33:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4282worker2 # [ 25.568118] nix-grpc-daemon[786]: ts=2026-09-20T13:33:29Z level=info event=scheduler_connected addr=lb:500514283worker2 # [ 25.571870] nix-grpc-daemon[786]: ts=2026-09-20T13:33:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4284worker1 # [ 25.647389] nix-grpc-daemon[783]: ts=2026-09-20T13:33:28Z level=info event=scheduler_connected addr=lb:500514285worker1 # [ 25.650468] nix-grpc-daemon[783]: ts=2026-09-20T13:33:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4286worker2 # [ 25.772935] nix-grpc-daemon[786]: ts=2026-09-20T13:33:29Z level=info event=scheduler_connected addr=lb:500514287worker2 # [ 25.776892] nix-grpc-daemon[786]: ts=2026-09-20T13:33:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4288worker1 # [ 25.848568] nix-grpc-daemon[783]: ts=2026-09-20T13:33:29Z level=info event=scheduler_connected addr=lb:500514289worker1 # [ 25.852754] nix-grpc-daemon[783]: ts=2026-09-20T13:33:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4290worker2 # [ 25.978100] nix-grpc-daemon[786]: ts=2026-09-20T13:33:30Z level=info event=scheduler_connected addr=lb:500514291worker2 # [ 25.982324] nix-grpc-daemon[786]: ts=2026-09-20T13:33:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4292worker1 # [ 26.053510] nix-grpc-daemon[783]: ts=2026-09-20T13:33:29Z level=info event=scheduler_connected addr=lb:500514293worker1 # [ 26.057337] nix-grpc-daemon[783]: ts=2026-09-20T13:33:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4294worker2 # [ 26.183421] nix-grpc-daemon[786]: ts=2026-09-20T13:33:30Z level=info event=scheduler_connected addr=lb:500514295worker2 # [ 26.187218] nix-grpc-daemon[786]: ts=2026-09-20T13:33:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4296worker1 # [ 26.258296] nix-grpc-daemon[783]: ts=2026-09-20T13:33:29Z level=info event=scheduler_connected addr=lb:500514297worker1 # [ 26.262219] nix-grpc-daemon[783]: ts=2026-09-20T13:33:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4298worker2 # [ 26.389955] nix-grpc-daemon[786]: ts=2026-09-20T13:33:30Z level=info event=scheduler_connected addr=lb:500514299worker2 # [ 26.393643] nix-grpc-daemon[786]: ts=2026-09-20T13:33:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4300worker1 # [ 26.462454] nix-grpc-daemon[783]: ts=2026-09-20T13:33:29Z level=info event=scheduler_connected addr=lb:500514301worker1 # [ 26.464144] nix-grpc-daemon[783]: ts=2026-09-20T13:33:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4302worker2 # [ 26.590869] nix-grpc-daemon[786]: ts=2026-09-20T13:33:30Z level=info event=scheduler_connected addr=lb:500514303worker2 # [ 26.594643] nix-grpc-daemon[786]: ts=2026-09-20T13:33:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4304worker1 # [ 26.666488] nix-grpc-daemon[783]: ts=2026-09-20T13:33:29Z level=info event=scheduler_connected addr=lb:500514305worker1 # [ 26.669750] nix-grpc-daemon[783]: ts=2026-09-20T13:33:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4306worker2 # [ 26.795313] nix-grpc-daemon[786]: ts=2026-09-20T13:33:30Z level=info event=scheduler_connected addr=lb:500514307worker2 # [ 26.797051] nix-grpc-daemon[786]: ts=2026-09-20T13:33:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4308worker1 # [ 26.866607] nix-grpc-daemon[783]: ts=2026-09-20T13:33:30Z level=info event=scheduler_connected addr=lb:500514309worker1 # [ 26.868629] nix-grpc-daemon[783]: ts=2026-09-20T13:33:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4310worker2 # [ 26.997871] nix-grpc-daemon[786]: ts=2026-09-20T13:33:31Z level=info event=scheduler_connected addr=lb:500514311worker2 # [ 26.999573] nix-grpc-daemon[786]: ts=2026-09-20T13:33:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4312worker1 # [ 27.069592] nix-grpc-daemon[783]: ts=2026-09-20T13:33:30Z level=info event=scheduler_connected addr=lb:500514313worker1 # [ 27.073335] nix-grpc-daemon[783]: ts=2026-09-20T13:33:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4314worker2 # [ 27.200252] nix-grpc-daemon[786]: ts=2026-09-20T13:33:31Z level=info event=scheduler_connected addr=lb:500514315worker2 # [ 27.202770] nix-grpc-daemon[786]: ts=2026-09-20T13:33:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4316worker1 # [ 27.274100] nix-grpc-daemon[783]: ts=2026-09-20T13:33:30Z level=info event=scheduler_connected addr=lb:500514317worker1 # [ 27.277919] nix-grpc-daemon[783]: ts=2026-09-20T13:33:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4318worker2 # [ 27.404162] nix-grpc-daemon[786]: ts=2026-09-20T13:33:31Z level=info event=scheduler_connected addr=lb:500514319worker2 # [ 27.407855] nix-grpc-daemon[786]: ts=2026-09-20T13:33:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4320worker1 # [ 27.479094] nix-grpc-daemon[783]: ts=2026-09-20T13:33:30Z level=info event=scheduler_connected addr=lb:500514321worker1 # [ 27.482846] nix-grpc-daemon[783]: ts=2026-09-20T13:33:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4322worker2 # [ 27.609123] nix-grpc-daemon[786]: ts=2026-09-20T13:33:31Z level=info event=scheduler_connected addr=lb:500514323worker2 # [ 27.613286] nix-grpc-daemon[786]: ts=2026-09-20T13:33:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4324worker1 # [ 27.684449] nix-grpc-daemon[783]: ts=2026-09-20T13:33:30Z level=info event=scheduler_connected addr=lb:500514325worker1 # [ 27.688297] nix-grpc-daemon[783]: ts=2026-09-20T13:33:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4326worker2 # [ 27.814232] nix-grpc-daemon[786]: ts=2026-09-20T13:33:31Z level=info event=scheduler_connected addr=lb:500514327worker2 # [ 27.818116] nix-grpc-daemon[786]: ts=2026-09-20T13:33:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4328lb # [ 27.882783] rustfs-bucket-start[1031]: mb s3://farm4329lb # [ 27.888937] systemd[1]: Finished rustfs-bucket.service.4330worker1 # [ 27.888676] nix-grpc-daemon[783]: ts=2026-09-20T13:33:31Z level=info event=scheduler_connected addr=lb:500514331worker1 # [ 27.890404] nix-grpc-daemon[783]: ts=2026-09-20T13:33:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4332lb # [ 27.895667] systemd[1]: Starting niks3 server...4333worker2 # [ 28.018854] nix-grpc-daemon[786]: ts=2026-09-20T13:33:32Z level=info event=scheduler_connected addr=lb:500514334worker2 # [ 28.020749] nix-grpc-daemon[786]: ts=2026-09-20T13:33:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4335worker1 # [ 28.090840] nix-grpc-daemon[783]: ts=2026-09-20T13:33:31Z level=info event=scheduler_connected addr=lb:500514336worker1 # [ 28.092772] nix-grpc-daemon[783]: ts=2026-09-20T13:33:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4337lb # [ 28.102305] postgres[1050]: [1050] ERROR: relation "goose_db_version" does not exist at character 364338lb # [ 28.103559] postgres[1050]: [1050] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4339lb # [ 28.141573] niks3-server[1040]: 2026/09/20 13:33:32 OK 20241026095416_initial_model.sql (19.05ms)4340lb # [ 28.145565] niks3-server[1040]: 2026/09/20 13:33:32 OK 20251210153512_drop_unused_gin_index.sql (2.12ms)4341lb # [ 28.151302] niks3-server[1040]: 2026/09/20 13:33:32 OK 20251218171726_add_pins.sql (4.07ms)4342lb # [ 28.156910] niks3-server[1040]: 2026/09/20 13:33:32 OK 20260628120000_add_object_size_and_stats.sql (5.77ms)4343lb # [ 28.161369] niks3-server[1040]: 2026/09/20 13:33:32 OK 20260905000000_add_claims.sql (3.08ms)4344lb # [ 28.165316] niks3-server[1040]: 2026/09/20 13:33:32 OK 20260920000000_drop_claims.sql (3.98ms)4345lb # [ 28.166735] niks3-server[1040]: 2026/09/20 13:33:32 goose: successfully migrated database to version: 202609200000004346lb # [ 28.171832] niks3-server[1040]: 2026/09/20 13:33:32 OK 1_commit_pending_closure.sql (5.12ms)4347lb # [ 28.174294] niks3-server[1040]: 2026/09/20 13:33:32 OK 2_object_stats_trigger.sql (1.3ms)4348lb # [ 28.175438] niks3-server[1040]: 2026/09/20 13:33:32 goose: up to current file version: 24349lb # [ 28.181029] niks3-server[1040]: 2026/09/20 13:33:32 INFO Loaded signing key name=farm-test-1 path=/nix/store/c745bsy529madd2s6hz4z5is2436ni04-key4350lb # [ 28.258582] niks3-server[1040]: 2026/09/20 13:33:32 INFO Read proxy enabled — serving cache objects from S34351lb # [ 28.261051] niks3-server[1040]: 2026/09/20 13:33:32 INFO Using socket-activated listener address=0.0.0.0:57514352worker2 # [ 28.221736] nix-grpc-daemon[786]: ts=2026-09-20T13:33:32Z level=info event=scheduler_connected addr=lb:500514353worker2 # [ 28.226438] nix-grpc-daemon[786]: ts=2026-09-20T13:33:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4354lb # [ 28.267789] niks3-server[1040]: 2026/09/20 13:33:32 INFO systemd watchdog enabled interval=15s4355lb # [ 28.268999] systemd[1]: Started niks3 server.4356lb # [ 28.269651] systemd[1]: Reached target Multi-User System.4357lb # [ 28.271905] systemd[1]: Startup finished in 859ms (kernel) + 4.125s (initrd) + 23.286s (userspace) = 28.271s.4358lb # [ 28.273813] niks3-server[1040]: 2026/09/20 13:33:32 INFO Starting HTTP server address=0.0.0.0:57514359lb # [ 28.279209] niks3-server[1040]: 2026/09/20 13:33:32 INFO lead: acquired remote=192.168.1.4:496544360worker2 # [ 28.246623] nix-grpc-daemon[786]: ts=2026-09-20T13:33:32Z level=info event=scheduler_take_over4361worker1 # [ 28.293525] nix-grpc-daemon[783]: ts=2026-09-20T13:33:31Z level=info event=scheduler_connected addr=lb:500514362worker1 # [ 28.297525] nix-grpc-daemon[783]: ts=2026-09-20T13:33:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4363worker2 # [ 28.428364] nix-grpc-daemon[786]: ts=2026-09-20T13:33:32Z level=info event=scheduler_connected addr=lb:500514364worker2 # [ 28.432077] nix-grpc-daemon[786]: ts=2026-09-20T13:33:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4365worker1 # [ 28.498308] nix-grpc-daemon[783]: ts=2026-09-20T13:33:31Z level=info event=scheduler_connected addr=lb:500514366worker1 # [ 28.502208] nix-grpc-daemon[783]: ts=2026-09-20T13:33:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4367worker2 # [ 28.633345] nix-grpc-daemon[786]: ts=2026-09-20T13:33:32Z level=info event=scheduler_connected addr=lb:500514368worker2 # [ 28.637411] nix-grpc-daemon[786]: ts=2026-09-20T13:33:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4369worker1 # [ 28.704522] nix-grpc-daemon[783]: ts=2026-09-20T13:33:31Z level=info event=scheduler_connected addr=lb:500514370worker1 # [ 28.707698] nix-grpc-daemon[783]: ts=2026-09-20T13:33:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4371worker2 # [ 28.838691] nix-grpc-daemon[786]: ts=2026-09-20T13:33:32Z level=info event=scheduler_connected addr=lb:500514372worker2 # [ 28.842699] nix-grpc-daemon[786]: ts=2026-09-20T13:33:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4373worker1 # [ 28.905220] nix-grpc-daemon[783]: ts=2026-09-20T13:33:32Z level=info event=scheduler_connected addr=lb:500514374worker1 # [ 28.909131] nix-grpc-daemon[783]: ts=2026-09-20T13:33:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4375worker2 # [ 29.044076] nix-grpc-daemon[786]: ts=2026-09-20T13:33:33Z level=info event=scheduler_connected addr=lb:500514376worker2 # [ 29.047968] nix-grpc-daemon[786]: ts=2026-09-20T13:33:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4377lb: (finished: waiting for unit niks3.service, in 18.40 seconds)4378lb: waiting for TCP port 5751 on localhost4379worker1 # [ 29.109827] nix-grpc-daemon[783]: ts=2026-09-20T13:33:32Z level=info event=scheduler_connected addr=lb:500514380worker1 # [ 29.114334] nix-grpc-daemon[783]: ts=2026-09-20T13:33:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4381lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!4382lb: (finished: waiting for TCP port 5751 on localhost, in 0.06 seconds)4383lb: waiting for unit envoy.service4384lb: (finished: waiting for unit envoy.service, in 0.05 seconds)4385lb: waiting for TCP port 50051 on localhost4386lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!4387lb # [ 29.253347] envoy[911]: conn peer=[::1]:47728 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=34388lb: (finished: waiting for TCP port 50051 on localhost, in 0.04 seconds)4389worker2 # [ 29.249813] nix-grpc-daemon[786]: ts=2026-09-20T13:33:33Z level=info event=scheduler_connected addr=lb:500514390worker2 # [ 29.274486] nix-grpc-daemon[786]: ts=2026-09-20T13:33:33Z level=info event=oidc_jwks provider=mock keys=14391worker2 # [ 29.276711] nix-grpc-daemon[786]: ts=2026-09-20T13:33:33Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:341084392worker1 # [ 29.315178] nix-grpc-daemon[783]: ts=2026-09-20T13:33:32Z level=info event=scheduler_connected addr=lb:500514393worker2 # [ 29.278965] nix-grpc-daemon[786]: ts=2026-09-20T13:33:33Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:341084394worker2 # [ 29.280964] nix-grpc-daemon[786]: ts=2026-09-20T13:33:33Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04395worker2 # [ 29.283474] nix-grpc-daemon[786]: ts=2026-09-20T13:33:33Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04396worker2 # [ 29.329850] systemd[1]: Starting scheduler-token.service...4397worker2 # [ 29.395843] systemd[1]: scheduler-token.service: Deactivated successfully.4398worker2 # [ 29.397635] systemd[1]: Finished scheduler-token.service.4399worker2 # [ 29.399067] systemd[1]: scheduler-token.service: Consumed 21ms CPU time over 68ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4400lb: waiting until exactly no endpoints of x86_64-linux are unhealthy4401lb: must succeed: curl -sf localhost:9901/clusters4402lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4403lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.08 seconds)4404subtest: one node schedules and both builders hold a WorkerSession on it4405lb: must succeed: curl -sf localhost:9901/clusters4406lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4407worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4408worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4409lb: must succeed: curl -sf localhost:9901/clusters4410lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4411lb: must succeed: curl -sf localhost:9901/clusters4412lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4413worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'4414worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.05 seconds)4415(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.24 seconds)4416subtest: balancer auth: client cert, bearer token, nothing, unknown CN4417worker1 # [ 30.070352] nix-grpc-daemon[783]: ts=2026-09-20T13:33:33Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:338964418worker1 # [ 30.076248] systemd[1]: Started Nix Daemon.4419worker1 # [ 30.257480] nix-daemon[841]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4420worker1 # [ 30.303180] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 0 ms (attempt 1/5)4421worker1 # [ 30.305497] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 161 ms (attempt 2/5)4422worker1 # [ 30.464601] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 395 ms (attempt 3/5)4423worker1 # [ 30.865959] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 169 ms (attempt 4/5)4424worker1 # [ 31.045867] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4425worker1 # [ 31.071686] nix-grpc-daemon[783]: ts=2026-09-20T13:33:34Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:338964426worker1 # [ 31.085223] nix-grpc-daemon[783]: ts=2026-09-20T13:33:34Z level=debug event=path_info_miss path=jwg6hsvasqr1bfb4p1nn6ayia3202nfr-nixos-system-client-test error="error: path '/nix/store/jwg6hsvasqr1bfb4p1nn6ayia3202nfr-nixos-system-client-test' is required, but there is no substituter that can build it"4427worker1 # [ 31.089047] nix-grpc-daemon[783]: ts=2026-09-20T13:33:34Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:338964428worker1 # [ 31.093764] nix-grpc-daemon[783]: ts=2026-09-20T13:33:34Z level=debug event=path_info_miss path=jwg6hsvasqr1bfb4p1nn6ayia3202nfr-nixos-system-client-test error="error: path '/nix/store/jwg6hsvasqr1bfb4p1nn6ayia3202nfr-nixos-system-client-test' is required, but there is no substituter that can build it"4429client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4430client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.07 seconds)4431client: must succeed: install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4432client: (finished: must succeed: install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.03 seconds)4433worker2 # [ 31.367190] nix-grpc-daemon[786]: ts=2026-09-20T13:33:35Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:341144434worker2 # [ 31.379887] systemd[1]: Started Nix Daemon.4435worker2 # [ 31.595967] nix-daemon[855]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4436worker2 # [ 31.624775] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 42 ms (attempt 1/5)4437worker2 # [ 31.669300] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 153 ms (attempt 2/5)4438worker2 # [ 31.825284] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 68 ms (attempt 3/5)4439worker2 # [ 31.893582] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 303 ms (attempt 4/5)4440worker2 # [ 32.201103] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4441worker2 # [ 32.216581] nix-grpc-daemon[786]: ts=2026-09-20T13:33:36Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:341144442worker2 # [ 32.227424] nix-grpc-daemon[786]: ts=2026-09-20T13:33:36Z level=debug event=path_info_miss path=jwg6hsvasqr1bfb4p1nn6ayia3202nfr-nixos-system-client-test error="error: path '/nix/store/jwg6hsvasqr1bfb4p1nn6ayia3202nfr-nixos-system-client-test' is required, but there is no substituter that can build it"4443worker1 # [ 32.282676] nix-grpc-daemon[783]: ts=2026-09-20T13:33:35Z level=info event=oidc_jwks provider=mock keys=14444worker1 # [ 32.286913] nix-grpc-daemon[783]: ts=2026-09-20T13:33:35Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:338964445worker1 # [ 32.298394] nix-grpc-daemon[783]: ts=2026-09-20T13:33:35Z level=debug event=path_info_miss path=jwg6hsvasqr1bfb4p1nn6ayia3202nfr-nixos-system-client-test error="error: path '/nix/store/jwg6hsvasqr1bfb4p1nn6ayia3202nfr-nixos-system-client-test' is required, but there is no substituter that can build it"4446client: must succeed: rm -r /var/lib/nix-grpc-store4447client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.03 seconds)4448worker1 # [ 32.541360] nix-grpc-daemon[783]: ts=2026-09-20T13:33:35Z level=info event=denied method=QueryMissing cn=- role=none4449worker2 # [ 32.733535] nix-grpc-daemon[786]: ts=2026-09-20T13:33:36Z level=info event=denied method=QueryMissing cn=stranger role=none4450(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.03 seconds)4451subtest: DAG build is scheduled across workers and lands in the cache4452client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24453worker2 # [ 32.973491] nix-grpc-daemon[786]: ts=2026-09-20T13:33:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:341144454client # don't know how to build these paths:4455client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4456worker1 # [ 33.027855] nix-grpc-daemon[783]: ts=2026-09-20T13:33:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:338964457worker2 # [ 33.030515] nix-grpc-daemon[786]: ts=2026-09-20T13:33:37Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04458worker2 # [ 33.032941] nix-grpc-daemon[786]: ts=2026-09-20T13:33:37Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.3:50051 clients=14459worker1 # [ 33.121853] nix-grpc-daemon[783]: ts=2026-09-20T13:33:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:338964460worker1 # [ 33.144171] nix-grpc-daemon[783]: ts=2026-09-20T13:33:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:338964461client # copying 1 paths to 'lb:50051'...4462worker1 # [ 33.156721] nix-grpc-daemon[783]: ts=2026-09-20T13:33:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:338964463client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4464worker1 # [ 33.177288] nix-daemon[841]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4465worker1 # [ 33.204468] nix-grpc-daemon[783]: ts=2026-09-20T13:33:36Z level=info event=niks3_push_started pid=8514466worker1 # [ 33.484898] nix-grpc-daemon[851]: time=2026-09-20T13:33:36.723Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""4467worker1 # [ 33.683474] nix-daemon[841]: accepted connection from pid 863, user nix-grpc-daemon (trusted)4468worker1 # [ 33.701242] nix-daemon[841]: reaped child process 866, status = succeeded4469worker1 # [ 33.845921] nix-daemon[841]: accepted connection from pid 868, user nix-grpc-daemon (trusted)4470worker1 # [ 33.858215] nix-daemon[841]: reaped child process 871, status = succeeded4471lb # [ 33.878679] niks3-server[1040]: 2026/09/20 13:33:37 INFO Received uploads request method=POST path=/api/pending_closures4472worker1 # [ 33.901293] nix-grpc-daemon[851]: time=2026-09-20T13:33:37.144Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4473worker1 # [ 33.903774] nix-grpc-daemon[851]: time=2026-09-20T13:33:37.146Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4474lb # [ 33.940711] niks3-server[1040]: 2026/09/20 13:33:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4475lb # [ 33.960343] niks3-server[1040]: 2026/09/20 13:33:37 INFO Signed narinfos id=1 count=14476worker1 # [ 33.959438] nix-grpc-daemon[851]: time=2026-09-20T13:33:37.202Z level=INFO msg="Uploading 1 narinfos"4477lb # [ 33.974934] niks3-server[1040]: 2026/09/20 13:33:37 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4478lb # [ 33.989646] niks3-server[1040]: 2026/09/20 13:33:37 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4479lb # [ 33.994293] niks3-server[1040]: 2026/09/20 13:33:37 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4480lb # [ 33.999661] niks3-server[1040]: 2026/09/20 13:33:37 INFO Completed upload id=14481lb # [ 34.001347] niks3-server[1040]: 2026/09/20 13:33:37 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo4482worker1 # [ 33.998641] nix-grpc-daemon[851]: time=2026-09-20T13:33:37.241Z level=INFO msg="Upload complete. (499ms)"4483worker1 # [ 34.006506] nix-grpc-daemon[783]: ts=2026-09-20T13:33:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33896 duration_s=0 paths=1 nar_bytes_in=5044484worker1 # [ 34.013224] nix-grpc-daemon[783]: ts=2026-09-20T13:33:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:338964485worker1 # [ 34.026120] nix-daemon[841]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4486worker1 # [ 34.033909] nix-daemon[841]: reaped child process 849, status = succeeded4487client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4488client # farm-a> node-a: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4489worker1 # [ 34.042428] nix-daemon[841]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4490client # farm-a> LOG-farm-a-t14491worker1 # [ 35.457540] nix-daemon[841]: reaped child process 875, status = succeeded4492worker1 # [ 35.628099] nix-daemon[841]: accepted connection from pid 891, user nix-grpc-daemon (trusted)4493worker1 # [ 35.639315] nix-daemon[841]: reaped child process 895, status = succeeded4494lb # [ 35.655348] niks3-server[1040]: 2026/09/20 13:33:39 INFO Received uploads request method=POST path=/api/pending_closures4495worker1 # [ 35.666425] nix-grpc-daemon[851]: time=2026-09-20T13:33:38.909Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4496worker1 # [ 35.668047] nix-grpc-daemon[851]: time=2026-09-20T13:33:38.911Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4497lb # [ 35.708668] niks3-server[1040]: 2026/09/20 13:33:39 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4498lb # [ 35.717291] niks3-server[1040]: 2026/09/20 13:33:39 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4499lb # [ 35.733167] niks3-server[1040]: 2026/09/20 13:33:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4500lb # [ 35.740823] niks3-server[1040]: 2026/09/20 13:33:39 INFO Signed narinfos id=2 count=14501worker1 # [ 35.740319] nix-grpc-daemon[851]: time=2026-09-20T13:33:38.982Z level=INFO msg="Uploading 1 narinfos"4502lb # [ 35.744318] niks3-server[1040]: 2026/09/20 13:33:39 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4503lb # [ 35.751647] niks3-server[1040]: 2026/09/20 13:33:39 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4504lb # [ 35.755596] niks3-server[1040]: 2026/09/20 13:33:39 INFO Completed upload id=24505worker1 # [ 35.754910] nix-grpc-daemon[851]: time=2026-09-20T13:33:38.997Z level=INFO msg="Upload complete. (303ms)"4506lb # [ 35.757717] niks3-server[1040]: 2026/09/20 13:33:39 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4507worker1 # [ 35.763318] nix-grpc-daemon[783]: ts=2026-09-20T13:33:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33896 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14508worker2 # [ 35.735280] nix-grpc-daemon[786]: ts=2026-09-20T13:33:39Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04509worker2 # [ 35.736957] nix-grpc-daemon[786]: ts=2026-09-20T13:33:39Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.3:50051 clients=14510worker1 # [ 35.776917] nix-daemon[841]: reaped child process 873, status = succeeded4511worker1 # [ 35.816923] nix-grpc-daemon[783]: ts=2026-09-20T13:33:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:338964512worker1 # [ 35.828315] nix-grpc-daemon[783]: ts=2026-09-20T13:33:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:338964513worker1 # [ 35.839181] nix-grpc-daemon[851]: time=2026-09-20T13:33:39.082Z level=INFO msg="All 1 paths already cached"4514client # copying 1 paths to 'lb:50051'...4515client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4516worker1 # [ 35.846653] nix-grpc-daemon[783]: ts=2026-09-20T13:33:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:338964517worker1 # [ 35.856302] nix-daemon[841]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4518worker1 # [ 36.022316] nix-daemon[841]: accepted connection from pid 899, user nix-grpc-daemon (trusted)4519worker1 # [ 36.036113] nix-daemon[841]: reaped child process 902, status = succeeded4520worker1 # [ 36.220604] nix-daemon[841]: accepted connection from pid 904, user nix-grpc-daemon (trusted)4521worker1 # [ 36.237521] nix-daemon[841]: reaped child process 907, status = succeeded4522lb # [ 36.251609] niks3-server[1040]: 2026/09/20 13:33:40 INFO Received uploads request method=POST path=/api/pending_closures4523worker1 # [ 36.257206] nix-grpc-daemon[851]: time=2026-09-20T13:33:39.500Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4524worker1 # [ 36.258849] nix-grpc-daemon[851]: time=2026-09-20T13:33:39.501Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4525lb # [ 36.271100] niks3-server[1040]: 2026/09/20 13:33:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4526lb # [ 36.273487] niks3-server[1040]: 2026/09/20 13:33:40 INFO Signed narinfos id=3 count=14527worker1 # [ 36.274136] nix-grpc-daemon[851]: time=2026-09-20T13:33:39.516Z level=INFO msg="Uploading 1 narinfos"4528lb # [ 36.280436] niks3-server[1040]: 2026/09/20 13:33:40 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4529lb # [ 36.285417] niks3-server[1040]: 2026/09/20 13:33:40 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4530lb # [ 36.295681] niks3-server[1040]: 2026/09/20 13:33:40 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4531lb # [ 36.302142] niks3-server[1040]: 2026/09/20 13:33:40 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4532lb # [ 36.305276] niks3-server[1040]: 2026/09/20 13:33:40 INFO Completed upload id=34533worker1 # [ 36.304346] nix-grpc-daemon[851]: time=2026-09-20T13:33:39.547Z level=INFO msg="Upload complete. (433ms)"4534worker1 # [ 36.307076] nix-grpc-daemon[783]: ts=2026-09-20T13:33:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33896 duration_s=0 paths=1 nar_bytes_in=6244535worker1 # [ 36.309540] nix-grpc-daemon[783]: ts=2026-09-20T13:33:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:338964536worker1 # [ 36.316552] nix-daemon[841]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4537worker1 # [ 36.319723] nix-daemon[841]: reaped child process 897, status = succeeded4538client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4539worker1 # [ 36.326313] nix-daemon[841]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4540client # farm-b> node-a: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4541lb # [ 36.897992] envoy[911]: conn peer=[2001:db8:1::1]:42392 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=715 tx=364 ms=10704542lb # [ 36.901724] envoy[911]: conn peer=[2001:db8:1::1]:42404 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=365 ms=9164543lb # [ 36.905110] envoy[911]: conn peer=[2001:db8:1::1]:42408 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=154544lb # [ 36.908399] envoy[911]: conn peer=[2001:db8:1::1]:42422 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=134545client # farm-b> LOG-farm-b-t14546worker1 # [ 37.347973] nix-daemon[841]: reaped child process 911, status = succeeded4547worker1 # [ 37.508072] nix-daemon[841]: accepted connection from pid 926, user nix-grpc-daemon (trusted)4548worker1 # [ 37.520666] nix-daemon[841]: reaped child process 929, status = succeeded4549lb # [ 37.534642] niks3-server[1040]: 2026/09/20 13:33:41 INFO Received uploads request method=POST path=/api/pending_closures4550worker1 # [ 37.542120] nix-grpc-daemon[851]: time=2026-09-20T13:33:40.784Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4551worker1 # [ 37.544245] nix-grpc-daemon[851]: time=2026-09-20T13:33:40.786Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4552lb # [ 37.569171] niks3-server[1040]: 2026/09/20 13:33:41 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4553lb # [ 37.582647] niks3-server[1040]: 2026/09/20 13:33:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4554lb # [ 37.590113] niks3-server[1040]: 2026/09/20 13:33:41 INFO Signed narinfos id=4 count=14555worker1 # [ 37.592578] nix-grpc-daemon[851]: time=2026-09-20T13:33:40.835Z level=INFO msg="Uploading 1 narinfos"4556lb # [ 37.595610] niks3-server[1040]: 2026/09/20 13:33:41 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4557lb # [ 37.603066] niks3-server[1040]: 2026/09/20 13:33:41 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4558lb # [ 37.610733] niks3-server[1040]: 2026/09/20 13:33:41 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4559lb # [ 37.620097] niks3-server[1040]: 2026/09/20 13:33:41 INFO Completed upload id=44560lb # [ 37.621800] niks3-server[1040]: 2026/09/20 13:33:41 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4561worker1 # [ 37.619520] nix-grpc-daemon[851]: time=2026-09-20T13:33:40.862Z level=INFO msg="Upload complete. (285ms)"4562worker1 # [ 37.625880] nix-grpc-daemon[783]: ts=2026-09-20T13:33:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33896 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14563worker2 # [ 37.593500] nix-grpc-daemon[786]: ts=2026-09-20T13:33:41Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04564worker2 # [ 37.595394] nix-grpc-daemon[786]: ts=2026-09-20T13:33:41Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.3:50051 clients=14565worker1 # [ 37.641243] nix-daemon[841]: reaped child process 909, status = succeeded4566worker1 # [ 37.682866] nix-grpc-daemon[783]: ts=2026-09-20T13:33:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:338964567worker1 # [ 37.698196] nix-grpc-daemon[783]: ts=2026-09-20T13:33:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:338964568worker1 # [ 37.711958] nix-grpc-daemon[851]: time=2026-09-20T13:33:40.954Z level=INFO msg="All 2 paths already cached"4569client # copying 1 paths to 'lb:50051'...4570worker1 # [ 37.720625] nix-grpc-daemon[783]: ts=2026-09-20T13:33:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:338964571client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4572worker1 # [ 37.743225] nix-daemon[841]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4573worker1 # [ 37.919407] nix-daemon[841]: accepted connection from pid 933, user nix-grpc-daemon (trusted)4574worker1 # [ 37.933469] nix-daemon[841]: reaped child process 936, status = succeeded4575worker1 # [ 38.091564] nix-daemon[841]: accepted connection from pid 938, user nix-grpc-daemon (trusted)4576worker1 # [ 38.104348] nix-daemon[841]: reaped child process 941, status = succeeded4577lb # [ 38.118211] niks3-server[1040]: 2026/09/20 13:33:42 INFO Received uploads request method=POST path=/api/pending_closures4578worker1 # [ 38.127701] nix-grpc-daemon[851]: time=2026-09-20T13:33:41.370Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4579worker1 # [ 38.129628] nix-grpc-daemon[851]: time=2026-09-20T13:33:41.372Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4580lb # [ 38.151491] niks3-server[1040]: 2026/09/20 13:33:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4581worker1 # [ 38.155787] nix-grpc-daemon[851]: time=2026-09-20T13:33:41.398Z level=INFO msg="Uploading 1 narinfos"4582lb # [ 38.157120] niks3-server[1040]: 2026/09/20 13:33:42 INFO Signed narinfos id=5 count=14583lb # [ 38.159805] niks3-server[1040]: 2026/09/20 13:33:42 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4584lb # [ 38.175556] niks3-server[1040]: 2026/09/20 13:33:42 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4585lb # [ 38.178363] niks3-server[1040]: 2026/09/20 13:33:42 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4586lb # [ 38.184808] niks3-server[1040]: 2026/09/20 13:33:42 INFO Completed upload id=54587worker1 # [ 38.184144] nix-grpc-daemon[851]: time=2026-09-20T13:33:41.426Z level=INFO msg="Upload complete. (425ms)"4588lb # [ 38.189746] niks3-server[1040]: 2026/09/20 13:33:42 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4589worker1 # [ 38.188162] nix-grpc-daemon[783]: ts=2026-09-20T13:33:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33896 duration_s=0 paths=1 nar_bytes_in=7524590worker1 # [ 38.194325] nix-grpc-daemon[783]: ts=2026-09-20T13:33:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:338964591worker1 # [ 38.203090] nix-daemon[841]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4592worker1 # [ 38.208881] nix-daemon[841]: reaped child process 931, status = succeeded4593client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4594client # farm-top> node-a: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4595worker1 # [ 38.215465] nix-daemon[841]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4596client # farm-top> LOG-farm-top-t14597worker1 # [ 39.031102] nix-daemon[841]: reaped child process 945, status = succeeded4598worker1 # [ 39.181367] nix-daemon[841]: accepted connection from pid 961, user nix-grpc-daemon (trusted)4599worker1 # [ 39.193651] nix-daemon[841]: reaped child process 964, status = succeeded4600lb # [ 39.209985] niks3-server[1040]: 2026/09/20 13:33:43 INFO Received uploads request method=POST path=/api/pending_closures4601worker1 # [ 39.219293] nix-grpc-daemon[851]: time=2026-09-20T13:33:42.460Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4602worker1 # [ 39.220698] nix-grpc-daemon[851]: time=2026-09-20T13:33:42.460Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4603lb # [ 39.245730] niks3-server[1040]: 2026/09/20 13:33:43 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4604lb # [ 39.254405] niks3-server[1040]: 2026/09/20 13:33:43 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4605lb # [ 39.265635] niks3-server[1040]: 2026/09/20 13:33:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4606lb # [ 39.270633] niks3-server[1040]: 2026/09/20 13:33:43 INFO Signed narinfos id=6 count=14607worker1 # [ 39.272030] nix-grpc-daemon[851]: time=2026-09-20T13:33:42.514Z level=INFO msg="Uploading 1 narinfos"4608lb # [ 39.275542] niks3-server[1040]: 2026/09/20 13:33:43 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4609lb # [ 39.284291] niks3-server[1040]: 2026/09/20 13:33:43 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4610lb # [ 39.286825] niks3-server[1040]: 2026/09/20 13:33:43 INFO Completed upload id=64611worker1 # [ 39.285916] nix-grpc-daemon[851]: time=2026-09-20T13:33:42.528Z level=INFO msg="Upload complete. (263ms)"4612lb # [ 39.289149] niks3-server[1040]: 2026/09/20 13:33:43 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4613worker1 # [ 39.288624] nix-grpc-daemon[783]: ts=2026-09-20T13:33:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33896 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14614worker1 # [ 39.294932] nix-daemon[841]: reaped child process 943, status = succeeded4615client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 6.53 seconds)4616client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4617client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.16 seconds)4618client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4619client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.01 seconds)4620client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4621client # copying 3 paths...4622client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4623client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4624client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4625client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.24 seconds)4626(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 7.02 seconds)4627subtest: repeat is answered Cached by the scheduler without building4628worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4629worker1 # finding garbage collector roots...4630worker1 # deleting specified paths...4631worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4632worker1 # deleting unused links...4633worker1 # note: hard linking is currently saving 0.0 KiB4634worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.23 seconds)4635worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4636worker2 # finding garbage collector roots...4637worker2 # deleting specified paths...4638worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4639worker2 # deleting unused links...4640worker2 # note: hard linking is currently saving 0.0 KiB4641worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.26 seconds)4642client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24643worker2 # [ 40.436467] nix-grpc-daemon[786]: ts=2026-09-20T13:33:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:341144644client # don't know how to build these paths:4645client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4646worker1 # [ 40.483595] nix-grpc-daemon[783]: ts=2026-09-20T13:33:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:338964647worker1 # [ 40.493555] nix-grpc-daemon[851]: time=2026-09-20T13:33:43.736Z level=INFO msg="All 2 paths already cached"4648worker2 # [ 40.471111] nix-grpc-daemon[786]: ts=2026-09-20T13:33:44Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14649client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 0.23 seconds)4650client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4651client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.20 seconds)4652lb: must succeed: curl -sf localhost:9901/clusters4653lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4654lb: must succeed: curl -sf localhost:9901/clusters4655lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4656worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4657worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4658(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.11 seconds)4659subtest: inputs already in the cache are not sent to the scheduler4660lb: must succeed: curl -sf localhost:9901/clusters4661lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4662lb: must succeed: curl -sf localhost:9901/clusters4663lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4664worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4665worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4666client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&24667worker1 # [ 41.254670] nix-grpc-daemon[783]: ts=2026-09-20T13:33:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:338964668client # don't know how to build these paths:4669client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4670worker1 # [ 41.260801] nix-grpc-daemon[783]: ts=2026-09-20T13:33:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:338964671worker1 # [ 41.271565] nix-grpc-daemon[851]: time=2026-09-20T13:33:44.514Z level=INFO msg="All 2 paths already cached"4672worker2 # [ 41.246260] nix-grpc-daemon[786]: ts=2026-09-20T13:33:45Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04673worker2 # [ 41.250634] nix-grpc-daemon[786]: ts=2026-09-20T13:33:45Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.3:50051 clients=14674worker1 # [ 41.327993] nix-grpc-daemon[783]: ts=2026-09-20T13:33:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:338964675worker1 # [ 41.341681] nix-grpc-daemon[783]: ts=2026-09-20T13:33:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:338964676worker1 # [ 41.354391] nix-grpc-daemon[851]: time=2026-09-20T13:33:44.597Z level=INFO msg="All 2 paths already cached"4677client # copying 1 paths to 'lb:50051'...4678client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4679worker1 # [ 41.363585] nix-grpc-daemon[783]: ts=2026-09-20T13:33:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:338964680worker1 # [ 41.377850] nix-daemon[841]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4681worker1 # [ 41.556315] nix-daemon[841]: accepted connection from pid 981, user nix-grpc-daemon (trusted)4682worker1 # [ 41.569247] nix-daemon[841]: reaped child process 984, status = succeeded4683worker1 # [ 41.730975] nix-daemon[841]: accepted connection from pid 986, user nix-grpc-daemon (trusted)4684worker1 # [ 41.743155] nix-daemon[841]: reaped child process 989, status = succeeded4685lb # [ 41.754443] niks3-server[1040]: 2026/09/20 13:33:45 INFO Received uploads request method=POST path=/api/pending_closures4686worker1 # [ 41.758190] nix-grpc-daemon[851]: time=2026-09-20T13:33:45.001Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4687worker1 # [ 41.759688] nix-grpc-daemon[851]: time=2026-09-20T13:33:45.002Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4688lb # [ 41.772364] niks3-server[1040]: 2026/09/20 13:33:45 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4689lb # [ 41.775068] niks3-server[1040]: 2026/09/20 13:33:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4690lb # [ 41.777912] niks3-server[1040]: 2026/09/20 13:33:45 INFO Signed narinfos id=7 count=14691worker1 # [ 41.777119] nix-grpc-daemon[851]: time=2026-09-20T13:33:45.020Z level=INFO msg="Uploading 1 narinfos"4692lb # [ 41.781073] niks3-server[1040]: 2026/09/20 13:33:45 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4693lb # [ 41.786629] niks3-server[1040]: 2026/09/20 13:33:45 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4694lb # [ 41.789856] niks3-server[1040]: 2026/09/20 13:33:45 INFO Completed upload id=74695worker1 # [ 41.790258] nix-grpc-daemon[851]: time=2026-09-20T13:33:45.033Z level=INFO msg="Upload complete. (396ms)"4696lb # [ 41.791727] niks3-server[1040]: 2026/09/20 13:33:45 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4697worker1 # [ 41.792354] nix-grpc-daemon[783]: ts=2026-09-20T13:33:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33896 duration_s=0 paths=1 nar_bytes_in=7524698worker1 # [ 41.795668] nix-grpc-daemon[783]: ts=2026-09-20T13:33:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:338964699worker1 # [ 41.801630] nix-daemon[841]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4700worker1 # [ 41.804845] nix-daemon[841]: reaped child process 979, status = succeeded4701worker1 # [ 41.809989] nix-daemon[841]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4702client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4703client # farm-top> node-a: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4704client # farm-top> LOG-farm-top-t1b4705worker1 # [ 42.823657] nix-daemon[841]: reaped child process 993, status = succeeded4706worker1 # [ 42.974842] nix-daemon[841]: accepted connection from pid 1009, user nix-grpc-daemon (trusted)4707worker1 # [ 42.986514] nix-daemon[841]: reaped child process 1012, status = succeeded4708lb # [ 43.003139] niks3-server[1040]: 2026/09/20 13:33:46 INFO Received uploads request method=POST path=/api/pending_closures4709worker1 # [ 43.009672] nix-grpc-daemon[851]: time=2026-09-20T13:33:46.252Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4710worker1 # [ 43.011382] nix-grpc-daemon[851]: time=2026-09-20T13:33:46.254Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4711lb # [ 43.036071] niks3-server[1040]: 2026/09/20 13:33:47 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4712lb # [ 43.052077] niks3-server[1040]: 2026/09/20 13:33:47 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4713lb # [ 43.061413] niks3-server[1040]: 2026/09/20 13:33:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4714lb # [ 43.066689] niks3-server[1040]: 2026/09/20 13:33:47 INFO Signed narinfos id=8 count=14715worker1 # [ 43.068319] nix-grpc-daemon[851]: time=2026-09-20T13:33:46.311Z level=INFO msg="Uploading 1 narinfos"4716lb # [ 43.071588] niks3-server[1040]: 2026/09/20 13:33:47 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4717lb # [ 43.081149] niks3-server[1040]: 2026/09/20 13:33:47 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4718lb # [ 43.086790] niks3-server[1040]: 2026/09/20 13:33:47 INFO Completed upload id=84719worker1 # [ 43.086095] nix-grpc-daemon[851]: time=2026-09-20T13:33:46.328Z level=INFO msg="Upload complete. (269ms)"4720lb # [ 43.089375] niks3-server[1040]: 2026/09/20 13:33:47 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4721worker1 # [ 43.092623] nix-grpc-daemon[783]: ts=2026-09-20T13:33:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33896 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14722worker1 # [ 43.104912] nix-daemon[841]: reaped child process 991, status = succeeded4723client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.07 seconds)4724client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath4725client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.21 seconds)4726lb: must succeed: curl -sf localhost:9901/clusters4727lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4728lb: must succeed: curl -sf localhost:9901/clusters4729lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4730worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4731worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.03 seconds)4732(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.56 seconds)4733subtest: two clients wanting the same drv build it once4734client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4735client # [ 43.498783] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4736client # Running as unit: dedup1.service; invocation ID: f0d0190195384284828e0c970f480c964737client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 0.05 seconds)4738client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4739worker1 # [ 43.718378] nix-grpc-daemon[783]: ts=2026-09-20T13:33:46Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:338964740client # [ 43.698650] nix[884]: don't know how to build these paths:4741client # [ 43.699626] nix[884]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4742worker2 # [ 43.700426] nix-grpc-daemon[786]: ts=2026-09-20T13:33:47Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04743worker2 # [ 43.706202] nix-grpc-daemon[786]: ts=2026-09-20T13:33:47Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.4:50051 clients=14744worker2 # [ 43.710916] nix-grpc-daemon[786]: ts=2026-09-20T13:33:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:341144745worker2 # [ 43.727585] nix-grpc-daemon[786]: ts=2026-09-20T13:33:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:341144746client # [ 43.749284] nix[884]: copying 1 paths to 'lb:50051'...4747client # [ 43.750503] nix[884]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4748worker2 # [ 43.739424] nix-grpc-daemon[786]: ts=2026-09-20T13:33:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:341144749worker2 # [ 43.751559] nix-daemon[855]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4750worker2 # [ 43.771301] nix-grpc-daemon[786]: ts=2026-09-20T13:33:47Z level=info event=niks3_push_started pid=9064751worker2 # [ 43.989659] nix-grpc-daemon[906]: time=2026-09-20T13:33:48.129Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""4752worker2 # [ 44.168041] nix-daemon[855]: accepted connection from pid 919, user nix-grpc-daemon (trusted)4753worker2 # [ 44.184138] nix-daemon[855]: reaped child process 922, status = succeeded4754worker2 # [ 44.316148] nix-daemon[855]: accepted connection from pid 924, user nix-grpc-daemon (trusted)4755worker2 # [ 44.328811] nix-daemon[855]: reaped child process 927, status = succeeded4756lb # [ 44.382439] niks3-server[1040]: 2026/09/20 13:33:48 INFO Received uploads request method=POST path=/api/pending_closures4757worker2 # [ 44.354103] nix-grpc-daemon[906]: time=2026-09-20T13:33:48.496Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4758worker2 # [ 44.356334] nix-grpc-daemon[906]: time=2026-09-20T13:33:48.498Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4759lb # [ 44.421966] niks3-server[1040]: 2026/09/20 13:33:48 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4760lb # [ 44.427535] niks3-server[1040]: 2026/09/20 13:33:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4761lb # [ 44.436463] niks3-server[1040]: 2026/09/20 13:33:48 INFO Signed narinfos id=9 count=14762worker2 # [ 44.401591] nix-grpc-daemon[906]: time=2026-09-20T13:33:48.543Z level=INFO msg="Uploading 1 narinfos"4763lb # [ 44.443676] niks3-server[1040]: 2026/09/20 13:33:48 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4764lb # [ 44.454290] niks3-server[1040]: 2026/09/20 13:33:48 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4765lb # [ 44.461381] niks3-server[1040]: 2026/09/20 13:33:48 INFO Completed upload id=94766worker2 # [ 44.422980] nix-grpc-daemon[906]: time=2026-09-20T13:33:48.564Z level=INFO msg="Upload complete. (421ms)"4767lb # [ 44.463828] niks3-server[1040]: 2026/09/20 13:33:48 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4768worker2 # [ 44.427118] nix-grpc-daemon[786]: ts=2026-09-20T13:33:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34114 duration_s=0 paths=1 nar_bytes_in=4324769worker2 # [ 44.433150] nix-grpc-daemon[786]: ts=2026-09-20T13:33:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:341144770worker2 # [ 44.441132] nix-daemon[855]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4771worker2 # [ 44.446944] nix-daemon[855]: reaped child process 904, status = succeeded4772worker2 # [ 44.451720] nix-daemon[855]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4773client # [ 44.462350] nix[884]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4774client # [ 44.574587] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4775client # Running as unit: dedup2.service; invocation ID: 67b3917829074bdb989fecac99c06c114776client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 1.09 seconds)4777worker1 # [ 44.871947] nix-grpc-daemon[783]: ts=2026-09-20T13:33:48Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:338964778client # [ 44.851551] nix[900]: don't know how to build these paths:4779client # [ 44.852724] nix[900]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4780worker2 # [ 44.853855] nix-grpc-daemon[786]: ts=2026-09-20T13:33:48Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04781client # [ 44.870294] nix[900]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4782worker2 # [ 44.858815] nix-grpc-daemon[786]: ts=2026-09-20T13:33:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:341144783client: must succeed: sleep 24784lb # [ 46.898206] envoy[911]: conn peer=[2001:db8:1::1]:52898 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9481 tx=3797 ms=63224785lb # [ 46.902184] envoy[911]: conn peer=[2001:db8:1::1]:52906 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=494786lb # [ 46.905659] envoy[911]: conn peer=[2001:db8:1::1]:52914 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4169 tx=1727 ms=18464787client: (finished: must succeed: sleep 2, in 2.05 seconds)4788client: waiting for success: ! systemctl is-active dedup1 dedup24789worker2 # [ 48.077470] nix-daemon[855]: reaped child process 931, status = succeeded4790worker2 # [ 48.220089] nix-daemon[855]: accepted connection from pid 962, user nix-grpc-daemon (trusted)4791worker2 # [ 48.231481] nix-daemon[855]: reaped child process 965, status = succeeded4792lb # [ 48.285948] niks3-server[1040]: 2026/09/20 13:33:52 INFO Received uploads request method=POST path=/api/pending_closures4793worker2 # [ 48.256634] nix-grpc-daemon[906]: time=2026-09-20T13:33:52.398Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4794worker2 # [ 48.258248] nix-grpc-daemon[906]: time=2026-09-20T13:33:52.400Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4795lb # [ 48.326950] niks3-server[1040]: 2026/09/20 13:33:52 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4796lb # [ 48.345316] niks3-server[1040]: 2026/09/20 13:33:52 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4797lb # [ 48.352520] niks3-server[1040]: 2026/09/20 13:33:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4798lb # [ 48.356767] niks3-server[1040]: 2026/09/20 13:33:52 INFO Signed narinfos id=10 count=14799worker2 # [ 48.320885] nix-grpc-daemon[906]: time=2026-09-20T13:33:52.462Z level=INFO msg="Uploading 1 narinfos"4800lb # [ 48.363743] niks3-server[1040]: 2026/09/20 13:33:52 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4801lb # [ 48.371357] niks3-server[1040]: 2026/09/20 13:33:52 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4802lb # [ 48.375657] niks3-server[1040]: 2026/09/20 13:33:52 INFO Completed upload id=104803worker2 # [ 48.337288] nix-grpc-daemon[906]: time=2026-09-20T13:33:52.479Z level=INFO msg="Upload complete. (271ms)"4804lb # [ 48.378149] niks3-server[1040]: 2026/09/20 13:33:52 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4805worker2 # [ 48.343873] nix-grpc-daemon[786]: ts=2026-09-20T13:33:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34114 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14806worker2 # [ 48.350245] nix-grpc-daemon[786]: ts=2026-09-20T13:33:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34114 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14807worker2 # [ 48.359327] nix-daemon[855]: reaped child process 929, status = succeeded4808client # [ 48.394328] systemd[1]: dedup2.service: Deactivated successfully.4809client # [ 48.397299] systemd[1]: dedup2.service: Consumed 144ms CPU time over 3.819s wall clock time, 31.2M memory peak, 7.5K incoming IP traffic, 6.5K outgoing IP traffic.4810client # [ 48.432045] systemd[1]: dedup1.service: Deactivated successfully.4811client # [ 48.433608] systemd[1]: dedup1.service: Consumed 121ms CPU time over 4.934s wall clock time, 39.4M memory peak, 8.6K incoming IP traffic, 9.1K outgoing IP traffic.4812client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.08 seconds)4813client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4814client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)4815worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4816worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.06 seconds)4817worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4818worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.06 seconds)4819worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4820worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4821worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4822worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4823(finished: subtest: two clients wanting the same drv build it once, in 5.85 seconds)4824subtest: build hook: nix-daemon with builders = grpc://lb4825client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log4826client # [ 49.498602] systemd[1]: Started Nix Daemon.4827client # [ 49.672555] nix-daemon[942]: accepted connection from pid 939, user root (trusted)4828worker1 # [ 49.887595] nix-grpc-daemon[783]: ts=2026-09-20T13:33:53Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:338964829worker2 # [ 49.855096] nix-grpc-daemon[786]: ts=2026-09-20T13:33:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:341144830worker2 # [ 49.864685] nix-grpc-daemon[786]: ts=2026-09-20T13:33:54Z level=debug event=want client=6 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=04831worker2 # [ 49.866657] nix-grpc-daemon[786]: ts=2026-09-20T13:33:54Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.3:50051 clients=14832worker1 # [ 49.946239] nix-grpc-daemon[783]: ts=2026-09-20T13:33:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:338964833worker1 # [ 49.958258] nix-grpc-daemon[783]: ts=2026-09-20T13:33:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:338964834worker1 # [ 49.968971] nix-grpc-daemon[783]: ts=2026-09-20T13:33:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:338964835worker1 # [ 49.984137] nix-daemon[841]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4836worker1 # [ 50.166645] nix-daemon[841]: accepted connection from pid 1037, user nix-grpc-daemon (trusted)4837worker1 # [ 50.180779] nix-daemon[841]: reaped child process 1040, status = succeeded4838worker1 # [ 50.322241] nix-daemon[841]: accepted connection from pid 1042, user nix-grpc-daemon (trusted)4839worker1 # [ 50.333721] nix-daemon[841]: reaped child process 1045, status = succeeded4840lb # [ 50.346115] niks3-server[1040]: 2026/09/20 13:33:54 INFO Received uploads request method=POST path=/api/pending_closures4841worker1 # [ 50.349572] nix-grpc-daemon[851]: time=2026-09-20T13:33:53.592Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4842worker1 # [ 50.350997] nix-grpc-daemon[851]: time=2026-09-20T13:33:53.594Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"4843lb # [ 50.368922] niks3-server[1040]: 2026/09/20 13:33:54 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst4844lb # [ 50.379561] niks3-server[1040]: 2026/09/20 13:33:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4845lb # [ 50.381883] niks3-server[1040]: 2026/09/20 13:33:54 INFO Signed narinfos id=11 count=14846worker1 # [ 50.381065] nix-grpc-daemon[851]: time=2026-09-20T13:33:53.623Z level=INFO msg="Uploading 1 narinfos"4847lb # [ 50.384894] niks3-server[1040]: 2026/09/20 13:33:54 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls4848lb # [ 50.388993] niks3-server[1040]: 2026/09/20 13:33:54 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4849lb # [ 50.391739] niks3-server[1040]: 2026/09/20 13:33:54 INFO Completed upload id=114850worker1 # [ 50.391794] nix-grpc-daemon[851]: time=2026-09-20T13:33:53.634Z level=INFO msg="Upload complete. (395ms)"4851lb # [ 50.393640] niks3-server[1040]: 2026/09/20 13:33:54 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo4852worker1 # [ 50.394069] nix-grpc-daemon[783]: ts=2026-09-20T13:33:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33896 duration_s=0 paths=1 nar_bytes_in=5124853worker1 # [ 50.397171] nix-grpc-daemon[783]: ts=2026-09-20T13:33:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:338964854worker1 # [ 50.404487] nix-daemon[841]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4855worker1 # [ 50.405748] nix-daemon[841]: reaped child process 1035, status = succeeded4856worker1 # [ 50.410761] nix-daemon[841]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4857worker1 # [ 51.225414] nix-daemon[841]: reaped child process 1049, status = succeeded4858worker1 # [ 51.381804] nix-daemon[841]: accepted connection from pid 1064, user nix-grpc-daemon (trusted)4859worker1 # [ 51.392713] nix-daemon[841]: reaped child process 1067, status = succeeded4860lb # [ 51.411538] niks3-server[1040]: 2026/09/20 13:33:55 INFO Received uploads request method=POST path=/api/pending_closures4861worker1 # [ 51.418621] nix-grpc-daemon[851]: time=2026-09-20T13:33:54.661Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4862worker1 # [ 51.420461] nix-grpc-daemon[851]: time=2026-09-20T13:33:54.663Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"4863lb # [ 51.451509] niks3-server[1040]: 2026/09/20 13:33:55 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4864lb # [ 51.460855] niks3-server[1040]: 2026/09/20 13:33:55 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv4865lb # [ 51.467265] niks3-server[1040]: 2026/09/20 13:33:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4866worker1 # [ 51.473050] nix-grpc-daemon[851]: time=2026-09-20T13:33:54.715Z level=INFO msg="Uploading 1 narinfos"4867lb # [ 51.474514] niks3-server[1040]: 2026/09/20 13:33:55 INFO Signed narinfos id=12 count=14868lb # [ 51.483965] niks3-server[1040]: 2026/09/20 13:33:55 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4869lb # [ 51.494759] niks3-server[1040]: 2026/09/20 13:33:55 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls4870lb # [ 51.496909] niks3-server[1040]: 2026/09/20 13:33:55 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo4871lb # [ 51.502766] niks3-server[1040]: 2026/09/20 13:33:55 INFO Completed upload id=124872worker1 # [ 51.502232] nix-grpc-daemon[851]: time=2026-09-20T13:33:54.745Z level=INFO msg="Upload complete. (284ms)"4873worker1 # [ 51.508605] nix-grpc-daemon[783]: ts=2026-09-20T13:33:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33896 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=6 outputs=14874worker1 # [ 51.520820] nix-daemon[841]: reaped child process 1047, status = succeeded4875worker1 # [ 51.552153] nix-grpc-daemon[783]: ts=2026-09-20T13:33:54Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:338964876worker1 # [ 51.563794] nix-grpc-daemon[783]: ts=2026-09-20T13:33:54Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:338964877worker1 # [ 51.568324] nix-grpc-daemon[783]: ts=2026-09-20T13:33:54Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:33896 duration_s=0 paths=1 nar_bytes_out=1284878worker1 # [ 51.816060] nix-grpc-daemon[783]: ts=2026-09-20T13:33:55Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:338964879worker2 # [ 51.783186] nix-grpc-daemon[786]: ts=2026-09-20T13:33:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:341144880worker2 # [ 51.803179] nix-grpc-daemon[786]: ts=2026-09-20T13:33:55Z level=debug event=want client=7 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=04881worker2 # [ 51.807531] nix-grpc-daemon[786]: ts=2026-09-20T13:33:55Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.3:50051 clients=14882worker1 # [ 51.851879] nix-grpc-daemon[783]: ts=2026-09-20T13:33:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:338964883worker1 # [ 51.860167] nix-grpc-daemon[783]: ts=2026-09-20T13:33:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:338964884worker1 # [ 51.872744] nix-grpc-daemon[851]: time=2026-09-20T13:33:55.115Z level=INFO msg="All 1 paths already cached"4885worker1 # [ 51.879505] nix-grpc-daemon[783]: ts=2026-09-20T13:33:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:338964886worker1 # [ 51.887753] nix-daemon[841]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4887worker1 # [ 52.067069] nix-daemon[841]: accepted connection from pid 1071, user nix-grpc-daemon (trusted)4888worker1 # [ 52.081146] nix-daemon[841]: reaped child process 1074, status = succeeded4889worker1 # [ 52.241062] nix-daemon[841]: accepted connection from pid 1076, user nix-grpc-daemon (trusted)4890worker1 # [ 52.252520] nix-daemon[841]: reaped child process 1079, status = succeeded4891lb # [ 52.269519] niks3-server[1040]: 2026/09/20 13:33:56 INFO Received uploads request method=POST path=/api/pending_closures4892worker1 # [ 52.273233] nix-grpc-daemon[851]: time=2026-09-20T13:33:55.516Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4893worker1 # [ 52.274614] nix-grpc-daemon[851]: time=2026-09-20T13:33:55.516Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"4894lb # [ 52.281599] niks3-server[1040]: 2026/09/20 13:33:56 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst4895lb # [ 52.286269] niks3-server[1040]: 2026/09/20 13:33:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4896lb # [ 52.289370] niks3-server[1040]: 2026/09/20 13:33:56 INFO Signed narinfos id=13 count=14897worker1 # [ 52.288481] nix-grpc-daemon[851]: time=2026-09-20T13:33:55.531Z level=INFO msg="Uploading 1 narinfos"4898lb # [ 52.292630] niks3-server[1040]: 2026/09/20 13:33:56 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls4899lb # [ 52.297621] niks3-server[1040]: 2026/09/20 13:33:56 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4900lb # [ 52.301317] niks3-server[1040]: 2026/09/20 13:33:56 INFO Completed upload id=134901worker1 # [ 52.300286] nix-grpc-daemon[851]: time=2026-09-20T13:33:55.543Z level=INFO msg="Upload complete. (398ms)"4902lb # [ 52.303028] niks3-server[1040]: 2026/09/20 13:33:56 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo4903worker1 # [ 52.303071] nix-grpc-daemon[783]: ts=2026-09-20T13:33:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33896 duration_s=0 paths=1 nar_bytes_in=6244904worker1 # [ 52.305721] nix-grpc-daemon[783]: ts=2026-09-20T13:33:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:338964905worker1 # [ 52.311822] nix-daemon[841]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4906worker1 # [ 52.313983] nix-daemon[841]: reaped child process 1069, status = succeeded4907worker1 # [ 52.319268] nix-daemon[841]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4908worker1 # [ 53.745905] nix-daemon[841]: reaped child process 1083, status = succeeded4909worker1 # [ 53.876788] nix-daemon[841]: accepted connection from pid 1098, user nix-grpc-daemon (trusted)4910worker1 # [ 53.888520] nix-daemon[841]: reaped child process 1101, status = succeeded4911lb # [ 53.905062] niks3-server[1040]: 2026/09/20 13:33:57 INFO Received uploads request method=POST path=/api/pending_closures4912worker1 # [ 53.908664] nix-grpc-daemon[851]: time=2026-09-20T13:33:57.151Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4913worker1 # [ 53.910267] nix-grpc-daemon[851]: time=2026-09-20T13:33:57.153Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"4914lb # [ 53.925257] niks3-server[1040]: 2026/09/20 13:33:57 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv4915lb # [ 53.929622] niks3-server[1040]: 2026/09/20 13:33:57 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst4916lb # [ 53.935412] niks3-server[1040]: 2026/09/20 13:33:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4917lb # [ 53.939102] niks3-server[1040]: 2026/09/20 13:33:57 INFO Signed narinfos id=14 count=14918worker1 # [ 53.939736] nix-grpc-daemon[851]: time=2026-09-20T13:33:57.182Z level=INFO msg="Uploading 1 narinfos"4919lb # [ 53.942466] niks3-server[1040]: 2026/09/20 13:33:57 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls4920lb # [ 53.947756] niks3-server[1040]: 2026/09/20 13:33:57 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4921lb # [ 53.950420] niks3-server[1040]: 2026/09/20 13:33:57 INFO Completed upload id=144922worker1 # [ 53.950648] nix-grpc-daemon[851]: time=2026-09-20T13:33:57.193Z level=INFO msg="Upload complete. (223ms)"4923worker1 # [ 53.952943] nix-grpc-daemon[783]: ts=2026-09-20T13:33:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33896 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=7 outputs=14924lb # [ 53.956257] niks3-server[1040]: 2026/09/20 13:33:57 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo4925worker1 # [ 53.959229] nix-daemon[841]: reaped child process 1081, status = succeeded4926worker2 # [ 53.921614] nix-grpc-daemon[786]: ts=2026-09-20T13:33:58Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:341144927worker2 # [ 53.934284] nix-grpc-daemon[786]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...4928worker2 # [ 53.950147] nix-grpc-daemon[786]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...4929worker1 # [ 54.009560] nix-grpc-daemon[783]: ts=2026-09-20T13:33:57Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:338964930worker1 # [ 54.011736] nix-grpc-daemon[783]: ts=2026-09-20T13:33:57Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:33896 duration_s=0 paths=1 nar_bytes_out=1764931worker2 # [ 54.155860] nix-grpc-daemon[786]: ts=2026-09-20T13:33:58Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:341144932worker1 # [ 54.196070] nix-grpc-daemon[783]: ts=2026-09-20T13:33:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:338964933worker1 # [ 54.203636] nix-grpc-daemon[851]: time=2026-09-20T13:33:57.446Z level=INFO msg="All 2 paths already cached"4934worker2 # [ 54.179890] nix-grpc-daemon[786]: ts=2026-09-20T13:33:58Z level=debug event=want client=8 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=04935worker2 # [ 54.184451] nix-grpc-daemon[786]: ts=2026-09-20T13:33:58Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.3:50051 clients=14936worker1 # [ 54.262614] nix-grpc-daemon[783]: ts=2026-09-20T13:33:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:338964937worker1 # [ 54.275210] nix-grpc-daemon[783]: ts=2026-09-20T13:33:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:338964938worker1 # [ 54.287574] nix-grpc-daemon[851]: time=2026-09-20T13:33:57.530Z level=INFO msg="All 2 paths already cached"4939worker1 # [ 54.294767] nix-grpc-daemon[783]: ts=2026-09-20T13:33:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:338964940worker1 # [ 54.305259] nix-daemon[841]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4941worker1 # [ 54.485442] nix-daemon[841]: accepted connection from pid 1105, user nix-grpc-daemon (trusted)4942worker1 # [ 54.499153] nix-daemon[841]: reaped child process 1108, status = succeeded4943worker1 # [ 54.662257] nix-daemon[841]: accepted connection from pid 1110, user nix-grpc-daemon (trusted)4944worker1 # [ 54.675198] nix-daemon[841]: reaped child process 1113, status = succeeded4945lb # [ 54.691198] niks3-server[1040]: 2026/09/20 13:33:58 INFO Received uploads request method=POST path=/api/pending_closures4946worker1 # [ 54.697865] nix-grpc-daemon[851]: time=2026-09-20T13:33:57.940Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4947worker1 # [ 54.699727] nix-grpc-daemon[851]: time=2026-09-20T13:33:57.942Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"4948lb # [ 54.719130] niks3-server[1040]: 2026/09/20 13:33:58 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst4949lb # [ 54.726598] niks3-server[1040]: 2026/09/20 13:33:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4950worker1 # [ 54.728121] nix-grpc-daemon[851]: time=2026-09-20T13:33:57.971Z level=INFO msg="Uploading 1 narinfos"4951lb # [ 54.729441] niks3-server[1040]: 2026/09/20 13:33:58 INFO Signed narinfos id=15 count=14952lb # [ 54.731674] niks3-server[1040]: 2026/09/20 13:33:58 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls4953lb # [ 54.736471] niks3-server[1040]: 2026/09/20 13:33:58 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4954lb # [ 54.739529] niks3-server[1040]: 2026/09/20 13:33:58 INFO Completed upload id=154955worker1 # [ 54.738369] nix-grpc-daemon[851]: time=2026-09-20T13:33:57.981Z level=INFO msg="Upload complete. (420ms)"4956lb # [ 54.741742] niks3-server[1040]: 2026/09/20 13:33:58 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo4957worker1 # [ 54.740345] nix-grpc-daemon[783]: ts=2026-09-20T13:33:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33896 duration_s=0 paths=1 nar_bytes_in=7524958worker1 # [ 54.743675] nix-grpc-daemon[783]: ts=2026-09-20T13:33:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:338964959worker1 # [ 54.749790] nix-daemon[841]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4960worker1 # [ 54.753376] nix-daemon[841]: reaped child process 1103, status = succeeded4961worker1 # [ 54.758394] nix-daemon[841]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4962worker1 # [ 55.972486] nix-daemon[841]: reaped child process 1117, status = succeeded4963worker1 # [ 56.115268] nix-daemon[841]: accepted connection from pid 1132, user nix-grpc-daemon (trusted)4964worker1 # [ 56.126935] nix-daemon[841]: reaped child process 1135, status = succeeded4965lb # [ 56.143953] niks3-server[1040]: 2026/09/20 13:34:00 INFO Received uploads request method=POST path=/api/pending_closures4966worker1 # [ 56.148281] nix-grpc-daemon[851]: time=2026-09-20T13:33:59.389Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4967worker1 # [ 56.151916] nix-grpc-daemon[851]: time=2026-09-20T13:33:59.389Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"4968lb # [ 56.165501] niks3-server[1040]: 2026/09/20 13:34:00 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv4969lb # [ 56.169966] niks3-server[1040]: 2026/09/20 13:34:00 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst4970lb # [ 56.172856] niks3-server[1040]: 2026/09/20 13:34:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4971lb # [ 56.175441] niks3-server[1040]: 2026/09/20 13:34:00 INFO Signed narinfos id=16 count=14972worker1 # [ 56.174605] nix-grpc-daemon[851]: time=2026-09-20T13:33:59.417Z level=INFO msg="Uploading 1 narinfos"4973lb # [ 56.177348] niks3-server[1040]: 2026/09/20 13:34:00 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls4974lb # [ 56.182220] niks3-server[1040]: 2026/09/20 13:34:00 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4975lb # [ 56.185851] niks3-server[1040]: 2026/09/20 13:34:00 INFO Completed upload id=164976worker1 # [ 56.184837] nix-grpc-daemon[851]: time=2026-09-20T13:33:59.427Z level=INFO msg="Upload complete. (219ms)"4977lb # [ 56.187382] niks3-server[1040]: 2026/09/20 13:34:00 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo4978worker1 # [ 56.187135] nix-grpc-daemon[783]: ts=2026-09-20T13:33:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33896 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=8 outputs=14979worker1 # [ 56.193432] nix-daemon[841]: reaped child process 1115, status = succeeded4980worker2 # [ 56.157904] nix-grpc-daemon[786]: ts=2026-09-20T13:34:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:341144981worker2 # [ 56.166969] nix-grpc-daemon[786]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...4982worker1 # [ 56.229059] nix-grpc-daemon[783]: ts=2026-09-20T13:33:59Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:338964983worker1 # [ 56.230718] nix-grpc-daemon[783]: ts=2026-09-20T13:33:59Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:33896 duration_s=0 paths=1 nar_bytes_out=2324984client # [ 56.313705] nix-daemon[942]: reaped child process 944, status = succeeded4985client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.02 seconds)4986client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top4987client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.03 seconds)4988client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4989client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.03 seconds)4990client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4991client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4992(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.11 seconds)4993subtest: an input only one worker has still reaches the builder4994client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only4995client # this derivation will be built:4996client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv4997client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...4998client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only, in 0.44 seconds)4999client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5000lb # [ 56.898778] envoy[911]: conn peer=[2001:db8:1::1]:33070 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1453 tx=1085 ms=35305001lb # [ 56.902515] envoy[911]: conn peer=[2001:db8:1::1]:33060 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2972 tx=1373 ms=47235002lb # [ 56.906109] envoy[911]: conn peer=[2001:db8:1::1]:33096 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=535003lb # [ 56.909516] envoy[911]: conn peer=[2001:db8:1::1]:33084 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3401 tx=1636 ms=17335004lb # [ 56.913190] envoy[911]: conn peer=[2001:db8:1::1]:33104 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3837 tx=1946 ms=22075005lb # [ 56.917204] envoy[911]: conn peer=[2001:db8:1::1]:40210 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=378 ms=85006lb # [ 56.920585] envoy[911]: conn peer=[2001:db8:1::1]:40214 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4285 tx=2205 ms=20905007lb # [ 56.924169] envoy[911]: conn peer=[2001:db8:1::1]:40220 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=525008client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.17 seconds)5009worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5010worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.22 seconds)5011worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5012worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.03 seconds)5013client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&25014client # [ 57.474035] nix-daemon[942]: accepted connection from pid 1024, user root (trusted)5015client # this derivation will be built:5016client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5017client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...5018worker2 # [ 57.640745] nix-grpc-daemon[786]: ts=2026-09-20T13:34:01Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:341145019worker2 # [ 57.646405] nix-grpc-daemon[786]: ts=2026-09-20T13:34:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:341145020client # copying 1 paths...5021client # copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' to 'grpc://lb:50051'...5022worker1 # [ 57.696697] nix-grpc-daemon[783]: ts=2026-09-20T13:34:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:338965023worker1 # [ 57.707786] nix-daemon[841]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5024worker1 # [ 57.879486] nix-daemon[841]: accepted connection from pid 1144, user nix-grpc-daemon (trusted)5025worker1 # [ 57.891180] nix-daemon[841]: reaped child process 1147, status = succeeded5026lb # [ 57.908314] niks3-server[1040]: 2026/09/20 13:34:01 INFO Received uploads request method=POST path=/api/pending_closures5027worker1 # [ 58.077941] nix-grpc-daemon[851]: time=2026-09-20T13:34:01.320Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5028worker1 # [ 58.082638] nix-grpc-daemon[851]: time=2026-09-20T13:34:01.324Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5029lb # [ 58.105611] niks3-server[1040]: 2026/09/20 13:34:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign5030lb # [ 58.112304] niks3-server[1040]: 2026/09/20 13:34:02 INFO Signed narinfos id=17 count=15031worker1 # [ 58.111853] nix-grpc-daemon[851]: time=2026-09-20T13:34:01.354Z level=INFO msg="Uploading 1 narinfos"5032lb # [ 58.118786] niks3-server[1040]: 2026/09/20 13:34:02 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5033lb # [ 58.134455] niks3-server[1040]: 2026/09/20 13:34:02 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete5034lb # [ 58.139158] niks3-server[1040]: 2026/09/20 13:34:02 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5035lb # [ 58.157421] niks3-server[1040]: 2026/09/20 13:34:02 INFO Completed upload id=175036worker1 # [ 58.156138] nix-grpc-daemon[851]: time=2026-09-20T13:34:01.399Z level=INFO msg="Upload complete. (438ms)"5037worker1 # [ 58.161170] nix-grpc-daemon[783]: ts=2026-09-20T13:34:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33896 duration_s=0 paths=1 nar_bytes_in=1205038lb # [ 58.171276] niks3-server[1040]: 2026/09/20 13:34:02 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5039worker1 # [ 58.173711] nix-daemon[841]: reaped child process 1142, status = succeeded5040worker2 # [ 58.139830] nix-grpc-daemon[786]: ts=2026-09-20T13:34:02Z level=debug event=want client=9 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05041worker2 # [ 58.142222] nix-grpc-daemon[786]: ts=2026-09-20T13:34:02Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=15042worker2 # [ 58.185435] nix-grpc-daemon[786]: ts=2026-09-20T13:34:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:341145043worker2 # [ 58.198242] nix-grpc-daemon[786]: ts=2026-09-20T13:34:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:341145044client # copying 2 paths to 'lb:50051'...5045worker2 # [ 58.214124] nix-grpc-daemon[786]: ts=2026-09-20T13:34:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:341145046client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5047client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5048worker2 # [ 58.235332] nix-daemon[855]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5049worker2 # [ 58.413119] nix-daemon[855]: accepted connection from pid 992, user nix-grpc-daemon (trusted)5050worker2 # [ 58.426115] nix-daemon[855]: reaped child process 995, status = succeeded5051worker2 # [ 58.582796] nix-daemon[855]: accepted connection from pid 997, user nix-grpc-daemon (trusted)5052worker2 # [ 58.594933] nix-daemon[855]: reaped child process 1000, status = succeeded5053lb # [ 58.647310] niks3-server[1040]: 2026/09/20 13:34:02 INFO Received uploads request method=POST path=/api/pending_closures5054lb # [ 58.655532] niks3-server[1040]: 2026/09/20 13:34:02 INFO Received uploads request method=POST path=/api/pending_closures5055worker2 # [ 58.625610] nix-grpc-daemon[906]: time=2026-09-20T13:34:02.766Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5056worker2 # [ 58.627068] nix-grpc-daemon[906]: time=2026-09-20T13:34:02.766Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5057worker2 # [ 58.628707] nix-grpc-daemon[906]: time=2026-09-20T13:34:02.766Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5058lb # [ 58.673974] niks3-server[1040]: 2026/09/20 13:34:02 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5059lb # [ 58.677326] niks3-server[1040]: 2026/09/20 13:34:02 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5060lb # [ 58.684342] niks3-server[1040]: 2026/09/20 13:34:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5061lb # [ 58.686296] niks3-server[1040]: 2026/09/20 13:34:02 INFO Signed narinfos id=18 count=15062lb # [ 58.687693] niks3-server[1040]: 2026/09/20 13:34:02 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5063lb # [ 58.690503] niks3-server[1040]: 2026/09/20 13:34:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5064lb # [ 58.692691] niks3-server[1040]: 2026/09/20 13:34:02 INFO Signed narinfos id=19 count=25065worker2 # [ 58.654889] nix-grpc-daemon[906]: time=2026-09-20T13:34:02.796Z level=INFO msg="Uploading 3 narinfos"5066lb # [ 58.695182] niks3-server[1040]: 2026/09/20 13:34:02 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5067lb # [ 58.706860] niks3-server[1040]: 2026/09/20 13:34:02 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5068lb # [ 58.709457] niks3-server[1040]: 2026/09/20 13:34:02 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5069lb # [ 58.713308] niks3-server[1040]: 2026/09/20 13:34:02 INFO Completed upload id=185070lb # [ 58.715283] niks3-server[1040]: 2026/09/20 13:34:02 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5071lb # [ 58.716823] niks3-server[1040]: 2026/09/20 13:34:02 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5072lb # [ 58.720577] niks3-server[1040]: 2026/09/20 13:34:02 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5073lb # [ 58.722726] niks3-server[1040]: 2026/09/20 13:34:02 INFO Completed upload id=195074worker2 # [ 58.684507] nix-grpc-daemon[906]: time=2026-09-20T13:34:02.826Z level=INFO msg="Upload complete. (433ms)"5075worker2 # [ 58.687239] nix-grpc-daemon[786]: ts=2026-09-20T13:34:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34114 duration_s=0 paths=2 nar_bytes_in=9765076worker2 # [ 58.692263] nix-grpc-daemon[786]: ts=2026-09-20T13:34:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:341145077worker2 # [ 58.698308] nix-daemon[855]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5078worker2 # [ 58.703900] nix-daemon[855]: reaped child process 990, status = succeeded5079worker2 # [ 58.708298] nix-grpc-daemon[786]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...5080worker2 # [ 58.715205] nix-daemon[855]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5081client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5082client # uses-input-w1onlya> worker2: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5083worker2 # [ 59.743457] nix-daemon[855]: reaped child process 1005, status = succeeded5084worker2 # [ 59.901441] nix-daemon[855]: accepted connection from pid 1020, user nix-grpc-daemon (trusted)5085worker2 # [ 59.914056] nix-daemon[855]: reaped child process 1023, status = succeeded5086lb # [ 59.969694] niks3-server[1040]: 2026/09/20 13:34:03 INFO Received uploads request method=POST path=/api/pending_closures5087worker2 # [ 59.939830] nix-grpc-daemon[906]: time=2026-09-20T13:34:04.081Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5088lb # [ 59.999439] niks3-server[1040]: 2026/09/20 13:34:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5089worker2 # [ 59.967846] nix-grpc-daemon[906]: time=2026-09-20T13:34:04.109Z level=INFO msg="Uploading 1 narinfos"5090lb # [ 60.007180] niks3-server[1040]: 2026/09/20 13:34:04 INFO Signed narinfos id=20 count=15091lb # [ 60.012692] niks3-server[1040]: 2026/09/20 13:34:04 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5092lb # [ 60.020849] niks3-server[1040]: 2026/09/20 13:34:04 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5093lb # [ 60.030285] niks3-server[1040]: 2026/09/20 13:34:04 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5094worker2 # [ 59.999414] nix-grpc-daemon[906]: time=2026-09-20T13:34:04.141Z level=INFO msg="Upload complete. (275ms)"5095lb # [ 60.037855] niks3-server[1040]: 2026/09/20 13:34:04 INFO Completed upload id=205096lb # [ 60.040790] niks3-server[1040]: 2026/09/20 13:34:04 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5097worker2 # [ 60.003174] nix-grpc-daemon[786]: ts=2026-09-20T13:34:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34114 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=9 outputs=15098worker2 # [ 60.011645] nix-daemon[855]: reaped child process 1002, status = succeeded5099client # copying 1 paths...5100worker2 # [ 60.050140] nix-grpc-daemon[786]: ts=2026-09-20T13:34:04Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:341145101client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5102worker2 # [ 60.072377] nix-grpc-daemon[786]: ts=2026-09-20T13:34:04Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:341145103worker2 # [ 60.077884] nix-grpc-daemon[786]: ts=2026-09-20T13:34:04Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34114 duration_s=0 paths=1 nar_bytes_out=1205104client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.92 seconds)5105client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&25106client # [ 60.363232] nix-daemon[942]: accepted connection from pid 1045, user root (trusted)5107client # [ 60.437050] nix-daemon[942]: reaped child process 1027, status = succeeded5108client # this derivation will be built:5109client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5110worker2 # [ 60.638565] nix-grpc-daemon[786]: ts=2026-09-20T13:34:04Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:341145111client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...5112worker2 # [ 60.644824] nix-grpc-daemon[786]: ts=2026-09-20T13:34:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:341145113worker2 # [ 60.649487] nix-grpc-daemon[906]: time=2026-09-20T13:34:04.791Z level=INFO msg="All 1 paths already cached"5114worker2 # [ 60.657665] nix-grpc-daemon[786]: ts=2026-09-20T13:34:04Z level=debug event=want client=10 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05115worker2 # [ 60.659591] nix-grpc-daemon[786]: ts=2026-09-20T13:34:04Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15116worker1 # [ 60.742177] nix-grpc-daemon[783]: ts=2026-09-20T13:34:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:338965117worker1 # [ 60.755712] nix-grpc-daemon[783]: ts=2026-09-20T13:34:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:338965118client # copying 1 paths to 'lb:50051'...5119client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5120worker1 # [ 60.776593] nix-grpc-daemon[783]: ts=2026-09-20T13:34:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:338965121worker1 # [ 60.788612] nix-daemon[841]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5122worker1 # [ 60.805504] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 22 ms (attempt 1/5)5123worker1 # [ 60.833480] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 149 ms (attempt 2/5)5124worker1 # [ 60.988684] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 273 ms (attempt 3/5)5125worker1 # [ 61.267835] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 70 ms (attempt 4/5)5126worker1 # [ 61.343946] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5127worker1 # [ 61.350179] nix-grpc-daemon[783]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5128worker1 # [ 61.538686] nix-daemon[841]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)5129worker1 # [ 61.550773] nix-daemon[841]: reaped child process 1158, status = succeeded5130worker1 # [ 61.719467] nix-daemon[841]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5131worker1 # [ 61.731942] nix-daemon[841]: reaped child process 1163, status = succeeded5132lb # [ 61.745719] niks3-server[1040]: 2026/09/20 13:34:05 INFO Received uploads request method=POST path=/api/pending_closures5133worker1 # [ 61.752889] nix-grpc-daemon[851]: time=2026-09-20T13:34:04.995Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5134worker1 # [ 61.754649] nix-grpc-daemon[851]: time=2026-09-20T13:34:04.997Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5135lb # [ 61.773595] niks3-server[1040]: 2026/09/20 13:34:05 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5136lb # [ 61.784452] niks3-server[1040]: 2026/09/20 13:34:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5137lb # [ 61.790307] niks3-server[1040]: 2026/09/20 13:34:05 INFO Signed narinfos id=21 count=15138worker1 # [ 61.792168] nix-grpc-daemon[851]: time=2026-09-20T13:34:05.035Z level=INFO msg="Uploading 1 narinfos"5139lb # [ 61.794808] niks3-server[1040]: 2026/09/20 13:34:05 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5140lb # [ 61.804283] niks3-server[1040]: 2026/09/20 13:34:05 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5141lb # [ 61.809427] niks3-server[1040]: 2026/09/20 13:34:05 INFO Completed upload id=215142worker1 # [ 61.809829] nix-grpc-daemon[851]: time=2026-09-20T13:34:05.052Z level=INFO msg="Upload complete. (436ms)"5143lb # [ 61.811206] niks3-server[1040]: 2026/09/20 13:34:05 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5144worker1 # [ 61.813503] nix-grpc-daemon[783]: ts=2026-09-20T13:34:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33896 duration_s=1 paths=1 nar_bytes_in=5605145worker1 # [ 61.818577] nix-grpc-daemon[783]: ts=2026-09-20T13:34:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:338965146worker1 # [ 61.826618] nix-daemon[841]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5147worker1 # [ 61.836340] nix-daemon[841]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5148client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5149worker1 # [ 61.838470] nix-daemon[841]: reaped child process 1149, status = succeeded5150client # uses-input-w1onlyb> node-a: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5151worker1 # [ 63.452378] nix-daemon[841]: reaped child process 1167, status = succeeded5152worker1 # [ 63.616262] nix-daemon[841]: accepted connection from pid 1182, user nix-grpc-daemon (trusted)5153worker1 # [ 63.627185] nix-daemon[841]: reaped child process 1185, status = succeeded5154lb # [ 63.646811] niks3-server[1040]: 2026/09/20 13:34:07 INFO Received uploads request method=POST path=/api/pending_closures5155worker1 # [ 63.653777] nix-grpc-daemon[851]: time=2026-09-20T13:34:06.896Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5156lb # [ 63.677171] niks3-server[1040]: 2026/09/20 13:34:07 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5157lb # [ 63.685714] niks3-server[1040]: 2026/09/20 13:34:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5158worker1 # [ 63.689360] nix-grpc-daemon[851]: time=2026-09-20T13:34:06.932Z level=INFO msg="Uploading 1 narinfos"5159lb # [ 63.690838] niks3-server[1040]: 2026/09/20 13:34:07 INFO Signed narinfos id=22 count=15160lb # [ 63.697649] niks3-server[1040]: 2026/09/20 13:34:07 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5161lb # [ 63.708809] niks3-server[1040]: 2026/09/20 13:34:07 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5162worker1 # [ 63.713328] nix-grpc-daemon[851]: time=2026-09-20T13:34:06.956Z level=INFO msg="Upload complete. (268ms)"5163lb # [ 63.714673] niks3-server[1040]: 2026/09/20 13:34:07 INFO Completed upload id=225164worker1 # [ 63.715799] nix-grpc-daemon[783]: ts=2026-09-20T13:34:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33896 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=10 outputs=15165lb # [ 63.719333] niks3-server[1040]: 2026/09/20 13:34:07 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5166worker1 # [ 63.727184] nix-daemon[841]: reaped child process 1165, status = succeeded5167client # copying 1 paths...5168worker2 # [ 63.726716] nix-grpc-daemon[786]: ts=2026-09-20T13:34:07Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:341145169worker2 # [ 63.739533] nix-grpc-daemon[786]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5170client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5171worker1 # [ 63.810175] nix-grpc-daemon[783]: ts=2026-09-20T13:34:07Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:338965172worker1 # [ 63.814788] nix-grpc-daemon[783]: ts=2026-09-20T13:34:07Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:33896 duration_s=0 paths=1 nar_bytes_out=1205173client # [ 63.851114] nix-daemon[942]: reaped child process 1048, status = succeeded5174client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.66 seconds)5175client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&25176client # [ 64.025973] nix-daemon[942]: accepted connection from pid 1067, user root (trusted)5177client # this derivation will be built:5178client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5179worker1 # [ 64.229710] nix-grpc-daemon[783]: ts=2026-09-20T13:34:07Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:338965180client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...5181worker2 # [ 64.195933] nix-grpc-daemon[786]: ts=2026-09-20T13:34:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:341145182worker2 # [ 64.206558] nix-grpc-daemon[906]: time=2026-09-20T13:34:08.348Z level=INFO msg="All 1 paths already cached"5183worker2 # [ 64.221933] nix-grpc-daemon[786]: ts=2026-09-20T13:34:08Z level=debug event=want client=11 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05184worker2 # [ 64.227435] nix-grpc-daemon[786]: ts=2026-09-20T13:34:08Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15185worker2 # [ 64.266332] nix-grpc-daemon[786]: ts=2026-09-20T13:34:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:341145186worker2 # [ 64.279602] nix-grpc-daemon[786]: ts=2026-09-20T13:34:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:341145187worker2 # [ 64.291452] nix-grpc-daemon[906]: time=2026-09-20T13:34:08.433Z level=INFO msg="All 1 paths already cached"5188client # copying 1 paths to 'lb:50051'...5189client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5190worker2 # [ 64.300820] nix-grpc-daemon[786]: ts=2026-09-20T13:34:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:341145191worker2 # [ 64.312307] nix-daemon[855]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5192worker2 # [ 64.474142] nix-daemon[855]: accepted connection from pid 1032, user nix-grpc-daemon (trusted)5193worker2 # [ 64.487577] nix-daemon[855]: reaped child process 1035, status = succeeded5194worker2 # [ 64.631071] nix-daemon[855]: accepted connection from pid 1037, user nix-grpc-daemon (trusted)5195worker2 # [ 64.643380] nix-daemon[855]: reaped child process 1040, status = succeeded5196lb # [ 64.697483] niks3-server[1040]: 2026/09/20 13:34:08 INFO Received uploads request method=POST path=/api/pending_closures5197worker2 # [ 64.666386] nix-grpc-daemon[906]: time=2026-09-20T13:34:08.808Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5198worker2 # [ 64.667773] nix-grpc-daemon[906]: time=2026-09-20T13:34:08.808Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5199lb # [ 64.723054] niks3-server[1040]: 2026/09/20 13:34:08 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5200lb # [ 64.736134] niks3-server[1040]: 2026/09/20 13:34:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5201worker2 # [ 64.703665] nix-grpc-daemon[906]: time=2026-09-20T13:34:08.845Z level=INFO msg="Uploading 1 narinfos"5202lb # [ 64.742141] niks3-server[1040]: 2026/09/20 13:34:08 INFO Signed narinfos id=23 count=15203lb # [ 64.748154] niks3-server[1040]: 2026/09/20 13:34:08 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5204lb # [ 64.758900] niks3-server[1040]: 2026/09/20 13:34:08 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5205worker2 # [ 64.727146] nix-grpc-daemon[906]: time=2026-09-20T13:34:08.869Z level=INFO msg="Upload complete. (402ms)"5206lb # [ 64.765438] niks3-server[1040]: 2026/09/20 13:34:08 INFO Completed upload id=235207lb # [ 64.768942] niks3-server[1040]: 2026/09/20 13:34:08 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5208worker2 # [ 64.731231] nix-grpc-daemon[786]: ts=2026-09-20T13:34:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34114 duration_s=0 paths=1 nar_bytes_in=5605209worker2 # [ 64.735743] nix-grpc-daemon[786]: ts=2026-09-20T13:34:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:341145210worker2 # [ 64.745585] nix-daemon[855]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5211worker2 # [ 64.749133] nix-daemon[855]: reaped child process 1030, status = succeeded5212worker2 # [ 64.754209] nix-daemon[855]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5213client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5214client # uses-input-w1onlyc> worker2: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5215worker2 # [ 65.568756] nix-daemon[855]: reaped child process 1044, status = succeeded5216worker2 # [ 65.689707] nix-daemon[855]: accepted connection from pid 1059, user nix-grpc-daemon (trusted)5217worker2 # [ 65.703208] nix-daemon[855]: reaped child process 1062, status = succeeded5218lb # [ 65.753978] niks3-server[1040]: 2026/09/20 13:34:09 INFO Received uploads request method=POST path=/api/pending_closures5219worker2 # [ 65.724535] nix-grpc-daemon[906]: time=2026-09-20T13:34:09.866Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5220lb # [ 65.792515] niks3-server[1040]: 2026/09/20 13:34:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5221lb # [ 65.797723] niks3-server[1040]: 2026/09/20 13:34:09 INFO Signed narinfos id=24 count=15222worker2 # [ 65.762587] nix-grpc-daemon[906]: time=2026-09-20T13:34:09.904Z level=INFO msg="Uploading 1 narinfos"5223lb # [ 65.802777] niks3-server[1040]: 2026/09/20 13:34:09 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5224lb # [ 65.811591] niks3-server[1040]: 2026/09/20 13:34:09 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5225lb # [ 65.821349] niks3-server[1040]: 2026/09/20 13:34:09 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5226lb # [ 65.827379] niks3-server[1040]: 2026/09/20 13:34:09 INFO Completed upload id=245227worker2 # [ 65.791688] nix-grpc-daemon[906]: time=2026-09-20T13:34:09.933Z level=INFO msg="Upload complete. (231ms)"5228lb # [ 65.830409] niks3-server[1040]: 2026/09/20 13:34:09 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5229worker2 # [ 65.797496] nix-grpc-daemon[786]: ts=2026-09-20T13:34:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34114 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=11 outputs=15230worker2 # [ 65.807971] nix-daemon[855]: reaped child process 1042, status = succeeded5231client # copying 1 paths...5232worker1 # [ 65.880729] nix-grpc-daemon[783]: ts=2026-09-20T13:34:09Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:338965233worker1 # [ 65.891780] nix-grpc-daemon[783]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5234client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5235worker2 # [ 65.884160] nix-grpc-daemon[786]: ts=2026-09-20T13:34:10Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:341145236worker2 # [ 65.888724] nix-grpc-daemon[786]: ts=2026-09-20T13:34:10Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34114 duration_s=0 paths=1 nar_bytes_out=1205237client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.10 seconds)5238(finished: subtest: an input only one worker has still reaches the builder, in 9.55 seconds)5239subtest: upload whose reference the worker lacks is completed from the cache5240client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache5241client # [ 65.979633] nix-daemon[942]: reaped child process 1070, status = succeeded5242client # this derivation will be built:5243client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5244client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5245client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache, in 0.36 seconds)5246client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache5247client # this derivation will be built:5248client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5249client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5250client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache, in 0.36 seconds)5251client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5252client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.15 seconds)5253worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5254lb # [ 66.898977] envoy[911]: conn peer=[2001:db8:1::1]:40240 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=575255lb # [ 66.903269] envoy[911]: conn peer=[2001:db8:1::1]:40224 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4195 tx=1752 ms=24915256lb # [ 66.907210] envoy[911]: conn peer=[2001:db8:1::1]:40248 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3714 tx=1916 ms=31485257lb # [ 66.911497] envoy[911]: conn peer=[2001:db8:1::1]:49886 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=195258lb # [ 66.915390] envoy[911]: conn peer=[2001:db8:1::1]:49900 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3702 tx=1916 ms=17085259lb # [ 66.920184] envoy[911]: conn peer=[2001:db8:1::1]:49910 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=421 tx=328 ms=185260worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.22 seconds)5261worker2 # [ 67.093236] systemd[1]: Stopping Nix worker-protocol over gRPC...5262worker2 # [ 67.167488] nix-grpc-daemon[786]: ts=2026-09-20T13:34:11Z level=info event=signal_exit5263worker2 # [ 67.410179] nix-grpc-daemon[786]: ts=2026-09-20T13:34:11Z level=info event=scheduler_yield5264lb # [ 67.451744] niks3-server[1040]: 2026/09/20 13:34:11 INFO lead: released remote=192.168.1.4:496545265worker2 # [ 67.414621] nix-grpc-daemon[786]: ts=2026-09-20T13:34:11Z level=info event=scheduler_restarting addr=lb:500515266worker1 # [ 67.456724] nix-grpc-daemon[783]: ts=2026-09-20T13:34:10Z level=info event=scheduler_restarting addr=lb:500515267worker2 # [ 67.420490] nix-grpc-daemon[786]: ts=2026-09-20T13:34:11Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:341085268worker2 # [ 67.423863] nix-grpc-daemon[786]: ts=2026-09-20T13:34:11Z level=info event=worker_gone addr=192.168.1.4:500515269worker2 # [ 67.426814] nix-grpc-daemon[786]: ts=2026-09-20T13:34:11Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:341085270worker2 # [ 67.430099] nix-grpc-daemon[786]: ts=2026-09-20T13:34:11Z level=info event=worker_gone addr=192.168.1.3:500515271worker2 # [ 67.614911] nix-grpc-daemon[786]: ts=2026-09-20T13:34:11Z level=info event=scheduler_connected addr=lb:500515272worker2 # [ 67.616721] nix-grpc-daemon[786]: ts=2026-09-20T13:34:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5273worker1 # [ 67.656987] nix-grpc-daemon[783]: ts=2026-09-20T13:34:10Z level=info event=scheduler_connected addr=lb:500515274worker1 # [ 67.660893] nix-grpc-daemon[783]: ts=2026-09-20T13:34:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5275worker2 # [ 67.817641] nix-grpc-daemon[786]: ts=2026-09-20T13:34:11Z level=info event=scheduler_connected addr=lb:500515276worker2 # [ 67.819934] nix-grpc-daemon[786]: ts=2026-09-20T13:34:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5277worker1 # [ 67.861366] nix-grpc-daemon[783]: ts=2026-09-20T13:34:11Z level=info event=scheduler_connected addr=lb:500515278worker1 # [ 67.863439] nix-grpc-daemon[783]: ts=2026-09-20T13:34:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5279worker2 # [ 68.020745] nix-grpc-daemon[786]: ts=2026-09-20T13:34:12Z level=info event=scheduler_connected addr=lb:500515280worker2 # [ 68.022824] nix-grpc-daemon[786]: ts=2026-09-20T13:34:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5281worker1 # [ 68.064096] nix-grpc-daemon[783]: ts=2026-09-20T13:34:11Z level=info event=scheduler_connected addr=lb:500515282worker1 # [ 68.065827] nix-grpc-daemon[783]: ts=2026-09-20T13:34:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5283worker2 # [ 68.224244] nix-grpc-daemon[786]: ts=2026-09-20T13:34:12Z level=info event=scheduler_connected addr=lb:500515284worker2 # [ 68.226220] nix-grpc-daemon[786]: ts=2026-09-20T13:34:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5285worker1 # [ 68.266788] nix-grpc-daemon[783]: ts=2026-09-20T13:34:11Z level=info event=scheduler_connected addr=lb:500515286worker1 # [ 68.270526] nix-grpc-daemon[783]: ts=2026-09-20T13:34:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5287lb # [ 68.278311] niks3-server[1040]: 2026/09/20 13:34:12 INFO lead: acquired remote=192.168.1.3:356685288worker1 # [ 68.277643] nix-grpc-daemon[783]: ts=2026-09-20T13:34:11Z level=info event=scheduler_take_over5289worker2 # [ 68.414957] nix-grpc-daemon[786]: ts=2026-09-20T13:34:12Z level=info event=scheduler_restarting5290worker2 # [ 68.432690] nix-grpc-daemon[786]: ts=2026-09-20T13:34:12Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5291worker1 # [ 68.472613] nix-grpc-daemon[783]: ts=2026-09-20T13:34:11Z level=info event=scheduler_connected addr=lb:500515292worker2 # [ 68.440227] nix-daemon[855]: reaped child process 857, status = succeeded5293worker1 # [ 68.475701] nix-grpc-daemon[783]: ts=2026-09-20T13:34:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5294worker1 # [ 68.672910] nix-grpc-daemon[783]: ts=2026-09-20T13:34:11Z level=info event=scheduler_connected addr=lb:500515295worker1 # [ 68.674635] nix-grpc-daemon[783]: ts=2026-09-20T13:34:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5296worker1 # [ 68.875261] nix-grpc-daemon[783]: ts=2026-09-20T13:34:12Z level=info event=scheduler_connected addr=lb:500515297worker1 # [ 68.876961] nix-grpc-daemon[783]: ts=2026-09-20T13:34:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5298worker1 # [ 69.077563] nix-grpc-daemon[783]: ts=2026-09-20T13:34:12Z level=info event=scheduler_connected addr=lb:500515299worker1 # [ 69.079686] nix-grpc-daemon[783]: ts=2026-09-20T13:34:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5300worker1 # [ 69.280523] nix-grpc-daemon[783]: ts=2026-09-20T13:34:12Z level=info event=scheduler_connected addr=lb:500515301worker1 # [ 69.293601] nix-grpc-daemon[783]: ts=2026-09-20T13:34:12Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:446325302worker1 # [ 69.298147] nix-grpc-daemon[783]: ts=2026-09-20T13:34:12Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05303worker2 # [ 69.280952] nix-grpc-daemon[786]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5304worker2 # [ 69.285146] nix-grpc-daemon[786]: E0000 00:00:1789911253.421047 829 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5305worker2 # [ 69.290689] nix-grpc-daemon[786]: E0000 00:00:1789911253.422731 829 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5306worker2 # [ 69.349592] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5307worker2 # [ 69.353734] systemd[1]: Stopped Nix worker-protocol over gRPC.5308worker2 # [ 69.357391] systemd[1]: nix-grpc-daemon.service: Consumed 1.358s CPU time over 52.406s wall clock time, 50M memory peak, 56.4K incoming IP traffic, 144.4K outgoing IP traffic.5309worker2 # [ 69.367750] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5310worker2 # [ 69.371136] systemd[1]: Closed Nix worker-protocol over gRPC.5311worker2 # [ 69.374115] systemd[1]: nix-grpc-daemon.socket: Consumed 291us CPU time over 1min 65ms wall clock time, 256K memory peak, 72.8K incoming IP traffic, 61.4K outgoing IP traffic.5312lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5313lb: must succeed: curl -sf localhost:9901/clusters5314lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5315lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.07 seconds)5316client: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25317worker1 # [ 69.676123] nix-grpc-daemon[783]: ts=2026-09-20T13:34:12Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:338965318client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5319worker1 # [ 69.681562] nix-grpc-daemon[783]: ts=2026-09-20T13:34:12Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:338965320client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.22 seconds)5321worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5322worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.01 seconds)5323worker2 # [ 69.722232] systemd[1]: Listening on Nix worker-protocol over gRPC.5324worker1 # [ 69.823696] systemd[1]: Stopping Nix worker-protocol over gRPC...5325worker1 # [ 70.106539] nix-grpc-daemon[783]: ts=2026-09-20T13:34:13Z level=info event=signal_exit5326worker1 # [ 70.359384] nix-grpc-daemon[783]: ts=2026-09-20T13:34:13Z level=info event=scheduler_yield5327worker1 # [ 70.363614] nix-grpc-daemon[783]: ts=2026-09-20T13:34:13Z level=info event=scheduler_restarting addr=lb:500515328lb # [ 70.365974] niks3-server[1040]: 2026/09/20 13:34:14 INFO lead: released remote=192.168.1.3:356685329worker1 # [ 70.368231] nix-grpc-daemon[783]: ts=2026-09-20T13:34:13Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:446325330worker2 # [ 70.336234] systemd[1]: Starting scheduler-token.service...5331worker1 # [ 70.371772] nix-grpc-daemon[783]: ts=2026-09-20T13:34:13Z level=info event=worker_gone addr=192.168.1.3:500515332worker2 # [ 70.405599] systemd[1]: scheduler-token.service: Deactivated successfully.5333worker2 # [ 70.406568] systemd[1]: Finished scheduler-token.service.5334worker2 # [ 70.407605] systemd[1]: scheduler-token.service: Consumed 17ms CPU time over 68ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5335worker2 # [ 70.412808] systemd[1]: Starting Nix worker-protocol over gRPC...5336worker1 # [ 70.564638] nix-grpc-daemon[783]: ts=2026-09-20T13:34:13Z level=info event=scheduler_connected addr=lb:500515337worker1 # [ 70.568616] nix-grpc-daemon[783]: ts=2026-09-20T13:34:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5338worker2 # [ 70.563471] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:14Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5339worker2 # [ 70.565337] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:14Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5340worker2 # [ 70.568634] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:14Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15341worker2 # [ 70.575059] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:14Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5342worker2 # [ 70.578481] systemd[1]: Started Nix worker-protocol over gRPC.5343lb # [ 70.625271] niks3-server[1040]: 2026/09/20 13:34:14 INFO lead: acquired remote=192.168.1.4:493685344worker2 # [ 70.590130] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:14Z level=info event=scheduler_take_over5345worker2 # [ 70.599087] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:14Z level=info event=scheduler_connected addr=lb:500515346worker2 # [ 70.602189] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5347worker1 # [ 70.770914] nix-grpc-daemon[783]: ts=2026-09-20T13:34:14Z level=info event=scheduler_connected addr=lb:500515348worker1 # [ 70.774212] nix-grpc-daemon[783]: ts=2026-09-20T13:34:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5349worker2 # [ 70.803662] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:14Z level=info event=scheduler_connected addr=lb:500515350worker2 # [ 70.807571] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5351worker1 # [ 70.971892] nix-grpc-daemon[783]: ts=2026-09-20T13:34:14Z level=info event=scheduler_connected addr=lb:500515352worker1 # [ 70.975951] nix-grpc-daemon[783]: ts=2026-09-20T13:34:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5353worker2 # [ 71.008630] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:15Z level=info event=scheduler_connected addr=lb:500515354worker2 # [ 71.012748] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5355worker1 # [ 71.176728] nix-grpc-daemon[783]: ts=2026-09-20T13:34:14Z level=info event=scheduler_connected addr=lb:500515356worker1 # [ 71.180799] nix-grpc-daemon[783]: ts=2026-09-20T13:34:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5357worker2 # [ 71.213844] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:15Z level=info event=scheduler_connected addr=lb:500515358worker2 # [ 71.217751] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5359worker1 # [ 71.362707] nix-grpc-daemon[783]: ts=2026-09-20T13:34:14Z level=info event=scheduler_restarting5360worker1 # [ 71.394761] nix-grpc-daemon[783]: ts=2026-09-20T13:34:14Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5361worker1 # [ 71.405260] nix-daemon[841]: reaped child process 843, status = succeeded5362worker2 # [ 71.418643] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:15Z level=info event=scheduler_connected addr=lb:500515363worker2 # [ 71.422636] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5364worker2 # [ 71.623282] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:15Z level=info event=scheduler_connected addr=lb:500515365worker2 # [ 71.632989] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:15Z level=info event=oidc_jwks provider=mock keys=15366worker2 # [ 71.636052] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:15Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:587445367worker2 # [ 71.640224] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:15Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05368worker1 # [ 72.221636] nix-grpc-daemon[783]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5369worker1 # [ 72.225247] nix-grpc-daemon[783]: E0000 00:00:1789911255.463133 826 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5370worker1 # [ 72.230097] nix-grpc-daemon[783]: E0000 00:00:1789911255.463845 826 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5371worker1 # [ 73.351633] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5372worker1 # [ 73.357467] systemd[1]: Stopped Nix worker-protocol over gRPC.5373worker1 # [ 73.359547] systemd[1]: nix-grpc-daemon.service: Consumed 2.626s CPU time over 56.462s wall clock time, 58.7M memory peak, 91.2K incoming IP traffic, 168.7K outgoing IP traffic.5374worker1 # [ 73.371364] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5375worker1 # [ 73.374903] systemd[1]: Closed Nix worker-protocol over gRPC.5376worker1 # [ 73.377619] systemd[1]: nix-grpc-daemon.socket: Consumed 301us CPU time over 1min 4.192s wall clock time, 256K memory peak, 80.1K incoming IP traffic, 66K outgoing IP traffic.5377lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5378lb: must succeed: curl -sf localhost:9901/clusters5379lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5380lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.07 seconds)5381client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25382worker2 # [ 73.654596] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:344725383worker2 # [ 73.666611] nix-daemon[855]: accepted connection from pid 1086, user nix-grpc-daemon (trusted)5384worker2 # [ 73.682736] nix-grpc-daemon[1086]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 45 ms (attempt 1/5)5385worker2 # [ 73.733876] nix-grpc-daemon[1086]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 162 ms (attempt 2/5)5386worker2 # [ 73.901829] nix-grpc-daemon[1086]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 375 ms (attempt 3/5)5387worker2 # [ 74.282910] nix-grpc-daemon[1086]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 456 ms (attempt 4/5)5388worker2 # [ 74.739582] nix-grpc-daemon[1086]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5389client # copying 2 paths...5390worker2 # [ 74.759130] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:344725391client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5392client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5393worker2 # [ 74.779303] nix-daemon[855]: accepted connection from pid 1086, user nix-grpc-daemon (trusted)5394worker2 # [ 74.799881] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:18Z level=info event=niks3_push_started pid=11425395worker2 # [ 75.001206] nix-grpc-daemon[1142]: time=2026-09-20T13:34:19.142Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""5396worker2 # [ 75.176091] nix-daemon[855]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)5397worker2 # [ 75.187188] nix-daemon[855]: reaped child process 1158, status = succeeded5398lb # [ 75.243826] niks3-server[1040]: 2026/09/20 13:34:19 INFO Received uploads request method=POST path=/api/pending_closures5399lb # [ 75.249367] niks3-server[1040]: 2026/09/20 13:34:19 INFO Received uploads request method=POST path=/api/pending_closures5400worker2 # [ 75.214743] nix-grpc-daemon[1142]: time=2026-09-20T13:34:19.356Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5401worker2 # [ 75.216267] nix-grpc-daemon[1142]: time=2026-09-20T13:34:19.358Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5402worker2 # [ 75.219056] nix-grpc-daemon[1142]: time=2026-09-20T13:34:19.360Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5403lb # [ 75.276701] niks3-server[1040]: 2026/09/20 13:34:19 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5404lb # [ 75.282096] niks3-server[1040]: 2026/09/20 13:34:19 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5405lb # [ 75.286619] niks3-server[1040]: 2026/09/20 13:34:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5406lb # [ 75.291099] niks3-server[1040]: 2026/09/20 13:34:19 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5407lb # [ 75.299256] niks3-server[1040]: 2026/09/20 13:34:19 INFO Signed narinfos id=26 count=25408lb # [ 75.300639] niks3-server[1040]: 2026/09/20 13:34:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5409lb # [ 75.304247] niks3-server[1040]: 2026/09/20 13:34:19 INFO Signed narinfos id=25 count=15410worker2 # [ 75.266783] nix-grpc-daemon[1142]: time=2026-09-20T13:34:19.407Z level=INFO msg="Uploading 3 narinfos"5411lb # [ 75.315068] niks3-server[1040]: 2026/09/20 13:34:19 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5412lb # [ 75.324509] niks3-server[1040]: 2026/09/20 13:34:19 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5413lb # [ 75.330475] niks3-server[1040]: 2026/09/20 13:34:19 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5414lb # [ 75.333668] niks3-server[1040]: 2026/09/20 13:34:19 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5415lb # [ 75.337028] niks3-server[1040]: 2026/09/20 13:34:19 INFO Completed upload id=255416lb # [ 75.339180] niks3-server[1040]: 2026/09/20 13:34:19 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5417lb # [ 75.341938] niks3-server[1040]: 2026/09/20 13:34:19 INFO Completed upload id=265418lb # [ 75.344127] niks3-server[1040]: 2026/09/20 13:34:19 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5419worker2 # [ 75.304963] nix-grpc-daemon[1142]: time=2026-09-20T13:34:19.446Z level=INFO msg="Upload complete. (300ms)"5420worker2 # [ 75.309927] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34472 duration_s=0 paths=2 nar_bytes_in=3125421worker2 # [ 75.320440] nix-daemon[855]: reaped child process 1140, status = succeeded5422client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.92 seconds)5423worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5424worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.03 seconds)5425worker1 # [ 75.476545] systemd[1]: Listening on Nix worker-protocol over gRPC.5426worker1 # [ 75.484788] systemd[1]: Starting Nix worker-protocol over gRPC...5427worker1 # [ 75.650358] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:18Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5428worker1 # [ 75.652832] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:18Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5429worker1 # [ 75.656927] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:18Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15430worker1 # [ 75.663738] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:18Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5431worker1 # [ 75.665361] systemd[1]: Started Nix worker-protocol over gRPC.5432lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5433lb: must succeed: curl -sf localhost:9901/clusters5434worker1 # [ 75.678202] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:18Z level=info event=scheduler_connected addr=lb:500515435worker2 # [ 75.643169] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:19Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:587445436worker2 # [ 75.646876] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:19Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05437lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5438lb: must succeed: curl -sf localhost:9901/clusters5439lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5440lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.11 seconds)5441(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.79 seconds)5442subtest: scheduler down: the other node takes the lock and keeps it5443lb: must succeed: curl -sf localhost:9901/clusters5444lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5445lb: must succeed: curl -sf localhost:9901/clusters5446lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5447lb: must succeed: curl -sf localhost:9901/clusters5448lb # [ 76.899616] envoy[911]: conn peer=[2001:db8:1::4]:55498 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=53781 tx=2574 ms=512545449lb # [ 76.901224] envoy[911]: conn peer=[2001:db8:1::1]:49926 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=519 ms=175450lb # [ 76.902687] envoy[911]: conn peer=[2001:db8:1::3]:46036 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=15306 tx=3102 ms=533655451lb # [ 76.904396] envoy[911]: conn peer=[2001:db8:1::1]:41908 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1009 tx=282 ms=16705452lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5453lb: must succeed: curl -sf localhost:9901/clusters5454lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5455worker2 # [ 76.967947] systemd[1]: Stopping Nix worker-protocol over gRPC...5456worker2 # [ 77.580409] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:21Z level=info event=signal_exit5457worker2 # [ 77.792159] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:21Z level=info event=scheduler_yield5458worker1 # [ 77.833900] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:21Z level=info event=scheduler_restarting addr=lb:500515459worker2 # [ 77.799387] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:21Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:587445460worker2 # [ 77.800747] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:21Z level=info event=worker_gone addr=192.168.1.3:500515461lb # [ 77.834425] niks3-server[1040]: 2026/09/20 13:34:21 INFO lead: released remote=192.168.1.4:493685462worker2 # [ 77.802514] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:21Z level=info event=scheduler_restarting addr=lb:500515463worker2 # [ 77.804526] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:21Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:587445464worker2 # [ 77.805910] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:21Z level=info event=worker_gone addr=192.168.1.4:500515465worker1 # [ 78.034277] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:21Z level=info event=scheduler_connected addr=lb:500515466worker1 # [ 78.038209] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5467worker2 # [ 78.003475] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:22Z level=info event=scheduler_connected addr=lb:500515468worker2 # [ 78.005201] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5469worker1 # [ 78.239245] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:21Z level=info event=scheduler_connected addr=lb:500515470worker2 # [ 78.205901] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:22Z level=info event=scheduler_connected addr=lb:500515471worker2 # [ 78.207584] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5472worker1 # [ 78.242997] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5473worker2 # [ 78.408263] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:22Z level=info event=scheduler_connected addr=lb:500515474worker1 # [ 78.443812] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:21Z level=info event=scheduler_connected addr=lb:500515475worker2 # [ 78.409959] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5476worker1 # [ 78.447804] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5477worker1 # [ 78.648817] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:21Z level=info event=scheduler_connected addr=lb:500515478worker2 # [ 78.612069] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:22Z level=info event=scheduler_connected addr=lb:500515479worker1 # [ 78.652663] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5480worker2 # [ 78.615901] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5481worker2 # [ 78.793303] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:22Z level=info event=scheduler_restarting5482worker1 # [ 78.854752] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:22Z level=info event=scheduler_connected addr=lb:500515483worker1 # [ 78.857930] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5484worker2 # [ 78.815118] nix-daemon[855]: reaped child process 1135, status = succeeded5485worker2 # [ 78.827123] nix-grpc-daemon[1086]: ts=2026-09-20T13:34:22Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5486worker1 # [ 79.055627] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:22Z level=info event=scheduler_connected addr=lb:500515487worker1 # [ 79.059550] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5488worker1 # [ 79.260406] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:22Z level=info event=scheduler_connected addr=lb:500515489worker1 # [ 79.264229] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5490worker1 # [ 79.464942] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:22Z level=info event=scheduler_connected addr=lb:500515491worker1 # [ 79.468900] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5492worker2 # [ 79.624513] nix-grpc-daemon[1086]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5493worker2 # [ 79.627967] nix-grpc-daemon[1086]: E0000 00:00:1789911263.765470 1129 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5494worker1 # [ 79.669598] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:22Z level=info event=scheduler_connected addr=lb:500515495worker2 # [ 79.632985] nix-grpc-daemon[1086]: E0000 00:00:1789911263.765635 1129 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5496worker1 # [ 79.673871] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5497worker1 # [ 79.875792] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:23Z level=info event=scheduler_connected addr=lb:500515498worker1 # [ 79.879598] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5499worker1 # [ 80.080390] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:23Z level=info event=scheduler_connected addr=lb:500515500worker1 # [ 80.084241] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5501worker1 # [ 80.284924] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:23Z level=info event=scheduler_connected addr=lb:500515502worker1 # [ 80.288972] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5503worker1 # [ 80.489682] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:23Z level=info event=scheduler_connected addr=lb:500515504worker1 # [ 80.493471] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5505worker2 # [ 80.614682] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5506worker2 # [ 80.618569] systemd[1]: Stopped Nix worker-protocol over gRPC.5507worker2 # [ 80.622829] systemd[1]: nix-grpc-daemon.service: Consumed 329ms CPU time over 10.200s wall clock time, 30.8M memory peak, 20.1K incoming IP traffic, 31.5K outgoing IP traffic.5508worker2 # [ 80.634204] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5509worker1 # [ 80.674113] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:23Z level=info event=scheduler_take_over5510lb # [ 80.675450] niks3-server[1040]: 2026/09/20 13:34:24 INFO lead: acquired remote=192.168.1.3:409345511worker2 # [ 80.638962] systemd[1]: Closed Nix worker-protocol over gRPC.5512worker2 # [ 80.642689] systemd[1]: nix-grpc-daemon.socket: Consumed 293us CPU time over 10.919s wall clock time, 256K memory peak, 20.7K incoming IP traffic, 15.3K outgoing IP traffic.5513worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5514worker1 # [ 80.694195] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:23Z level=info event=scheduler_connected addr=lb:500515515worker1 # [ 80.698595] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5516worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.07 seconds)5517worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5518worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5519worker1 # [ 80.900258] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:24Z level=info event=scheduler_connected addr=lb:500515520worker1 # [ 80.903963] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5521worker1 # [ 81.105165] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:24Z level=info event=scheduler_connected addr=lb:500515522worker1 # [ 81.109067] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5523worker1 # [ 81.309432] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:24Z level=info event=scheduler_connected addr=lb:500515524worker1 # [ 81.311273] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5525worker1 # [ 81.512254] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:24Z level=info event=scheduler_connected addr=lb:500515526worker1 # [ 81.519186] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:24Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:530825527worker1 # [ 81.522721] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:24Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05528worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5529worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5530client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&25531worker1 # [ 82.099342] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:530845532worker1 # [ 82.110820] nix-daemon[841]: accepted connection from pid 1209, user nix-grpc-daemon (trusted)5533client # don't know how to build these paths:5534client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5535worker1 # [ 82.125396] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:530845536worker1 # [ 82.137642] nix-grpc-daemon[1209]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 91 ms (attempt 1/5)5537worker1 # [ 82.234471] nix-grpc-daemon[1209]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 33 ms (attempt 2/5)5538worker1 # [ 82.273308] nix-grpc-daemon[1209]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 301 ms (attempt 3/5)5539worker1 # [ 82.574693] nix-grpc-daemon[1209]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 451 ms (attempt 4/5)5540worker1 # [ 83.031734] nix-grpc-daemon[1209]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5541worker1 # [ 83.057644] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:26Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05542worker1 # [ 83.063490] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:26Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=15543worker1 # [ 83.107421] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:530845544worker1 # [ 83.119116] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:530845545client # copying 1 paths to 'lb:50051'...5546client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5547worker1 # [ 83.130155] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:530845548worker1 # [ 83.141482] nix-daemon[841]: accepted connection from pid 1209, user nix-grpc-daemon (trusted)5549worker1 # [ 83.158611] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:26Z level=info event=niks3_push_started pid=12795550worker1 # [ 83.338288] nix-grpc-daemon[1279]: time=2026-09-20T13:34:26.580Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""5551worker1 # [ 83.490957] nix-daemon[841]: accepted connection from pid 1292, user nix-grpc-daemon (trusted)5552worker1 # [ 83.501572] nix-daemon[841]: reaped child process 1295, status = succeeded5553worker1 # [ 83.641827] nix-daemon[841]: accepted connection from pid 1297, user nix-grpc-daemon (trusted)5554worker1 # [ 83.653533] nix-daemon[841]: reaped child process 1300, status = succeeded5555lb # [ 83.668814] niks3-server[1040]: 2026/09/20 13:34:27 INFO Received uploads request method=POST path=/api/pending_closures5556worker1 # [ 83.679697] nix-grpc-daemon[1279]: time=2026-09-20T13:34:26.921Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5557worker1 # [ 83.681092] nix-grpc-daemon[1279]: time=2026-09-20T13:34:26.921Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5558lb # [ 83.712488] niks3-server[1040]: 2026/09/20 13:34:27 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5559lb # [ 83.721375] niks3-server[1040]: 2026/09/20 13:34:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5560lb # [ 83.729349] niks3-server[1040]: 2026/09/20 13:34:27 INFO Signed narinfos id=27 count=15561worker1 # [ 83.731337] nix-grpc-daemon[1279]: time=2026-09-20T13:34:26.974Z level=INFO msg="Uploading 1 narinfos"5562lb # [ 83.733876] niks3-server[1040]: 2026/09/20 13:34:27 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5563lb # [ 83.745228] niks3-server[1040]: 2026/09/20 13:34:27 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5564lb # [ 83.748945] niks3-server[1040]: 2026/09/20 13:34:27 INFO Completed upload id=275565worker1 # [ 83.748033] nix-grpc-daemon[1279]: time=2026-09-20T13:34:26.991Z level=INFO msg="Upload complete. (405ms)"5566lb # [ 83.751029] niks3-server[1040]: 2026/09/20 13:34:27 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5567worker1 # [ 83.751257] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53084 duration_s=0 paths=1 nar_bytes_in=5205568worker1 # [ 83.756230] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:530845569worker1 # [ 83.765767] nix-daemon[841]: accepted connection from pid 1209, user nix-grpc-daemon (trusted)5570worker1 # [ 83.770858] nix-daemon[841]: reaped child process 1277, status = succeeded5571client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5572worker1 # [ 83.776586] nix-daemon[841]: accepted connection from pid 1209, user nix-grpc-daemon (trusted)5573client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5574client # farm-a> LOG-farm-a-on-standby5575worker1 # [ 85.189531] nix-daemon[841]: reaped child process 1304, status = succeeded5576worker1 # [ 85.301923] nix-daemon[841]: accepted connection from pid 1319, user nix-grpc-daemon (trusted)5577worker1 # [ 85.315526] nix-daemon[841]: reaped child process 1322, status = succeeded5578lb # [ 85.329725] niks3-server[1040]: 2026/09/20 13:34:29 INFO Received uploads request method=POST path=/api/pending_closures5579worker1 # [ 85.337773] nix-grpc-daemon[1279]: time=2026-09-20T13:34:28.580Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5580worker1 # [ 85.341070] nix-grpc-daemon[1279]: time=2026-09-20T13:34:28.582Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5581lb # [ 85.373927] niks3-server[1040]: 2026/09/20 13:34:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5582lb # [ 85.383109] niks3-server[1040]: 2026/09/20 13:34:29 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5583worker1 # [ 85.387949] nix-grpc-daemon[1279]: time=2026-09-20T13:34:28.630Z level=INFO msg="Uploading 1 narinfos"5584lb # [ 85.390215] niks3-server[1040]: 2026/09/20 13:34:29 INFO Signed narinfos id=28 count=15585lb # [ 85.393864] niks3-server[1040]: 2026/09/20 13:34:29 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5586lb # [ 85.408444] niks3-server[1040]: 2026/09/20 13:34:29 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5587lb # [ 85.413569] niks3-server[1040]: 2026/09/20 13:34:29 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5588lb # [ 85.416718] niks3-server[1040]: 2026/09/20 13:34:29 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5589lb # [ 85.420633] niks3-server[1040]: 2026/09/20 13:34:29 INFO Completed upload id=285590worker1 # [ 85.419885] nix-grpc-daemon[1279]: time=2026-09-20T13:34:28.662Z level=INFO msg="Upload complete. (237ms)"5591worker1 # [ 85.433993] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53084 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15592worker1 # [ 85.445886] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:28Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05593worker1 # [ 85.448254] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:28Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=15594worker1 # [ 85.451580] nix-daemon[841]: reaped child process 1302, status = succeeded5595worker1 # [ 85.489353] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:530845596worker1 # [ 85.497822] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:530845597worker1 # [ 85.507746] nix-grpc-daemon[1279]: time=2026-09-20T13:34:28.750Z level=INFO msg="All 1 paths already cached"5598client # copying 1 paths to 'lb:50051'...5599client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5600worker1 # [ 85.515724] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:530845601worker1 # [ 85.528174] nix-daemon[841]: accepted connection from pid 1209, user nix-grpc-daemon (trusted)5602worker1 # [ 85.686488] nix-daemon[841]: accepted connection from pid 1328, user nix-grpc-daemon (trusted)5603worker1 # [ 85.699223] nix-daemon[841]: reaped child process 1331, status = succeeded5604worker1 # [ 85.825155] nix-daemon[841]: accepted connection from pid 1333, user nix-grpc-daemon (trusted)5605worker1 # [ 85.836616] nix-daemon[841]: reaped child process 1336, status = succeeded5606lb # [ 85.852888] niks3-server[1040]: 2026/09/20 13:34:29 INFO Received uploads request method=POST path=/api/pending_closures5607worker1 # [ 85.864145] nix-grpc-daemon[1279]: time=2026-09-20T13:34:29.106Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5608worker1 # [ 85.865514] nix-grpc-daemon[1279]: time=2026-09-20T13:34:29.106Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5609lb # [ 85.884373] niks3-server[1040]: 2026/09/20 13:34:29 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5610lb # [ 85.894932] niks3-server[1040]: 2026/09/20 13:34:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5611lb # [ 85.901057] niks3-server[1040]: 2026/09/20 13:34:29 INFO Signed narinfos id=29 count=15612worker1 # [ 85.903355] nix-grpc-daemon[1279]: time=2026-09-20T13:34:29.146Z level=INFO msg="Uploading 1 narinfos"5613lb # [ 85.906139] niks3-server[1040]: 2026/09/20 13:34:29 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5614lb # [ 85.917292] niks3-server[1040]: 2026/09/20 13:34:29 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5615worker1 # [ 85.921929] nix-grpc-daemon[1279]: time=2026-09-20T13:34:29.164Z level=INFO msg="Upload complete. (382ms)"5616lb # [ 85.923181] niks3-server[1040]: 2026/09/20 13:34:29 INFO Completed upload id=295617worker1 # [ 85.924744] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53084 duration_s=0 paths=1 nar_bytes_in=6405618lb # [ 85.926830] niks3-server[1040]: 2026/09/20 13:34:29 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5619worker1 # [ 85.930143] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:530845620worker1 # [ 85.939494] nix-daemon[841]: accepted connection from pid 1209, user nix-grpc-daemon (trusted)5621worker1 # [ 85.943702] nix-daemon[841]: reaped child process 1326, status = succeeded5622worker1 # [ 85.948824] nix-daemon[841]: accepted connection from pid 1209, user nix-grpc-daemon (trusted)5623client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5624client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5625client # farm-b> LOG-farm-b-on-standby5626worker1 # [ 86.774473] nix-daemon[841]: reaped child process 1340, status = succeeded5627lb # [ 86.899884] envoy[911]: conn peer=[2001:db8:1::4]:41702 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=8603 tx=535 ms=82295628worker1 # [ 86.927734] nix-daemon[841]: accepted connection from pid 1355, user nix-grpc-daemon (trusted)5629worker1 # [ 86.938615] nix-daemon[841]: reaped child process 1358, status = succeeded5630lb # [ 86.955381] niks3-server[1040]: 2026/09/20 13:34:30 INFO Received uploads request method=POST path=/api/pending_closures5631worker1 # [ 86.963151] nix-grpc-daemon[1279]: time=2026-09-20T13:34:30.205Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5632worker1 # [ 86.964806] nix-grpc-daemon[1279]: time=2026-09-20T13:34:30.207Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"5633lb # [ 86.987201] niks3-server[1040]: 2026/09/20 13:34:30 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5634lb # [ 87.007124] niks3-server[1040]: 2026/09/20 13:34:31 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst5635lb # [ 87.010840] niks3-server[1040]: 2026/09/20 13:34:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5636lb # [ 87.016301] niks3-server[1040]: 2026/09/20 13:34:31 INFO Signed narinfos id=30 count=15637worker1 # [ 87.018363] nix-grpc-daemon[1279]: time=2026-09-20T13:34:30.261Z level=INFO msg="Uploading 1 narinfos"5638lb # [ 87.022339] niks3-server[1040]: 2026/09/20 13:34:31 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls5639lb # [ 87.030263] niks3-server[1040]: 2026/09/20 13:34:31 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5640lb # [ 87.034150] niks3-server[1040]: 2026/09/20 13:34:31 INFO Completed upload id=305641lb # [ 87.035692] niks3-server[1040]: 2026/09/20 13:34:31 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo5642worker1 # [ 87.033298] nix-grpc-daemon[1279]: time=2026-09-20T13:34:30.276Z level=INFO msg="Upload complete. (277ms)"5643worker1 # [ 87.041690] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53084 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=15644worker1 # [ 87.050895] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:30Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=05645worker1 # [ 87.055737] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:30Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=15646worker1 # [ 87.063848] nix-daemon[841]: reaped child process 1338, status = succeeded5647worker1 # [ 87.095228] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:530845648worker1 # [ 87.105985] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:530845649worker1 # [ 87.118996] nix-grpc-daemon[1279]: time=2026-09-20T13:34:30.362Z level=INFO msg="All 2 paths already cached"5650client # copying 1 paths to 'lb:50051'...5651client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...5652worker1 # [ 87.128344] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:530845653worker1 # [ 87.136539] nix-daemon[841]: accepted connection from pid 1209, user nix-grpc-daemon (trusted)5654worker1 # [ 87.295267] nix-daemon[841]: accepted connection from pid 1364, user nix-grpc-daemon (trusted)5655worker1 # [ 87.307673] nix-daemon[841]: reaped child process 1367, status = succeeded5656worker1 # [ 87.466138] nix-daemon[841]: accepted connection from pid 1369, user nix-grpc-daemon (trusted)5657worker1 # [ 87.478175] nix-daemon[841]: reaped child process 1372, status = succeeded5658lb # [ 87.492849] niks3-server[1040]: 2026/09/20 13:34:31 INFO Received uploads request method=POST path=/api/pending_closures5659worker1 # [ 87.500176] nix-grpc-daemon[1279]: time=2026-09-20T13:34:30.743Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5660worker1 # [ 87.501685] nix-grpc-daemon[1279]: time=2026-09-20T13:34:30.744Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"5661lb # [ 87.524298] niks3-server[1040]: 2026/09/20 13:34:31 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst5662lb # [ 87.532941] niks3-server[1040]: 2026/09/20 13:34:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5663lb # [ 87.538092] niks3-server[1040]: 2026/09/20 13:34:31 INFO Signed narinfos id=31 count=15664worker1 # [ 87.540168] nix-grpc-daemon[1279]: time=2026-09-20T13:34:30.783Z level=INFO msg="Uploading 1 narinfos"5665lb # [ 87.547568] niks3-server[1040]: 2026/09/20 13:34:31 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls5666lb # [ 87.556500] niks3-server[1040]: 2026/09/20 13:34:31 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5667worker1 # [ 87.567439] nix-grpc-daemon[1279]: time=2026-09-20T13:34:30.810Z level=INFO msg="Upload complete. (419ms)"5668lb # [ 87.568378] niks3-server[1040]: 2026/09/20 13:34:31 INFO Completed upload id=315669worker1 # [ 87.569793] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53084 duration_s=0 paths=1 nar_bytes_in=7685670lb # [ 87.572592] niks3-server[1040]: 2026/09/20 13:34:31 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo5671worker1 # [ 87.576136] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:530845672worker1 # [ 87.586117] nix-daemon[841]: accepted connection from pid 1209, user nix-grpc-daemon (trusted)5673worker1 # [ 87.594660] nix-daemon[841]: reaped child process 1362, status = succeeded5674worker1 # [ 87.598900] nix-daemon[841]: accepted connection from pid 1209, user nix-grpc-daemon (trusted)5675client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...5676client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5677client # farm-top> LOG-farm-top-on-standby5678worker1 # [ 88.624699] nix-daemon[841]: reaped child process 1376, status = succeeded5679worker1 # [ 88.765746] nix-daemon[841]: accepted connection from pid 1391, user nix-grpc-daemon (trusted)5680worker1 # [ 88.776709] nix-daemon[841]: reaped child process 1394, status = succeeded5681lb # [ 88.793715] niks3-server[1040]: 2026/09/20 13:34:32 INFO Received uploads request method=POST path=/api/pending_closures5682worker1 # [ 88.802387] nix-grpc-daemon[1279]: time=2026-09-20T13:34:32.045Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5683worker1 # [ 88.803898] nix-grpc-daemon[1279]: time=2026-09-20T13:34:32.046Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"5684lb # [ 88.830654] niks3-server[1040]: 2026/09/20 13:34:32 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5685lb # [ 88.839864] niks3-server[1040]: 2026/09/20 13:34:32 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst5686lb # [ 88.850294] niks3-server[1040]: 2026/09/20 13:34:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5687worker1 # [ 88.854552] nix-grpc-daemon[1279]: time=2026-09-20T13:34:32.097Z level=INFO msg="Uploading 1 narinfos"5688lb # [ 88.855613] niks3-server[1040]: 2026/09/20 13:34:32 INFO Signed narinfos id=32 count=15689lb # [ 88.860054] niks3-server[1040]: 2026/09/20 13:34:32 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls5690lb # [ 88.870378] niks3-server[1040]: 2026/09/20 13:34:32 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5691lb # [ 88.875182] niks3-server[1040]: 2026/09/20 13:34:32 INFO Completed upload id=325692worker1 # [ 88.874507] nix-grpc-daemon[1279]: time=2026-09-20T13:34:32.117Z level=INFO msg="Upload complete. (268ms)"5693lb # [ 88.877583] niks3-server[1040]: 2026/09/20 13:34:32 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo5694worker1 # [ 88.882269] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53084 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=15695worker1 # [ 88.894266] nix-daemon[841]: reaped child process 1374, status = succeeded5696client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&2, in 7.06 seconds)5697client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath5698client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath, in 0.20 seconds)5699worker2 # [ 89.146734] systemd[1]: Listening on Nix worker-protocol over gRPC.5700worker2 # [ 89.153066] systemd[1]: Starting scheduler-token.service...5701worker2 # [ 89.221071] systemd[1]: scheduler-token.service: Deactivated successfully.5702worker2 # [ 89.222249] systemd[1]: Finished scheduler-token.service.5703worker2 # [ 89.223713] systemd[1]: scheduler-token.service: Consumed 19ms CPU time over 68ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5704worker2 # [ 89.228901] systemd[1]: Starting Nix worker-protocol over gRPC...5705worker2 # [ 89.374836] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:33Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5706worker2 # [ 89.377155] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:33Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5707worker2 # [ 89.383810] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:33Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15708worker2 # [ 89.391055] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:33Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5709worker2 # [ 89.394698] systemd[1]: Started Nix worker-protocol over gRPC.5710lb: must succeed: curl -sf localhost:9901/clusters5711worker2 # [ 89.409332] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:33Z level=info event=scheduler_connected addr=lb:500515712worker1 # [ 89.455171] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:32Z level=info event=oidc_jwks provider=mock keys=15713worker1 # [ 89.458990] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:32Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:530825714worker1 # [ 89.462902] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:32Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05715lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5716worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5717worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5718lb: must succeed: curl -sf localhost:9901/clusters5719lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5720lb: must succeed: curl -sf localhost:9901/clusters5721lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5722worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5723worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5724(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.97 seconds)5725subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5726client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs5727client # [ 89.780521] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs.5728client # Running as unit: rs.service; invocation ID: cec36657fb6e48f49869f560b11006175729client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs, in 0.07 seconds)5730worker2 # [ 89.990159] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:34Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:524165731worker2 # [ 90.002497] nix-daemon[855]: accepted connection from pid 1182, user nix-grpc-daemon (trusted)5732client # [ 90.026379] nix[1171]: don't know how to build these paths:5733client # [ 90.027303] nix[1171]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5734worker1 # [ 90.064632] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:33Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=05735worker1 # [ 90.068549] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:33Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=15736worker1 # [ 90.107385] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:530845737worker1 # [ 90.119771] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:530845738client # [ 90.104209] nix[1171]: copying 1 paths to 'lb:50051'...5739worker1 # [ 90.131699] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:530845740client # [ 90.107951] nix[1171]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...5741worker1 # [ 90.142338] nix-daemon[841]: accepted connection from pid 1209, user nix-grpc-daemon (trusted)5742worker1 # [ 90.298784] nix-daemon[841]: accepted connection from pid 1407, user nix-grpc-daemon (trusted)5743worker1 # [ 90.311293] nix-daemon[841]: reaped child process 1410, status = succeeded5744worker1 # [ 90.454457] nix-daemon[841]: accepted connection from pid 1412, user nix-grpc-daemon (trusted)5745worker1 # [ 90.465116] nix-daemon[841]: reaped child process 1415, status = succeeded5746lb # [ 90.482611] niks3-server[1040]: 2026/09/20 13:34:34 INFO Received uploads request method=POST path=/api/pending_closures5747worker1 # [ 90.491337] nix-grpc-daemon[1279]: time=2026-09-20T13:34:33.734Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5748worker1 # [ 90.492879] nix-grpc-daemon[1279]: time=2026-09-20T13:34:33.735Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"5749lb # [ 90.507486] niks3-server[1040]: 2026/09/20 13:34:34 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst5750lb # [ 90.515636] niks3-server[1040]: 2026/09/20 13:34:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5751lb # [ 90.517930] niks3-server[1040]: 2026/09/20 13:34:34 INFO Signed narinfos id=33 count=15752worker1 # [ 90.516917] nix-grpc-daemon[1279]: time=2026-09-20T13:34:33.759Z level=INFO msg="Uploading 1 narinfos"5753lb # [ 90.520965] niks3-server[1040]: 2026/09/20 13:34:34 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls5754lb # [ 90.526260] niks3-server[1040]: 2026/09/20 13:34:34 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5755lb # [ 90.529960] niks3-server[1040]: 2026/09/20 13:34:34 INFO Completed upload id=335756worker1 # [ 90.528998] nix-grpc-daemon[1279]: time=2026-09-20T13:34:33.772Z level=INFO msg="Upload complete. (374ms)"5757worker1 # [ 90.530931] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53084 duration_s=0 paths=1 nar_bytes_in=4165758lb # [ 90.532479] niks3-server[1040]: 2026/09/20 13:34:34 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo5759worker1 # [ 90.533814] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:530845760worker1 # [ 90.540263] nix-daemon[841]: accepted connection from pid 1209, user nix-grpc-daemon (trusted)5761worker1 # [ 90.543075] nix-daemon[841]: reaped child process 1405, status = succeeded5762worker1 # [ 90.547794] nix-daemon[841]: accepted connection from pid 1209, user nix-grpc-daemon (trusted)5763client # [ 90.521444] nix[1171]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...5764lb: must succeed: curl -sf localhost:9901/clusters5765lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5766lb: must succeed: curl -sf localhost:9901/clusters5767lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5768lb: must succeed: curl -sf localhost:9901/clusters5769lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5770lb: must succeed: curl -sf localhost:9901/clusters5771lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5772worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5773worker2 # [ 92.580445] systemd-journald[286]: Received client request to rotate journal, rotating.5774worker2 # [ 92.597257] systemd-journald[286]: Vacuuming done, freed 0B of archived journals from /var/log/journal/1f5f40f8ec944b5f836ea4a215e385d5.5775worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.06 seconds)5776worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5777worker1 # [ 92.549940] systemd[1]: Stopping Nix worker-protocol over gRPC...5778worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)5779client: must succeed: sleep 5.05780worker1 # [ 92.678386] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:35Z level=info event=draining5781lb # [ 96.900404] envoy[911]: conn peer=[2001:db8:1::1]:41910 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9534 tx=3852 ms=68165782client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5783worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5784worker1 # [ 97.978652] nix-daemon[841]: reaped child process 1419, status = succeeded5785worker1 # [ 98.145671] nix-daemon[841]: accepted connection from pid 1459, user nix-grpc-daemon (trusted)5786worker1 # [ 98.157616] nix-daemon[841]: reaped child process 1462, status = succeeded5787lb # [ 98.178322] niks3-server[1040]: 2026/09/20 13:34:42 INFO Received uploads request method=POST path=/api/pending_closures5788worker1 # [ 98.187289] nix-grpc-daemon[1279]: time=2026-09-20T13:34:41.430Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5789worker1 # [ 98.188816] nix-grpc-daemon[1279]: time=2026-09-20T13:34:41.431Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"5790lb # [ 98.220821] niks3-server[1040]: 2026/09/20 13:34:42 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5791lb # [ 98.228074] niks3-server[1040]: 2026/09/20 13:34:42 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5792lb # [ 98.240684] niks3-server[1040]: 2026/09/20 13:34:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5793lb # [ 98.244280] niks3-server[1040]: 2026/09/20 13:34:42 INFO Signed narinfos id=34 count=15794worker1 # [ 98.245882] nix-grpc-daemon[1279]: time=2026-09-20T13:34:41.488Z level=INFO msg="Uploading 1 narinfos"5795lb # [ 98.249415] niks3-server[1040]: 2026/09/20 13:34:42 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls5796lb # [ 98.258326] niks3-server[1040]: 2026/09/20 13:34:42 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5797lb # [ 98.264171] niks3-server[1040]: 2026/09/20 13:34:42 INFO Completed upload id=345798worker1 # [ 98.263292] nix-grpc-daemon[1279]: time=2026-09-20T13:34:41.506Z level=INFO msg="Upload complete. (301ms)"5799lb # [ 98.266055] niks3-server[1040]: 2026/09/20 13:34:42 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo5800worker1 # [ 98.270602] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53084 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=15801worker1 # [ 98.283922] nix-daemon[841]: reaped child process 1417, status = succeeded5802client # [ 98.279659] systemd[1]: rs.service: Deactivated successfully.5803client # [ 98.283857] systemd[1]: rs.service: Consumed 120ms CPU time over 8.501s wall clock time, 37.1M memory peak, 8.6K incoming IP traffic, 9.1K outgoing IP traffic.5804worker1 # [ 98.679475] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:41Z level=info event=signal_exit5805worker1 # [ 98.898199] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:42Z level=info event=scheduler_yield5806lb # [ 98.901180] niks3-server[1040]: 2026/09/20 13:34:42 INFO lead: released remote=192.168.1.3:409345807worker1 # [ 98.906804] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:42Z level=info event=scheduler_restarting addr=lb:500515808worker2 # [ 98.871526] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:43Z level=info event=scheduler_restarting addr=lb:500515809worker1 # [ 98.910764] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:42Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:530825810worker1 # [ 98.914472] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:42Z level=info event=worker_gone addr=192.168.1.3:500515811worker1 # [ 98.917522] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:42Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:530825812worker1 # [ 98.920769] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:42Z level=info event=worker_gone addr=192.168.1.4:500515813worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 1.10 seconds)5814client: waiting for success: ! systemctl is-active rs5815client: (finished: waiting for success: ! systemctl is-active rs, in 0.04 seconds)5816client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5817client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.03 seconds)5818client: must fail: journalctl -u rs | grep -q 'reconnecting'5819client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.02 seconds)5820lb: must succeed: curl -sf localhost:9901/clusters5821worker1 # [ 99.107454] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:42Z level=info event=scheduler_connected addr=lb:500515822worker2 # [ 99.071080] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:43Z level=info event=scheduler_connected addr=lb:500515823worker1 # [ 99.112937] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:42Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5824worker2 # [ 99.075869] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:43Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5825lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5826worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5827worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5828worker2 # [ 99.277378] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:43Z level=info event=scheduler_connected addr=lb:500515829worker1 # [ 99.315807] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:42Z level=info event=scheduler_connected addr=lb:500515830worker2 # [ 99.281507] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:43Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5831worker1 # [ 99.319888] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:42Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5832lb # [ 99.440319] niks3-server[1040]: 2026/09/20 13:34:43 INFO lead: acquired remote=192.168.1.4:398425833worker2 # [ 99.405636] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:43Z level=info event=scheduler_take_over5834worker1 # [ 99.517635] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:42Z level=info event=scheduler_connected addr=lb:500515835worker2 # [ 99.482617] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:43Z level=info event=scheduler_connected addr=lb:500515836worker1 # [ 99.521591] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5837worker2 # [ 99.486751] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5838worker1 # [ 99.722336] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:42Z level=info event=scheduler_connected addr=lb:500515839worker2 # [ 99.687655] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:43Z level=info event=scheduler_connected addr=lb:500515840worker1 # [ 99.726379] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5841worker2 # [ 99.691517] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5842worker1 # [ 99.902939] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:43Z level=info event=scheduler_restarting5843worker1 # [ 99.924441] nix-daemon[841]: reaped child process 1270, status = succeeded5844worker2 # [ 99.894086] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:44Z level=info event=scheduler_connected addr=lb:500515845worker2 # [ 99.897335] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5846worker1 # [ 99.936911] nix-grpc-daemon[1209]: ts=2026-09-20T13:34:43Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5847worker2 # [ 100.095234] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:44Z level=info event=scheduler_connected addr=lb:500515848worker2 # [ 100.099140] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5849lb: must succeed: curl -sf localhost:9901/clusters5850lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5851worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5852worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5853worker2 # [ 100.299918] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:44Z level=info event=scheduler_connected addr=lb:500515854worker2 # [ 100.303794] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5855worker1 # [ 100.393183] nix-grpc-daemon[1209]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5856worker1 # [ 100.396691] nix-grpc-daemon[1209]: E0000 00:00:1789911283.635217 1252 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5857worker1 # [ 100.401445] nix-grpc-daemon[1209]: E0000 00:00:1789911283.635480 1252 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5858worker2 # [ 100.504877] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:44Z level=info event=scheduler_connected addr=lb:500515859worker2 # [ 100.508927] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5860worker2 # [ 100.709906] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:44Z level=info event=scheduler_connected addr=lb:500515861worker2 # [ 100.721133] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:44Z level=info event=oidc_jwks provider=mock keys=15862worker2 # [ 100.725412] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:44Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:361645863worker2 # [ 100.729368] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:44Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05864worker1 # [ 101.205573] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5865worker1 # [ 101.210616] systemd[1]: Stopped Nix worker-protocol over gRPC.5866worker1 # [ 101.212673] systemd[1]: nix-grpc-daemon.service: Consumed 1.276s CPU time over 25.720s wall clock time, 33.4M memory peak, 49.7K incoming IP traffic, 101.8K outgoing IP traffic.5867worker1 # [ 101.226670] systemd[1]: Starting Nix worker-protocol over gRPC...5868lb: must succeed: curl -sf localhost:9901/clusters5869lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5870worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5871worker1 # [ 101.391275] nix-grpc-daemon[1472]: ts=2026-09-20T13:34:44Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5872worker1 # [ 101.393192] nix-grpc-daemon[1472]: ts=2026-09-20T13:34:44Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5873worker1 # [ 101.396567] nix-grpc-daemon[1472]: ts=2026-09-20T13:34:44Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15874worker1 # [ 101.402875] nix-grpc-daemon[1472]: ts=2026-09-20T13:34:44Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5875worker1 # [ 101.405571] systemd[1]: Started Nix worker-protocol over gRPC.5876worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5877worker1 # [ 101.418166] nix-grpc-daemon[1472]: ts=2026-09-20T13:34:44Z level=info event=scheduler_connected addr=lb:500515878worker2 # [ 101.382446] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:45Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:361645879worker2 # [ 101.383943] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:45Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05880lb: must succeed: curl -sf localhost:9901/clusters5881lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5882worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5883worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5884(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.77 seconds)5885subtest: scheduler restart mid-build: build finishes, no double build5886client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid5887client # [ 102.553606] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid.5888client # Running as unit: mid.service; invocation ID: ec07d2db695a491dadc240f26610888c5889client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid, in 0.07 seconds)5890worker1 # [ 102.767562] nix-grpc-daemon[1472]: ts=2026-09-20T13:34:46Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:359425891worker1 # [ 102.778847] nix-daemon[841]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)5892client # [ 102.764354] nix[1207]: don't know how to build these paths:5893client # [ 102.765245] nix[1207]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5894worker2 # [ 102.764430] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:46Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05895worker2 # [ 102.768521] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:46Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=15896worker2 # [ 102.805502] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524165897worker2 # [ 102.812054] nix-grpc-daemon[1182]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 80 ms (attempt 1/5)5898worker2 # [ 102.894782] nix-grpc-daemon[1182]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 70 ms (attempt 2/5)5899worker2 # [ 102.967283] nix-grpc-daemon[1182]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 8 ms (attempt 3/5)5900worker2 # [ 102.975358] nix-grpc-daemon[1182]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 640 ms (attempt 4/5)5901worker2 # [ 103.618168] nix-grpc-daemon[1182]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5902worker2 # [ 103.632545] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524165903client # [ 103.650251] nix[1207]: copying 1 paths to 'lb:50051'...5904worker2 # [ 103.643040] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524165905client # [ 103.654145] nix[1207]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...5906worker2 # [ 103.651219] nix-daemon[855]: accepted connection from pid 1182, user nix-grpc-daemon (trusted)5907worker2 # [ 103.666587] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:47Z level=info event=niks3_push_started pid=12975908worker2 # [ 103.863500] nix-grpc-daemon[1297]: time=2026-09-20T13:34:48.004Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""5909worker2 # [ 104.036642] nix-daemon[855]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)5910worker2 # [ 104.047528] nix-daemon[855]: reaped child process 1317, status = succeeded5911worker2 # [ 104.219440] nix-daemon[855]: accepted connection from pid 1319, user nix-grpc-daemon (trusted)5912worker2 # [ 104.231276] nix-daemon[855]: reaped child process 1322, status = succeeded5913lb # [ 104.287828] niks3-server[1040]: 2026/09/20 13:34:48 INFO Received uploads request method=POST path=/api/pending_closures5914worker2 # [ 104.260459] nix-grpc-daemon[1297]: time=2026-09-20T13:34:48.402Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5915worker2 # [ 104.262976] nix-grpc-daemon[1297]: time=2026-09-20T13:34:48.403Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"5916lb # [ 104.331788] niks3-server[1040]: 2026/09/20 13:34:48 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst5917lb # [ 104.336708] niks3-server[1040]: 2026/09/20 13:34:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5918worker2 # [ 104.303546] nix-grpc-daemon[1297]: time=2026-09-20T13:34:48.445Z level=INFO msg="Uploading 1 narinfos"5919lb # [ 104.342375] niks3-server[1040]: 2026/09/20 13:34:48 INFO Signed narinfos id=35 count=15920lb # [ 104.353896] niks3-server[1040]: 2026/09/20 13:34:48 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls5921lb # [ 104.357163] niks3-server[1040]: 2026/09/20 13:34:48 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5922lb # [ 104.360799] niks3-server[1040]: 2026/09/20 13:34:48 INFO Completed upload id=355923worker2 # [ 104.322602] nix-grpc-daemon[1297]: time=2026-09-20T13:34:48.464Z level=INFO msg="Upload complete. (455ms)"5924lb # [ 104.363723] niks3-server[1040]: 2026/09/20 13:34:48 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo5925worker2 # [ 104.326251] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52416 duration_s=0 paths=1 nar_bytes_in=4245926worker2 # [ 104.332270] nix-grpc-daemon[1182]: ts=2026-09-20T13:34:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524165927worker2 # [ 104.341862] nix-daemon[855]: accepted connection from pid 1182, user nix-grpc-daemon (trusted)5928worker2 # [ 104.343549] nix-daemon[855]: reaped child process 1295, status = succeeded5929worker2 # [ 104.348522] nix-daemon[855]: accepted connection from pid 1182, user nix-grpc-daemon (trusted)5930client # [ 104.359927] nix[1207]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5931lb: must succeed: curl -sf localhost:9901/clusters5932lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5933lb: must succeed: curl -sf localhost:9901/clusters5934lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5935worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5936worker2 # [ 106.285555] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1182 (nix-grpc-daemon) on client request.5937worker2 # [ 106.289378] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5938lb # [ 106.334730] niks3-server[1040]: 2026/09/20 13:34:50 INFO lead: released remote=192.168.1.4:398425939worker1 # [ 106.341170] nix-grpc-daemon[1472]: ts=2026-09-20T13:34:49Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5940worker2 # [ 106.308304] systemd[1]: Starting scheduler-token.service...5941client # [ 106.316503] nix[1207]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5942client # [ 106.321349] nix[1207]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5943worker2 # [ 106.321034] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5944worker2 # [ 106.325801] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5945worker2 # [ 106.327263] systemd[1]: nix-grpc-daemon.service: Consumed 436ms CPU time over 17.090s wall clock time, 31.1M memory peak, 17.1K incoming IP traffic, 31.5K outgoing IP traffic.5946worker2 # [ 106.333507] nix-daemon[855]: reaped child process 1324, status = succeeded5947worker2 # [ 106.334691] nix-daemon[855]: reaped child process 1238, status = succeeded5948worker2 # [ 106.343280] nix-daemon[1326]: unexpected Nix daemon error: error: interrupted by the user5949worker2 # [ 106.348571] nix-daemon[855]: reaped child process 1326, status = failed with exit code 15950worker1 # [ 106.412165] nix-grpc-daemon[1472]: ts=2026-09-20T13:34:49Z level=info event=scheduler_take_over5951lb # [ 106.412969] niks3-server[1040]: 2026/09/20 13:34:50 INFO lead: acquired remote=192.168.1.3:559845952worker2 # [ 106.396721] systemd[1]: scheduler-token.service: Deactivated successfully.5953worker2 # [ 106.398185] systemd[1]: Finished scheduler-token.service.5954worker2 # [ 106.398957] systemd[1]: scheduler-token.service: Consumed 17ms CPU time over 87ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5955worker2 # [ 106.401300] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.5956worker2 # [ 106.405607] systemd[1]: Starting scheduler-token.service...5957worker2 # [ 106.469225] systemd[1]: scheduler-token.service: Deactivated successfully.5958worker2 # [ 106.470298] systemd[1]: Finished scheduler-token.service.5959worker2 # [ 106.471935] systemd[1]: scheduler-token.service: Consumed 16ms CPU time over 63ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5960worker2 # [ 106.476818] systemd[1]: Starting Nix worker-protocol over gRPC...5961worker1 # [ 106.541863] nix-grpc-daemon[1472]: ts=2026-09-20T13:34:49Z level=info event=scheduler_connected addr=lb:500515962worker2 # [ 106.629755] nix-grpc-daemon[1368]: ts=2026-09-20T13:34:50Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5963worker2 # [ 106.631450] nix-grpc-daemon[1368]: ts=2026-09-20T13:34:50Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5964worker2 # [ 106.634630] nix-grpc-daemon[1368]: ts=2026-09-20T13:34:50Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15965worker2 # [ 106.641626] nix-grpc-daemon[1368]: ts=2026-09-20T13:34:50Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5966worker2 # [ 106.643232] systemd[1]: Started Nix worker-protocol over gRPC.5967worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.40 seconds)5968client: must succeed: sleep 5.05969worker2 # [ 106.662553] nix-grpc-daemon[1368]: ts=2026-09-20T13:34:50Z level=info event=scheduler_connected addr=lb:500515970worker2 # [ 106.666134] nix-grpc-daemon[1368]: ts=2026-09-20T13:34:50Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5971worker1 # [ 106.702196] nix-grpc-daemon[1472]: ts=2026-09-20T13:34:49Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5972client # [ 106.823754] nix[1207]: scheduler at lb:50051: no healthy upstream, reconnecting5973lb # [ 106.901165] envoy[911]: conn peer=[2001:db8:1::1]:42100 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2934 tx=1349 ms=82585974worker1 # [ 106.903180] nix-grpc-daemon[1472]: ts=2026-09-20T13:34:50Z level=info event=scheduler_connected addr=lb:500515975worker2 # [ 106.867632] nix-grpc-daemon[1368]: ts=2026-09-20T13:34:51Z level=info event=scheduler_connected addr=lb:500515976lb # [ 106.906704] envoy[911]: conn peer=[2001:db8:1::3]:39928 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=5767 tx=1349 ms=242545977worker1 # [ 106.907328] nix-grpc-daemon[1472]: ts=2026-09-20T13:34:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5978worker2 # [ 106.871837] nix-grpc-daemon[1368]: ts=2026-09-20T13:34:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5979lb # [ 106.910471] envoy[911]: conn peer=[2001:db8:1::4]:48134 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=8646 tx=646 ms=168995980worker1 # [ 107.108162] nix-grpc-daemon[1472]: ts=2026-09-20T13:34:50Z level=info event=scheduler_connected addr=lb:500515981worker2 # [ 107.073114] nix-grpc-daemon[1368]: ts=2026-09-20T13:34:51Z level=info event=scheduler_connected addr=lb:500515982worker1 # [ 107.112247] nix-grpc-daemon[1472]: ts=2026-09-20T13:34:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5983worker2 # [ 107.076879] nix-grpc-daemon[1368]: ts=2026-09-20T13:34:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5984worker1 # [ 107.312913] nix-grpc-daemon[1472]: ts=2026-09-20T13:34:50Z level=info event=scheduler_connected addr=lb:500515985worker2 # [ 107.277927] nix-grpc-daemon[1368]: ts=2026-09-20T13:34:51Z level=info event=scheduler_connected addr=lb:500515986worker1 # [ 107.316887] nix-grpc-daemon[1472]: ts=2026-09-20T13:34:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5987worker2 # [ 107.281881] nix-grpc-daemon[1368]: ts=2026-09-20T13:34:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5988worker1 # [ 107.518487] nix-grpc-daemon[1472]: ts=2026-09-20T13:34:50Z level=info event=scheduler_connected addr=lb:500515989worker2 # [ 107.482987] nix-grpc-daemon[1368]: ts=2026-09-20T13:34:51Z level=info event=scheduler_connected addr=lb:500515990worker1 # [ 107.522309] nix-grpc-daemon[1472]: ts=2026-09-20T13:34:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5991worker2 # [ 107.486974] nix-grpc-daemon[1368]: ts=2026-09-20T13:34:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5992worker1 # [ 107.723130] nix-grpc-daemon[1472]: ts=2026-09-20T13:34:50Z level=info event=scheduler_connected addr=lb:500515993worker2 # [ 107.688947] nix-grpc-daemon[1368]: ts=2026-09-20T13:34:51Z level=info event=scheduler_connected addr=lb:500515994worker1 # [ 107.726926] nix-grpc-daemon[1472]: ts=2026-09-20T13:34:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5995worker2 # [ 107.692381] nix-grpc-daemon[1368]: ts=2026-09-20T13:34:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5996client # [ 107.827453] nix[1207]: scheduler at lb:50051: no healthy upstream, reconnecting5997worker2 # [ 107.889848] nix-grpc-daemon[1368]: ts=2026-09-20T13:34:52Z level=info event=scheduler_connected addr=lb:500515998worker1 # [ 107.927594] nix-grpc-daemon[1472]: ts=2026-09-20T13:34:51Z level=info event=scheduler_connected addr=lb:500515999worker2 # [ 107.893938] nix-grpc-daemon[1368]: ts=2026-09-20T13:34:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6000worker1 # [ 107.931617] nix-grpc-daemon[1472]: ts=2026-09-20T13:34:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6001worker2 # [ 108.094377] nix-grpc-daemon[1368]: ts=2026-09-20T13:34:52Z level=info event=scheduler_connected addr=lb:500516002worker2 # [ 108.096145] nix-grpc-daemon[1368]: ts=2026-09-20T13:34:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6003worker1 # [ 108.132432] nix-grpc-daemon[1472]: ts=2026-09-20T13:34:51Z level=info event=scheduler_connected addr=lb:500516004worker1 # [ 108.136304] nix-grpc-daemon[1472]: ts=2026-09-20T13:34:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6005worker2 # [ 108.297026] nix-grpc-daemon[1368]: ts=2026-09-20T13:34:52Z level=info event=scheduler_connected addr=lb:500516006worker1 # [ 108.337147] nix-grpc-daemon[1472]: ts=2026-09-20T13:34:51Z level=info event=scheduler_connected addr=lb:500516007worker1 # [ 108.349820] nix-grpc-daemon[1472]: ts=2026-09-20T13:34:51Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:432666008worker1 # [ 108.355482] nix-grpc-daemon[1472]: ts=2026-09-20T13:34:51Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06009worker1 # [ 108.363349] nix-grpc-daemon[1472]: ts=2026-09-20T13:34:51Z level=info event=oidc_jwks provider=mock keys=16010worker1 # [ 108.367288] nix-grpc-daemon[1472]: ts=2026-09-20T13:34:51Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:432666011worker1 # [ 108.371239] nix-grpc-daemon[1472]: ts=2026-09-20T13:34:51Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06012worker1 # [ 109.862836] nix-grpc-daemon[1472]: ts=2026-09-20T13:34:53Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06013worker1 # [ 109.868676] nix-grpc-daemon[1472]: ts=2026-09-20T13:34:53Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=16014worker1 # [ 109.910599] nix-grpc-daemon[1472]: ts=2026-09-20T13:34:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:359426015worker1 # [ 109.923361] nix-grpc-daemon[1472]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 34 ms (attempt 1/5)6016worker1 # [ 109.963599] nix-grpc-daemon[1472]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 183 ms (attempt 2/5)6017worker1 # [ 110.152475] nix-grpc-daemon[1472]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 383 ms (attempt 3/5)6018worker1 # [ 110.538116] nix-grpc-daemon[1472]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 537 ms (attempt 4/5)6019worker1 # [ 111.076538] nix-grpc-daemon[1472]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6020worker1 # [ 111.091495] nix-grpc-daemon[1472]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' from 'http://lb:5751'...6021worker1 # [ 111.114290] nix-daemon[841]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6022client # [ 111.100487] nix[1207]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6023worker1 # [ 111.127434] nix-daemon[841]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6024client: (finished: must succeed: sleep 5.0, in 5.05 seconds)6025client: waiting for success: ! systemctl is-active mid6026worker1 # [ 202.635896] nix-grpc-daemon[1472]: ts=2026-09-20T13:36:25Z level=info event=niks3_push_started pid=15756027worker1 # [ 202.665523] nix-daemon[841]: reaped child process 1551, status = succeeded6028worker1 # [ 202.792498] nix-grpc-daemon[1575]: time=2026-09-20T13:36:26.035Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""6029worker1 # [ 202.916447] nix-daemon[841]: accepted connection from pid 1588, user nix-grpc-daemon (trusted)6030worker1 # [ 202.928393] nix-daemon[841]: reaped child process 1591, status = succeeded6031lb # [ 202.943554] niks3-server[1040]: 2026/09/20 13:36:26 INFO Received uploads request method=POST path=/api/pending_closures6032worker1 # [ 202.954908] nix-grpc-daemon[1575]: time=2026-09-20T13:36:26.197Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6033worker1 # [ 202.956493] nix-grpc-daemon[1575]: time=2026-09-20T13:36:26.199Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6034lb # [ 203.000185] niks3-server[1040]: 2026/09/20 13:36:26 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6035lb # [ 203.007889] niks3-server[1040]: 2026/09/20 13:36:27 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6036lb # [ 203.014453] niks3-server[1040]: 2026/09/20 13:36:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign6037worker1 # [ 203.017948] nix-grpc-daemon[1575]: time=2026-09-20T13:36:26.261Z level=INFO msg="Uploading 1 narinfos"6038lb # [ 203.019382] niks3-server[1040]: 2026/09/20 13:36:27 INFO Signed narinfos id=36 count=16039lb # [ 203.023266] niks3-server[1040]: 2026/09/20 13:36:27 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6040lb # [ 203.028566] niks3-server[1040]: 2026/09/20 13:36:27 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete6041lb # [ 203.034424] niks3-server[1040]: 2026/09/20 13:36:27 INFO Completed upload id=366042worker1 # [ 203.033585] nix-grpc-daemon[1575]: time=2026-09-20T13:36:26.276Z level=INFO msg="Upload complete. (237ms)"6043lb # [ 203.036092] niks3-server[1040]: 2026/09/20 13:36:27 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo6044worker1 # [ 203.042314] nix-grpc-daemon[1472]: ts=2026-09-20T13:36:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35942 duration_s=93 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16045worker1 # [ 203.052918] nix-daemon[841]: reaped child process 1549, status = succeeded6046client # [ 203.044169] systemd[1]: mid.service: Deactivated successfully.6047client # [ 203.045895] systemd[1]: mid.service: Consumed 216ms CPU time over 1min 40.490s wall clock time, 39.3M memory peak, 4K written to disk, 10K incoming IP traffic, 11.3K outgoing IP traffic.6048client: (finished: waiting for success: ! systemctl is-active mid, in 91.21 seconds)6049client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6050client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.03 seconds)6051(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 100.70 seconds)6052subtest: interrupting the client stops the build on the worker6053client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr6054client # [ 203.228103] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr.6055client # Running as unit: intr.service; invocation ID: fb5ac7c6ed2f48c2b613e1b02eb98e686056client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr, in 0.03 seconds)6057worker2 # [ 203.393907] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:374146058worker2 # [ 203.405586] nix-daemon[855]: accepted connection from pid 1368, user nix-grpc-daemon (trusted)6059client # [ 203.429183] nix[1673]: don't know how to build these paths:6060client # [ 203.430131] nix[1673]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6061worker1 # [ 203.463981] nix-grpc-daemon[1472]: ts=2026-09-20T13:36:26Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06062worker1 # [ 203.468498] nix-grpc-daemon[1472]: ts=2026-09-20T13:36:26Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=16063worker2 # [ 203.472131] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374146064worker2 # [ 203.483826] nix-grpc-daemon[1368]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 2 ms (attempt 1/5)6065worker2 # [ 203.491616] nix-grpc-daemon[1368]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 35 ms (attempt 2/5)6066worker2 # [ 203.532673] nix-grpc-daemon[1368]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 302 ms (attempt 3/5)6067worker2 # [ 203.835296] nix-grpc-daemon[1368]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 25 ms (attempt 4/5)6068worker2 # [ 203.866804] nix-grpc-daemon[1368]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6069worker2 # [ 203.884468] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374146070client # [ 203.906385] nix[1673]: copying 1 paths to 'lb:50051'...6071worker2 # [ 203.895346] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:374146072client # [ 203.907218] nix[1673]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6073worker2 # [ 203.901341] nix-daemon[855]: accepted connection from pid 1368, user nix-grpc-daemon (trusted)6074worker2 # [ 203.911500] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:28Z level=info event=niks3_push_started pid=14366075worker2 # [ 204.055675] nix-grpc-daemon[1436]: time=2026-09-20T13:36:28.196Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""6076worker2 # [ 204.199520] nix-daemon[855]: accepted connection from pid 1449, user nix-grpc-daemon (trusted)6077worker2 # [ 204.212255] nix-daemon[855]: reaped child process 1452, status = succeeded6078worker2 # [ 204.365243] nix-daemon[855]: accepted connection from pid 1454, user nix-grpc-daemon (trusted)6079worker2 # [ 204.377194] nix-daemon[855]: reaped child process 1457, status = succeeded6080lb # [ 204.432814] niks3-server[1040]: 2026/09/20 13:36:28 INFO Received uploads request method=POST path=/api/pending_closures6081worker2 # [ 204.402236] nix-grpc-daemon[1436]: time=2026-09-20T13:36:28.544Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6082worker2 # [ 204.404026] nix-grpc-daemon[1436]: time=2026-09-20T13:36:28.545Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6083lb # [ 204.467146] niks3-server[1040]: 2026/09/20 13:36:28 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6084lb # [ 204.474145] niks3-server[1040]: 2026/09/20 13:36:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign6085lb # [ 204.479270] niks3-server[1040]: 2026/09/20 13:36:28 INFO Signed narinfos id=37 count=16086worker2 # [ 204.444483] nix-grpc-daemon[1436]: time=2026-09-20T13:36:28.586Z level=INFO msg="Uploading 1 narinfos"6087lb # [ 204.490160] niks3-server[1040]: 2026/09/20 13:36:28 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6088lb # [ 204.498078] niks3-server[1040]: 2026/09/20 13:36:28 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete6089lb # [ 204.505909] niks3-server[1040]: 2026/09/20 13:36:28 INFO Completed upload id=376090worker2 # [ 204.469847] nix-grpc-daemon[1436]: time=2026-09-20T13:36:28.611Z level=INFO msg="Upload complete. (411ms)"6091lb # [ 204.508543] niks3-server[1040]: 2026/09/20 13:36:28 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6092worker2 # [ 204.471565] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37414 duration_s=0 paths=1 nar_bytes_in=4246093worker2 # [ 204.476649] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374146094worker2 # [ 204.482288] nix-daemon[855]: accepted connection from pid 1368, user nix-grpc-daemon (trusted)6095worker2 # [ 204.484559] nix-daemon[855]: reaped child process 1434, status = succeeded6096client # [ 204.500478] nix[1673]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6097worker2 # [ 204.489522] nix-daemon[855]: accepted connection from pid 1368, user nix-grpc-daemon (trusted)6098client: must succeed: systemctl kill -s INT intr6099client # [ 205.617604] systemd[1]: intr.service: Sent signal SIGINT to main process 1673 (nix) on client request.6100worker2 # [ 205.612985] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:29Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6101client # [ 205.626446] nix[1673]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6102client: (finished: must succeed: systemctl kill -s INT intr, in 0.05 seconds)6103client # [ 205.632839] nix[1673]: error: interrupted by the user6104client # [ 205.664435] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6105client # [ 205.665970] systemd[1]: intr.service: Failed with result 'exit-code'.6106client # [ 205.668110] systemd[1]: intr.service: Consumed 112ms CPU time over 2.439s wall clock time, 39.3M memory peak, 7.7K incoming IP traffic, 8.7K outgoing IP traffic.6107worker2 # [ 205.715193] nix-daemon[1465]: unexpected Nix daemon error: error: interrupted by the user6108worker2 # [ 205.716974] nix-daemon[855]: reaped child process 1463, status = succeeded6109worker2 # [ 205.723163] nix-daemon[855]: reaped child process 1465, status = failed with exit code 16110(finished: subtest: interrupting the client stops the build on the worker, in 2.54 seconds)6111subtest: a deploy mid-build drains: the build finishes, the new generation takes over6112client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw6113client # [ 205.772471] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw.6114client # Running as unit: sw.service; invocation ID: 6e6c3062b36f42568aec10190614c0b66115client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw, in 0.04 seconds)6116worker2 # [ 205.952364] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:374146117client # [ 205.965645] nix[1694]: don't know how to build these paths:6118client # [ 205.966565] nix[1694]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6119worker1 # [ 206.003731] nix-grpc-daemon[1472]: ts=2026-09-20T13:36:29Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06120worker1 # [ 206.007781] nix-grpc-daemon[1472]: ts=2026-09-20T13:36:29Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=16121worker1 # [ 206.045711] nix-grpc-daemon[1472]: ts=2026-09-20T13:36:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:359426122worker1 # [ 206.061734] nix-grpc-daemon[1472]: ts=2026-09-20T13:36:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:359426123client # [ 206.045191] nix[1694]: copying 1 paths to 'lb:50051'...6124worker1 # [ 206.072730] nix-grpc-daemon[1472]: ts=2026-09-20T13:36:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:359426125client # [ 206.047792] nix[1694]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6126worker1 # [ 206.085322] nix-daemon[841]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6127worker1 # [ 206.252230] nix-daemon[841]: accepted connection from pid 1621, user nix-grpc-daemon (trusted)6128worker1 # [ 206.264445] nix-daemon[841]: reaped child process 1624, status = succeeded6129worker1 # [ 206.410901] nix-daemon[841]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)6130worker1 # [ 206.422748] nix-daemon[841]: reaped child process 1629, status = succeeded6131lb # [ 206.432940] niks3-server[1040]: 2026/09/20 13:36:30 INFO Received uploads request method=POST path=/api/pending_closures6132worker1 # [ 206.440350] nix-grpc-daemon[1575]: time=2026-09-20T13:36:29.683Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6133worker1 # [ 206.442268] nix-grpc-daemon[1575]: time=2026-09-20T13:36:29.684Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6134lb # [ 206.463848] niks3-server[1040]: 2026/09/20 13:36:30 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6135lb # [ 206.473998] niks3-server[1040]: 2026/09/20 13:36:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6136lb # [ 206.480498] niks3-server[1040]: 2026/09/20 13:36:30 INFO Signed narinfos id=38 count=16137worker1 # [ 206.482808] nix-grpc-daemon[1575]: time=2026-09-20T13:36:29.725Z level=INFO msg="Uploading 1 narinfos"6138lb # [ 206.487586] niks3-server[1040]: 2026/09/20 13:36:30 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6139lb # [ 206.498349] niks3-server[1040]: 2026/09/20 13:36:30 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6140worker1 # [ 206.503601] nix-grpc-daemon[1575]: time=2026-09-20T13:36:29.746Z level=INFO msg="Upload complete. (407ms)"6141lb # [ 206.504778] niks3-server[1040]: 2026/09/20 13:36:30 INFO Completed upload id=386142worker1 # [ 206.506133] nix-grpc-daemon[1472]: ts=2026-09-20T13:36:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35942 duration_s=0 paths=1 nar_bytes_in=4166143lb # [ 206.508229] niks3-server[1040]: 2026/09/20 13:36:30 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6144worker1 # [ 206.510611] nix-grpc-daemon[1472]: ts=2026-09-20T13:36:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:359426145worker1 # [ 206.514846] nix-daemon[841]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6146worker1 # [ 206.519118] nix-daemon[841]: reaped child process 1619, status = succeeded6147worker1 # [ 206.523756] nix-daemon[841]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6148client # [ 206.497063] nix[1694]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6149lb # [ 206.905595] envoy[911]: conn peer=[2001:db8:1::1]:59630 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4192 tx=1757 ms=1002886150lb # [ 206.909435] envoy[911]: conn peer=[2001:db8:1::1]:50742 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2937 tx=734 ms=22446151worker1: must succeed: systemctl reload nix-grpc-daemon.service6152worker1 # [ 208.281624] systemd[1]: Reloading Nix worker-protocol over gRPC...6153worker1 # [ 208.321308] systemd[1]: Reloaded Nix worker-protocol over gRPC.6154worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.08 seconds)6155worker1: must succeed: pgrep -f 'read -t [9]0 x'6156worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)6157client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&26158worker1 # [ 208.437965] nix-grpc-daemon[1472]: ts=2026-09-20T13:36:31Z level=info event=draining6159worker2 # [ 208.565806] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:374146160client # don't know how to build these paths:6161client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6162worker2 # [ 208.574360] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374146163worker1 # [ 208.628956] nix-grpc-daemon[1472]: ts=2026-09-20T13:36:31Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06164worker1 # [ 208.630823] nix-grpc-daemon[1472]: ts=2026-09-20T13:36:31Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=16165worker2 # [ 208.640798] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374146166worker2 # [ 208.651885] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374146167client # copying 1 paths to 'lb:50051'...6168worker2 # [ 208.662745] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:374146169client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6170worker2 # [ 208.685240] nix-daemon[855]: accepted connection from pid 1368, user nix-grpc-daemon (trusted)6171worker2 # [ 208.859585] nix-daemon[855]: accepted connection from pid 1507, user nix-grpc-daemon (trusted)6172worker2 # [ 208.872565] nix-daemon[855]: reaped child process 1510, status = succeeded6173worker2 # [ 209.033612] nix-daemon[855]: accepted connection from pid 1512, user nix-grpc-daemon (trusted)6174worker2 # [ 209.047303] nix-daemon[855]: reaped child process 1515, status = succeeded6175lb # [ 209.100630] niks3-server[1040]: 2026/09/20 13:36:33 INFO Received uploads request method=POST path=/api/pending_closures6176worker2 # [ 209.072250] nix-grpc-daemon[1436]: time=2026-09-20T13:36:33.213Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6177worker2 # [ 209.073774] nix-grpc-daemon[1436]: time=2026-09-20T13:36:33.213Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6178lb # [ 209.128893] niks3-server[1040]: 2026/09/20 13:36:33 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6179lb # [ 209.139992] niks3-server[1040]: 2026/09/20 13:36:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6180lb # [ 209.144464] niks3-server[1040]: 2026/09/20 13:36:33 INFO Signed narinfos id=39 count=16181worker2 # [ 209.108701] nix-grpc-daemon[1436]: time=2026-09-20T13:36:33.250Z level=INFO msg="Uploading 1 narinfos"6182lb # [ 209.153509] niks3-server[1040]: 2026/09/20 13:36:33 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6183lb # [ 209.161637] niks3-server[1040]: 2026/09/20 13:36:33 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6184lb # [ 209.164710] niks3-server[1040]: 2026/09/20 13:36:33 INFO Completed upload id=396185worker2 # [ 209.126399] nix-grpc-daemon[1436]: time=2026-09-20T13:36:33.268Z level=INFO msg="Upload complete. (429ms)"6186lb # [ 209.166718] niks3-server[1040]: 2026/09/20 13:36:33 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6187worker2 # [ 209.131195] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37414 duration_s=0 paths=1 nar_bytes_in=5286188worker2 # [ 209.135797] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374146189worker2 # [ 209.143536] nix-daemon[855]: accepted connection from pid 1368, user nix-grpc-daemon (trusted)6190worker2 # [ 209.148329] nix-daemon[855]: reaped child process 1505, status = succeeded6191client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6192worker2 # [ 209.154223] nix-daemon[855]: accepted connection from pid 1368, user nix-grpc-daemon (trusted)6193client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6194client # farm-a> LOG-farm-a-during-drain6195worker2 # [ 210.370433] nix-daemon[855]: reaped child process 1519, status = succeeded6196worker2 # [ 210.515188] nix-daemon[855]: accepted connection from pid 1534, user nix-grpc-daemon (trusted)6197worker2 # [ 210.527713] nix-daemon[855]: reaped child process 1537, status = succeeded6198lb # [ 210.581720] niks3-server[1040]: 2026/09/20 13:36:34 INFO Received uploads request method=POST path=/api/pending_closures6199worker2 # [ 210.551513] nix-grpc-daemon[1436]: time=2026-09-20T13:36:34.693Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6200worker2 # [ 210.553064] nix-grpc-daemon[1436]: time=2026-09-20T13:36:34.695Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6201lb # [ 210.611766] niks3-server[1040]: 2026/09/20 13:36:34 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6202lb # [ 210.629917] niks3-server[1040]: 2026/09/20 13:36:34 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6203lb # [ 210.639444] niks3-server[1040]: 2026/09/20 13:36:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6204lb # [ 210.644084] niks3-server[1040]: 2026/09/20 13:36:34 INFO Signed narinfos id=40 count=16205worker2 # [ 210.608823] nix-grpc-daemon[1436]: time=2026-09-20T13:36:34.750Z level=INFO msg="Uploading 1 narinfos"6206lb # [ 210.649716] niks3-server[1040]: 2026/09/20 13:36:34 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6207lb # [ 210.659532] niks3-server[1040]: 2026/09/20 13:36:34 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6208lb # [ 210.664949] niks3-server[1040]: 2026/09/20 13:36:34 INFO Completed upload id=406209worker2 # [ 210.627888] nix-grpc-daemon[1436]: time=2026-09-20T13:36:34.769Z level=INFO msg="Upload complete. (266ms)"6210lb # [ 210.667058] niks3-server[1040]: 2026/09/20 13:36:34 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6211worker2 # [ 210.630334] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37414 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16212worker1 # [ 210.673895] nix-grpc-daemon[1472]: ts=2026-09-20T13:36:33Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06213worker2 # [ 210.637725] nix-daemon[855]: reaped child process 1517, status = succeeded6214worker1 # [ 210.675696] nix-grpc-daemon[1472]: ts=2026-09-20T13:36:33Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=16215worker2 # [ 210.680154] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374146216worker2 # [ 210.693638] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374146217worker2 # [ 210.706645] nix-grpc-daemon[1436]: time=2026-09-20T13:36:34.848Z level=INFO msg="All 1 paths already cached"6218client # copying 1 paths to 'lb:50051'...6219worker2 # [ 210.714854] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:374146220client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6221worker2 # [ 210.734170] nix-daemon[855]: accepted connection from pid 1368, user nix-grpc-daemon (trusted)6222worker2 # [ 210.904771] nix-daemon[855]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6223worker2 # [ 210.917216] nix-daemon[855]: reaped child process 1544, status = succeeded6224worker2 # [ 211.056670] nix-daemon[855]: accepted connection from pid 1546, user nix-grpc-daemon (trusted)6225worker2 # [ 211.069150] nix-daemon[855]: reaped child process 1549, status = succeeded6226lb # [ 211.118179] niks3-server[1040]: 2026/09/20 13:36:35 INFO Received uploads request method=POST path=/api/pending_closures6227worker2 # [ 211.083981] nix-grpc-daemon[1436]: time=2026-09-20T13:36:35.225Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6228worker2 # [ 211.086098] nix-grpc-daemon[1436]: time=2026-09-20T13:36:35.227Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6229lb # [ 211.132558] niks3-server[1040]: 2026/09/20 13:36:35 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6230lb # [ 211.136054] niks3-server[1040]: 2026/09/20 13:36:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6231lb # [ 211.138418] niks3-server[1040]: 2026/09/20 13:36:35 INFO Signed narinfos id=41 count=16232worker2 # [ 211.101465] nix-grpc-daemon[1436]: time=2026-09-20T13:36:35.243Z level=INFO msg="Uploading 1 narinfos"6233lb # [ 211.140581] niks3-server[1040]: 2026/09/20 13:36:35 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6234lb # [ 211.145635] niks3-server[1040]: 2026/09/20 13:36:35 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6235lb # [ 211.148493] niks3-server[1040]: 2026/09/20 13:36:35 INFO Completed upload id=416236worker2 # [ 211.111443] nix-grpc-daemon[1436]: time=2026-09-20T13:36:35.253Z level=INFO msg="Upload complete. (365ms)"6237lb # [ 211.150710] niks3-server[1040]: 2026/09/20 13:36:35 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6238worker2 # [ 211.113323] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37414 duration_s=0 paths=1 nar_bytes_in=6406239worker2 # [ 211.116598] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374146240worker2 # [ 211.122987] nix-daemon[855]: accepted connection from pid 1368, user nix-grpc-daemon (trusted)6241worker2 # [ 211.125369] nix-daemon[855]: reaped child process 1539, status = succeeded6242worker2 # [ 211.130343] nix-daemon[855]: accepted connection from pid 1368, user nix-grpc-daemon (trusted)6243client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6244client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6245client # farm-b> LOG-farm-b-during-drain6246worker2 # [ 212.545091] nix-daemon[855]: reaped child process 1553, status = succeeded6247worker2 # [ 212.671340] nix-daemon[855]: accepted connection from pid 1568, user nix-grpc-daemon (trusted)6248worker2 # [ 212.682670] nix-daemon[855]: reaped child process 1571, status = succeeded6249lb # [ 212.732730] niks3-server[1040]: 2026/09/20 13:36:36 INFO Received uploads request method=POST path=/api/pending_closures6250worker2 # [ 212.702406] nix-grpc-daemon[1436]: time=2026-09-20T13:36:36.844Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6251worker2 # [ 212.703879] nix-grpc-daemon[1436]: time=2026-09-20T13:36:36.845Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6252lb # [ 212.764573] niks3-server[1040]: 2026/09/20 13:36:36 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6253lb # [ 212.777384] niks3-server[1040]: 2026/09/20 13:36:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6254worker2 # [ 212.743594] nix-grpc-daemon[1436]: time=2026-09-20T13:36:36.885Z level=INFO msg="Uploading 1 narinfos"6255lb # [ 212.783076] niks3-server[1040]: 2026/09/20 13:36:36 INFO Signed narinfos id=42 count=16256lb # [ 212.793333] niks3-server[1040]: 2026/09/20 13:36:36 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6257lb # [ 212.802797] niks3-server[1040]: 2026/09/20 13:36:36 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6258lb # [ 212.806510] niks3-server[1040]: 2026/09/20 13:36:36 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6259lb # [ 212.814028] niks3-server[1040]: 2026/09/20 13:36:36 INFO Completed upload id=426260worker2 # [ 212.776881] nix-grpc-daemon[1436]: time=2026-09-20T13:36:36.918Z level=INFO msg="Upload complete. (239ms)"6261lb # [ 212.815751] niks3-server[1040]: 2026/09/20 13:36:36 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6262worker2 # [ 212.779463] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37414 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16263worker2 # [ 212.786392] nix-daemon[855]: reaped child process 1551, status = succeeded6264worker1 # [ 212.823502] nix-grpc-daemon[1472]: ts=2026-09-20T13:36:36Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06265worker1 # [ 212.825883] nix-grpc-daemon[1472]: ts=2026-09-20T13:36:36Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=16266worker2 # [ 212.830308] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374146267worker2 # [ 212.838441] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374146268worker2 # [ 212.846605] nix-grpc-daemon[1436]: time=2026-09-20T13:36:36.988Z level=INFO msg="All 2 paths already cached"6269client # copying 1 paths to 'lb:50051'...6270worker2 # [ 212.850842] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:374146271client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6272worker2 # [ 212.858989] nix-daemon[855]: accepted connection from pid 1368, user nix-grpc-daemon (trusted)6273worker2 # [ 213.016664] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)6274worker2 # [ 213.029249] nix-daemon[855]: reaped child process 1578, status = succeeded6275worker2 # [ 213.165093] nix-daemon[855]: accepted connection from pid 1580, user nix-grpc-daemon (trusted)6276worker2 # [ 213.177814] nix-daemon[855]: reaped child process 1583, status = succeeded6277lb # [ 213.224804] niks3-server[1040]: 2026/09/20 13:36:37 INFO Received uploads request method=POST path=/api/pending_closures6278worker2 # [ 213.194154] nix-grpc-daemon[1436]: time=2026-09-20T13:36:37.335Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6279worker2 # [ 213.195608] nix-grpc-daemon[1436]: time=2026-09-20T13:36:37.335Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6280lb # [ 213.248575] niks3-server[1040]: 2026/09/20 13:36:37 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6281lb # [ 213.259819] niks3-server[1040]: 2026/09/20 13:36:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6282lb # [ 213.265627] niks3-server[1040]: 2026/09/20 13:36:37 INFO Signed narinfos id=43 count=16283worker2 # [ 213.230402] nix-grpc-daemon[1436]: time=2026-09-20T13:36:37.372Z level=INFO msg="Uploading 1 narinfos"6284lb # [ 213.269129] niks3-server[1040]: 2026/09/20 13:36:37 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6285lb # [ 213.282360] niks3-server[1040]: 2026/09/20 13:36:37 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6286worker2 # [ 213.248954] nix-grpc-daemon[1436]: time=2026-09-20T13:36:37.390Z level=INFO msg="Upload complete. (383ms)"6287lb # [ 213.287295] niks3-server[1040]: 2026/09/20 13:36:37 INFO Completed upload id=436288worker2 # [ 213.250908] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37414 duration_s=0 paths=1 nar_bytes_in=7686289lb # [ 213.292255] niks3-server[1040]: 2026/09/20 13:36:37 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6290worker2 # [ 213.258508] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374146291worker2 # [ 213.262655] nix-daemon[855]: accepted connection from pid 1368, user nix-grpc-daemon (trusted)6292worker2 # [ 213.264928] nix-daemon[855]: reaped child process 1573, status = succeeded6293worker2 # [ 213.270376] nix-daemon[855]: accepted connection from pid 1368, user nix-grpc-daemon (trusted)6294client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6295client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6296worker2 # [ 214.082643] nix-daemon[855]: reaped child process 1587, status = succeeded6297client # farm-top> LOG-farm-top-during-drain6298worker2 # [ 214.223357] nix-daemon[855]: accepted connection from pid 1602, user nix-grpc-daemon (trusted)6299worker2 # [ 214.235619] nix-daemon[855]: reaped child process 1605, status = succeeded6300lb # [ 214.289578] niks3-server[1040]: 2026/09/20 13:36:38 INFO Received uploads request method=POST path=/api/pending_closures6301worker2 # [ 214.256303] nix-grpc-daemon[1436]: time=2026-09-20T13:36:38.398Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6302worker2 # [ 214.257859] nix-grpc-daemon[1436]: time=2026-09-20T13:36:38.399Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6303lb # [ 214.310910] niks3-server[1040]: 2026/09/20 13:36:38 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6304lb # [ 214.314987] niks3-server[1040]: 2026/09/20 13:36:38 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6305lb # [ 214.318670] niks3-server[1040]: 2026/09/20 13:36:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6306lb # [ 214.320541] niks3-server[1040]: 2026/09/20 13:36:38 INFO Signed narinfos id=44 count=16307worker2 # [ 214.283645] nix-grpc-daemon[1436]: time=2026-09-20T13:36:38.425Z level=INFO msg="Uploading 1 narinfos"6308lb # [ 214.324577] niks3-server[1040]: 2026/09/20 13:36:38 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6309lb # [ 214.330415] niks3-server[1040]: 2026/09/20 13:36:38 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6310lb # [ 214.333774] niks3-server[1040]: 2026/09/20 13:36:38 INFO Completed upload id=446311worker2 # [ 214.296788] nix-grpc-daemon[1436]: time=2026-09-20T13:36:38.438Z level=INFO msg="Upload complete. (220ms)"6312lb # [ 214.335616] niks3-server[1040]: 2026/09/20 13:36:38 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6313worker2 # [ 214.299431] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37414 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16314worker2 # [ 214.306093] nix-daemon[855]: reaped child process 1585, status = succeeded6315client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&2, in 6.01 seconds)6316client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath6317client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath, in 0.19 seconds)6318worker1: must succeed: pkill -f 'read -t [9]0 x'6319worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6320client: waiting for success: ! systemctl is-active sw6321worker1 # [ 214.763779] nix-daemon[841]: reaped child process 1633, status = succeeded6322worker1 # [ 214.898304] nix-daemon[841]: accepted connection from pid 1679, user nix-grpc-daemon (trusted)6323worker1 # [ 214.910912] nix-daemon[841]: reaped child process 1682, status = succeeded6324lb # [ 214.926098] niks3-server[1040]: 2026/09/20 13:36:38 INFO Received uploads request method=POST path=/api/pending_closures6325worker1 # [ 214.934197] nix-grpc-daemon[1575]: time=2026-09-20T13:36:38.177Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6326worker1 # [ 214.935924] nix-grpc-daemon[1575]: time=2026-09-20T13:36:38.178Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6327lb # [ 214.955862] niks3-server[1040]: 2026/09/20 13:36:38 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6328lb # [ 214.968520] niks3-server[1040]: 2026/09/20 13:36:38 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6329lb # [ 214.977998] niks3-server[1040]: 2026/09/20 13:36:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6330lb # [ 214.984177] niks3-server[1040]: 2026/09/20 13:36:38 INFO Signed narinfos id=45 count=16331worker1 # [ 214.986148] nix-grpc-daemon[1575]: time=2026-09-20T13:36:38.228Z level=INFO msg="Uploading 1 narinfos"6332lb # [ 214.989518] niks3-server[1040]: 2026/09/20 13:36:38 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6333lb # [ 214.999205] niks3-server[1040]: 2026/09/20 13:36:38 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6334lb # [ 215.004741] niks3-server[1040]: 2026/09/20 13:36:39 INFO Completed upload id=456335lb # [ 215.006964] niks3-server[1040]: 2026/09/20 13:36:39 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6336worker1 # [ 215.005699] nix-grpc-daemon[1575]: time=2026-09-20T13:36:38.248Z level=INFO msg="Upload complete. (267ms)"6337worker1 # [ 215.014563] nix-grpc-daemon[1472]: ts=2026-09-20T13:36:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35942 duration_s=8 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16338worker1 # [ 215.025146] nix-daemon[841]: reaped child process 1631, status = succeeded6339client # [ 215.024863] systemd[1]: sw.service: Deactivated successfully.6340client # [ 215.028609] systemd[1]: sw.service: Consumed 122ms CPU time over 9.254s wall clock time, 39.1M memory peak, 8.6K incoming IP traffic, 9K outgoing IP traffic.6341worker1 # [ 215.440683] nix-grpc-daemon[1472]: ts=2026-09-20T13:36:38Z level=info event=signal_exit6342client: (finished: waiting for success: ! systemctl is-active sw, in 1.09 seconds)6343client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6344worker1 # [ 215.717941] nix-grpc-daemon[1472]: ts=2026-09-20T13:36:38Z level=info event=scheduler_yield6345lb # [ 215.721457] niks3-server[1040]: 2026/09/20 13:36:39 INFO lead: released remote=192.168.1.3:559846346worker2 # [ 215.685556] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:39Z level=info event=scheduler_restarting addr=lb:500516347worker1 # [ 215.723638] nix-grpc-daemon[1472]: ts=2026-09-20T13:36:38Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:432666348worker1 # [ 215.728806] nix-grpc-daemon[1472]: ts=2026-09-20T13:36:38Z level=info event=worker_gone addr=192.168.1.4:500516349worker1 # [ 215.731781] nix-grpc-daemon[1472]: ts=2026-09-20T13:36:38Z level=info event=scheduler_restarting addr=lb:500516350worker1 # [ 215.734702] nix-grpc-daemon[1472]: ts=2026-09-20T13:36:38Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:432666351worker1 # [ 215.738005] nix-grpc-daemon[1472]: ts=2026-09-20T13:36:38Z level=info event=worker_gone addr=192.168.1.3:500516352client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.04 seconds)6353worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6354worker2 # [ 215.885911] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:40Z level=info event=scheduler_connected addr=lb:500516355worker2 # [ 215.888726] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6356worker1 # [ 215.929247] nix-grpc-daemon[1472]: ts=2026-09-20T13:36:39Z level=info event=scheduler_connected addr=lb:500516357worker1 # [ 215.933824] nix-grpc-daemon[1472]: ts=2026-09-20T13:36:39Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6358worker2 # [ 216.089654] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:40Z level=info event=scheduler_connected addr=lb:500516359worker2 # [ 216.092324] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6360worker1 # [ 216.136176] nix-grpc-daemon[1472]: ts=2026-09-20T13:36:39Z level=info event=scheduler_connected addr=lb:500516361worker1 # [ 216.139791] nix-grpc-daemon[1472]: ts=2026-09-20T13:36:39Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6362worker2 # [ 216.293180] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:40Z level=info event=scheduler_connected addr=lb:500516363worker2 # [ 216.295329] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6364worker1 # [ 216.337465] nix-grpc-daemon[1472]: ts=2026-09-20T13:36:39Z level=info event=scheduler_connected addr=lb:500516365worker1 # [ 216.342068] nix-grpc-daemon[1472]: ts=2026-09-20T13:36:39Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6366worker2 # [ 216.496248] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:40Z level=info event=scheduler_connected addr=lb:500516367worker2 # [ 216.497971] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6368worker1 # [ 216.542978] nix-grpc-daemon[1472]: ts=2026-09-20T13:36:39Z level=info event=scheduler_connected addr=lb:500516369worker1 # [ 216.546937] nix-grpc-daemon[1472]: ts=2026-09-20T13:36:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6370worker2 # [ 216.652583] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:40Z level=info event=scheduler_take_over6371lb # [ 216.690587] niks3-server[1040]: 2026/09/20 13:36:40 INFO lead: acquired remote=192.168.1.4:460006372worker1 # [ 216.719201] nix-grpc-daemon[1472]: ts=2026-09-20T13:36:39Z level=info event=scheduler_restarting6373worker2 # [ 216.698874] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:40Z level=info event=scheduler_connected addr=lb:500516374worker2 # [ 216.700498] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6375worker1 # [ 216.739180] nix-daemon[841]: reaped child process 1525, status = succeeded6376worker1 # [ 216.756895] nix-grpc-daemon[1472]: ts=2026-09-20T13:36:39Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6377lb # [ 216.907307] envoy[911]: conn peer=[2001:db8:1::1]:50758 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9585 tx=3874 ms=57496378lb # [ 216.910999] envoy[911]: conn peer=[2001:db8:1::1]:50746 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2949 tx=1349 ms=90356379lb # [ 216.914568] envoy[911]: conn peer=[2001:db8:1::3]:60342 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=3588 tx=1007 ms=1153356380worker2 # [ 216.901062] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:41Z level=info event=scheduler_connected addr=lb:500516381worker2 # [ 216.902787] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6382worker2 # [ 217.103535] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:41Z level=info event=scheduler_connected addr=lb:500516383worker2 # [ 217.105338] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6384worker2 # [ 217.306151] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:41Z level=info event=scheduler_connected addr=lb:500516385worker2 # [ 217.307923] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6386worker1 # [ 217.355976] nix-grpc-daemon[1472]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6387worker1 # [ 217.359511] nix-grpc-daemon[1472]: E0000 00:00:1789911400.598036 1515 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true6388worker1 # [ 217.364128] nix-grpc-daemon[1472]: E0000 00:00:1789911400.598323 1515 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6389worker1 # [ 217.540632] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6390worker2 # [ 217.508610] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:41Z level=info event=scheduler_connected addr=lb:500516391worker2 # [ 217.510391] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6392worker1 # [ 217.544579] systemd[1]: nix-grpc-daemon.service: Consumed 873ms CPU time over 1min 56.313s wall clock time, 30.9M memory peak, 28.9K incoming IP traffic, 38.4K outgoing IP traffic.6393worker1 # [ 217.560764] systemd[1]: Starting Nix worker-protocol over gRPC...6394worker1 # [ 217.705028] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:40Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6395worker1 # [ 217.707116] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:40Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6396worker1 # [ 217.710662] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:40Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16397worker1 # [ 217.717863] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:40Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6398worker1 # [ 217.720373] systemd[1]: Started Nix worker-protocol over gRPC.6399worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 1.98 seconds)6400lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6401lb: must succeed: curl -sf localhost:9901/clusters6402worker1 # [ 217.733446] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:40Z level=info event=scheduler_connected addr=lb:500516403worker1 # [ 217.735343] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6404worker2 # [ 217.712347] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:41Z level=info event=scheduler_connected addr=lb:500516405worker2 # [ 217.716094] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6406lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6407worker1 # [ 217.935929] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:41Z level=info event=scheduler_connected addr=lb:500516408worker2 # [ 217.908448] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:42Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:365326409worker2 # [ 217.912636] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:42Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06410worker2 # [ 217.921271] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:42Z level=info event=scheduler_connected addr=lb:500516411worker2 # [ 217.928142] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:42Z level=info event=oidc_jwks provider=mock keys=16412worker2 # [ 217.932231] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:42Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:365326413worker2 # [ 217.936130] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:42Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06414lb: must succeed: curl -sf localhost:9901/clusters6415lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6416lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.14 seconds)6417lb: must succeed: curl -sf localhost:9901/clusters6418lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6419worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6420worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6421(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 13.20 seconds)6422subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6423client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16424worker1 # [ 219.160745] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:42Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:514506425worker1 # [ 219.166051] nix-daemon[841]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6426worker2 # [ 219.138571] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374146427worker2 # [ 219.182848] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:43Z level=debug event=want client=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06428worker2 # [ 219.186741] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:43Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16429worker2 # [ 219.191135] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:43Z level=debug event=want client=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06430worker2 # [ 219.194904] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:43Z level=debug event=want client=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06431worker1 # [ 219.265681] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514506432worker1 # [ 219.279989] nix-grpc-daemon[1696]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 33 ms (attempt 1/5)6433worker1 # [ 219.318760] nix-grpc-daemon[1696]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 125 ms (attempt 2/5)6434worker1 # [ 219.449640] nix-grpc-daemon[1696]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 315 ms (attempt 3/5)6435worker1 # [ 219.770647] nix-grpc-daemon[1696]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 81 ms (attempt 4/5)6436worker1 # [ 219.857867] nix-grpc-daemon[1696]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6437worker1 # [ 219.874647] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514506438worker1 # [ 219.886187] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:514506439worker1 # [ 219.896136] nix-daemon[841]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6440worker1 # [ 219.915367] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:43Z level=info event=niks3_push_started pid=17526441worker1 # [ 220.093608] nix-grpc-daemon[1752]: time=2026-09-20T13:36:43.336Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""6442worker1 # [ 220.225192] nix-daemon[841]: accepted connection from pid 1765, user nix-grpc-daemon (trusted)6443worker1 # [ 220.235647] nix-daemon[841]: reaped child process 1768, status = succeeded6444worker1 # [ 220.368702] nix-daemon[841]: accepted connection from pid 1770, user nix-grpc-daemon (trusted)6445worker1 # [ 220.380162] nix-daemon[841]: reaped child process 1773, status = succeeded6446lb # [ 220.395365] niks3-server[1040]: 2026/09/20 13:36:44 INFO Received uploads request method=POST path=/api/pending_closures6447worker1 # [ 220.405073] nix-grpc-daemon[1752]: time=2026-09-20T13:36:43.647Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6448worker1 # [ 220.406428] nix-grpc-daemon[1752]: time=2026-09-20T13:36:43.647Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6449lb # [ 220.428933] niks3-server[1040]: 2026/09/20 13:36:44 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6450lb # [ 220.440963] niks3-server[1040]: 2026/09/20 13:36:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6451lb # [ 220.444914] niks3-server[1040]: 2026/09/20 13:36:44 INFO Signed narinfos id=46 count=16452worker1 # [ 220.447600] nix-grpc-daemon[1752]: time=2026-09-20T13:36:43.690Z level=INFO msg="Uploading 1 narinfos"6453lb # [ 220.451918] niks3-server[1040]: 2026/09/20 13:36:44 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6454lb # [ 220.462985] niks3-server[1040]: 2026/09/20 13:36:44 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6455lb # [ 220.465938] niks3-server[1040]: 2026/09/20 13:36:44 INFO Completed upload id=466456worker1 # [ 220.464963] nix-grpc-daemon[1752]: time=2026-09-20T13:36:43.708Z level=INFO msg="Upload complete. (367ms)"6457worker1 # [ 220.467208] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51450 duration_s=0 paths=1 nar_bytes_in=5046458worker1 # [ 220.470646] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514506459lb # [ 220.476345] niks3-server[1040]: 2026/09/20 13:36:44 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6460worker1 # [ 220.477112] nix-daemon[841]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6461worker1 # [ 220.479197] nix-daemon[841]: reaped child process 1750, status = succeeded6462worker1 # [ 220.484517] nix-daemon[841]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6463worker1 # [ 221.507917] nix-daemon[841]: reaped child process 1777, status = succeeded6464worker1 # [ 221.660360] nix-daemon[841]: accepted connection from pid 1792, user nix-grpc-daemon (trusted)6465worker1 # [ 221.670829] nix-daemon[841]: reaped child process 1795, status = succeeded6466lb # [ 221.686203] niks3-server[1040]: 2026/09/20 13:36:45 INFO Received uploads request method=POST path=/api/pending_closures6467worker1 # [ 221.697639] nix-grpc-daemon[1752]: time=2026-09-20T13:36:44.940Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6468worker1 # [ 221.699241] nix-grpc-daemon[1752]: time=2026-09-20T13:36:44.942Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6469lb # [ 221.730408] niks3-server[1040]: 2026/09/20 13:36:45 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6470lb # [ 221.740081] niks3-server[1040]: 2026/09/20 13:36:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6471lb # [ 221.749976] niks3-server[1040]: 2026/09/20 13:36:45 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6472lb # [ 221.753917] niks3-server[1040]: 2026/09/20 13:36:45 INFO Signed narinfos id=47 count=16473worker1 # [ 221.755877] nix-grpc-daemon[1752]: time=2026-09-20T13:36:44.998Z level=INFO msg="Uploading 1 narinfos"6474lb # [ 221.773079] niks3-server[1040]: 2026/09/20 13:36:45 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6475lb # [ 221.778516] niks3-server[1040]: 2026/09/20 13:36:45 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6476lb # [ 221.781433] niks3-server[1040]: 2026/09/20 13:36:45 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6477lb # [ 221.784582] niks3-server[1040]: 2026/09/20 13:36:45 INFO Completed upload id=476478worker1 # [ 221.783597] nix-grpc-daemon[1752]: time=2026-09-20T13:36:45.026Z level=INFO msg="Upload complete. (292ms)"6479worker1 # [ 221.786292] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51450 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=1 outputs=16480worker1 # [ 221.792288] nix-daemon[841]: reaped child process 1775, status = succeeded6481worker2 # [ 221.756460] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:45Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16482worker2 # [ 221.761083] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:45Z level=debug event=want client=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06483worker1 # [ 221.836736] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514506484worker1 # [ 221.849930] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514506485worker1 # [ 221.861310] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:514506486worker1 # [ 221.872236] nix-daemon[841]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6487worker1 # [ 222.037318] nix-daemon[841]: accepted connection from pid 1799, user nix-grpc-daemon (trusted)6488worker1 # [ 222.049169] nix-daemon[841]: reaped child process 1802, status = succeeded6489worker1 # [ 222.201141] nix-daemon[841]: accepted connection from pid 1804, user nix-grpc-daemon (trusted)6490worker1 # [ 222.212786] nix-daemon[841]: reaped child process 1807, status = succeeded6491lb # [ 222.227374] niks3-server[1040]: 2026/09/20 13:36:46 INFO Received uploads request method=POST path=/api/pending_closures6492worker1 # [ 222.231076] nix-grpc-daemon[1752]: time=2026-09-20T13:36:45.474Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6493worker1 # [ 222.232733] nix-grpc-daemon[1752]: time=2026-09-20T13:36:45.475Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6494lb # [ 222.244786] niks3-server[1040]: 2026/09/20 13:36:46 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6495lb # [ 222.246908] niks3-server[1040]: 2026/09/20 13:36:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6496lb # [ 222.249375] niks3-server[1040]: 2026/09/20 13:36:46 INFO Signed narinfos id=48 count=16497worker1 # [ 222.249800] nix-grpc-daemon[1752]: time=2026-09-20T13:36:45.492Z level=INFO msg="Uploading 1 narinfos"6498lb # [ 222.252758] niks3-server[1040]: 2026/09/20 13:36:46 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6499lb # [ 222.257743] niks3-server[1040]: 2026/09/20 13:36:46 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6500lb # [ 222.261619] niks3-server[1040]: 2026/09/20 13:36:46 INFO Completed upload id=486501worker1 # [ 222.260655] nix-grpc-daemon[1752]: time=2026-09-20T13:36:45.503Z level=INFO msg="Upload complete. (376ms)"6502lb # [ 222.263172] niks3-server[1040]: 2026/09/20 13:36:46 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6503worker1 # [ 222.262727] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51450 duration_s=0 paths=1 nar_bytes_in=5046504worker1 # [ 222.266518] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514506505worker1 # [ 222.272518] nix-daemon[841]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6506worker1 # [ 222.274686] nix-daemon[841]: reaped child process 1797, status = succeeded6507worker1 # [ 222.279822] nix-daemon[841]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6508worker1 # [ 223.091495] nix-daemon[841]: reaped child process 1811, status = succeeded6509worker1 # [ 223.226250] nix-daemon[841]: accepted connection from pid 1826, user nix-grpc-daemon (trusted)6510worker1 # [ 223.237839] nix-daemon[841]: reaped child process 1829, status = succeeded6511lb # [ 223.252183] niks3-server[1040]: 2026/09/20 13:36:47 INFO Received uploads request method=POST path=/api/pending_closures6512worker1 # [ 223.256967] nix-grpc-daemon[1752]: time=2026-09-20T13:36:46.499Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6513worker1 # [ 223.258465] nix-grpc-daemon[1752]: time=2026-09-20T13:36:46.501Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6514lb # [ 223.271472] niks3-server[1040]: 2026/09/20 13:36:47 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6515lb # [ 223.275685] niks3-server[1040]: 2026/09/20 13:36:47 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6516lb # [ 223.279593] niks3-server[1040]: 2026/09/20 13:36:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6517lb # [ 223.281259] niks3-server[1040]: 2026/09/20 13:36:47 INFO Signed narinfos id=49 count=16518worker1 # [ 223.280642] nix-grpc-daemon[1752]: time=2026-09-20T13:36:46.523Z level=INFO msg="Uploading 1 narinfos"6519lb # [ 223.285836] niks3-server[1040]: 2026/09/20 13:36:47 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6520lb # [ 223.289567] niks3-server[1040]: 2026/09/20 13:36:47 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6521lb # [ 223.294715] niks3-server[1040]: 2026/09/20 13:36:47 INFO Completed upload id=496522worker1 # [ 223.293781] nix-grpc-daemon[1752]: time=2026-09-20T13:36:46.536Z level=INFO msg="Upload complete. (208ms)"6523lb # [ 223.296415] niks3-server[1040]: 2026/09/20 13:36:47 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6524worker1 # [ 223.296043] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51450 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=2 outputs=16525worker1 # [ 223.302436] nix-daemon[841]: reaped child process 1809, status = succeeded6526worker2 # [ 223.262319] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:47Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16527worker2 # [ 223.270828] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:47Z level=debug event=want client=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06528worker1 # [ 223.349269] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514506529worker1 # [ 223.355644] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514506530worker1 # [ 223.361480] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:514506531worker1 # [ 223.365371] nix-daemon[841]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6532worker1 # [ 223.507790] nix-daemon[841]: accepted connection from pid 1833, user nix-grpc-daemon (trusted)6533worker1 # [ 223.519690] nix-daemon[841]: reaped child process 1836, status = succeeded6534worker1 # [ 223.671927] nix-daemon[841]: accepted connection from pid 1838, user nix-grpc-daemon (trusted)6535worker1 # [ 223.683492] nix-daemon[841]: reaped child process 1841, status = succeeded6536lb # [ 223.698285] niks3-server[1040]: 2026/09/20 13:36:47 INFO Received uploads request method=POST path=/api/pending_closures6537worker1 # [ 223.705249] nix-grpc-daemon[1752]: time=2026-09-20T13:36:46.948Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6538worker1 # [ 223.706677] nix-grpc-daemon[1752]: time=2026-09-20T13:36:46.948Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6539lb # [ 223.723996] niks3-server[1040]: 2026/09/20 13:36:47 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6540lb # [ 223.733867] niks3-server[1040]: 2026/09/20 13:36:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6541worker1 # [ 223.738589] nix-grpc-daemon[1752]: time=2026-09-20T13:36:46.981Z level=INFO msg="Uploading 1 narinfos"6542lb # [ 223.740067] niks3-server[1040]: 2026/09/20 13:36:47 INFO Signed narinfos id=50 count=16543lb # [ 223.752747] niks3-server[1040]: 2026/09/20 13:36:47 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6544lb # [ 223.759430] niks3-server[1040]: 2026/09/20 13:36:47 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6545worker1 # [ 223.762808] nix-grpc-daemon[1752]: time=2026-09-20T13:36:47.005Z level=INFO msg="Upload complete. (391ms)"6546lb # [ 223.763867] niks3-server[1040]: 2026/09/20 13:36:47 INFO Completed upload id=506547worker1 # [ 223.764721] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51450 duration_s=0 paths=1 nar_bytes_in=5046548lb # [ 223.767514] niks3-server[1040]: 2026/09/20 13:36:47 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6549worker1 # [ 223.771296] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514506550worker1 # [ 223.772981] nix-daemon[841]: reaped child process 1831, status = succeeded6551worker1 # [ 223.776051] nix-daemon[841]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6552worker1 # [ 223.782373] nix-daemon[841]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6553worker1 # [ 225.004662] nix-daemon[841]: reaped child process 1845, status = succeeded6554worker1 # [ 225.139575] nix-daemon[841]: accepted connection from pid 1860, user nix-grpc-daemon (trusted)6555worker1 # [ 225.151267] nix-daemon[841]: reaped child process 1863, status = succeeded6556lb # [ 225.164824] niks3-server[1040]: 2026/09/20 13:36:49 INFO Received uploads request method=POST path=/api/pending_closures6557worker1 # [ 225.168392] nix-grpc-daemon[1752]: time=2026-09-20T13:36:48.411Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6558worker1 # [ 225.169851] nix-grpc-daemon[1752]: time=2026-09-20T13:36:48.412Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6559lb # [ 225.186063] niks3-server[1040]: 2026/09/20 13:36:49 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv6560lb # [ 225.188195] niks3-server[1040]: 2026/09/20 13:36:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6561lb # [ 225.190331] niks3-server[1040]: 2026/09/20 13:36:49 INFO Signed narinfos id=51 count=16562worker1 # [ 225.189295] nix-grpc-daemon[1752]: time=2026-09-20T13:36:48.432Z level=INFO msg="Uploading 1 narinfos"6563lb # [ 225.192766] niks3-server[1040]: 2026/09/20 13:36:49 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6564lb # [ 225.196213] niks3-server[1040]: 2026/09/20 13:36:49 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls6565lb # [ 225.201265] niks3-server[1040]: 2026/09/20 13:36:49 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6566lb # [ 225.203837] niks3-server[1040]: 2026/09/20 13:36:49 INFO Completed upload id=516567worker1 # [ 225.204064] nix-grpc-daemon[1752]: time=2026-09-20T13:36:48.447Z level=INFO msg="Upload complete. (214ms)"6568lb # [ 225.205526] niks3-server[1040]: 2026/09/20 13:36:49 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo6569worker1 # [ 225.207237] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51450 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=3 outputs=16570worker1 # [ 225.213383] nix-daemon[841]: reaped child process 1843, status = succeeded6571worker2 # [ 225.171368] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:49Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=16572worker2 # [ 225.180907] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:49Z level=debug event=want client=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=06573worker1 # [ 225.250243] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514506574worker1 # [ 225.257262] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514506575worker1 # [ 225.266270] nix-grpc-daemon[1752]: time=2026-09-20T13:36:48.509Z level=INFO msg="All 1 paths already cached"6576worker1 # [ 225.271729] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:514506577worker1 # [ 225.280148] nix-daemon[841]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6578worker1 # [ 225.411853] nix-daemon[841]: accepted connection from pid 1867, user nix-grpc-daemon (trusted)6579worker1 # [ 225.423585] nix-daemon[841]: reaped child process 1870, status = succeeded6580worker1 # [ 225.570499] nix-daemon[841]: accepted connection from pid 1872, user nix-grpc-daemon (trusted)6581worker1 # [ 225.582402] nix-daemon[841]: reaped child process 1875, status = succeeded6582lb # [ 225.595086] niks3-server[1040]: 2026/09/20 13:36:49 INFO Received uploads request method=POST path=/api/pending_closures6583worker1 # [ 225.598312] nix-grpc-daemon[1752]: time=2026-09-20T13:36:48.841Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6584worker1 # [ 225.599775] nix-grpc-daemon[1752]: time=2026-09-20T13:36:48.842Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"6585lb # [ 225.611550] niks3-server[1040]: 2026/09/20 13:36:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6586lb # [ 225.614148] niks3-server[1040]: 2026/09/20 13:36:49 INFO Signed narinfos id=52 count=16587worker1 # [ 225.613212] nix-grpc-daemon[1752]: time=2026-09-20T13:36:48.856Z level=INFO msg="Uploading 1 narinfos"6588lb # [ 225.616515] niks3-server[1040]: 2026/09/20 13:36:49 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst6589lb # [ 225.618577] niks3-server[1040]: 2026/09/20 13:36:49 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls6590lb # [ 225.624690] niks3-server[1040]: 2026/09/20 13:36:49 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6591lb # [ 225.627337] niks3-server[1040]: 2026/09/20 13:36:49 INFO Completed upload id=526592worker1 # [ 225.626349] nix-grpc-daemon[1752]: time=2026-09-20T13:36:48.869Z level=INFO msg="Upload complete. (341ms)"6593lb # [ 225.629872] niks3-server[1040]: 2026/09/20 13:36:49 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo6594worker1 # [ 225.628271] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51450 duration_s=0 paths=1 nar_bytes_in=6246595worker1 # [ 225.631165] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514506596worker1 # [ 225.636860] nix-daemon[841]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6597worker1 # [ 225.639978] nix-daemon[841]: reaped child process 1865, status = succeeded6598worker1 # [ 225.644861] nix-daemon[841]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6599worker1 # [ 226.659325] nix-daemon[841]: reaped child process 1879, status = succeeded6600worker1 # [ 226.793129] nix-daemon[841]: accepted connection from pid 1894, user nix-grpc-daemon (trusted)6601worker1 # [ 226.804902] nix-daemon[841]: reaped child process 1897, status = succeeded6602lb # [ 226.817711] niks3-server[1040]: 2026/09/20 13:36:50 INFO Received uploads request method=POST path=/api/pending_closures6603worker1 # [ 226.821408] nix-grpc-daemon[1752]: time=2026-09-20T13:36:50.064Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6604worker1 # [ 226.824575] nix-grpc-daemon[1752]: time=2026-09-20T13:36:50.065Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"6605lb # [ 226.839761] niks3-server[1040]: 2026/09/20 13:36:50 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv6606lb # [ 226.842423] niks3-server[1040]: 2026/09/20 13:36:50 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst6607lb # [ 226.846480] niks3-server[1040]: 2026/09/20 13:36:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6608lb # [ 226.848098] niks3-server[1040]: 2026/09/20 13:36:50 INFO Signed narinfos id=53 count=16609worker1 # [ 226.848517] nix-grpc-daemon[1752]: time=2026-09-20T13:36:50.091Z level=INFO msg="Uploading 1 narinfos"6610lb # [ 226.852783] niks3-server[1040]: 2026/09/20 13:36:50 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls6611lb # [ 226.857139] niks3-server[1040]: 2026/09/20 13:36:50 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6612lb # [ 226.860582] niks3-server[1040]: 2026/09/20 13:36:50 INFO Completed upload id=536613worker1 # [ 226.859613] nix-grpc-daemon[1752]: time=2026-09-20T13:36:50.102Z level=INFO msg="Upload complete. (207ms)"6614lb # [ 226.862424] niks3-server[1040]: 2026/09/20 13:36:50 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo6615worker1 # [ 226.861908] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51450 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=4 outputs=16616worker1 # [ 226.868529] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514506617worker1 # [ 226.871824] nix-daemon[841]: reaped child process 1877, status = succeeded6618worker2 # [ 226.830929] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:50Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=16619worker1 # [ 226.875463] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514506620worker2 # [ 226.837634] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:50Z level=debug event=want client=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=06621worker1 # [ 226.880909] nix-grpc-daemon[1752]: time=2026-09-20T13:36:50.124Z level=INFO msg="All 1 paths already cached"6622worker1 # [ 226.884697] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:514506623worker1 # [ 226.888476] nix-daemon[841]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6624worker1 # [ 227.033251] nix-daemon[841]: accepted connection from pid 1901, user nix-grpc-daemon (trusted)6625worker1 # [ 227.045474] nix-daemon[841]: reaped child process 1904, status = succeeded6626worker1 # [ 227.193609] nix-daemon[841]: accepted connection from pid 1906, user nix-grpc-daemon (trusted)6627worker1 # [ 227.205216] nix-daemon[841]: reaped child process 1909, status = succeeded6628lb # [ 227.219127] niks3-server[1040]: 2026/09/20 13:36:51 INFO Received uploads request method=POST path=/api/pending_closures6629worker1 # [ 227.222578] nix-grpc-daemon[1752]: time=2026-09-20T13:36:50.465Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6630worker1 # [ 227.224307] nix-grpc-daemon[1752]: time=2026-09-20T13:36:50.465Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"6631lb # [ 227.233031] niks3-server[1040]: 2026/09/20 13:36:51 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst6632lb # [ 227.237227] niks3-server[1040]: 2026/09/20 13:36:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6633lb # [ 227.239167] niks3-server[1040]: 2026/09/20 13:36:51 INFO Signed narinfos id=54 count=16634worker1 # [ 227.238138] nix-grpc-daemon[1752]: time=2026-09-20T13:36:50.481Z level=INFO msg="Uploading 1 narinfos"6635lb # [ 227.241846] niks3-server[1040]: 2026/09/20 13:36:51 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls6636lb # [ 227.246165] niks3-server[1040]: 2026/09/20 13:36:51 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6637lb # [ 227.249703] niks3-server[1040]: 2026/09/20 13:36:51 INFO Completed upload id=546638worker1 # [ 227.251386] nix-grpc-daemon[1752]: time=2026-09-20T13:36:50.494Z level=INFO msg="Upload complete. (357ms)"6639lb # [ 227.254483] niks3-server[1040]: 2026/09/20 13:36:51 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo6640worker1 # [ 227.253427] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51450 duration_s=0 paths=1 nar_bytes_in=6246641worker1 # [ 227.257376] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514506642worker1 # [ 227.262884] nix-daemon[841]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6643worker1 # [ 227.265063] nix-daemon[841]: reaped child process 1899, status = succeeded6644worker1 # [ 227.270162] nix-daemon[841]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6645worker1 # [ 228.284190] nix-daemon[841]: reaped child process 1913, status = succeeded6646worker1 # [ 228.408352] nix-daemon[841]: accepted connection from pid 1928, user nix-grpc-daemon (trusted)6647worker1 # [ 228.420449] nix-daemon[841]: reaped child process 1931, status = succeeded6648lb # [ 228.433602] niks3-server[1040]: 2026/09/20 13:36:52 INFO Received uploads request method=POST path=/api/pending_closures6649worker1 # [ 228.441134] nix-grpc-daemon[1752]: time=2026-09-20T13:36:51.683Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6650worker1 # [ 228.442599] nix-grpc-daemon[1752]: time=2026-09-20T13:36:51.685Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"6651lb # [ 228.470941] niks3-server[1040]: 2026/09/20 13:36:52 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv6652lb # [ 228.481766] niks3-server[1040]: 2026/09/20 13:36:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6653lb # [ 228.486612] niks3-server[1040]: 2026/09/20 13:36:52 INFO Signed narinfos id=55 count=16654worker1 # [ 228.488797] nix-grpc-daemon[1752]: time=2026-09-20T13:36:51.731Z level=INFO msg="Uploading 1 narinfos"6655lb # [ 228.495732] niks3-server[1040]: 2026/09/20 13:36:52 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst6656lb # [ 228.499486] niks3-server[1040]: 2026/09/20 13:36:52 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls6657lb # [ 228.508595] niks3-server[1040]: 2026/09/20 13:36:52 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6658lb # [ 228.515832] niks3-server[1040]: 2026/09/20 13:36:52 INFO Completed upload id=556659lb # [ 228.517310] niks3-server[1040]: 2026/09/20 13:36:52 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo6660worker1 # [ 228.516333] nix-grpc-daemon[1752]: time=2026-09-20T13:36:51.759Z level=INFO msg="Upload complete. (239ms)"6661worker2 # [ 228.484154] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:52Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=16662worker2 # [ 228.489095] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:52Z level=debug event=want client=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=06663worker1 # [ 228.523966] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51450 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=5 outputs=16664worker1 # [ 228.531357] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514506665worker1 # [ 228.540381] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514506666worker1 # [ 228.551189] nix-daemon[841]: reaped child process 1911, status = succeeded6667worker1 # [ 228.553656] nix-grpc-daemon[1752]: time=2026-09-20T13:36:51.796Z level=INFO msg="All 1 paths already cached"6668worker1 # [ 228.557360] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:514506669worker1 # [ 228.561821] nix-daemon[841]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6670worker1 # [ 228.695228] nix-daemon[841]: accepted connection from pid 1935, user nix-grpc-daemon (trusted)6671worker1 # [ 228.706907] nix-daemon[841]: reaped child process 1938, status = succeeded6672worker1 # [ 228.839344] nix-daemon[841]: accepted connection from pid 1940, user nix-grpc-daemon (trusted)6673worker1 # [ 228.850788] nix-daemon[841]: reaped child process 1943, status = succeeded6674lb # [ 228.864484] niks3-server[1040]: 2026/09/20 13:36:52 INFO Received uploads request method=POST path=/api/pending_closures6675worker1 # [ 228.868456] nix-grpc-daemon[1752]: time=2026-09-20T13:36:52.111Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6676worker1 # [ 228.870043] nix-grpc-daemon[1752]: time=2026-09-20T13:36:52.113Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"6677lb # [ 228.885227] niks3-server[1040]: 2026/09/20 13:36:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6678lb # [ 228.886964] niks3-server[1040]: 2026/09/20 13:36:52 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst6679lb # [ 228.889667] niks3-server[1040]: 2026/09/20 13:36:52 INFO Signed narinfos id=56 count=16680worker1 # [ 228.888749] nix-grpc-daemon[1752]: time=2026-09-20T13:36:52.131Z level=INFO msg="Uploading 1 narinfos"6681lb # [ 228.892811] niks3-server[1040]: 2026/09/20 13:36:52 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls6682lb # [ 228.898499] niks3-server[1040]: 2026/09/20 13:36:52 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6683lb # [ 228.902115] niks3-server[1040]: 2026/09/20 13:36:52 INFO Completed upload id=566684worker1 # [ 228.902415] nix-grpc-daemon[1752]: time=2026-09-20T13:36:52.145Z level=INFO msg="Upload complete. (336ms)"6685lb # [ 228.903883] niks3-server[1040]: 2026/09/20 13:36:52 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo6686worker1 # [ 228.903849] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51450 duration_s=0 paths=1 nar_bytes_in=6246687worker1 # [ 228.910412] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514506688worker1 # [ 228.914039] nix-daemon[841]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6689worker1 # [ 228.916212] nix-daemon[841]: reaped child process 1933, status = succeeded6690worker1 # [ 228.921385] nix-daemon[841]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6691worker1 # [ 229.947549] nix-daemon[841]: reaped child process 1947, status = succeeded6692worker1 # [ 230.096307] nix-daemon[841]: accepted connection from pid 1962, user nix-grpc-daemon (trusted)6693worker1 # [ 230.107690] nix-daemon[841]: reaped child process 1965, status = succeeded6694lb # [ 230.122337] niks3-server[1040]: 2026/09/20 13:36:54 INFO Received uploads request method=POST path=/api/pending_closures6695worker1 # [ 230.129643] nix-grpc-daemon[1752]: time=2026-09-20T13:36:53.372Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6696worker1 # [ 230.131176] nix-grpc-daemon[1752]: time=2026-09-20T13:36:53.374Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"6697lb # [ 230.160686] niks3-server[1040]: 2026/09/20 13:36:54 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst6698lb # [ 230.169294] niks3-server[1040]: 2026/09/20 13:36:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6699lb # [ 230.174905] niks3-server[1040]: 2026/09/20 13:36:54 INFO Signed narinfos id=57 count=16700worker1 # [ 230.178125] nix-grpc-daemon[1752]: time=2026-09-20T13:36:53.421Z level=INFO msg="Uploading 1 narinfos"6701lb # [ 230.184262] niks3-server[1040]: 2026/09/20 13:36:54 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv6702lb # [ 230.188492] niks3-server[1040]: 2026/09/20 13:36:54 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls6703lb # [ 230.197938] niks3-server[1040]: 2026/09/20 13:36:54 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6704lb # [ 230.202983] niks3-server[1040]: 2026/09/20 13:36:54 INFO Completed upload id=576705worker1 # [ 230.202324] nix-grpc-daemon[1752]: time=2026-09-20T13:36:53.445Z level=INFO msg="Upload complete. (273ms)"6706lb # [ 230.206166] niks3-server[1040]: 2026/09/20 13:36:54 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo6707worker2 # [ 230.173318] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:54Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=16708worker1 # [ 230.209668] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51450 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=6 outputs=16709worker1 # [ 230.216735] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514506710worker2 # [ 230.180786] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:54Z level=debug event=want client=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=06711worker1 # [ 230.226595] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514506712worker1 # [ 230.233872] nix-daemon[841]: reaped child process 1945, status = succeeded6713worker1 # [ 230.239854] nix-grpc-daemon[1752]: time=2026-09-20T13:36:53.482Z level=INFO msg="All 2 paths already cached"6714worker1 # [ 230.246191] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:514506715worker1 # [ 230.252803] nix-daemon[841]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6716worker1 # [ 230.393966] nix-daemon[841]: accepted connection from pid 1969, user nix-grpc-daemon (trusted)6717worker1 # [ 230.405801] nix-daemon[841]: reaped child process 1972, status = succeeded6718worker1 # [ 230.549723] nix-daemon[841]: accepted connection from pid 1974, user nix-grpc-daemon (trusted)6719worker1 # [ 230.562057] nix-daemon[841]: reaped child process 1977, status = succeeded6720lb # [ 230.574700] niks3-server[1040]: 2026/09/20 13:36:54 INFO Received uploads request method=POST path=/api/pending_closures6721worker1 # [ 230.582564] nix-grpc-daemon[1752]: time=2026-09-20T13:36:53.825Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6722worker1 # [ 230.584123] nix-grpc-daemon[1752]: time=2026-09-20T13:36:53.827Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"6723lb # [ 230.606927] niks3-server[1040]: 2026/09/20 13:36:54 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst6724lb # [ 230.616198] niks3-server[1040]: 2026/09/20 13:36:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6725lb # [ 230.620610] niks3-server[1040]: 2026/09/20 13:36:54 INFO Signed narinfos id=58 count=16726worker1 # [ 230.622346] nix-grpc-daemon[1752]: time=2026-09-20T13:36:53.865Z level=INFO msg="Uploading 1 narinfos"6727lb # [ 230.627764] niks3-server[1040]: 2026/09/20 13:36:54 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls6728lb # [ 230.639684] niks3-server[1040]: 2026/09/20 13:36:54 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6729worker1 # [ 230.643255] nix-grpc-daemon[1752]: time=2026-09-20T13:36:53.886Z level=INFO msg="Upload complete. (384ms)"6730lb # [ 230.644701] niks3-server[1040]: 2026/09/20 13:36:54 INFO Completed upload id=586731worker1 # [ 230.645871] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51450 duration_s=0 paths=1 nar_bytes_in=7526732lb # [ 230.648377] niks3-server[1040]: 2026/09/20 13:36:54 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo6733worker1 # [ 230.652154] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514506734worker1 # [ 230.660273] nix-daemon[841]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6735worker1 # [ 230.668602] nix-daemon[841]: reaped child process 1967, status = succeeded6736worker1 # [ 230.671638] nix-daemon[841]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6737worker1 # [ 231.686311] nix-daemon[841]: reaped child process 1981, status = succeeded6738worker1 # [ 231.817687] nix-daemon[841]: accepted connection from pid 1996, user nix-grpc-daemon (trusted)6739worker1 # [ 231.828579] nix-daemon[841]: reaped child process 1999, status = succeeded6740lb # [ 231.843642] niks3-server[1040]: 2026/09/20 13:36:55 INFO Received uploads request method=POST path=/api/pending_closures6741worker1 # [ 231.851677] nix-grpc-daemon[1752]: time=2026-09-20T13:36:55.094Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6742worker1 # [ 231.853259] nix-grpc-daemon[1752]: time=2026-09-20T13:36:55.096Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"6743lb # [ 231.884320] niks3-server[1040]: 2026/09/20 13:36:55 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst6744lb # [ 231.893113] niks3-server[1040]: 2026/09/20 13:36:55 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv6745lb # [ 231.902561] niks3-server[1040]: 2026/09/20 13:36:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6746lb # [ 231.909121] niks3-server[1040]: 2026/09/20 13:36:55 INFO Signed narinfos id=59 count=16747worker1 # [ 231.910550] nix-grpc-daemon[1752]: time=2026-09-20T13:36:55.153Z level=INFO msg="Uploading 1 narinfos"6748lb # [ 231.914652] niks3-server[1040]: 2026/09/20 13:36:55 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls6749lb # [ 231.923458] niks3-server[1040]: 2026/09/20 13:36:55 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6750lb # [ 231.930058] niks3-server[1040]: 2026/09/20 13:36:55 INFO Completed upload id=596751lb # [ 231.932563] niks3-server[1040]: 2026/09/20 13:36:55 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo6752worker1 # [ 231.929215] nix-grpc-daemon[1752]: time=2026-09-20T13:36:55.172Z level=INFO msg="Upload complete. (250ms)"6753worker2 # [ 231.897045] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:56Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=16754worker1 # [ 231.938221] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51450 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=7 outputs=16755worker1 # [ 231.944094] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514506756worker1 # [ 231.954573] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514506757worker1 # [ 231.960512] nix-daemon[841]: reaped child process 1979, status = succeeded6758worker1 # [ 231.966585] nix-grpc-daemon[1752]: time=2026-09-20T13:36:55.209Z level=INFO msg="All 2 paths already cached"6759worker1 # [ 231.972788] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:514506760worker1 # [ 231.979856] nix-daemon[841]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6761worker1 # [ 232.123305] nix-daemon[841]: accepted connection from pid 2003, user nix-grpc-daemon (trusted)6762worker1 # [ 232.135542] nix-daemon[841]: reaped child process 2006, status = succeeded6763worker1 # [ 232.287704] nix-daemon[841]: accepted connection from pid 2008, user nix-grpc-daemon (trusted)6764worker1 # [ 232.299861] nix-daemon[841]: reaped child process 2011, status = succeeded6765lb # [ 232.313710] niks3-server[1040]: 2026/09/20 13:36:56 INFO Received uploads request method=POST path=/api/pending_closures6766worker1 # [ 232.321295] nix-grpc-daemon[1752]: time=2026-09-20T13:36:55.564Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6767worker1 # [ 232.322702] nix-grpc-daemon[1752]: time=2026-09-20T13:36:55.564Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"6768lb # [ 232.340912] niks3-server[1040]: 2026/09/20 13:36:56 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst6769lb # [ 232.351900] niks3-server[1040]: 2026/09/20 13:36:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6770lb # [ 232.357493] niks3-server[1040]: 2026/09/20 13:36:56 INFO Signed narinfos id=60 count=16771worker1 # [ 232.359583] nix-grpc-daemon[1752]: time=2026-09-20T13:36:55.602Z level=INFO msg="Uploading 1 narinfos"6772lb # [ 232.362303] niks3-server[1040]: 2026/09/20 13:36:56 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls6773lb # [ 232.372718] niks3-server[1040]: 2026/09/20 13:36:56 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6774worker1 # [ 232.378372] nix-grpc-daemon[1752]: time=2026-09-20T13:36:55.621Z level=INFO msg="Upload complete. (392ms)"6775lb # [ 232.379332] niks3-server[1040]: 2026/09/20 13:36:56 INFO Completed upload id=606776worker1 # [ 232.380388] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51450 duration_s=0 paths=1 nar_bytes_in=7526777lb # [ 232.383716] niks3-server[1040]: 2026/09/20 13:36:56 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo6778worker1 # [ 232.386874] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514506779worker1 # [ 232.394386] nix-daemon[841]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6780worker1 # [ 232.396661] nix-daemon[841]: reaped child process 2001, status = succeeded6781worker1 # [ 232.401929] nix-daemon[841]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6782worker1 # [ 233.829979] nix-daemon[841]: reaped child process 2015, status = succeeded6783worker1 # [ 233.957728] nix-daemon[841]: accepted connection from pid 2030, user nix-grpc-daemon (trusted)6784worker1 # [ 233.968375] nix-daemon[841]: reaped child process 2033, status = succeeded6785lb # [ 233.979963] niks3-server[1040]: 2026/09/20 13:36:57 INFO Received uploads request method=POST path=/api/pending_closures6786worker1 # [ 233.987941] nix-grpc-daemon[1752]: time=2026-09-20T13:36:57.230Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6787worker1 # [ 233.989487] nix-grpc-daemon[1752]: time=2026-09-20T13:36:57.232Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"6788lb # [ 234.017924] niks3-server[1040]: 2026/09/20 13:36:58 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv6789lb # [ 234.030259] niks3-server[1040]: 2026/09/20 13:36:58 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst6790lb # [ 234.038242] niks3-server[1040]: 2026/09/20 13:36:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6791lb # [ 234.043424] niks3-server[1040]: 2026/09/20 13:36:58 INFO Signed narinfos id=61 count=16792worker1 # [ 234.044979] nix-grpc-daemon[1752]: time=2026-09-20T13:36:57.287Z level=INFO msg="Uploading 1 narinfos"6793lb # [ 234.045895] niks3-server[1040]: 2026/09/20 13:36:58 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls6794lb # [ 234.057021] niks3-server[1040]: 2026/09/20 13:36:58 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6795lb # [ 234.059446] niks3-server[1040]: 2026/09/20 13:36:58 INFO Completed upload id=616796lb # [ 234.062400] niks3-server[1040]: 2026/09/20 13:36:58 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo6797worker1 # [ 234.060108] nix-grpc-daemon[1752]: time=2026-09-20T13:36:57.303Z level=INFO msg="Upload complete. (248ms)"6798worker2 # [ 234.029419] nix-grpc-daemon[1368]: ts=2026-09-20T13:36:58Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=16799worker1 # [ 234.068241] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51450 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=8 outputs=16800worker1 # [ 234.080222] nix-daemon[841]: reaped child process 2013, status = succeeded6801worker1 # [ 234.109687] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514506802worker1 # [ 234.122319] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514506803worker1 # [ 234.135438] nix-grpc-daemon[1752]: time=2026-09-20T13:36:57.378Z level=INFO msg="All 2 paths already cached"6804worker1 # [ 234.142757] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:514506805worker1 # [ 234.153184] nix-daemon[841]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6806worker1 # [ 234.299917] nix-daemon[841]: accepted connection from pid 2037, user nix-grpc-daemon (trusted)6807worker1 # [ 234.312441] nix-daemon[841]: reaped child process 2040, status = succeeded6808worker1 # [ 234.447611] nix-daemon[841]: accepted connection from pid 2042, user nix-grpc-daemon (trusted)6809worker1 # [ 234.460707] nix-daemon[841]: reaped child process 2045, status = succeeded6810lb # [ 234.470654] niks3-server[1040]: 2026/09/20 13:36:58 INFO Received uploads request method=POST path=/api/pending_closures6811worker1 # [ 234.477614] nix-grpc-daemon[1752]: time=2026-09-20T13:36:57.720Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6812worker1 # [ 234.479458] nix-grpc-daemon[1752]: time=2026-09-20T13:36:57.722Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"6813lb # [ 234.504146] niks3-server[1040]: 2026/09/20 13:36:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6814lb # [ 234.509054] niks3-server[1040]: 2026/09/20 13:36:58 INFO Signed narinfos id=62 count=16815worker1 # [ 234.511168] nix-grpc-daemon[1752]: time=2026-09-20T13:36:57.754Z level=INFO msg="Uploading 1 narinfos"6816lb # [ 234.516830] niks3-server[1040]: 2026/09/20 13:36:58 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls6817lb # [ 234.523204] niks3-server[1040]: 2026/09/20 13:36:58 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst6818lb # [ 234.536416] niks3-server[1040]: 2026/09/20 13:36:58 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6819lb # [ 234.541570] niks3-server[1040]: 2026/09/20 13:36:58 INFO Completed upload id=626820worker1 # [ 234.542931] nix-grpc-daemon[1752]: time=2026-09-20T13:36:57.785Z level=INFO msg="Upload complete. (378ms)"6821lb # [ 234.544428] niks3-server[1040]: 2026/09/20 13:36:58 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo6822worker1 # [ 234.546933] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51450 duration_s=0 paths=1 nar_bytes_in=7526823worker1 # [ 234.552276] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514506824worker1 # [ 234.561813] nix-daemon[841]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6825worker1 # [ 234.567469] nix-daemon[841]: reaped child process 2035, status = succeeded6826worker1 # [ 234.572510] nix-daemon[841]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6827worker1 # [ 235.793817] nix-daemon[841]: reaped child process 2049, status = succeeded6828worker1 # [ 235.935267] nix-daemon[841]: accepted connection from pid 2064, user nix-grpc-daemon (trusted)6829worker1 # [ 235.947413] nix-daemon[841]: reaped child process 2067, status = succeeded6830lb # [ 235.962866] niks3-server[1040]: 2026/09/20 13:36:59 INFO Received uploads request method=POST path=/api/pending_closures6831worker1 # [ 235.972101] nix-grpc-daemon[1752]: time=2026-09-20T13:36:59.214Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6832worker1 # [ 235.973611] nix-grpc-daemon[1752]: time=2026-09-20T13:36:59.216Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"6833lb # [ 236.009722] niks3-server[1040]: 2026/09/20 13:37:00 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst6834lb # [ 236.014696] niks3-server[1040]: 2026/09/20 13:37:00 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv6835lb # [ 236.025484] niks3-server[1040]: 2026/09/20 13:37:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6836worker1 # [ 236.029170] nix-grpc-daemon[1752]: time=2026-09-20T13:36:59.272Z level=INFO msg="Uploading 1 narinfos"6837lb # [ 236.031201] niks3-server[1040]: 2026/09/20 13:37:00 INFO Signed narinfos id=63 count=16838lb # [ 236.035327] niks3-server[1040]: 2026/09/20 13:37:00 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls6839lb # [ 236.047825] niks3-server[1040]: 2026/09/20 13:37:00 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6840lb # [ 236.053986] niks3-server[1040]: 2026/09/20 13:37:00 INFO Completed upload id=636841worker1 # [ 236.053548] nix-grpc-daemon[1752]: time=2026-09-20T13:36:59.296Z level=INFO msg="Upload complete. (274ms)"6842lb # [ 236.056350] niks3-server[1040]: 2026/09/20 13:37:00 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo6843worker1 # [ 236.058938] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51450 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=9 outputs=16844worker1 # [ 236.072706] nix-daemon[841]: reaped child process 2047, status = succeeded6845client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 17.19 seconds)6846worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6847worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6848client: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&16849worker2 # [ 236.374688] nix-grpc-daemon[1368]: ts=2026-09-20T13:37:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:374146850worker1 # [ 236.415411] nix-grpc-daemon[1696]: ts=2026-09-20T13:36:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514506851worker2 # [ 236.401625] nix-grpc-daemon[1368]: ts=2026-09-20T13:37:00Z level=debug event=want client=2 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=06852lb # [ 236.908538] envoy[911]: conn peer=[2001:db8:1::1]:43970 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27115 tx=10660 ms=169566853client: (finished: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&1, in 30.26 seconds)6854(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 47.51 seconds)6855subtest: low disk drains a worker and builds go to the other6856worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6857lb # [ 266.909987] envoy[911]: conn peer=[2001:db8:1::1]:32924 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=895 tx=498 ms=300466858worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.99 seconds)6859worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6860worker1 # [ 267.738580] nix-grpc-daemon[1696]: ts=2026-09-20T13:37:30Z level=info event=unhealthy reason=min_free6861worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.09 seconds)6862client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&26863worker1 # [ 268.759253] nix-grpc-daemon[1696]: ts=2026-09-20T13:37:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:514506864client # don't know how to build these paths:6865client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6866worker1 # [ 268.769602] nix-grpc-daemon[1696]: ts=2026-09-20T13:37:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514506867worker2 # [ 268.751908] nix-grpc-daemon[1368]: ts=2026-09-20T13:37:32Z level=debug event=want client=3 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=06868worker2 # [ 268.754467] nix-grpc-daemon[1368]: ts=2026-09-20T13:37:32Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=16869worker2 # [ 268.795065] nix-grpc-daemon[1368]: ts=2026-09-20T13:37:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374146870worker2 # [ 268.803812] nix-grpc-daemon[1368]: ts=2026-09-20T13:37:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374146871client # copying 1 paths to 'lb:50051'...6872worker2 # [ 268.812439] nix-grpc-daemon[1368]: ts=2026-09-20T13:37:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:374146873client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...6874worker2 # [ 268.820175] nix-daemon[855]: accepted connection from pid 1368, user nix-grpc-daemon (trusted)6875worker2 # [ 268.957673] nix-daemon[855]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)6876worker2 # [ 268.974849] nix-daemon[855]: reaped child process 1633, status = succeeded6877worker2 # [ 269.129141] nix-daemon[855]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6878worker2 # [ 269.143828] nix-daemon[855]: reaped child process 1638, status = succeeded6879lb # [ 269.197194] niks3-server[1040]: 2026/09/20 13:37:33 INFO Received uploads request method=POST path=/api/pending_closures6880worker2 # [ 269.167804] nix-grpc-daemon[1436]: time=2026-09-20T13:37:33.309Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6881worker2 # [ 269.169296] nix-grpc-daemon[1436]: time=2026-09-20T13:37:33.309Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"6882lb # [ 269.230075] niks3-server[1040]: 2026/09/20 13:37:33 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst6883lb # [ 269.236688] niks3-server[1040]: 2026/09/20 13:37:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6884worker2 # [ 269.204688] nix-grpc-daemon[1436]: time=2026-09-20T13:37:33.346Z level=INFO msg="Uploading 1 narinfos"6885lb # [ 269.243122] niks3-server[1040]: 2026/09/20 13:37:33 INFO Signed narinfos id=64 count=16886lb # [ 269.251741] niks3-server[1040]: 2026/09/20 13:37:33 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls6887lb # [ 269.260520] niks3-server[1040]: 2026/09/20 13:37:33 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6888worker2 # [ 269.228294] nix-grpc-daemon[1436]: time=2026-09-20T13:37:33.370Z level=INFO msg="Upload complete. (402ms)"6889lb # [ 269.266755] niks3-server[1040]: 2026/09/20 13:37:33 INFO Completed upload id=646890worker2 # [ 269.230435] nix-grpc-daemon[1368]: ts=2026-09-20T13:37:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37414 duration_s=0 paths=1 nar_bytes_in=5126891lb # [ 269.270570] niks3-server[1040]: 2026/09/20 13:37:33 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo6892worker2 # [ 269.236484] nix-grpc-daemon[1368]: ts=2026-09-20T13:37:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374146893worker2 # [ 269.242184] nix-daemon[855]: accepted connection from pid 1368, user nix-grpc-daemon (trusted)6894worker2 # [ 269.244549] nix-daemon[855]: reaped child process 1628, status = succeeded6895worker2 # [ 269.249965] nix-daemon[855]: accepted connection from pid 1368, user nix-grpc-daemon (trusted)6896client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...6897client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6898client # farm-a> LOG-farm-a-drain6899worker2 # [ 270.879516] nix-daemon[855]: reaped child process 1642, status = succeeded6900worker2 # [ 271.035684] nix-daemon[855]: accepted connection from pid 1657, user nix-grpc-daemon (trusted)6901worker2 # [ 271.050093] nix-daemon[855]: reaped child process 1660, status = succeeded6902lb # [ 271.098883] niks3-server[1040]: 2026/09/20 13:37:35 INFO Received uploads request method=POST path=/api/pending_closures6903worker2 # [ 271.069967] nix-grpc-daemon[1436]: time=2026-09-20T13:37:35.211Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6904worker2 # [ 271.072516] nix-grpc-daemon[1436]: time=2026-09-20T13:37:35.213Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"6905lb # [ 271.134956] niks3-server[1040]: 2026/09/20 13:37:35 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6906lb # [ 271.145469] niks3-server[1040]: 2026/09/20 13:37:35 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6907lb # [ 271.156284] niks3-server[1040]: 2026/09/20 13:37:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6908lb # [ 271.161784] niks3-server[1040]: 2026/09/20 13:37:35 INFO Signed narinfos id=65 count=16909worker2 # [ 271.126351] nix-grpc-daemon[1436]: time=2026-09-20T13:37:35.268Z level=INFO msg="Uploading 1 narinfos"6910lb # [ 271.166671] niks3-server[1040]: 2026/09/20 13:37:35 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls6911lb # [ 271.176128] niks3-server[1040]: 2026/09/20 13:37:35 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6912lb # [ 271.181661] niks3-server[1040]: 2026/09/20 13:37:35 INFO Completed upload id=656913worker2 # [ 271.143254] nix-grpc-daemon[1436]: time=2026-09-20T13:37:35.285Z level=INFO msg="Upload complete. (283ms)"6914lb # [ 271.184093] niks3-server[1040]: 2026/09/20 13:37:35 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo6915worker2 # [ 271.151595] nix-grpc-daemon[1368]: ts=2026-09-20T13:37:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37414 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=10 outputs=16916worker2 # [ 271.160510] nix-grpc-daemon[1368]: ts=2026-09-20T13:37:35Z level=debug event=want client=3 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=06917worker2 # [ 271.164361] nix-grpc-daemon[1368]: ts=2026-09-20T13:37:35Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=16918worker2 # [ 271.173716] nix-daemon[855]: reaped child process 1640, status = succeeded6919worker2 # [ 271.206537] nix-grpc-daemon[1368]: ts=2026-09-20T13:37:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374146920worker2 # [ 271.218759] nix-grpc-daemon[1368]: ts=2026-09-20T13:37:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374146921worker2 # [ 271.231854] nix-grpc-daemon[1436]: time=2026-09-20T13:37:35.373Z level=INFO msg="All 1 paths already cached"6922client # copying 1 paths to 'lb:50051'...6923client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...6924worker2 # [ 271.240138] nix-grpc-daemon[1368]: ts=2026-09-20T13:37:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:374146925worker2 # [ 271.249333] nix-daemon[855]: accepted connection from pid 1368, user nix-grpc-daemon (trusted)6926worker2 # [ 271.417683] nix-daemon[855]: accepted connection from pid 1666, user nix-grpc-daemon (trusted)6927worker2 # [ 271.432703] nix-daemon[855]: reaped child process 1669, status = succeeded6928worker2 # [ 271.586239] nix-daemon[855]: accepted connection from pid 1671, user nix-grpc-daemon (trusted)6929worker2 # [ 271.599861] nix-daemon[855]: reaped child process 1674, status = succeeded6930lb # [ 271.652780] niks3-server[1040]: 2026/09/20 13:37:35 INFO Received uploads request method=POST path=/api/pending_closures6931worker2 # [ 271.622326] nix-grpc-daemon[1436]: time=2026-09-20T13:37:35.764Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6932worker2 # [ 271.623887] nix-grpc-daemon[1436]: time=2026-09-20T13:37:35.765Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"6933lb # [ 271.680186] niks3-server[1040]: 2026/09/20 13:37:35 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst6934lb # [ 271.691252] niks3-server[1040]: 2026/09/20 13:37:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6935worker2 # [ 271.659382] nix-grpc-daemon[1436]: time=2026-09-20T13:37:35.801Z level=INFO msg="Uploading 1 narinfos"6936lb # [ 271.697616] niks3-server[1040]: 2026/09/20 13:37:35 INFO Signed narinfos id=66 count=16937lb # [ 271.704278] niks3-server[1040]: 2026/09/20 13:37:35 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls6938lb # [ 271.714064] niks3-server[1040]: 2026/09/20 13:37:35 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6939worker2 # [ 271.681460] nix-grpc-daemon[1436]: time=2026-09-20T13:37:35.823Z level=INFO msg="Upload complete. (419ms)"6940lb # [ 271.719787] niks3-server[1040]: 2026/09/20 13:37:35 INFO Completed upload id=666941worker2 # [ 271.683864] nix-grpc-daemon[1368]: ts=2026-09-20T13:37:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37414 duration_s=0 paths=1 nar_bytes_in=6246942lb # [ 271.725160] niks3-server[1040]: 2026/09/20 13:37:35 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo6943worker2 # [ 271.690296] nix-grpc-daemon[1368]: ts=2026-09-20T13:37:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374146944worker2 # [ 271.696286] nix-daemon[855]: accepted connection from pid 1368, user nix-grpc-daemon (trusted)6945worker2 # [ 271.702219] nix-daemon[855]: reaped child process 1664, status = succeeded6946worker2 # [ 271.707397] nix-daemon[855]: accepted connection from pid 1368, user nix-grpc-daemon (trusted)6947client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...6948client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6949worker2 # [ 272.922599] nix-daemon[855]: reaped child process 1678, status = succeeded6950client # farm-b> LOG-farm-b-drain6951worker2 # [ 273.059274] nix-daemon[855]: accepted connection from pid 1693, user nix-grpc-daemon (trusted)6952worker2 # [ 273.073407] nix-daemon[855]: reaped child process 1696, status = succeeded6953lb # [ 273.127376] niks3-server[1040]: 2026/09/20 13:37:37 INFO Received uploads request method=POST path=/api/pending_closures6954worker2 # [ 273.097408] nix-grpc-daemon[1436]: time=2026-09-20T13:37:37.239Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6955worker2 # [ 273.098901] nix-grpc-daemon[1436]: time=2026-09-20T13:37:37.240Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"6956lb # [ 273.165887] niks3-server[1040]: 2026/09/20 13:37:37 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst6957lb # [ 273.178163] niks3-server[1040]: 2026/09/20 13:37:37 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6958lb # [ 273.182428] niks3-server[1040]: 2026/09/20 13:37:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6959worker2 # [ 273.151834] nix-grpc-daemon[1436]: time=2026-09-20T13:37:37.293Z level=INFO msg="Uploading 1 narinfos"6960lb # [ 273.189815] niks3-server[1040]: 2026/09/20 13:37:37 INFO Signed narinfos id=67 count=16961lb # [ 273.195577] niks3-server[1040]: 2026/09/20 13:37:37 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls6962lb # [ 273.205387] niks3-server[1040]: 2026/09/20 13:37:37 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6963lb # [ 273.210430] niks3-server[1040]: 2026/09/20 13:37:37 INFO Completed upload id=676964lb # [ 273.211915] niks3-server[1040]: 2026/09/20 13:37:37 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo6965worker2 # [ 273.173968] nix-grpc-daemon[1436]: time=2026-09-20T13:37:37.315Z level=INFO msg="Upload complete. (259ms)"6966worker2 # [ 273.183628] nix-grpc-daemon[1368]: ts=2026-09-20T13:37:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37414 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=11 outputs=16967worker2 # [ 273.191428] nix-grpc-daemon[1368]: ts=2026-09-20T13:37:37Z level=debug event=want client=3 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=06968worker2 # [ 273.195529] nix-grpc-daemon[1368]: ts=2026-09-20T13:37:37Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=16969worker2 # [ 273.205261] nix-daemon[855]: reaped child process 1676, status = succeeded6970worker2 # [ 273.235736] nix-grpc-daemon[1368]: ts=2026-09-20T13:37:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374146971worker2 # [ 273.248161] nix-grpc-daemon[1368]: ts=2026-09-20T13:37:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374146972worker2 # [ 273.260609] nix-grpc-daemon[1436]: time=2026-09-20T13:37:37.402Z level=INFO msg="All 2 paths already cached"6973client # copying 1 paths to 'lb:50051'...6974client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...6975worker2 # [ 273.269189] nix-grpc-daemon[1368]: ts=2026-09-20T13:37:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:374146976worker2 # [ 273.279341] nix-daemon[855]: accepted connection from pid 1368, user nix-grpc-daemon (trusted)6977worker2 # [ 273.423835] nix-daemon[855]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6978worker2 # [ 273.438906] nix-daemon[855]: reaped child process 1705, status = succeeded6979worker2 # [ 273.577458] nix-daemon[855]: accepted connection from pid 1707, user nix-grpc-daemon (trusted)6980worker2 # [ 273.591689] nix-daemon[855]: reaped child process 1710, status = succeeded6981lb # [ 273.643333] niks3-server[1040]: 2026/09/20 13:37:37 INFO Received uploads request method=POST path=/api/pending_closures6982worker2 # [ 273.614799] nix-grpc-daemon[1436]: time=2026-09-20T13:37:37.756Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6983worker2 # [ 273.616236] nix-grpc-daemon[1436]: time=2026-09-20T13:37:37.756Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"6984lb # [ 273.669595] niks3-server[1040]: 2026/09/20 13:37:37 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst6985lb # [ 273.680869] niks3-server[1040]: 2026/09/20 13:37:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6986worker2 # [ 273.650501] nix-grpc-daemon[1436]: time=2026-09-20T13:37:37.792Z level=INFO msg="Uploading 1 narinfos"6987lb # [ 273.688761] niks3-server[1040]: 2026/09/20 13:37:37 INFO Signed narinfos id=68 count=16988lb # [ 273.693661] niks3-server[1040]: 2026/09/20 13:37:37 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls6989lb # [ 273.703806] niks3-server[1040]: 2026/09/20 13:37:37 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6990worker2 # [ 273.672665] nix-grpc-daemon[1436]: time=2026-09-20T13:37:37.814Z level=INFO msg="Upload complete. (382ms)"6991lb # [ 273.711067] niks3-server[1040]: 2026/09/20 13:37:37 INFO Completed upload id=686992worker2 # [ 273.674926] nix-grpc-daemon[1368]: ts=2026-09-20T13:37:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37414 duration_s=0 paths=1 nar_bytes_in=7526993lb # [ 273.714632] niks3-server[1040]: 2026/09/20 13:37:37 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo6994worker2 # [ 273.681474] nix-grpc-daemon[1368]: ts=2026-09-20T13:37:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374146995worker2 # [ 273.688266] nix-daemon[855]: accepted connection from pid 1368, user nix-grpc-daemon (trusted)6996worker2 # [ 273.690680] nix-daemon[855]: reaped child process 1700, status = succeeded6997worker2 # [ 273.696076] nix-daemon[855]: accepted connection from pid 1368, user nix-grpc-daemon (trusted)6998client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...6999client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7000client # farm-top> LOG-farm-top-drain7001worker2 # [ 274.725261] nix-daemon[855]: reaped child process 1714, status = succeeded7002worker2 # [ 274.883712] nix-daemon[855]: accepted connection from pid 1729, user nix-grpc-daemon (trusted)7003worker2 # [ 274.896241] nix-daemon[855]: reaped child process 1732, status = succeeded7004lb # [ 274.949814] niks3-server[1040]: 2026/09/20 13:37:38 INFO Received uploads request method=POST path=/api/pending_closures7005worker2 # [ 274.920525] nix-grpc-daemon[1436]: time=2026-09-20T13:37:39.062Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7006worker2 # [ 274.922097] nix-grpc-daemon[1436]: time=2026-09-20T13:37:39.064Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"7007lb # [ 274.987632] niks3-server[1040]: 2026/09/20 13:37:38 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7008lb # [ 274.994612] niks3-server[1040]: 2026/09/20 13:37:38 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7009lb # [ 275.008423] niks3-server[1040]: 2026/09/20 13:37:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign7010lb # [ 275.012886] niks3-server[1040]: 2026/09/20 13:37:39 INFO Signed narinfos id=69 count=17011worker2 # [ 274.979246] nix-grpc-daemon[1436]: time=2026-09-20T13:37:39.118Z level=INFO msg="Uploading 1 narinfos"7012lb # [ 275.017497] niks3-server[1040]: 2026/09/20 13:37:39 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7013lb # [ 275.028983] niks3-server[1040]: 2026/09/20 13:37:39 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete7014lb # [ 275.033715] niks3-server[1040]: 2026/09/20 13:37:39 INFO Completed upload id=697015worker2 # [ 274.995610] nix-grpc-daemon[1436]: time=2026-09-20T13:37:39.137Z level=INFO msg="Upload complete. (291ms)"7016lb # [ 275.036154] niks3-server[1040]: 2026/09/20 13:37:39 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7017worker2 # [ 275.003444] nix-grpc-daemon[1368]: ts=2026-09-20T13:37:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37414 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=12 outputs=17018worker2 # [ 275.017265] nix-daemon[855]: reaped child process 1712, status = succeeded7019client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&2, in 6.53 seconds)7020client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath7021client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath, in 0.19 seconds)7022worker1: must succeed: rm /nix/.rw-store/fill7023worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.13 seconds)7024worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7025worker1 # [ 275.741807] nix-grpc-daemon[1696]: ts=2026-09-20T13:37:38Z level=info event=healthy reason=min_free7026worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.12 seconds)7027(finished: subtest: low disk drains a worker and builds go to the other, in 10.12 seconds)7028(finished: run the VM test script, in 277.50 seconds)7029test script finished in 277.72s7030cleanup7031kill QemuMachine (pid 50)7032client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7033lb # [ 276.910662] envoy[911]: conn peer=[2001:db8:1::1]:52000 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9516 tx=3809 ms=62967034kill QemuMachine (pid 49)7035lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7036kill QemuMachine (pid 51)7037worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7038kill QemuMachine (pid 52)7039worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7040(finished: cleanup, in 0.97 seconds)7041additionally exposed symbols:7042 client, lb, worker1, worker2,7043 vlan1,7044 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh