vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #301
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker1 # Disk image does not exist, creating the virtualisation disk image...13worker1 # Formatting '/build/vm-state-worker1/tmp.kmLUpmOZaJ', fmt=raw size=107374182414lb: QEMU running (pid 50)15worker1 # mke2fs 1.47.4 (6-Mar-2025)16worker1: QEMU running (pid 51)17worker1 # Discarding device blocks: 0/262144 done18lb # Disk image does not exist, creating the virtualisation disk image...19worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes20lb # Formatting '/build/vm-state-lb/tmp.8uApV1Ox8u', fmt=raw size=107374182421worker1 # Filesystem UUID: 6272a74b-6e08-443c-9df6-f1c5107e85cc22lb # mke2fs 1.47.4 (6-Mar-2025)23worker1 # Superblock backups stored on blocks:24lb # Discarding device blocks: 0/262144 done25client # Disk image does not exist, creating the virtualisation disk image...26lb # Creating filesystem with 262144 4k blocks and 65536 inodes27client # Formatting '/build/vm-state-client/tmp.nck1W1jbTO', fmt=raw size=107374182428lb # Filesystem UUID: 2f7b137c-1b6c-4e85-82ac-d6a601e4fc0e29worker1 # 32768, 98304, 163840, 22937630client # mke2fs 1.47.4 (6-Mar-2025)31client: QEMU running (pid 49)32worker2 # Disk image does not exist, creating the virtualisation disk image...33worker2: QEMU running (pid 52)34lb # Superblock backups stored on blocks:35worker2 # Formatting '/build/vm-state-worker2/tmp.WRo1gGnmDZ', fmt=raw size=107374182436worker1 # 37client # Discarding device blocks: 0/262144 done38lb # 32768, 98304, 163840, 22937639worker2 # mke2fs 1.47.4 (6-Mar-2025)40client # Creating filesystem with 262144 4k blocks and 65536 inodes41lb # 42lb # Allocating group tables: 0/8 done43lb # Writing inode tables: 0/8 done44(finished: start all VMs, in 0.22 seconds)45client # Filesystem UUID: b18bc253-14ab-4c61-a333-fff67ad4222946lb # Creating journal (8192 blocks): done47client: waiting for unit network-addresses-eth1.service48client # Superblock backups stored on blocks:49lb # Writing superblocks and filesystem accounting information: 0/8 done50client: waiting for the VM to finish booting51client # 32768, 98304, 163840, 22937652lb # 53worker1 # Allocating group tables: 0/8 done54worker2 # Discarding device blocks: 0/262144 done55client # 56lb # Virtualisation disk image created.57worker1 # Writing inode tables: 0/8 done58worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes59worker1 # Creating journal (8192 blocks): done60worker2 # Filesystem UUID: 7ed55d24-b19d-4478-ba55-3d2a753afb3a61worker1 # Writing superblocks and filesystem accounting information: 0/8 done62worker2 # Superblock backups stored on blocks:63worker1 # 64worker2 # 32768, 98304, 163840, 22937665worker1 # Virtualisation disk image created.66worker2 # 67client # Allocating group tables: 0/8 done68worker2 # Allocating group tables: 0/8 done69client # Writing inode tables: 0/8 done70worker2 # Writing inode tables: 0/8 done71client # Creating journal (8192 blocks): done72worker2 # Creating journal (8192 blocks): done73client # Writing superblocks and filesystem accounting information: 0/8 done74worker2 # Writing superblocks and filesystem accounting information: 0/8 done75client # 76worker2 # 77client # Virtualisation disk image created.78worker2 # Virtualisation disk image created.79worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)80client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)81lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)82worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)83worker1 # 84worker1 # 85worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0086worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...87worker1 # 88worker1 # 89worker1 # 90worker1 # 91worker1 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB0092client # 93client # 94client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0095worker1 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...96worker1 # 97worker1 # 98lb # 99lb # 100client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...101client # 102client # 103client # 104client # 105lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00106client # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00107lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...108lb # 109lb # 110lb # 111lb # 112lb # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00113client # Press Ctrl-B to configure iPXE (PCI 00:08.0)...114client # 115client # 116worker2 # 117worker2 # 118worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00119lb # Press Ctrl-B to configure iPXE (PCI 00:08.0)...120lb # 121lb # 122worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...123worker2 # 124worker2 # 125worker2 # 126worker2 # 127worker2 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00128worker2 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...129worker2 # 130worker2 # 131worker1 # Booting from ROM...132client # Booting from ROM...133lb # Booting from ROM...134worker2 # Booting from ROM...135worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026136worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/9vpb52xhb70b1gg4gv26lpf9869r95q7-nixos-system-worker1-test/init regInfo=/nix/store/az75mwmf4lqycv6f4hmfwc0nayrgxhwf-closure-info/registration console=ttyS0,115200n8 console=tty0137worker1 # [ 0.000000] BIOS-provided physical RAM map:138worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable139worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved140worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved141worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable142worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved143worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved144worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved145worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved146client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026147worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved148worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved149worker1 # [ 0.000000] NX (Execute Disable) protection: active150worker1 # [ 0.000000] APIC: Static calls initialized151worker1 # [ 0.000000] SMBIOS 2.8 present.152client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/bgb9qwzhklrxfjy4awx25fzniv721g1y-nixos-system-client-test/init regInfo=/nix/store/941m2hwqpcjqiysnxdfsqwdw595s87vk-closure-info/registration console=ttyS0,115200n8 console=tty0153worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014154client # [ 0.000000] BIOS-provided physical RAM map:155worker1 # [ 0.000000] DMI: Memory slots populated: 1/1156worker1 # [ 0.000000] Hypervisor detected: KVM157client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable158worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000159client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved160worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00161client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved162worker1 # [ 0.000001] kvm-clock: using sched offset of 449291314 cycles163client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable164worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns165client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved166lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026167worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor168client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved169worker1 # [ 0.000805] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000170client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved171worker1 # [ 0.000832] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs172client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved173worker1 # [ 0.000834] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT174client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved175worker1 # [ 0.002733] found SMP MP-table at [mem 0x000f5450-0x000f545f]176lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/xzrwmx591i25vlijx1m7lk76hjmbxra6-nixos-system-lb-test/init regInfo=/nix/store/5b1jdrkcd54zycf7j131lh8wai3gzg44-closure-info/registration console=ttyS0,115200n8 console=tty0177lb # [ 0.000000] BIOS-provided physical RAM map:178worker1 # [ 0.002744] Using GB pages for direct mapping179client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved180worker1 # [ 0.002894] RAMDISK: [mem 0x5e367000-0x5ffcffff]181client # [ 0.000000] NX (Execute Disable) protection: active182lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable183client # [ 0.000000] APIC: Static calls initialized184worker1 # [ 0.002904] ACPI: Early table checksum verification disabled185client # [ 0.000000] SMBIOS 2.8 present.186lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved187worker1 # [ 0.002906] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )188lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved189client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014190worker1 # [ 0.002910] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)191lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable192client # [ 0.000000] DMI: Memory slots populated: 1/1193client # [ 0.000000] Hypervisor detected: KVM194worker1 # [ 0.002914] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)195lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved196client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000197lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved198worker1 # [ 0.002921] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)199client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00200worker1 # [ 0.002923] ACPI: FACS 0x000000005FFE0000 000040201client # [ 0.000000] kvm-clock: using sched offset of 453918911 cycles202lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved203worker1 # [ 0.002925] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)204lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved205client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns206client # [ 0.000005] tsc: Detected 2400.012 MHz processor207lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved208worker1 # [ 0.002926] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)209worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026210client # [ 0.000812] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000211lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved212worker1 # [ 0.002928] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)213lb # [ 0.000000] NX (Execute Disable) protection: active214client # [ 0.000838] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs215lb # [ 0.000000] APIC: Static calls initialized216worker1 # [ 0.002929] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)217lb # [ 0.000000] SMBIOS 2.8 present.218client # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT219worker1 # [ 0.002931] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]220client # [ 0.002732] found SMP MP-table at [mem 0x000f5450-0x000f545f]221lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014222client # [ 0.002743] Using GB pages for direct mapping223worker1 # [ 0.002932] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]224lb # [ 0.000000] DMI: Memory slots populated: 1/1225worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/g6bjvn1fpihxjl3rkk64li3qwyfil5lr-nixos-system-worker2-test/init regInfo=/nix/store/mddi5sa1bwc5cd417i20xm8sk5njknhg-closure-info/registration console=ttyS0,115200n8 console=tty0226client # [ 0.002899] RAMDISK: [mem 0x5e366000-0x5ffcffff]227lb # [ 0.000000] Hypervisor detected: KVM228worker2 # [ 0.000000] BIOS-provided physical RAM map:229client # [ 0.002908] ACPI: Early table checksum verification disabled230lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000231worker1 # [ 0.002932] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]232worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable233worker1 # [ 0.002933] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]234lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00235client # [ 0.002910] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )236worker1 # [ 0.002933] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]237lb # [ 0.000000] kvm-clock: using sched offset of 480142271 cycles238worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved239client # [ 0.002914] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)240worker1 # [ 0.002934] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]241worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved242client # [ 0.002918] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)243lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns244worker1 # [ 0.002934] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]245worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable246lb # [ 0.000004] tsc: Detected 2400.012 MHz processor247worker1 # [ 0.003172] No NUMA configuration found248client # [ 0.002926] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)249lb # [ 0.000803] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000250worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved251client # [ 0.002928] ACPI: FACS 0x000000005FFE0000 000040252worker1 # [ 0.003173] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]253worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved254lb # [ 0.000830] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs255worker1 # [ 0.003175] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]256worker1 # [ 0.005333] Zone ranges:257client # [ 0.002930] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)258worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved259lb # [ 0.000832] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT260worker1 # [ 0.005334] DMA [mem 0x0000000000001000-0x0000000000ffffff]261client # [ 0.002932] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)262worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved263lb # [ 0.002724] found SMP MP-table at [mem 0x000f5450-0x000f545f]264worker1 # [ 0.005336] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]265lb # [ 0.002735] Using GB pages for direct mapping266worker1 # [ 0.005337] Normal empty267worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved268client # [ 0.002934] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)269worker1 # [ 0.005338] Device empty270lb # [ 0.002804] RAMDISK: [mem 0x5e367000-0x5ffcffff]271worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved272worker1 # [ 0.005338] Movable zone start for each node273lb # [ 0.002812] ACPI: Early table checksum verification disabled274client # [ 0.002935] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)275worker1 # [ 0.005339] Early memory node ranges276worker2 # [ 0.000000] NX (Execute Disable) protection: active277lb # [ 0.002815] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )278worker2 # [ 0.000000] APIC: Static calls initialized279worker1 # [ 0.005339] node 0: [mem 0x0000000000001000-0x000000000009efff]280client # [ 0.002937] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]281worker2 # [ 0.000000] SMBIOS 2.8 present.282lb # [ 0.002819] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)283worker1 # [ 0.005340] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]284client # [ 0.002939] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]285worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014286worker1 # [ 0.005341] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]287lb # [ 0.002824] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)288client # [ 0.002939] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]289worker2 # [ 0.000000] DMI: Memory slots populated: 1/1290worker1 # [ 0.005359] On node 0, zone DMA: 1 pages in unavailable ranges291worker2 # [ 0.000000] Hypervisor detected: KVM292client # [ 0.002940] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]293lb # [ 0.002830] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)294worker1 # [ 0.005606] On node 0, zone DMA: 97 pages in unavailable ranges295worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000296lb # [ 0.002832] ACPI: FACS 0x000000005FFE0000 000040297client # [ 0.002940] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]298worker1 # [ 0.030731] On node 0, zone DMA32: 40 pages in unavailable ranges299worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00300client # [ 0.002941] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]301lb # [ 0.002834] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)302worker1 # [ 0.031195] ACPI: PM-Timer IO Port: 0x608303worker2 # [ 0.000000] kvm-clock: using sched offset of 489715112 cycles304worker1 # [ 0.031206] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])305client # [ 0.002941] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]306lb # [ 0.002835] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)307client # [ 0.003169] No NUMA configuration found308worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns309worker1 # [ 0.031234] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23310worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor311client # [ 0.003169] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]312lb # [ 0.002837] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)313worker1 # [ 0.031237] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)314worker2 # [ 0.000816] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000315client # [ 0.003172] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]316worker1 # [ 0.031238] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)317lb # [ 0.002838] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)318client # [ 0.005284] Zone ranges:319worker2 # [ 0.000843] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs320worker1 # [ 0.031239] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)321lb # [ 0.002840] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]322client # [ 0.005285] DMA [mem 0x0000000000001000-0x0000000000ffffff]323worker2 # [ 0.000845] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT324client # [ 0.005286] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]325worker1 # [ 0.031240] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)326lb # [ 0.002841] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]327client # [ 0.005287] Normal empty328client # [ 0.005288] Device empty329worker2 # [ 0.002747] found SMP MP-table at [mem 0x000f5450-0x000f545f]330worker1 # [ 0.031241] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)331lb # [ 0.002841] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]332client # [ 0.005288] Movable zone start for each node333worker2 # [ 0.002757] Using GB pages for direct mapping334client # [ 0.005289] Early memory node ranges335worker1 # [ 0.031243] ACPI: Using ACPI (MADT) for SMP configuration information336worker2 # [ 0.002908] RAMDISK: [mem 0x5e367000-0x5ffcffff]337lb # [ 0.002842] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]338worker1 # [ 0.031244] ACPI: HPET id: 0x8086a201 base: 0xfed00000339worker2 # [ 0.002916] ACPI: Early table checksum verification disabled340client # [ 0.005289] node 0: [mem 0x0000000000001000-0x000000000009efff]341lb # [ 0.002842] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]342worker1 # [ 0.031247] TSC deadline timer available343worker2 # [ 0.002918] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )344client # [ 0.005291] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]345worker1 # [ 0.031251] CPU topo: Max. logical packages: 1346lb # [ 0.002843] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]347worker1 # [ 0.031252] CPU topo: Max. logical dies: 1348client # [ 0.005292] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]349worker2 # [ 0.002922] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)350lb # [ 0.002843] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]351worker1 # [ 0.031253] CPU topo: Max. dies per package: 1352client # [ 0.005314] On node 0, zone DMA: 1 pages in unavailable ranges353lb # [ 0.003070] No NUMA configuration found354worker1 # [ 0.031256] CPU topo: Max. threads per core: 1355worker2 # [ 0.002926] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)356client # [ 0.005567] On node 0, zone DMA: 97 pages in unavailable ranges357worker1 # [ 0.031257] CPU topo: Num. cores per package: 1358lb # [ 0.003070] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]359worker1 # [ 0.031257] CPU topo: Num. threads per package: 1360client # [ 0.030572] On node 0, zone DMA32: 40 pages in unavailable ranges361worker2 # [ 0.002933] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)362lb # [ 0.003073] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]363client # [ 0.031025] ACPI: PM-Timer IO Port: 0x608364worker2 # [ 0.002935] ACPI: FACS 0x000000005FFE0000 000040365lb # [ 0.005215] Zone ranges:366worker1 # [ 0.031257] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs367client # [ 0.031037] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])368lb # [ 0.005216] DMA [mem 0x0000000000001000-0x0000000000ffffff]369worker1 # [ 0.031275] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()370worker2 # [ 0.002936] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)371client # [ 0.031068] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23372lb # [ 0.005218] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]373lb # [ 0.005219] Normal empty374worker1 # [ 0.031306] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]375worker2 # [ 0.002938] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)376lb # [ 0.005219] Device empty377client # [ 0.031072] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)378lb # [ 0.005220] Movable zone start for each node379worker1 # [ 0.031307] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]380client # [ 0.031074] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)381lb # [ 0.005221] Early memory node ranges382worker2 # [ 0.002939] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)383worker1 # [ 0.031309] [mem 0x60000000-0xafffffff] available for PCI devices384lb # [ 0.005221] node 0: [mem 0x0000000000001000-0x000000000009efff]385client # [ 0.031076] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)386worker1 # [ 0.031310] Booting paravirtualized kernel on KVM387worker2 # [ 0.002941] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)388client # [ 0.031077] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)389lb # [ 0.005222] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]390worker2 # [ 0.002942] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]391worker1 # [ 0.031313] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns392lb # [ 0.005223] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]393client # [ 0.031077] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)394worker2 # [ 0.002943] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]395lb # [ 0.005240] On node 0, zone DMA: 1 pages in unavailable ranges396client # [ 0.031080] ACPI: Using ACPI (MADT) for SMP configuration information397worker1 # [ 0.035760] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1398worker2 # [ 0.002944] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]399lb # [ 0.005486] On node 0, zone DMA: 97 pages in unavailable ranges400client # [ 0.031081] ACPI: HPET id: 0x8086a201 base: 0xfed00000401worker1 # [ 0.037814] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152402client # [ 0.031086] TSC deadline timer available403worker2 # [ 0.002944] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]404lb # [ 0.032499] On node 0, zone DMA32: 40 pages in unavailable ranges405worker1 # [ 0.037857] kvm-guest: PV spinlocks disabled, single CPU406client # [ 0.031090] CPU topo: Max. logical packages: 1407lb # [ 0.032972] ACPI: PM-Timer IO Port: 0x608408worker2 # [ 0.002945] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]409client # [ 0.031091] CPU topo: Max. logical dies: 1410lb # [ 0.032983] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])411client # [ 0.031091] CPU topo: Max. dies per package: 1412worker2 # [ 0.002945] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]413client # [ 0.031095] CPU topo: Max. threads per core: 1414lb # [ 0.033009] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23415worker2 # [ 0.002946] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]416client # [ 0.031096] CPU topo: Num. cores per package: 1417lb # [ 0.033012] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)418worker2 # [ 0.003172] No NUMA configuration found419client # [ 0.031096] CPU topo: Num. threads per package: 1420lb # [ 0.033014] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)421worker2 # [ 0.003173] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]422client # [ 0.031096] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs423worker1 # [ 0.037858] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/9vpb52xhb70b1gg4gv26lpf9869r95q7-nixos-system-worker1-test/init regInfo=/nix/store/az75mwmf4lqycv6f4hmfwc0nayrgxhwf-closure-info/registration console=ttyS0,115200n8 console=tty0424lb # [ 0.033015] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)425worker2 # [ 0.003175] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]426client # [ 0.031114] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()427worker2 # [ 0.005726] Zone ranges:428lb # [ 0.033016] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)429worker2 # [ 0.005727] DMA [mem 0x0000000000001000-0x0000000000ffffff]430worker1 # [ 0.037947] Unknown kernel command line parameters "regInfo=/nix/store/az75mwmf4lqycv6f4hmfwc0nayrgxhwf-closure-info/registration", will be passed to user space.431client # [ 0.031144] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]432lb # [ 0.033016] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)433worker1 # [ 0.037960] random: crng init done434worker2 # [ 0.005729] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]435client # [ 0.031146] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]436worker2 # [ 0.005730] Normal empty437lb # [ 0.033019] ACPI: Using ACPI (MADT) for SMP configuration information438worker2 # [ 0.005731] Device empty439worker1 # [ 0.037961] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes440client # [ 0.031148] [mem 0x60000000-0xafffffff] available for PCI devices441lb # [ 0.033020] ACPI: HPET id: 0x8086a201 base: 0xfed00000442worker2 # [ 0.005731] Movable zone start for each node443client # [ 0.031149] Booting paravirtualized kernel on KVM444worker2 # [ 0.005732] Early memory node ranges445lb # [ 0.033024] TSC deadline timer available446worker1 # [ 0.040227] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)447lb # [ 0.033028] CPU topo: Max. logical packages: 1448worker2 # [ 0.005732] node 0: [mem 0x0000000000001000-0x000000000009efff]449worker1 # [ 0.041245] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)450lb # [ 0.033029] CPU topo: Max. logical dies: 1451client # [ 0.031152] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns452worker1 # [ 0.041275] Fallback order for Node 0: 0453worker2 # [ 0.005733] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]454lb # [ 0.033029] CPU topo: Max. dies per package: 1455client # [ 0.035606] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1456lb # [ 0.033033] CPU topo: Max. threads per core: 1457worker1 # [ 0.041277] Built 1 zonelists, mobility grouping on. Total pages: 393078458worker2 # [ 0.005734] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]459worker1 # [ 0.041278] Policy zone: DMA32460lb # [ 0.033034] CPU topo: Num. cores per package: 1461client # [ 0.037653] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152462worker2 # [ 0.005756] On node 0, zone DMA: 1 pages in unavailable ranges463lb # [ 0.033034] CPU topo: Num. threads per package: 1464worker1 # [ 0.043649] mem auto-init: stack:all(zero), heap alloc:on, heap free:off465client # [ 0.037693] kvm-guest: PV spinlocks disabled, single CPU466worker2 # [ 0.006059] On node 0, zone DMA: 97 pages in unavailable ranges467lb # [ 0.033035] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs468worker1 # [ 0.046612] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1469worker2 # [ 0.032250] On node 0, zone DMA32: 40 pages in unavailable ranges470worker1 # [ 0.049780] allocated 3145728 bytes of page_ext471worker2 # [ 0.032731] ACPI: PM-Timer IO Port: 0x608472lb # [ 0.033055] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()473worker1 # [ 0.059694] ftrace: allocating 48731 entries in 192 pages474worker2 # [ 0.032743] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])475lb # [ 0.033087] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]476worker1 # [ 0.059696] ftrace: allocated 192 pages with 2 groups477worker2 # [ 0.032772] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23478worker1 # [ 0.060506] Dynamic Preempt: lazy479lb # [ 0.033088] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]480worker1 # [ 0.060647] rcu: Preemptible hierarchical RCU implementation.481worker2 # [ 0.032775] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)482client # [ 0.037694] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/bgb9qwzhklrxfjy4awx25fzniv721g1y-nixos-system-client-test/init regInfo=/nix/store/941m2hwqpcjqiysnxdfsqwdw595s87vk-closure-info/registration console=ttyS0,115200n8 console=tty0483lb # [ 0.033090] [mem 0x60000000-0xafffffff] available for PCI devices484worker1 # [ 0.060648] rcu: RCU event tracing is enabled.485worker2 # [ 0.032777] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)486lb # [ 0.033091] Booting paravirtualized kernel on KVM487worker1 # [ 0.060649] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.488worker2 # [ 0.032778] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)489client # [ 0.037787] Unknown kernel command line parameters "regInfo=/nix/store/941m2hwqpcjqiysnxdfsqwdw595s87vk-closure-info/registration", will be passed to user space.490worker1 # [ 0.060650] Trampoline variant of Tasks RCU enabled.491lb # [ 0.033095] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns492client # [ 0.037800] random: crng init done493worker2 # [ 0.032779] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)494worker1 # [ 0.060650] Rude variant of Tasks RCU enabled.495worker1 # [ 0.060651] Tracing variant of Tasks RCU enabled.496lb # [ 0.037559] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1497client # [ 0.037801] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes498worker2 # [ 0.032779] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)499lb # [ 0.039774] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152500worker2 # [ 0.032782] ACPI: Using ACPI (MADT) for SMP configuration information501worker1 # [ 0.060651] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.502client # [ 0.039818] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)503worker2 # [ 0.032783] ACPI: HPET id: 0x8086a201 base: 0xfed00000504lb # [ 0.039814] kvm-guest: PV spinlocks disabled, single CPU505worker1 # [ 0.060652] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1506client # [ 0.040972] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)507worker2 # [ 0.032787] TSC deadline timer available508client # [ 0.041003] Fallback order for Node 0: 0509worker2 # [ 0.032791] CPU topo: Max. logical packages: 1510worker1 # [ 0.060672] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.511worker2 # [ 0.032791] CPU topo: Max. logical dies: 1512client # [ 0.041005] Built 1 zonelists, mobility grouping on. Total pages: 393078513client # [ 0.041006] Policy zone: DMA32514worker2 # [ 0.032792] CPU topo: Max. dies per package: 1515worker1 # [ 0.060674] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.516worker2 # [ 0.032795] CPU topo: Max. threads per core: 1517client # [ 0.043368] mem auto-init: stack:all(zero), heap alloc:on, heap free:off518worker2 # [ 0.032796] CPU topo: Num. cores per package: 1519worker1 # [ 0.060675] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.520lb # [ 0.039816] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/xzrwmx591i25vlijx1m7lk76hjmbxra6-nixos-system-lb-test/init regInfo=/nix/store/5b1jdrkcd54zycf7j131lh8wai3gzg44-closure-info/registration console=ttyS0,115200n8 console=tty0521client # [ 0.046455] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1522worker2 # [ 0.032796] CPU topo: Num. threads per package: 1523worker1 # [ 0.065029] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16524client # [ 0.049592] allocated 3145728 bytes of page_ext525worker2 # [ 0.032796] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs526client # [ 0.059116] ftrace: allocating 48731 entries in 192 pages527worker1 # [ 0.065299] rcu: srcu_init: Setting srcu_struct sizes based on contention.528lb # [ 0.039915] Unknown kernel command line parameters "regInfo=/nix/store/5b1jdrkcd54zycf7j131lh8wai3gzg44-closure-info/registration", will be passed to user space.529worker2 # [ 0.032815] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()530client # [ 0.059118] ftrace: allocated 192 pages with 2 groups531lb # [ 0.039928] random: crng init done532client # [ 0.059912] Dynamic Preempt: lazy533worker1 # [ 0.065305] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns534worker2 # [ 0.032845] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]535client # [ 0.060037] rcu: Preemptible hierarchical RCU implementation.536lb # [ 0.039929] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes537client # [ 0.060038] rcu: RCU event tracing is enabled.538worker2 # [ 0.032847] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]539worker1 # [ 0.065406] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)540lb # [ 0.041938] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)541worker1 # [ 0.068928] Console: colour VGA+ 80x25542client # [ 0.060038] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.543worker2 # [ 0.032848] [mem 0x60000000-0xafffffff] available for PCI devices544worker1 # [ 0.068931] printk: legacy console [tty0] enabled545lb # [ 0.042943] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)546client # [ 0.060039] Trampoline variant of Tasks RCU enabled.547worker2 # [ 0.032850] Booting paravirtualized kernel on KVM548worker1 # [ 0.098435] printk: legacy console [ttyS0] enabled549lb # [ 0.042977] Fallback order for Node 0: 0550client # [ 0.060040] Rude variant of Tasks RCU enabled.551worker1 # [ 0.196298] ACPI: Core revision 20250807552client # [ 0.060040] Tracing variant of Tasks RCU enabled.553lb # [ 0.042980] Built 1 zonelists, mobility grouping on. Total pages: 393078554worker2 # [ 0.032852] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns555lb # [ 0.042981] Policy zone: DMA32556client # [ 0.060041] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.557worker2 # [ 0.037333] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1558lb # [ 0.045304] mem auto-init: stack:all(zero), heap alloc:on, heap free:off559worker1 # [ 0.197203] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns560client # [ 0.060042] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1561worker2 # [ 0.039405] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152562lb # [ 0.048407] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1563worker2 # [ 0.039452] kvm-guest: PV spinlocks disabled, single CPU564worker1 # [ 0.198690] APIC: Switch to symmetric I/O mode setup565lb # [ 0.051659] allocated 3145728 bytes of page_ext566client # [ 0.060096] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.567worker1 # [ 0.199623] x2apic enabled568lb # [ 0.061615] ftrace: allocating 48731 entries in 192 pages569lb # [ 0.061617] ftrace: allocated 192 pages with 2 groups570client # [ 0.060098] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.571lb # [ 0.062424] Dynamic Preempt: lazy572worker1 # [ 0.200366] APIC: Switched APIC routing to: physical x2apic573client # [ 0.060099] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.574lb # [ 0.062570] rcu: Preemptible hierarchical RCU implementation.575lb # [ 0.062571] rcu: RCU event tracing is enabled.576client # [ 0.064531] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16577worker2 # [ 0.039453] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/g6bjvn1fpihxjl3rkk64li3qwyfil5lr-nixos-system-worker2-test/init regInfo=/nix/store/mddi5sa1bwc5cd417i20xm8sk5njknhg-closure-info/registration console=ttyS0,115200n8 console=tty0578worker1 # [ 0.202146] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1579lb # [ 0.062571] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.580client # [ 0.064807] rcu: srcu_init: Setting srcu_struct sizes based on contention.581lb # [ 0.062573] Trampoline variant of Tasks RCU enabled.582lb # [ 0.062573] Rude variant of Tasks RCU enabled.583client # [ 0.064814] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns584worker2 # [ 0.039543] Unknown kernel command line parameters "regInfo=/nix/store/mddi5sa1bwc5cd417i20xm8sk5njknhg-closure-info/registration", will be passed to user space.585worker1 # [ 0.203093] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns586lb # [ 0.062573] Tracing variant of Tasks RCU enabled.587worker2 # [ 0.039556] random: crng init done588client # [ 0.064920] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)589lb # [ 0.062574] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.590worker2 # [ 0.039557] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes591worker1 # [ 0.204739] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)592client # [ 0.068451] Console: colour VGA+ 80x25593lb # [ 0.062575] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1594client # [ 0.068454] printk: legacy console [tty0] enabled595worker2 # [ 0.041585] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)596worker1 # [ 0.205823] x86/cpu: User Mode Instruction Prevention (UMIP) activated597client # [ 0.097952] printk: legacy console [ttyS0] enabled598lb # [ 0.062594] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.599worker2 # [ 0.042736] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)600client # [ 0.194832] ACPI: Core revision 20250807601worker1 # [ 0.206868] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127602worker2 # [ 0.042768] Fallback order for Node 0: 0603lb # [ 0.062595] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.604worker2 # [ 0.042771] Built 1 zonelists, mobility grouping on. Total pages: 393078605worker1 # [ 0.207736] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0606worker2 # [ 0.042772] Policy zone: DMA32607client # [ 0.195712] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns608lb # [ 0.062596] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.609worker2 # [ 0.045132] mem auto-init: stack:all(zero), heap alloc:on, heap free:off610lb # [ 0.066954] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16611client # [ 0.197201] APIC: Switch to symmetric I/O mode setup612worker1 # [ 0.208739] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto613worker2 # [ 0.048118] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1614client # [ 0.198143] x2apic enabled615lb # [ 0.067229] rcu: srcu_init: Setting srcu_struct sizes based on contention.616worker2 # [ 0.051274] allocated 3145728 bytes of page_ext617worker1 # [ 0.209736] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl618worker2 # [ 0.061112] ftrace: allocating 48731 entries in 192 pages619lb # [ 0.067235] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns620client # [ 0.198885] APIC: Switched APIC routing to: physical x2apic621worker2 # [ 0.061115] ftrace: allocated 192 pages with 2 groups622worker1 # [ 0.210736] Transient Scheduler Attacks: Vulnerable: No microcode623worker2 # [ 0.061935] Dynamic Preempt: lazy624lb # [ 0.067332] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)625worker2 # [ 0.062101] rcu: Preemptible hierarchical RCU implementation.626lb # [ 0.070850] Console: colour VGA+ 80x25627client # [ 0.200643] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1628worker1 # [ 0.211736] Spectre V2 : Mitigation: Enhanced / Automatic IBRS629worker2 # [ 0.062102] rcu: RCU event tracing is enabled.630lb # [ 0.070852] printk: legacy console [tty0] enabled631worker1 # [ 0.212736] Speculative Return Stack Overflow: Mitigation: Safe RET632lb # [ 0.100434] printk: legacy console [ttyS0] enabled633worker2 # [ 0.062103] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.634client # [ 0.201586] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns635worker2 # [ 0.062104] Trampoline variant of Tasks RCU enabled.636lb # [ 0.197013] ACPI: Core revision 20250807637worker2 # [ 0.062105] Rude variant of Tasks RCU enabled.638worker2 # [ 0.062105] Tracing variant of Tasks RCU enabled.639client # [ 0.203201] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)640lb # [ 0.197914] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns641worker2 # [ 0.062106] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.642client # [ 0.204295] x86/cpu: User Mode Instruction Prevention (UMIP) activated643lb # [ 0.199453] APIC: Switch to symmetric I/O mode setup644worker2 # [ 0.062106] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1645worker1 # [ 0.213736] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization646lb # [ 0.200405] x2apic enabled647client # [ 0.205340] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127648worker1 # [ 0.215744] Spectre V2 : Enabling IBPB for BPF649worker2 # [ 0.062120] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.650client # [ 0.206198] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0651lb # [ 0.201143] APIC: Switched APIC routing to: physical x2apic652worker2 # [ 0.062122] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.653worker1 # [ 0.216461] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier654worker1 # [ 0.217736] active return thunk: srso_alias_return_thunk655client # [ 0.207202] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto656worker2 # [ 0.062122] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.657lb # [ 0.202920] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1658worker2 # [ 0.066631] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16659worker1 # [ 0.218575] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'660client # [ 0.208199] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl661worker2 # [ 0.066905] rcu: srcu_init: Setting srcu_struct sizes based on contention.662lb # [ 0.203869] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns663worker1 # [ 0.219736] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'664client # [ 0.209198] Transient Scheduler Attacks: Vulnerable: No microcode665worker2 # [ 0.066911] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns666worker1 # [ 0.220735] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'667lb # [ 0.205513] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)668client # [ 0.210198] Spectre V2 : Mitigation: Enhanced / Automatic IBRS669worker2 # [ 0.067014] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)670worker1 # [ 0.221736] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'671client # [ 0.211198] Speculative Return Stack Overflow: Mitigation: Safe RET672lb # [ 0.206602] x86/cpu: User Mode Instruction Prevention (UMIP) activated673worker2 # [ 0.070633] Console: colour VGA+ 80x25674worker2 # [ 0.070637] printk: legacy console [tty0] enabled675worker1 # [ 0.222735] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'676lb # [ 0.207627] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127677worker2 # [ 0.100433] printk: legacy console [ttyS0] enabled678worker1 # [ 0.223736] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'679worker2 # [ 0.196586] ACPI: Core revision 20250807680lb # [ 0.208510] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0681worker1 # [ 0.224735] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'682worker2 # [ 0.197439] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns683client # [ 0.212198] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization684lb # [ 0.209513] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto685worker1 # [ 0.225736] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256686client # [ 0.214206] Spectre V2 : Enabling IBPB for BPF687worker2 # [ 0.199021] APIC: Switch to symmetric I/O mode setup688worker1 # [ 0.226735] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64689lb # [ 0.210510] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl690worker2 # [ 0.199992] x2apic enabled691client # [ 0.214928] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier692worker1 # [ 0.227735] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512693lb # [ 0.211510] Transient Scheduler Attacks: Vulnerable: No microcode694worker2 # [ 0.200777] APIC: Switched APIC routing to: physical x2apic695client # [ 0.216199] active return thunk: srso_alias_return_thunk696worker1 # [ 0.228735] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024697lb # [ 0.212509] Spectre V2 : Mitigation: Enhanced / Automatic IBRS698worker1 # [ 0.229735] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8699client # [ 0.217054] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'700lb # [ 0.213510] Speculative Return Stack Overflow: Mitigation: Safe RET701worker2 # [ 0.202591] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1702client # [ 0.218198] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'703worker1 # [ 0.230736] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.704client # [ 0.219198] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'705worker2 # [ 0.203546] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns706client # [ 0.220198] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'707worker2 # [ 0.205230] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)708client # [ 0.221198] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'709lb # [ 0.214510] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization710worker2 # [ 0.206316] x86/cpu: User Mode Instruction Prevention (UMIP) activated711lb # [ 0.216517] Spectre V2 : Enabling IBPB for BPF712client # [ 0.222198] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'713worker2 # [ 0.207349] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127714client # [ 0.223198] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'715lb # [ 0.217510] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier716worker2 # [ 0.208956] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0717lb # [ 0.218510] active return thunk: srso_alias_return_thunk718client # [ 0.224198] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256719client # [ 0.225198] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64720worker2 # [ 0.210005] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto721lb # [ 0.219531] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'722client # [ 0.226198] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512723lb # [ 0.220510] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'724worker2 # [ 0.211229] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl725client # [ 0.227198] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024726lb # [ 0.222268] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'727worker2 # [ 0.212228] Transient Scheduler Attacks: Vulnerable: No microcode728client # [ 0.228198] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8729lb # [ 0.223284] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'730worker2 # [ 0.213954] Spectre V2 : Mitigation: Enhanced / Automatic IBRS731lb # [ 0.224259] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'732worker2 # [ 0.214944] Speculative Return Stack Overflow: Mitigation: Safe RET733client # [ 0.229198] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.734lb # [ 0.225274] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'735lb # [ 0.226363] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'736lb # [ 0.227510] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256737worker2 # [ 0.215228] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization738lb # [ 0.228510] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64739worker2 # [ 0.217236] Spectre V2 : Enabling IBPB for BPF740lb # [ 0.229509] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512741worker2 # [ 0.218229] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier742lb # [ 0.230510] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024743worker2 # [ 0.219229] active return thunk: srso_alias_return_thunk744lb # [ 0.231510] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8745worker2 # [ 0.220965] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'746lb # [ 0.232509] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.747worker2 # [ 0.222228] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'748worker2 # [ 0.223228] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'749worker2 # [ 0.224228] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'750worker2 # [ 0.225228] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'751worker2 # [ 0.226228] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'752worker2 # [ 0.227228] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'753worker2 # [ 0.228228] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256754worker2 # [ 0.230027] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64755worker2 # [ 0.231228] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512756worker2 # [ 0.232227] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024757worker2 # [ 0.233046] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8758worker2 # [ 0.234228] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.759worker1 # [ 0.264897] Freeing SMP alternatives memory: 44K760worker1 # [ 0.265696] pid_max: default: 32768 minimum: 301761worker1 # [ 0.266399] LSM: initializing lsm=capability,landlock,yama,bpf,ima762worker1 # [ 0.267628] landlock: Up and running.763worker1 # [ 0.268221] Yama: becoming mindful.764worker1 # [ 0.268963] LSM support for eBPF active765worker1 # [ 0.269839] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)766worker1 # [ 0.270771] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)767worker1 # [ 0.273550] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)768client # [ 0.263237] Freeing SMP alternatives memory: 44K769worker1 # [ 0.274255] Performance Events: Fam17h+ core perfctr, AMD PMU driver.770client # [ 0.264044] pid_max: default: 32768 minimum: 301771worker1 # [ 0.274740] ... version: 2772worker1 # [ 0.275449] ... bit width: 48773client # [ 0.264861] LSM: initializing lsm=capability,landlock,yama,bpf,ima774worker1 # [ 0.275739] ... generic counters: 6775client # [ 0.265299] landlock: Up and running.776worker1 # [ 0.276455] ... generic bitmap: 000000000000003f777client # [ 0.266650] Yama: becoming mindful.778worker1 # [ 0.276748] ... fixed-purpose counters: 0779client # [ 0.267416] LSM support for eBPF active780worker1 # [ 0.277454] ... fixed-purpose bitmap: 0000000000000000781client # [ 0.268321] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)782worker1 # [ 0.277738] ... value mask: 0000ffffffffffff783worker1 # [ 0.278652] ... max period: 00007fffffffffff784client # [ 0.269232] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)785worker1 # [ 0.279448] ... global_ctrl mask: 000000000000003f786worker1 # [ 0.279847] signal: max sigframe size: 3376787client # [ 0.271887] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)788lb # [ 0.267045] Freeing SMP alternatives memory: 44K789worker1 # [ 0.280702] rcu: Hierarchical SRCU implementation.790lb # [ 0.267512] pid_max: default: 32768 minimum: 301791worker1 # [ 0.281345] rcu: Max phase no-delay instances is 400.792client # [ 0.272691] Performance Events: Fam17h+ core perfctr, AMD PMU driver.793lb # [ 0.268590] LSM: initializing lsm=capability,landlock,yama,bpf,ima794client # [ 0.273203] ... version: 2795lb # [ 0.269608] landlock: Up and running.796client # [ 0.273941] ... bit width: 48797lb # [ 0.270510] Yama: becoming mindful.798client # [ 0.274201] ... generic counters: 6799lb # [ 0.271376] LSM support for eBPF active800client # [ 0.274915] ... generic bitmap: 000000000000003f801client # [ 0.275266] ... fixed-purpose counters: 0802lb # [ 0.272112] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)803worker1 # [ 0.286190] smp: Bringing up secondary CPUs ...804client # [ 0.275976] ... fixed-purpose bitmap: 0000000000000000805worker1 # [ 0.286762] smp: Brought up 1 node, 1 CPU806lb # [ 0.272547] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)807client # [ 0.276201] ... value mask: 0000ffffffffffff808worker1 # [ 0.287420] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)809client # [ 0.277109] ... max period: 00007fffffffffff810client # [ 0.277908] ... global_ctrl mask: 000000000000003f811lb # [ 0.274838] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)812client # [ 0.278336] signal: max sigframe size: 3376813worker1 # [ 0.287936] Memory: 1454552K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)814client # [ 0.279121] rcu: Hierarchical SRCU implementation.815worker1 # [ 0.288977] devtmpfs: initialized816lb # [ 0.276038] Performance Events: Fam17h+ core perfctr, AMD PMU driver.817client # [ 0.279804] rcu: Max phase no-delay instances is 400.818lb # [ 0.276519] ... version: 2819worker1 # [ 0.289700] x86/mm: Memory block size: 128MB820lb # [ 0.277283] ... bit width: 48821lb # [ 0.277513] ... generic counters: 6822worker2 # [ 0.269482] Freeing SMP alternatives memory: 44K823worker1 # [ 0.291327] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)824lb # [ 0.278280] ... generic bitmap: 000000000000003f825worker2 # [ 0.270231] pid_max: default: 32768 minimum: 301826lb # [ 0.278512] ... fixed-purpose counters: 0827worker1 # [ 0.291762] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).828worker2 # [ 0.271051] LSM: initializing lsm=capability,landlock,yama,bpf,ima829lb # [ 0.279228] ... fixed-purpose bitmap: 0000000000000000830worker2 # [ 0.272064] landlock: Up and running.831worker1 # [ 0.292821] pinctrl core: initialized pinctrl subsystem832lb # [ 0.279512] ... value mask: 0000ffffffffffff833client # [ 0.284640] smp: Bringing up secondary CPUs ...834worker2 # [ 0.272661] Yama: becoming mindful.835client # [ 0.285214] smp: Brought up 1 node, 1 CPU836worker1 # [ 0.293929] PM: RTC time: 10:51:22, date: 2026-09-20837lb # [ 0.280382] ... max period: 00007fffffffffff838worker2 # [ 0.273451] LSM support for eBPF active839lb # [ 0.281156] ... global_ctrl mask: 000000000000003f840client # [ 0.285865] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)841worker2 # [ 0.274338] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)842lb # [ 0.281608] signal: max sigframe size: 3376843lb # [ 0.282393] rcu: Hierarchical SRCU implementation.844worker2 # [ 0.275262] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)845client # [ 0.286412] Memory: 1454568K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109348K reserved, 0K cma-reserved)846lb # [ 0.283119] rcu: Max phase no-delay instances is 400.847worker1 # [ 0.297302] NET: Registered PF_NETLINK/PF_ROUTE protocol family848client # [ 0.287484] devtmpfs: initialized849client # [ 0.288250] x86/mm: Memory block size: 128MB850worker1 # [ 0.298211] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations851worker2 # [ 0.277982] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)852worker1 # [ 0.298757] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations853worker2 # [ 0.278730] Performance Events: Fam17h+ core perfctr, AMD PMU driver.854client # [ 0.289975] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)855worker2 # [ 0.279233] ... version: 2856worker1 # [ 0.299996] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations857worker2 # [ 0.279944] ... bit width: 48858client # [ 0.291131] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).859lb # [ 0.287967] smp: Bringing up secondary CPUs ...860worker1 # [ 0.300749] audit: initializing netlink subsys (disabled)861worker2 # [ 0.280230] ... generic counters: 6862lb # [ 0.288525] smp: Brought up 1 node, 1 CPU863client # [ 0.292271] pinctrl core: initialized pinctrl subsystem864worker2 # [ 0.280914] ... generic bitmap: 000000000000003f865worker1 # [ 0.301808] thermal_sys: Registered thermal governor 'fair_share'866lb # [ 0.289214] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)867worker2 # [ 0.281241] ... fixed-purpose counters: 0868client # [ 0.293368] PM: RTC time: 10:51:22, date: 2026-09-20869worker1 # [ 0.301810] thermal_sys: Registered thermal governor 'bang_bang'870worker2 # [ 0.281929] ... fixed-purpose bitmap: 0000000000000000871worker1 # [ 0.302740] thermal_sys: Registered thermal governor 'step_wise'872worker2 # [ 0.282230] ... value mask: 0000ffffffffffff873lb # [ 0.289665] Memory: 1454560K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)874worker1 # [ 0.303649] thermal_sys: Registered thermal governor 'user_space'875worker2 # [ 0.283104] ... max period: 00007fffffffffff876lb # [ 0.290729] devtmpfs: initialized877client # [ 0.296694] NET: Registered PF_NETLINK/PF_ROUTE protocol family878worker1 # [ 0.304459] audit: type=2000 audit(1789901483.215:1): state=initialized audit_enabled=0 res=1879worker2 # [ 0.283921] ... global_ctrl mask: 000000000000003f880lb # [ 0.291573] x86/mm: Memory block size: 128MB881worker2 # [ 0.284339] signal: max sigframe size: 3376882worker1 # [ 0.305740] thermal_sys: Registered thermal governor 'power_allocator'883client # [ 0.297704] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations884worker2 # [ 0.285101] rcu: Hierarchical SRCU implementation.885worker1 # [ 0.305767] cpuidle: using governor menu886worker2 # [ 0.285796] rcu: Max phase no-delay instances is 400.887lb # [ 0.293275] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)888client # [ 0.298220] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations889worker1 # [ 0.307952] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5890lb # [ 0.294430] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).891client # [ 0.299457] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations892lb # [ 0.295589] pinctrl core: initialized pinctrl subsystem893worker1 # [ 0.308971] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]894client # [ 0.300211] audit: initializing netlink subsys (disabled)895lb # [ 0.296707] PM: RTC time: 10:51:22, date: 2026-09-20896worker1 # [ 0.309741] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry897client # [ 0.301202] audit: type=2000 audit(1789901483.228:1): state=initialized audit_enabled=0 res=1898worker1 # [ 0.310749] PCI: Using configuration type 1 for base access899worker2 # [ 0.290700] smp: Bringing up secondary CPUs ...900client # [ 0.302338] thermal_sys: Registered thermal governor 'fair_share'901worker2 # [ 0.291244] smp: Brought up 1 node, 1 CPU902worker1 # [ 0.311808] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.903client # [ 0.302341] thermal_sys: Registered thermal governor 'bang_bang'904worker2 # [ 0.291912] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)905client # [ 0.303201] thermal_sys: Registered thermal governor 'step_wise'906lb # [ 0.300263] NET: Registered PF_NETLINK/PF_ROUTE protocol family907client # [ 0.304127] thermal_sys: Registered thermal governor 'user_space'908lb # [ 0.301719] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations909worker2 # [ 0.292376] Memory: 1454544K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)910client # [ 0.304923] thermal_sys: Registered thermal governor 'power_allocator'911client # [ 0.305214] cpuidle: using governor menu912worker2 # [ 0.293450] devtmpfs: initialized913lb # [ 0.302531] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations914worker2 # [ 0.294286] x86/mm: Memory block size: 128MB915client # [ 0.307430] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5916lb # [ 0.303779] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations917lb # [ 0.304521] audit: initializing netlink subsys (disabled)918client # [ 0.308419] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]919worker2 # [ 0.296013] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)920worker1 # [ 0.318969] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages921lb # [ 0.305687] thermal_sys: Registered thermal governor 'fair_share'922client # [ 0.309204] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry923worker1 # [ 0.319740] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page924worker2 # [ 0.297206] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).925lb # [ 0.305689] thermal_sys: Registered thermal governor 'bang_bang'926client # [ 0.310213] PCI: Using configuration type 1 for base access927worker1 # [ 0.320713] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages928worker2 # [ 0.298306] pinctrl core: initialized pinctrl subsystem929lb # [ 0.306513] thermal_sys: Registered thermal governor 'step_wise'930client # [ 0.311275] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.931worker2 # [ 0.299451] PM: RTC time: 10:51:22, date: 2026-09-20932lb # [ 0.307457] thermal_sys: Registered thermal governor 'user_space'933lb # [ 0.308260] audit: type=2000 audit(1789901483.238:1): state=initialized audit_enabled=0 res=1934lb # [ 0.309515] thermal_sys: Registered thermal governor 'power_allocator'935lb # [ 0.309530] cpuidle: using governor menu936worker2 # [ 0.302836] NET: Registered PF_NETLINK/PF_ROUTE protocol family937worker1 # [ 0.325739] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page938lb # [ 0.311762] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5939worker2 # [ 0.303698] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations940worker2 # [ 0.304250] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations941lb # [ 0.312757] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]942client # [ 0.318226] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages943lb # [ 0.313515] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry944worker2 # [ 0.305491] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations945client # [ 0.319202] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page946lb # [ 0.314524] PCI: Using configuration type 1 for base access947worker2 # [ 0.306241] audit: initializing netlink subsys (disabled)948worker2 # [ 0.307428] thermal_sys: Registered thermal governor 'fair_share'949lb # [ 0.315601] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.950worker2 # [ 0.307430] thermal_sys: Registered thermal governor 'bang_bang'951worker1 # [ 0.334156] ACPI: Added _OSI(Module Device)952worker2 # [ 0.308233] thermal_sys: Registered thermal governor 'step_wise'953worker1 # [ 0.334750] ACPI: Added _OSI(Processor Device)954client # [ 0.324202] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages955worker2 # [ 0.309234] audit: type=2000 audit(1789901483.253:1): state=initialized audit_enabled=0 res=1956worker1 # [ 0.335481] ACPI: Added _OSI(Processor Aggregator Device)957client # [ 0.325201] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page958worker2 # [ 0.311235] thermal_sys: Registered thermal governor 'user_space'959worker2 # [ 0.311237] thermal_sys: Registered thermal governor 'power_allocator'960lb # [ 0.320783] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages961worker2 # [ 0.312215] cpuidle: using governor menu962lb # [ 0.321513] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page963worker2 # [ 0.314246] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5964worker2 # [ 0.315466] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]965worker2 # [ 0.316233] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry966worker1 # [ 0.342249] ACPI: 1 ACPI AML tables successfully acquired and loaded967worker2 # [ 0.317243] PCI: Using configuration type 1 for base access968lb # [ 0.326513] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages969lb # [ 0.327512] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page970worker2 # [ 0.318316] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.971client # [ 0.336436] ACPI: Added _OSI(Module Device)972worker1 # [ 0.346139] ACPI: Interpreter enabled973client # [ 0.337167] ACPI: Added _OSI(Processor Device)974worker1 # [ 0.346754] ACPI: PM: (supports S0 S3 S4 S5)975client # [ 0.337762] ACPI: Added _OSI(Processor Aggregator Device)976worker1 # [ 0.347490] ACPI: Using IOAPIC for interrupt routing977worker1 # [ 0.349773] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug978worker2 # [ 0.325512] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages979worker1 # [ 0.350738] PCI: Using E820 reservations for host bridge windows980worker2 # [ 0.326232] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page981lb # [ 0.337938] ACPI: Added _OSI(Module Device)982client # [ 0.343870] ACPI: 1 ACPI AML tables successfully acquired and loaded983lb # [ 0.338514] ACPI: Added _OSI(Processor Device)984worker1 # [ 0.353849] ACPI: Enabled 2 GPEs in block 00 to 3F985lb # [ 0.339299] ACPI: Added _OSI(Processor Aggregator Device)986worker2 # [ 0.331231] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages987client # [ 0.347610] ACPI: Interpreter enabled988worker2 # [ 0.332232] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page989client # [ 0.348216] ACPI: PM: (supports S0 S3 S4 S5)990client # [ 0.348929] ACPI: Using IOAPIC for interrupt routing991client # [ 0.351274] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug992worker1 # [ 0.362433] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])993client # [ 0.352201] PCI: Using E820 reservations for host bridge windows994lb # [ 0.348057] ACPI: 1 ACPI AML tables successfully acquired and loaded995worker1 # [ 0.362745] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]996worker1 # [ 0.363816] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]997worker2 # [ 0.340847] ACPI: Added _OSI(Module Device)998client # [ 0.355306] ACPI: Enabled 2 GPEs in block 00 to 3F999worker2 # [ 0.341233] ACPI: Added _OSI(Processor Device)1000worker1 # [ 0.364863] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1001worker2 # [ 0.341969] ACPI: Added _OSI(Processor Aggregator Device)1002worker1 # [ 0.366076] PCI host bridge to bus 0000:001003lb # [ 0.351922] ACPI: Interpreter enabled1004lb # [ 0.352528] ACPI: PM: (supports S0 S3 S4 S5)1005worker1 # [ 0.366743] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1006lb # [ 0.353240] ACPI: Using IOAPIC for interrupt routing1007worker1 # [ 0.367739] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1008lb # [ 0.353546] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1009worker1 # [ 0.368738] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1010lb # [ 0.354513] PCI: Using E820 reservations for host bridge windows1011worker1 # [ 0.369739] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1012worker1 # [ 0.370739] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1013client # [ 0.362742] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1014worker1 # [ 0.371738] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1015lb # [ 0.357638] ACPI: Enabled 2 GPEs in block 00 to 3F1016worker2 # [ 0.348838] ACPI: 1 ACPI AML tables successfully acquired and loaded1017client # [ 0.363207] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1018worker1 # [ 0.372738] pci_bus 0000:00: root bus resource [bus 00-ff]1019client # [ 0.364281] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1020worker1 # [ 0.373753] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1021client # [ 0.365323] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1022worker2 # [ 0.352168] ACPI: Interpreter enabled1023client # [ 0.366531] PCI host bridge to bus 0000:001024worker1 # [ 0.375197] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1025worker2 # [ 0.352721] ACPI: PM: (supports S0 S3 S4 S5)1026client # [ 0.367206] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1027worker2 # [ 0.353232] ACPI: Using IOAPIC for interrupt routing1028client # [ 0.368201] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1029worker2 # [ 0.354103] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1030client # [ 0.369201] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1031client # [ 0.370201] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1032lb # [ 0.366183] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1033client # [ 0.371201] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1034lb # [ 0.366519] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1035worker1 # [ 0.377829] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1036client # [ 0.372201] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1037worker1 # [ 0.378751] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1038worker2 # [ 0.359232] PCI: Using E820 reservations for host bridge windows1039lb # [ 0.367807] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1040client # [ 0.373201] pci_bus 0000:00: root bus resource [bus 00-ff]1041worker1 # [ 0.379730] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1042worker2 # [ 0.360304] ACPI: Enabled 2 GPEs in block 00 to 3F1043lb # [ 0.368635] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1044client # [ 0.374157] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1045lb # [ 0.369861] PCI host bridge to bus 0000:001046worker1 # [ 0.380943] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1047lb # [ 0.370516] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1048client # [ 0.375652] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1049lb # [ 0.371512] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1050worker1 # [ 0.382562] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1051lb # [ 0.372512] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1052lb # [ 0.373512] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1053lb # [ 0.374512] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1054worker1 # [ 0.385768] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1055worker1 # [ 0.386651] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1056lb # [ 0.375512] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1057worker2 # [ 0.368923] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1058client # [ 0.378241] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1059lb # [ 0.376513] pci_bus 0000:00: root bus resource [bus 00-ff]1060worker1 # [ 0.387519] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1061client # [ 0.379213] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1062worker1 # [ 0.388578] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1063worker2 # [ 0.370055] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1064lb # [ 0.377501] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1065client # [ 0.380222] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1066worker2 # [ 0.371311] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1067worker1 # [ 0.390164] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1068client # [ 0.381425] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1069lb # [ 0.378955] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1070worker2 # [ 0.372357] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1071worker2 # [ 0.373584] PCI host bridge to bus 0000:001072client # [ 0.383051] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1073worker2 # [ 0.374236] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1074worker1 # [ 0.392844] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1075worker2 # [ 0.375231] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1076worker1 # [ 0.393745] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1077client # [ 0.385250] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1078worker2 # [ 0.376232] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1079client # [ 0.386100] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1080worker1 # [ 0.394718] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1081worker2 # [ 0.377231] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1082lb # [ 0.381535] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1083client # [ 0.386937] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1084lb # [ 0.382549] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1085worker2 # [ 0.378231] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1086worker1 # [ 0.396140] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1087client # [ 0.387228] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1088lb # [ 0.383461] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1089worker2 # [ 0.379231] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1090worker2 # [ 0.380231] pci_bus 0000:00: root bus resource [bus 00-ff]1091client # [ 0.388887] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1092lb # [ 0.384403] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1093worker1 # [ 0.397747] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1094worker2 # [ 0.381151] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1095worker1 # [ 0.398588] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1096lb # [ 0.386590] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1097worker1 # [ 0.399514] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1098worker2 # [ 0.382693] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1099client # [ 0.392207] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1100client # [ 0.393114] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1101worker1 # [ 0.401846] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1102lb # [ 0.389549] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1103client # [ 0.393983] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1104lb # [ 0.390375] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1105client # [ 0.395738] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1106lb # [ 0.391320] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1107worker1 # [ 0.403746] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1108lb # [ 0.392332] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1109worker1 # [ 0.404629] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1110worker1 # [ 0.405481] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1111client # [ 0.397232] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1112worker2 # [ 0.385362] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1113lb # [ 0.393849] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1114client # [ 0.398054] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1115worker2 # [ 0.386285] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1116worker1 # [ 0.407166] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1117worker2 # [ 0.387242] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1118client # [ 0.398958] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1119worker2 # [ 0.388535] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1120lb # [ 0.396547] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1121client # [ 0.399784] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1122worker1 # [ 0.409660] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1123lb # [ 0.397409] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1124worker1 # [ 0.410745] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1125worker2 # [ 0.389961] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1126lb # [ 0.398340] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1127client # [ 0.401208] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1128worker1 # [ 0.411693] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1129client # [ 0.402018] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1130lb # [ 0.399906] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1131client # [ 0.402926] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1132worker2 # [ 0.393256] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1133worker1 # [ 0.413236] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1134worker2 # [ 0.394160] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1135client # [ 0.403769] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1136lb # [ 0.402429] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1137worker2 # [ 0.395005] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1138worker1 # [ 0.414768] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1139lb # [ 0.403160] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1140worker2 # [ 0.396086] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1141worker1 # [ 0.415625] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1142worker1 # [ 0.416530] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1143client # [ 0.405455] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1144lb # [ 0.403534] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1145worker2 # [ 0.397561] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1146client # [ 0.406208] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1147client # [ 0.407179] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1148lb # [ 0.405082] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1149worker1 # [ 0.418079] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1150worker2 # [ 0.400140] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1151client # [ 0.408620] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1152worker2 # [ 0.400871] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1153lb # [ 0.407343] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1154worker1 # [ 0.420767] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1155worker2 # [ 0.401253] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1156lb # [ 0.408200] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1157worker1 # [ 0.421679] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1158client # [ 0.410238] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1159worker1 # [ 0.422444] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1160lb # [ 0.408534] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1161client # [ 0.411089] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1162worker2 # [ 0.402815] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1163worker1 # [ 0.422744] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1164client # [ 0.411927] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1165lb # [ 0.410063] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1166worker1 # [ 0.424279] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1167client # [ 0.412774] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1168worker2 # [ 0.405257] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1169lb # [ 0.412520] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1170worker1 # [ 0.425764] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1171client # [ 0.414229] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1172worker2 # [ 0.406238] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1173lb # [ 0.413361] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1174client # [ 0.415045] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1175worker1 # [ 0.426760] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1176worker2 # [ 0.407153] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1177lb # [ 0.414246] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1178client # [ 0.415893] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1179worker1 # [ 0.428313] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1180client # [ 0.416207] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1181worker2 # [ 0.408624] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1182lb # [ 0.415103] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1183client # [ 0.417701] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1184worker1 # [ 0.429776] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1185worker2 # [ 0.411108] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1186worker1 # [ 0.430658] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1187lb # [ 0.416548] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1188worker2 # [ 0.411897] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1189client # [ 0.419213] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1190worker1 # [ 0.431492] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1191lb # [ 0.417387] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1192worker2 # [ 0.412253] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1193client # [ 0.420152] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1194worker1 # [ 0.432278] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1195lb # [ 0.418236] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1196client # [ 0.421736] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1197worker2 # [ 0.413787] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1198lb # [ 0.419138] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1199worker1 # [ 0.434491] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1200client # [ 0.423246] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1201worker1 # [ 0.434764] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1202client # [ 0.424120] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1203worker2 # [ 0.415985] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1204lb # [ 0.421226] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1205worker1 # [ 0.435702] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1206worker2 # [ 0.416238] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1207client # [ 0.424961] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1208lb # [ 0.421519] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1209worker2 # [ 0.417163] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1210worker1 # [ 0.437102] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1211lb # [ 0.422517] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1212client # [ 0.425737] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1213lb # [ 0.423518] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1214worker2 # [ 0.418671] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1215worker1 # [ 0.438384] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1216lb # [ 0.425049] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1217worker1 # [ 0.438946] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1218client # [ 0.427237] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1219client # [ 0.428207] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1220worker1 # [ 0.440369] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1221client # [ 0.429200] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1222worker2 # [ 0.421295] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1223lb # [ 0.427181] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1224worker2 # [ 0.422239] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1225lb # [ 0.427536] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1226client # [ 0.430769] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1227worker2 # [ 0.423201] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1228client # [ 0.431731] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1229lb # [ 0.429054] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1230worker2 # [ 0.424668] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1231client # [ 0.432401] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1232client # [ 0.433721] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1233lb # [ 0.430561] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1234worker2 # [ 0.427001] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1235lb # [ 0.431403] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1236worker2 # [ 0.427237] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1237lb # [ 0.432287] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1238worker2 # [ 0.428236] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1239worker2 # [ 0.429236] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1240lb # [ 0.434733] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1241worker1 # [ 0.440948] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1242worker2 # [ 0.431720] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1243lb # [ 0.437211] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1244lb # [ 0.437520] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1245worker2 # [ 0.433269] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1246worker1 # [ 0.444341] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1247lb # [ 0.438487] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1248worker2 # [ 0.434252] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1249worker1 # [ 0.445008] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1250lb # [ 0.439921] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1251worker2 # [ 0.435785] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1252worker1 # [ 0.446746] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1253client # [ 0.434389] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1254lb # [ 0.441174] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1255worker2 # [ 0.437262] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1256worker1 # [ 0.447924] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1257lb # [ 0.441734] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1258worker1 # [ 0.449037] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1259worker2 # [ 0.438115] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1260client # [ 0.437729] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1261worker2 # [ 0.439253] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1262lb # [ 0.443345] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1263worker1 # [ 0.450008] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1264client # [ 0.438443] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1265worker2 # [ 0.440804] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1266client # [ 0.440067] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1267worker1 # [ 0.452792] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1268worker1 # [ 0.453680] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1269worker2 # [ 0.442912] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1270client # [ 0.441236] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1271worker2 # [ 0.443237] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1272worker1 # [ 0.454794] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1273client # [ 0.442507] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1274worker2 # [ 0.444219] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1275worker1 # [ 0.456568] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1276client # [ 0.443470] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1277worker2 # [ 0.445654] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1278client # [ 0.445818] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1279worker2 # [ 0.447283] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1280client # [ 0.446208] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1281worker1 # [ 0.460400] ACPI: PCI: Interrupt link LNKA configured for IRQ 101282worker2 # [ 0.448366] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1283lb # [ 0.444346] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1284client # [ 0.447495] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1285worker1 # [ 0.460847] ACPI: PCI: Interrupt link LNKB configured for IRQ 101286worker2 # [ 0.449936] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1287worker1 # [ 0.461837] ACPI: PCI: Interrupt link LNKC configured for IRQ 111288lb # [ 0.446221] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1289client # [ 0.448874] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1290worker1 # [ 0.462847] ACPI: PCI: Interrupt link LNKD configured for IRQ 111291lb # [ 0.446743] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1292worker1 # [ 0.463838] ACPI: PCI: Interrupt link LNKE configured for IRQ 101293worker1 # [ 0.464835] ACPI: PCI: Interrupt link LNKF configured for IRQ 101294lb # [ 0.448160] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1295client # [ 0.452145] ACPI: PCI: Interrupt link LNKA configured for IRQ 101296worker1 # [ 0.465840] ACPI: PCI: Interrupt link LNKG configured for IRQ 111297worker1 # [ 0.466839] ACPI: PCI: Interrupt link LNKH configured for IRQ 111298lb # [ 0.448788] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1299client # [ 0.453028] ACPI: PCI: Interrupt link LNKB configured for IRQ 101300worker1 # [ 0.467777] ACPI: PCI: Interrupt link GSIA configured for IRQ 161301client # [ 0.454302] ACPI: PCI: Interrupt link LNKC configured for IRQ 111302lb # [ 0.449831] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1303worker1 # [ 0.468751] ACPI: PCI: Interrupt link GSIB configured for IRQ 171304client # [ 0.455324] ACPI: PCI: Interrupt link LNKD configured for IRQ 111305worker1 # [ 0.469710] ACPI: PCI: Interrupt link GSIC configured for IRQ 181306lb # [ 0.450758] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1307client # [ 0.456308] ACPI: PCI: Interrupt link LNKE configured for IRQ 101308worker1 # [ 0.470653] ACPI: PCI: Interrupt link GSID configured for IRQ 191309client # [ 0.457298] ACPI: PCI: Interrupt link LNKF configured for IRQ 101310worker1 # [ 0.471593] ACPI: PCI: Interrupt link GSIE configured for IRQ 201311lb # [ 0.452574] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1312client # [ 0.458306] ACPI: PCI: Interrupt link LNKG configured for IRQ 111313worker1 # [ 0.472480] ACPI: PCI: Interrupt link GSIF configured for IRQ 211314lb # [ 0.453439] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1315client # [ 0.459302] ACPI: PCI: Interrupt link LNKH configured for IRQ 111316worker1 # [ 0.472754] ACPI: PCI: Interrupt link GSIG configured for IRQ 221317worker2 # [ 0.450437] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1318client # [ 0.460245] ACPI: PCI: Interrupt link GSIA configured for IRQ 161319lb # [ 0.454571] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1320worker1 # [ 0.473677] ACPI: PCI: Interrupt link GSIH configured for IRQ 231321client # [ 0.461168] ACPI: PCI: Interrupt link GSIB configured for IRQ 171322worker2 # [ 0.452931] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1323client # [ 0.461930] ACPI: PCI: Interrupt link GSIC configured for IRQ 181324lb # [ 0.456541] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1325worker1 # [ 0.475545] iommu: Default domain type: Translated1326client # [ 0.462213] ACPI: PCI: Interrupt link GSID configured for IRQ 191327worker2 # [ 0.453464] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1328worker1 # [ 0.476308] iommu: DMA domain TLB invalidation policy: lazy mode1329client # [ 0.463213] ACPI: PCI: Interrupt link GSIE configured for IRQ 201330worker1 # [ 0.477007] ACPI: bus type USB registered1331worker2 # [ 0.454919] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1332client # [ 0.464217] ACPI: PCI: Interrupt link GSIF configured for IRQ 211333worker1 # [ 0.477724] usbcore: registered new interface driver usbfs1334client # [ 0.465190] ACPI: PCI: Interrupt link GSIG configured for IRQ 221335worker2 # [ 0.455506] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1336worker1 # [ 0.478452] usbcore: registered new interface driver hub1337lb # [ 0.460264] ACPI: PCI: Interrupt link LNKA configured for IRQ 101338client # [ 0.465912] ACPI: PCI: Interrupt link GSIH configured for IRQ 231339worker1 # [ 0.478749] usbcore: registered new device driver usb1340lb # [ 0.460617] ACPI: PCI: Interrupt link LNKB configured for IRQ 101341worker2 # [ 0.456515] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1342lb # [ 0.461612] ACPI: PCI: Interrupt link LNKC configured for IRQ 111343worker1 # [ 0.480471] NetLabel: Initializing1344client # [ 0.467195] iommu: Default domain type: Translated1345worker2 # [ 0.457492] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1346worker1 # [ 0.480744] NetLabel: domain hash size = 1281347lb # [ 0.462613] ACPI: PCI: Interrupt link LNKD configured for IRQ 111348client # [ 0.468048] iommu: DMA domain TLB invalidation policy: lazy mode1349worker1 # [ 0.481483] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1350client # [ 0.469175] ACPI: bus type USB registered1351lb # [ 0.463612] ACPI: PCI: Interrupt link LNKE configured for IRQ 101352worker1 # [ 0.481796] NetLabel: unlabeled traffic allowed by default1353worker2 # [ 0.459258] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1354client # [ 0.469721] usbcore: registered new interface driver usbfs1355lb # [ 0.464610] ACPI: PCI: Interrupt link LNKF configured for IRQ 101356worker1 # [ 0.482703] PCI: Using ACPI for IRQ routing1357worker2 # [ 0.460101] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1358client # [ 0.470216] usbcore: registered new interface driver hub1359lb # [ 0.465620] ACPI: PCI: Interrupt link LNKG configured for IRQ 111360client # [ 0.471046] usbcore: registered new device driver usb1361lb # [ 0.466618] ACPI: PCI: Interrupt link LNKH configured for IRQ 111362worker2 # [ 0.461246] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1363client # [ 0.472702] NetLabel: Initializing1364lb # [ 0.467550] ACPI: PCI: Interrupt link GSIA configured for IRQ 161365client # [ 0.473201] NetLabel: domain hash size = 1281366worker2 # [ 0.463046] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1367lb # [ 0.468527] ACPI: PCI: Interrupt link GSIB configured for IRQ 171368client # [ 0.473909] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1369lb # [ 0.469444] ACPI: PCI: Interrupt link GSIC configured for IRQ 181370client # [ 0.474258] NetLabel: unlabeled traffic allowed by default1371client # [ 0.475184] PCI: Using ACPI for IRQ routing1372lb # [ 0.470289] ACPI: PCI: Interrupt link GSID configured for IRQ 191373lb # [ 0.470527] ACPI: PCI: Interrupt link GSIE configured for IRQ 201374worker2 # [ 0.466796] ACPI: PCI: Interrupt link LNKA configured for IRQ 101375lb # [ 0.471525] ACPI: PCI: Interrupt link GSIF configured for IRQ 211376lb # [ 0.472491] ACPI: PCI: Interrupt link GSIG configured for IRQ 221377worker2 # [ 0.467339] ACPI: PCI: Interrupt link LNKB configured for IRQ 101378lb # [ 0.473285] ACPI: PCI: Interrupt link GSIH configured for IRQ 231379worker2 # [ 0.468328] ACPI: PCI: Interrupt link LNKC configured for IRQ 111380worker2 # [ 0.469331] ACPI: PCI: Interrupt link LNKD configured for IRQ 111381lb # [ 0.474600] iommu: Default domain type: Translated1382worker2 # [ 0.470328] ACPI: PCI: Interrupt link LNKE configured for IRQ 101383lb # [ 0.475432] iommu: DMA domain TLB invalidation policy: lazy mode1384worker2 # [ 0.471327] ACPI: PCI: Interrupt link LNKF configured for IRQ 101385lb # [ 0.476539] ACPI: bus type USB registered1386worker2 # [ 0.472329] ACPI: PCI: Interrupt link LNKG configured for IRQ 111387lb # [ 0.477286] usbcore: registered new interface driver usbfs1388worker2 # [ 0.473331] ACPI: PCI: Interrupt link LNKH configured for IRQ 111389lb # [ 0.477524] usbcore: registered new interface driver hub1390lb # [ 0.478422] usbcore: registered new device driver usb1391worker2 # [ 0.474269] ACPI: PCI: Interrupt link GSIA configured for IRQ 161392worker2 # [ 0.475245] ACPI: PCI: Interrupt link GSIB configured for IRQ 171393lb # [ 0.479950] NetLabel: Initializing1394worker2 # [ 0.476244] ACPI: PCI: Interrupt link GSIC configured for IRQ 181395lb # [ 0.480513] NetLabel: domain hash size = 1281396lb # [ 0.481243] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1397worker2 # [ 0.477242] ACPI: PCI: Interrupt link GSID configured for IRQ 191398worker2 # [ 0.478236] ACPI: PCI: Interrupt link GSIE configured for IRQ 201399lb # [ 0.481565] NetLabel: unlabeled traffic allowed by default1400lb # [ 0.482675] PCI: Using ACPI for IRQ routing1401worker2 # [ 0.479208] ACPI: PCI: Interrupt link GSIF configured for IRQ 211402worker2 # [ 0.480148] ACPI: PCI: Interrupt link GSIG configured for IRQ 221403worker2 # [ 0.481053] ACPI: PCI: Interrupt link GSIH configured for IRQ 231404worker2 # [ 0.483038] iommu: Default domain type: Translated1405worker2 # [ 0.483893] iommu: DMA domain TLB invalidation policy: lazy mode1406worker2 # [ 0.484478] ACPI: bus type USB registered1407worker2 # [ 0.485260] usbcore: registered new interface driver usbfs1408worker2 # [ 0.486222] usbcore: registered new interface driver hub1409worker2 # [ 0.486937] usbcore: registered new device driver usb1410worker2 # [ 0.488128] NetLabel: Initializing1411worker2 # [ 0.488800] NetLabel: domain hash size = 1281412worker2 # [ 0.489235] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1413worker2 # [ 0.490302] NetLabel: unlabeled traffic allowed by default1414worker2 # [ 0.491241] PCI: Using ACPI for IRQ routing1415worker1 # [ 0.527233] pci 0000:00:01.0: vgaarb: setting as boot VGA device1416worker1 # [ 0.527734] pci 0000:00:01.0: vgaarb: bridge control possible1417worker1 # [ 0.527734] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1418worker1 # [ 0.527743] vgaarb: loaded1419worker1 # [ 0.528518] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01420client # [ 0.519102] pci 0000:00:01.0: vgaarb: setting as boot VGA device1421worker1 # [ 0.528740] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1422client # [ 0.519197] pci 0000:00:01.0: vgaarb: bridge control possible1423client # [ 0.519197] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1424client # [ 0.519206] vgaarb: loaded1425client # [ 0.519909] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01426worker1 # [ 0.531824] clocksource: Switched to clocksource kvm-clock1427client # [ 0.520201] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1428worker1 # [ 0.535310] VFS: Disk quotas dquot_6.6.01429worker1 # [ 0.536055] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1430worker1 # [ 0.537493] pnp: PnP ACPI init1431client # [ 0.524295] clocksource: Switched to clocksource kvm-clock1432worker1 # [ 0.538292] ACPI: IRQ 4 override to edge(!), high(!)1433worker1 # [ 0.539337] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1434client # [ 0.527703] VFS: Disk quotas dquot_6.6.01435worker1 # [ 0.540928] pnp: PnP ACPI: found 6 devices1436client # [ 0.528417] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1437client # [ 0.529740] pnp: PnP ACPI init1438client # [ 0.530531] ACPI: IRQ 4 override to edge(!), high(!)1439lb # [ 0.526810] pci 0000:00:01.0: vgaarb: setting as boot VGA device1440client # [ 0.531490] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1441lb # [ 0.527508] pci 0000:00:01.0: vgaarb: bridge control possible1442client # [ 0.532958] pnp: PnP ACPI: found 6 devices1443lb # [ 0.527508] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1444lb # [ 0.527516] vgaarb: loaded1445lb # [ 0.528279] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01446worker1 # [ 0.548451] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1447lb # [ 0.528514] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1448worker1 # [ 0.549888] clocksource: Switched to clocksource acpi_pm1449worker1 # [ 0.550879] NET: Registered PF_INET protocol family1450worker1 # [ 0.552081] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1451client # [ 0.540545] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1452lb # [ 0.532598] clocksource: Switched to clocksource kvm-clock1453client # [ 0.541967] clocksource: Switched to clocksource acpi_pm1454client # [ 0.542911] NET: Registered PF_INET protocol family1455lb # [ 0.534219] VFS: Disk quotas dquot_6.6.01456client # [ 0.544030] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1457worker2 # [ 0.535786] pci 0000:00:01.0: vgaarb: setting as boot VGA device1458lb # [ 0.536679] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1459worker2 # [ 0.536227] pci 0000:00:01.0: vgaarb: bridge control possible1460lb # [ 0.538027] pnp: PnP ACPI init1461lb # [ 0.538818] ACPI: IRQ 4 override to edge(!), high(!)1462worker2 # [ 0.536227] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1463worker2 # [ 0.536234] vgaarb: loaded1464lb # [ 0.539834] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1465worker2 # [ 0.537040] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01466lb # [ 0.541361] pnp: PnP ACPI: found 6 devices1467worker2 # [ 0.537901] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1468worker1 # [ 0.568286] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1469worker2 # [ 0.542315] clocksource: Switched to clocksource kvm-clock1470worker1 # [ 0.569884] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1471lb # [ 0.548914] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1472worker1 # [ 0.571237] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1473lb # [ 0.550341] clocksource: Switched to clocksource acpi_pm1474worker2 # [ 0.545721] VFS: Disk quotas dquot_6.6.01475worker1 # [ 0.572596] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1476lb # [ 0.551265] NET: Registered PF_INET protocol family1477worker2 # [ 0.546421] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1478worker1 # [ 0.573831] TCP: Hash tables configured (established 16384 bind 16384)1479client # [ 0.560356] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1480worker2 # [ 0.547724] pnp: PnP ACPI init1481lb # [ 0.552386] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1482worker1 # [ 0.574927] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1483worker2 # [ 0.548499] ACPI: IRQ 4 override to edge(!), high(!)1484client # [ 0.561853] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1485worker1 # [ 0.576191] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1486client # [ 0.563115] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1487worker2 # [ 0.549451] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1488worker1 # [ 0.577243] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1489worker2 # [ 0.550960] pnp: PnP ACPI: found 6 devices1490client # [ 0.564418] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1491worker1 # [ 0.578463] NET: Registered PF_UNIX/PF_LOCAL protocol family1492client # [ 0.565606] TCP: Hash tables configured (established 16384 bind 16384)1493worker1 # [ 0.579472] NET: Registered PF_XDP protocol family1494client # [ 0.566696] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1495worker1 # [ 0.580305] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1496client # [ 0.567916] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1497worker1 # [ 0.581329] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1498client # [ 0.568942] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1499worker1 # [ 0.582355] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1500client # [ 0.570082] NET: Registered PF_UNIX/PF_LOCAL protocol family1501worker1 # [ 0.583509] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1502client # [ 0.571005] NET: Registered PF_XDP protocol family1503worker1 # [ 0.584669] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1504client # [ 0.571824] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1505worker2 # [ 0.558482] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1506worker1 # [ 0.585812] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1507client # [ 0.572795] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1508worker2 # [ 0.559915] clocksource: Switched to clocksource acpi_pm1509client # [ 0.573744] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1510worker1 # [ 0.587472] ACPI: \_SB_.GSIA: Enabled at IRQ 161511worker2 # [ 0.560903] NET: Registered PF_INET protocol family1512client # [ 0.574805] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1513client # [ 0.575860] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1514worker2 # [ 0.562079] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1515worker1 # [ 0.589873] ACPI: \_SB_.GSIB: Enabled at IRQ 171516client # [ 0.576914] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1517lb # [ 0.568454] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1518client # [ 0.578598] ACPI: \_SB_.GSIA: Enabled at IRQ 161519worker1 # [ 0.592014] ACPI: \_SB_.GSIC: Enabled at IRQ 181520lb # [ 0.569968] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1521lb # [ 0.571238] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1522client # [ 0.580707] ACPI: \_SB_.GSIB: Enabled at IRQ 171523worker1 # [ 0.594069] ACPI: \_SB_.GSID: Enabled at IRQ 191524lb # [ 0.572563] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1525worker1 # [ 0.595734] PCI: CLS 0 bytes, default 641526client # [ 0.582566] ACPI: \_SB_.GSIC: Enabled at IRQ 181527lb # [ 0.573790] TCP: Hash tables configured (established 16384 bind 16384)1528lb # [ 0.574879] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1529worker1 # [ 0.596712] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1530client # [ 0.584446] ACPI: \_SB_.GSID: Enabled at IRQ 191531lb # [ 0.576099] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1532worker1 # [ 0.598466] Trying to unpack rootfs image as initramfs...1533client # [ 0.586025] PCI: CLS 0 bytes, default 641534lb # [ 0.577164] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1535lb # [ 0.578356] NET: Registered PF_UNIX/PF_LOCAL protocol family1536client # [ 0.586954] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1537lb # [ 0.579296] NET: Registered PF_XDP protocol family1538client # [ 0.588708] Trying to unpack rootfs image as initramfs...1539lb # [ 0.580077] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1540lb # [ 0.581130] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1541lb # [ 0.582098] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1542lb # [ 0.583198] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1543worker2 # [ 0.578095] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1544lb # [ 0.584266] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1545worker2 # [ 0.579628] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1546lb # [ 0.585353] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1547worker2 # [ 0.580968] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1548lb # [ 0.587018] ACPI: \_SB_.GSIA: Enabled at IRQ 161549worker2 # [ 0.582370] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1550worker2 # [ 0.583672] TCP: Hash tables configured (established 16384 bind 16384)1551lb # [ 0.589596] ACPI: \_SB_.GSIB: Enabled at IRQ 171552worker2 # [ 0.584845] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1553worker2 # [ 0.586154] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1554lb # [ 0.591452] ACPI: \_SB_.GSIC: Enabled at IRQ 181555worker2 # [ 0.587266] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1556lb # [ 0.593381] ACPI: \_SB_.GSID: Enabled at IRQ 191557worker2 # [ 0.588595] NET: Registered PF_UNIX/PF_LOCAL protocol family1558worker2 # [ 0.589600] NET: Registered PF_XDP protocol family1559lb # [ 0.594970] PCI: CLS 0 bytes, default 641560worker2 # [ 0.590475] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1561lb # [ 0.595852] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1562worker2 # [ 0.591502] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1563lb # [ 0.597531] Trying to unpack rootfs image as initramfs...1564worker2 # [ 0.592539] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1565worker2 # [ 0.593672] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1566worker2 # [ 0.594825] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1567worker2 # [ 0.595977] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1568worker2 # [ 0.597779] ACPI: \_SB_.GSIA: Enabled at IRQ 161569worker2 # [ 0.600235] ACPI: \_SB_.GSIB: Enabled at IRQ 171570worker2 # [ 0.602092] ACPI: \_SB_.GSIC: Enabled at IRQ 181571worker2 # [ 0.604004] ACPI: \_SB_.GSID: Enabled at IRQ 191572worker2 # [ 0.605644] PCI: CLS 0 bytes, default 641573worker2 # [ 0.606543] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1574worker2 # [ 0.608307] Trying to unpack rootfs image as initramfs...1575worker1 # [ 0.644895] Initialise system trusted keyrings1576client # [ 0.632671] Initialise system trusted keyrings1577worker1 # [ 0.645863] workingset: timestamp_bits=40 max_order=19 bucket_order=01578client # [ 0.635467] workingset: timestamp_bits=40 max_order=19 bucket_order=01579lb # [ 0.643722] Initialise system trusted keyrings1580lb # [ 0.646350] workingset: timestamp_bits=40 max_order=19 bucket_order=01581worker1 # [ 0.669925] Key type asymmetric registered1582worker1 # [ 0.670688] Asymmetric key parser 'x509' registered1583client # [ 0.657626] Key type asymmetric registered1584client # [ 0.658343] Asymmetric key parser 'x509' registered1585client # [ 0.659267] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1586worker1 # [ 0.673498] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1587worker1 # [ 0.674958] io scheduler mq-deadline registered1588worker1 # [ 0.675784] io scheduler kyber registered1589client # [ 0.664644] io scheduler mq-deadline registered1590worker1 # [ 0.678977] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1591client # [ 0.667434] io scheduler kyber registered1592worker1 # [ 0.680362] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1593worker2 # [ 0.654752] Initialise system trusted keyrings1594client # [ 0.668733] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1595worker2 # [ 0.655694] workingset: timestamp_bits=40 max_order=19 bucket_order=01596client # [ 0.670019] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1597worker1 # [ 0.686096] Linux agpgart interface v0.1031598worker1 # [ 0.686913] ACPI: bus type drm_connector registered1599lb # [ 0.667594] Key type asymmetric registered1600worker1 # [ 0.689878] usbcore: registered new interface driver usbserial_generic1601lb # [ 0.668338] Asymmetric key parser 'x509' registered1602worker1 # [ 0.690992] usbserial: USB Serial support registered for generic1603lb # [ 0.669169] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1604client # [ 0.680700] Linux agpgart interface v0.1031605worker1 # [ 0.693474] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1606client # [ 0.681520] ACPI: bus type drm_connector registered1607worker1 # [ 0.693474] Please enable it if your BIOS has the CPPC option.1608worker1 # [ 0.695797] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1609lb # [ 0.674404] io scheduler mq-deadline registered1610lb # [ 0.675157] io scheduler kyber registered1611client # [ 0.684795] usbcore: registered new interface driver usbserial_generic1612client # [ 0.685932] usbserial: USB Serial support registered for generic1613worker1 # [ 0.699626] drop_monitor: Initializing network drop monitor service1614lb # [ 0.677813] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1615worker1 # [ 0.700797] NET: Registered PF_INET6 protocol family1616lb # [ 0.679054] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1617client # [ 0.689408] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1618client # [ 0.689408] Please enable it if your BIOS has the CPPC option.1619worker2 # [ 0.677424] Key type asymmetric registered1620worker1 # [ 0.704886] Segment Routing with IPv61621worker2 # [ 0.678117] Asymmetric key parser 'x509' registered1622client # [ 0.691688] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1623worker2 # [ 0.678962] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1624lb # [ 0.684688] Linux agpgart interface v0.1031625worker1 # [ 0.707464] In-situ OAM (IOAM) with IPv61626lb # [ 0.685468] ACPI: bus type drm_connector registered1627client # [ 0.694539] drop_monitor: Initializing network drop monitor service1628worker1 # [ 0.708493] IPI shorthand broadcast: enabled1629client # [ 0.695739] NET: Registered PF_INET6 protocol family1630lb # [ 0.688751] usbcore: registered new interface driver usbserial_generic1631worker2 # [ 0.684608] io scheduler mq-deadline registered1632client # [ 0.698863] Segment Routing with IPv61633lb # [ 0.689811] usbserial: USB Serial support registered for generic1634worker2 # [ 0.687340] io scheduler kyber registered1635client # [ 0.701441] In-situ OAM (IOAM) with IPv61636client # [ 0.702532] IPI shorthand broadcast: enabled1637worker2 # [ 0.688677] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1638lb # [ 0.693316] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1639lb # [ 0.693316] Please enable it if your BIOS has the CPPC option.1640worker1 # [ 0.716547] sched_clock: Marking stable (581016700, 135429642)->(790829786, -74383444)1641worker2 # [ 0.689905] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1642lb # [ 0.695472] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1643lb # [ 0.696847] drop_monitor: Initializing network drop monitor service1644worker1 # [ 0.719530] registered taskstats version 11645lb # [ 0.697973] NET: Registered PF_INET6 protocol family1646worker1 # [ 0.720516] Loading compiled-in X.509 certificates1647worker2 # [ 0.696481] Linux agpgart interface v0.1031648lb # [ 0.701785] Segment Routing with IPv61649worker2 # [ 0.697282] ACPI: bus type drm_connector registered1650client # [ 0.710747] sched_clock: Marking stable (576013778, 134384088)->(785517262, -75119396)1651lb # [ 0.705315] In-situ OAM (IOAM) with IPv61652worker2 # [ 0.700472] usbcore: registered new interface driver usbserial_generic1653lb # [ 0.706313] IPI shorthand broadcast: enabled1654client # [ 0.715607] registered taskstats version 11655worker2 # [ 0.701597] usbserial: USB Serial support registered for generic1656client # [ 0.716572] Loading compiled-in X.509 certificates1657worker2 # [ 0.704347] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1658worker2 # [ 0.704347] Please enable it if your BIOS has the CPPC option.1659worker2 # [ 0.706654] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1660lb # [ 0.712239] sched_clock: Marking stable (577014262, 134274628)->(789639401, -78350511)1661worker1 # [ 0.738463] Demotion targets for Node 0: null1662lb # [ 0.716456] registered taskstats version 11663worker2 # [ 0.711474] drop_monitor: Initializing network drop monitor service1664lb # [ 0.717424] Loading compiled-in X.509 certificates1665worker2 # [ 0.712625] NET: Registered PF_INET6 protocol family1666worker1 # [ 0.740500] Key type .fscrypt registered1667worker1 # [ 0.741167] Key type fscrypt-provisioning registered1668worker1 # [ 0.742128] ima: No TPM chip found, activating TPM-bypass!1669worker2 # [ 0.715764] Segment Routing with IPv61670worker2 # [ 0.716460] In-situ OAM (IOAM) with IPv61671worker1 # [ 0.746463] ima: Allocated hash algorithm: sha11672worker2 # [ 0.719593] IPI shorthand broadcast: enabled1673worker1 # [ 0.747245] ima: No architecture policies found1674client # [ 0.735416] Demotion targets for Node 0: null1675worker1 # [ 0.749455] PM: Magic number: 10:919:8811676client # [ 0.736334] Key type .fscrypt registered1677client # [ 0.737030] Key type fscrypt-provisioning registered1678worker1 # [ 0.751034] RAS: Correctable Errors collector initialized.1679client # [ 0.738014] ima: No TPM chip found, activating TPM-bypass!1680worker2 # [ 0.727512] sched_clock: Marking stable (593015114, 134316048)->(797983560, -70652398)1681client # [ 0.742427] ima: Allocated hash algorithm: sha11682client # [ 0.743268] ima: No architecture policies found1683worker2 # [ 0.730411] registered taskstats version 11684client # [ 0.744283] PM: Magic number: 10:919:8811685lb # [ 0.736303] Demotion targets for Node 0: null1686worker2 # [ 0.731345] Loading compiled-in X.509 certificates1687lb # [ 0.737135] Key type .fscrypt registered1688lb # [ 0.737861] Key type fscrypt-provisioning registered1689worker1 # [ 0.761135] clk: Disabling unused clocks1690lb # [ 0.738851] ima: No TPM chip found, activating TPM-bypass!1691client # [ 0.748025] RAS: Correctable Errors collector initialized.1692worker1 # [ 0.763464] PM: genpd: Disabling unused power domains1693lb # [ 0.743305] ima: Allocated hash algorithm: sha11694lb # [ 0.744113] ima: No architecture policies found1695lb # [ 0.745073] PM: Magic number: 10:919:8811696client # [ 0.756670] clk: Disabling unused clocks1697lb # [ 0.748716] RAS: Correctable Errors collector initialized.1698client # [ 0.759434] PM: genpd: Disabling unused power domains1699worker2 # [ 0.749344] Demotion targets for Node 0: null1700worker2 # [ 0.750212] Key type .fscrypt registered1701lb # [ 0.757403] clk: Disabling unused clocks1702lb # [ 0.758143] PM: genpd: Disabling unused power domains1703worker2 # [ 0.753352] Key type fscrypt-provisioning registered1704worker2 # [ 0.754259] ima: No TPM chip found, activating TPM-bypass!1705worker2 # [ 0.755197] ima: Allocated hash algorithm: sha11706worker2 # [ 0.757370] ima: No architecture policies found1707worker2 # [ 0.758304] PM: Magic number: 10:919:8811708worker2 # [ 0.761943] RAS: Correctable Errors collector initialized.1709worker2 # [ 0.770621] clk: Disabling unused clocks1710worker2 # [ 0.773346] PM: genpd: Disabling unused power domains1711worker1 # [ 0.918207] Freeing initrd memory: 29092K1712worker1 # [ 0.921288] Freeing unused decrypted memory: 2028K1713worker1 # [ 0.923840] Freeing unused kernel image (initmem) memory: 3644K1714worker1 # [ 0.924967] Write protecting the kernel read-only data: 32768k1715worker1 # [ 0.926841] Freeing unused kernel image (text/rodata gap) memory: 1208K1716worker1 # [ 0.928349] Freeing unused kernel image (rodata/data gap) memory: 752K1717client # [ 0.921901] Freeing initrd memory: 29096K1718lb # [ 0.913606] Freeing initrd memory: 29092K1719client # [ 0.924993] Freeing unused decrypted memory: 2028K1720lb # [ 0.916722] Freeing unused decrypted memory: 2028K1721client # [ 0.927561] Freeing unused kernel image (initmem) memory: 3644K1722lb # [ 0.919305] Freeing unused kernel image (initmem) memory: 3644K1723client # [ 0.928677] Write protecting the kernel read-only data: 32768k1724lb # [ 0.920400] Write protecting the kernel read-only data: 32768k1725client # [ 0.930557] Freeing unused kernel image (text/rodata gap) memory: 1208K1726lb # [ 0.922246] Freeing unused kernel image (text/rodata gap) memory: 1208K1727client # [ 0.932091] Freeing unused kernel image (rodata/data gap) memory: 752K1728lb # [ 0.923773] Freeing unused kernel image (rodata/data gap) memory: 752K1729worker2 # [ 0.926996] Freeing initrd memory: 29092K1730worker2 # [ 0.930092] Freeing unused decrypted memory: 2028K1731worker2 # [ 0.932642] Freeing unused kernel image (initmem) memory: 3644K1732worker2 # [ 0.933742] Write protecting the kernel read-only data: 32768k1733worker2 # [ 0.935626] Freeing unused kernel image (text/rodata gap) memory: 1208K1734worker2 # [ 0.937139] Freeing unused kernel image (rodata/data gap) memory: 752K1735worker1 # [ 0.979422] x86/mm: Checked W+X mappings: passed, no W+X pages found.1736worker1 # [ 0.980558] Run /init as init process1737worker1 # [ 0.990364] systemd[1]: Inserted module 'autofs4'1738client # [ 0.983257] x86/mm: Checked W+X mappings: passed, no W+X pages found.1739lb # [ 0.974850] x86/mm: Checked W+X mappings: passed, no W+X pages found.1740client # [ 0.984356] Run /init as init process1741lb # [ 0.975930] Run /init as init process1742worker1 # [ 1.006540] fuse: init (API version 7.45)1743client # [ 0.994523] systemd[1]: Inserted module 'autofs4'1744lb # [ 0.985680] systemd[1]: Inserted module 'autofs4'1745worker1 # [ 1.012304] ACPI: \_SB_.GSIG: Enabled at IRQ 221746worker1 # [ 1.014670] ACPI: \_SB_.GSIH: Enabled at IRQ 231747worker2 # [ 0.988405] x86/mm: Checked W+X mappings: passed, no W+X pages found.1748worker2 # [ 0.989495] Run /init as init process1749worker1 # [ 1.018172] ACPI: \_SB_.GSIE: Enabled at IRQ 201750worker1 # [ 1.020868] ACPI: \_SB_.GSIF: Enabled at IRQ 211751lb # [ 1.001802] fuse: init (API version 7.45)1752client # [ 1.010689] fuse: init (API version 7.45)1753worker2 # [ 0.999356] systemd[1]: Inserted module 'autofs4'1754lb # [ 1.007558] ACPI: \_SB_.GSIG: Enabled at IRQ 221755client # [ 1.016502] ACPI: \_SB_.GSIG: Enabled at IRQ 221756client # [ 1.018842] ACPI: \_SB_.GSIH: Enabled at IRQ 231757lb # [ 1.010124] ACPI: \_SB_.GSIH: Enabled at IRQ 231758lb # [ 1.013458] ACPI: \_SB_.GSIE: Enabled at IRQ 201759client # [ 1.022702] ACPI: \_SB_.GSIE: Enabled at IRQ 201760lb # [ 1.015802] ACPI: \_SB_.GSIF: Enabled at IRQ 211761client # [ 1.025445] ACPI: \_SB_.GSIF: Enabled at IRQ 211762worker2 # [ 1.015458] fuse: init (API version 7.45)1763worker2 # [ 1.021384] ACPI: \_SB_.GSIG: Enabled at IRQ 221764worker2 # [ 1.023839] ACPI: \_SB_.GSIH: Enabled at IRQ 231765worker1 # [ 1.052229] systemd[1]: Successfully made /usr/ read-only.1766worker2 # [ 1.026993] ACPI: \_SB_.GSIE: Enabled at IRQ 201767worker2 # [ 1.029484] ACPI: \_SB_.GSIF: Enabled at IRQ 211768lb # [ 1.047477] systemd[1]: Successfully made /usr/ read-only.1769client # [ 1.057181] systemd[1]: Successfully made /usr/ read-only.1770worker2 # [ 1.059755] systemd[1]: Successfully made /usr/ read-only.1771worker1 # [ 1.388861] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1772worker1 # [ 1.400950] systemd[1]: Detected virtualization kvm.1773worker1 # [ 1.403058] systemd[1]: Detected architecture x86-64.1774worker1 # [ 1.405526] systemd[1]: Running in initrd.1775worker1 # [ 1.408642] systemd[1]: Initializing machine ID from random generator.1776worker1 # [ 1.412118] systemd[1]: Hostname set to <worker1>.1777client # [ 1.393746] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1778lb # [ 1.384632] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1779client # [ 1.406227] systemd[1]: Detected virtualization kvm.1780client # [ 1.408493] systemd[1]: Detected architecture x86-64.1781lb # [ 1.399029] systemd[1]: Detected virtualization kvm.1782client # [ 1.410616] systemd[1]: Running in initrd.1783lb # [ 1.401805] systemd[1]: Detected architecture x86-64.1784lb # [ 1.404538] systemd[1]: Running in initrd.1785client # [ 1.413203] systemd[1]: Initializing machine ID from random generator.1786worker2 # [ 1.395447] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1787client # [ 1.416161] systemd[1]: Hostname set to <client>.1788worker2 # [ 1.403717] systemd[1]: Detected virtualization kvm.1789lb # [ 1.407916] systemd[1]: Initializing machine ID from random generator.1790worker2 # [ 1.405839] systemd[1]: Detected architecture x86-64.1791lb # [ 1.411833] systemd[1]: Hostname set to <lb>.1792worker2 # [ 1.408091] systemd[1]: Running in initrd.1793worker2 # [ 1.410643] systemd[1]: Initializing machine ID from random generator.1794worker2 # [ 1.413540] systemd[1]: Hostname set to <worker2>.1795worker1 # [ 1.641645] systemd[1]: bpf-restrict-fs: LSM BPF program attached1796lb # [ 1.627614] systemd[1]: bpf-restrict-fs: LSM BPF program attached1797client # [ 1.639299] systemd[1]: bpf-restrict-fs: LSM BPF program attached1798worker2 # [ 1.632996] systemd[1]: bpf-restrict-fs: LSM BPF program attached1799worker1 # [ 1.695981] systemd[1]: Queued start job for default target Initrd Default Target.1800worker1 # [ 1.700773] systemd[1]: Created slice Slice /system/modprobe.1801worker1 # [ 1.701989] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1802worker1 # [ 1.703520] systemd[1]: Expecting device /dev/disk/by-label/nixos...1803lb # [ 1.682041] systemd[1]: Queued start job for default target Initrd Default Target.1804worker1 # [ 1.704714] systemd[1]: Reached target Path Units.1805worker1 # [ 1.705618] systemd[1]: Reached target Slice Units.1806worker1 # [ 1.706535] systemd[1]: Reached target Swaps.1807client # [ 1.692875] systemd[1]: Queued start job for default target Initrd Default Target.1808worker1 # [ 1.707292] systemd[1]: Reached target Timer Units.1809worker1 # [ 1.708283] systemd[1]: Listening on D-Bus System Message Bus Socket.1810lb # [ 1.686731] systemd[1]: Created slice Slice /system/modprobe.1811worker1 # [ 1.709571] systemd[1]: Listening on Journal Socket (/dev/log).1812client # [ 1.696715] systemd[1]: Created slice Slice /system/modprobe.1813worker1 # [ 1.710767] systemd[1]: Listening on Journal Sockets.1814lb # [ 1.688421] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1815client # [ 1.697925] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1816worker1 # [ 1.711872] systemd[1]: Listening on udev Control Socket.1817lb # [ 1.689927] systemd[1]: Expecting device /dev/disk/by-label/nixos...1818client # [ 1.699294] systemd[1]: Expecting device /dev/disk/by-label/nixos...1819worker1 # [ 1.712895] systemd[1]: Listening on udev Kernel Socket.1820lb # [ 1.691080] systemd[1]: Reached target Path Units.1821worker2 # [ 1.685975] systemd[1]: Queued start job for default target Initrd Default Target.1822client # [ 1.700411] systemd[1]: Reached target Path Units.1823worker1 # [ 1.713820] systemd[1]: Reached target Socket Units.1824lb # [ 1.691912] systemd[1]: Reached target Slice Units.1825client # [ 1.701229] systemd[1]: Reached target Slice Units.1826lb # [ 1.692829] systemd[1]: Reached target Swaps.1827client # [ 1.702115] systemd[1]: Reached target Swaps.1828lb # [ 1.693704] systemd[1]: Reached target Timer Units.1829worker1 # [ 1.715595] systemd[1]: Starting Create List of Static Device Nodes...1830client # [ 1.702912] systemd[1]: Reached target Timer Units.1831worker2 # [ 1.689608] systemd[1]: Created slice Slice /system/modprobe.1832lb # [ 1.694795] systemd[1]: Listening on D-Bus System Message Bus Socket.1833client # [ 1.703853] systemd[1]: Listening on D-Bus System Message Bus Socket.1834lb # [ 1.696019] systemd[1]: Listening on Journal Socket (/dev/log).1835worker2 # [ 1.690816] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1836client # [ 1.705029] systemd[1]: Listening on Journal Socket (/dev/log).1837lb # [ 1.697174] systemd[1]: Listening on Journal Sockets.1838worker2 # [ 1.692179] systemd[1]: Expecting device /dev/disk/by-label/nixos...1839client # [ 1.706126] systemd[1]: Listening on Journal Sockets.1840worker2 # [ 1.693229] systemd[1]: Reached target Path Units.1841client # [ 1.707067] systemd[1]: Listening on udev Control Socket.1842lb # [ 1.698165] systemd[1]: Listening on udev Control Socket.1843worker1 # [ 1.720296] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1844worker2 # [ 1.694046] systemd[1]: Reached target Slice Units.1845client # [ 1.708014] systemd[1]: Listening on udev Kernel Socket.1846lb # [ 1.699128] systemd[1]: Listening on udev Kernel Socket.1847worker2 # [ 1.694912] systemd[1]: Reached target Swaps.1848client # [ 1.708923] systemd[1]: Reached target Socket Units.1849lb # [ 1.700043] systemd[1]: Reached target Socket Units.1850worker2 # [ 1.695709] systemd[1]: Reached target Timer Units.1851worker2 # [ 1.696682] systemd[1]: Listening on D-Bus System Message Bus Socket.1852client # [ 1.710633] systemd[1]: Starting Create List of Static Device Nodes...1853lb # [ 1.702231] systemd[1]: Starting Create List of Static Device Nodes...1854worker2 # [ 1.697926] systemd[1]: Listening on Journal Socket (/dev/log).1855worker2 # [ 1.699121] systemd[1]: Listening on Journal Sockets.1856worker2 # [ 1.700169] systemd[1]: Listening on udev Control Socket.1857worker1 # [ 1.727556] systemd[1]: Starting Load Kernel Module configfs...1858worker2 # [ 1.701196] systemd[1]: Listening on udev Kernel Socket.1859lb # [ 1.706223] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1860client # [ 1.715263] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1861worker2 # [ 1.702132] systemd[1]: Reached target Socket Units.1862worker2 # [ 1.703936] systemd[1]: Starting Create List of Static Device Nodes...1863worker2 # [ 1.708403] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1864client # [ 1.722530] systemd[1]: Starting Load Kernel Module configfs...1865lb # [ 1.715381] systemd[1]: Starting Load Kernel Module configfs...1866worker1 # [ 1.740551] systemd[1]: Starting Journal Service...1867worker2 # [ 1.716006] systemd[1]: Starting Load Kernel Module configfs...1868client # [ 1.731297] systemd[1]: Starting Journal Service...1869lb # [ 1.726515] systemd[1]: Starting Journal Service...1870worker1 # [ 1.754529] systemd[1]: Starting Load Kernel Modules...1871worker2 # [ 1.727964] systemd[1]: Starting Journal Service...1872client # [ 1.745472] systemd[1]: Starting Load Kernel Modules...1873lb # [ 1.737470] systemd[1]: Starting Load Kernel Modules...1874worker2 # [ 1.742418] systemd[1]: Starting Load Kernel Modules...1875worker1 # [ 1.761547] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1876lb # [ 1.743375] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1877client # [ 1.752479] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1878client # [ 1.770461] netfs: FS-Cache loaded1879worker1 # [ 1.785397] netfs: FS-Cache loaded1880lb # [ 1.765364] netfs: FS-Cache loaded1881client # [ 1.775165] 9pnet: Installing 9P2000 support1882worker2 # [ 1.749802] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1883worker1 # [ 1.790333] 9pnet: Installing 9P2000 support1884lb # [ 1.770052] 9pnet: Installing 9P2000 support1885worker2 # [ 1.765941] netfs: FS-Cache loaded1886client # [ 1.785675] systemd[1]: Starting Coldplug All udev Devices...1887worker1 # [ 1.800671] systemd[1]: Starting Coldplug All udev Devices...1888worker2 # [ 1.774808] 9pnet: Installing 9P2000 support1889lb # [ 1.780362] systemd[1]: Starting Coldplug All udev Devices...1890worker2 # [ 1.786112] systemd-journald[67]: Collecting audit messages is disabled.1891worker2 # [ 1.788552] systemd[1]: Starting Coldplug All udev Devices...1892client # [ 1.802494] systemd[1]: Finished Create List of Static Device Nodes.1893worker1 # [ 1.818502] systemd[1]: Finished Create List of Static Device Nodes.1894lb # [ 1.797599] systemd[1]: Finished Create List of Static Device Nodes.1895worker1 # [ 1.823403] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1896client # [ 1.811582] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1897lb # [ 1.802945] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1898worker2 # [ 1.800914] systemd[1]: Finished Create List of Static Device Nodes.1899worker1 # [ 1.830843] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1900client # [ 1.819921] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1901lb # [ 1.811732] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1902worker1 # [ 1.833922] systemd-journald[67]: Collecting audit messages is disabled.1903worker2 # [ 1.806914] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1904lb # [ 1.815247] systemd-journald[67]: Collecting audit messages is disabled.1905worker1 # [ 1.837904] systemd[1]: modprobe@configfs.service: Deactivated successfully.1906lb # [ 1.817775] systemd[1]: modprobe@configfs.service: Deactivated successfully.1907worker2 # [ 1.813743] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1908client # [ 1.829990] systemd[1]: modprobe@configfs.service: Deactivated successfully.1909worker1 # [ 1.844825] systemd[1]: Finished Load Kernel Module configfs.1910client # [ 1.833296] systemd-journald[67]: Collecting audit messages is disabled.1911worker2 # [ 1.819712] systemd[1]: modprobe@configfs.service: Deactivated successfully.1912lb # [ 1.825840] systemd[1]: Finished Load Kernel Module configfs.1913client # [ 1.835774] systemd[1]: Finished Load Kernel Module configfs.1914worker1 # [ 1.849989] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1915worker1 # [ 1.852547] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1916worker2 # [ 1.826685] systemd[1]: Finished Load Kernel Module configfs.1917lb # [ 1.830916] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1918client # [ 1.841707] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1919lb # [ 1.835224] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1920client # [ 1.845798] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1921worker2 # [ 1.832758] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1922worker1 # [ 1.861636] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1923worker1 # [ 1.863479] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1924worker2 # [ 1.838360] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1925client # [ 1.853506] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1926lb # [ 1.845580] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1927client # [ 1.856560] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1928worker2 # [ 1.844425] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1929lb # [ 1.849501] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1930worker2 # [ 1.845775] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1931worker1 # [ 1.893052] systemd[1]: Finished Load Kernel Modules.1932worker1 # [ 1.899692] systemd[1]: Starting Apply Kernel Variables...1933client # [ 1.888957] systemd[1]: Finished Load Kernel Modules.1934worker2 # [ 1.876846] systemd[1]: Finished Load Kernel Modules.1935client # [ 1.893533] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1936lb # [ 1.886077] systemd[1]: Finished Load Kernel Modules.1937worker2 # [ 1.884437] systemd[1]: Starting Apply Kernel Variables...1938worker1 # [ 1.911111] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1939client # [ 1.901662] systemd[1]: Starting Apply Kernel Variables...1940lb # [ 1.895747] systemd[1]: Starting Apply Kernel Variables...1941worker2 # [ 1.892495] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1942worker1 # [ 1.920693] systemd[1]: Starting Create Static Device Nodes in /dev...1943worker2 # [ 1.899691] systemd[1]: Started Journal Service.1944client # [ 1.914486] systemd[1]: Starting Create Static Device Nodes in /dev...1945worker2 # [ 1.763810] systemd-modules-load[68]: Inserted module 'dm_mod'1946worker2 # [ 1.768889] systemd-modules-load[68]: Inserted module 'virtio_balloon'1947worker2 # [ 1.769938] systemd-modules-load[68]: Inserted module 'virtio_gpu'1948lb # [ 1.909028] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1949worker1 # [ 1.938646] systemd[1]: Finished Apply Kernel Variables.1950worker2 # [ 1.779070] systemd[1]: Starting Create Static Device Nodes in /dev...1951lb # [ 1.921853] systemd[1]: Starting Create Static Device Nodes in /dev...1952worker2 # [ 1.789076] systemd[1]: Finished Apply Kernel Variables.1953worker1 # [ 1.955026] systemd[1]: Finished Create Static Device Nodes in /dev.1954client # [ 1.945588] systemd[1]: Finished Apply Kernel Variables.1955worker1 # [ 1.823572] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.960130] systemd[1]: Started Journal Service.1956worker1 # 1957worker1 # [ 1.826321] systemd-modules-load[68]: Inserted module 'virtio_balloon'1958client # [ 1.949499] systemd[1]: Finished Create Static Device Nodes in /dev.1959worker2 # [ 1.803056] systemd[1]: Finished Create Static Device Nodes in /dev.1960worker1 # [ 1.829279] systemd-modules-load[68]: Inserted module 'virtio_gpu'1961worker2 # [ 1.804391] systemd[1]: Reached target Preparation for Local File Systems.1962worker1 # [ 1.830294] systemd[1]: Reached target Preparation for Local File Systems.1963client # [ 1.816614] systemd-modules-load[68]: Inserted module 'dm_mod'1964worker1 # [ 1.831525] systemd[1]: Reached target Local File Systems.1965client # [ 1.819381] systemd-modules-load[68]: Inserted module 'virtio_balloon'1966worker2 # [ 1.806369] systemd[1]: Reached target Local File Systems.1967lb # [ 1.945641] systemd[1]: Finished Apply Kernel Variables.1968client # [ 1.954835] systemd[1]: Started Journal Service.1969worker1 # [ 1.833334] systemd[1]: Starting Create System Files and Directories...1970client # [ 1.821412] systemd-modules-load[68]: Inserted module 'virtio_gpu'1971worker2 # [ 1.809951] systemd[1]: Starting Create System Files and Directories...1972lb # [ 1.952437] systemd[1]: Started Journal Service.1973worker1 # [ 1.840101] systemd[1]: Starting Rule-based Manager for Device Events and Files...1974lb # [ 1.820240] systemd-modules-load[68]: Inserted module 'dm_mod'1975worker2 # [ 1.817063] systemd[1]: Starting Rule-based Manager for Device Events and Files...1976lb # [ 1.822949] systemd-modules-load[68]: Inserted module 'virtio_balloon'1977client # [ 1.831894] systemd[1]: Reached target Preparation for Local File Systems.1978client # [ 1.833671] systemd[1]: Reached target Local File Systems.1979lb # [ 1.826521] systemd-modules-load[68]: Inserted module 'virtio_gpu'1980client # [ 1.837102] systemd[1]: Starting Create System Files and Directories...1981client # [ 1.844078] systemd[1]: Starting Rule-based Manager for Device Events and Files...1982lb # [ 1.838166] systemd[1]: Finished Create Static Device Nodes in /dev.1983lb # [ 1.842081] systemd[1]: Reached target Preparation for Local File Systems.1984lb # [ 1.843140] systemd[1]: Reached target Local File Systems.1985worker2 # [ 1.840327] systemd[1]: Finished Create System Files and Directories.1986lb # [ 1.845838] systemd[1]: Starting Create System Files and Directories...1987worker1 # [ 1.869275] systemd[1]: Finished Create System Files and Directories.1988lb # [ 1.850906] systemd[1]: Starting Rule-based Manager for Device Events and Files...1989client # [ 1.866604] systemd[1]: Finished Create System Files and Directories.1990worker2 # [ 1.860433] systemd-udevd[83]: Using default interface naming scheme 'v261'.1991worker1 # [ 1.902403] systemd-udevd[77]: Using default interface naming scheme 'v261'.1992client # [ 1.890698] systemd-udevd[83]: Using default interface naming scheme 'v261'.1993lb # [ 1.882289] systemd[1]: Finished Create System Files and Directories.1994worker2 # [ 1.885315] systemd[1]: Started Rule-based Manager for Device Events and Files.1995worker1 # [ 1.926512] systemd[1]: Started Rule-based Manager for Device Events and Files.1996client # [ 1.916187] systemd[1]: Started Rule-based Manager for Device Events and Files.1997lb # [ 1.910730] systemd-udevd[83]: Using default interface naming scheme 'v261'.1998lb # [ 1.935539] systemd[1]: Started Rule-based Manager for Device Events and Files.1999worker2 # [ 1.933643] systemd[1]: Finished Coldplug All udev Devices.2000worker2 # [ 1.935704] systemd[1]: Reached target System Initialization.
loading 1,953 hidden lines…
3954lb # [ 14.863470] systemd[1]: Reached target PostgreSQL.3955worker2 # [ 15.983238] dhcpcd[690]: eth0: leased 10.0.2.15 for 86400 seconds3956worker2 # [ 15.986091] dhcpcd[690]: eth0: adding route to 10.0.2.0/243957worker2 # [ 15.988273] dhcpcd[690]: eth0: adding default route via 10.0.2.23958worker2 # [ 16.103989] systemd[1]: Started DHCP Client.3959worker2 # [ 16.106143] systemd[1]: Reached target Multi-User System.3960worker2 # [ 16.106882] systemd[1]: Startup finished in 858ms (kernel) + 4.046s (initrd) + 11.201s (userspace) = 16.105s.3961client # [ 16.651203] dhcpcd[565]: eth0: leased 10.0.2.15 for 86400 seconds3962client # [ 16.652120] dhcpcd[565]: eth0: adding route to 10.0.2.0/243963client # [ 16.652821] dhcpcd[565]: eth0: adding default route via 10.0.2.23964client # [ 16.751770] systemd[1]: Started DHCP Client.3965client # [ 16.753756] systemd[1]: Reached target Multi-User System.3966client # [ 16.754911] systemd[1]: Startup finished in 853ms (kernel) + 4.978s (initrd) + 10.922s (userspace) = 16.754s.3967lb # [ 16.800075] dhcpcd[709]: eth0: leased 10.0.2.15 for 86400 seconds3968lb # [ 16.802816] dhcpcd[709]: eth0: adding route to 10.0.2.0/243969lb # [ 16.805148] dhcpcd[709]: eth0: adding default route via 10.0.2.23970lb # [ 16.933389] systemd[1]: Started DHCP Client.3971lb # [ 16.935199] systemd[1]: Reached target Network is Online.3972lb # [ 16.939398] systemd[1]: Started Envoy reverse proxy.3973lb # [ 16.943653] systemd[1]: Started mock-oidc.service.3974lb # [ 17.061470] mock-oidc-server[911]: Mock OIDC Server running3975lb # [ 17.063496] mock-oidc-server[911]: OIDC Address: 192.168.1.2:80803976lb # [ 17.064947] mock-oidc-server[911]: Issue Address: 0.0.0.0:80813977lb # [ 17.065894] mock-oidc-server[911]: Issuer: http://192.168.1.2:8080/oidc3978lb # [ 17.067456] mock-oidc-server[911]: JWKS: http://192.168.1.2:8080/oidc/.well-known/jwks.json3979lb # [ 17.068924] mock-oidc-server[911]: Discovery: http://192.168.1.2:8080/oidc/.well-known/openid-configuration3980lb # [ 17.070559] mock-oidc-server[911]: Issue tokens: http://0.0.0.0:8081/issue?sub=...3981worker1 # [ 17.216534] dhcpcd[690]: eth0: leased 10.0.2.15 for 86400 seconds3982worker1 # [ 17.218708] dhcpcd[690]: eth0: adding route to 10.0.2.0/243983worker1 # [ 17.221073] dhcpcd[690]: eth0: adding default route via 10.0.2.23984worker1 # [ 17.332644] systemd[1]: Started DHCP Client.3985worker1 # [ 17.334619] systemd[1]: Reached target Multi-User System.3986worker1 # [ 17.335589] systemd[1]: Startup finished in 848ms (kernel) + 5.000s (initrd) + 11.485s (userspace) = 17.334s.3987lb # [ 17.537312] envoy[910]: [2026-09-20 10:51:40.461][910][info][main] [source/server/server.cc:467] initializing epoch 0 (base id=0, hot restart version=11.104)3988lb # [ 17.539303] envoy[910]: [2026-09-20 10:51:40.464][910][info][main] [source/server/server.cc:469] statically linked extensions:3989lb # [ 17.541034] envoy[910]: [2026-09-20 10:51:40.464][910][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router3990lb # [ 17.542776] envoy[910]: [2026-09-20 10:51:40.464][910][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service3991lb # [ 17.546704] envoy[910]: [2026-09-20 10:51:40.464][910][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule3992lb # [ 17.549161] envoy[910]: [2026-09-20 10:51:40.464][910][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port3993lb # [ 17.550770] envoy[910]: [2026-09-20 10:51:40.464][910][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router3994lb # [ 17.552444] envoy[910]: [2026-09-20 10:51:40.465][910][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter3995lb # [ 17.554307] envoy[910]: [2026-09-20 10:51:40.465][910][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value3996lb # [ 17.557525] envoy[910]: [2026-09-20 10:51:40.465][910][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor3997lb # [ 17.560162] envoy[910]: [2026-09-20 10:51:40.465][910][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities3998lb # [ 17.562068] envoy[910]: [2026-09-20 10:51:40.465][910][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file3999lb # [ 17.564100] envoy[910]: [2026-09-20 10:51:40.465][910][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation4000lb # [ 17.566171] envoy[910]: [2026-09-20 10:51:40.465][910][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata4001lb # [ 17.568160] envoy[910]: [2026-09-20 10:51:40.465][910][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple4002lb # [ 17.570251] envoy[910]: [2026-09-20 10:51:40.465][910][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based4003lb # [ 17.574079] envoy[910]: [2026-09-20 10:51:40.465][910][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default4004lb # [ 17.576092] envoy[910]: [2026-09-20 10:51:40.465][910][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder4005lb # [ 17.578339] envoy[910]: [2026-09-20 10:51:40.465][910][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default4006lb # [ 17.580218] envoy[910]: [2026-09-20 10:51:40.465][910][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default4007lb # [ 17.582038] envoy[910]: [2026-09-20 10:51:40.465][910][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple4008lb # [ 17.584162] envoy[910]: [2026-09-20 10:51:40.465][910][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection4009lb # [ 17.585945] envoy[910]: [2026-09-20 10:51:40.465][910][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name4010lb # [ 17.588248] envoy[910]: [2026-09-20 10:51:40.465][910][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port4011lb # [ 17.605473] envoy[910]: [2026-09-20 10:51:40.465][910][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response4012lb # [ 17.607499] envoy[910]: [2026-09-20 10:51:40.465][910][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router4013lb # [ 17.610451] envoy[910]: [2026-09-20 10:51:40.465][910][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift4014lb # [ 17.614407] envoy[910]: [2026-09-20 10:51:40.465][910][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header4015lb # [ 17.616801] envoy[910]: [2026-09-20 10:51:40.465][910][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override4016lb # [ 17.619406] envoy[910]: [2026-09-20 10:51:40.465][910][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default4017lb # [ 17.621157] envoy[910]: [2026-09-20 10:51:40.465][910][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed4018lb # [ 17.622766] envoy[910]: [2026-09-20 10:51:40.465][910][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin4019lb # [ 17.625664] envoy[910]: [2026-09-20 10:51:40.467][910][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter4020lb # [ 17.627375] envoy[910]: [2026-09-20 10:51:40.467][910][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts4021lb # [ 17.630077] envoy[910]: [2026-09-20 10:51:40.467][910][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params4022lb # [ 17.639218] envoy[910]: [2026-09-20 10:51:40.467][910][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd4023lb # [ 17.643659] envoy[910]: [2026-09-20 10:51:40.467][910][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json4024lb # [ 17.645630] envoy[910]: [2026-09-20 10:51:40.467][910][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid4025lb # [ 17.647250] envoy[910]: [2026-09-20 10:51:40.467][910][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap4026lb # [ 17.651729] envoy[910]: [2026-09-20 10:51:40.482][910][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san4027lb # [ 17.658496] envoy[910]: [2026-09-20 10:51:40.482][910][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls4028lb # [ 17.664053] envoy[910]: [2026-09-20 10:51:40.482][910][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe4029lb # [ 17.666537] envoy[910]: [2026-09-20 10:51:40.482][910][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind4030lb # [ 17.668281] envoy[910]: [2026-09-20 10:51:40.482][910][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re24031lb # [ 17.670034] envoy[910]: [2026-09-20 10:51:40.482][910][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain4032lb # [ 17.672045] envoy[910]: [2026-09-20 10:51:40.482][910][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router4033lb # [ 17.697674] envoy[910]: [2026-09-20 10:51:40.484][910][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats4034lb # [ 17.700073] envoy[910]: [2026-09-20 10:51:40.484][910][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher4035lb # [ 17.702438] envoy[910]: [2026-09-20 10:51:40.484][910][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4036lb # [ 17.704876] envoy[910]: [2026-09-20 10:51:40.484][910][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default4037lb # [ 17.706663] envoy[910]: [2026-09-20 10:51:40.485][910][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality4038lb # [ 17.713140] envoy[910]: [2026-09-20 10:51:40.485][910][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input4039lb # [ 17.714992] envoy[910]: [2026-09-20 10:51:40.485][910][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls4040lb # [ 17.719285] envoy[910]: [2026-09-20 10:51:40.486][910][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4041lb # [ 17.721384] envoy[910]: [2026-09-20 10:51:40.486][910][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche4042lb # [ 17.723408] envoy[910]: [2026-09-20 10:51:40.486][910][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory4043lb # [ 17.726108] envoy[910]: [2026-09-20 10:51:40.486][910][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource4044lb # [ 17.728883] envoy[910]: [2026-09-20 10:51:40.486][910][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher4045lb # [ 17.730762] envoy[910]: [2026-09-20 10:51:40.486][910][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based4046lb # [ 17.732986] envoy[910]: [2026-09-20 10:51:40.486][910][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation4047lb # [ 17.734930] envoy[910]: [2026-09-20 10:51:40.486][910][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal4048lb # [ 17.736530] envoy[910]: [2026-09-20 10:51:40.486][910][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic4049lb # [ 17.738248] envoy[910]: [2026-09-20 10:51:40.486][910][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection4050lb # [ 17.742701] envoy[910]: [2026-09-20 10:51:40.486][910][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme4051lb # [ 17.745734] envoy[910]: [2026-09-20 10:51:40.488][910][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector4052lb # [ 17.751503] envoy[910]: [2026-09-20 10:51:40.488][910][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4053lb # [ 17.753930] envoy[910]: [2026-09-20 10:51:40.488][910][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state4054lb # [ 17.756833] envoy[910]: [2026-09-20 10:51:40.488][910][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor4055lb # [ 17.759303] envoy[910]: [2026-09-20 10:51:40.488][910][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy4056lb # [ 17.761855] envoy[910]: [2026-09-20 10:51:40.488][910][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input4057lb # [ 17.763729] envoy[910]: [2026-09-20 10:51:40.488][910][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction4058lb # [ 17.767886] envoy[910]: [2026-09-20 10:51:40.488][910][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua4059lb # [ 17.769530] envoy[910]: [2026-09-20 10:51:40.488][910][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory4060lb # [ 17.771899] envoy[910]: [2026-09-20 10:51:40.488][910][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name4061lb # [ 17.774242] envoy[910]: [2026-09-20 10:51:40.488][910][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config4062lb # [ 17.777507] envoy[910]: [2026-09-20 10:51:40.488][910][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth24063lb # [ 17.779807] envoy[910]: [2026-09-20 10:51:40.488][910][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed4064lb # [ 17.782186] envoy[910]: [2026-09-20 10:51:40.488][910][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle4065lb # [ 17.784167] envoy[910]: [2026-09-20 10:51:40.488][910][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options4066lb # [ 17.788414] envoy[910]: [2026-09-20 10:51:40.488][910][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr4067lb # [ 17.790203] envoy[910]: [2026-09-20 10:51:40.488][910][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit4068lb # [ 17.792695] envoy[910]: [2026-09-20 10:51:40.488][910][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4069lb # [ 17.794733] envoy[910]: [2026-09-20 10:51:40.488][910][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff4070lb # [ 17.797066] envoy[910]: [2026-09-20 10:51:40.488][910][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case4071lb # [ 17.799293] envoy[910]: [2026-09-20 10:51:40.488][910][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override4072lb # [ 17.801813] envoy[910]: [2026-09-20 10:51:40.488][910][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso4073lb # [ 17.803875] envoy[910]: [2026-09-20 10:51:40.488][910][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default4074lb # [ 17.805872] envoy[910]: [2026-09-20 10:51:40.488][910][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest4075lb # [ 17.810583] envoy[910]: [2026-09-20 10:51:40.488][910][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default4076lb # [ 17.812182] envoy[910]: [2026-09-20 10:51:40.488][910][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable4077lb # [ 17.814373] envoy[910]: [2026-09-20 10:51:40.488][910][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm4078lb # [ 17.820264] envoy[910]: [2026-09-20 10:51:40.488][910][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator4079lb # [ 17.822599] envoy[910]: [2026-09-20 10:51:40.488][910][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated4080lb # [ 17.824754] envoy[910]: [2026-09-20 10:51:40.488][910][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log4081lb # [ 17.830736] envoy[910]: [2026-09-20 10:51:40.488][910][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert4082lb # [ 17.832453] envoy[910]: [2026-09-20 10:51:40.488][910][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http14083lb # [ 17.834510] envoy[910]: [2026-09-20 10:51:40.498][910][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default4084lb # [ 17.836155] envoy[910]: [2026-09-20 10:51:40.498][910][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default4085lb # [ 17.837645] envoy[910]: [2026-09-20 10:51:40.498][910][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy4086lb # [ 17.848547] envoy[910]: [2026-09-20 10:51:40.498][910][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action4087lb # [ 17.850385] envoy[910]: [2026-09-20 10:51:40.498][910][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy4088lb # [ 17.852820] envoy[910]: [2026-09-20 10:51:40.498][910][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v84089lb # [ 17.854693] envoy[910]: [2026-09-20 10:51:40.500][910][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query4090lb # [ 17.857842] envoy[910]: [2026-09-20 10:51:40.500][910][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator4091lb # [ 17.860337] envoy[910]: [2026-09-20 10:51:40.500][910][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian24092lb # [ 17.862065] envoy[910]: [2026-09-20 10:51:40.503][910][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo4093lb # [ 17.863768] envoy[910]: [2026-09-20 10:51:40.583][910][info][main] [source/server/server.cc:531] HTTP header map info:4094lb # [ 17.865205] envoy[910]: [2026-09-20 10:51:40.648][910][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id4095lb # [ 17.878295] envoy[910]: [2026-09-20 10:51:40.648][910][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:4096lb # [ 17.879753] envoy[910]: [2026-09-20 10:51:40.648][910][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id4097lb # [ 17.887376] envoy[910]: [2026-09-20 10:51:40.648][910][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status4098lb # [ 17.919647] envoy[910]: [2026-09-20 10:51:40.844][910][info][main] [source/server/server.cc:968] runtime: {}4099lb # [ 17.926689] envoy[910]: [2026-09-20 10:51:40.851][910][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.4100lb # [ 17.928877] envoy[910]: [2026-09-20 10:51:40.853][910][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99014101lb # [ 17.931553] envoy[910]: [2026-09-20 10:51:40.856][910][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99014102lb # [ 17.934911] envoy[910]: [2026-09-20 10:51:40.859][910][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration4103lb # [ 17.936556] envoy[910]: [2026-09-20 10:51:40.861][910][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)4104lb # [ 17.938197] envoy[910]: [2026-09-20 10:51:40.861][910][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)4105lb # [ 17.989821] envoy[910]: [2026-09-20 10:51:40.914][910][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)4106worker1 # [ 18.013140] systemd[1]: Starting Nix worker-protocol over gRPC...4107worker2 # [ 17.993096] systemd[1]: Starting Nix worker-protocol over gRPC...4108lb # [ 18.014719] envoy[910]: [2026-09-20 10:51:40.939][910][info][config] [source/server/configuration_impl.cc:164] loading stats configuration4109lb # [ 18.017078] envoy[910]: [2026-09-20 10:51:40.941][910][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.4110lb # [ 18.020030] envoy[910]: [2026-09-20 10:51:40.944][910][info][main] [source/server/server.cc:1077] starting main dispatch loop4111worker1 # [ 18.217663] nix-grpc-daemon[782]: ts=2026-09-20T10:51:41Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4112worker1 # [ 18.230995] nix-grpc-daemon[782]: ts=2026-09-20T10:51:41Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14113worker2 # [ 18.214307] nix-grpc-daemon[783]: ts=2026-09-20T10:51:40Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4114worker2 # [ 18.226569] nix-grpc-daemon[783]: ts=2026-09-20T10:51:40Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14115worker1 # [ 18.268585] nix-grpc-daemon[782]: ts=2026-09-20T10:51:41Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4116worker1 # [ 18.273774] systemd[1]: Started Nix worker-protocol over gRPC.4117worker2 # [ 18.266789] nix-grpc-daemon[783]: ts=2026-09-20T10:51:40Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4118worker2 # [ 18.270845] systemd[1]: Started Nix worker-protocol over gRPC.4119lb # [ 18.304783] envoy[910]: [2026-09-20 10:51:41.229][910][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization4120lb # [ 18.309252] envoy[910]: [2026-09-20 10:51:41.234][910][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized4121lb # [ 18.313618] envoy[910]: [2026-09-20 10:51:41.234][910][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager4122lb # [ 18.317782] envoy[910]: [2026-09-20 10:51:41.238][910][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers4123worker1 # [ 18.341159] nix-grpc-daemon[782]: ts=2026-09-20T10:51:41Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4124worker2 # [ 18.341716] nix-grpc-daemon[783]: ts=2026-09-20T10:51:40Z level=info event=scheduler_connected addr=lb:500514125worker2 # [ 18.353129] nix-grpc-daemon[783]: ts=2026-09-20T10:51:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4126worker1 # [ 18.542512] nix-grpc-daemon[782]: ts=2026-09-20T10:51:41Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4127worker2 # [ 18.554328] nix-grpc-daemon[783]: ts=2026-09-20T10:51:40Z level=info event=scheduler_connected addr=lb:500514128worker2 # [ 18.558473] nix-grpc-daemon[783]: ts=2026-09-20T10:51:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4129worker2 # [ 18.759201] nix-grpc-daemon[783]: ts=2026-09-20T10:51:41Z level=info event=scheduler_connected addr=lb:500514130worker2 # [ 18.763125] nix-grpc-daemon[783]: ts=2026-09-20T10:51:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4131worker1 # [ 18.943280] nix-grpc-daemon[782]: ts=2026-09-20T10:51:42Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4132worker2 # [ 18.964093] nix-grpc-daemon[783]: ts=2026-09-20T10:51:41Z level=info event=scheduler_connected addr=lb:500514133worker2 # [ 18.967957] nix-grpc-daemon[783]: ts=2026-09-20T10:51:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4134worker2 # [ 19.168812] nix-grpc-daemon[783]: ts=2026-09-20T10:51:41Z level=info event=scheduler_connected addr=lb:500514135worker2 # [ 19.172731] nix-grpc-daemon[783]: ts=2026-09-20T10:51:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4136worker2 # [ 19.374359] nix-grpc-daemon[783]: ts=2026-09-20T10:51:41Z level=info event=scheduler_connected addr=lb:500514137worker2 # [ 19.377939] nix-grpc-daemon[783]: ts=2026-09-20T10:51:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4138worker2 # [ 19.578482] nix-grpc-daemon[783]: ts=2026-09-20T10:51:41Z level=info event=scheduler_connected addr=lb:500514139worker2 # [ 19.582505] nix-grpc-daemon[783]: ts=2026-09-20T10:51:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4140worker1 # [ 19.747280] nix-grpc-daemon[782]: ts=2026-09-20T10:51:42Z level=info event=scheduler_connected addr=lb:500514141worker1 # [ 19.751803] nix-grpc-daemon[782]: ts=2026-09-20T10:51:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4142worker2 # [ 19.783437] nix-grpc-daemon[783]: ts=2026-09-20T10:51:42Z level=info event=scheduler_connected addr=lb:500514143worker2 # [ 19.787266] nix-grpc-daemon[783]: ts=2026-09-20T10:51:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4144worker1 # [ 19.953139] nix-grpc-daemon[782]: ts=2026-09-20T10:51:43Z level=info event=scheduler_connected addr=lb:500514145worker1 # [ 19.957084] nix-grpc-daemon[782]: ts=2026-09-20T10:51:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4146worker2 # [ 19.987996] nix-grpc-daemon[783]: ts=2026-09-20T10:51:42Z level=info event=scheduler_connected addr=lb:500514147worker2 # [ 19.991888] nix-grpc-daemon[783]: ts=2026-09-20T10:51:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4148worker1 # [ 20.157974] nix-grpc-daemon[782]: ts=2026-09-20T10:51:43Z level=info event=scheduler_connected addr=lb:500514149worker1 # [ 20.161895] nix-grpc-daemon[782]: ts=2026-09-20T10:51:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4150worker2 # [ 20.192797] nix-grpc-daemon[783]: ts=2026-09-20T10:51:42Z level=info event=scheduler_connected addr=lb:500514151worker2 # [ 20.196939] nix-grpc-daemon[783]: ts=2026-09-20T10:51:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4152worker1 # [ 20.362488] nix-grpc-daemon[782]: ts=2026-09-20T10:51:43Z level=info event=scheduler_connected addr=lb:500514153worker1 # [ 20.366441] nix-grpc-daemon[782]: ts=2026-09-20T10:51:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4154worker2 # [ 20.398449] nix-grpc-daemon[783]: ts=2026-09-20T10:51:42Z level=info event=scheduler_connected addr=lb:500514155worker2 # [ 20.401911] nix-grpc-daemon[783]: ts=2026-09-20T10:51:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4156worker1 # [ 20.567314] nix-grpc-daemon[782]: ts=2026-09-20T10:51:43Z level=info event=scheduler_connected addr=lb:500514157worker1 # [ 20.571223] nix-grpc-daemon[782]: ts=2026-09-20T10:51:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4158worker2 # [ 20.603078] nix-grpc-daemon[783]: ts=2026-09-20T10:51:42Z level=info event=scheduler_connected addr=lb:500514159worker2 # [ 20.607116] nix-grpc-daemon[783]: ts=2026-09-20T10:51:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4160worker1 # [ 20.773417] nix-grpc-daemon[782]: ts=2026-09-20T10:51:43Z level=info event=scheduler_connected addr=lb:500514161worker1 # [ 20.776516] nix-grpc-daemon[782]: ts=2026-09-20T10:51:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4162worker2 # [ 20.807805] nix-grpc-daemon[783]: ts=2026-09-20T10:51:43Z level=info event=scheduler_connected addr=lb:500514163worker2 # [ 20.811751] nix-grpc-daemon[783]: ts=2026-09-20T10:51:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4164worker1 # [ 20.974225] nix-grpc-daemon[782]: ts=2026-09-20T10:51:44Z level=info event=scheduler_connected addr=lb:500514165worker1 # [ 20.978251] nix-grpc-daemon[782]: ts=2026-09-20T10:51:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4166worker2 # [ 21.012752] nix-grpc-daemon[783]: ts=2026-09-20T10:51:43Z level=info event=scheduler_connected addr=lb:500514167worker2 # [ 21.016714] nix-grpc-daemon[783]: ts=2026-09-20T10:51:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4168worker1 # [ 21.179096] nix-grpc-daemon[782]: ts=2026-09-20T10:51:44Z level=info event=scheduler_connected addr=lb:500514169worker1 # [ 21.183159] nix-grpc-daemon[782]: ts=2026-09-20T10:51:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4170worker2 # [ 21.217485] nix-grpc-daemon[783]: ts=2026-09-20T10:51:43Z level=info event=scheduler_connected addr=lb:500514171worker2 # [ 21.221292] nix-grpc-daemon[783]: ts=2026-09-20T10:51:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4172worker1 # [ 21.383930] nix-grpc-daemon[782]: ts=2026-09-20T10:51:44Z level=info event=scheduler_connected addr=lb:500514173worker1 # [ 21.387850] nix-grpc-daemon[782]: ts=2026-09-20T10:51:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4174worker2 # [ 21.423433] nix-grpc-daemon[783]: ts=2026-09-20T10:51:43Z level=info event=scheduler_connected addr=lb:500514175worker2 # [ 21.426586] nix-grpc-daemon[783]: ts=2026-09-20T10:51:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4176worker1 # [ 21.588746] nix-grpc-daemon[782]: ts=2026-09-20T10:51:44Z level=info event=scheduler_connected addr=lb:500514177worker1 # [ 21.592550] nix-grpc-daemon[782]: ts=2026-09-20T10:51:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4178worker2 # [ 21.623772] nix-grpc-daemon[783]: ts=2026-09-20T10:51:43Z level=info event=scheduler_connected addr=lb:500514179worker2 # [ 21.627730] nix-grpc-daemon[783]: ts=2026-09-20T10:51:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4180worker1 # [ 21.794934] nix-grpc-daemon[782]: ts=2026-09-20T10:51:44Z level=info event=scheduler_connected addr=lb:500514181worker1 # [ 21.798213] nix-grpc-daemon[782]: ts=2026-09-20T10:51:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4182worker2 # [ 21.828593] nix-grpc-daemon[783]: ts=2026-09-20T10:51:44Z level=info event=scheduler_connected addr=lb:500514183worker2 # [ 21.832491] nix-grpc-daemon[783]: ts=2026-09-20T10:51:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4184worker1 # [ 21.996074] nix-grpc-daemon[782]: ts=2026-09-20T10:51:45Z level=info event=scheduler_connected addr=lb:500514185worker1 # [ 22.000111] nix-grpc-daemon[782]: ts=2026-09-20T10:51:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4186worker2 # [ 22.033165] nix-grpc-daemon[783]: ts=2026-09-20T10:51:44Z level=info event=scheduler_connected addr=lb:500514187worker2 # [ 22.036407] nix-grpc-daemon[783]: ts=2026-09-20T10:51:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4188worker1 # [ 22.200969] nix-grpc-daemon[782]: ts=2026-09-20T10:51:45Z level=info event=scheduler_connected addr=lb:500514189worker1 # [ 22.204825] nix-grpc-daemon[782]: ts=2026-09-20T10:51:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4190worker2 # [ 22.236875] nix-grpc-daemon[783]: ts=2026-09-20T10:51:44Z level=info event=scheduler_connected addr=lb:500514191worker2 # [ 22.238970] nix-grpc-daemon[783]: ts=2026-09-20T10:51:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4192worker1 # [ 22.405595] nix-grpc-daemon[782]: ts=2026-09-20T10:51:45Z level=info event=scheduler_connected addr=lb:500514193worker1 # [ 22.409514] nix-grpc-daemon[782]: ts=2026-09-20T10:51:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4194worker2 # [ 22.440031] nix-grpc-daemon[783]: ts=2026-09-20T10:51:44Z level=info event=scheduler_connected addr=lb:500514195worker2 # [ 22.441687] nix-grpc-daemon[783]: ts=2026-09-20T10:51:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4196worker1 # [ 22.610564] nix-grpc-daemon[782]: ts=2026-09-20T10:51:45Z level=info event=scheduler_connected addr=lb:500514197worker1 # [ 22.614519] nix-grpc-daemon[782]: ts=2026-09-20T10:51:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4198worker2 # [ 22.643669] nix-grpc-daemon[783]: ts=2026-09-20T10:51:44Z level=info event=scheduler_connected addr=lb:500514199worker2 # [ 22.647536] nix-grpc-daemon[783]: ts=2026-09-20T10:51:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4200worker1 # [ 22.816105] nix-grpc-daemon[782]: ts=2026-09-20T10:51:45Z level=info event=scheduler_connected addr=lb:500514201worker1 # [ 22.819855] nix-grpc-daemon[782]: ts=2026-09-20T10:51:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4202worker2 # [ 22.847967] nix-grpc-daemon[783]: ts=2026-09-20T10:51:45Z level=info event=scheduler_connected addr=lb:500514203worker2 # [ 22.849701] nix-grpc-daemon[783]: ts=2026-09-20T10:51:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4204worker1 # [ 23.020651] nix-grpc-daemon[782]: ts=2026-09-20T10:51:46Z level=info event=scheduler_connected addr=lb:500514205worker1 # [ 23.024538] nix-grpc-daemon[782]: ts=2026-09-20T10:51:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4206worker2 # [ 23.051908] nix-grpc-daemon[783]: ts=2026-09-20T10:51:45Z level=info event=scheduler_connected addr=lb:500514207worker2 # [ 23.053706] nix-grpc-daemon[783]: ts=2026-09-20T10:51:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4208worker1 # [ 23.225394] nix-grpc-daemon[782]: ts=2026-09-20T10:51:46Z level=info event=scheduler_connected addr=lb:500514209worker1 # [ 23.229153] nix-grpc-daemon[782]: ts=2026-09-20T10:51:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4210worker2 # [ 23.254257] nix-grpc-daemon[783]: ts=2026-09-20T10:51:45Z level=info event=scheduler_connected addr=lb:500514211worker2 # [ 23.256209] nix-grpc-daemon[783]: ts=2026-09-20T10:51:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4212worker1 # [ 23.429940] nix-grpc-daemon[782]: ts=2026-09-20T10:51:46Z level=info event=scheduler_connected addr=lb:500514213worker1 # [ 23.433799] nix-grpc-daemon[782]: ts=2026-09-20T10:51:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4214worker2 # [ 23.457188] nix-grpc-daemon[783]: ts=2026-09-20T10:51:45Z level=info event=scheduler_connected addr=lb:500514215worker2 # [ 23.458861] nix-grpc-daemon[783]: ts=2026-09-20T10:51:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4216worker1 # [ 23.634646] nix-grpc-daemon[782]: ts=2026-09-20T10:51:46Z level=info event=scheduler_connected addr=lb:500514217worker1 # [ 23.638482] nix-grpc-daemon[782]: ts=2026-09-20T10:51:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4218worker2 # [ 23.659396] nix-grpc-daemon[783]: ts=2026-09-20T10:51:45Z level=info event=scheduler_connected addr=lb:500514219worker2 # [ 23.661176] nix-grpc-daemon[783]: ts=2026-09-20T10:51:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4220worker1 # [ 23.841152] nix-grpc-daemon[782]: ts=2026-09-20T10:51:46Z level=info event=scheduler_connected addr=lb:500514221worker1 # [ 23.843933] nix-grpc-daemon[782]: ts=2026-09-20T10:51:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4222worker2 # [ 23.861697] nix-grpc-daemon[783]: ts=2026-09-20T10:51:46Z level=info event=scheduler_connected addr=lb:500514223worker2 # [ 23.863446] nix-grpc-daemon[783]: ts=2026-09-20T10:51:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4224worker1 # [ 24.041569] nix-grpc-daemon[782]: ts=2026-09-20T10:51:47Z level=info event=scheduler_connected addr=lb:500514225worker1 # [ 24.043305] nix-grpc-daemon[782]: ts=2026-09-20T10:51:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4226worker2 # [ 24.064394] nix-grpc-daemon[783]: ts=2026-09-20T10:51:46Z level=info event=scheduler_connected addr=lb:500514227worker2 # [ 24.068164] nix-grpc-daemon[783]: ts=2026-09-20T10:51:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4228worker1 # [ 24.243756] nix-grpc-daemon[782]: ts=2026-09-20T10:51:47Z level=info event=scheduler_connected addr=lb:500514229worker1 # [ 24.245427] nix-grpc-daemon[782]: ts=2026-09-20T10:51:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4230worker2 # [ 24.269083] nix-grpc-daemon[783]: ts=2026-09-20T10:51:46Z level=info event=scheduler_connected addr=lb:500514231worker2 # [ 24.273300] nix-grpc-daemon[783]: ts=2026-09-20T10:51:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4232worker1 # [ 24.445920] nix-grpc-daemon[782]: ts=2026-09-20T10:51:47Z level=info event=scheduler_connected addr=lb:500514233worker1 # [ 24.447584] nix-grpc-daemon[782]: ts=2026-09-20T10:51:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4234worker2 # [ 24.474119] nix-grpc-daemon[783]: ts=2026-09-20T10:51:46Z level=info event=scheduler_connected addr=lb:500514235worker2 # [ 24.475976] nix-grpc-daemon[783]: ts=2026-09-20T10:51:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4236worker1 # [ 24.648201] nix-grpc-daemon[782]: ts=2026-09-20T10:51:47Z level=info event=scheduler_connected addr=lb:500514237worker1 # [ 24.650158] nix-grpc-daemon[782]: ts=2026-09-20T10:51:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4238worker2 # [ 24.676998] nix-grpc-daemon[783]: ts=2026-09-20T10:51:46Z level=info event=scheduler_connected addr=lb:500514239worker2 # [ 24.680937] nix-grpc-daemon[783]: ts=2026-09-20T10:51:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4240worker1 # [ 24.851230] nix-grpc-daemon[782]: ts=2026-09-20T10:51:47Z level=info event=scheduler_connected addr=lb:500514241worker1 # [ 24.852865] nix-grpc-daemon[782]: ts=2026-09-20T10:51:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4242worker2 # [ 24.881390] nix-grpc-daemon[783]: ts=2026-09-20T10:51:47Z level=info event=scheduler_connected addr=lb:500514243worker2 # [ 24.884125] nix-grpc-daemon[783]: ts=2026-09-20T10:51:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4244worker1 # [ 25.054095] nix-grpc-daemon[782]: ts=2026-09-20T10:51:48Z level=info event=scheduler_connected addr=lb:500514245worker1 # [ 25.057981] nix-grpc-daemon[782]: ts=2026-09-20T10:51:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4246worker2 # [ 25.084520] nix-grpc-daemon[783]: ts=2026-09-20T10:51:47Z level=info event=scheduler_connected addr=lb:500514247worker2 # [ 25.086294] nix-grpc-daemon[783]: ts=2026-09-20T10:51:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4248worker1 # [ 25.258845] nix-grpc-daemon[782]: ts=2026-09-20T10:51:48Z level=info event=scheduler_connected addr=lb:500514249worker1 # [ 25.263110] nix-grpc-daemon[782]: ts=2026-09-20T10:51:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4250worker2 # [ 25.287224] nix-grpc-daemon[783]: ts=2026-09-20T10:51:47Z level=info event=scheduler_connected addr=lb:500514251worker2 # [ 25.290984] nix-grpc-daemon[783]: ts=2026-09-20T10:51:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4252worker1 # [ 25.463338] nix-grpc-daemon[782]: ts=2026-09-20T10:51:48Z level=info event=scheduler_connected addr=lb:500514253worker1 # [ 25.465031] nix-grpc-daemon[782]: ts=2026-09-20T10:51:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4254worker2 # [ 25.491780] nix-grpc-daemon[783]: ts=2026-09-20T10:51:47Z level=info event=scheduler_connected addr=lb:500514255worker2 # [ 25.493505] nix-grpc-daemon[783]: ts=2026-09-20T10:51:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4256worker1 # [ 25.665585] nix-grpc-daemon[782]: ts=2026-09-20T10:51:48Z level=info event=scheduler_connected addr=lb:500514257worker1 # [ 25.667393] nix-grpc-daemon[782]: ts=2026-09-20T10:51:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4258worker2 # [ 25.694411] nix-grpc-daemon[783]: ts=2026-09-20T10:51:47Z level=info event=scheduler_connected addr=lb:500514259worker2 # [ 25.698189] nix-grpc-daemon[783]: ts=2026-09-20T10:51:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4260worker1 # [ 25.869105] nix-grpc-daemon[782]: ts=2026-09-20T10:51:49Z level=info event=scheduler_connected addr=lb:500514261worker1 # [ 25.871992] nix-grpc-daemon[782]: ts=2026-09-20T10:51:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4262worker2 # [ 25.898826] nix-grpc-daemon[783]: ts=2026-09-20T10:51:48Z level=info event=scheduler_connected addr=lb:500514263worker2 # [ 25.902355] nix-grpc-daemon[783]: ts=2026-09-20T10:51:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4264worker1 # [ 26.072443] nix-grpc-daemon[782]: ts=2026-09-20T10:51:49Z level=info event=scheduler_connected addr=lb:500514265worker1 # [ 26.074199] nix-grpc-daemon[782]: ts=2026-09-20T10:51:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4266worker2 # [ 26.102829] nix-grpc-daemon[783]: ts=2026-09-20T10:51:48Z level=info event=scheduler_connected addr=lb:500514267worker2 # [ 26.106790] nix-grpc-daemon[783]: ts=2026-09-20T10:51:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4268worker1 # [ 26.275161] nix-grpc-daemon[782]: ts=2026-09-20T10:51:49Z level=info event=scheduler_connected addr=lb:500514269worker1 # [ 26.279649] nix-grpc-daemon[782]: ts=2026-09-20T10:51:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4270worker2 # [ 26.307714] nix-grpc-daemon[783]: ts=2026-09-20T10:51:48Z level=info event=scheduler_connected addr=lb:500514271worker2 # [ 26.311851] nix-grpc-daemon[783]: ts=2026-09-20T10:51:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4272worker1 # [ 26.480474] nix-grpc-daemon[782]: ts=2026-09-20T10:51:49Z level=info event=scheduler_connected addr=lb:500514273worker1 # [ 26.484447] nix-grpc-daemon[782]: ts=2026-09-20T10:51:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4274worker2 # [ 26.513396] nix-grpc-daemon[783]: ts=2026-09-20T10:51:48Z level=info event=scheduler_connected addr=lb:500514275worker2 # [ 26.516974] nix-grpc-daemon[783]: ts=2026-09-20T10:51:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4276worker1 # [ 26.685603] nix-grpc-daemon[782]: ts=2026-09-20T10:51:49Z level=info event=scheduler_connected addr=lb:500514277worker1 # [ 26.689566] nix-grpc-daemon[782]: ts=2026-09-20T10:51:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4278worker2 # [ 26.717824] nix-grpc-daemon[783]: ts=2026-09-20T10:51:49Z level=info event=scheduler_connected addr=lb:500514279worker2 # [ 26.721598] nix-grpc-daemon[783]: ts=2026-09-20T10:51:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4280worker1 # [ 26.891324] nix-grpc-daemon[782]: ts=2026-09-20T10:51:50Z level=info event=scheduler_connected addr=lb:500514281worker1 # [ 26.895144] nix-grpc-daemon[782]: ts=2026-09-20T10:51:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4282worker2 # [ 26.922406] nix-grpc-daemon[783]: ts=2026-09-20T10:51:49Z level=info event=scheduler_connected addr=lb:500514283worker2 # [ 26.926186] nix-grpc-daemon[783]: ts=2026-09-20T10:51:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4284worker1 # [ 27.095719] nix-grpc-daemon[782]: ts=2026-09-20T10:51:50Z level=info event=scheduler_connected addr=lb:500514285worker1 # [ 27.099503] nix-grpc-daemon[782]: ts=2026-09-20T10:51:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4286worker2 # [ 27.126950] nix-grpc-daemon[783]: ts=2026-09-20T10:51:49Z level=info event=scheduler_connected addr=lb:500514287worker2 # [ 27.130710] nix-grpc-daemon[783]: ts=2026-09-20T10:51:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4288worker1 # [ 27.299966] nix-grpc-daemon[782]: ts=2026-09-20T10:51:50Z level=info event=scheduler_connected addr=lb:500514289worker1 # [ 27.303832] nix-grpc-daemon[782]: ts=2026-09-20T10:51:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4290worker2 # [ 27.331717] nix-grpc-daemon[783]: ts=2026-09-20T10:51:49Z level=info event=scheduler_connected addr=lb:500514291worker2 # [ 27.335583] nix-grpc-daemon[783]: ts=2026-09-20T10:51:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4292worker1 # [ 27.504703] nix-grpc-daemon[782]: ts=2026-09-20T10:51:50Z level=info event=scheduler_connected addr=lb:500514293worker1 # [ 27.508717] nix-grpc-daemon[782]: ts=2026-09-20T10:51:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4294worker2 # [ 27.537511] nix-grpc-daemon[783]: ts=2026-09-20T10:51:49Z level=info event=scheduler_connected addr=lb:500514295worker2 # [ 27.540982] nix-grpc-daemon[783]: ts=2026-09-20T10:51:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4296worker1 # [ 27.709476] nix-grpc-daemon[782]: ts=2026-09-20T10:51:50Z level=info event=scheduler_connected addr=lb:500514297worker1 # [ 27.713300] nix-grpc-daemon[782]: ts=2026-09-20T10:51:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4298worker2 # [ 27.741883] nix-grpc-daemon[783]: ts=2026-09-20T10:51:50Z level=info event=scheduler_connected addr=lb:500514299worker2 # [ 27.745785] nix-grpc-daemon[783]: ts=2026-09-20T10:51:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4300worker1 # [ 27.914956] nix-grpc-daemon[782]: ts=2026-09-20T10:51:51Z level=info event=scheduler_connected addr=lb:500514301worker1 # [ 27.918776] nix-grpc-daemon[782]: ts=2026-09-20T10:51:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4302worker2 # [ 27.946754] nix-grpc-daemon[783]: ts=2026-09-20T10:51:50Z level=info event=scheduler_connected addr=lb:500514303worker2 # [ 27.950641] nix-grpc-daemon[783]: ts=2026-09-20T10:51:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4304worker1 # [ 28.119468] nix-grpc-daemon[782]: ts=2026-09-20T10:51:51Z level=info event=scheduler_connected addr=lb:500514305worker1 # [ 28.123366] nix-grpc-daemon[782]: ts=2026-09-20T10:51:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4306worker2 # [ 28.151588] nix-grpc-daemon[783]: ts=2026-09-20T10:51:50Z level=info event=scheduler_connected addr=lb:500514307worker2 # [ 28.155380] nix-grpc-daemon[783]: ts=2026-09-20T10:51:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4308worker1 # [ 28.323907] nix-grpc-daemon[782]: ts=2026-09-20T10:51:51Z level=info event=scheduler_connected addr=lb:500514309worker1 # [ 28.327875] nix-grpc-daemon[782]: ts=2026-09-20T10:51:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4310worker2 # [ 28.356115] nix-grpc-daemon[783]: ts=2026-09-20T10:51:50Z level=info event=scheduler_connected addr=lb:500514311worker2 # [ 28.360298] nix-grpc-daemon[783]: ts=2026-09-20T10:51:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4312worker1 # [ 28.528632] nix-grpc-daemon[782]: ts=2026-09-20T10:51:51Z level=info event=scheduler_connected addr=lb:500514313worker1 # [ 28.532497] nix-grpc-daemon[782]: ts=2026-09-20T10:51:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4314worker2 # [ 28.561788] nix-grpc-daemon[783]: ts=2026-09-20T10:51:50Z level=info event=scheduler_connected addr=lb:500514315worker2 # [ 28.565939] nix-grpc-daemon[783]: ts=2026-09-20T10:51:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4316worker1 # [ 28.733282] nix-grpc-daemon[782]: ts=2026-09-20T10:51:51Z level=info event=scheduler_connected addr=lb:500514317worker1 # [ 28.737247] nix-grpc-daemon[782]: ts=2026-09-20T10:51:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4318lb # [ 28.757061] rustfs-bucket-start[1030]: mb s3://farm4319lb # [ 28.763158] systemd[1]: Finished rustfs-bucket.service.4320lb # [ 28.770315] systemd[1]: Starting niks3 server...4321worker2 # [ 28.766451] nix-grpc-daemon[783]: ts=2026-09-20T10:51:51Z level=info event=scheduler_connected addr=lb:500514322worker2 # [ 28.768163] nix-grpc-daemon[783]: ts=2026-09-20T10:51:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4323worker1 # [ 28.937990] nix-grpc-daemon[782]: ts=2026-09-20T10:51:52Z level=info event=scheduler_connected addr=lb:500514324worker1 # [ 28.939924] nix-grpc-daemon[782]: ts=2026-09-20T10:51:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4325worker2 # [ 28.969111] nix-grpc-daemon[783]: ts=2026-09-20T10:51:51Z level=info event=scheduler_connected addr=lb:500514326worker2 # [ 28.973530] nix-grpc-daemon[783]: ts=2026-09-20T10:51:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4327lb # [ 29.009334] postgres[1050]: [1050] ERROR: relation "goose_db_version" does not exist at character 364328lb # [ 29.010560] postgres[1050]: [1050] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4329lb # [ 29.049767] niks3-server[1039]: 2026/09/20 10:51:51 OK 20241026095416_initial_model.sql (21.35ms)4330lb # [ 29.053626] niks3-server[1039]: 2026/09/20 10:51:51 OK 20251210153512_drop_unused_gin_index.sql (3.93ms)4331lb # [ 29.058898] niks3-server[1039]: 2026/09/20 10:51:51 OK 20251218171726_add_pins.sql (3.76ms)4332lb # [ 29.064265] niks3-server[1039]: 2026/09/20 10:51:51 OK 20260628120000_add_object_size_and_stats.sql (5.29ms)4333lb # [ 29.068588] niks3-server[1039]: 2026/09/20 10:51:51 OK 20260905000000_add_claims.sql (4.35ms)4334lb # [ 29.072734] niks3-server[1039]: 2026/09/20 10:51:51 OK 20260920000000_drop_claims.sql (4.18ms)4335lb # [ 29.074240] niks3-server[1039]: 2026/09/20 10:51:51 goose: successfully migrated database to version: 202609200000004336lb # [ 29.080165] niks3-server[1039]: 2026/09/20 10:51:52 OK 1_commit_pending_closure.sql (6.03ms)4337lb # [ 29.082927] niks3-server[1039]: 2026/09/20 10:51:52 OK 2_object_stats_trigger.sql (1.51ms)4338lb # [ 29.084182] niks3-server[1039]: 2026/09/20 10:51:52 goose: up to current file version: 24339lb # [ 29.091157] niks3-server[1039]: 2026/09/20 10:51:52 INFO Loaded signing key name=farm-test-1 path=/nix/store/c745bsy529madd2s6hz4z5is2436ni04-key4340worker1 # [ 29.140888] nix-grpc-daemon[782]: ts=2026-09-20T10:51:52Z level=info event=scheduler_connected addr=lb:500514341worker1 # [ 29.144724] nix-grpc-daemon[782]: ts=2026-09-20T10:51:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4342lb # [ 29.173090] niks3-server[1039]: 2026/09/20 10:51:52 INFO Read proxy enabled — serving cache objects from S34343lb # [ 29.174952] niks3-server[1039]: 2026/09/20 10:51:52 INFO Using socket-activated listener address=0.0.0.0:57514344lb # [ 29.181989] systemd[1]: Started niks3 server.4345worker2 # [ 29.174613] nix-grpc-daemon[783]: ts=2026-09-20T10:51:51Z level=info event=scheduler_connected addr=lb:500514346lb # [ 29.182647] niks3-server[1039]: 2026/09/20 10:51:52 INFO systemd watchdog enabled interval=15s4347lb # [ 29.183855] systemd[1]: Reached target Multi-User System.4348lb # [ 29.185806] systemd[1]: Startup finished in 844ms (kernel) + 5.136s (initrd) + 23.203s (userspace) = 29.185s.4349worker2 # [ 29.179108] nix-grpc-daemon[783]: ts=2026-09-20T10:51:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4350lb # [ 29.187669] niks3-server[1039]: 2026/09/20 10:51:52 INFO Starting HTTP server address=0.0.0.0:57514351lb # [ 29.193290] niks3-server[1039]: 2026/09/20 10:51:52 INFO lead: acquired remote=192.168.1.4:331204352worker2 # [ 29.194641] nix-grpc-daemon[783]: ts=2026-09-20T10:51:51Z level=info event=scheduler_take_over4353worker1 # [ 29.344864] nix-grpc-daemon[782]: ts=2026-09-20T10:51:52Z level=info event=scheduler_connected addr=lb:500514354worker1 # [ 29.346541] nix-grpc-daemon[782]: ts=2026-09-20T10:51:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4355worker2 # [ 29.379277] nix-grpc-daemon[783]: ts=2026-09-20T10:51:51Z level=info event=scheduler_connected addr=lb:500514356worker2 # [ 29.391769] nix-grpc-daemon[783]: ts=2026-09-20T10:51:51Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04357worker1 # [ 29.547031] nix-grpc-daemon[782]: ts=2026-09-20T10:51:52Z level=info event=scheduler_connected addr=lb:500514358worker2 # [ 29.524292] nix-grpc-daemon[783]: ts=2026-09-20T10:51:51Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04359lb: (finished: waiting for unit niks3.service, in 18.48 seconds)4360lb: waiting for TCP port 5751 on localhost4361lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!4362lb: (finished: waiting for TCP port 5751 on localhost, in 0.06 seconds)4363lb: waiting for unit envoy.service4364lb: (finished: waiting for unit envoy.service, in 0.06 seconds)4365lb: waiting for TCP port 50051 on localhost4366lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!4367lb # [ 30.214562] envoy[910]: conn peer=[::1]:46286 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=24368lb: (finished: waiting for TCP port 50051 on localhost, in 0.03 seconds)4369lb: waiting until exactly no endpoints of x86_64-linux are unhealthy4370??? Warning (UserWarning): retry(): The 'timeout_seconds' argument is deprecated. Use 'timeout' instead.4371 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 1284372lb: must succeed: curl -sf localhost:9901/clusters4373lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4374lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.07 seconds)4375subtest: one node schedules and both builders hold a WorkerSession on it4376lb: must succeed: curl -sf localhost:9901/clusters4377lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4378worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4379worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4380(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.12 seconds)4381subtest: balancer auth: client cert, bearer token, nothing, unknown CN4382worker1 # [ 30.859496] nix-grpc-daemon[782]: ts=2026-09-20T10:51:54Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:487824383worker1 # [ 30.867293] systemd[1]: Started Nix Daemon.4384worker1 # [ 31.067265] nix-daemon[840]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4385worker1 # [ 31.122227] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 63 ms (attempt 1/5)4386worker1 # [ 31.187761] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 112 ms (attempt 2/5)4387worker1 # [ 31.302608] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 340 ms (attempt 3/5)4388worker1 # [ 31.648588] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 296 ms (attempt 4/5)4389worker1 # [ 31.954524] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4390worker1 # [ 31.978462] nix-grpc-daemon[782]: ts=2026-09-20T10:51:55Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:487824391worker1 # [ 31.996627] nix-grpc-daemon[782]: ts=2026-09-20T10:51:55Z level=debug event=path_info_miss path=bgb9qwzhklrxfjy4awx25fzniv721g1y-nixos-system-client-test error="error: path '/nix/store/bgb9qwzhklrxfjy4awx25fzniv721g1y-nixos-system-client-test' is required, but there is no substituter that can build it"4392worker1 # [ 32.005707] nix-grpc-daemon[782]: ts=2026-09-20T10:51:55Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:487824393worker1 # [ 32.014778] nix-grpc-daemon[782]: ts=2026-09-20T10:51:55Z level=debug event=path_info_miss path=bgb9qwzhklrxfjy4awx25fzniv721g1y-nixos-system-client-test error="error: path '/nix/store/bgb9qwzhklrxfjy4awx25fzniv721g1y-nixos-system-client-test' is required, but there is no substituter that can build it"4394client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4395client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.09 seconds)4396client: must succeed: install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4397client: (finished: must succeed: install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds)4398worker1 # [ 32.370284] nix-grpc-daemon[782]: ts=2026-09-20T10:51:55Z level=info event=oidc_jwks provider=mock keys=14399worker1 # [ 32.372557] nix-grpc-daemon[782]: ts=2026-09-20T10:51:55Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:487824400worker1 # [ 32.381197] nix-grpc-daemon[782]: ts=2026-09-20T10:51:55Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:487824401worker1 # [ 32.386776] nix-grpc-daemon[782]: ts=2026-09-20T10:51:55Z level=debug event=path_info_miss path=bgb9qwzhklrxfjy4awx25fzniv721g1y-nixos-system-client-test error="error: path '/nix/store/bgb9qwzhklrxfjy4awx25fzniv721g1y-nixos-system-client-test' is required, but there is no substituter that can build it"4402worker2 # [ 32.385291] nix-grpc-daemon[783]: ts=2026-09-20T10:51:54Z level=info event=oidc_jwks provider=mock keys=14403worker2 # [ 32.388797] nix-grpc-daemon[783]: ts=2026-09-20T10:51:54Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:581504404worker2 # [ 32.398280] systemd[1]: Started Nix Daemon.4405worker2 # [ 32.586325] nix-daemon[845]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4406worker2 # [ 32.614372] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 35 ms (attempt 1/5)4407worker2 # [ 32.651751] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 157 ms (attempt 2/5)4408worker2 # [ 32.811377] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 197 ms (attempt 3/5)4409worker2 # [ 33.009122] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 372 ms (attempt 4/5)4410worker2 # [ 33.393383] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4411worker2 # [ 33.413607] nix-grpc-daemon[783]: ts=2026-09-20T10:51:55Z level=debug event=path_info_miss path=bgb9qwzhklrxfjy4awx25fzniv721g1y-nixos-system-client-test error="error: path '/nix/store/bgb9qwzhklrxfjy4awx25fzniv721g1y-nixos-system-client-test' is required, but there is no substituter that can build it"4412client: must succeed: rm -r /var/lib/nix-grpc-store4413client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.04 seconds)4414worker1 # [ 33.695835] nix-grpc-daemon[782]: ts=2026-09-20T10:51:56Z level=info event=denied method=QueryMissing cn=- role=none4415worker1 # [ 33.892896] nix-grpc-daemon[782]: ts=2026-09-20T10:51:57Z level=info event=denied method=QueryMissing cn=stranger role=none4416(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.38 seconds)4417subtest: DAG build is scheduled across workers and lands in the cache4418client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24419worker1 # [ 34.105624] nix-grpc-daemon[782]: ts=2026-09-20T10:51:57Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:487824420client # don't know how to build these paths:4421client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4422worker1 # [ 34.121266] nix-grpc-daemon[782]: ts=2026-09-20T10:51:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:487824423worker2 # [ 34.131321] nix-grpc-daemon[783]: ts=2026-09-20T10:51:56Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04424worker2 # [ 34.133490] nix-grpc-daemon[783]: ts=2026-09-20T10:51:56Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.4:50051 clients=14425worker2 # [ 34.184759] nix-grpc-daemon[783]: ts=2026-09-20T10:51:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581504426worker2 # [ 34.203770] nix-grpc-daemon[783]: ts=2026-09-20T10:51:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581504427client # copying 1 paths to 'lb:50051'...4428worker2 # [ 34.212976] nix-grpc-daemon[783]: ts=2026-09-20T10:51:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581504429client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4430worker2 # [ 34.239722] nix-daemon[845]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4431worker2 # [ 34.267445] nix-grpc-daemon[783]: ts=2026-09-20T10:51:56Z level=info event=niks3_push_started pid=8564432worker2 # [ 34.687715] nix-daemon[845]: accepted connection from pid 867, user nix-grpc-daemon (trusted)4433worker2 # [ 34.706113] nix-daemon[845]: reaped child process 871, status = succeeded4434worker2 # [ 34.860964] nix-daemon[845]: accepted connection from pid 873, user nix-grpc-daemon (trusted)4435worker2 # [ 34.874098] nix-daemon[845]: reaped child process 876, status = succeeded4436lb # [ 34.893324] niks3-server[1039]: 2026/09/20 10:51:57 INFO Received uploads request method=POST path=/api/pending_closures4437worker2 # [ 34.907976] nix-grpc-daemon[856]: time=2026-09-20T10:51:57.197Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4438worker2 # [ 34.910409] nix-grpc-daemon[856]: time=2026-09-20T10:51:57.200Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4439lb # [ 34.955726] niks3-server[1039]: 2026/09/20 10:51:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4440lb # [ 34.959341] niks3-server[1039]: 2026/09/20 10:51:57 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4441lb # [ 34.974809] niks3-server[1039]: 2026/09/20 10:51:57 INFO Signed narinfos id=1 count=14442worker2 # [ 34.971543] nix-grpc-daemon[856]: time=2026-09-20T10:51:57.262Z level=INFO msg="Uploading 1 narinfos"4443lb # [ 34.985536] niks3-server[1039]: 2026/09/20 10:51:57 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4444lb # [ 34.987728] niks3-server[1039]: 2026/09/20 10:51:57 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4445lb # [ 34.994298] niks3-server[1039]: 2026/09/20 10:51:57 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo4446lb # [ 35.002865] niks3-server[1039]: 2026/09/20 10:51:57 INFO Completed upload id=14447worker2 # [ 34.998314] nix-grpc-daemon[856]: time=2026-09-20T10:51:57.289Z level=INFO msg="Upload complete. (481ms)"4448worker2 # [ 35.003658] nix-grpc-daemon[783]: ts=2026-09-20T10:51:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=0 paths=1 nar_bytes_in=5044449worker2 # [ 35.011663] nix-grpc-daemon[783]: ts=2026-09-20T10:51:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581504450worker2 # [ 35.022261] nix-daemon[845]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4451worker2 # [ 35.032221] nix-daemon[845]: reaped child process 854, status = succeeded4452worker2 # [ 35.036557] nix-daemon[845]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4453client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4454client # farm-a> worker2: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4455client # farm-a> LOG-farm-a-t14456worker2 # [ 35.647850] nix-daemon[845]: reaped child process 880, status = succeeded4457worker2 # [ 35.791540] nix-daemon[845]: accepted connection from pid 896, user nix-grpc-daemon (trusted)4458worker2 # [ 35.802979] nix-daemon[845]: reaped child process 899, status = succeeded4459lb # [ 35.822941] niks3-server[1039]: 2026/09/20 10:51:58 INFO Received uploads request method=POST path=/api/pending_closures4460worker2 # [ 35.832873] nix-grpc-daemon[856]: time=2026-09-20T10:51:58.124Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4461worker2 # [ 35.834672] nix-grpc-daemon[856]: time=2026-09-20T10:51:58.125Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4462lb # [ 35.879103] niks3-server[1039]: 2026/09/20 10:51:58 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4463lb # [ 35.883776] niks3-server[1039]: 2026/09/20 10:51:58 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4464lb # [ 35.895335] niks3-server[1039]: 2026/09/20 10:51:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4465worker2 # [ 35.895046] nix-grpc-daemon[856]: time=2026-09-20T10:51:58.186Z level=INFO msg="Uploading 1 narinfos"4466lb # [ 35.900321] niks3-server[1039]: 2026/09/20 10:51:58 INFO Signed narinfos id=2 count=14467lb # [ 35.905650] niks3-server[1039]: 2026/09/20 10:51:58 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4468lb # [ 35.914977] niks3-server[1039]: 2026/09/20 10:51:58 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4469lb # [ 35.919651] niks3-server[1039]: 2026/09/20 10:51:58 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4470lb # [ 35.923182] niks3-server[1039]: 2026/09/20 10:51:58 INFO Completed upload id=24471worker2 # [ 35.919434] nix-grpc-daemon[856]: time=2026-09-20T10:51:58.209Z level=INFO msg="Upload complete. (275ms)"4472worker2 # [ 35.931420] nix-grpc-daemon[783]: ts=2026-09-20T10:51:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=0 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14473worker2 # [ 35.946248] nix-grpc-daemon[783]: ts=2026-09-20T10:51:58Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04474worker2 # [ 35.951430] nix-grpc-daemon[783]: ts=2026-09-20T10:51:58Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.4:50051 clients=14475worker2 # [ 35.956261] nix-daemon[845]: reaped child process 878, status = succeeded4476worker2 # [ 35.991199] nix-grpc-daemon[783]: ts=2026-09-20T10:51:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581504477worker2 # [ 36.004529] nix-grpc-daemon[783]: ts=2026-09-20T10:51:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581504478worker2 # [ 36.017659] nix-grpc-daemon[856]: time=2026-09-20T10:51:58.308Z level=INFO msg="All 1 paths already cached"4479client # copying 1 paths to 'lb:50051'...4480worker2 # [ 36.025812] nix-grpc-daemon[783]: ts=2026-09-20T10:51:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581504481client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4482worker2 # [ 36.046103] nix-daemon[845]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4483worker2 # [ 36.221619] nix-daemon[845]: accepted connection from pid 905, user nix-grpc-daemon (trusted)4484worker2 # [ 36.235400] nix-daemon[845]: reaped child process 908, status = succeeded4485worker2 # [ 36.389167] nix-daemon[845]: accepted connection from pid 911, user nix-grpc-daemon (trusted)4486worker2 # [ 36.401602] nix-daemon[845]: reaped child process 914, status = succeeded4487lb # [ 36.420087] niks3-server[1039]: 2026/09/20 10:51:59 INFO Received uploads request method=POST path=/api/pending_closures4488worker2 # [ 36.426886] nix-grpc-daemon[856]: time=2026-09-20T10:51:58.718Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4489worker2 # [ 36.428484] nix-grpc-daemon[856]: time=2026-09-20T10:51:58.719Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4490lb # [ 36.456648] niks3-server[1039]: 2026/09/20 10:51:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4491lb # [ 36.461627] niks3-server[1039]: 2026/09/20 10:51:59 INFO Signed narinfos id=3 count=14492worker2 # [ 36.460026] nix-grpc-daemon[856]: time=2026-09-20T10:51:58.751Z level=INFO msg="Uploading 1 narinfos"4493lb # [ 36.469581] niks3-server[1039]: 2026/09/20 10:51:59 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4494lb # [ 36.478375] niks3-server[1039]: 2026/09/20 10:51:59 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4495lb # [ 36.486231] niks3-server[1039]: 2026/09/20 10:51:59 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4496worker2 # [ 36.489934] nix-grpc-daemon[856]: time=2026-09-20T10:51:58.781Z level=INFO msg="Upload complete. (430ms)"4497lb # [ 36.494703] niks3-server[1039]: 2026/09/20 10:51:59 INFO Completed upload id=34498worker2 # [ 36.492938] nix-grpc-daemon[783]: ts=2026-09-20T10:51:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=0 paths=1 nar_bytes_in=6244499lb # [ 36.497814] niks3-server[1039]: 2026/09/20 10:51:59 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4500worker2 # [ 36.496027] nix-grpc-daemon[783]: ts=2026-09-20T10:51:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581504501worker2 # [ 36.499761] nix-daemon[845]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4502worker2 # [ 36.504855] nix-daemon[845]: reaped child process 903, status = succeeded4503client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4504worker2 # [ 36.509898] nix-daemon[845]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4505client # farm-b> worker2: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4506worker2 # [ 37.721863] nix-daemon[845]: reaped child process 918, status = succeeded4507client # farm-b> LOG-farm-b-t14508worker2 # [ 37.858094] nix-daemon[845]: accepted connection from pid 933, user nix-grpc-daemon (trusted)4509worker2 # [ 37.869548] nix-daemon[845]: reaped child process 936, status = succeeded4510lb # [ 37.889603] niks3-server[1039]: 2026/09/20 10:52:00 INFO Received uploads request method=POST path=/api/pending_closures4511worker2 # [ 37.894062] nix-grpc-daemon[856]: time=2026-09-20T10:52:00.184Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4512worker2 # [ 37.895557] nix-grpc-daemon[856]: time=2026-09-20T10:52:00.184Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4513lb # [ 37.910874] niks3-server[1039]: 2026/09/20 10:52:00 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4514lb # [ 37.916722] niks3-server[1039]: 2026/09/20 10:52:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4515lb # [ 37.920131] niks3-server[1039]: 2026/09/20 10:52:00 INFO Signed narinfos id=4 count=14516worker2 # [ 37.915581] nix-grpc-daemon[856]: time=2026-09-20T10:52:00.206Z level=INFO msg="Uploading 1 narinfos"4517lb # [ 37.922991] niks3-server[1039]: 2026/09/20 10:52:00 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4518lb # [ 37.925915] niks3-server[1039]: 2026/09/20 10:52:00 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4519lb # [ 37.931063] niks3-server[1039]: 2026/09/20 10:52:00 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4520lb # [ 37.935515] niks3-server[1039]: 2026/09/20 10:52:00 INFO Completed upload id=44521worker2 # [ 37.930783] nix-grpc-daemon[856]: time=2026-09-20T10:52:00.221Z level=INFO msg="Upload complete. (212ms)"4522lb # [ 37.937331] niks3-server[1039]: 2026/09/20 10:52:00 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4523worker2 # [ 37.933068] nix-grpc-daemon[783]: ts=2026-09-20T10:52:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14524worker2 # [ 37.940583] nix-grpc-daemon[783]: ts=2026-09-20T10:52:00Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04525worker2 # [ 37.942552] nix-grpc-daemon[783]: ts=2026-09-20T10:52:00Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.4:50051 clients=14526worker2 # [ 37.946097] nix-daemon[845]: reaped child process 916, status = succeeded4527worker2 # [ 37.984369] nix-grpc-daemon[783]: ts=2026-09-20T10:52:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581504528lb # [ 37.998865] envoy[910]: conn peer=[2001:db8:1::1]:60940 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=715 tx=365 ms=12054529worker2 # [ 37.994154] nix-grpc-daemon[783]: ts=2026-09-20T10:52:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581504530lb # [ 38.000561] envoy[910]: conn peer=[2001:db8:1::1]:60944 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2630 tx=421 ms=10964531lb # [ 38.002158] envoy[910]: conn peer=[2001:db8:1::1]:37414 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=104532lb # [ 38.004770] envoy[910]: conn peer=[2001:db8:1::1]:37422 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=84533worker2 # [ 38.005987] nix-grpc-daemon[856]: time=2026-09-20T10:52:00.297Z level=INFO msg="All 2 paths already cached"4534client # copying 1 paths to 'lb:50051'...4535worker2 # [ 38.013417] nix-grpc-daemon[783]: ts=2026-09-20T10:52:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581504536client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4537worker2 # [ 38.032114] nix-daemon[845]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4538worker2 # [ 38.189914] nix-daemon[845]: accepted connection from pid 942, user nix-grpc-daemon (trusted)4539worker2 # [ 38.201799] nix-daemon[845]: reaped child process 945, status = succeeded4540worker2 # [ 38.333955] nix-daemon[845]: accepted connection from pid 947, user nix-grpc-daemon (trusted)4541worker2 # [ 38.345461] nix-daemon[845]: reaped child process 950, status = succeeded4542lb # [ 38.363551] niks3-server[1039]: 2026/09/20 10:52:01 INFO Received uploads request method=POST path=/api/pending_closures4543worker2 # [ 38.362935] nix-grpc-daemon[856]: time=2026-09-20T10:52:00.654Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4544worker2 # [ 38.364450] nix-grpc-daemon[856]: time=2026-09-20T10:52:00.655Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4545lb # [ 38.379536] niks3-server[1039]: 2026/09/20 10:52:01 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4546lb # [ 38.382804] niks3-server[1039]: 2026/09/20 10:52:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4547lb # [ 38.384809] niks3-server[1039]: 2026/09/20 10:52:01 INFO Signed narinfos id=5 count=14548worker2 # [ 38.380178] nix-grpc-daemon[856]: time=2026-09-20T10:52:00.671Z level=INFO msg="Uploading 1 narinfos"4549lb # [ 38.388257] niks3-server[1039]: 2026/09/20 10:52:01 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4550lb # [ 38.393259] niks3-server[1039]: 2026/09/20 10:52:01 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4551lb # [ 38.396202] niks3-server[1039]: 2026/09/20 10:52:01 INFO Completed upload id=54552worker2 # [ 38.392660] nix-grpc-daemon[856]: time=2026-09-20T10:52:00.683Z level=INFO msg="Upload complete. (350ms)"4553lb # [ 38.398337] niks3-server[1039]: 2026/09/20 10:52:01 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4554worker2 # [ 38.394413] nix-grpc-daemon[783]: ts=2026-09-20T10:52:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=0 paths=1 nar_bytes_in=7524555worker2 # [ 38.398233] nix-grpc-daemon[783]: ts=2026-09-20T10:52:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581504556worker2 # [ 38.404813] nix-daemon[845]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4557worker2 # [ 38.406123] nix-daemon[845]: reaped child process 940, status = succeeded4558worker2 # [ 38.411152] nix-daemon[845]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4559client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4560client # farm-top> worker2: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4561client # farm-top> LOG-farm-top-t14562worker2 # [ 39.034126] nix-daemon[845]: reaped child process 954, status = succeeded4563worker2 # [ 39.190434] nix-daemon[845]: accepted connection from pid 970, user nix-grpc-daemon (trusted)4564worker2 # [ 39.200852] nix-daemon[845]: reaped child process 973, status = succeeded4565lb # [ 39.221788] niks3-server[1039]: 2026/09/20 10:52:02 INFO Received uploads request method=POST path=/api/pending_closures4566worker2 # [ 39.225672] nix-grpc-daemon[856]: time=2026-09-20T10:52:01.516Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4567worker2 # [ 39.227368] nix-grpc-daemon[856]: time=2026-09-20T10:52:01.518Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4568lb # [ 39.259776] niks3-server[1039]: 2026/09/20 10:52:02 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4569lb # [ 39.268722] niks3-server[1039]: 2026/09/20 10:52:02 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4570lb # [ 39.278082] niks3-server[1039]: 2026/09/20 10:52:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4571lb # [ 39.283721] niks3-server[1039]: 2026/09/20 10:52:02 INFO Signed narinfos id=6 count=14572worker2 # [ 39.278571] nix-grpc-daemon[856]: time=2026-09-20T10:52:01.569Z level=INFO msg="Uploading 1 narinfos"4573lb # [ 39.288634] niks3-server[1039]: 2026/09/20 10:52:02 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4574lb # [ 39.297762] niks3-server[1039]: 2026/09/20 10:52:02 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4575lb # [ 39.304546] niks3-server[1039]: 2026/09/20 10:52:02 INFO Completed upload id=64576lb # [ 39.305752] niks3-server[1039]: 2026/09/20 10:52:02 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4577worker2 # [ 39.300088] nix-grpc-daemon[856]: time=2026-09-20T10:52:01.591Z level=INFO msg="Upload complete. (282ms)"4578worker2 # [ 39.306814] nix-grpc-daemon[783]: ts=2026-09-20T10:52:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=0 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14579worker2 # [ 39.319889] nix-daemon[845]: reaped child process 952, status = succeeded4580client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 5.46 seconds)4581client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4582client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.18 seconds)4583client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4584client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.02 seconds)4585client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4586client # copying 3 paths...4587client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4588client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4589client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4590client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.30 seconds)4591(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 6.04 seconds)4592subtest: repeat is answered Cached by the scheduler without building4593worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4594worker1 # finding garbage collector roots...4595worker1 # deleting specified paths...4596worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4597worker1 # deleting unused links...4598worker1 # note: hard linking is currently saving 0.0 KiB4599worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.25 seconds)4600worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4601worker2 # finding garbage collector roots...4602worker2 # deleting specified paths...4603worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4604worker2 # deleting unused links...4605worker2 # note: hard linking is currently saving 0.0 KiB4606worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.24 seconds)4607client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24608worker2 # [ 40.638314] nix-grpc-daemon[783]: ts=2026-09-20T10:52:02Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:581504609client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4610client # /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4611worker1 # [ 40.682960] nix-grpc-daemon[782]: ts=2026-09-20T10:52:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:487824612worker2 # [ 40.684509] nix-grpc-daemon[783]: ts=2026-09-20T10:52:02Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14613client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 0.29 seconds)4614client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4615client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.19 seconds)4616lb: must succeed: curl -sf localhost:9901/clusters4617lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4618lb: must succeed: curl -sf localhost:9901/clusters4619lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4620worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4621worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4622(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.19 seconds)4623subtest: inputs already in the cache are not sent to the scheduler4624lb: must succeed: curl -sf localhost:9901/clusters4625lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4626lb: must succeed: curl -sf localhost:9901/clusters4627lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4628worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4629worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4630client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&24631client # don't know how to build these paths:4632worker1 # [ 41.486408] nix-grpc-daemon[782]: ts=2026-09-20T10:52:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:487824633client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4634worker2 # [ 41.464496] nix-grpc-daemon[783]: ts=2026-09-20T10:52:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581504635worker2 # [ 41.473093] nix-grpc-daemon[856]: time=2026-09-20T10:52:03.763Z level=INFO msg="All 2 paths already cached"4636worker2 # [ 41.484631] nix-grpc-daemon[783]: ts=2026-09-20T10:52:03Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04637worker2 # [ 41.489473] nix-grpc-daemon[783]: ts=2026-09-20T10:52:03Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.4:50051 clients=14638worker2 # [ 41.528404] nix-grpc-daemon[783]: ts=2026-09-20T10:52:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581504639worker2 # [ 41.538076] nix-grpc-daemon[783]: ts=2026-09-20T10:52:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581504640worker2 # [ 41.547266] nix-grpc-daemon[856]: time=2026-09-20T10:52:03.838Z level=INFO msg="All 2 paths already cached"4641client # copying 1 paths to 'lb:50051'...4642client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4643worker2 # [ 41.554494] nix-grpc-daemon[783]: ts=2026-09-20T10:52:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581504644worker2 # [ 41.565316] nix-daemon[845]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4645worker2 # [ 41.720592] nix-daemon[845]: accepted connection from pid 1002, user nix-grpc-daemon (trusted)4646worker2 # [ 41.732923] nix-daemon[845]: reaped child process 1005, status = succeeded4647worker2 # [ 41.893944] nix-daemon[845]: accepted connection from pid 1007, user nix-grpc-daemon (trusted)4648worker2 # [ 41.911751] nix-daemon[845]: reaped child process 1010, status = succeeded4649lb # [ 41.930902] niks3-server[1039]: 2026/09/20 10:52:04 INFO Received uploads request method=POST path=/api/pending_closures4650worker2 # [ 41.930195] nix-grpc-daemon[856]: time=2026-09-20T10:52:04.221Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4651worker2 # [ 41.931763] nix-grpc-daemon[856]: time=2026-09-20T10:52:04.222Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4652lb # [ 41.947033] niks3-server[1039]: 2026/09/20 10:52:04 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4653lb # [ 41.951034] niks3-server[1039]: 2026/09/20 10:52:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4654lb # [ 41.952828] niks3-server[1039]: 2026/09/20 10:52:04 INFO Signed narinfos id=7 count=14655worker2 # [ 41.949139] nix-grpc-daemon[856]: time=2026-09-20T10:52:04.240Z level=INFO msg="Uploading 1 narinfos"4656lb # [ 41.956487] niks3-server[1039]: 2026/09/20 10:52:04 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4657lb # [ 41.961256] niks3-server[1039]: 2026/09/20 10:52:04 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4658lb # [ 41.964645] niks3-server[1039]: 2026/09/20 10:52:04 INFO Completed upload id=74659worker2 # [ 41.961046] nix-grpc-daemon[856]: time=2026-09-20T10:52:04.252Z level=INFO msg="Upload complete. (384ms)"4660lb # [ 41.966302] niks3-server[1039]: 2026/09/20 10:52:04 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4661worker2 # [ 41.963254] nix-grpc-daemon[783]: ts=2026-09-20T10:52:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=0 paths=1 nar_bytes_in=7524662worker2 # [ 41.966143] nix-grpc-daemon[783]: ts=2026-09-20T10:52:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581504663worker2 # [ 41.970636] nix-daemon[845]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4664worker2 # [ 41.974963] nix-daemon[845]: reaped child process 1000, status = succeeded4665client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4666client # farm-top> worker2: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4667worker2 # [ 41.979781] nix-daemon[845]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4668client # farm-top> LOG-farm-top-t1b4669worker2 # [ 43.200977] nix-daemon[845]: reaped child process 1015, status = succeeded4670worker2 # [ 43.336598] nix-daemon[845]: accepted connection from pid 1030, user nix-grpc-daemon (trusted)4671worker2 # [ 43.348576] nix-daemon[845]: reaped child process 1033, status = succeeded4672lb # [ 43.369462] niks3-server[1039]: 2026/09/20 10:52:06 INFO Received uploads request method=POST path=/api/pending_closures4673worker2 # [ 43.373089] nix-grpc-daemon[856]: time=2026-09-20T10:52:05.664Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4674worker2 # [ 43.374699] nix-grpc-daemon[856]: time=2026-09-20T10:52:05.665Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4675lb # [ 43.411158] niks3-server[1039]: 2026/09/20 10:52:06 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4676lb # [ 43.418123] niks3-server[1039]: 2026/09/20 10:52:06 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4677lb # [ 43.425736] niks3-server[1039]: 2026/09/20 10:52:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4678lb # [ 43.429917] niks3-server[1039]: 2026/09/20 10:52:06 INFO Signed narinfos id=8 count=14679worker2 # [ 43.428148] nix-grpc-daemon[856]: time=2026-09-20T10:52:05.719Z level=INFO msg="Uploading 1 narinfos"4680lb # [ 43.435753] niks3-server[1039]: 2026/09/20 10:52:06 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4681lb # [ 43.444949] niks3-server[1039]: 2026/09/20 10:52:06 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4682lb # [ 43.449738] niks3-server[1039]: 2026/09/20 10:52:06 INFO Completed upload id=84683worker2 # [ 43.447261] nix-grpc-daemon[856]: time=2026-09-20T10:52:05.738Z level=INFO msg="Upload complete. (261ms)"4684lb # [ 43.454496] niks3-server[1039]: 2026/09/20 10:52:06 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4685worker2 # [ 43.451762] nix-grpc-daemon[783]: ts=2026-09-20T10:52:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14686worker2 # [ 43.463805] nix-daemon[845]: reaped child process 1013, status = succeeded4687client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.25 seconds)4688client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath4689client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.16 seconds)4690lb: must succeed: curl -sf localhost:9901/clusters4691lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4692lb: must succeed: curl -sf localhost:9901/clusters4693lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4694worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4695worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4696(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.70 seconds)4697subtest: two clients wanting the same drv build it once4698client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4699client # [ 43.888153] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4700client # Running as unit: dedup1.service; invocation ID: 14684b2f937247aa892f7f395ad1ad684701client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 0.07 seconds)4702client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4703worker2 # [ 44.059784] nix-grpc-daemon[783]: ts=2026-09-20T10:52:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:581504704client # [ 44.075130] nix[886]: don't know how to build these paths:4705client # [ 44.075986] nix[886]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4706worker2 # [ 44.068704] nix-grpc-daemon[783]: ts=2026-09-20T10:52:06Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04707worker2 # [ 44.071205] nix-grpc-daemon[783]: ts=2026-09-20T10:52:06Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.3:50051 clients=14708worker1 # [ 44.101130] nix-grpc-daemon[782]: ts=2026-09-20T10:52:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487824709worker1 # [ 44.116455] nix-grpc-daemon[782]: ts=2026-09-20T10:52:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:487824710client # [ 44.114480] nix[886]: copying 1 paths to 'lb:50051'...4711client # [ 44.115346] nix[886]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4712worker1 # [ 44.127533] nix-grpc-daemon[782]: ts=2026-09-20T10:52:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:487824713worker1 # [ 44.139146] nix-daemon[840]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4714worker1 # [ 44.160840] nix-grpc-daemon[782]: ts=2026-09-20T10:52:07Z level=info event=niks3_push_started pid=8664715worker1 # [ 44.591709] nix-daemon[840]: accepted connection from pid 877, user nix-grpc-daemon (trusted)4716worker1 # [ 44.606396] nix-daemon[840]: reaped child process 881, status = succeeded4717worker1 # [ 44.763041] nix-daemon[840]: accepted connection from pid 883, user nix-grpc-daemon (trusted)4718worker1 # [ 44.774755] nix-daemon[840]: reaped child process 886, status = succeeded4719lb # [ 44.770829] niks3-server[1039]: 2026/09/20 10:52:07 INFO Received uploads request method=POST path=/api/pending_closures4720worker1 # [ 44.799579] nix-grpc-daemon[866]: time=2026-09-20T10:52:07.943Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4721worker1 # [ 44.803063] nix-grpc-daemon[866]: time=2026-09-20T10:52:07.947Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4722lb # [ 44.799462] niks3-server[1039]: 2026/09/20 10:52:07 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4723lb # [ 44.802602] niks3-server[1039]: 2026/09/20 10:52:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4724lb # [ 44.805490] niks3-server[1039]: 2026/09/20 10:52:07 INFO Signed narinfos id=9 count=14725worker1 # [ 44.826656] nix-grpc-daemon[866]: time=2026-09-20T10:52:07.972Z level=INFO msg="Uploading 1 narinfos"4726lb # [ 44.808259] niks3-server[1039]: 2026/09/20 10:52:07 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4727lb # [ 44.814961] niks3-server[1039]: 2026/09/20 10:52:07 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4728lb # [ 44.817884] niks3-server[1039]: 2026/09/20 10:52:07 INFO Completed upload id=94729worker1 # [ 44.840368] nix-grpc-daemon[866]: time=2026-09-20T10:52:07.986Z level=INFO msg="Upload complete. (423ms)"4730lb # [ 44.819829] niks3-server[1039]: 2026/09/20 10:52:07 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4731worker1 # [ 44.843049] nix-grpc-daemon[782]: ts=2026-09-20T10:52:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48782 duration_s=0 paths=1 nar_bytes_in=4324732worker1 # [ 44.845714] nix-grpc-daemon[782]: ts=2026-09-20T10:52:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487824733worker1 # [ 44.851711] nix-daemon[840]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4734worker1 # [ 44.854537] nix-daemon[840]: reaped child process 864, status = succeeded4735worker1 # [ 44.859491] nix-daemon[840]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4736client # [ 44.846796] nix[886]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4737client # [ 44.967351] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4738client # Running as unit: dedup2.service; invocation ID: 6272af397c894e0098bbe2a82d17bb3f4739client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 1.09 seconds)4740worker2 # [ 45.206140] nix-grpc-daemon[783]: ts=2026-09-20T10:52:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:581504741client # [ 45.222706] nix[902]: don't know how to build these paths:4742client # [ 45.223837] nix[902]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4743worker2 # [ 45.216277] nix-grpc-daemon[783]: ts=2026-09-20T10:52:07Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04744worker1 # [ 45.247523] nix-grpc-daemon[782]: ts=2026-09-20T10:52:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487824745client # [ 45.240173] nix[902]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4746client: must succeed: sleep 24747lb # [ 47.999409] envoy[910]: conn peer=[2001:db8:1::1]:37424 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9477 tx=3785 ms=52774748lb # [ 48.001194] envoy[910]: conn peer=[2001:db8:1::1]:37430 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=614749lb # [ 48.002668] envoy[910]: conn peer=[2001:db8:1::1]:37436 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4139 tx=1708 ms=20074750client: (finished: must succeed: sleep 2, in 2.05 seconds)4751??? Warning (UserWarning): wait_until_succeeds(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4752 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394753client: waiting for success: ! systemctl is-active dedup1 dedup24754??? Warning (UserWarning): execute(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4755 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394756worker1 # [ 49.689940] nix-daemon[840]: reaped child process 890, status = succeeded4757worker1 # [ 49.868366] nix-daemon[840]: accepted connection from pid 924, user nix-grpc-daemon (trusted)4758worker1 # [ 49.880267] nix-daemon[840]: reaped child process 927, status = succeeded4759lb # [ 49.878187] niks3-server[1039]: 2026/09/20 10:52:12 INFO Received uploads request method=POST path=/api/pending_closures4760worker1 # [ 49.907974] nix-grpc-daemon[866]: time=2026-09-20T10:52:13.053Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4761worker1 # [ 49.909650] nix-grpc-daemon[866]: time=2026-09-20T10:52:13.055Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4762lb # [ 49.929532] niks3-server[1039]: 2026/09/20 10:52:12 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4763lb # [ 49.936397] niks3-server[1039]: 2026/09/20 10:52:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4764lb # [ 49.941465] niks3-server[1039]: 2026/09/20 10:52:12 INFO Signed narinfos id=10 count=14765worker1 # [ 49.966092] nix-grpc-daemon[866]: time=2026-09-20T10:52:13.111Z level=INFO msg="Uploading 1 narinfos"4766lb # [ 49.950097] niks3-server[1039]: 2026/09/20 10:52:12 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4767lb # [ 49.955402] niks3-server[1039]: 2026/09/20 10:52:12 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4768lb # [ 49.960070] niks3-server[1039]: 2026/09/20 10:52:12 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4769lb # [ 49.965288] niks3-server[1039]: 2026/09/20 10:52:12 INFO Completed upload id=104770worker1 # [ 49.986475] nix-grpc-daemon[866]: time=2026-09-20T10:52:13.132Z level=INFO msg="Upload complete. (309ms)"4771lb # [ 49.967131] niks3-server[1039]: 2026/09/20 10:52:12 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4772worker1 # [ 49.991692] nix-grpc-daemon[782]: ts=2026-09-20T10:52:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48782 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14773worker1 # [ 49.998273] nix-grpc-daemon[782]: ts=2026-09-20T10:52:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48782 duration_s=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14774worker1 # [ 50.007610] nix-daemon[840]: reaped child process 888, status = succeeded4775client # [ 50.022808] systemd[1]: dedup2.service: Deactivated successfully.4776client # [ 50.027149] systemd[1]: dedup2.service: Consumed 121ms CPU time over 5.058s wall clock time, 31.2M memory peak, 7.5K incoming IP traffic, 6.5K outgoing IP traffic.4777client # [ 50.063888] systemd[1]: dedup1.service: Deactivated successfully.4778client # [ 50.065631] systemd[1]: dedup1.service: Consumed 108ms CPU time over 6.178s wall clock time, 39.3M memory peak, 8.6K incoming IP traffic, 9.2K outgoing IP traffic.4779client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.10 seconds)4780client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4781client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)4782worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4783worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.06 seconds)4784worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4785worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.06 seconds)4786worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4787worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4788worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4789worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4790(finished: subtest: two clients wanting the same drv build it once, in 7.14 seconds)4791subtest: build hook: nix-daemon with builders = grpc://lb4792client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log4793client # [ 51.176911] systemd[1]: Started Nix Daemon.4794client # [ 51.350331] nix-daemon[944]: accepted connection from pid 941, user root (trusted)4795worker1 # [ 51.541799] nix-grpc-daemon[782]: ts=2026-09-20T10:52:14Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:487824796worker1 # [ 51.550836] nix-grpc-daemon[782]: ts=2026-09-20T10:52:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:487824797worker2 # [ 51.539909] nix-grpc-daemon[783]: ts=2026-09-20T10:52:13Z level=debug event=want client=6 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=04798worker2 # [ 51.545595] nix-grpc-daemon[783]: ts=2026-09-20T10:52:13Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.4:50051 clients=14799worker2 # [ 51.587141] nix-grpc-daemon[783]: ts=2026-09-20T10:52:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581504800worker2 # [ 51.598390] nix-grpc-daemon[783]: ts=2026-09-20T10:52:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581504801worker2 # [ 51.608944] nix-grpc-daemon[783]: ts=2026-09-20T10:52:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581504802worker2 # [ 51.621512] nix-daemon[845]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4803worker2 # [ 51.804643] nix-daemon[845]: accepted connection from pid 1073, user nix-grpc-daemon (trusted)4804worker2 # [ 51.817695] nix-daemon[845]: reaped child process 1076, status = succeeded4805worker2 # [ 51.984981] nix-daemon[845]: accepted connection from pid 1078, user nix-grpc-daemon (trusted)4806worker2 # [ 51.996402] nix-daemon[845]: reaped child process 1081, status = succeeded4807lb # [ 52.019930] niks3-server[1039]: 2026/09/20 10:52:14 INFO Received uploads request method=POST path=/api/pending_closures4808worker2 # [ 52.021757] nix-grpc-daemon[856]: time=2026-09-20T10:52:14.312Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4809worker2 # [ 52.023401] nix-grpc-daemon[856]: time=2026-09-20T10:52:14.314Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"4810lb # [ 52.046621] niks3-server[1039]: 2026/09/20 10:52:14 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst4811lb # [ 52.059459] niks3-server[1039]: 2026/09/20 10:52:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4812lb # [ 52.065107] niks3-server[1039]: 2026/09/20 10:52:14 INFO Signed narinfos id=11 count=14813worker2 # [ 52.063407] nix-grpc-daemon[856]: time=2026-09-20T10:52:14.354Z level=INFO msg="Uploading 1 narinfos"4814lb # [ 52.072297] niks3-server[1039]: 2026/09/20 10:52:14 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls4815lb # [ 52.082090] niks3-server[1039]: 2026/09/20 10:52:15 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4816worker2 # [ 52.083987] nix-grpc-daemon[856]: time=2026-09-20T10:52:14.375Z level=INFO msg="Upload complete. (449ms)"4817lb # [ 52.089105] niks3-server[1039]: 2026/09/20 10:52:15 INFO Completed upload id=114818worker2 # [ 52.085916] nix-grpc-daemon[783]: ts=2026-09-20T10:52:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=0 paths=1 nar_bytes_in=5124819lb # [ 52.092716] niks3-server[1039]: 2026/09/20 10:52:15 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo4820worker2 # [ 52.090293] nix-grpc-daemon[783]: ts=2026-09-20T10:52:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581504821worker2 # [ 52.095910] nix-daemon[845]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4822worker2 # [ 52.098056] nix-daemon[845]: reaped child process 1071, status = succeeded4823worker2 # [ 52.103405] nix-daemon[845]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4824worker2 # [ 53.114836] nix-daemon[845]: reaped child process 1085, status = succeeded4825worker2 # [ 53.253652] nix-daemon[845]: accepted connection from pid 1100, user nix-grpc-daemon (trusted)4826worker2 # [ 53.265313] nix-daemon[845]: reaped child process 1103, status = succeeded4827lb # [ 53.285902] niks3-server[1039]: 2026/09/20 10:52:16 INFO Received uploads request method=POST path=/api/pending_closures4828worker2 # [ 53.289755] nix-grpc-daemon[856]: time=2026-09-20T10:52:15.580Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4829worker2 # [ 53.295177] nix-grpc-daemon[856]: time=2026-09-20T10:52:15.582Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"4830lb # [ 53.322521] niks3-server[1039]: 2026/09/20 10:52:16 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv4831lb # [ 53.337494] niks3-server[1039]: 2026/09/20 10:52:16 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4832lb # [ 53.344305] niks3-server[1039]: 2026/09/20 10:52:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4833lb # [ 53.350213] niks3-server[1039]: 2026/09/20 10:52:16 INFO Signed narinfos id=12 count=14834worker2 # [ 53.348090] nix-grpc-daemon[856]: time=2026-09-20T10:52:15.639Z level=INFO msg="Uploading 1 narinfos"4835lb # [ 53.353076] niks3-server[1039]: 2026/09/20 10:52:16 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls4836lb # [ 53.363269] niks3-server[1039]: 2026/09/20 10:52:16 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4837lb # [ 53.366182] niks3-server[1039]: 2026/09/20 10:52:16 INFO Completed upload id=124838worker2 # [ 53.361617] nix-grpc-daemon[856]: time=2026-09-20T10:52:15.652Z level=INFO msg="Upload complete. (252ms)"4839lb # [ 53.368138] niks3-server[1039]: 2026/09/20 10:52:16 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo4840worker2 # [ 53.368721] nix-grpc-daemon[783]: ts=2026-09-20T10:52:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=6 outputs=14841worker2 # [ 53.375297] nix-grpc-daemon[783]: ts=2026-09-20T10:52:15Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:581504842worker2 # [ 53.383962] nix-daemon[845]: reaped child process 1083, status = succeeded4843worker1 # [ 53.415977] nix-grpc-daemon[782]: ts=2026-09-20T10:52:16Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:487824844worker1 # [ 53.425329] nix-grpc-daemon[782]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...4845worker1 # [ 53.443548] nix-grpc-daemon[782]: ts=2026-09-20T10:52:16Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48782 duration_s=0 paths=1 nar_bytes_out=1284846worker2 # [ 53.667855] nix-grpc-daemon[783]: ts=2026-09-20T10:52:15Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:581504847worker1 # [ 53.701167] nix-grpc-daemon[782]: ts=2026-09-20T10:52:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:487824848worker1 # [ 53.711312] nix-grpc-daemon[866]: time=2026-09-20T10:52:16.856Z level=INFO msg="All 1 paths already cached"4849worker2 # [ 53.699392] nix-grpc-daemon[783]: ts=2026-09-20T10:52:15Z level=debug event=want client=7 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=04850worker2 # [ 53.704761] nix-grpc-daemon[783]: ts=2026-09-20T10:52:15Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.4:50051 clients=14851worker2 # [ 53.709997] nix-grpc-daemon[783]: ts=2026-09-20T10:52:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581504852worker2 # [ 53.721866] nix-grpc-daemon[783]: ts=2026-09-20T10:52:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581504853worker2 # [ 53.733838] nix-grpc-daemon[856]: time=2026-09-20T10:52:16.024Z level=INFO msg="All 1 paths already cached"4854worker2 # [ 53.741165] nix-grpc-daemon[783]: ts=2026-09-20T10:52:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581504855worker2 # [ 53.749864] nix-daemon[845]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4856worker2 # [ 53.917062] nix-daemon[845]: accepted connection from pid 1109, user nix-grpc-daemon (trusted)4857worker2 # [ 53.929502] nix-daemon[845]: reaped child process 1112, status = succeeded4858worker2 # [ 54.079602] nix-daemon[845]: accepted connection from pid 1114, user nix-grpc-daemon (trusted)4859worker2 # [ 54.091241] nix-daemon[845]: reaped child process 1117, status = succeeded4860lb # [ 54.109743] niks3-server[1039]: 2026/09/20 10:52:17 INFO Received uploads request method=POST path=/api/pending_closures4861worker2 # [ 54.113158] nix-grpc-daemon[856]: time=2026-09-20T10:52:16.404Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4862worker2 # [ 54.114681] nix-grpc-daemon[856]: time=2026-09-20T10:52:16.405Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"4863lb # [ 54.139485] niks3-server[1039]: 2026/09/20 10:52:17 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst4864lb # [ 54.151856] niks3-server[1039]: 2026/09/20 10:52:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4865worker2 # [ 54.152380] nix-grpc-daemon[856]: time=2026-09-20T10:52:16.443Z level=INFO msg="Uploading 1 narinfos"4866lb # [ 54.157583] niks3-server[1039]: 2026/09/20 10:52:17 INFO Signed narinfos id=13 count=14867lb # [ 54.163943] niks3-server[1039]: 2026/09/20 10:52:17 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls4868lb # [ 54.174532] niks3-server[1039]: 2026/09/20 10:52:17 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4869lb # [ 54.182537] niks3-server[1039]: 2026/09/20 10:52:17 INFO Completed upload id=134870worker2 # [ 54.177479] nix-grpc-daemon[856]: time=2026-09-20T10:52:16.468Z level=INFO msg="Upload complete. (413ms)"4871lb # [ 54.185535] niks3-server[1039]: 2026/09/20 10:52:17 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo4872worker2 # [ 54.181180] nix-grpc-daemon[783]: ts=2026-09-20T10:52:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=0 paths=1 nar_bytes_in=6244873worker2 # [ 54.186947] nix-grpc-daemon[783]: ts=2026-09-20T10:52:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581504874worker2 # [ 54.195897] nix-daemon[845]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4875worker2 # [ 54.201234] nix-daemon[845]: reaped child process 1107, status = succeeded4876worker2 # [ 54.205486] nix-daemon[845]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4877worker2 # [ 55.629562] nix-daemon[845]: reaped child process 1121, status = succeeded4878worker2 # [ 55.791498] nix-daemon[845]: accepted connection from pid 1136, user nix-grpc-daemon (trusted)4879worker2 # [ 55.803710] nix-daemon[845]: reaped child process 1139, status = succeeded4880lb # [ 55.824293] niks3-server[1039]: 2026/09/20 10:52:18 INFO Received uploads request method=POST path=/api/pending_closures4881worker2 # [ 55.828382] nix-grpc-daemon[856]: time=2026-09-20T10:52:18.119Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4882worker2 # [ 55.829850] nix-grpc-daemon[856]: time=2026-09-20T10:52:18.121Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"4883lb # [ 55.864990] niks3-server[1039]: 2026/09/20 10:52:18 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv4884lb # [ 55.873595] niks3-server[1039]: 2026/09/20 10:52:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4885worker2 # [ 55.876329] nix-grpc-daemon[856]: time=2026-09-20T10:52:18.167Z level=INFO msg="Uploading 1 narinfos"4886lb # [ 55.881053] niks3-server[1039]: 2026/09/20 10:52:18 INFO Signed narinfos id=14 count=14887lb # [ 55.888770] niks3-server[1039]: 2026/09/20 10:52:18 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst4888lb # [ 55.893159] niks3-server[1039]: 2026/09/20 10:52:18 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls4889lb # [ 55.903834] niks3-server[1039]: 2026/09/20 10:52:18 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4890lb # [ 55.906602] niks3-server[1039]: 2026/09/20 10:52:18 INFO Completed upload id=144891worker2 # [ 55.901933] nix-grpc-daemon[856]: time=2026-09-20T10:52:18.193Z level=INFO msg="Upload complete. (287ms)"4892lb # [ 55.908170] niks3-server[1039]: 2026/09/20 10:52:18 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo4893worker2 # [ 55.905377] nix-grpc-daemon[783]: ts=2026-09-20T10:52:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=7 outputs=14894worker2 # [ 55.911093] nix-daemon[845]: reaped child process 1119, status = succeeded4895worker1 # [ 55.977511] nix-grpc-daemon[782]: ts=2026-09-20T10:52:19Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:487824896worker1 # [ 55.988093] nix-grpc-daemon[782]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...4897worker2 # [ 55.995057] nix-grpc-daemon[783]: ts=2026-09-20T10:52:18Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:581504898worker2 # [ 55.998035] nix-grpc-daemon[783]: ts=2026-09-20T10:52:18Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=0 paths=1 nar_bytes_out=1764899worker2 # [ 56.223440] nix-grpc-daemon[783]: ts=2026-09-20T10:52:18Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:581504900worker2 # [ 56.227612] nix-grpc-daemon[783]: ts=2026-09-20T10:52:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581504901worker2 # [ 56.234183] nix-grpc-daemon[856]: time=2026-09-20T10:52:18.525Z level=INFO msg="All 2 paths already cached"4902worker2 # [ 56.244985] nix-grpc-daemon[783]: ts=2026-09-20T10:52:18Z level=debug event=want client=8 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=04903worker2 # [ 56.247236] nix-grpc-daemon[783]: ts=2026-09-20T10:52:18Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.4:50051 clients=14904worker2 # [ 56.249795] nix-grpc-daemon[783]: ts=2026-09-20T10:52:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581504905worker2 # [ 56.256940] nix-grpc-daemon[783]: ts=2026-09-20T10:52:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581504906worker2 # [ 56.270025] nix-grpc-daemon[856]: time=2026-09-20T10:52:18.561Z level=INFO msg="All 2 paths already cached"4907worker2 # [ 56.274162] nix-grpc-daemon[783]: ts=2026-09-20T10:52:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581504908worker2 # [ 56.279071] nix-daemon[845]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4909worker2 # [ 56.417532] nix-daemon[845]: accepted connection from pid 1145, user nix-grpc-daemon (trusted)4910worker2 # [ 56.429485] nix-daemon[845]: reaped child process 1148, status = succeeded4911worker2 # [ 56.587100] nix-daemon[845]: accepted connection from pid 1150, user nix-grpc-daemon (trusted)4912worker2 # [ 56.598560] nix-daemon[845]: reaped child process 1153, status = succeeded4913lb # [ 56.618079] niks3-server[1039]: 2026/09/20 10:52:19 INFO Received uploads request method=POST path=/api/pending_closures4914worker2 # [ 56.621438] nix-grpc-daemon[856]: time=2026-09-20T10:52:18.912Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4915worker2 # [ 56.622816] nix-grpc-daemon[856]: time=2026-09-20T10:52:18.912Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"4916lb # [ 56.644904] niks3-server[1039]: 2026/09/20 10:52:19 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst4917lb # [ 56.651396] niks3-server[1039]: 2026/09/20 10:52:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4918lb # [ 56.655756] niks3-server[1039]: 2026/09/20 10:52:19 INFO Signed narinfos id=15 count=14919worker2 # [ 56.653909] nix-grpc-daemon[856]: time=2026-09-20T10:52:18.944Z level=INFO msg="Uploading 1 narinfos"4920lb # [ 56.666619] niks3-server[1039]: 2026/09/20 10:52:19 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls4921lb # [ 56.672614] niks3-server[1039]: 2026/09/20 10:52:19 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4922worker2 # [ 56.673748] nix-grpc-daemon[856]: time=2026-09-20T10:52:18.964Z level=INFO msg="Upload complete. (389ms)"4923lb # [ 56.678617] niks3-server[1039]: 2026/09/20 10:52:19 INFO Completed upload id=154924worker2 # [ 56.675673] nix-grpc-daemon[783]: ts=2026-09-20T10:52:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=0 paths=1 nar_bytes_in=7524925lb # [ 56.684234] niks3-server[1039]: 2026/09/20 10:52:19 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo4926worker2 # [ 56.679870] nix-grpc-daemon[783]: ts=2026-09-20T10:52:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581504927worker2 # [ 56.687797] nix-daemon[845]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4928worker2 # [ 56.689959] nix-daemon[845]: reaped child process 1143, status = succeeded4929worker2 # [ 56.695378] nix-daemon[845]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4930worker2 # [ 57.718508] nix-daemon[845]: reaped child process 1157, status = succeeded4931worker2 # [ 57.880214] nix-daemon[845]: accepted connection from pid 1172, user nix-grpc-daemon (trusted)4932worker2 # [ 57.890617] nix-daemon[845]: reaped child process 1175, status = succeeded4933lb # [ 57.912475] niks3-server[1039]: 2026/09/20 10:52:20 INFO Received uploads request method=POST path=/api/pending_closures4934worker2 # [ 57.915446] nix-grpc-daemon[856]: time=2026-09-20T10:52:20.206Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4935worker2 # [ 57.916999] nix-grpc-daemon[856]: time=2026-09-20T10:52:20.208Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"4936lb # [ 57.952618] niks3-server[1039]: 2026/09/20 10:52:20 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst4937lb # [ 57.961334] niks3-server[1039]: 2026/09/20 10:52:20 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv4938lb # [ 57.971162] niks3-server[1039]: 2026/09/20 10:52:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4939worker2 # [ 57.970627] nix-grpc-daemon[856]: time=2026-09-20T10:52:20.261Z level=INFO msg="Uploading 1 narinfos"4940lb # [ 57.975498] niks3-server[1039]: 2026/09/20 10:52:20 INFO Signed narinfos id=16 count=14941lb # [ 57.979765] niks3-server[1039]: 2026/09/20 10:52:20 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls4942lb # [ 57.988469] niks3-server[1039]: 2026/09/20 10:52:20 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4943lb # [ 57.994866] niks3-server[1039]: 2026/09/20 10:52:20 INFO Completed upload id=164944worker2 # [ 57.990237] nix-grpc-daemon[856]: time=2026-09-20T10:52:20.281Z level=INFO msg="Upload complete. (287ms)"4945lb # [ 57.997315] niks3-server[1039]: 2026/09/20 10:52:20 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo4946lb # [ 57.999847] envoy[910]: conn peer=[2001:db8:1::1]:34878 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1440 tx=1085 ms=47844947lb # [ 58.001428] envoy[910]: conn peer=[2001:db8:1::1]:34876 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2972 tx=1370 ms=59724948lb # [ 58.002903] envoy[910]: conn peer=[2001:db8:1::1]:34886 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3384 tx=1643 ms=19514949lb # [ 58.004297] envoy[910]: conn peer=[2001:db8:1::1]:33746 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=334 ms=814950worker2 # [ 57.995831] nix-grpc-daemon[783]: ts=2026-09-20T10:52:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=8 outputs=14951lb # [ 58.005642] envoy[910]: conn peer=[2001:db8:1::1]:33764 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=378 ms=174952lb # [ 58.007266] envoy[910]: conn peer=[2001:db8:1::1]:33750 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3880 tx=1969 ms=23444953worker2 # [ 58.008891] nix-daemon[845]: reaped child process 1155, status = succeeded4954worker1 # [ 58.033330] nix-grpc-daemon[782]: ts=2026-09-20T10:52:21Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:487824955worker1 # [ 58.040858] nix-grpc-daemon[782]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...4956worker2 # [ 58.034513] nix-grpc-daemon[783]: ts=2026-09-20T10:52:20Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:581504957worker2 # [ 58.039482] nix-grpc-daemon[783]: ts=2026-09-20T10:52:20Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=0 paths=1 nar_bytes_out=2324958client # [ 58.146434] nix-daemon[944]: reaped child process 946, status = succeeded4959client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.18 seconds)4960client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top4961client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.02 seconds)4962client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4963client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.04 seconds)4964client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4965client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4966(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.27 seconds)4967subtest: an input only one worker has still reaches the builder4968client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only4969client # this derivation will be built:4970client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv4971client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...4972client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only, in 0.40 seconds)4973client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure4974client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.16 seconds)4975worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4976worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.20 seconds)4977worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only4978worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)4979client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&24980client # [ 59.195720] nix-daemon[944]: accepted connection from pid 1027, user root (trusted)4981client # this derivation will be built:4982client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv4983worker2 # [ 59.343324] nix-grpc-daemon[783]: ts=2026-09-20T10:52:21Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:581504984client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux'...4985worker1 # [ 59.376269] nix-grpc-daemon[782]: ts=2026-09-20T10:52:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:487824986worker1 # [ 59.571714] nix-daemon[840]: accepted connection from pid 946, user nix-grpc-daemon (trusted)4987worker1 # [ 59.581982] nix-daemon[840]: reaped child process 949, status = succeeded4988lb # [ 59.578614] niks3-server[1039]: 2026/09/20 10:52:22 INFO Received uploads request method=POST path=/api/pending_closures4989worker1 # [ 59.608394] nix-grpc-daemon[866]: time=2026-09-20T10:52:22.753Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4990worker1 # [ 59.609874] nix-grpc-daemon[866]: time=2026-09-20T10:52:22.753Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"4991lb # [ 59.607168] niks3-server[1039]: 2026/09/20 10:52:22 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4992lb # [ 59.616546] niks3-server[1039]: 2026/09/20 10:52:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4993lb # [ 59.621551] niks3-server[1039]: 2026/09/20 10:52:22 INFO Signed narinfos id=17 count=14994worker1 # [ 59.645810] nix-grpc-daemon[866]: time=2026-09-20T10:52:22.791Z level=INFO msg="Uploading 1 narinfos"4995lb # [ 59.628290] niks3-server[1039]: 2026/09/20 10:52:22 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls4996lb # [ 59.638345] niks3-server[1039]: 2026/09/20 10:52:22 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4997lb # [ 59.644096] niks3-server[1039]: 2026/09/20 10:52:22 INFO Completed upload id=174998worker1 # [ 59.665331] nix-grpc-daemon[866]: time=2026-09-20T10:52:22.810Z level=INFO msg="Upload complete. (283ms)"4999lb # [ 59.649405] niks3-server[1039]: 2026/09/20 10:52:22 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5000worker2 # [ 59.656820] nix-grpc-daemon[783]: ts=2026-09-20T10:52:21Z level=debug event=want client=9 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05001worker2 # [ 59.661365] nix-grpc-daemon[783]: ts=2026-09-20T10:52:21Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=15002worker1 # [ 59.727600] nix-grpc-daemon[782]: ts=2026-09-20T10:52:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487825003worker1 # [ 59.741161] nix-grpc-daemon[782]: ts=2026-09-20T10:52:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:487825004client # copying 2 paths to 'lb:50051'...5005client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5006worker1 # [ 59.758114] nix-grpc-daemon[782]: ts=2026-09-20T10:52:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:487825007client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5008worker1 # [ 59.771225] nix-daemon[840]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5009worker1 # [ 59.951367] nix-daemon[840]: accepted connection from pid 953, user nix-grpc-daemon (trusted)5010worker1 # [ 59.966315] nix-daemon[840]: reaped child process 956, status = succeeded5011worker1 # [ 60.107837] nix-daemon[840]: accepted connection from pid 958, user nix-grpc-daemon (trusted)5012worker1 # [ 60.119369] nix-daemon[840]: reaped child process 961, status = succeeded5013lb # [ 60.113481] niks3-server[1039]: 2026/09/20 10:52:23 INFO Received uploads request method=POST path=/api/pending_closures5014lb # [ 60.118170] niks3-server[1039]: 2026/09/20 10:52:23 INFO Received uploads request method=POST path=/api/pending_closures5015worker1 # [ 60.142814] nix-grpc-daemon[866]: time=2026-09-20T10:52:23.288Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5016worker1 # [ 60.145161] nix-grpc-daemon[866]: time=2026-09-20T10:52:23.290Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5017worker1 # [ 60.146717] nix-grpc-daemon[866]: time=2026-09-20T10:52:23.290Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5018lb # [ 60.136116] niks3-server[1039]: 2026/09/20 10:52:23 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5019lb # [ 60.142560] niks3-server[1039]: 2026/09/20 10:52:23 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5020lb # [ 60.146700] niks3-server[1039]: 2026/09/20 10:52:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5021lb # [ 60.148883] niks3-server[1039]: 2026/09/20 10:52:23 INFO Signed narinfos id=18 count=15022lb # [ 60.151981] niks3-server[1039]: 2026/09/20 10:52:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5023lb # [ 60.154304] niks3-server[1039]: 2026/09/20 10:52:23 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5024lb # [ 60.156355] niks3-server[1039]: 2026/09/20 10:52:23 INFO Signed narinfos id=19 count=25025worker1 # [ 60.177544] nix-grpc-daemon[866]: time=2026-09-20T10:52:23.322Z level=INFO msg="Uploading 3 narinfos"5026lb # [ 60.158637] niks3-server[1039]: 2026/09/20 10:52:23 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5027lb # [ 60.169729] niks3-server[1039]: 2026/09/20 10:52:23 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5028lb # [ 60.171943] niks3-server[1039]: 2026/09/20 10:52:23 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5029lb # [ 60.177579] niks3-server[1039]: 2026/09/20 10:52:23 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5030lb # [ 60.180097] niks3-server[1039]: 2026/09/20 10:52:23 INFO Completed upload id=185031lb # [ 60.182184] niks3-server[1039]: 2026/09/20 10:52:23 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5032lb # [ 60.184503] niks3-server[1039]: 2026/09/20 10:52:23 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5033lb # [ 60.186612] niks3-server[1039]: 2026/09/20 10:52:23 INFO Completed upload id=195034worker1 # [ 60.209086] nix-grpc-daemon[866]: time=2026-09-20T10:52:23.354Z level=INFO msg="Upload complete. (422ms)"5035worker1 # [ 60.214126] nix-grpc-daemon[782]: ts=2026-09-20T10:52:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48782 duration_s=0 paths=2 nar_bytes_in=9765036worker1 # [ 60.218870] nix-grpc-daemon[782]: ts=2026-09-20T10:52:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487825037worker1 # [ 60.225693] nix-daemon[840]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5038worker1 # [ 60.230643] nix-daemon[840]: reaped child process 951, status = succeeded5039client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5040client # uses-input-w1onlya> node-a: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5041worker1 # [ 60.236725] nix-daemon[840]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5042worker1 # [ 61.448193] nix-daemon[840]: reaped child process 965, status = succeeded5043worker1 # [ 61.584207] nix-daemon[840]: accepted connection from pid 980, user nix-grpc-daemon (trusted)5044worker1 # [ 61.595761] nix-daemon[840]: reaped child process 983, status = succeeded5045lb # [ 61.592659] niks3-server[1039]: 2026/09/20 10:52:24 INFO Received uploads request method=POST path=/api/pending_closures5046worker1 # [ 61.623249] nix-grpc-daemon[866]: time=2026-09-20T10:52:24.768Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5047lb # [ 61.621690] niks3-server[1039]: 2026/09/20 10:52:24 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5048lb # [ 61.634355] niks3-server[1039]: 2026/09/20 10:52:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5049lb # [ 61.639555] niks3-server[1039]: 2026/09/20 10:52:24 INFO Signed narinfos id=20 count=15050worker1 # [ 61.663317] nix-grpc-daemon[866]: time=2026-09-20T10:52:24.808Z level=INFO msg="Uploading 1 narinfos"5051lb # [ 61.645574] niks3-server[1039]: 2026/09/20 10:52:24 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5052lb # [ 61.654151] niks3-server[1039]: 2026/09/20 10:52:24 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5053lb # [ 61.658776] niks3-server[1039]: 2026/09/20 10:52:24 INFO Completed upload id=205054worker1 # [ 61.679966] nix-grpc-daemon[866]: time=2026-09-20T10:52:24.825Z level=INFO msg="Upload complete. (237ms)"5055lb # [ 61.660499] niks3-server[1039]: 2026/09/20 10:52:24 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5056worker1 # [ 61.684440] nix-grpc-daemon[782]: ts=2026-09-20T10:52:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48782 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=9 outputs=15057worker1 # [ 61.695043] nix-daemon[840]: reaped child process 963, status = succeeded5058client # copying 1 paths...5059worker2 # [ 61.706400] nix-grpc-daemon[783]: ts=2026-09-20T10:52:23Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:581505060worker2 # [ 61.721587] nix-grpc-daemon[783]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5061client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5062worker2 # [ 61.754727] nix-grpc-daemon[783]: ts=2026-09-20T10:52:24Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:581505063worker2 # [ 61.760106] nix-grpc-daemon[783]: ts=2026-09-20T10:52:24Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=0 paths=1 nar_bytes_out=1205064client # [ 61.864223] nix-daemon[944]: reaped child process 1030, status = succeeded5065client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.85 seconds)5066client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&25067client # [ 62.031286] nix-daemon[944]: accepted connection from pid 1049, user root (trusted)5068client # this derivation will be built:5069client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5070worker1 # [ 62.217678] nix-grpc-daemon[782]: ts=2026-09-20T10:52:25Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:487825071client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux'...5072worker1 # [ 62.223493] nix-grpc-daemon[782]: ts=2026-09-20T10:52:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:487825073worker1 # [ 62.229233] nix-grpc-daemon[866]: time=2026-09-20T10:52:25.374Z level=INFO msg="All 1 paths already cached"5074worker2 # [ 62.217172] nix-grpc-daemon[783]: ts=2026-09-20T10:52:24Z level=debug event=want client=10 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05075worker2 # [ 62.221391] nix-grpc-daemon[783]: ts=2026-09-20T10:52:24Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=15076worker2 # [ 62.229188] nix-grpc-daemon[783]: ts=2026-09-20T10:52:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581505077worker2 # [ 62.241461] nix-grpc-daemon[783]: ts=2026-09-20T10:52:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581505078client # copying 1 paths to 'lb:50051'...5079client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5080worker2 # [ 62.259483] nix-grpc-daemon[783]: ts=2026-09-20T10:52:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581505081worker2 # [ 62.270154] nix-daemon[845]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5082worker2 # [ 62.285593] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 31 ms (attempt 1/5)5083worker2 # [ 62.321556] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 171 ms (attempt 2/5)5084worker2 # [ 62.498423] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 127 ms (attempt 3/5)5085worker2 # [ 62.631178] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 442 ms (attempt 4/5)5086worker2 # [ 63.079937] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5087worker2 # [ 63.086161] nix-grpc-daemon[783]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5088worker2 # [ 63.271443] nix-daemon[845]: accepted connection from pid 1191, user nix-grpc-daemon (trusted)5089worker2 # [ 63.282740] nix-daemon[845]: reaped child process 1194, status = succeeded5090worker2 # [ 63.419644] nix-daemon[845]: accepted connection from pid 1196, user nix-grpc-daemon (trusted)5091worker2 # [ 63.430973] nix-daemon[845]: reaped child process 1199, status = succeeded5092lb # [ 63.451506] niks3-server[1039]: 2026/09/20 10:52:26 INFO Received uploads request method=POST path=/api/pending_closures5093worker2 # [ 63.455965] nix-grpc-daemon[856]: time=2026-09-20T10:52:25.746Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5094worker2 # [ 63.457338] nix-grpc-daemon[856]: time=2026-09-20T10:52:25.746Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5095lb # [ 63.480199] niks3-server[1039]: 2026/09/20 10:52:26 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5096lb # [ 63.490996] niks3-server[1039]: 2026/09/20 10:52:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5097worker2 # [ 63.491971] nix-grpc-daemon[856]: time=2026-09-20T10:52:25.783Z level=INFO msg="Uploading 1 narinfos"5098lb # [ 63.496619] niks3-server[1039]: 2026/09/20 10:52:26 INFO Signed narinfos id=21 count=15099lb # [ 63.504263] niks3-server[1039]: 2026/09/20 10:52:26 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5100lb # [ 63.514992] niks3-server[1039]: 2026/09/20 10:52:26 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5101lb # [ 63.521404] niks3-server[1039]: 2026/09/20 10:52:26 INFO Completed upload id=215102worker2 # [ 63.516652] nix-grpc-daemon[856]: time=2026-09-20T10:52:25.807Z level=INFO msg="Upload complete. (411ms)"5103lb # [ 63.524464] niks3-server[1039]: 2026/09/20 10:52:26 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5104worker2 # [ 63.521125] nix-grpc-daemon[783]: ts=2026-09-20T10:52:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=1 paths=1 nar_bytes_in=5605105worker2 # [ 63.526424] nix-grpc-daemon[783]: ts=2026-09-20T10:52:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581505106worker2 # [ 63.535172] nix-daemon[845]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5107worker2 # [ 63.539850] nix-daemon[845]: reaped child process 1185, status = succeeded5108worker2 # [ 63.548207] nix-grpc-daemon[783]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...5109worker2 # [ 63.557158] nix-daemon[845]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5110client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5111client # uses-input-w1onlyb> worker2: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5112worker2 # [ 64.383864] nix-daemon[845]: reaped child process 1204, status = succeeded5113worker2 # [ 64.534373] nix-daemon[845]: accepted connection from pid 1219, user nix-grpc-daemon (trusted)5114worker2 # [ 64.544608] nix-daemon[845]: reaped child process 1222, status = succeeded5115lb # [ 64.564604] niks3-server[1039]: 2026/09/20 10:52:27 INFO Received uploads request method=POST path=/api/pending_closures5116worker2 # [ 64.567835] nix-grpc-daemon[856]: time=2026-09-20T10:52:26.858Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5117lb # [ 64.596836] niks3-server[1039]: 2026/09/20 10:52:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5118lb # [ 64.602625] niks3-server[1039]: 2026/09/20 10:52:27 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5119worker2 # [ 64.604289] nix-grpc-daemon[856]: time=2026-09-20T10:52:26.895Z level=INFO msg="Uploading 1 narinfos"5120lb # [ 64.609882] niks3-server[1039]: 2026/09/20 10:52:27 INFO Signed narinfos id=22 count=15121lb # [ 64.616963] niks3-server[1039]: 2026/09/20 10:52:27 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5122lb # [ 64.627140] niks3-server[1039]: 2026/09/20 10:52:27 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5123lb # [ 64.632652] niks3-server[1039]: 2026/09/20 10:52:27 INFO Completed upload id=225124worker2 # [ 64.627987] nix-grpc-daemon[856]: time=2026-09-20T10:52:26.919Z level=INFO msg="Upload complete. (263ms)"5125lb # [ 64.637465] niks3-server[1039]: 2026/09/20 10:52:27 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5126worker2 # [ 64.634124] nix-grpc-daemon[783]: ts=2026-09-20T10:52:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=10 outputs=15127client # copying 1 paths...5128worker2 # [ 64.643893] nix-grpc-daemon[783]: ts=2026-09-20T10:52:26Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:581505129client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5130worker2 # [ 64.648605] nix-daemon[845]: reaped child process 1201, status = succeeded5131worker2 # [ 64.658883] nix-grpc-daemon[783]: ts=2026-09-20T10:52:26Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:581505132worker2 # [ 64.662742] nix-grpc-daemon[783]: ts=2026-09-20T10:52:26Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=0 paths=1 nar_bytes_out=1205133client # [ 64.780443] nix-daemon[944]: reaped child process 1052, status = succeeded5134client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.92 seconds)5135client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&25136client # [ 64.943788] nix-daemon[944]: accepted connection from pid 1071, user root (trusted)5137client # this derivation will be built:5138client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5139worker2 # [ 65.118162] nix-grpc-daemon[783]: ts=2026-09-20T10:52:27Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:581505140client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux'...5141worker1 # [ 65.148095] nix-grpc-daemon[782]: ts=2026-09-20T10:52:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:487825142worker1 # [ 65.158472] nix-grpc-daemon[866]: time=2026-09-20T10:52:28.303Z level=INFO msg="All 1 paths already cached"5143worker2 # [ 65.146865] nix-grpc-daemon[783]: ts=2026-09-20T10:52:27Z level=debug event=want client=11 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05144worker2 # [ 65.151308] nix-grpc-daemon[783]: ts=2026-09-20T10:52:27Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=15145worker1 # [ 65.216717] nix-grpc-daemon[782]: ts=2026-09-20T10:52:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487825146worker1 # [ 65.230162] nix-grpc-daemon[782]: ts=2026-09-20T10:52:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:487825147worker1 # [ 65.243135] nix-grpc-daemon[866]: time=2026-09-20T10:52:28.388Z level=INFO msg="All 1 paths already cached"5148client # copying 1 paths to 'lb:50051'...5149client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5150worker1 # [ 65.251791] nix-grpc-daemon[782]: ts=2026-09-20T10:52:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:487825151worker1 # [ 65.263226] nix-daemon[840]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5152worker1 # [ 65.439369] nix-daemon[840]: accepted connection from pid 987, user nix-grpc-daemon (trusted)5153worker1 # [ 65.451197] nix-daemon[840]: reaped child process 990, status = succeeded5154worker1 # [ 65.593271] nix-daemon[840]: accepted connection from pid 992, user nix-grpc-daemon (trusted)5155worker1 # [ 65.605334] nix-daemon[840]: reaped child process 995, status = succeeded5156lb # [ 65.601363] niks3-server[1039]: 2026/09/20 10:52:28 INFO Received uploads request method=POST path=/api/pending_closures5157worker1 # [ 65.627593] nix-grpc-daemon[866]: time=2026-09-20T10:52:28.773Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5158worker1 # [ 65.628940] nix-grpc-daemon[866]: time=2026-09-20T10:52:28.773Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5159lb # [ 65.617651] niks3-server[1039]: 2026/09/20 10:52:28 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5160lb # [ 65.621132] niks3-server[1039]: 2026/09/20 10:52:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5161lb # [ 65.623527] niks3-server[1039]: 2026/09/20 10:52:28 INFO Signed narinfos id=23 count=15162worker1 # [ 65.644497] nix-grpc-daemon[866]: time=2026-09-20T10:52:28.790Z level=INFO msg="Uploading 1 narinfos"5163lb # [ 65.626466] niks3-server[1039]: 2026/09/20 10:52:28 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5164lb # [ 65.631932] niks3-server[1039]: 2026/09/20 10:52:28 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5165lb # [ 65.634912] niks3-server[1039]: 2026/09/20 10:52:28 INFO Completed upload id=235166worker1 # [ 65.655806] nix-grpc-daemon[866]: time=2026-09-20T10:52:28.801Z level=INFO msg="Upload complete. (380ms)"5167worker1 # [ 65.657925] nix-grpc-daemon[782]: ts=2026-09-20T10:52:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48782 duration_s=0 paths=1 nar_bytes_in=5605168worker1 # [ 65.659898] nix-grpc-daemon[866]: time=2026-09-20T10:52:28.803Z level=WARN msg="Failed to register uploaded object" key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo error="server returned 404: object is not pending upload\n"5169worker1 # [ 65.663202] nix-grpc-daemon[782]: ts=2026-09-20T10:52:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487825170worker1 # [ 65.666861] nix-daemon[840]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5171client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5172worker1 # [ 65.677062] nix-daemon[840]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5173worker1 # [ 65.678217] nix-daemon[840]: reaped child process 985, status = succeeded5174client # uses-input-w1onlyc> node-a: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5175worker1 # [ 66.503743] nix-daemon[840]: reaped child process 999, status = succeeded5176worker1 # [ 66.655412] nix-daemon[840]: accepted connection from pid 1014, user nix-grpc-daemon (trusted)5177worker1 # [ 66.665708] nix-daemon[840]: reaped child process 1017, status = succeeded5178lb # [ 66.657518] niks3-server[1039]: 2026/09/20 10:52:29 INFO Received uploads request method=POST path=/api/pending_closures5179worker1 # [ 66.691105] nix-grpc-daemon[866]: time=2026-09-20T10:52:29.836Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5180lb # [ 66.693669] niks3-server[1039]: 2026/09/20 10:52:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5181lb # [ 66.699924] niks3-server[1039]: 2026/09/20 10:52:29 INFO Signed narinfos id=24 count=15182worker1 # [ 66.724046] nix-grpc-daemon[866]: time=2026-09-20T10:52:29.869Z level=INFO msg="Uploading 1 narinfos"5183lb # [ 66.707572] niks3-server[1039]: 2026/09/20 10:52:29 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5184lb # [ 66.713375] niks3-server[1039]: 2026/09/20 10:52:29 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5185lb # [ 66.725270] niks3-server[1039]: 2026/09/20 10:52:29 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5186lb # [ 66.731540] niks3-server[1039]: 2026/09/20 10:52:29 INFO Completed upload id=245187worker1 # [ 66.752067] nix-grpc-daemon[866]: time=2026-09-20T10:52:29.897Z level=INFO msg="Upload complete. (267ms)"5188lb # [ 66.734597] niks3-server[1039]: 2026/09/20 10:52:29 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5189client # copying 1 paths...5190worker1 # [ 66.758109] nix-grpc-daemon[782]: ts=2026-09-20T10:52:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48782 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=11 outputs=15191worker2 # [ 66.737743] nix-grpc-daemon[783]: ts=2026-09-20T10:52:29Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:581505192worker1 # [ 66.770404] nix-daemon[840]: reaped child process 997, status = succeeded5193worker2 # [ 66.746956] nix-grpc-daemon[783]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5194client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5195worker2 # [ 66.771604] nix-grpc-daemon[783]: ts=2026-09-20T10:52:29Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:581505196worker2 # [ 66.776461] nix-grpc-daemon[783]: ts=2026-09-20T10:52:29Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:58150 duration_s=0 paths=1 nar_bytes_out=1205197client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.10 seconds)5198(finished: subtest: an input only one worker has still reaches the builder, in 8.65 seconds)5199subtest: upload whose reference the worker lacks is completed from the cache5200client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache5201client # [ 66.916230] nix-daemon[944]: reaped child process 1074, status = succeeded5202client # this derivation will be built:5203client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5204client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5205client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache, in 0.33 seconds)5206client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache5207client # this derivation will be built:5208client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5209client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5210client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache, in 0.33 seconds)5211client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5212client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.17 seconds)5213worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5214worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.18 seconds)5215worker2 # [ 67.946548] systemd[1]: Stopping Nix worker-protocol over gRPC...5216lb # [ 68.000412] envoy[910]: conn peer=[2001:db8:1::1]:33772 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4315 tx=2218 ms=18675217lb # [ 68.004434] envoy[910]: conn peer=[2001:db8:1::1]:33774 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=515218lb # [ 68.008380] envoy[910]: conn peer=[2001:db8:1::1]:33778 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3866 tx=1876 ms=24715219lb # [ 68.012521] envoy[910]: conn peer=[2001:db8:1::1]:33786 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=565220lb # [ 68.016523] envoy[910]: conn peer=[2001:db8:1::1]:45778 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=545221lb # [ 68.020715] envoy[910]: conn peer=[2001:db8:1::1]:33788 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3688 tx=1827 ms=25235222lb # [ 68.024874] envoy[910]: conn peer=[2001:db8:1::1]:45786 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3690 tx=1916 ms=17105223lb # [ 68.029879] envoy[910]: conn peer=[2001:db8:1::1]:45792 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=545224worker2 # [ 68.293646] nix-grpc-daemon[783]: ts=2026-09-20T10:52:30Z level=info event=signal_exit5225worker2 # [ 68.537771] nix-grpc-daemon[783]: ts=2026-09-20T10:52:30Z level=info event=scheduler_yield5226worker1 # [ 68.568385] nix-grpc-daemon[782]: ts=2026-09-20T10:52:31Z level=info event=scheduler_restarting addr=lb:500515227lb # [ 68.548347] niks3-server[1039]: 2026/09/20 10:52:31 INFO lead: released remote=192.168.1.4:331205228worker2 # [ 68.548641] nix-grpc-daemon[783]: ts=2026-09-20T10:52:30Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:581425229worker2 # [ 68.554448] nix-grpc-daemon[783]: ts=2026-09-20T10:52:30Z level=info event=worker_gone addr=192.168.1.3:500515230worker2 # [ 68.557453] nix-grpc-daemon[783]: ts=2026-09-20T10:52:30Z level=info event=scheduler_restarting addr=lb:500515231worker2 # [ 68.560373] nix-grpc-daemon[783]: ts=2026-09-20T10:52:30Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:581425232worker2 # [ 68.563595] nix-grpc-daemon[783]: ts=2026-09-20T10:52:30Z level=info event=worker_gone addr=192.168.1.4:500515233worker1 # [ 68.768522] nix-grpc-daemon[782]: ts=2026-09-20T10:52:31Z level=info event=scheduler_connected addr=lb:500515234worker1 # [ 68.772374] nix-grpc-daemon[782]: ts=2026-09-20T10:52:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5235worker2 # [ 68.754119] nix-grpc-daemon[783]: ts=2026-09-20T10:52:31Z level=info event=scheduler_connected addr=lb:500515236worker2 # [ 68.757955] nix-grpc-daemon[783]: ts=2026-09-20T10:52:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5237worker1 # [ 68.972926] nix-grpc-daemon[782]: ts=2026-09-20T10:52:32Z level=info event=scheduler_connected addr=lb:500515238worker1 # [ 68.976806] nix-grpc-daemon[782]: ts=2026-09-20T10:52:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5239worker2 # [ 68.958819] nix-grpc-daemon[783]: ts=2026-09-20T10:52:31Z level=info event=scheduler_connected addr=lb:500515240worker2 # [ 68.962717] nix-grpc-daemon[783]: ts=2026-09-20T10:52:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5241worker1 # [ 69.177644] nix-grpc-daemon[782]: ts=2026-09-20T10:52:32Z level=info event=scheduler_connected addr=lb:500515242worker1 # [ 69.181613] nix-grpc-daemon[782]: ts=2026-09-20T10:52:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5243worker2 # [ 69.163546] nix-grpc-daemon[783]: ts=2026-09-20T10:52:31Z level=info event=scheduler_connected addr=lb:500515244worker2 # [ 69.167433] nix-grpc-daemon[783]: ts=2026-09-20T10:52:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5245lb # [ 69.193120] niks3-server[1039]: 2026/09/20 10:52:32 INFO lead: acquired remote=192.168.1.3:470785246worker1 # [ 69.214587] nix-grpc-daemon[782]: ts=2026-09-20T10:52:32Z level=info event=scheduler_take_over5247worker1 # [ 69.382559] nix-grpc-daemon[782]: ts=2026-09-20T10:52:32Z level=info event=scheduler_connected addr=lb:500515248worker1 # [ 69.386398] nix-grpc-daemon[782]: ts=2026-09-20T10:52:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5249worker2 # [ 69.369581] nix-grpc-daemon[783]: ts=2026-09-20T10:52:31Z level=info event=scheduler_connected addr=lb:500515250worker2 # [ 69.372647] nix-grpc-daemon[783]: ts=2026-09-20T10:52:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5251worker2 # [ 69.541933] nix-grpc-daemon[783]: ts=2026-09-20T10:52:31Z level=info event=scheduler_restarting5252worker2 # [ 69.560556] nix-daemon[845]: reaped child process 847, status = succeeded5253worker1 # [ 69.587912] nix-grpc-daemon[782]: ts=2026-09-20T10:52:32Z level=info event=scheduler_connected addr=lb:500515254worker1 # [ 69.591626] nix-grpc-daemon[782]: ts=2026-09-20T10:52:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5255worker2 # [ 69.575972] nix-grpc-daemon[783]: ts=2026-09-20T10:52:31Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5256worker1 # [ 69.789621] nix-grpc-daemon[782]: ts=2026-09-20T10:52:32Z level=info event=scheduler_connected addr=lb:500515257worker1 # [ 69.793536] nix-grpc-daemon[782]: ts=2026-09-20T10:52:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5258worker1 # [ 69.994408] nix-grpc-daemon[782]: ts=2026-09-20T10:52:33Z level=info event=scheduler_connected addr=lb:500515259worker1 # [ 69.998341] nix-grpc-daemon[782]: ts=2026-09-20T10:52:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5260worker1 # [ 70.198918] nix-grpc-daemon[782]: ts=2026-09-20T10:52:33Z level=info event=scheduler_connected addr=lb:500515261worker1 # [ 70.202778] nix-grpc-daemon[782]: ts=2026-09-20T10:52:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5262worker1 # [ 70.403601] nix-grpc-daemon[782]: ts=2026-09-20T10:52:33Z level=info event=scheduler_connected addr=lb:500515263worker1 # [ 70.408724] nix-grpc-daemon[782]: ts=2026-09-20T10:52:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5264worker2 # [ 70.424975] nix-grpc-daemon[783]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5265worker2 # [ 70.428463] nix-grpc-daemon[783]: E0000 00:00:1789901552.714749 826 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5266worker2 # [ 70.433109] nix-grpc-daemon[783]: E0000 00:00:1789901552.715475 826 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5267worker1 # [ 70.610756] nix-grpc-daemon[782]: ts=2026-09-20T10:52:33Z level=info event=scheduler_connected addr=lb:500515268worker1 # [ 70.623287] nix-grpc-daemon[782]: ts=2026-09-20T10:52:33Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05269worker2 # [ 71.006550] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5270worker2 # [ 71.010138] systemd[1]: Stopped Nix worker-protocol over gRPC.5271worker2 # [ 71.014390] systemd[1]: nix-grpc-daemon.service: Consumed 2.503s CPU time over 53.014s wall clock time, 63.8M memory peak, 91.3K incoming IP traffic, 179.3K outgoing IP traffic.5272worker2 # [ 71.025983] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5273worker2 # [ 71.029805] systemd[1]: Closed Nix worker-protocol over gRPC.5274worker2 # [ 71.032565] systemd[1]: nix-grpc-daemon.socket: Consumed 282us CPU time over 1min 1.888s wall clock time, 256K memory peak, 88.2K incoming IP traffic, 70.5K outgoing IP traffic.5275lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5276lb: must succeed: curl -sf localhost:9901/clusters5277lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5278lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.08 seconds)5279client: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25280worker1 # [ 71.337559] nix-grpc-daemon[782]: ts=2026-09-20T10:52:34Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:487825281worker1 # [ 71.341801] nix-grpc-daemon[782]: ts=2026-09-20T10:52:34Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:487825282client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5283client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.22 seconds)5284worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5285worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.03 seconds)5286worker2 # [ 71.423758] systemd[1]: Listening on Nix worker-protocol over gRPC.5287worker1 # [ 71.488458] systemd[1]: Stopping Nix worker-protocol over gRPC...5288worker2 # [ 71.995946] systemd[1]: Starting Nix worker-protocol over gRPC...5289worker2 # [ 72.172810] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:34Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5290worker2 # [ 72.175971] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:34Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15291worker2 # [ 72.182611] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:34Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5292worker2 # [ 72.186291] systemd[1]: Started Nix worker-protocol over gRPC.5293worker2 # [ 72.214219] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:34Z level=info event=scheduler_connected addr=lb:500515294worker1 # [ 72.243756] nix-grpc-daemon[782]: ts=2026-09-20T10:52:35Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05295worker1 # [ 72.295602] nix-grpc-daemon[782]: ts=2026-09-20T10:52:35Z level=info event=signal_exit5296worker1 # [ 72.547403] nix-grpc-daemon[782]: ts=2026-09-20T10:52:35Z level=info event=scheduler_yield5297worker2 # [ 72.523859] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:34Z level=info event=scheduler_restarting addr=lb:500515298lb # [ 72.530330] niks3-server[1039]: 2026/09/20 10:52:35 INFO lead: released remote=192.168.1.3:470785299worker1 # [ 72.552396] nix-grpc-daemon[782]: ts=2026-09-20T10:52:35Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:369405300worker1 # [ 72.556823] nix-grpc-daemon[782]: ts=2026-09-20T10:52:35Z level=info event=worker_gone addr=192.168.1.4:500515301worker1 # [ 72.561266] nix-grpc-daemon[782]: ts=2026-09-20T10:52:35Z level=info event=scheduler_restarting addr=lb:500515302worker1 # [ 72.564268] nix-grpc-daemon[782]: ts=2026-09-20T10:52:35Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:369405303worker1 # [ 72.567533] nix-grpc-daemon[782]: ts=2026-09-20T10:52:35Z level=info event=worker_gone addr=192.168.1.3:500515304worker2 # [ 72.724431] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:35Z level=info event=scheduler_connected addr=lb:500515305worker2 # [ 72.728283] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5306worker1 # [ 72.757812] nix-grpc-daemon[782]: ts=2026-09-20T10:52:35Z level=info event=scheduler_connected addr=lb:500515307worker1 # [ 72.761639] nix-grpc-daemon[782]: ts=2026-09-20T10:52:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5308worker2 # [ 72.928907] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:35Z level=info event=scheduler_connected addr=lb:500515309worker2 # [ 72.932782] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5310worker1 # [ 72.962447] nix-grpc-daemon[782]: ts=2026-09-20T10:52:36Z level=info event=scheduler_connected addr=lb:500515311worker1 # [ 72.966374] nix-grpc-daemon[782]: ts=2026-09-20T10:52:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5312worker2 # [ 73.133573] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:35Z level=info event=scheduler_connected addr=lb:500515313worker2 # [ 73.137480] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5314worker1 # [ 73.167083] nix-grpc-daemon[782]: ts=2026-09-20T10:52:36Z level=info event=scheduler_connected addr=lb:500515315worker1 # [ 73.170809] nix-grpc-daemon[782]: ts=2026-09-20T10:52:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5316worker2 # [ 73.339149] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:35Z level=info event=scheduler_connected addr=lb:500515317worker2 # [ 73.342822] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5318worker1 # [ 73.373167] nix-grpc-daemon[782]: ts=2026-09-20T10:52:36Z level=info event=scheduler_connected addr=lb:500515319worker1 # [ 73.376379] nix-grpc-daemon[782]: ts=2026-09-20T10:52:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5320worker1 # [ 73.551411] nix-grpc-daemon[782]: ts=2026-09-20T10:52:36Z level=info event=scheduler_restarting5321worker2 # [ 73.543790] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:35Z level=info event=scheduler_connected addr=lb:500515322worker2 # [ 73.547628] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5323worker1 # [ 73.585733] nix-grpc-daemon[782]: ts=2026-09-20T10:52:36Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5324worker1 # [ 73.590983] nix-daemon[840]: reaped child process 842, status = succeeded5325worker2 # [ 73.748568] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:36Z level=info event=scheduler_connected addr=lb:500515326worker2 # [ 73.752403] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5327worker2 # [ 73.952834] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:36Z level=info event=scheduler_connected addr=lb:500515328worker2 # [ 73.954748] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5329worker2 # [ 74.155229] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:36Z level=info event=scheduler_connected addr=lb:500515330worker2 # [ 74.156927] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5331worker2 # [ 74.357807] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:36Z level=info event=scheduler_connected addr=lb:500515332worker2 # [ 74.359485] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5333worker1 # [ 74.419588] nix-grpc-daemon[782]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5334worker1 # [ 74.423096] nix-grpc-daemon[782]: E0000 00:00:1789901557.563617 825 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5335worker1 # [ 74.427710] nix-grpc-daemon[782]: E0000 00:00:1789901557.564488 825 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5336worker2 # [ 74.560054] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:36Z level=info event=scheduler_connected addr=lb:500515337worker2 # [ 74.562133] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5338worker2 # [ 74.762650] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:37Z level=info event=scheduler_connected addr=lb:500515339worker2 # [ 74.764356] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5340worker1 # [ 74.976732] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5341worker1 # [ 74.979584] systemd[1]: Stopped Nix worker-protocol over gRPC.5342worker1 # [ 74.982743] systemd[1]: nix-grpc-daemon.service: Consumed 1.425s CPU time over 56.963s wall clock time, 50M memory peak, 51.7K incoming IP traffic, 95.1K outgoing IP traffic.5343worker2 # [ 74.965294] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:37Z level=info event=scheduler_connected addr=lb:500515344worker2 # [ 74.969118] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5345worker1 # [ 74.995250] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5346worker1 # [ 74.998673] systemd[1]: Closed Nix worker-protocol over gRPC.5347worker1 # [ 75.001365] systemd[1]: nix-grpc-daemon.socket: Consumed 296us CPU time over 1min 4.727s wall clock time, 256K memory peak, 63.7K incoming IP traffic, 56.4K outgoing IP traffic.5348lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5349lb: must succeed: curl -sf localhost:9901/clusters5350lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5351lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.08 seconds)5352client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25353worker2 # [ 75.169853] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:37Z level=info event=scheduler_connected addr=lb:500515354worker2 # [ 75.174128] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5355worker2 # [ 75.272705] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:565545356worker2 # [ 75.284140] nix-daemon[845]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5357worker2 # [ 75.300139] nix-grpc-daemon[1246]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 36 ms (attempt 1/5)5358worker2 # [ 75.341864] nix-grpc-daemon[1246]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 36 ms (attempt 2/5)5359worker2 # [ 75.375515] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:37Z level=info event=scheduler_connected addr=lb:500515360worker2 # [ 75.379429] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5361worker2 # [ 75.383636] nix-grpc-daemon[1246]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 388 ms (attempt 3/5)5362worker2 # [ 75.580128] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:37Z level=info event=scheduler_connected addr=lb:500515363worker2 # [ 75.584149] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5364worker2 # [ 75.777295] nix-grpc-daemon[1246]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 217 ms (attempt 4/5)5365worker2 # [ 75.784733] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:38Z level=info event=scheduler_connected addr=lb:500515366worker2 # [ 75.788500] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5367worker2 # [ 75.989293] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:38Z level=info event=scheduler_connected addr=lb:500515368worker2 # [ 75.993322] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5369worker2 # [ 76.000527] nix-grpc-daemon[1246]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5370client # copying 2 paths...5371client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5372worker2 # [ 76.018456] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:565545373client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5374worker2 # [ 76.029935] nix-daemon[845]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5375worker2 # [ 76.051150] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:38Z level=info event=niks3_push_started pid=13025376worker2 # [ 76.193535] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:38Z level=info event=scheduler_connected addr=lb:500515377worker2 # [ 76.196408] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5378worker2 # [ 76.383514] nix-daemon[845]: accepted connection from pid 1315, user nix-grpc-daemon (trusted)5379worker2 # [ 76.394161] nix-daemon[845]: reaped child process 1318, status = succeeded5380worker2 # [ 76.397137] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:38Z level=info event=scheduler_connected addr=lb:500515381worker2 # [ 76.398953] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5382lb # [ 76.419556] niks3-server[1039]: 2026/09/20 10:52:39 INFO Received uploads request method=POST path=/api/pending_closures5383lb # [ 76.428295] niks3-server[1039]: 2026/09/20 10:52:39 INFO Received uploads request method=POST path=/api/pending_closures5384worker2 # [ 76.432284] nix-grpc-daemon[1302]: time=2026-09-20T10:52:38.722Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5385worker2 # [ 76.435823] nix-grpc-daemon[1302]: time=2026-09-20T10:52:38.722Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5386worker2 # [ 76.437721] nix-grpc-daemon[1302]: time=2026-09-20T10:52:38.722Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5387lb # [ 76.468697] niks3-server[1039]: 2026/09/20 10:52:39 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5388lb # [ 76.487376] niks3-server[1039]: 2026/09/20 10:52:39 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5389lb # [ 76.495108] niks3-server[1039]: 2026/09/20 10:52:39 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5390lb # [ 76.502389] niks3-server[1039]: 2026/09/20 10:52:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5391lb # [ 76.504500] niks3-server[1039]: 2026/09/20 10:52:39 INFO Signed narinfos id=25 count=15392lb # [ 76.506194] niks3-server[1039]: 2026/09/20 10:52:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5393lb # [ 76.509067] niks3-server[1039]: 2026/09/20 10:52:39 INFO Signed narinfos id=26 count=25394lb # [ 76.510826] niks3-server[1039]: 2026/09/20 10:52:39 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5395worker2 # [ 76.505132] nix-grpc-daemon[1302]: time=2026-09-20T10:52:38.795Z level=INFO msg="Uploading 3 narinfos"5396lb # [ 76.525485] niks3-server[1039]: 2026/09/20 10:52:39 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5397lb # [ 76.528527] niks3-server[1039]: 2026/09/20 10:52:39 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5398lb # [ 76.530957] niks3-server[1039]: 2026/09/20 10:52:39 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5399lb # [ 76.534425] niks3-server[1039]: 2026/09/20 10:52:39 INFO Completed upload id=255400lb # [ 76.535728] niks3-server[1039]: 2026/09/20 10:52:39 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5401lb # [ 76.539037] niks3-server[1039]: 2026/09/20 10:52:39 INFO Completed upload id=265402lb # [ 76.540880] niks3-server[1039]: 2026/09/20 10:52:39 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5403worker2 # [ 76.534597] nix-grpc-daemon[1302]: time=2026-09-20T10:52:38.825Z level=INFO msg="Upload complete. (300ms)"5404worker2 # [ 76.539938] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56554 duration_s=0 paths=2 nar_bytes_in=3125405worker2 # [ 76.552489] nix-daemon[845]: reaped child process 1300, status = succeeded5406client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.52 seconds)5407worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5408worker2 # [ 76.599875] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:38Z level=info event=scheduler_connected addr=lb:500515409worker2 # [ 76.604543] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5410worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.03 seconds)5411worker1 # [ 76.696456] systemd[1]: Listening on Nix worker-protocol over gRPC.5412worker1 # [ 76.703153] systemd[1]: Starting Nix worker-protocol over gRPC...5413worker2 # [ 76.805098] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:39Z level=info event=scheduler_connected addr=lb:500515414worker2 # [ 76.809368] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5415worker1 # [ 76.864849] nix-grpc-daemon[1041]: ts=2026-09-20T10:52:40Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5416worker1 # [ 76.867642] nix-grpc-daemon[1041]: ts=2026-09-20T10:52:40Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15417worker1 # [ 76.874160] nix-grpc-daemon[1041]: ts=2026-09-20T10:52:40Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5418worker1 # [ 76.875743] systemd[1]: Started Nix worker-protocol over gRPC.5419lb # [ 76.863741] niks3-server[1039]: 2026/09/20 10:52:39 INFO lead: acquired remote=192.168.1.3:575745420worker1 # [ 76.888444] nix-grpc-daemon[1041]: ts=2026-09-20T10:52:40Z level=info event=scheduler_take_over5421lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5422lb: must succeed: curl -sf localhost:9901/clusters5423worker1 # [ 76.905552] nix-grpc-daemon[1041]: ts=2026-09-20T10:52:40Z level=info event=scheduler_connected addr=lb:500515424worker1 # [ 76.907781] nix-grpc-daemon[1041]: ts=2026-09-20T10:52:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5425lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5426worker2 # [ 77.010241] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:39Z level=info event=scheduler_connected addr=lb:500515427worker1 # [ 77.041964] nix-grpc-daemon[1041]: ts=2026-09-20T10:52:40Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05428worker1 # [ 77.108735] nix-grpc-daemon[1041]: ts=2026-09-20T10:52:40Z level=info event=scheduler_connected addr=lb:500515429worker1 # [ 77.112691] nix-grpc-daemon[1041]: ts=2026-09-20T10:52:40Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05430lb: must succeed: curl -sf localhost:9901/clusters5431lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5432lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.12 seconds)5433(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.11 seconds)5434subtest: scheduler down: the other node takes the lock and keeps it5435lb: must succeed: curl -sf localhost:9901/clusters5436lb # [ 78.000071] envoy[910]: conn peer=[2001:db8:1::4]:49782 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=13363 tx=2811 ms=512485437lb # [ 78.002374] envoy[910]: conn peer=[2001:db8:1::1]:45796 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=155438lb # [ 78.004081] envoy[910]: conn peer=192.168.1.3:49726 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=13574 tx=2567 ms=542355439lb # [ 78.006362] envoy[910]: conn peer=[2001:db8:1::1]:40682 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1008 tx=282 ms=12815440lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5441lb: must succeed: curl -sf localhost:9901/clusters5442lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5443lb: must succeed: curl -sf localhost:9901/clusters5444lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5445lb: must succeed: curl -sf localhost:9901/clusters5446lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5447worker1 # [ 78.244118] systemd[1]: Stopping Nix worker-protocol over gRPC...5448worker1 # [ 78.875527] nix-grpc-daemon[1041]: ts=2026-09-20T10:52:42Z level=info event=signal_exit5449worker1 # [ 79.083992] nix-grpc-daemon[1041]: ts=2026-09-20T10:52:42Z level=info event=scheduler_yield5450worker1 # [ 79.088244] nix-grpc-daemon[1041]: ts=2026-09-20T10:52:42Z level=info event=scheduler_restarting addr=lb:500515451lb # [ 79.065888] niks3-server[1039]: 2026/09/20 10:52:41 INFO lead: released remote=192.168.1.3:575745452worker2 # [ 79.067566] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:41Z level=info event=scheduler_restarting addr=lb:500515453worker1 # [ 79.092306] nix-grpc-daemon[1041]: ts=2026-09-20T10:52:42Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:347825454worker1 # [ 79.096723] nix-grpc-daemon[1041]: ts=2026-09-20T10:52:42Z level=info event=worker_gone addr=192.168.1.3:500515455worker1 # [ 79.099848] nix-grpc-daemon[1041]: ts=2026-09-20T10:52:42Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:347825456worker1 # [ 79.103129] nix-grpc-daemon[1041]: ts=2026-09-20T10:52:42Z level=info event=worker_gone addr=192.168.1.4:500515457worker1 # [ 79.288910] nix-grpc-daemon[1041]: ts=2026-09-20T10:52:42Z level=info event=scheduler_connected addr=lb:500515458worker2 # [ 79.267696] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:41Z level=info event=scheduler_connected addr=lb:500515459worker1 # [ 79.292865] nix-grpc-daemon[1041]: ts=2026-09-20T10:52:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5460worker2 # [ 79.269484] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5461worker1 # [ 79.493569] nix-grpc-daemon[1041]: ts=2026-09-20T10:52:42Z level=info event=scheduler_connected addr=lb:500515462worker2 # [ 79.469901] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:41Z level=info event=scheduler_connected addr=lb:500515463worker2 # [ 79.471675] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5464worker1 # [ 79.497678] nix-grpc-daemon[1041]: ts=2026-09-20T10:52:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5465worker1 # [ 79.698315] nix-grpc-daemon[1041]: ts=2026-09-20T10:52:42Z level=info event=scheduler_connected addr=lb:500515466worker2 # [ 79.672654] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:41Z level=info event=scheduler_connected addr=lb:500515467worker1 # [ 79.702191] nix-grpc-daemon[1041]: ts=2026-09-20T10:52:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5468worker2 # [ 79.676629] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5469worker2 # [ 79.877558] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:42Z level=info event=scheduler_connected addr=lb:500515470worker1 # [ 79.903667] nix-grpc-daemon[1041]: ts=2026-09-20T10:52:43Z level=info event=scheduler_connected addr=lb:500515471worker2 # [ 79.881417] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5472worker1 # [ 79.907411] nix-grpc-daemon[1041]: ts=2026-09-20T10:52:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5473worker1 # [ 80.085285] nix-grpc-daemon[1041]: ts=2026-09-20T10:52:43Z level=info event=scheduler_restarting5474worker2 # [ 80.083645] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:42Z level=info event=scheduler_connected addr=lb:500515475worker2 # [ 80.086761] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5476worker2 # [ 80.284749] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:42Z level=info event=scheduler_connected addr=lb:500515477worker2 # [ 80.288520] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5478worker2 # [ 80.489425] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:42Z level=info event=scheduler_connected addr=lb:500515479worker2 # [ 80.493298] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5480worker2 # [ 80.694139] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:42Z level=info event=scheduler_connected addr=lb:500515481worker2 # [ 80.698132] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5482worker1 # [ 80.906731] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5483worker1 # [ 80.911130] systemd[1]: Stopped Nix worker-protocol over gRPC.5484worker1 # [ 80.914571] systemd[1]: nix-grpc-daemon.service: Consumed 134ms CPU time over 4.202s wall clock time, 12.1M memory peak, 7.4K incoming IP traffic, 7.9K outgoing IP traffic.5485worker1 # [ 80.923123] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5486worker2 # [ 80.898973] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:43Z level=info event=scheduler_connected addr=lb:500515487worker1 # [ 80.927436] systemd[1]: Closed Nix worker-protocol over gRPC.5488worker2 # [ 80.902965] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5489worker1 # [ 80.930971] systemd[1]: nix-grpc-daemon.socket: Consumed 563us CPU time over 4.238s wall clock time, 256K memory peak, 11.5K incoming IP traffic, 11K outgoing IP traffic.5490worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5491worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.07 seconds)5492worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5493worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5494worker2 # [ 81.104570] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:43Z level=info event=scheduler_connected addr=lb:500515495worker2 # [ 81.108228] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5496worker2 # [ 81.305716] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:43Z level=info event=scheduler_connected addr=lb:500515497worker2 # [ 81.307441] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5498worker2 # [ 81.508683] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:43Z level=info event=scheduler_connected addr=lb:500515499worker2 # [ 81.513183] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5500worker2 # [ 81.713938] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:44Z level=info event=scheduler_connected addr=lb:500515501worker2 # [ 81.718410] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5502worker2 # [ 81.919166] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:44Z level=info event=scheduler_connected addr=lb:500515503worker2 # [ 81.923190] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5504worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5505worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5506worker2 # [ 82.124690] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:44Z level=info event=scheduler_connected addr=lb:500515507worker2 # [ 82.128626] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5508worker2 # [ 82.195823] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:44Z level=info event=scheduler_take_over5509lb # [ 82.200979] niks3-server[1039]: 2026/09/20 10:52:45 INFO lead: acquired remote=192.168.1.4:335125510worker2 # [ 82.329474] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:44Z level=info event=scheduler_connected addr=lb:500515511worker2 # [ 82.341486] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:44Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05512worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5513worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5514client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&25515worker2 # [ 83.336611] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:565545516client # don't know how to build these paths:5517client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5518worker2 # [ 83.341835] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:565545519worker2 # [ 83.359040] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:45Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05520worker2 # [ 83.361315] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:45Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.4:50051 clients=15521worker2 # [ 83.401597] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565545522worker2 # [ 83.408300] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:565545523client # copying 1 paths to 'lb:50051'...5524worker2 # [ 83.414905] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:565545525client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5526worker2 # [ 83.421986] nix-daemon[845]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5527worker2 # [ 83.567992] nix-daemon[845]: accepted connection from pid 1342, user nix-grpc-daemon (trusted)5528worker2 # [ 83.579401] nix-daemon[845]: reaped child process 1345, status = succeeded5529worker2 # [ 83.729961] nix-daemon[845]: accepted connection from pid 1347, user nix-grpc-daemon (trusted)5530worker2 # [ 83.741498] nix-daemon[845]: reaped child process 1350, status = succeeded5531lb # [ 83.759869] niks3-server[1039]: 2026/09/20 10:52:46 INFO Received uploads request method=POST path=/api/pending_closures5532worker2 # [ 83.761894] nix-grpc-daemon[1302]: time=2026-09-20T10:52:46.052Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5533worker2 # [ 83.763341] nix-grpc-daemon[1302]: time=2026-09-20T10:52:46.052Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5534lb # [ 83.786114] niks3-server[1039]: 2026/09/20 10:52:46 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5535lb # [ 83.792943] niks3-server[1039]: 2026/09/20 10:52:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5536lb # [ 83.803215] niks3-server[1039]: 2026/09/20 10:52:46 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5537lb # [ 83.806992] niks3-server[1039]: 2026/09/20 10:52:46 INFO Signed narinfos id=27 count=15538worker2 # [ 83.802487] nix-grpc-daemon[1302]: time=2026-09-20T10:52:46.093Z level=INFO msg="Uploading 1 narinfos"5539lb # [ 83.812759] niks3-server[1039]: 2026/09/20 10:52:46 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5540lb # [ 83.818428] niks3-server[1039]: 2026/09/20 10:52:46 INFO Completed upload id=275541worker2 # [ 83.813700] nix-grpc-daemon[1302]: time=2026-09-20T10:52:46.104Z level=INFO msg="Upload complete. (386ms)"5542lb # [ 83.823080] niks3-server[1039]: 2026/09/20 10:52:46 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5543worker2 # [ 83.815692] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56554 duration_s=0 paths=1 nar_bytes_in=5205544worker2 # [ 83.821402] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565545545worker2 # [ 83.825893] nix-daemon[845]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5546worker2 # [ 83.828217] nix-daemon[845]: reaped child process 1340, status = succeeded5547worker2 # [ 83.833092] nix-daemon[845]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5548client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5549client # farm-a> worker2: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5550client # farm-a> LOG-farm-a-on-standby5551worker2 # [ 84.658319] nix-daemon[845]: reaped child process 1354, status = succeeded5552worker2 # [ 84.785929] nix-daemon[845]: accepted connection from pid 1369, user nix-grpc-daemon (trusted)5553worker2 # [ 84.797399] nix-daemon[845]: reaped child process 1372, status = succeeded5554lb # [ 84.815475] niks3-server[1039]: 2026/09/20 10:52:47 INFO Received uploads request method=POST path=/api/pending_closures5555worker2 # [ 84.821179] nix-grpc-daemon[1302]: time=2026-09-20T10:52:47.112Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5556worker2 # [ 84.822644] nix-grpc-daemon[1302]: time=2026-09-20T10:52:47.113Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5557lb # [ 84.854685] niks3-server[1039]: 2026/09/20 10:52:47 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5558lb # [ 84.862118] niks3-server[1039]: 2026/09/20 10:52:47 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5559lb # [ 84.872304] niks3-server[1039]: 2026/09/20 10:52:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5560lb # [ 84.876509] niks3-server[1039]: 2026/09/20 10:52:47 INFO Signed narinfos id=28 count=15561worker2 # [ 84.876196] nix-grpc-daemon[1302]: time=2026-09-20T10:52:47.166Z level=INFO msg="Uploading 1 narinfos"5562lb # [ 84.883762] niks3-server[1039]: 2026/09/20 10:52:47 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5563lb # [ 84.891592] niks3-server[1039]: 2026/09/20 10:52:47 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5564lb # [ 84.898273] niks3-server[1039]: 2026/09/20 10:52:47 INFO Completed upload id=285565lb # [ 84.899428] niks3-server[1039]: 2026/09/20 10:52:47 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5566worker2 # [ 84.893642] nix-grpc-daemon[1302]: time=2026-09-20T10:52:47.184Z level=INFO msg="Upload complete. (253ms)"5567worker2 # [ 84.899837] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56554 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15568worker2 # [ 84.912297] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:47Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05569worker2 # [ 84.916223] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:47Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.4:50051 clients=15570worker2 # [ 84.924237] nix-daemon[845]: reaped child process 1352, status = succeeded5571worker2 # [ 84.955368] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565545572worker2 # [ 84.966676] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:565545573worker2 # [ 84.975076] nix-grpc-daemon[1302]: time=2026-09-20T10:52:47.266Z level=INFO msg="All 1 paths already cached"5574client # copying 1 paths to 'lb:50051'...5575client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5576worker2 # [ 84.982538] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:565545577worker2 # [ 84.992131] nix-daemon[845]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5578worker2 # [ 85.136969] nix-daemon[845]: accepted connection from pid 1378, user nix-grpc-daemon (trusted)5579worker2 # [ 85.148742] nix-daemon[845]: reaped child process 1381, status = succeeded5580worker2 # [ 85.289594] nix-daemon[845]: accepted connection from pid 1383, user nix-grpc-daemon (trusted)5581worker2 # [ 85.301173] nix-daemon[845]: reaped child process 1386, status = succeeded5582lb # [ 85.318873] niks3-server[1039]: 2026/09/20 10:52:48 INFO Received uploads request method=POST path=/api/pending_closures5583worker2 # [ 85.322184] nix-grpc-daemon[1302]: time=2026-09-20T10:52:47.613Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5584worker2 # [ 85.323542] nix-grpc-daemon[1302]: time=2026-09-20T10:52:47.613Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5585lb # [ 85.342830] niks3-server[1039]: 2026/09/20 10:52:48 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5586lb # [ 85.354902] niks3-server[1039]: 2026/09/20 10:52:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5587lb # [ 85.362112] niks3-server[1039]: 2026/09/20 10:52:48 INFO Signed narinfos id=29 count=15588worker2 # [ 85.360472] nix-grpc-daemon[1302]: time=2026-09-20T10:52:47.651Z level=INFO msg="Uploading 1 narinfos"5589lb # [ 85.367064] niks3-server[1039]: 2026/09/20 10:52:48 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5590lb # [ 85.377084] niks3-server[1039]: 2026/09/20 10:52:48 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5591worker2 # [ 85.379097] nix-grpc-daemon[1302]: time=2026-09-20T10:52:47.670Z level=INFO msg="Upload complete. (377ms)"5592lb # [ 85.384367] niks3-server[1039]: 2026/09/20 10:52:48 INFO Completed upload id=295593worker2 # [ 85.380988] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56554 duration_s=0 paths=1 nar_bytes_in=6405594lb # [ 85.387946] niks3-server[1039]: 2026/09/20 10:52:48 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5595worker2 # [ 85.387581] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565545596worker2 # [ 85.393126] nix-daemon[845]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5597worker2 # [ 85.396114] nix-daemon[845]: reaped child process 1376, status = succeeded5598worker2 # [ 85.400835] nix-daemon[845]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5599client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5600client # farm-b> worker2: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5601worker2 # [ 86.413159] nix-daemon[845]: reaped child process 1390, status = succeeded5602client # farm-b> LOG-farm-b-on-standby5603worker2 # [ 86.540538] nix-daemon[845]: accepted connection from pid 1405, user nix-grpc-daemon (trusted)5604worker2 # [ 86.552924] nix-daemon[845]: reaped child process 1408, status = succeeded5605lb # [ 86.570622] niks3-server[1039]: 2026/09/20 10:52:49 INFO Received uploads request method=POST path=/api/pending_closures5606worker2 # [ 86.570831] nix-grpc-daemon[1302]: time=2026-09-20T10:52:48.861Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5607worker2 # [ 86.572193] nix-grpc-daemon[1302]: time=2026-09-20T10:52:48.861Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"5608lb # [ 86.585152] niks3-server[1039]: 2026/09/20 10:52:49 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5609lb # [ 86.589951] niks3-server[1039]: 2026/09/20 10:52:49 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst5610lb # [ 86.592322] niks3-server[1039]: 2026/09/20 10:52:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5611lb # [ 86.594362] niks3-server[1039]: 2026/09/20 10:52:49 INFO Signed narinfos id=30 count=15612worker2 # [ 86.589594] nix-grpc-daemon[1302]: time=2026-09-20T10:52:48.880Z level=INFO msg="Uploading 1 narinfos"5613lb # [ 86.597723] niks3-server[1039]: 2026/09/20 10:52:49 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls5614lb # [ 86.602050] niks3-server[1039]: 2026/09/20 10:52:49 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5615lb # [ 86.604977] niks3-server[1039]: 2026/09/20 10:52:49 INFO Completed upload id=305616worker2 # [ 86.601323] nix-grpc-daemon[1302]: time=2026-09-20T10:52:48.892Z level=INFO msg="Upload complete. (194ms)"5617lb # [ 86.606856] niks3-server[1039]: 2026/09/20 10:52:49 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo5618worker2 # [ 86.603633] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56554 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=15619worker2 # [ 86.610511] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:48Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=05620worker2 # [ 86.612471] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:48Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.4:50051 clients=15621worker2 # [ 86.615797] nix-daemon[845]: reaped child process 1388, status = succeeded5622worker2 # [ 86.654503] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565545623worker2 # [ 86.666217] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:565545624worker2 # [ 86.677312] nix-grpc-daemon[1302]: time=2026-09-20T10:52:48.968Z level=INFO msg="All 2 paths already cached"5625client # copying 1 paths to 'lb:50051'...5626client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...5627worker2 # [ 86.684387] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:565545628worker2 # [ 86.692135] nix-daemon[845]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5629worker2 # [ 86.834768] nix-daemon[845]: accepted connection from pid 1414, user nix-grpc-daemon (trusted)5630worker2 # [ 86.846378] nix-daemon[845]: reaped child process 1417, status = succeeded5631worker2 # [ 86.991343] nix-daemon[845]: accepted connection from pid 1419, user nix-grpc-daemon (trusted)5632worker2 # [ 87.002956] nix-daemon[845]: reaped child process 1422, status = succeeded5633lb # [ 87.021597] niks3-server[1039]: 2026/09/20 10:52:49 INFO Received uploads request method=POST path=/api/pending_closures5634worker2 # [ 87.024936] nix-grpc-daemon[1302]: time=2026-09-20T10:52:49.315Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5635worker2 # [ 87.026522] nix-grpc-daemon[1302]: time=2026-09-20T10:52:49.315Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"5636lb # [ 87.045363] niks3-server[1039]: 2026/09/20 10:52:49 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst5637lb # [ 87.058741] niks3-server[1039]: 2026/09/20 10:52:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5638lb # [ 87.065369] niks3-server[1039]: 2026/09/20 10:52:49 INFO Signed narinfos id=31 count=15639worker2 # [ 87.063769] nix-grpc-daemon[1302]: time=2026-09-20T10:52:49.354Z level=INFO msg="Uploading 1 narinfos"5640lb # [ 87.069978] niks3-server[1039]: 2026/09/20 10:52:49 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls5641lb # [ 87.082639] niks3-server[1039]: 2026/09/20 10:52:50 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5642lb # [ 87.089408] niks3-server[1039]: 2026/09/20 10:52:50 INFO Completed upload id=315643worker2 # [ 87.084685] nix-grpc-daemon[1302]: time=2026-09-20T10:52:49.375Z level=INFO msg="Upload complete. (383ms)"5644lb # [ 87.091823] niks3-server[1039]: 2026/09/20 10:52:50 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo5645worker2 # [ 87.088872] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56554 duration_s=0 paths=1 nar_bytes_in=7685646worker2 # [ 87.094664] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565545647worker2 # [ 87.104412] nix-daemon[845]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5648worker2 # [ 87.108319] nix-daemon[845]: reaped child process 1412, status = succeeded5649worker2 # [ 87.113240] nix-daemon[845]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5650client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...5651client # farm-top> worker2: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5652lb # [ 88.000947] envoy[910]: conn peer=[2001:db8:1::3]:43492 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1547 tx=400 ms=32105653client # farm-top> LOG-farm-top-on-standby5654worker2 # [ 88.124940] nix-daemon[845]: reaped child process 1426, status = succeeded5655worker2 # [ 88.259432] nix-daemon[845]: accepted connection from pid 1441, user nix-grpc-daemon (trusted)5656worker2 # [ 88.271158] nix-daemon[845]: reaped child process 1444, status = succeeded5657lb # [ 88.290209] niks3-server[1039]: 2026/09/20 10:52:51 INFO Received uploads request method=POST path=/api/pending_closures5658worker2 # [ 88.295272] nix-grpc-daemon[1302]: time=2026-09-20T10:52:50.585Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5659worker2 # [ 88.296606] nix-grpc-daemon[1302]: time=2026-09-20T10:52:50.585Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"5660lb # [ 88.318409] niks3-server[1039]: 2026/09/20 10:52:51 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5661lb # [ 88.333189] niks3-server[1039]: 2026/09/20 10:52:51 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst5662lb # [ 88.345364] niks3-server[1039]: 2026/09/20 10:52:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5663lb # [ 88.349094] niks3-server[1039]: 2026/09/20 10:52:51 INFO Signed narinfos id=32 count=15664worker2 # [ 88.346805] nix-grpc-daemon[1302]: time=2026-09-20T10:52:50.637Z level=INFO msg="Uploading 1 narinfos"5665lb # [ 88.353701] niks3-server[1039]: 2026/09/20 10:52:51 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls5666lb # [ 88.362384] niks3-server[1039]: 2026/09/20 10:52:51 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5667lb # [ 88.367768] niks3-server[1039]: 2026/09/20 10:52:51 INFO Completed upload id=325668worker2 # [ 88.363228] nix-grpc-daemon[1302]: time=2026-09-20T10:52:50.654Z level=INFO msg="Upload complete. (242ms)"5669lb # [ 88.369420] niks3-server[1039]: 2026/09/20 10:52:51 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo5670worker2 # [ 88.370373] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56554 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=15671worker2 # [ 88.382449] nix-daemon[845]: reaped child process 1424, status = succeeded5672client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&2, in 5.27 seconds)5673client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath5674client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath, in 0.21 seconds)5675worker1 # [ 88.698808] systemd[1]: Listening on Nix worker-protocol over gRPC.5676worker1 # [ 88.703280] systemd[1]: Starting Nix worker-protocol over gRPC...5677worker1 # [ 88.873508] nix-grpc-daemon[1105]: ts=2026-09-20T10:52:52Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5678worker1 # [ 88.876855] nix-grpc-daemon[1105]: ts=2026-09-20T10:52:52Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15679worker1 # [ 88.883878] nix-grpc-daemon[1105]: ts=2026-09-20T10:52:52Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5680worker1 # [ 88.885417] systemd[1]: Started Nix worker-protocol over gRPC.5681lb: must succeed: curl -sf localhost:9901/clusters5682worker1 # [ 88.914601] nix-grpc-daemon[1105]: ts=2026-09-20T10:52:52Z level=info event=scheduler_connected addr=lb:500515683worker2 # [ 88.890585] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:51Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05684lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5685worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5686worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5687lb: must succeed: curl -sf localhost:9901/clusters5688lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5689lb: must succeed: curl -sf localhost:9901/clusters5690lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5691worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5692worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5693(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 11.20 seconds)5694subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5695client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs5696client # [ 89.254774] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs.5697client # Running as unit: rs.service; invocation ID: cdd78b47a739451eb7dec2558ac995535698client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs, in 0.07 seconds)5699worker1 # [ 89.475920] nix-grpc-daemon[1105]: ts=2026-09-20T10:52:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:450005700worker1 # [ 89.487569] nix-daemon[840]: accepted connection from pid 1105, user nix-grpc-daemon (trusted)5701client # [ 89.488129] nix[1175]: don't know how to build these paths:5702client # [ 89.489401] nix[1175]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5703worker2 # [ 89.486662] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:51Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=05704worker2 # [ 89.490638] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:51Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.4:50051 clients=15705worker2 # [ 89.528982] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565545706worker2 # [ 89.542308] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:565545707client # [ 89.566322] nix[1175]: copying 1 paths to 'lb:50051'...5708worker2 # [ 89.553566] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:565545709client # [ 89.569196] nix[1175]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...5710worker2 # [ 89.564318] nix-daemon[845]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5711worker2 # [ 89.740668] nix-daemon[845]: accepted connection from pid 1457, user nix-grpc-daemon (trusted)5712worker2 # [ 89.752460] nix-daemon[845]: reaped child process 1460, status = succeeded5713worker2 # [ 89.911225] nix-daemon[845]: accepted connection from pid 1462, user nix-grpc-daemon (trusted)5714worker2 # [ 89.922492] nix-daemon[845]: reaped child process 1465, status = succeeded5715lb # [ 89.946524] niks3-server[1039]: 2026/09/20 10:52:52 INFO Received uploads request method=POST path=/api/pending_closures5716worker2 # [ 89.948896] nix-grpc-daemon[1302]: time=2026-09-20T10:52:52.239Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5717worker2 # [ 89.950288] nix-grpc-daemon[1302]: time=2026-09-20T10:52:52.239Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"5718lb # [ 89.974453] niks3-server[1039]: 2026/09/20 10:52:52 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst5719lb # [ 89.983395] niks3-server[1039]: 2026/09/20 10:52:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5720lb # [ 89.988683] niks3-server[1039]: 2026/09/20 10:52:52 INFO Signed narinfos id=33 count=15721worker2 # [ 89.986462] nix-grpc-daemon[1302]: time=2026-09-20T10:52:52.277Z level=INFO msg="Uploading 1 narinfos"5722lb # [ 89.999540] niks3-server[1039]: 2026/09/20 10:52:52 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls5723lb # [ 90.003156] niks3-server[1039]: 2026/09/20 10:52:52 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5724lb # [ 90.008139] niks3-server[1039]: 2026/09/20 10:52:52 INFO Completed upload id=335725worker2 # [ 90.004713] nix-grpc-daemon[1302]: time=2026-09-20T10:52:52.295Z level=INFO msg="Upload complete. (429ms)"5726lb # [ 90.010581] niks3-server[1039]: 2026/09/20 10:52:52 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo5727worker2 # [ 90.008088] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56554 duration_s=0 paths=1 nar_bytes_in=4165728worker2 # [ 90.013258] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565545729worker2 # [ 90.022057] nix-daemon[845]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5730worker2 # [ 90.026052] nix-daemon[845]: reaped child process 1455, status = succeeded5731worker2 # [ 90.030760] nix-daemon[845]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5732client # [ 90.043810] nix[1175]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...5733lb: must succeed: curl -sf localhost:9901/clusters5734lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5735lb: must succeed: curl -sf localhost:9901/clusters5736lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5737lb: must succeed: curl -sf localhost:9901/clusters5738lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5739lb: must succeed: curl -sf localhost:9901/clusters5740lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5741worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5742worker1 # [ 92.118092] systemd-journald[286]: Received client request to rotate journal, rotating.5743worker1 # [ 92.139674] systemd-journald[286]: Vacuuming done, freed 0B of archived journals from /var/log/journal/09e51b09083c49d5a4cb601044ebcaf3.5744worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.07 seconds)5745worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service5746worker2 # [ 92.028409] systemd[1]: Stopping Nix worker-protocol over gRPC...5747worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)5748??? Warning (UserWarning): sleep(): The 'secs' argument is deprecated. Use 'duration' instead.5749 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 5735750client: must succeed: sleep 5.05751worker2 # [ 92.189255] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:54Z level=info event=draining5752client: (finished: must succeed: sleep 5.0, in 5.04 seconds)5753worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5754worker2 # [ 97.450756] nix-daemon[845]: reaped child process 1469, status = succeeded5755worker2 # [ 97.586157] nix-daemon[845]: accepted connection from pid 1509, user nix-grpc-daemon (trusted)5756worker2 # [ 97.597609] nix-daemon[845]: reaped child process 1512, status = succeeded5757lb # [ 97.615879] niks3-server[1039]: 2026/09/20 10:53:00 INFO Received uploads request method=POST path=/api/pending_closures5758worker2 # [ 97.620817] nix-grpc-daemon[1302]: time=2026-09-20T10:52:59.911Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5759worker2 # [ 97.622262] nix-grpc-daemon[1302]: time=2026-09-20T10:52:59.913Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"5760lb # [ 97.656322] niks3-server[1039]: 2026/09/20 10:53:00 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5761lb # [ 97.663456] niks3-server[1039]: 2026/09/20 10:53:00 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5762lb # [ 97.675367] niks3-server[1039]: 2026/09/20 10:53:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5763lb # [ 97.680800] niks3-server[1039]: 2026/09/20 10:53:00 INFO Signed narinfos id=34 count=15764worker2 # [ 97.676175] nix-grpc-daemon[1302]: time=2026-09-20T10:52:59.967Z level=INFO msg="Uploading 1 narinfos"5765lb # [ 97.684953] niks3-server[1039]: 2026/09/20 10:53:00 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls5766lb # [ 97.695729] niks3-server[1039]: 2026/09/20 10:53:00 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5767lb # [ 97.700714] niks3-server[1039]: 2026/09/20 10:53:00 INFO Completed upload id=345768lb # [ 97.702793] niks3-server[1039]: 2026/09/20 10:53:00 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo5769worker2 # [ 97.697245] nix-grpc-daemon[1302]: time=2026-09-20T10:52:59.988Z level=INFO msg="Upload complete. (253ms)"5770worker2 # [ 97.704830] nix-grpc-daemon[1246]: ts=2026-09-20T10:52:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56554 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=15771worker2 # [ 97.717683] nix-daemon[845]: reaped child process 1467, status = succeeded5772client # [ 97.747719] systemd[1]: rs.service: Deactivated successfully.5773client # [ 97.749517] systemd[1]: rs.service: Consumed 107ms CPU time over 8.494s wall clock time, 37.1M memory peak, 8.5K incoming IP traffic, 9K outgoing IP traffic.5774lb # [ 98.002582] envoy[910]: conn peer=[2001:db8:1::1]:59314 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9560 tx=3804 ms=50465775lb # [ 98.006902] envoy[910]: conn peer=[2001:db8:1::1]:59326 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2947 tx=1351 ms=82675776worker2 # [ 98.190462] nix-grpc-daemon[1246]: ts=2026-09-20T10:53:00Z level=info event=signal_exit5777worker2 # [ 98.416899] nix-grpc-daemon[1246]: ts=2026-09-20T10:53:00Z level=info event=scheduler_yield5778worker1 # [ 98.445256] nix-grpc-daemon[1105]: ts=2026-09-20T10:53:01Z level=info event=scheduler_restarting addr=lb:500515779lb # [ 98.424717] niks3-server[1039]: 2026/09/20 10:53:01 INFO lead: released remote=192.168.1.4:335125780worker2 # [ 98.420925] nix-grpc-daemon[1246]: ts=2026-09-20T10:53:00Z level=info event=scheduler_restarting addr=lb:500515781worker2 # [ 98.425653] nix-grpc-daemon[1246]: ts=2026-09-20T10:53:00Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:565705782worker2 # [ 98.428951] nix-grpc-daemon[1246]: ts=2026-09-20T10:53:00Z level=info event=worker_gone addr=192.168.1.3:500515783worker2 # [ 98.431931] nix-grpc-daemon[1246]: ts=2026-09-20T10:53:00Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:565705784worker2 # [ 98.435218] nix-grpc-daemon[1246]: ts=2026-09-20T10:53:00Z level=info event=worker_gone addr=192.168.1.4:500515785worker1 # [ 98.645791] nix-grpc-daemon[1105]: ts=2026-09-20T10:53:01Z level=info event=scheduler_connected addr=lb:500515786worker2 # [ 98.621834] nix-grpc-daemon[1246]: ts=2026-09-20T10:53:00Z level=info event=scheduler_connected addr=lb:500515787worker1 # [ 98.650091] nix-grpc-daemon[1105]: ts=2026-09-20T10:53:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5788worker2 # [ 98.625810] nix-grpc-daemon[1246]: ts=2026-09-20T10:53:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5789worker1 # [ 98.850273] nix-grpc-daemon[1105]: ts=2026-09-20T10:53:01Z level=info event=scheduler_connected addr=lb:500515790worker1 # [ 98.852129] nix-grpc-daemon[1105]: ts=2026-09-20T10:53:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5791worker2 # [ 98.827966] nix-grpc-daemon[1246]: ts=2026-09-20T10:53:01Z level=info event=scheduler_connected addr=lb:500515792worker2 # [ 98.831579] nix-grpc-daemon[1246]: ts=2026-09-20T10:53:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5793worker1 # [ 98.894441] nix-grpc-daemon[1105]: ts=2026-09-20T10:53:02Z level=info event=scheduler_take_over5794lb # [ 98.873712] niks3-server[1039]: 2026/09/20 10:53:01 INFO lead: acquired remote=192.168.1.3:381445795worker1 # [ 99.052505] nix-grpc-daemon[1105]: ts=2026-09-20T10:53:02Z level=info event=scheduler_connected addr=lb:500515796worker1 # [ 99.054330] nix-grpc-daemon[1105]: ts=2026-09-20T10:53:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5797worker2 # [ 99.029130] nix-grpc-daemon[1246]: ts=2026-09-20T10:53:01Z level=info event=scheduler_connected addr=lb:500515798worker2 # [ 99.032784] nix-grpc-daemon[1246]: ts=2026-09-20T10:53:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5799worker1 # [ 99.254818] nix-grpc-daemon[1105]: ts=2026-09-20T10:53:02Z level=info event=scheduler_connected addr=lb:500515800worker1 # [ 99.256873] nix-grpc-daemon[1105]: ts=2026-09-20T10:53:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5801worker2 # [ 99.230061] nix-grpc-daemon[1246]: ts=2026-09-20T10:53:01Z level=info event=scheduler_connected addr=lb:500515802worker2 # [ 99.234718] nix-grpc-daemon[1246]: ts=2026-09-20T10:53:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5803worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.14 seconds)5804client: waiting for success: ! systemctl is-active rs5805worker2 # [ 99.418191] nix-grpc-daemon[1246]: ts=2026-09-20T10:53:01Z level=info event=scheduler_restarting5806worker1 # [ 99.458420] nix-grpc-daemon[1105]: ts=2026-09-20T10:53:02Z level=info event=scheduler_connected addr=lb:500515807worker1 # [ 99.462189] nix-grpc-daemon[1105]: ts=2026-09-20T10:53:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5808worker2 # [ 99.449126] nix-daemon[845]: reaped child process 1295, status = succeeded5809worker2 # [ 99.451499] nix-grpc-daemon[1246]: ts=2026-09-20T10:53:01Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5810client: (finished: waiting for success: ! systemctl is-active rs, in 0.05 seconds)5811client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5812client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.04 seconds)5813client: must fail: journalctl -u rs | grep -q 'reconnecting'5814client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.02 seconds)5815lb: must succeed: curl -sf localhost:9901/clusters5816lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5817worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5818worker1 # [ 99.662925] nix-grpc-daemon[1105]: ts=2026-09-20T10:53:02Z level=info event=scheduler_connected addr=lb:500515819worker1 # [ 99.667132] nix-grpc-daemon[1105]: ts=2026-09-20T10:53:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5820worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5821worker1 # [ 99.867374] nix-grpc-daemon[1105]: ts=2026-09-20T10:53:03Z level=info event=scheduler_connected addr=lb:500515822worker1 # [ 99.869493] nix-grpc-daemon[1105]: ts=2026-09-20T10:53:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5823worker1 # [ 100.069850] nix-grpc-daemon[1105]: ts=2026-09-20T10:53:03Z level=info event=scheduler_connected addr=lb:500515824worker1 # [ 100.074258] nix-grpc-daemon[1105]: ts=2026-09-20T10:53:03Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05825worker2 # [ 100.243841] nix-grpc-daemon[1246]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5826worker2 # [ 100.247249] nix-grpc-daemon[1246]: E0000 00:00:1789901582.533875 1289 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5827worker2 # [ 100.252008] nix-grpc-daemon[1246]: E0000 00:00:1789901582.534243 1289 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5828lb: must succeed: curl -sf localhost:9901/clusters5829lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5830worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5831worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5832worker2 # [ 101.062540] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5833worker2 # [ 101.066243] systemd[1]: Stopped Nix worker-protocol over gRPC.5834worker2 # [ 101.069774] systemd[1]: nix-grpc-daemon.service: Consumed 1.350s CPU time over 29.066s wall clock time, 35.1M memory peak, 60.3K incoming IP traffic, 129.9K outgoing IP traffic.5835worker2 # [ 101.085561] systemd[1]: Starting Nix worker-protocol over gRPC...5836worker2 # [ 101.261294] nix-grpc-daemon[1519]: ts=2026-09-20T10:53:03Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5837worker2 # [ 101.264846] nix-grpc-daemon[1519]: ts=2026-09-20T10:53:03Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15838worker2 # [ 101.272066] nix-grpc-daemon[1519]: ts=2026-09-20T10:53:03Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5839worker2 # [ 101.275528] systemd[1]: Started Nix worker-protocol over gRPC.5840worker2 # [ 101.296696] nix-grpc-daemon[1519]: ts=2026-09-20T10:53:03Z level=info event=scheduler_connected addr=lb:500515841worker1 # [ 101.325348] nix-grpc-daemon[1105]: ts=2026-09-20T10:53:04Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05842lb: must succeed: curl -sf localhost:9901/clusters5843lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5844worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5845worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5846(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.71 seconds)5847subtest: scheduler restart mid-build: build finishes, no double build5848client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid5849client # [ 101.942515] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid.5850client # Running as unit: mid.service; invocation ID: 584b8154180e4ea188306b356a5d26cd5851client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid, in 0.04 seconds)5852worker2 # [ 102.114831] nix-grpc-daemon[1519]: ts=2026-09-20T10:53:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:487265853worker2 # [ 102.126114] nix-daemon[845]: accepted connection from pid 1519, user nix-grpc-daemon (trusted)5854client # [ 102.151609] nix[1211]: don't know how to build these paths:5855client # [ 102.152592] nix[1211]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5856worker1 # [ 102.171681] nix-grpc-daemon[1105]: ts=2026-09-20T10:53:05Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05857worker1 # [ 102.176960] nix-grpc-daemon[1105]: ts=2026-09-20T10:53:05Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=15858worker1 # [ 102.216536] nix-grpc-daemon[1105]: ts=2026-09-20T10:53:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:450005859worker1 # [ 102.228450] nix-grpc-daemon[1105]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 7 ms (attempt 1/5)5860worker1 # [ 102.241148] nix-grpc-daemon[1105]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 81 ms (attempt 2/5)5861worker1 # [ 102.328646] nix-grpc-daemon[1105]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 216 ms (attempt 3/5)5862worker1 # [ 102.550278] nix-grpc-daemon[1105]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 274 ms (attempt 4/5)5863worker1 # [ 102.825460] nix-grpc-daemon[1105]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5864worker1 # [ 102.842373] nix-grpc-daemon[1105]: ts=2026-09-20T10:53:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:450005865client # [ 102.839900] nix[1211]: copying 1 paths to 'lb:50051'...5866client # [ 102.842940] nix[1211]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...5867worker1 # [ 102.856130] nix-grpc-daemon[1105]: ts=2026-09-20T10:53:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:450005868worker1 # [ 102.867462] nix-daemon[840]: accepted connection from pid 1105, user nix-grpc-daemon (trusted)5869worker1 # [ 102.885737] nix-grpc-daemon[1105]: ts=2026-09-20T10:53:06Z level=info event=niks3_push_started pid=12265870worker1 # [ 103.255341] nix-daemon[840]: accepted connection from pid 1243, user nix-grpc-daemon (trusted)5871worker1 # [ 103.266630] nix-daemon[840]: reaped child process 1246, status = succeeded5872worker1 # [ 103.436772] nix-daemon[840]: accepted connection from pid 1248, user nix-grpc-daemon (trusted)5873worker1 # [ 103.448269] nix-daemon[840]: reaped child process 1251, status = succeeded5874lb # [ 103.444658] niks3-server[1039]: 2026/09/20 10:53:06 INFO Received uploads request method=POST path=/api/pending_closures5875worker1 # [ 103.476033] nix-grpc-daemon[1226]: time=2026-09-20T10:53:06.621Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5876worker1 # [ 103.479088] nix-grpc-daemon[1226]: time=2026-09-20T10:53:06.623Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"5877lb # [ 103.485959] niks3-server[1039]: 2026/09/20 10:53:06 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst5878lb # [ 103.491262] niks3-server[1039]: 2026/09/20 10:53:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5879worker1 # [ 103.518525] nix-grpc-daemon[1226]: time=2026-09-20T10:53:06.663Z level=INFO msg="Uploading 1 narinfos"5880lb # [ 103.497101] niks3-server[1039]: 2026/09/20 10:53:06 INFO Signed narinfos id=35 count=15881lb # [ 103.510603] niks3-server[1039]: 2026/09/20 10:53:06 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5882worker1 # [ 103.539404] nix-grpc-daemon[1226]: time=2026-09-20T10:53:06.685Z level=INFO msg="Upload complete. (446ms)"5883lb # [ 103.518848] niks3-server[1039]: 2026/09/20 10:53:06 INFO Completed upload id=355884worker1 # [ 103.542846] nix-grpc-daemon[1105]: ts=2026-09-20T10:53:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45000 duration_s=0 paths=1 nar_bytes_in=4245885worker1 # [ 103.548260] nix-grpc-daemon[1105]: ts=2026-09-20T10:53:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:450005886worker1 # [ 103.555220] nix-daemon[840]: accepted connection from pid 1105, user nix-grpc-daemon (trusted)5887worker1 # [ 103.556410] nix-grpc-daemon[1226]: time=2026-09-20T10:53:06.700Z level=WARN msg="Failed to register uploaded object" key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo error="server returned 404: object is not pending upload\n"5888lb # [ 103.537953] niks3-server[1039]: 2026/09/20 10:53:06 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls5889worker1 # [ 103.561258] nix-daemon[840]: reaped child process 1224, status = succeeded5890worker1 # [ 103.567058] nix-daemon[840]: accepted connection from pid 1105, user nix-grpc-daemon (trusted)5891client # [ 103.554476] nix[1211]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5892lb: must succeed: curl -sf localhost:9901/clusters5893lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5894lb: must succeed: curl -sf localhost:9901/clusters5895lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5896worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5897worker1 # [ 105.722292] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1105 (nix-grpc-daemon) on client request.5898lb # [ 105.703372] niks3-server[1039]: 2026/09/20 10:53:08 INFO lead: released remote=192.168.1.3:381445899worker1 # [ 105.724172] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5900client # [ 105.717742] nix[1211]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5901client # [ 105.719708] nix[1211]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5902worker2 # [ 105.703741] nix-grpc-daemon[1519]: ts=2026-09-20T10:53:07Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5903worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.06 seconds)5904client: must succeed: sleep 5.05905worker1 # [ 105.749389] nix-daemon[840]: reaped child process 1253, status = succeeded5906worker1 # [ 105.750523] nix-daemon[840]: reaped child process 1161, status = succeeded5907worker1 # [ 105.756072] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5908worker1 # [ 105.757558] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5909worker1 # [ 105.759158] systemd[1]: nix-grpc-daemon.service: Consumed 409ms CPU time over 17.052s wall clock time, 32.8M memory peak, 15.3K incoming IP traffic, 24.6K outgoing IP traffic.5910worker1 # [ 105.769211] nix-daemon[1255]: unexpected Nix daemon error: error: interrupted by the user5911worker1 # [ 105.773692] nix-daemon[840]: reaped child process 1255, status = failed with exit code 15912worker2 # [ 105.904160] nix-grpc-daemon[1519]: ts=2026-09-20T10:53:08Z level=info event=scheduler_connected addr=lb:500515913worker1 # [ 105.934401] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.5914worker1 # [ 105.943168] systemd[1]: Starting Nix worker-protocol over gRPC...5915worker1 # [ 106.129557] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:09Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5916worker1 # [ 106.132767] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:09Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15917worker1 # [ 106.139630] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:09Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5918worker1 # [ 106.143149] systemd[1]: Started Nix worker-protocol over gRPC.5919worker1 # [ 106.153045] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:09Z level=info event=scheduler_take_over5920lb # [ 106.132137] niks3-server[1039]: 2026/09/20 10:53:09 INFO lead: acquired remote=192.168.1.3:428325921worker1 # [ 106.156934] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:09Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05922worker1 # [ 106.174919] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:09Z level=info event=scheduler_connected addr=lb:500515923worker1 # [ 106.176437] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:09Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05924worker1 # [ 106.241827] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:09Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05925worker1 # [ 106.245926] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:09Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=15926worker2 # [ 106.259413] nix-grpc-daemon[1519]: ts=2026-09-20T10:53:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487265927worker2 # [ 106.269814] nix-grpc-daemon[1519]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 70 ms (attempt 1/5)5928worker2 # [ 106.345843] nix-grpc-daemon[1519]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 167 ms (attempt 2/5)5929worker2 # [ 106.518795] nix-grpc-daemon[1519]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 312 ms (attempt 3/5)5930worker2 # [ 106.833855] nix-grpc-daemon[1519]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 87 ms (attempt 4/5)5931worker2 # [ 106.923619] nix-grpc-daemon[1519]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5932worker2 # [ 106.934279] nix-grpc-daemon[1519]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' from 'http://lb:5751'...5933worker2 # [ 106.949550] nix-daemon[845]: accepted connection from pid 1519, user nix-grpc-daemon (trusted)5934worker2 # [ 106.955191] nix-daemon[845]: accepted connection from pid 1519, user nix-grpc-daemon (trusted)5935client # [ 106.970363] nix[1211]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5936lb # [ 108.002499] envoy[910]: conn peer=[2001:db8:1::4]:41476 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=9588 tx=2016 ms=272295937lb # [ 108.006994] envoy[910]: conn peer=[2001:db8:1::3]:43410 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2187 tx=590 ms=168205938client: (finished: must succeed: sleep 5.0, in 5.03 seconds)5939client: waiting for success: ! systemctl is-active mid5940worker2 # [ 111.170293] nix-grpc-daemon[1519]: ts=2026-09-20T10:53:13Z level=info event=niks3_push_started pid=16205941worker2 # [ 111.199087] nix-daemon[845]: reaped child process 1596, status = succeeded5942worker2 # [ 111.514295] nix-daemon[845]: accepted connection from pid 1633, user nix-grpc-daemon (trusted)5943worker2 # [ 111.526091] nix-daemon[845]: reaped child process 1636, status = succeeded5944lb # [ 111.548631] niks3-server[1039]: 2026/09/20 10:53:14 INFO Received uploads request method=POST path=/api/pending_closures5945worker2 # [ 111.554968] nix-grpc-daemon[1620]: time=2026-09-20T10:53:13.845Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5946worker2 # [ 111.558057] nix-grpc-daemon[1620]: time=2026-09-20T10:53:13.847Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"5947lb # [ 111.600868] niks3-server[1039]: 2026/09/20 10:53:14 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5948lb # [ 111.606095] niks3-server[1039]: 2026/09/20 10:53:14 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5949lb # [ 111.613472] niks3-server[1039]: 2026/09/20 10:53:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5950lb # [ 111.619127] niks3-server[1039]: 2026/09/20 10:53:14 INFO Signed narinfos id=36 count=15951worker2 # [ 111.614916] nix-grpc-daemon[1620]: time=2026-09-20T10:53:13.905Z level=INFO msg="Uploading 1 narinfos"5952lb # [ 111.623957] niks3-server[1039]: 2026/09/20 10:53:14 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls5953lb # [ 111.631290] niks3-server[1039]: 2026/09/20 10:53:14 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5954lb # [ 111.633430] niks3-server[1039]: 2026/09/20 10:53:14 INFO Completed upload id=365955worker2 # [ 111.628846] nix-grpc-daemon[1620]: time=2026-09-20T10:53:13.919Z level=INFO msg="Upload complete. (265ms)"5956lb # [ 111.636367] niks3-server[1039]: 2026/09/20 10:53:14 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo5957worker2 # [ 111.634112] nix-grpc-daemon[1519]: ts=2026-09-20T10:53:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48726 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=15958worker2 # [ 111.645903] nix-daemon[845]: reaped child process 1594, status = succeeded5959client # [ 111.682752] systemd[1]: mid.service: Deactivated successfully.5960client # [ 111.684538] systemd[1]: mid.service: Consumed 108ms CPU time over 9.741s wall clock time, 37.3M memory peak, 9.8K incoming IP traffic, 10.9K outgoing IP traffic.5961client: (finished: waiting for success: ! systemctl is-active mid, in 1.09 seconds)5962client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5963client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.04 seconds)5964(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.22 seconds)5965subtest: interrupting the client stops the build on the worker5966client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr5967client # [ 112.155895] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr.5968client # Running as unit: intr.service; invocation ID: f6b88a7ca6204907a9c343e1f2aef1905969client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr, in 0.04 seconds)5970client # [ 112.351749] nix[1247]: don't know how to build these paths:5971client # [ 112.352650] nix[1247]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv5972worker2 # [ 112.336981] nix-grpc-daemon[1519]: ts=2026-09-20T10:53:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:487265973worker1 # [ 112.371075] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:15Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=05974worker1 # [ 112.376169] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:15Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.3:50051 clients=15975worker1 # [ 112.420128] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564325976worker1 # [ 112.430709] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)5977worker1 # [ 112.447233] nix-grpc-daemon[1291]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 4 ms (attempt 1/5)5978worker1 # [ 112.453901] nix-grpc-daemon[1291]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 156 ms (attempt 2/5)5979worker1 # [ 112.614278] nix-grpc-daemon[1291]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 30 ms (attempt 3/5)5980worker1 # [ 112.644475] nix-grpc-daemon[1291]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 572 ms (attempt 4/5)5981worker1 # [ 113.222592] nix-grpc-daemon[1291]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5982worker1 # [ 113.239282] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564325983client # [ 113.237456] nix[1247]: copying 1 paths to 'lb:50051'...5984client # [ 113.240977] nix[1247]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...5985worker1 # [ 113.253296] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:564325986worker1 # [ 113.263160] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)5987worker1 # [ 113.280853] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:16Z level=info event=niks3_push_started pid=13635988worker1 # [ 113.589586] nix-daemon[840]: accepted connection from pid 1380, user nix-grpc-daemon (trusted)5989worker1 # [ 113.601080] nix-daemon[840]: reaped child process 1383, status = succeeded5990worker1 # [ 113.773885] nix-daemon[840]: accepted connection from pid 1385, user nix-grpc-daemon (trusted)5991worker1 # [ 113.785453] nix-daemon[840]: reaped child process 1388, status = succeeded5992lb # [ 113.775886] niks3-server[1039]: 2026/09/20 10:53:16 INFO Received uploads request method=POST path=/api/pending_closures5993worker1 # [ 113.802877] nix-grpc-daemon[1363]: time=2026-09-20T10:53:16.947Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5994worker1 # [ 113.804446] nix-grpc-daemon[1363]: time=2026-09-20T10:53:16.947Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"5995lb # [ 113.794682] niks3-server[1039]: 2026/09/20 10:53:16 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst5996lb # [ 113.799564] niks3-server[1039]: 2026/09/20 10:53:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5997lb # [ 113.803047] niks3-server[1039]: 2026/09/20 10:53:16 INFO Signed narinfos id=37 count=15998worker1 # [ 113.824314] nix-grpc-daemon[1363]: time=2026-09-20T10:53:16.969Z level=INFO msg="Uploading 1 narinfos"5999lb # [ 113.810396] niks3-server[1039]: 2026/09/20 10:53:16 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete6000lb # [ 113.814525] niks3-server[1039]: 2026/09/20 10:53:16 INFO Completed upload id=376001worker1 # [ 113.835574] nix-grpc-daemon[1363]: time=2026-09-20T10:53:16.981Z level=INFO msg="Upload complete. (388ms)"6002lb # [ 113.815687] niks3-server[1039]: 2026/09/20 10:53:16 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6003worker1 # [ 113.837787] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56432 duration_s=0 paths=1 nar_bytes_in=4246004worker1 # [ 113.841796] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564326005worker1 # [ 113.843840] nix-grpc-daemon[1363]: time=2026-09-20T10:53:16.989Z level=WARN msg="Failed to register uploaded object" key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo error="server returned 404: object is not pending upload\n"6006worker1 # [ 113.850700] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6007worker1 # [ 113.851970] nix-daemon[840]: reaped child process 1361, status = succeeded6008worker1 # [ 113.856883] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6009client # [ 113.844923] nix[1247]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6010client: must succeed: systemctl kill -s INT intr6011client # [ 115.630573] systemd[1]: intr.service: Sent signal SIGINT to main process 1247 (nix) on client request.6012client # [ 115.636780] nix[1247]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6013client: (finished: must succeed: systemctl kill -s INT intr, in 0.05 seconds)6014client # [ 115.643852] nix[1247]: error: interrupted by the user6015worker1 # [ 115.678216] nix-daemon[840]: reaped child process 1390, status = succeeded6016worker1 # [ 115.680160] nix-daemon[1392]: unexpected Nix daemon error: error: interrupted by the user6017worker1 # [ 115.685613] nix-daemon[840]: reaped child process 1392, status = failed with exit code 16018client # [ 115.673513] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6019client # [ 115.674817] systemd[1]: intr.service: Failed with result 'exit-code'.6020client # [ 115.676727] systemd[1]: intr.service: Consumed 102ms CPU time over 3.517s wall clock time, 39.1M memory peak, 7.7K incoming IP traffic, 8.7K outgoing IP traffic.6021(finished: subtest: interrupting the client stops the build on the worker, in 3.61 seconds)6022subtest: a deploy mid-build drains: the build finishes, the new generation takes over6023client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw6024client # [ 115.779348] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw.6025client # Running as unit: sw.service; invocation ID: 5f96b36f45ba474190663d87156d1bf36026client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw, in 0.05 seconds)6027worker1 # [ 115.980655] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:564326028client # [ 115.974829] nix[1267]: don't know how to build these paths:6029client # [ 115.975749] nix[1267]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6030worker1 # [ 115.996105] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:19Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06031worker1 # [ 116.000240] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:19Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.4:50051 clients=16032worker2 # [ 116.015759] nix-grpc-daemon[1519]: ts=2026-09-20T10:53:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487266033worker2 # [ 116.029086] nix-grpc-daemon[1519]: ts=2026-09-20T10:53:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:487266034client # [ 116.053231] nix[1267]: copying 1 paths to 'lb:50051'...6035worker2 # [ 116.040721] nix-grpc-daemon[1519]: ts=2026-09-20T10:53:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:487266036client # [ 116.055443] nix[1267]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6037worker2 # [ 116.053289] nix-daemon[845]: accepted connection from pid 1519, user nix-grpc-daemon (trusted)6038worker2 # [ 116.230182] nix-daemon[845]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6039worker2 # [ 116.241892] nix-daemon[845]: reaped child process 1667, status = succeeded6040worker2 # [ 116.406806] nix-daemon[845]: accepted connection from pid 1669, user nix-grpc-daemon (trusted)6041worker2 # [ 116.418193] nix-daemon[845]: reaped child process 1672, status = succeeded6042lb # [ 116.440870] niks3-server[1039]: 2026/09/20 10:53:19 INFO Received uploads request method=POST path=/api/pending_closures6043worker2 # [ 116.447501] nix-grpc-daemon[1620]: time=2026-09-20T10:53:18.738Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6044worker2 # [ 116.449042] nix-grpc-daemon[1620]: time=2026-09-20T10:53:18.740Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6045lb # [ 116.481827] niks3-server[1039]: 2026/09/20 10:53:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6046lb # [ 116.490892] niks3-server[1039]: 2026/09/20 10:53:19 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6047lb # [ 116.498344] niks3-server[1039]: 2026/09/20 10:53:19 INFO Signed narinfos id=38 count=16048worker2 # [ 116.496232] nix-grpc-daemon[1620]: time=2026-09-20T10:53:18.787Z level=INFO msg="Uploading 1 narinfos"6049lb # [ 116.501992] niks3-server[1039]: 2026/09/20 10:53:19 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6050lb # [ 116.513196] niks3-server[1039]: 2026/09/20 10:53:19 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6051lb # [ 116.521803] niks3-server[1039]: 2026/09/20 10:53:19 INFO Completed upload id=386052worker2 # [ 116.517296] nix-grpc-daemon[1620]: time=2026-09-20T10:53:18.808Z level=INFO msg="Upload complete. (452ms)"6053lb # [ 116.524072] niks3-server[1039]: 2026/09/20 10:53:19 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6054worker2 # [ 116.521850] nix-grpc-daemon[1519]: ts=2026-09-20T10:53:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48726 duration_s=0 paths=1 nar_bytes_in=4166055worker2 # [ 116.528364] nix-grpc-daemon[1519]: ts=2026-09-20T10:53:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487266056worker2 # [ 116.538126] nix-daemon[845]: accepted connection from pid 1519, user nix-grpc-daemon (trusted)6057worker2 # [ 116.547581] nix-daemon[845]: reaped child process 1662, status = succeeded6058worker2 # [ 116.550686] nix-daemon[845]: accepted connection from pid 1519, user nix-grpc-daemon (trusted)6059client # [ 116.563778] nix[1267]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6060lb # [ 118.003068] envoy[910]: conn peer=[2001:db8:1::1]:41004 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3852 tx=1677 ms=95436061lb # [ 118.006904] envoy[910]: conn peer=[2001:db8:1::1]:50008 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2924 tx=753 ms=33096062worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&26063worker2 # Checking switch inhibitors... done6064worker2 # [ 118.910381] nixos[1707]: switching to system configuration /nix/store/plyqnl92wps9ak4cwrw9g13lkn5mvgaq-nixos-system-worker2-test6065worker2 # [ 118.912443] systemd[1]: Stopped target Remote File Systems.6066worker2 # activating the configuration...6067worker2 # [ 118.914818] systemd[1]: Stopped target Local File Systems.6068worker2 # setting up /etc...6069worker2 # [ 119.499519] systemd[1]: Reload requested from client PID 1707 ('.switch-to-conf') (unit backdoor.service)...6070worker2 # [ 119.501102] systemd[1]: Reloading...6071worker2 # [ 120.222901] systemd[1]: Reloading finished in 719 ms.6072worker2 # restarting sysinit-reactivation.target6073worker2 # [ 120.276446] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....6074worker2 # [ 120.400511] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..6075worker2 # [ 120.404143] systemd[1]: Reached target Reactivate sysinit units.6076worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service6077worker2 # [ 120.409195] systemd[1]: Reloading D-Bus System Message Bus...6078worker2 # [ 120.412112] dbus-broker-launch[533]: Caught SIGHUP, trigger reload.6079worker2 # [ 120.414571] systemd[1]: Reloading Nix worker-protocol over gRPC...6080worker2 # [ 120.442450] dbus-broker-launch[533]: Looking up NSS user entry for 'systemd-timesync'...6081worker2 # [ 120.444932] dbus-broker-launch[533]: NSS returned no entry for 'systemd-timesync'6082worker2 # [ 120.445954] dbus-broker-launch[533]: Invalid user-name in /nix/store/nl02z4vnl25dc1df60lqifjnw6c89zpg-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"6083worker2 # [ 120.454742] systemd[1]: Reloaded Nix worker-protocol over gRPC.6084worker2 # [ 120.471308] systemd[1]: Reloaded D-Bus System Message Bus.6085worker2 # [ 120.478208] systemd[1]: Reached target Local File Systems.6086worker2 # [ 120.479043] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore6087worker2 # [ 120.485468] systemd[1]: Starting Create SUID/SGID Wrappers...6088worker2 # [ 120.486284] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.6089worker2 # [ 120.487445] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc676090worker2 # [ 120.489440] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore6091worker2 # [ 120.490850] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6092worker2 # [ 120.492857] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6093worker2 # [ 120.495637] systemd[1]: Reached target Remote File Systems.6094worker2 # [ 120.505170] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os6095worker2 # [ 120.506458] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki6096worker2 # [ 120.911125] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.6097worker2 # [ 120.912256] systemd[1]: Finished Create SUID/SGID Wrappers.6098worker2 # [ 120.915632] systemd[1]: Starting linger-users.service...6099worker2 # [ 120.959456] systemd[1]: linger-users.service: Deactivated successfully.6100worker2 # [ 120.963485] systemd[1]: Finished linger-users.service.6101worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service6102worker2 # [ 121.217792] nixos[1707]: finished switching to system configuration /nix/store/plyqnl92wps9ak4cwrw9g13lkn5mvgaq-nixos-system-worker2-test6103worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 3.00 seconds)6104worker2: must succeed: pgrep -f 'read -t [9]0 x'6105worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.04 seconds)6106client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&26107worker2 # [ 121.280211] nix-grpc-daemon[1519]: ts=2026-09-20T10:53:23Z level=info event=draining6108worker1 # [ 121.477227] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:564326109client # don't know how to build these paths:6110client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6111worker1 # [ 121.485319] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564326112worker1 # [ 121.506258] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:24Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06113worker1 # [ 121.511787] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:24Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.3:50051 clients=16114worker1 # [ 121.552925] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564326115worker1 # [ 121.564295] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564326116client # copying 1 paths to 'lb:50051'...6117worker1 # [ 121.575217] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:564326118client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6119worker1 # [ 121.587201] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6120worker1 # [ 121.764191] nix-daemon[840]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6121worker1 # [ 121.776278] nix-daemon[840]: reaped child process 1445, status = succeeded6122worker1 # [ 121.944107] nix-daemon[840]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)6123worker1 # [ 121.955929] nix-daemon[840]: reaped child process 1450, status = succeeded6124lb # [ 121.952873] niks3-server[1039]: 2026/09/20 10:53:24 INFO Received uploads request method=POST path=/api/pending_closures6125worker1 # [ 122.002889] nix-grpc-daemon[1363]: time=2026-09-20T10:53:25.148Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6126worker1 # [ 122.004321] nix-grpc-daemon[1363]: time=2026-09-20T10:53:25.148Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6127lb # [ 122.003128] niks3-server[1039]: 2026/09/20 10:53:24 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6128lb # [ 122.014689] niks3-server[1039]: 2026/09/20 10:53:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6129lb # [ 122.020505] niks3-server[1039]: 2026/09/20 10:53:24 INFO Signed narinfos id=39 count=16130worker1 # [ 122.045256] nix-grpc-daemon[1363]: time=2026-09-20T10:53:25.190Z level=INFO msg="Uploading 1 narinfos"6131lb # [ 122.032743] niks3-server[1039]: 2026/09/20 10:53:24 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6132lb # [ 122.043303] niks3-server[1039]: 2026/09/20 10:53:24 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6133worker1 # [ 122.071724] nix-grpc-daemon[1363]: time=2026-09-20T10:53:25.217Z level=INFO msg="Upload complete. (473ms)"6134lb # [ 122.050878] niks3-server[1039]: 2026/09/20 10:53:24 INFO Completed upload id=396135lb # [ 122.052565] niks3-server[1039]: 2026/09/20 10:53:24 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6136worker1 # [ 122.074070] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56432 duration_s=0 paths=1 nar_bytes_in=5286137worker1 # [ 122.077138] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564326138worker1 # [ 122.083549] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6139worker1 # [ 122.085747] nix-daemon[840]: reaped child process 1440, status = succeeded6140client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6141client # farm-a> node-a: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6142worker1 # [ 122.090879] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6143client # farm-a> LOG-farm-a-during-drain6144worker1 # [ 123.115408] nix-daemon[840]: reaped child process 1454, status = succeeded6145worker1 # [ 123.274308] nix-daemon[840]: accepted connection from pid 1469, user nix-grpc-daemon (trusted)6146worker1 # [ 123.286531] nix-daemon[840]: reaped child process 1472, status = succeeded6147lb # [ 123.281628] niks3-server[1039]: 2026/09/20 10:53:26 INFO Received uploads request method=POST path=/api/pending_closures6148worker1 # [ 123.310499] nix-grpc-daemon[1363]: time=2026-09-20T10:53:26.455Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6149worker1 # [ 123.312050] nix-grpc-daemon[1363]: time=2026-09-20T10:53:26.457Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6150lb # [ 123.320367] niks3-server[1039]: 2026/09/20 10:53:26 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6151lb # [ 123.327758] niks3-server[1039]: 2026/09/20 10:53:26 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6152lb # [ 123.337635] niks3-server[1039]: 2026/09/20 10:53:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6153lb # [ 123.342957] niks3-server[1039]: 2026/09/20 10:53:26 INFO Signed narinfos id=40 count=16154worker1 # [ 123.367461] nix-grpc-daemon[1363]: time=2026-09-20T10:53:26.512Z level=INFO msg="Uploading 1 narinfos"6155lb # [ 123.348350] niks3-server[1039]: 2026/09/20 10:53:26 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6156lb # [ 123.358095] niks3-server[1039]: 2026/09/20 10:53:26 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6157lb # [ 123.364883] niks3-server[1039]: 2026/09/20 10:53:26 INFO Completed upload id=406158worker1 # [ 123.385930] nix-grpc-daemon[1363]: time=2026-09-20T10:53:26.531Z level=INFO msg="Upload complete. (287ms)"6159lb # [ 123.366675] niks3-server[1039]: 2026/09/20 10:53:26 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6160worker1 # [ 123.395700] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56432 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16161worker1 # [ 123.403230] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:26Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06162worker1 # [ 123.406319] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:26Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.3:50051 clients=16163worker1 # [ 123.414143] nix-daemon[840]: reaped child process 1452, status = succeeded6164worker1 # [ 123.450819] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564326165worker1 # [ 123.463063] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564326166worker1 # [ 123.475796] nix-grpc-daemon[1363]: time=2026-09-20T10:53:26.621Z level=INFO msg="All 1 paths already cached"6167client # copying 1 paths to 'lb:50051'...6168worker1 # [ 123.483749] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:564326169client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6170worker1 # [ 123.503227] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6171worker1 # [ 123.665785] nix-daemon[840]: accepted connection from pid 1478, user nix-grpc-daemon (trusted)6172worker1 # [ 123.677489] nix-daemon[840]: reaped child process 1481, status = succeeded6173worker1 # [ 123.829459] nix-daemon[840]: accepted connection from pid 1483, user nix-grpc-daemon (trusted)6174worker1 # [ 123.840741] nix-daemon[840]: reaped child process 1486, status = succeeded6175lb # [ 123.834898] niks3-server[1039]: 2026/09/20 10:53:26 INFO Received uploads request method=POST path=/api/pending_closures6176worker1 # [ 123.864221] nix-grpc-daemon[1363]: time=2026-09-20T10:53:27.009Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6177worker1 # [ 123.865990] nix-grpc-daemon[1363]: time=2026-09-20T10:53:27.011Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6178lb # [ 123.862899] niks3-server[1039]: 2026/09/20 10:53:26 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6179lb # [ 123.874703] niks3-server[1039]: 2026/09/20 10:53:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6180worker1 # [ 123.900558] nix-grpc-daemon[1363]: time=2026-09-20T10:53:27.046Z level=INFO msg="Uploading 1 narinfos"6181lb # [ 123.879625] niks3-server[1039]: 2026/09/20 10:53:26 INFO Signed narinfos id=41 count=16182lb # [ 123.886200] niks3-server[1039]: 2026/09/20 10:53:26 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6183lb # [ 123.897433] niks3-server[1039]: 2026/09/20 10:53:26 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6184lb # [ 123.905592] niks3-server[1039]: 2026/09/20 10:53:26 INFO Completed upload id=416185worker1 # [ 123.926928] nix-grpc-daemon[1363]: time=2026-09-20T10:53:27.072Z level=INFO msg="Upload complete. (411ms)"6186lb # [ 123.908958] niks3-server[1039]: 2026/09/20 10:53:26 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6187worker1 # [ 123.930791] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56432 duration_s=0 paths=1 nar_bytes_in=6406188worker1 # [ 123.936071] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564326189worker1 # [ 123.947871] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6190worker1 # [ 123.950252] nix-daemon[840]: reaped child process 1476, status = succeeded6191worker1 # [ 123.955277] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6192client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6193client # farm-b> node-a: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6194client # farm-b> LOG-farm-b-during-drain6195worker1 # [ 124.574134] nix-daemon[840]: reaped child process 1490, status = succeeded6196worker1 # [ 124.700538] nix-daemon[840]: accepted connection from pid 1505, user nix-grpc-daemon (trusted)6197worker1 # [ 124.712111] nix-daemon[840]: reaped child process 1508, status = succeeded6198lb # [ 124.706623] niks3-server[1039]: 2026/09/20 10:53:27 INFO Received uploads request method=POST path=/api/pending_closures6199worker1 # [ 124.736294] nix-grpc-daemon[1363]: time=2026-09-20T10:53:27.881Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6200worker1 # [ 124.737944] nix-grpc-daemon[1363]: time=2026-09-20T10:53:27.883Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6201lb # [ 124.739528] niks3-server[1039]: 2026/09/20 10:53:27 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6202lb # [ 124.755285] niks3-server[1039]: 2026/09/20 10:53:27 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6203lb # [ 124.765839] niks3-server[1039]: 2026/09/20 10:53:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6204lb # [ 124.771121] niks3-server[1039]: 2026/09/20 10:53:27 INFO Signed narinfos id=42 count=16205worker1 # [ 124.792583] nix-grpc-daemon[1363]: time=2026-09-20T10:53:27.937Z level=INFO msg="Uploading 1 narinfos"6206lb # [ 124.776226] niks3-server[1039]: 2026/09/20 10:53:27 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6207lb # [ 124.785349] niks3-server[1039]: 2026/09/20 10:53:27 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6208lb # [ 124.791553] niks3-server[1039]: 2026/09/20 10:53:27 INFO Completed upload id=426209worker1 # [ 124.812841] nix-grpc-daemon[1363]: time=2026-09-20T10:53:27.958Z level=INFO msg="Upload complete. (251ms)"6210lb # [ 124.794134] niks3-server[1039]: 2026/09/20 10:53:27 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6211worker1 # [ 124.821959] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56432 duration_s=0 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16212worker1 # [ 124.831170] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:27Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06213worker1 # [ 124.835223] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:27Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.3:50051 clients=16214worker1 # [ 124.843211] nix-daemon[840]: reaped child process 1488, status = succeeded6215worker1 # [ 124.874878] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564326216worker1 # [ 124.884796] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564326217worker1 # [ 124.897642] nix-grpc-daemon[1363]: time=2026-09-20T10:53:28.043Z level=INFO msg="All 2 paths already cached"6218client # copying 1 paths to 'lb:50051'...6219client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6220worker1 # [ 124.905751] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:564326221worker1 # [ 124.918136] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6222worker1 # [ 125.081652] nix-daemon[840]: accepted connection from pid 1514, user nix-grpc-daemon (trusted)6223worker1 # [ 125.093382] nix-daemon[840]: reaped child process 1517, status = succeeded6224worker1 # [ 125.249554] nix-daemon[840]: accepted connection from pid 1519, user nix-grpc-daemon (trusted)6225worker1 # [ 125.260960] nix-daemon[840]: reaped child process 1522, status = succeeded6226lb # [ 125.255345] niks3-server[1039]: 2026/09/20 10:53:28 INFO Received uploads request method=POST path=/api/pending_closures6227worker1 # [ 125.285828] nix-grpc-daemon[1363]: time=2026-09-20T10:53:28.431Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6228worker1 # [ 125.287304] nix-grpc-daemon[1363]: time=2026-09-20T10:53:28.431Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6229lb # [ 125.284861] niks3-server[1039]: 2026/09/20 10:53:28 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6230lb # [ 125.293256] niks3-server[1039]: 2026/09/20 10:53:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6231lb # [ 125.298162] niks3-server[1039]: 2026/09/20 10:53:28 INFO Signed narinfos id=43 count=16232worker1 # [ 125.323676] nix-grpc-daemon[1363]: time=2026-09-20T10:53:28.469Z level=INFO msg="Uploading 1 narinfos"6233lb # [ 125.307247] niks3-server[1039]: 2026/09/20 10:53:28 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6234lb # [ 125.318400] niks3-server[1039]: 2026/09/20 10:53:28 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6235lb # [ 125.325728] niks3-server[1039]: 2026/09/20 10:53:28 INFO Completed upload id=436236worker1 # [ 125.346914] nix-grpc-daemon[1363]: time=2026-09-20T10:53:28.492Z level=INFO msg="Upload complete. (415ms)"6237lb # [ 125.329388] niks3-server[1039]: 2026/09/20 10:53:28 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6238worker1 # [ 125.351242] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56432 duration_s=0 paths=1 nar_bytes_in=7686239worker1 # [ 125.356504] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564326240worker1 # [ 125.366131] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6241worker1 # [ 125.370259] nix-daemon[840]: reaped child process 1512, status = succeeded6242worker1 # [ 125.375128] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6243client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6244client # farm-top> node-a: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6245client # farm-top> LOG-farm-top-during-drain6246worker1 # [ 126.600840] nix-daemon[840]: reaped child process 1526, status = succeeded6247worker1 # [ 126.745354] nix-daemon[840]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6248worker1 # [ 126.757108] nix-daemon[840]: reaped child process 1544, status = succeeded6249lb # [ 126.752736] niks3-server[1039]: 2026/09/20 10:53:29 INFO Received uploads request method=POST path=/api/pending_closures6250worker1 # [ 126.783075] nix-grpc-daemon[1363]: time=2026-09-20T10:53:29.928Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6251worker1 # [ 126.784667] nix-grpc-daemon[1363]: time=2026-09-20T10:53:29.930Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6252lb # [ 126.784048] niks3-server[1039]: 2026/09/20 10:53:29 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6253lb # [ 126.786605] niks3-server[1039]: 2026/09/20 10:53:29 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6254lb # [ 126.791115] niks3-server[1039]: 2026/09/20 10:53:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6255worker1 # [ 126.816696] nix-grpc-daemon[1363]: time=2026-09-20T10:53:29.962Z level=INFO msg="Uploading 1 narinfos"6256lb # [ 126.795179] niks3-server[1039]: 2026/09/20 10:53:29 INFO Signed narinfos id=44 count=16257lb # [ 126.800656] niks3-server[1039]: 2026/09/20 10:53:29 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6258lb # [ 126.806182] niks3-server[1039]: 2026/09/20 10:53:29 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6259lb # [ 126.808669] niks3-server[1039]: 2026/09/20 10:53:29 INFO Completed upload id=446260worker1 # [ 126.830909] nix-grpc-daemon[1363]: time=2026-09-20T10:53:29.976Z level=INFO msg="Upload complete. (248ms)"6261lb # [ 126.810166] niks3-server[1039]: 2026/09/20 10:53:29 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6262worker1 # [ 126.834640] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56432 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16263worker1 # [ 126.841108] nix-daemon[840]: reaped child process 1524, status = succeeded6264client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&2, in 5.57 seconds)6265client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath6266client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath, in 0.20 seconds)6267worker2: must succeed: pkill -f 'read -t [9]0 x'6268worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.04 seconds)6269client: waiting for success: ! systemctl is-active sw6270worker2 # [ 127.184215] nix-daemon[845]: reaped child process 1676, status = succeeded6271worker2 # [ 127.308671] nix-daemon[845]: accepted connection from pid 1847, user nix-grpc-daemon (trusted)6272worker2 # [ 127.318908] nix-daemon[845]: reaped child process 1850, status = succeeded6273lb # [ 127.337703] niks3-server[1039]: 2026/09/20 10:53:30 INFO Received uploads request method=POST path=/api/pending_closures6274worker2 # [ 127.340541] nix-grpc-daemon[1620]: time=2026-09-20T10:53:29.631Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6275worker2 # [ 127.342358] nix-grpc-daemon[1620]: time=2026-09-20T10:53:29.633Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6276lb # [ 127.372950] niks3-server[1039]: 2026/09/20 10:53:30 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6277lb # [ 127.383941] niks3-server[1039]: 2026/09/20 10:53:30 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6278lb # [ 127.391212] niks3-server[1039]: 2026/09/20 10:53:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6279worker2 # [ 127.391682] nix-grpc-daemon[1620]: time=2026-09-20T10:53:29.682Z level=INFO msg="Uploading 1 narinfos"6280lb # [ 127.396357] niks3-server[1039]: 2026/09/20 10:53:30 INFO Signed narinfos id=45 count=16281lb # [ 127.402432] niks3-server[1039]: 2026/09/20 10:53:30 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6282lb # [ 127.412103] niks3-server[1039]: 2026/09/20 10:53:30 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6283lb # [ 127.418232] niks3-server[1039]: 2026/09/20 10:53:30 INFO Completed upload id=456284worker2 # [ 127.413524] nix-grpc-daemon[1620]: time=2026-09-20T10:53:29.704Z level=INFO msg="Upload complete. (243ms)"6285lb # [ 127.421912] niks3-server[1039]: 2026/09/20 10:53:30 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6286worker2 # [ 127.419161] nix-grpc-daemon[1519]: ts=2026-09-20T10:53:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48726 duration_s=10 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16287worker2 # [ 127.430551] nix-daemon[845]: reaped child process 1674, status = succeeded6288client # [ 127.472486] systemd[1]: sw.service: Deactivated successfully.6289client # [ 127.476191] systemd[1]: sw.service: Consumed 106ms CPU time over 11.695s wall clock time, 39.1M memory peak, 8.4K incoming IP traffic, 8.9K outgoing IP traffic.6290lb # [ 128.003164] envoy[910]: conn peer=[2001:db8:1::1]:54970 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9585 tx=3820 ms=53766291lb # [ 128.007383] envoy[910]: conn peer=[2001:db8:1::1]:54958 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2932 tx=1310 ms=114756292client: (finished: waiting for success: ! systemctl is-active sw, in 1.09 seconds)6293client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6294client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.04 seconds)6295worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6296worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 0.05 seconds)6297lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6298lb: must succeed: curl -sf localhost:9901/clusters6299worker2 # [ 128.281901] nix-grpc-daemon[1519]: ts=2026-09-20T10:53:30Z level=info event=signal_exit6300lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6301lb: must succeed: curl -sf localhost:9901/clusters6302lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6303worker1 # [ 129.538855] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:32Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:564066304worker1 # [ 129.542879] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:32Z level=info event=worker_gone addr=192.168.1.4:500516305worker2 # [ 129.516489] nix-grpc-daemon[1519]: ts=2026-09-20T10:53:31Z level=info event=scheduler_restarting6306worker2 # [ 129.526510] nix-grpc-daemon[1519]: ts=2026-09-20T10:53:31Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6307worker2 # [ 129.545684] nix-daemon[845]: reaped child process 1572, status = succeeded6308worker2 # [ 130.100358] nix-grpc-daemon[1519]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6309worker2 # [ 130.104252] nix-grpc-daemon[1519]: E0000 00:00:1789901612.391377 1562 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true6310worker2 # [ 130.109078] nix-grpc-daemon[1519]: E0000 00:00:1789901612.395338 1562 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6311lb: must succeed: curl -sf localhost:9901/clusters6312lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6313worker2 # [ 131.311894] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6314worker2 # [ 131.313948] systemd[1]: nix-grpc-daemon.service: Consumed 619ms CPU time over 30.227s wall clock time, 32.3M memory peak, 20.8K incoming IP traffic, 27.3K outgoing IP traffic.6315lb: must succeed: curl -sf localhost:9901/clusters6316lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6317worker2 # [ 132.305833] systemd[1]: Starting Nix worker-protocol over gRPC...6318worker2 # [ 132.486598] nix-grpc-daemon[1862]: ts=2026-09-20T10:53:34Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6319worker2 # [ 132.490500] nix-grpc-daemon[1862]: ts=2026-09-20T10:53:34Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16320worker2 # [ 132.496589] nix-grpc-daemon[1862]: ts=2026-09-20T10:53:34Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6321worker2 # [ 132.500305] systemd[1]: Started Nix worker-protocol over gRPC.6322worker2 # [ 132.526273] nix-grpc-daemon[1862]: ts=2026-09-20T10:53:34Z level=info event=scheduler_connected addr=lb:500516323worker1 # [ 132.555073] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:35Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06324lb: must succeed: curl -sf localhost:9901/clusters6325lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6326lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 4.36 seconds)6327lb: must succeed: curl -sf localhost:9901/clusters6328lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6329worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6330worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6331(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 17.01 seconds)6332subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6333client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16334worker2 # [ 132.917983] nix-grpc-daemon[1862]: ts=2026-09-20T10:53:35Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:405146335worker2 # [ 132.923394] nix-daemon[845]: accepted connection from pid 1862, user nix-grpc-daemon (trusted)6336worker2 # [ 132.930179] nix-grpc-daemon[1862]: ts=2026-09-20T10:53:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:405146337worker2 # [ 132.935713] nix-grpc-daemon[1862]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 95 ms (attempt 1/5)6338worker2 # [ 133.033261] nix-grpc-daemon[1862]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 36 ms (attempt 2/5)6339worker2 # [ 133.071841] nix-grpc-daemon[1862]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 6 ms (attempt 3/5)6340worker2 # [ 133.080214] nix-grpc-daemon[1862]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 762 ms (attempt 4/5)6341worker2 # [ 133.845820] nix-grpc-daemon[1862]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6342worker1 # [ 133.918333] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:37Z level=debug event=want client=5 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06343worker1 # [ 133.922414] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:37Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16344worker1 # [ 133.926663] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:37Z level=debug event=want client=5 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06345worker1 # [ 133.930454] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:37Z level=debug event=want client=5 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06346worker1 # [ 133.961069] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564326347worker1 # [ 133.973524] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564326348worker1 # [ 133.984605] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:564326349worker1 # [ 133.996212] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6350worker1 # [ 134.162846] nix-daemon[840]: accepted connection from pid 1553, user nix-grpc-daemon (trusted)6351worker1 # [ 134.174627] nix-daemon[840]: reaped child process 1556, status = succeeded6352worker1 # [ 134.320498] nix-daemon[840]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6353worker1 # [ 134.332401] nix-daemon[840]: reaped child process 1561, status = succeeded6354lb # [ 134.322902] niks3-server[1039]: 2026/09/20 10:53:37 INFO Received uploads request method=POST path=/api/pending_closures6355worker1 # [ 134.353079] nix-grpc-daemon[1363]: time=2026-09-20T10:53:37.498Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6356worker1 # [ 134.354437] nix-grpc-daemon[1363]: time=2026-09-20T10:53:37.498Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6357lb # [ 134.353932] niks3-server[1039]: 2026/09/20 10:53:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6358worker1 # [ 134.381222] nix-grpc-daemon[1363]: time=2026-09-20T10:53:37.526Z level=INFO msg="Uploading 1 narinfos"6359lb # [ 134.360683] niks3-server[1039]: 2026/09/20 10:53:37 INFO Signed narinfos id=46 count=16360lb # [ 134.367271] niks3-server[1039]: 2026/09/20 10:53:37 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6361lb # [ 134.376889] niks3-server[1039]: 2026/09/20 10:53:37 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6362lb # [ 134.384909] niks3-server[1039]: 2026/09/20 10:53:37 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6363worker1 # [ 134.413549] nix-grpc-daemon[1363]: time=2026-09-20T10:53:37.559Z level=INFO msg="Upload complete. (406ms)"6364lb # [ 134.392730] niks3-server[1039]: 2026/09/20 10:53:37 INFO Completed upload id=466365lb # [ 134.396866] niks3-server[1039]: 2026/09/20 10:53:37 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6366worker1 # [ 134.417133] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56432 duration_s=0 paths=1 nar_bytes_in=5046367worker1 # [ 134.422497] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564326368worker1 # [ 134.433629] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6369worker1 # [ 134.436073] nix-daemon[840]: reaped child process 1551, status = succeeded6370worker1 # [ 134.441421] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6371worker1 # [ 135.456944] nix-daemon[840]: reaped child process 1565, status = succeeded6372worker1 # [ 135.601037] nix-daemon[840]: accepted connection from pid 1580, user nix-grpc-daemon (trusted)6373worker1 # [ 135.613588] nix-daemon[840]: reaped child process 1583, status = succeeded6374lb # [ 135.609334] niks3-server[1039]: 2026/09/20 10:53:38 INFO Received uploads request method=POST path=/api/pending_closures6375worker1 # [ 135.640305] nix-grpc-daemon[1363]: time=2026-09-20T10:53:38.784Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6376worker1 # [ 135.641641] nix-grpc-daemon[1363]: time=2026-09-20T10:53:38.784Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6377lb # [ 135.643616] niks3-server[1039]: 2026/09/20 10:53:38 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6378lb # [ 135.655131] niks3-server[1039]: 2026/09/20 10:53:38 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6379lb # [ 135.668402] niks3-server[1039]: 2026/09/20 10:53:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6380worker1 # [ 135.693576] nix-grpc-daemon[1363]: time=2026-09-20T10:53:38.839Z level=INFO msg="Uploading 1 narinfos"6381lb # [ 135.672436] niks3-server[1039]: 2026/09/20 10:53:38 INFO Signed narinfos id=47 count=16382lb # [ 135.677829] niks3-server[1039]: 2026/09/20 10:53:38 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6383lb # [ 135.685849] niks3-server[1039]: 2026/09/20 10:53:38 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6384lb # [ 135.688535] niks3-server[1039]: 2026/09/20 10:53:38 INFO Completed upload id=476385worker1 # [ 135.711052] nix-grpc-daemon[1363]: time=2026-09-20T10:53:38.856Z level=INFO msg="Upload complete. (261ms)"6386lb # [ 135.691451] niks3-server[1039]: 2026/09/20 10:53:38 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6387worker1 # [ 135.714373] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56432 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=7 outputs=16388worker1 # [ 135.718948] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:38Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16389worker1 # [ 135.723317] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:38Z level=debug event=want client=5 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06390worker1 # [ 135.728597] nix-daemon[840]: reaped child process 1563, status = succeeded6391worker1 # [ 135.762411] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564326392worker1 # [ 135.770672] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564326393worker1 # [ 135.778704] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:564326394worker1 # [ 135.783242] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6395worker1 # [ 135.939239] nix-daemon[840]: accepted connection from pid 1589, user nix-grpc-daemon (trusted)6396worker1 # [ 135.951628] nix-daemon[840]: reaped child process 1592, status = succeeded6397worker1 # [ 136.112552] nix-daemon[840]: accepted connection from pid 1594, user nix-grpc-daemon (trusted)6398worker1 # [ 136.124058] nix-daemon[840]: reaped child process 1597, status = succeeded6399lb # [ 136.119463] niks3-server[1039]: 2026/09/20 10:53:39 INFO Received uploads request method=POST path=/api/pending_closures6400worker1 # [ 136.225736] nix-grpc-daemon[1363]: time=2026-09-20T10:53:39.371Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6401worker1 # [ 136.228043] nix-grpc-daemon[1363]: time=2026-09-20T10:53:39.373Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6402lb # [ 136.224712] niks3-server[1039]: 2026/09/20 10:53:39 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6403lb # [ 136.238352] niks3-server[1039]: 2026/09/20 10:53:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6404lb # [ 136.243502] niks3-server[1039]: 2026/09/20 10:53:39 INFO Signed narinfos id=48 count=16405worker1 # [ 136.268167] nix-grpc-daemon[1363]: time=2026-09-20T10:53:39.413Z level=INFO msg="Uploading 1 narinfos"6406lb # [ 136.251862] niks3-server[1039]: 2026/09/20 10:53:39 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6407lb # [ 136.260976] niks3-server[1039]: 2026/09/20 10:53:39 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6408lb # [ 136.267081] niks3-server[1039]: 2026/09/20 10:53:39 INFO Completed upload id=486409worker1 # [ 136.288335] nix-grpc-daemon[1363]: time=2026-09-20T10:53:39.433Z level=INFO msg="Upload complete. (497ms)"6410lb # [ 136.272388] niks3-server[1039]: 2026/09/20 10:53:39 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6411worker1 # [ 136.293148] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56432 duration_s=0 paths=1 nar_bytes_in=5046412worker1 # [ 136.298961] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564326413worker1 # [ 136.311133] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6414worker1 # [ 136.319590] nix-daemon[840]: reaped child process 1587, status = succeeded6415worker1 # [ 136.330228] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6416worker1 # [ 137.547809] nix-daemon[840]: reaped child process 1601, status = succeeded6417worker1 # [ 137.683054] nix-daemon[840]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)6418worker1 # [ 137.694680] nix-daemon[840]: reaped child process 1619, status = succeeded6419lb # [ 137.688117] niks3-server[1039]: 2026/09/20 10:53:40 INFO Received uploads request method=POST path=/api/pending_closures6420worker1 # [ 137.714365] nix-grpc-daemon[1363]: time=2026-09-20T10:53:40.859Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6421worker1 # [ 137.718650] nix-grpc-daemon[1363]: time=2026-09-20T10:53:40.859Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6422lb # [ 137.712168] niks3-server[1039]: 2026/09/20 10:53:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6423lb # [ 137.714687] niks3-server[1039]: 2026/09/20 10:53:40 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6424lb # [ 137.717390] niks3-server[1039]: 2026/09/20 10:53:40 INFO Signed narinfos id=49 count=16425worker1 # [ 137.738315] nix-grpc-daemon[1363]: time=2026-09-20T10:53:40.883Z level=INFO msg="Uploading 1 narinfos"6426lb # [ 137.718649] niks3-server[1039]: 2026/09/20 10:53:40 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6427lb # [ 137.722363] niks3-server[1039]: 2026/09/20 10:53:40 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6428lb # [ 137.725839] niks3-server[1039]: 2026/09/20 10:53:40 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6429lb # [ 137.729907] niks3-server[1039]: 2026/09/20 10:53:40 INFO Completed upload id=496430worker1 # [ 137.750807] nix-grpc-daemon[1363]: time=2026-09-20T10:53:40.896Z level=INFO msg="Upload complete. (209ms)"6431lb # [ 137.731470] niks3-server[1039]: 2026/09/20 10:53:40 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6432worker1 # [ 137.753075] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56432 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=8 outputs=16433worker1 # [ 137.757837] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:40Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16434worker1 # [ 137.759914] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564326435worker1 # [ 137.762577] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:40Z level=debug event=want client=5 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06436worker1 # [ 137.768277] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564326437worker1 # [ 137.770582] nix-daemon[840]: reaped child process 1599, status = succeeded6438worker1 # [ 137.773665] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:564326439worker1 # [ 137.777407] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6440worker1 # [ 137.931657] nix-daemon[840]: accepted connection from pid 1625, user nix-grpc-daemon (trusted)6441worker1 # [ 137.944268] nix-daemon[840]: reaped child process 1628, status = succeeded6442lb # [ 138.003044] envoy[910]: conn peer=[2001:db8:1::4]:50218 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1104 tx=425 ms=282226443worker1 # [ 138.095754] nix-daemon[840]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)6444worker1 # [ 138.108325] nix-daemon[840]: reaped child process 1633, status = succeeded6445lb # [ 138.100449] niks3-server[1039]: 2026/09/20 10:53:41 INFO Received uploads request method=POST path=/api/pending_closures6446worker1 # [ 138.125914] nix-grpc-daemon[1363]: time=2026-09-20T10:53:41.271Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6447worker1 # [ 138.127313] nix-grpc-daemon[1363]: time=2026-09-20T10:53:41.271Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6448lb # [ 138.112916] niks3-server[1039]: 2026/09/20 10:53:41 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6449lb # [ 138.116337] niks3-server[1039]: 2026/09/20 10:53:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6450lb # [ 138.118770] niks3-server[1039]: 2026/09/20 10:53:41 INFO Signed narinfos id=50 count=16451worker1 # [ 138.141127] nix-grpc-daemon[1363]: time=2026-09-20T10:53:41.286Z level=INFO msg="Uploading 1 narinfos"6452lb # [ 138.120871] niks3-server[1039]: 2026/09/20 10:53:41 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6453lb # [ 138.126644] niks3-server[1039]: 2026/09/20 10:53:41 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6454lb # [ 138.129054] niks3-server[1039]: 2026/09/20 10:53:41 INFO Completed upload id=506455worker1 # [ 138.150309] nix-grpc-daemon[1363]: time=2026-09-20T10:53:41.295Z level=INFO msg="Upload complete. (367ms)"6456lb # [ 138.130551] niks3-server[1039]: 2026/09/20 10:53:41 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6457worker1 # [ 138.152214] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56432 duration_s=0 paths=1 nar_bytes_in=5046458worker1 # [ 138.156211] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564326459worker1 # [ 138.161939] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6460worker1 # [ 138.164200] nix-daemon[840]: reaped child process 1623, status = succeeded6461worker1 # [ 138.169303] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6462worker1 # [ 139.195442] nix-daemon[840]: reaped child process 1637, status = succeeded6463worker1 # [ 139.348486] nix-daemon[840]: accepted connection from pid 1652, user nix-grpc-daemon (trusted)6464worker1 # [ 139.360141] nix-daemon[840]: reaped child process 1655, status = succeeded6465lb # [ 139.354163] niks3-server[1039]: 2026/09/20 10:53:42 INFO Received uploads request method=POST path=/api/pending_closures6466worker1 # [ 139.383621] nix-grpc-daemon[1363]: time=2026-09-20T10:53:42.529Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6467worker1 # [ 139.385103] nix-grpc-daemon[1363]: time=2026-09-20T10:53:42.530Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6468lb # [ 139.391432] niks3-server[1039]: 2026/09/20 10:53:42 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv6469lb # [ 139.398827] niks3-server[1039]: 2026/09/20 10:53:42 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6470lb # [ 139.408928] niks3-server[1039]: 2026/09/20 10:53:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6471lb # [ 139.415902] niks3-server[1039]: 2026/09/20 10:53:42 INFO Signed narinfos id=51 count=16472worker1 # [ 139.439445] nix-grpc-daemon[1363]: time=2026-09-20T10:53:42.584Z level=INFO msg="Uploading 1 narinfos"6473lb # [ 139.419592] niks3-server[1039]: 2026/09/20 10:53:42 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls6474lb # [ 139.430291] niks3-server[1039]: 2026/09/20 10:53:42 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6475lb # [ 139.436033] niks3-server[1039]: 2026/09/20 10:53:42 INFO Completed upload id=516476worker1 # [ 139.457070] nix-grpc-daemon[1363]: time=2026-09-20T10:53:42.602Z level=INFO msg="Upload complete. (280ms)"6477lb # [ 139.437802] niks3-server[1039]: 2026/09/20 10:53:42 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo6478worker1 # [ 139.463722] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56432 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=9 outputs=16479worker1 # [ 139.470269] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:42Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=16480worker1 # [ 139.480490] nix-daemon[840]: reaped child process 1635, status = succeeded6481worker1 # [ 139.505997] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564326482worker1 # [ 139.512531] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:42Z level=debug event=want client=5 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=06483worker1 # [ 139.523339] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564326484worker1 # [ 139.535933] nix-grpc-daemon[1363]: time=2026-09-20T10:53:42.681Z level=INFO msg="All 1 paths already cached"6485worker1 # [ 139.542917] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:564326486worker1 # [ 139.552155] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6487worker1 # [ 139.709723] nix-daemon[840]: accepted connection from pid 1661, user nix-grpc-daemon (trusted)6488worker1 # [ 139.721683] nix-daemon[840]: reaped child process 1664, status = succeeded6489worker1 # [ 139.870202] nix-daemon[840]: accepted connection from pid 1666, user nix-grpc-daemon (trusted)6490worker1 # [ 139.881790] nix-daemon[840]: reaped child process 1669, status = succeeded6491lb # [ 139.873425] niks3-server[1039]: 2026/09/20 10:53:42 INFO Received uploads request method=POST path=/api/pending_closures6492worker1 # [ 139.902472] nix-grpc-daemon[1363]: time=2026-09-20T10:53:43.047Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6493worker1 # [ 139.903836] nix-grpc-daemon[1363]: time=2026-09-20T10:53:43.047Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"6494lb # [ 139.897632] niks3-server[1039]: 2026/09/20 10:53:42 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst6495lb # [ 139.908591] niks3-server[1039]: 2026/09/20 10:53:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6496worker1 # [ 139.934804] nix-grpc-daemon[1363]: time=2026-09-20T10:53:43.080Z level=INFO msg="Uploading 1 narinfos"6497lb # [ 139.913726] niks3-server[1039]: 2026/09/20 10:53:42 INFO Signed narinfos id=52 count=16498lb # [ 139.919784] niks3-server[1039]: 2026/09/20 10:53:42 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls6499lb # [ 139.931471] niks3-server[1039]: 2026/09/20 10:53:42 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6500worker1 # [ 139.958489] nix-grpc-daemon[1363]: time=2026-09-20T10:53:43.104Z level=INFO msg="Upload complete. (396ms)"6501lb # [ 139.937693] niks3-server[1039]: 2026/09/20 10:53:42 INFO Completed upload id=526502lb # [ 139.941132] niks3-server[1039]: 2026/09/20 10:53:42 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo6503worker1 # [ 139.961366] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56432 duration_s=0 paths=1 nar_bytes_in=6246504worker1 # [ 139.966535] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564326505worker1 # [ 139.976135] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6506worker1 # [ 139.979735] nix-daemon[840]: reaped child process 1659, status = succeeded6507worker1 # [ 139.984523] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6508worker1 # [ 141.002964] nix-daemon[840]: reaped child process 1673, status = succeeded6509worker1 # [ 141.157417] nix-daemon[840]: accepted connection from pid 1688, user nix-grpc-daemon (trusted)6510worker1 # [ 141.167826] nix-daemon[840]: reaped child process 1691, status = succeeded6511lb # [ 141.162318] niks3-server[1039]: 2026/09/20 10:53:44 INFO Received uploads request method=POST path=/api/pending_closures6512worker1 # [ 141.193824] nix-grpc-daemon[1363]: time=2026-09-20T10:53:44.339Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6513worker1 # [ 141.195544] nix-grpc-daemon[1363]: time=2026-09-20T10:53:44.341Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"6514lb # [ 141.203160] niks3-server[1039]: 2026/09/20 10:53:44 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst6515lb # [ 141.205887] niks3-server[1039]: 2026/09/20 10:53:44 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv6516lb # [ 141.209626] niks3-server[1039]: 2026/09/20 10:53:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6517lb # [ 141.211575] niks3-server[1039]: 2026/09/20 10:53:44 INFO Signed narinfos id=53 count=16518worker1 # [ 141.234072] nix-grpc-daemon[1363]: time=2026-09-20T10:53:44.379Z level=INFO msg="Uploading 1 narinfos"6519lb # [ 141.216382] niks3-server[1039]: 2026/09/20 10:53:44 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls6520lb # [ 141.220065] niks3-server[1039]: 2026/09/20 10:53:44 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6521lb # [ 141.223893] niks3-server[1039]: 2026/09/20 10:53:44 INFO Completed upload id=536522worker1 # [ 141.244770] nix-grpc-daemon[1363]: time=2026-09-20T10:53:44.390Z level=INFO msg="Upload complete. (252ms)"6523lb # [ 141.226747] niks3-server[1039]: 2026/09/20 10:53:44 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo6524worker1 # [ 141.248607] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56432 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=10 outputs=16525worker1 # [ 141.252128] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:44Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=16526worker1 # [ 141.255281] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564326527worker1 # [ 141.257532] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:44Z level=debug event=want client=5 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=06528worker1 # [ 141.263283] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564326529worker1 # [ 141.265899] nix-daemon[840]: reaped child process 1671, status = succeeded6530worker1 # [ 141.269472] nix-grpc-daemon[1363]: time=2026-09-20T10:53:44.415Z level=INFO msg="All 1 paths already cached"6531worker1 # [ 141.273255] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:564326532worker1 # [ 141.278058] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6533worker1 # [ 141.432064] nix-daemon[840]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)6534worker1 # [ 141.444907] nix-daemon[840]: reaped child process 1700, status = succeeded6535worker1 # [ 141.597502] nix-daemon[840]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6536worker1 # [ 141.608901] nix-daemon[840]: reaped child process 1705, status = succeeded6537lb # [ 141.602352] niks3-server[1039]: 2026/09/20 10:53:44 INFO Received uploads request method=POST path=/api/pending_closures6538worker1 # [ 141.632377] nix-grpc-daemon[1363]: time=2026-09-20T10:53:44.777Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6539worker1 # [ 141.633708] nix-grpc-daemon[1363]: time=2026-09-20T10:53:44.777Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"6540lb # [ 141.631834] niks3-server[1039]: 2026/09/20 10:53:44 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst6541lb # [ 141.644896] niks3-server[1039]: 2026/09/20 10:53:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6542lb # [ 141.651684] niks3-server[1039]: 2026/09/20 10:53:44 INFO Signed narinfos id=54 count=16543worker1 # [ 141.675478] nix-grpc-daemon[1363]: time=2026-09-20T10:53:44.820Z level=INFO msg="Uploading 1 narinfos"6544lb # [ 141.656814] niks3-server[1039]: 2026/09/20 10:53:44 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls6545lb # [ 141.663050] niks3-server[1039]: 2026/09/20 10:53:44 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6546lb # [ 141.665902] niks3-server[1039]: 2026/09/20 10:53:44 INFO Completed upload id=546547worker1 # [ 141.686789] nix-grpc-daemon[1363]: time=2026-09-20T10:53:44.832Z level=INFO msg="Upload complete. (403ms)"6548lb # [ 141.668890] niks3-server[1039]: 2026/09/20 10:53:44 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo6549worker1 # [ 141.690524] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56432 duration_s=0 paths=1 nar_bytes_in=6246550worker1 # [ 141.696259] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564326551worker1 # [ 141.702833] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6552worker1 # [ 141.707418] nix-daemon[840]: reaped child process 1695, status = succeeded6553worker1 # [ 141.712397] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6554worker1 # [ 143.327975] nix-daemon[840]: reaped child process 1709, status = succeeded6555worker1 # [ 143.466977] nix-daemon[840]: accepted connection from pid 1724, user nix-grpc-daemon (trusted)6556worker1 # [ 143.478517] nix-daemon[840]: reaped child process 1727, status = succeeded6557lb # [ 143.474981] niks3-server[1039]: 2026/09/20 10:53:46 INFO Received uploads request method=POST path=/api/pending_closures6558worker1 # [ 143.504154] nix-grpc-daemon[1363]: time=2026-09-20T10:53:46.649Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6559worker1 # [ 143.506551] nix-grpc-daemon[1363]: time=2026-09-20T10:53:46.651Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"6560lb # [ 143.512602] niks3-server[1039]: 2026/09/20 10:53:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6561worker1 # [ 143.540219] nix-grpc-daemon[1363]: time=2026-09-20T10:53:46.685Z level=INFO msg="Uploading 1 narinfos"6562lb # [ 143.518752] niks3-server[1039]: 2026/09/20 10:53:46 INFO Signed narinfos id=55 count=16563lb # [ 143.526529] niks3-server[1039]: 2026/09/20 10:53:46 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv6564lb # [ 143.539762] niks3-server[1039]: 2026/09/20 10:53:46 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls6565lb # [ 143.545709] niks3-server[1039]: 2026/09/20 10:53:46 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6566lb # [ 143.548903] niks3-server[1039]: 2026/09/20 10:53:46 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst6567lb # [ 143.554396] niks3-server[1039]: 2026/09/20 10:53:46 INFO Completed upload id=556568worker1 # [ 143.575514] nix-grpc-daemon[1363]: time=2026-09-20T10:53:46.721Z level=INFO msg="Upload complete. (255ms)"6569lb # [ 143.556256] niks3-server[1039]: 2026/09/20 10:53:46 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo6570worker1 # [ 143.578264] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56432 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=11 outputs=16571worker1 # [ 143.583600] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:46Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=16572worker1 # [ 143.587725] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564326573worker1 # [ 143.591034] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:46Z level=debug event=want client=5 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=06574worker1 # [ 143.597304] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564326575worker1 # [ 143.599914] nix-daemon[840]: reaped child process 1707, status = succeeded6576worker1 # [ 143.603628] nix-grpc-daemon[1363]: time=2026-09-20T10:53:46.749Z level=INFO msg="All 1 paths already cached"6577worker1 # [ 143.607962] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:564326578worker1 # [ 143.613557] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6579worker1 # [ 143.773294] nix-daemon[840]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6580worker1 # [ 143.784995] nix-daemon[840]: reaped child process 1736, status = succeeded6581worker1 # [ 143.952594] nix-daemon[840]: accepted connection from pid 1738, user nix-grpc-daemon (trusted)6582worker1 # [ 143.964150] nix-daemon[840]: reaped child process 1741, status = succeeded6583lb # [ 143.959915] niks3-server[1039]: 2026/09/20 10:53:46 INFO Received uploads request method=POST path=/api/pending_closures6584worker1 # [ 143.990478] nix-grpc-daemon[1363]: time=2026-09-20T10:53:47.135Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6585worker1 # [ 143.991808] nix-grpc-daemon[1363]: time=2026-09-20T10:53:47.135Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"6586lb # [ 143.986902] niks3-server[1039]: 2026/09/20 10:53:46 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst6587lb # [ 143.996479] niks3-server[1039]: 2026/09/20 10:53:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6588lb # [ 144.001799] niks3-server[1039]: 2026/09/20 10:53:46 INFO Signed narinfos id=56 count=16589worker1 # [ 144.025822] nix-grpc-daemon[1363]: time=2026-09-20T10:53:47.171Z level=INFO msg="Uploading 1 narinfos"6590lb # [ 144.008561] niks3-server[1039]: 2026/09/20 10:53:46 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls6591lb # [ 144.018622] niks3-server[1039]: 2026/09/20 10:53:46 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6592worker1 # [ 144.044758] nix-grpc-daemon[1363]: time=2026-09-20T10:53:47.190Z level=INFO msg="Upload complete. (425ms)"6593lb # [ 144.023877] niks3-server[1039]: 2026/09/20 10:53:46 INFO Completed upload id=566594worker1 # [ 144.046818] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56432 duration_s=0 paths=1 nar_bytes_in=6246595lb # [ 144.029092] niks3-server[1039]: 2026/09/20 10:53:46 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo6596worker1 # [ 144.050592] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564326597worker1 # [ 144.058539] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6598worker1 # [ 144.061744] nix-daemon[840]: reaped child process 1731, status = succeeded6599worker1 # [ 144.066970] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6600worker1 # [ 145.486897] nix-daemon[840]: reaped child process 1745, status = succeeded6601worker1 # [ 145.643429] nix-daemon[840]: accepted connection from pid 1761, user nix-grpc-daemon (trusted)6602worker1 # [ 145.654918] nix-daemon[840]: reaped child process 1764, status = succeeded6603lb # [ 145.650378] niks3-server[1039]: 2026/09/20 10:53:48 INFO Received uploads request method=POST path=/api/pending_closures6604worker1 # [ 145.679866] nix-grpc-daemon[1363]: time=2026-09-20T10:53:48.825Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6605worker1 # [ 145.681360] nix-grpc-daemon[1363]: time=2026-09-20T10:53:48.827Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"6606lb # [ 145.689541] niks3-server[1039]: 2026/09/20 10:53:48 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv6607lb # [ 145.697196] niks3-server[1039]: 2026/09/20 10:53:48 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst6608lb # [ 145.707660] niks3-server[1039]: 2026/09/20 10:53:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6609worker1 # [ 145.735455] nix-grpc-daemon[1363]: time=2026-09-20T10:53:48.881Z level=INFO msg="Uploading 1 narinfos"6610lb # [ 145.714221] niks3-server[1039]: 2026/09/20 10:53:48 INFO Signed narinfos id=57 count=16611lb # [ 145.718473] niks3-server[1039]: 2026/09/20 10:53:48 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls6612lb # [ 145.728514] niks3-server[1039]: 2026/09/20 10:53:48 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6613lb # [ 145.733540] niks3-server[1039]: 2026/09/20 10:53:48 INFO Completed upload id=576614worker1 # [ 145.754814] nix-grpc-daemon[1363]: time=2026-09-20T10:53:48.900Z level=INFO msg="Upload complete. (280ms)"6615lb # [ 145.735907] niks3-server[1039]: 2026/09/20 10:53:48 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo6616worker1 # [ 145.765705] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56432 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=12 outputs=16617worker1 # [ 145.772308] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:48Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=16618worker1 # [ 145.777368] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:48Z level=debug event=want client=5 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=06619worker1 # [ 145.785441] nix-daemon[840]: reaped child process 1743, status = succeeded6620worker1 # [ 145.803847] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564326621worker1 # [ 145.812285] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564326622worker1 # [ 145.821541] nix-grpc-daemon[1363]: time=2026-09-20T10:53:48.967Z level=INFO msg="All 2 paths already cached"6623worker1 # [ 145.826992] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:564326624worker1 # [ 145.834878] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6625worker1 # [ 145.986548] nix-daemon[840]: accepted connection from pid 1770, user nix-grpc-daemon (trusted)6626worker1 # [ 145.998364] nix-daemon[840]: reaped child process 1773, status = succeeded6627worker1 # [ 146.157078] nix-daemon[840]: accepted connection from pid 1775, user nix-grpc-daemon (trusted)6628worker1 # [ 146.168811] nix-daemon[840]: reaped child process 1778, status = succeeded6629lb # [ 146.156910] niks3-server[1039]: 2026/09/20 10:53:49 INFO Received uploads request method=POST path=/api/pending_closures6630worker1 # [ 146.181955] nix-grpc-daemon[1363]: time=2026-09-20T10:53:49.327Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6631worker1 # [ 146.183710] nix-grpc-daemon[1363]: time=2026-09-20T10:53:49.329Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"6632lb # [ 146.170481] niks3-server[1039]: 2026/09/20 10:53:49 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst6633lb # [ 146.173710] niks3-server[1039]: 2026/09/20 10:53:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6634lb # [ 146.175413] niks3-server[1039]: 2026/09/20 10:53:49 INFO Signed narinfos id=58 count=16635worker1 # [ 146.196717] nix-grpc-daemon[1363]: time=2026-09-20T10:53:49.342Z level=INFO msg="Uploading 1 narinfos"6636lb # [ 146.181064] niks3-server[1039]: 2026/09/20 10:53:49 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6637lb # [ 146.184252] niks3-server[1039]: 2026/09/20 10:53:49 INFO Completed upload id=586638worker1 # [ 146.206540] nix-grpc-daemon[1363]: time=2026-09-20T10:53:49.352Z level=INFO msg="Upload complete. (364ms)"6639lb # [ 146.185980] niks3-server[1039]: 2026/09/20 10:53:49 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls6640worker1 # [ 146.208375] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56432 duration_s=0 paths=1 nar_bytes_in=7526641lb # [ 146.188405] niks3-server[1039]: 2026/09/20 10:53:49 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo6642worker1 # [ 146.211708] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564326643worker1 # [ 146.217924] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6644worker1 # [ 146.220131] nix-daemon[840]: reaped child process 1768, status = succeeded6645worker1 # [ 146.225382] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6646worker1 # [ 147.452661] nix-daemon[840]: reaped child process 1782, status = succeeded6647worker1 # [ 147.608090] nix-daemon[840]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6648worker1 # [ 147.619288] nix-daemon[840]: reaped child process 1800, status = succeeded6649lb # [ 147.613205] niks3-server[1039]: 2026/09/20 10:53:50 INFO Received uploads request method=POST path=/api/pending_closures6650worker1 # [ 147.638858] nix-grpc-daemon[1363]: time=2026-09-20T10:53:50.784Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6651worker1 # [ 147.640727] nix-grpc-daemon[1363]: time=2026-09-20T10:53:50.785Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"6652lb # [ 147.631358] niks3-server[1039]: 2026/09/20 10:53:50 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv6653lb # [ 147.635447] niks3-server[1039]: 2026/09/20 10:53:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6654lb # [ 147.638065] niks3-server[1039]: 2026/09/20 10:53:50 INFO Signed narinfos id=59 count=16655worker1 # [ 147.659027] nix-grpc-daemon[1363]: time=2026-09-20T10:53:50.804Z level=INFO msg="Uploading 1 narinfos"6656lb # [ 147.640140] niks3-server[1039]: 2026/09/20 10:53:50 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst6657lb # [ 147.644049] niks3-server[1039]: 2026/09/20 10:53:50 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls6658lb # [ 147.647691] niks3-server[1039]: 2026/09/20 10:53:50 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6659lb # [ 147.650788] niks3-server[1039]: 2026/09/20 10:53:50 INFO Completed upload id=596660worker1 # [ 147.671890] nix-grpc-daemon[1363]: time=2026-09-20T10:53:50.817Z level=INFO msg="Upload complete. (238ms)"6661lb # [ 147.653530] niks3-server[1039]: 2026/09/20 10:53:50 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo6662worker1 # [ 147.674275] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56432 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=13 outputs=16663worker1 # [ 147.678363] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:50Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=16664worker1 # [ 147.680669] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564326665worker1 # [ 147.686601] nix-daemon[840]: reaped child process 1780, status = succeeded6666worker1 # [ 147.688340] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564326667worker1 # [ 147.694049] nix-grpc-daemon[1363]: time=2026-09-20T10:53:50.839Z level=INFO msg="All 2 paths already cached"6668worker1 # [ 147.697789] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:564326669worker1 # [ 147.703112] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6670worker1 # [ 147.862951] nix-daemon[840]: accepted connection from pid 1804, user nix-grpc-daemon (trusted)6671worker1 # [ 147.874667] nix-daemon[840]: reaped child process 1807, status = succeeded6672worker1 # [ 148.045681] nix-daemon[840]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)6673worker1 # [ 148.057670] nix-daemon[840]: reaped child process 1812, status = succeeded6674lb # [ 148.050202] niks3-server[1039]: 2026/09/20 10:53:50 INFO Received uploads request method=POST path=/api/pending_closures6675worker1 # [ 148.075818] nix-grpc-daemon[1363]: time=2026-09-20T10:53:51.221Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6676worker1 # [ 148.077169] nix-grpc-daemon[1363]: time=2026-09-20T10:53:51.221Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"6677lb # [ 148.063223] niks3-server[1039]: 2026/09/20 10:53:50 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst6678lb # [ 148.065656] niks3-server[1039]: 2026/09/20 10:53:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6679lb # [ 148.068102] niks3-server[1039]: 2026/09/20 10:53:50 INFO Signed narinfos id=60 count=16680worker1 # [ 148.089315] nix-grpc-daemon[1363]: time=2026-09-20T10:53:51.234Z level=INFO msg="Uploading 1 narinfos"6681lb # [ 148.071475] niks3-server[1039]: 2026/09/20 10:53:50 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls6682lb # [ 148.076309] niks3-server[1039]: 2026/09/20 10:53:51 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6683lb # [ 148.079185] niks3-server[1039]: 2026/09/20 10:53:51 INFO Completed upload id=606684worker1 # [ 148.100350] nix-grpc-daemon[1363]: time=2026-09-20T10:53:51.246Z level=INFO msg="Upload complete. (392ms)"6685lb # [ 148.080848] niks3-server[1039]: 2026/09/20 10:53:51 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo6686worker1 # [ 148.102341] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56432 duration_s=0 paths=1 nar_bytes_in=7526687worker1 # [ 148.105943] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564326688worker1 # [ 148.111936] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6689worker1 # [ 148.114446] nix-daemon[840]: reaped child process 1802, status = succeeded6690worker1 # [ 148.119735] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6691worker1 # [ 149.340241] nix-daemon[840]: reaped child process 1816, status = succeeded6692worker1 # [ 149.503196] nix-daemon[840]: accepted connection from pid 1831, user nix-grpc-daemon (trusted)6693worker1 # [ 149.514912] nix-daemon[840]: reaped child process 1834, status = succeeded6694lb # [ 149.510637] niks3-server[1039]: 2026/09/20 10:53:52 INFO Received uploads request method=POST path=/api/pending_closures6695worker1 # [ 149.539318] nix-grpc-daemon[1363]: time=2026-09-20T10:53:52.684Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6696worker1 # [ 149.540910] nix-grpc-daemon[1363]: time=2026-09-20T10:53:52.686Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"6697lb # [ 149.549485] niks3-server[1039]: 2026/09/20 10:53:52 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst6698lb # [ 149.557576] niks3-server[1039]: 2026/09/20 10:53:52 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv6699lb # [ 149.562464] niks3-server[1039]: 2026/09/20 10:53:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6700lb # [ 149.567494] niks3-server[1039]: 2026/09/20 10:53:52 INFO Signed narinfos id=61 count=16701worker1 # [ 149.591974] nix-grpc-daemon[1363]: time=2026-09-20T10:53:52.737Z level=INFO msg="Uploading 1 narinfos"6702lb # [ 149.579836] niks3-server[1039]: 2026/09/20 10:53:52 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6703lb # [ 149.586190] niks3-server[1039]: 2026/09/20 10:53:52 INFO Completed upload id=616704worker1 # [ 149.609781] nix-grpc-daemon[1363]: time=2026-09-20T10:53:52.755Z level=INFO msg="Upload complete. (282ms)"6705lb # [ 149.590312] niks3-server[1039]: 2026/09/20 10:53:52 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls6706lb # [ 149.593446] niks3-server[1039]: 2026/09/20 10:53:52 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo6707worker1 # [ 149.618342] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56432 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=14 outputs=16708worker1 # [ 149.624255] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:52Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=16709worker1 # [ 149.633668] nix-daemon[840]: reaped child process 1814, status = succeeded6710worker1 # [ 149.659731] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564326711worker1 # [ 149.671839] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564326712worker1 # [ 149.683600] nix-grpc-daemon[1363]: time=2026-09-20T10:53:52.829Z level=INFO msg="All 2 paths already cached"6713worker1 # [ 149.690996] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:564326714worker1 # [ 149.699470] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6715worker1 # [ 149.874194] nix-daemon[840]: accepted connection from pid 1838, user nix-grpc-daemon (trusted)6716worker1 # [ 149.886386] nix-daemon[840]: reaped child process 1841, status = succeeded6717worker1 # [ 150.049322] nix-daemon[840]: accepted connection from pid 1843, user nix-grpc-daemon (trusted)6718worker1 # [ 150.061080] nix-daemon[840]: reaped child process 1846, status = succeeded6719lb # [ 150.054609] niks3-server[1039]: 2026/09/20 10:53:52 INFO Received uploads request method=POST path=/api/pending_closures6720worker1 # [ 150.082952] nix-grpc-daemon[1363]: time=2026-09-20T10:53:53.228Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6721worker1 # [ 150.084409] nix-grpc-daemon[1363]: time=2026-09-20T10:53:53.230Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"6722lb # [ 150.081138] niks3-server[1039]: 2026/09/20 10:53:53 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst6723lb # [ 150.092525] niks3-server[1039]: 2026/09/20 10:53:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6724worker1 # [ 150.118113] nix-grpc-daemon[1363]: time=2026-09-20T10:53:53.263Z level=INFO msg="Uploading 1 narinfos"6725lb # [ 150.098413] niks3-server[1039]: 2026/09/20 10:53:53 INFO Signed narinfos id=62 count=16726lb # [ 150.103294] niks3-server[1039]: 2026/09/20 10:53:53 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls6727lb # [ 150.114164] niks3-server[1039]: 2026/09/20 10:53:53 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6728lb # [ 150.120787] niks3-server[1039]: 2026/09/20 10:53:53 INFO Completed upload id=626729worker1 # [ 150.141868] nix-grpc-daemon[1363]: time=2026-09-20T10:53:53.287Z level=INFO msg="Upload complete. (429ms)"6730lb # [ 150.123118] niks3-server[1039]: 2026/09/20 10:53:53 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo6731worker1 # [ 150.145152] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56432 duration_s=0 paths=1 nar_bytes_in=7526732worker1 # [ 150.150485] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564326733worker1 # [ 150.158268] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6734worker1 # [ 150.166063] nix-daemon[840]: reaped child process 1836, status = succeeded6735worker1 # [ 150.169144] nix-daemon[840]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6736worker1 # [ 150.982538] nix-daemon[840]: reaped child process 1850, status = succeeded6737worker1 # [ 151.132975] nix-daemon[840]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)6738worker1 # [ 151.144592] nix-daemon[840]: reaped child process 1868, status = succeeded6739lb # [ 151.139347] niks3-server[1039]: 2026/09/20 10:53:54 INFO Received uploads request method=POST path=/api/pending_closures6740worker1 # [ 151.168197] nix-grpc-daemon[1363]: time=2026-09-20T10:53:54.313Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6741worker1 # [ 151.169673] nix-grpc-daemon[1363]: time=2026-09-20T10:53:54.315Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"6742lb # [ 151.174505] niks3-server[1039]: 2026/09/20 10:53:54 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv6743lb # [ 151.183611] niks3-server[1039]: 2026/09/20 10:53:54 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst6744lb # [ 151.193117] niks3-server[1039]: 2026/09/20 10:53:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6745worker1 # [ 151.220377] nix-grpc-daemon[1363]: time=2026-09-20T10:53:54.365Z level=INFO msg="Uploading 1 narinfos"6746lb # [ 151.199220] niks3-server[1039]: 2026/09/20 10:53:54 INFO Signed narinfos id=63 count=16747lb # [ 151.202649] niks3-server[1039]: 2026/09/20 10:53:54 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls6748lb # [ 151.212398] niks3-server[1039]: 2026/09/20 10:53:54 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6749lb # [ 151.218932] niks3-server[1039]: 2026/09/20 10:53:54 INFO Completed upload id=636750worker1 # [ 151.240230] nix-grpc-daemon[1363]: time=2026-09-20T10:53:54.385Z level=INFO msg="Upload complete. (264ms)"6751lb # [ 151.221040] niks3-server[1039]: 2026/09/20 10:53:54 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo6752worker1 # [ 151.247527] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56432 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=15 outputs=16753worker1 # [ 151.259533] nix-daemon[840]: reaped child process 1848, status = succeeded6754client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 18.53 seconds)6755worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6756worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6757client: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&16758worker2 # [ 151.535121] nix-grpc-daemon[1862]: ts=2026-09-20T10:53:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:405146759worker2 # [ 151.542381] nix-grpc-daemon[1862]: ts=2026-09-20T10:53:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:405146760worker1 # [ 151.589274] nix-grpc-daemon[1291]: ts=2026-09-20T10:53:54Z level=debug event=want client=6 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=06761lb # [ 158.004152] envoy[910]: conn peer=[2001:db8:1::1]:49226 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27111 tx=10689 ms=183196762client: (finished: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&1, in 30.27 seconds)6763(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 48.87 seconds)6764subtest: low disk drains a worker and builds go to the other6765worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6766worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.02 seconds)6767worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6768worker1 # [ 183.167649] nix-grpc-daemon[1291]: ts=2026-09-20T10:54:26Z level=info event=unhealthy reason=min_free6769worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.08 seconds)6770client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&26771worker1 # [ 183.919396] nix-grpc-daemon[1291]: ts=2026-09-20T10:54:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:564326772client # don't know how to build these paths:6773client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6774worker2 # [ 183.900105] nix-grpc-daemon[1862]: ts=2026-09-20T10:54:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:405146775worker1 # [ 183.943043] nix-grpc-daemon[1291]: ts=2026-09-20T10:54:27Z level=debug event=want client=7 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=06776worker1 # [ 183.944833] nix-grpc-daemon[1291]: ts=2026-09-20T10:54:27Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=16777worker2 # [ 183.961164] nix-grpc-daemon[1862]: ts=2026-09-20T10:54:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:405146778worker2 # [ 183.969312] nix-grpc-daemon[1862]: ts=2026-09-20T10:54:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:405146779client # copying 1 paths to 'lb:50051'...6780worker2 # [ 183.977850] nix-grpc-daemon[1862]: ts=2026-09-20T10:54:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:405146781client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...6782worker2 # [ 183.984529] nix-daemon[845]: accepted connection from pid 1862, user nix-grpc-daemon (trusted)6783worker2 # [ 183.997296] nix-grpc-daemon[1862]: ts=2026-09-20T10:54:26Z level=info event=niks3_push_started pid=19186784worker2 # [ 184.335233] nix-daemon[845]: accepted connection from pid 1931, user nix-grpc-daemon (trusted)6785worker2 # [ 184.346656] nix-daemon[845]: reaped child process 1934, status = succeeded6786worker2 # [ 184.499166] nix-daemon[845]: accepted connection from pid 1936, user nix-grpc-daemon (trusted)6787worker2 # [ 184.511156] nix-daemon[845]: reaped child process 1939, status = succeeded6788lb # [ 184.529869] niks3-server[1039]: 2026/09/20 10:54:27 INFO Received uploads request method=POST path=/api/pending_closures6789worker2 # [ 184.536311] nix-grpc-daemon[1918]: time=2026-09-20T10:54:26.826Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6790worker2 # [ 184.537740] nix-grpc-daemon[1918]: time=2026-09-20T10:54:26.826Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"6791lb # [ 184.569727] niks3-server[1039]: 2026/09/20 10:54:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6792lb # [ 184.580506] niks3-server[1039]: 2026/09/20 10:54:27 INFO Signed narinfos id=64 count=16793worker2 # [ 184.579886] nix-grpc-daemon[1918]: time=2026-09-20T10:54:26.870Z level=INFO msg="Uploading 1 narinfos"6794lb # [ 184.585877] niks3-server[1039]: 2026/09/20 10:54:27 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst6795lb # [ 184.596253] niks3-server[1039]: 2026/09/20 10:54:27 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls6796lb # [ 184.607246] niks3-server[1039]: 2026/09/20 10:54:27 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6797lb # [ 184.611669] niks3-server[1039]: 2026/09/20 10:54:27 INFO Completed upload id=646798lb # [ 184.614050] niks3-server[1039]: 2026/09/20 10:54:27 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo6799worker2 # [ 184.608446] nix-grpc-daemon[1918]: time=2026-09-20T10:54:26.899Z level=INFO msg="Upload complete. (430ms)"6800worker2 # [ 184.613269] nix-grpc-daemon[1862]: ts=2026-09-20T10:54:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40514 duration_s=0 paths=1 nar_bytes_in=5126801worker2 # [ 184.617924] nix-grpc-daemon[1862]: ts=2026-09-20T10:54:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:405146802worker2 # [ 184.628249] nix-daemon[845]: accepted connection from pid 1862, user nix-grpc-daemon (trusted)6803worker2 # [ 184.632157] nix-daemon[845]: reaped child process 1916, status = succeeded6804worker2 # [ 184.637203] nix-daemon[845]: accepted connection from pid 1862, user nix-grpc-daemon (trusted)6805client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...6806client # farm-a> worker2-next: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6807client # farm-a> LOG-farm-a-drain6808worker2 # [ 186.060865] nix-daemon[845]: reaped child process 1943, status = succeeded6809worker2 # [ 186.217916] nix-daemon[845]: accepted connection from pid 1958, user nix-grpc-daemon (trusted)6810worker2 # [ 186.228475] nix-daemon[845]: reaped child process 1961, status = succeeded6811lb # [ 186.250754] niks3-server[1039]: 2026/09/20 10:54:29 INFO Received uploads request method=POST path=/api/pending_closures6812worker2 # [ 186.251488] nix-grpc-daemon[1918]: time=2026-09-20T10:54:28.542Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6813worker2 # [ 186.253065] nix-grpc-daemon[1918]: time=2026-09-20T10:54:28.544Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"6814lb # [ 186.274258] niks3-server[1039]: 2026/09/20 10:54:29 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6815lb # [ 186.277119] niks3-server[1039]: 2026/09/20 10:54:29 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6816lb # [ 186.281470] niks3-server[1039]: 2026/09/20 10:54:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6817lb # [ 186.283534] niks3-server[1039]: 2026/09/20 10:54:29 INFO Signed narinfos id=65 count=16818worker2 # [ 186.280535] nix-grpc-daemon[1918]: time=2026-09-20T10:54:28.571Z level=INFO msg="Uploading 1 narinfos"6819lb # [ 186.286300] niks3-server[1039]: 2026/09/20 10:54:29 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls6820lb # [ 186.292600] niks3-server[1039]: 2026/09/20 10:54:29 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6821lb # [ 186.295843] niks3-server[1039]: 2026/09/20 10:54:29 INFO Completed upload id=656822worker2 # [ 186.292124] nix-grpc-daemon[1918]: time=2026-09-20T10:54:28.583Z level=INFO msg="Upload complete. (246ms)"6823lb # [ 186.297355] niks3-server[1039]: 2026/09/20 10:54:29 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo6824worker2 # [ 186.294583] nix-grpc-daemon[1862]: ts=2026-09-20T10:54:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40514 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=16 outputs=16825worker2 # [ 186.300695] nix-daemon[845]: reaped child process 1941, status = succeeded6826worker1 # [ 186.327799] nix-grpc-daemon[1291]: ts=2026-09-20T10:54:29Z level=debug event=want client=7 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=06827worker1 # [ 186.332260] nix-grpc-daemon[1291]: ts=2026-09-20T10:54:29Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=16828worker2 # [ 186.346142] nix-grpc-daemon[1862]: ts=2026-09-20T10:54:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:405146829worker2 # [ 186.358982] nix-grpc-daemon[1862]: ts=2026-09-20T10:54:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:405146830worker2 # [ 186.372087] nix-grpc-daemon[1918]: time=2026-09-20T10:54:28.663Z level=INFO msg="All 1 paths already cached"6831client # copying 1 paths to 'lb:50051'...6832client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...6833worker2 # [ 186.380385] nix-grpc-daemon[1862]: ts=2026-09-20T10:54:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:405146834worker2 # [ 186.388634] nix-daemon[845]: accepted connection from pid 1862, user nix-grpc-daemon (trusted)6835worker2 # [ 186.557056] nix-daemon[845]: accepted connection from pid 1965, user nix-grpc-daemon (trusted)6836worker2 # [ 186.568674] nix-daemon[845]: reaped child process 1968, status = succeeded6837worker2 # [ 186.724688] nix-daemon[845]: accepted connection from pid 1970, user nix-grpc-daemon (trusted)6838worker2 # [ 186.735387] nix-daemon[845]: reaped child process 1973, status = succeeded6839lb # [ 186.755397] niks3-server[1039]: 2026/09/20 10:54:29 INFO Received uploads request method=POST path=/api/pending_closures6840worker2 # [ 186.759384] nix-grpc-daemon[1918]: time=2026-09-20T10:54:29.050Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6841worker2 # [ 186.760729] nix-grpc-daemon[1918]: time=2026-09-20T10:54:29.050Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"6842lb # [ 186.781953] niks3-server[1039]: 2026/09/20 10:54:29 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst6843lb # [ 186.795394] niks3-server[1039]: 2026/09/20 10:54:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6844lb # [ 186.801106] niks3-server[1039]: 2026/09/20 10:54:29 INFO Signed narinfos id=66 count=16845worker2 # [ 186.799729] nix-grpc-daemon[1918]: time=2026-09-20T10:54:29.090Z level=INFO msg="Uploading 1 narinfos"6846lb # [ 186.804890] niks3-server[1039]: 2026/09/20 10:54:29 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls6847lb # [ 186.818901] niks3-server[1039]: 2026/09/20 10:54:29 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6848lb # [ 186.825420] niks3-server[1039]: 2026/09/20 10:54:29 INFO Completed upload id=666849worker2 # [ 186.820519] nix-grpc-daemon[1918]: time=2026-09-20T10:54:29.111Z level=INFO msg="Upload complete. (419ms)"6850lb # [ 186.828310] niks3-server[1039]: 2026/09/20 10:54:29 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo6851worker2 # [ 186.824066] nix-grpc-daemon[1862]: ts=2026-09-20T10:54:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40514 duration_s=0 paths=1 nar_bytes_in=6246852worker2 # [ 186.830162] nix-grpc-daemon[1862]: ts=2026-09-20T10:54:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:405146853worker2 # [ 186.840502] nix-daemon[845]: accepted connection from pid 1862, user nix-grpc-daemon (trusted)6854worker2 # [ 186.843734] nix-daemon[845]: reaped child process 1963, status = succeeded6855worker2 # [ 186.849112] nix-daemon[845]: accepted connection from pid 1862, user nix-grpc-daemon (trusted)6856client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...6857client # farm-b> worker2-next: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6858client # farm-b> LOG-farm-b-drain6859worker2 # [ 187.669661] nix-daemon[845]: reaped child process 1977, status = succeeded6860worker2 # [ 187.797523] nix-daemon[845]: accepted connection from pid 1992, user nix-grpc-daemon (trusted)6861worker2 # [ 187.808976] nix-daemon[845]: reaped child process 1995, status = succeeded6862lb # [ 187.831592] niks3-server[1039]: 2026/09/20 10:54:30 INFO Received uploads request method=POST path=/api/pending_closures6863worker2 # [ 187.834387] nix-grpc-daemon[1918]: time=2026-09-20T10:54:30.125Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6864worker2 # [ 187.835890] nix-grpc-daemon[1918]: time=2026-09-20T10:54:30.127Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"6865lb # [ 187.871452] niks3-server[1039]: 2026/09/20 10:54:30 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst6866lb # [ 187.881126] niks3-server[1039]: 2026/09/20 10:54:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6867lb # [ 187.888909] niks3-server[1039]: 2026/09/20 10:54:30 INFO Signed narinfos id=67 count=16868worker2 # [ 187.887342] nix-grpc-daemon[1918]: time=2026-09-20T10:54:30.178Z level=INFO msg="Uploading 1 narinfos"6869lb # [ 187.895570] niks3-server[1039]: 2026/09/20 10:54:30 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls6870lb # [ 187.899796] niks3-server[1039]: 2026/09/20 10:54:30 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6871lb # [ 187.908989] niks3-server[1039]: 2026/09/20 10:54:30 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6872lb # [ 187.915546] niks3-server[1039]: 2026/09/20 10:54:30 INFO Completed upload id=676873worker2 # [ 187.910970] nix-grpc-daemon[1918]: time=2026-09-20T10:54:30.202Z level=INFO msg="Upload complete. (255ms)"6874lb # [ 187.917243] niks3-server[1039]: 2026/09/20 10:54:30 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo6875worker2 # [ 187.917133] nix-grpc-daemon[1862]: ts=2026-09-20T10:54:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40514 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=17 outputs=16876worker1 # [ 187.947982] nix-grpc-daemon[1291]: ts=2026-09-20T10:54:31Z level=debug event=want client=7 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=06877worker2 # [ 187.925187] nix-grpc-daemon[1862]: ts=2026-09-20T10:54:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:405146878worker1 # [ 187.952975] nix-grpc-daemon[1291]: ts=2026-09-20T10:54:31Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=16879worker2 # [ 187.937962] nix-daemon[845]: reaped child process 1975, status = succeeded6880worker2 # [ 187.940690] nix-grpc-daemon[1862]: ts=2026-09-20T10:54:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:405146881worker2 # [ 187.950237] nix-grpc-daemon[1918]: time=2026-09-20T10:54:30.241Z level=INFO msg="All 2 paths already cached"6882client # copying 1 paths to 'lb:50051'...6883client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...6884worker2 # [ 187.957332] nix-grpc-daemon[1862]: ts=2026-09-20T10:54:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:405146885worker2 # [ 187.963949] nix-daemon[845]: accepted connection from pid 1862, user nix-grpc-daemon (trusted)6886lb # [ 188.006807] envoy[910]: conn peer=[2001:db8:1::1]:47022 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=895 tx=498 ms=300466887worker2 # [ 188.121335] nix-daemon[845]: accepted connection from pid 1999, user nix-grpc-daemon (trusted)6888worker2 # [ 188.132929] nix-daemon[845]: reaped child process 2002, status = succeeded6889worker2 # [ 188.294691] nix-daemon[845]: accepted connection from pid 2004, user nix-grpc-daemon (trusted)6890worker2 # [ 188.306223] nix-daemon[845]: reaped child process 2007, status = succeeded6891lb # [ 188.326979] niks3-server[1039]: 2026/09/20 10:54:31 INFO Received uploads request method=POST path=/api/pending_closures6892worker2 # [ 188.330924] nix-grpc-daemon[1918]: time=2026-09-20T10:54:30.622Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6893worker2 # [ 188.332840] nix-grpc-daemon[1918]: time=2026-09-20T10:54:30.623Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"6894lb # [ 188.353615] niks3-server[1039]: 2026/09/20 10:54:31 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst6895lb # [ 188.357862] niks3-server[1039]: 2026/09/20 10:54:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6896lb # [ 188.359586] niks3-server[1039]: 2026/09/20 10:54:31 INFO Signed narinfos id=68 count=16897worker2 # [ 188.356126] nix-grpc-daemon[1918]: time=2026-09-20T10:54:30.647Z level=INFO msg="Uploading 1 narinfos"6898lb # [ 188.367316] niks3-server[1039]: 2026/09/20 10:54:31 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls6899lb # [ 188.369991] niks3-server[1039]: 2026/09/20 10:54:31 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6900lb # [ 188.373700] niks3-server[1039]: 2026/09/20 10:54:31 INFO Completed upload id=686901worker2 # [ 188.368903] nix-grpc-daemon[1918]: time=2026-09-20T10:54:30.660Z level=INFO msg="Upload complete. (395ms)"6902worker2 # [ 188.370812] nix-grpc-daemon[1862]: ts=2026-09-20T10:54:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40514 duration_s=0 paths=1 nar_bytes_in=7526903lb # [ 188.376166] niks3-server[1039]: 2026/09/20 10:54:31 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo6904worker2 # [ 188.373664] nix-grpc-daemon[1862]: ts=2026-09-20T10:54:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:405146905worker2 # [ 188.381476] nix-daemon[845]: accepted connection from pid 1862, user nix-grpc-daemon (trusted)6906worker2 # [ 188.383084] nix-daemon[845]: reaped child process 1997, status = succeeded6907client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...6908client # farm-top> worker2-next: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6909worker2 # [ 188.388148] nix-daemon[845]: accepted connection from pid 1862, user nix-grpc-daemon (trusted)6910worker2 # [ 189.599864] nix-daemon[845]: reaped child process 2011, status = succeeded6911client # farm-top> LOG-farm-top-drain6912worker2 # [ 189.747170] nix-daemon[845]: accepted connection from pid 2026, user nix-grpc-daemon (trusted)6913worker2 # [ 189.759219] nix-daemon[845]: reaped child process 2029, status = succeeded6914lb # [ 189.779897] niks3-server[1039]: 2026/09/20 10:54:32 INFO Received uploads request method=POST path=/api/pending_closures6915worker2 # [ 189.784357] nix-grpc-daemon[1918]: time=2026-09-20T10:54:32.075Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6916worker2 # [ 189.785817] nix-grpc-daemon[1918]: time=2026-09-20T10:54:32.077Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"6917lb # [ 189.814159] niks3-server[1039]: 2026/09/20 10:54:32 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6918lb # [ 189.830173] niks3-server[1039]: 2026/09/20 10:54:32 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst6919lb # [ 189.839614] niks3-server[1039]: 2026/09/20 10:54:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6920worker2 # [ 189.839489] nix-grpc-daemon[1918]: time=2026-09-20T10:54:32.130Z level=INFO msg="Uploading 1 narinfos"6921lb # [ 189.845129] niks3-server[1039]: 2026/09/20 10:54:32 INFO Signed narinfos id=69 count=16922lb # [ 189.849460] niks3-server[1039]: 2026/09/20 10:54:32 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls6923lb # [ 189.858890] niks3-server[1039]: 2026/09/20 10:54:32 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6924lb # [ 189.864108] niks3-server[1039]: 2026/09/20 10:54:32 INFO Completed upload id=696925worker2 # [ 189.859543] nix-grpc-daemon[1918]: time=2026-09-20T10:54:32.150Z level=INFO msg="Upload complete. (263ms)"6926lb # [ 189.866712] niks3-server[1039]: 2026/09/20 10:54:32 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo6927worker2 # [ 189.865662] nix-grpc-daemon[1862]: ts=2026-09-20T10:54:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40514 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=18 outputs=16928worker2 # [ 189.878181] nix-daemon[845]: reaped child process 2009, status = succeeded6929client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&2, in 6.20 seconds)6930client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath6931client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath, in 0.20 seconds)6932worker1: must succeed: rm /nix/.rw-store/fill6933worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.13 seconds)6934worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6935worker1 # [ 191.175079] nix-grpc-daemon[1291]: ts=2026-09-20T10:54:34Z level=info event=healthy reason=min_free6936worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.08 seconds)6937(finished: subtest: low disk drains a worker and builds go to the other, in 9.78 seconds)6938(finished: run the VM test script, in 192.27 seconds)6939test script finished in 192.47s6940cleanup6941kill QemuMachine (pid 49)6942client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6943kill QemuMachine (pid 50)6944lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6945kill QemuMachine (pid 51)6946worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6947kill QemuMachine (pid 52)6948worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6949(finished: cleanup, in 0.87 seconds)6950additionally exposed symbols:6951 client, lb, worker1, worker2,6952 vlan1,6953 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh