vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #547
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker2 # Disk image does not exist, creating the virtualisation disk image...13worker2: QEMU running (pid 52)14worker2 # Formatting '/build/vm-state-worker2/tmp.FBi8oysBIH', fmt=raw size=107374182415worker2 # mke2fs 1.47.4 (6-Mar-2025)16worker2 # Discarding device blocks: 0/262144 done17worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes18worker2 # Filesystem UUID: 99b73c50-b0c8-4fbd-a58c-cb4c1b03797119worker2 # Superblock backups stored on blocks:20worker2 # 32768, 98304, 163840, 22937621client # Disk image does not exist, creating the virtualisation disk image...22client: QEMU running (pid 49)23client # Formatting '/build/vm-state-client/tmp.pK5bNSNY26', fmt=raw size=107374182424worker2 # 25client # mke2fs 1.47.4 (6-Mar-2025)26worker2 # Allocating group tables: 0/8 done27client # Discarding device blocks: 0/262144 done28worker2 # Writing inode tables: 0/8 done29client # Creating filesystem with 262144 4k blocks and 65536 inodes30worker2 # Creating journal (8192 blocks): done31client # Filesystem UUID: 395dedcd-1a69-4569-8b1c-9b65c0bdba4632worker2 # Writing superblocks and filesystem accounting information: 0/8 done33client # Superblock backups stored on blocks:34worker2 # 35client # 32768, 98304, 163840, 22937636worker2 # Virtualisation disk image created.37client # 38worker2 # Starting virtiofs daemons...39client # Allocating group tables: 0/8 done40worker2 # [2026-09-23T15:59:29Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)41client # Writing inode tables: 0/8 done42worker2 # [2026-09-23T15:59:29Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether43client # Creating journal (8192 blocks): done44worker2 # [2026-09-23T15:59:29Z INFO virtiofsd] Waiting for vhost-user socket connection...45client # Writing superblocks and filesystem accounting information: 0/8 done46worker2 # [2026-09-23T15:59:29Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)47client # 48worker2 # [2026-09-23T15:59:29Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether49client # Virtualisation disk image created.50worker2 # [2026-09-23T15:59:29Z INFO virtiofsd] Waiting for vhost-user socket connection...51client # Starting virtiofs daemons...52worker2 # [2026-09-23T15:59:29Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)53client # [2026-09-23T15:59:29Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)54worker2 # [2026-09-23T15:59:29Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether55client # [2026-09-23T15:59:29Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether56worker2 # [2026-09-23T15:59:29Z INFO virtiofsd] Waiting for vhost-user socket connection...57client # [2026-09-23T15:59:29Z INFO virtiofsd] Waiting for vhost-user socket connection...58worker2 # [2026-09-23T15:59:29Z INFO virtiofsd] Client connected, servicing requests59client # [2026-09-23T15:59:29Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)60worker2 # [2026-09-23T15:59:29Z INFO virtiofsd] Client connected, servicing requests61client # [2026-09-23T15:59:29Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether62worker2 # [2026-09-23T15:59:29Z INFO virtiofsd] Client connected, servicing requests63client # [2026-09-23T15:59:29Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)64client # [2026-09-23T15:59:29Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether65client # [2026-09-23T15:59:29Z INFO virtiofsd] Waiting for vhost-user socket connection...66client # [2026-09-23T15:59:29Z INFO virtiofsd] Waiting for vhost-user socket connection...67client # [2026-09-23T15:59:29Z INFO virtiofsd] Client connected, servicing requests68client # [2026-09-23T15:59:29Z INFO virtiofsd] Client connected, servicing requests69client # [2026-09-23T15:59:29Z INFO virtiofsd] Client connected, servicing requests70lb # Disk image does not exist, creating the virtualisation disk image...71lb: QEMU running (pid 50)72lb # Formatting '/build/vm-state-lb/tmp.bBZe9ifhjK', fmt=raw size=107374182473lb # mke2fs 1.47.4 (6-Mar-2025)74lb # Discarding device blocks: 0/262144 done75lb # Creating filesystem with 262144 4k blocks and 65536 inodes76lb # Filesystem UUID: 0462966c-9ade-45a3-b61a-be0249fbfa1177lb # Superblock backups stored on blocks:78lb # 32768, 98304, 163840, 22937679lb # 80lb # Allocating group tables: 0/8 done81lb # Writing inode tables: 0/8 done82lb # Creating journal (8192 blocks): done83lb # Writing superblocks and filesystem accounting information: 0/8 done84lb # 85lb # Virtualisation disk image created.86lb # Starting virtiofs daemons...87lb # [2026-09-23T15:59:29Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)88lb # [2026-09-23T15:59:29Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether89lb # [2026-09-23T15:59:29Z INFO virtiofsd] Waiting for vhost-user socket connection...90lb # [2026-09-23T15:59:29Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)91lb # [2026-09-23T15:59:29Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether92lb # [2026-09-23T15:59:29Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)93lb # [2026-09-23T15:59:29Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether94lb # [2026-09-23T15:59:29Z INFO virtiofsd] Waiting for vhost-user socket connection...95lb # [2026-09-23T15:59:29Z INFO virtiofsd] Waiting for vhost-user socket connection...96lb # [2026-09-23T15:59:29Z INFO virtiofsd] Client connected, servicing requests97lb # [2026-09-23T15:59:29Z INFO virtiofsd] Client connected, servicing requests98lb # [2026-09-23T15:59:29Z INFO virtiofsd] Client connected, servicing requests99worker1: QEMU running (pid 51)100worker1 # Disk image does not exist, creating the virtualisation disk image...101(finished: start all VMs, in 0.26 seconds)102worker1 # Formatting '/build/vm-state-worker1/tmp.XE6rDlsdpV', fmt=raw size=1073741824103client: waiting for unit network-addresses-eth1.service104worker1 # mke2fs 1.47.4 (6-Mar-2025)105worker1 # Discarding device blocks: 0/262144 done106worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes107worker1 # Filesystem UUID: b86d2bb0-fb2f-4811-a307-f0d372f4dda3108client: waiting for the VM to finish booting109worker1 # Superblock backups stored on blocks:110worker1 # 32768, 98304, 163840, 229376111worker1 # 112worker1 # Allocating group tables: 0/8 done113worker1 # Writing inode tables: 0/8 done114worker1 # Creating journal (8192 blocks): done115worker1 # Writing superblocks and filesystem accounting information: 0/8 done116worker1 # 117worker1 # Virtualisation disk image created.118worker1 # Starting virtiofs daemons...119worker1 # [2026-09-23T15:59:29Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120worker1 # [2026-09-23T15:59:29Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether121worker1 # [2026-09-23T15:59:29Z INFO virtiofsd] Waiting for vhost-user socket connection...122worker1 # [2026-09-23T15:59:29Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)123worker1 # [2026-09-23T15:59:29Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether124worker1 # [2026-09-23T15:59:29Z INFO virtiofsd] Waiting for vhost-user socket connection...125worker1 # [2026-09-23T15:59:29Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)126worker1 # [2026-09-23T15:59:29Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether127worker1 # [2026-09-23T15:59:29Z INFO virtiofsd] Waiting for vhost-user socket connection...128worker1 # [2026-09-23T15:59:29Z INFO virtiofsd] Client connected, servicing requests129worker1 # [2026-09-23T15:59:29Z INFO virtiofsd] Client connected, servicing requests130worker1 # [2026-09-23T15:59:29Z INFO virtiofsd] Client connected, servicing requests131worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)132worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)133client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)134lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)135worker2 # 136worker2 # 137worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00138worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...139worker2 # 140worker2 # 141worker2 # 142worker2 # 143worker2 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00144worker2 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...145worker2 # 146worker2 # 147worker1 # 148worker1 # 149worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00150worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...151worker1 # 152worker1 # 153worker1 # 154worker1 # 155worker1 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00156client # 157client # 158worker1 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...159client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00160worker1 # 161worker1 # 162lb # 163lb # 164client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...165client # 166client # 167client # 168client # 169lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00170client # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00171lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...172lb # 173lb # 174lb # 175lb # 176lb # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00177client # Press Ctrl-B to configure iPXE (PCI 00:05.0)...178client # 179client # 180worker2 # Booting from ROM...181lb # Press Ctrl-B to configure iPXE (PCI 00:05.0)...182lb # 183lb # 184worker1 # Booting from ROM...185client # Booting from ROM...186lb # Booting from ROM...187worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026188worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ryyw8nza55ayn45yacz76nd2ajf36w90-nixos-system-worker2-test/init regInfo=/nix/store/b4kcswdscn3288f20kcibxgb5i0sxv2l-closure-info/registration console=ttyS0,115200n8 console=tty0189worker2 # [ 0.000000] BIOS-provided physical RAM map:190worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable191worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved192worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved193worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable194worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved195worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved196worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved197worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved198worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved199worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved200worker2 # [ 0.000000] NX (Execute Disable) protection: active201worker2 # [ 0.000000] APIC: Static calls initialized202worker2 # [ 0.000000] SMBIOS 2.8 present.203worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014204worker2 # [ 0.000000] DMI: Memory slots populated: 1/1205worker2 # [ 0.000000] Hypervisor detected: KVM206worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000207worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00208worker2 # [ 0.000000] kvm-clock: using sched offset of 464506469 cycles209worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns210worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor211worker2 # [ 0.000809] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000212worker2 # [ 0.000837] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs213worker2 # [ 0.000840] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT214worker2 # [ 0.002733] found SMP MP-table at [mem 0x000f5450-0x000f545f]215worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026216worker2 # [ 0.002744] Using GB pages for direct mapping217worker2 # [ 0.002824] RAMDISK: [mem 0x5e36f000-0x5ffcffff]218worker2 # [ 0.002833] ACPI: Early table checksum verification disabled219worker2 # [ 0.002836] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )220worker2 # [ 0.002839] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)221worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/x5xm43clszqqn1s52xaz4fgg5ff2y8g4-nixos-system-worker1-test/init regInfo=/nix/store/7qz9sz7kdv8kd3dah4cq47kqnn15rajm-closure-info/registration console=ttyS0,115200n8 console=tty0222worker2 # [ 0.002843] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)223worker1 # [ 0.000000] BIOS-provided physical RAM map:224worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable225worker2 # [ 0.002849] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)226worker2 # [ 0.002851] ACPI: FACS 0x000000005FFE0000 000040227worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved228worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved229worker2 # [ 0.002853] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)230worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable231worker2 # [ 0.002854] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)232worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved233worker2 # [ 0.002856] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)234worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved235worker2 # [ 0.002857] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)236worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved237worker2 # [ 0.002859] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]238worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved239client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026240worker2 # [ 0.002860] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]241worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved242worker2 # [ 0.002860] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]243worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved244worker1 # [ 0.000000] NX (Execute Disable) protection: active245worker2 # [ 0.002861] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]246worker1 # [ 0.000000] APIC: Static calls initialized247worker1 # [ 0.000000] SMBIOS 2.8 present.248worker2 # [ 0.002861] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]249worker2 # [ 0.002862] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]250client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/hcd7c1gssanjyr6daz78amrlyss1f6ps-nixos-system-client-test/init regInfo=/nix/store/ahb1sfxh6ggw7bfwmy5j8nfvv6w9sgga-closure-info/registration console=ttyS0,115200n8 console=tty0251worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014252client # [ 0.000000] BIOS-provided physical RAM map:253worker1 # [ 0.000000] DMI: Memory slots populated: 1/1254worker2 # [ 0.002862] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]255worker2 # [ 0.003085] No NUMA configuration found256worker1 # [ 0.000000] Hypervisor detected: KVM257client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable258worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000259worker2 # [ 0.003086] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]260client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved261lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026262worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00263worker2 # [ 0.003088] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]264worker2 # [ 0.005317] Zone ranges:265worker1 # [ 0.000000] kvm-clock: using sched offset of 468131861 cycles266client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved267worker2 # [ 0.005318] DMA [mem 0x0000000000001000-0x0000000000ffffff]268client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable269worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns270worker2 # [ 0.005319] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]271worker2 # [ 0.005320] Normal empty272client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved273worker1 # [ 0.000004] tsc: Detected 2400.012 MHz processor274worker2 # [ 0.005321] Device empty275client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved276worker2 # [ 0.005322] Movable zone start for each node277worker1 # [ 0.000808] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000278lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/b57s96hna1n87xmygcaz6qx4i7f2zyzx-nixos-system-lb-test/init regInfo=/nix/store/jpd2qccg07hyc3sd9msmabx1jknjxkc6-closure-info/registration console=ttyS0,115200n8 console=tty0279worker2 # [ 0.005323] Early memory node ranges280client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved281lb # [ 0.000000] BIOS-provided physical RAM map:282worker1 # [ 0.000835] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs283worker2 # [ 0.005323] node 0: [mem 0x0000000000001000-0x000000000009efff]284lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable285client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved286worker1 # [ 0.000838] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT287worker2 # [ 0.005324] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]288client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved289lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved290worker1 # [ 0.002734] found SMP MP-table at [mem 0x000f5450-0x000f545f]291worker2 # [ 0.005325] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]292worker1 # [ 0.002745] Using GB pages for direct mapping293lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved294client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved295worker2 # [ 0.005344] On node 0, zone DMA: 1 pages in unavailable ranges296worker1 # [ 0.002869] RAMDISK: [mem 0x5e370000-0x5ffcffff]297client # [ 0.000000] NX (Execute Disable) protection: active298lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable299worker2 # [ 0.005599] On node 0, zone DMA: 97 pages in unavailable ranges300worker1 # [ 0.002879] ACPI: Early table checksum verification disabled301client # [ 0.000000] APIC: Static calls initialized302client # [ 0.000000] SMBIOS 2.8 present.303worker2 # [ 0.031167] On node 0, zone DMA32: 40 pages in unavailable ranges304lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved305worker1 # [ 0.002881] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )306worker2 # [ 0.031625] ACPI: PM-Timer IO Port: 0x608307lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved308client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014309worker2 # [ 0.031635] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])310worker1 # [ 0.002885] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)311client # [ 0.000000] DMI: Memory slots populated: 1/1312lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved313worker2 # [ 0.031661] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23314client # [ 0.000000] Hypervisor detected: KVM315worker1 # [ 0.002889] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)316lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved317worker2 # [ 0.031663] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)318client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000319worker1 # [ 0.002896] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)320client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00321lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved322worker2 # [ 0.031665] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)323worker1 # [ 0.002898] ACPI: FACS 0x000000005FFE0000 000040324client # [ 0.000001] kvm-clock: using sched offset of 497489634 cycles325lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved326worker2 # [ 0.031666] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)327worker1 # [ 0.002899] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)328lb # [ 0.000000] NX (Execute Disable) protection: active329worker2 # [ 0.031667] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)330lb # [ 0.000000] APIC: Static calls initialized331client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns332lb # [ 0.000000] SMBIOS 2.8 present.333worker1 # [ 0.002901] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)334worker2 # [ 0.031668] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)335client # [ 0.000005] tsc: Detected 2400.012 MHz processor336client # [ 0.000811] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000337worker2 # [ 0.031670] ACPI: Using ACPI (MADT) for SMP configuration information338worker1 # [ 0.002903] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)339lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014340worker2 # [ 0.031671] ACPI: HPET id: 0x8086a201 base: 0xfed00000341lb # [ 0.000000] DMI: Memory slots populated: 1/1342client # [ 0.000837] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs343worker2 # [ 0.031674] TSC deadline timer available344worker1 # [ 0.002904] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)345lb # [ 0.000000] Hypervisor detected: KVM346worker2 # [ 0.031678] CPU topo: Max. logical packages: 1347client # [ 0.000840] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT348lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000349worker1 # [ 0.002905] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]350worker2 # [ 0.031679] CPU topo: Max. logical dies: 1351client # [ 0.002721] found SMP MP-table at [mem 0x000f5450-0x000f545f]352lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00353worker2 # [ 0.031679] CPU topo: Max. dies per package: 1354worker1 # [ 0.002907] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]355client # [ 0.002732] Using GB pages for direct mapping356lb # [ 0.000000] kvm-clock: using sched offset of 497082595 cycles357worker2 # [ 0.031682] CPU topo: Max. threads per core: 1358client # [ 0.002886] RAMDISK: [mem 0x5e370000-0x5ffcffff]359worker1 # [ 0.002907] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]360worker2 # [ 0.031683] CPU topo: Num. cores per package: 1361client # [ 0.002897] ACPI: Early table checksum verification disabled362worker2 # [ 0.031683] CPU topo: Num. threads per package: 1363lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns364worker1 # [ 0.002908] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]365client # [ 0.002900] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )366worker2 # [ 0.031684] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs367lb # [ 0.000004] tsc: Detected 2400.012 MHz processor368worker1 # [ 0.002908] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]369lb # [ 0.000810] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000370client # [ 0.002904] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)371worker2 # [ 0.031701] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()372worker1 # [ 0.002908] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]373lb # [ 0.000836] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs374worker2 # [ 0.031730] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]375client # [ 0.002908] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)376worker1 # [ 0.002909] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]377lb # [ 0.000839] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT378worker1 # [ 0.003140] No NUMA configuration found379worker2 # [ 0.031731] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]380client # [ 0.002916] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)381lb # [ 0.002724] found SMP MP-table at [mem 0x000f5450-0x000f545f]382worker1 # [ 0.003141] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]383worker2 # [ 0.031732] [mem 0x60000000-0xafffffff] available for PCI devices384client # [ 0.002919] ACPI: FACS 0x000000005FFE0000 000040385lb # [ 0.002734] Using GB pages for direct mapping386worker1 # [ 0.003144] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]387worker2 # [ 0.031734] Booting paravirtualized kernel on KVM388lb # [ 0.002863] RAMDISK: [mem 0x5e36e000-0x5ffcffff]389worker1 # [ 0.005417] Zone ranges:390client # [ 0.002920] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)391lb # [ 0.002872] ACPI: Early table checksum verification disabled392worker1 # [ 0.005418] DMA [mem 0x0000000000001000-0x0000000000ffffff]393worker2 # [ 0.031736] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns394client # [ 0.002922] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)395lb # [ 0.002874] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )396worker1 # [ 0.005419] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]397worker2 # [ 0.036194] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1398worker1 # [ 0.005420] Normal empty399worker1 # [ 0.005421] Device empty400client # [ 0.002923] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)401lb # [ 0.002878] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)402worker2 # [ 0.038456] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152403worker1 # [ 0.005421] Movable zone start for each node404worker1 # [ 0.005422] Early memory node ranges405worker2 # [ 0.038498] kvm-guest: PV spinlocks disabled, single CPU406client # [ 0.002925] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)407lb # [ 0.002883] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)408worker1 # [ 0.005423] node 0: [mem 0x0000000000001000-0x000000000009efff]409client # [ 0.002926] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]410lb # [ 0.002890] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)411worker1 # [ 0.005424] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]412client # [ 0.002927] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]413lb # [ 0.002892] ACPI: FACS 0x000000005FFE0000 000040414worker1 # [ 0.005425] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]415client # [ 0.002928] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]416lb # [ 0.002894] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)417worker1 # [ 0.005445] On node 0, zone DMA: 1 pages in unavailable ranges418client # [ 0.002928] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]419worker1 # [ 0.005705] On node 0, zone DMA: 97 pages in unavailable ranges420worker2 # [ 0.038500] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ryyw8nza55ayn45yacz76nd2ajf36w90-nixos-system-worker2-test/init regInfo=/nix/store/b4kcswdscn3288f20kcibxgb5i0sxv2l-closure-info/registration console=ttyS0,115200n8 console=tty0421lb # [ 0.002895] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)422client # [ 0.002929] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]423worker1 # [ 0.032102] On node 0, zone DMA32: 40 pages in unavailable ranges424lb # [ 0.002897] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)425worker1 # [ 0.032562] ACPI: PM-Timer IO Port: 0x608426client # [ 0.002929] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]427worker2 # [ 0.038591] Unknown kernel command line parameters "regInfo=/nix/store/b4kcswdscn3288f20kcibxgb5i0sxv2l-closure-info/registration", will be passed to user space.428worker1 # [ 0.032573] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])429worker2 # [ 0.038604] random: crng init done430lb # [ 0.002898] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)431client # [ 0.002930] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]432worker1 # [ 0.032600] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23433client # [ 0.003161] No NUMA configuration found434lb # [ 0.002900] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]435worker2 # [ 0.038605] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes436worker1 # [ 0.032603] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)437client # [ 0.003162] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]438lb # [ 0.002901] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]439worker2 # [ 0.040658] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)440client # [ 0.003164] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]441worker1 # [ 0.032604] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)442client # [ 0.005443] Zone ranges:443lb # [ 0.002901] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]444worker2 # [ 0.041691] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)445worker1 # [ 0.032605] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)446client # [ 0.005444] DMA [mem 0x0000000000001000-0x0000000000ffffff]447worker2 # [ 0.041721] Fallback order for Node 0: 0448lb # [ 0.002902] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]449worker1 # [ 0.032607] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)450client # [ 0.005447] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]451worker2 # [ 0.041724] Built 1 zonelists, mobility grouping on. Total pages: 393078452client # [ 0.005448] Normal empty453lb # [ 0.002902] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]454worker2 # [ 0.041725] Policy zone: DMA32455worker1 # [ 0.032607] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)456client # [ 0.005449] Device empty457lb # [ 0.002903] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]458client # [ 0.005449] Movable zone start for each node459worker2 # [ 0.044128] mem auto-init: stack:all(zero), heap alloc:on, heap free:off460worker1 # [ 0.032610] ACPI: Using ACPI (MADT) for SMP configuration information461client # [ 0.005450] Early memory node ranges462worker1 # [ 0.032611] ACPI: HPET id: 0x8086a201 base: 0xfed00000463lb # [ 0.002903] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]464worker2 # [ 0.047080] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1465client # [ 0.005451] node 0: [mem 0x0000000000001000-0x000000000009efff]466worker1 # [ 0.032614] TSC deadline timer available467lb # [ 0.003127] No NUMA configuration found468worker2 # [ 0.050483] allocated 3145728 bytes of page_ext469worker1 # [ 0.032618] CPU topo: Max. logical packages: 1470client # [ 0.005452] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]471worker2 # [ 0.059995] ftrace: allocating 48787 entries in 192 pages472lb # [ 0.003128] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]473worker1 # [ 0.032619] CPU topo: Max. logical dies: 1474worker2 # [ 0.059997] ftrace: allocated 192 pages with 2 groups475client # [ 0.005453] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]476lb # [ 0.003131] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]477worker1 # [ 0.032619] CPU topo: Max. dies per package: 1478worker2 # [ 0.060812] Dynamic Preempt: lazy479lb # [ 0.005415] Zone ranges:480client # [ 0.005472] On node 0, zone DMA: 1 pages in unavailable ranges481worker1 # [ 0.032622] CPU topo: Max. threads per core: 1482worker2 # [ 0.060963] rcu: Preemptible hierarchical RCU implementation.483lb # [ 0.005416] DMA [mem 0x0000000000001000-0x0000000000ffffff]484worker1 # [ 0.032623] CPU topo: Num. cores per package: 1485client # [ 0.005737] On node 0, zone DMA: 97 pages in unavailable ranges486worker2 # [ 0.060964] rcu: RCU event tracing is enabled.487worker1 # [ 0.032623] CPU topo: Num. threads per package: 1488lb # [ 0.005418] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]489client # [ 0.032036] On node 0, zone DMA32: 40 pages in unavailable ranges490lb # [ 0.005419] Normal empty491worker2 # [ 0.060965] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.492lb # [ 0.005420] Device empty493worker1 # [ 0.032624] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs494client # [ 0.032498] ACPI: PM-Timer IO Port: 0x608495worker2 # [ 0.060966] Trampoline variant of Tasks RCU enabled.496lb # [ 0.005420] Movable zone start for each node497client # [ 0.032510] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])498worker2 # [ 0.060967] Rude variant of Tasks RCU enabled.499lb # [ 0.005421] Early memory node ranges500worker1 # [ 0.032642] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()501worker2 # [ 0.060967] Tracing variant of Tasks RCU enabled.502client # [ 0.032539] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23503lb # [ 0.005421] node 0: [mem 0x0000000000001000-0x000000000009efff]504worker1 # [ 0.032674] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]505worker2 # [ 0.060968] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.506client # [ 0.032542] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)507lb # [ 0.005422] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]508worker1 # [ 0.032676] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]509worker2 # [ 0.060968] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1510lb # [ 0.005423] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]511client # [ 0.032544] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)512worker1 # [ 0.032677] [mem 0x60000000-0xafffffff] available for PCI devices513worker1 # [ 0.032679] Booting paravirtualized kernel on KVM514lb # [ 0.005443] On node 0, zone DMA: 1 pages in unavailable ranges515worker2 # [ 0.061024] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.516client # [ 0.032551] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)517lb # [ 0.005710] On node 0, zone DMA: 97 pages in unavailable ranges518client # [ 0.032552] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)519worker2 # [ 0.061026] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.520worker1 # [ 0.032682] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns521lb # [ 0.031982] On node 0, zone DMA32: 40 pages in unavailable ranges522client # [ 0.032553] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)523lb # [ 0.032443] ACPI: PM-Timer IO Port: 0x608524worker1 # [ 0.037149] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1525worker2 # [ 0.061027] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.526client # [ 0.032555] ACPI: Using ACPI (MADT) for SMP configuration information527lb # [ 0.032455] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])528worker1 # [ 0.039286] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152529worker2 # [ 0.065345] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16530client # [ 0.032556] ACPI: HPET id: 0x8086a201 base: 0xfed00000531lb # [ 0.032488] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23532client # [ 0.032560] TSC deadline timer available533worker1 # [ 0.039330] kvm-guest: PV spinlocks disabled, single CPU534worker2 # [ 0.065622] rcu: srcu_init: Setting srcu_struct sizes based on contention.535client # [ 0.032564] CPU topo: Max. logical packages: 1536lb # [ 0.032490] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)537client # [ 0.032564] CPU topo: Max. logical dies: 1538worker2 # [ 0.065628] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns539lb # [ 0.032492] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)540client # [ 0.032565] CPU topo: Max. dies per package: 1541client # [ 0.032568] CPU topo: Max. threads per core: 1542lb # [ 0.032494] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)543worker2 # [ 0.065726] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)544client # [ 0.032568] CPU topo: Num. cores per package: 1545worker2 # [ 0.069190] Console: colour VGA+ 80x25546lb # [ 0.032495] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)547client # [ 0.032569] CPU topo: Num. threads per package: 1548worker2 # [ 0.069193] printk: legacy console [tty0] enabled549worker1 # [ 0.039331] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/x5xm43clszqqn1s52xaz4fgg5ff2y8g4-nixos-system-worker1-test/init regInfo=/nix/store/7qz9sz7kdv8kd3dah4cq47kqnn15rajm-closure-info/registration console=ttyS0,115200n8 console=tty0550lb # [ 0.032495] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)551client # [ 0.032569] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs552worker2 # [ 0.098678] printk: legacy console [ttyS0] enabled553lb # [ 0.032498] ACPI: Using ACPI (MADT) for SMP configuration information554client # [ 0.032588] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()555worker2 # [ 0.197323] ACPI: Core revision 20250807556lb # [ 0.032499] ACPI: HPET id: 0x8086a201 base: 0xfed00000557worker1 # [ 0.039425] Unknown kernel command line parameters "regInfo=/nix/store/7qz9sz7kdv8kd3dah4cq47kqnn15rajm-closure-info/registration", will be passed to user space.558lb # [ 0.032502] TSC deadline timer available559worker1 # [ 0.039437] random: crng init done560client # [ 0.032623] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]561lb # [ 0.032506] CPU topo: Max. logical packages: 1562worker2 # [ 0.198191] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns563client # [ 0.032624] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]564worker1 # [ 0.039438] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes565lb # [ 0.032507] CPU topo: Max. logical dies: 1566lb # [ 0.032507] CPU topo: Max. dies per package: 1567client # [ 0.032626] [mem 0x60000000-0xafffffff] available for PCI devices568worker2 # [ 0.199701] APIC: Switch to symmetric I/O mode setup569worker1 # [ 0.041536] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)570lb # [ 0.032510] CPU topo: Max. threads per core: 1571client # [ 0.032627] Booting paravirtualized kernel on KVM572worker2 # [ 0.200665] x2apic enabled573lb # [ 0.032511] CPU topo: Num. cores per package: 1574worker1 # [ 0.042777] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)575worker1 # [ 0.042808] Fallback order for Node 0: 0576lb # [ 0.032511] CPU topo: Num. threads per package: 1577worker2 # [ 0.201402] APIC: Switched APIC routing to: physical x2apic578client # [ 0.032630] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns579lb # [ 0.032512] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs580worker1 # [ 0.042811] Built 1 zonelists, mobility grouping on. Total pages: 393078581client # [ 0.037149] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1582worker1 # [ 0.042812] Policy zone: DMA32583lb # [ 0.032530] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()584client # [ 0.039296] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152585worker1 # [ 0.045281] mem auto-init: stack:all(zero), heap alloc:on, heap free:off586worker2 # [ 0.203195] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1587lb # [ 0.032559] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]588client # [ 0.039341] kvm-guest: PV spinlocks disabled, single CPU589worker1 # [ 0.048420] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1590worker1 # [ 0.051703] allocated 3145728 bytes of page_ext591lb # [ 0.032561] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]592worker2 # [ 0.204162] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns593worker1 # [ 0.061625] ftrace: allocating 48787 entries in 192 pages594lb # [ 0.032562] [mem 0x60000000-0xafffffff] available for PCI devices595worker1 # [ 0.061627] ftrace: allocated 192 pages with 2 groups596lb # [ 0.032564] Booting paravirtualized kernel on KVM597worker1 # [ 0.062456] Dynamic Preempt: lazy598worker2 # [ 0.205807] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)599worker1 # [ 0.062593] rcu: Preemptible hierarchical RCU implementation.600lb # [ 0.032566] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns601worker2 # [ 0.206890] x86/cpu: User Mode Instruction Prevention (UMIP) activated602worker1 # [ 0.062594] rcu: RCU event tracing is enabled.603client # [ 0.039343] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/hcd7c1gssanjyr6daz78amrlyss1f6ps-nixos-system-client-test/init regInfo=/nix/store/ahb1sfxh6ggw7bfwmy5j8nfvv6w9sgga-closure-info/registration console=ttyS0,115200n8 console=tty0604lb # [ 0.037033] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1605worker1 # [ 0.062594] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.606worker2 # [ 0.207925] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127607worker1 # [ 0.062596] Trampoline variant of Tasks RCU enabled.608lb # [ 0.039240] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152609client # [ 0.039437] Unknown kernel command line parameters "regInfo=/nix/store/ahb1sfxh6ggw7bfwmy5j8nfvv6w9sgga-closure-info/registration", will be passed to user space.610worker1 # [ 0.062596] Rude variant of Tasks RCU enabled.611worker2 # [ 0.208804] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0612lb # [ 0.039285] kvm-guest: PV spinlocks disabled, single CPU613client # [ 0.039449] random: crng init done614worker1 # [ 0.062596] Tracing variant of Tasks RCU enabled.615client # [ 0.039449] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes616worker1 # [ 0.062597] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.617worker2 # [ 0.209808] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto618client # [ 0.041569] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)619worker1 # [ 0.062598] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1620client # [ 0.042659] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)621worker2 # [ 0.210804] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl622worker1 # [ 0.062619] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.623client # [ 0.042689] Fallback order for Node 0: 0624lb # [ 0.039287] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/b57s96hna1n87xmygcaz6qx4i7f2zyzx-nixos-system-lb-test/init regInfo=/nix/store/jpd2qccg07hyc3sd9msmabx1jknjxkc6-closure-info/registration console=ttyS0,115200n8 console=tty0625worker2 # [ 0.211804] Transient Scheduler Attacks: Vulnerable: No microcode626client # [ 0.042692] Built 1 zonelists, mobility grouping on. Total pages: 393078627worker1 # [ 0.062621] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.628client # [ 0.042693] Policy zone: DMA32629worker2 # [ 0.212804] Spectre V2 : Mitigation: Enhanced / Automatic IBRS630client # [ 0.045329] mem auto-init: stack:all(zero), heap alloc:on, heap free:off631worker1 # [ 0.062621] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.632lb # [ 0.039380] Unknown kernel command line parameters "regInfo=/nix/store/jpd2qccg07hyc3sd9msmabx1jknjxkc6-closure-info/registration", will be passed to user space.633lb # [ 0.039393] random: crng init done634worker2 # [ 0.213804] Speculative Return Stack Overflow: Mitigation: Safe RET635worker1 # [ 0.067225] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16636client # [ 0.048431] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1637client # [ 0.051893] allocated 3145728 bytes of page_ext638lb # [ 0.039394] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes639worker1 # [ 0.067492] rcu: srcu_init: Setting srcu_struct sizes based on contention.640client # [ 0.061849] ftrace: allocating 48787 entries in 192 pages641lb # [ 0.041505] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)642client # [ 0.061851] ftrace: allocated 192 pages with 2 groups643worker1 # [ 0.067500] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns644client # [ 0.062680] Dynamic Preempt: lazy645lb # [ 0.042737] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)646client # [ 0.062819] rcu: Preemptible hierarchical RCU implementation.647lb # [ 0.042771] Fallback order for Node 0: 0648worker1 # [ 0.067603] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)649client # [ 0.062820] rcu: RCU event tracing is enabled.650worker1 # [ 0.071088] Console: colour VGA+ 80x25651lb # [ 0.042774] Built 1 zonelists, mobility grouping on. Total pages: 393078652worker2 # [ 0.214804] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization653lb # [ 0.042775] Policy zone: DMA32654worker1 # [ 0.071091] printk: legacy console [tty0] enabled655client # [ 0.062821] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.656worker2 # [ 0.216811] Spectre V2 : Enabling IBPB for BPF657worker1 # [ 0.100508] printk: legacy console [ttyS0] enabled658client # [ 0.062822] Trampoline variant of Tasks RCU enabled.659lb # [ 0.045252] mem auto-init: stack:all(zero), heap alloc:on, heap free:off660client # [ 0.062823] Rude variant of Tasks RCU enabled.661worker1 # [ 0.197847] ACPI: Core revision 20250807662lb # [ 0.048352] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1663worker2 # [ 0.217805] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier664client # [ 0.062823] Tracing variant of Tasks RCU enabled.665lb # [ 0.051644] allocated 3145728 bytes of page_ext666worker2 # [ 0.218805] active return thunk: srso_alias_return_thunk667client # [ 0.062824] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.668lb # [ 0.061506] ftrace: allocating 48787 entries in 192 pages669worker1 # [ 0.198703] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns670lb # [ 0.061508] ftrace: allocated 192 pages with 2 groups671client # [ 0.062825] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1672lb # [ 0.062321] Dynamic Preempt: lazy673worker1 # [ 0.200198] APIC: Switch to symmetric I/O mode setup674worker2 # [ 0.219825] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'675lb # [ 0.062512] rcu: Preemptible hierarchical RCU implementation.676client # [ 0.062880] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.677worker1 # [ 0.201165] x2apic enabled678worker2 # [ 0.220804] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'679lb # [ 0.062513] rcu: RCU event tracing is enabled.680client # [ 0.062882] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.681worker1 # [ 0.201932] APIC: Switched APIC routing to: physical x2apic682lb # [ 0.062513] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.683worker2 # [ 0.221804] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'684lb # [ 0.062515] Trampoline variant of Tasks RCU enabled.685client # [ 0.062883] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.686worker2 # [ 0.222804] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'687lb # [ 0.062515] Rude variant of Tasks RCU enabled.688client # [ 0.067248] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16689lb # [ 0.062516] Tracing variant of Tasks RCU enabled.690worker1 # [ 0.203751] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1691worker2 # [ 0.223804] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'692client # [ 0.067517] rcu: srcu_init: Setting srcu_struct sizes based on contention.693lb # [ 0.062516] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.694worker2 # [ 0.224804] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'695lb # [ 0.062517] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1696worker1 # [ 0.204730] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns697client # [ 0.067525] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns698worker2 # [ 0.225804] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'699lb # [ 0.062538] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.700client # [ 0.067635] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)701worker1 # [ 0.206408] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)702client # [ 0.071184] Console: colour VGA+ 80x25703worker2 # [ 0.226804] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256704lb # [ 0.062540] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.705client # [ 0.071186] printk: legacy console [tty0] enabled706worker1 # [ 0.207493] x86/cpu: User Mode Instruction Prevention (UMIP) activated707worker2 # [ 0.227804] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64708client # [ 0.100693] printk: legacy console [ttyS0] enabled709lb # [ 0.062541] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.710worker1 # [ 0.208535] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127711client # [ 0.197852] ACPI: Core revision 20250807712worker2 # [ 0.228804] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512713lb # [ 0.067060] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16714worker1 # [ 0.209405] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0715lb # [ 0.067336] rcu: srcu_init: Setting srcu_struct sizes based on contention.716worker2 # [ 0.229804] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024717client # [ 0.198750] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns718worker2 # [ 0.230804] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8719lb # [ 0.067343] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns720worker1 # [ 0.210409] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto721client # [ 0.200257] APIC: Switch to symmetric I/O mode setup722client # [ 0.201227] x2apic enabled723lb # [ 0.067445] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)724worker2 # [ 0.231804] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.725lb # [ 0.071009] Console: colour VGA+ 80x25726worker1 # [ 0.212405] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl727client # [ 0.201999] APIC: Switched APIC routing to: physical x2apic728lb # [ 0.071012] printk: legacy console [tty0] enabled729worker1 # [ 0.213405] Transient Scheduler Attacks: Vulnerable: No microcode730lb # [ 0.100486] printk: legacy console [ttyS0] enabled731lb # [ 0.197503] ACPI: Core revision 20250807732worker1 # [ 0.214404] Spectre V2 : Mitigation: Enhanced / Automatic IBRS733client # [ 0.203812] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1734worker1 # [ 0.215405] Speculative Return Stack Overflow: Mitigation: Safe RET735lb # [ 0.198440] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns736client # [ 0.204870] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns737lb # [ 0.199999] APIC: Switch to symmetric I/O mode setup738lb # [ 0.201038] x2apic enabled739client # [ 0.206605] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)740lb # [ 0.201788] APIC: Switched APIC routing to: physical x2apic741client # [ 0.207690] x86/cpu: User Mode Instruction Prevention (UMIP) activated742worker1 # [ 0.216404] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization743worker1 # [ 0.218411] Spectre V2 : Enabling IBPB for BPF744client # [ 0.209657] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127745lb # [ 0.203675] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1746worker1 # [ 0.219406] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier747client # [ 0.210602] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0748worker1 # [ 0.220406] active return thunk: srso_alias_return_thunk749lb # [ 0.204689] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns750client # [ 0.211606] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto751worker1 # [ 0.222051] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'752lb # [ 0.206387] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)753worker1 # [ 0.223299] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'754client # [ 0.212602] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl755lb # [ 0.207474] x86/cpu: User Mode Instruction Prevention (UMIP) activated756worker1 # [ 0.224164] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'757client # [ 0.214602] Transient Scheduler Attacks: Vulnerable: No microcode758lb # [ 0.209296] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127759worker1 # [ 0.225131] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'760client # [ 0.215602] Spectre V2 : Mitigation: Enhanced / Automatic IBRS761lb # [ 0.210077] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0762worker1 # [ 0.226147] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'763client # [ 0.216602] Speculative Return Stack Overflow: Mitigation: Safe RET764worker1 # [ 0.227147] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'765lb # [ 0.211158] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto766worker1 # [ 0.228218] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'767lb # [ 0.212384] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl768worker1 # [ 0.229405] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256769lb # [ 0.213384] Transient Scheduler Attacks: Vulnerable: No microcode770client # [ 0.217602] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization771worker1 # [ 0.230405] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64772lb # [ 0.214384] Spectre V2 : Mitigation: Enhanced / Automatic IBRS773client # [ 0.219610] Spectre V2 : Enabling IBPB for BPF774worker1 # [ 0.231404] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512775lb # [ 0.215287] Speculative Return Stack Overflow: Mitigation: Safe RET776client # [ 0.220338] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier777worker1 # [ 0.232404] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024778client # [ 0.221603] active return thunk: srso_alias_return_thunk779worker1 # [ 0.233405] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8780client # [ 0.222625] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'781worker1 # [ 0.234405] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.782client # [ 0.223602] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'783lb # [ 0.216135] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization784lb # [ 0.217391] Spectre V2 : Enabling IBPB for BPF785client # [ 0.224602] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'786client # [ 0.225602] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'787lb # [ 0.218937] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier788client # [ 0.226602] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'789lb # [ 0.219384] active return thunk: srso_alias_return_thunk790client # [ 0.227602] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'791lb # [ 0.220270] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'792client # [ 0.228602] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'793lb # [ 0.221384] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'794client # [ 0.230550] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256795lb # [ 0.222384] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'796client # [ 0.231602] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64797lb # [ 0.223384] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'798client # [ 0.232565] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512799worker2 # [ 0.266356] Freeing SMP alternatives memory: 44K800lb # [ 0.224384] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'801worker2 # [ 0.266807] pid_max: default: 32768 minimum: 301802client # [ 0.233602] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024803lb # [ 0.225384] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'804worker2 # [ 0.267887] LSM: initializing lsm=capability,landlock,yama,bpf,ima805client # [ 0.234543] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8806lb # [ 0.226384] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'807worker2 # [ 0.268913] landlock: Up and running.808worker2 # [ 0.269530] Yama: becoming mindful.809client # [ 0.235340] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.810lb # [ 0.227384] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256811worker2 # [ 0.270438] LSM support for eBPF active812lb # [ 0.228384] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64813lb # [ 0.229384] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512814worker2 # [ 0.271389] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)815lb # [ 0.230384] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024816worker2 # [ 0.271840] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)817lb # [ 0.231384] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8818lb # [ 0.232384] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.819worker2 # [ 0.275046] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)820worker2 # [ 0.276363] Performance Events: Fam17h+ core perfctr, AMD PMU driver.821worker2 # [ 0.276813] ... version: 2822worker2 # [ 0.277572] ... bit width: 48823worker2 # [ 0.277807] ... generic counters: 6824worker2 # [ 0.278538] ... generic bitmap: 000000000000003f825worker2 # [ 0.278806] ... fixed-purpose counters: 0826worker2 # [ 0.279566] ... fixed-purpose bitmap: 0000000000000000827worker2 # [ 0.279806] ... value mask: 0000ffffffffffff828worker2 # [ 0.280763] ... max period: 00007fffffffffff829worker2 # [ 0.281558] ... global_ctrl mask: 000000000000003f830worker2 # [ 0.281901] signal: max sigframe size: 3376831worker2 # [ 0.282754] rcu: Hierarchical SRCU implementation.832worker2 # [ 0.283448] rcu: Max phase no-delay instances is 400.833worker1 # [ 0.268630] Freeing SMP alternatives memory: 44K834worker1 # [ 0.269407] pid_max: default: 32768 minimum: 301835worker1 # [ 0.270253] LSM: initializing lsm=capability,landlock,yama,bpf,ima836worker2 # [ 0.288378] smp: Bringing up secondary CPUs ...837worker1 # [ 0.271470] landlock: Up and running.838worker1 # [ 0.272079] Yama: becoming mindful.839worker2 # [ 0.288821] smp: Brought up 1 node, 1 CPU840worker1 # [ 0.272617] LSM support for eBPF active841worker2 # [ 0.289526] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)842worker1 # [ 0.273513] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)843worker2 # [ 0.290015] Memory: 1454600K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109312K reserved, 0K cma-reserved)844worker1 # [ 0.274443] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)845worker2 # [ 0.291030] devtmpfs: initialized846worker2 # [ 0.291895] x86/mm: Memory block size: 128MB847worker1 # [ 0.277253] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)848worker2 # [ 0.293675] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)849worker1 # [ 0.278009] Performance Events: Fam17h+ core perfctr, AMD PMU driver.850worker1 # [ 0.278409] ... version: 2851worker2 # [ 0.294770] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).852worker1 # [ 0.279101] ... bit width: 48853client # [ 0.269974] Freeing SMP alternatives memory: 44K854worker1 # [ 0.279407] ... generic counters: 6855worker2 # [ 0.295894] pinctrl core: initialized pinctrl subsystem856client # [ 0.270601] pid_max: default: 32768 minimum: 301857worker1 # [ 0.280123] ... generic bitmap: 000000000000003f858worker2 # [ 0.296950] PM: RTC time: 15:59:29, date: 2026-09-23859client # [ 0.271668] LSM: initializing lsm=capability,landlock,yama,bpf,ima860worker1 # [ 0.280417] ... fixed-purpose counters: 0861client # [ 0.272706] landlock: Up and running.862worker1 # [ 0.281109] ... fixed-purpose bitmap: 0000000000000000863client # [ 0.273333] Yama: becoming mindful.864worker1 # [ 0.281407] ... value mask: 0000ffffffffffff865client # [ 0.273815] LSM support for eBPF active866worker1 # [ 0.282348] ... max period: 00007fffffffffff867lb # [ 0.267279] Freeing SMP alternatives memory: 44K868client # [ 0.274697] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)869worker2 # [ 0.300673] NET: Registered PF_NETLINK/PF_ROUTE protocol family870worker1 # [ 0.283121] ... global_ctrl mask: 000000000000003f871lb # [ 0.267935] pid_max: default: 32768 minimum: 301872worker1 # [ 0.283517] signal: max sigframe size: 3376873client # [ 0.275645] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)874worker2 # [ 0.301957] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations875lb # [ 0.268492] LSM: initializing lsm=capability,landlock,yama,bpf,ima876worker1 # [ 0.284295] rcu: Hierarchical SRCU implementation.877lb # [ 0.269489] landlock: Up and running.878worker1 # [ 0.285006] rcu: Max phase no-delay instances is 400.879worker2 # [ 0.302825] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations880lb # [ 0.270385] Yama: becoming mindful.881client # [ 0.278492] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)882lb # [ 0.271472] LSM support for eBPF active883worker2 # [ 0.304073] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations884lb # [ 0.272210] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)885client # [ 0.279116] Performance Events: Fam17h+ core perfctr, AMD PMU driver.886worker2 # [ 0.304816] audit: initializing netlink subsys (disabled)887client # [ 0.279607] ... version: 2888lb # [ 0.273424] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)889worker2 # [ 0.305982] thermal_sys: Registered thermal governor 'fair_share'890client # [ 0.280308] ... bit width: 48891client # [ 0.280605] ... generic counters: 6892worker2 # [ 0.305984] thermal_sys: Registered thermal governor 'bang_bang'893worker1 # [ 0.289920] smp: Bringing up secondary CPUs ...894client # [ 0.281345] ... generic bitmap: 000000000000003f895worker2 # [ 0.306808] thermal_sys: Registered thermal governor 'step_wise'896worker1 # [ 0.290420] smp: Brought up 1 node, 1 CPU897lb # [ 0.275713] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)898client # [ 0.281653] ... fixed-purpose counters: 0899worker2 # [ 0.307800] thermal_sys: Registered thermal governor 'user_space'900worker1 # [ 0.291106] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)901client # [ 0.282386] ... fixed-purpose bitmap: 0000000000000000902lb # [ 0.276933] Performance Events: Fam17h+ core perfctr, AMD PMU driver.903client # [ 0.282606] ... value mask: 0000ffffffffffff904worker2 # [ 0.308732] audit: type=2000 audit(1790179170.376:1): state=initialized audit_enabled=0 res=1905lb # [ 0.277393] ... version: 2906client # [ 0.283475] ... max period: 00007fffffffffff907worker2 # [ 0.310809] thermal_sys: Registered thermal governor 'power_allocator'908lb # [ 0.278100] ... bit width: 48909worker1 # [ 0.291577] Memory: 1454624K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)910worker2 # [ 0.310824] cpuidle: using governor menu911client # [ 0.284284] ... global_ctrl mask: 000000000000003f912lb # [ 0.278386] ... generic counters: 6913worker1 # [ 0.292643] devtmpfs: initialized914client # [ 0.284724] signal: max sigframe size: 3376915lb # [ 0.279112] ... generic bitmap: 000000000000003f916worker1 # [ 0.293445] x86/mm: Memory block size: 128MB917worker2 # [ 0.313009] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5918client # [ 0.285488] rcu: Hierarchical SRCU implementation.919lb # [ 0.279386] ... fixed-purpose counters: 0920client # [ 0.286169] rcu: Max phase no-delay instances is 400.921lb # [ 0.280114] ... fixed-purpose bitmap: 0000000000000000922worker2 # [ 0.314036] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]923worker1 # [ 0.295191] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)924lb # [ 0.280386] ... value mask: 0000ffffffffffff925worker2 # [ 0.314810] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry926lb # [ 0.281246] ... max period: 00007fffffffffff927worker1 # [ 0.296295] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).928worker2 # [ 0.315818] PCI: Using configuration type 1 for base access929lb # [ 0.282041] ... global_ctrl mask: 000000000000003f930worker1 # [ 0.297492] pinctrl core: initialized pinctrl subsystem931lb # [ 0.282498] signal: max sigframe size: 3376932worker2 # [ 0.316924] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.933worker1 # [ 0.298636] PM: RTC time: 15:59:29, date: 2026-09-23934lb # [ 0.283292] rcu: Hierarchical SRCU implementation.935lb # [ 0.283983] rcu: Max phase no-delay instances is 400.936client # [ 0.291235] smp: Bringing up secondary CPUs ...937client # [ 0.291619] smp: Brought up 1 node, 1 CPU938client # [ 0.292320] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)939worker1 # [ 0.302183] NET: Registered PF_NETLINK/PF_ROUTE protocol family940worker2 # [ 0.322055] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages941client # [ 0.292768] Memory: 1454600K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)942worker1 # [ 0.303704] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations943client # [ 0.293919] devtmpfs: initialized944worker2 # [ 0.322807] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page945lb # [ 0.288966] smp: Bringing up secondary CPUs ...946worker1 # [ 0.304432] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations947lb # [ 0.289401] smp: Brought up 1 node, 1 CPU948client # [ 0.294792] x86/mm: Memory block size: 128MB949lb # [ 0.290087] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)950worker1 # [ 0.305681] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations951worker1 # [ 0.306416] audit: initializing netlink subsys (disabled)952client # [ 0.296576] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)953lb # [ 0.290554] Memory: 1454572K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109316K reserved, 0K cma-reserved)954worker1 # [ 0.307549] thermal_sys: Registered thermal governor 'fair_share'955client # [ 0.297512] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).956worker1 # [ 0.307551] thermal_sys: Registered thermal governor 'bang_bang'957lb # [ 0.291647] devtmpfs: initialized958worker2 # [ 0.327807] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages959client # [ 0.298681] pinctrl core: initialized pinctrl subsystem960worker1 # [ 0.308408] thermal_sys: Registered thermal governor 'step_wise'961lb # [ 0.292499] x86/mm: Memory block size: 128MB962worker2 # [ 0.328807] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page963client # [ 0.299714] PM: RTC time: 15:59:29, date: 2026-09-23964worker1 # [ 0.309302] thermal_sys: Registered thermal governor 'user_space'965lb # [ 0.294255] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)966worker1 # [ 0.310097] audit: type=2000 audit(1790179170.402:1): state=initialized audit_enabled=0 res=1967worker1 # [ 0.311410] thermal_sys: Registered thermal governor 'power_allocator'968lb # [ 0.295314] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).969worker1 # [ 0.311432] cpuidle: using governor menu970client # [ 0.303280] NET: Registered PF_NETLINK/PF_ROUTE protocol family971lb # [ 0.296464] pinctrl core: initialized pinctrl subsystem972worker1 # [ 0.313630] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5973lb # [ 0.297645] PM: RTC time: 15:59:29, date: 2026-09-23974client # [ 0.304107] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations975client # [ 0.304629] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations976worker1 # [ 0.314681] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]977worker1 # [ 0.315410] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry978client # [ 0.305885] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations979worker1 # [ 0.316419] PCI: Using configuration type 1 for base access980lb # [ 0.301099] NET: Registered PF_NETLINK/PF_ROUTE protocol family981client # [ 0.306615] audit: initializing netlink subsys (disabled)982worker2 # [ 0.339126] ACPI: Added _OSI(Module Device)983worker2 # [ 0.339808] ACPI: Added _OSI(Processor Device)984worker1 # [ 0.317507] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.985client # [ 0.307607] audit: type=2000 audit(1790179170.414:1): state=initialized audit_enabled=0 res=1986lb # [ 0.301897] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations987worker2 # [ 0.340536] ACPI: Added _OSI(Processor Aggregator Device)988client # [ 0.308752] thermal_sys: Registered thermal governor 'fair_share'989lb # [ 0.302409] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations990client # [ 0.308754] thermal_sys: Registered thermal governor 'bang_bang'991lb # [ 0.303658] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations992client # [ 0.309606] thermal_sys: Registered thermal governor 'step_wise'993lb # [ 0.304397] audit: initializing netlink subsys (disabled)994client # [ 0.310606] thermal_sys: Registered thermal governor 'user_space'995lb # [ 0.305580] thermal_sys: Registered thermal governor 'fair_share'996client # [ 0.311587] thermal_sys: Registered thermal governor 'power_allocator'997client # [ 0.312403] cpuidle: using governor menu998lb # [ 0.305582] thermal_sys: Registered thermal governor 'bang_bang'999lb # [ 0.306387] thermal_sys: Registered thermal governor 'step_wise'1000worker1 # [ 0.324662] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1001client # [ 0.314147] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.51002worker2 # [ 0.346767] ACPI: 1 ACPI AML tables successfully acquired and loaded1003worker1 # [ 0.325408] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1004lb # [ 0.307389] audit: type=2000 audit(1790179170.419:1): state=initialized audit_enabled=0 res=11005client # [ 0.314831] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]1006lb # [ 0.309389] thermal_sys: Registered thermal governor 'user_space'1007client # [ 0.315608] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1008lb # [ 0.309391] thermal_sys: Registered thermal governor 'power_allocator'1009client # [ 0.316617] PCI: Using configuration type 1 for base access1010lb # [ 0.310389] cpuidle: using governor menu1011lb # [ 0.312620] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.51012client # [ 0.317743] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1013worker1 # [ 0.330408] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1014worker2 # [ 0.353210] ACPI: Interpreter enabled1015worker1 # [ 0.331409] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1016lb # [ 0.313626] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]1017worker2 # [ 0.353823] ACPI: PM: (supports S0 S3 S4 S5)1018lb # [ 0.314390] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1019worker2 # [ 0.354558] ACPI: Using IOAPIC for interrupt routing1020lb # [ 0.315398] PCI: Using configuration type 1 for base access1021worker2 # [ 0.354884] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1022lb # [ 0.316529] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1023worker2 # [ 0.355807] PCI: Using E820 reservations for host bridge windows1024client # [ 0.324619] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1025client # [ 0.325605] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1026worker2 # [ 0.358952] ACPI: Enabled 2 GPEs in block 00 to 3F1027worker1 # [ 0.340011] ACPI: Added _OSI(Module Device)1028lb # [ 0.321467] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1029worker1 # [ 0.340409] ACPI: Added _OSI(Processor Device)1030lb # [ 0.322387] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1031worker1 # [ 0.341164] ACPI: Added _OSI(Processor Aggregator Device)1032client # [ 0.330607] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1033client # [ 0.331606] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1034lb # [ 0.327388] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1035worker2 # [ 0.367583] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1036lb # [ 0.328386] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1037worker2 # [ 0.368621] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1038worker1 # [ 0.347984] ACPI: 1 ACPI AML tables successfully acquired and loaded1039worker2 # [ 0.370419] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1040worker2 # [ 0.370929] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1041worker1 # [ 0.351332] ACPI: Interpreter enabled1042worker2 # [ 0.372157] PCI host bridge to bus 0000:001043worker1 # [ 0.351872] ACPI: PM: (supports S0 S3 S4 S5)1044worker2 # [ 0.372811] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1045worker1 # [ 0.352408] ACPI: Using IOAPIC for interrupt routing1046worker2 # [ 0.373807] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1047client # [ 0.342832] ACPI: Added _OSI(Module Device)1048worker1 # [ 0.353282] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1049client # [ 0.343566] ACPI: Added _OSI(Processor Device)1050worker2 # [ 0.374807] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1051client # [ 0.344144] ACPI: Added _OSI(Processor Aggregator Device)1052worker2 # [ 0.375807] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1053lb # [ 0.338731] ACPI: Added _OSI(Module Device)1054lb # [ 0.339388] ACPI: Added _OSI(Processor Device)1055worker2 # [ 0.376807] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1056lb # [ 0.340122] ACPI: Added _OSI(Processor Aggregator Device)1057worker2 # [ 0.377812] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1058worker1 # [ 0.358408] PCI: Using E820 reservations for host bridge windows1059worker2 # [ 0.378832] pci_bus 0000:00: root bus resource [bus 00-ff]1060worker1 # [ 0.359536] ACPI: Enabled 2 GPEs in block 00 to 3F1061worker2 # [ 0.379855] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1062client # [ 0.350118] ACPI: 1 ACPI AML tables successfully acquired and loaded1063worker2 # [ 0.381248] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1064client # [ 0.354014] ACPI: Interpreter enabled1065client # [ 0.354620] ACPI: PM: (supports S0 S3 S4 S5)1066lb # [ 0.346376] ACPI: 1 ACPI AML tables successfully acquired and loaded1067client # [ 0.355327] ACPI: Using IOAPIC for interrupt routing1068worker1 # [ 0.368100] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1069worker2 # [ 0.383860] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1070worker1 # [ 0.368771] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1071lb # [ 0.352827] ACPI: Interpreter enabled1072client # [ 0.357653] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1073worker2 # [ 0.384818] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1074lb # [ 0.353402] ACPI: PM: (supports S0 S3 S4 S5)1075client # [ 0.358605] PCI: Using E820 reservations for host bridge windows1076worker1 # [ 0.369484] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1077lb # [ 0.354096] ACPI: Using IOAPIC for interrupt routing1078worker2 # [ 0.385761] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1079worker1 # [ 0.370530] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1080worker1 # [ 0.371809] PCI host bridge to bus 0000:001081worker2 # [ 0.386773] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1082client # [ 0.361657] ACPI: Enabled 2 GPEs in block 00 to 3F1083worker1 # [ 0.372412] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1084lb # [ 0.356444] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1085worker1 # [ 0.373408] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1086worker2 # [ 0.388526] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1087lb # [ 0.357388] PCI: Using E820 reservations for host bridge windows1088worker1 # [ 0.374408] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1089worker1 # [ 0.375407] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1090worker2 # [ 0.391849] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1091lb # [ 0.360464] ACPI: Enabled 2 GPEs in block 00 to 3F1092worker1 # [ 0.376407] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1093worker2 # [ 0.392695] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1094worker1 # [ 0.377408] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1095worker2 # [ 0.393550] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1096worker1 # [ 0.378408] pci_bus 0000:00: root bus resource [bus 00-ff]1097client # [ 0.369123] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1098worker2 # [ 0.393812] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1099worker1 # [ 0.379382] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1100client # [ 0.369611] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1101worker2 # [ 0.395319] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1102client # [ 0.370683] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1103worker1 # [ 0.380841] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1104client # [ 0.371736] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1105lb # [ 0.367961] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1106client # [ 0.372953] PCI host bridge to bus 0000:001107worker2 # [ 0.397848] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1108client # [ 0.373610] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1109lb # [ 0.368394] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1110worker2 # [ 0.398718] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1111client # [ 0.374605] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1112worker2 # [ 0.399558] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1113lb # [ 0.369465] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1114client # [ 0.375605] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1115lb # [ 0.370518] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1116worker2 # [ 0.400482] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1117client # [ 0.376605] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1118lb # [ 0.371761] PCI host bridge to bus 0000:001119worker1 # [ 0.383492] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1120client # [ 0.377605] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1121lb # [ 0.372391] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1122worker1 # [ 0.384419] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1123lb # [ 0.373387] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1124client # [ 0.378605] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1125worker1 # [ 0.385377] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1126client # [ 0.379605] pci_bus 0000:00: root bus resource [bus 00-ff]1127lb # [ 0.374386] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1128client # [ 0.380569] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1129lb # [ 0.375386] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1130worker1 # [ 0.386356] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1131worker2 # [ 0.404154] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1132lb # [ 0.376387] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1133client # [ 0.382052] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1134worker2 # [ 0.405709] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1135worker1 # [ 0.388316] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1136lb # [ 0.377392] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1137worker2 # [ 0.406645] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1138lb # [ 0.378387] pci_bus 0000:00: root bus resource [bus 00-ff]1139lb # [ 0.379377] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1140worker2 # [ 0.408393] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1141worker1 # [ 0.391437] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1142worker1 # [ 0.392329] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1143lb # [ 0.380880] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1144worker1 # [ 0.393161] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1145worker2 # [ 0.410760] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1146worker1 # [ 0.393413] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1147worker2 # [ 0.411531] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1148client # [ 0.384694] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1149client # [ 0.385617] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1150worker2 # [ 0.411829] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1151worker1 # [ 0.394976] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1152client # [ 0.386552] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1153worker2 # [ 0.412813] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1154worker2 # [ 0.414374] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1155client # [ 0.387644] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1156worker1 # [ 0.398399] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1157worker1 # [ 0.399024] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1158lb # [ 0.383475] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1159worker2 # [ 0.415848] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1160worker1 # [ 0.399429] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1161lb # [ 0.384399] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1162client # [ 0.389593] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1163worker2 # [ 0.416823] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1164lb # [ 0.385353] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1165worker1 # [ 0.400986] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1166worker2 # [ 0.418404] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1167lb # [ 0.386348] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1168client # [ 0.392643] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1169client # [ 0.393512] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1170lb # [ 0.388114] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1171client # [ 0.394399] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1172worker2 # [ 0.420310] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1173client # [ 0.395481] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1174worker1 # [ 0.404261] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1175worker2 # [ 0.420854] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1176worker1 # [ 0.405056] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1177lb # [ 0.391426] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1178worker1 # [ 0.405430] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1179worker2 # [ 0.422398] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1180client # [ 0.396941] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1181lb # [ 0.392302] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1182lb # [ 0.393147] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1183worker1 # [ 0.406958] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1184worker2 # [ 0.423695] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1185lb # [ 0.394231] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1186worker2 # [ 0.424508] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1187client # [ 0.400641] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1188client # [ 0.401596] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1189worker1 # [ 0.409305] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1190worker2 # [ 0.425381] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1191lb # [ 0.395786] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1192client # [ 0.402364] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1193worker1 # [ 0.410012] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1194worker1 # [ 0.410429] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1195client # [ 0.403202] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1196worker2 # [ 0.426819] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1197lb # [ 0.398424] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1198worker1 # [ 0.411413] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1199worker2 # [ 0.427829] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1200lb # [ 0.399301] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1201lb # [ 0.400174] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1202worker1 # [ 0.413010] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1203worker2 # [ 0.429385] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1204lb # [ 0.401855] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1205client # [ 0.405972] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1206worker1 # [ 0.414420] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1207worker2 # [ 0.430824] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1208client # [ 0.407263] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1209worker2 # [ 0.431674] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1210worker1 # [ 0.415375] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1211client # [ 0.407627] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1212worker2 # [ 0.432562] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1213worker1 # [ 0.416781] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1214client # [ 0.409181] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1215lb # [ 0.404395] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1216worker2 # [ 0.433383] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1217lb # [ 0.406040] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1218lb # [ 0.406433] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1219worker1 # [ 0.418444] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1220client # [ 0.411528] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1221worker2 # [ 0.435696] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1222client # [ 0.412261] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1223worker1 # [ 0.419430] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1224lb # [ 0.407933] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1225worker2 # [ 0.436459] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1226client # [ 0.412627] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1227worker2 # [ 0.436828] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1228worker1 # [ 0.420972] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1229lb # [ 0.409424] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1230client # [ 0.413611] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1231worker2 # [ 0.438411] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1232lb # [ 0.410338] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1233worker1 # [ 0.422447] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1234lb # [ 0.411144] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1235worker2 # [ 0.439463] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1236client # [ 0.415185] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1237worker1 # [ 0.423429] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1238lb # [ 0.411392] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1239worker2 # [ 0.440031] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1240worker1 # [ 0.424989] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1241client # [ 0.416648] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1242worker2 # [ 0.441421] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1243lb # [ 0.412931] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1244client # [ 0.417628] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1245worker2 # [ 0.442425] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1246worker1 # [ 0.426420] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1247client # [ 0.419201] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1248lb # [ 0.414388] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1249worker1 # [ 0.427429] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1250worker2 # [ 0.443375] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1251lb # [ 0.415356] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1252worker1 # [ 0.428963] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1253lb # [ 0.416849] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1254client # [ 0.420649] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1255client # [ 0.421664] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1256lb # [ 0.418399] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1257worker1 # [ 0.431230] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1258client # [ 0.423170] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1259worker1 # [ 0.432080] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1260lb # [ 0.419392] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1261worker1 # [ 0.432430] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1262client # [ 0.424618] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1263lb # [ 0.420994] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1264client # [ 0.425627] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1265worker1 # [ 0.433979] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1266lb # [ 0.422407] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1267client # [ 0.427173] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1268lb # [ 0.423409] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1269worker1 # [ 0.436140] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1270worker2 # [ 0.444031] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1271worker1 # [ 0.436414] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1272lb # [ 0.424948] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1273client # [ 0.428648] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1274worker1 # [ 0.437371] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1275worker2 # [ 0.446333] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1276client # [ 0.430337] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1277lb # [ 0.426400] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1278worker2 # [ 0.447115] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1279worker1 # [ 0.438789] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1280client # [ 0.431144] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1281lb # [ 0.427348] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1282worker2 # [ 0.448101] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1283worker1 # [ 0.440164] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1284worker2 # [ 0.449216] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1285lb # [ 0.428948] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1286worker1 # [ 0.440609] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1287client # [ 0.433474] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1288client # [ 0.434278] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1289worker2 # [ 0.450713] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1290lb # [ 0.430404] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1291worker1 # [ 0.442920] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1292worker2 # [ 0.451486] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1293client # [ 0.434627] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1294lb # [ 0.431221] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1295worker1 # [ 0.443617] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1296worker2 # [ 0.452126] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1297lb # [ 0.432143] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1298client # [ 0.436144] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1299worker1 # [ 0.445113] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1300worker2 # [ 0.453437] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1301lb # [ 0.432955] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1302client # [ 0.438402] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1303client # [ 0.439345] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1304lb # [ 0.435359] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1305worker2 # [ 0.457226] ACPI: PCI: Interrupt link LNKA configured for IRQ 101306client # [ 0.439627] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1307lb # [ 0.436091] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1308worker2 # [ 0.457910] ACPI: PCI: Interrupt link LNKB configured for IRQ 101309lb # [ 0.436408] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1310worker2 # [ 0.458907] ACPI: PCI: Interrupt link LNKC configured for IRQ 111311client # [ 0.441224] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1312worker2 # [ 0.459906] ACPI: PCI: Interrupt link LNKD configured for IRQ 111313lb # [ 0.437982] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1314client # [ 0.442414] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1315worker2 # [ 0.460937] ACPI: PCI: Interrupt link LNKE configured for IRQ 101316worker2 # [ 0.461908] ACPI: PCI: Interrupt link LNKF configured for IRQ 101317lb # [ 0.439118] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1318client # [ 0.443460] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1319worker2 # [ 0.462908] ACPI: PCI: Interrupt link LNKG configured for IRQ 111320lb # [ 0.439606] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1321worker2 # [ 0.463904] ACPI: PCI: Interrupt link LNKH configured for IRQ 111322client # [ 0.445455] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1323lb # [ 0.441027] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1324worker2 # [ 0.464847] ACPI: PCI: Interrupt link GSIA configured for IRQ 161325worker1 # [ 0.445625] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1326client # [ 0.446440] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1327worker2 # [ 0.465818] ACPI: PCI: Interrupt link GSIB configured for IRQ 171328worker1 # [ 0.448174] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1329lb # [ 0.441580] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1330worker2 # [ 0.466820] ACPI: PCI: Interrupt link GSIC configured for IRQ 181331client # [ 0.448359] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1332worker2 # [ 0.467821] ACPI: PCI: Interrupt link GSID configured for IRQ 191333worker1 # [ 0.448682] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1334lb # [ 0.443060] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1335worker2 # [ 0.468823] ACPI: PCI: Interrupt link GSIE configured for IRQ 201336worker1 # [ 0.449887] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1337worker2 # [ 0.469822] ACPI: PCI: Interrupt link GSIF configured for IRQ 211338worker2 # [ 0.470821] ACPI: PCI: Interrupt link GSIG configured for IRQ 221339worker1 # [ 0.450702] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1340worker2 # [ 0.471819] ACPI: PCI: Interrupt link GSIH configured for IRQ 231341worker1 # [ 0.452435] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1342worker2 # [ 0.474161] iommu: Default domain type: Translated1343worker1 # [ 0.453340] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1344worker2 # [ 0.474820] iommu: DMA domain TLB invalidation policy: lazy mode1345worker2 # [ 0.476118] ACPI: bus type USB registered1346worker1 # [ 0.454502] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1347worker2 # [ 0.476880] usbcore: registered new interface driver usbfs1348worker1 # [ 0.456116] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1349worker2 # [ 0.477817] usbcore: registered new interface driver hub1350worker2 # [ 0.478710] usbcore: registered new device driver usb1351worker2 # [ 0.480427] NetLabel: Initializing1352client # [ 0.448862] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1353lb # [ 0.443642] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1354worker2 # [ 0.480808] NetLabel: domain hash size = 1281355worker2 # [ 0.481530] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1356client # [ 0.451399] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1357worker1 # [ 0.459923] ACPI: PCI: Interrupt link LNKA configured for IRQ 101358lb # [ 0.446020] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1359worker2 # [ 0.481899] NetLabel: unlabeled traffic allowed by default1360worker1 # [ 0.460516] ACPI: PCI: Interrupt link LNKB configured for IRQ 101361worker2 # [ 0.482794] PCI: Using ACPI for IRQ routing1362client # [ 0.452553] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1363lb # [ 0.446628] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1364worker1 # [ 0.461508] ACPI: PCI: Interrupt link LNKC configured for IRQ 111365worker1 # [ 0.462503] ACPI: PCI: Interrupt link LNKD configured for IRQ 111366client # [ 0.453883] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1367lb # [ 0.447671] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1368worker1 # [ 0.463508] ACPI: PCI: Interrupt link LNKE configured for IRQ 101369client # [ 0.454903] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1370lb # [ 0.448655] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1371worker1 # [ 0.464503] ACPI: PCI: Interrupt link LNKF configured for IRQ 101372worker1 # [ 0.465507] ACPI: PCI: Interrupt link LNKG configured for IRQ 111373client # [ 0.456633] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1374lb # [ 0.450410] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1375worker1 # [ 0.466511] ACPI: PCI: Interrupt link LNKH configured for IRQ 111376client # [ 0.457465] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1377lb # [ 0.451240] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1378worker1 # [ 0.467449] ACPI: PCI: Interrupt link GSIA configured for IRQ 161379lb # [ 0.452424] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1380client # [ 0.458688] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1381worker1 # [ 0.468424] ACPI: PCI: Interrupt link GSIB configured for IRQ 171382worker1 # [ 0.469422] ACPI: PCI: Interrupt link GSIC configured for IRQ 181383lb # [ 0.454088] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1384client # [ 0.460344] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1385worker1 # [ 0.470419] ACPI: PCI: Interrupt link GSID configured for IRQ 191386worker1 # [ 0.471422] ACPI: PCI: Interrupt link GSIE configured for IRQ 201387worker1 # [ 0.472420] ACPI: PCI: Interrupt link GSIF configured for IRQ 211388worker1 # [ 0.473419] ACPI: PCI: Interrupt link GSIG configured for IRQ 221389lb # [ 0.457450] ACPI: PCI: Interrupt link LNKA configured for IRQ 101390client # [ 0.463703] ACPI: PCI: Interrupt link LNKA configured for IRQ 101391worker1 # [ 0.474421] ACPI: PCI: Interrupt link GSIH configured for IRQ 231392lb # [ 0.458492] ACPI: PCI: Interrupt link LNKB configured for IRQ 101393client # [ 0.464710] ACPI: PCI: Interrupt link LNKB configured for IRQ 101394lb # [ 0.459483] ACPI: PCI: Interrupt link LNKC configured for IRQ 111395worker1 # [ 0.476450] iommu: Default domain type: Translated1396client # [ 0.465703] ACPI: PCI: Interrupt link LNKC configured for IRQ 111397lb # [ 0.460494] ACPI: PCI: Interrupt link LNKD configured for IRQ 111398worker1 # [ 0.477240] iommu: DMA domain TLB invalidation policy: lazy mode1399client # [ 0.466728] ACPI: PCI: Interrupt link LNKD configured for IRQ 111400worker1 # [ 0.478421] ACPI: bus type USB registered1401lb # [ 0.461487] ACPI: PCI: Interrupt link LNKE configured for IRQ 101402client # [ 0.467731] ACPI: PCI: Interrupt link LNKE configured for IRQ 101403worker1 # [ 0.479182] usbcore: registered new interface driver usbfs1404lb # [ 0.462484] ACPI: PCI: Interrupt link LNKF configured for IRQ 101405client # [ 0.468704] ACPI: PCI: Interrupt link LNKF configured for IRQ 101406worker1 # [ 0.479420] usbcore: registered new interface driver hub1407lb # [ 0.463488] ACPI: PCI: Interrupt link LNKG configured for IRQ 111408worker1 # [ 0.480304] usbcore: registered new device driver usb1409client # [ 0.469704] ACPI: PCI: Interrupt link LNKG configured for IRQ 111410lb # [ 0.464488] ACPI: PCI: Interrupt link LNKH configured for IRQ 111411client # [ 0.470725] ACPI: PCI: Interrupt link LNKH configured for IRQ 111412worker1 # [ 0.481852] NetLabel: Initializing1413lb # [ 0.465423] ACPI: PCI: Interrupt link GSIA configured for IRQ 161414worker1 # [ 0.482409] NetLabel: domain hash size = 1281415client # [ 0.471647] ACPI: PCI: Interrupt link GSIA configured for IRQ 161416worker1 # [ 0.483148] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1417lb # [ 0.466389] ACPI: PCI: Interrupt link GSIB configured for IRQ 171418client # [ 0.472608] ACPI: PCI: Interrupt link GSIB configured for IRQ 171419worker1 # [ 0.483447] NetLabel: unlabeled traffic allowed by default1420lb # [ 0.467401] ACPI: PCI: Interrupt link GSIC configured for IRQ 181421client # [ 0.473570] ACPI: PCI: Interrupt link GSIC configured for IRQ 181422worker1 # [ 0.484332] PCI: Using ACPI for IRQ routing1423lb # [ 0.468400] ACPI: PCI: Interrupt link GSID configured for IRQ 191424client # [ 0.474338] ACPI: PCI: Interrupt link GSID configured for IRQ 191425lb # [ 0.469404] ACPI: PCI: Interrupt link GSIE configured for IRQ 201426client # [ 0.474617] ACPI: PCI: Interrupt link GSIE configured for IRQ 201427lb # [ 0.470395] ACPI: PCI: Interrupt link GSIF configured for IRQ 211428client # [ 0.475618] ACPI: PCI: Interrupt link GSIF configured for IRQ 211429lb # [ 0.471401] ACPI: PCI: Interrupt link GSIG configured for IRQ 221430client # [ 0.476586] ACPI: PCI: Interrupt link GSIG configured for IRQ 221431lb # [ 0.472363] ACPI: PCI: Interrupt link GSIH configured for IRQ 231432client # [ 0.477385] ACPI: PCI: Interrupt link GSIH configured for IRQ 231433lb # [ 0.474266] iommu: Default domain type: Translated1434client # [ 0.478665] iommu: Default domain type: Translated1435lb # [ 0.474995] iommu: DMA domain TLB invalidation policy: lazy mode1436client # [ 0.479497] iommu: DMA domain TLB invalidation policy: lazy mode1437lb # [ 0.475696] ACPI: bus type USB registered1438client # [ 0.480704] ACPI: bus type USB registered1439lb # [ 0.476445] usbcore: registered new interface driver usbfs1440client # [ 0.481471] usbcore: registered new interface driver usbfs1441lb # [ 0.477379] usbcore: registered new interface driver hub1442client # [ 0.482317] usbcore: registered new interface driver hub1443lb # [ 0.478097] usbcore: registered new device driver usb1444client # [ 0.482615] usbcore: registered new device driver usb1445lb # [ 0.479200] NetLabel: Initializing1446client # [ 0.484697] NetLabel: Initializing1447lb # [ 0.479866] NetLabel: domain hash size = 1281448client # [ 0.485375] NetLabel: domain hash size = 1281449lb # [ 0.480386] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1450client # [ 0.485605] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1451lb # [ 0.481482] NetLabel: unlabeled traffic allowed by default1452client # [ 0.486654] NetLabel: unlabeled traffic allowed by default1453lb # [ 0.482528] PCI: Using ACPI for IRQ routing1454client # [ 0.487598] PCI: Using ACPI for IRQ routing1455worker2 # [ 0.527086] pci 0000:00:01.0: vgaarb: setting as boot VGA device1456worker2 # [ 0.527803] pci 0000:00:01.0: vgaarb: bridge control possible1457worker2 # [ 0.527803] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1458worker2 # [ 0.527813] vgaarb: loaded1459worker2 # [ 0.528654] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01460worker2 # [ 0.529464] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1461worker2 # [ 0.533921] clocksource: Switched to clocksource kvm-clock1462worker2 # [ 0.538015] VFS: Disk quotas dquot_6.6.01463worker2 # [ 0.538769] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1464worker2 # [ 0.540225] pnp: PnP ACPI init1465worker2 # [ 0.541088] ACPI: IRQ 4 override to edge(!), high(!)1466worker2 # [ 0.542096] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1467worker2 # [ 0.543703] pnp: PnP ACPI: found 5 devices1468worker1 # [ 0.528333] pci 0000:00:01.0: vgaarb: setting as boot VGA device1469worker1 # [ 0.528403] pci 0000:00:01.0: vgaarb: bridge control possible1470worker1 # [ 0.528403] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1471worker2 # [ 0.551486] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1472worker1 # [ 0.528412] vgaarb: loaded1473worker2 # [ 0.552941] clocksource: Switched to clocksource acpi_pm1474worker1 # [ 0.529184] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01475worker2 # [ 0.553969] NET: Registered PF_INET protocol family1476worker1 # [ 0.529408] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1477worker2 # [ 0.555216] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1478worker1 # [ 0.533497] clocksource: Switched to clocksource kvm-clock1479worker1 # [ 0.537175] VFS: Disk quotas dquot_6.6.01480worker1 # [ 0.537851] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1481worker1 # [ 0.539268] pnp: PnP ACPI init1482worker1 # [ 0.540079] ACPI: IRQ 4 override to edge(!), high(!)1483client # [ 0.532246] pci 0000:00:01.0: vgaarb: setting as boot VGA device1484lb # [ 0.527831] pci 0000:00:01.0: vgaarb: setting as boot VGA device1485worker1 # [ 0.541038] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1486client # [ 0.532601] pci 0000:00:01.0: vgaarb: bridge control possible1487lb # [ 0.528383] pci 0000:00:01.0: vgaarb: bridge control possible1488worker1 # [ 0.542535] pnp: PnP ACPI: found 5 devices1489client # [ 0.532601] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1490client # [ 0.532610] vgaarb: loaded1491lb # [ 0.528383] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1492lb # [ 0.528394] vgaarb: loaded1493client # [ 0.533333] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01494lb # [ 0.529219] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01495client # [ 0.533605] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1496lb # [ 0.530044] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1497worker2 # [ 0.572555] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1498worker2 # [ 0.574219] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1499worker2 # [ 0.575609] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1500lb # [ 0.533503] clocksource: Switched to clocksource kvm-clock1501client # [ 0.537687] clocksource: Switched to clocksource kvm-clock1502worker1 # [ 0.550162] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1503worker2 # [ 0.576960] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1504worker1 # [ 0.551572] clocksource: Switched to clocksource acpi_pm1505worker2 # [ 0.578235] TCP: Hash tables configured (established 16384 bind 16384)1506worker1 # [ 0.552558] NET: Registered PF_INET protocol family1507worker2 # [ 0.579392] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1508lb # [ 0.537290] VFS: Disk quotas dquot_6.6.01509client # [ 0.541294] VFS: Disk quotas dquot_6.6.01510worker1 # [ 0.553761] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1511client # [ 0.541983] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1512worker2 # [ 0.580673] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1513lb # [ 0.538048] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1514client # [ 0.543327] pnp: PnP ACPI init1515lb # [ 0.539488] pnp: PnP ACPI init1516worker2 # [ 0.581748] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1517client # [ 0.544108] ACPI: IRQ 4 override to edge(!), high(!)1518lb # [ 0.540387] ACPI: IRQ 4 override to edge(!), high(!)1519worker2 # [ 0.583036] NET: Registered PF_UNIX/PF_LOCAL protocol family1520worker2 # [ 0.584103] NET: Registered PF_XDP protocol family1521client # [ 0.545026] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1522lb # [ 0.541433] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1523worker2 # [ 0.584925] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1524client # [ 0.546538] pnp: PnP ACPI: found 5 devices1525lb # [ 0.542953] pnp: PnP ACPI: found 5 devices1526worker2 # [ 0.585959] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1527worker2 # [ 0.587026] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1528worker2 # [ 0.588215] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1529worker2 # [ 0.589373] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1530worker2 # [ 0.590537] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1531worker2 # [ 0.592432] ACPI: \_SB_.GSIA: Enabled at IRQ 161532client # [ 0.554159] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1533lb # [ 0.550665] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1534client # [ 0.555596] clocksource: Switched to clocksource acpi_pm1535lb # [ 0.552116] clocksource: Switched to clocksource acpi_pm1536worker2 # [ 0.594992] ACPI: \_SB_.GSIB: Enabled at IRQ 171537client # [ 0.556641] NET: Registered PF_INET protocol family1538lb # [ 0.553150] NET: Registered PF_INET protocol family1539worker1 # [ 0.570251] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1540worker2 # [ 0.596950] ACPI: \_SB_.GSIC: Enabled at IRQ 181541client # [ 0.557841] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1542lb # [ 0.554402] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1543worker1 # [ 0.571767] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1544worker2 # [ 0.598910] ACPI: \_SB_.GSID: Enabled at IRQ 191545worker1 # [ 0.573133] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1546worker2 # [ 0.600845] PCI: CLS 0 bytes, default 641547worker1 # [ 0.574524] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1548worker1 # [ 0.575774] TCP: Hash tables configured (established 16384 bind 16384)1549worker2 # [ 0.601764] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1550worker1 # [ 0.576926] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1551worker2 # [ 0.603563] Trying to unpack rootfs image as initramfs...1552worker1 # [ 0.578224] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1553worker1 # [ 0.579401] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1554worker1 # [ 0.580665] NET: Registered PF_UNIX/PF_LOCAL protocol family1555worker1 # [ 0.581679] NET: Registered PF_XDP protocol family1556worker1 # [ 0.582616] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1557worker1 # [ 0.583665] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1558worker1 # [ 0.584719] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1559worker1 # [ 0.585876] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1560worker1 # [ 0.587056] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1561client # [ 0.574540] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1562lb # [ 0.571231] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1563worker1 # [ 0.588174] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1564client # [ 0.576041] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1565worker1 # [ 0.589827] ACPI: \_SB_.GSIA: Enabled at IRQ 161566lb # [ 0.572741] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1567client # [ 0.577363] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1568lb # [ 0.574033] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1569client # [ 0.578695] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1570worker1 # [ 0.592544] ACPI: \_SB_.GSIB: Enabled at IRQ 171571client # [ 0.579895] TCP: Hash tables configured (established 16384 bind 16384)1572lb # [ 0.576335] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1573client # [ 0.581068] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1574worker1 # [ 0.594477] ACPI: \_SB_.GSIC: Enabled at IRQ 181575lb # [ 0.578076] TCP: Hash tables configured (established 16384 bind 16384)1576client # [ 0.582340] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1577client # [ 0.583417] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1578worker1 # [ 0.596466] ACPI: \_SB_.GSID: Enabled at IRQ 191579lb # [ 0.579342] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1580client # [ 0.584601] NET: Registered PF_UNIX/PF_LOCAL protocol family1581lb # [ 0.580647] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1582worker1 # [ 0.598120] PCI: CLS 0 bytes, default 641583client # [ 0.585547] NET: Registered PF_XDP protocol family1584lb # [ 0.581829] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1585worker1 # [ 0.599030] Trying to unpack rootfs image as initramfs...1586client # [ 0.586394] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1587lb # [ 0.583136] NET: Registered PF_UNIX/PF_LOCAL protocol family1588client # [ 0.587406] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1589lb # [ 0.584133] NET: Registered PF_XDP protocol family1590client # [ 0.588399] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1591lb # [ 0.585015] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1592client # [ 0.589531] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1593lb # [ 0.586086] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1594client # [ 0.590637] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1595lb # [ 0.587159] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1596worker1 # [ 0.603213] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1597client # [ 0.591682] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1598lb # [ 0.588238] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1599client # [ 0.593399] ACPI: \_SB_.GSIA: Enabled at IRQ 161600lb # [ 0.589374] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1601lb # [ 0.590505] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1602client # [ 0.596051] ACPI: \_SB_.GSIB: Enabled at IRQ 171603lb # [ 0.592446] ACPI: \_SB_.GSIA: Enabled at IRQ 161604client # [ 0.598133] ACPI: \_SB_.GSIC: Enabled at IRQ 181605lb # [ 0.595402] ACPI: \_SB_.GSIB: Enabled at IRQ 171606client # [ 0.600060] ACPI: \_SB_.GSID: Enabled at IRQ 191607lb # [ 0.597387] ACPI: \_SB_.GSIC: Enabled at IRQ 181608client # [ 0.601773] PCI: CLS 0 bytes, default 641609lb # [ 0.599379] ACPI: \_SB_.GSID: Enabled at IRQ 191610client # [ 0.602638] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1611client # [ 0.604396] Trying to unpack rootfs image as initramfs...1612lb # [ 0.601025] PCI: CLS 0 bytes, default 641613lb # [ 0.601991] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1614lb # [ 0.603841] Trying to unpack rootfs image as initramfs...1615worker2 # [ 0.649409] Initialise system trusted keyrings1616worker2 # [ 0.652259] workingset: timestamp_bits=40 max_order=19 bucket_order=01617worker1 # [ 0.644342] Initialise system trusted keyrings1618worker1 # [ 0.645331] workingset: timestamp_bits=40 max_order=19 bucket_order=01619worker2 # [ 0.672572] Key type asymmetric registered1620worker2 # [ 0.675211] Asymmetric key parser 'x509' registered1621worker2 # [ 0.676116] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1622worker2 # [ 0.679415] io scheduler mq-deadline registered1623worker2 # [ 0.682210] io scheduler kyber registered1624worker2 # [ 0.683685] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1625worker2 # [ 0.685020] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1626client # [ 0.648044] Initialise system trusted keyrings1627client # [ 0.651148] workingset: timestamp_bits=40 max_order=19 bucket_order=01628worker1 # [ 0.664882] Key type asymmetric registered1629worker2 # [ 0.691204] Linux agpgart interface v0.1031630worker2 # [ 0.691993] ACPI: bus type drm_connector registered1631lb # [ 0.649530] Initialise system trusted keyrings1632worker2 # [ 0.694719] usbcore: registered new interface driver usbserial_generic1633worker1 # [ 0.669937] Asymmetric key parser 'x509' registered1634lb # [ 0.653116] workingset: timestamp_bits=40 max_order=19 bucket_order=01635worker2 # [ 0.695897] usbserial: USB Serial support registered for generic1636worker1 # [ 0.670729] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1637worker2 # [ 0.698216] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1638worker2 # [ 0.698216] Please enable it if your BIOS has the CPPC option.1639worker1 # [ 0.674071] io scheduler mq-deadline registered1640worker1 # [ 0.674833] io scheduler kyber registered1641worker2 # [ 0.700556] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1642worker1 # [ 0.677457] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1643worker1 # [ 0.678763] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1644worker2 # [ 0.706372] drop_monitor: Initializing network drop monitor service1645worker2 # [ 0.707704] NET: Registered PF_INET6 protocol family1646worker1 # [ 0.684709] Linux agpgart interface v0.1031647worker2 # [ 0.710727] Segment Routing with IPv61648client # [ 0.672433] Key type asymmetric registered1649worker1 # [ 0.685478] ACPI: bus type drm_connector registered1650client # [ 0.675116] Asymmetric key parser 'x509' registered1651worker2 # [ 0.714221] In-situ OAM (IOAM) with IPv61652worker1 # [ 0.688399] usbcore: registered new interface driver usbserial_generic1653client # [ 0.676002] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1654worker2 # [ 0.715344] IPI shorthand broadcast: enabled1655worker1 # [ 0.689523] usbserial: USB Serial support registered for generic1656lb # [ 0.674406] Key type asymmetric registered1657lb # [ 0.675195] Asymmetric key parser 'x509' registered1658client # [ 0.679254] io scheduler mq-deadline registered1659client # [ 0.680023] io scheduler kyber registered1660worker1 # [ 0.691942] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1661worker1 # [ 0.691942] Please enable it if your BIOS has the CPPC option.1662worker1 # [ 0.694232] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1663lb # [ 0.678091] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1664client # [ 0.682688] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1665client # [ 0.683952] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1666lb # [ 0.681116] io scheduler mq-deadline registered1667worker2 # [ 0.723577] sched_clock: Marking stable (587014756, 136174859)->(794655217, -71465602)1668lb # [ 0.681898] io scheduler kyber registered1669worker1 # [ 0.700060] drop_monitor: Initializing network drop monitor service1670worker1 # [ 0.701315] NET: Registered PF_INET6 protocol family1671lb # [ 0.684748] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1672worker2 # [ 0.728449] registered taskstats version 11673client # [ 0.689876] Linux agpgart interface v0.1031674client # [ 0.690650] ACPI: bus type drm_connector registered1675worker2 # [ 0.729477] Loading compiled-in X.509 certificates1676worker1 # [ 0.704420] Segment Routing with IPv61677lb # [ 0.688078] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1678client # [ 0.693551] usbcore: registered new interface driver usbserial_generic1679client # [ 0.694706] usbserial: USB Serial support registered for generic1680worker1 # [ 0.707943] In-situ OAM (IOAM) with IPv61681lb # [ 0.691837] Linux agpgart interface v0.1031682worker1 # [ 0.708990] IPI shorthand broadcast: enabled1683lb # [ 0.692644] ACPI: bus type drm_connector registered1684client # [ 0.698117] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1685client # [ 0.698117] Please enable it if your BIOS has the CPPC option.1686lb # [ 0.695527] usbcore: registered new interface driver usbserial_generic1687lb # [ 0.696642] usbserial: USB Serial support registered for generic1688client # [ 0.700465] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1689client # [ 0.703271] drop_monitor: Initializing network drop monitor service1690lb # [ 0.699075] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1691client # [ 0.704454] NET: Registered PF_INET6 protocol family1692lb # [ 0.699075] Please enable it if your BIOS has the CPPC option.1693worker1 # [ 0.717190] sched_clock: Marking stable (582014082, 134903716)->(791170804, -74253006)1694lb # [ 0.701359] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1695client # [ 0.707580] Segment Routing with IPv61696worker2 # [ 0.747204] Demotion targets for Node 0: null1697worker1 # [ 0.722011] registered taskstats version 11698client # [ 0.710132] In-situ OAM (IOAM) with IPv61699worker1 # [ 0.722992] Loading compiled-in X.509 certificates1700lb # [ 0.706226] drop_monitor: Initializing network drop monitor service1701client # [ 0.711166] IPI shorthand broadcast: enabled1702lb # [ 0.707411] NET: Registered PF_INET6 protocol family1703worker2 # [ 0.750246] Key type .fscrypt registered1704worker2 # [ 0.750903] Key type fscrypt-provisioning registered1705worker2 # [ 0.751919] ima: No TPM chip found, activating TPM-bypass!1706lb # [ 0.710569] Segment Routing with IPv61707worker2 # [ 0.754222] ima: Allocated hash algorithm: sha11708worker2 # [ 0.755040] ima: No architecture policies found1709lb # [ 0.713082] In-situ OAM (IOAM) with IPv61710lb # [ 0.714244] IPI shorthand broadcast: enabled1711client # [ 0.719489] sched_clock: Marking stable (584014827, 135076974)->(794514097, -75422296)1712worker2 # [ 0.759202] PM: Magic number: 10:780:9931713worker2 # [ 0.760975] RAS: Correctable Errors collector initialized.1714client # [ 0.724195] registered taskstats version 11715client # [ 0.725200] Loading compiled-in X.509 certificates1716worker1 # [ 0.739353] Demotion targets for Node 0: null1717lb # [ 0.722804] sched_clock: Marking stable (588017467, 134039795)->(795372993, -73315731)1718worker1 # [ 0.742964] Key type .fscrypt registered1719lb # [ 0.726282] registered taskstats version 11720worker1 # [ 0.743616] Key type fscrypt-provisioning registered1721worker2 # [ 0.769962] clk: Disabling unused clocks1722lb # [ 0.727321] Loading compiled-in X.509 certificates1723worker1 # [ 0.744605] ima: No TPM chip found, activating TPM-bypass!1724worker2 # [ 0.772221] PM: genpd: Disabling unused power domains1725worker1 # [ 0.746941] ima: Allocated hash algorithm: sha11726worker1 # [ 0.747725] ima: No architecture policies found1727worker1 # [ 0.749930] PM: Magic number: 10:780:9931728worker1 # [ 0.751616] RAS: Correctable Errors collector initialized.1729client # [ 0.743104] Demotion targets for Node 0: null1730client # [ 0.746145] Key type .fscrypt registered1731client # [ 0.746837] Key type fscrypt-provisioning registered1732worker1 # [ 0.760605] clk: Disabling unused clocks1733client # [ 0.747832] ima: No TPM chip found, activating TPM-bypass!1734lb # [ 0.746077] Demotion targets for Node 0: null1735client # [ 0.750108] ima: Allocated hash algorithm: sha11736client # [ 0.750899] ima: No architecture policies found1737worker1 # [ 0.764950] PM: genpd: Disabling unused power domains1738lb # [ 0.749110] Key type .fscrypt registered1739lb # [ 0.749838] Key type fscrypt-provisioning registered1740lb # [ 0.750824] ima: No TPM chip found, activating TPM-bypass!1741client # [ 0.755109] PM: Magic number: 10:780:9931742client # [ 0.756688] RAS: Correctable Errors collector initialized.1743lb # [ 0.753077] ima: Allocated hash algorithm: sha11744lb # [ 0.753907] ima: No architecture policies found1745lb # [ 0.758068] PM: Magic number: 10:780:9931746lb # [ 0.759815] RAS: Correctable Errors collector initialized.1747client # [ 0.765815] clk: Disabling unused clocks1748client # [ 0.768212] PM: genpd: Disabling unused power domains1749lb # [ 0.768856] clk: Disabling unused clocks1750lb # [ 0.771077] PM: genpd: Disabling unused power domains1751worker2 # [ 0.940673] Freeing initrd memory: 29060K1752worker2 # [ 0.944207] Freeing unused decrypted memory: 2028K1753worker2 # [ 0.947152] Freeing unused kernel image (initmem) memory: 3652K1754worker2 # [ 0.948302] Write protecting the kernel read-only data: 32768k1755worker2 # [ 0.950301] Freeing unused kernel image (text/rodata gap) memory: 1184K1756worker2 # [ 0.951893] Freeing unused kernel image (rodata/data gap) memory: 720K1757worker1 # [ 0.928127] Freeing initrd memory: 29056K1758worker1 # [ 0.931272] Freeing unused decrypted memory: 2028K1759worker1 # [ 0.933937] Freeing unused kernel image (initmem) memory: 3652K1760worker1 # [ 0.935030] Write protecting the kernel read-only data: 32768k1761worker1 # [ 0.936898] Freeing unused kernel image (text/rodata gap) memory: 1184K1762worker1 # [ 0.938462] Freeing unused kernel image (rodata/data gap) memory: 720K1763client # [ 0.931496] Freeing initrd memory: 29056K1764client # [ 0.934691] Freeing unused decrypted memory: 2028K1765client # [ 0.937315] Freeing unused kernel image (initmem) memory: 3652K1766client # [ 0.938455] Write protecting the kernel read-only data: 32768k1767lb # [ 0.936249] Freeing initrd memory: 29064K1768client # [ 0.940343] Freeing unused kernel image (text/rodata gap) memory: 1184K1769client # [ 0.941860] Freeing unused kernel image (rodata/data gap) memory: 720K1770lb # [ 0.939618] Freeing unused decrypted memory: 2028K1771lb # [ 0.942455] Freeing unused kernel image (initmem) memory: 3652K1772lb # [ 0.943597] Write protecting the kernel read-only data: 32768k1773lb # [ 0.945536] Freeing unused kernel image (text/rodata gap) memory: 1184K1774lb # [ 0.947106] Freeing unused kernel image (rodata/data gap) memory: 720K1775worker2 # [ 1.002949] x86/mm: Checked W+X mappings: passed, no W+X pages found.1776worker2 # [ 1.004140] Run /init as init process1777worker2 # [ 1.016013] systemd[1]: Inserted module 'autofs4'1778worker1 # [ 0.990737] x86/mm: Checked W+X mappings: passed, no W+X pages found.1779worker1 # [ 0.991870] Run /init as init process1780worker1 # [ 1.002670] systemd[1]: Inserted module 'autofs4'1781client # [ 0.992899] x86/mm: Checked W+X mappings: passed, no W+X pages found.1782client # [ 0.994014] Run /init as init process1783worker2 # [ 1.036249] fuse: init (API version 7.45)1784lb # [ 0.998159] x86/mm: Checked W+X mappings: passed, no W+X pages found.1785lb # [ 0.999289] Run /init as init process1786client # [ 1.004840] systemd[1]: Inserted module 'autofs4'1787worker2 # [ 1.043823] ACPI: \_SB_.GSIG: Enabled at IRQ 221788worker2 # [ 1.046263] ACPI: \_SB_.GSIH: Enabled at IRQ 231789worker1 # [ 1.022491] fuse: init (API version 7.45)1790worker2 # [ 1.049428] ACPI: \_SB_.GSIE: Enabled at IRQ 201791worker2 # [ 1.051415] ACPI: \_SB_.GSIF: Enabled at IRQ 211792lb # [ 1.010762] systemd[1]: Inserted module 'autofs4'1793worker1 # [ 1.029946] ACPI: \_SB_.GSIG: Enabled at IRQ 221794worker2 # [ 1.056542] virtiofs virtio5: discovered new tag: nix-store1795worker1 # [ 1.032388] ACPI: \_SB_.GSIH: Enabled at IRQ 231796worker2 # [ 1.058129] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1797worker1 # [ 1.035476] ACPI: \_SB_.GSIE: Enabled at IRQ 201798client # [ 1.024641] fuse: init (API version 7.45)1799worker1 # [ 1.037519] ACPI: \_SB_.GSIF: Enabled at IRQ 211800worker2 # [ 1.065511] virtiofs virtio6: discovered new tag: shared1801worker2 # [ 1.067250] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1802worker1 # [ 1.043014] virtiofs virtio5: discovered new tag: nix-store1803worker2 # [ 1.070469] virtiofs virtio7: discovered new tag: xchg1804client # [ 1.032030] ACPI: \_SB_.GSIG: Enabled at IRQ 221805worker1 # [ 1.044524] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1806worker2 # [ 1.071911] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1807client # [ 1.034521] ACPI: \_SB_.GSIH: Enabled at IRQ 231808lb # [ 1.030772] fuse: init (API version 7.45)1809client # [ 1.037636] ACPI: \_SB_.GSIE: Enabled at IRQ 201810worker1 # [ 1.051326] virtiofs virtio6: discovered new tag: shared1811client # [ 1.039679] ACPI: \_SB_.GSIF: Enabled at IRQ 211812worker1 # [ 1.052847] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1813lb # [ 1.038295] ACPI: \_SB_.GSIG: Enabled at IRQ 221814worker1 # [ 1.056435] virtiofs virtio7: discovered new tag: xchg1815client # [ 1.044782] virtiofs virtio5: discovered new tag: nix-store1816lb # [ 1.041414] ACPI: \_SB_.GSIH: Enabled at IRQ 231817worker1 # [ 1.058003] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1818client # [ 1.046394] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1819lb # [ 1.044619] ACPI: \_SB_.GSIE: Enabled at IRQ 201820lb # [ 1.046862] ACPI: \_SB_.GSIF: Enabled at IRQ 211821worker2 # [ 1.091104] systemd[1]: Successfully made /usr/ read-only.1822client # [ 1.053565] virtiofs virtio6: discovered new tag: shared1823client # [ 1.055365] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1824lb # [ 1.052575] virtiofs virtio5: discovered new tag: nix-store1825lb # [ 1.054458] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1826client # [ 1.058739] virtiofs virtio7: discovered new tag: xchg1827client # [ 1.060234] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1828worker1 # [ 1.077648] systemd[1]: Successfully made /usr/ read-only.1829lb # [ 1.062563] virtiofs virtio6: discovered new tag: shared1830lb # [ 1.064108] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1831lb # [ 1.067755] virtiofs virtio7: discovered new tag: xchg1832lb # [ 1.069487] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1833client # [ 1.080895] systemd[1]: Successfully made /usr/ read-only.1834lb # [ 1.091076] systemd[1]: Successfully made /usr/ read-only.1835worker2 # [ 1.427712] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1836worker2 # [ 1.439774] systemd[1]: Detected virtualization kvm.1837worker2 # [ 1.441953] systemd[1]: Detected architecture x86-64.1838worker1 # [ 1.413323] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1839worker1 # [ 1.418387] systemd[1]: Detected virtualization kvm.1840worker2 # [ 1.444078] systemd[1]: Running in initrd.1841worker1 # [ 1.419247] systemd[1]: Detected architecture x86-64.1842worker1 # [ 1.420104] systemd[1]: Running in initrd.1843worker1 # [ 1.421183] systemd[1]: Initializing machine ID from random generator.1844worker2 # [ 1.446662] systemd[1]: Initializing machine ID from random generator.1845worker1 # [ 1.422457] systemd[1]: Hostname set to <worker1>.1846worker2 # [ 1.449582] systemd[1]: Hostname set to <worker2>.1847client # [ 1.417389] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1848client # [ 1.429417] systemd[1]: Detected virtualization kvm.1849client # [ 1.431520] systemd[1]: Detected architecture x86-64.1850client # [ 1.433629] systemd[1]: Running in initrd.1851client # [ 1.436167] systemd[1]: Initializing machine ID from random generator.1852client # [ 1.438973] systemd[1]: Hostname set to <client>.1853lb # [ 1.427732] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1854lb # [ 1.439712] systemd[1]: Detected virtualization kvm.1855lb # [ 1.441880] systemd[1]: Detected architecture x86-64.1856lb # [ 1.444087] systemd[1]: Running in initrd.1857lb # [ 1.446565] systemd[1]: Initializing machine ID from random generator.1858lb # [ 1.449440] systemd[1]: Hostname set to <lb>.1859worker1 # [ 1.605224] systemd[1]: bpf-restrict-fs: LSM BPF program attached1860worker1 # [ 1.642707] systemd[1]: Queued start job for default target Initrd Default Target.1861worker1 # [ 1.647184] systemd[1]: Created slice Slice /system/modprobe.1862worker1 # [ 1.648424] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1863worker1 # [ 1.649891] systemd[1]: Expecting device /dev/disk/by-label/nixos...1864worker2 # [ 1.674830] systemd[1]: bpf-restrict-fs: LSM BPF program attached1865worker1 # [ 1.651072] systemd[1]: Reached target Path Units.1866worker1 # [ 1.651941] systemd[1]: Reached target Slice Units.1867worker1 # [ 1.652804] systemd[1]: Reached target Swaps.1868worker1 # [ 1.653612] systemd[1]: Reached target Timer Units.1869worker1 # [ 1.654605] systemd[1]: Listening on D-Bus System Message Bus Socket.1870worker1 # [ 1.655871] systemd[1]: Listening on Journal Socket (/dev/log).1871worker1 # [ 1.657069] systemd[1]: Listening on Journal Sockets.1872worker1 # [ 1.658099] systemd[1]: Listening on udev Control Socket.1873worker1 # [ 1.659188] systemd[1]: Listening on udev Kernel Socket.1874worker1 # [ 1.660201] systemd[1]: Reached target Socket Units.1875worker1 # [ 1.661983] systemd[1]: Starting Create List of Static Device Nodes...1876worker1 # [ 1.665744] systemd[1]: Starting Load Kernel Module configfs...1877client # [ 1.661723] systemd[1]: bpf-restrict-fs: LSM BPF program attached1878worker1 # [ 1.675313] systemd[1]: Starting Journal Service...1879lb # [ 1.674293] systemd[1]: bpf-restrict-fs: LSM BPF program attached1880worker1 # [ 1.697111] systemd[1]: Starting Load Kernel Modules...1881worker2 # [ 1.729272] systemd[1]: Queued start job for default target Initrd Default Target.1882worker1 # [ 1.703070] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1883worker2 # [ 1.732576] systemd[1]: Created slice Slice /system/modprobe.1884worker1 # [ 1.708096] systemd[1]: Starting Coldplug All udev Devices...1885worker2 # [ 1.733815] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1886worker2 # [ 1.735230] systemd[1]: Expecting device /dev/disk/by-label/nixos...1887worker2 # [ 1.736387] systemd[1]: Reached target Path Units.1888worker2 # [ 1.737346] systemd[1]: Reached target Slice Units.1889worker2 # [ 1.738244] systemd[1]: Reached target Swaps.1890worker2 # [ 1.739039] systemd[1]: Reached target Timer Units.1891worker2 # [ 1.740054] systemd[1]: Listening on D-Bus System Message Bus Socket.1892worker2 # [ 1.741315] systemd[1]: Listening on Journal Socket (/dev/log).1893worker2 # [ 1.742597] systemd[1]: Listening on Journal Sockets.1894worker2 # [ 1.743629] systemd[1]: Listening on udev Control Socket.1895worker2 # [ 1.744709] systemd[1]: Listening on udev Kernel Socket.1896worker2 # [ 1.745725] systemd[1]: Reached target Socket Units.1897worker2 # [ 1.747597] systemd[1]: Starting Create List of Static Device Nodes...1898worker1 # [ 1.722240] systemd[1]: Finished Create List of Static Device Nodes.1899worker1 # [ 1.725320] systemd-journald[66]: Collecting audit messages is disabled.1900worker2 # [ 1.752262] systemd[1]: Starting Load Kernel Module configfs...1901client # [ 1.714363] systemd[1]: Queued start job for default target Initrd Default Target.1902worker1 # [ 1.729181] systemd[1]: modprobe@configfs.service: Deactivated successfully.1903client # [ 1.718360] systemd[1]: Created slice Slice /system/modprobe.1904client # [ 1.719584] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1905client # [ 1.721018] systemd[1]: Expecting device /dev/disk/by-label/nixos...1906client # [ 1.722192] systemd[1]: Reached target Path Units.1907client # [ 1.723103] systemd[1]: Reached target Slice Units.1908client # [ 1.723994] systemd[1]: Reached target Swaps.1909client # [ 1.724799] systemd[1]: Reached target Timer Units.1910worker1 # [ 1.738291] systemd[1]: Finished Load Kernel Module configfs.1911client # [ 1.725794] systemd[1]: Listening on D-Bus System Message Bus Socket.1912client # [ 1.727046] systemd[1]: Listening on Journal Socket (/dev/log).1913client # [ 1.728285] systemd[1]: Listening on Journal Sockets.1914client # [ 1.729350] systemd[1]: Listening on udev Control Socket.1915client # [ 1.730407] systemd[1]: Listening on udev Kernel Socket.1916worker1 # [ 1.742344] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1917client # [ 1.731397] systemd[1]: Reached target Socket Units.1918lb # [ 1.728303] systemd[1]: Queued start job for default target Initrd Default Target.1919client # [ 1.733278] systemd[1]: Starting Create List of Static Device Nodes...1920worker2 # [ 1.761882] systemd[1]: Starting Journal Service...1921lb # [ 1.732407] systemd[1]: Created slice Slice /system/modprobe.1922client # [ 1.737172] systemd[1]: Starting Load Kernel Module configfs...1923lb # [ 1.733572] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1924lb # [ 1.734946] systemd[1]: Expecting device /dev/disk/by-label/nixos...1925lb # [ 1.736028] systemd[1]: Reached target Path Units.1926lb # [ 1.736897] systemd[1]: Reached target Slice Units.1927lb # [ 1.737756] systemd[1]: Reached target Swaps.1928lb # [ 1.738523] systemd[1]: Reached target Timer Units.1929worker1 # [ 1.755010] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1930lb # [ 1.739447] systemd[1]: Listening on D-Bus System Message Bus Socket.1931lb # [ 1.740622] systemd[1]: Listening on Journal Socket (/dev/log).1932lb # [ 1.741775] systemd[1]: Listening on Journal Sockets.1933worker2 # [ 1.784389] systemd[1]: Starting Load Kernel Modules...1934lb # [ 1.742726] systemd[1]: Listening on udev Control Socket.1935lb # [ 1.743688] systemd[1]: Listening on udev Kernel Socket.1936lb # [ 1.744593] systemd[1]: Reached target Socket Units.1937worker1 # [ 1.760408] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1938lb # [ 1.746354] systemd[1]: Starting Create List of Static Device Nodes...1939worker2 # [ 1.791328] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1940lb # [ 1.750131] systemd[1]: Starting Load Kernel Module configfs...1941client # [ 1.745680] systemd[1]: Starting Journal Service...1942worker2 # [ 1.796375] systemd[1]: Starting Coldplug All udev Devices...1943worker1 # [ 1.771072] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1944worker2 # [ 1.809747] systemd-journald[66]: Collecting audit messages is disabled.1945client # [ 1.771281] systemd[1]: Starting Load Kernel Modules...1946worker2 # [ 1.811147] systemd[1]: Finished Create List of Static Device Nodes.1947lb # [ 1.759131] systemd[1]: Starting Journal Service...1948client # [ 1.775201] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1949worker2 # [ 1.817152] systemd[1]: modprobe@configfs.service: Deactivated successfully.1950worker1 # [ 1.793464] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1951client # [ 1.783336] systemd[1]: Starting Coldplug All udev Devices...1952worker2 # [ 1.823591] systemd[1]: Finished Load Kernel Module configfs.1953lb # [ 1.784126] systemd[1]: Starting Load Kernel Modules...1954worker1 # [ 1.802120] systemd[1]: Starting Create Static Device Nodes in /dev...1955worker2 # [ 1.829073] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1956lb # [ 1.788226] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1957client # [ 1.794452] systemd-journald[66]: Collecting audit messages is disabled.1958client # [ 1.796342] systemd[1]: Finished Create List of Static Device Nodes.1959worker1 # [ 1.813985] systemd[1]: Finished Load Kernel Modules.1960lb # [ 1.797528] systemd[1]: Starting Coldplug All udev Devices...1961client # [ 1.803557] systemd[1]: modprobe@configfs.service: Deactivated successfully.1962worker2 # [ 1.843260] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1963worker1 # [ 1.820140] systemd[1]: Starting Apply Kernel Variables...1964lb # [ 1.804634] systemd-journald[66]: Collecting audit messages is disabled.1965client # [ 1.809478] systemd[1]: Finished Load Kernel Module configfs.1966worker2 # [ 1.848426] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1967lb # [ 1.812155] systemd[1]: Finished Create List of Static Device Nodes.1968client # [ 1.816508] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1969lb # [ 1.815770] systemd[1]: modprobe@configfs.service: Deactivated successfully.1970worker2 # [ 1.859409] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1971lb # [ 1.822466] systemd[1]: Finished Load Kernel Module configfs.1972client # [ 1.828191] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1973worker1 # [ 1.841484] systemd[1]: Finished Create Static Device Nodes in /dev.1974lb # [ 1.827445] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1975client # [ 1.831997] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1976worker1 # [ 1.847566] systemd[1]: Reached target Preparation for Local File Systems.1977worker1 # [ 1.714450] systemd-modules-load[67]: Inserted module 'dm_mod'1978worker1 # [ 1.718265] systemd-modules-load[67]: Inserted module 'virtio_balloon'1979worker1 # [ 1.719254] systemd-modules-load[67]: Inserted module 'virtio_gpu'1980worker1 # [ 1.855639] systemd[1]: Reached target Local File Systems.1981lb # [ 1.840183] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1982worker2 # [ 1.883780] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1983client # [ 1.845307] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1984worker1 # [ 1.860112] systemd[1]: Starting Rule-based Manager for Device Events and Files...1985worker1 # [ 1.866084] systemd[1]: Started Journal Service.1986lb # [ 1.848383] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1987worker1 # [ 1.736216] systemd[1]: Finished Apply Kernel Variables.1988worker2 # [ 1.897287] systemd[1]: Starting Create Static Device Nodes in /dev...1989lb # [ 1.858097] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1990client # [ 1.867755] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1991worker1 # [ 1.747151] systemd[1]: Starting Create System Files and Directories...1992worker2 # [ 1.908315] systemd[1]: Finished Load Kernel Modules.1993client # [ 1.876290] systemd[1]: Starting Create Static Device Nodes in /dev...1994worker2 # [ 1.915539] systemd[1]: Starting Apply Kernel Variables...1995lb # [ 1.877616] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1996client # [ 1.888256] systemd[1]: Finished Load Kernel Modules.1997lb # [ 1.887284] systemd[1]: Starting Create Static Device Nodes in /dev...1998worker1 # [ 1.772372] systemd[1]: Finished Create System Files and Directories.1999worker2 # [ 1.798253] systemd-modules-load[67]: Inserted module 'dm_mod'2000client # [ 1.896377] systemd[1]: Starting Apply Kernel Variables...
loading 2,483 hidden lines…
4484client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath4485client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath, in 0.12 seconds)4486client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4487client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.01 seconds)4488client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4489client # copying 3 paths...4490client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4491client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4492client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4493client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.16 seconds)4494(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 4.77 seconds)4495subtest: repeat is answered Cached by the scheduler without building4496worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4497worker1 # finding garbage collector roots...4498worker1 # deleting specified paths...4499worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4500worker1 # deleting unused links...4501worker1 # note: hard linking is currently saving 0.0 KiB4502worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.17 seconds)4503worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4504worker2 # finding garbage collector roots...4505worker2 # deleting specified paths...4506worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4507worker2 # deleting unused links...4508worker2 # note: hard linking is currently saving 0.0 KiB4509worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.16 seconds)4510client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&24511worker2 # [ 44.424559] nix-grpc-daemon[801]: ts=2026-09-23T16:00:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:424524512client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4513client # /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4514worker2 # [ 44.443819] nix-grpc-daemon[801]: ts=2026-09-23T16:00:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:424524515worker2 # [ 44.452915] nix-grpc-daemon[871]: time=2026-09-23T16:00:14.457Z level=INFO msg="All 2 paths already cached"4516worker1 # [ 44.444567] nix-grpc-daemon[512]: ts=2026-09-23T16:00:13Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14517client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2, in 0.20 seconds)4518client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath4519client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath, in 0.10 seconds)4520lb: must succeed: curl -sf localhost:9901/clusters4521lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4522lb: must succeed: curl -sf localhost:9901/clusters4523lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4524worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4525worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4526(finished: subtest: repeat is answered Cached by the scheduler without building, in 0.81 seconds)4527subtest: inputs already in the cache are not sent to the scheduler4528lb: must succeed: curl -sf localhost:9901/clusters4529lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4530lb: must succeed: curl -sf localhost:9901/clusters4531lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4532worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4533worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.04 seconds)4534client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&24535worker1 # [ 44.974438] nix-grpc-daemon[512]: ts=2026-09-23T16:00:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:455564536client # don't know how to build these paths:4537client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4538worker2 # [ 45.009297] nix-grpc-daemon[801]: ts=2026-09-23T16:00:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:424524539worker2 # [ 45.019443] nix-grpc-daemon[871]: time=2026-09-23T16:00:15.023Z level=INFO msg="All 2 paths already cached"4540worker1 # [ 45.008283] nix-grpc-daemon[512]: ts=2026-09-23T16:00:14Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04541worker1 # [ 45.012383] nix-grpc-daemon[512]: ts=2026-09-23T16:00:14Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.4:50051 clients=14542worker2 # [ 45.044610] nix-grpc-daemon[801]: ts=2026-09-23T16:00:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:424524543worker2 # [ 45.058836] nix-grpc-daemon[801]: ts=2026-09-23T16:00:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:424524544worker2 # [ 45.073437] nix-grpc-daemon[871]: time=2026-09-23T16:00:15.078Z level=INFO msg="All 2 paths already cached"4545client # copying 1 paths to 'lb:50051'...4546client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4547worker2 # [ 45.082856] nix-grpc-daemon[801]: ts=2026-09-23T16:00:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:424524548worker2 # [ 45.094171] nix-daemon[858]: accepted connection from pid 801, user nix-grpc-daemon (trusted)4549worker2 # [ 45.202858] nix-daemon[858]: accepted connection from pid 988, user nix-grpc-daemon (trusted)4550worker2 # [ 45.216565] nix-daemon[858]: reaped child process 991, status = succeeded4551lb # [ 45.186184] niks3-server[950]: 2026/09/23 16:00:15 INFO Received push request method=POST path=/api/pushes4552worker2 # [ 45.234691] nix-grpc-daemon[871]: time=2026-09-23T16:00:15.239Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4553worker2 # [ 45.236709] nix-grpc-daemon[871]: time=2026-09-23T16:00:15.240Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4554lb # [ 45.221843] niks3-server[950]: 2026/09/23 16:00:15 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4555lb # [ 45.227117] niks3-server[950]: 2026/09/23 16:00:15 INFO Received sign narinfos request method=POST path=/api/pushes/7/sign4556worker2 # [ 45.273503] nix-grpc-daemon[871]: time=2026-09-23T16:00:15.278Z level=INFO msg="Uploading 1 narinfos"4557lb # [ 45.232596] niks3-server[950]: 2026/09/23 16:00:15 INFO Signed narinfos id=7 count=14558lb # [ 45.238416] niks3-server[950]: 2026/09/23 16:00:15 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4559lb # [ 45.248567] niks3-server[950]: 2026/09/23 16:00:15 INFO Received complete push request method=POST path=/api/pushes/7/complete4560lb # [ 45.255656] niks3-server[950]: 2026/09/23 16:00:15 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4561worker2 # [ 45.295718] nix-grpc-daemon[871]: time=2026-09-23T16:00:15.300Z level=INFO msg="Upload complete. (188ms)"4562worker2 # [ 45.299826] nix-grpc-daemon[801]: ts=2026-09-23T16:00:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42452 duration_s=0 paths=1 nar_bytes_in=7524563worker2 # [ 45.304621] nix-grpc-daemon[801]: ts=2026-09-23T16:00:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:424524564worker2 # [ 45.316024] nix-daemon[858]: accepted connection from pid 801, user nix-grpc-daemon (trusted)4565worker2 # [ 45.319789] nix-daemon[858]: reaped child process 986, status = succeeded4566worker2 # [ 45.324451] nix-daemon[858]: accepted connection from pid 801, user nix-grpc-daemon (trusted)4567client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4568client # farm-top> worker2: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4569client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4570client # farm-top> LOG-farm-top-t1b4571worker2 # [ 46.752955] nix-daemon[858]: reaped child process 995, status = succeeded4572worker2 # [ 46.837810] nix-daemon[858]: accepted connection from pid 1011, user nix-grpc-daemon (trusted)4573worker2 # [ 46.851087] nix-daemon[858]: reaped child process 1014, status = succeeded4574lb # [ 46.817984] envoy[918]: conn peer=[2001:db8:1::1]:45588 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=406 ms=19354575lb # [ 46.822977] envoy[918]: conn peer=[2001:db8:1::1]:45598 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=694576lb # [ 46.826505] envoy[918]: conn peer=[2001:db8:1::1]:45608 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=114577lb # [ 46.831524] envoy[918]: conn peer=[2001:db8:1::1]:45622 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=124578lb # [ 46.835267] envoy[918]: conn peer=[2001:db8:1::1]:45630 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9464 tx=4373 ms=42754579lb # [ 46.840521] envoy[918]: conn peer=[2001:db8:1::1]:45382 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=584580lb # [ 46.844725] niks3-server[950]: 2026/09/23 16:00:16 INFO Received push request method=POST path=/api/pushes4581worker2 # [ 46.892609] nix-grpc-daemon[871]: time=2026-09-23T16:00:16.897Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4582worker2 # [ 46.896794] nix-grpc-daemon[871]: time=2026-09-23T16:00:16.898Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4583lb # [ 46.883767] niks3-server[950]: 2026/09/23 16:00:16 INFO Received sign narinfos request method=POST path=/api/pushes/8/sign4584lb # [ 46.888209] niks3-server[950]: 2026/09/23 16:00:16 INFO Signed narinfos id=8 count=14585worker2 # [ 46.931613] nix-grpc-daemon[871]: time=2026-09-23T16:00:16.936Z level=INFO msg="Uploading 1 narinfos"4586lb # [ 46.898141] niks3-server[950]: 2026/09/23 16:00:16 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4587lb # [ 46.902400] niks3-server[950]: 2026/09/23 16:00:16 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4588lb # [ 46.908062] niks3-server[950]: 2026/09/23 16:00:16 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4589lb # [ 46.909781] niks3-server[950]: 2026/09/23 16:00:16 INFO Received complete push request method=POST path=/api/pushes/8/complete4590lb # [ 46.913735] niks3-server[950]: 2026/09/23 16:00:16 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4591worker2 # [ 46.953497] nix-grpc-daemon[871]: time=2026-09-23T16:00:16.958Z level=INFO msg="Upload complete. (221ms)"4592worker2 # [ 46.959749] nix-grpc-daemon[801]: ts=2026-09-23T16:00:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42452 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14593worker2 # [ 46.974396] nix-daemon[858]: reaped child process 993, status = succeeded4594client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.13 seconds)4595client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath4596client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.11 seconds)4597lb: must succeed: curl -sf localhost:9901/clusters4598lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4599lb: must succeed: curl -sf localhost:9901/clusters4600lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4601worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4602worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.04 seconds)4603(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.45 seconds)4604subtest: outputs read back through any worker carry the cache signature4605client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&24606worker2 # [ 47.358521] nix-grpc-daemon[801]: ts=2026-09-23T16:00:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:424524607client # don't know how to build these paths:4608client # /nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv4609worker1 # [ 47.344655] nix-grpc-daemon[512]: ts=2026-09-23T16:00:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455564610worker1 # [ 47.367230] nix-grpc-daemon[512]: ts=2026-09-23T16:00:16Z level=debug event=want client=4 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv cached=04611worker1 # [ 47.372768] nix-grpc-daemon[512]: ts=2026-09-23T16:00:16Z level=debug event=assigned drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv worker=192.168.1.3:50051 clients=14612worker1 # [ 47.377907] nix-grpc-daemon[512]: ts=2026-09-23T16:00:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455564613worker1 # [ 47.390119] nix-grpc-daemon[512]: ts=2026-09-23T16:00:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455564614client # copying 1 paths to 'lb:50051'...4615client # copying path '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' to 'lb:50051'...4616worker1 # [ 47.402316] nix-grpc-daemon[512]: ts=2026-09-23T16:00:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:455564617worker1 # [ 47.417543] nix-daemon[865]: accepted connection from pid 512, user nix-grpc-daemon (trusted)4618worker1 # [ 47.434712] nix-grpc-daemon[512]: ts=2026-09-23T16:00:16Z level=info event=niks3_push_started pid=9174619worker1 # [ 47.559798] nix-grpc-daemon[917]: time=2026-09-23T16:00:16.769Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""4620worker1 # [ 47.650535] nix-daemon[865]: accepted connection from pid 930, user nix-grpc-daemon (trusted)4621worker1 # [ 47.664196] nix-daemon[865]: reaped child process 933, status = succeeded4622lb # [ 47.653338] niks3-server[950]: 2026/09/23 16:00:17 INFO Received push request method=POST path=/api/pushes4623worker1 # [ 47.677742] nix-grpc-daemon[917]: time=2026-09-23T16:00:16.888Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4624worker1 # [ 47.679290] nix-grpc-daemon[917]: time=2026-09-23T16:00:16.889Z level=INFO msg="Uploading lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv (504B)"4625lb # [ 47.693358] niks3-server[950]: 2026/09/23 16:00:17 INFO Received sign narinfos request method=POST path=/api/pushes/9/sign4626lb # [ 47.701730] niks3-server[950]: 2026/09/23 16:00:17 INFO Registered completed upload object_key=nar/0bddczcnpndyzwmafx9yqfqrkrp1jg9rr5hsixz6rpy99kg84g1g.nar.zst4627lb # [ 47.707150] niks3-server[950]: 2026/09/23 16:00:17 INFO Signed narinfos id=9 count=14628worker1 # [ 47.727294] nix-grpc-daemon[917]: time=2026-09-23T16:00:16.937Z level=INFO msg="Uploading 1 narinfos"4629lb # [ 47.714581] niks3-server[950]: 2026/09/23 16:00:17 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.ls4630lb # [ 47.728396] niks3-server[950]: 2026/09/23 16:00:17 INFO Received complete push request method=POST path=/api/pushes/9/complete4631worker1 # [ 47.754319] nix-grpc-daemon[917]: time=2026-09-23T16:00:16.964Z level=INFO msg="Upload complete. (190ms)"4632lb # [ 47.740220] niks3-server[950]: 2026/09/23 16:00:17 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.narinfo4633worker1 # [ 47.757349] nix-grpc-daemon[512]: ts=2026-09-23T16:00:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45556 duration_s=0 paths=1 nar_bytes_in=5044634worker1 # [ 47.763672] nix-grpc-daemon[512]: ts=2026-09-23T16:00:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455564635worker1 # [ 47.774647] nix-daemon[865]: accepted connection from pid 512, user nix-grpc-daemon (trusted)4636worker1 # [ 47.779993] nix-daemon[865]: reaped child process 915, status = succeeded4637worker1 # [ 47.783206] nix-daemon[865]: accepted connection from pid 512, user nix-grpc-daemon (trusted)4638client # building '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' on lb:50051...4639client # farm-a> node-a: building lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv4640client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4641client # farm-a> LOG-farm-a-sig4642worker1 # [ 48.621250] nix-daemon[865]: reaped child process 937, status = succeeded4643worker1 # [ 48.696550] nix-daemon[865]: accepted connection from pid 953, user nix-grpc-daemon (trusted)4644worker1 # [ 48.710722] nix-daemon[865]: reaped child process 956, status = succeeded4645lb # [ 48.701274] niks3-server[950]: 2026/09/23 16:00:18 INFO Received push request method=POST path=/api/pushes4646worker1 # [ 48.725349] nix-grpc-daemon[917]: time=2026-09-23T16:00:17.935Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4647worker1 # [ 48.726954] nix-grpc-daemon[917]: time=2026-09-23T16:00:17.937Z level=INFO msg="Uploading qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a (128B)"4648lb # [ 48.740172] niks3-server[950]: 2026/09/23 16:00:18 INFO Received sign narinfos request method=POST path=/api/pushes/10/sign4649lb # [ 48.745769] niks3-server[950]: 2026/09/23 16:00:18 INFO Signed narinfos id=10 count=14650worker1 # [ 48.764795] nix-grpc-daemon[917]: time=2026-09-23T16:00:17.974Z level=INFO msg="Uploading 1 narinfos"4651lb # [ 48.758723] niks3-server[950]: 2026/09/23 16:00:18 INFO Registered completed upload object_key=log/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv4652lb # [ 48.763152] niks3-server[950]: 2026/09/23 16:00:18 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4653lb # [ 48.772838] niks3-server[950]: 2026/09/23 16:00:18 INFO Received complete push request method=POST path=/api/pushes/10/complete4654lb # [ 48.779431] niks3-server[950]: 2026/09/23 16:00:18 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.ls4655lb # [ 48.785636] niks3-server[950]: 2026/09/23 16:00:18 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.narinfo4656worker1 # [ 48.800996] nix-grpc-daemon[917]: time=2026-09-23T16:00:18.011Z level=INFO msg="Upload complete. (210ms)"4657worker1 # [ 48.807844] nix-grpc-daemon[512]: ts=2026-09-23T16:00:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45556 duration_s=1 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv assign_id=5 outputs=14658worker1 # [ 48.818727] nix-grpc-daemon[512]: ts=2026-09-23T16:00:18Z level=debug event=want client=4 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv cached=04659worker1 # [ 48.824267] nix-grpc-daemon[512]: ts=2026-09-23T16:00:18Z level=debug event=assigned drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv worker=192.168.1.3:50051 clients=14660worker1 # [ 48.828772] nix-grpc-daemon[512]: ts=2026-09-23T16:00:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455564661worker1 # [ 48.837604] nix-daemon[865]: reaped child process 935, status = succeeded4662worker1 # [ 48.839186] nix-grpc-daemon[512]: ts=2026-09-23T16:00:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455564663worker1 # [ 48.846447] nix-grpc-daemon[917]: time=2026-09-23T16:00:18.056Z level=INFO msg="All 1 paths already cached"4664client # copying 1 paths to 'lb:50051'...4665worker1 # [ 48.852097] nix-grpc-daemon[512]: ts=2026-09-23T16:00:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:455564666client # copying path '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' to 'lb:50051'...4667worker1 # [ 48.857054] nix-daemon[865]: accepted connection from pid 512, user nix-grpc-daemon (trusted)4668worker1 # [ 48.943209] nix-daemon[865]: accepted connection from pid 962, user nix-grpc-daemon (trusted)4669worker1 # [ 48.958229] nix-daemon[865]: reaped child process 965, status = succeeded4670lb # [ 48.947660] niks3-server[950]: 2026/09/23 16:00:19 INFO Received push request method=POST path=/api/pushes4671worker1 # [ 48.971097] nix-grpc-daemon[917]: time=2026-09-23T16:00:18.181Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4672worker1 # [ 48.972752] nix-grpc-daemon[917]: time=2026-09-23T16:00:18.182Z level=INFO msg="Uploading 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv (624B)"4673lb # [ 48.979266] niks3-server[950]: 2026/09/23 16:00:19 INFO Registered completed upload object_key=nar/1m1mvw8bmb91glp26qj699lxqv98q2iz8wm4ik3miz0xj6vi3mw6.nar.zst4674lb # [ 48.985295] niks3-server[950]: 2026/09/23 16:00:19 INFO Received sign narinfos request method=POST path=/api/pushes/11/sign4675lb # [ 48.989805] niks3-server[950]: 2026/09/23 16:00:19 INFO Signed narinfos id=11 count=14676worker1 # [ 49.009040] nix-grpc-daemon[917]: time=2026-09-23T16:00:18.219Z level=INFO msg="Uploading 1 narinfos"4677lb # [ 49.003860] niks3-server[950]: 2026/09/23 16:00:19 INFO Received complete push request method=POST path=/api/pushes/11/complete4678lb # [ 49.012805] niks3-server[950]: 2026/09/23 16:00:19 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.ls4679lb # [ 49.017361] niks3-server[950]: 2026/09/23 16:00:19 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.narinfo4680worker1 # [ 49.037719] nix-grpc-daemon[917]: time=2026-09-23T16:00:18.247Z level=INFO msg="Upload complete. (174ms)"4681worker1 # [ 49.042515] nix-grpc-daemon[512]: ts=2026-09-23T16:00:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45556 duration_s=0 paths=1 nar_bytes_in=6244682worker1 # [ 49.049332] nix-grpc-daemon[512]: ts=2026-09-23T16:00:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455564683worker1 # [ 49.057519] nix-daemon[865]: accepted connection from pid 512, user nix-grpc-daemon (trusted)4684worker1 # [ 49.066361] nix-daemon[865]: reaped child process 960, status = succeeded4685worker1 # [ 49.071377] nix-daemon[865]: accepted connection from pid 512, user nix-grpc-daemon (trusted)4686client # building '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' on lb:50051...4687client # farm-b> node-a: building 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv4688client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4689client # farm-b> LOG-farm-b-sig4690worker1 # [ 50.486810] nix-daemon[865]: reaped child process 969, status = succeeded4691worker1 # [ 50.561898] nix-daemon[865]: accepted connection from pid 985, user nix-grpc-daemon (trusted)4692worker1 # [ 50.572995] nix-daemon[865]: reaped child process 988, status = succeeded4693lb # [ 50.565419] niks3-server[950]: 2026/09/23 16:00:20 INFO Received push request method=POST path=/api/pushes4694worker1 # [ 50.590216] nix-grpc-daemon[917]: time=2026-09-23T16:00:19.800Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4695worker1 # [ 50.591966] nix-grpc-daemon[917]: time=2026-09-23T16:00:19.802Z level=INFO msg="Uploading 2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b (176B)"4696lb # [ 50.603838] niks3-server[950]: 2026/09/23 16:00:20 INFO Received sign narinfos request method=POST path=/api/pushes/12/sign4697lb # [ 50.607326] niks3-server[950]: 2026/09/23 16:00:20 INFO Signed narinfos id=12 count=14698worker1 # [ 50.626416] nix-grpc-daemon[917]: time=2026-09-23T16:00:19.836Z level=INFO msg="Uploading 1 narinfos"4699lb # [ 50.616100] niks3-server[950]: 2026/09/23 16:00:20 INFO Registered completed upload object_key=nar/0y3xlf15z3z540fgrc9sgnn2rl0aj3w6x9mjqd50hhqgmaxbiivd.nar.zst4700lb # [ 50.621845] niks3-server[950]: 2026/09/23 16:00:20 INFO Registered completed upload object_key=log/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv4701lb # [ 50.626926] niks3-server[950]: 2026/09/23 16:00:20 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.ls4702lb # [ 50.633954] niks3-server[950]: 2026/09/23 16:00:20 INFO Received complete push request method=POST path=/api/pushes/12/complete4703worker1 # [ 50.653519] nix-grpc-daemon[917]: time=2026-09-23T16:00:19.863Z level=INFO msg="Upload complete. (174ms)"4704lb # [ 50.640148] niks3-server[950]: 2026/09/23 16:00:20 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.narinfo4705worker1 # [ 50.657477] nix-grpc-daemon[512]: ts=2026-09-23T16:00:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45556 duration_s=1 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv assign_id=6 outputs=14706worker1 # [ 50.668662] nix-grpc-daemon[512]: ts=2026-09-23T16:00:19Z level=debug event=want client=4 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv cached=04707worker1 # [ 50.673638] nix-grpc-daemon[512]: ts=2026-09-23T16:00:19Z level=debug event=assigned drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv worker=192.168.1.3:50051 clients=14708worker1 # [ 50.676956] nix-grpc-daemon[512]: ts=2026-09-23T16:00:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455564709worker1 # [ 50.682741] nix-daemon[865]: reaped child process 967, status = succeeded4710worker1 # [ 50.684173] nix-grpc-daemon[512]: ts=2026-09-23T16:00:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455564711worker1 # [ 50.689816] nix-grpc-daemon[917]: time=2026-09-23T16:00:19.900Z level=INFO msg="All 2 paths already cached"4712client # copying 1 paths to 'lb:50051'...4713worker1 # [ 50.695155] nix-grpc-daemon[512]: ts=2026-09-23T16:00:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:455564714client # copying path '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' to 'lb:50051'...4715worker1 # [ 50.700155] nix-daemon[865]: accepted connection from pid 512, user nix-grpc-daemon (trusted)4716worker1 # [ 50.784044] nix-daemon[865]: accepted connection from pid 994, user nix-grpc-daemon (trusted)4717worker1 # [ 50.798190] nix-daemon[865]: reaped child process 997, status = succeeded4718lb # [ 50.787193] niks3-server[950]: 2026/09/23 16:00:20 INFO Received push request method=POST path=/api/pushes4719worker1 # [ 50.811194] nix-grpc-daemon[917]: time=2026-09-23T16:00:20.021Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4720worker1 # [ 50.812769] nix-grpc-daemon[917]: time=2026-09-23T16:00:20.022Z level=INFO msg="Uploading cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv (752B)"4721lb # [ 50.817477] niks3-server[950]: 2026/09/23 16:00:20 INFO Received sign narinfos request method=POST path=/api/pushes/13/sign4722lb # [ 50.822587] niks3-server[950]: 2026/09/23 16:00:20 INFO Signed narinfos id=13 count=14723worker1 # [ 50.841099] nix-grpc-daemon[917]: time=2026-09-23T16:00:20.051Z level=INFO msg="Uploading 1 narinfos"4724lb # [ 50.826938] niks3-server[950]: 2026/09/23 16:00:20 INFO Registered completed upload object_key=nar/0w21abkvlzg18nqkl2x4zxs7d0376kbld1n0fkr68vcpqpq1g5zg.nar.zst4725lb # [ 50.834993] niks3-server[950]: 2026/09/23 16:00:20 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.ls4726lb # [ 50.840522] niks3-server[950]: 2026/09/23 16:00:20 INFO Received complete push request method=POST path=/api/pushes/13/complete4727worker1 # [ 50.859448] nix-grpc-daemon[917]: time=2026-09-23T16:00:20.069Z level=INFO msg="Upload complete. (153ms)"4728worker1 # [ 50.862367] nix-grpc-daemon[512]: ts=2026-09-23T16:00:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45556 duration_s=0 paths=1 nar_bytes_in=7524729lb # [ 50.847049] niks3-server[950]: 2026/09/23 16:00:20 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.narinfo4730worker1 # [ 50.865232] nix-grpc-daemon[512]: ts=2026-09-23T16:00:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455564731worker1 # [ 50.873162] nix-daemon[865]: accepted connection from pid 512, user nix-grpc-daemon (trusted)4732worker1 # [ 50.874468] nix-daemon[865]: reaped child process 992, status = succeeded4733worker1 # [ 50.879639] nix-daemon[865]: accepted connection from pid 512, user nix-grpc-daemon (trusted)4734client # building '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' on lb:50051...4735client # farm-top> node-a: building cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv4736client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4737client # farm-top> LOG-farm-top-sig4738worker1 # [ 51.720709] nix-daemon[865]: reaped child process 1001, status = succeeded4739worker1 # [ 51.798354] nix-daemon[865]: accepted connection from pid 1017, user nix-grpc-daemon (trusted)4740worker1 # [ 51.811746] nix-daemon[865]: reaped child process 1020, status = succeeded4741lb # [ 51.802659] niks3-server[950]: 2026/09/23 16:00:21 INFO Received push request method=POST path=/api/pushes4742worker1 # [ 51.826631] nix-grpc-daemon[917]: time=2026-09-23T16:00:21.036Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4743worker1 # [ 51.828146] nix-grpc-daemon[917]: time=2026-09-23T16:00:21.038Z level=INFO msg="Uploading dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top (232B)"4744lb # [ 51.843295] niks3-server[950]: 2026/09/23 16:00:21 INFO Registered completed upload object_key=nar/0h4zvhqjxsvdqli2l6i5ajh5sh40hz57gzchdirnvbisq2aks0k8.nar.zst4745lb # [ 51.851654] niks3-server[950]: 2026/09/23 16:00:21 INFO Received sign narinfos request method=POST path=/api/pushes/14/sign4746lb # [ 51.857151] niks3-server[950]: 2026/09/23 16:00:21 INFO Signed narinfos id=14 count=14747worker1 # [ 51.878302] nix-grpc-daemon[917]: time=2026-09-23T16:00:21.088Z level=INFO msg="Uploading 1 narinfos"4748lb # [ 51.865227] niks3-server[950]: 2026/09/23 16:00:21 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.ls4749lb # [ 51.870159] niks3-server[950]: 2026/09/23 16:00:21 INFO Registered completed upload object_key=log/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv4750lb # [ 51.880731] niks3-server[950]: 2026/09/23 16:00:21 INFO Received complete push request method=POST path=/api/pushes/14/complete4751lb # [ 51.888537] niks3-server[950]: 2026/09/23 16:00:21 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.narinfo4752worker1 # [ 51.903138] nix-grpc-daemon[917]: time=2026-09-23T16:00:21.113Z level=INFO msg="Upload complete. (216ms)"4753worker1 # [ 51.909668] nix-grpc-daemon[512]: ts=2026-09-23T16:00:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45556 duration_s=1 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv assign_id=7 outputs=14754worker1 # [ 51.925748] nix-daemon[865]: reaped child process 999, status = succeeded4755client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&2, in 4.76 seconds)4756client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath4757client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath, in 0.12 seconds)4758client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top4759worker1 # [ 52.170352] nix-grpc-daemon[512]: ts=2026-09-23T16:00:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:342744760worker1 # [ 52.176794] nix-grpc-daemon[512]: ts=2026-09-23T16:00:21Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:342744761client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.12 seconds)4762client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top4763worker2 # [ 52.319798] nix-grpc-daemon[801]: ts=2026-09-23T16:00:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:520844764client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):4765client # /nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a4766client # /nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b4767client # /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top4768worker2 # [ 52.343536] nix-grpc-daemon[801]: ts=2026-09-23T16:00:22Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:520844769worker2 # [ 52.350395] nix-grpc-daemon[801]: copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'http://lb:5751'...4770worker2 # [ 52.364830] nix-grpc-daemon[801]: copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'http://lb:5751'...4771worker2 # [ 52.377618] nix-grpc-daemon[801]: copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'http://lb:5751'...4772client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.20 seconds)4773client: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top4774client # finding garbage collector roots...4775client # removing stale temporary roots file "/nix/var/nix/temproots/886"4776client # removing stale temporary roots file "/nix/var/nix/temproots/878"4777client # removing stale temporary roots file "/nix/var/nix/temproots/845"4778client # removing stale temporary roots file "/nix/var/nix/temproots/824"4779client # removing stale temporary roots file "/nix/var/nix/temproots/808"4780client # removing stale temporary roots file "/nix/var/nix/temproots/857"4781client # removing stale temporary roots file "/nix/var/nix/temproots/865"4782client # removing stale temporary roots file "/nix/var/nix/temproots/901"4783client # deleting specified paths...4784client # deleting '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top'4785client # deleting unused links...4786client # note: hard linking is currently saving 0.0 KiB4787worker2 # [ 52.659210] nix-grpc-daemon[801]: ts=2026-09-23T16:00:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:424524788worker1 # [ 52.637840] nix-grpc-daemon[512]: ts=2026-09-23T16:00:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455564789worker1 # [ 52.646920] nix-grpc-daemon[917]: time=2026-09-23T16:00:21.857Z level=INFO msg="All 1 paths already cached"4790worker2 # [ 52.678434] nix-grpc-daemon[801]: ts=2026-09-23T16:00:22Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:424524791worker2 # [ 52.685783] nix-grpc-daemon[801]: ts=2026-09-23T16:00:22Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:424524792client # copying 3 paths...4793worker2 # [ 52.695808] nix-grpc-daemon[801]: ts=2026-09-23T16:00:22Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:424524794client # copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'grpc://lb:50051'...4795worker2 # [ 52.711847] nix-grpc-daemon[801]: ts=2026-09-23T16:00:22Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:424524796worker2 # [ 52.717483] nix-grpc-daemon[801]: ts=2026-09-23T16:00:22Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:42452 duration_s=0 paths=1 nar_bytes_out=2324797worker2 # [ 52.723949] nix-grpc-daemon[801]: ts=2026-09-23T16:00:22Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:424524798worker2 # [ 52.728294] nix-grpc-daemon[801]: ts=2026-09-23T16:00:22Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:42452 duration_s=0 paths=1 nar_bytes_out=1764799worker2 # [ 52.733705] nix-grpc-daemon[801]: ts=2026-09-23T16:00:22Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:424524800worker2 # [ 52.737590] nix-grpc-daemon[801]: ts=2026-09-23T16:00:22Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:42452 duration_s=0 paths=1 nar_bytes_out=1284801client # copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'grpc://lb:50051'...4802client # copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'grpc://lb:50051'...4803client: (finished: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.35 seconds)4804(finished: subtest: outputs read back through any worker carry the cache signature, in 5.55 seconds)4805subtest: two clients wanting the same drv build it once4806client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup4807client # [ 52.751578] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.4808client # Running as unit: dedup1.service; invocation ID: 1c389f50430b43f694bd28b2dc0e90734809client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 0.03 seconds)4810client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup4811client # [ 52.878860] nix[954]: don't know how to build these paths:4812client # [ 52.879745] nix[954]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4813worker2 # [ 52.915194] nix-grpc-daemon[801]: ts=2026-09-23T16:00:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:424524814worker1 # [ 52.899357] nix-grpc-daemon[512]: ts=2026-09-23T16:00:22Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04815worker2 # [ 52.926208] nix-grpc-daemon[801]: ts=2026-09-23T16:00:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:424524816worker1 # [ 52.903592] nix-grpc-daemon[512]: ts=2026-09-23T16:00:22Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.4:50051 clients=14817worker2 # [ 52.937832] nix-grpc-daemon[801]: ts=2026-09-23T16:00:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:424524818client # [ 52.910103] nix[954]: copying 1 paths to 'lb:50051'...4819client # [ 52.910901] nix[954]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4820worker2 # [ 52.948984] nix-grpc-daemon[801]: ts=2026-09-23T16:00:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:424524821worker2 # [ 52.957877] nix-daemon[858]: accepted connection from pid 801, user nix-grpc-daemon (trusted)4822worker2 # [ 53.071142] nix-daemon[858]: accepted connection from pid 1021, user nix-grpc-daemon (trusted)4823worker2 # [ 53.085400] nix-daemon[858]: reaped child process 1024, status = succeeded4824lb # [ 53.054369] niks3-server[950]: 2026/09/23 16:00:23 INFO Received push request method=POST path=/api/pushes4825worker2 # [ 53.103388] nix-grpc-daemon[871]: time=2026-09-23T16:00:23.107Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4826worker2 # [ 53.105367] nix-grpc-daemon[871]: time=2026-09-23T16:00:23.109Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4827lb # [ 53.085479] niks3-server[950]: 2026/09/23 16:00:23 INFO Received sign narinfos request method=POST path=/api/pushes/15/sign4828worker2 # [ 53.135488] nix-grpc-daemon[871]: time=2026-09-23T16:00:23.140Z level=INFO msg="Uploading 1 narinfos"4829lb # [ 53.094877] niks3-server[950]: 2026/09/23 16:00:23 INFO Signed narinfos id=15 count=14830lb # [ 53.102166] niks3-server[950]: 2026/09/23 16:00:23 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4831lb # [ 53.108534] niks3-server[950]: 2026/09/23 16:00:23 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4832lb # [ 53.119136] niks3-server[950]: 2026/09/23 16:00:23 INFO Received complete push request method=POST path=/api/pushes/15/complete4833worker2 # [ 53.168084] nix-grpc-daemon[871]: time=2026-09-23T16:00:23.172Z level=INFO msg="Upload complete. (195ms)"4834lb # [ 53.128870] niks3-server[950]: 2026/09/23 16:00:23 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4835worker2 # [ 53.172140] nix-grpc-daemon[801]: ts=2026-09-23T16:00:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42452 duration_s=0 paths=1 nar_bytes_in=4324836worker2 # [ 53.176912] nix-grpc-daemon[801]: ts=2026-09-23T16:00:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:424524837worker2 # [ 53.187147] nix-daemon[858]: accepted connection from pid 801, user nix-grpc-daemon (trusted)4838worker2 # [ 53.196132] nix-daemon[858]: reaped child process 1019, status = succeeded4839worker2 # [ 53.199749] nix-daemon[858]: accepted connection from pid 801, user nix-grpc-daemon (trusted)4840client # [ 53.161934] nix[954]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4841client # [ 53.805942] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.4842client # Running as unit: dedup2.service; invocation ID: fba723f66fc844858b5db91d78f8b01b4843client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 1.07 seconds)4844client: must succeed: sleep 24845worker2 # [ 54.011584] nix-grpc-daemon[801]: ts=2026-09-23T16:00:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:424524846client # [ 53.984345] nix[970]: this derivation will be built:4847worker2 # [ 54.024968] nix-grpc-daemon[801]: ts=2026-09-23T16:00:24Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:424524848client # [ 53.990437] nix[970]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4849worker2 # [ 54.038840] nix-grpc-daemon[801]: ts=2026-09-23T16:00:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:424524850worker1 # [ 54.012848] nix-grpc-daemon[512]: ts=2026-09-23T16:00:23Z level=debug event=want client=6 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04851client # [ 54.003650] nix[970]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4852client: (finished: must succeed: sleep 2, in 2.05 seconds)4853client: waiting for success: ! systemctl is-active dedup1 dedup24854worker2 # [ 56.234315] nix-daemon[858]: reaped child process 1028, status = succeeded4855worker2 # [ 56.320961] nix-daemon[858]: accepted connection from pid 1054, user nix-grpc-daemon (trusted)4856worker2 # [ 56.334734] nix-daemon[858]: reaped child process 1057, status = succeeded4857lb # [ 56.302990] niks3-server[950]: 2026/09/23 16:00:26 INFO Received push request method=POST path=/api/pushes4858worker2 # [ 56.352323] nix-grpc-daemon[871]: time=2026-09-23T16:00:26.356Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4859worker2 # [ 56.353958] nix-grpc-daemon[871]: time=2026-09-23T16:00:26.358Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4860lb # [ 56.349864] niks3-server[950]: 2026/09/23 16:00:26 INFO Received sign narinfos request method=POST path=/api/pushes/16/sign4861lb # [ 56.357336] niks3-server[950]: 2026/09/23 16:00:26 INFO Signed narinfos id=16 count=14862worker2 # [ 56.401755] nix-grpc-daemon[871]: time=2026-09-23T16:00:26.406Z level=INFO msg="Uploading 1 narinfos"4863lb # [ 56.363706] niks3-server[950]: 2026/09/23 16:00:26 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4864lb # [ 56.369782] niks3-server[950]: 2026/09/23 16:00:26 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4865lb # [ 56.373395] niks3-server[950]: 2026/09/23 16:00:26 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4866lb # [ 56.381905] niks3-server[950]: 2026/09/23 16:00:26 INFO Received complete push request method=POST path=/api/pushes/16/complete4867lb # [ 56.388575] niks3-server[950]: 2026/09/23 16:00:26 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4868worker2 # [ 56.432359] nix-grpc-daemon[871]: time=2026-09-23T16:00:26.436Z level=INFO msg="Upload complete. (222ms)"4869worker2 # [ 56.439931] nix-grpc-daemon[801]: ts=2026-09-23T16:00:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42452 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=14870worker2 # [ 56.447514] nix-grpc-daemon[801]: ts=2026-09-23T16:00:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42452 duration_s=2 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=14871worker2 # [ 56.460642] nix-daemon[858]: reaped child process 1026, status = succeeded4872client # [ 56.446678] systemd[1]: dedup2.service: Deactivated successfully.4873client # [ 56.450871] systemd[1]: dedup2.service: Consumed 136ms CPU time over 2.643s wall clock time, 33.4M memory peak, 6.5K incoming IP traffic, 5.9K outgoing IP traffic.4874client # [ 56.457294] systemd[1]: dedup1.service: Deactivated successfully.4875client # [ 56.460681] systemd[1]: dedup1.service: Consumed 113ms CPU time over 3.708s wall clock time, 41.5M memory peak, 7.1K incoming IP traffic, 8.3K outgoing IP traffic.4876lb # [ 56.818238] envoy[918]: conn peer=[2001:db8:1::1]:45390 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4221 tx=1896 ms=19744877lb # [ 56.821893] envoy[918]: conn peer=[2001:db8:1::1]:45406 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9507 tx=4355 ms=45924878lb # [ 56.825609] envoy[918]: conn peer=[2001:db8:1::1]:45422 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1034 tx=2870 ms=1014879lb # [ 56.829136] envoy[918]: conn peer=[2001:db8:1::1]:45436 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=384 tx=410 ms=464880lb # [ 56.833202] envoy[918]: conn peer=[2001:db8:1::1]:45446 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=329 ms=334881lb # [ 56.836568] envoy[918]: conn peer=[2001:db8:1::1]:45440 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=376 ms=404882lb # [ 56.840087] envoy[918]: conn peer=[2001:db8:1::1]:34666 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2981 tx=1521 ms=35374883lb # [ 56.843569] envoy[918]: conn peer=[2001:db8:1::1]:34678 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1576 tx=1595 ms=24414884client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.08 seconds)4885client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4886client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)4887worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4888worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4889worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4890worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4891worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4892worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4893worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4894worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4895(finished: subtest: two clients wanting the same drv build it once, in 4.65 seconds)4896subtest: --timeout and --max-silent-time reach the builder4897worker1 # [ 57.530803] nix-grpc-daemon[512]: ts=2026-09-23T16:00:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:455564898worker1 # [ 57.541941] nix-grpc-daemon[512]: ts=2026-09-23T16:00:26Z level=debug event=want client=7 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv cached=04899worker1 # [ 57.547867] nix-grpc-daemon[512]: ts=2026-09-23T16:00:26Z level=debug event=assigned drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv worker=192.168.1.3:50051 clients=14900worker1 # [ 57.553126] nix-grpc-daemon[512]: ts=2026-09-23T16:00:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455564901worker1 # [ 57.564693] nix-grpc-daemon[512]: ts=2026-09-23T16:00:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455564902worker1 # [ 57.576165] nix-grpc-daemon[512]: ts=2026-09-23T16:00:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:455564903worker1 # [ 57.584669] nix-daemon[865]: accepted connection from pid 512, user nix-grpc-daemon (trusted)4904worker1 # [ 57.681913] nix-daemon[865]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)4905worker1 # [ 57.694542] nix-daemon[865]: reaped child process 1050, status = succeeded4906lb # [ 57.686873] niks3-server[950]: 2026/09/23 16:00:27 INFO Received push request method=POST path=/api/pushes4907worker1 # [ 57.711743] nix-grpc-daemon[917]: time=2026-09-23T16:00:26.921Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4908worker1 # [ 57.713736] nix-grpc-daemon[917]: time=2026-09-23T16:00:26.923Z level=INFO msg="Uploading qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv (424B)"4909lb # [ 57.717424] niks3-server[950]: 2026/09/23 16:00:27 INFO Received sign narinfos request method=POST path=/api/pushes/17/sign4910lb # [ 57.728820] niks3-server[950]: 2026/09/23 16:00:27 INFO Signed narinfos id=17 count=14911worker1 # [ 57.748225] nix-grpc-daemon[917]: time=2026-09-23T16:00:26.958Z level=INFO msg="Uploading 1 narinfos"4912lb # [ 57.733641] niks3-server[950]: 2026/09/23 16:00:27 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.ls4913lb # [ 57.742723] niks3-server[950]: 2026/09/23 16:00:27 INFO Registered completed upload object_key=nar/1s4sciw269ksj1s7ymwz5cy4dw9n7bxv3k8wqw69h694spdh4ha2.nar.zst4914lb # [ 57.751964] niks3-server[950]: 2026/09/23 16:00:27 INFO Received complete push request method=POST path=/api/pushes/17/complete4915lb # [ 57.761758] niks3-server[950]: 2026/09/23 16:00:27 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.narinfo4916worker1 # [ 57.776951] nix-grpc-daemon[917]: time=2026-09-23T16:00:26.987Z level=INFO msg="Upload complete. (179ms)"4917worker1 # [ 57.781835] nix-grpc-daemon[512]: ts=2026-09-23T16:00:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45556 duration_s=0 paths=1 nar_bytes_in=4244918worker1 # [ 57.787712] nix-grpc-daemon[512]: ts=2026-09-23T16:00:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455564919worker1 # [ 57.800116] nix-daemon[865]: accepted connection from pid 512, user nix-grpc-daemon (trusted)4920worker1 # [ 57.809758] nix-daemon[865]: reaped child process 1045, status = succeeded4921worker1 # [ 57.812568] nix-daemon[865]: accepted connection from pid 512, user nix-grpc-daemon (trusted)4922worker1 # [ 63.624242] nix-grpc-daemon[512]: ts=2026-09-23T16:00:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45556 duration_s=5 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv assign_id=9 outputs=04923worker1 # [ 63.636295] nix-daemon[865]: reaped child process 1052, status = succeeded4924worker1 # [ 63.637528] nix-daemon[865]: reaped child process 1054, status = succeeded4925worker2 # [ 63.808951] nix-grpc-daemon[801]: ts=2026-09-23T16:00:33Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:424524926worker1 # [ 63.799183] nix-grpc-daemon[512]: ts=2026-09-23T16:00:33Z level=debug event=want client=8 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv cached=04927worker2 # [ 63.826585] nix-grpc-daemon[801]: ts=2026-09-23T16:00:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:424524928worker1 # [ 63.803761] nix-grpc-daemon[512]: ts=2026-09-23T16:00:33Z level=debug event=assigned drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv worker=192.168.1.4:50051 clients=14929worker2 # [ 63.839320] nix-grpc-daemon[801]: ts=2026-09-23T16:00:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:424524930worker2 # [ 63.850467] nix-grpc-daemon[801]: ts=2026-09-23T16:00:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:424524931worker2 # [ 63.861821] nix-daemon[858]: accepted connection from pid 801, user nix-grpc-daemon (trusted)4932worker2 # [ 63.973368] nix-daemon[858]: accepted connection from pid 1070, user nix-grpc-daemon (trusted)4933worker2 # [ 63.988700] nix-daemon[858]: reaped child process 1073, status = succeeded4934lb # [ 63.955653] niks3-server[950]: 2026/09/23 16:00:34 INFO Received push request method=POST path=/api/pushes4935worker2 # [ 64.005248] nix-grpc-daemon[871]: time=2026-09-23T16:00:34.009Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4936worker2 # [ 64.006731] nix-grpc-daemon[871]: time=2026-09-23T16:00:34.009Z level=INFO msg="Uploading fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv (424B)"4937lb # [ 63.988178] niks3-server[950]: 2026/09/23 16:00:34 INFO Received sign narinfos request method=POST path=/api/pushes/18/sign4938worker2 # [ 64.038157] nix-grpc-daemon[871]: time=2026-09-23T16:00:34.042Z level=INFO msg="Uploading 1 narinfos"4939lb # [ 63.997331] niks3-server[950]: 2026/09/23 16:00:34 INFO Signed narinfos id=18 count=14940lb # [ 64.006078] niks3-server[950]: 2026/09/23 16:00:34 INFO Registered completed upload object_key=nar/013y5sy2r6v0qdwv0dzpgvdfy08bds4mfh2qpvbgjazzrg2v71hp.nar.zst4941lb # [ 64.012159] niks3-server[950]: 2026/09/23 16:00:34 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.ls4942lb # [ 64.025103] niks3-server[950]: 2026/09/23 16:00:34 INFO Received complete push request method=POST path=/api/pushes/18/complete4943lb # [ 64.033797] niks3-server[950]: 2026/09/23 16:00:34 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.narinfo4944worker2 # [ 64.079055] nix-grpc-daemon[871]: time=2026-09-23T16:00:34.083Z level=INFO msg="Upload complete. (205ms)"4945worker2 # [ 64.084151] nix-grpc-daemon[801]: ts=2026-09-23T16:00:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42452 duration_s=0 paths=1 nar_bytes_in=4244946worker2 # [ 64.090333] nix-grpc-daemon[801]: ts=2026-09-23T16:00:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:424524947worker2 # [ 64.102114] nix-daemon[858]: accepted connection from pid 801, user nix-grpc-daemon (trusted)4948worker2 # [ 64.113191] nix-daemon[858]: reaped child process 1068, status = succeeded4949worker2 # [ 64.116575] nix-daemon[858]: accepted connection from pid 801, user nix-grpc-daemon (trusted)4950lb # [ 66.819166] envoy[918]: conn peer=[2001:db8:1::1]:34686 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2953 tx=933 ms=61084951worker2 # [ 70.330912] nix-grpc-daemon[801]: ts=2026-09-23T16:00:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42452 duration_s=6 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv assign_id=10 outputs=04952worker2 # [ 70.356444] nix-daemon[858]: reaped child process 1075, status = succeeded4953worker2 # [ 70.358842] nix-daemon[858]: reaped child process 1077, status = succeeded4954(finished: subtest: --timeout and --max-silent-time reach the builder, in 13.04 seconds)4955subtest: the first client leaving does not fail the second4956client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share4957client # [ 70.460899] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.4958client # Running as unit: share1.service; invocation ID: 28c973019cab43d6afe276e62d8a64c84959client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 0.06 seconds)4960client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share4961worker2 # [ 70.640310] nix-grpc-daemon[801]: ts=2026-09-23T16:00:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:424524962client # [ 70.608775] nix[1043]: don't know how to build these paths:4963client # [ 70.609665] nix[1043]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv4964worker1 # [ 70.631326] nix-grpc-daemon[512]: ts=2026-09-23T16:00:39Z level=debug event=want client=9 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=04965worker1 # [ 70.637212] nix-grpc-daemon[512]: ts=2026-09-23T16:00:39Z level=debug event=assigned drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv worker=192.168.1.3:50051 clients=14966worker1 # [ 70.641617] nix-grpc-daemon[512]: ts=2026-09-23T16:00:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455564967worker1 # [ 70.653695] nix-grpc-daemon[512]: ts=2026-09-23T16:00:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455564968client # [ 70.650957] nix[1043]: copying 1 paths to 'lb:50051'...4969client # [ 70.651799] nix[1043]: copying path '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' to 'lb:50051'...4970worker1 # [ 70.664731] nix-grpc-daemon[512]: ts=2026-09-23T16:00:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:455564971worker1 # [ 70.675672] nix-daemon[865]: accepted connection from pid 512, user nix-grpc-daemon (trusted)4972worker1 # [ 70.775716] nix-daemon[865]: accepted connection from pid 1081, user nix-grpc-daemon (trusted)4973worker1 # [ 70.788522] nix-daemon[865]: reaped child process 1084, status = succeeded4974lb # [ 70.781203] niks3-server[950]: 2026/09/23 16:00:40 INFO Received push request method=POST path=/api/pushes4975worker1 # [ 70.808580] nix-grpc-daemon[917]: time=2026-09-23T16:00:40.018Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4976worker1 # [ 70.809985] nix-grpc-daemon[917]: time=2026-09-23T16:00:40.018Z level=INFO msg="Uploading j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv (432B)"4977lb # [ 70.814160] niks3-server[950]: 2026/09/23 16:00:40 INFO Received sign narinfos request method=POST path=/api/pushes/19/sign4978worker1 # [ 70.839222] nix-grpc-daemon[917]: time=2026-09-23T16:00:40.049Z level=INFO msg="Uploading 1 narinfos"4979lb # [ 70.823246] niks3-server[950]: 2026/09/23 16:00:40 INFO Signed narinfos id=19 count=14980lb # [ 70.830819] niks3-server[950]: 2026/09/23 16:00:40 INFO Registered completed upload object_key=nar/0s0b0gfhn36p0aylk1qa64cgw01bi7a8ca93iv4zg2q7wqcjcqhs.nar.zst4981lb # [ 70.839734] niks3-server[950]: 2026/09/23 16:00:40 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.ls4982lb # [ 70.852784] niks3-server[950]: 2026/09/23 16:00:40 INFO Received complete push request method=POST path=/api/pushes/19/complete4983worker1 # [ 70.875418] nix-grpc-daemon[917]: time=2026-09-23T16:00:40.085Z level=INFO msg="Upload complete. (188ms)"4984lb # [ 70.860153] niks3-server[950]: 2026/09/23 16:00:40 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.narinfo4985worker1 # [ 70.879542] nix-grpc-daemon[512]: ts=2026-09-23T16:00:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45556 duration_s=0 paths=1 nar_bytes_in=4324986worker1 # [ 70.886342] nix-grpc-daemon[512]: ts=2026-09-23T16:00:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455564987worker1 # [ 70.895760] nix-daemon[865]: accepted connection from pid 512, user nix-grpc-daemon (trusted)4988worker1 # [ 70.905827] nix-daemon[865]: reaped child process 1079, status = succeeded4989worker1 # [ 70.909811] nix-daemon[865]: accepted connection from pid 512, user nix-grpc-daemon (trusted)4990client # [ 70.896951] nix[1043]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...4991client # [ 71.528578] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.4992client # Running as unit: share2.service; invocation ID: 82fc203aeb764286a25346f0202341094993client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 1.07 seconds)4994client: must succeed: sleep 2; systemctl kill -s INT share14995worker2 # [ 71.722811] nix-grpc-daemon[801]: ts=2026-09-23T16:00:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:424524996client # [ 71.690954] nix[1059]: don't know how to build these paths:4997client # [ 71.691852] nix[1059]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv4998worker1 # [ 71.710133] nix-grpc-daemon[512]: ts=2026-09-23T16:00:40Z level=debug event=want client=10 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=04999worker1 # [ 71.714445] nix-grpc-daemon[512]: ts=2026-09-23T16:00:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455565000client # [ 71.704284] nix[1059]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5001client # [ 73.688568] systemd[1]: share1.service: Sent signal SIGINT to main process 1043 (nix) on client request.5002client # [ 73.696113] nix[1043]: shutting downj2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5003client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.06 seconds)5004client: waiting for success: ! systemctl is-active share15005client # [ 73.703477] nix[1043]: error: interrupted by the user5006client # [ 73.744749] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE5007client # [ 73.746159] systemd[1]: share1.service: Failed with result 'exit-code'.5008client # [ 73.747772] systemd[1]: share1.service: Consumed 114ms CPU time over 3.283s wall clock time, 41.4M memory peak, 6K incoming IP traffic, 8K outgoing IP traffic.5009client: (finished: waiting for success: ! systemctl is-active share1, in 1.08 seconds)5010client: waiting for success: ! systemctl is-active share25011worker1 # [ 75.156552] nix-daemon[865]: reaped child process 1088, status = succeeded5012worker1 # [ 75.238879] nix-daemon[865]: accepted connection from pid 1120, user nix-grpc-daemon (trusted)5013worker1 # [ 75.252947] nix-daemon[865]: reaped child process 1123, status = succeeded5014lb # [ 75.244162] niks3-server[950]: 2026/09/23 16:00:45 INFO Received push request method=POST path=/api/pushes5015worker1 # [ 75.268054] nix-grpc-daemon[917]: time=2026-09-23T16:00:44.478Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5016worker1 # [ 75.270910] nix-grpc-daemon[917]: time=2026-09-23T16:00:44.479Z level=INFO msg="Uploading b9lg6znd0rinl36gy6ixq1dcj6b8bhhw-slow-share (120B)"5017lb # [ 75.289066] niks3-server[950]: 2026/09/23 16:00:45 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.ls5018lb # [ 75.293434] niks3-server[950]: 2026/09/23 16:00:45 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst5019lb # [ 75.302231] niks3-server[950]: 2026/09/23 16:00:45 INFO Received sign narinfos request method=POST path=/api/pushes/20/sign5020lb # [ 75.309265] niks3-server[950]: 2026/09/23 16:00:45 INFO Signed narinfos id=20 count=15021worker1 # [ 75.325753] nix-grpc-daemon[917]: time=2026-09-23T16:00:44.535Z level=INFO msg="Uploading 1 narinfos"5022lb # [ 75.315140] niks3-server[950]: 2026/09/23 16:00:45 INFO Registered completed upload object_key=log/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5023lb # [ 75.325298] niks3-server[950]: 2026/09/23 16:00:45 INFO Received complete push request method=POST path=/api/pushes/20/complete5024lb # [ 75.330400] niks3-server[950]: 2026/09/23 16:00:45 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.narinfo5025worker1 # [ 75.345818] nix-grpc-daemon[917]: time=2026-09-23T16:00:44.556Z level=INFO msg="Upload complete. (224ms)"5026worker1 # [ 75.353777] nix-grpc-daemon[512]: ts=2026-09-23T16:00:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45556 duration_s=4 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=15027worker1 # [ 75.360620] nix-grpc-daemon[512]: ts=2026-09-23T16:00:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45556 duration_s=3 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=15028worker1 # [ 75.373751] nix-daemon[865]: reaped child process 1086, status = succeeded5029client # [ 75.377176] systemd[1]: share2.service: Deactivated successfully.5030client # [ 75.380776] systemd[1]: share2.service: Consumed 124ms CPU time over 3.851s wall clock time, 37.5M memory peak, 6.1K incoming IP traffic, 5.7K outgoing IP traffic.5031client: (finished: waiting for success: ! systemctl is-active share2, in 1.08 seconds)5032client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }5033client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.03 seconds)5034(finished: subtest: the first client leaving does not fail the second, in 5.66 seconds)5035subtest: build hook: nix-daemon with builders = grpc://lb5036client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log5037client # [ 76.163910] systemd[1]: Started Nix Daemon.5038client # [ 76.251675] nix-daemon[1111]: accepted connection from pid 1108, user root (trusted)5039worker2 # [ 76.403456] nix-grpc-daemon[801]: ts=2026-09-23T16:00:46Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:424525040worker1 # [ 76.387202] nix-grpc-daemon[512]: ts=2026-09-23T16:00:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455565041worker1 # [ 76.403586] nix-grpc-daemon[512]: ts=2026-09-23T16:00:45Z level=debug event=want client=11 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=05042worker2 # [ 76.431362] nix-grpc-daemon[801]: ts=2026-09-23T16:00:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:424525043worker1 # [ 76.408177] nix-grpc-daemon[512]: ts=2026-09-23T16:00:45Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.4:50051 clients=15044worker2 # [ 76.445526] nix-grpc-daemon[801]: ts=2026-09-23T16:00:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:424525045worker2 # [ 76.457127] nix-grpc-daemon[801]: ts=2026-09-23T16:00:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:424525046worker2 # [ 76.468479] nix-daemon[858]: accepted connection from pid 801, user nix-grpc-daemon (trusted)5047worker2 # [ 76.577273] nix-daemon[858]: accepted connection from pid 1112, user nix-grpc-daemon (trusted)5048worker2 # [ 76.591469] nix-daemon[858]: reaped child process 1115, status = succeeded5049lb # [ 76.560308] niks3-server[950]: 2026/09/23 16:00:46 INFO Received push request method=POST path=/api/pushes5050worker2 # [ 76.608592] nix-grpc-daemon[871]: time=2026-09-23T16:00:46.613Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5051worker2 # [ 76.610525] nix-grpc-daemon[871]: time=2026-09-23T16:00:46.614Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"5052lb # [ 76.587432] niks3-server[950]: 2026/09/23 16:00:46 INFO Received sign narinfos request method=POST path=/api/pushes/21/sign5053worker2 # [ 76.639619] nix-grpc-daemon[871]: time=2026-09-23T16:00:46.644Z level=INFO msg="Uploading 1 narinfos"5054lb # [ 76.598750] niks3-server[950]: 2026/09/23 16:00:46 INFO Signed narinfos id=21 count=15055lb # [ 76.605487] niks3-server[950]: 2026/09/23 16:00:46 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst5056lb # [ 76.611102] niks3-server[950]: 2026/09/23 16:00:46 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls5057lb # [ 76.621153] niks3-server[950]: 2026/09/23 16:00:46 INFO Received complete push request method=POST path=/api/pushes/21/complete5058worker2 # [ 76.665530] nix-grpc-daemon[871]: time=2026-09-23T16:00:46.670Z level=INFO msg="Upload complete. (185ms)"5059lb # [ 76.627233] niks3-server[950]: 2026/09/23 16:00:46 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo5060worker2 # [ 76.669571] nix-grpc-daemon[801]: ts=2026-09-23T16:00:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42452 duration_s=0 paths=1 nar_bytes_in=5125061worker2 # [ 76.675253] nix-grpc-daemon[801]: ts=2026-09-23T16:00:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:424525062worker2 # [ 76.684481] nix-daemon[858]: accepted connection from pid 801, user nix-grpc-daemon (trusted)5063worker2 # [ 76.691151] nix-daemon[858]: reaped child process 1110, status = succeeded5064worker2 # [ 76.694056] nix-daemon[858]: accepted connection from pid 801, user nix-grpc-daemon (trusted)5065lb # [ 76.819657] envoy[918]: conn peer=[2001:db8:1::1]:33220 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2952 tx=936 ms=65345066lb # [ 76.823453] envoy[918]: conn peer=[2001:db8:1::1]:33232 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2965 tx=770 ms=31155067lb # [ 76.827153] envoy[918]: conn peer=[2001:db8:1::1]:33234 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1427 tx=1197 ms=36725068worker2 # [ 77.733283] nix-daemon[858]: reaped child process 1119, status = succeeded5069worker2 # [ 77.816921] nix-daemon[858]: accepted connection from pid 1135, user nix-grpc-daemon (trusted)5070worker2 # [ 77.830728] nix-daemon[858]: reaped child process 1138, status = succeeded5071lb # [ 77.798256] niks3-server[950]: 2026/09/23 16:00:47 INFO Received push request method=POST path=/api/pushes5072worker2 # [ 77.847209] nix-grpc-daemon[871]: time=2026-09-23T16:00:47.851Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5073worker2 # [ 77.848710] nix-grpc-daemon[871]: time=2026-09-23T16:00:47.853Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"5074lb # [ 77.841784] niks3-server[950]: 2026/09/23 16:00:47 INFO Received sign narinfos request method=POST path=/api/pushes/22/sign5075lb # [ 77.847428] niks3-server[950]: 2026/09/23 16:00:47 INFO Signed narinfos id=22 count=15076worker2 # [ 77.890701] nix-grpc-daemon[871]: time=2026-09-23T16:00:47.895Z level=INFO msg="Uploading 1 narinfos"5077lb # [ 77.856778] niks3-server[950]: 2026/09/23 16:00:47 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv5078lb # [ 77.862789] niks3-server[950]: 2026/09/23 16:00:47 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst5079lb # [ 77.867459] niks3-server[950]: 2026/09/23 16:00:47 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls5080lb # [ 77.874275] niks3-server[950]: 2026/09/23 16:00:47 INFO Received complete push request method=POST path=/api/pushes/22/complete5081worker2 # [ 77.921292] nix-grpc-daemon[871]: time=2026-09-23T16:00:47.925Z level=INFO msg="Upload complete. (220ms)"5082lb # [ 77.883611] niks3-server[950]: 2026/09/23 16:00:47 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo5083worker2 # [ 77.927423] nix-grpc-daemon[801]: ts=2026-09-23T16:00:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42452 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=12 outputs=15084worker1 # [ 77.908249] nix-grpc-daemon[512]: ts=2026-09-23T16:00:47Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:455565085worker2 # [ 77.941583] nix-daemon[858]: reaped child process 1117, status = succeeded5086worker1 # [ 77.918382] nix-grpc-daemon[512]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...5087worker2 # [ 77.966692] nix-grpc-daemon[801]: ts=2026-09-23T16:00:47Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:424525088worker2 # [ 77.972128] nix-grpc-daemon[801]: ts=2026-09-23T16:00:47Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:42452 duration_s=0 paths=1 nar_bytes_out=1285089worker1 # [ 78.117864] nix-grpc-daemon[512]: ts=2026-09-23T16:00:47Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:455565090worker1 # [ 78.125696] nix-grpc-daemon[512]: ts=2026-09-23T16:00:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455565091worker1 # [ 78.137432] nix-grpc-daemon[917]: time=2026-09-23T16:00:47.347Z level=INFO msg="All 1 paths already cached"5092worker1 # [ 78.151918] nix-grpc-daemon[512]: ts=2026-09-23T16:00:47Z level=debug event=want client=12 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=05093worker1 # [ 78.156937] nix-grpc-daemon[512]: ts=2026-09-23T16:00:47Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.4:50051 clients=15094worker2 # [ 78.184393] nix-grpc-daemon[801]: ts=2026-09-23T16:00:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:424525095worker2 # [ 78.197171] nix-grpc-daemon[801]: ts=2026-09-23T16:00:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:424525096worker2 # [ 78.209759] nix-grpc-daemon[871]: time=2026-09-23T16:00:48.214Z level=INFO msg="All 1 paths already cached"5097worker2 # [ 78.217322] nix-grpc-daemon[801]: ts=2026-09-23T16:00:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:424525098worker2 # [ 78.225647] nix-daemon[858]: accepted connection from pid 801, user nix-grpc-daemon (trusted)5099worker2 # [ 78.338281] nix-daemon[858]: accepted connection from pid 1142, user nix-grpc-daemon (trusted)5100worker2 # [ 78.352823] nix-daemon[858]: reaped child process 1145, status = succeeded5101lb # [ 78.319962] niks3-server[950]: 2026/09/23 16:00:48 INFO Received push request method=POST path=/api/pushes5102worker2 # [ 78.369965] nix-grpc-daemon[871]: time=2026-09-23T16:00:48.374Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5103worker2 # [ 78.371482] nix-grpc-daemon[871]: time=2026-09-23T16:00:48.376Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"5104lb # [ 78.351781] niks3-server[950]: 2026/09/23 16:00:48 INFO Received sign narinfos request method=POST path=/api/pushes/23/sign5105worker2 # [ 78.399424] nix-grpc-daemon[871]: time=2026-09-23T16:00:48.404Z level=INFO msg="Uploading 1 narinfos"5106lb # [ 78.359282] niks3-server[950]: 2026/09/23 16:00:48 INFO Signed narinfos id=23 count=15107lb # [ 78.368156] niks3-server[950]: 2026/09/23 16:00:48 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls5108lb # [ 78.373562] niks3-server[950]: 2026/09/23 16:00:48 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst5109lb # [ 78.387140] niks3-server[950]: 2026/09/23 16:00:48 INFO Received complete push request method=POST path=/api/pushes/23/complete5110lb # [ 78.394161] niks3-server[950]: 2026/09/23 16:00:48 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo5111worker2 # [ 78.434479] nix-grpc-daemon[871]: time=2026-09-23T16:00:48.439Z level=INFO msg="Upload complete. (194ms)"5112worker2 # [ 78.438564] nix-grpc-daemon[801]: ts=2026-09-23T16:00:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42452 duration_s=0 paths=1 nar_bytes_in=6245113worker2 # [ 78.445181] nix-grpc-daemon[801]: ts=2026-09-23T16:00:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:424525114worker2 # [ 78.452619] nix-daemon[858]: accepted connection from pid 801, user nix-grpc-daemon (trusted)5115worker2 # [ 78.463116] nix-daemon[858]: reaped child process 1140, status = succeeded5116worker2 # [ 78.466332] nix-daemon[858]: accepted connection from pid 801, user nix-grpc-daemon (trusted)5117worker2 # [ 79.682827] nix-daemon[858]: reaped child process 1149, status = succeeded5118worker2 # [ 79.765184] nix-daemon[858]: accepted connection from pid 1165, user nix-grpc-daemon (trusted)5119worker2 # [ 79.778914] nix-daemon[858]: reaped child process 1168, status = succeeded5120lb # [ 79.746872] niks3-server[950]: 2026/09/23 16:00:49 INFO Received push request method=POST path=/api/pushes5121worker2 # [ 79.796341] nix-grpc-daemon[871]: time=2026-09-23T16:00:49.800Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5122worker2 # [ 79.797988] nix-grpc-daemon[871]: time=2026-09-23T16:00:49.802Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"5123lb # [ 79.792535] niks3-server[950]: 2026/09/23 16:00:49 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst5124lb # [ 79.798356] niks3-server[950]: 2026/09/23 16:00:49 INFO Received sign narinfos request method=POST path=/api/pushes/24/sign5125lb # [ 79.803349] niks3-server[950]: 2026/09/23 16:00:49 INFO Signed narinfos id=24 count=15126worker2 # [ 79.847292] nix-grpc-daemon[871]: time=2026-09-23T16:00:49.851Z level=INFO msg="Uploading 1 narinfos"5127lb # [ 79.808744] niks3-server[950]: 2026/09/23 16:00:49 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls5128lb # [ 79.816139] niks3-server[950]: 2026/09/23 16:00:49 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv5129lb # [ 79.826283] niks3-server[950]: 2026/09/23 16:00:49 INFO Received complete push request method=POST path=/api/pushes/24/complete5130worker2 # [ 79.873460] nix-grpc-daemon[871]: time=2026-09-23T16:00:49.878Z level=INFO msg="Upload complete. (199ms)"5131lb # [ 79.833503] niks3-server[950]: 2026/09/23 16:00:49 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo5132worker2 # [ 79.876408] nix-grpc-daemon[801]: ts=2026-09-23T16:00:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42452 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=13 outputs=15133worker2 # [ 79.883913] nix-daemon[858]: reaped child process 1147, status = succeeded5134worker1 # [ 79.859285] nix-grpc-daemon[512]: ts=2026-09-23T16:00:49Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:455565135worker1 # [ 79.867487] nix-grpc-daemon[512]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...5136worker2 # [ 79.914377] nix-grpc-daemon[801]: ts=2026-09-23T16:00:49Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:424525137worker2 # [ 79.916501] nix-grpc-daemon[801]: ts=2026-09-23T16:00:49Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:42452 duration_s=0 paths=1 nar_bytes_out=1765138worker2 # [ 80.081337] nix-grpc-daemon[801]: ts=2026-09-23T16:00:50Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:424525139worker1 # [ 80.060088] nix-grpc-daemon[512]: ts=2026-09-23T16:00:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455565140worker1 # [ 80.070665] nix-grpc-daemon[917]: time=2026-09-23T16:00:49.280Z level=INFO msg="All 2 paths already cached"5141worker1 # [ 80.085214] nix-grpc-daemon[512]: ts=2026-09-23T16:00:49Z level=debug event=want client=13 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=05142worker2 # [ 80.116520] nix-grpc-daemon[801]: ts=2026-09-23T16:00:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:424525143worker1 # [ 80.089802] nix-grpc-daemon[512]: ts=2026-09-23T16:00:49Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.4:50051 clients=15144worker2 # [ 80.125142] nix-grpc-daemon[801]: ts=2026-09-23T16:00:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:424525145worker2 # [ 80.134464] nix-grpc-daemon[871]: time=2026-09-23T16:00:50.139Z level=INFO msg="All 2 paths already cached"5146worker2 # [ 80.139289] nix-grpc-daemon[801]: ts=2026-09-23T16:00:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:424525147worker2 # [ 80.142919] nix-daemon[858]: accepted connection from pid 801, user nix-grpc-daemon (trusted)5148worker2 # [ 80.235317] nix-daemon[858]: accepted connection from pid 1172, user nix-grpc-daemon (trusted)5149worker2 # [ 80.248329] nix-daemon[858]: reaped child process 1175, status = succeeded5150lb # [ 80.216719] niks3-server[950]: 2026/09/23 16:00:50 INFO Received push request method=POST path=/api/pushes5151worker2 # [ 80.266210] nix-grpc-daemon[871]: time=2026-09-23T16:00:50.270Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5152worker2 # [ 80.267934] nix-grpc-daemon[871]: time=2026-09-23T16:00:50.272Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"5153lb # [ 80.253324] niks3-server[950]: 2026/09/23 16:00:50 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls5154lb # [ 80.261302] niks3-server[950]: 2026/09/23 16:00:50 INFO Received sign narinfos request method=POST path=/api/pushes/25/sign5155lb # [ 80.266987] niks3-server[950]: 2026/09/23 16:00:50 INFO Signed narinfos id=25 count=15156worker2 # [ 80.310965] nix-grpc-daemon[871]: time=2026-09-23T16:00:50.315Z level=INFO msg="Uploading 1 narinfos"5157lb # [ 80.274351] niks3-server[950]: 2026/09/23 16:00:50 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst5158lb # [ 80.285238] niks3-server[950]: 2026/09/23 16:00:50 INFO Received complete push request method=POST path=/api/pushes/25/complete5159worker2 # [ 80.335079] nix-grpc-daemon[871]: time=2026-09-23T16:00:50.339Z level=INFO msg="Upload complete. (185ms)"5160lb # [ 80.296037] niks3-server[950]: 2026/09/23 16:00:50 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo5161worker2 # [ 80.337279] nix-grpc-daemon[801]: ts=2026-09-23T16:00:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42452 duration_s=0 paths=1 nar_bytes_in=7525162worker2 # [ 80.341659] nix-grpc-daemon[801]: ts=2026-09-23T16:00:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:424525163worker2 # [ 80.349098] nix-daemon[858]: accepted connection from pid 801, user nix-grpc-daemon (trusted)5164worker2 # [ 80.350394] nix-daemon[858]: reaped child process 1170, status = succeeded5165worker2 # [ 80.355680] nix-daemon[858]: accepted connection from pid 801, user nix-grpc-daemon (trusted)5166worker2 # [ 81.599721] nix-daemon[858]: reaped child process 1179, status = succeeded5167worker2 # [ 81.682106] nix-daemon[858]: accepted connection from pid 1195, user nix-grpc-daemon (trusted)5168worker2 # [ 81.694751] nix-daemon[858]: reaped child process 1198, status = succeeded5169lb # [ 81.662613] niks3-server[950]: 2026/09/23 16:00:51 INFO Received push request method=POST path=/api/pushes5170worker2 # [ 81.713887] nix-grpc-daemon[871]: time=2026-09-23T16:00:51.718Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5171worker2 # [ 81.715436] nix-grpc-daemon[871]: time=2026-09-23T16:00:51.720Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"5172lb # [ 81.707217] niks3-server[950]: 2026/09/23 16:00:51 INFO Received sign narinfos request method=POST path=/api/pushes/26/sign5173worker2 # [ 81.754466] nix-grpc-daemon[871]: time=2026-09-23T16:00:51.759Z level=INFO msg="Uploading 1 narinfos"5174lb # [ 81.713681] niks3-server[950]: 2026/09/23 16:00:51 INFO Signed narinfos id=26 count=15175lb # [ 81.722290] niks3-server[950]: 2026/09/23 16:00:51 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls5176lb # [ 81.728196] niks3-server[950]: 2026/09/23 16:00:51 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst5177lb # [ 81.738436] niks3-server[950]: 2026/09/23 16:00:51 INFO Received complete push request method=POST path=/api/pushes/26/complete5178worker2 # [ 81.787284] nix-grpc-daemon[871]: time=2026-09-23T16:00:51.791Z level=INFO msg="Upload complete. (223ms)"5179lb # [ 81.748185] niks3-server[950]: 2026/09/23 16:00:51 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo5180lb # [ 81.751044] niks3-server[950]: 2026/09/23 16:00:51 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv5181worker2 # [ 81.795159] nix-grpc-daemon[801]: ts=2026-09-23T16:00:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42452 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=14 outputs=15182worker1 # [ 81.776594] nix-grpc-daemon[512]: ts=2026-09-23T16:00:50Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:455565183worker2 # [ 81.809930] nix-daemon[858]: reaped child process 1177, status = succeeded5184worker1 # [ 81.784282] nix-grpc-daemon[512]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...5185worker2 # [ 81.833605] nix-grpc-daemon[801]: ts=2026-09-23T16:00:51Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:424525186worker2 # [ 81.838809] nix-grpc-daemon[801]: ts=2026-09-23T16:00:51Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:42452 duration_s=0 paths=1 nar_bytes_out=2325187client # [ 81.872201] nix-daemon[1111]: reaped child process 1113, status = succeeded5188client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log, in 5.81 seconds)5189client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5190client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.02 seconds)5191client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5192client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.02 seconds)5193client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5194client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.02 seconds)5195(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 5.86 seconds)5196subtest: an input only one worker has still reaches the builder5197client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only5198client # this derivation will be built:5199client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5200client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5201client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only, in 0.24 seconds)5202client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5203client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.10 seconds)5204worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5205worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.13 seconds)5206worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5207worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)5208client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&25209client # [ 82.547911] nix-daemon[1111]: accepted connection from pid 1193, user root (trusted)5210client # this derivation will be built:5211client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5212worker2 # [ 82.702182] nix-grpc-daemon[801]: ts=2026-09-23T16:00:52Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:424525213client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5214worker1 # [ 82.685742] nix-grpc-daemon[512]: ts=2026-09-23T16:00:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455565215worker1 # [ 82.793693] nix-daemon[865]: accepted connection from pid 1139, user nix-grpc-daemon (trusted)5216worker1 # [ 82.806457] nix-daemon[865]: reaped child process 1142, status = succeeded5217lb # [ 82.799523] niks3-server[950]: 2026/09/23 16:00:52 INFO Received push request method=POST path=/api/pushes5218worker1 # [ 82.823858] nix-grpc-daemon[917]: time=2026-09-23T16:00:52.033Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5219worker1 # [ 82.825822] nix-grpc-daemon[917]: time=2026-09-23T16:00:52.035Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5220lb # [ 82.832817] niks3-server[950]: 2026/09/23 16:00:52 INFO Received sign narinfos request method=POST path=/api/pushes/27/sign5221lb # [ 82.842334] niks3-server[950]: 2026/09/23 16:00:52 INFO Signed narinfos id=27 count=15222worker1 # [ 82.861607] nix-grpc-daemon[917]: time=2026-09-23T16:00:52.071Z level=INFO msg="Uploading 1 narinfos"5223lb # [ 82.846069] niks3-server[950]: 2026/09/23 16:00:52 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5224lb # [ 82.851271] niks3-server[950]: 2026/09/23 16:00:52 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5225lb # [ 82.867157] niks3-server[950]: 2026/09/23 16:00:52 INFO Received complete push request method=POST path=/api/pushes/27/complete5226worker1 # [ 82.892786] nix-grpc-daemon[917]: time=2026-09-23T16:00:52.103Z level=INFO msg="Upload complete. (202ms)"5227lb # [ 82.877862] niks3-server[950]: 2026/09/23 16:00:52 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5228worker1 # [ 82.911601] nix-grpc-daemon[512]: ts=2026-09-23T16:00:52Z level=debug event=want client=14 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05229worker1 # [ 82.917371] nix-grpc-daemon[512]: ts=2026-09-23T16:00:52Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=15230worker1 # [ 82.923752] nix-grpc-daemon[512]: ts=2026-09-23T16:00:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455565231worker1 # [ 82.933930] nix-grpc-daemon[512]: ts=2026-09-23T16:00:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455565232client # copying 2 paths to 'lb:50051'...5233worker1 # [ 82.947908] nix-grpc-daemon[512]: ts=2026-09-23T16:00:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:455565234client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5235client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5236worker1 # [ 82.957876] nix-daemon[865]: accepted connection from pid 512, user nix-grpc-daemon (trusted)5237worker1 # [ 83.054197] nix-daemon[865]: accepted connection from pid 1148, user nix-grpc-daemon (trusted)5238worker1 # [ 83.067627] nix-daemon[865]: reaped child process 1151, status = succeeded5239lb # [ 83.060068] niks3-server[950]: 2026/09/23 16:00:53 INFO Received push request method=POST path=/api/pushes5240worker1 # [ 83.084807] nix-grpc-daemon[917]: time=2026-09-23T16:00:52.295Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5241worker1 # [ 83.086860] nix-grpc-daemon[917]: time=2026-09-23T16:00:52.296Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5242worker1 # [ 83.088556] nix-grpc-daemon[917]: time=2026-09-23T16:00:52.296Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5243lb # [ 83.108706] niks3-server[950]: 2026/09/23 16:00:53 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5244lb # [ 83.118411] niks3-server[950]: 2026/09/23 16:00:53 INFO Received sign narinfos request method=POST path=/api/pushes/28/sign5245lb # [ 83.122449] niks3-server[950]: 2026/09/23 16:00:53 INFO Signed narinfos id=28 count=25246worker1 # [ 83.140981] nix-grpc-daemon[917]: time=2026-09-23T16:00:52.351Z level=INFO msg="Uploading 2 narinfos"5247lb # [ 83.130120] niks3-server[950]: 2026/09/23 16:00:53 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5248lb # [ 83.134787] niks3-server[950]: 2026/09/23 16:00:53 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5249lb # [ 83.143959] niks3-server[950]: 2026/09/23 16:00:53 INFO Received complete push request method=POST path=/api/pushes/28/complete5250lb # [ 83.152769] niks3-server[950]: 2026/09/23 16:00:53 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5251worker1 # [ 83.167297] nix-grpc-daemon[917]: time=2026-09-23T16:00:52.377Z level=INFO msg="Upload complete. (197ms)"5252lb # [ 83.155200] niks3-server[950]: 2026/09/23 16:00:53 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5253lb # [ 83.157771] niks3-server[950]: 2026/09/23 16:00:53 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5254worker1 # [ 83.172869] nix-grpc-daemon[512]: ts=2026-09-23T16:00:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45556 duration_s=0 paths=2 nar_bytes_in=9765255worker1 # [ 83.179850] nix-grpc-daemon[512]: ts=2026-09-23T16:00:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455565256worker1 # [ 83.190474] nix-daemon[865]: accepted connection from pid 512, user nix-grpc-daemon (trusted)5257worker1 # [ 83.199945] nix-daemon[865]: reaped child process 1146, status = succeeded5258worker1 # [ 83.205481] nix-daemon[865]: accepted connection from pid 512, user nix-grpc-daemon (trusted)5259client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5260client # uses-input-w1onlya> node-a: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5261client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5262worker1 # [ 84.421347] nix-daemon[865]: reaped child process 1155, status = succeeded5263worker1 # [ 84.497413] nix-daemon[865]: accepted connection from pid 1171, user nix-grpc-daemon (trusted)5264worker1 # [ 84.511155] nix-daemon[865]: reaped child process 1174, status = succeeded5265lb # [ 84.503227] niks3-server[950]: 2026/09/23 16:00:54 INFO Received push request method=POST path=/api/pushes5266worker1 # [ 84.527302] nix-grpc-daemon[917]: time=2026-09-23T16:00:53.737Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5267lb # [ 84.533580] niks3-server[950]: 2026/09/23 16:00:54 INFO Received sign narinfos request method=POST path=/api/pushes/29/sign5268worker1 # [ 84.559117] nix-grpc-daemon[917]: time=2026-09-23T16:00:53.769Z level=INFO msg="Uploading 1 narinfos"5269lb # [ 84.543312] niks3-server[950]: 2026/09/23 16:00:54 INFO Signed narinfos id=29 count=15270lb # [ 84.549420] niks3-server[950]: 2026/09/23 16:00:54 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5271lb # [ 84.554853] niks3-server[950]: 2026/09/23 16:00:54 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5272lb # [ 84.567140] niks3-server[950]: 2026/09/23 16:00:54 INFO Received complete push request method=POST path=/api/pushes/29/complete5273lb # [ 84.574775] niks3-server[950]: 2026/09/23 16:00:54 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5274worker1 # [ 84.594688] nix-grpc-daemon[917]: time=2026-09-23T16:00:53.804Z level=INFO msg="Upload complete. (182ms)"5275client # copying 1 paths...5276worker1 # [ 84.601760] nix-grpc-daemon[512]: ts=2026-09-23T16:00:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45556 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=15 outputs=15277worker1 # [ 84.607094] nix-grpc-daemon[512]: ts=2026-09-23T16:00:53Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:455565278client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5279worker1 # [ 84.616661] nix-daemon[865]: reaped child process 1153, status = succeeded5280worker2 # [ 84.653361] nix-grpc-daemon[801]: ts=2026-09-23T16:00:54Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:424525281worker2 # [ 84.664482] nix-grpc-daemon[801]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5282worker2 # [ 84.677623] nix-grpc-daemon[801]: ts=2026-09-23T16:00:54Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:42452 duration_s=0 paths=1 nar_bytes_out=1205283client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.29 seconds)5284client # [ 84.714365] nix-daemon[1111]: reaped child process 1196, status = succeeded5285client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&25286client # [ 84.807811] nix-daemon[1111]: accepted connection from pid 1215, user root (trusted)5287client # this derivation will be built:5288client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5289worker1 # [ 84.937172] nix-grpc-daemon[512]: ts=2026-09-23T16:00:54Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:455565290client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5291worker2 # [ 84.965861] nix-grpc-daemon[801]: ts=2026-09-23T16:00:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:424525292worker1 # [ 84.965244] nix-grpc-daemon[512]: ts=2026-09-23T16:00:54Z level=debug event=want client=15 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05293worker1 # [ 84.969377] nix-grpc-daemon[512]: ts=2026-09-23T16:00:54Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=15294worker2 # [ 85.002260] nix-grpc-daemon[801]: ts=2026-09-23T16:00:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:424525295worker2 # [ 85.014396] nix-grpc-daemon[801]: ts=2026-09-23T16:00:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:424525296client # copying 1 paths to 'lb:50051'...5297client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5298worker2 # [ 85.033324] nix-grpc-daemon[801]: ts=2026-09-23T16:00:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:424525299worker2 # [ 85.044195] nix-daemon[858]: accepted connection from pid 801, user nix-grpc-daemon (trusted)5300worker2 # [ 85.060762] nix-grpc-daemon[801]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 62 ms (attempt 1/5)5301worker2 # [ 85.130534] nix-grpc-daemon[801]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 116 ms (attempt 2/5)5302worker2 # [ 85.248751] nix-grpc-daemon[801]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 341 ms (attempt 3/5)5303worker2 # [ 85.597735] nix-grpc-daemon[801]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 453 ms (attempt 4/5)5304worker2 # [ 86.053865] nix-grpc-daemon[801]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5305worker2 # [ 86.057910] nix-grpc-daemon[801]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5306worker2 # [ 86.063318] nix-grpc-daemon[801]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5307worker2 # [ 86.185129] nix-daemon[858]: accepted connection from pid 1211, user nix-grpc-daemon (trusted)5308worker2 # [ 86.196983] nix-daemon[858]: reaped child process 1214, status = succeeded5309lb # [ 86.165812] niks3-server[950]: 2026/09/23 16:00:56 INFO Received push request method=POST path=/api/pushes5310worker2 # [ 86.215163] nix-grpc-daemon[871]: time=2026-09-23T16:00:56.219Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5311worker2 # [ 86.217146] nix-grpc-daemon[871]: time=2026-09-23T16:00:56.221Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5312lb # [ 86.196334] niks3-server[950]: 2026/09/23 16:00:56 INFO Received sign narinfos request method=POST path=/api/pushes/30/sign5313worker2 # [ 86.244187] nix-grpc-daemon[871]: time=2026-09-23T16:00:56.248Z level=INFO msg="Uploading 1 narinfos"5314lb # [ 86.203938] niks3-server[950]: 2026/09/23 16:00:56 INFO Signed narinfos id=30 count=15315lb # [ 86.211219] niks3-server[950]: 2026/09/23 16:00:56 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5316lb # [ 86.218546] niks3-server[950]: 2026/09/23 16:00:56 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5317lb # [ 86.229327] niks3-server[950]: 2026/09/23 16:00:56 INFO Received complete push request method=POST path=/api/pushes/30/complete5318worker2 # [ 86.280644] nix-grpc-daemon[871]: time=2026-09-23T16:00:56.285Z level=INFO msg="Upload complete. (198ms)"5319lb # [ 86.243268] niks3-server[950]: 2026/09/23 16:00:56 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5320worker2 # [ 86.285344] nix-grpc-daemon[801]: ts=2026-09-23T16:00:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42452 duration_s=1 paths=1 nar_bytes_in=5605321worker2 # [ 86.292214] nix-grpc-daemon[801]: ts=2026-09-23T16:00:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:424525322worker2 # [ 86.305180] nix-daemon[858]: accepted connection from pid 801, user nix-grpc-daemon (trusted)5323worker2 # [ 86.313761] nix-daemon[858]: reaped child process 1204, status = succeeded5324worker2 # [ 86.317414] nix-grpc-daemon[801]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...5325worker2 # [ 86.327451] nix-daemon[858]: accepted connection from pid 801, user nix-grpc-daemon (trusted)5326client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5327client # uses-input-w1onlyb> worker2: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5328lb # [ 86.820733] envoy[918]: conn peer=[2001:db8:1::1]:47182 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=245329lb # [ 86.824450] envoy[918]: conn peer=[2001:db8:1::1]:47176 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3412 tx=1924 ms=15925330lb # [ 86.827956] envoy[918]: conn peer=[2001:db8:1::1]:47188 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3833 tx=2149 ms=17855331lb # [ 86.831528] envoy[918]: conn peer=[2001:db8:1::1]:47194 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=378 ms=135332lb # [ 86.835288] envoy[918]: conn peer=[2001:db8:1::1]:47206 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4325 tx=2373 ms=17685333lb # [ 86.839558] envoy[918]: conn peer=[2001:db8:1::1]:47222 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=411 tx=410 ms=195334lb # [ 86.843099] envoy[918]: conn peer=[2001:db8:1::1]:47226 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3867 tx=2039 ms=19875335lb # [ 86.846586] envoy[918]: conn peer=[2001:db8:1::1]:42256 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=421 tx=329 ms=395336client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5337worker2 # [ 87.567743] nix-daemon[858]: reaped child process 1219, status = succeeded5338worker2 # [ 87.658651] nix-daemon[858]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)5339worker2 # [ 87.671268] nix-daemon[858]: reaped child process 1238, status = succeeded5340lb # [ 87.638899] niks3-server[950]: 2026/09/23 16:00:57 INFO Received push request method=POST path=/api/pushes5341worker2 # [ 87.688160] nix-grpc-daemon[871]: time=2026-09-23T16:00:57.692Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5342lb # [ 87.668218] niks3-server[950]: 2026/09/23 16:00:57 INFO Received sign narinfos request method=POST path=/api/pushes/31/sign5343lb # [ 87.670293] niks3-server[950]: 2026/09/23 16:00:57 INFO Signed narinfos id=31 count=15344worker2 # [ 87.710992] nix-grpc-daemon[871]: time=2026-09-23T16:00:57.715Z level=INFO msg="Uploading 1 narinfos"5345lb # [ 87.674123] niks3-server[950]: 2026/09/23 16:00:57 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5346lb # [ 87.680412] niks3-server[950]: 2026/09/23 16:00:57 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5347lb # [ 87.682515] niks3-server[950]: 2026/09/23 16:00:57 INFO Received complete push request method=POST path=/api/pushes/31/complete5348worker2 # [ 87.728220] nix-grpc-daemon[871]: time=2026-09-23T16:00:57.732Z level=INFO msg="Upload complete. (191ms)"5349lb # [ 87.689593] niks3-server[950]: 2026/09/23 16:00:57 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5350worker2 # [ 87.730988] nix-grpc-daemon[801]: ts=2026-09-23T16:00:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42452 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=16 outputs=15351client # copying 1 paths...5352worker2 # [ 87.738467] nix-daemon[858]: reaped child process 1216, status = succeeded5353worker1 # [ 87.715772] nix-grpc-daemon[512]: ts=2026-09-23T16:00:56Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:455565354worker1 # [ 87.724517] nix-grpc-daemon[512]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5355client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5356worker2 # [ 87.775316] nix-grpc-daemon[801]: ts=2026-09-23T16:00:57Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:424525357worker2 # [ 87.780248] nix-grpc-daemon[801]: ts=2026-09-23T16:00:57Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:42452 duration_s=0 paths=1 nar_bytes_out=1205358client # [ 87.813464] nix-daemon[1111]: reaped child process 1218, status = succeeded5359client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.10 seconds)5360client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&25361client # [ 87.910753] nix-daemon[1111]: accepted connection from pid 1237, user root (trusted)5362client # this derivation will be built:5363client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5364worker2 # [ 88.060769] nix-grpc-daemon[801]: ts=2026-09-23T16:00:58Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:424525365client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5366worker2 # [ 88.068507] nix-grpc-daemon[801]: ts=2026-09-23T16:00:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:424525367worker2 # [ 88.078368] nix-grpc-daemon[871]: time=2026-09-23T16:00:58.083Z level=INFO msg="All 1 paths already cached"5368worker1 # [ 88.068170] nix-grpc-daemon[512]: ts=2026-09-23T16:00:57Z level=debug event=want client=16 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05369worker1 # [ 88.074630] nix-grpc-daemon[512]: ts=2026-09-23T16:00:57Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=15370worker1 # [ 88.080257] nix-grpc-daemon[512]: ts=2026-09-23T16:00:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455565371worker1 # [ 88.091676] nix-grpc-daemon[512]: ts=2026-09-23T16:00:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455565372client # copying 1 paths to 'lb:50051'...5373worker1 # [ 88.106072] nix-grpc-daemon[917]: time=2026-09-23T16:00:57.315Z level=INFO msg="All 1 paths already cached"5374client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5375worker1 # [ 88.111633] nix-grpc-daemon[512]: ts=2026-09-23T16:00:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:455565376worker1 # [ 88.122710] nix-daemon[865]: accepted connection from pid 512, user nix-grpc-daemon (trusted)5377worker1 # [ 88.219197] nix-daemon[865]: accepted connection from pid 1183, user nix-grpc-daemon (trusted)5378worker1 # [ 88.232493] nix-daemon[865]: reaped child process 1186, status = succeeded5379lb # [ 88.223758] niks3-server[950]: 2026/09/23 16:00:58 INFO Received push request method=POST path=/api/pushes5380worker1 # [ 88.249615] nix-grpc-daemon[917]: time=2026-09-23T16:00:57.459Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5381worker1 # [ 88.251031] nix-grpc-daemon[917]: time=2026-09-23T16:00:57.459Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5382lb # [ 88.255057] niks3-server[950]: 2026/09/23 16:00:58 INFO Received sign narinfos request method=POST path=/api/pushes/32/sign5383lb # [ 88.261186] niks3-server[950]: 2026/09/23 16:00:58 INFO Signed narinfos id=32 count=15384worker1 # [ 88.282270] nix-grpc-daemon[917]: time=2026-09-23T16:00:57.492Z level=INFO msg="Uploading 1 narinfos"5385lb # [ 88.268148] niks3-server[950]: 2026/09/23 16:00:58 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5386lb # [ 88.275098] niks3-server[950]: 2026/09/23 16:00:58 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5387lb # [ 88.285365] niks3-server[950]: 2026/09/23 16:00:58 INFO Received complete push request method=POST path=/api/pushes/32/complete5388worker1 # [ 88.304265] nix-grpc-daemon[917]: time=2026-09-23T16:00:57.514Z level=INFO msg="Upload complete. (170ms)"5389lb # [ 88.289656] niks3-server[950]: 2026/09/23 16:00:58 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5390worker1 # [ 88.308263] nix-grpc-daemon[512]: ts=2026-09-23T16:00:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45556 duration_s=0 paths=1 nar_bytes_in=5605391worker1 # [ 88.312993] nix-grpc-daemon[512]: ts=2026-09-23T16:00:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455565392worker1 # [ 88.322533] nix-daemon[865]: accepted connection from pid 512, user nix-grpc-daemon (trusted)5393worker1 # [ 88.328191] nix-daemon[865]: reaped child process 1181, status = succeeded5394worker1 # [ 88.333250] nix-daemon[865]: accepted connection from pid 512, user nix-grpc-daemon (trusted)5395client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5396client # uses-input-w1onlyc> node-a: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5397client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5398worker1 # [ 89.171860] nix-daemon[865]: reaped child process 1190, status = succeeded5399worker1 # [ 89.251536] nix-daemon[865]: accepted connection from pid 1206, user nix-grpc-daemon (trusted)5400worker1 # [ 89.264407] nix-daemon[865]: reaped child process 1209, status = succeeded5401lb # [ 89.256782] niks3-server[950]: 2026/09/23 16:00:59 INFO Received push request method=POST path=/api/pushes5402worker1 # [ 89.282323] nix-grpc-daemon[917]: time=2026-09-23T16:00:58.492Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5403lb # [ 89.292578] niks3-server[950]: 2026/09/23 16:00:59 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5404lb # [ 89.296136] niks3-server[950]: 2026/09/23 16:00:59 INFO Received sign narinfos request method=POST path=/api/pushes/33/sign5405lb # [ 89.301077] niks3-server[950]: 2026/09/23 16:00:59 INFO Signed narinfos id=33 count=15406worker1 # [ 89.320058] nix-grpc-daemon[917]: time=2026-09-23T16:00:58.530Z level=INFO msg="Uploading 1 narinfos"5407lb # [ 89.313645] niks3-server[950]: 2026/09/23 16:00:59 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5408lb # [ 89.325448] niks3-server[950]: 2026/09/23 16:00:59 INFO Received complete push request method=POST path=/api/pushes/33/complete5409lb # [ 89.333733] niks3-server[950]: 2026/09/23 16:00:59 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5410worker1 # [ 89.353357] nix-grpc-daemon[917]: time=2026-09-23T16:00:58.563Z level=INFO msg="Upload complete. (212ms)"5411worker1 # [ 89.359707] nix-grpc-daemon[512]: ts=2026-09-23T16:00:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45556 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=17 outputs=15412client # copying 1 paths...5413worker2 # [ 89.395934] nix-grpc-daemon[801]: ts=2026-09-23T16:00:59Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:424525414worker1 # [ 89.376578] nix-daemon[865]: reaped child process 1188, status = succeeded5415worker2 # [ 89.410140] nix-grpc-daemon[801]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5416client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5417worker1 # [ 89.414602] nix-grpc-daemon[512]: ts=2026-09-23T16:00:58Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:455565418worker1 # [ 89.420682] nix-grpc-daemon[512]: ts=2026-09-23T16:00:58Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45556 duration_s=0 paths=1 nar_bytes_out=1205419client # [ 89.475650] nix-daemon[1111]: reaped child process 1240, status = succeeded5420client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&2, in 1.66 seconds)5421(finished: subtest: an input only one worker has still reaches the builder, in 7.54 seconds)5422subtest: upload whose reference the worker lacks is completed from the cache5423client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache5424client # this derivation will be built:5425client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5426client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5427client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache, in 0.24 seconds)5428client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache5429client # this derivation will be built:5430client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5431client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5432client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache, in 0.23 seconds)5433client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5434client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.09 seconds)5435worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5436worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.13 seconds)5437worker2 # [ 90.260192] systemd[1]: Stopping Nix worker-protocol over gRPC...5438worker2 # [ 90.808666] nix-grpc-daemon[801]: ts=2026-09-23T16:01:00Z level=info event=signal_exit5439worker1 # [ 92.776780] nix-grpc-daemon[512]: ts=2026-09-23T16:01:01Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:560245440worker1 # [ 92.783618] nix-grpc-daemon[512]: ts=2026-09-23T16:01:01Z level=info event=worker_gone addr=192.168.1.4:500515441worker2 # [ 92.799817] nix-grpc-daemon[801]: ts=2026-09-23T16:01:02Z level=info event=scheduler_restarting5442worker2 # [ 92.819569] nix-grpc-daemon[801]: ts=2026-09-23T16:01:02Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5443worker2 # [ 92.841201] nix-daemon[858]: reaped child process 860, status = succeeded5444worker2 # [ 93.476874] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5445worker2 # [ 93.481194] systemd[1]: Stopped Nix worker-protocol over gRPC.5446worker2 # [ 93.485847] systemd[1]: nix-grpc-daemon.service: Consumed 2.330s CPU time over 1min 1.843s wall clock time, 65.7M memory peak, 178.6K incoming IP traffic, 239.7K outgoing IP traffic.5447lb: waiting until exactly ['worker1'] are healthy members of x86_64-linux5448lb: must succeed: curl -sf localhost:9901/clusters5449lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5450lb: (finished: waiting until exactly ['worker1'] are healthy members of x86_64-linux, in 0.06 seconds)5451client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25452worker1 # [ 93.678265] nix-grpc-daemon[512]: ts=2026-09-23T16:01:02Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:455565453client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5454worker1 # [ 93.683615] nix-grpc-daemon[512]: ts=2026-09-23T16:01:02Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:455565455client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.16 seconds)5456worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5457worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds)5458worker2 # [ 93.789761] systemd[1]: Starting scheduler-token.service...5459worker2 # [ 93.829705] systemd[1]: scheduler-token.service: Deactivated successfully.5460worker2 # [ 93.831401] systemd[1]: Finished scheduler-token.service.5461worker2 # [ 93.832741] systemd[1]: scheduler-token.service: Consumed 17ms CPU time over 41ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5462worker2 # [ 93.837097] systemd[1]: Starting Nix worker-protocol over gRPC...5463worker2 # [ 93.923292] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:03Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5464worker2 # [ 93.927795] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:03Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5465worker2 # [ 93.931427] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:03Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15466worker2 # [ 93.933414] nix-grpc-daemon[1263]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5467worker2 # [ 93.934698] nix-grpc-daemon[1263]: E0000 00:00:1790179263.935338 1263 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5468worker2 # [ 93.938172] nix-grpc-daemon[1263]: E0000 00:00:1790179263.942779 1263 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5469worker2 # [ 93.946061] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:03Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5470worker2 # [ 93.948969] systemd[1]: Started Nix worker-protocol over gRPC.5471worker2 # [ 93.951114] nix-grpc-daemon[1263]: E0000 00:00:1790179263.954937 1306 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5472worker2 # [ 93.966094] nix-grpc-daemon[1263]: E0000 00:00:1790179263.969241 1306 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5473worker2 # [ 93.970143] nix-grpc-daemon[1263]: E0000 00:00:1790179263.969556 1306 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5474worker2 # [ 93.979727] nix-grpc-daemon[1263]: E0000 00:00:1790179263.984401 1306 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5475worker2 # [ 93.990268] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:03Z level=info event=scheduler_connected addr=lb:500515476worker1 # [ 93.970517] nix-grpc-daemon[512]: ts=2026-09-23T16:01:03Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:560245477worker1 # [ 93.978657] nix-grpc-daemon[512]: ts=2026-09-23T16:01:03Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05478worker1 # [ 94.000842] systemd[1]: Stopping Nix worker-protocol over gRPC...5479worker1 # [ 94.834710] nix-grpc-daemon[512]: ts=2026-09-23T16:01:04Z level=info event=signal_exit5480lb # [ 95.787898] niks3-server[950]: 2026/09/23 16:01:05 INFO lead: released remote=192.168.1.3:548325481worker1 # [ 96.804930] nix-grpc-daemon[512]: ts=2026-09-23T16:01:06Z level=info event=scheduler_restarting5482lb # [ 96.787838] envoy[918]: [2026-09-23 16:01:06.867][918][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5483worker2 # [ 96.837965] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:06Z level=info event=scheduler_restarting addr=lb:500515484worker1 # [ 96.813673] nix-grpc-daemon[512]: ts=2026-09-23T16:01:06Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:560245485worker1 # [ 96.818248] nix-grpc-daemon[512]: ts=2026-09-23T16:01:06Z level=info event=worker_gone addr=192.168.1.4:500515486worker1 # [ 96.822487] nix-grpc-daemon[512]: ts=2026-09-23T16:01:06Z level=info event=scheduler_restarting addr=lb:500515487worker1 # [ 96.826340] nix-grpc-daemon[512]: ts=2026-09-23T16:01:06Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:560245488worker1 # [ 96.829824] nix-grpc-daemon[512]: ts=2026-09-23T16:01:06Z level=info event=worker_gone addr=192.168.1.3:500515489lb # [ 96.823148] envoy[918]: conn peer=[2001:db8:1::1]:42264 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3745 tx=2124 ms=28295490lb # [ 96.826811] envoy[918]: conn peer=[2001:db8:1::1]:42278 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=421 tx=328 ms=155491lb # [ 96.830340] envoy[918]: conn peer=[2001:db8:1::1]:42292 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=175492lb # [ 96.833832] envoy[918]: conn peer=[2001:db8:1::1]:42286 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3730 tx=2084 ms=13965493worker1 # [ 96.853089] nix-daemon[865]: reaped child process 867, status = succeeded5494lb # [ 96.838693] envoy[918]: conn peer=[2001:db8:1::4]:38626 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=3121 tx=1072 ms=609725495lb # [ 96.843240] envoy[918]: conn peer=[2001:db8:1::1]:53508 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=165496worker2 # [ 97.038407] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:07Z level=info event=scheduler_connected addr=lb:500515497worker2 # [ 97.042994] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5498worker1 # [ 97.028294] nix-grpc-daemon[512]: ts=2026-09-23T16:01:06Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5499worker2 # [ 97.244285] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:07Z level=info event=scheduler_connected addr=lb:500515500worker2 # [ 97.248358] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5501worker1 # [ 97.412329] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5502worker1 # [ 97.416114] systemd[1]: Stopped Nix worker-protocol over gRPC.5503worker1 # [ 97.420575] systemd[1]: nix-grpc-daemon.service: Consumed 2.068s CPU time over 1min 28.866s wall clock time, 64.8M memory peak, 531.1K incoming IP traffic, 261K outgoing IP traffic.5504worker2 # [ 97.449620] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:07Z level=info event=scheduler_connected addr=lb:500515505worker2 # [ 97.453781] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5506lb: waiting until exactly ['worker2'] are healthy members of x86_64-linux5507lb: must succeed: curl -sf localhost:9901/clusters5508lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5509lb: (finished: waiting until exactly ['worker2'] are healthy members of x86_64-linux, in 0.05 seconds)5510client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25511worker2 # [ 97.655178] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:07Z level=info event=scheduler_connected addr=lb:500515512worker2 # [ 97.662359] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5513worker2 # [ 97.666984] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:392085514worker2 # [ 97.677819] nix-daemon[858]: accepted connection from pid 1263, user nix-grpc-daemon (trusted)5515worker2 # [ 97.695834] nix-grpc-daemon[1263]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 50 ms (attempt 1/5)5516worker2 # [ 97.754130] nix-grpc-daemon[1263]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 21 ms (attempt 2/5)5517worker2 # [ 97.776675] nix-grpc-daemon[1263]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 362 ms (attempt 3/5)5518worker2 # [ 97.861977] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:07Z level=info event=scheduler_connected addr=lb:500515519worker2 # [ 97.865982] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5520worker2 # [ 98.066977] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:08Z level=info event=scheduler_connected addr=lb:500515521worker2 # [ 98.071211] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5522worker2 # [ 98.141109] nix-grpc-daemon[1263]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 634 ms (attempt 4/5)5523worker2 # [ 98.272448] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:08Z level=info event=scheduler_connected addr=lb:500515524worker2 # [ 98.276695] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5525worker2 # [ 98.478091] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:08Z level=info event=scheduler_connected addr=lb:500515526worker2 # [ 98.482245] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5527worker2 # [ 98.683562] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:08Z level=info event=scheduler_connected addr=lb:500515528worker2 # [ 98.687905] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5529worker2 # [ 98.777822] nix-grpc-daemon[1263]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5530worker2 # [ 98.781193] nix-grpc-daemon[1263]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5531client # copying 2 paths...5532client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5533worker2 # [ 98.806782] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:392085534client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5535worker2 # [ 98.820393] nix-daemon[858]: accepted connection from pid 1263, user nix-grpc-daemon (trusted)5536worker2 # [ 98.843171] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:08Z level=info event=niks3_push_started pid=13235537worker2 # [ 98.890599] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:08Z level=info event=scheduler_connected addr=lb:500515538worker2 # [ 98.892523] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5539lb # [ 98.908424] niks3-server[950]: 2026/09/23 16:01:08 INFO lead: acquired remote=192.168.1.4:554345540worker2 # [ 98.952082] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:08Z level=info event=scheduler_take_over5541worker2 # [ 98.966956] nix-grpc-daemon[1323]: time=2026-09-23T16:01:08.971Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""5542worker2 # [ 99.071982] nix-daemon[858]: accepted connection from pid 1336, user nix-grpc-daemon (trusted)5543worker2 # [ 99.085252] nix-daemon[858]: reaped child process 1339, status = succeeded5544worker2 # [ 99.092932] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:09Z level=info event=scheduler_connected addr=lb:500515545lb # [ 99.052075] niks3-server[950]: 2026/09/23 16:01:09 INFO Received push request method=POST path=/api/pushes5546worker2 # [ 99.108106] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:09Z level=info event=oidc_jwks provider=mock keys=15547worker2 # [ 99.109955] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:09Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:392205548worker2 # [ 99.113061] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:09Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05549worker2 # [ 99.115470] nix-grpc-daemon[1323]: time=2026-09-23T16:01:09.116Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5550worker2 # [ 99.116898] nix-grpc-daemon[1323]: time=2026-09-23T16:01:09.117Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5551worker2 # [ 99.118542] nix-grpc-daemon[1323]: time=2026-09-23T16:01:09.117Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5552lb # [ 99.125910] niks3-server[950]: 2026/09/23 16:01:09 INFO Received sign narinfos request method=POST path=/api/pushes/34/sign5553lb # [ 99.128285] niks3-server[950]: 2026/09/23 16:01:09 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5554lb # [ 99.132089] niks3-server[950]: 2026/09/23 16:01:09 INFO Signed narinfos id=34 count=25555worker2 # [ 99.173398] nix-grpc-daemon[1323]: time=2026-09-23T16:01:09.177Z level=INFO msg="Uploading 2 narinfos"5556lb # [ 99.134559] niks3-server[950]: 2026/09/23 16:01:09 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5557lb # [ 99.149958] niks3-server[950]: 2026/09/23 16:01:09 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5558lb # [ 99.154640] niks3-server[950]: 2026/09/23 16:01:09 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5559lb # [ 99.157943] niks3-server[950]: 2026/09/23 16:01:09 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5560lb # [ 99.160702] niks3-server[950]: 2026/09/23 16:01:09 INFO Received complete push request method=POST path=/api/pushes/34/complete5561lb # [ 99.165921] niks3-server[950]: 2026/09/23 16:01:09 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5562worker2 # [ 99.211354] nix-grpc-daemon[1323]: time=2026-09-23T16:01:09.215Z level=INFO msg="Upload complete. (240ms)"5563worker2 # [ 99.216677] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39208 duration_s=0 paths=2 nar_bytes_in=3125564worker2 # [ 99.230324] nix-daemon[858]: reaped child process 1321, status = succeeded5565client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.74 seconds)5566worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5567worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds)5568worker1 # [ 99.292371] systemd[1]: Starting Nix worker-protocol over gRPC...5569worker1 # [ 99.386354] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:08Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5570worker1 # [ 99.388547] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:08Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5571worker1 # [ 99.392365] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:08Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15572worker1 # [ 99.396050] nix-grpc-daemon[1232]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5573worker1 # [ 99.397368] nix-grpc-daemon[1232]: E0000 00:00:1790179268.606350 1232 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5574worker1 # [ 99.401465] nix-grpc-daemon[1232]: E0000 00:00:1790179268.611700 1232 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5575worker1 # [ 99.406050] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:08Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5576worker1 # [ 99.411346] systemd[1]: Started Nix worker-protocol over gRPC.5577worker1 # [ 99.414095] nix-grpc-daemon[1232]: E0000 00:00:1790179268.623570 1275 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5578worker1 # [ 99.427271] nix-grpc-daemon[1232]: E0000 00:00:1790179268.637535 1275 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5579worker1 # [ 99.430155] nix-grpc-daemon[1232]: E0000 00:00:1790179268.639759 1275 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5580lb: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux5581lb: must succeed: curl -sf localhost:9901/clusters5582worker1 # [ 99.441783] nix-grpc-daemon[1232]: E0000 00:00:1790179268.652069 1275 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5583worker1 # [ 99.456544] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:08Z level=info event=scheduler_connected addr=lb:500515584worker2 # [ 99.483334] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:09Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:392205585worker2 # [ 99.487185] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:09Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05586lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5587lb: must succeed: curl -sf localhost:9901/clusters5588lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5589lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux, in 1.10 seconds)5590(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.04 seconds)5591subtest: a niks3 restart does not cost the leader its role5592lb: must succeed: curl -sf localhost:9901/clusters5593lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5594lb: must succeed: curl -sf localhost:9901/clusters5595lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5596lb # [ 100.592930] niks3-server[950]: 2026/09/23 16:01:10 INFO Shutdown signal received, draining in-flight requests timeout=10s5597lb # [ 100.597355] systemd[1]: Stopping niks3 server...5598lb # [ 100.600853] niks3-server[950]: 2026/09/23 16:01:10 INFO lead: released remote=192.168.1.4:554345599worker2 # [ 100.642584] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:10Z level=info event=scheduler_lock_lost status=2005600worker1 # [ 100.618274] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:09Z level=info event=scheduler_lock_lost status=2005601lb # [ 100.613149] systemd[1]: niks3.service: Deactivated successfully.5602lb # [ 100.615235] systemd[1]: Stopped niks3 server.5603lb # [ 100.615900] systemd[1]: niks3.service: Consumed 769ms CPU time over 1min 23.056s wall clock time, 28.8M memory peak, 97K incoming IP traffic, 139.7K outgoing IP traffic.5604lb # [ 100.623349] systemd[1]: Starting niks3 server...5605lb # [ 100.675364] niks3-server[1128]: 2026/09/23 16:01:10 goose: no migrations to run. current version: 202609231200005606lb # [ 100.682186] niks3-server[1128]: 2026/09/23 16:01:10 OK 1_commit_pending_closure.sql (4.92ms)5607lb # [ 100.685791] niks3-server[1128]: 2026/09/23 16:01:10 OK 2_object_stats_trigger.sql (2.3ms)5608lb # [ 100.688925] niks3-server[1128]: 2026/09/23 16:01:10 OK 3_commit_push.sql (3.19ms)5609lb # [ 100.690451] niks3-server[1128]: 2026/09/23 16:01:10 goose: up to current file version: 35610lb # [ 100.692158] niks3-server[1128]: 2026/09/23 16:01:10 INFO Loaded signing key name=farm-test-1 path=/nix/store/hk0nhypl37jvkn3gr36v3y7qx593p39i-key5611lb # [ 100.703698] niks3-server[1128]: 2026/09/23 16:01:10 INFO Read proxy enabled — serving cache objects from S35612lb # [ 100.705216] niks3-server[1128]: 2026/09/23 16:01:10 INFO Using socket-activated listener address=0.0.0.0:57515613lb # [ 100.707133] systemd[1]: Started niks3 server.5614lb # [ 100.708246] niks3-server[1128]: 2026/09/23 16:01:10 INFO systemd watchdog enabled interval=15s5615lb # [ 100.711941] niks3-server[1128]: 2026/09/23 16:01:10 INFO Starting HTTP server address=0.0.0.0:57515616lb: waiting for unit niks3.service5617lb: (finished: waiting for unit niks3.service, in 0.02 seconds)5618lb: must succeed: curl -sf localhost:9901/clusters5619lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5620worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5621worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5622lb: must succeed: curl -sf localhost:9901/clusters5623lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5624lb: must succeed: curl -sf localhost:9901/clusters5625lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5626worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5627worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.05 seconds)5628client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&25629worker1 # [ 101.064150] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:555765630worker1 # [ 101.076389] nix-daemon[865]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)5631client # don't know how to build these paths:5632client # /nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5633worker1 # [ 101.091368] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:555765634worker1 # [ 101.101782] nix-grpc-daemon[1232]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 55 ms (attempt 1/5)5635worker1 # [ 101.164171] nix-grpc-daemon[1232]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 46 ms (attempt 2/5)5636worker1 # [ 101.211943] nix-grpc-daemon[1232]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 210 ms (attempt 3/5)5637worker1 # [ 101.423980] nix-grpc-daemon[1232]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 585 ms (attempt 4/5)5638lb # [ 101.608806] niks3-server[1128]: 2026/09/23 16:01:11 INFO lead: acquired remote=192.168.1.4:554905639worker1 # [ 102.011858] nix-grpc-daemon[1232]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5640worker1 # [ 102.015835] nix-grpc-daemon[1232]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5641worker2 # [ 102.080547] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:12Z level=debug event=want client=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv cached=05642worker2 # [ 102.082774] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:12Z level=debug event=assigned drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv worker=192.168.1.4:50051 clients=15643worker2 # [ 102.085373] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:392085644worker2 # [ 102.095187] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:392085645client # copying 1 paths to 'lb:50051'...5646worker2 # [ 102.104562] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:392085647client # copying path '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' to 'lb:50051'...5648worker2 # [ 102.111697] nix-daemon[858]: accepted connection from pid 1263, user nix-grpc-daemon (trusted)5649worker2 # [ 102.205628] nix-daemon[858]: accepted connection from pid 1358, user nix-grpc-daemon (trusted)5650worker2 # [ 102.218458] nix-daemon[858]: reaped child process 1361, status = succeeded5651lb # [ 102.187812] niks3-server[1128]: 2026/09/23 16:01:12 INFO Received push request method=POST path=/api/pushes5652worker2 # [ 102.241621] nix-grpc-daemon[1323]: time=2026-09-23T16:01:12.246Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5653worker2 # [ 102.243841] nix-grpc-daemon[1323]: time=2026-09-23T16:01:12.247Z level=INFO msg="Uploading 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv (536B)"5654lb # [ 102.230708] niks3-server[1128]: 2026/09/23 16:01:12 INFO Received sign narinfos request method=POST path=/api/pushes/35/sign5655lb # [ 102.258954] niks3-server[1128]: 2026/09/23 16:01:12 INFO Signed narinfos id=35 count=15656lb # [ 102.260598] niks3-server[1128]: 2026/09/23 16:01:12 INFO Registered completed upload object_key=nar/0wjrljh500qj2dh74gryz4ik45bn1iv9w72490z0f2w0l2x4c87h.nar.zst5657worker2 # [ 102.301791] nix-grpc-daemon[1323]: time=2026-09-23T16:01:12.305Z level=INFO msg="Uploading 1 narinfos"5658lb # [ 102.263737] niks3-server[1128]: 2026/09/23 16:01:12 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.ls5659lb # [ 102.269532] niks3-server[1128]: 2026/09/23 16:01:12 INFO Received complete push request method=POST path=/api/pushes/35/complete5660lb # [ 102.277083] niks3-server[1128]: 2026/09/23 16:01:12 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.narinfo5661worker2 # [ 102.321254] nix-grpc-daemon[1323]: time=2026-09-23T16:01:12.325Z level=INFO msg="Upload complete. (204ms)"5662worker2 # [ 102.323488] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39208 duration_s=0 paths=1 nar_bytes_in=5365663worker2 # [ 102.326654] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:392085664worker2 # [ 102.334042] nix-daemon[858]: accepted connection from pid 1263, user nix-grpc-daemon (trusted)5665worker2 # [ 102.336689] nix-daemon[858]: reaped child process 1356, status = succeeded5666worker2 # [ 102.341673] nix-daemon[858]: accepted connection from pid 1263, user nix-grpc-daemon (trusted)5667client # building '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' on lb:50051...5668client # farm-a> worker2: building 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5669client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5670client # farm-a> LOG-farm-a-after-niks3-restart5671worker2 # [ 103.156747] nix-daemon[858]: reaped child process 1365, status = succeeded5672worker2 # [ 103.238560] nix-daemon[858]: accepted connection from pid 1381, user nix-grpc-daemon (trusted)5673worker2 # [ 103.252913] nix-daemon[858]: reaped child process 1384, status = succeeded5674lb # [ 103.219322] niks3-server[1128]: 2026/09/23 16:01:13 INFO Received push request method=POST path=/api/pushes5675worker2 # [ 103.269429] nix-grpc-daemon[1323]: time=2026-09-23T16:01:13.273Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5676worker2 # [ 103.270956] nix-grpc-daemon[1323]: time=2026-09-23T16:01:13.275Z level=INFO msg="Uploading vllq1krpcvis69h349hy98vfz55ld1wv-farm-a (144B)"5677lb # [ 103.269163] niks3-server[1128]: 2026/09/23 16:01:13 INFO Received sign narinfos request method=POST path=/api/pushes/36/sign5678lb # [ 103.275437] niks3-server[1128]: 2026/09/23 16:01:13 INFO Signed narinfos id=36 count=15679worker2 # [ 103.318732] nix-grpc-daemon[1323]: time=2026-09-23T16:01:13.323Z level=INFO msg="Uploading 1 narinfos"5680lb # [ 103.278604] niks3-server[1128]: 2026/09/23 16:01:13 INFO Registered completed upload object_key=log/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5681lb # [ 103.288906] niks3-server[1128]: 2026/09/23 16:01:13 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.ls5682lb # [ 103.293330] niks3-server[1128]: 2026/09/23 16:01:13 INFO Received complete push request method=POST path=/api/pushes/36/complete5683lb # [ 103.304376] niks3-server[1128]: 2026/09/23 16:01:13 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5684lb # [ 103.310587] niks3-server[1128]: 2026/09/23 16:01:13 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.narinfo5685worker2 # [ 103.353793] nix-grpc-daemon[1323]: time=2026-09-23T16:01:13.358Z level=INFO msg="Upload complete. (206ms)"5686worker2 # [ 103.356727] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39208 duration_s=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv assign_id=1 outputs=15687worker2 # [ 103.364443] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:13Z level=debug event=want client=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv cached=05688worker2 # [ 103.367188] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:13Z level=debug event=assigned drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv worker=192.168.1.4:50051 clients=15689worker2 # [ 103.368965] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:392085690worker2 # [ 103.375226] nix-daemon[858]: reaped child process 1363, status = succeeded5691worker2 # [ 103.376790] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:392085692worker2 # [ 103.383092] nix-grpc-daemon[1323]: time=2026-09-23T16:01:13.387Z level=INFO msg="All 1 paths already cached"5693client # copying 1 paths to 'lb:50051'...5694worker2 # [ 103.388125] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:392085695client # copying path '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' to 'lb:50051'...5696worker2 # [ 103.393145] nix-daemon[858]: accepted connection from pid 1263, user nix-grpc-daemon (trusted)5697worker2 # [ 103.484971] nix-daemon[858]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)5698worker2 # [ 103.498140] nix-daemon[858]: reaped child process 1393, status = succeeded5699lb # [ 103.466149] niks3-server[1128]: 2026/09/23 16:01:13 INFO Received push request method=POST path=/api/pushes5700worker2 # [ 103.518131] nix-grpc-daemon[1323]: time=2026-09-23T16:01:13.522Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5701worker2 # [ 103.520142] nix-grpc-daemon[1323]: time=2026-09-23T16:01:13.524Z level=INFO msg="Uploading 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv (656B)"5702lb # [ 103.502232] niks3-server[1128]: 2026/09/23 16:01:13 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.ls5703lb # [ 103.508509] niks3-server[1128]: 2026/09/23 16:01:13 INFO Received sign narinfos request method=POST path=/api/pushes/37/sign5704lb # [ 103.513494] niks3-server[1128]: 2026/09/23 16:01:13 INFO Signed narinfos id=37 count=15705worker2 # [ 103.557358] nix-grpc-daemon[1323]: time=2026-09-23T16:01:13.561Z level=INFO msg="Uploading 1 narinfos"5706lb # [ 103.526579] niks3-server[1128]: 2026/09/23 16:01:13 INFO Registered completed upload object_key=nar/0sjnwmbwn7i8g7g7czwn9r2pzaaa1j1yx3dfa4gw29ynhmpz9v3j.nar.zst5707lb # [ 103.532724] niks3-server[1128]: 2026/09/23 16:01:13 INFO Received complete push request method=POST path=/api/pushes/37/complete5708lb # [ 103.542482] niks3-server[1128]: 2026/09/23 16:01:13 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.narinfo5709worker2 # [ 103.586686] nix-grpc-daemon[1323]: time=2026-09-23T16:01:13.591Z level=INFO msg="Upload complete. (187ms)"5710worker2 # [ 103.588890] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39208 duration_s=0 paths=1 nar_bytes_in=6565711worker2 # [ 103.591990] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:392085712worker2 # [ 103.598929] nix-daemon[858]: accepted connection from pid 1263, user nix-grpc-daemon (trusted)5713worker2 # [ 103.603082] nix-daemon[858]: reaped child process 1388, status = succeeded5714worker2 # [ 103.607329] nix-daemon[858]: accepted connection from pid 1263, user nix-grpc-daemon (trusted)5715client # building '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' on lb:50051...5716client # farm-b> worker2: building 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv5717client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5718worker2 # [ 104.365219] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:14Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:482145719client # farm-b> LOG-farm-b-after-niks3-restart5720worker2 # [ 104.622534] nix-daemon[858]: reaped child process 1397, status = succeeded5721worker2 # [ 104.704759] nix-daemon[858]: accepted connection from pid 1413, user nix-grpc-daemon (trusted)5722worker2 # [ 104.718219] nix-daemon[858]: reaped child process 1416, status = succeeded5723lb # [ 104.686976] niks3-server[1128]: 2026/09/23 16:01:14 INFO Received push request method=POST path=/api/pushes5724worker2 # [ 104.739529] nix-grpc-daemon[1323]: time=2026-09-23T16:01:14.744Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5725worker2 # [ 104.741658] nix-grpc-daemon[1323]: time=2026-09-23T16:01:14.745Z level=INFO msg="Uploading vkqg7lcax8h1mxs0g750ficqiii1falw-farm-b (192B)"5726lb # [ 104.733096] niks3-server[1128]: 2026/09/23 16:01:14 INFO Registered completed upload object_key=log/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv5727lb # [ 104.738600] niks3-server[1128]: 2026/09/23 16:01:14 INFO Registered completed upload object_key=nar/12zl6l8xhwx5hdivnvf1vw18yzvz3764mhdd9n03x81phzqblxj0.nar.zst5728lb # [ 104.741876] niks3-server[1128]: 2026/09/23 16:01:14 INFO Received sign narinfos request method=POST path=/api/pushes/38/sign5729worker2 # [ 104.789685] nix-grpc-daemon[1323]: time=2026-09-23T16:01:14.794Z level=INFO msg="Uploading 1 narinfos"5730lb # [ 104.748849] niks3-server[1128]: 2026/09/23 16:01:14 INFO Signed narinfos id=38 count=15731lb # [ 104.754307] niks3-server[1128]: 2026/09/23 16:01:14 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.ls5732lb # [ 104.765181] niks3-server[1128]: 2026/09/23 16:01:14 INFO Received complete push request method=POST path=/api/pushes/38/complete5733worker2 # [ 104.809482] nix-grpc-daemon[1323]: time=2026-09-23T16:01:14.814Z level=INFO msg="Upload complete. (194ms)"5734lb # [ 104.770038] niks3-server[1128]: 2026/09/23 16:01:14 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.narinfo5735worker2 # [ 104.812228] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39208 duration_s=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv assign_id=2 outputs=15736worker2 # [ 104.818317] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:14Z level=debug event=want client=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv cached=05737worker2 # [ 104.820900] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:14Z level=debug event=assigned drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv worker=192.168.1.4:50051 clients=15738worker2 # [ 104.823468] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:392085739worker2 # [ 104.829364] nix-daemon[858]: reaped child process 1395, status = succeeded5740worker2 # [ 104.833165] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:392085741worker2 # [ 104.839592] nix-grpc-daemon[1323]: time=2026-09-23T16:01:14.844Z level=INFO msg="All 2 paths already cached"5742client # copying 1 paths to 'lb:50051'...5743worker2 # [ 104.844834] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:392085744client # copying path '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' to 'lb:50051'...5745worker2 # [ 104.849941] nix-daemon[858]: accepted connection from pid 1263, user nix-grpc-daemon (trusted)5746worker2 # [ 104.942101] nix-daemon[858]: accepted connection from pid 1422, user nix-grpc-daemon (trusted)5747worker2 # [ 104.955719] nix-daemon[858]: reaped child process 1425, status = succeeded5748lb # [ 104.923342] niks3-server[1128]: 2026/09/23 16:01:15 INFO Received push request method=POST path=/api/pushes5749worker2 # [ 104.974062] nix-grpc-daemon[1323]: time=2026-09-23T16:01:14.978Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5750worker2 # [ 104.975884] nix-grpc-daemon[1323]: time=2026-09-23T16:01:14.980Z level=INFO msg="Uploading v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv (784B)"5751lb # [ 104.960696] niks3-server[1128]: 2026/09/23 16:01:15 INFO Received sign narinfos request method=POST path=/api/pushes/39/sign5752lb # [ 104.966923] niks3-server[1128]: 2026/09/23 16:01:15 INFO Signed narinfos id=39 count=15753worker2 # [ 105.010302] nix-grpc-daemon[1323]: time=2026-09-23T16:01:15.014Z level=INFO msg="Uploading 1 narinfos"5754lb # [ 104.976964] niks3-server[1128]: 2026/09/23 16:01:15 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.ls5755lb # [ 104.987347] niks3-server[1128]: 2026/09/23 16:01:15 INFO Received complete push request method=POST path=/api/pushes/39/complete5756lb # [ 104.991583] niks3-server[1128]: 2026/09/23 16:01:15 INFO Registered completed upload object_key=nar/0ppnnsmr641nka691g6vrp1crps1bbn31zm7nc5za9pl2y2iwb0f.nar.zst5757worker2 # [ 105.036397] nix-grpc-daemon[1323]: time=2026-09-23T16:01:15.041Z level=INFO msg="Upload complete. (180ms)"5758worker2 # [ 105.038481] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39208 duration_s=0 paths=1 nar_bytes_in=7845759lb # [ 105.000130] niks3-server[1128]: 2026/09/23 16:01:15 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.narinfo5760worker2 # [ 105.043326] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:392085761worker2 # [ 105.049529] nix-daemon[858]: accepted connection from pid 1263, user nix-grpc-daemon (trusted)5762worker2 # [ 105.051917] nix-daemon[858]: reaped child process 1420, status = succeeded5763worker2 # [ 105.057207] nix-daemon[858]: accepted connection from pid 1263, user nix-grpc-daemon (trusted)5764client # building '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' on lb:50051...5765client # farm-top> worker2: building v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5766client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5767client # farm-top> LOG-farm-top-after-niks3-restart5768worker2 # [ 106.273691] nix-daemon[858]: reaped child process 1429, status = succeeded5769worker2 # [ 106.354908] nix-daemon[858]: accepted connection from pid 1445, user nix-grpc-daemon (trusted)5770worker2 # [ 106.368476] nix-daemon[858]: reaped child process 1448, status = succeeded5771lb # [ 106.338523] niks3-server[1128]: 2026/09/23 16:01:16 INFO Received push request method=POST path=/api/pushes5772worker2 # [ 106.389960] nix-grpc-daemon[1323]: time=2026-09-23T16:01:16.394Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5773worker2 # [ 106.391785] nix-grpc-daemon[1323]: time=2026-09-23T16:01:16.396Z level=INFO msg="Uploading lyi0sw9pvmhznkzgfl39callq9c0n87r-farm-top (248B)"5774lb # [ 106.382167] niks3-server[1128]: 2026/09/23 16:01:16 INFO Received sign narinfos request method=POST path=/api/pushes/40/sign5775worker2 # [ 106.433297] nix-grpc-daemon[1323]: time=2026-09-23T16:01:16.437Z level=INFO msg="Uploading 1 narinfos"5776lb # [ 106.392331] niks3-server[1128]: 2026/09/23 16:01:16 INFO Signed narinfos id=40 count=15777lb # [ 106.396968] niks3-server[1128]: 2026/09/23 16:01:16 INFO Registered completed upload object_key=log/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5778lb # [ 106.401626] niks3-server[1128]: 2026/09/23 16:01:16 INFO Registered completed upload object_key=nar/141220x0digv3f58fbh9c79avwg0fwgsk5jyac9ymkxmr8zs2prw.nar.zst5779lb # [ 106.406642] niks3-server[1128]: 2026/09/23 16:01:16 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.ls5780lb # [ 106.414270] niks3-server[1128]: 2026/09/23 16:01:16 INFO Received complete push request method=POST path=/api/pushes/40/complete5781worker2 # [ 106.459824] nix-grpc-daemon[1323]: time=2026-09-23T16:01:16.464Z level=INFO msg="Upload complete. (196ms)"5782lb # [ 106.420247] niks3-server[1128]: 2026/09/23 16:01:16 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.narinfo5783worker2 # [ 106.462665] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39208 duration_s=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv assign_id=3 outputs=15784worker2 # [ 106.471480] nix-daemon[858]: reaped child process 1427, status = succeeded5785client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&2, in 5.56 seconds)5786client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath5787client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath, in 0.12 seconds)5788(finished: subtest: a niks3 restart does not cost the leader its role, in 6.05 seconds)5789subtest: scheduler down: the other node takes the lock and keeps it5790lb: must succeed: curl -sf localhost:9901/clusters5791lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5792lb: must succeed: curl -sf localhost:9901/clusters5793lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5794lb: must succeed: curl -sf localhost:9901/clusters5795lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5796lb: must succeed: curl -sf localhost:9901/clusters5797lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5798worker2 # [ 106.759904] systemd[1]: Stopping Nix worker-protocol over gRPC...5799lb # [ 106.824481] envoy[918]: conn peer=[2001:db8:1::3]:37446 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2093 tx=969 ms=646525800lb # [ 106.828286] envoy[918]: conn peer=[2001:db8:1::1]:53520 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1009 tx=282 ms=15765801lb # [ 106.831745] envoy[918]: conn peer=[2001:db8:1::1]:53532 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9679 tx=4420 ms=53925802worker2 # [ 106.946368] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:16Z level=info event=signal_exit5803lb # [ 107.612951] niks3-server[1128]: 2026/09/23 16:01:17 INFO lead: released remote=192.168.1.4:554905804worker2 # [ 108.651511] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:18Z level=info event=scheduler_restarting5805lb # [ 108.612754] envoy[918]: [2026-09-23 16:01:18.691][918][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5806worker2 # [ 108.660168] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:18Z level=info event=scheduler_restarting addr=lb:500515807worker2 # [ 108.663150] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:18Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:392205808worker1 # [ 108.637760] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:17Z level=info event=scheduler_restarting addr=lb:500515809worker2 # [ 108.664679] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:18Z level=info event=worker_gone addr=192.168.1.4:500515810worker2 # [ 108.666082] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:18Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:392205811worker2 # [ 108.667669] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:18Z level=info event=worker_gone addr=192.168.1.3:500515812worker2 # [ 108.681757] nix-daemon[858]: reaped child process 1315, status = succeeded5813worker1 # [ 108.837666] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:18Z level=info event=scheduler_connected addr=lb:500515814worker2 # [ 108.866903] nix-grpc-daemon[1263]: ts=2026-09-23T16:01:18Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5815worker1 # [ 108.841782] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5816worker1 # [ 109.042750] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:18Z level=info event=scheduler_connected addr=lb:500515817worker1 # [ 109.048302] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5818worker1 # [ 109.248657] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:18Z level=info event=scheduler_connected addr=lb:500515819worker1 # [ 109.252796] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5820worker1 # [ 109.453708] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:18Z level=info event=scheduler_connected addr=lb:500515821worker1 # [ 109.457661] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5822worker1 # [ 109.659547] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:18Z level=info event=scheduler_connected addr=lb:500515823worker1 # [ 109.663414] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5824worker1 # [ 109.864323] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:19Z level=info event=scheduler_connected addr=lb:500515825worker1 # [ 109.868321] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5826worker1 # [ 110.068814] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:19Z level=info event=scheduler_connected addr=lb:500515827worker1 # [ 110.072669] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5828worker1 # [ 110.273727] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:19Z level=info event=scheduler_connected addr=lb:500515829worker1 # [ 110.277782] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5830worker1 # [ 110.478546] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:19Z level=info event=scheduler_connected addr=lb:500515831worker1 # [ 110.482519] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5832worker1 # [ 110.684429] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:19Z level=info event=scheduler_connected addr=lb:500515833worker1 # [ 110.688372] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5834worker2 # [ 110.867174] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5835worker2 # [ 110.868919] systemd[1]: Stopped Nix worker-protocol over gRPC.5836worker2 # [ 110.870818] systemd[1]: nix-grpc-daemon.service: Consumed 872ms CPU time over 17.031s wall clock time, 38.6M memory peak, 200.6K incoming IP traffic, 124K outgoing IP traffic.5837worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5838worker1 # [ 110.890431] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:20Z level=info event=scheduler_connected addr=lb:500515839worker1 # [ 110.894341] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5840worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.06 seconds)5841worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5842worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)5843worker1 # [ 111.094842] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:20Z level=info event=scheduler_connected addr=lb:500515844worker1 # [ 111.098946] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5845worker1 # [ 111.299620] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:20Z level=info event=scheduler_connected addr=lb:500515846worker1 # [ 111.303539] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5847worker1 # [ 111.504628] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:20Z level=info event=scheduler_connected addr=lb:500515848worker1 # [ 111.508616] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5849worker1 # [ 111.626653] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:20Z level=info event=scheduler_take_over5850lb # [ 111.610811] niks3-server[1128]: 2026/09/23 16:01:21 INFO lead: acquired remote=192.168.1.3:337585851worker1 # [ 111.710424] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:20Z level=info event=scheduler_connected addr=lb:500515852worker1 # [ 111.714254] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5853worker1 # [ 111.915171] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:21Z level=info event=scheduler_connected addr=lb:500515854worker1 # [ 111.919236] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5855worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5856worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5857worker1 # [ 112.119839] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:21Z level=info event=scheduler_connected addr=lb:500515858worker1 # [ 112.132184] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:21Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:573525859worker1 # [ 112.135729] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:21Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05860worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5861worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5862client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&25863worker1 # [ 113.171200] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:555765864client # don't know how to build these paths:5865client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5866worker1 # [ 113.179830] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:555765867worker1 # [ 113.203075] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:22Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05868worker1 # [ 113.209529] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:22Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=15869worker1 # [ 113.214491] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555765870worker1 # [ 113.225209] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:555765871client # copying 1 paths to 'lb:50051'...5872client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5873worker1 # [ 113.237352] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:555765874worker1 # [ 113.248258] nix-daemon[865]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)5875worker1 # [ 113.265619] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:22Z level=info event=niks3_push_started pid=13115876worker1 # [ 113.366367] nix-grpc-daemon[1311]: time=2026-09-23T16:01:22.576Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""5877worker1 # [ 113.456548] nix-daemon[865]: accepted connection from pid 1324, user nix-grpc-daemon (trusted)5878worker1 # [ 113.468646] nix-daemon[865]: reaped child process 1327, status = succeeded5879lb # [ 113.462401] niks3-server[1128]: 2026/09/23 16:01:23 INFO Received push request method=POST path=/api/pushes5880worker1 # [ 113.487091] nix-grpc-daemon[1311]: time=2026-09-23T16:01:22.697Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5881worker1 # [ 113.488592] nix-grpc-daemon[1311]: time=2026-09-23T16:01:22.698Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5882lb # [ 113.501895] niks3-server[1128]: 2026/09/23 16:01:23 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5883lb # [ 113.508147] niks3-server[1128]: 2026/09/23 16:01:23 INFO Received sign narinfos request method=POST path=/api/pushes/41/sign5884lb # [ 113.513504] niks3-server[1128]: 2026/09/23 16:01:23 INFO Signed narinfos id=41 count=15885worker1 # [ 113.533047] nix-grpc-daemon[1311]: time=2026-09-23T16:01:22.743Z level=INFO msg="Uploading 1 narinfos"5886lb # [ 113.525984] niks3-server[1128]: 2026/09/23 16:01:23 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5887lb # [ 113.537536] niks3-server[1128]: 2026/09/23 16:01:23 INFO Received complete push request method=POST path=/api/pushes/41/complete5888worker1 # [ 113.561939] nix-grpc-daemon[1311]: time=2026-09-23T16:01:22.772Z level=INFO msg="Upload complete. (192ms)"5889lb # [ 113.548239] niks3-server[1128]: 2026/09/23 16:01:23 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5890worker1 # [ 113.565349] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55576 duration_s=0 paths=1 nar_bytes_in=5205891worker1 # [ 113.570621] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555765892worker1 # [ 113.580111] nix-daemon[865]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)5893worker1 # [ 113.585520] nix-daemon[865]: reaped child process 1309, status = succeeded5894worker1 # [ 113.590343] nix-daemon[865]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)5895client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5896client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5897client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5898client # farm-a> LOG-farm-a-on-standby5899worker1 # [ 114.624600] nix-daemon[865]: reaped child process 1331, status = succeeded5900worker1 # [ 114.713639] nix-daemon[865]: accepted connection from pid 1347, user nix-grpc-daemon (trusted)5901worker1 # [ 114.724475] nix-daemon[865]: reaped child process 1350, status = succeeded5902lb # [ 114.718268] niks3-server[1128]: 2026/09/23 16:01:24 INFO Received push request method=POST path=/api/pushes5903worker1 # [ 114.743597] nix-grpc-daemon[1311]: time=2026-09-23T16:01:23.953Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5904worker1 # [ 114.747473] nix-grpc-daemon[1311]: time=2026-09-23T16:01:23.955Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5905lb # [ 114.764634] niks3-server[1128]: 2026/09/23 16:01:24 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5906lb # [ 114.773256] niks3-server[1128]: 2026/09/23 16:01:24 INFO Received sign narinfos request method=POST path=/api/pushes/42/sign5907worker1 # [ 114.795064] nix-grpc-daemon[1311]: time=2026-09-23T16:01:24.005Z level=INFO msg="Uploading 1 narinfos"5908lb # [ 114.778898] niks3-server[1128]: 2026/09/23 16:01:24 INFO Signed narinfos id=42 count=15909lb # [ 114.783093] niks3-server[1128]: 2026/09/23 16:01:24 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5910lb # [ 114.788407] niks3-server[1128]: 2026/09/23 16:01:24 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5911lb # [ 114.798158] niks3-server[1128]: 2026/09/23 16:01:24 INFO Received complete push request method=POST path=/api/pushes/42/complete5912lb # [ 114.805812] niks3-server[1128]: 2026/09/23 16:01:24 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5913worker1 # [ 114.820214] nix-grpc-daemon[1311]: time=2026-09-23T16:01:24.030Z level=INFO msg="Upload complete. (222ms)"5914worker1 # [ 114.826607] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55576 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15915worker1 # [ 114.841329] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:24Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05916worker1 # [ 114.845497] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:24Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=15917worker1 # [ 114.849408] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555765918worker1 # [ 114.859292] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:555765919worker1 # [ 114.862047] nix-daemon[865]: reaped child process 1329, status = succeeded5920worker1 # [ 114.867996] nix-grpc-daemon[1311]: time=2026-09-23T16:01:24.078Z level=INFO msg="All 1 paths already cached"5921client # copying 1 paths to 'lb:50051'...5922worker1 # [ 114.873745] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:555765923client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5924worker1 # [ 114.878836] nix-daemon[865]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)5925worker1 # [ 114.964936] nix-daemon[865]: accepted connection from pid 1356, user nix-grpc-daemon (trusted)5926worker1 # [ 114.977967] nix-daemon[865]: reaped child process 1359, status = succeeded5927lb # [ 114.970147] niks3-server[1128]: 2026/09/23 16:01:25 INFO Received push request method=POST path=/api/pushes5928worker1 # [ 114.996939] nix-grpc-daemon[1311]: time=2026-09-23T16:01:24.207Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5929worker1 # [ 114.998992] nix-grpc-daemon[1311]: time=2026-09-23T16:01:24.208Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5930lb # [ 115.008404] niks3-server[1128]: 2026/09/23 16:01:25 INFO Received sign narinfos request method=POST path=/api/pushes/43/sign5931worker1 # [ 115.030244] nix-grpc-daemon[1311]: time=2026-09-23T16:01:24.240Z level=INFO msg="Uploading 1 narinfos"5932lb # [ 115.014186] niks3-server[1128]: 2026/09/23 16:01:25 INFO Signed narinfos id=43 count=15933lb # [ 115.022580] niks3-server[1128]: 2026/09/23 16:01:25 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5934lb # [ 115.026994] niks3-server[1128]: 2026/09/23 16:01:25 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5935lb # [ 115.039074] niks3-server[1128]: 2026/09/23 16:01:25 INFO Received complete push request method=POST path=/api/pushes/43/complete5936lb # [ 115.044405] niks3-server[1128]: 2026/09/23 16:01:25 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5937worker1 # [ 115.059509] nix-grpc-daemon[1311]: time=2026-09-23T16:01:24.269Z level=INFO msg="Upload complete. (175ms)"5938worker1 # [ 115.063555] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55576 duration_s=0 paths=1 nar_bytes_in=6405939worker1 # [ 115.068180] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555765940worker1 # [ 115.080135] nix-daemon[865]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)5941worker1 # [ 115.083592] nix-daemon[865]: reaped child process 1354, status = succeeded5942worker1 # [ 115.088503] nix-daemon[865]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)5943client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5944client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5945client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5946client # farm-b> LOG-farm-b-on-standby5947worker1 # [ 116.324781] nix-daemon[865]: reaped child process 1363, status = succeeded5948worker1 # [ 116.412134] nix-daemon[865]: accepted connection from pid 1379, user nix-grpc-daemon (trusted)5949worker1 # [ 116.425788] nix-daemon[865]: reaped child process 1382, status = succeeded5950lb # [ 116.415689] niks3-server[1128]: 2026/09/23 16:01:26 INFO Received push request method=POST path=/api/pushes5951worker1 # [ 116.442576] nix-grpc-daemon[1311]: time=2026-09-23T16:01:25.652Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5952worker1 # [ 116.444048] nix-grpc-daemon[1311]: time=2026-09-23T16:01:25.654Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"5953lb # [ 116.452086] niks3-server[1128]: 2026/09/23 16:01:26 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls5954lb # [ 116.465839] niks3-server[1128]: 2026/09/23 16:01:26 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5955lb # [ 116.472668] niks3-server[1128]: 2026/09/23 16:01:26 INFO Received sign narinfos request method=POST path=/api/pushes/44/sign5956lb # [ 116.479686] niks3-server[1128]: 2026/09/23 16:01:26 INFO Signed narinfos id=44 count=15957worker1 # [ 116.496080] nix-grpc-daemon[1311]: time=2026-09-23T16:01:25.705Z level=INFO msg="Uploading 1 narinfos"5958lb # [ 116.487878] niks3-server[1128]: 2026/09/23 16:01:26 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst5959lb # [ 116.495714] niks3-server[1128]: 2026/09/23 16:01:26 INFO Received complete push request method=POST path=/api/pushes/44/complete5960lb # [ 116.503704] niks3-server[1128]: 2026/09/23 16:01:26 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo5961worker1 # [ 116.518397] nix-grpc-daemon[1311]: time=2026-09-23T16:01:25.728Z level=INFO msg="Upload complete. (221ms)"5962worker1 # [ 116.524615] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55576 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=15963worker1 # [ 116.536827] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:25Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=05964worker1 # [ 116.541780] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:25Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=15965worker1 # [ 116.546281] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555765966worker1 # [ 116.556545] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:555765967worker1 # [ 116.559584] nix-daemon[865]: reaped child process 1361, status = succeeded5968worker1 # [ 116.565052] nix-grpc-daemon[1311]: time=2026-09-23T16:01:25.775Z level=INFO msg="All 2 paths already cached"5969client # copying 1 paths to 'lb:50051'...5970worker1 # [ 116.570751] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:555765971client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...5972worker1 # [ 116.577096] nix-daemon[865]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)5973worker1 # [ 116.662308] nix-daemon[865]: accepted connection from pid 1388, user nix-grpc-daemon (trusted)5974worker1 # [ 116.675933] nix-daemon[865]: reaped child process 1391, status = succeeded5975lb # [ 116.666427] niks3-server[1128]: 2026/09/23 16:01:26 INFO Received push request method=POST path=/api/pushes5976worker1 # [ 116.690584] nix-grpc-daemon[1311]: time=2026-09-23T16:01:25.900Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5977worker1 # [ 116.692594] nix-grpc-daemon[1311]: time=2026-09-23T16:01:25.902Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"5978lb # [ 116.695578] niks3-server[1128]: 2026/09/23 16:01:26 INFO Received sign narinfos request method=POST path=/api/pushes/45/sign5979worker1 # [ 116.716997] nix-grpc-daemon[1311]: time=2026-09-23T16:01:25.927Z level=INFO msg="Uploading 1 narinfos"5980lb # [ 116.700816] niks3-server[1128]: 2026/09/23 16:01:26 INFO Signed narinfos id=45 count=15981lb # [ 116.711235] niks3-server[1128]: 2026/09/23 16:01:26 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst5982lb # [ 116.720239] niks3-server[1128]: 2026/09/23 16:01:26 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls5983lb # [ 116.726899] niks3-server[1128]: 2026/09/23 16:01:26 INFO Received complete push request method=POST path=/api/pushes/45/complete5984worker1 # [ 116.751330] nix-grpc-daemon[1311]: time=2026-09-23T16:01:25.961Z level=INFO msg="Upload complete. (169ms)"5985lb # [ 116.737822] niks3-server[1128]: 2026/09/23 16:01:26 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo5986worker1 # [ 116.754823] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55576 duration_s=0 paths=1 nar_bytes_in=7685987worker1 # [ 116.759678] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555765988worker1 # [ 116.768673] nix-daemon[865]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)5989worker1 # [ 116.774539] nix-daemon[865]: reaped child process 1386, status = succeeded5990worker1 # [ 116.779435] nix-daemon[865]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)5991client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...5992client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5993lb # [ 116.824712] envoy[918]: conn peer=[2001:db8:1::4]:34320 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=10946 tx=959 ms=148785994client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5995client # farm-top> LOG-farm-top-on-standby5996worker1 # [ 117.994627] nix-daemon[865]: reaped child process 1395, status = succeeded5997worker1 # [ 118.071716] nix-daemon[865]: accepted connection from pid 1411, user nix-grpc-daemon (trusted)5998worker1 # [ 118.085526] nix-daemon[865]: reaped child process 1414, status = succeeded5999lb # [ 118.076744] niks3-server[1128]: 2026/09/23 16:01:28 INFO Received push request method=POST path=/api/pushes6000worker1 # [ 118.101426] nix-grpc-daemon[1311]: time=2026-09-23T16:01:27.311Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6001worker1 # [ 118.102905] nix-grpc-daemon[1311]: time=2026-09-23T16:01:27.313Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"6002lb # [ 118.117934] niks3-server[1128]: 2026/09/23 16:01:28 INFO Received sign narinfos request method=POST path=/api/pushes/46/sign6003worker1 # [ 118.138812] nix-grpc-daemon[1311]: time=2026-09-23T16:01:27.348Z level=INFO msg="Uploading 1 narinfos"6004lb # [ 118.123160] niks3-server[1128]: 2026/09/23 16:01:28 INFO Signed narinfos id=46 count=16005lb # [ 118.133677] niks3-server[1128]: 2026/09/23 16:01:28 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls6006lb # [ 118.147955] niks3-server[1128]: 2026/09/23 16:01:28 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6007lb # [ 118.151920] niks3-server[1128]: 2026/09/23 16:01:28 INFO Received complete push request method=POST path=/api/pushes/46/complete6008worker1 # [ 118.175338] nix-grpc-daemon[1311]: time=2026-09-23T16:01:27.385Z level=INFO msg="Upload complete. (188ms)"6009lb # [ 118.160493] niks3-server[1128]: 2026/09/23 16:01:28 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo6010lb # [ 118.164246] niks3-server[1128]: 2026/09/23 16:01:28 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst6011worker1 # [ 118.181606] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55576 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=16012worker1 # [ 118.197740] nix-daemon[865]: reaped child process 1393, status = succeeded6013client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&2, in 5.19 seconds)6014client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath6015client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath, in 0.11 seconds)6016worker2 # [ 118.413493] systemd[1]: Starting scheduler-token.service...6017worker2 # [ 118.460974] systemd[1]: scheduler-token.service: Deactivated successfully.6018worker2 # [ 118.462662] systemd[1]: Finished scheduler-token.service.6019worker2 # [ 118.463892] systemd[1]: scheduler-token.service: Consumed 23ms CPU time over 49ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6020worker2 # [ 118.468345] systemd[1]: Starting Nix worker-protocol over gRPC...6021worker2 # [ 118.552154] nix-grpc-daemon[1469]: ts=2026-09-23T16:01:28Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6022worker2 # [ 118.554177] nix-grpc-daemon[1469]: ts=2026-09-23T16:01:28Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6023worker2 # [ 118.557887] nix-grpc-daemon[1469]: ts=2026-09-23T16:01:28Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16024worker2 # [ 118.562057] nix-grpc-daemon[1469]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6025worker2 # [ 118.563421] nix-grpc-daemon[1469]: E0000 00:00:1790179288.566355 1469 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6026worker2 # [ 118.567061] nix-grpc-daemon[1469]: E0000 00:00:1790179288.571684 1469 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6027worker2 # [ 118.572108] nix-grpc-daemon[1469]: ts=2026-09-23T16:01:28Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket6028worker2 # [ 118.575494] systemd[1]: Started Nix worker-protocol over gRPC.6029worker2 # [ 118.579096] nix-grpc-daemon[1469]: E0000 00:00:1790179288.583465 1511 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6030worker2 # [ 118.592863] nix-grpc-daemon[1469]: E0000 00:00:1790179288.597541 1511 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6031lb: must succeed: curl -sf localhost:9901/clusters6032worker2 # [ 118.597156] nix-grpc-daemon[1469]: E0000 00:00:1790179288.599796 1511 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6033worker2 # [ 118.607409] nix-grpc-daemon[1469]: E0000 00:00:1790179288.612081 1511 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6034worker2 # [ 118.619206] nix-grpc-daemon[1469]: ts=2026-09-23T16:01:28Z level=info event=scheduler_connected addr=lb:500516035worker1 # [ 118.604178] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:27Z level=info event=oidc_jwks provider=mock keys=16036worker1 # [ 118.607325] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:27Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:573526037worker1 # [ 118.611655] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:27Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06038lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6039worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6040worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6041lb: must succeed: curl -sf localhost:9901/clusters6042lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6043lb: must succeed: curl -sf localhost:9901/clusters6044lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6045worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6046worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6047(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.20 seconds)6048subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise6049client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs6050client # [ 118.815614] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs.6051client # Running as unit: rs.service; invocation ID: 579572c43ed84a2caaf70cec484cf7166052client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs, in 0.06 seconds)6053client # [ 118.954727] nix[1364]: don't know how to build these paths:6054client # [ 118.955595] nix[1364]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6055worker1 # [ 118.966466] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:555766056worker1 # [ 118.979136] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:28Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=06057worker1 # [ 118.983961] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:28Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=16058worker1 # [ 118.989944] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555766059worker1 # [ 119.001927] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:555766060client # [ 118.998901] nix[1364]: copying 1 paths to 'lb:50051'...6061client # [ 118.999770] nix[1364]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...6062worker1 # [ 119.012957] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:555766063worker1 # [ 119.023731] nix-daemon[865]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)6064worker1 # [ 119.120883] nix-daemon[865]: accepted connection from pid 1427, user nix-grpc-daemon (trusted)6065worker1 # [ 119.134608] nix-daemon[865]: reaped child process 1430, status = succeeded6066lb # [ 119.125362] niks3-server[1128]: 2026/09/23 16:01:29 INFO Received push request method=POST path=/api/pushes6067worker1 # [ 119.154230] nix-grpc-daemon[1311]: time=2026-09-23T16:01:28.363Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6068worker1 # [ 119.155612] nix-grpc-daemon[1311]: time=2026-09-23T16:01:28.364Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"6069lb # [ 119.159698] niks3-server[1128]: 2026/09/23 16:01:29 INFO Received sign narinfos request method=POST path=/api/pushes/47/sign6070worker1 # [ 119.180900] nix-grpc-daemon[1311]: time=2026-09-23T16:01:28.391Z level=INFO msg="Uploading 1 narinfos"6071lb # [ 119.165471] niks3-server[1128]: 2026/09/23 16:01:29 INFO Signed narinfos id=47 count=16072lb # [ 119.172859] niks3-server[1128]: 2026/09/23 16:01:29 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst6073lb # [ 119.182046] niks3-server[1128]: 2026/09/23 16:01:29 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls6074lb # [ 119.189712] niks3-server[1128]: 2026/09/23 16:01:29 INFO Received complete push request method=POST path=/api/pushes/47/complete6075lb # [ 119.198748] niks3-server[1128]: 2026/09/23 16:01:29 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo6076worker1 # [ 119.218188] nix-grpc-daemon[1311]: time=2026-09-23T16:01:28.428Z level=INFO msg="Upload complete. (183ms)"6077worker1 # [ 119.223471] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55576 duration_s=0 paths=1 nar_bytes_in=4166078worker1 # [ 119.230353] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555766079worker1 # [ 119.241135] nix-daemon[865]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)6080worker1 # [ 119.252128] nix-daemon[865]: reaped child process 1425, status = succeeded6081worker1 # [ 119.255274] nix-daemon[865]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)6082client # [ 119.242421] nix[1364]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...6083lb: must succeed: curl -sf localhost:9901/clusters6084lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6085lb: must succeed: curl -sf localhost:9901/clusters6086lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6087lb: must succeed: curl -sf localhost:9901/clusters6088lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6089lb: must succeed: curl -sf localhost:9901/clusters6090lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6091worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true6092worker2 # [ 121.549539] systemd-journald[284]: Received client request to rotate journal, rotating.6093worker2 # [ 121.569617] systemd-journald[284]: Vacuuming done, freed 0B of archived journals from /var/log/journal/ad8a82e305c049cea52ff8c75b5ac7c1.6094worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.06 seconds)6095worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service6096worker1 # [ 121.451290] systemd[1]: Stopping Nix worker-protocol over gRPC...6097worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.03 seconds)6098client: must succeed: sleep 5.06099worker1 # [ 122.413727] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:31Z level=info event=draining6100client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6101worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='6102worker1 # [ 126.677525] nix-daemon[865]: reaped child process 1434, status = succeeded6103worker1 # [ 126.754062] nix-daemon[865]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6104worker1 # [ 126.765857] nix-daemon[865]: reaped child process 1478, status = succeeded6105lb # [ 126.760166] niks3-server[1128]: 2026/09/23 16:01:36 INFO Received push request method=POST path=/api/pushes6106worker1 # [ 126.783090] nix-grpc-daemon[1311]: time=2026-09-23T16:01:35.993Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6107worker1 # [ 126.784795] nix-grpc-daemon[1311]: time=2026-09-23T16:01:35.994Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"6108lb # [ 126.802075] niks3-server[1128]: 2026/09/23 16:01:36 INFO Received sign narinfos request method=POST path=/api/pushes/48/sign6109worker1 # [ 126.830188] nix-grpc-daemon[1311]: time=2026-09-23T16:01:36.040Z level=INFO msg="Uploading 1 narinfos"6110lb # [ 126.814387] niks3-server[1128]: 2026/09/23 16:01:36 INFO Signed narinfos id=48 count=16111lb # [ 126.825101] niks3-server[1128]: 2026/09/23 16:01:36 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst6112lb # [ 126.828833] envoy[918]: conn peer=[2001:db8:1::1]:45224 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9633 tx=4403 ms=50276113lb # [ 126.834451] niks3-server[1128]: 2026/09/23 16:01:36 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6114lb # [ 126.841904] niks3-server[1128]: 2026/09/23 16:01:36 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls6115lb # [ 126.845589] niks3-server[1128]: 2026/09/23 16:01:36 INFO Received complete push request method=POST path=/api/pushes/48/complete6116lb # [ 126.851558] niks3-server[1128]: 2026/09/23 16:01:36 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo6117worker1 # [ 126.870809] nix-grpc-daemon[1311]: time=2026-09-23T16:01:36.081Z level=INFO msg="Upload complete. (202ms)"6118worker1 # [ 126.878821] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55576 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=16119worker1 # [ 126.894155] nix-daemon[865]: reaped child process 1432, status = succeeded6120client # [ 126.899956] systemd[1]: rs.service: Deactivated successfully.6121client # [ 126.903719] systemd[1]: rs.service: Consumed 124ms CPU time over 8.087s wall clock time, 43.4M memory peak, 7.2K incoming IP traffic, 8.3K outgoing IP traffic.6122worker1 # [ 127.417084] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:36Z level=info event=signal_exit6123lb # [ 127.615331] niks3-server[1128]: 2026/09/23 16:01:37 INFO lead: released remote=192.168.1.3:337586124lb # [ 128.540827] niks3-server[1128]: 2026/09/23 16:01:38 INFO lead: acquired remote=192.168.1.4:540326125worker2 # [ 128.585331] nix-grpc-daemon[1469]: ts=2026-09-23T16:01:38Z level=info event=scheduler_take_over6126worker2 # [ 128.657465] nix-grpc-daemon[1469]: ts=2026-09-23T16:01:38Z level=info event=scheduler_restarting addr=lb:500516127worker1 # [ 128.630641] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:37Z level=info event=scheduler_restarting6128worker1 # [ 128.640770] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:37Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:573526129worker1 # [ 128.644328] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:37Z level=info event=worker_gone addr=192.168.1.4:500516130worker1 # [ 128.647553] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:37Z level=info event=scheduler_restarting addr=lb:500516131worker1 # [ 128.650782] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:37Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:573526132worker1 # [ 128.654432] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:37Z level=info event=worker_gone addr=192.168.1.3:500516133worker1 # [ 128.669662] nix-daemon[865]: reaped child process 1284, status = succeeded6134worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.11 seconds)6135client: waiting for success: ! systemctl is-active rs6136client: (finished: waiting for success: ! systemctl is-active rs, in 0.04 seconds)6137client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }6138worker2 # [ 128.859171] nix-grpc-daemon[1469]: ts=2026-09-23T16:01:38Z level=info event=scheduler_connected addr=lb:500516139worker2 # [ 128.863224] nix-grpc-daemon[1469]: ts=2026-09-23T16:01:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6140worker1 # [ 128.848820] nix-grpc-daemon[1232]: ts=2026-09-23T16:01:38Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6141client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.03 seconds)6142client: must fail: journalctl -u rs | grep -q 'reconnecting'6143client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.01 seconds)6144lb: must succeed: curl -sf localhost:9901/clusters6145lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6146worker2 # [ 129.064291] nix-grpc-daemon[1469]: ts=2026-09-23T16:01:39Z level=info event=scheduler_connected addr=lb:500516147worker2 # [ 129.068328] nix-grpc-daemon[1469]: ts=2026-09-23T16:01:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6148worker2 # [ 129.269540] nix-grpc-daemon[1469]: ts=2026-09-23T16:01:39Z level=info event=scheduler_connected addr=lb:500516149worker2 # [ 129.273672] nix-grpc-daemon[1469]: ts=2026-09-23T16:01:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6150worker2 # [ 129.474912] nix-grpc-daemon[1469]: ts=2026-09-23T16:01:39Z level=info event=scheduler_connected addr=lb:500516151worker2 # [ 129.478944] nix-grpc-daemon[1469]: ts=2026-09-23T16:01:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6152worker2 # [ 129.681201] nix-grpc-daemon[1469]: ts=2026-09-23T16:01:39Z level=info event=scheduler_connected addr=lb:500516153worker2 # [ 129.685188] nix-grpc-daemon[1469]: ts=2026-09-23T16:01:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6154worker2 # [ 129.886680] nix-grpc-daemon[1469]: ts=2026-09-23T16:01:39Z level=info event=scheduler_connected addr=lb:500516155worker2 # [ 129.890689] nix-grpc-daemon[1469]: ts=2026-09-23T16:01:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6156lb: must succeed: curl -sf localhost:9901/clusters6157lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6158worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6159worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6160worker2 # [ 130.091538] nix-grpc-daemon[1469]: ts=2026-09-23T16:01:40Z level=info event=scheduler_connected addr=lb:500516161worker2 # [ 130.104142] nix-grpc-daemon[1469]: ts=2026-09-23T16:01:40Z level=info event=oidc_jwks provider=mock keys=16162worker2 # [ 130.108158] nix-grpc-daemon[1469]: ts=2026-09-23T16:01:40Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:538926163worker2 # [ 130.112177] nix-grpc-daemon[1469]: ts=2026-09-23T16:01:40Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06164worker1 # [ 130.698951] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6165worker1 # [ 130.702162] systemd[1]: Stopped Nix worker-protocol over gRPC.6166worker1 # [ 130.705340] systemd[1]: nix-grpc-daemon.service: Consumed 1.078s CPU time over 31.405s wall clock time, 37.3M memory peak, 97.3K incoming IP traffic, 149.8K outgoing IP traffic.6167worker1 # [ 130.720158] systemd[1]: Starting Nix worker-protocol over gRPC...6168worker1 # [ 130.810700] nix-grpc-daemon[1486]: ts=2026-09-23T16:01:40Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6169worker1 # [ 130.815189] nix-grpc-daemon[1486]: ts=2026-09-23T16:01:40Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6170worker1 # [ 130.820051] nix-grpc-daemon[1486]: ts=2026-09-23T16:01:40Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16171worker1 # [ 130.822031] nix-grpc-daemon[1486]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6172worker1 # [ 130.823352] nix-grpc-daemon[1486]: E0000 00:00:1790179300.028107 1486 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6173worker1 # [ 130.825674] nix-grpc-daemon[1486]: E0000 00:00:1790179300.035936 1486 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6174worker1 # [ 130.832462] nix-grpc-daemon[1486]: ts=2026-09-23T16:01:40Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket6175worker1 # [ 130.834536] systemd[1]: Started Nix worker-protocol over gRPC.6176worker1 # [ 130.836142] nix-grpc-daemon[1486]: E0000 00:00:1790179300.045876 1529 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6177worker1 # [ 130.847512] nix-grpc-daemon[1486]: E0000 00:00:1790179300.057790 1529 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6178worker1 # [ 130.852061] nix-grpc-daemon[1486]: E0000 00:00:1790179300.060106 1529 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6179worker1 # [ 130.861452] nix-grpc-daemon[1486]: E0000 00:00:1790179300.071741 1529 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6180worker1 # [ 130.870728] nix-grpc-daemon[1486]: ts=2026-09-23T16:01:40Z level=info event=scheduler_connected addr=lb:500516181worker2 # [ 130.898448] nix-grpc-daemon[1469]: ts=2026-09-23T16:01:40Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:538926182worker2 # [ 130.902734] nix-grpc-daemon[1469]: ts=2026-09-23T16:01:40Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06183lb: must succeed: curl -sf localhost:9901/clusters6184lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6185worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6186worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6187(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.32 seconds)6188subtest: scheduler restart mid-build: build finishes, no double build6189client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid6190client # [ 131.130919] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid.6191client # Running as unit: mid.service; invocation ID: 0eeafa19ecef434e82388c7e68e01e336192client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid, in 0.06 seconds)6193worker2 # [ 131.226504] nix-grpc-daemon[1469]: ts=2026-09-23T16:01:41Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:539006194worker2 # [ 131.310599] nix-grpc-daemon[1469]: ts=2026-09-23T16:01:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:539026195worker2 # [ 131.322160] nix-daemon[858]: accepted connection from pid 1469, user nix-grpc-daemon (trusted)6196client # [ 131.295416] nix[1400]: don't know how to build these paths:6197client # [ 131.296358] nix[1400]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6198worker2 # [ 131.339135] nix-grpc-daemon[1469]: ts=2026-09-23T16:01:41Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06199worker2 # [ 131.343100] nix-grpc-daemon[1469]: ts=2026-09-23T16:01:41Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=16200worker2 # [ 131.348339] nix-grpc-daemon[1469]: ts=2026-09-23T16:01:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:539026201worker2 # [ 131.355757] nix-grpc-daemon[1469]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 85 ms (attempt 1/5)6202worker2 # [ 131.445441] nix-grpc-daemon[1469]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 67 ms (attempt 2/5)6203worker2 # [ 131.514688] nix-grpc-daemon[1469]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 106 ms (attempt 3/5)6204worker2 # [ 131.622775] nix-grpc-daemon[1469]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 562 ms (attempt 4/5)6205worker2 # [ 132.187905] nix-grpc-daemon[1469]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6206worker2 # [ 132.192178] nix-grpc-daemon[1469]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6207worker2 # [ 132.210469] nix-grpc-daemon[1469]: ts=2026-09-23T16:01:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:539026208client # [ 132.182799] nix[1400]: copying 1 paths to 'lb:50051'...6209client # [ 132.187135] nix[1400]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...6210worker2 # [ 132.227153] nix-grpc-daemon[1469]: ts=2026-09-23T16:01:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:539026211worker2 # [ 132.236168] nix-daemon[858]: accepted connection from pid 1469, user nix-grpc-daemon (trusted)6212worker2 # [ 132.259226] nix-grpc-daemon[1469]: ts=2026-09-23T16:01:42Z level=info event=niks3_push_started pid=15916213worker2 # [ 132.374429] nix-grpc-daemon[1591]: time=2026-09-23T16:01:42.378Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""6214worker2 # [ 132.469449] nix-daemon[858]: accepted connection from pid 1608, user nix-grpc-daemon (trusted)6215worker2 # [ 132.482713] nix-daemon[858]: reaped child process 1611, status = succeeded6216lb # [ 132.447743] niks3-server[1128]: 2026/09/23 16:01:42 INFO Received push request method=POST path=/api/pushes6217worker2 # [ 132.498077] nix-grpc-daemon[1591]: time=2026-09-23T16:01:42.501Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6218worker2 # [ 132.499470] nix-grpc-daemon[1591]: time=2026-09-23T16:01:42.501Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"6219lb # [ 132.485518] niks3-server[1128]: 2026/09/23 16:01:42 INFO Received sign narinfos request method=POST path=/api/pushes/49/sign6220lb # [ 132.492299] niks3-server[1128]: 2026/09/23 16:01:42 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst6221lb # [ 132.499308] niks3-server[1128]: 2026/09/23 16:01:42 INFO Signed narinfos id=49 count=16222worker2 # [ 132.543836] nix-grpc-daemon[1591]: time=2026-09-23T16:01:42.547Z level=INFO msg="Uploading 1 narinfos"6223lb # [ 132.503102] niks3-server[1128]: 2026/09/23 16:01:42 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls6224lb # [ 132.517183] niks3-server[1128]: 2026/09/23 16:01:42 INFO Received complete push request method=POST path=/api/pushes/49/complete6225worker2 # [ 132.569416] nix-grpc-daemon[1591]: time=2026-09-23T16:01:42.574Z level=INFO msg="Upload complete. (192ms)"6226lb # [ 132.530321] niks3-server[1128]: 2026/09/23 16:01:42 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo6227worker2 # [ 132.572190] nix-grpc-daemon[1469]: ts=2026-09-23T16:01:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53902 duration_s=0 paths=1 nar_bytes_in=4246228worker2 # [ 132.577764] nix-grpc-daemon[1469]: ts=2026-09-23T16:01:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:539026229worker2 # [ 132.588871] nix-daemon[858]: accepted connection from pid 1469, user nix-grpc-daemon (trusted)6230worker2 # [ 132.595463] nix-daemon[858]: reaped child process 1589, status = succeeded6231worker2 # [ 132.598311] nix-daemon[858]: accepted connection from pid 1469, user nix-grpc-daemon (trusted)6232client # [ 132.561394] nix[1400]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6233lb: must succeed: curl -sf localhost:9901/clusters6234lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6235lb: must succeed: curl -sf localhost:9901/clusters6236lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6237worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6238worker2 # [ 133.715283] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1469 (nix-grpc-daemon) on client request.6239worker2 # [ 133.719382] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6240lb # [ 133.688277] niks3-server[1128]: 2026/09/23 16:01:43 INFO lead: released remote=192.168.1.4:540326241client # [ 133.698669] nix[1400]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again6242lb # [ 133.694149] envoy[918]: [2026-09-23 16:01:43.771][918][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 13, upstream_reset_after_response_started{connection_termination}6243worker2 # [ 133.740321] systemd[1]: Starting scheduler-token.service...6244worker1 # [ 133.711565] nix-grpc-daemon[1486]: ts=2026-09-23T16:01:42Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"6245worker2 # [ 133.741130] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6246worker2 # [ 133.742445] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6247client # [ 133.703366] nix[1400]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6248worker2 # [ 133.745106] systemd[1]: nix-grpc-daemon.service: Consumed 405ms CPU time over 15.269s wall clock time, 36M memory peak, 159.8K incoming IP traffic, 57.1K outgoing IP traffic.6249worker2 # [ 133.771658] nix-daemon[858]: reaped child process 1613, status = succeeded6250worker2 # [ 133.773402] nix-daemon[858]: reaped child process 1581, status = succeeded6251worker2 # [ 133.781272] nix-daemon[1615]: unexpected Nix daemon error: error: interrupted by the user6252worker2 # [ 133.791181] nix-daemon[858]: reaped child process 1615, status = failed with exit code 16253worker2 # [ 133.803180] systemd[1]: scheduler-token.service: Deactivated successfully.6254worker2 # [ 133.804701] systemd[1]: Finished scheduler-token.service.6255worker2 # [ 133.806176] systemd[1]: scheduler-token.service: Consumed 18ms CPU time over 67ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6256worker2 # [ 133.809071] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6257worker2 # [ 133.812777] systemd[1]: Starting scheduler-token.service...6258worker2 # [ 133.847833] systemd[1]: scheduler-token.service: Deactivated successfully.6259worker2 # [ 133.849441] systemd[1]: Finished scheduler-token.service.6260worker2 # [ 133.851159] systemd[1]: scheduler-token.service: Consumed 17ms CPU time over 36ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6261worker2 # [ 133.855316] systemd[1]: Starting Nix worker-protocol over gRPC...6262worker2 # [ 133.933396] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:43Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6263worker2 # [ 133.935403] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:43Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6264worker1 # [ 133.911832] nix-grpc-daemon[1486]: ts=2026-09-23T16:01:43Z level=info event=scheduler_connected addr=lb:500516265worker2 # [ 133.939253] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:43Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16266worker2 # [ 133.944054] nix-grpc-daemon[1654]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6267worker2 # [ 133.945354] nix-grpc-daemon[1654]: E0000 00:00:1790179303.948396 1654 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6268worker2 # [ 133.948649] nix-grpc-daemon[1654]: E0000 00:00:1790179303.953288 1654 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6269worker2 # [ 133.952465] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:43Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket6270worker2 # [ 133.955429] systemd[1]: Started Nix worker-protocol over gRPC.6271worker2 # [ 133.962074] nix-grpc-daemon[1654]: E0000 00:00:1790179303.965239 1696 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6272lb # [ 133.921310] niks3-server[1128]: 2026/09/23 16:01:44 INFO lead: acquired remote=192.168.1.4:486046273worker2 # [ 133.965097] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:43Z level=info event=scheduler_take_over6274worker2 # [ 133.975291] nix-grpc-daemon[1654]: E0000 00:00:1790179303.980026 1696 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6275worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.29 seconds)6276worker2 # [ 133.979207] nix-grpc-daemon[1654]: E0000 00:00:1790179303.982549 1696 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6277client: must succeed: sleep 5.06278worker2 # [ 133.984512] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:43Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:569386279worker2 # [ 133.987093] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:43Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06280worker2 # [ 133.996157] nix-grpc-daemon[1654]: E0000 00:00:1790179304.000786 1696 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6281worker2 # [ 134.005231] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:44Z level=info event=scheduler_connected addr=lb:500516282worker2 # [ 134.010057] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:44Z level=info event=oidc_jwks provider=mock keys=16283worker2 # [ 134.011500] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:44Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:569386284worker2 # [ 134.012998] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:44Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06285worker2 # [ 134.250384] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:44Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06286worker2 # [ 134.255181] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:44Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=16287worker1 # [ 134.240848] nix-grpc-daemon[1486]: ts=2026-09-23T16:01:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:523346288worker1 # [ 134.252460] nix-daemon[865]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6289worker1 # [ 134.268762] nix-grpc-daemon[1486]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 95 ms (attempt 1/5)6290worker2 # [ 134.312573] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:44Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:569726291worker1 # [ 134.371486] nix-grpc-daemon[1486]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 106 ms (attempt 2/5)6292worker1 # [ 134.485259] nix-grpc-daemon[1486]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 255 ms (attempt 3/5)6293worker1 # [ 134.743230] nix-grpc-daemon[1486]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 391 ms (attempt 4/5)6294worker1 # [ 135.135827] nix-grpc-daemon[1486]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6295worker1 # [ 135.139323] nix-grpc-daemon[1486]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6296worker1 # [ 135.154479] nix-grpc-daemon[1486]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' from 'http://lb:5751'...6297worker1 # [ 135.176914] nix-daemon[865]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6298client # [ 135.179653] nix[1400]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6299worker1 # [ 135.192736] nix-daemon[865]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6300lb # [ 136.825099] envoy[918]: conn peer=[2001:db8:1::1]:45232 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2943 tx=1497 ms=79296301lb # [ 136.829470] envoy[918]: conn peer=[2001:db8:1::3]:44112 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=4857 tx=1250 ms=293906302lb # [ 136.833201] envoy[918]: conn peer=[2001:db8:1::4]:49786 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=7013 tx=556 ms=151226303client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6304client: waiting for success: ! systemctl is-active mid6305worker1 # [ 139.212660] nix-grpc-daemon[1486]: ts=2026-09-23T16:01:48Z level=info event=niks3_push_started pid=15886306worker1 # [ 139.237918] nix-daemon[865]: reaped child process 1563, status = succeeded6307worker1 # [ 139.320609] nix-grpc-daemon[1588]: time=2026-09-23T16:01:48.530Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6308worker1 # [ 139.409334] nix-daemon[865]: accepted connection from pid 1601, user nix-grpc-daemon (trusted)6309worker1 # [ 139.422473] nix-daemon[865]: reaped child process 1604, status = succeeded6310lb # [ 139.416680] niks3-server[1128]: 2026/09/23 16:01:49 INFO Received push request method=POST path=/api/pushes6311worker1 # [ 139.442756] nix-grpc-daemon[1588]: time=2026-09-23T16:01:48.653Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6312worker1 # [ 139.444312] nix-grpc-daemon[1588]: time=2026-09-23T16:01:48.654Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6313lb # [ 139.466664] niks3-server[1128]: 2026/09/23 16:01:49 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6314lb # [ 139.473454] niks3-server[1128]: 2026/09/23 16:01:49 INFO Received sign narinfos request method=POST path=/api/pushes/50/sign6315worker1 # [ 139.497494] nix-grpc-daemon[1588]: time=2026-09-23T16:01:48.707Z level=INFO msg="Uploading 1 narinfos"6316lb # [ 139.481969] niks3-server[1128]: 2026/09/23 16:01:49 INFO Signed narinfos id=50 count=16317lb # [ 139.493359] niks3-server[1128]: 2026/09/23 16:01:49 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6318lb # [ 139.501175] niks3-server[1128]: 2026/09/23 16:01:49 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6319lb # [ 139.506575] niks3-server[1128]: 2026/09/23 16:01:49 INFO Received complete push request method=POST path=/api/pushes/50/complete6320worker1 # [ 139.529406] nix-grpc-daemon[1588]: time=2026-09-23T16:01:48.739Z level=INFO msg="Upload complete. (205ms)"6321worker1 # [ 139.535531] nix-grpc-daemon[1486]: ts=2026-09-23T16:01:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52334 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16322worker1 # [ 139.549563] nix-daemon[865]: reaped child process 1561, status = succeeded6323client # [ 139.555439] systemd[1]: mid.service: Deactivated successfully.6324client # [ 139.559207] systemd[1]: mid.service: Consumed 142ms CPU time over 8.427s wall clock time, 43.5M memory peak, 8.6K incoming IP traffic, 10.2K outgoing IP traffic.6325client: (finished: waiting for success: ! systemctl is-active mid, in 1.08 seconds)6326client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6327client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.03 seconds)6328(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 9.18 seconds)6329subtest: interrupting the client stops the build on the worker6330client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr6331client # [ 140.284402] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr.6332client # Running as unit: intr.service; invocation ID: a88ccb2047e747d6bdec3da8649f5eaa6333client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr, in 0.03 seconds)6334worker2 # [ 140.445709] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:50Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:569806335worker2 # [ 140.454826] nix-daemon[858]: accepted connection from pid 1654, user nix-grpc-daemon (trusted)6336client # [ 140.427661] nix[1436]: don't know how to build these paths:6337client # [ 140.428635] nix[1436]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6338worker2 # [ 140.471771] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:50Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06339worker2 # [ 140.475187] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:50Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=16340worker2 # [ 140.480980] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569806341worker2 # [ 140.485857] nix-grpc-daemon[1654]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 4 ms (attempt 1/5)6342worker2 # [ 140.492793] nix-grpc-daemon[1654]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 86 ms (attempt 2/5)6343worker2 # [ 140.583334] nix-grpc-daemon[1654]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 301 ms (attempt 3/5)6344worker2 # [ 140.892109] nix-grpc-daemon[1654]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 742 ms (attempt 4/5)6345worker2 # [ 141.658978] nix-grpc-daemon[1654]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6346worker2 # [ 141.662739] nix-grpc-daemon[1654]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6347worker2 # [ 141.679729] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:569806348client # [ 141.652631] nix[1436]: copying 1 paths to 'lb:50051'...6349client # [ 141.655904] nix[1436]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6350worker2 # [ 141.694073] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:569806351worker2 # [ 141.705494] nix-daemon[858]: accepted connection from pid 1654, user nix-grpc-daemon (trusted)6352worker2 # [ 141.726252] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:51Z level=info event=niks3_push_started pid=17346353worker2 # [ 141.832350] nix-grpc-daemon[1734]: time=2026-09-23T16:01:51.835Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""6354worker2 # [ 141.919147] nix-daemon[858]: accepted connection from pid 1747, user nix-grpc-daemon (trusted)6355worker2 # [ 141.931963] nix-daemon[858]: reaped child process 1750, status = succeeded6356lb # [ 141.899224] niks3-server[1128]: 2026/09/23 16:01:51 INFO Received push request method=POST path=/api/pushes6357worker2 # [ 141.951514] nix-grpc-daemon[1734]: time=2026-09-23T16:01:51.956Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6358worker2 # [ 141.953044] nix-grpc-daemon[1734]: time=2026-09-23T16:01:51.957Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6359lb # [ 141.942429] niks3-server[1128]: 2026/09/23 16:01:52 INFO Received sign narinfos request method=POST path=/api/pushes/51/sign6360lb # [ 141.954153] niks3-server[1128]: 2026/09/23 16:01:52 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6361lb # [ 141.961232] niks3-server[1128]: 2026/09/23 16:01:52 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6362worker2 # [ 142.013201] nix-grpc-daemon[1734]: time=2026-09-23T16:01:52.017Z level=INFO msg="Uploading 1 narinfos"6363lb # [ 141.972293] niks3-server[1128]: 2026/09/23 16:01:52 INFO Signed narinfos id=51 count=16364lb # [ 141.983727] niks3-server[1128]: 2026/09/23 16:01:52 INFO Received complete push request method=POST path=/api/pushes/51/complete6365lb # [ 141.994773] niks3-server[1128]: 2026/09/23 16:01:52 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6366worker2 # [ 142.039751] nix-grpc-daemon[1734]: time=2026-09-23T16:01:52.044Z level=INFO msg="Upload complete. (207ms)"6367worker2 # [ 142.044096] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56980 duration_s=0 paths=1 nar_bytes_in=4246368worker2 # [ 142.050246] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569806369worker2 # [ 142.059259] nix-daemon[858]: accepted connection from pid 1654, user nix-grpc-daemon (trusted)6370worker2 # [ 142.067373] nix-daemon[858]: reaped child process 1732, status = succeeded6371worker2 # [ 142.070088] nix-daemon[858]: accepted connection from pid 1654, user nix-grpc-daemon (trusted)6372client # [ 142.032405] nix[1436]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6373client: must succeed: systemctl kill -s INT intr6374client # [ 143.712993] systemd[1]: intr.service: Sent signal SIGINT to main process 1436 (nix) on client request.6375worker2 # [ 143.753222] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:53Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6376client # [ 143.719691] nix[1436]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6377client # [ 143.724769] nix[1436]: error: interrupted by the user6378client: (finished: must succeed: systemctl kill -s INT intr, in 0.05 seconds)6379client # [ 143.752843] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6380client # [ 143.755716] systemd[1]: intr.service: Failed with result 'exit-code'.6381client # [ 143.759448] systemd[1]: intr.service: Consumed 117ms CPU time over 3.471s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic.6382worker2 # [ 143.890461] nix-daemon[1754]: unexpected Nix daemon error: error: interrupted by the user6383worker2 # [ 143.895186] nix-daemon[858]: reaped child process 1752, status = succeeded6384worker2 # [ 143.896338] nix-daemon[858]: reaped child process 1754, status = failed with exit code 16385(finished: subtest: interrupting the client stops the build on the worker, in 4.65 seconds)6386subtest: a deploy mid-build drains: the build finishes, the new generation takes over6387client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw6388client # [ 144.958154] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw.6389client # Running as unit: sw.service; invocation ID: 49b13b30de344a0f8ad9e5c254823c7d6390client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw, in 0.06 seconds)6391client # [ 145.084591] nix[1456]: don't know how to build these paths:6392client # [ 145.085482] nix[1456]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6393worker2 # [ 145.121556] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:569806394worker2 # [ 145.134158] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:55Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06395worker2 # [ 145.138327] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:55Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=16396worker1 # [ 145.121546] nix-grpc-daemon[1486]: ts=2026-09-23T16:01:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:523346397worker1 # [ 145.134796] nix-grpc-daemon[1486]: ts=2026-09-23T16:01:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:523346398client # [ 145.131984] nix[1456]: copying 1 paths to 'lb:50051'...6399client # [ 145.132878] nix[1456]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6400worker1 # [ 145.146136] nix-grpc-daemon[1486]: ts=2026-09-23T16:01:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:523346401worker1 # [ 145.156797] nix-daemon[865]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6402worker1 # [ 145.258437] nix-daemon[865]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)6403worker1 # [ 145.271153] nix-daemon[865]: reaped child process 1639, status = succeeded6404lb # [ 145.263728] niks3-server[1128]: 2026/09/23 16:01:55 INFO Received push request method=POST path=/api/pushes6405worker1 # [ 145.290534] nix-grpc-daemon[1588]: time=2026-09-23T16:01:54.500Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6406worker1 # [ 145.291998] nix-grpc-daemon[1588]: time=2026-09-23T16:01:54.500Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6407lb # [ 145.292717] niks3-server[1128]: 2026/09/23 16:01:55 INFO Received sign narinfos request method=POST path=/api/pushes/52/sign6408lb # [ 145.298974] niks3-server[1128]: 2026/09/23 16:01:55 INFO Signed narinfos id=52 count=16409worker1 # [ 145.318624] nix-grpc-daemon[1588]: time=2026-09-23T16:01:54.528Z level=INFO msg="Uploading 1 narinfos"6410lb # [ 145.308334] niks3-server[1128]: 2026/09/23 16:01:55 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6411lb # [ 145.317908] niks3-server[1128]: 2026/09/23 16:01:55 INFO Received complete push request method=POST path=/api/pushes/52/complete6412lb # [ 145.330260] niks3-server[1128]: 2026/09/23 16:01:55 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6413worker1 # [ 145.351926] nix-grpc-daemon[1588]: time=2026-09-23T16:01:54.562Z level=INFO msg="Upload complete. (183ms)"6414lb # [ 145.340054] niks3-server[1128]: 2026/09/23 16:01:55 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6415worker1 # [ 145.356703] nix-grpc-daemon[1486]: ts=2026-09-23T16:01:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52334 duration_s=0 paths=1 nar_bytes_in=4166416worker1 # [ 145.361395] nix-grpc-daemon[1486]: ts=2026-09-23T16:01:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:523346417worker1 # [ 145.373141] nix-daemon[865]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6418worker1 # [ 145.376353] nix-daemon[865]: reaped child process 1634, status = succeeded6419worker1 # [ 145.381627] nix-daemon[865]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6420client # [ 145.368634] nix[1456]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6421lb # [ 146.825650] envoy[918]: conn peer=[2001:db8:1::1]:49242 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3878 tx=1973 ms=82656422lb # [ 146.830509] envoy[918]: conn peer=[2001:db8:1::1]:34486 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2980 tx=829 ms=33366423worker1: must succeed: systemctl reload nix-grpc-daemon.service6424worker1 # [ 147.404208] systemd[1]: Reloading Nix worker-protocol over gRPC...6425worker1 # [ 147.432677] systemd[1]: Reloaded Nix worker-protocol over gRPC.6426worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.06 seconds)6427worker1: must succeed: pgrep -f 'read -t [9]0 x'6428worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.03 seconds)6429client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&26430worker2 # [ 147.639569] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:57Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:569806431client # don't know how to build these paths:6432client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6433worker2 # [ 147.650805] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:569806434worker2 # [ 147.672723] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:57Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06435worker2 # [ 147.677709] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:57Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=16436worker2 # [ 147.683661] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569806437worker2 # [ 147.694728] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:569806438client # copying 1 paths to 'lb:50051'...6439worker2 # [ 147.705991] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:569806440client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6441worker2 # [ 147.724276] nix-daemon[858]: accepted connection from pid 1654, user nix-grpc-daemon (trusted)6442worker2 # [ 147.821597] nix-daemon[858]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)6443worker2 # [ 147.835362] nix-daemon[858]: reaped child process 1812, status = succeeded6444lb # [ 147.802597] niks3-server[1128]: 2026/09/23 16:01:57 INFO Received push request method=POST path=/api/pushes6445worker2 # [ 147.851855] nix-grpc-daemon[1734]: time=2026-09-23T16:01:57.856Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6446worker2 # [ 147.853684] nix-grpc-daemon[1734]: time=2026-09-23T16:01:57.858Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6447worker1 # [ 147.836785] nix-grpc-daemon[1486]: ts=2026-09-23T16:01:57Z level=info event=draining6448lb # [ 147.842181] niks3-server[1128]: 2026/09/23 16:01:57 INFO Received sign narinfos request method=POST path=/api/pushes/53/sign6449worker2 # [ 147.891579] nix-grpc-daemon[1734]: time=2026-09-23T16:01:57.895Z level=INFO msg="Uploading 1 narinfos"6450lb # [ 147.850411] niks3-server[1128]: 2026/09/23 16:01:57 INFO Signed narinfos id=53 count=16451lb # [ 147.859832] niks3-server[1128]: 2026/09/23 16:01:57 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6452lb # [ 147.864217] niks3-server[1128]: 2026/09/23 16:01:57 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6453lb # [ 147.877654] niks3-server[1128]: 2026/09/23 16:01:57 INFO Received complete push request method=POST path=/api/pushes/53/complete6454lb # [ 147.881793] niks3-server[1128]: 2026/09/23 16:01:57 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6455worker2 # [ 147.921320] nix-grpc-daemon[1734]: time=2026-09-23T16:01:57.925Z level=INFO msg="Upload complete. (185ms)"6456worker2 # [ 147.926624] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56980 duration_s=0 paths=1 nar_bytes_in=5286457worker2 # [ 147.932490] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569806458worker2 # [ 147.944158] nix-daemon[858]: accepted connection from pid 1654, user nix-grpc-daemon (trusted)6459worker2 # [ 147.952034] nix-daemon[858]: reaped child process 1807, status = succeeded6460client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6461worker2 # [ 147.957570] nix-daemon[858]: accepted connection from pid 1654, user nix-grpc-daemon (trusted)6462client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6463client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6464client # farm-a> LOG-farm-a-during-drain6465worker2 # [ 149.194746] nix-daemon[858]: reaped child process 1816, status = succeeded6466worker2 # [ 149.274095] nix-daemon[858]: accepted connection from pid 1832, user nix-grpc-daemon (trusted)6467worker2 # [ 149.286787] nix-daemon[858]: reaped child process 1835, status = succeeded6468lb # [ 149.253655] niks3-server[1128]: 2026/09/23 16:01:59 INFO Received push request method=POST path=/api/pushes6469worker2 # [ 149.303248] nix-grpc-daemon[1734]: time=2026-09-23T16:01:59.307Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6470worker2 # [ 149.306037] nix-grpc-daemon[1734]: time=2026-09-23T16:01:59.309Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6471lb # [ 149.290396] niks3-server[1128]: 2026/09/23 16:01:59 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6472lb # [ 149.302452] niks3-server[1128]: 2026/09/23 16:01:59 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6473lb # [ 149.311170] niks3-server[1128]: 2026/09/23 16:01:59 INFO Received sign narinfos request method=POST path=/api/pushes/54/sign6474lb # [ 149.316362] niks3-server[1128]: 2026/09/23 16:01:59 INFO Signed narinfos id=54 count=16475worker2 # [ 149.356804] nix-grpc-daemon[1734]: time=2026-09-23T16:01:59.361Z level=INFO msg="Uploading 1 narinfos"6476lb # [ 149.325625] niks3-server[1128]: 2026/09/23 16:01:59 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6477lb # [ 149.334751] niks3-server[1128]: 2026/09/23 16:01:59 INFO Received complete push request method=POST path=/api/pushes/54/complete6478lb # [ 149.339334] niks3-server[1128]: 2026/09/23 16:01:59 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6479worker2 # [ 149.378620] nix-grpc-daemon[1734]: time=2026-09-23T16:01:59.383Z level=INFO msg="Upload complete. (213ms)"6480worker2 # [ 149.385980] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56980 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16481worker2 # [ 149.397763] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:59Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06482worker2 # [ 149.402244] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:59Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=16483worker2 # [ 149.407212] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569806484worker2 # [ 149.417318] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:569806485worker2 # [ 149.419970] nix-daemon[858]: reaped child process 1814, status = succeeded6486worker2 # [ 149.425868] nix-grpc-daemon[1734]: time=2026-09-23T16:01:59.430Z level=INFO msg="All 1 paths already cached"6487client # copying 1 paths to 'lb:50051'...6488worker2 # [ 149.431721] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:569806489client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6490worker2 # [ 149.447526] nix-daemon[858]: accepted connection from pid 1654, user nix-grpc-daemon (trusted)6491worker2 # [ 149.535213] nix-daemon[858]: accepted connection from pid 1841, user nix-grpc-daemon (trusted)6492worker2 # [ 149.549439] nix-daemon[858]: reaped child process 1844, status = succeeded6493lb # [ 149.515965] niks3-server[1128]: 2026/09/23 16:01:59 INFO Received push request method=POST path=/api/pushes6494worker2 # [ 149.565254] nix-grpc-daemon[1734]: time=2026-09-23T16:01:59.569Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6495worker2 # [ 149.567143] nix-grpc-daemon[1734]: time=2026-09-23T16:01:59.571Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6496lb # [ 149.547992] niks3-server[1128]: 2026/09/23 16:01:59 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6497lb # [ 149.554495] niks3-server[1128]: 2026/09/23 16:01:59 INFO Received sign narinfos request method=POST path=/api/pushes/55/sign6498lb # [ 149.561216] niks3-server[1128]: 2026/09/23 16:01:59 INFO Signed narinfos id=55 count=16499worker2 # [ 149.605733] nix-grpc-daemon[1734]: time=2026-09-23T16:01:59.610Z level=INFO msg="Uploading 1 narinfos"6500lb # [ 149.568931] niks3-server[1128]: 2026/09/23 16:01:59 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6501lb # [ 149.582771] niks3-server[1128]: 2026/09/23 16:01:59 INFO Received complete push request method=POST path=/api/pushes/55/complete6502lb # [ 149.594913] niks3-server[1128]: 2026/09/23 16:01:59 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6503worker2 # [ 149.633918] nix-grpc-daemon[1734]: time=2026-09-23T16:01:59.638Z level=INFO msg="Upload complete. (180ms)"6504worker2 # [ 149.638702] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56980 duration_s=0 paths=1 nar_bytes_in=6406505worker2 # [ 149.644266] nix-grpc-daemon[1654]: ts=2026-09-23T16:01:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569806506worker2 # [ 149.656129] nix-daemon[858]: accepted connection from pid 1654, user nix-grpc-daemon (trusted)6507worker2 # [ 149.663128] nix-daemon[858]: reaped child process 1839, status = succeeded6508worker2 # [ 149.666374] nix-daemon[858]: accepted connection from pid 1654, user nix-grpc-daemon (trusted)6509client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6510client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6511client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6512client # farm-b> LOG-farm-b-during-drain6513worker2 # [ 150.302985] nix-daemon[858]: reaped child process 1848, status = succeeded6514worker2 # [ 150.388219] nix-daemon[858]: accepted connection from pid 1864, user nix-grpc-daemon (trusted)6515worker2 # [ 150.402056] nix-daemon[858]: reaped child process 1867, status = succeeded6516lb # [ 150.369726] niks3-server[1128]: 2026/09/23 16:02:00 INFO Received push request method=POST path=/api/pushes6517worker2 # [ 150.419136] nix-grpc-daemon[1734]: time=2026-09-23T16:02:00.423Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6518worker2 # [ 150.420827] nix-grpc-daemon[1734]: time=2026-09-23T16:02:00.425Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6519lb # [ 150.413453] niks3-server[1128]: 2026/09/23 16:02:00 INFO Received sign narinfos request method=POST path=/api/pushes/56/sign6520lb # [ 150.424495] niks3-server[1128]: 2026/09/23 16:02:00 INFO Signed narinfos id=56 count=16521worker2 # [ 150.467711] nix-grpc-daemon[1734]: time=2026-09-23T16:02:00.472Z level=INFO msg="Uploading 1 narinfos"6522lb # [ 150.429578] niks3-server[1128]: 2026/09/23 16:02:00 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6523lb # [ 150.433976] niks3-server[1128]: 2026/09/23 16:02:00 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6524lb # [ 150.438975] niks3-server[1128]: 2026/09/23 16:02:00 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6525lb # [ 150.446862] niks3-server[1128]: 2026/09/23 16:02:00 INFO Received complete push request method=POST path=/api/pushes/56/complete6526lb # [ 150.453785] niks3-server[1128]: 2026/09/23 16:02:00 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6527worker2 # [ 150.493276] nix-grpc-daemon[1734]: time=2026-09-23T16:02:00.497Z level=INFO msg="Upload complete. (218ms)"6528worker2 # [ 150.500570] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56980 duration_s=0 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16529worker2 # [ 150.511606] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:00Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06530worker2 # [ 150.515872] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:00Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=16531worker2 # [ 150.520249] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569806532worker2 # [ 150.530377] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:569806533worker2 # [ 150.533366] nix-daemon[858]: reaped child process 1846, status = succeeded6534worker2 # [ 150.538147] nix-grpc-daemon[1734]: time=2026-09-23T16:02:00.542Z level=INFO msg="All 2 paths already cached"6535client # copying 1 paths to 'lb:50051'...6536worker2 # [ 150.544255] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:569806537client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6538worker2 # [ 150.562475] nix-daemon[858]: accepted connection from pid 1654, user nix-grpc-daemon (trusted)6539worker2 # [ 150.647563] nix-daemon[858]: accepted connection from pid 1873, user nix-grpc-daemon (trusted)6540worker2 # [ 150.662231] nix-daemon[858]: reaped child process 1876, status = succeeded6541lb # [ 150.627493] niks3-server[1128]: 2026/09/23 16:02:00 INFO Received push request method=POST path=/api/pushes6542worker2 # [ 150.676064] nix-grpc-daemon[1734]: time=2026-09-23T16:02:00.680Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6543worker2 # [ 150.677703] nix-grpc-daemon[1734]: time=2026-09-23T16:02:00.682Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6544lb # [ 150.657928] niks3-server[1128]: 2026/09/23 16:02:00 INFO Received sign narinfos request method=POST path=/api/pushes/57/sign6545worker2 # [ 150.705133] nix-grpc-daemon[1734]: time=2026-09-23T16:02:00.709Z level=INFO msg="Uploading 1 narinfos"6546lb # [ 150.664350] niks3-server[1128]: 2026/09/23 16:02:00 INFO Signed narinfos id=57 count=16547lb # [ 150.673763] niks3-server[1128]: 2026/09/23 16:02:00 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6548lb # [ 150.684607] niks3-server[1128]: 2026/09/23 16:02:00 INFO Received complete push request method=POST path=/api/pushes/57/complete6549worker2 # [ 150.736294] nix-grpc-daemon[1734]: time=2026-09-23T16:02:00.740Z level=INFO msg="Upload complete. (167ms)"6550lb # [ 150.696595] niks3-server[1128]: 2026/09/23 16:02:00 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6551lb # [ 150.700916] niks3-server[1128]: 2026/09/23 16:02:00 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6552worker2 # [ 150.739391] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56980 duration_s=0 paths=1 nar_bytes_in=7686553worker2 # [ 150.744224] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569806554worker2 # [ 150.753872] nix-daemon[858]: accepted connection from pid 1654, user nix-grpc-daemon (trusted)6555worker2 # [ 150.760219] nix-daemon[858]: reaped child process 1871, status = succeeded6556worker2 # [ 150.764186] nix-daemon[858]: accepted connection from pid 1654, user nix-grpc-daemon (trusted)6557client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6558client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6559client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6560client # farm-top> LOG-farm-top-during-drain6561worker2 # [ 151.782175] nix-daemon[858]: reaped child process 1880, status = succeeded6562worker2 # [ 151.856888] nix-daemon[858]: accepted connection from pid 1896, user nix-grpc-daemon (trusted)6563worker2 # [ 151.871296] nix-daemon[858]: reaped child process 1899, status = succeeded6564lb # [ 151.837913] niks3-server[1128]: 2026/09/23 16:02:01 INFO Received push request method=POST path=/api/pushes6565worker2 # [ 151.890126] nix-grpc-daemon[1734]: time=2026-09-23T16:02:01.894Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6566worker2 # [ 151.891618] nix-grpc-daemon[1734]: time=2026-09-23T16:02:01.896Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6567lb # [ 151.885190] niks3-server[1128]: 2026/09/23 16:02:01 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6568lb # [ 151.889609] niks3-server[1128]: 2026/09/23 16:02:01 INFO Received sign narinfos request method=POST path=/api/pushes/58/sign6569lb # [ 151.892634] niks3-server[1128]: 2026/09/23 16:02:01 INFO Signed narinfos id=58 count=16570worker2 # [ 151.934811] nix-grpc-daemon[1734]: time=2026-09-23T16:02:01.939Z level=INFO msg="Uploading 1 narinfos"6571lb # [ 151.897226] niks3-server[1128]: 2026/09/23 16:02:01 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6572lb # [ 151.900988] niks3-server[1128]: 2026/09/23 16:02:01 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6573lb # [ 151.904932] niks3-server[1128]: 2026/09/23 16:02:01 INFO Received complete push request method=POST path=/api/pushes/58/complete6574lb # [ 151.912399] niks3-server[1128]: 2026/09/23 16:02:01 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6575worker2 # [ 151.952488] nix-grpc-daemon[1734]: time=2026-09-23T16:02:01.957Z level=INFO msg="Upload complete. (181ms)"6576worker2 # [ 151.959704] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56980 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16577worker2 # [ 151.972084] nix-daemon[858]: reaped child process 1878, status = succeeded6578client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&2, in 4.50 seconds)6579client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath6580client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath, in 0.11 seconds)6581worker1: must succeed: pkill -f 'read -t [9]0 x'6582worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.04 seconds)6583client: waiting for success: ! systemctl is-active sw6584worker1 # [ 152.201878] nix-daemon[865]: reaped child process 1643, status = succeeded6585worker1 # [ 152.278866] nix-daemon[865]: accepted connection from pid 1684, user nix-grpc-daemon (trusted)6586worker1 # [ 152.291780] nix-daemon[865]: reaped child process 1687, status = succeeded6587lb # [ 152.283536] niks3-server[1128]: 2026/09/23 16:02:02 INFO Received push request method=POST path=/api/pushes6588worker1 # [ 152.308659] nix-grpc-daemon[1588]: time=2026-09-23T16:02:01.518Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6589worker1 # [ 152.310554] nix-grpc-daemon[1588]: time=2026-09-23T16:02:01.520Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6590lb # [ 152.320305] niks3-server[1128]: 2026/09/23 16:02:02 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6591lb # [ 152.332155] niks3-server[1128]: 2026/09/23 16:02:02 INFO Received sign narinfos request method=POST path=/api/pushes/59/sign6592lb # [ 152.341202] niks3-server[1128]: 2026/09/23 16:02:02 INFO Signed narinfos id=59 count=16593worker1 # [ 152.360990] nix-grpc-daemon[1588]: time=2026-09-23T16:02:01.571Z level=INFO msg="Uploading 1 narinfos"6594lb # [ 152.345196] niks3-server[1128]: 2026/09/23 16:02:02 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6595lb # [ 152.350456] niks3-server[1128]: 2026/09/23 16:02:02 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6596lb # [ 152.360678] niks3-server[1128]: 2026/09/23 16:02:02 INFO Received complete push request method=POST path=/api/pushes/59/complete6597lb # [ 152.367190] niks3-server[1128]: 2026/09/23 16:02:02 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6598worker1 # [ 152.386873] nix-grpc-daemon[1588]: time=2026-09-23T16:02:01.597Z level=INFO msg="Upload complete. (191ms)"6599worker1 # [ 152.393597] nix-grpc-daemon[1486]: ts=2026-09-23T16:02:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52334 duration_s=7 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16600worker1 # [ 152.408513] nix-daemon[865]: reaped child process 1641, status = succeeded6601client # [ 152.412575] systemd[1]: sw.service: Deactivated successfully.6602client # [ 152.417164] systemd[1]: sw.service: Consumed 121ms CPU time over 7.458s wall clock time, 43.4M memory peak, 7K incoming IP traffic, 8.3K outgoing IP traffic.6603worker1 # [ 152.838624] nix-grpc-daemon[1486]: ts=2026-09-23T16:02:02Z level=info event=signal_exit6604client: (finished: waiting for success: ! systemctl is-active sw, in 1.06 seconds)6605client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6606client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds)6607worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6608worker2 # [ 154.870374] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:04Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:569386609worker2 # [ 154.876961] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:04Z level=info event=worker_gone addr=192.168.1.3:500516610worker1 # [ 154.848385] nix-grpc-daemon[1486]: ts=2026-09-23T16:02:04Z level=info event=scheduler_restarting6611worker1 # [ 154.861539] nix-grpc-daemon[1486]: ts=2026-09-23T16:02:04Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6612worker1 # [ 154.881532] nix-daemon[865]: reaped child process 1554, status = succeeded6613lb # [ 156.825968] envoy[918]: conn peer=[2001:db8:1::1]:55690 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9564 tx=4348 ms=43346614lb # [ 156.829776] envoy[918]: conn peer=[2001:db8:1::1]:55676 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2958 tx=1498 ms=73066615lb # [ 156.833328] envoy[918]: conn peer=[2001:db8:1::3]:55914 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1148 tx=459 ms=239796616worker1 # [ 156.867547] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6617worker1 # [ 156.871600] systemd[1]: nix-grpc-daemon.service: Consumed 580ms CPU time over 26.150s wall clock time, 34.5M memory peak, 47.8K incoming IP traffic, 57.1K outgoing IP traffic.6618worker1 # [ 156.884825] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6619worker1 # [ 156.894773] systemd[1]: Starting Nix worker-protocol over gRPC...6620worker1 # [ 156.984158] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:06Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6621worker1 # [ 156.986129] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:06Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6622worker1 # [ 156.989315] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:06Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16623worker1 # [ 156.993571] nix-grpc-daemon[1702]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6624worker1 # [ 156.994913] nix-grpc-daemon[1702]: E0000 00:00:1790179326.203685 1702 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6625worker1 # [ 156.998772] nix-grpc-daemon[1702]: E0000 00:00:1790179326.208938 1702 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6626worker1 # [ 157.001689] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:06Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket6627worker1 # [ 157.005829] systemd[1]: Started Nix worker-protocol over gRPC.6628worker1 # [ 157.011092] nix-grpc-daemon[1702]: E0000 00:00:1790179326.220989 1744 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6629worker1 # [ 157.025078] nix-grpc-daemon[1702]: E0000 00:00:1790179326.234847 1744 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6630worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 3.81 seconds)6631lb: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux6632lb: must succeed: curl -sf localhost:9901/clusters6633worker1 # [ 157.029127] nix-grpc-daemon[1702]: E0000 00:00:1790179326.234958 1744 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6634worker1 # [ 157.038833] nix-grpc-daemon[1702]: E0000 00:00:1790179326.249115 1744 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6635worker1 # [ 157.050336] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:06Z level=info event=scheduler_connected addr=lb:500516636worker2 # [ 157.077849] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:07Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:569386637worker2 # [ 157.082296] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:07Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06638lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6639lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux, in 0.06 seconds)6640lb: must succeed: curl -sf localhost:9901/clusters6641lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6642worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6643worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6644(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 12.23 seconds)6645subtest: queue gauges show a burst while every build blocks6646lb: must succeed: curl -sf localhost:9901/clusters6647lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6648lb: must succeed: curl -sf localhost:9901/clusters6649lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6650worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6651worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6652client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]'6653client # [ 157.279997] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]".6654client # Running as unit: burst.service; invocation ID: d8d6c6b137894f6caa574e5ef9b0cb966655client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.06 seconds)6656lb: must succeed: curl -sf localhost:9901/clusters6657lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6658lb: must succeed: curl -sf localhost:9901/clusters6659lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6660worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6661worker1 # [ 157.429755] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:377906662worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6663worker1 # [ 157.441455] nix-daemon[865]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6664client # [ 157.440778] nix[1512]: don't know how to build these paths:6665client # [ 157.441659] nix[1512]: /nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv6666client # [ 157.442585] nix[1512]: /nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv6667client # [ 157.444108] nix[1512]: /nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv6668client # [ 157.445084] nix[1512]: /nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv6669worker2 # [ 157.491282] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:07Z level=debug event=want client=5 drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv cached=06670worker2 # [ 157.495270] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:07Z level=debug event=assigned drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv worker=192.168.1.4:50051 clients=16671worker1 # [ 157.478327] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377906672worker2 # [ 157.501460] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:07Z level=debug event=want client=5 drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv cached=06673worker2 # [ 157.507376] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:07Z level=debug event=assigned drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv worker=192.168.1.3:50051 clients=16674worker2 # [ 157.512123] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:07Z level=debug event=want client=5 drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv cached=06675worker2 # [ 157.515917] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:07Z level=debug event=want client=5 drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv cached=06676worker1 # [ 157.489510] nix-grpc-daemon[1702]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 25 ms (attempt 1/5)6677worker2 # [ 157.519362] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569806678worker2 # [ 157.529226] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:569806679client # [ 157.500229] nix[1512]: copying 1 paths to 'lb:50051'...6680worker2 # [ 157.538749] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:569806681client # [ 157.502368] nix[1512]: copying path '/nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv' to 'lb:50051'...6682worker2 # [ 157.547913] nix-daemon[858]: accepted connection from pid 1654, user nix-grpc-daemon (trusted)6683worker1 # [ 157.521706] nix-grpc-daemon[1702]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 137 ms (attempt 2/5)6684worker2 # [ 157.638858] nix-daemon[858]: accepted connection from pid 1914, user nix-grpc-daemon (trusted)6685worker2 # [ 157.653590] nix-daemon[858]: reaped child process 1917, status = succeeded6686lb # [ 157.618771] niks3-server[1128]: 2026/09/23 16:02:07 INFO Received push request method=POST path=/api/pushes6687worker2 # [ 157.667971] nix-grpc-daemon[1734]: time=2026-09-23T16:02:07.672Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6688worker2 # [ 157.669464] nix-grpc-daemon[1734]: time=2026-09-23T16:02:07.672Z level=INFO msg="Uploading 2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv (416B)"6689worker1 # [ 157.660936] nix-grpc-daemon[1702]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 209 ms (attempt 3/5)6690lb # [ 157.648666] niks3-server[1128]: 2026/09/23 16:02:07 INFO Received sign narinfos request method=POST path=/api/pushes/60/sign6691worker2 # [ 157.694991] nix-grpc-daemon[1734]: time=2026-09-23T16:02:07.699Z level=INFO msg="Uploading 1 narinfos"6692lb # [ 157.654125] niks3-server[1128]: 2026/09/23 16:02:07 INFO Signed narinfos id=60 count=16693lb # [ 157.664066] niks3-server[1128]: 2026/09/23 16:02:07 INFO Registered completed upload object_key=2s9psgqm7awsyhlfyaclma7yr7qmrk0g.ls6694lb # [ 157.674755] niks3-server[1128]: 2026/09/23 16:02:07 INFO Registered completed upload object_key=nar/0raymp7y3ngiv7lj2qq6nkp63sms2510yhhlnr7hnz7ldnl7b3qc.nar.zst6695lb # [ 157.686050] niks3-server[1128]: 2026/09/23 16:02:07 INFO Received complete push request method=POST path=/api/pushes/60/complete6696lb # [ 157.692786] niks3-server[1128]: 2026/09/23 16:02:07 INFO Registered completed upload object_key=2s9psgqm7awsyhlfyaclma7yr7qmrk0g.narinfo6697worker2 # [ 157.732729] nix-grpc-daemon[1734]: time=2026-09-23T16:02:07.737Z level=INFO msg="Upload complete. (174ms)"6698worker2 # [ 157.737933] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56980 duration_s=0 paths=1 nar_bytes_in=4166699worker2 # [ 157.743729] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569806700worker2 # [ 157.756120] nix-daemon[858]: accepted connection from pid 1654, user nix-grpc-daemon (trusted)6701worker2 # [ 157.760804] nix-daemon[858]: reaped child process 1912, status = succeeded6702worker2 # [ 157.765744] nix-daemon[858]: accepted connection from pid 1654, user nix-grpc-daemon (trusted)6703client # [ 157.727920] nix[1512]: building '/nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv' on lb:50051...6704worker1 # [ 157.872262] nix-grpc-daemon[1702]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 524 ms (attempt 4/5)6705worker1 # [ 158.400428] nix-grpc-daemon[1702]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6706worker1 # [ 158.404067] nix-grpc-daemon[1702]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6707worker1 # [ 158.423439] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:377906708lb: must succeed: curl -sf localhost:9901/clusters6709client # [ 158.421293] nix[1512]: copying 1 paths to 'lb:50051'...6710worker1 # [ 158.435314] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:377906711client # [ 158.425180] nix[1512]: copying path '/nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv' to 'lb:50051'...6712worker1 # [ 158.447204] nix-daemon[865]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6713worker1 # [ 158.467491] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:07Z level=info event=niks3_push_started pid=17626714lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6715lb: must succeed: curl -sf localhost:9901/clusters6716lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6717worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6718worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6719worker1 # [ 158.575522] nix-grpc-daemon[1762]: time=2026-09-23T16:02:07.785Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6720worker1 # [ 158.671961] nix-daemon[865]: accepted connection from pid 1778, user nix-grpc-daemon (trusted)6721worker1 # [ 158.683290] nix-daemon[865]: reaped child process 1781, status = succeeded6722lb # [ 158.678988] niks3-server[1128]: 2026/09/23 16:02:08 INFO Received push request method=POST path=/api/pushes6723worker1 # [ 158.703092] nix-grpc-daemon[1762]: time=2026-09-23T16:02:07.913Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6724worker1 # [ 158.704614] nix-grpc-daemon[1762]: time=2026-09-23T16:02:07.914Z level=INFO msg="Uploading 885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv (416B)"6725lb # [ 158.715148] niks3-server[1128]: 2026/09/23 16:02:08 INFO Registered completed upload object_key=885n55y9srkrjs4p487lr255wvhla7hq.ls6726lb # [ 158.719597] niks3-server[1128]: 2026/09/23 16:02:08 INFO Received sign narinfos request method=POST path=/api/pushes/61/sign6727lb # [ 158.725357] niks3-server[1128]: 2026/09/23 16:02:08 INFO Signed narinfos id=61 count=16728worker1 # [ 158.743509] nix-grpc-daemon[1762]: time=2026-09-23T16:02:07.953Z level=INFO msg="Uploading 1 narinfos"6729lb # [ 158.733509] niks3-server[1128]: 2026/09/23 16:02:08 INFO Registered completed upload object_key=nar/0mmcg3cvna05ngbx6v5vspkmc5gj8lna2zc48rllhxr0g8i9mg6y.nar.zst6730lb # [ 158.743386] niks3-server[1128]: 2026/09/23 16:02:08 INFO Received complete push request method=POST path=/api/pushes/61/complete6731worker1 # [ 158.763203] nix-grpc-daemon[1762]: time=2026-09-23T16:02:07.973Z level=INFO msg="Upload complete. (184ms)"6732lb # [ 158.748073] niks3-server[1128]: 2026/09/23 16:02:08 INFO Registered completed upload object_key=885n55y9srkrjs4p487lr255wvhla7hq.narinfo6733worker1 # [ 158.765504] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37790 duration_s=0 paths=1 nar_bytes_in=4166734worker1 # [ 158.768518] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377906735worker1 # [ 158.776391] nix-daemon[865]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6736worker1 # [ 158.778366] nix-daemon[865]: reaped child process 1760, status = succeeded6737worker1 # [ 158.783459] nix-daemon[865]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6738client # [ 158.770851] nix[1512]: building '/nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv' on lb:50051...6739worker2 # [ 160.006601] nix-daemon[858]: reaped child process 1921, status = succeeded6740worker2 # [ 160.084092] nix-daemon[858]: accepted connection from pid 1949, user nix-grpc-daemon (trusted)6741worker2 # [ 160.096920] nix-daemon[858]: reaped child process 1952, status = succeeded6742lb # [ 160.064673] niks3-server[1128]: 2026/09/23 16:02:10 INFO Received push request method=POST path=/api/pushes6743worker2 # [ 160.112459] nix-grpc-daemon[1734]: time=2026-09-23T16:02:10.116Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6744worker2 # [ 160.114406] nix-grpc-daemon[1734]: time=2026-09-23T16:02:10.118Z level=INFO msg="Uploading vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq-slow-q3 (120B)"6745lb # [ 160.109132] niks3-server[1128]: 2026/09/23 16:02:10 INFO Received sign narinfos request method=POST path=/api/pushes/62/sign6746lb # [ 160.114128] niks3-server[1128]: 2026/09/23 16:02:10 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst6747worker2 # [ 160.159820] nix-grpc-daemon[1734]: time=2026-09-23T16:02:10.164Z level=INFO msg="Uploading 1 narinfos"6748lb # [ 160.119204] niks3-server[1128]: 2026/09/23 16:02:10 INFO Signed narinfos id=62 count=16749lb # [ 160.125779] niks3-server[1128]: 2026/09/23 16:02:10 INFO Registered completed upload object_key=log/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv6750lb # [ 160.130961] niks3-server[1128]: 2026/09/23 16:02:10 INFO Registered completed upload object_key=vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq.ls6751lb # [ 160.139534] niks3-server[1128]: 2026/09/23 16:02:10 INFO Received complete push request method=POST path=/api/pushes/62/complete6752worker2 # [ 160.184137] nix-grpc-daemon[1734]: time=2026-09-23T16:02:10.188Z level=INFO msg="Upload complete. (209ms)"6753lb # [ 160.144708] niks3-server[1128]: 2026/09/23 16:02:10 INFO Registered completed upload object_key=vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq.narinfo6754worker2 # [ 160.191422] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56980 duration_s=2 drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv assign_id=7 outputs=16755worker2 # [ 160.197275] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:10Z level=debug event=assigned drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv worker=192.168.1.4:50051 clients=16756worker2 # [ 160.201566] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569806757worker2 # [ 160.210353] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:569806758worker2 # [ 160.213051] nix-daemon[858]: reaped child process 1919, status = succeeded6759client # [ 160.178796] nix[1512]: copying 1 paths to 'lb:50051'...6760worker2 # [ 160.218914] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:569806761client # [ 160.181724] nix[1512]: copying path '/nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv' to 'lb:50051'...6762worker2 # [ 160.225400] nix-daemon[858]: accepted connection from pid 1654, user nix-grpc-daemon (trusted)6763worker2 # [ 160.310168] nix-daemon[858]: accepted connection from pid 1956, user nix-grpc-daemon (trusted)6764worker2 # [ 160.324767] nix-daemon[858]: reaped child process 1959, status = succeeded6765lb # [ 160.289157] niks3-server[1128]: 2026/09/23 16:02:10 INFO Received push request method=POST path=/api/pushes6766worker2 # [ 160.338332] nix-grpc-daemon[1734]: time=2026-09-23T16:02:10.343Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6767worker2 # [ 160.340157] nix-grpc-daemon[1734]: time=2026-09-23T16:02:10.344Z level=INFO msg="Uploading 8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv (416B)"6768lb # [ 160.317174] niks3-server[1128]: 2026/09/23 16:02:10 INFO Received sign narinfos request method=POST path=/api/pushes/63/sign6769worker2 # [ 160.362477] nix-grpc-daemon[1734]: time=2026-09-23T16:02:10.367Z level=INFO msg="Uploading 1 narinfos"6770lb # [ 160.322162] niks3-server[1128]: 2026/09/23 16:02:10 INFO Signed narinfos id=63 count=16771lb # [ 160.335416] niks3-server[1128]: 2026/09/23 16:02:10 INFO Registered completed upload object_key=8v867csm7d9sip4xd4nlg9jwvrpqqwv9.ls6772lb # [ 160.339877] niks3-server[1128]: 2026/09/23 16:02:10 INFO Registered completed upload object_key=nar/1xibay372y9by9a678y9h7bfzwahsxi0vq51acgywak31q6q2m87.nar.zst6773lb # [ 160.349691] niks3-server[1128]: 2026/09/23 16:02:10 INFO Received complete push request method=POST path=/api/pushes/63/complete6774worker2 # [ 160.397858] nix-grpc-daemon[1734]: time=2026-09-23T16:02:10.402Z level=INFO msg="Upload complete. (166ms)"6775lb # [ 160.357955] niks3-server[1128]: 2026/09/23 16:02:10 INFO Registered completed upload object_key=8v867csm7d9sip4xd4nlg9jwvrpqqwv9.narinfo6776worker2 # [ 160.400503] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56980 duration_s=0 paths=1 nar_bytes_in=4166777worker2 # [ 160.403171] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569806778worker2 # [ 160.409789] nix-daemon[858]: accepted connection from pid 1654, user nix-grpc-daemon (trusted)6779worker2 # [ 160.417707] nix-daemon[858]: accepted connection from pid 1654, user nix-grpc-daemon (trusted)6780client # [ 160.379706] nix[1512]: building '/nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv' on lb:50051...6781worker2 # [ 160.419909] nix-daemon[858]: reaped child process 1954, status = succeeded6782worker1 # [ 161.015631] nix-daemon[865]: reaped child process 1785, status = succeeded6783worker1 # [ 161.092957] nix-daemon[865]: accepted connection from pid 1810, user nix-grpc-daemon (trusted)6784worker1 # [ 161.105869] nix-daemon[865]: reaped child process 1813, status = succeeded6785lb # [ 161.100706] niks3-server[1128]: 2026/09/23 16:02:11 INFO Received push request method=POST path=/api/pushes6786worker1 # [ 161.125178] nix-grpc-daemon[1762]: time=2026-09-23T16:02:10.335Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6787worker1 # [ 161.127431] nix-grpc-daemon[1762]: time=2026-09-23T16:02:10.336Z level=INFO msg="Uploading nv0kp5b571hsjibnd9v06981i024iima-slow-q2 (120B)"6788lb # [ 161.144985] niks3-server[1128]: 2026/09/23 16:02:11 INFO Received sign narinfos request method=POST path=/api/pushes/64/sign6789lb # [ 161.150653] niks3-server[1128]: 2026/09/23 16:02:11 INFO Signed narinfos id=64 count=16790worker1 # [ 161.169313] nix-grpc-daemon[1762]: time=2026-09-23T16:02:10.379Z level=INFO msg="Uploading 1 narinfos"6791lb # [ 161.155598] niks3-server[1128]: 2026/09/23 16:02:11 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst6792lb # [ 161.160881] niks3-server[1128]: 2026/09/23 16:02:11 INFO Registered completed upload object_key=nv0kp5b571hsjibnd9v06981i024iima.ls6793lb # [ 161.169491] niks3-server[1128]: 2026/09/23 16:02:11 INFO Received complete push request method=POST path=/api/pushes/64/complete6794lb # [ 161.175877] niks3-server[1128]: 2026/09/23 16:02:11 INFO Registered completed upload object_key=nv0kp5b571hsjibnd9v06981i024iima.narinfo6795worker1 # [ 161.191055] nix-grpc-daemon[1762]: time=2026-09-23T16:02:10.401Z level=INFO msg="Upload complete. (199ms)"6796lb # [ 161.178375] niks3-server[1128]: 2026/09/23 16:02:11 INFO Registered completed upload object_key=log/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv6797worker1 # [ 161.195931] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37790 duration_s=2 drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv assign_id=8 outputs=16798worker2 # [ 161.222926] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:11Z level=debug event=assigned drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv worker=192.168.1.3:50051 clients=16799worker1 # [ 161.205773] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377906800worker1 # [ 161.211050] nix-daemon[865]: reaped child process 1783, status = succeeded6801worker1 # [ 161.214740] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:377906802worker1 # [ 161.220324] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:377906803client # [ 161.207857] nix[1512]: copying 1 paths to 'lb:50051'...6804worker1 # [ 161.225169] nix-daemon[865]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6805client # [ 161.210589] nix[1512]: copying path '/nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv' to 'lb:50051'...6806worker1 # [ 161.311497] nix-daemon[865]: accepted connection from pid 1817, user nix-grpc-daemon (trusted)6807worker1 # [ 161.325426] nix-daemon[865]: reaped child process 1820, status = succeeded6808lb # [ 161.315382] niks3-server[1128]: 2026/09/23 16:02:11 INFO Received push request method=POST path=/api/pushes6809worker1 # [ 161.339163] nix-grpc-daemon[1762]: time=2026-09-23T16:02:10.549Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6810worker1 # [ 161.341090] nix-grpc-daemon[1762]: time=2026-09-23T16:02:10.550Z level=INFO msg="Uploading qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv (416B)"6811lb # [ 161.346861] niks3-server[1128]: 2026/09/23 16:02:11 INFO Received sign narinfos request method=POST path=/api/pushes/65/sign6812lb # [ 161.353086] niks3-server[1128]: 2026/09/23 16:02:11 INFO Signed narinfos id=65 count=16813worker1 # [ 161.372569] nix-grpc-daemon[1762]: time=2026-09-23T16:02:10.582Z level=INFO msg="Uploading 1 narinfos"6814lb # [ 161.357970] niks3-server[1128]: 2026/09/23 16:02:11 INFO Registered completed upload object_key=qs0cgdr560r5cllf2djj9bb4gz0vrp4p.ls6815lb # [ 161.363517] niks3-server[1128]: 2026/09/23 16:02:11 INFO Registered completed upload object_key=nar/1spxkiwzvlj9sgbjal4jf4p68h5kbylxymfww73s5va157qxgafi.nar.zst6816lb # [ 161.376447] niks3-server[1128]: 2026/09/23 16:02:11 INFO Received complete push request method=POST path=/api/pushes/65/complete6817worker1 # [ 161.402460] nix-grpc-daemon[1762]: time=2026-09-23T16:02:10.612Z level=INFO msg="Upload complete. (171ms)"6818lb # [ 161.387805] niks3-server[1128]: 2026/09/23 16:02:11 INFO Registered completed upload object_key=qs0cgdr560r5cllf2djj9bb4gz0vrp4p.narinfo6819worker1 # [ 161.404573] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37790 duration_s=0 paths=1 nar_bytes_in=4166820worker1 # [ 161.409462] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377906821worker1 # [ 161.415260] nix-daemon[865]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6822worker1 # [ 161.417528] nix-daemon[865]: reaped child process 1815, status = succeeded6823worker1 # [ 161.422635] nix-daemon[865]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6824client # [ 161.409475] nix[1512]: building '/nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv' on lb:50051...6825worker2 # [ 162.269214] nix-daemon[858]: reaped child process 1963, status = succeeded6826worker2 # [ 162.333942] nix-daemon[858]: accepted connection from pid 1988, user nix-grpc-daemon (trusted)6827worker2 # [ 162.348162] nix-daemon[858]: reaped child process 1991, status = succeeded6828lb # [ 162.313885] niks3-server[1128]: 2026/09/23 16:02:12 INFO Received push request method=POST path=/api/pushes6829worker2 # [ 162.363120] nix-grpc-daemon[1734]: time=2026-09-23T16:02:12.367Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6830worker2 # [ 162.364556] nix-grpc-daemon[1734]: time=2026-09-23T16:02:12.369Z level=INFO msg="Uploading wmw8972gq2nf739xhg7is7lylxf8299y-slow-q0 (120B)"6831lb # [ 162.358699] niks3-server[1128]: 2026/09/23 16:02:12 INFO Received sign narinfos request method=POST path=/api/pushes/66/sign6832lb # [ 162.367555] niks3-server[1128]: 2026/09/23 16:02:12 INFO Registered completed upload object_key=log/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv6833worker2 # [ 162.417735] nix-grpc-daemon[1734]: time=2026-09-23T16:02:12.422Z level=INFO msg="Uploading 1 narinfos"6834lb # [ 162.376531] niks3-server[1128]: 2026/09/23 16:02:12 INFO Signed narinfos id=66 count=16835lb # [ 162.380469] niks3-server[1128]: 2026/09/23 16:02:12 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst6836lb # [ 162.384411] niks3-server[1128]: 2026/09/23 16:02:12 INFO Registered completed upload object_key=wmw8972gq2nf739xhg7is7lylxf8299y.ls6837lb # [ 162.394893] niks3-server[1128]: 2026/09/23 16:02:12 INFO Received complete push request method=POST path=/api/pushes/66/complete6838lb # [ 162.403770] niks3-server[1128]: 2026/09/23 16:02:12 INFO Registered completed upload object_key=wmw8972gq2nf739xhg7is7lylxf8299y.narinfo6839worker2 # [ 162.443879] nix-grpc-daemon[1734]: time=2026-09-23T16:02:12.448Z level=INFO msg="Upload complete. (213ms)"6840worker2 # [ 162.449903] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56980 duration_s=2 drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv assign_id=9 outputs=16841worker2 # [ 162.461993] nix-daemon[858]: reaped child process 1961, status = succeeded6842worker1 # [ 163.261844] nix-daemon[865]: reaped child process 1824, status = succeeded6843worker1 # [ 163.333826] nix-daemon[865]: accepted connection from pid 1849, user nix-grpc-daemon (trusted)6844worker1 # [ 163.345134] nix-daemon[865]: reaped child process 1852, status = succeeded6845lb # [ 163.338437] niks3-server[1128]: 2026/09/23 16:02:13 INFO Received push request method=POST path=/api/pushes6846worker1 # [ 163.362980] nix-grpc-daemon[1762]: time=2026-09-23T16:02:12.573Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6847worker1 # [ 163.364885] nix-grpc-daemon[1762]: time=2026-09-23T16:02:12.574Z level=INFO msg="Uploading 106i40yblbkazayvn9ppswrq6kqs26sj-slow-q1 (120B)"6848lb # [ 163.381703] niks3-server[1128]: 2026/09/23 16:02:13 INFO Received sign narinfos request method=POST path=/api/pushes/67/sign6849worker1 # [ 163.403479] nix-grpc-daemon[1762]: time=2026-09-23T16:02:12.613Z level=INFO msg="Uploading 1 narinfos"6850lb # [ 163.387283] niks3-server[1128]: 2026/09/23 16:02:13 INFO Signed narinfos id=67 count=16851lb # [ 163.395378] niks3-server[1128]: 2026/09/23 16:02:13 INFO Registered completed upload object_key=log/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv6852lb # [ 163.401522] niks3-server[1128]: 2026/09/23 16:02:13 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst6853lb # [ 163.407072] niks3-server[1128]: 2026/09/23 16:02:13 INFO Registered completed upload object_key=106i40yblbkazayvn9ppswrq6kqs26sj.ls6854lb # [ 163.416132] niks3-server[1128]: 2026/09/23 16:02:13 INFO Received complete push request method=POST path=/api/pushes/67/complete6855worker1 # [ 163.440120] nix-grpc-daemon[1762]: time=2026-09-23T16:02:12.650Z level=INFO msg="Upload complete. (209ms)"6856lb # [ 163.425940] niks3-server[1128]: 2026/09/23 16:02:13 INFO Registered completed upload object_key=106i40yblbkazayvn9ppswrq6kqs26sj.narinfo6857worker1 # [ 163.446873] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37790 duration_s=2 drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv assign_id=10 outputs=16858worker1 # [ 163.460940] nix-daemon[865]: reaped child process 1822, status = succeeded6859client # [ 163.465279] systemd[1]: burst.service: Deactivated successfully.6860client # [ 163.468301] systemd[1]: burst.service: Consumed 160ms CPU time over 6.184s wall clock time, 51.9M memory peak, 14K incoming IP traffic, 19K outgoing IP traffic.6861client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }6862client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.04 seconds)6863lb: must succeed: curl -sf localhost:9901/clusters6864lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6865lb: must succeed: curl -sf localhost:9901/clusters6866lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6867worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6868worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6869(finished: subtest: queue gauges show a burst while every build blocks, in 7.37 seconds)6870subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6871client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16872worker1 # [ 164.660912] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:377906873worker1 # [ 164.667265] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:377906874worker2 # [ 164.734939] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:14Z level=debug event=want client=6 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06875worker2 # [ 164.739698] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:14Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16876worker1 # [ 164.717178] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377906877worker2 # [ 164.744166] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:14Z level=debug event=want client=6 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06878worker2 # [ 164.748093] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:14Z level=debug event=want client=6 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06879worker1 # [ 164.729911] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:377906880worker1 # [ 164.741720] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:377906881worker1 # [ 164.750700] nix-daemon[865]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6882worker1 # [ 164.855984] nix-daemon[865]: accepted connection from pid 1861, user nix-grpc-daemon (trusted)6883worker1 # [ 164.868972] nix-daemon[865]: reaped child process 1864, status = succeeded6884lb # [ 164.861788] niks3-server[1128]: 2026/09/23 16:02:14 INFO Received push request method=POST path=/api/pushes6885worker1 # [ 164.887251] nix-grpc-daemon[1762]: time=2026-09-23T16:02:14.097Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6886worker1 # [ 164.889292] nix-grpc-daemon[1762]: time=2026-09-23T16:02:14.099Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6887lb # [ 164.896921] niks3-server[1128]: 2026/09/23 16:02:14 INFO Received sign narinfos request method=POST path=/api/pushes/68/sign6888worker1 # [ 164.917080] nix-grpc-daemon[1762]: time=2026-09-23T16:02:14.127Z level=INFO msg="Uploading 1 narinfos"6889lb # [ 164.900753] niks3-server[1128]: 2026/09/23 16:02:14 INFO Signed narinfos id=68 count=16890lb # [ 164.909492] niks3-server[1128]: 2026/09/23 16:02:14 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6891lb # [ 164.916510] niks3-server[1128]: 2026/09/23 16:02:14 INFO Received complete push request method=POST path=/api/pushes/68/complete6892worker1 # [ 164.939279] nix-grpc-daemon[1762]: time=2026-09-23T16:02:14.149Z level=INFO msg="Upload complete. (174ms)"6893lb # [ 164.924890] niks3-server[1128]: 2026/09/23 16:02:15 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6894worker1 # [ 164.941406] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37790 duration_s=0 paths=1 nar_bytes_in=5046895lb # [ 164.927843] niks3-server[1128]: 2026/09/23 16:02:15 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6896worker1 # [ 164.946596] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377906897worker1 # [ 164.953072] nix-daemon[865]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6898worker1 # [ 164.954389] nix-daemon[865]: reaped child process 1858, status = succeeded6899worker1 # [ 164.959550] nix-daemon[865]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6900worker1 # [ 166.175885] nix-daemon[865]: reaped child process 1868, status = succeeded6901worker1 # [ 166.252659] nix-daemon[865]: accepted connection from pid 1884, user nix-grpc-daemon (trusted)6902worker1 # [ 166.264346] nix-daemon[865]: reaped child process 1887, status = succeeded6903lb # [ 166.256599] niks3-server[1128]: 2026/09/23 16:02:16 INFO Received push request method=POST path=/api/pushes6904worker1 # [ 166.280780] nix-grpc-daemon[1762]: time=2026-09-23T16:02:15.490Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6905worker1 # [ 166.282440] nix-grpc-daemon[1762]: time=2026-09-23T16:02:15.492Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6906lb # [ 166.302319] niks3-server[1128]: 2026/09/23 16:02:16 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6907lb # [ 166.307274] niks3-server[1128]: 2026/09/23 16:02:16 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6908lb # [ 166.311672] niks3-server[1128]: 2026/09/23 16:02:16 INFO Received sign narinfos request method=POST path=/api/pushes/69/sign6909lb # [ 166.317351] niks3-server[1128]: 2026/09/23 16:02:16 INFO Signed narinfos id=69 count=16910worker1 # [ 166.332962] nix-grpc-daemon[1762]: time=2026-09-23T16:02:15.543Z level=INFO msg="Uploading 1 narinfos"6911lb # [ 166.323686] niks3-server[1128]: 2026/09/23 16:02:16 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6912lb # [ 166.332578] niks3-server[1128]: 2026/09/23 16:02:16 INFO Received complete push request method=POST path=/api/pushes/69/complete6913lb # [ 166.339295] niks3-server[1128]: 2026/09/23 16:02:16 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6914worker1 # [ 166.353573] nix-grpc-daemon[1762]: time=2026-09-23T16:02:15.563Z level=INFO msg="Upload complete. (186ms)"6915worker1 # [ 166.360089] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37790 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=11 outputs=16916worker2 # [ 166.389586] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:16Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16917worker1 # [ 166.370335] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377906918worker2 # [ 166.395465] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:16Z level=debug event=want client=6 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06919worker1 # [ 166.379703] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:377906920worker1 # [ 166.387725] nix-daemon[865]: reaped child process 1866, status = succeeded6921worker1 # [ 166.390064] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:377906922worker1 # [ 166.393914] nix-daemon[865]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6923worker1 # [ 166.480541] nix-daemon[865]: accepted connection from pid 1891, user nix-grpc-daemon (trusted)6924worker1 # [ 166.494323] nix-daemon[865]: reaped child process 1894, status = succeeded6925lb # [ 166.484532] niks3-server[1128]: 2026/09/23 16:02:16 INFO Received push request method=POST path=/api/pushes6926worker1 # [ 166.508056] nix-grpc-daemon[1762]: time=2026-09-23T16:02:15.718Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6927worker1 # [ 166.510067] nix-grpc-daemon[1762]: time=2026-09-23T16:02:15.719Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6928lb # [ 166.511979] niks3-server[1128]: 2026/09/23 16:02:16 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6929lb # [ 166.519644] niks3-server[1128]: 2026/09/23 16:02:16 INFO Received sign narinfos request method=POST path=/api/pushes/70/sign6930lb # [ 166.525207] niks3-server[1128]: 2026/09/23 16:02:16 INFO Signed narinfos id=70 count=16931worker1 # [ 166.545062] nix-grpc-daemon[1762]: time=2026-09-23T16:02:15.755Z level=INFO msg="Uploading 1 narinfos"6932lb # [ 166.538181] niks3-server[1128]: 2026/09/23 16:02:16 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6933lb # [ 166.544708] niks3-server[1128]: 2026/09/23 16:02:16 INFO Received complete push request method=POST path=/api/pushes/70/complete6934lb # [ 166.555948] niks3-server[1128]: 2026/09/23 16:02:16 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6935worker1 # [ 166.571153] nix-grpc-daemon[1762]: time=2026-09-23T16:02:15.781Z level=INFO msg="Upload complete. (171ms)"6936worker1 # [ 166.575096] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37790 duration_s=0 paths=1 nar_bytes_in=5046937worker1 # [ 166.580247] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377906938worker1 # [ 166.589649] nix-daemon[865]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6939worker1 # [ 166.595130] nix-daemon[865]: reaped child process 1889, status = succeeded6940worker1 # [ 166.599540] nix-daemon[865]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6941lb # [ 166.827343] envoy[918]: conn peer=[2001:db8:1::1]:60992 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9647 tx=5187 ms=60316942worker1 # [ 167.640596] nix-daemon[865]: reaped child process 1898, status = succeeded6943worker1 # [ 167.723334] nix-daemon[865]: accepted connection from pid 1914, user nix-grpc-daemon (trusted)6944worker1 # [ 167.736084] nix-daemon[865]: reaped child process 1917, status = succeeded6945lb # [ 167.728500] niks3-server[1128]: 2026/09/23 16:02:17 INFO Received push request method=POST path=/api/pushes6946worker1 # [ 167.755409] nix-grpc-daemon[1762]: time=2026-09-23T16:02:16.965Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6947worker1 # [ 167.756941] nix-grpc-daemon[1762]: time=2026-09-23T16:02:16.967Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6948lb # [ 167.778864] niks3-server[1128]: 2026/09/23 16:02:17 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6949lb # [ 167.782759] niks3-server[1128]: 2026/09/23 16:02:17 INFO Received sign narinfos request method=POST path=/api/pushes/71/sign6950worker1 # [ 167.802998] nix-grpc-daemon[1762]: time=2026-09-23T16:02:17.013Z level=INFO msg="Uploading 1 narinfos"6951lb # [ 167.787417] niks3-server[1128]: 2026/09/23 16:02:17 INFO Signed narinfos id=71 count=16952lb # [ 167.792520] niks3-server[1128]: 2026/09/23 16:02:17 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6953lb # [ 167.804499] niks3-server[1128]: 2026/09/23 16:02:17 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6954lb # [ 167.811250] niks3-server[1128]: 2026/09/23 16:02:17 INFO Received complete push request method=POST path=/api/pushes/71/complete6955lb # [ 167.820587] niks3-server[1128]: 2026/09/23 16:02:17 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6956worker1 # [ 167.835553] nix-grpc-daemon[1762]: time=2026-09-23T16:02:17.045Z level=INFO msg="Upload complete. (228ms)"6957worker1 # [ 167.841603] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37790 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=12 outputs=16958worker2 # [ 167.871212] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:17Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16959worker1 # [ 167.850358] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377906960worker2 # [ 167.877094] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:17Z level=debug event=want client=6 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06961worker1 # [ 167.862056] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:377906962worker1 # [ 167.870357] nix-daemon[865]: reaped child process 1896, status = succeeded6963worker1 # [ 167.873145] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:377906964worker1 # [ 167.881070] nix-daemon[865]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6965worker1 # [ 167.965228] nix-daemon[865]: accepted connection from pid 1921, user nix-grpc-daemon (trusted)6966worker1 # [ 167.978377] nix-daemon[865]: reaped child process 1924, status = succeeded6967lb # [ 167.969221] niks3-server[1128]: 2026/09/23 16:02:18 INFO Received push request method=POST path=/api/pushes6968worker1 # [ 167.993351] nix-grpc-daemon[1762]: time=2026-09-23T16:02:17.203Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6969worker1 # [ 167.994843] nix-grpc-daemon[1762]: time=2026-09-23T16:02:17.205Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6970lb # [ 167.996272] niks3-server[1128]: 2026/09/23 16:02:18 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6971lb # [ 168.006360] niks3-server[1128]: 2026/09/23 16:02:18 INFO Received sign narinfos request method=POST path=/api/pushes/72/sign6972lb # [ 168.010691] niks3-server[1128]: 2026/09/23 16:02:18 INFO Signed narinfos id=72 count=16973worker1 # [ 168.029629] nix-grpc-daemon[1762]: time=2026-09-23T16:02:17.239Z level=INFO msg="Uploading 1 narinfos"6974lb # [ 168.020530] niks3-server[1128]: 2026/09/23 16:02:18 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6975lb # [ 168.032311] niks3-server[1128]: 2026/09/23 16:02:18 INFO Received complete push request method=POST path=/api/pushes/72/complete6976worker1 # [ 168.054735] nix-grpc-daemon[1762]: time=2026-09-23T16:02:17.265Z level=INFO msg="Upload complete. (169ms)"6977lb # [ 168.040565] niks3-server[1128]: 2026/09/23 16:02:18 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6978worker1 # [ 168.057928] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37790 duration_s=0 paths=1 nar_bytes_in=5046979worker1 # [ 168.063457] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377906980worker1 # [ 168.071974] nix-daemon[865]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6981worker1 # [ 168.077425] nix-daemon[865]: reaped child process 1919, status = succeeded6982worker1 # [ 168.083785] nix-daemon[865]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6983worker1 # [ 169.116423] nix-daemon[865]: reaped child process 1928, status = succeeded6984worker1 # [ 169.195091] nix-daemon[865]: accepted connection from pid 1944, user nix-grpc-daemon (trusted)6985worker1 # [ 169.208088] nix-daemon[865]: reaped child process 1947, status = succeeded6986lb # [ 169.199963] niks3-server[1128]: 2026/09/23 16:02:19 INFO Received push request method=POST path=/api/pushes6987worker1 # [ 169.222206] nix-grpc-daemon[1762]: time=2026-09-23T16:02:18.432Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6988worker1 # [ 169.225151] nix-grpc-daemon[1762]: time=2026-09-23T16:02:18.434Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6989lb # [ 169.239165] niks3-server[1128]: 2026/09/23 16:02:19 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls6990lb # [ 169.242864] niks3-server[1128]: 2026/09/23 16:02:19 INFO Received sign narinfos request method=POST path=/api/pushes/73/sign6991worker1 # [ 169.264696] nix-grpc-daemon[1762]: time=2026-09-23T16:02:18.474Z level=INFO msg="Uploading 1 narinfos"6992lb # [ 169.248950] niks3-server[1128]: 2026/09/23 16:02:19 INFO Signed narinfos id=73 count=16993lb # [ 169.254858] niks3-server[1128]: 2026/09/23 16:02:19 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6994lb # [ 169.259502] niks3-server[1128]: 2026/09/23 16:02:19 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv6995lb # [ 169.267056] niks3-server[1128]: 2026/09/23 16:02:19 INFO Received complete push request method=POST path=/api/pushes/73/complete6996lb # [ 169.273333] niks3-server[1128]: 2026/09/23 16:02:19 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo6997worker1 # [ 169.288592] nix-grpc-daemon[1762]: time=2026-09-23T16:02:18.498Z level=INFO msg="Upload complete. (198ms)"6998worker1 # [ 169.293777] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37790 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=13 outputs=16999worker1 # [ 169.300735] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377907000worker2 # [ 169.323622] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:19Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=17001worker2 # [ 169.329391] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:19Z level=debug event=want client=6 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=07002worker1 # [ 169.311287] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:377907003worker1 # [ 169.315163] nix-daemon[865]: reaped child process 1926, status = succeeded7004worker1 # [ 169.317381] nix-grpc-daemon[1762]: time=2026-09-23T16:02:18.527Z level=INFO msg="All 1 paths already cached"7005worker1 # [ 169.321537] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:377907006worker1 # [ 169.326258] nix-daemon[865]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)7007worker1 # [ 169.411294] nix-daemon[865]: accepted connection from pid 1951, user nix-grpc-daemon (trusted)7008worker1 # [ 169.426946] nix-daemon[865]: reaped child process 1954, status = succeeded7009lb # [ 169.415969] niks3-server[1128]: 2026/09/23 16:02:19 INFO Received push request method=POST path=/api/pushes7010worker1 # [ 169.439518] nix-grpc-daemon[1762]: time=2026-09-23T16:02:18.649Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7011worker1 # [ 169.440990] nix-grpc-daemon[1762]: time=2026-09-23T16:02:18.651Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"7012lb # [ 169.450660] niks3-server[1128]: 2026/09/23 16:02:19 INFO Received sign narinfos request method=POST path=/api/pushes/74/sign7013lb # [ 169.457160] niks3-server[1128]: 2026/09/23 16:02:19 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst7014lb # [ 169.463295] niks3-server[1128]: 2026/09/23 16:02:19 INFO Signed narinfos id=74 count=17015worker1 # [ 169.482541] nix-grpc-daemon[1762]: time=2026-09-23T16:02:18.692Z level=INFO msg="Uploading 1 narinfos"7016lb # [ 169.470413] niks3-server[1128]: 2026/09/23 16:02:19 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls7017lb # [ 169.480819] niks3-server[1128]: 2026/09/23 16:02:19 INFO Received complete push request method=POST path=/api/pushes/74/complete7018worker1 # [ 169.505976] nix-grpc-daemon[1762]: time=2026-09-23T16:02:18.716Z level=INFO msg="Upload complete. (175ms)"7019lb # [ 169.492062] niks3-server[1128]: 2026/09/23 16:02:19 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo7020worker1 # [ 169.510064] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37790 duration_s=0 paths=1 nar_bytes_in=6247021worker1 # [ 169.515189] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377907022worker1 # [ 169.524514] nix-daemon[865]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)7023worker1 # [ 169.533193] nix-daemon[865]: reaped child process 1949, status = succeeded7024worker1 # [ 169.538214] nix-daemon[865]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)7025worker1 # [ 170.564958] nix-daemon[865]: reaped child process 1958, status = succeeded7026worker1 # [ 170.638280] nix-daemon[865]: accepted connection from pid 1974, user nix-grpc-daemon (trusted)7027worker1 # [ 170.649528] nix-daemon[865]: reaped child process 1977, status = succeeded7028lb # [ 170.644362] niks3-server[1128]: 2026/09/23 16:02:20 INFO Received push request method=POST path=/api/pushes7029worker1 # [ 170.667412] nix-grpc-daemon[1762]: time=2026-09-23T16:02:19.877Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7030worker1 # [ 170.669065] nix-grpc-daemon[1762]: time=2026-09-23T16:02:19.879Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"7031lb # [ 170.687583] niks3-server[1128]: 2026/09/23 16:02:20 INFO Received sign narinfos request method=POST path=/api/pushes/75/sign7032lb # [ 170.695742] niks3-server[1128]: 2026/09/23 16:02:20 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls7033worker1 # [ 170.716868] nix-grpc-daemon[1762]: time=2026-09-23T16:02:19.927Z level=INFO msg="Uploading 1 narinfos"7034lb # [ 170.701683] niks3-server[1128]: 2026/09/23 16:02:20 INFO Signed narinfos id=75 count=17035lb # [ 170.705190] niks3-server[1128]: 2026/09/23 16:02:20 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst7036lb # [ 170.711989] niks3-server[1128]: 2026/09/23 16:02:20 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv7037lb # [ 170.724542] niks3-server[1128]: 2026/09/23 16:02:20 INFO Received complete push request method=POST path=/api/pushes/75/complete7038worker1 # [ 170.745559] nix-grpc-daemon[1762]: time=2026-09-23T16:02:19.955Z level=INFO msg="Upload complete. (200ms)"7039lb # [ 170.731251] niks3-server[1128]: 2026/09/23 16:02:20 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo7040worker1 # [ 170.751915] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37790 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=14 outputs=17041worker2 # [ 170.781295] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:20Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=17042worker1 # [ 170.760707] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377907043worker2 # [ 170.787196] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:20Z level=debug event=want client=6 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=07044worker1 # [ 170.770870] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:377907045worker1 # [ 170.780199] nix-daemon[865]: reaped child process 1956, status = succeeded7046worker1 # [ 170.782190] nix-grpc-daemon[1762]: time=2026-09-23T16:02:19.992Z level=INFO msg="All 1 paths already cached"7047worker1 # [ 170.786777] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:377907048worker1 # [ 170.791170] nix-daemon[865]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)7049worker1 # [ 170.877232] nix-daemon[865]: accepted connection from pid 1981, user nix-grpc-daemon (trusted)7050worker1 # [ 170.890288] nix-daemon[865]: reaped child process 1984, status = succeeded7051lb # [ 170.881862] niks3-server[1128]: 2026/09/23 16:02:20 INFO Received push request method=POST path=/api/pushes7052worker1 # [ 170.905560] nix-grpc-daemon[1762]: time=2026-09-23T16:02:20.115Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7053worker1 # [ 170.907518] nix-grpc-daemon[1762]: time=2026-09-23T16:02:20.117Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"7054lb # [ 170.910266] niks3-server[1128]: 2026/09/23 16:02:20 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst7055lb # [ 170.922709] niks3-server[1128]: 2026/09/23 16:02:21 INFO Received sign narinfos request method=POST path=/api/pushes/76/sign7056worker1 # [ 170.943769] nix-grpc-daemon[1762]: time=2026-09-23T16:02:20.153Z level=INFO msg="Uploading 1 narinfos"7057lb # [ 170.928137] niks3-server[1128]: 2026/09/23 16:02:21 INFO Signed narinfos id=76 count=17058lb # [ 170.933609] niks3-server[1128]: 2026/09/23 16:02:21 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls7059lb # [ 170.947099] niks3-server[1128]: 2026/09/23 16:02:21 INFO Received complete push request method=POST path=/api/pushes/76/complete7060lb # [ 170.956052] niks3-server[1128]: 2026/09/23 16:02:21 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo7061worker1 # [ 170.975817] nix-grpc-daemon[1762]: time=2026-09-23T16:02:20.186Z level=INFO msg="Upload complete. (178ms)"7062worker1 # [ 170.980517] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37790 duration_s=0 paths=1 nar_bytes_in=6247063worker1 # [ 170.987243] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377907064worker1 # [ 170.996484] nix-daemon[865]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)7065worker1 # [ 171.008392] nix-daemon[865]: reaped child process 1979, status = succeeded7066worker1 # [ 171.012130] nix-daemon[865]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)7067worker1 # [ 172.451256] nix-daemon[865]: reaped child process 1988, status = succeeded7068worker1 # [ 172.534888] nix-daemon[865]: accepted connection from pid 2004, user nix-grpc-daemon (trusted)7069worker1 # [ 172.548086] nix-daemon[865]: reaped child process 2007, status = succeeded7070lb # [ 172.540665] niks3-server[1128]: 2026/09/23 16:02:22 INFO Received push request method=POST path=/api/pushes7071worker1 # [ 172.565177] nix-grpc-daemon[1762]: time=2026-09-23T16:02:21.775Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7072worker1 # [ 172.566888] nix-grpc-daemon[1762]: time=2026-09-23T16:02:21.777Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"7073lb # [ 172.584528] niks3-server[1128]: 2026/09/23 16:02:22 INFO Received sign narinfos request method=POST path=/api/pushes/77/sign7074lb # [ 172.593053] niks3-server[1128]: 2026/09/23 16:02:22 INFO Signed narinfos id=77 count=17075worker1 # [ 172.612162] nix-grpc-daemon[1762]: time=2026-09-23T16:02:21.822Z level=INFO msg="Uploading 1 narinfos"7076lb # [ 172.603052] niks3-server[1128]: 2026/09/23 16:02:22 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv7077lb # [ 172.608384] niks3-server[1128]: 2026/09/23 16:02:22 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst7078lb # [ 172.615060] niks3-server[1128]: 2026/09/23 16:02:22 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls7079lb # [ 172.623061] niks3-server[1128]: 2026/09/23 16:02:22 INFO Received complete push request method=POST path=/api/pushes/77/complete7080worker1 # [ 172.642454] nix-grpc-daemon[1762]: time=2026-09-23T16:02:21.852Z level=INFO msg="Upload complete. (222ms)"7081lb # [ 172.629097] niks3-server[1128]: 2026/09/23 16:02:22 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo7082worker1 # [ 172.648479] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37790 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=15 outputs=17083worker2 # [ 172.677946] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:22Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=17084worker1 # [ 172.658370] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377907085worker2 # [ 172.683966] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:22Z level=debug event=want client=6 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=07086worker1 # [ 172.668295] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:377907087worker1 # [ 172.677130] nix-daemon[865]: reaped child process 1986, status = succeeded7088worker1 # [ 172.679300] nix-grpc-daemon[1762]: time=2026-09-23T16:02:21.887Z level=INFO msg="All 1 paths already cached"7089worker1 # [ 172.681881] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:377907090worker1 # [ 172.686604] nix-daemon[865]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)7091worker1 # [ 172.771979] nix-daemon[865]: accepted connection from pid 2011, user nix-grpc-daemon (trusted)7092worker1 # [ 172.786122] nix-daemon[865]: reaped child process 2014, status = succeeded7093lb # [ 172.775490] niks3-server[1128]: 2026/09/23 16:02:22 INFO Received push request method=POST path=/api/pushes7094worker1 # [ 172.799457] nix-grpc-daemon[1762]: time=2026-09-23T16:02:22.009Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7095worker1 # [ 172.801443] nix-grpc-daemon[1762]: time=2026-09-23T16:02:22.011Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"7096lb # [ 172.804433] niks3-server[1128]: 2026/09/23 16:02:22 INFO Received sign narinfos request method=POST path=/api/pushes/78/sign7097worker1 # [ 172.829120] nix-grpc-daemon[1762]: time=2026-09-23T16:02:22.039Z level=INFO msg="Uploading 1 narinfos"7098lb # [ 172.814340] niks3-server[1128]: 2026/09/23 16:02:22 INFO Signed narinfos id=78 count=17099lb # [ 172.818555] niks3-server[1128]: 2026/09/23 16:02:22 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst7100lb # [ 172.825402] niks3-server[1128]: 2026/09/23 16:02:22 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls7101lb # [ 172.837218] niks3-server[1128]: 2026/09/23 16:02:22 INFO Received complete push request method=POST path=/api/pushes/78/complete7102worker1 # [ 172.861357] nix-grpc-daemon[1762]: time=2026-09-23T16:02:22.071Z level=INFO msg="Upload complete. (170ms)"7103lb # [ 172.846190] niks3-server[1128]: 2026/09/23 16:02:22 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo7104worker1 # [ 172.864998] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37790 duration_s=0 paths=1 nar_bytes_in=6247105worker1 # [ 172.869914] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377907106worker1 # [ 172.880791] nix-daemon[865]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)7107worker1 # [ 172.884810] nix-daemon[865]: reaped child process 2009, status = succeeded7108worker1 # [ 172.889707] nix-daemon[865]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)7109worker1 # [ 174.127633] nix-daemon[865]: reaped child process 2018, status = succeeded7110worker1 # [ 174.214435] nix-daemon[865]: accepted connection from pid 2034, user nix-grpc-daemon (trusted)7111worker1 # [ 174.225525] nix-daemon[865]: reaped child process 2037, status = succeeded7112lb # [ 174.219541] niks3-server[1128]: 2026/09/23 16:02:24 INFO Received push request method=POST path=/api/pushes7113worker1 # [ 174.243858] nix-grpc-daemon[1762]: time=2026-09-23T16:02:23.453Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7114worker1 # [ 174.246578] nix-grpc-daemon[1762]: time=2026-09-23T16:02:23.455Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"7115lb # [ 174.258195] niks3-server[1128]: 2026/09/23 16:02:24 INFO Received sign narinfos request method=POST path=/api/pushes/79/sign7116worker1 # [ 174.280943] nix-grpc-daemon[1762]: time=2026-09-23T16:02:23.491Z level=INFO msg="Uploading 1 narinfos"7117lb # [ 174.264721] niks3-server[1128]: 2026/09/23 16:02:24 INFO Signed narinfos id=79 count=17118lb # [ 174.274476] niks3-server[1128]: 2026/09/23 16:02:24 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls7119lb # [ 174.278887] niks3-server[1128]: 2026/09/23 16:02:24 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst7120lb # [ 174.290412] niks3-server[1128]: 2026/09/23 16:02:24 INFO Received complete push request method=POST path=/api/pushes/79/complete7121lb # [ 174.293916] niks3-server[1128]: 2026/09/23 16:02:24 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv7122lb # [ 174.301278] niks3-server[1128]: 2026/09/23 16:02:24 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo7123worker1 # [ 174.316329] nix-grpc-daemon[1762]: time=2026-09-23T16:02:23.526Z level=INFO msg="Upload complete. (218ms)"7124worker1 # [ 174.322630] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37790 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=16 outputs=17125worker2 # [ 174.352547] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:24Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=17126worker1 # [ 174.332321] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377907127worker2 # [ 174.358696] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:24Z level=debug event=want client=6 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=07128worker1 # [ 174.344514] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:377907129worker1 # [ 174.350715] nix-daemon[865]: reaped child process 2016, status = succeeded7130worker1 # [ 174.356889] nix-grpc-daemon[1762]: time=2026-09-23T16:02:23.567Z level=INFO msg="All 2 paths already cached"7131worker1 # [ 174.363826] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:377907132worker1 # [ 174.370390] nix-daemon[865]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)7133worker1 # [ 174.457403] nix-daemon[865]: accepted connection from pid 2041, user nix-grpc-daemon (trusted)7134worker1 # [ 174.471833] nix-daemon[865]: reaped child process 2044, status = succeeded7135lb # [ 174.463066] niks3-server[1128]: 2026/09/23 16:02:24 INFO Received push request method=POST path=/api/pushes7136worker1 # [ 174.486908] nix-grpc-daemon[1762]: time=2026-09-23T16:02:23.697Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7137worker1 # [ 174.489026] nix-grpc-daemon[1762]: time=2026-09-23T16:02:23.698Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"7138lb # [ 174.491692] niks3-server[1128]: 2026/09/23 16:02:24 INFO Received sign narinfos request method=POST path=/api/pushes/80/sign7139worker1 # [ 174.513097] nix-grpc-daemon[1762]: time=2026-09-23T16:02:23.723Z level=INFO msg="Uploading 1 narinfos"7140lb # [ 174.497144] niks3-server[1128]: 2026/09/23 16:02:24 INFO Signed narinfos id=80 count=17141lb # [ 174.505143] niks3-server[1128]: 2026/09/23 16:02:24 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls7142lb # [ 174.518948] niks3-server[1128]: 2026/09/23 16:02:24 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst7143lb # [ 174.524539] niks3-server[1128]: 2026/09/23 16:02:24 INFO Received complete push request method=POST path=/api/pushes/80/complete7144worker1 # [ 174.548106] nix-grpc-daemon[1762]: time=2026-09-23T16:02:23.758Z level=INFO msg="Upload complete. (171ms)"7145lb # [ 174.533895] niks3-server[1128]: 2026/09/23 16:02:24 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo7146worker1 # [ 174.551740] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37790 duration_s=0 paths=1 nar_bytes_in=7527147worker1 # [ 174.556704] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377907148worker1 # [ 174.566479] nix-daemon[865]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)7149worker1 # [ 174.571790] nix-daemon[865]: reaped child process 2039, status = succeeded7150worker1 # [ 174.576463] nix-daemon[865]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)7151worker1 # [ 175.811539] nix-daemon[865]: reaped child process 2048, status = succeeded7152worker1 # [ 175.892728] nix-daemon[865]: accepted connection from pid 2064, user nix-grpc-daemon (trusted)7153worker1 # [ 175.905610] nix-daemon[865]: reaped child process 2067, status = succeeded7154lb # [ 175.897640] niks3-server[1128]: 2026/09/23 16:02:25 INFO Received push request method=POST path=/api/pushes7155worker1 # [ 175.923781] nix-grpc-daemon[1762]: time=2026-09-23T16:02:25.133Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7156worker1 # [ 175.926061] nix-grpc-daemon[1762]: time=2026-09-23T16:02:25.135Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"7157lb # [ 175.940712] niks3-server[1128]: 2026/09/23 16:02:26 INFO Received sign narinfos request method=POST path=/api/pushes/81/sign7158worker1 # [ 175.961726] nix-grpc-daemon[1762]: time=2026-09-23T16:02:25.171Z level=INFO msg="Uploading 1 narinfos"7159lb # [ 175.945614] niks3-server[1128]: 2026/09/23 16:02:26 INFO Signed narinfos id=81 count=17160lb # [ 175.952934] niks3-server[1128]: 2026/09/23 16:02:26 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls7161lb # [ 175.960962] niks3-server[1128]: 2026/09/23 16:02:26 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv7162lb # [ 175.963426] niks3-server[1128]: 2026/09/23 16:02:26 INFO Received complete push request method=POST path=/api/pushes/81/complete7163worker1 # [ 175.984702] nix-grpc-daemon[1762]: time=2026-09-23T16:02:25.194Z level=INFO msg="Upload complete. (198ms)"7164lb # [ 175.969965] niks3-server[1128]: 2026/09/23 16:02:26 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst7165lb # [ 175.972867] niks3-server[1128]: 2026/09/23 16:02:26 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo7166worker1 # [ 175.991129] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37790 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=17 outputs=17167worker2 # [ 176.017872] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:26Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=17168worker1 # [ 175.997429] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377907169worker1 # [ 176.008059] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:377907170worker1 # [ 176.010378] nix-daemon[865]: reaped child process 2046, status = succeeded7171worker1 # [ 176.014656] nix-grpc-daemon[1762]: time=2026-09-23T16:02:25.225Z level=INFO msg="All 2 paths already cached"7172worker1 # [ 176.018967] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:377907173worker1 # [ 176.023846] nix-daemon[865]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)7174worker1 # [ 176.107981] nix-daemon[865]: accepted connection from pid 2071, user nix-grpc-daemon (trusted)7175worker1 # [ 176.121922] nix-daemon[865]: reaped child process 2074, status = succeeded7176lb # [ 176.113454] niks3-server[1128]: 2026/09/23 16:02:26 INFO Received push request method=POST path=/api/pushes7177worker1 # [ 176.137936] nix-grpc-daemon[1762]: time=2026-09-23T16:02:25.348Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7178worker1 # [ 176.139932] nix-grpc-daemon[1762]: time=2026-09-23T16:02:25.349Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"7179lb # [ 176.144145] niks3-server[1128]: 2026/09/23 16:02:26 INFO Received sign narinfos request method=POST path=/api/pushes/82/sign7180lb # [ 176.148699] niks3-server[1128]: 2026/09/23 16:02:26 INFO Signed narinfos id=82 count=17181worker1 # [ 176.167125] nix-grpc-daemon[1762]: time=2026-09-23T16:02:25.377Z level=INFO msg="Uploading 1 narinfos"7182lb # [ 176.154187] niks3-server[1128]: 2026/09/23 16:02:26 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls7183lb # [ 176.164172] niks3-server[1128]: 2026/09/23 16:02:26 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst7184lb # [ 176.167533] niks3-server[1128]: 2026/09/23 16:02:26 INFO Received complete push request method=POST path=/api/pushes/82/complete7185worker1 # [ 176.188144] nix-grpc-daemon[1762]: time=2026-09-23T16:02:25.398Z level=INFO msg="Upload complete. (159ms)"7186worker1 # [ 176.190115] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37790 duration_s=0 paths=1 nar_bytes_in=7527187lb # [ 176.174837] niks3-server[1128]: 2026/09/23 16:02:26 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo7188worker1 # [ 176.192926] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377907189worker1 # [ 176.200157] nix-daemon[865]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)7190worker1 # [ 176.203551] nix-daemon[865]: reaped child process 2069, status = succeeded7191worker1 # [ 176.207493] nix-daemon[865]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)7192worker1 # [ 177.446782] nix-daemon[865]: reaped child process 2078, status = succeeded7193worker1 # [ 177.529404] nix-daemon[865]: accepted connection from pid 2094, user nix-grpc-daemon (trusted)7194worker1 # [ 177.542549] nix-daemon[865]: reaped child process 2097, status = succeeded7195lb # [ 177.534339] niks3-server[1128]: 2026/09/23 16:02:27 INFO Received push request method=POST path=/api/pushes7196worker1 # [ 177.559226] nix-grpc-daemon[1762]: time=2026-09-23T16:02:26.769Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7197worker1 # [ 177.560701] nix-grpc-daemon[1762]: time=2026-09-23T16:02:26.771Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"7198lb # [ 177.576368] niks3-server[1128]: 2026/09/23 16:02:27 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls7199lb # [ 177.580923] niks3-server[1128]: 2026/09/23 16:02:27 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv7200lb # [ 177.587136] niks3-server[1128]: 2026/09/23 16:02:27 INFO Received sign narinfos request method=POST path=/api/pushes/83/sign7201worker1 # [ 177.607573] nix-grpc-daemon[1762]: time=2026-09-23T16:02:26.817Z level=INFO msg="Uploading 1 narinfos"7202lb # [ 177.592099] niks3-server[1128]: 2026/09/23 16:02:27 INFO Signed narinfos id=83 count=17203lb # [ 177.597074] niks3-server[1128]: 2026/09/23 16:02:27 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst7204lb # [ 177.602762] niks3-server[1128]: 2026/09/23 16:02:27 INFO Received complete push request method=POST path=/api/pushes/83/complete7205worker1 # [ 177.623140] nix-grpc-daemon[1762]: time=2026-09-23T16:02:26.833Z level=INFO msg="Upload complete. (208ms)"7206lb # [ 177.608870] niks3-server[1128]: 2026/09/23 16:02:27 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo7207worker1 # [ 177.628942] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37790 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=18 outputs=17208worker2 # [ 177.655703] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:27Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=17209worker1 # [ 177.637364] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377907210worker1 # [ 177.648525] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:377907211worker1 # [ 177.652113] nix-daemon[865]: reaped child process 2076, status = succeeded7212worker1 # [ 177.656303] nix-grpc-daemon[1762]: time=2026-09-23T16:02:26.866Z level=INFO msg="All 2 paths already cached"7213worker1 # [ 177.660488] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:377907214worker1 # [ 177.664435] nix-daemon[865]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)7215worker1 # [ 177.750135] nix-daemon[865]: accepted connection from pid 2101, user nix-grpc-daemon (trusted)7216worker1 # [ 177.764395] nix-daemon[865]: reaped child process 2104, status = succeeded7217lb # [ 177.753558] niks3-server[1128]: 2026/09/23 16:02:27 INFO Received push request method=POST path=/api/pushes7218worker1 # [ 177.777926] nix-grpc-daemon[1762]: time=2026-09-23T16:02:26.988Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7219worker1 # [ 177.779976] nix-grpc-daemon[1762]: time=2026-09-23T16:02:26.989Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"7220lb # [ 177.785318] niks3-server[1128]: 2026/09/23 16:02:27 INFO Received sign narinfos request method=POST path=/api/pushes/84/sign7221lb # [ 177.792651] niks3-server[1128]: 2026/09/23 16:02:27 INFO Signed narinfos id=84 count=17222worker1 # [ 177.811979] nix-grpc-daemon[1762]: time=2026-09-23T16:02:27.022Z level=INFO msg="Uploading 1 narinfos"7223lb # [ 177.797994] niks3-server[1128]: 2026/09/23 16:02:27 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls7224lb # [ 177.804764] niks3-server[1128]: 2026/09/23 16:02:27 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst7225lb # [ 177.815213] niks3-server[1128]: 2026/09/23 16:02:27 INFO Received complete push request method=POST path=/api/pushes/84/complete7226worker1 # [ 177.835020] nix-grpc-daemon[1762]: time=2026-09-23T16:02:27.045Z level=INFO msg="Upload complete. (165ms)"7227lb # [ 177.821344] niks3-server[1128]: 2026/09/23 16:02:27 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo7228worker1 # [ 177.837159] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37790 duration_s=0 paths=1 nar_bytes_in=7527229worker1 # [ 177.842088] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:377907230worker1 # [ 177.851910] nix-daemon[865]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)7231worker1 # [ 177.856963] nix-daemon[865]: reaped child process 2099, status = succeeded7232worker1 # [ 177.860272] nix-daemon[865]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)7233worker1 # [ 178.674939] nix-daemon[865]: reaped child process 2108, status = succeeded7234worker1 # [ 178.753091] nix-daemon[865]: accepted connection from pid 2124, user nix-grpc-daemon (trusted)7235worker1 # [ 178.765390] nix-daemon[865]: reaped child process 2127, status = succeeded7236lb # [ 178.757790] niks3-server[1128]: 2026/09/23 16:02:28 INFO Received push request method=POST path=/api/pushes7237worker1 # [ 178.781853] nix-grpc-daemon[1762]: time=2026-09-23T16:02:27.991Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7238worker1 # [ 178.784060] nix-grpc-daemon[1762]: time=2026-09-23T16:02:27.993Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"7239lb # [ 178.799360] niks3-server[1128]: 2026/09/23 16:02:28 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv7240lb # [ 178.808076] niks3-server[1128]: 2026/09/23 16:02:28 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst7241lb # [ 178.812327] niks3-server[1128]: 2026/09/23 16:02:28 INFO Received sign narinfos request method=POST path=/api/pushes/85/sign7242lb # [ 178.819502] niks3-server[1128]: 2026/09/23 16:02:28 INFO Signed narinfos id=85 count=17243worker1 # [ 178.835946] nix-grpc-daemon[1762]: time=2026-09-23T16:02:28.045Z level=INFO msg="Uploading 1 narinfos"7244lb # [ 178.825228] niks3-server[1128]: 2026/09/23 16:02:28 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls7245lb # [ 178.835161] niks3-server[1128]: 2026/09/23 16:02:28 INFO Received complete push request method=POST path=/api/pushes/85/complete7246worker1 # [ 178.856350] nix-grpc-daemon[1762]: time=2026-09-23T16:02:28.066Z level=INFO msg="Upload complete. (189ms)"7247lb # [ 178.841316] niks3-server[1128]: 2026/09/23 16:02:28 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo7248worker1 # [ 178.861878] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37790 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=19 outputs=17249worker1 # [ 178.876515] nix-daemon[865]: reaped child process 2106, status = succeeded7250client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 14.39 seconds)7251worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7252worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7253client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]'7254client # [ 178.979737] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7255client # Running as unit: gpu.service; invocation ID: 5db48ed53a084288add6f21fc65380497256client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.06 seconds)7257lb: must succeed: curl -sf localhost:9901/clusters7258lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7259lb: must succeed: curl -sf localhost:9901/clusters7260lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7261worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7262client # [ 179.114792] nix[1586]: don't know how to build these paths:7263client # [ 179.115648] nix[1586]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv7264worker1 # [ 179.126499] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:377907265worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7266worker1 # [ 179.132405] nix-grpc-daemon[1702]: ts=2026-09-23T16:02:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:377907267client # [ 179.140250] nix[1586]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}, waiting up to 30s for a worker7268worker2 # [ 179.177894] nix-grpc-daemon[1654]: ts=2026-09-23T16:02:29Z level=debug event=want client=7 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=07269lb: must succeed: curl -sf localhost:9901/clusters7270lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7271lb: must succeed: curl -sf localhost:9901/clusters7272lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7273worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7274worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7275client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7276lb # [ 186.828211] envoy[918]: conn peer=[2001:db8:1::1]:53720 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27094 tx=12312 ms=142187277client # [ 209.141413] nix[1586]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}7278client # [ 209.152352] nix[1586]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed7279client # [ 209.186609] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7280client # [ 209.190570] systemd[1]: gpu.service: Failed with result 'exit-code'.7281client # [ 209.194452] systemd[1]: gpu.service: Consumed 153ms CPU time over 30.206s wall clock time, 37.6M memory peak, 5.6K incoming IP traffic, 5.3K outgoing IP traffic.7282client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.43 seconds)7283client: must succeed: journalctl -u gpu -o cat7284client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds)7285lb: must succeed: curl -sf localhost:9901/clusters7286lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7287lb: must succeed: curl -sf localhost:9901/clusters7288lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7289worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7290worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7291(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 45.32 seconds)7292subtest: low disk drains a worker and builds go to the other7293worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7294worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.94 seconds)7295worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7296worker1 # [ 211.022508] nix-grpc-daemon[1702]: ts=2026-09-23T16:03:00Z level=info event=unhealthy reason=min_free7297worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.09 seconds)7298client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&27299client # don't know how to build these paths:7300client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7301worker2 # [ 212.034416] nix-grpc-daemon[1654]: ts=2026-09-23T16:03:02Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:569807302worker2 # [ 212.040917] nix-grpc-daemon[1654]: ts=2026-09-23T16:03:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:569807303worker2 # [ 212.062182] nix-grpc-daemon[1654]: ts=2026-09-23T16:03:02Z level=debug event=want client=8 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=07304worker2 # [ 212.067982] nix-grpc-daemon[1654]: ts=2026-09-23T16:03:02Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=17305worker2 # [ 212.073418] nix-grpc-daemon[1654]: ts=2026-09-23T16:03:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569807306worker2 # [ 212.084747] nix-grpc-daemon[1654]: ts=2026-09-23T16:03:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:569807307client # copying 1 paths to 'lb:50051'...7308client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...7309worker2 # [ 212.096543] nix-grpc-daemon[1654]: ts=2026-09-23T16:03:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:569807310worker2 # [ 212.109140] nix-daemon[858]: accepted connection from pid 1654, user nix-grpc-daemon (trusted)7311worker2 # [ 212.208935] nix-daemon[858]: accepted connection from pid 2033, user nix-grpc-daemon (trusted)7312worker2 # [ 212.224179] nix-daemon[858]: reaped child process 2036, status = succeeded7313lb # [ 212.191275] niks3-server[1128]: 2026/09/23 16:03:02 INFO Received push request method=POST path=/api/pushes7314worker2 # [ 212.242321] nix-grpc-daemon[1734]: time=2026-09-23T16:03:02.246Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7315worker2 # [ 212.244447] nix-grpc-daemon[1734]: time=2026-09-23T16:03:02.248Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"7316lb # [ 212.233813] niks3-server[1128]: 2026/09/23 16:03:02 INFO Received sign narinfos request method=POST path=/api/pushes/86/sign7317worker2 # [ 212.280435] nix-grpc-daemon[1734]: time=2026-09-23T16:03:02.284Z level=INFO msg="Uploading 1 narinfos"7318lb # [ 212.241120] niks3-server[1128]: 2026/09/23 16:03:02 INFO Signed narinfos id=86 count=17319lb # [ 212.246265] niks3-server[1128]: 2026/09/23 16:03:02 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls7320lb # [ 212.260238] niks3-server[1128]: 2026/09/23 16:03:02 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst7321lb # [ 212.266364] niks3-server[1128]: 2026/09/23 16:03:02 INFO Received complete push request method=POST path=/api/pushes/86/complete7322lb # [ 212.271291] niks3-server[1128]: 2026/09/23 16:03:02 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo7323worker2 # [ 212.315553] nix-grpc-daemon[1734]: time=2026-09-23T16:03:02.320Z level=INFO msg="Upload complete. (196ms)"7324worker2 # [ 212.321146] nix-grpc-daemon[1654]: ts=2026-09-23T16:03:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56980 duration_s=0 paths=1 nar_bytes_in=5127325worker2 # [ 212.327095] nix-grpc-daemon[1654]: ts=2026-09-23T16:03:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569807326worker2 # [ 212.340121] nix-daemon[858]: accepted connection from pid 1654, user nix-grpc-daemon (trusted)7327worker2 # [ 212.348823] nix-daemon[858]: reaped child process 2031, status = succeeded7328worker2 # [ 212.353984] nix-daemon[858]: accepted connection from pid 1654, user nix-grpc-daemon (trusted)7329client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...7330client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7331client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7332client # farm-a> LOG-farm-a-drain7333worker2 # [ 212.968521] nix-daemon[858]: reaped child process 2040, status = succeeded7334worker2 # [ 213.050778] nix-daemon[858]: accepted connection from pid 2056, user nix-grpc-daemon (trusted)7335worker2 # [ 213.063666] nix-daemon[858]: reaped child process 2059, status = succeeded7336lb # [ 213.030411] niks3-server[1128]: 2026/09/23 16:03:03 INFO Received push request method=POST path=/api/pushes7337worker2 # [ 213.081351] nix-grpc-daemon[1734]: time=2026-09-23T16:03:03.086Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7338worker2 # [ 213.082898] nix-grpc-daemon[1734]: time=2026-09-23T16:03:03.087Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"7339lb # [ 213.073064] niks3-server[1128]: 2026/09/23 16:03:03 INFO Received sign narinfos request method=POST path=/api/pushes/87/sign7340lb # [ 213.080646] niks3-server[1128]: 2026/09/23 16:03:03 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7341worker2 # [ 213.125190] nix-grpc-daemon[1734]: time=2026-09-23T16:03:03.129Z level=INFO msg="Uploading 1 narinfos"7342lb # [ 213.084724] niks3-server[1128]: 2026/09/23 16:03:03 INFO Signed narinfos id=87 count=17343lb # [ 213.089492] niks3-server[1128]: 2026/09/23 16:03:03 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls7344lb # [ 213.093668] niks3-server[1128]: 2026/09/23 16:03:03 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7345lb # [ 213.100083] niks3-server[1128]: 2026/09/23 16:03:03 INFO Received complete push request method=POST path=/api/pushes/87/complete7346lb # [ 213.107099] niks3-server[1128]: 2026/09/23 16:03:03 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo7347worker2 # [ 213.147209] nix-grpc-daemon[1734]: time=2026-09-23T16:03:03.151Z level=INFO msg="Upload complete. (187ms)"7348worker2 # [ 213.155405] nix-grpc-daemon[1654]: ts=2026-09-23T16:03:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56980 duration_s=0 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=20 outputs=17349worker2 # [ 213.164886] nix-grpc-daemon[1654]: ts=2026-09-23T16:03:03Z level=debug event=want client=8 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=07350worker2 # [ 213.169502] nix-grpc-daemon[1654]: ts=2026-09-23T16:03:03Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=17351worker2 # [ 213.171786] nix-grpc-daemon[1654]: ts=2026-09-23T16:03:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569807352worker2 # [ 213.178770] nix-daemon[858]: reaped child process 2038, status = succeeded7353worker2 # [ 213.180648] nix-grpc-daemon[1654]: ts=2026-09-23T16:03:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:569807354worker2 # [ 213.185904] nix-grpc-daemon[1734]: time=2026-09-23T16:03:03.190Z level=INFO msg="All 1 paths already cached"7355client # copying 1 paths to 'lb:50051'...7356worker2 # [ 213.191109] nix-grpc-daemon[1654]: ts=2026-09-23T16:03:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:569807357client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...7358worker2 # [ 213.197427] nix-daemon[858]: accepted connection from pid 1654, user nix-grpc-daemon (trusted)7359worker2 # [ 213.287566] nix-daemon[858]: accepted connection from pid 2065, user nix-grpc-daemon (trusted)7360worker2 # [ 213.302047] nix-daemon[858]: reaped child process 2068, status = succeeded7361lb # [ 213.269380] niks3-server[1128]: 2026/09/23 16:03:03 INFO Received push request method=POST path=/api/pushes7362worker2 # [ 213.319416] nix-grpc-daemon[1734]: time=2026-09-23T16:03:03.323Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7363worker2 # [ 213.321540] nix-grpc-daemon[1734]: time=2026-09-23T16:03:03.325Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"7364lb # [ 213.299216] niks3-server[1128]: 2026/09/23 16:03:03 INFO Received sign narinfos request method=POST path=/api/pushes/88/sign7365worker2 # [ 213.345648] nix-grpc-daemon[1734]: time=2026-09-23T16:03:03.350Z level=INFO msg="Uploading 1 narinfos"7366lb # [ 213.304952] niks3-server[1128]: 2026/09/23 16:03:03 INFO Signed narinfos id=88 count=17367lb # [ 213.311090] niks3-server[1128]: 2026/09/23 16:03:03 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls7368lb # [ 213.316786] niks3-server[1128]: 2026/09/23 16:03:03 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst7369lb # [ 213.324059] niks3-server[1128]: 2026/09/23 16:03:03 INFO Received complete push request method=POST path=/api/pushes/88/complete7370worker2 # [ 213.369351] nix-grpc-daemon[1734]: time=2026-09-23T16:03:03.374Z level=INFO msg="Upload complete. (166ms)"7371lb # [ 213.329909] niks3-server[1128]: 2026/09/23 16:03:03 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo7372worker2 # [ 213.371548] nix-grpc-daemon[1654]: ts=2026-09-23T16:03:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56980 duration_s=0 paths=1 nar_bytes_in=6247373worker2 # [ 213.375098] nix-grpc-daemon[1654]: ts=2026-09-23T16:03:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569807374worker2 # [ 213.383772] nix-daemon[858]: accepted connection from pid 1654, user nix-grpc-daemon (trusted)7375worker2 # [ 213.385491] nix-daemon[858]: reaped child process 2063, status = succeeded7376worker2 # [ 213.390804] nix-daemon[858]: accepted connection from pid 1654, user nix-grpc-daemon (trusted)7377client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...7378client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7379client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7380client # farm-b> LOG-farm-b-drain7381worker2 # [ 214.832264] nix-daemon[858]: reaped child process 2072, status = succeeded7382worker2 # [ 214.919620] nix-daemon[858]: accepted connection from pid 2088, user nix-grpc-daemon (trusted)7383worker2 # [ 214.933929] nix-daemon[858]: reaped child process 2091, status = succeeded7384lb # [ 214.902283] niks3-server[1128]: 2026/09/23 16:03:04 INFO Received push request method=POST path=/api/pushes7385worker2 # [ 214.951509] nix-grpc-daemon[1734]: time=2026-09-23T16:03:04.955Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7386worker2 # [ 214.954638] nix-grpc-daemon[1734]: time=2026-09-23T16:03:04.957Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"7387lb # [ 214.942839] niks3-server[1128]: 2026/09/23 16:03:05 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls7388lb # [ 214.955206] niks3-server[1128]: 2026/09/23 16:03:05 INFO Received sign narinfos request method=POST path=/api/pushes/89/sign7389worker2 # [ 215.003384] nix-grpc-daemon[1734]: time=2026-09-23T16:03:05.007Z level=INFO msg="Uploading 1 narinfos"7390lb # [ 214.962992] niks3-server[1128]: 2026/09/23 16:03:05 INFO Signed narinfos id=89 count=17391lb # [ 214.967148] niks3-server[1128]: 2026/09/23 16:03:05 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst7392lb # [ 214.972924] niks3-server[1128]: 2026/09/23 16:03:05 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7393lb # [ 214.982181] niks3-server[1128]: 2026/09/23 16:03:05 INFO Received complete push request method=POST path=/api/pushes/89/complete7394worker2 # [ 215.027295] nix-grpc-daemon[1734]: time=2026-09-23T16:03:05.031Z level=INFO msg="Upload complete. (226ms)"7395lb # [ 214.988523] niks3-server[1128]: 2026/09/23 16:03:05 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo7396worker2 # [ 215.035398] nix-grpc-daemon[1654]: ts=2026-09-23T16:03:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56980 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=21 outputs=17397worker2 # [ 215.046788] nix-grpc-daemon[1654]: ts=2026-09-23T16:03:05Z level=debug event=want client=8 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=07398worker2 # [ 215.051574] nix-grpc-daemon[1654]: ts=2026-09-23T16:03:05Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=17399worker2 # [ 215.055249] nix-grpc-daemon[1654]: ts=2026-09-23T16:03:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569807400worker2 # [ 215.062932] nix-daemon[858]: reaped child process 2070, status = succeeded7401worker2 # [ 215.064525] nix-grpc-daemon[1654]: ts=2026-09-23T16:03:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:569807402worker2 # [ 215.071105] nix-grpc-daemon[1734]: time=2026-09-23T16:03:05.075Z level=INFO msg="All 2 paths already cached"7403client # copying 1 paths to 'lb:50051'...7404client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...7405worker2 # [ 215.077501] nix-grpc-daemon[1654]: ts=2026-09-23T16:03:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:569807406worker2 # [ 215.082643] nix-daemon[858]: accepted connection from pid 1654, user nix-grpc-daemon (trusted)7407worker2 # [ 215.175932] nix-daemon[858]: accepted connection from pid 2097, user nix-grpc-daemon (trusted)7408worker2 # [ 215.191508] nix-daemon[858]: reaped child process 2100, status = succeeded7409lb # [ 215.158475] niks3-server[1128]: 2026/09/23 16:03:05 INFO Received push request method=POST path=/api/pushes7410worker2 # [ 215.207805] nix-grpc-daemon[1734]: time=2026-09-23T16:03:05.212Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7411worker2 # [ 215.209936] nix-grpc-daemon[1734]: time=2026-09-23T16:03:05.214Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"7412lb # [ 215.190537] niks3-server[1128]: 2026/09/23 16:03:05 INFO Received sign narinfos request method=POST path=/api/pushes/90/sign7413lb # [ 215.195907] niks3-server[1128]: 2026/09/23 16:03:05 INFO Signed narinfos id=90 count=17414worker2 # [ 215.239941] nix-grpc-daemon[1734]: time=2026-09-23T16:03:05.244Z level=INFO msg="Uploading 1 narinfos"7415lb # [ 215.205731] niks3-server[1128]: 2026/09/23 16:03:05 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst7416lb # [ 215.211504] niks3-server[1128]: 2026/09/23 16:03:05 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls7417lb # [ 215.223123] niks3-server[1128]: 2026/09/23 16:03:05 INFO Received complete push request method=POST path=/api/pushes/90/complete7418worker2 # [ 215.270213] nix-grpc-daemon[1734]: time=2026-09-23T16:03:05.274Z level=INFO msg="Upload complete. (181ms)"7419lb # [ 215.232103] niks3-server[1128]: 2026/09/23 16:03:05 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo7420worker2 # [ 215.274417] nix-grpc-daemon[1654]: ts=2026-09-23T16:03:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56980 duration_s=0 paths=1 nar_bytes_in=7527421worker2 # [ 215.280265] nix-grpc-daemon[1654]: ts=2026-09-23T16:03:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569807422worker2 # [ 215.288738] nix-daemon[858]: accepted connection from pid 1654, user nix-grpc-daemon (trusted)7423worker2 # [ 215.296922] nix-daemon[858]: reaped child process 2095, status = succeeded7424worker2 # [ 215.300845] nix-daemon[858]: accepted connection from pid 1654, user nix-grpc-daemon (trusted)7425client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...7426client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7427client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7428client # farm-top> LOG-farm-top-drain7429worker2 # [ 216.739570] nix-daemon[858]: reaped child process 2104, status = succeeded7430worker2 # [ 216.829155] nix-daemon[858]: accepted connection from pid 2120, user nix-grpc-daemon (trusted)7431worker2 # [ 216.843395] nix-daemon[858]: reaped child process 2123, status = succeeded7432lb # [ 216.811373] niks3-server[1128]: 2026/09/23 16:03:06 INFO Received push request method=POST path=/api/pushes7433worker2 # [ 216.861387] nix-grpc-daemon[1734]: time=2026-09-23T16:03:06.865Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7434worker2 # [ 216.862958] nix-grpc-daemon[1734]: time=2026-09-23T16:03:06.867Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"7435lb # [ 216.828939] envoy[918]: conn peer=[2001:db8:1::1]:35474 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=929 tx=596 ms=300527436lb # [ 216.858147] niks3-server[1128]: 2026/09/23 16:03:06 INFO Received sign narinfos request method=POST path=/api/pushes/91/sign7437worker2 # [ 216.903270] nix-grpc-daemon[1734]: time=2026-09-23T16:03:06.907Z level=INFO msg="Uploading 1 narinfos"7438lb # [ 216.863117] niks3-server[1128]: 2026/09/23 16:03:06 INFO Signed narinfos id=91 count=17439lb # [ 216.867594] niks3-server[1128]: 2026/09/23 16:03:06 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7440lb # [ 216.875411] niks3-server[1128]: 2026/09/23 16:03:06 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7441lb # [ 216.882902] niks3-server[1128]: 2026/09/23 16:03:06 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7442lb # [ 216.888179] niks3-server[1128]: 2026/09/23 16:03:06 INFO Received complete push request method=POST path=/api/pushes/91/complete7443worker2 # [ 216.933802] nix-grpc-daemon[1734]: time=2026-09-23T16:03:06.938Z level=INFO msg="Upload complete. (225ms)"7444lb # [ 216.894693] niks3-server[1128]: 2026/09/23 16:03:06 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7445worker2 # [ 216.940959] nix-grpc-daemon[1654]: ts=2026-09-23T16:03:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56980 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=22 outputs=17446worker2 # [ 216.956621] nix-daemon[858]: reaped child process 2102, status = succeeded7447client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&2, in 5.09 seconds)7448client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath7449client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath, in 0.11 seconds)7450worker1: must succeed: rm /nix/.rw-store/fill7451worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.12 seconds)7452worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7453worker1 # [ 218.026673] nix-grpc-daemon[1702]: ts=2026-09-23T16:03:07Z level=info event=healthy reason=min_free7454worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.08 seconds)7455(finished: subtest: low disk drains a worker and builds go to the other, in 8.49 seconds)7456(finished: run the VM test script, in 219.25 seconds)7457test script finished in 219.42s7458cleanup7459kill QemuMachine (pid 49)7460client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7461client # [2026-09-23T16:03:08Z INFO virtiofsd] Client disconnected, shutting down7462client # [2026-09-23T16:03:08Z INFO virtiofsd] Client disconnected, shutting down7463client # [2026-09-23T16:03:08Z INFO virtiofsd] Client disconnected, shutting down7464kill QemuMachine (pid 50)7465lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7466lb # [2026-09-23T16:03:09Z INFO virtiofsd] Client disconnected, shutting down7467lb # [2026-09-23T16:03:09Z INFO virtiofsd] Client disconnected, shutting down7468lb # [2026-09-23T16:03:09Z INFO virtiofsd] Client disconnected, shutting down7469kill QemuMachine (pid 51)7470worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7471worker1 # [2026-09-23T16:03:09Z INFO virtiofsd] Client disconnected, shutting down7472worker1 # [2026-09-23T16:03:09Z INFO virtiofsd] Client disconnected, shutting down7473worker1 # [2026-09-23T16:03:09Z INFO virtiofsd] Client disconnected, shutting down7474kill QemuMachine (pid 52)7475worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7476worker2 # [2026-09-23T16:03:09Z INFO virtiofsd] Client disconnected, shutting down7477worker2 # [2026-09-23T16:03:09Z INFO virtiofsd] Client disconnected, shutting down7478worker2 # [2026-09-23T16:03:09Z INFO virtiofsd] Client disconnected, shutting down7479(finished: cleanup, in 1.08 seconds)7480additionally exposed symbols:7481 client, lb, worker1, worker2,7482 vlan1,7483 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh