vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #511
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.01 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12client # Disk image does not exist, creating the virtualisation disk image...13client # Formatting '/build/vm-state-client/tmp.ECrY0ty3sf', fmt=raw size=107374182414client # mke2fs 1.47.4 (6-Mar-2025)15client # Discarding device blocks: 0/262144 done16client # Creating filesystem with 262144 4k blocks and 65536 inodes17client: QEMU running (pid 50)18client # Filesystem UUID: a6b3967d-e387-46e9-8a66-aa5be603204619client # Superblock backups stored on blocks:20client # 32768, 98304, 163840, 22937621client # 22client # Allocating group tables: 0/8 done23lb # Disk image does not exist, creating the virtualisation disk image...24client # Writing inode tables: 0/8 done25lb # Formatting '/build/vm-state-lb/tmp.039mOYFrb5', fmt=raw size=107374182426lb # mke2fs 1.47.4 (6-Mar-2025)27worker2: QEMU running (pid 52)28worker2 # Disk image does not exist, creating the virtualisation disk image...29lb # Discarding device blocks: 0/262144 done30client # Creating journal (8192 blocks): done31lb: QEMU running (pid 49)32worker2 # Formatting '/build/vm-state-worker2/tmp.50yweYKqmo', fmt=raw size=107374182433worker1 # Disk image does not exist, creating the virtualisation disk image...34worker1: QEMU running (pid 51)35worker2 # mke2fs 1.47.4 (6-Mar-2025)36worker1 # Formatting '/build/vm-state-worker1/tmp.Sex9gwYnjF', fmt=raw size=107374182437lb # Creating filesystem with 262144 4k blocks and 65536 inodes38client # Writing superblocks and filesystem accounting information: 0/8 done39worker2 # Discarding device blocks: 0/262144 done40worker1 # mke2fs 1.47.4 (6-Mar-2025)41client # 42worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes43worker1 # Discarding device blocks: 0/262144 done44client # Virtualisation disk image created.45worker2 # Filesystem UUID: a1ed688d-48df-4b59-a3bf-1c1f1024573146worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes47client # Starting virtiofs daemons...48worker2 # Superblock backups stored on blocks:49worker1 # Filesystem UUID: 3ebdc43b-31ee-4da5-a711-4800e87d895750client # [2026-09-23T04:11:15Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)51worker2 # 32768, 98304, 163840, 22937652worker1 # Superblock backups stored on blocks:53client # [2026-09-23T04:11:15Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether54worker2 # 55worker1 # 32768, 98304, 163840, 22937656client # [2026-09-23T04:11:15Z INFO virtiofsd] Waiting for vhost-user socket connection...57worker2 # Allocating group tables: 0/8 done58worker1 # 59client # [2026-09-23T04:11:15Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)60worker2 # Writing inode tables: 0/8 done61worker1 # Allocating group tables: 0/8 done62client # [2026-09-23T04:11:15Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether63worker2 # Creating journal (8192 blocks): done64worker1 # Writing inode tables: 0/8 done65client # [2026-09-23T04:11:15Z INFO virtiofsd] Waiting for vhost-user socket connection...66worker2 # Writing superblocks and filesystem accounting information: 0/8 done67worker1 # Creating journal (8192 blocks): done68client # [2026-09-23T04:11:15Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)69worker2 # 70worker1 # Writing superblocks and filesystem accounting information: 0/8 done71client # [2026-09-23T04:11:15Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether72worker2 # Virtualisation disk image created.73worker1 # 74client # [2026-09-23T04:11:15Z INFO virtiofsd] Waiting for vhost-user socket connection...75worker2 # Starting virtiofs daemons...76worker1 # Virtualisation disk image created.77client # [2026-09-23T04:11:15Z INFO virtiofsd] Client connected, servicing requests78worker2 # [2026-09-23T04:11:15Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)79worker1 # Starting virtiofs daemons...80client # [2026-09-23T04:11:15Z INFO virtiofsd] Client connected, servicing requests81worker2 # [2026-09-23T04:11:15Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether82worker1 # [2026-09-23T04:11:15Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)83client # [2026-09-23T04:11:15Z INFO virtiofsd] Client connected, servicing requests84worker2 # [2026-09-23T04:11:15Z INFO virtiofsd] Waiting for vhost-user socket connection...85worker1 # [2026-09-23T04:11:15Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether86(finished: start all VMs, in 0.52 seconds)87lb # Filesystem UUID: 5a3560a7-6b93-41e5-a941-6e0aac61785488worker2 # [2026-09-23T04:11:15Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)89worker1 # [2026-09-23T04:11:15Z INFO virtiofsd] Waiting for vhost-user socket connection...90client: waiting for unit network-addresses-eth1.service91lb # Superblock backups stored on blocks:92worker2 # [2026-09-23T04:11:15Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether93worker1 # [2026-09-23T04:11:15Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)94client: waiting for the VM to finish booting95lb # 32768, 98304, 163840, 22937696worker2 # [2026-09-23T04:11:15Z INFO virtiofsd] Waiting for vhost-user socket connection...97lb # 98worker2 # [2026-09-23T04:11:15Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)99lb # Allocating group tables: 0/8 done100worker2 # [2026-09-23T04:11:15Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether101lb # Writing inode tables: 0/8 done102worker2 # [2026-09-23T04:11:15Z INFO virtiofsd] Waiting for vhost-user socket connection...103lb # Creating journal (8192 blocks): done104worker2 # [2026-09-23T04:11:15Z INFO virtiofsd] Client connected, servicing requests105lb # Writing superblocks and filesystem accounting information: 0/8 done106worker2 # [2026-09-23T04:11:15Z INFO virtiofsd] Client connected, servicing requests107lb # 108worker2 # [2026-09-23T04:11:15Z INFO virtiofsd] Client connected, servicing requests109lb # Virtualisation disk image created.110worker1 # [2026-09-23T04:11:15Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether111lb # Starting virtiofs daemons...112worker1 # [2026-09-23T04:11:15Z INFO virtiofsd] Waiting for vhost-user socket connection...113lb # [2026-09-23T04:11:15Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)114worker1 # [2026-09-23T04:11:15Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)115lb # [2026-09-23T04:11:15Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether116worker1 # [2026-09-23T04:11:15Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether117lb # [2026-09-23T04:11:15Z INFO virtiofsd] Waiting for vhost-user socket connection...118worker1 # [2026-09-23T04:11:15Z INFO virtiofsd] Waiting for vhost-user socket connection...119lb # [2026-09-23T04:11:15Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120worker1 # [2026-09-23T04:11:15Z INFO virtiofsd] Client connected, servicing requests121lb # [2026-09-23T04:11:15Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether122worker1 # [2026-09-23T04:11:15Z INFO virtiofsd] Client connected, servicing requests123lb # [2026-09-23T04:11:15Z INFO virtiofsd] Waiting for vhost-user socket connection...124worker1 # [2026-09-23T04:11:15Z INFO virtiofsd] Client connected, servicing requests125lb # [2026-09-23T04:11:15Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)126lb # [2026-09-23T04:11:15Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether127lb # [2026-09-23T04:11:15Z INFO virtiofsd] Waiting for vhost-user socket connection...128lb # [2026-09-23T04:11:15Z INFO virtiofsd] Client connected, servicing requests129lb # [2026-09-23T04:11:15Z INFO virtiofsd] Client connected, servicing requests130lb # [2026-09-23T04:11:15Z INFO virtiofsd] Client connected, servicing requests131lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)132worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)133worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)134client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)135lb # 136lb # 137lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00138lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...139lb # 140lb # 141lb # 142lb # 143worker2 # 144lb # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00145worker2 # 146worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00147lb # Press Ctrl-B to configure iPXE (PCI 00:05.0)...148lb # 149lb # 150worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...151worker2 # 152worker2 # 153worker2 # 154worker2 # 155worker2 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00156worker1 # 157worker1 # 158worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00159worker2 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...160worker2 # 161worker2 # 162worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...163worker1 # 164worker1 # 165worker1 # 166worker1 # 167worker1 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00168worker1 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...169worker1 # 170worker1 # 171worker1 # Booting from ROM...172client # 173client # 174client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00175client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...176client # 177client # 178client # 179client # 180client # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00181client # Press Ctrl-B to configure iPXE (PCI 00:05.0)...182client # 183client # 184lb # Booting from ROM...185worker2 # Booting from ROM...186client # Booting from ROM...187client # Probing EDD (edd=off to disable)... ok188lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026189lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ggsnp37phfgq4b11zirffk4vxgicg7h8-nixos-system-lb-test/init regInfo=/nix/store/94h72crvqzbw5sjsl7v7v31gd90khbjb-closure-info/registration console=ttyS0,115200n8 console=tty0190lb # [ 0.000000] BIOS-provided physical RAM map:191lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable192lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved193lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved194lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable195lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved196lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved197lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved198lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved199lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved200lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved201lb # [ 0.000000] NX (Execute Disable) protection: active202lb # [ 0.000000] APIC: Static calls initialized203lb # [ 0.000000] SMBIOS 2.8 present.204lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014205lb # [ 0.000000] DMI: Memory slots populated: 1/1206lb # [ 0.000000] Hypervisor detected: KVM207lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000208lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00209lb # [ 0.000000] kvm-clock: using sched offset of 522132230 cycles210lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns211lb # [ 0.000004] tsc: Detected 2400.012 MHz processor212lb # [ 0.000804] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000213worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026214lb # [ 0.000830] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs215lb # [ 0.000832] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT216lb # [ 0.002720] found SMP MP-table at [mem 0x000f5450-0x000f545f]217lb # [ 0.002731] Using GB pages for direct mapping218lb # [ 0.002840] RAMDISK: [mem 0x5e36e000-0x5ffcffff]219lb # [ 0.002848] ACPI: Early table checksum verification disabled220worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/gvbzy0g4j2sf3am2wkwfga8lrrdci67y-nixos-system-worker2-test/init regInfo=/nix/store/788l1h41glyfk32if77z9ksf8qk2j16n-closure-info/registration console=ttyS0,115200n8 console=tty0221lb # [ 0.002850] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )222worker2 # [ 0.000000] BIOS-provided physical RAM map:223worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable224lb # [ 0.002854] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)225worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved226lb # [ 0.002858] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)227worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved228lb # [ 0.002864] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)229worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable230lb # [ 0.002873] ACPI: FACS 0x000000005FFE0000 000040231worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved232lb # [ 0.002875] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)233worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved234worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026235lb # [ 0.002876] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)236worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved237lb # [ 0.002878] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)238worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved239worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved240lb # [ 0.002879] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)241worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved242lb # [ 0.002881] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]243worker2 # [ 0.000000] NX (Execute Disable) protection: active244worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/dyab6c8i372v5waficb63zp79vnsaxrg-nixos-system-worker1-test/init regInfo=/nix/store/a01b4wkhhfw171hhg3lf8p5ygrq1fswf-closure-info/registration console=ttyS0,115200n8 console=tty0245lb # [ 0.002882] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]246worker2 # [ 0.000000] APIC: Static calls initialized247worker1 # [ 0.000000] BIOS-provided physical RAM map:248worker2 # [ 0.000000] SMBIOS 2.8 present.249lb # [ 0.002882] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]250worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable251worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014252lb # [ 0.002883] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]253worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved254worker2 # [ 0.000000] DMI: Memory slots populated: 1/1255lb # [ 0.002883] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]256worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved257worker2 # [ 0.000000] Hypervisor detected: KVM258worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000259worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable260lb # [ 0.002884] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]261worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00262worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved263lb # [ 0.002884] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]264worker2 # [ 0.000001] kvm-clock: using sched offset of 539873391 cycles265lb # [ 0.003104] No NUMA configuration found266worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved267lb # [ 0.003105] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]268worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved269worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns270lb # [ 0.003107] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]271worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor272lb # [ 0.005359] Zone ranges:273worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved274worker2 # [ 0.000812] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000275lb # [ 0.005360] DMA [mem 0x0000000000001000-0x0000000000ffffff]276worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved277lb # [ 0.005361] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]278worker2 # [ 0.000838] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs279worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved280lb # [ 0.005362] Normal empty281lb # [ 0.005363] Device empty282worker1 # [ 0.000000] NX (Execute Disable) protection: active283worker2 # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT284lb # [ 0.005364] Movable zone start for each node285worker1 # [ 0.000000] APIC: Static calls initialized286worker2 # [ 0.002732] found SMP MP-table at [mem 0x000f5450-0x000f545f]287lb # [ 0.005364] Early memory node ranges288worker1 # [ 0.000000] SMBIOS 2.8 present.289worker2 # [ 0.002742] Using GB pages for direct mapping290lb # [ 0.005365] node 0: [mem 0x0000000000001000-0x000000000009efff]291worker2 # [ 0.002856] RAMDISK: [mem 0x5e36f000-0x5ffcffff]292worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014293lb # [ 0.005366] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]294worker2 # [ 0.002865] ACPI: Early table checksum verification disabled295worker1 # [ 0.000000] DMI: Memory slots populated: 1/1296worker1 # [ 0.000000] Hypervisor detected: KVM297lb # [ 0.005367] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]298worker2 # [ 0.002867] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )299worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000300lb # [ 0.005385] On node 0, zone DMA: 1 pages in unavailable ranges301worker2 # [ 0.002872] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)302worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00303lb # [ 0.005650] On node 0, zone DMA: 97 pages in unavailable ranges304worker1 # [ 0.000001] kvm-clock: using sched offset of 549436863 cycles305worker2 # [ 0.002876] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)306lb # [ 0.032513] On node 0, zone DMA32: 40 pages in unavailable ranges307lb # [ 0.032974] ACPI: PM-Timer IO Port: 0x608308client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026309worker2 # [ 0.002883] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)310worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns311lb # [ 0.032984] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])312worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor313worker2 # [ 0.002885] ACPI: FACS 0x000000005FFE0000 000040314lb # [ 0.033011] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23315worker1 # [ 0.000804] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000316worker2 # [ 0.002887] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)317lb # [ 0.033013] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)318worker1 # [ 0.000832] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs319lb # [ 0.033015] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)320worker2 # [ 0.002888] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)321worker1 # [ 0.000835] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT322lb # [ 0.033016] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)323client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/g90cn3cn0g95ry9pa0dhwf3r60ijvjrh-nixos-system-client-test/init regInfo=/nix/store/s9k7mpd3day04qzf19r71x8dq0q2ycs2-closure-info/registration console=ttyS0,115200n8 console=tty0324worker2 # [ 0.002890] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)325worker1 # [ 0.002722] found SMP MP-table at [mem 0x000f5450-0x000f545f]326client # [ 0.000000] BIOS-provided physical RAM map:327lb # [ 0.033017] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)328worker1 # [ 0.002734] Using GB pages for direct mapping329worker2 # [ 0.002891] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)330worker1 # [ 0.002882] RAMDISK: [mem 0x5e370000-0x5ffcffff]331client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable332lb # [ 0.033018] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)333worker2 # [ 0.002892] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]334worker1 # [ 0.002904] ACPI: Early table checksum verification disabled335client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved336lb # [ 0.033020] ACPI: Using ACPI (MADT) for SMP configuration information337worker1 # [ 0.002907] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )338worker2 # [ 0.002893] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]339lb # [ 0.033021] ACPI: HPET id: 0x8086a201 base: 0xfed00000340client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved341lb # [ 0.033024] TSC deadline timer available342worker2 # [ 0.002894] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]343worker1 # [ 0.002912] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)344client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable345lb # [ 0.033028] CPU topo: Max. logical packages: 1346lb # [ 0.033029] CPU topo: Max. logical dies: 1347worker2 # [ 0.002894] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]348worker1 # [ 0.002916] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)349client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved350lb # [ 0.033029] CPU topo: Max. dies per package: 1351worker2 # [ 0.002895] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]352client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved353worker1 # [ 0.002923] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)354lb # [ 0.033033] CPU topo: Max. threads per core: 1355worker2 # [ 0.002896] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]356lb # [ 0.033033] CPU topo: Num. cores per package: 1357worker1 # [ 0.002925] ACPI: FACS 0x000000005FFE0000 000040358client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved359lb # [ 0.033034] CPU topo: Num. threads per package: 1360worker2 # [ 0.002896] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]361worker1 # [ 0.002926] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)362client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved363worker2 # [ 0.003127] No NUMA configuration found364lb # [ 0.033034] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs365client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved366worker1 # [ 0.002928] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)367lb # [ 0.033051] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()368worker2 # [ 0.003128] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]369client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved370worker2 # [ 0.003130] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]371worker1 # [ 0.002929] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)372worker2 # [ 0.005351] Zone ranges:373lb # [ 0.033079] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]374client # [ 0.000000] NX (Execute Disable) protection: active375client # [ 0.000000] APIC: Static calls initialized376worker2 # [ 0.005352] DMA [mem 0x0000000000001000-0x0000000000ffffff]377lb # [ 0.033081] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]378worker1 # [ 0.002931] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)379client # [ 0.000000] SMBIOS 2.8 present.380worker2 # [ 0.005354] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]381lb # [ 0.033082] [mem 0x60000000-0xafffffff] available for PCI devices382worker1 # [ 0.002932] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]383worker2 # [ 0.005355] Normal empty384lb # [ 0.033084] Booting paravirtualized kernel on KVM385worker2 # [ 0.005356] Device empty386client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014387worker2 # [ 0.005356] Movable zone start for each node388worker1 # [ 0.002933] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]389client # [ 0.000000] DMI: Memory slots populated: 1/1390worker2 # [ 0.005357] Early memory node ranges391client # [ 0.000000] Hypervisor detected: KVM392lb # [ 0.033086] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns393worker1 # [ 0.002934] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]394worker2 # [ 0.005357] node 0: [mem 0x0000000000001000-0x000000000009efff]395client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000396lb # [ 0.037548] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1397worker1 # [ 0.002934] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]398client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00399worker2 # [ 0.005359] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]400lb # [ 0.039999] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152401worker1 # [ 0.002935] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]402client # [ 0.000000] kvm-clock: using sched offset of 575867191 cycles403worker2 # [ 0.005359] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]404lb # [ 0.040043] kvm-guest: PV spinlocks disabled, single CPU405worker1 # [ 0.002935] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]406worker2 # [ 0.005382] On node 0, zone DMA: 1 pages in unavailable ranges407client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns408worker2 # [ 0.005642] On node 0, zone DMA: 97 pages in unavailable ranges409worker1 # [ 0.002936] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]410client # [ 0.000004] tsc: Detected 2400.012 MHz processor411worker1 # [ 0.003163] No NUMA configuration found412worker2 # [ 0.031680] On node 0, zone DMA32: 40 pages in unavailable ranges413client # [ 0.000824] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000414worker2 # [ 0.032144] ACPI: PM-Timer IO Port: 0x608415worker1 # [ 0.003164] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]416client # [ 0.000851] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs417worker2 # [ 0.032155] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])418worker1 # [ 0.003166] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]419worker1 # [ 0.005391] Zone ranges:420lb # [ 0.040045] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ggsnp37phfgq4b11zirffk4vxgicg7h8-nixos-system-lb-test/init regInfo=/nix/store/94h72crvqzbw5sjsl7v7v31gd90khbjb-closure-info/registration console=ttyS0,115200n8 console=tty0421worker2 # [ 0.032185] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23422client # [ 0.000854] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT423worker1 # [ 0.005392] DMA [mem 0x0000000000001000-0x0000000000ffffff]424client # [ 0.002740] found SMP MP-table at [mem 0x000f5450-0x000f545f]425worker2 # [ 0.032188] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)426worker1 # [ 0.005394] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]427client # [ 0.002750] Using GB pages for direct mapping428worker1 # [ 0.005395] Normal empty429worker2 # [ 0.032190] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)430worker1 # [ 0.005396] Device empty431lb # [ 0.040135] Unknown kernel command line parameters "regInfo=/nix/store/94h72crvqzbw5sjsl7v7v31gd90khbjb-closure-info/registration", will be passed to user space.432client # [ 0.002853] RAMDISK: [mem 0x5e370000-0x5ffcffff]433lb # [ 0.040148] random: crng init done434worker1 # [ 0.005396] Movable zone start for each node435worker2 # [ 0.032191] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)436client # [ 0.002862] ACPI: Early table checksum verification disabled437worker1 # [ 0.005397] Early memory node ranges438lb # [ 0.040148] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes439worker2 # [ 0.032192] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)440client # [ 0.002865] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )441worker1 # [ 0.005397] node 0: [mem 0x0000000000001000-0x000000000009efff]442lb # [ 0.042284] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)443worker2 # [ 0.032193] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)444worker1 # [ 0.005398] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]445client # [ 0.002869] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)446worker2 # [ 0.032195] ACPI: Using ACPI (MADT) for SMP configuration information447lb # [ 0.043351] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)448worker1 # [ 0.005400] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]449client # [ 0.002873] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)450lb # [ 0.043382] Fallback order for Node 0: 0451worker2 # [ 0.032196] ACPI: HPET id: 0x8086a201 base: 0xfed00000452worker1 # [ 0.005422] On node 0, zone DMA: 1 pages in unavailable ranges453worker2 # [ 0.032199] TSC deadline timer available454lb # [ 0.043384] Built 1 zonelists, mobility grouping on. Total pages: 393078455client # [ 0.002881] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)456worker1 # [ 0.005678] On node 0, zone DMA: 97 pages in unavailable ranges457worker2 # [ 0.032203] CPU topo: Max. logical packages: 1458lb # [ 0.043385] Policy zone: DMA32459client # [ 0.002883] ACPI: FACS 0x000000005FFE0000 000040460worker2 # [ 0.032204] CPU topo: Max. logical dies: 1461worker1 # [ 0.031846] On node 0, zone DMA32: 40 pages in unavailable ranges462lb # [ 0.045857] mem auto-init: stack:all(zero), heap alloc:on, heap free:off463worker2 # [ 0.032205] CPU topo: Max. dies per package: 1464worker1 # [ 0.032309] ACPI: PM-Timer IO Port: 0x608465client # [ 0.002885] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)466lb # [ 0.048918] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1467worker2 # [ 0.032208] CPU topo: Max. threads per core: 1468worker1 # [ 0.032319] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])469lb # [ 0.052522] allocated 3145728 bytes of page_ext470worker2 # [ 0.032208] CPU topo: Num. cores per package: 1471client # [ 0.002886] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)472worker1 # [ 0.032349] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23473worker2 # [ 0.032209] CPU topo: Num. threads per package: 1474lb # [ 0.062138] ftrace: allocating 48787 entries in 192 pages475worker1 # [ 0.032352] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)476lb # [ 0.062140] ftrace: allocated 192 pages with 2 groups477client # [ 0.002888] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)478worker2 # [ 0.032209] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs479lb # [ 0.062967] Dynamic Preempt: lazy480worker1 # [ 0.032354] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)481worker2 # [ 0.032228] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()482client # [ 0.002889] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)483lb # [ 0.063122] rcu: Preemptible hierarchical RCU implementation.484worker1 # [ 0.032355] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)485lb # [ 0.063123] rcu: RCU event tracing is enabled.486worker2 # [ 0.032258] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]487client # [ 0.002891] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]488worker1 # [ 0.032356] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)489lb # [ 0.063124] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.490client # [ 0.002892] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]491worker2 # [ 0.032259] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]492lb # [ 0.063125] Trampoline variant of Tasks RCU enabled.493worker1 # [ 0.032357] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)494worker2 # [ 0.032261] [mem 0x60000000-0xafffffff] available for PCI devices495client # [ 0.002892] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]496lb # [ 0.063126] Rude variant of Tasks RCU enabled.497worker1 # [ 0.032359] ACPI: Using ACPI (MADT) for SMP configuration information498worker2 # [ 0.032262] Booting paravirtualized kernel on KVM499lb # [ 0.063126] Tracing variant of Tasks RCU enabled.500client # [ 0.002893] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]501worker1 # [ 0.032360] ACPI: HPET id: 0x8086a201 base: 0xfed00000502worker1 # [ 0.032364] TSC deadline timer available503client # [ 0.002893] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]504lb # [ 0.063126] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.505worker2 # [ 0.032265] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns506worker1 # [ 0.032368] CPU topo: Max. logical packages: 1507lb # [ 0.063127] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1508worker1 # [ 0.032369] CPU topo: Max. logical dies: 1509worker2 # [ 0.036763] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1510client # [ 0.002894] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]511worker1 # [ 0.032369] CPU topo: Max. dies per package: 1512lb # [ 0.063181] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.513client # [ 0.002894] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]514worker2 # [ 0.038897] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152515worker1 # [ 0.032373] CPU topo: Max. threads per core: 1516client # [ 0.003126] No NUMA configuration found517worker2 # [ 0.038944] kvm-guest: PV spinlocks disabled, single CPU518worker1 # [ 0.032373] CPU topo: Num. cores per package: 1519lb # [ 0.063183] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.520client # [ 0.003127] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]521worker1 # [ 0.032373] CPU topo: Num. threads per package: 1522client # [ 0.003129] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]523worker1 # [ 0.032374] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs524client # [ 0.005361] Zone ranges:525lb # [ 0.063184] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.526lb # [ 0.067507] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16527client # [ 0.005362] DMA [mem 0x0000000000001000-0x0000000000ffffff]528worker1 # [ 0.032394] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()529lb # [ 0.067795] rcu: srcu_init: Setting srcu_struct sizes based on contention.530client # [ 0.005364] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]531worker1 # [ 0.032424] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]532client # [ 0.005365] Normal empty533worker2 # [ 0.038946] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/gvbzy0g4j2sf3am2wkwfga8lrrdci67y-nixos-system-worker2-test/init regInfo=/nix/store/788l1h41glyfk32if77z9ksf8qk2j16n-closure-info/registration console=ttyS0,115200n8 console=tty0534client # [ 0.005366] Device empty535lb # [ 0.067801] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns536client # [ 0.005366] Movable zone start for each node537worker1 # [ 0.032426] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]538client # [ 0.005367] Early memory node ranges539worker1 # [ 0.032427] [mem 0x60000000-0xafffffff] available for PCI devices540lb # [ 0.067906] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)541worker2 # [ 0.039042] Unknown kernel command line parameters "regInfo=/nix/store/788l1h41glyfk32if77z9ksf8qk2j16n-closure-info/registration", will be passed to user space.542client # [ 0.005367] node 0: [mem 0x0000000000001000-0x000000000009efff]543worker1 # [ 0.032429] Booting paravirtualized kernel on KVM544worker2 # [ 0.039054] random: crng init done545lb # [ 0.071404] Console: colour VGA+ 80x25546client # [ 0.005368] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]547lb # [ 0.071406] printk: legacy console [tty0] enabled548worker1 # [ 0.032431] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns549worker2 # [ 0.039054] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes550lb # [ 0.100874] printk: legacy console [ttyS0] enabled551client # [ 0.005369] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]552worker1 # [ 0.036920] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1553worker2 # [ 0.041315] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)554lb # [ 0.199493] ACPI: Core revision 20250807555client # [ 0.005390] On node 0, zone DMA: 1 pages in unavailable ranges556worker1 # [ 0.039250] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152557client # [ 0.005653] On node 0, zone DMA: 97 pages in unavailable ranges558worker2 # [ 0.042359] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)559worker1 # [ 0.039296] kvm-guest: PV spinlocks disabled, single CPU560worker2 # [ 0.042390] Fallback order for Node 0: 0561client # [ 0.031656] On node 0, zone DMA32: 40 pages in unavailable ranges562lb # [ 0.200352] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns563client # [ 0.032114] ACPI: PM-Timer IO Port: 0x608564worker2 # [ 0.042393] Built 1 zonelists, mobility grouping on. Total pages: 393078565worker2 # [ 0.042394] Policy zone: DMA32566client # [ 0.032125] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])567lb # [ 0.201913] APIC: Switch to symmetric I/O mode setup568worker2 # [ 0.044844] mem auto-init: stack:all(zero), heap alloc:on, heap free:off569lb # [ 0.202860] x2apic enabled570client # [ 0.032153] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23571worker2 # [ 0.047938] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1572client # [ 0.032156] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)573lb # [ 0.203619] APIC: Switched APIC routing to: physical x2apic574worker2 # [ 0.051311] allocated 3145728 bytes of page_ext575worker1 # [ 0.039298] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/dyab6c8i372v5waficb63zp79vnsaxrg-nixos-system-worker1-test/init regInfo=/nix/store/a01b4wkhhfw171hhg3lf8p5ygrq1fswf-closure-info/registration console=ttyS0,115200n8 console=tty0576client # [ 0.032157] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)577worker2 # [ 0.061562] ftrace: allocating 48787 entries in 192 pages578client # [ 0.032158] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)579worker2 # [ 0.061564] ftrace: allocated 192 pages with 2 groups580worker2 # [ 0.062586] Dynamic Preempt: lazy581lb # [ 0.205431] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1582client # [ 0.032160] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)583worker1 # [ 0.039393] Unknown kernel command line parameters "regInfo=/nix/store/a01b4wkhhfw171hhg3lf8p5ygrq1fswf-closure-info/registration", will be passed to user space.584worker2 # [ 0.062749] rcu: Preemptible hierarchical RCU implementation.585worker1 # [ 0.039405] random: crng init done586client # [ 0.032160] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)587worker2 # [ 0.062749] rcu: RCU event tracing is enabled.588worker1 # [ 0.039406] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes589lb # [ 0.206382] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns590client # [ 0.032163] ACPI: Using ACPI (MADT) for SMP configuration information591worker2 # [ 0.062750] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.592client # [ 0.032164] ACPI: HPET id: 0x8086a201 base: 0xfed00000593worker2 # [ 0.062752] Trampoline variant of Tasks RCU enabled.594worker1 # [ 0.041509] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)595client # [ 0.032167] TSC deadline timer available596lb # [ 0.207989] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)597worker2 # [ 0.062752] Rude variant of Tasks RCU enabled.598worker1 # [ 0.042557] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)599client # [ 0.032171] CPU topo: Max. logical packages: 1600worker2 # [ 0.062752] Tracing variant of Tasks RCU enabled.601worker1 # [ 0.042587] Fallback order for Node 0: 0602client # [ 0.032172] CPU topo: Max. logical dies: 1603lb # [ 0.209074] x86/cpu: User Mode Instruction Prevention (UMIP) activated604worker2 # [ 0.062753] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.605client # [ 0.032172] CPU topo: Max. dies per package: 1606worker1 # [ 0.042590] Built 1 zonelists, mobility grouping on. Total pages: 393078607client # [ 0.032175] CPU topo: Max. threads per core: 1608worker1 # [ 0.042592] Policy zone: DMA32609lb # [ 0.210154] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127610worker2 # [ 0.062754] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1611client # [ 0.032176] CPU topo: Num. cores per package: 1612worker1 # [ 0.045047] mem auto-init: stack:all(zero), heap alloc:on, heap free:off613lb # [ 0.210988] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0614client # [ 0.032176] CPU topo: Num. threads per package: 1615worker2 # [ 0.062776] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.616worker1 # [ 0.048143] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1617client # [ 0.032177] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs618worker1 # [ 0.051575] allocated 3145728 bytes of page_ext619worker2 # [ 0.062778] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.620client # [ 0.032193] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()621lb # [ 0.211991] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto622worker1 # [ 0.061699] ftrace: allocating 48787 entries in 192 pages623worker1 # [ 0.061701] ftrace: allocated 192 pages with 2 groups624worker2 # [ 0.062778] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.625client # [ 0.032229] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]626worker1 # [ 0.062545] Dynamic Preempt: lazy627lb # [ 0.212987] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl628worker2 # [ 0.067165] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16629worker1 # [ 0.062702] rcu: Preemptible hierarchical RCU implementation.630client # [ 0.032231] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]631worker1 # [ 0.062703] rcu: RCU event tracing is enabled.632worker2 # [ 0.067428] rcu: srcu_init: Setting srcu_struct sizes based on contention.633lb # [ 0.213987] Transient Scheduler Attacks: Vulnerable: No microcode634client # [ 0.032232] [mem 0x60000000-0xafffffff] available for PCI devices635worker1 # [ 0.062704] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.636lb # [ 0.214987] Spectre V2 : Mitigation: Enhanced / Automatic IBRS637client # [ 0.032234] Booting paravirtualized kernel on KVM638worker2 # [ 0.067434] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns639worker1 # [ 0.062705] Trampoline variant of Tasks RCU enabled.640worker1 # [ 0.062706] Rude variant of Tasks RCU enabled.641lb # [ 0.215987] Speculative Return Stack Overflow: Mitigation: Safe RET642client # [ 0.032237] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns643worker2 # [ 0.067538] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)644worker1 # [ 0.062706] Tracing variant of Tasks RCU enabled.645worker2 # [ 0.071061] Console: colour VGA+ 80x25646client # [ 0.036719] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1647worker1 # [ 0.062707] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.648worker2 # [ 0.071063] printk: legacy console [tty0] enabled649client # [ 0.039011] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152650worker2 # [ 0.100616] printk: legacy console [ttyS0] enabled651worker1 # [ 0.062707] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1652client # [ 0.039055] kvm-guest: PV spinlocks disabled, single CPU653worker2 # [ 0.196979] ACPI: Core revision 20250807654worker1 # [ 0.062729] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.655lb # [ 0.216987] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization656worker1 # [ 0.062730] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.657worker2 # [ 0.197841] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns658lb # [ 0.218994] Spectre V2 : Enabling IBPB for BPF659worker2 # [ 0.199381] APIC: Switch to symmetric I/O mode setup660worker1 # [ 0.062731] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.661worker1 # [ 0.067146] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16662lb # [ 0.219695] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier663worker2 # [ 0.200379] x2apic enabled664client # [ 0.039057] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/g90cn3cn0g95ry9pa0dhwf3r60ijvjrh-nixos-system-client-test/init regInfo=/nix/store/s9k7mpd3day04qzf19r71x8dq0q2ycs2-closure-info/registration console=ttyS0,115200n8 console=tty0665worker1 # [ 0.067417] rcu: srcu_init: Setting srcu_struct sizes based on contention.666lb # [ 0.220988] active return thunk: srso_alias_return_thunk667worker2 # [ 0.201151] APIC: Switched APIC routing to: physical x2apic668worker1 # [ 0.067425] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns669client # [ 0.039154] Unknown kernel command line parameters "regInfo=/nix/store/s9k7mpd3day04qzf19r71x8dq0q2ycs2-closure-info/registration", will be passed to user space.670lb # [ 0.222004] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'671client # [ 0.039166] random: crng init done672lb # [ 0.222987] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'673worker1 # [ 0.067528] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)674worker2 # [ 0.203000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1675client # [ 0.039167] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes676worker1 # [ 0.071080] Console: colour VGA+ 80x25677lb # [ 0.223987] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'678worker1 # [ 0.071083] printk: legacy console [tty0] enabled679client # [ 0.041257] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)680worker1 # [ 0.100574] printk: legacy console [ttyS0] enabled681worker2 # [ 0.203977] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns682lb # [ 0.224988] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'683client # [ 0.042294] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)684client # [ 0.042325] Fallback order for Node 0: 0685worker1 # [ 0.195717] ACPI: Core revision 20250807686lb # [ 0.225987] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'687worker2 # [ 0.205636] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)688client # [ 0.042328] Built 1 zonelists, mobility grouping on. Total pages: 393078689client # [ 0.042329] Policy zone: DMA32690lb # [ 0.226987] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'691worker1 # [ 0.196590] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns692worker2 # [ 0.206721] x86/cpu: User Mode Instruction Prevention (UMIP) activated693client # [ 0.045118] mem auto-init: stack:all(zero), heap alloc:on, heap free:off694worker1 # [ 0.198101] APIC: Switch to symmetric I/O mode setup695lb # [ 0.227987] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'696client # [ 0.048157] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1697worker2 # [ 0.207800] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127698worker1 # [ 0.199075] x2apic enabled699client # [ 0.051586] allocated 3145728 bytes of page_ext700lb # [ 0.228987] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256701worker2 # [ 0.209298] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0702client # [ 0.061717] ftrace: allocating 48787 entries in 192 pages703worker1 # [ 0.199829] APIC: Switched APIC routing to: physical x2apic704client # [ 0.061719] ftrace: allocated 192 pages with 2 groups705lb # [ 0.229987] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64706client # [ 0.062565] Dynamic Preempt: lazy707worker2 # [ 0.209637] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto708lb # [ 0.230987] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512709client # [ 0.062704] rcu: Preemptible hierarchical RCU implementation.710client # [ 0.062705] rcu: RCU event tracing is enabled.711worker1 # [ 0.201616] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1712lb # [ 0.231987] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024713worker2 # [ 0.211633] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl714client # [ 0.062706] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.715lb # [ 0.232987] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8716client # [ 0.062707] Trampoline variant of Tasks RCU enabled.717worker1 # [ 0.202586] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns718worker2 # [ 0.212634] Transient Scheduler Attacks: Vulnerable: No microcode719client # [ 0.062708] Rude variant of Tasks RCU enabled.720client # [ 0.062708] Tracing variant of Tasks RCU enabled.721lb # [ 0.233892] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.722worker2 # [ 0.213632] Spectre V2 : Mitigation: Enhanced / Automatic IBRS723worker1 # [ 0.204216] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)724client # [ 0.062709] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.725worker2 # [ 0.214633] Speculative Return Stack Overflow: Mitigation: Safe RET726worker1 # [ 0.205303] x86/cpu: User Mode Instruction Prevention (UMIP) activated727client # [ 0.062709] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1728worker1 # [ 0.206361] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127729client # [ 0.062719] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.730worker1 # [ 0.207213] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0731client # [ 0.062720] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.732worker2 # [ 0.215633] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization733client # [ 0.062721] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.734worker1 # [ 0.208217] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto735client # [ 0.067083] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16736worker2 # [ 0.217640] Spectre V2 : Enabling IBPB for BPF737client # [ 0.067391] rcu: srcu_init: Setting srcu_struct sizes based on contention.738worker1 # [ 0.210213] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl739worker2 # [ 0.218634] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier740worker1 # [ 0.211213] Transient Scheduler Attacks: Vulnerable: No microcode741client # [ 0.067400] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns742worker2 # [ 0.219633] active return thunk: srso_alias_return_thunk743worker1 # [ 0.212213] Spectre V2 : Mitigation: Enhanced / Automatic IBRS744client # [ 0.067499] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)745worker2 # [ 0.220655] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'746client # [ 0.071057] Console: colour VGA+ 80x25747worker1 # [ 0.213213] Speculative Return Stack Overflow: Mitigation: Safe RET748client # [ 0.071060] printk: legacy console [tty0] enabled749worker2 # [ 0.221633] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'750client # [ 0.100500] printk: legacy console [ttyS0] enabled751worker2 # [ 0.223405] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'752client # [ 0.197182] ACPI: Core revision 20250807753worker2 # [ 0.224422] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'754client # [ 0.198073] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns755worker2 # [ 0.225439] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'756worker1 # [ 0.214213] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization757client # [ 0.199619] APIC: Switch to symmetric I/O mode setup758worker1 # [ 0.216221] Spectre V2 : Enabling IBPB for BPF759worker2 # [ 0.226372] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'760client # [ 0.200627] x2apic enabled761worker1 # [ 0.217214] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier762worker2 # [ 0.227437] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'763client # [ 0.201413] APIC: Switched APIC routing to: physical x2apic764worker1 # [ 0.218214] active return thunk: srso_alias_return_thunk765worker2 # [ 0.228633] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256766worker1 # [ 0.219233] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'767worker2 # [ 0.229632] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64768client # [ 0.203267] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1769worker1 # [ 0.220213] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'770worker2 # [ 0.230633] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512771client # [ 0.204269] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns772worker1 # [ 0.221213] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'773worker2 # [ 0.231633] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024774worker1 # [ 0.222213] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'775worker2 # [ 0.232632] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8776client # [ 0.205932] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)777worker1 # [ 0.223213] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'778client # [ 0.207018] x86/cpu: User Mode Instruction Prevention (UMIP) activated779worker2 # [ 0.233632] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.780worker1 # [ 0.224213] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'781client # [ 0.208841] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127782worker1 # [ 0.225213] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'783client # [ 0.209656] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0784worker1 # [ 0.226213] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256785worker1 # [ 0.227923] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64786client # [ 0.210735] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto787worker1 # [ 0.228921] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512788client # [ 0.211929] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl789worker1 # [ 0.229213] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024790client # [ 0.212931] Transient Scheduler Attacks: Vulnerable: No microcode791worker1 # [ 0.230213] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8792client # [ 0.213929] Spectre V2 : Mitigation: Enhanced / Automatic IBRS793worker1 # [ 0.231213] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.794client # [ 0.214929] Speculative Return Stack Overflow: Mitigation: Safe RET795lb # [ 0.268094] Freeing SMP alternatives memory: 44K796client # [ 0.215929] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization797lb # [ 0.268856] pid_max: default: 32768 minimum: 301798client # [ 0.217938] Spectre V2 : Enabling IBPB for BPF799lb # [ 0.269609] LSM: initializing lsm=capability,landlock,yama,bpf,ima800lb # [ 0.270089] landlock: Up and running.801client # [ 0.218685] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier802lb # [ 0.270988] Yama: becoming mindful.803client # [ 0.219930] active return thunk: srso_alias_return_thunk804lb # [ 0.272199] LSM support for eBPF active805client # [ 0.220951] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'806lb # [ 0.273072] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)807client # [ 0.221929] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'808lb # [ 0.274032] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)809client # [ 0.222929] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'810client # [ 0.223929] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'811client # [ 0.224929] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'812lb # [ 0.276585] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)813client # [ 0.225929] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'814lb # [ 0.277523] Performance Events: Fam17h+ core perfctr, AMD PMU driver.815lb # [ 0.277992] ... version: 2816client # [ 0.226929] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'817lb # [ 0.278729] ... bit width: 48818client # [ 0.228930] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256819lb # [ 0.278990] ... generic counters: 6820client # [ 0.229930] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64821lb # [ 0.279699] ... generic bitmap: 000000000000003f822client # [ 0.230929] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512823lb # [ 0.280001] ... fixed-purpose counters: 0824lb # [ 0.280738] ... fixed-purpose bitmap: 0000000000000000825client # [ 0.231929] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024826lb # [ 0.280989] ... value mask: 0000ffffffffffff827client # [ 0.232929] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8828lb # [ 0.281901] ... max period: 00007fffffffffff829client # [ 0.233929] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.830lb # [ 0.282704] ... global_ctrl mask: 000000000000003f831lb # [ 0.283100] signal: max sigframe size: 3376832lb # [ 0.283932] rcu: Hierarchical SRCU implementation.833lb # [ 0.284603] rcu: Max phase no-delay instances is 400.834worker2 # [ 0.268080] Freeing SMP alternatives memory: 44K835worker2 # [ 0.268636] pid_max: default: 32768 minimum: 301836worker2 # [ 0.269737] LSM: initializing lsm=capability,landlock,yama,bpf,ima837worker2 # [ 0.270780] landlock: Up and running.838worker2 # [ 0.271633] Yama: becoming mindful.839lb # [ 0.289499] smp: Bringing up secondary CPUs ...840worker2 # [ 0.272465] LSM support for eBPF active841lb # [ 0.290003] smp: Brought up 1 node, 1 CPU842worker2 # [ 0.273186] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)843lb # [ 0.290689] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)844worker2 # [ 0.274647] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)845lb # [ 0.291176] Memory: 1454584K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109316K reserved, 0K cma-reserved)846worker1 # [ 0.266456] Freeing SMP alternatives memory: 44K847worker1 # [ 0.267217] pid_max: default: 32768 minimum: 301848lb # [ 0.292270] devtmpfs: initialized849worker2 # [ 0.276971] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)850lb # [ 0.293085] x86/mm: Memory block size: 128MB851worker1 # [ 0.268032] LSM: initializing lsm=capability,landlock,yama,bpf,ima852worker1 # [ 0.269035] landlock: Up and running.853worker2 # [ 0.278194] Performance Events: Fam17h+ core perfctr, AMD PMU driver.854worker1 # [ 0.269634] Yama: becoming mindful.855worker2 # [ 0.278642] ... version: 2856worker1 # [ 0.270425] LSM support for eBPF active857lb # [ 0.294849] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)858worker2 # [ 0.279363] ... bit width: 48859worker2 # [ 0.279635] ... generic counters: 6860worker1 # [ 0.271334] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)861lb # [ 0.295879] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).862worker2 # [ 0.280346] ... generic bitmap: 000000000000003f863lb # [ 0.297068] pinctrl core: initialized pinctrl subsystem864worker1 # [ 0.272249] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)865worker2 # [ 0.280635] ... fixed-purpose counters: 0866lb # [ 0.298138] PM: RTC time: 04:11:15, date: 2026-09-23867worker2 # [ 0.281351] ... fixed-purpose bitmap: 0000000000000000868worker2 # [ 0.281635] ... value mask: 0000ffffffffffff869worker1 # [ 0.274784] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)870worker2 # [ 0.282558] ... max period: 00007fffffffffff871worker2 # [ 0.283356] ... global_ctrl mask: 000000000000003f872worker1 # [ 0.275777] Performance Events: Fam17h+ core perfctr, AMD PMU driver.873worker2 # [ 0.283732] signal: max sigframe size: 3376874worker1 # [ 0.276218] ... version: 2875lb # [ 0.302095] NET: Registered PF_NETLINK/PF_ROUTE protocol family876worker1 # [ 0.276922] ... bit width: 48877worker2 # [ 0.284508] rcu: Hierarchical SRCU implementation.878worker1 # [ 0.277216] ... generic counters: 6879worker2 # [ 0.285217] rcu: Max phase no-delay instances is 400.880lb # [ 0.303477] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations881worker1 # [ 0.277886] ... generic bitmap: 000000000000003f882lb # [ 0.304012] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations883worker1 # [ 0.278220] ... fixed-purpose counters: 0884worker1 # [ 0.278975] ... fixed-purpose bitmap: 0000000000000000885lb # [ 0.305265] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations886worker1 # [ 0.279216] ... value mask: 0000ffffffffffff887lb # [ 0.305999] audit: initializing netlink subsys (disabled)888worker1 # [ 0.280095] ... max period: 00007fffffffffff889worker1 # [ 0.280897] ... global_ctrl mask: 000000000000003f890lb # [ 0.307208] thermal_sys: Registered thermal governor 'fair_share'891worker2 # [ 0.290212] smp: Bringing up secondary CPUs ...892worker1 # [ 0.281333] signal: max sigframe size: 3376893worker2 # [ 0.290649] smp: Brought up 1 node, 1 CPU894lb # [ 0.307210] thermal_sys: Registered thermal governor 'bang_bang'895worker1 # [ 0.282085] rcu: Hierarchical SRCU implementation.896lb # [ 0.307991] thermal_sys: Registered thermal governor 'step_wise'897worker2 # [ 0.291303] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)898worker1 # [ 0.282767] rcu: Max phase no-delay instances is 400.899lb # [ 0.308913] thermal_sys: Registered thermal governor 'user_space'900client # [ 0.268680] Freeing SMP alternatives memory: 44K901worker2 # [ 0.291834] Memory: 1454616K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109312K reserved, 0K cma-reserved)902lb # [ 0.309711] audit: type=2000 audit(1790136676.463:1): state=initialized audit_enabled=0 res=1903client # [ 0.268933] pid_max: default: 32768 minimum: 301904worker2 # [ 0.292941] devtmpfs: initialized905lb # [ 0.310992] thermal_sys: Registered thermal governor 'power_allocator'906client # [ 0.270028] LSM: initializing lsm=capability,landlock,yama,bpf,ima907lb # [ 0.311019] cpuidle: using governor menu908worker2 # [ 0.293681] x86/mm: Memory block size: 128MB909client # [ 0.271051] landlock: Up and running.910client # [ 0.271930] Yama: becoming mindful.911lb # [ 0.313204] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5912client # [ 0.272981] LSM support for eBPF active913worker1 # [ 0.287894] smp: Bringing up secondary CPUs ...914worker2 # [ 0.295457] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)915worker1 # [ 0.288229] smp: Brought up 1 node, 1 CPU916client # [ 0.273718] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)917lb # [ 0.314218] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]918worker2 # [ 0.296537] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).919worker1 # [ 0.288865] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)920lb # [ 0.314993] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry921client # [ 0.274939] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)922worker2 # [ 0.297729] pinctrl core: initialized pinctrl subsystem923lb # [ 0.316002] PCI: Using configuration type 1 for base access924worker2 # [ 0.298848] PM: RTC time: 04:11:15, date: 2026-09-23925worker1 # [ 0.289352] Memory: 1454604K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)926worker1 # [ 0.290510] devtmpfs: initialized927lb # [ 0.317111] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.928client # [ 0.277151] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)929worker1 # [ 0.291282] x86/mm: Memory block size: 128MB930client # [ 0.278464] Performance Events: Fam17h+ core perfctr, AMD PMU driver.931client # [ 0.278938] ... version: 2932worker2 # [ 0.302317] NET: Registered PF_NETLINK/PF_ROUTE protocol family933worker1 # [ 0.293023] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)934client # [ 0.279665] ... bit width: 48935client # [ 0.279932] ... generic counters: 6936worker2 # [ 0.303139] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations937worker1 # [ 0.294094] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).938client # [ 0.280645] ... generic bitmap: 000000000000003f939worker1 # [ 0.295299] pinctrl core: initialized pinctrl subsystem940worker2 # [ 0.303659] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations941client # [ 0.280932] ... fixed-purpose counters: 0942worker1 # [ 0.296461] PM: RTC time: 04:11:15, date: 2026-09-23943client # [ 0.281660] ... fixed-purpose bitmap: 0000000000000000944lb # [ 0.324276] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages945client # [ 0.281931] ... value mask: 0000ffffffffffff946worker2 # [ 0.304903] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations947client # [ 0.282786] ... max period: 00007fffffffffff948lb # [ 0.324990] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page949worker2 # [ 0.305702] audit: initializing netlink subsys (disabled)950client # [ 0.283587] ... global_ctrl mask: 000000000000003f951client # [ 0.284043] signal: max sigframe size: 3376952worker2 # [ 0.307065] thermal_sys: Registered thermal governor 'fair_share'953worker1 # [ 0.300000] NET: Registered PF_NETLINK/PF_ROUTE protocol family954client # [ 0.284800] rcu: Hierarchical SRCU implementation.955worker2 # [ 0.307067] thermal_sys: Registered thermal governor 'bang_bang'956client # [ 0.285560] rcu: Max phase no-delay instances is 400.957worker1 # [ 0.301432] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations958worker2 # [ 0.307637] thermal_sys: Registered thermal governor 'step_wise'959worker2 # [ 0.308575] thermal_sys: Registered thermal governor 'user_space'960worker1 # [ 0.302239] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations961lb # [ 0.329991] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages962worker2 # [ 0.309360] audit: type=2000 audit(1790136676.488:1): state=initialized audit_enabled=0 res=1963lb # [ 0.330990] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page964worker1 # [ 0.303488] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations965worker2 # [ 0.310639] thermal_sys: Registered thermal governor 'power_allocator'966worker1 # [ 0.304226] audit: initializing netlink subsys (disabled)967worker2 # [ 0.310657] cpuidle: using governor menu968worker1 # [ 0.305350] thermal_sys: Registered thermal governor 'fair_share'969client # [ 0.290507] smp: Bringing up secondary CPUs ...970worker2 # [ 0.312936] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5971worker1 # [ 0.305352] thermal_sys: Registered thermal governor 'bang_bang'972client # [ 0.290947] smp: Brought up 1 node, 1 CPU973worker1 # [ 0.306217] thermal_sys: Registered thermal governor 'step_wise'974client # [ 0.291635] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)975worker2 # [ 0.313898] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]976worker1 # [ 0.307145] thermal_sys: Registered thermal governor 'user_space'977worker2 # [ 0.314639] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry978worker1 # [ 0.307956] audit: type=2000 audit(1790136676.497:1): state=initialized audit_enabled=0 res=1979worker2 # [ 0.315648] PCI: Using configuration type 1 for base access980client # [ 0.292199] Memory: 1454600K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)981worker1 # [ 0.309218] thermal_sys: Registered thermal governor 'power_allocator'982client # [ 0.293190] devtmpfs: initialized983worker1 # [ 0.309231] cpuidle: using governor menu984lb # [ 0.339621] ACPI: Added _OSI(Module Device)985worker2 # [ 0.316757] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.986client # [ 0.293995] x86/mm: Memory block size: 128MB987lb # [ 0.339991] ACPI: Added _OSI(Processor Device)988worker1 # [ 0.311441] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5989lb # [ 0.340801] ACPI: Added _OSI(Processor Aggregator Device)990client # [ 0.295761] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)991worker1 # [ 0.312444] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]992client # [ 0.296875] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).993worker1 # [ 0.313219] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry994client # [ 0.298025] pinctrl core: initialized pinctrl subsystem995worker1 # [ 0.314227] PCI: Using configuration type 1 for base access996worker2 # [ 0.321891] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages997client # [ 0.299167] PM: RTC time: 04:11:15, date: 2026-09-23998worker2 # [ 0.322636] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page999worker1 # [ 0.315321] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1000client # [ 0.302829] NET: Registered PF_NETLINK/PF_ROUTE protocol family1001lb # [ 0.350268] ACPI: 1 ACPI AML tables successfully acquired and loaded1002worker2 # [ 0.327636] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1003client # [ 0.304408] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations1004worker1 # [ 0.320486] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1005worker2 # [ 0.328635] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1006lb # [ 0.351925] ACPI: Interpreter enabled1007worker1 # [ 0.321216] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1008client # [ 0.304956] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations1009lb # [ 0.352438] ACPI: PM: (supports S0 S3 S4 S5)1010client # [ 0.306205] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations1011client # [ 0.306941] audit: initializing netlink subsys (disabled)1012lb # [ 0.354991] ACPI: Using IOAPIC for interrupt routing1013client # [ 0.308171] thermal_sys: Registered thermal governor 'fair_share'1014lb # [ 0.355898] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1015client # [ 0.308174] thermal_sys: Registered thermal governor 'bang_bang'1016worker1 # [ 0.326217] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1017client # [ 0.308933] thermal_sys: Registered thermal governor 'step_wise'1018worker1 # [ 0.327215] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1019client # [ 0.309869] thermal_sys: Registered thermal governor 'user_space'1020lb # [ 0.358990] PCI: Using E820 reservations for host bridge windows1021client # [ 0.310677] audit: type=2000 audit(1790136676.518:1): state=initialized audit_enabled=0 res=11022lb # [ 0.360081] ACPI: Enabled 2 GPEs in block 00 to 3F1023client # [ 0.311935] thermal_sys: Registered thermal governor 'power_allocator'1024worker2 # [ 0.339026] ACPI: Added _OSI(Module Device)1025client # [ 0.311948] cpuidle: using governor menu1026worker2 # [ 0.339637] ACPI: Added _OSI(Processor Device)1027worker2 # [ 0.340374] ACPI: Added _OSI(Processor Aggregator Device)1028client # [ 0.314189] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.51029client # [ 0.315146] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]1030client # [ 0.315935] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1031client # [ 0.316944] PCI: Using configuration type 1 for base access1032worker1 # [ 0.337587] ACPI: Added _OSI(Module Device)1033worker1 # [ 0.338217] ACPI: Added _OSI(Processor Device)1034client # [ 0.318058] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1035lb # [ 0.368722] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1036worker1 # [ 0.338956] ACPI: Added _OSI(Processor Aggregator Device)1037lb # [ 0.369260] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1038worker2 # [ 0.349191] ACPI: 1 ACPI AML tables successfully acquired and loaded1039lb # [ 0.370067] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1040lb # [ 0.371113] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1041client # [ 0.323017] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1042lb # [ 0.372347] PCI host bridge to bus 0000:001043client # [ 0.323933] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1044lb # [ 0.372995] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1045worker1 # [ 0.345137] ACPI: 1 ACPI AML tables successfully acquired and loaded1046worker2 # [ 0.353037] ACPI: Interpreter enabled1047lb # [ 0.373990] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1048worker2 # [ 0.353650] ACPI: PM: (supports S0 S3 S4 S5)1049worker2 # [ 0.354345] ACPI: Using IOAPIC for interrupt routing1050lb # [ 0.374990] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1051lb # [ 0.375990] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1052worker2 # [ 0.354700] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1053client # [ 0.328933] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1054worker2 # [ 0.355636] PCI: Using E820 reservations for host bridge windows1055lb # [ 0.376990] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1056client # [ 0.329932] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1057lb # [ 0.377989] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1058worker1 # [ 0.351622] ACPI: Interpreter enabled1059lb # [ 0.378991] pci_bus 0000:00: root bus resource [bus 00-ff]1060worker1 # [ 0.352231] ACPI: PM: (supports S0 S3 S4 S5)1061worker2 # [ 0.358678] ACPI: Enabled 2 GPEs in block 00 to 3F1062lb # [ 0.379980] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1063worker1 # [ 0.352940] ACPI: Using IOAPIC for interrupt routing1064worker1 # [ 0.353281] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1065lb # [ 0.381415] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1066worker1 # [ 0.354216] PCI: Using E820 reservations for host bridge windows1067worker1 # [ 0.357327] ACPI: Enabled 2 GPEs in block 00 to 3F1068client # [ 0.340322] ACPI: Added _OSI(Module Device)1069client # [ 0.340934] ACPI: Added _OSI(Processor Device)1070worker2 # [ 0.367282] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1071client # [ 0.341652] ACPI: Added _OSI(Processor Aggregator Device)1072worker2 # [ 0.367642] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1073lb # [ 0.384052] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1074worker2 # [ 0.368885] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1075lb # [ 0.385002] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1076worker2 # [ 0.369757] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1077lb # [ 0.385973] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1078worker2 # [ 0.371014] PCI host bridge to bus 0000:001079lb # [ 0.386963] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1080worker2 # [ 0.371640] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1081worker1 # [ 0.365958] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1082worker2 # [ 0.372636] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1083worker1 # [ 0.366222] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1084client # [ 0.347874] ACPI: 1 ACPI AML tables successfully acquired and loaded1085lb # [ 0.388773] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1086worker2 # [ 0.373635] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1087worker1 # [ 0.367818] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1088worker2 # [ 0.374635] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1089worker1 # [ 0.368339] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1090worker2 # [ 0.375635] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1091lb # [ 0.391007] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1092worker1 # [ 0.369580] PCI host bridge to bus 0000:001093worker2 # [ 0.376636] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1094lb # [ 0.391870] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1095worker1 # [ 0.370220] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1096worker2 # [ 0.377636] pci_bus 0000:00: root bus resource [bus 00-ff]1097lb # [ 0.392736] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1098worker1 # [ 0.371216] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1099client # [ 0.354354] ACPI: Interpreter enabled1100lb # [ 0.392995] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1101worker2 # [ 0.378601] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1102client # [ 0.354949] ACPI: PM: (supports S0 S3 S4 S5)1103worker1 # [ 0.372216] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1104client # [ 0.355668] ACPI: Using IOAPIC for interrupt routing1105worker2 # [ 0.380078] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1106worker1 # [ 0.373215] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1107lb # [ 0.394645] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1108worker1 # [ 0.374216] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1109worker1 # [ 0.375216] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1110client # [ 0.357984] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1111lb # [ 0.397006] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1112worker1 # [ 0.376221] pci_bus 0000:00: root bus resource [bus 00-ff]1113client # [ 0.358933] PCI: Using E820 reservations for host bridge windows1114lb # [ 0.397853] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1115worker1 # [ 0.377165] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1116lb # [ 0.398794] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1117worker1 # [ 0.378702] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1118client # [ 0.362017] ACPI: Enabled 2 GPEs in block 00 to 3F1119lb # [ 0.400418] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1120worker2 # [ 0.382725] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1121worker2 # [ 0.383647] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1122worker2 # [ 0.384565] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1123lb # [ 0.402998] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1124worker2 # [ 0.385736] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1125lb # [ 0.403996] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1126lb # [ 0.404941] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1127client # [ 0.369148] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1128worker2 # [ 0.387554] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1129worker1 # [ 0.381304] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1130lb # [ 0.406421] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1131client # [ 0.369938] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1132worker1 # [ 0.382228] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1133worker1 # [ 0.383167] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1134client # [ 0.371011] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1135worker2 # [ 0.389673] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1136lb # [ 0.408008] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1137worker2 # [ 0.390523] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1138client # [ 0.372054] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1139worker1 # [ 0.384199] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1140lb # [ 0.408834] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1141client # [ 0.373286] PCI host bridge to bus 0000:001142worker2 # [ 0.391358] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1143lb # [ 0.409706] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1144worker2 # [ 0.391641] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1145client # [ 0.373937] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1146worker1 # [ 0.385902] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1147client # [ 0.374932] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1148lb # [ 0.409996] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1149worker2 # [ 0.393140] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1150client # [ 0.375932] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1151worker1 # [ 0.388256] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1152lb # [ 0.411532] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1153client # [ 0.376932] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1154worker1 # [ 0.389064] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1155client # [ 0.377932] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1156worker2 # [ 0.394670] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1157worker1 # [ 0.389919] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1158lb # [ 0.413012] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1159worker2 # [ 0.395565] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1160client # [ 0.378932] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1161worker1 # [ 0.390222] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1162lb # [ 0.413946] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1163client # [ 0.379933] pci_bus 0000:00: root bus resource [bus 00-ff]1164worker2 # [ 0.396355] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1165lb # [ 0.415394] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1166client # [ 0.380954] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1167worker1 # [ 0.391717] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1168worker2 # [ 0.397190] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1169client # [ 0.382398] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1170lb # [ 0.417011] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1171worker1 # [ 0.393224] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1172worker1 # [ 0.394032] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1173lb # [ 0.417973] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1174worker1 # [ 0.394945] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1175worker2 # [ 0.399944] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1176lb # [ 0.419566] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1177worker2 # [ 0.401284] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1178worker1 # [ 0.395787] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1179worker2 # [ 0.401658] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1180lb # [ 0.421011] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1181lb # [ 0.422012] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1182worker2 # [ 0.403208] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1183client # [ 0.385066] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1184lb # [ 0.423524] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1185worker1 # [ 0.398254] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1186worker2 # [ 0.404670] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1187client # [ 0.385946] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1188worker1 # [ 0.399867] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1189worker2 # [ 0.405479] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1190lb # [ 0.425012] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1191client # [ 0.386922] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1192worker1 # [ 0.400237] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1193worker2 # [ 0.406335] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1194worker2 # [ 0.406641] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1195lb # [ 0.425965] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1196client # [ 0.388295] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1197worker1 # [ 0.401760] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1198lb # [ 0.427552] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1199client # [ 0.389655] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1200worker2 # [ 0.408545] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1201worker1 # [ 0.403252] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1202worker1 # [ 0.404071] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1203lb # [ 0.429008] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1204worker2 # [ 0.410648] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1205client # [ 0.392968] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1206worker1 # [ 0.404914] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1207lb # [ 0.429844] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1208worker2 # [ 0.411620] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1209client # [ 0.393804] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1210lb # [ 0.430688] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1211worker1 # [ 0.405222] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1212client # [ 0.394660] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1213worker2 # [ 0.413033] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1214lb # [ 0.431528] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1215client # [ 0.394938] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1216worker1 # [ 0.406760] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1217worker2 # [ 0.414648] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1218client # [ 0.396995] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1219worker1 # [ 0.408228] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1220worker2 # [ 0.415573] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1221lb # [ 0.433301] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1222worker1 # [ 0.409184] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1223lb # [ 0.434648] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1224worker2 # [ 0.417233] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1225client # [ 0.398966] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1226lb # [ 0.435011] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1227worker1 # [ 0.410652] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1228client # [ 0.399780] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1229worker2 # [ 0.418648] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1230client # [ 0.400712] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1231lb # [ 0.436571] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1232worker1 # [ 0.412228] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1233worker2 # [ 0.419592] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1234lb # [ 0.437716] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1235client # [ 0.402346] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1236worker1 # [ 0.413180] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1237worker2 # [ 0.420995] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1238lb # [ 0.438210] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1239worker1 # [ 0.414767] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1240lb # [ 0.439650] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1241worker2 # [ 0.422648] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1242worker1 # [ 0.416260] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1243client # [ 0.404940] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1244worker2 # [ 0.423597] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1245lb # [ 0.440188] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1246client # [ 0.406585] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1247worker1 # [ 0.417221] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1248client # [ 0.406954] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1249lb # [ 0.441651] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1250worker2 # [ 0.425200] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1251worker1 # [ 0.418756] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1252client # [ 0.408492] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1253worker2 # [ 0.426652] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1254worker1 # [ 0.420259] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1255worker2 # [ 0.427468] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1256client # [ 0.409962] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1257worker2 # [ 0.428369] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1258worker1 # [ 0.421215] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1259client # [ 0.410806] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1260worker2 # [ 0.429212] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1261client # [ 0.411710] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1262worker1 # [ 0.422748] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1263client # [ 0.412755] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1264worker1 # [ 0.424232] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1265worker1 # [ 0.425079] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1266worker2 # [ 0.431016] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1267client # [ 0.414503] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1268worker1 # [ 0.425945] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1269worker2 # [ 0.432280] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1270worker2 # [ 0.432657] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1271client # [ 0.415945] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1272lb # [ 0.442246] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1273worker1 # [ 0.426763] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1274client # [ 0.416865] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1275lb # [ 0.444580] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1276worker2 # [ 0.434185] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1277lb # [ 0.445233] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1278client # [ 0.418361] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1279worker2 # [ 0.435369] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1280worker1 # [ 0.428254] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1281lb # [ 0.446275] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1282worker2 # [ 0.435844] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1283worker1 # [ 0.429859] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1284client # [ 0.419970] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1285worker1 # [ 0.430237] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1286lb # [ 0.447242] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1287worker2 # [ 0.437267] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1288client # [ 0.420954] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1289worker1 # [ 0.431813] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1290lb # [ 0.448990] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1291worker2 # [ 0.437853] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1292lb # [ 0.449838] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1293client # [ 0.422531] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1294worker1 # [ 0.432945] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1295worker2 # [ 0.439347] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1296worker1 # [ 0.433411] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1297lb # [ 0.451051] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1298client # [ 0.423968] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1299client # [ 0.424937] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1300lb # [ 0.452606] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1301worker1 # [ 0.434890] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1302client # [ 0.426469] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1303worker1 # [ 0.435404] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1304worker1 # [ 0.436934] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1305client # [ 0.427973] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1306lb # [ 0.456045] ACPI: PCI: Interrupt link LNKA configured for IRQ 101307lb # [ 0.457097] ACPI: PCI: Interrupt link LNKB configured for IRQ 101308client # [ 0.428954] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1309lb # [ 0.458091] ACPI: PCI: Interrupt link LNKC configured for IRQ 111310client # [ 0.431322] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1311lb # [ 0.459087] ACPI: PCI: Interrupt link LNKD configured for IRQ 111312client # [ 0.432946] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1313worker2 # [ 0.439855] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1314lb # [ 0.460128] ACPI: PCI: Interrupt link LNKE configured for IRQ 101315client # [ 0.433819] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1316lb # [ 0.461087] ACPI: PCI: Interrupt link LNKF configured for IRQ 101317worker2 # [ 0.442246] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1318client # [ 0.434709] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1319lb # [ 0.462089] ACPI: PCI: Interrupt link LNKG configured for IRQ 111320worker2 # [ 0.442944] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1321lb # [ 0.463088] ACPI: PCI: Interrupt link LNKH configured for IRQ 111322client # [ 0.436334] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1323worker2 # [ 0.443922] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1324lb # [ 0.464024] ACPI: PCI: Interrupt link GSIA configured for IRQ 161325worker1 # [ 0.437471] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1326lb # [ 0.465007] ACPI: PCI: Interrupt link GSIB configured for IRQ 171327worker2 # [ 0.444904] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1328lb # [ 0.466000] ACPI: PCI: Interrupt link GSIC configured for IRQ 181329worker1 # [ 0.439798] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1330lb # [ 0.466982] ACPI: PCI: Interrupt link GSID configured for IRQ 191331worker2 # [ 0.446664] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1332client # [ 0.438939] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1333lb # [ 0.467952] ACPI: PCI: Interrupt link GSIE configured for IRQ 201334client # [ 0.439953] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1335lb # [ 0.468728] ACPI: PCI: Interrupt link GSIF configured for IRQ 211336worker2 # [ 0.447477] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1337worker1 # [ 0.440462] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1338client # [ 0.440954] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1339lb # [ 0.469002] ACPI: PCI: Interrupt link GSIG configured for IRQ 221340lb # [ 0.469944] ACPI: PCI: Interrupt link GSIH configured for IRQ 231341worker2 # [ 0.448984] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1342client # [ 0.442503] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1343worker1 # [ 0.441530] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1344lb # [ 0.471684] iommu: Default domain type: Translated1345worker2 # [ 0.450663] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1346client # [ 0.443960] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1347worker1 # [ 0.442486] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1348lb # [ 0.471997] iommu: DMA domain TLB invalidation policy: lazy mode1349lb # [ 0.473172] ACPI: bus type USB registered1350lb # [ 0.473910] usbcore: registered new interface driver usbfs1351worker1 # [ 0.444274] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1352lb # [ 0.474716] usbcore: registered new interface driver hub1353client # [ 0.445019] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1354worker1 # [ 0.445089] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1355lb # [ 0.475006] usbcore: registered new device driver usb1356client # [ 0.446824] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1357worker1 # [ 0.446244] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1358lb # [ 0.476682] NetLabel: Initializing1359worker2 # [ 0.455493] ACPI: PCI: Interrupt link LNKA configured for IRQ 101360lb # [ 0.476996] NetLabel: domain hash size = 1281361client # [ 0.447802] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1362worker2 # [ 0.456474] ACPI: PCI: Interrupt link LNKB configured for IRQ 101363lb # [ 0.477726] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1364worker1 # [ 0.448243] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1365worker2 # [ 0.457453] ACPI: PCI: Interrupt link LNKC configured for IRQ 111366lb # [ 0.478025] NetLabel: unlabeled traffic allowed by default1367client # [ 0.449711] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1368lb # [ 0.478933] PCI: Using ACPI for IRQ routing1369worker2 # [ 0.458456] ACPI: PCI: Interrupt link LNKD configured for IRQ 111370worker2 # [ 0.459453] ACPI: PCI: Interrupt link LNKE configured for IRQ 101371worker2 # [ 0.460459] ACPI: PCI: Interrupt link LNKF configured for IRQ 101372worker2 # [ 0.461431] ACPI: PCI: Interrupt link LNKG configured for IRQ 111373worker1 # [ 0.453004] ACPI: PCI: Interrupt link LNKA configured for IRQ 101374worker2 # [ 0.462422] ACPI: PCI: Interrupt link LNKH configured for IRQ 111375worker1 # [ 0.454046] ACPI: PCI: Interrupt link LNKB configured for IRQ 101376worker2 # [ 0.463364] ACPI: PCI: Interrupt link GSIA configured for IRQ 161377worker1 # [ 0.455040] ACPI: PCI: Interrupt link LNKC configured for IRQ 111378worker2 # [ 0.464429] ACPI: PCI: Interrupt link GSIB configured for IRQ 171379worker1 # [ 0.456039] ACPI: PCI: Interrupt link LNKD configured for IRQ 111380worker2 # [ 0.465432] ACPI: PCI: Interrupt link GSIC configured for IRQ 181381worker1 # [ 0.457012] ACPI: PCI: Interrupt link LNKE configured for IRQ 101382worker2 # [ 0.466409] ACPI: PCI: Interrupt link GSID configured for IRQ 191383worker1 # [ 0.458023] ACPI: PCI: Interrupt link LNKF configured for IRQ 101384worker2 # [ 0.467439] ACPI: PCI: Interrupt link GSIE configured for IRQ 201385worker1 # [ 0.459008] ACPI: PCI: Interrupt link LNKG configured for IRQ 111386worker2 # [ 0.468437] ACPI: PCI: Interrupt link GSIF configured for IRQ 211387worker1 # [ 0.460009] ACPI: PCI: Interrupt link LNKH configured for IRQ 111388worker2 # [ 0.469430] ACPI: PCI: Interrupt link GSIG configured for IRQ 221389worker1 # [ 0.461256] ACPI: PCI: Interrupt link GSIA configured for IRQ 161390client # [ 0.450166] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1391worker2 # [ 0.470434] ACPI: PCI: Interrupt link GSIH configured for IRQ 231392worker1 # [ 0.462235] ACPI: PCI: Interrupt link GSIB configured for IRQ 171393worker1 # [ 0.463229] ACPI: PCI: Interrupt link GSIC configured for IRQ 181394client # [ 0.452658] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1395worker2 # [ 0.472400] iommu: Default domain type: Translated1396worker1 # [ 0.464220] ACPI: PCI: Interrupt link GSID configured for IRQ 191397worker2 # [ 0.472643] iommu: DMA domain TLB invalidation policy: lazy mode1398worker1 # [ 0.465203] ACPI: PCI: Interrupt link GSIE configured for IRQ 201399client # [ 0.453208] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1400worker2 # [ 0.473904] ACPI: bus type USB registered1401worker1 # [ 0.465963] ACPI: PCI: Interrupt link GSIF configured for IRQ 211402worker2 # [ 0.474688] usbcore: registered new interface driver usbfs1403client # [ 0.454220] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1404worker1 # [ 0.466228] ACPI: PCI: Interrupt link GSIG configured for IRQ 221405worker2 # [ 0.475589] usbcore: registered new interface driver hub1406worker1 # [ 0.467178] ACPI: PCI: Interrupt link GSIH configured for IRQ 231407worker2 # [ 0.476289] usbcore: registered new device driver usb1408client # [ 0.455202] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1409worker2 # [ 0.477476] NetLabel: Initializing1410worker1 # [ 0.469100] iommu: Default domain type: Translated1411worker2 # [ 0.478122] NetLabel: domain hash size = 1281412client # [ 0.456961] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1413worker1 # [ 0.469789] iommu: DMA domain TLB invalidation policy: lazy mode1414worker2 # [ 0.478635] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1415client # [ 0.457820] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1416worker1 # [ 0.470484] ACPI: bus type USB registered1417worker2 # [ 0.479607] NetLabel: unlabeled traffic allowed by default1418worker1 # [ 0.471203] usbcore: registered new interface driver usbfs1419worker2 # [ 0.480369] PCI: Using ACPI for IRQ routing1420client # [ 0.459024] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1421worker1 # [ 0.471874] usbcore: registered new interface driver hub1422worker1 # [ 0.472225] usbcore: registered new device driver usb1423client # [ 0.460812] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1424worker1 # [ 0.473888] NetLabel: Initializing1425worker1 # [ 0.474222] NetLabel: domain hash size = 1281426worker1 # [ 0.474916] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1427worker1 # [ 0.475260] NetLabel: unlabeled traffic allowed by default1428worker1 # [ 0.476150] PCI: Using ACPI for IRQ routing1429client # [ 0.464724] ACPI: PCI: Interrupt link LNKA configured for IRQ 101430client # [ 0.465824] ACPI: PCI: Interrupt link LNKB configured for IRQ 101431client # [ 0.466796] ACPI: PCI: Interrupt link LNKC configured for IRQ 111432client # [ 0.467794] ACPI: PCI: Interrupt link LNKD configured for IRQ 111433client # [ 0.468820] ACPI: PCI: Interrupt link LNKE configured for IRQ 101434client # [ 0.469812] ACPI: PCI: Interrupt link LNKF configured for IRQ 101435client # [ 0.471042] ACPI: PCI: Interrupt link LNKG configured for IRQ 111436client # [ 0.472032] ACPI: PCI: Interrupt link LNKH configured for IRQ 111437client # [ 0.472983] ACPI: PCI: Interrupt link GSIA configured for IRQ 161438client # [ 0.473948] ACPI: PCI: Interrupt link GSIB configured for IRQ 171439client # [ 0.474943] ACPI: PCI: Interrupt link GSIC configured for IRQ 181440client # [ 0.475944] ACPI: PCI: Interrupt link GSID configured for IRQ 191441lb # [ 0.522543] pci 0000:00:01.0: vgaarb: setting as boot VGA device1442client # [ 0.476947] ACPI: PCI: Interrupt link GSIE configured for IRQ 201443lb # [ 0.522986] pci 0000:00:01.0: vgaarb: bridge control possible1444client # [ 0.477938] ACPI: PCI: Interrupt link GSIF configured for IRQ 211445client # [ 0.478925] ACPI: PCI: Interrupt link GSIG configured for IRQ 221446lb # [ 0.522986] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1447lb # [ 0.522996] vgaarb: loaded1448client # [ 0.479678] ACPI: PCI: Interrupt link GSIH configured for IRQ 231449lb # [ 0.523771] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01450lb # [ 0.523992] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1451client # [ 0.480972] iommu: Default domain type: Translated1452client # [ 0.481759] iommu: DMA domain TLB invalidation policy: lazy mode1453client # [ 0.482968] ACPI: bus type USB registered1454client # [ 0.483728] usbcore: registered new interface driver usbfs1455client # [ 0.484646] usbcore: registered new interface driver hub1456client # [ 0.484941] usbcore: registered new device driver usb1457lb # [ 0.528071] clocksource: Switched to clocksource kvm-clock1458client # [ 0.486670] NetLabel: Initializing1459client # [ 0.486938] NetLabel: domain hash size = 1281460client # [ 0.487691] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1461lb # [ 0.531550] VFS: Disk quotas dquot_6.6.01462client # [ 0.487971] NetLabel: unlabeled traffic allowed by default1463lb # [ 0.532267] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1464client # [ 0.488859] PCI: Using ACPI for IRQ routing1465lb # [ 0.533672] pnp: PnP ACPI init1466lb # [ 0.534473] ACPI: IRQ 4 override to edge(!), high(!)1467lb # [ 0.535440] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1468lb # [ 0.537039] pnp: PnP ACPI: found 5 devices1469worker2 # [ 0.523609] pci 0000:00:01.0: vgaarb: setting as boot VGA device1470worker2 # [ 0.523632] pci 0000:00:01.0: vgaarb: bridge control possible1471worker2 # [ 0.523632] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1472lb # [ 0.544684] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1473worker2 # [ 0.523641] vgaarb: loaded1474lb # [ 0.546114] clocksource: Switched to clocksource acpi_pm1475worker2 # [ 0.524387] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01476lb # [ 0.547139] NET: Registered PF_INET protocol family1477worker2 # [ 0.524635] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1478worker1 # [ 0.519851] pci 0000:00:01.0: vgaarb: setting as boot VGA device1479lb # [ 0.548368] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1480worker1 # [ 0.520212] pci 0000:00:01.0: vgaarb: bridge control possible1481worker1 # [ 0.520212] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1482worker1 # [ 0.520220] vgaarb: loaded1483worker2 # [ 0.527721] clocksource: Switched to clocksource kvm-clock1484worker1 # [ 0.520982] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01485worker1 # [ 0.521217] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1486worker2 # [ 0.531166] VFS: Disk quotas dquot_6.6.01487worker2 # [ 0.531982] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1488worker2 # [ 0.533366] pnp: PnP ACPI init1489worker1 # [ 0.524297] clocksource: Switched to clocksource kvm-clock1490worker2 # [ 0.534187] ACPI: IRQ 4 override to edge(!), high(!)1491worker2 # [ 0.535188] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1492worker2 # [ 0.536795] pnp: PnP ACPI: found 5 devices1493worker1 # [ 0.527818] VFS: Disk quotas dquot_6.6.01494worker1 # [ 0.528499] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1495worker1 # [ 0.529851] pnp: PnP ACPI init1496worker1 # [ 0.530638] ACPI: IRQ 4 override to edge(!), high(!)1497worker1 # [ 0.531581] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1498lb # [ 0.564791] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1499worker1 # [ 0.533104] pnp: PnP ACPI: found 5 devices1500lb # [ 0.566348] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1501lb # [ 0.567761] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1502worker2 # [ 0.544349] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1503worker2 # [ 0.545822] clocksource: Switched to clocksource acpi_pm1504lb # [ 0.569138] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1505worker2 # [ 0.546770] NET: Registered PF_INET protocol family1506lb # [ 0.570418] TCP: Hash tables configured (established 16384 bind 16384)1507worker2 # [ 0.547922] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1508lb # [ 0.571573] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1509lb # [ 0.572872] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1510worker1 # [ 0.540623] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1511lb # [ 0.574019] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1512worker1 # [ 0.542035] clocksource: Switched to clocksource acpi_pm1513lb # [ 0.575288] NET: Registered PF_UNIX/PF_LOCAL protocol family1514worker1 # [ 0.542952] NET: Registered PF_INET protocol family1515lb # [ 0.576288] NET: Registered PF_XDP protocol family1516client # [ 0.532363] pci 0000:00:01.0: vgaarb: setting as boot VGA device1517lb # [ 0.577157] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1518worker1 # [ 0.544062] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1519client # [ 0.532928] pci 0000:00:01.0: vgaarb: bridge control possible1520lb # [ 0.578128] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1521client # [ 0.532928] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1522lb # [ 0.579163] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1523client # [ 0.532937] vgaarb: loaded1524lb # [ 0.580294] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1525client # [ 0.533661] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01526client # [ 0.533933] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1527lb # [ 0.581443] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1528lb # [ 0.582533] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1529lb # [ 0.584312] ACPI: \_SB_.GSIA: Enabled at IRQ 161530lb # [ 0.586463] ACPI: \_SB_.GSIB: Enabled at IRQ 171531client # [ 0.538030] clocksource: Switched to clocksource kvm-clock1532lb # [ 0.588438] ACPI: \_SB_.GSIC: Enabled at IRQ 181533worker2 # [ 0.564821] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1534client # [ 0.541727] VFS: Disk quotas dquot_6.6.01535lb # [ 0.590431] ACPI: \_SB_.GSID: Enabled at IRQ 191536worker2 # [ 0.566380] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1537client # [ 0.542448] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1538worker2 # [ 0.567646] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1539lb # [ 0.592117] PCI: CLS 0 bytes, default 641540client # [ 0.543847] pnp: PnP ACPI init1541worker2 # [ 0.569008] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1542client # [ 0.544694] ACPI: IRQ 4 override to edge(!), high(!)1543lb # [ 0.593100] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1544worker1 # [ 0.560476] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1545worker2 # [ 0.570292] TCP: Hash tables configured (established 16384 bind 16384)1546client # [ 0.545616] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1547lb # [ 0.594817] Trying to unpack rootfs image as initramfs...1548worker1 # [ 0.561984] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1549worker2 # [ 0.571384] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1550client # [ 0.547117] pnp: PnP ACPI: found 5 devices1551worker2 # [ 0.572577] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1552worker1 # [ 0.563240] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1553worker2 # [ 0.573637] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1554worker1 # [ 0.564548] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1555worker2 # [ 0.574852] NET: Registered PF_UNIX/PF_LOCAL protocol family1556worker1 # [ 0.565737] TCP: Hash tables configured (established 16384 bind 16384)1557worker2 # [ 0.575843] NET: Registered PF_XDP protocol family1558worker1 # [ 0.566815] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1559worker2 # [ 0.576679] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1560worker1 # [ 0.568039] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1561worker2 # [ 0.577716] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1562worker1 # [ 0.569097] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1563worker2 # [ 0.578750] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1564worker1 # [ 0.570278] NET: Registered PF_UNIX/PF_LOCAL protocol family1565worker2 # [ 0.579872] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1566client # [ 0.554696] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1567worker1 # [ 0.571223] NET: Registered PF_XDP protocol family1568client # [ 0.556113] clocksource: Switched to clocksource acpi_pm1569worker2 # [ 0.580977] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1570worker1 # [ 0.572049] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1571client # [ 0.557077] NET: Registered PF_INET protocol family1572worker2 # [ 0.582066] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1573worker1 # [ 0.573245] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1574worker2 # [ 0.583786] ACPI: \_SB_.GSIA: Enabled at IRQ 161575client # [ 0.558271] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1576worker1 # [ 0.574381] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1577worker1 # [ 0.575480] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1578worker2 # [ 0.585871] ACPI: \_SB_.GSIB: Enabled at IRQ 171579worker1 # [ 0.576624] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1580worker1 # [ 0.577718] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1581worker2 # [ 0.587916] ACPI: \_SB_.GSIC: Enabled at IRQ 181582worker1 # [ 0.579597] ACPI: \_SB_.GSIA: Enabled at IRQ 161583worker2 # [ 0.590010] ACPI: \_SB_.GSID: Enabled at IRQ 191584worker1 # [ 0.581771] ACPI: \_SB_.GSIB: Enabled at IRQ 171585worker2 # [ 0.591810] PCI: CLS 0 bytes, default 641586worker1 # [ 0.583871] ACPI: \_SB_.GSIC: Enabled at IRQ 181587worker2 # [ 0.592792] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1588worker2 # [ 0.594537] Trying to unpack rootfs image as initramfs...1589worker1 # [ 0.585911] ACPI: \_SB_.GSID: Enabled at IRQ 191590worker1 # [ 0.587596] PCI: CLS 0 bytes, default 641591worker1 # [ 0.588543] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1592worker1 # [ 0.590382] Trying to unpack rootfs image as initramfs...1593client # [ 0.575019] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1594client # [ 0.576651] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1595client # [ 0.578036] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1596client # [ 0.579442] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1597client # [ 0.580749] TCP: Hash tables configured (established 16384 bind 16384)1598client # [ 0.581914] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1599client # [ 0.583218] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1600client # [ 0.584356] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1601client # [ 0.585797] NET: Registered PF_UNIX/PF_LOCAL protocol family1602client # [ 0.586816] NET: Registered PF_XDP protocol family1603client # [ 0.587711] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1604client # [ 0.588760] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1605lb # [ 0.638624] Initialise system trusted keyrings1606client # [ 0.589820] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1607client # [ 0.590897] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1608client # [ 0.592054] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1609client # [ 0.593213] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1610lb # [ 0.642257] workingset: timestamp_bits=40 max_order=19 bucket_order=01611client # [ 0.595050] ACPI: \_SB_.GSIA: Enabled at IRQ 161612client # [ 0.597134] ACPI: \_SB_.GSIB: Enabled at IRQ 171613client # [ 0.599165] ACPI: \_SB_.GSIC: Enabled at IRQ 181614client # [ 0.601183] ACPI: \_SB_.GSID: Enabled at IRQ 191615client # [ 0.602917] PCI: CLS 0 bytes, default 641616client # [ 0.603922] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1617client # [ 0.605722] Trying to unpack rootfs image as initramfs...1618lb # [ 0.661600] Key type asymmetric registered1619worker2 # [ 0.639623] Initialise system trusted keyrings1620lb # [ 0.664222] Asymmetric key parser 'x509' registered1621lb # [ 0.665124] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1622worker2 # [ 0.643281] workingset: timestamp_bits=40 max_order=19 bucket_order=01623worker1 # [ 0.636220] Initialise system trusted keyrings1624lb # [ 0.670388] io scheduler mq-deadline registered1625lb # [ 0.671221] io scheduler kyber registered1626worker1 # [ 0.639871] workingset: timestamp_bits=40 max_order=19 bucket_order=01627lb # [ 0.673747] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1628lb # [ 0.675084] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1629lb # [ 0.680979] Linux agpgart interface v0.1031630lb # [ 0.681814] ACPI: bus type drm_connector registered1631lb # [ 0.684702] usbcore: registered new interface driver usbserial_generic1632worker2 # [ 0.662479] Key type asymmetric registered1633lb # [ 0.685836] usbserial: USB Serial support registered for generic1634worker2 # [ 0.665232] Asymmetric key parser 'x509' registered1635lb # [ 0.689225] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1636worker2 # [ 0.666138] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1637lb # [ 0.689225] Please enable it if your BIOS has the CPPC option.1638worker1 # [ 0.659078] Key type asymmetric registered1639lb # [ 0.691466] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1640worker1 # [ 0.659770] Asymmetric key parser 'x509' registered1641worker2 # [ 0.669365] io scheduler mq-deadline registered1642worker2 # [ 0.670131] io scheduler kyber registered1643lb # [ 0.695362] drop_monitor: Initializing network drop monitor service1644worker1 # [ 0.662854] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1645lb # [ 0.696596] NET: Registered PF_INET6 protocol family1646worker2 # [ 0.673833] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1647worker1 # [ 0.665864] io scheduler mq-deadline registered1648worker1 # [ 0.666660] io scheduler kyber registered1649lb # [ 0.699690] Segment Routing with IPv61650client # [ 0.651433] Initialise system trusted keyrings1651worker2 # [ 0.677234] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1652client # [ 0.652415] workingset: timestamp_bits=40 max_order=19 bucket_order=01653lb # [ 0.702237] In-situ OAM (IOAM) with IPv61654worker1 # [ 0.669568] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1655lb # [ 0.703340] IPI shorthand broadcast: enabled1656worker2 # [ 0.680880] Linux agpgart interface v0.1031657worker2 # [ 0.681683] ACPI: bus type drm_connector registered1658worker1 # [ 0.672972] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1659worker2 # [ 0.684704] usbcore: registered new interface driver usbserial_generic1660worker2 # [ 0.685827] usbserial: USB Serial support registered for generic1661worker1 # [ 0.678115] Linux agpgart interface v0.1031662worker1 # [ 0.678911] ACPI: bus type drm_connector registered1663lb # [ 0.711712] sched_clock: Marking stable (575014612, 136185749)->(787480272, -76279911)1664worker2 # [ 0.689235] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1665worker2 # [ 0.689235] Please enable it if your BIOS has the CPPC option.1666worker1 # [ 0.682172] usbcore: registered new interface driver usbserial_generic1667worker2 # [ 0.691510] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1668worker1 # [ 0.683272] usbserial: USB Serial support registered for generic1669lb # [ 0.717385] registered taskstats version 11670lb # [ 0.718385] Loading compiled-in X.509 certificates1671worker2 # [ 0.696366] drop_monitor: Initializing network drop monitor service1672worker1 # [ 0.686827] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1673worker2 # [ 0.697544] NET: Registered PF_INET6 protocol family1674worker1 # [ 0.686827] Please enable it if your BIOS has the CPPC option.1675worker1 # [ 0.689111] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1676client # [ 0.675603] Key type asymmetric registered1677worker2 # [ 0.700785] Segment Routing with IPv61678worker2 # [ 0.701464] In-situ OAM (IOAM) with IPv61679client # [ 0.676331] Asymmetric key parser 'x509' registered1680worker1 # [ 0.691996] drop_monitor: Initializing network drop monitor service1681worker1 # [ 0.693142] NET: Registered PF_INET6 protocol family1682client # [ 0.677160] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1683worker2 # [ 0.704510] IPI shorthand broadcast: enabled1684client # [ 0.679666] io scheduler mq-deadline registered1685worker1 # [ 0.696253] Segment Routing with IPv61686client # [ 0.680458] io scheduler kyber registered1687worker1 # [ 0.698827] In-situ OAM (IOAM) with IPv61688client # [ 0.683175] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1689worker1 # [ 0.699903] IPI shorthand broadcast: enabled1690client # [ 0.684458] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1691lb # [ 0.734696] Demotion targets for Node 0: null1692worker2 # [ 0.710772] sched_clock: Marking stable (576014446, 134197779)->(786954125, -76741900)1693lb # [ 0.738248] Key type .fscrypt registered1694client # [ 0.690200] Linux agpgart interface v0.1031695lb # [ 0.738986] Key type fscrypt-provisioning registered1696client # [ 0.690959] ACPI: bus type drm_connector registered1697lb # [ 0.739928] ima: No TPM chip found, activating TPM-bypass!1698worker2 # [ 0.717405] registered taskstats version 11699worker1 # [ 0.708135] sched_clock: Marking stable (575013999, 132787167)->(784029299, -76228133)1700worker2 # [ 0.718410] Loading compiled-in X.509 certificates1701lb # [ 0.742212] ima: Allocated hash algorithm: sha11702client # [ 0.694031] usbcore: registered new interface driver usbserial_generic1703lb # [ 0.743055] ima: No architecture policies found1704client # [ 0.695175] usbserial: USB Serial support registered for generic1705worker1 # [ 0.713051] registered taskstats version 11706worker1 # [ 0.714014] Loading compiled-in X.509 certificates1707lb # [ 0.747219] PM: Magic number: 10:784:1611708client # [ 0.698570] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1709client # [ 0.698570] Please enable it if your BIOS has the CPPC option.1710lb # [ 0.748804] RAS: Correctable Errors collector initialized.1711client # [ 0.700846] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1712client # [ 0.705694] drop_monitor: Initializing network drop monitor service1713client # [ 0.706917] NET: Registered PF_INET6 protocol family1714lb # [ 0.757860] clk: Disabling unused clocks1715client # [ 0.710003] Segment Routing with IPv61716lb # [ 0.760224] PM: genpd: Disabling unused power domains1717worker2 # [ 0.737221] Demotion targets for Node 0: null1718client # [ 0.712585] In-situ OAM (IOAM) with IPv61719worker2 # [ 0.738124] Key type .fscrypt registered1720client # [ 0.713680] IPI shorthand broadcast: enabled1721worker2 # [ 0.738855] Key type fscrypt-provisioning registered1722worker2 # [ 0.739818] ima: No TPM chip found, activating TPM-bypass!1723worker1 # [ 0.731821] Demotion targets for Node 0: null1724worker1 # [ 0.732740] Key type .fscrypt registered1725worker2 # [ 0.744223] ima: Allocated hash algorithm: sha11726worker2 # [ 0.745053] ima: No architecture policies found1727worker1 # [ 0.735808] Key type fscrypt-provisioning registered1728worker1 # [ 0.736719] ima: No TPM chip found, activating TPM-bypass!1729worker1 # [ 0.737647] ima: Allocated hash algorithm: sha11730worker2 # [ 0.747232] PM: Magic number: 10:784:1611731client # [ 0.721929] sched_clock: Marking stable (588014534, 133528287)->(794846175, -73303354)1732worker2 # [ 0.748870] RAS: Correctable Errors collector initialized.1733worker1 # [ 0.739936] ima: No architecture policies found1734worker1 # [ 0.740907] PM: Magic number: 10:784:1611735client # [ 0.725805] registered taskstats version 11736client # [ 0.726810] Loading compiled-in X.509 certificates1737worker1 # [ 0.742569] RAS: Correctable Errors collector initialized.1738worker2 # [ 0.758880] clk: Disabling unused clocks1739worker2 # [ 0.761231] PM: genpd: Disabling unused power domains1740worker1 # [ 0.753493] clk: Disabling unused clocks1741worker1 # [ 0.755833] PM: genpd: Disabling unused power domains1742client # [ 0.746553] Demotion targets for Node 0: null1743client # [ 0.747502] Key type .fscrypt registered1744client # [ 0.750556] Key type fscrypt-provisioning registered1745client # [ 0.751564] ima: No TPM chip found, activating TPM-bypass!1746client # [ 0.752509] ima: Allocated hash algorithm: sha11747client # [ 0.754578] ima: No architecture policies found1748client # [ 0.757570] PM: Magic number: 10:784:1611749client # [ 0.759238] RAS: Correctable Errors collector initialized.1750client # [ 0.767226] clk: Disabling unused clocks1751client # [ 0.771558] PM: genpd: Disabling unused power domains1752lb # [ 0.926477] Freeing initrd memory: 29064K1753lb # [ 0.929664] Freeing unused decrypted memory: 2028K1754lb # [ 0.932269] Freeing unused kernel image (initmem) memory: 3652K1755lb # [ 0.933369] Write protecting the kernel read-only data: 32768k1756lb # [ 0.935276] Freeing unused kernel image (text/rodata gap) memory: 1184K1757lb # [ 0.936838] Freeing unused kernel image (rodata/data gap) memory: 720K1758worker2 # [ 0.921718] Freeing initrd memory: 29060K1759worker1 # [ 0.914295] Freeing initrd memory: 29056K1760worker2 # [ 0.925001] Freeing unused decrypted memory: 2028K1761worker1 # [ 0.917498] Freeing unused decrypted memory: 2028K1762worker2 # [ 0.927771] Freeing unused kernel image (initmem) memory: 3652K1763worker2 # [ 0.928893] Write protecting the kernel read-only data: 32768k1764worker1 # [ 0.920195] Freeing unused kernel image (initmem) memory: 3652K1765worker1 # [ 0.921309] Write protecting the kernel read-only data: 32768k1766worker2 # [ 0.930827] Freeing unused kernel image (text/rodata gap) memory: 1184K1767worker2 # [ 0.932378] Freeing unused kernel image (rodata/data gap) memory: 720K1768worker1 # [ 0.923247] Freeing unused kernel image (text/rodata gap) memory: 1184K1769worker1 # [ 0.924821] Freeing unused kernel image (rodata/data gap) memory: 720K1770client # [ 0.930718] Freeing initrd memory: 29056K1771client # [ 0.933952] Freeing unused decrypted memory: 2028K1772client # [ 0.936589] Freeing unused kernel image (initmem) memory: 3652K1773client # [ 0.937710] Write protecting the kernel read-only data: 32768k1774lb # [ 0.987466] x86/mm: Checked W+X mappings: passed, no W+X pages found.1775lb # [ 0.988565] Run /init as init process1776client # [ 0.939652] Freeing unused kernel image (text/rodata gap) memory: 1184K1777client # [ 0.941197] Freeing unused kernel image (rodata/data gap) memory: 720K1778lb # [ 0.999447] systemd[1]: Inserted module 'autofs4'1779worker2 # [ 0.983222] x86/mm: Checked W+X mappings: passed, no W+X pages found.1780worker2 # [ 0.984328] Run /init as init process1781worker1 # [ 0.975711] x86/mm: Checked W+X mappings: passed, no W+X pages found.1782worker1 # [ 0.976819] Run /init as init process1783worker2 # [ 0.994994] systemd[1]: Inserted module 'autofs4'1784lb # [ 1.019171] fuse: init (API version 7.45)1785worker1 # [ 0.987491] systemd[1]: Inserted module 'autofs4'1786lb # [ 1.026566] ACPI: \_SB_.GSIG: Enabled at IRQ 221787lb # [ 1.028905] ACPI: \_SB_.GSIH: Enabled at IRQ 231788lb # [ 1.031920] ACPI: \_SB_.GSIE: Enabled at IRQ 201789lb # [ 1.033931] ACPI: \_SB_.GSIF: Enabled at IRQ 211790worker2 # [ 1.014840] fuse: init (API version 7.45)1791lb # [ 1.039246] virtiofs virtio5: discovered new tag: nix-store1792worker1 # [ 1.007332] fuse: init (API version 7.45)1793client # [ 0.992076] x86/mm: Checked W+X mappings: passed, no W+X pages found.1794lb # [ 1.040900] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1795client # [ 0.993170] Run /init as init process1796worker2 # [ 1.022299] ACPI: \_SB_.GSIG: Enabled at IRQ 221797worker1 # [ 1.014855] ACPI: \_SB_.GSIG: Enabled at IRQ 221798lb # [ 1.047809] virtiofs virtio6: discovered new tag: shared1799worker2 # [ 1.024628] ACPI: \_SB_.GSIH: Enabled at IRQ 231800lb # [ 1.049435] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1801worker1 # [ 1.017202] ACPI: \_SB_.GSIH: Enabled at IRQ 231802worker2 # [ 1.027620] ACPI: \_SB_.GSIE: Enabled at IRQ 201803client # [ 1.003894] systemd[1]: Inserted module 'autofs4'1804lb # [ 1.052953] virtiofs virtio7: discovered new tag: xchg1805worker2 # [ 1.029549] ACPI: \_SB_.GSIF: Enabled at IRQ 211806worker1 # [ 1.020181] ACPI: \_SB_.GSIE: Enabled at IRQ 201807lb # [ 1.054385] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1808worker1 # [ 1.022099] ACPI: \_SB_.GSIF: Enabled at IRQ 211809worker2 # [ 1.034906] virtiofs virtio5: discovered new tag: nix-store1810worker1 # [ 1.027110] virtiofs virtio5: discovered new tag: nix-store1811worker2 # [ 1.036557] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1812worker1 # [ 1.028721] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1813worker2 # [ 1.043577] virtiofs virtio6: discovered new tag: shared1814worker2 # [ 1.045248] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1815worker1 # [ 1.036204] virtiofs virtio6: discovered new tag: shared1816worker1 # [ 1.037780] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1817worker2 # [ 1.048433] virtiofs virtio7: discovered new tag: xchg1818client # [ 1.023751] fuse: init (API version 7.45)1819worker2 # [ 1.049989] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1820worker1 # [ 1.042427] virtiofs virtio7: discovered new tag: xchg1821lb # [ 1.076563] systemd[1]: Successfully made /usr/ read-only.1822worker1 # [ 1.044111] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1823client # [ 1.031148] ACPI: \_SB_.GSIG: Enabled at IRQ 221824client # [ 1.033656] ACPI: \_SB_.GSIH: Enabled at IRQ 231825client # [ 1.036589] ACPI: \_SB_.GSIE: Enabled at IRQ 201826client # [ 1.038694] ACPI: \_SB_.GSIF: Enabled at IRQ 211827client # [ 1.043878] virtiofs virtio5: discovered new tag: nix-store1828client # [ 1.045419] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1829worker2 # [ 1.073338] systemd[1]: Successfully made /usr/ read-only.1830worker1 # [ 1.065953] systemd[1]: Successfully made /usr/ read-only.1831client # [ 1.052898] virtiofs virtio6: discovered new tag: shared1832client # [ 1.054689] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1833client # [ 1.059493] virtiofs virtio7: discovered new tag: xchg1834client # [ 1.061159] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1835client # [ 1.082600] systemd[1]: Successfully made /usr/ read-only.1836lb # [ 1.413077] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1837lb # [ 1.425085] systemd[1]: Detected virtualization kvm.1838lb # [ 1.427261] systemd[1]: Detected architecture x86-64.1839lb # [ 1.429409] systemd[1]: Running in initrd.1840lb # [ 1.431830] systemd[1]: Initializing machine ID from random generator.1841lb # [ 1.434687] systemd[1]: Hostname set to <lb>.1842worker2 # [ 1.409063] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1843worker2 # [ 1.414101] systemd[1]: Detected virtualization kvm.1844worker2 # [ 1.415003] systemd[1]: Detected architecture x86-64.1845worker2 # [ 1.415947] systemd[1]: Running in initrd.1846worker2 # [ 1.417088] systemd[1]: Initializing machine ID from random generator.1847worker2 # [ 1.418345] systemd[1]: Hostname set to <worker2>.1848worker1 # [ 1.402952] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1849worker1 # [ 1.415116] systemd[1]: Detected virtualization kvm.1850worker1 # [ 1.417293] systemd[1]: Detected architecture x86-64.1851worker1 # [ 1.419478] systemd[1]: Running in initrd.1852worker1 # [ 1.422089] systemd[1]: Initializing machine ID from random generator.1853worker1 # [ 1.424984] systemd[1]: Hostname set to <worker1>.1854client # [ 1.419396] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1855client # [ 1.431503] systemd[1]: Detected virtualization kvm.1856client # [ 1.433749] systemd[1]: Detected architecture x86-64.1857client # [ 1.435949] systemd[1]: Running in initrd.1858client # [ 1.438389] systemd[1]: Initializing machine ID from random generator.1859client # [ 1.441290] systemd[1]: Hostname set to <client>.1860worker2 # [ 1.612906] systemd[1]: bpf-restrict-fs: LSM BPF program attached1861lb # [ 1.660098] systemd[1]: bpf-restrict-fs: LSM BPF program attached1862worker2 # [ 1.652262] systemd[1]: Queued start job for default target Initrd Default Target.1863worker2 # [ 1.655535] systemd[1]: Created slice Slice /system/modprobe.1864worker2 # [ 1.656736] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1865worker2 # [ 1.658186] systemd[1]: Expecting device /dev/disk/by-label/nixos...1866worker2 # [ 1.659388] systemd[1]: Reached target Path Units.1867worker2 # [ 1.660340] systemd[1]: Reached target Slice Units.1868worker2 # [ 1.661237] systemd[1]: Reached target Swaps.1869worker1 # [ 1.650191] systemd[1]: bpf-restrict-fs: LSM BPF program attached1870worker2 # [ 1.662045] systemd[1]: Reached target Timer Units.1871worker2 # [ 1.663056] systemd[1]: Listening on D-Bus System Message Bus Socket.1872worker2 # [ 1.664351] systemd[1]: Listening on Journal Socket (/dev/log).1873worker2 # [ 1.665598] systemd[1]: Listening on Journal Sockets.1874worker2 # [ 1.666645] systemd[1]: Listening on udev Control Socket.1875worker2 # [ 1.667659] systemd[1]: Listening on udev Kernel Socket.1876worker2 # [ 1.668627] systemd[1]: Reached target Socket Units.1877worker2 # [ 1.670510] systemd[1]: Starting Create List of Static Device Nodes...1878worker2 # [ 1.674287] systemd[1]: Starting Load Kernel Module configfs...1879lb # [ 1.713074] systemd[1]: Queued start job for default target Initrd Default Target.1880client # [ 1.666289] systemd[1]: bpf-restrict-fs: LSM BPF program attached1881lb # [ 1.717448] systemd[1]: Created slice Slice /system/modprobe.1882worker2 # [ 1.683289] systemd[1]: Starting Journal Service...1883lb # [ 1.718568] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1884lb # [ 1.719970] systemd[1]: Expecting device /dev/disk/by-label/nixos...1885lb # [ 1.721042] systemd[1]: Reached target Path Units.1886lb # [ 1.721907] systemd[1]: Reached target Slice Units.1887lb # [ 1.722757] systemd[1]: Reached target Swaps.1888lb # [ 1.723514] systemd[1]: Reached target Timer Units.1889lb # [ 1.724428] systemd[1]: Listening on D-Bus System Message Bus Socket.1890lb # [ 1.725562] systemd[1]: Listening on Journal Socket (/dev/log).1891lb # [ 1.726665] systemd[1]: Listening on Journal Sockets.1892lb # [ 1.727662] systemd[1]: Listening on udev Control Socket.1893lb # [ 1.728618] systemd[1]: Listening on udev Kernel Socket.1894lb # [ 1.729507] systemd[1]: Reached target Socket Units.1895lb # [ 1.731172] systemd[1]: Starting Create List of Static Device Nodes...1896worker2 # [ 1.710411] systemd[1]: Starting Load Kernel Modules...1897lb # [ 1.735018] systemd[1]: Starting Load Kernel Module configfs...1898worker2 # [ 1.714320] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1899worker1 # [ 1.708757] systemd[1]: Queued start job for default target Initrd Default Target.1900worker1 # [ 1.713170] systemd[1]: Created slice Slice /system/modprobe.1901worker2 # [ 1.723292] systemd[1]: Starting Coldplug All udev Devices...1902worker1 # [ 1.714405] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1903worker1 # [ 1.715854] systemd[1]: Expecting device /dev/disk/by-label/nixos...1904worker1 # [ 1.717049] systemd[1]: Reached target Path Units.1905worker1 # [ 1.718006] systemd[1]: Reached target Slice Units.1906worker1 # [ 1.718891] systemd[1]: Reached target Swaps.1907worker1 # [ 1.719675] systemd[1]: Reached target Timer Units.1908worker1 # [ 1.720703] systemd[1]: Listening on D-Bus System Message Bus Socket.1909worker1 # [ 1.721936] systemd[1]: Listening on Journal Socket (/dev/log).1910lb # [ 1.743863] systemd[1]: Starting Journal Service...1911worker1 # [ 1.723206] systemd[1]: Listening on Journal Sockets.1912worker2 # [ 1.733157] systemd-journald[66]: Collecting audit messages is disabled.1913worker1 # [ 1.724326] systemd[1]: Listening on udev Control Socket.1914worker1 # [ 1.725390] systemd[1]: Listening on udev Kernel Socket.1915worker1 # [ 1.726401] systemd[1]: Reached target Socket Units.1916worker1 # [ 1.728381] systemd[1]: Starting Create List of Static Device Nodes...1917worker2 # [ 1.741486] systemd[1]: Finished Create List of Static Device Nodes.1918worker1 # [ 1.732872] systemd[1]: Starting Load Kernel Module configfs...1919lb # [ 1.769380] systemd[1]: Starting Load Kernel Modules...1920client # [ 1.720915] systemd[1]: Queued start job for default target Initrd Default Target.1921worker2 # [ 1.747890] systemd[1]: modprobe@configfs.service: Deactivated successfully.1922lb # [ 1.772320] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1923client # [ 1.724875] systemd[1]: Created slice Slice /system/modprobe.1924client # [ 1.726125] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1925client # [ 1.727585] systemd[1]: Expecting device /dev/disk/by-label/nixos...1926client # [ 1.728745] systemd[1]: Reached target Path Units.1927client # [ 1.729674] systemd[1]: Reached target Slice Units.1928client # [ 1.730606] systemd[1]: Reached target Swaps.1929worker2 # [ 1.755601] systemd[1]: Finished Load Kernel Module configfs.1930client # [ 1.731428] systemd[1]: Reached target Timer Units.1931lb # [ 1.780272] systemd[1]: Starting Coldplug All udev Devices...1932client # [ 1.732389] systemd[1]: Listening on D-Bus System Message Bus Socket.1933client # [ 1.733747] systemd[1]: Listening on Journal Socket (/dev/log).1934client # [ 1.734969] systemd[1]: Listening on Journal Sockets.1935client # [ 1.736009] systemd[1]: Listening on udev Control Socket.1936client # [ 1.737070] systemd[1]: Listening on udev Kernel Socket.1937worker2 # [ 1.761647] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1938client # [ 1.738085] systemd[1]: Reached target Socket Units.1939worker1 # [ 1.742504] systemd[1]: Starting Journal Service...1940client # [ 1.739999] systemd[1]: Starting Create List of Static Device Nodes...1941lb # [ 1.792634] systemd-journald[66]: Collecting audit messages is disabled.1942lb # [ 1.794020] systemd[1]: Finished Create List of Static Device Nodes.1943client # [ 1.745692] systemd[1]: Starting Load Kernel Module configfs...1944worker2 # [ 1.773286] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1945lb # [ 1.799611] systemd[1]: modprobe@configfs.service: Deactivated successfully.1946worker1 # [ 1.767083] systemd[1]: Starting Load Kernel Modules...1947worker2 # [ 1.775800] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1948lb # [ 1.806559] systemd[1]: Finished Load Kernel Module configfs.1949worker1 # [ 1.773925] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1950worker2 # [ 1.785392] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1951lb # [ 1.809649] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1952worker1 # [ 1.780712] systemd[1]: Starting Coldplug All udev Devices...1953client # [ 1.754288] systemd[1]: Starting Journal Service...1954lb # [ 1.820968] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1955worker1 # [ 1.792613] systemd-journald[66]: Collecting audit messages is disabled.1956client # [ 1.780183] systemd[1]: Starting Load Kernel Modules...1957worker1 # [ 1.796182] systemd[1]: Finished Create List of Static Device Nodes.1958client # [ 1.784686] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1959lb # [ 1.833720] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1960worker1 # [ 1.801727] systemd[1]: modprobe@configfs.service: Deactivated successfully.1961worker2 # [ 1.815849] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1962lb # [ 1.842520] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1963client # [ 1.794789] systemd[1]: Starting Coldplug All udev Devices...1964worker1 # [ 1.811261] systemd[1]: Finished Load Kernel Module configfs.1965worker1 # [ 1.815245] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1966worker2 # [ 1.825427] systemd[1]: Starting Create Static Device Nodes in /dev...1967client # [ 1.804293] systemd-journald[66]: Collecting audit messages is disabled.1968client # [ 1.807623] systemd[1]: Finished Create List of Static Device Nodes.1969worker2 # [ 1.835777] systemd[1]: Finished Load Kernel Modules.1970worker1 # [ 1.828042] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1971lb # [ 1.863781] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1972client # [ 1.815222] systemd[1]: modprobe@configfs.service: Deactivated successfully.1973worker2 # [ 1.846469] systemd[1]: Starting Apply Kernel Variables...1974worker1 # [ 1.836167] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1975client # [ 1.822323] systemd[1]: Finished Load Kernel Module configfs.1976lb # [ 1.872386] systemd[1]: Starting Create Static Device Nodes in /dev...1977client # [ 1.827003] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1978worker2 # [ 1.856014] systemd[1]: Started Journal Service.1979worker2 # [ 1.718692] systemd-modules-load[68]: Inserted module 'dm_mod'1980worker2 # [ 1.723526] systemd-modules-load[68]: Inserted module 'virtio_balloon'1981worker1 # [ 1.848157] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1982worker2 # [ 1.724568] systemd-modules-load[68]: Inserted module 'virtio_gpu'1983lb # [ 1.884275] systemd[1]: Finished Load Kernel Modules.1984client # [ 1.839618] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1985lb # [ 1.892436] systemd[1]: Starting Apply Kernel Variables...1986worker2 # [ 1.734976] systemd[1]: Finished Create Static Device Nodes in /dev.1987client # [ 1.849474] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1988worker2 # [ 1.743093] systemd[1]: Reached target Preparation for Local File Systems.1989worker2 # [ 1.744174] systemd[1]: Reached target Local File Systems.1990worker1 # [ 1.870352] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1991worker2 # [ 1.749082] systemd[1]: Starting Create System Files and Directories...1992client # [ 1.859788] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1993worker1 # [ 1.880009] systemd[1]: Starting Create Static Device Nodes in /dev...1994lb # [ 1.913294] systemd[1]: Finished Create Static Device Nodes in /dev.1995worker2 # [ 1.755658] systemd[1]: Starting Rule-based Manager for Device Events and Files...1996lb # [ 1.774813] systemd-modules-load[68]: Inserted module 'dm_mod'1997lb # [ 1.778955] systemd-modules-load[68]: Inserted module 'virtio_balloon'1998lb # [ 1.781629] systemd-modules-load[68]: Inserted module 'virtio_gpu'1999lb # [ 1.919410] systemd[1]: Started Journal Service.2000worker2 # [ 1.762695] systemd[1]: Finished Apply Kernel Variables.
loading 2,884 hidden lines…
4885worker2 # [ 45.263902] nix-grpc-daemon[923]: time=2026-09-23T04:12:01.390Z level=INFO msg="Uploading lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv (504B)"4886lb # [ 45.312918] niks3-server[979]: 2026/09/23 04:12:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4887lb # [ 45.321600] niks3-server[979]: 2026/09/23 04:12:01 INFO Signed narinfos id=9 count=14888worker2 # [ 45.303117] nix-grpc-daemon[923]: time=2026-09-23T04:12:01.429Z level=INFO msg="Uploading 1 narinfos"4889lb # [ 45.325284] niks3-server[979]: 2026/09/23 04:12:01 INFO Registered completed upload object_key=nar/0bddczcnpndyzwmafx9yqfqrkrp1jg9rr5hsixz6rpy99kg84g1g.nar.zst4890lb # [ 45.333752] niks3-server[979]: 2026/09/23 04:12:01 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.ls4891lb # [ 45.342627] niks3-server[979]: 2026/09/23 04:12:01 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4892worker2 # [ 45.328095] nix-grpc-daemon[923]: time=2026-09-23T04:12:01.454Z level=INFO msg="Upload complete. (188ms)"4893lb # [ 45.349954] niks3-server[979]: 2026/09/23 04:12:01 INFO Completed upload id=94894worker2 # [ 45.330989] nix-grpc-daemon[784]: ts=2026-09-23T04:12:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52284 duration_s=0 paths=1 nar_bytes_in=5044895lb # [ 45.352580] niks3-server[979]: 2026/09/23 04:12:01 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.narinfo4896worker2 # [ 45.336992] nix-grpc-daemon[784]: ts=2026-09-23T04:12:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:522844897worker2 # [ 45.342725] nix-daemon[870]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4898worker2 # [ 45.345063] nix-daemon[870]: reaped child process 921, status = succeeded4899worker2 # [ 45.350166] nix-daemon[870]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4900client # building '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' on lb:50051...4901client # farm-a> worker2: building lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv4902lb # [ 45.891169] envoy[914]: conn peer=[2001:db8:1::1]:53036 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=74903lb # [ 45.894646] envoy[914]: conn peer=[2001:db8:1::1]:53046 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=104904lb # [ 45.898345] envoy[914]: conn peer=[2001:db8:1::1]:53056 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9451 tx=4323 ms=48254905lb # [ 45.902616] envoy[914]: conn peer=[2001:db8:1::1]:53066 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=494906lb # [ 45.906092] envoy[914]: conn peer=[2001:db8:1::1]:53284 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4165 tx=1877 ms=13544907client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4908client # farm-a> LOG-farm-a-sig4909worker2 # [ 46.391272] nix-daemon[870]: reaped child process 943, status = succeeded4910worker2 # [ 46.474361] nix-daemon[870]: accepted connection from pid 959, user nix-grpc-daemon (trusted)4911worker2 # [ 46.486968] nix-daemon[870]: reaped child process 962, status = succeeded4912lb # [ 46.517790] niks3-server[979]: 2026/09/23 04:12:02 INFO Received uploads request method=POST path=/api/pending_closures4913worker2 # [ 46.504953] nix-grpc-daemon[923]: time=2026-09-23T04:12:02.631Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4914worker2 # [ 46.507058] nix-grpc-daemon[923]: time=2026-09-23T04:12:02.632Z level=INFO msg="Uploading qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a (128B)"4915lb # [ 46.558925] niks3-server[979]: 2026/09/23 04:12:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4916lb # [ 46.563912] niks3-server[979]: 2026/09/23 04:12:02 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.ls4917lb # [ 46.575199] niks3-server[979]: 2026/09/23 04:12:02 INFO Signed narinfos id=10 count=14918worker2 # [ 46.556993] nix-grpc-daemon[923]: time=2026-09-23T04:12:02.683Z level=INFO msg="Uploading 1 narinfos"4919lb # [ 46.579108] niks3-server[979]: 2026/09/23 04:12:02 INFO Registered completed upload object_key=log/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv4920lb # [ 46.585857] niks3-server[979]: 2026/09/23 04:12:02 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4921lb # [ 46.595822] niks3-server[979]: 2026/09/23 04:12:02 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4922lb # [ 46.599252] niks3-server[979]: 2026/09/23 04:12:02 INFO Completed upload id=104923lb # [ 46.601190] niks3-server[979]: 2026/09/23 04:12:02 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.narinfo4924worker2 # [ 46.579155] nix-grpc-daemon[923]: time=2026-09-23T04:12:02.705Z level=INFO msg="Upload complete. (220ms)"4925worker2 # [ 46.586787] nix-grpc-daemon[784]: ts=2026-09-23T04:12:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52284 duration_s=1 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv assign_id=5 outputs=14926worker2 # [ 46.597348] nix-grpc-daemon[784]: ts=2026-09-23T04:12:02Z level=debug event=want client=4 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv cached=04927worker2 # [ 46.601352] nix-grpc-daemon[784]: ts=2026-09-23T04:12:02Z level=debug event=assigned drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv worker=192.168.1.4:50051 clients=14928worker2 # [ 46.609724] nix-daemon[870]: reaped child process 941, status = succeeded4929worker2 # [ 46.642523] nix-grpc-daemon[784]: ts=2026-09-23T04:12:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:522844930worker2 # [ 46.654665] nix-grpc-daemon[784]: ts=2026-09-23T04:12:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:522844931worker2 # [ 46.668113] nix-grpc-daemon[923]: time=2026-09-23T04:12:02.794Z level=INFO msg="All 1 paths already cached"4932client # copying 1 paths to 'lb:50051'...4933client # copying path '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' to 'lb:50051'...4934worker2 # [ 46.676730] nix-grpc-daemon[784]: ts=2026-09-23T04:12:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:522844935worker2 # [ 46.686169] nix-daemon[870]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4936worker2 # [ 46.783508] nix-daemon[870]: accepted connection from pid 968, user nix-grpc-daemon (trusted)4937worker2 # [ 46.797787] nix-daemon[870]: reaped child process 971, status = succeeded4938lb # [ 46.826372] niks3-server[979]: 2026/09/23 04:12:03 INFO Received uploads request method=POST path=/api/pending_closures4939worker2 # [ 46.812679] nix-grpc-daemon[923]: time=2026-09-23T04:12:02.939Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4940worker2 # [ 46.814167] nix-grpc-daemon[923]: time=2026-09-23T04:12:02.940Z level=INFO msg="Uploading 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv (624B)"4941lb # [ 46.855553] niks3-server[979]: 2026/09/23 04:12:03 INFO Registered completed upload object_key=nar/1m1mvw8bmb91glp26qj699lxqv98q2iz8wm4ik3miz0xj6vi3mw6.nar.zst4942lb # [ 46.868120] niks3-server[979]: 2026/09/23 04:12:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4943lb # [ 46.874691] niks3-server[979]: 2026/09/23 04:12:03 INFO Signed narinfos id=11 count=14944worker2 # [ 46.856280] nix-grpc-daemon[923]: time=2026-09-23T04:12:02.982Z level=INFO msg="Uploading 1 narinfos"4945lb # [ 46.880129] niks3-server[979]: 2026/09/23 04:12:03 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.ls4946lb # [ 46.890976] niks3-server[979]: 2026/09/23 04:12:03 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4947lb # [ 46.897749] niks3-server[979]: 2026/09/23 04:12:03 INFO Completed upload id=114948worker2 # [ 46.876403] nix-grpc-daemon[923]: time=2026-09-23T04:12:03.002Z level=INFO msg="Upload complete. (178ms)"4949lb # [ 46.903699] niks3-server[979]: 2026/09/23 04:12:03 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.narinfo4950worker2 # [ 46.880500] nix-grpc-daemon[784]: ts=2026-09-23T04:12:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52284 duration_s=0 paths=1 nar_bytes_in=6244951worker2 # [ 46.885806] nix-grpc-daemon[784]: ts=2026-09-23T04:12:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:522844952worker2 # [ 46.897166] nix-daemon[870]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4953worker2 # [ 46.900745] nix-daemon[870]: reaped child process 966, status = succeeded4954worker2 # [ 46.905690] nix-daemon[870]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4955client # building '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' on lb:50051...4956client # farm-b> worker2: building 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv4957client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4958client # farm-b> LOG-farm-b-sig4959worker2 # [ 48.146167] nix-daemon[870]: reaped child process 975, status = succeeded4960worker2 # [ 48.230174] nix-daemon[870]: accepted connection from pid 991, user nix-grpc-daemon (trusted)4961worker2 # [ 48.242531] nix-daemon[870]: reaped child process 994, status = succeeded4962lb # [ 48.273421] niks3-server[979]: 2026/09/23 04:12:04 INFO Received uploads request method=POST path=/api/pending_closures4963worker2 # [ 48.260206] nix-grpc-daemon[923]: time=2026-09-23T04:12:04.386Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4964worker2 # [ 48.261681] nix-grpc-daemon[923]: time=2026-09-23T04:12:04.388Z level=INFO msg="Uploading 2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b (176B)"4965lb # [ 48.315676] niks3-server[979]: 2026/09/23 04:12:04 INFO Registered completed upload object_key=nar/0y3xlf15z3z540fgrc9sgnn2rl0aj3w6x9mjqd50hhqgmaxbiivd.nar.zst4966lb # [ 48.322139] niks3-server[979]: 2026/09/23 04:12:04 INFO Registered completed upload object_key=log/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv4967lb # [ 48.325888] niks3-server[979]: 2026/09/23 04:12:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4968lb # [ 48.330541] niks3-server[979]: 2026/09/23 04:12:04 INFO Signed narinfos id=12 count=14969worker2 # [ 48.311904] nix-grpc-daemon[923]: time=2026-09-23T04:12:04.438Z level=INFO msg="Uploading 1 narinfos"4970lb # [ 48.337909] niks3-server[979]: 2026/09/23 04:12:04 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.ls4971lb # [ 48.349332] niks3-server[979]: 2026/09/23 04:12:04 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4972lb # [ 48.353112] niks3-server[979]: 2026/09/23 04:12:04 INFO Completed upload id=124973lb # [ 48.354906] niks3-server[979]: 2026/09/23 04:12:04 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.narinfo4974worker2 # [ 48.333265] nix-grpc-daemon[923]: time=2026-09-23T04:12:04.459Z level=INFO msg="Upload complete. (218ms)"4975worker2 # [ 48.340131] nix-grpc-daemon[784]: ts=2026-09-23T04:12:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52284 duration_s=1 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv assign_id=6 outputs=14976worker2 # [ 48.351941] nix-grpc-daemon[784]: ts=2026-09-23T04:12:04Z level=debug event=want client=4 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv cached=04977worker2 # [ 48.355988] nix-grpc-daemon[784]: ts=2026-09-23T04:12:04Z level=debug event=assigned drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv worker=192.168.1.4:50051 clients=14978worker2 # [ 48.365123] nix-daemon[870]: reaped child process 973, status = succeeded4979worker2 # [ 48.396790] nix-grpc-daemon[784]: ts=2026-09-23T04:12:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:522844980worker2 # [ 48.408939] nix-grpc-daemon[784]: ts=2026-09-23T04:12:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:522844981worker2 # [ 48.422570] nix-grpc-daemon[923]: time=2026-09-23T04:12:04.548Z level=INFO msg="All 2 paths already cached"4982client # copying 1 paths to 'lb:50051'...4983client # copying path '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' to 'lb:50051'...4984worker2 # [ 48.432331] nix-grpc-daemon[784]: ts=2026-09-23T04:12:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:522844985worker2 # [ 48.440628] nix-daemon[870]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4986worker2 # [ 48.538702] nix-daemon[870]: accepted connection from pid 1000, user nix-grpc-daemon (trusted)4987worker2 # [ 48.551405] nix-daemon[870]: reaped child process 1003, status = succeeded4988lb # [ 48.581267] niks3-server[979]: 2026/09/23 04:12:04 INFO Received uploads request method=POST path=/api/pending_closures4989worker2 # [ 48.567970] nix-grpc-daemon[923]: time=2026-09-23T04:12:04.694Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4990worker2 # [ 48.569483] nix-grpc-daemon[923]: time=2026-09-23T04:12:04.695Z level=INFO msg="Uploading cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv (752B)"4991lb # [ 48.611271] niks3-server[979]: 2026/09/23 04:12:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4992worker2 # [ 48.600166] nix-grpc-daemon[923]: time=2026-09-23T04:12:04.726Z level=INFO msg="Uploading 1 narinfos"4993lb # [ 48.621117] niks3-server[979]: 2026/09/23 04:12:04 INFO Signed narinfos id=13 count=14994lb # [ 48.627783] niks3-server[979]: 2026/09/23 04:12:04 INFO Registered completed upload object_key=nar/0w21abkvlzg18nqkl2x4zxs7d0376kbld1n0fkr68vcpqpq1g5zg.nar.zst4995lb # [ 48.633938] niks3-server[979]: 2026/09/23 04:12:04 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.ls4996lb # [ 48.645158] niks3-server[979]: 2026/09/23 04:12:04 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4997lb # [ 48.649983] niks3-server[979]: 2026/09/23 04:12:04 INFO Completed upload id=134998worker2 # [ 48.628446] nix-grpc-daemon[923]: time=2026-09-23T04:12:04.754Z level=INFO msg="Upload complete. (175ms)"4999worker2 # [ 48.632486] nix-grpc-daemon[784]: ts=2026-09-23T04:12:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52284 duration_s=0 paths=1 nar_bytes_in=7525000lb # [ 48.655492] niks3-server[979]: 2026/09/23 04:12:04 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.narinfo5001worker2 # [ 48.638383] nix-grpc-daemon[784]: ts=2026-09-23T04:12:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:522845002worker2 # [ 48.646642] nix-daemon[870]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5003worker2 # [ 48.652643] nix-daemon[870]: reaped child process 998, status = succeeded5004worker2 # [ 48.656813] nix-daemon[870]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5005client # building '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' on lb:50051...5006client # farm-top> worker2: building cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv5007client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5008client # farm-top> LOG-farm-top-sig5009worker2 # [ 49.690625] nix-daemon[870]: reaped child process 1007, status = succeeded5010worker2 # [ 49.768088] nix-daemon[870]: accepted connection from pid 1023, user nix-grpc-daemon (trusted)5011worker2 # [ 49.780058] nix-daemon[870]: reaped child process 1026, status = succeeded5012lb # [ 49.810145] niks3-server[979]: 2026/09/23 04:12:06 INFO Received uploads request method=POST path=/api/pending_closures5013worker2 # [ 49.796708] nix-grpc-daemon[923]: time=2026-09-23T04:12:05.922Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5014worker2 # [ 49.799060] nix-grpc-daemon[923]: time=2026-09-23T04:12:05.924Z level=INFO msg="Uploading dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top (232B)"5015lb # [ 49.853487] niks3-server[979]: 2026/09/23 04:12:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign5016lb # [ 49.858329] niks3-server[979]: 2026/09/23 04:12:06 INFO Signed narinfos id=14 count=15017worker2 # [ 49.845062] nix-grpc-daemon[923]: time=2026-09-23T04:12:05.971Z level=INFO msg="Uploading 1 narinfos"5018lb # [ 49.869288] niks3-server[979]: 2026/09/23 04:12:06 INFO Registered completed upload object_key=nar/0h4zvhqjxsvdqli2l6i5ajh5sh40hz57gzchdirnvbisq2aks0k8.nar.zst5019lb # [ 49.877111] niks3-server[979]: 2026/09/23 04:12:06 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.ls5020lb # [ 49.884928] niks3-server[979]: 2026/09/23 04:12:06 INFO Registered completed upload object_key=log/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv5021lb # [ 49.888709] niks3-server[979]: 2026/09/23 04:12:06 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete5022lb # [ 49.893550] niks3-server[979]: 2026/09/23 04:12:06 INFO Completed upload id=145023worker2 # [ 49.872268] nix-grpc-daemon[923]: time=2026-09-23T04:12:05.998Z level=INFO msg="Upload complete. (208ms)"5024lb # [ 49.895660] niks3-server[979]: 2026/09/23 04:12:06 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.narinfo5025worker2 # [ 49.878678] nix-grpc-daemon[784]: ts=2026-09-23T04:12:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52284 duration_s=1 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv assign_id=7 outputs=15026worker2 # [ 49.893539] nix-daemon[870]: reaped child process 1005, status = succeeded5027client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&2, in 5.13 seconds)5028client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath5029client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath, in 0.12 seconds)5030client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5031worker1 # [ 50.136848] nix-grpc-daemon[769]: ts=2026-09-23T04:12:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:454925032client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):5033client # /nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a5034client # /nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b5035client # /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5036worker1 # [ 50.164440] nix-grpc-daemon[769]: ts=2026-09-23T04:12:06Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:454925037worker1 # [ 50.171078] nix-grpc-daemon[769]: copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'http://lb:5751'...5038worker1 # [ 50.186645] nix-grpc-daemon[769]: copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'http://lb:5751'...5039worker1 # [ 50.200793] nix-grpc-daemon[769]: copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'http://lb:5751'...5040client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.21 seconds)5041client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5042worker2 # [ 50.363810] nix-grpc-daemon[784]: ts=2026-09-23T04:12:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:431745043worker2 # [ 50.369968] nix-grpc-daemon[784]: ts=2026-09-23T04:12:06Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:431745044client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.17 seconds)5045client: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5046client # finding garbage collector roots...5047client # removing stale temporary roots file "/nix/var/nix/temproots/882"5048client # removing stale temporary roots file "/nix/var/nix/temproots/869"5049client # removing stale temporary roots file "/nix/var/nix/temproots/890"5050client # removing stale temporary roots file "/nix/var/nix/temproots/905"5051client # removing stale temporary roots file "/nix/var/nix/temproots/849"5052client # removing stale temporary roots file "/nix/var/nix/temproots/811"5053client # removing stale temporary roots file "/nix/var/nix/temproots/828"5054client # removing stale temporary roots file "/nix/var/nix/temproots/861"5055client # deleting specified paths...5056client # deleting '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top'5057client # deleting unused links...5058client # note: hard linking is currently saving 0.0 KiB5059worker2 # [ 50.668334] nix-grpc-daemon[784]: ts=2026-09-23T04:12:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:522845060worker1 # [ 50.663285] nix-grpc-daemon[769]: ts=2026-09-23T04:12:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536025061worker1 # [ 50.674521] nix-grpc-daemon[860]: time=2026-09-23T04:12:06.759Z level=INFO msg="All 1 paths already cached"5062worker2 # [ 50.688831] nix-grpc-daemon[784]: ts=2026-09-23T04:12:06Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:522845063worker1 # [ 50.688473] nix-grpc-daemon[769]: ts=2026-09-23T04:12:06Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:536025064client # copying 3 paths...5065worker2 # [ 50.706696] nix-grpc-daemon[784]: ts=2026-09-23T04:12:06Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:522845066client # copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'grpc://lb:50051'...5067worker2 # [ 50.723114] nix-grpc-daemon[784]: ts=2026-09-23T04:12:06Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:522845068worker2 # [ 50.728732] nix-grpc-daemon[784]: ts=2026-09-23T04:12:06Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:52284 duration_s=0 paths=1 nar_bytes_out=2325069worker1 # [ 50.722812] nix-grpc-daemon[769]: ts=2026-09-23T04:12:06Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:536025070worker1 # [ 50.728534] nix-grpc-daemon[769]: ts=2026-09-23T04:12:06Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53602 duration_s=0 paths=1 nar_bytes_out=1765071worker2 # [ 50.737698] nix-grpc-daemon[784]: ts=2026-09-23T04:12:06Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:522845072worker2 # [ 50.742142] nix-grpc-daemon[784]: ts=2026-09-23T04:12:06Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:52284 duration_s=0 paths=1 nar_bytes_out=1285073client # copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'grpc://lb:50051'...5074client # copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'grpc://lb:50051'...5075client: (finished: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.40 seconds)5076(finished: subtest: outputs read back through any worker carry the cache signature, in 6.03 seconds)5077subtest: two clients wanting the same drv build it once5078client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup5079client # [ 50.824515] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.5080client # Running as unit: dedup1.service; invocation ID: 118f14da33a34081933bb06baf4886635081client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 0.03 seconds)5082client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup5083client # [ 50.958788] nix[958]: don't know how to build these paths:5084client # [ 50.960159] nix[958]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5085worker1 # [ 50.974368] nix-grpc-daemon[769]: ts=2026-09-23T04:12:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:536025086worker2 # [ 50.993922] nix-grpc-daemon[784]: ts=2026-09-23T04:12:07Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=05087worker2 # [ 50.998366] nix-grpc-daemon[784]: ts=2026-09-23T04:12:07Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.3:50051 clients=15088worker1 # [ 51.029573] nix-grpc-daemon[769]: ts=2026-09-23T04:12:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536025089worker1 # [ 51.041634] nix-grpc-daemon[769]: ts=2026-09-23T04:12:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536025090client # [ 51.036228] nix[958]: copying 1 paths to 'lb:50051'...5091worker1 # [ 51.053512] nix-grpc-daemon[769]: ts=2026-09-23T04:12:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536025092client # [ 51.038421] nix[958]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...5093worker1 # [ 51.067387] nix-daemon[846]: accepted connection from pid 769, user nix-grpc-daemon (trusted)5094worker1 # [ 51.179125] nix-daemon[846]: accepted connection from pid 1011, user nix-grpc-daemon (trusted)5095worker1 # [ 51.195971] nix-daemon[846]: reaped child process 1014, status = succeeded5096lb # [ 51.230918] niks3-server[979]: 2026/09/23 04:12:07 INFO Received uploads request method=POST path=/api/pending_closures5097worker1 # [ 51.209664] nix-grpc-daemon[860]: time=2026-09-23T04:12:07.295Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5098worker1 # [ 51.211430] nix-grpc-daemon[860]: time=2026-09-23T04:12:07.297Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"5099lb # [ 51.263894] niks3-server[979]: 2026/09/23 04:12:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign5100lb # [ 51.272582] niks3-server[979]: 2026/09/23 04:12:07 INFO Signed narinfos id=15 count=15101worker1 # [ 51.246464] nix-grpc-daemon[860]: time=2026-09-23T04:12:07.331Z level=INFO msg="Uploading 1 narinfos"5102lb # [ 51.278565] niks3-server[979]: 2026/09/23 04:12:07 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls5103lb # [ 51.284122] niks3-server[979]: 2026/09/23 04:12:07 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst5104lb # [ 51.299548] niks3-server[979]: 2026/09/23 04:12:07 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete5105lb # [ 51.307220] niks3-server[979]: 2026/09/23 04:12:07 INFO Completed upload id=155106worker1 # [ 51.278070] nix-grpc-daemon[860]: time=2026-09-23T04:12:07.363Z level=INFO msg="Upload complete. (198ms)"5107lb # [ 51.310861] niks3-server[979]: 2026/09/23 04:12:07 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo5108worker1 # [ 51.282811] nix-grpc-daemon[769]: ts=2026-09-23T04:12:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53602 duration_s=0 paths=1 nar_bytes_in=4325109worker1 # [ 51.289319] nix-grpc-daemon[769]: ts=2026-09-23T04:12:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536025110worker1 # [ 51.298832] nix-daemon[846]: accepted connection from pid 769, user nix-grpc-daemon (trusted)5111worker1 # [ 51.309138] nix-daemon[846]: reaped child process 1009, status = succeeded5112worker1 # [ 51.312584] nix-daemon[846]: accepted connection from pid 769, user nix-grpc-daemon (trusted)5113client # [ 51.295941] nix[958]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...5114client # [ 51.881748] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.5115client # Running as unit: dedup2.service; invocation ID: d4b784c78de2426bb2e9eecf9ed9caef5116client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 1.07 seconds)5117worker2 # [ 52.074236] nix-grpc-daemon[784]: ts=2026-09-23T04:12:08Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:522845118client # [ 52.057346] nix[974]: don't know how to build these paths:5119client # [ 52.058241] nix[974]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5120worker2 # [ 52.091941] nix-grpc-daemon[784]: ts=2026-09-23T04:12:08Z level=debug event=want client=6 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=05121worker1 # [ 52.087331] nix-grpc-daemon[769]: ts=2026-09-23T04:12:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536025122client # [ 52.076063] nix[974]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...5123client: must succeed: sleep 25124client: (finished: must succeed: sleep 2, in 2.04 seconds)5125client: waiting for success: ! systemctl is-active dedup1 dedup25126worker1 # [ 55.340098] nix-daemon[846]: reaped child process 1018, status = succeeded5127worker1 # [ 55.425390] nix-daemon[846]: accepted connection from pid 1048, user nix-grpc-daemon (trusted)5128worker1 # [ 55.438495] nix-daemon[846]: reaped child process 1051, status = succeeded5129lb # [ 55.475635] niks3-server[979]: 2026/09/23 04:12:11 INFO Received uploads request method=POST path=/api/pending_closures5130worker1 # [ 55.454619] nix-grpc-daemon[860]: time=2026-09-23T04:12:11.540Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5131worker1 # [ 55.456590] nix-grpc-daemon[860]: time=2026-09-23T04:12:11.542Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"5132lb # [ 55.523696] niks3-server[979]: 2026/09/23 04:12:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign5133lb # [ 55.527889] niks3-server[979]: 2026/09/23 04:12:11 INFO Signed narinfos id=16 count=15134worker1 # [ 55.501697] nix-grpc-daemon[860]: time=2026-09-23T04:12:11.587Z level=INFO msg="Uploading 1 narinfos"5135lb # [ 55.539458] niks3-server[979]: 2026/09/23 04:12:11 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls5136lb # [ 55.544181] niks3-server[979]: 2026/09/23 04:12:11 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst5137lb # [ 55.548899] niks3-server[979]: 2026/09/23 04:12:11 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5138lb # [ 55.558607] niks3-server[979]: 2026/09/23 04:12:11 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete5139lb # [ 55.564373] niks3-server[979]: 2026/09/23 04:12:11 INFO Completed upload id=165140lb # [ 55.566720] niks3-server[979]: 2026/09/23 04:12:11 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo5141worker1 # [ 55.536448] nix-grpc-daemon[860]: time=2026-09-23T04:12:11.622Z level=INFO msg="Upload complete. (212ms)"5142worker1 # [ 55.543850] nix-grpc-daemon[769]: ts=2026-09-23T04:12:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53602 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=15143worker1 # [ 55.551361] nix-grpc-daemon[769]: ts=2026-09-23T04:12:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53602 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=15144worker1 # [ 55.564514] nix-daemon[846]: reaped child process 1016, status = succeeded5145client # [ 55.559724] systemd[1]: dedup1.service: Deactivated successfully.5146client # [ 55.564540] systemd[1]: dedup1.service: Consumed 122ms CPU time over 4.738s wall clock time, 43.5M memory peak, 7.1K incoming IP traffic, 8.3K outgoing IP traffic.5147client # [ 55.597603] systemd[1]: dedup2.service: Deactivated successfully.5148client # [ 55.600896] systemd[1]: dedup2.service: Consumed 144ms CPU time over 3.718s wall clock time, 35.4M memory peak, 6K incoming IP traffic, 5.8K outgoing IP traffic.5149lb # [ 55.891977] envoy[914]: conn peer=[2001:db8:1::1]:53286 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9481 tx=4354 ms=49865150lb # [ 55.896357] envoy[914]: conn peer=[2001:db8:1::1]:53294 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1051 tx=2889 ms=1415151lb # [ 55.900416] envoy[914]: conn peer=[2001:db8:1::1]:53316 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=330 ms=705152lb # [ 55.903819] envoy[914]: conn peer=[2001:db8:1::1]:53306 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=408 tx=376 ms=785153lb # [ 55.907754] envoy[914]: conn peer=[2001:db8:1::1]:53302 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=410 ms=865154lb # [ 55.911175] envoy[914]: conn peer=[2001:db8:1::1]:53332 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2981 tx=1519 ms=45835155lb # [ 55.914988] envoy[914]: conn peer=[2001:db8:1::1]:53344 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1440 tx=1194 ms=35375156client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.09 seconds)5157client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }5158client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)5159worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true5160worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)5161worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true5162worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)5163worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5164worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5165worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5166worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5167(finished: subtest: two clients wanting the same drv build it once, in 5.76 seconds)5168subtest: --timeout and --max-silent-time reach the builder5169worker2 # [ 56.729828] nix-grpc-daemon[784]: ts=2026-09-23T04:12:12Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:522845170worker2 # [ 56.740164] nix-grpc-daemon[784]: ts=2026-09-23T04:12:12Z level=debug event=want client=7 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv cached=05171worker2 # [ 56.745561] nix-grpc-daemon[784]: ts=2026-09-23T04:12:12Z level=debug event=assigned drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv worker=192.168.1.4:50051 clients=15172worker2 # [ 56.751378] nix-grpc-daemon[784]: ts=2026-09-23T04:12:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:522845173worker2 # [ 56.762113] nix-grpc-daemon[784]: ts=2026-09-23T04:12:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:522845174worker2 # [ 56.773065] nix-grpc-daemon[784]: ts=2026-09-23T04:12:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:522845175worker2 # [ 56.783506] nix-daemon[870]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5176worker2 # [ 56.879991] nix-daemon[870]: accepted connection from pid 1057, user nix-grpc-daemon (trusted)5177worker2 # [ 56.893971] nix-daemon[870]: reaped child process 1060, status = succeeded5178lb # [ 56.922559] niks3-server[979]: 2026/09/23 04:12:13 INFO Received uploads request method=POST path=/api/pending_closures5179worker2 # [ 56.909685] nix-grpc-daemon[923]: time=2026-09-23T04:12:13.035Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5180worker2 # [ 56.911637] nix-grpc-daemon[923]: time=2026-09-23T04:12:13.037Z level=INFO msg="Uploading qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv (424B)"5181lb # [ 56.953095] niks3-server[979]: 2026/09/23 04:12:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign5182worker2 # [ 56.938046] nix-grpc-daemon[923]: time=2026-09-23T04:12:13.064Z level=INFO msg="Uploading 1 narinfos"5183lb # [ 56.959194] niks3-server[979]: 2026/09/23 04:12:13 INFO Signed narinfos id=17 count=15184lb # [ 56.968476] niks3-server[979]: 2026/09/23 04:12:13 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.ls5185lb # [ 56.978701] niks3-server[979]: 2026/09/23 04:12:13 INFO Registered completed upload object_key=nar/1s4sciw269ksj1s7ymwz5cy4dw9n7bxv3k8wqw69h694spdh4ha2.nar.zst5186lb # [ 56.987179] niks3-server[979]: 2026/09/23 04:12:13 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete5187lb # [ 56.992785] niks3-server[979]: 2026/09/23 04:12:13 INFO Completed upload id=175188worker2 # [ 56.971313] nix-grpc-daemon[923]: time=2026-09-23T04:12:13.097Z level=INFO msg="Upload complete. (176ms)"5189worker2 # [ 56.975911] nix-grpc-daemon[784]: ts=2026-09-23T04:12:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52284 duration_s=0 paths=1 nar_bytes_in=4245190worker2 # [ 56.982348] nix-grpc-daemon[784]: ts=2026-09-23T04:12:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:522845191worker2 # [ 56.991475] nix-daemon[870]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5192worker2 # [ 56.996089] nix-daemon[870]: reaped child process 1055, status = succeeded5193worker2 # [ 57.001120] nix-daemon[870]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5194worker2 # [ 63.213593] nix-grpc-daemon[784]: ts=2026-09-23T04:12:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52284 duration_s=6 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv assign_id=9 outputs=05195worker2 # [ 63.225190] nix-daemon[870]: reaped child process 1064, status = succeeded5196worker2 # [ 63.226263] nix-daemon[870]: reaped child process 1062, status = succeeded5197worker1 # [ 63.416619] nix-grpc-daemon[769]: ts=2026-09-23T04:12:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:536025198worker2 # [ 63.432407] nix-grpc-daemon[784]: ts=2026-09-23T04:12:19Z level=debug event=want client=8 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv cached=05199worker2 # [ 63.436962] nix-grpc-daemon[784]: ts=2026-09-23T04:12:19Z level=debug event=assigned drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv worker=192.168.1.3:50051 clients=15200worker1 # [ 63.431387] nix-grpc-daemon[769]: ts=2026-09-23T04:12:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536025201worker1 # [ 63.443551] nix-grpc-daemon[769]: ts=2026-09-23T04:12:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536025202worker1 # [ 63.454383] nix-grpc-daemon[769]: ts=2026-09-23T04:12:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536025203worker1 # [ 63.465152] nix-daemon[846]: accepted connection from pid 769, user nix-grpc-daemon (trusted)5204worker1 # [ 63.571990] nix-daemon[846]: accepted connection from pid 1064, user nix-grpc-daemon (trusted)5205worker1 # [ 63.586049] nix-daemon[846]: reaped child process 1067, status = succeeded5206lb # [ 63.625878] niks3-server[979]: 2026/09/23 04:12:19 INFO Received uploads request method=POST path=/api/pending_closures5207worker1 # [ 63.603058] nix-grpc-daemon[860]: time=2026-09-23T04:12:19.688Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5208worker1 # [ 63.604906] nix-grpc-daemon[860]: time=2026-09-23T04:12:19.690Z level=INFO msg="Uploading fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv (424B)"5209lb # [ 63.653818] niks3-server[979]: 2026/09/23 04:12:19 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.ls5210lb # [ 63.669976] niks3-server[979]: 2026/09/23 04:12:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5211worker1 # [ 63.647651] nix-grpc-daemon[860]: time=2026-09-23T04:12:19.733Z level=INFO msg="Uploading 1 narinfos"5212lb # [ 63.677190] niks3-server[979]: 2026/09/23 04:12:19 INFO Signed narinfos id=18 count=15213lb # [ 63.681488] niks3-server[979]: 2026/09/23 04:12:19 INFO Registered completed upload object_key=nar/013y5sy2r6v0qdwv0dzpgvdfy08bds4mfh2qpvbgjazzrg2v71hp.nar.zst5214lb # [ 63.693727] niks3-server[979]: 2026/09/23 04:12:19 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5215lb # [ 63.698886] niks3-server[979]: 2026/09/23 04:12:19 INFO Completed upload id=185216worker1 # [ 63.669873] nix-grpc-daemon[860]: time=2026-09-23T04:12:19.755Z level=INFO msg="Upload complete. (192ms)"5217worker1 # [ 63.674812] nix-grpc-daemon[769]: ts=2026-09-23T04:12:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53602 duration_s=0 paths=1 nar_bytes_in=4245218worker1 # [ 63.681333] nix-grpc-daemon[769]: ts=2026-09-23T04:12:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536025219worker1 # [ 63.690637] nix-daemon[846]: accepted connection from pid 769, user nix-grpc-daemon (trusted)5220worker1 # [ 63.700782] nix-daemon[846]: reaped child process 1062, status = succeeded5221worker1 # [ 63.704793] nix-daemon[846]: accepted connection from pid 769, user nix-grpc-daemon (trusted)5222lb # [ 65.892423] envoy[914]: conn peer=[2001:db8:1::1]:41968 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2966 tx=959 ms=65395223worker1 # [ 69.315884] nix-grpc-daemon[769]: ts=2026-09-23T04:12:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53602 duration_s=5 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv assign_id=10 outputs=05224worker1 # [ 69.327345] nix-daemon[846]: reaped child process 1069, status = succeeded5225worker1 # [ 69.328375] nix-daemon[846]: reaped child process 1071, status = succeeded5226(finished: subtest: --timeout and --max-silent-time reach the builder, in 12.90 seconds)5227subtest: the first client leaving does not fail the second5228client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share5229client # [ 69.508538] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.5230client # Running as unit: share1.service; invocation ID: d9504acc238945e7a09e9fe51fda30045231client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 0.06 seconds)5232client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share5233client # [ 69.661814] nix[1046]: don't know how to build these paths:5234client # [ 69.662892] nix[1046]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5235worker2 # [ 69.685251] nix-grpc-daemon[784]: ts=2026-09-23T04:12:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:522845236worker2 # [ 69.695605] nix-grpc-daemon[784]: ts=2026-09-23T04:12:25Z level=debug event=want client=9 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05237worker2 # [ 69.702266] nix-grpc-daemon[784]: ts=2026-09-23T04:12:25Z level=debug event=assigned drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv worker=192.168.1.4:50051 clients=15238worker2 # [ 69.707114] nix-grpc-daemon[784]: ts=2026-09-23T04:12:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:522845239worker2 # [ 69.715132] nix-grpc-daemon[784]: ts=2026-09-23T04:12:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:522845240client # [ 69.700103] nix[1046]: copying 1 paths to 'lb:50051'...5241client # [ 69.701289] nix[1046]: copying path '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' to 'lb:50051'...5242worker2 # [ 69.725918] nix-grpc-daemon[784]: ts=2026-09-23T04:12:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:522845243worker2 # [ 69.737175] nix-daemon[870]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5244worker2 # [ 69.834437] nix-daemon[870]: accepted connection from pid 1091, user nix-grpc-daemon (trusted)5245worker2 # [ 69.847052] nix-daemon[870]: reaped child process 1094, status = succeeded5246lb # [ 69.877450] niks3-server[979]: 2026/09/23 04:12:26 INFO Received uploads request method=POST path=/api/pending_closures5247worker2 # [ 69.864859] nix-grpc-daemon[923]: time=2026-09-23T04:12:25.990Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5248worker2 # [ 69.866823] nix-grpc-daemon[923]: time=2026-09-23T04:12:25.992Z level=INFO msg="Uploading j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv (432B)"5249lb # [ 69.910945] niks3-server[979]: 2026/09/23 04:12:26 INFO Registered completed upload object_key=nar/0s0b0gfhn36p0aylk1qa64cgw01bi7a8ca93iv4zg2q7wqcjcqhs.nar.zst5250lb # [ 69.920068] niks3-server[979]: 2026/09/23 04:12:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5251worker2 # [ 69.905541] nix-grpc-daemon[923]: time=2026-09-23T04:12:26.031Z level=INFO msg="Uploading 1 narinfos"5252lb # [ 69.926862] niks3-server[979]: 2026/09/23 04:12:26 INFO Signed narinfos id=19 count=15253lb # [ 69.943115] niks3-server[979]: 2026/09/23 04:12:26 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5254lb # [ 69.947221] niks3-server[979]: 2026/09/23 04:12:26 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.ls5255lb # [ 69.953981] niks3-server[979]: 2026/09/23 04:12:26 INFO Completed upload id=195256worker2 # [ 69.932502] nix-grpc-daemon[923]: time=2026-09-23T04:12:26.058Z level=INFO msg="Upload complete. (184ms)"5257lb # [ 69.957635] niks3-server[979]: 2026/09/23 04:12:26 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.narinfo5258worker2 # [ 69.937895] nix-grpc-daemon[784]: ts=2026-09-23T04:12:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52284 duration_s=0 paths=1 nar_bytes_in=4325259worker2 # [ 69.943384] nix-grpc-daemon[784]: ts=2026-09-23T04:12:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:522845260worker2 # [ 69.956166] nix-daemon[870]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5261worker2 # [ 69.963960] nix-daemon[870]: reaped child process 1089, status = succeeded5262worker2 # [ 69.968790] nix-daemon[870]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5263client # [ 69.944094] nix[1046]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5264client # [ 70.577554] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.5265client # Running as unit: share2.service; invocation ID: 056c13ab2cec470089d89af3612234385266client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 1.08 seconds)5267client # [ 70.728980] nix[1062]: don't know how to build these paths:5268client # [ 70.729882] nix[1062]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5269worker1 # [ 70.744747] nix-grpc-daemon[769]: ts=2026-09-23T04:12:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:536025270worker2 # [ 70.760533] nix-grpc-daemon[784]: ts=2026-09-23T04:12:26Z level=debug event=want client=10 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05271client # [ 70.741815] nix[1062]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5272worker2 # [ 70.766249] nix-grpc-daemon[784]: ts=2026-09-23T04:12:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:522845273client: must succeed: sleep 2; systemctl kill -s INT share15274client # [ 73.850308] systemd[1]: share1.service: Sent signal SIGINT to main process 1046 (nix) on client request.5275client # [ 73.858198] nix[1046]: shutting downj2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5276client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.06 seconds)5277client: waiting for success: ! systemctl is-active share15278client # [ 73.865186] nix[1046]: error: interrupted by the user5279client # [ 73.909904] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE5280client # [ 73.911401] systemd[1]: share1.service: Failed with result 'exit-code'.5281client # [ 73.913418] systemd[1]: share1.service: Consumed 124ms CPU time over 4.405s wall clock time, 41.4M memory peak, 6.1K incoming IP traffic, 8.1K outgoing IP traffic.5282client: (finished: waiting for success: ! systemctl is-active share1, in 1.09 seconds)5283client: waiting for success: ! systemctl is-active share25284worker2 # [ 75.188185] nix-daemon[870]: reaped child process 1098, status = succeeded5285worker2 # [ 75.265635] nix-daemon[870]: accepted connection from pid 1134, user nix-grpc-daemon (trusted)5286worker2 # [ 75.276458] nix-daemon[870]: reaped child process 1137, status = succeeded5287lb # [ 75.307716] niks3-server[979]: 2026/09/23 04:12:31 INFO Received uploads request method=POST path=/api/pending_closures5288worker2 # [ 75.296147] nix-grpc-daemon[923]: time=2026-09-23T04:12:31.422Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5289worker2 # [ 75.297933] nix-grpc-daemon[923]: time=2026-09-23T04:12:31.424Z level=INFO msg="Uploading b9lg6znd0rinl36gy6ixq1dcj6b8bhhw-slow-share (120B)"5290lb # [ 75.347124] niks3-server[979]: 2026/09/23 04:12:31 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst5291lb # [ 75.362328] niks3-server[979]: 2026/09/23 04:12:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5292worker2 # [ 75.346304] nix-grpc-daemon[923]: time=2026-09-23T04:12:31.472Z level=INFO msg="Uploading 1 narinfos"5293lb # [ 75.368133] niks3-server[979]: 2026/09/23 04:12:31 INFO Signed narinfos id=20 count=15294lb # [ 75.373569] niks3-server[979]: 2026/09/23 04:12:31 INFO Registered completed upload object_key=log/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5295lb # [ 75.378926] niks3-server[979]: 2026/09/23 04:12:31 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.ls5296lb # [ 75.390337] niks3-server[979]: 2026/09/23 04:12:31 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5297lb # [ 75.396031] niks3-server[979]: 2026/09/23 04:12:31 INFO Completed upload id=205298worker2 # [ 75.375995] nix-grpc-daemon[923]: time=2026-09-23T04:12:31.502Z level=INFO msg="Upload complete. (196ms)"5299lb # [ 75.398494] niks3-server[979]: 2026/09/23 04:12:31 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.narinfo5300worker2 # [ 75.382512] nix-grpc-daemon[784]: ts=2026-09-23T04:12:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52284 duration_s=5 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=15301worker2 # [ 75.387222] nix-grpc-daemon[784]: ts=2026-09-23T04:12:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52284 duration_s=4 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=15302worker2 # [ 75.398624] nix-daemon[870]: reaped child process 1096, status = succeeded5303client # [ 75.432436] systemd[1]: share2.service: Deactivated successfully.5304client # [ 75.435958] systemd[1]: share2.service: Consumed 125ms CPU time over 4.854s wall clock time, 35.4M memory peak, 6.1K incoming IP traffic, 5.8K outgoing IP traffic.5305lb # [ 75.892262] envoy[914]: conn peer=[2001:db8:1::1]:38708 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2965 tx=933 ms=59545306lb # [ 75.896649] envoy[914]: conn peer=[2001:db8:1::1]:38716 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2978 tx=796 ms=42235307lb # [ 75.900960] envoy[914]: conn peer=[2001:db8:1::1]:38730 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1453 tx=1197 ms=46825308client: (finished: waiting for success: ! systemctl is-active share2, in 1.09 seconds)5309client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }5310client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.03 seconds)5311(finished: subtest: the first client leaving does not fail the second, in 6.78 seconds)5312subtest: build hook: nix-daemon with builders = grpc://lb5313client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log5314client # [ 76.332473] systemd[1]: Started Nix Daemon.5315client # [ 76.422171] nix-daemon[1114]: accepted connection from pid 1111, user root (trusted)5316worker2 # [ 76.561616] nix-grpc-daemon[784]: ts=2026-09-23T04:12:32Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:522845317worker2 # [ 76.569059] nix-grpc-daemon[784]: ts=2026-09-23T04:12:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:522845318worker2 # [ 76.584143] nix-grpc-daemon[784]: ts=2026-09-23T04:12:32Z level=debug event=want client=11 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=05319worker2 # [ 76.588565] nix-grpc-daemon[784]: ts=2026-09-23T04:12:32Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.3:50051 clients=15320worker1 # [ 76.584290] nix-grpc-daemon[769]: ts=2026-09-23T04:12:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536025321worker1 # [ 76.596319] nix-grpc-daemon[769]: ts=2026-09-23T04:12:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536025322worker1 # [ 76.607319] nix-grpc-daemon[769]: ts=2026-09-23T04:12:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536025323worker1 # [ 76.618416] nix-daemon[846]: accepted connection from pid 769, user nix-grpc-daemon (trusted)5324worker1 # [ 76.721933] nix-daemon[846]: accepted connection from pid 1110, user nix-grpc-daemon (trusted)5325worker1 # [ 76.734756] nix-daemon[846]: reaped child process 1113, status = succeeded5326lb # [ 76.771255] niks3-server[979]: 2026/09/23 04:12:32 INFO Received uploads request method=POST path=/api/pending_closures5327worker1 # [ 76.749894] nix-grpc-daemon[860]: time=2026-09-23T04:12:32.835Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5328worker1 # [ 76.751808] nix-grpc-daemon[860]: time=2026-09-23T04:12:32.837Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"5329lb # [ 76.806149] niks3-server[979]: 2026/09/23 04:12:33 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls5330lb # [ 76.809723] niks3-server[979]: 2026/09/23 04:12:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5331worker1 # [ 76.789743] nix-grpc-daemon[860]: time=2026-09-23T04:12:32.875Z level=INFO msg="Uploading 1 narinfos"5332lb # [ 76.818807] niks3-server[979]: 2026/09/23 04:12:33 INFO Signed narinfos id=21 count=15333lb # [ 76.829158] niks3-server[979]: 2026/09/23 04:12:33 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst5334lb # [ 76.837871] niks3-server[979]: 2026/09/23 04:12:33 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5335lb # [ 76.843560] niks3-server[979]: 2026/09/23 04:12:33 INFO Completed upload id=215336worker1 # [ 76.814482] nix-grpc-daemon[860]: time=2026-09-23T04:12:32.900Z level=INFO msg="Upload complete. (183ms)"5337lb # [ 76.850731] niks3-server[979]: 2026/09/23 04:12:33 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo5338worker1 # [ 76.818943] nix-grpc-daemon[769]: ts=2026-09-23T04:12:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53602 duration_s=0 paths=1 nar_bytes_in=5125339worker1 # [ 76.825300] nix-grpc-daemon[769]: ts=2026-09-23T04:12:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536025340worker1 # [ 76.837186] nix-daemon[846]: accepted connection from pid 769, user nix-grpc-daemon (trusted)5341worker1 # [ 76.845067] nix-daemon[846]: reaped child process 1108, status = succeeded5342worker1 # [ 76.850115] nix-daemon[846]: accepted connection from pid 769, user nix-grpc-daemon (trusted)5343worker1 # [ 77.663647] nix-daemon[846]: reaped child process 1117, status = succeeded5344worker1 # [ 77.739712] nix-daemon[846]: accepted connection from pid 1133, user nix-grpc-daemon (trusted)5345worker1 # [ 77.752653] nix-daemon[846]: reaped child process 1136, status = succeeded5346lb # [ 77.790521] niks3-server[979]: 2026/09/23 04:12:33 INFO Received uploads request method=POST path=/api/pending_closures5347worker1 # [ 77.768207] nix-grpc-daemon[860]: time=2026-09-23T04:12:33.853Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5348worker1 # [ 77.770254] nix-grpc-daemon[860]: time=2026-09-23T04:12:33.855Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"5349lb # [ 77.828666] niks3-server[979]: 2026/09/23 04:12:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5350lb # [ 77.836201] niks3-server[979]: 2026/09/23 04:12:34 INFO Signed narinfos id=22 count=15351worker1 # [ 77.810174] nix-grpc-daemon[860]: time=2026-09-23T04:12:33.895Z level=INFO msg="Uploading 1 narinfos"5352lb # [ 77.844418] niks3-server[979]: 2026/09/23 04:12:34 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv5353lb # [ 77.850977] niks3-server[979]: 2026/09/23 04:12:34 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst5354lb # [ 77.855910] niks3-server[979]: 2026/09/23 04:12:34 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls5355lb # [ 77.866570] niks3-server[979]: 2026/09/23 04:12:34 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5356lb # [ 77.872687] niks3-server[979]: 2026/09/23 04:12:34 INFO Completed upload id=225357worker1 # [ 77.843500] nix-grpc-daemon[860]: time=2026-09-23T04:12:33.929Z level=INFO msg="Upload complete. (186ms)"5358lb # [ 77.874637] niks3-server[979]: 2026/09/23 04:12:34 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo5359worker1 # [ 77.849866] nix-grpc-daemon[769]: ts=2026-09-23T04:12:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53602 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=12 outputs=15360worker2 # [ 77.863261] nix-grpc-daemon[784]: ts=2026-09-23T04:12:33Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:522845361worker1 # [ 77.863979] nix-daemon[846]: reaped child process 1115, status = succeeded5362worker2 # [ 77.871916] nix-grpc-daemon[784]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...5363worker1 # [ 77.889931] nix-grpc-daemon[769]: ts=2026-09-23T04:12:33Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:536025364worker1 # [ 77.895246] nix-grpc-daemon[769]: ts=2026-09-23T04:12:33Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53602 duration_s=0 paths=1 nar_bytes_out=1285365worker1 # [ 78.071291] nix-grpc-daemon[769]: ts=2026-09-23T04:12:34Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:536025366worker2 # [ 78.083130] nix-grpc-daemon[784]: ts=2026-09-23T04:12:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:522845367worker2 # [ 78.093201] nix-grpc-daemon[923]: time=2026-09-23T04:12:34.218Z level=INFO msg="All 1 paths already cached"5368worker2 # [ 78.106601] nix-grpc-daemon[784]: ts=2026-09-23T04:12:34Z level=debug event=want client=12 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=05369worker2 # [ 78.111132] nix-grpc-daemon[784]: ts=2026-09-23T04:12:34Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.3:50051 clients=15370worker1 # [ 78.105580] nix-grpc-daemon[769]: ts=2026-09-23T04:12:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536025371worker1 # [ 78.117115] nix-grpc-daemon[769]: ts=2026-09-23T04:12:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536025372worker1 # [ 78.129615] nix-grpc-daemon[860]: time=2026-09-23T04:12:34.215Z level=INFO msg="All 1 paths already cached"5373worker1 # [ 78.137461] nix-grpc-daemon[769]: ts=2026-09-23T04:12:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536025374worker1 # [ 78.146325] nix-daemon[846]: accepted connection from pid 769, user nix-grpc-daemon (trusted)5375worker1 # [ 78.252264] nix-daemon[846]: accepted connection from pid 1140, user nix-grpc-daemon (trusted)5376worker1 # [ 78.266728] nix-daemon[846]: reaped child process 1143, status = succeeded5377lb # [ 78.301884] niks3-server[979]: 2026/09/23 04:12:34 INFO Received uploads request method=POST path=/api/pending_closures5378worker1 # [ 78.280739] nix-grpc-daemon[860]: time=2026-09-23T04:12:34.366Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5379worker1 # [ 78.282289] nix-grpc-daemon[860]: time=2026-09-23T04:12:34.368Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"5380lb # [ 78.335482] niks3-server[979]: 2026/09/23 04:12:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5381worker1 # [ 78.311643] nix-grpc-daemon[860]: time=2026-09-23T04:12:34.397Z level=INFO msg="Uploading 1 narinfos"5382lb # [ 78.341677] niks3-server[979]: 2026/09/23 04:12:34 INFO Signed narinfos id=23 count=15383lb # [ 78.346747] niks3-server[979]: 2026/09/23 04:12:34 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls5384lb # [ 78.351152] niks3-server[979]: 2026/09/23 04:12:34 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst5385lb # [ 78.364649] niks3-server[979]: 2026/09/23 04:12:34 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5386lb # [ 78.371221] niks3-server[979]: 2026/09/23 04:12:34 INFO Completed upload id=235387worker1 # [ 78.341728] nix-grpc-daemon[860]: time=2026-09-23T04:12:34.427Z level=INFO msg="Upload complete. (181ms)"5388lb # [ 78.374051] niks3-server[979]: 2026/09/23 04:12:34 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo5389worker1 # [ 78.345141] nix-grpc-daemon[769]: ts=2026-09-23T04:12:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53602 duration_s=0 paths=1 nar_bytes_in=6245390worker1 # [ 78.351338] nix-grpc-daemon[769]: ts=2026-09-23T04:12:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536025391worker1 # [ 78.359075] nix-daemon[846]: accepted connection from pid 769, user nix-grpc-daemon (trusted)5392worker1 # [ 78.366587] nix-daemon[846]: reaped child process 1138, status = succeeded5393worker1 # [ 78.372077] nix-daemon[846]: accepted connection from pid 769, user nix-grpc-daemon (trusted)5394worker1 # [ 78.985937] nix-daemon[846]: reaped child process 1147, status = succeeded5395worker1 # [ 79.058458] nix-daemon[846]: accepted connection from pid 1163, user nix-grpc-daemon (trusted)5396worker1 # [ 79.071099] nix-daemon[846]: reaped child process 1166, status = succeeded5397lb # [ 79.107989] niks3-server[979]: 2026/09/23 04:12:35 INFO Received uploads request method=POST path=/api/pending_closures5398worker1 # [ 79.087337] nix-grpc-daemon[860]: time=2026-09-23T04:12:35.172Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5399worker1 # [ 79.089032] nix-grpc-daemon[860]: time=2026-09-23T04:12:35.174Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"5400lb # [ 79.148607] niks3-server[979]: 2026/09/23 04:12:35 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv5401lb # [ 79.156322] niks3-server[979]: 2026/09/23 04:12:35 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst5402lb # [ 79.166962] niks3-server[979]: 2026/09/23 04:12:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5403worker1 # [ 79.141908] nix-grpc-daemon[860]: time=2026-09-23T04:12:35.227Z level=INFO msg="Uploading 1 narinfos"5404lb # [ 79.171460] niks3-server[979]: 2026/09/23 04:12:35 INFO Signed narinfos id=24 count=15405lb # [ 79.177416] niks3-server[979]: 2026/09/23 04:12:35 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls5406lb # [ 79.186637] niks3-server[979]: 2026/09/23 04:12:35 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5407lb # [ 79.189633] niks3-server[979]: 2026/09/23 04:12:35 INFO Completed upload id=245408worker1 # [ 79.160271] nix-grpc-daemon[860]: time=2026-09-23T04:12:35.245Z level=INFO msg="Upload complete. (183ms)"5409lb # [ 79.192553] niks3-server[979]: 2026/09/23 04:12:35 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo5410worker1 # [ 79.165907] nix-grpc-daemon[769]: ts=2026-09-23T04:12:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53602 duration_s=0 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=13 outputs=15411worker1 # [ 79.174894] nix-grpc-daemon[769]: ts=2026-09-23T04:12:35Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:536025412worker1 # [ 79.184970] nix-daemon[846]: reaped child process 1145, status = succeeded5413worker2 # [ 79.196400] nix-grpc-daemon[784]: ts=2026-09-23T04:12:35Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:522845414worker2 # [ 79.204633] nix-grpc-daemon[784]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...5415worker2 # [ 79.216267] nix-grpc-daemon[784]: ts=2026-09-23T04:12:35Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:52284 duration_s=0 paths=1 nar_bytes_out=1765416worker1 # [ 79.420168] nix-grpc-daemon[769]: ts=2026-09-23T04:12:35Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:536025417worker1 # [ 79.426294] nix-grpc-daemon[769]: ts=2026-09-23T04:12:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536025418worker1 # [ 79.432125] nix-grpc-daemon[860]: time=2026-09-23T04:12:35.517Z level=INFO msg="All 2 paths already cached"5419worker2 # [ 79.455177] nix-grpc-daemon[784]: ts=2026-09-23T04:12:35Z level=debug event=want client=13 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=05420worker2 # [ 79.459547] nix-grpc-daemon[784]: ts=2026-09-23T04:12:35Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.3:50051 clients=15421worker1 # [ 79.492344] nix-grpc-daemon[769]: ts=2026-09-23T04:12:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536025422worker1 # [ 79.504865] nix-grpc-daemon[769]: ts=2026-09-23T04:12:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536025423worker1 # [ 79.518167] nix-grpc-daemon[860]: time=2026-09-23T04:12:35.603Z level=INFO msg="All 2 paths already cached"5424worker1 # [ 79.526590] nix-grpc-daemon[769]: ts=2026-09-23T04:12:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536025425worker1 # [ 79.536169] nix-daemon[846]: accepted connection from pid 769, user nix-grpc-daemon (trusted)5426worker1 # [ 79.637931] nix-daemon[846]: accepted connection from pid 1170, user nix-grpc-daemon (trusted)5427worker1 # [ 79.650850] nix-daemon[846]: reaped child process 1173, status = succeeded5428lb # [ 79.688387] niks3-server[979]: 2026/09/23 04:12:35 INFO Received uploads request method=POST path=/api/pending_closures5429worker1 # [ 79.667452] nix-grpc-daemon[860]: time=2026-09-23T04:12:35.753Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5430worker1 # [ 79.668923] nix-grpc-daemon[860]: time=2026-09-23T04:12:35.754Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"5431lb # [ 79.722324] niks3-server[979]: 2026/09/23 04:12:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5432lb # [ 79.728061] niks3-server[979]: 2026/09/23 04:12:35 INFO Signed narinfos id=25 count=15433worker1 # [ 79.701955] nix-grpc-daemon[860]: time=2026-09-23T04:12:35.787Z level=INFO msg="Uploading 1 narinfos"5434lb # [ 79.737110] niks3-server[979]: 2026/09/23 04:12:35 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst5435lb # [ 79.749119] niks3-server[979]: 2026/09/23 04:12:35 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls5436lb # [ 79.751919] niks3-server[979]: 2026/09/23 04:12:35 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5437lb # [ 79.758940] niks3-server[979]: 2026/09/23 04:12:35 INFO Completed upload id=255438worker1 # [ 79.729810] nix-grpc-daemon[860]: time=2026-09-23T04:12:35.815Z level=INFO msg="Upload complete. (181ms)"5439lb # [ 79.765174] niks3-server[979]: 2026/09/23 04:12:35 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo5440worker1 # [ 79.733802] nix-grpc-daemon[769]: ts=2026-09-23T04:12:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53602 duration_s=0 paths=1 nar_bytes_in=7525441worker1 # [ 79.741158] nix-grpc-daemon[769]: ts=2026-09-23T04:12:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536025442worker1 # [ 79.749654] nix-daemon[846]: accepted connection from pid 769, user nix-grpc-daemon (trusted)5443worker1 # [ 79.759170] nix-daemon[846]: reaped child process 1168, status = succeeded5444worker1 # [ 79.762450] nix-daemon[846]: accepted connection from pid 769, user nix-grpc-daemon (trusted)5445worker1 # [ 80.389940] nix-daemon[846]: reaped child process 1177, status = succeeded5446worker1 # [ 80.459904] nix-daemon[846]: accepted connection from pid 1193, user nix-grpc-daemon (trusted)5447worker1 # [ 80.472073] nix-daemon[846]: reaped child process 1196, status = succeeded5448lb # [ 80.508214] niks3-server[979]: 2026/09/23 04:12:36 INFO Received uploads request method=POST path=/api/pending_closures5449worker1 # [ 80.487455] nix-grpc-daemon[860]: time=2026-09-23T04:12:36.573Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5450worker1 # [ 80.489137] nix-grpc-daemon[860]: time=2026-09-23T04:12:36.574Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"5451lb # [ 80.551358] niks3-server[979]: 2026/09/23 04:12:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5452lb # [ 80.557859] niks3-server[979]: 2026/09/23 04:12:36 INFO Signed narinfos id=26 count=15453worker1 # [ 80.531645] nix-grpc-daemon[860]: time=2026-09-23T04:12:36.617Z level=INFO msg="Uploading 1 narinfos"5454lb # [ 80.565720] niks3-server[979]: 2026/09/23 04:12:36 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls5455lb # [ 80.570375] niks3-server[979]: 2026/09/23 04:12:36 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv5456lb # [ 80.574600] niks3-server[979]: 2026/09/23 04:12:36 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst5457lb # [ 80.584902] niks3-server[979]: 2026/09/23 04:12:36 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5458lb # [ 80.591211] niks3-server[979]: 2026/09/23 04:12:36 INFO Completed upload id=265459worker1 # [ 80.563336] nix-grpc-daemon[860]: time=2026-09-23T04:12:36.649Z level=INFO msg="Upload complete. (194ms)"5460lb # [ 80.595459] niks3-server[979]: 2026/09/23 04:12:36 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo5461worker1 # [ 80.570260] nix-grpc-daemon[769]: ts=2026-09-23T04:12:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53602 duration_s=0 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=14 outputs=15462worker1 # [ 80.576824] nix-grpc-daemon[769]: ts=2026-09-23T04:12:36Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:536025463worker1 # [ 80.588721] nix-daemon[846]: reaped child process 1175, status = succeeded5464worker2 # [ 80.603921] nix-grpc-daemon[784]: ts=2026-09-23T04:12:36Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:522845465worker2 # [ 80.613639] nix-grpc-daemon[784]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...5466worker2 # [ 80.625220] nix-grpc-daemon[784]: ts=2026-09-23T04:12:36Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:52284 duration_s=0 paths=1 nar_bytes_out=2325467client # [ 80.713887] nix-daemon[1114]: reaped child process 1116, status = succeeded5468client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log, in 4.48 seconds)5469client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5470client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.02 seconds)5471client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5472client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.02 seconds)5473client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5474client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.02 seconds)5475(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 4.55 seconds)5476subtest: an input only one worker has still reaches the builder5477client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only5478client # this derivation will be built:5479client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5480client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5481client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only, in 0.25 seconds)5482client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5483client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.10 seconds)5484worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5485worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.12 seconds)5486worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5487worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)5488client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&25489client # [ 81.412453] nix-daemon[1114]: accepted connection from pid 1197, user root (trusted)5490client # this derivation will be built:5491client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5492worker1 # [ 81.554437] nix-grpc-daemon[769]: ts=2026-09-23T04:12:37Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:536025493client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5494worker2 # [ 81.565402] nix-grpc-daemon[784]: ts=2026-09-23T04:12:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:522845495client # copying 1 paths...5496client # copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' to 'grpc://lb:50051'...5497worker2 # [ 81.578358] nix-grpc-daemon[784]: ts=2026-09-23T04:12:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:522845498worker2 # [ 81.590253] nix-daemon[870]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5499worker2 # [ 81.704207] nix-daemon[870]: accepted connection from pid 1153, user nix-grpc-daemon (trusted)5500worker2 # [ 81.718592] nix-daemon[870]: reaped child process 1156, status = succeeded5501lb # [ 81.746821] niks3-server[979]: 2026/09/23 04:12:37 INFO Received uploads request method=POST path=/api/pending_closures5502worker2 # [ 81.734169] nix-grpc-daemon[923]: time=2026-09-23T04:12:37.859Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5503worker2 # [ 81.735589] nix-grpc-daemon[923]: time=2026-09-23T04:12:37.860Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5504lb # [ 81.774087] niks3-server[979]: 2026/09/23 04:12:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5505lb # [ 81.781696] niks3-server[979]: 2026/09/23 04:12:37 INFO Signed narinfos id=27 count=15506worker2 # [ 81.763286] nix-grpc-daemon[923]: time=2026-09-23T04:12:37.889Z level=INFO msg="Uploading 1 narinfos"5507lb # [ 81.791141] niks3-server[979]: 2026/09/23 04:12:37 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5508lb # [ 81.801932] niks3-server[979]: 2026/09/23 04:12:38 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5509lb # [ 81.810343] niks3-server[979]: 2026/09/23 04:12:38 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5510lb # [ 81.815099] niks3-server[979]: 2026/09/23 04:12:38 INFO Completed upload id=275511worker2 # [ 81.796668] nix-grpc-daemon[923]: time=2026-09-23T04:12:37.922Z level=INFO msg="Upload complete. (194ms)"5512lb # [ 81.819144] niks3-server[979]: 2026/09/23 04:12:38 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5513worker2 # [ 81.802243] nix-grpc-daemon[784]: ts=2026-09-23T04:12:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52284 duration_s=0 paths=1 nar_bytes_in=1205514worker2 # [ 81.814379] nix-daemon[870]: reaped child process 1151, status = succeeded5515worker2 # [ 81.818400] nix-grpc-daemon[784]: ts=2026-09-23T04:12:37Z level=debug event=want client=14 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05516worker2 # [ 81.822709] nix-grpc-daemon[784]: ts=2026-09-23T04:12:37Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=15517worker2 # [ 81.863248] nix-grpc-daemon[784]: ts=2026-09-23T04:12:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:522845518worker2 # [ 81.875963] nix-grpc-daemon[784]: ts=2026-09-23T04:12:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:522845519client # copying 2 paths to 'lb:50051'...5520client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5521worker2 # [ 81.891524] nix-grpc-daemon[784]: ts=2026-09-23T04:12:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:522845522client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5523worker2 # [ 81.905171] nix-daemon[870]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5524worker2 # [ 82.005289] nix-daemon[870]: accepted connection from pid 1162, user nix-grpc-daemon (trusted)5525worker2 # [ 82.018643] nix-daemon[870]: reaped child process 1165, status = succeeded5526lb # [ 82.047717] niks3-server[979]: 2026/09/23 04:12:38 INFO Received uploads request method=POST path=/api/pending_closures5527lb # [ 82.057330] niks3-server[979]: 2026/09/23 04:12:38 INFO Received uploads request method=POST path=/api/pending_closures5528worker2 # [ 82.044721] nix-grpc-daemon[923]: time=2026-09-23T04:12:38.171Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5529worker2 # [ 82.046250] nix-grpc-daemon[923]: time=2026-09-23T04:12:38.172Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5530worker2 # [ 82.048026] nix-grpc-daemon[923]: time=2026-09-23T04:12:38.174Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5531lb # [ 82.101059] niks3-server[979]: 2026/09/23 04:12:38 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5532lb # [ 82.111645] niks3-server[979]: 2026/09/23 04:12:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5533lb # [ 82.113967] niks3-server[979]: 2026/09/23 04:12:38 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5534lb # [ 82.117338] niks3-server[979]: 2026/09/23 04:12:38 INFO Signed narinfos id=28 count=15535lb # [ 82.119630] niks3-server[979]: 2026/09/23 04:12:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5536lb # [ 82.121679] niks3-server[979]: 2026/09/23 04:12:38 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5537lb # [ 82.123627] niks3-server[979]: 2026/09/23 04:12:38 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5538lb # [ 82.125193] niks3-server[979]: 2026/09/23 04:12:38 INFO Signed narinfos id=29 count=25539worker2 # [ 82.102885] nix-grpc-daemon[923]: time=2026-09-23T04:12:38.228Z level=INFO msg="Uploading 3 narinfos"5540lb # [ 82.137713] niks3-server[979]: 2026/09/23 04:12:38 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5541lb # [ 82.140840] niks3-server[979]: 2026/09/23 04:12:38 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5542lb # [ 82.143087] niks3-server[979]: 2026/09/23 04:12:38 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5543lb # [ 82.145880] niks3-server[979]: 2026/09/23 04:12:38 INFO Completed upload id=285544lb # [ 82.147369] niks3-server[979]: 2026/09/23 04:12:38 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5545lb # [ 82.149918] niks3-server[979]: 2026/09/23 04:12:38 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5546lb # [ 82.153176] niks3-server[979]: 2026/09/23 04:12:38 INFO Completed upload id=295547worker2 # [ 82.131843] nix-grpc-daemon[923]: time=2026-09-23T04:12:38.258Z level=INFO msg="Upload complete. (213ms)"5548worker2 # [ 82.136787] nix-grpc-daemon[784]: ts=2026-09-23T04:12:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52284 duration_s=0 paths=2 nar_bytes_in=9765549worker2 # [ 82.143277] nix-grpc-daemon[784]: ts=2026-09-23T04:12:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:522845550worker2 # [ 82.154225] nix-daemon[870]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5551worker2 # [ 82.164622] nix-daemon[870]: reaped child process 1160, status = succeeded5552worker2 # [ 82.171544] nix-daemon[870]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5553client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5554client # uses-input-w1onlya> worker2: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5555client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5556worker2 # [ 82.985170] nix-daemon[870]: reaped child process 1169, status = succeeded5557worker2 # [ 83.068134] nix-daemon[870]: accepted connection from pid 1185, user nix-grpc-daemon (trusted)5558worker2 # [ 83.083499] nix-daemon[870]: reaped child process 1188, status = succeeded5559lb # [ 83.111480] niks3-server[979]: 2026/09/23 04:12:39 INFO Received uploads request method=POST path=/api/pending_closures5560worker2 # [ 83.100532] nix-grpc-daemon[923]: time=2026-09-23T04:12:39.226Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5561lb # [ 83.143156] niks3-server[979]: 2026/09/23 04:12:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5562worker2 # [ 83.124964] nix-grpc-daemon[923]: time=2026-09-23T04:12:39.251Z level=INFO msg="Uploading 1 narinfos"5563lb # [ 83.148159] niks3-server[979]: 2026/09/23 04:12:39 INFO Signed narinfos id=30 count=15564lb # [ 83.152772] niks3-server[979]: 2026/09/23 04:12:39 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5565lb # [ 83.157417] niks3-server[979]: 2026/09/23 04:12:39 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5566lb # [ 83.159164] niks3-server[979]: 2026/09/23 04:12:39 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5567lb # [ 83.164686] niks3-server[979]: 2026/09/23 04:12:39 INFO Completed upload id=305568worker2 # [ 83.144617] nix-grpc-daemon[923]: time=2026-09-23T04:12:39.270Z level=INFO msg="Upload complete. (165ms)"5569lb # [ 83.166568] niks3-server[979]: 2026/09/23 04:12:39 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5570worker2 # [ 83.149058] nix-grpc-daemon[784]: ts=2026-09-23T04:12:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52284 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=15 outputs=15571client # copying 1 paths...5572worker2 # [ 83.155350] nix-grpc-daemon[784]: ts=2026-09-23T04:12:39Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:522845573worker2 # [ 83.158565] nix-daemon[870]: reaped child process 1167, status = succeeded5574client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5575worker1 # [ 83.166623] nix-grpc-daemon[769]: ts=2026-09-23T04:12:39Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:536025576worker1 # [ 83.175828] nix-grpc-daemon[769]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5577worker1 # [ 83.187968] nix-grpc-daemon[769]: ts=2026-09-23T04:12:39Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53602 duration_s=0 paths=1 nar_bytes_out=1205578client # [ 83.286359] nix-daemon[1114]: reaped child process 1200, status = succeeded5579client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.00 seconds)5580client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&25581client # [ 83.393897] nix-daemon[1114]: accepted connection from pid 1218, user root (trusted)5582client # this derivation will be built:5583client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5584worker2 # [ 83.541740] nix-grpc-daemon[784]: ts=2026-09-23T04:12:39Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:522845585client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5586worker2 # [ 83.547318] nix-grpc-daemon[784]: ts=2026-09-23T04:12:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:522845587worker2 # [ 83.557407] nix-grpc-daemon[923]: time=2026-09-23T04:12:39.683Z level=INFO msg="All 1 paths already cached"5588worker2 # [ 83.572483] nix-grpc-daemon[784]: ts=2026-09-23T04:12:39Z level=debug event=want client=15 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05589worker2 # [ 83.577353] nix-grpc-daemon[784]: ts=2026-09-23T04:12:39Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15590worker1 # [ 83.609518] nix-grpc-daemon[769]: ts=2026-09-23T04:12:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536025591worker1 # [ 83.623867] nix-grpc-daemon[769]: ts=2026-09-23T04:12:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536025592client # copying 1 paths to 'lb:50051'...5593client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5594worker1 # [ 83.644357] nix-grpc-daemon[769]: ts=2026-09-23T04:12:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536025595worker1 # [ 83.656722] nix-daemon[846]: accepted connection from pid 769, user nix-grpc-daemon (trusted)5596worker1 # [ 83.673577] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 95 ms (attempt 1/5)5597worker1 # [ 83.776679] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 46 ms (attempt 2/5)5598worker1 # [ 83.824273] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 122 ms (attempt 3/5)5599worker1 # [ 83.947350] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 471 ms (attempt 4/5)5600worker1 # [ 84.424201] nix-grpc-daemon[769]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5601worker1 # [ 84.426707] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5602worker1 # [ 84.431135] nix-grpc-daemon[769]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5603worker1 # [ 84.547421] nix-daemon[846]: accepted connection from pid 1211, user nix-grpc-daemon (trusted)5604worker1 # [ 84.562794] nix-daemon[846]: reaped child process 1214, status = succeeded5605lb # [ 84.600168] niks3-server[979]: 2026/09/23 04:12:40 INFO Received uploads request method=POST path=/api/pending_closures5606worker1 # [ 84.581131] nix-grpc-daemon[860]: time=2026-09-23T04:12:40.666Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5607worker1 # [ 84.582736] nix-grpc-daemon[860]: time=2026-09-23T04:12:40.668Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5608lb # [ 84.636526] niks3-server[979]: 2026/09/23 04:12:40 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5609lb # [ 84.642378] niks3-server[979]: 2026/09/23 04:12:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5610worker1 # [ 84.619919] nix-grpc-daemon[860]: time=2026-09-23T04:12:40.705Z level=INFO msg="Uploading 1 narinfos"5611lb # [ 84.649257] niks3-server[979]: 2026/09/23 04:12:40 INFO Signed narinfos id=31 count=15612lb # [ 84.660485] niks3-server[979]: 2026/09/23 04:12:40 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5613lb # [ 84.668973] niks3-server[979]: 2026/09/23 04:12:40 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5614lb # [ 84.675345] niks3-server[979]: 2026/09/23 04:12:40 INFO Completed upload id=315615worker1 # [ 84.646226] nix-grpc-daemon[860]: time=2026-09-23T04:12:40.731Z level=INFO msg="Upload complete. (197ms)"5616lb # [ 84.676749] niks3-server[979]: 2026/09/23 04:12:40 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5617worker1 # [ 84.650591] nix-grpc-daemon[769]: ts=2026-09-23T04:12:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53602 duration_s=1 paths=1 nar_bytes_in=5605618worker1 # [ 84.657313] nix-grpc-daemon[769]: ts=2026-09-23T04:12:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536025619worker1 # [ 84.667959] nix-daemon[846]: accepted connection from pid 769, user nix-grpc-daemon (trusted)5620worker1 # [ 84.678529] nix-daemon[846]: reaped child process 1204, status = succeeded5621worker1 # [ 84.682646] nix-daemon[846]: accepted connection from pid 769, user nix-grpc-daemon (trusted)5622client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5623client # uses-input-w1onlyb> node-a: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5624client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5625worker1 # [ 85.495756] nix-daemon[846]: reaped child process 1218, status = succeeded5626worker1 # [ 85.571089] nix-daemon[846]: accepted connection from pid 1234, user nix-grpc-daemon (trusted)5627worker1 # [ 85.582549] nix-daemon[846]: reaped child process 1237, status = succeeded5628lb # [ 85.621317] niks3-server[979]: 2026/09/23 04:12:41 INFO Received uploads request method=POST path=/api/pending_closures5629worker1 # [ 85.598825] nix-grpc-daemon[860]: time=2026-09-23T04:12:41.684Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5630lb # [ 85.650152] niks3-server[979]: 2026/09/23 04:12:41 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5631lb # [ 85.660212] niks3-server[979]: 2026/09/23 04:12:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5632lb # [ 85.666641] niks3-server[979]: 2026/09/23 04:12:41 INFO Signed narinfos id=32 count=15633worker1 # [ 85.640862] nix-grpc-daemon[860]: time=2026-09-23T04:12:41.726Z level=INFO msg="Uploading 1 narinfos"5634lb # [ 85.672383] niks3-server[979]: 2026/09/23 04:12:41 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5635lb # [ 85.685422] niks3-server[979]: 2026/09/23 04:12:41 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5636lb # [ 85.691745] niks3-server[979]: 2026/09/23 04:12:41 INFO Completed upload id=325637worker1 # [ 85.662456] nix-grpc-daemon[860]: time=2026-09-23T04:12:41.748Z level=INFO msg="Upload complete. (174ms)"5638lb # [ 85.694282] niks3-server[979]: 2026/09/23 04:12:41 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5639worker1 # [ 85.667442] nix-grpc-daemon[769]: ts=2026-09-23T04:12:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53602 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=16 outputs=15640worker1 # [ 85.679647] nix-daemon[846]: reaped child process 1216, status = succeeded5641client # copying 1 paths...5642worker1 # [ 85.713058] nix-grpc-daemon[769]: ts=2026-09-23T04:12:41Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:536025643client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5644worker1 # [ 85.734769] nix-grpc-daemon[769]: ts=2026-09-23T04:12:41Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:536025645worker1 # [ 85.739587] nix-grpc-daemon[769]: ts=2026-09-23T04:12:41Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53602 duration_s=0 paths=1 nar_bytes_out=1205646client # [ 85.834442] nix-daemon[1114]: reaped child process 1221, status = succeeded5647client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.55 seconds)5648client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&25649lb # [ 85.892941] envoy[914]: conn peer=[2001:db8:1::1]:46606 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=235650lb # [ 85.896724] envoy[914]: conn peer=[2001:db8:1::1]:46594 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3380 tx=1922 ms=13605651lb # [ 85.900338] envoy[914]: conn peer=[2001:db8:1::1]:46620 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=379 ms=725652lb # [ 85.903790] envoy[914]: conn peer=[2001:db8:1::1]:46616 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3850 tx=2139 ms=11895653lb # [ 85.907616] envoy[914]: conn peer=[2001:db8:1::1]:46626 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4311 tx=2370 ms=12495654lb # [ 85.911119] envoy[914]: conn peer=[2001:db8:1::1]:46634 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=411 ms=745655lb # [ 85.915198] envoy[914]: conn peer=[2001:db8:1::1]:46638 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4216 tx=2054 ms=16865656lb # [ 85.918720] envoy[914]: conn peer=[2001:db8:1::1]:36720 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=329 ms=755657lb # [ 85.922149] envoy[914]: conn peer=[2001:db8:1::1]:36726 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3741 tx=2124 ms=22585658lb # [ 85.925656] envoy[914]: conn peer=[2001:db8:1::1]:36742 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=575659client # [ 85.941608] nix-daemon[1114]: accepted connection from pid 1240, user root (trusted)5660client # this derivation will be built:5661client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5662client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5663worker2 # [ 86.090948] nix-grpc-daemon[784]: ts=2026-09-23T04:12:42Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:522845664worker2 # [ 86.097686] nix-grpc-daemon[784]: ts=2026-09-23T04:12:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:522845665worker2 # [ 86.103556] nix-grpc-daemon[923]: time=2026-09-23T04:12:42.228Z level=INFO msg="All 1 paths already cached"5666worker2 # [ 86.113130] nix-grpc-daemon[784]: ts=2026-09-23T04:12:42Z level=debug event=want client=16 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05667worker2 # [ 86.119983] nix-grpc-daemon[784]: ts=2026-09-23T04:12:42Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15668worker2 # [ 86.125276] nix-grpc-daemon[784]: ts=2026-09-23T04:12:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:522845669worker2 # [ 86.133079] nix-grpc-daemon[784]: ts=2026-09-23T04:12:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:522845670worker2 # [ 86.145713] nix-grpc-daemon[923]: time=2026-09-23T04:12:42.271Z level=INFO msg="All 1 paths already cached"5671client # copying 1 paths to 'lb:50051'...5672client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5673worker2 # [ 86.153357] nix-grpc-daemon[784]: ts=2026-09-23T04:12:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:522845674worker2 # [ 86.164333] nix-daemon[870]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5675worker2 # [ 86.260493] nix-daemon[870]: accepted connection from pid 1196, user nix-grpc-daemon (trusted)5676worker2 # [ 86.274757] nix-daemon[870]: reaped child process 1199, status = succeeded5677lb # [ 86.303586] niks3-server[979]: 2026/09/23 04:12:42 INFO Received uploads request method=POST path=/api/pending_closures5678worker2 # [ 86.290145] nix-grpc-daemon[923]: time=2026-09-23T04:12:42.416Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5679worker2 # [ 86.291594] nix-grpc-daemon[923]: time=2026-09-23T04:12:42.417Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5680lb # [ 86.337369] niks3-server[979]: 2026/09/23 04:12:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5681lb # [ 86.344565] niks3-server[979]: 2026/09/23 04:12:42 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5682worker2 # [ 86.330333] nix-grpc-daemon[923]: time=2026-09-23T04:12:42.456Z level=INFO msg="Uploading 1 narinfos"5683lb # [ 86.351451] niks3-server[979]: 2026/09/23 04:12:42 INFO Signed narinfos id=33 count=15684lb # [ 86.360364] niks3-server[979]: 2026/09/23 04:12:42 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5685lb # [ 86.367251] niks3-server[979]: 2026/09/23 04:12:42 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5686lb # [ 86.375104] niks3-server[979]: 2026/09/23 04:12:42 INFO Completed upload id=335687worker2 # [ 86.354063] nix-grpc-daemon[923]: time=2026-09-23T04:12:42.480Z level=INFO msg="Upload complete. (179ms)"5688lb # [ 86.380066] niks3-server[979]: 2026/09/23 04:12:42 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5689worker2 # [ 86.358054] nix-grpc-daemon[784]: ts=2026-09-23T04:12:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52284 duration_s=0 paths=1 nar_bytes_in=5605690worker2 # [ 86.363254] nix-grpc-daemon[784]: ts=2026-09-23T04:12:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:522845691worker2 # [ 86.372891] nix-daemon[870]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5692worker2 # [ 86.380263] nix-daemon[870]: reaped child process 1194, status = succeeded5693worker2 # [ 86.385283] nix-daemon[870]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5694client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5695client # uses-input-w1onlyc> worker2: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5696client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5697worker2 # [ 87.201256] nix-daemon[870]: reaped child process 1203, status = succeeded5698worker2 # [ 87.279834] nix-daemon[870]: accepted connection from pid 1219, user nix-grpc-daemon (trusted)5699worker2 # [ 87.294632] nix-daemon[870]: reaped child process 1222, status = succeeded5700lb # [ 87.322569] niks3-server[979]: 2026/09/23 04:12:43 INFO Received uploads request method=POST path=/api/pending_closures5701worker2 # [ 87.310330] nix-grpc-daemon[923]: time=2026-09-23T04:12:43.435Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5702lb # [ 87.349745] niks3-server[979]: 2026/09/23 04:12:43 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5703lb # [ 87.357064] niks3-server[979]: 2026/09/23 04:12:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5704lb # [ 87.364257] niks3-server[979]: 2026/09/23 04:12:43 INFO Signed narinfos id=34 count=15705worker2 # [ 87.345909] nix-grpc-daemon[923]: time=2026-09-23T04:12:43.472Z level=INFO msg="Uploading 1 narinfos"5706lb # [ 87.377209] niks3-server[979]: 2026/09/23 04:12:43 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5707lb # [ 87.384667] niks3-server[979]: 2026/09/23 04:12:43 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5708lb # [ 87.391646] niks3-server[979]: 2026/09/23 04:12:43 INFO Completed upload id=345709worker2 # [ 87.370210] nix-grpc-daemon[923]: time=2026-09-23T04:12:43.496Z level=INFO msg="Upload complete. (178ms)"5710lb # [ 87.395149] niks3-server[979]: 2026/09/23 04:12:43 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5711worker2 # [ 87.376242] nix-grpc-daemon[784]: ts=2026-09-23T04:12:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52284 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=17 outputs=15712client # copying 1 paths...5713worker2 # [ 87.385571] nix-grpc-daemon[784]: ts=2026-09-23T04:12:43Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:522845714client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5715worker2 # [ 87.393332] nix-daemon[870]: reaped child process 1201, status = succeeded5716worker2 # [ 87.404199] nix-grpc-daemon[784]: ts=2026-09-23T04:12:43Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:522845717worker2 # [ 87.406336] nix-grpc-daemon[784]: ts=2026-09-23T04:12:43Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:52284 duration_s=0 paths=1 nar_bytes_out=1205718client # [ 87.493284] nix-daemon[1114]: reaped child process 1243, status = succeeded5719client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&2, in 1.66 seconds)5720(finished: subtest: an input only one worker has still reaches the builder, in 6.71 seconds)5721subtest: upload whose reference the worker lacks is completed from the cache5722client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache5723client # this derivation will be built:5724client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5725client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5726client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache, in 0.25 seconds)5727client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache5728client # this derivation will be built:5729client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5730client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5731client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache, in 0.25 seconds)5732client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5733client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.10 seconds)5734worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5735worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.12 seconds)5736worker2 # [ 88.302505] systemd[1]: Stopping Nix worker-protocol over gRPC...5737worker2 # [ 88.430791] nix-grpc-daemon[784]: ts=2026-09-23T04:12:44Z level=info event=signal_exit5738lb # [ 88.464779] niks3-server[979]: 2026/09/23 04:12:44 INFO lead: released remote=192.168.1.4:417045739worker1 # [ 89.439649] nix-grpc-daemon[769]: ts=2026-09-23T04:12:45Z level=info event=scheduler_restarting addr=lb:500515740worker2 # [ 89.449501] nix-grpc-daemon[784]: ts=2026-09-23T04:12:45Z level=info event=scheduler_restarting5741worker2 # [ 89.453929] nix-grpc-daemon[784]: ts=2026-09-23T04:12:45Z level=info event=scheduler_restarting addr=lb:500515742worker2 # [ 89.457117] nix-grpc-daemon[784]: ts=2026-09-23T04:12:45Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:350325743worker2 # [ 89.460560] nix-grpc-daemon[784]: ts=2026-09-23T04:12:45Z level=info event=worker_gone addr=192.168.1.4:500515744worker2 # [ 89.463879] nix-grpc-daemon[784]: ts=2026-09-23T04:12:45Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:350325745worker2 # [ 89.467422] nix-grpc-daemon[784]: ts=2026-09-23T04:12:45Z level=info event=worker_gone addr=192.168.1.3:500515746worker2 # [ 89.489922] nix-daemon[870]: reaped child process 872, status = succeeded5747worker2 # [ 89.579152] nix-grpc-daemon[784]: E0000 00:00:1790136765.705107 830 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true5748worker2 # [ 89.584371] nix-grpc-daemon[784]: E0000 00:00:1790136765.705321 830 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5749worker1 # [ 89.639611] nix-grpc-daemon[769]: ts=2026-09-23T04:12:45Z level=info event=scheduler_connected addr=lb:500515750worker1 # [ 89.643554] nix-grpc-daemon[769]: ts=2026-09-23T04:12:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5751worker2 # [ 89.660443] nix-grpc-daemon[784]: ts=2026-09-23T04:12:45Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5752worker1 # [ 89.844698] nix-grpc-daemon[769]: ts=2026-09-23T04:12:45Z level=info event=scheduler_connected addr=lb:500515753worker1 # [ 89.848752] nix-grpc-daemon[769]: ts=2026-09-23T04:12:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5754worker1 # [ 90.049499] nix-grpc-daemon[769]: ts=2026-09-23T04:12:46Z level=info event=scheduler_connected addr=lb:500515755worker1 # [ 90.053345] nix-grpc-daemon[769]: ts=2026-09-23T04:12:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5756worker1 # [ 90.253992] nix-grpc-daemon[769]: ts=2026-09-23T04:12:46Z level=info event=scheduler_connected addr=lb:500515757worker1 # [ 90.258211] nix-grpc-daemon[769]: ts=2026-09-23T04:12:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5758lb # [ 90.459851] niks3-server[979]: 2026/09/23 04:12:46 INFO lead: acquired remote=192.168.1.3:430925759worker1 # [ 90.433740] nix-grpc-daemon[769]: ts=2026-09-23T04:12:46Z level=info event=scheduler_take_over5760worker1 # [ 90.459532] nix-grpc-daemon[769]: ts=2026-09-23T04:12:46Z level=info event=scheduler_connected addr=lb:500515761worker1 # [ 90.463713] nix-grpc-daemon[769]: ts=2026-09-23T04:12:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5762worker2 # [ 90.649084] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5763worker2 # [ 90.652534] systemd[1]: Stopped Nix worker-protocol over gRPC.5764worker2 # [ 90.656450] systemd[1]: nix-grpc-daemon.service: Consumed 2.047s CPU time over 1min 14.384s wall clock time, 64M memory peak, 93.9K incoming IP traffic, 246.5K outgoing IP traffic.5765worker2 # [ 90.667957] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5766worker2 # [ 90.672325] systemd[1]: Closed Nix worker-protocol over gRPC.5767worker1 # [ 90.664644] nix-grpc-daemon[769]: ts=2026-09-23T04:12:46Z level=info event=scheduler_connected addr=lb:500515768worker2 # [ 90.675720] systemd[1]: nix-grpc-daemon.socket: Consumed 284us CPU time over 1min 21.625s wall clock time, 256K memory peak, 110.2K incoming IP traffic, 93.1K outgoing IP traffic.5769worker1 # [ 90.669219] nix-grpc-daemon[769]: ts=2026-09-23T04:12:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5770lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5771lb: must succeed: curl -sf localhost:9901/clusters5772lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5773lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.05 seconds)5774client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25775worker1 # [ 90.867602] nix-grpc-daemon[769]: ts=2026-09-23T04:12:46Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:536025776client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5777worker1 # [ 90.874627] nix-grpc-daemon[769]: ts=2026-09-23T04:12:46Z level=info event=scheduler_connected addr=lb:500515778worker1 # [ 90.878397] nix-grpc-daemon[769]: ts=2026-09-23T04:12:46Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:536025779worker1 # [ 90.882761] nix-grpc-daemon[769]: ts=2026-09-23T04:12:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5780client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.17 seconds)5781worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5782worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds)5783worker2 # [ 90.964459] systemd[1]: Listening on Nix worker-protocol over gRPC.5784worker1 # [ 91.005492] systemd[1]: Stopping Nix worker-protocol over gRPC...5785worker1 # [ 91.040664] nix-grpc-daemon[769]: ts=2026-09-23T04:12:47Z level=info event=signal_exit5786worker1 # [ 91.074888] nix-grpc-daemon[769]: ts=2026-09-23T04:12:47Z level=info event=scheduler_connected addr=lb:500515787worker1 # [ 91.079252] nix-grpc-daemon[769]: ts=2026-09-23T04:12:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5788worker1 # [ 91.280224] nix-grpc-daemon[769]: ts=2026-09-23T04:12:47Z level=info event=scheduler_connected addr=lb:500515789worker1 # [ 91.284794] nix-grpc-daemon[769]: ts=2026-09-23T04:12:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5790lb # [ 91.467861] niks3-server[979]: 2026/09/23 04:12:47 INFO lead: released remote=192.168.1.3:430925791worker1 # [ 91.486505] nix-grpc-daemon[769]: ts=2026-09-23T04:12:47Z level=info event=scheduler_connected addr=lb:500515792worker1 # [ 91.490647] nix-grpc-daemon[769]: ts=2026-09-23T04:12:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5793worker2 # [ 91.645433] systemd[1]: Starting scheduler-token.service...5794worker1 # [ 91.688356] nix-grpc-daemon[769]: ts=2026-09-23T04:12:47Z level=info event=scheduler_connected addr=lb:500515795worker1 # [ 91.692812] nix-grpc-daemon[769]: ts=2026-09-23T04:12:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5796worker2 # [ 91.707109] systemd[1]: scheduler-token.service: Deactivated successfully.5797worker2 # [ 91.709358] systemd[1]: Finished scheduler-token.service.5798worker2 # [ 91.712538] systemd[1]: scheduler-token.service: Consumed 35ms CPU time over 61ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5799worker2 # [ 91.720528] systemd[1]: Starting Nix worker-protocol over gRPC...5800worker2 # [ 91.805424] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:47Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5801worker2 # [ 91.807594] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:47Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5802worker2 # [ 91.810964] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:47Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15803worker2 # [ 91.816057] nix-grpc-daemon[1246]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5804worker2 # [ 91.817413] nix-grpc-daemon[1246]: E0000 00:00:1790136767.941980 1246 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5805worker2 # [ 91.824068] nix-grpc-daemon[1246]: E0000 00:00:1790136767.947170 1246 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5806worker2 # [ 91.826264] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:47Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5807worker2 # [ 91.827834] systemd[1]: Started Nix worker-protocol over gRPC.5808worker2 # [ 91.834069] nix-grpc-daemon[1246]: E0000 00:00:1790136767.960036 1288 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5809worker2 # [ 91.838051] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:47Z level=info event=scheduler_take_over5810lb # [ 91.858298] niks3-server[979]: 2026/09/23 04:12:48 INFO lead: acquired remote=192.168.1.4:397405811worker2 # [ 91.849760] nix-grpc-daemon[1246]: E0000 00:00:1790136767.976045 1288 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5812worker2 # [ 91.854055] nix-grpc-daemon[1246]: E0000 00:00:1790136767.978378 1288 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5813worker2 # [ 91.863957] nix-grpc-daemon[1246]: E0000 00:00:1790136767.990246 1288 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5814worker2 # [ 91.873176] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:47Z level=info event=scheduler_connected addr=lb:500515815worker2 # [ 91.880061] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:48Z level=info event=oidc_jwks provider=mock keys=15816worker2 # [ 91.881732] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:48Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:336105817worker2 # [ 91.883375] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:48Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05818worker2 # [ 91.902853] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:48Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:336105819worker1 # [ 91.893685] nix-grpc-daemon[769]: ts=2026-09-23T04:12:47Z level=info event=scheduler_connected addr=lb:500515820worker2 # [ 91.904330] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:48Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05821worker1 # [ 92.436581] nix-grpc-daemon[769]: ts=2026-09-23T04:12:48Z level=info event=scheduler_restarting5822worker2 # [ 92.451883] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:48Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:336105823worker2 # [ 92.455519] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:48Z level=info event=worker_gone addr=192.168.1.3:500515824worker1 # [ 92.457298] nix-grpc-daemon[769]: ts=2026-09-23T04:12:48Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5825worker1 # [ 92.477220] nix-daemon[846]: reaped child process 848, status = succeeded5826worker1 # [ 92.591297] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5827worker1 # [ 92.596790] systemd[1]: Stopped Nix worker-protocol over gRPC.5828worker1 # [ 92.599914] systemd[1]: nix-grpc-daemon.service: Consumed 2.377s CPU time over 1min 16.733s wall clock time, 65.3M memory peak, 102.4K incoming IP traffic, 193.8K outgoing IP traffic.5829worker1 # [ 92.613382] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5830worker1 # [ 92.617342] systemd[1]: Closed Nix worker-protocol over gRPC.5831worker1 # [ 92.620274] systemd[1]: nix-grpc-daemon.socket: Consumed 289us CPU time over 1min 23.970s wall clock time, 256K memory peak, 95.2K incoming IP traffic, 84.7K outgoing IP traffic.5832lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5833lb: must succeed: curl -sf localhost:9901/clusters5834lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5835lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.05 seconds)5836client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25837worker2 # [ 92.839728] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:332565838worker2 # [ 92.853320] nix-daemon[870]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5839worker2 # [ 92.869857] nix-grpc-daemon[1246]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 87 ms (attempt 1/5)5840worker2 # [ 92.964963] nix-grpc-daemon[1246]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 156 ms (attempt 2/5)5841worker2 # [ 93.123659] nix-grpc-daemon[1246]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 234 ms (attempt 3/5)5842worker2 # [ 93.365568] nix-grpc-daemon[1246]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 183 ms (attempt 4/5)5843worker2 # [ 93.551276] nix-grpc-daemon[1246]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5844worker2 # [ 93.554396] nix-grpc-daemon[1246]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5845client # copying 2 paths...5846client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5847worker2 # [ 93.577756] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:332565848client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5849worker2 # [ 93.588589] nix-daemon[870]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5850worker2 # [ 93.612484] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:49Z level=info event=niks3_push_started pid=13075851worker2 # [ 93.722604] nix-grpc-daemon[1307]: time=2026-09-23T04:12:49.848Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""5852worker2 # [ 93.815148] nix-daemon[870]: accepted connection from pid 1320, user nix-grpc-daemon (trusted)5853worker2 # [ 93.829258] nix-daemon[870]: reaped child process 1323, status = succeeded5854lb # [ 93.858315] niks3-server[979]: 2026/09/23 04:12:50 INFO Received uploads request method=POST path=/api/pending_closures5855lb # [ 93.868514] niks3-server[979]: 2026/09/23 04:12:50 INFO Received uploads request method=POST path=/api/pending_closures5856worker2 # [ 93.856142] nix-grpc-daemon[1307]: time=2026-09-23T04:12:49.982Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5857worker2 # [ 93.857689] nix-grpc-daemon[1307]: time=2026-09-23T04:12:49.984Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5858worker2 # [ 93.859488] nix-grpc-daemon[1307]: time=2026-09-23T04:12:49.985Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5859lb # [ 93.927272] niks3-server[979]: 2026/09/23 04:12:50 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5860lb # [ 93.931637] niks3-server[979]: 2026/09/23 04:12:50 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5861lb # [ 93.934121] niks3-server[979]: 2026/09/23 04:12:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5862lb # [ 93.945144] niks3-server[979]: 2026/09/23 04:12:50 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5863lb # [ 93.947644] niks3-server[979]: 2026/09/23 04:12:50 INFO Signed narinfos id=35 count=15864lb # [ 93.949525] niks3-server[979]: 2026/09/23 04:12:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5865lb # [ 93.953042] niks3-server[979]: 2026/09/23 04:12:50 INFO Signed narinfos id=36 count=25866worker2 # [ 93.931531] nix-grpc-daemon[1307]: time=2026-09-23T04:12:50.057Z level=INFO msg="Uploading 3 narinfos"5867lb # [ 93.955496] niks3-server[979]: 2026/09/23 04:12:50 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5868lb # [ 93.971296] niks3-server[979]: 2026/09/23 04:12:50 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5869lb # [ 93.979533] niks3-server[979]: 2026/09/23 04:12:50 INFO Completed upload id=355870lb # [ 93.981204] niks3-server[979]: 2026/09/23 04:12:50 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5871lb # [ 93.983412] niks3-server[979]: 2026/09/23 04:12:50 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5872lb # [ 93.986447] niks3-server[979]: 2026/09/23 04:12:50 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5873lb # [ 93.989146] niks3-server[979]: 2026/09/23 04:12:50 INFO Completed upload id=365874worker2 # [ 93.968156] nix-grpc-daemon[1307]: time=2026-09-23T04:12:50.094Z level=INFO msg="Upload complete. (241ms)"5875lb # [ 93.991656] niks3-server[979]: 2026/09/23 04:12:50 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5876worker2 # [ 93.974719] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33256 duration_s=0 paths=2 nar_bytes_in=3125877worker2 # [ 93.989737] nix-daemon[870]: reaped child process 1305, status = succeeded5878client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.32 seconds)5879worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5880worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds)5881worker1 # [ 94.067885] systemd[1]: Listening on Nix worker-protocol over gRPC.5882worker1 # [ 94.075390] systemd[1]: Starting Nix worker-protocol over gRPC...5883worker1 # [ 94.162198] nix-grpc-daemon[1260]: ts=2026-09-23T04:12:50Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5884worker1 # [ 94.164158] nix-grpc-daemon[1260]: ts=2026-09-23T04:12:50Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5885worker1 # [ 94.167454] nix-grpc-daemon[1260]: ts=2026-09-23T04:12:50Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15886worker1 # [ 94.171822] nix-grpc-daemon[1260]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5887worker1 # [ 94.173310] nix-grpc-daemon[1260]: E0000 00:00:1790136770.257437 1260 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5888worker1 # [ 94.179688] nix-grpc-daemon[1260]: E0000 00:00:1790136770.263024 1260 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5889worker1 # [ 94.185188] nix-grpc-daemon[1260]: ts=2026-09-23T04:12:50Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5890worker1 # [ 94.187657] systemd[1]: Started Nix worker-protocol over gRPC.5891worker1 # [ 94.190148] nix-grpc-daemon[1260]: E0000 00:00:1790136770.274628 1302 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5892worker1 # [ 94.205043] nix-grpc-daemon[1260]: E0000 00:00:1790136770.289783 1302 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5893lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5894lb: must succeed: curl -sf localhost:9901/clusters5895worker1 # [ 94.208803] nix-grpc-daemon[1260]: E0000 00:00:1790136770.293384 1302 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5896worker1 # [ 94.222310] nix-grpc-daemon[1260]: E0000 00:00:1790136770.308049 1302 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5897worker1 # [ 94.235132] nix-grpc-daemon[1260]: ts=2026-09-23T04:12:50Z level=info event=scheduler_connected addr=lb:500515898worker2 # [ 94.244344] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:50Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:336105899worker2 # [ 94.249773] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:50Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05900lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5901lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.05 seconds)5902(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 6.74 seconds)5903subtest: a niks3 restart does not cost the leader its role5904lb: must succeed: curl -sf localhost:9901/clusters5905lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5906lb: must succeed: curl -sf localhost:9901/clusters5907lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5908lb # [ 94.350275] systemd[1]: Stopping niks3 server...5909lb # [ 94.351250] niks3-server[979]: 2026/09/23 04:12:50 INFO Shutdown signal received, draining in-flight requests timeout=10s5910lb # [ 94.352584] niks3-server[979]: 2026/09/23 04:12:50 INFO lead: released remote=192.168.1.4:397405911worker2 # [ 94.329335] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:50Z level=info event=scheduler_lock_lost status=2005912worker1 # [ 94.321244] nix-grpc-daemon[1260]: ts=2026-09-23T04:12:50Z level=info event=scheduler_lock_lost status=2005913lb # [ 94.368574] systemd[1]: niks3.service: Deactivated successfully.5914lb # [ 94.371191] systemd[1]: Stopped niks3 server.5915lb # [ 94.371911] systemd[1]: niks3.service: Consumed 744ms CPU time over 1min 14.145s wall clock time, 28.7M memory peak, 92.1K incoming IP traffic, 134.5K outgoing IP traffic.5916lb # [ 94.378920] systemd[1]: Starting niks3 server...5917lb # [ 94.430807] niks3-server[1119]: 2026/09/23 04:12:50 goose: no migrations to run. current version: 202609200000005918lb # [ 94.437143] niks3-server[1119]: 2026/09/23 04:12:50 OK 1_commit_pending_closure.sql (4.38ms)5919lb # [ 94.441193] niks3-server[1119]: 2026/09/23 04:12:50 OK 2_object_stats_trigger.sql (2.39ms)5920lb # [ 94.442270] niks3-server[1119]: 2026/09/23 04:12:50 goose: up to current file version: 25921lb # [ 94.443693] niks3-server[1119]: 2026/09/23 04:12:50 INFO Loaded signing key name=farm-test-1 path=/nix/store/hk0nhypl37jvkn3gr36v3y7qx593p39i-key5922lb # [ 94.454099] niks3-server[1119]: 2026/09/23 04:12:50 INFO Read proxy enabled — serving cache objects from S35923lb # [ 94.455330] niks3-server[1119]: 2026/09/23 04:12:50 INFO Using socket-activated listener address=0.0.0.0:57515924lb # [ 94.458383] niks3-server[1119]: 2026/09/23 04:12:50 INFO systemd watchdog enabled interval=15s5925lb # [ 94.459479] niks3-server[1119]: 2026/09/23 04:12:50 INFO Starting HTTP server address=0.0.0.0:57515926lb # [ 94.460911] systemd[1]: Started niks3 server.5927lb: waiting for unit niks3.service5928lb: (finished: waiting for unit niks3.service, in 0.02 seconds)5929lb: must succeed: curl -sf localhost:9901/clusters5930lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5931worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5932worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5933lb: must succeed: curl -sf localhost:9901/clusters5934lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5935lb: must succeed: curl -sf localhost:9901/clusters5936lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5937worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5938worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.05 seconds)5939client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&25940worker1 # [ 94.765179] nix-grpc-daemon[1260]: ts=2026-09-23T04:12:50Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:388085941worker1 # [ 94.776898] nix-daemon[846]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5942client # don't know how to build these paths:5943client # /nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5944worker1 # [ 94.791985] nix-grpc-daemon[1260]: ts=2026-09-23T04:12:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:388085945worker1 # [ 94.802398] nix-grpc-daemon[1260]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 83 ms (attempt 1/5)5946worker1 # [ 94.893592] nix-grpc-daemon[1260]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 173 ms (attempt 2/5)5947worker1 # [ 95.068886] nix-grpc-daemon[1260]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 309 ms (attempt 3/5)5948lb # [ 95.356752] niks3-server[1119]: 2026/09/23 04:12:51 INFO lead: acquired remote=192.168.1.4:469585949worker1 # [ 95.380760] nix-grpc-daemon[1260]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 771 ms (attempt 4/5)5950lb # [ 95.892821] envoy[914]: conn peer=[2001:db8:1::1]:36748 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3725 tx=2084 ms=13685951lb # [ 95.896678] envoy[914]: conn peer=[2001:db8:1::1]:36762 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=535952lb # [ 95.900229] envoy[914]: conn peer=[2001:db8:1::4]:58006 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=59575 tx=3200 ms=731945953lb # [ 95.903906] envoy[914]: conn peer=[2001:db8:1::1]:36774 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=165954lb # [ 95.907328] envoy[914]: conn peer=[2001:db8:1::3]:59310 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=19111 tx=3871 ms=763815955lb # [ 95.911233] envoy[914]: conn peer=[2001:db8:1::1]:34680 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1005 tx=305 ms=11495956worker1 # [ 96.153835] nix-grpc-daemon[1260]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5957worker1 # [ 96.157486] nix-grpc-daemon[1260]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5958worker2 # [ 96.205629] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:52Z level=debug event=want client=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv cached=05959worker2 # [ 96.209758] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:52Z level=debug event=assigned drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv worker=192.168.1.4:50051 clients=15960worker2 # [ 96.249139] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:332565961worker2 # [ 96.263989] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:332565962client # copying 1 paths to 'lb:50051'...5963client # copying path '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' to 'lb:50051'...5964worker2 # [ 96.277158] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:332565965worker2 # [ 96.288316] nix-daemon[870]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5966worker2 # [ 96.394643] nix-daemon[870]: accepted connection from pid 1342, user nix-grpc-daemon (trusted)5967worker2 # [ 96.409172] nix-daemon[870]: reaped child process 1345, status = succeeded5968lb # [ 96.438701] niks3-server[1119]: 2026/09/23 04:12:52 INFO Received uploads request method=POST path=/api/pending_closures5969worker2 # [ 96.430123] nix-grpc-daemon[1307]: time=2026-09-23T04:12:52.556Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5970worker2 # [ 96.432365] nix-grpc-daemon[1307]: time=2026-09-23T04:12:52.557Z level=INFO msg="Uploading 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv (536B)"5971lb # [ 96.483547] niks3-server[1119]: 2026/09/23 04:12:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5972lb # [ 96.492907] niks3-server[1119]: 2026/09/23 04:12:52 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.ls5973lb # [ 96.511553] niks3-server[1119]: 2026/09/23 04:12:52 INFO Signed narinfos id=37 count=15974worker2 # [ 96.492311] nix-grpc-daemon[1307]: time=2026-09-23T04:12:52.618Z level=INFO msg="Uploading 1 narinfos"5975lb # [ 96.518808] niks3-server[1119]: 2026/09/23 04:12:52 INFO Registered completed upload object_key=nar/0wjrljh500qj2dh74gryz4ik45bn1iv9w72490z0f2w0l2x4c87h.nar.zst5976lb # [ 96.522200] niks3-server[1119]: 2026/09/23 04:12:52 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5977lb # [ 96.528076] niks3-server[1119]: 2026/09/23 04:12:52 INFO Completed upload id=375978worker2 # [ 96.506837] nix-grpc-daemon[1307]: time=2026-09-23T04:12:52.633Z level=INFO msg="Upload complete. (206ms)"5979lb # [ 96.531757] niks3-server[1119]: 2026/09/23 04:12:52 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.narinfo5980worker2 # [ 96.511977] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33256 duration_s=0 paths=1 nar_bytes_in=5365981worker2 # [ 96.518208] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:332565982worker2 # [ 96.531146] nix-daemon[870]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5983worker2 # [ 96.538917] nix-daemon[870]: reaped child process 1340, status = succeeded5984worker2 # [ 96.546589] nix-daemon[870]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5985client # building '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' on lb:50051...5986client # farm-a> worker2: building 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5987client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5988client # farm-a> LOG-farm-a-after-niks3-restart5989worker2 # [ 97.964100] nix-daemon[870]: reaped child process 1349, status = succeeded5990worker2 # [ 98.042851] nix-daemon[870]: accepted connection from pid 1365, user nix-grpc-daemon (trusted)5991worker2 # [ 98.057339] nix-daemon[870]: reaped child process 1368, status = succeeded5992lb # [ 98.085911] niks3-server[1119]: 2026/09/23 04:12:54 INFO Received uploads request method=POST path=/api/pending_closures5993worker2 # [ 98.075536] nix-grpc-daemon[1307]: time=2026-09-23T04:12:54.201Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5994worker2 # [ 98.077097] nix-grpc-daemon[1307]: time=2026-09-23T04:12:54.203Z level=INFO msg="Uploading vllq1krpcvis69h349hy98vfz55ld1wv-farm-a (144B)"5995lb # [ 98.130703] niks3-server[1119]: 2026/09/23 04:12:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5996lb # [ 98.142731] niks3-server[1119]: 2026/09/23 04:12:54 INFO Signed narinfos id=38 count=15997worker2 # [ 98.123689] nix-grpc-daemon[1307]: time=2026-09-23T04:12:54.250Z level=INFO msg="Uploading 1 narinfos"5998lb # [ 98.145491] niks3-server[1119]: 2026/09/23 04:12:54 INFO Registered completed upload object_key=log/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5999lb # [ 98.155573] niks3-server[1119]: 2026/09/23 04:12:54 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst6000lb # [ 98.162631] niks3-server[1119]: 2026/09/23 04:12:54 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.ls6001lb # [ 98.169699] niks3-server[1119]: 2026/09/23 04:12:54 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6002lb # [ 98.174981] niks3-server[1119]: 2026/09/23 04:12:54 INFO Completed upload id=386003lb # [ 98.176379] niks3-server[1119]: 2026/09/23 04:12:54 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.narinfo6004worker2 # [ 98.153826] nix-grpc-daemon[1307]: time=2026-09-23T04:12:54.280Z level=INFO msg="Upload complete. (199ms)"6005worker2 # [ 98.161135] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33256 duration_s=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv assign_id=1 outputs=16006worker2 # [ 98.175185] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:54Z level=debug event=want client=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv cached=06007worker2 # [ 98.179230] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:54Z level=debug event=assigned drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv worker=192.168.1.4:50051 clients=16008worker2 # [ 98.189081] nix-daemon[870]: reaped child process 1347, status = succeeded6009worker2 # [ 98.218498] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:332566010worker2 # [ 98.232421] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:332566011worker2 # [ 98.246381] nix-grpc-daemon[1307]: time=2026-09-23T04:12:54.372Z level=INFO msg="All 1 paths already cached"6012client # copying 1 paths to 'lb:50051'...6013client # copying path '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' to 'lb:50051'...6014worker2 # [ 98.255166] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:332566015worker2 # [ 98.265154] nix-daemon[870]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)6016worker2 # [ 98.365087] nix-daemon[870]: accepted connection from pid 1374, user nix-grpc-daemon (trusted)6017worker2 # [ 98.379497] nix-daemon[870]: reaped child process 1377, status = succeeded6018lb # [ 98.407988] niks3-server[1119]: 2026/09/23 04:12:54 INFO Received uploads request method=POST path=/api/pending_closures6019worker2 # [ 98.396307] nix-grpc-daemon[1307]: time=2026-09-23T04:12:54.522Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6020worker2 # [ 98.397877] nix-grpc-daemon[1307]: time=2026-09-23T04:12:54.524Z level=INFO msg="Uploading 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv (656B)"6021lb # [ 98.439684] niks3-server[1119]: 2026/09/23 04:12:54 INFO Registered completed upload object_key=nar/0sjnwmbwn7i8g7g7czwn9r2pzaaa1j1yx3dfa4gw29ynhmpz9v3j.nar.zst6022lb # [ 98.449987] niks3-server[1119]: 2026/09/23 04:12:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6023lb # [ 98.455962] niks3-server[1119]: 2026/09/23 04:12:54 INFO Signed narinfos id=39 count=16024worker2 # [ 98.437712] nix-grpc-daemon[1307]: time=2026-09-23T04:12:54.564Z level=INFO msg="Uploading 1 narinfos"6025lb # [ 98.463155] niks3-server[1119]: 2026/09/23 04:12:54 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.ls6026lb # [ 98.477311] niks3-server[1119]: 2026/09/23 04:12:54 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6027lb # [ 98.483739] niks3-server[1119]: 2026/09/23 04:12:54 INFO Completed upload id=396028worker2 # [ 98.462467] nix-grpc-daemon[1307]: time=2026-09-23T04:12:54.588Z level=INFO msg="Upload complete. (186ms)"6029lb # [ 98.487521] niks3-server[1119]: 2026/09/23 04:12:54 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.narinfo6030worker2 # [ 98.466742] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33256 duration_s=0 paths=1 nar_bytes_in=6566031worker2 # [ 98.473210] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:332566032worker2 # [ 98.482541] nix-daemon[870]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)6033worker2 # [ 98.492156] nix-daemon[870]: reaped child process 1372, status = succeeded6034worker2 # [ 98.495523] nix-daemon[870]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)6035client # building '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' on lb:50051...6036client # farm-b> worker2: building 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv6037client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6038client # farm-b> LOG-farm-b-after-niks3-restart6039worker2 # [ 99.311173] nix-daemon[870]: reaped child process 1381, status = succeeded6040worker2 # [ 99.391621] nix-daemon[870]: accepted connection from pid 1397, user nix-grpc-daemon (trusted)6041worker2 # [ 99.404628] nix-daemon[870]: reaped child process 1400, status = succeeded6042lb # [ 99.433502] niks3-server[1119]: 2026/09/23 04:12:55 INFO Received uploads request method=POST path=/api/pending_closures6043worker2 # [ 99.420534] nix-grpc-daemon[1307]: time=2026-09-23T04:12:55.546Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6044worker2 # [ 99.422083] nix-grpc-daemon[1307]: time=2026-09-23T04:12:55.548Z level=INFO msg="Uploading vkqg7lcax8h1mxs0g750ficqiii1falw-farm-b (192B)"6045lb # [ 99.471751] niks3-server[1119]: 2026/09/23 04:12:55 INFO Registered completed upload object_key=nar/12zl6l8xhwx5hdivnvf1vw18yzvz3764mhdd9n03x81phzqblxj0.nar.zst6046lb # [ 99.483695] niks3-server[1119]: 2026/09/23 04:12:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6047lb # [ 99.492715] niks3-server[1119]: 2026/09/23 04:12:55 INFO Signed narinfos id=40 count=16048worker2 # [ 99.473989] nix-grpc-daemon[1307]: time=2026-09-23T04:12:55.600Z level=INFO msg="Uploading 1 narinfos"6049lb # [ 99.497291] niks3-server[1119]: 2026/09/23 04:12:55 INFO Registered completed upload object_key=log/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv6050lb # [ 99.502154] niks3-server[1119]: 2026/09/23 04:12:55 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.ls6051lb # [ 99.513648] niks3-server[1119]: 2026/09/23 04:12:55 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6052lb # [ 99.520478] niks3-server[1119]: 2026/09/23 04:12:55 INFO Completed upload id=406053worker2 # [ 99.499334] nix-grpc-daemon[1307]: time=2026-09-23T04:12:55.625Z level=INFO msg="Upload complete. (197ms)"6054lb # [ 99.524219] niks3-server[1119]: 2026/09/23 04:12:55 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.narinfo6055worker2 # [ 99.506795] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33256 duration_s=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv assign_id=2 outputs=16056worker2 # [ 99.518934] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:55Z level=debug event=want client=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv cached=06057worker2 # [ 99.523160] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:55Z level=debug event=assigned drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv worker=192.168.1.4:50051 clients=16058worker2 # [ 99.533451] nix-daemon[870]: reaped child process 1379, status = succeeded6059worker2 # [ 99.564114] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:332566060worker2 # [ 99.576639] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:332566061worker2 # [ 99.589936] nix-grpc-daemon[1307]: time=2026-09-23T04:12:55.716Z level=INFO msg="All 2 paths already cached"6062client # copying 1 paths to 'lb:50051'...6063client # copying path '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' to 'lb:50051'...6064worker2 # [ 99.599582] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:332566065worker2 # [ 99.609335] nix-daemon[870]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)6066worker2 # [ 99.710294] nix-daemon[870]: accepted connection from pid 1406, user nix-grpc-daemon (trusted)6067worker2 # [ 99.726220] nix-daemon[870]: reaped child process 1409, status = succeeded6068lb # [ 99.752513] niks3-server[1119]: 2026/09/23 04:12:55 INFO Received uploads request method=POST path=/api/pending_closures6069worker2 # [ 99.739035] nix-grpc-daemon[1307]: time=2026-09-23T04:12:55.865Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6070worker2 # [ 99.740570] nix-grpc-daemon[1307]: time=2026-09-23T04:12:55.866Z level=INFO msg="Uploading v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv (784B)"6071lb # [ 99.782771] niks3-server[1119]: 2026/09/23 04:12:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6072worker2 # [ 99.769279] nix-grpc-daemon[1307]: time=2026-09-23T04:12:55.895Z level=INFO msg="Uploading 1 narinfos"6073lb # [ 99.790379] niks3-server[1119]: 2026/09/23 04:12:55 INFO Signed narinfos id=41 count=16074lb # [ 99.798077] niks3-server[1119]: 2026/09/23 04:12:56 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.ls6075lb # [ 99.809878] niks3-server[1119]: 2026/09/23 04:12:56 INFO Registered completed upload object_key=nar/0ppnnsmr641nka691g6vrp1crps1bbn31zm7nc5za9pl2y2iwb0f.nar.zst6076lb # [ 99.825700] niks3-server[1119]: 2026/09/23 04:12:56 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6077lb # [ 99.829728] niks3-server[1119]: 2026/09/23 04:12:56 INFO Completed upload id=416078worker2 # [ 99.808461] nix-grpc-daemon[1307]: time=2026-09-23T04:12:55.934Z level=INFO msg="Upload complete. (187ms)"6079lb # [ 99.831626] niks3-server[1119]: 2026/09/23 04:12:56 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.narinfo6080worker2 # [ 99.813156] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33256 duration_s=0 paths=1 nar_bytes_in=7846081worker2 # [ 99.819755] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:332566082worker2 # [ 99.830131] nix-daemon[870]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)6083worker2 # [ 99.842938] nix-daemon[870]: reaped child process 1404, status = succeeded6084worker2 # [ 99.846354] nix-daemon[870]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)6085client # building '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' on lb:50051...6086client # farm-top> worker2: building v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv6087client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6088client # farm-top> LOG-farm-top-after-niks3-restart6089worker2 # [ 100.684238] nix-daemon[870]: reaped child process 1413, status = succeeded6090worker2 # [ 100.772955] nix-daemon[870]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6091worker2 # [ 100.786143] nix-daemon[870]: reaped child process 1432, status = succeeded6092lb # [ 100.815594] niks3-server[1119]: 2026/09/23 04:12:57 INFO Received uploads request method=POST path=/api/pending_closures6093worker2 # [ 100.802684] nix-grpc-daemon[1307]: time=2026-09-23T04:12:56.928Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6094worker2 # [ 100.804409] nix-grpc-daemon[1307]: time=2026-09-23T04:12:56.930Z level=INFO msg="Uploading lyi0sw9pvmhznkzgfl39callq9c0n87r-farm-top (248B)"6095lb # [ 100.856173] niks3-server[1119]: 2026/09/23 04:12:57 INFO Registered completed upload object_key=log/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv6096lb # [ 100.862816] niks3-server[1119]: 2026/09/23 04:12:57 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.ls6097lb # [ 100.870665] niks3-server[1119]: 2026/09/23 04:12:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6098lb # [ 100.875605] niks3-server[1119]: 2026/09/23 04:12:57 INFO Signed narinfos id=42 count=16099worker2 # [ 100.854652] nix-grpc-daemon[1307]: time=2026-09-23T04:12:56.980Z level=INFO msg="Uploading 1 narinfos"6100lb # [ 100.881532] niks3-server[1119]: 2026/09/23 04:12:57 INFO Registered completed upload object_key=nar/141220x0digv3f58fbh9c79avwg0fwgsk5jyac9ymkxmr8zs2prw.nar.zst6101lb # [ 100.891946] niks3-server[1119]: 2026/09/23 04:12:57 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6102lb # [ 100.894331] niks3-server[1119]: 2026/09/23 04:12:57 INFO Completed upload id=426103lb # [ 100.896974] niks3-server[1119]: 2026/09/23 04:12:57 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.narinfo6104worker2 # [ 100.874242] nix-grpc-daemon[1307]: time=2026-09-23T04:12:57.000Z level=INFO msg="Upload complete. (218ms)"6105worker2 # [ 100.882332] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33256 duration_s=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv assign_id=3 outputs=16106worker2 # [ 100.895195] nix-daemon[870]: reaped child process 1411, status = succeeded6107client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&2, in 6.29 seconds)6108client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath6109client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath, in 0.12 seconds)6110(finished: subtest: a niks3 restart does not cost the leader its role, in 6.77 seconds)6111subtest: scheduler down: the other node takes the lock and keeps it6112lb: must succeed: curl -sf localhost:9901/clusters6113lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6114lb: must succeed: curl -sf localhost:9901/clusters6115lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6116lb: must succeed: curl -sf localhost:9901/clusters6117lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6118lb: must succeed: curl -sf localhost:9901/clusters6119lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6120worker2 # [ 101.198069] systemd[1]: Stopping Nix worker-protocol over gRPC...6121worker2 # [ 101.827642] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:57Z level=info event=signal_exit6122lb # [ 102.362587] niks3-server[1119]: 2026/09/23 04:12:58 INFO lead: released remote=192.168.1.4:469586123worker1 # [ 103.337525] nix-grpc-daemon[1260]: ts=2026-09-23T04:12:59Z level=info event=scheduler_restarting addr=lb:500516124worker2 # [ 103.348548] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:59Z level=info event=scheduler_restarting6125worker2 # [ 103.353152] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:59Z level=info event=scheduler_restarting addr=lb:500516126worker2 # [ 103.357223] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:59Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:336106127worker2 # [ 103.360608] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:59Z level=info event=worker_gone addr=192.168.1.3:500516128worker2 # [ 103.363969] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:59Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:336106129worker2 # [ 103.367375] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:59Z level=info event=worker_gone addr=192.168.1.4:500516130worker2 # [ 103.389925] nix-daemon[870]: reaped child process 1299, status = succeeded6131worker1 # [ 103.537119] nix-grpc-daemon[1260]: ts=2026-09-23T04:12:59Z level=info event=scheduler_connected addr=lb:500516132worker1 # [ 103.541441] nix-grpc-daemon[1260]: ts=2026-09-23T04:12:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6133worker2 # [ 103.552502] nix-grpc-daemon[1246]: ts=2026-09-23T04:12:59Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6134worker1 # [ 103.742362] nix-grpc-daemon[1260]: ts=2026-09-23T04:12:59Z level=info event=scheduler_connected addr=lb:500516135worker1 # [ 103.746488] nix-grpc-daemon[1260]: ts=2026-09-23T04:12:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6136worker2 # [ 103.871424] nix-grpc-daemon[1246]: E0000 00:00:1790136779.997596 1292 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true6137worker2 # [ 103.873841] nix-grpc-daemon[1246]: E0000 00:00:1790136779.999966 1292 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6138worker1 # [ 103.947422] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:00Z level=info event=scheduler_connected addr=lb:500516139worker1 # [ 103.951564] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6140worker1 # [ 104.152435] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:00Z level=info event=scheduler_connected addr=lb:500516141worker1 # [ 104.156520] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6142worker1 # [ 104.358742] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:00Z level=info event=scheduler_connected addr=lb:500516143worker1 # [ 104.361941] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6144worker2 # [ 104.566926] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6145worker1 # [ 104.559835] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:00Z level=info event=scheduler_connected addr=lb:500516146worker2 # [ 104.570360] systemd[1]: Stopped Nix worker-protocol over gRPC.6147worker1 # [ 104.563838] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6148worker2 # [ 104.574538] systemd[1]: nix-grpc-daemon.service: Consumed 920ms CPU time over 12.845s wall clock time, 37.3M memory peak, 45.5K incoming IP traffic, 79.3K outgoing IP traffic.6149worker2 # [ 104.585806] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.6150worker2 # [ 104.590297] systemd[1]: Closed Nix worker-protocol over gRPC.6151worker2 # [ 104.594553] systemd[1]: nix-grpc-daemon.socket: Consumed 538us CPU time over 13.630s wall clock time, 256K memory peak, 36.4K incoming IP traffic, 26.7K outgoing IP traffic.6152worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over6153worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.05 seconds)6154worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6155worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)6156worker1 # [ 104.764808] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:00Z level=info event=scheduler_connected addr=lb:500516157worker1 # [ 104.768764] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6158worker1 # [ 104.969679] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:01Z level=info event=scheduler_connected addr=lb:500516159worker1 # [ 104.973793] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6160worker1 # [ 105.174514] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:01Z level=info event=scheduler_connected addr=lb:500516161worker1 # [ 105.178596] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6162worker1 # [ 105.333807] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:01Z level=info event=scheduler_take_over6163lb # [ 105.363246] niks3-server[1119]: 2026/09/23 04:13:01 INFO lead: acquired remote=192.168.1.3:518326164worker1 # [ 105.380330] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:01Z level=info event=scheduler_connected addr=lb:500516165worker1 # [ 105.384125] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6166worker1 # [ 105.585135] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:01Z level=info event=scheduler_connected addr=lb:500516167worker1 # [ 105.589153] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6168worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6169worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6170worker1 # [ 105.790080] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:01Z level=info event=scheduler_connected addr=lb:500516171worker1 # [ 105.794540] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6172lb # [ 105.893379] envoy[914]: conn peer=[2001:db8:1::1]:34696 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9679 tx=4408 ms=61226173lb # [ 105.897140] envoy[914]: conn peer=[2001:db8:1::4]:49534 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=1772 tx=442 ms=116786174worker1 # [ 105.995521] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:02Z level=info event=scheduler_connected addr=lb:500516175worker1 # [ 105.999970] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6176worker1 # [ 106.200828] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:02Z level=info event=scheduler_connected addr=lb:500516177worker1 # [ 106.205261] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6178worker1 # [ 106.407159] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:02Z level=info event=scheduler_connected addr=lb:500516179worker1 # [ 106.414774] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:02Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:542326180worker1 # [ 106.419568] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:02Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06181worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6182worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6183client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&26184worker1 # [ 106.924368] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:388086185client # don't know how to build these paths:6186client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6187worker1 # [ 106.933613] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:388086188worker1 # [ 106.956247] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:03Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=06189worker1 # [ 106.961578] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:03Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=16190worker1 # [ 107.000702] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:388086191worker1 # [ 107.014368] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:388086192client # copying 1 paths to 'lb:50051'...6193client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...6194worker1 # [ 107.028250] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:388086195worker1 # [ 107.037924] nix-daemon[846]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)6196worker1 # [ 107.062870] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:03Z level=info event=niks3_push_started pid=13406197worker1 # [ 107.168690] nix-grpc-daemon[1340]: time=2026-09-23T04:13:03.254Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6198worker1 # [ 107.256509] nix-daemon[846]: accepted connection from pid 1353, user nix-grpc-daemon (trusted)6199worker1 # [ 107.268498] nix-daemon[846]: reaped child process 1356, status = succeeded6200lb # [ 107.305873] niks3-server[1119]: 2026/09/23 04:13:03 INFO Received uploads request method=POST path=/api/pending_closures6201worker1 # [ 107.285429] nix-grpc-daemon[1340]: time=2026-09-23T04:13:03.371Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6202worker1 # [ 107.286895] nix-grpc-daemon[1340]: time=2026-09-23T04:13:03.372Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"6203lb # [ 107.342310] niks3-server[1119]: 2026/09/23 04:13:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6204lb # [ 107.347745] niks3-server[1119]: 2026/09/23 04:13:03 INFO Signed narinfos id=43 count=16205worker1 # [ 107.321233] nix-grpc-daemon[1340]: time=2026-09-23T04:13:03.407Z level=INFO msg="Uploading 1 narinfos"6206lb # [ 107.357863] niks3-server[1119]: 2026/09/23 04:13:03 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst6207lb # [ 107.364519] niks3-server[1119]: 2026/09/23 04:13:03 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls6208lb # [ 107.378188] niks3-server[1119]: 2026/09/23 04:13:03 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6209lb # [ 107.383831] niks3-server[1119]: 2026/09/23 04:13:03 INFO Completed upload id=436210worker1 # [ 107.354629] nix-grpc-daemon[1340]: time=2026-09-23T04:13:03.440Z level=INFO msg="Upload complete. (182ms)"6211worker1 # [ 107.357480] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38808 duration_s=0 paths=1 nar_bytes_in=5206212lb # [ 107.388862] niks3-server[1119]: 2026/09/23 04:13:03 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo6213worker1 # [ 107.364378] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:388086214worker1 # [ 107.367887] nix-daemon[846]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)6215worker1 # [ 107.370060] nix-daemon[846]: reaped child process 1338, status = succeeded6216worker1 # [ 107.375111] nix-daemon[846]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)6217client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...6218client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv6219client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6220client # farm-a> LOG-farm-a-on-standby6221worker1 # [ 108.815283] nix-daemon[846]: reaped child process 1360, status = succeeded6222worker1 # [ 108.905426] nix-daemon[846]: accepted connection from pid 1376, user nix-grpc-daemon (trusted)6223worker1 # [ 108.916466] nix-daemon[846]: reaped child process 1379, status = succeeded6224lb # [ 108.953942] niks3-server[1119]: 2026/09/23 04:13:05 INFO Received uploads request method=POST path=/api/pending_closures6225worker1 # [ 108.933586] nix-grpc-daemon[1340]: time=2026-09-23T04:13:05.019Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6226worker1 # [ 108.936063] nix-grpc-daemon[1340]: time=2026-09-23T04:13:05.020Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"6227lb # [ 108.996296] niks3-server[1119]: 2026/09/23 04:13:05 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv6228lb # [ 109.004223] niks3-server[1119]: 2026/09/23 04:13:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6229lb # [ 109.012079] niks3-server[1119]: 2026/09/23 04:13:05 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst6230lb # [ 109.025164] niks3-server[1119]: 2026/09/23 04:13:05 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls6231lb # [ 109.031109] niks3-server[1119]: 2026/09/23 04:13:05 INFO Signed narinfos id=44 count=16232worker1 # [ 109.002419] nix-grpc-daemon[1340]: time=2026-09-23T04:13:05.087Z level=INFO msg="Uploading 1 narinfos"6233lb # [ 109.038688] niks3-server[1119]: 2026/09/23 04:13:05 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6234lb # [ 109.045417] niks3-server[1119]: 2026/09/23 04:13:05 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo6235lb # [ 109.049068] niks3-server[1119]: 2026/09/23 04:13:05 INFO Completed upload id=446236worker1 # [ 109.019946] nix-grpc-daemon[1340]: time=2026-09-23T04:13:05.105Z level=INFO msg="Upload complete. (234ms)"6237worker1 # [ 109.029115] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38808 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=16238worker1 # [ 109.043809] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:05Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=06239worker1 # [ 109.049260] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:05Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=16240worker1 # [ 109.060807] nix-daemon[846]: reaped child process 1358, status = succeeded6241worker1 # [ 109.086630] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:388086242worker1 # [ 109.100749] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:388086243worker1 # [ 109.115422] nix-grpc-daemon[1340]: time=2026-09-23T04:13:05.201Z level=INFO msg="All 1 paths already cached"6244client # copying 1 paths to 'lb:50051'...6245client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...6246worker1 # [ 109.125065] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:388086247worker1 # [ 109.135369] nix-daemon[846]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)6248worker1 # [ 109.231080] nix-daemon[846]: accepted connection from pid 1385, user nix-grpc-daemon (trusted)6249worker1 # [ 109.243845] nix-daemon[846]: reaped child process 1388, status = succeeded6250lb # [ 109.281370] niks3-server[1119]: 2026/09/23 04:13:05 INFO Received uploads request method=POST path=/api/pending_closures6251worker1 # [ 109.262041] nix-grpc-daemon[1340]: time=2026-09-23T04:13:05.347Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6252worker1 # [ 109.264087] nix-grpc-daemon[1340]: time=2026-09-23T04:13:05.349Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"6253lb # [ 109.313693] niks3-server[1119]: 2026/09/23 04:13:05 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls6254lb # [ 109.321692] niks3-server[1119]: 2026/09/23 04:13:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6255worker1 # [ 109.297715] nix-grpc-daemon[1340]: time=2026-09-23T04:13:05.383Z level=INFO msg="Uploading 1 narinfos"6256lb # [ 109.326997] niks3-server[1119]: 2026/09/23 04:13:05 INFO Signed narinfos id=45 count=16257lb # [ 109.333134] niks3-server[1119]: 2026/09/23 04:13:05 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst6258lb # [ 109.347327] niks3-server[1119]: 2026/09/23 04:13:05 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6259lb # [ 109.354622] niks3-server[1119]: 2026/09/23 04:13:05 INFO Completed upload id=456260worker1 # [ 109.325392] nix-grpc-daemon[1340]: time=2026-09-23T04:13:05.411Z level=INFO msg="Upload complete. (177ms)"6261lb # [ 109.358167] niks3-server[1119]: 2026/09/23 04:13:05 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo6262worker1 # [ 109.329874] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38808 duration_s=0 paths=1 nar_bytes_in=6406263worker1 # [ 109.336345] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:388086264worker1 # [ 109.347179] nix-daemon[846]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)6265worker1 # [ 109.358132] nix-daemon[846]: reaped child process 1383, status = succeeded6266worker1 # [ 109.361057] nix-daemon[846]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)6267client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...6268client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6269client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6270client # farm-b> LOG-farm-b-on-standby6271worker1 # [ 110.176568] nix-daemon[846]: reaped child process 1392, status = succeeded6272worker1 # [ 110.250173] nix-daemon[846]: accepted connection from pid 1408, user nix-grpc-daemon (trusted)6273worker1 # [ 110.262911] nix-daemon[846]: reaped child process 1411, status = succeeded6274lb # [ 110.300310] niks3-server[1119]: 2026/09/23 04:13:06 INFO Received uploads request method=POST path=/api/pending_closures6275worker1 # [ 110.282248] nix-grpc-daemon[1340]: time=2026-09-23T04:13:06.367Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6276worker1 # [ 110.283790] nix-grpc-daemon[1340]: time=2026-09-23T04:13:06.369Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"6277lb # [ 110.344151] niks3-server[1119]: 2026/09/23 04:13:06 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls6278lb # [ 110.350181] niks3-server[1119]: 2026/09/23 04:13:06 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6279lb # [ 110.355837] niks3-server[1119]: 2026/09/23 04:13:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6280lb # [ 110.365658] niks3-server[1119]: 2026/09/23 04:13:06 INFO Signed narinfos id=46 count=16281worker1 # [ 110.336882] nix-grpc-daemon[1340]: time=2026-09-23T04:13:06.422Z level=INFO msg="Uploading 1 narinfos"6282lb # [ 110.371149] niks3-server[1119]: 2026/09/23 04:13:06 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst6283lb # [ 110.383385] niks3-server[1119]: 2026/09/23 04:13:06 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6284lb # [ 110.387491] niks3-server[1119]: 2026/09/23 04:13:06 INFO Completed upload id=466285lb # [ 110.389690] niks3-server[1119]: 2026/09/23 04:13:06 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo6286worker1 # [ 110.358340] nix-grpc-daemon[1340]: time=2026-09-23T04:13:06.444Z level=INFO msg="Upload complete. (191ms)"6287worker1 # [ 110.368891] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38808 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=16288worker1 # [ 110.378223] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:06Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=06289worker1 # [ 110.382252] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:06Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=16290worker1 # [ 110.391575] nix-daemon[846]: reaped child process 1390, status = succeeded6291worker1 # [ 110.422412] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:388086292worker1 # [ 110.434880] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:388086293worker1 # [ 110.448374] nix-grpc-daemon[1340]: time=2026-09-23T04:13:06.534Z level=INFO msg="All 2 paths already cached"6294client # copying 1 paths to 'lb:50051'...6295client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...6296worker1 # [ 110.457689] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:388086297worker1 # [ 110.465753] nix-daemon[846]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)6298worker1 # [ 110.561187] nix-daemon[846]: accepted connection from pid 1417, user nix-grpc-daemon (trusted)6299worker1 # [ 110.573564] nix-daemon[846]: reaped child process 1420, status = succeeded6300lb # [ 110.610702] niks3-server[1119]: 2026/09/23 04:13:06 INFO Received uploads request method=POST path=/api/pending_closures6301worker1 # [ 110.590272] nix-grpc-daemon[1340]: time=2026-09-23T04:13:06.676Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6302worker1 # [ 110.591733] nix-grpc-daemon[1340]: time=2026-09-23T04:13:06.677Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"6303lb # [ 110.643514] niks3-server[1119]: 2026/09/23 04:13:06 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst6304lb # [ 110.656355] niks3-server[1119]: 2026/09/23 04:13:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6305worker1 # [ 110.631360] nix-grpc-daemon[1340]: time=2026-09-23T04:13:06.717Z level=INFO msg="Uploading 1 narinfos"6306lb # [ 110.660966] niks3-server[1119]: 2026/09/23 04:13:06 INFO Signed narinfos id=47 count=16307lb # [ 110.670074] niks3-server[1119]: 2026/09/23 04:13:06 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls6308lb # [ 110.684842] niks3-server[1119]: 2026/09/23 04:13:06 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6309lb # [ 110.689255] niks3-server[1119]: 2026/09/23 04:13:06 INFO Completed upload id=476310worker1 # [ 110.660291] nix-grpc-daemon[1340]: time=2026-09-23T04:13:06.745Z level=INFO msg="Upload complete. (181ms)"6311lb # [ 110.692684] niks3-server[1119]: 2026/09/23 04:13:06 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo6312worker1 # [ 110.664407] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38808 duration_s=0 paths=1 nar_bytes_in=7686313worker1 # [ 110.669295] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:388086314worker1 # [ 110.678493] nix-daemon[846]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)6315worker1 # [ 110.687890] nix-daemon[846]: reaped child process 1415, status = succeeded6316worker1 # [ 110.692689] nix-daemon[846]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)6317client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...6318client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6319client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6320worker1 # [ 111.306213] nix-daemon[846]: reaped child process 1424, status = succeeded6321client # farm-top> LOG-farm-top-on-standby6322worker1 # [ 111.379791] nix-daemon[846]: accepted connection from pid 1440, user nix-grpc-daemon (trusted)6323worker1 # [ 111.393184] nix-daemon[846]: reaped child process 1443, status = succeeded6324lb # [ 111.429437] niks3-server[1119]: 2026/09/23 04:13:07 INFO Received uploads request method=POST path=/api/pending_closures6325worker1 # [ 111.410542] nix-grpc-daemon[1340]: time=2026-09-23T04:13:07.496Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6326worker1 # [ 111.412096] nix-grpc-daemon[1340]: time=2026-09-23T04:13:07.497Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"6327lb # [ 111.470994] niks3-server[1119]: 2026/09/23 04:13:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6328worker1 # [ 111.448900] nix-grpc-daemon[1340]: time=2026-09-23T04:13:07.534Z level=INFO msg="Uploading 1 narinfos"6329lb # [ 111.478378] niks3-server[1119]: 2026/09/23 04:13:07 INFO Signed narinfos id=48 count=16330lb # [ 111.484730] niks3-server[1119]: 2026/09/23 04:13:07 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls6331lb # [ 111.490692] niks3-server[1119]: 2026/09/23 04:13:07 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst6332lb # [ 111.494677] niks3-server[1119]: 2026/09/23 04:13:07 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6333lb # [ 111.501070] niks3-server[1119]: 2026/09/23 04:13:07 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6334lb # [ 111.505507] niks3-server[1119]: 2026/09/23 04:13:07 INFO Completed upload id=486335worker1 # [ 111.476322] nix-grpc-daemon[1340]: time=2026-09-23T04:13:07.562Z level=INFO msg="Upload complete. (177ms)"6336lb # [ 111.507049] niks3-server[1119]: 2026/09/23 04:13:07 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo6337worker1 # [ 111.482503] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38808 duration_s=0 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=16338worker1 # [ 111.494531] nix-daemon[846]: reaped child process 1422, status = succeeded6339client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&2, in 4.75 seconds)6340client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath6341client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath, in 0.12 seconds)6342worker2 # [ 111.696588] systemd[1]: Listening on Nix worker-protocol over gRPC.6343worker2 # [ 111.702227] systemd[1]: Starting scheduler-token.service...6344worker2 # [ 111.744819] systemd[1]: scheduler-token.service: Deactivated successfully.6345worker2 # [ 111.746559] systemd[1]: Finished scheduler-token.service.6346worker2 # [ 111.749256] systemd[1]: scheduler-token.service: Consumed 21ms CPU time over 44ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6347worker2 # [ 111.752421] systemd[1]: Starting Nix worker-protocol over gRPC...6348worker2 # [ 111.833873] nix-grpc-daemon[1453]: ts=2026-09-23T04:13:07Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6349worker2 # [ 111.838397] nix-grpc-daemon[1453]: ts=2026-09-23T04:13:07Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6350worker2 # [ 111.843197] nix-grpc-daemon[1453]: ts=2026-09-23T04:13:07Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16351worker2 # [ 111.849127] nix-grpc-daemon[1453]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6352worker2 # [ 111.850537] nix-grpc-daemon[1453]: E0000 00:00:1790136787.967871 1453 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6353worker2 # [ 111.852790] nix-grpc-daemon[1453]: E0000 00:00:1790136787.969248 1453 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6354worker2 # [ 111.857753] nix-grpc-daemon[1453]: ts=2026-09-23T04:13:07Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6355worker2 # [ 111.864121] systemd[1]: Started Nix worker-protocol over gRPC.6356worker2 # [ 111.868097] nix-grpc-daemon[1453]: E0000 00:00:1790136787.986316 1495 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6357worker2 # [ 111.874779] nix-grpc-daemon[1453]: E0000 00:00:1790136788.001018 1495 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6358worker2 # [ 111.879154] nix-grpc-daemon[1453]: E0000 00:00:1790136788.003562 1495 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6359lb: must succeed: curl -sf localhost:9901/clusters6360worker2 # [ 111.889569] nix-grpc-daemon[1453]: E0000 00:00:1790136788.015860 1495 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6361worker2 # [ 111.902237] nix-grpc-daemon[1453]: ts=2026-09-23T04:13:08Z level=info event=scheduler_connected addr=lb:500516362worker1 # [ 111.906612] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:07Z level=info event=oidc_jwks provider=mock keys=16363worker1 # [ 111.909799] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:07Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:542326364worker1 # [ 111.914235] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:07Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06365lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6366worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6367worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6368lb: must succeed: curl -sf localhost:9901/clusters6369lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6370lb: must succeed: curl -sf localhost:9901/clusters6371lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6372worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6373worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6374(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 11.04 seconds)6375subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise6376client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs6377client # [ 112.095895] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs.6378client # Running as unit: rs.service; invocation ID: 9ef0dbed6c6047b492fb689f27d9f7b86379client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs, in 0.07 seconds)6380worker1 # [ 112.253248] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:08Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:388086381client # [ 112.243837] nix[1366]: don't know how to build these paths:6382client # [ 112.244932] nix[1366]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6383worker1 # [ 112.266884] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:08Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=06384worker1 # [ 112.272416] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:08Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=16385worker1 # [ 112.311130] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:388086386worker1 # [ 112.324248] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:388086387client # [ 112.318206] nix[1366]: copying 1 paths to 'lb:50051'...6388worker1 # [ 112.335993] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:388086389client # [ 112.322078] nix[1366]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...6390worker1 # [ 112.347560] nix-daemon[846]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)6391worker1 # [ 112.444920] nix-daemon[846]: accepted connection from pid 1456, user nix-grpc-daemon (trusted)6392worker1 # [ 112.459303] nix-daemon[846]: reaped child process 1459, status = succeeded6393lb # [ 112.493681] niks3-server[1119]: 2026/09/23 04:13:08 INFO Received uploads request method=POST path=/api/pending_closures6394worker1 # [ 112.472456] nix-grpc-daemon[1340]: time=2026-09-23T04:13:08.558Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6395worker1 # [ 112.474433] nix-grpc-daemon[1340]: time=2026-09-23T04:13:08.559Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"6396lb # [ 112.520737] niks3-server[1119]: 2026/09/23 04:13:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6397worker1 # [ 112.500828] nix-grpc-daemon[1340]: time=2026-09-23T04:13:08.586Z level=INFO msg="Uploading 1 narinfos"6398lb # [ 112.529951] niks3-server[1119]: 2026/09/23 04:13:08 INFO Signed narinfos id=49 count=16399lb # [ 112.538116] niks3-server[1119]: 2026/09/23 04:13:08 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls6400lb # [ 112.544245] niks3-server[1119]: 2026/09/23 04:13:08 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst6401lb # [ 112.555742] niks3-server[1119]: 2026/09/23 04:13:08 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6402worker1 # [ 112.533088] nix-grpc-daemon[1340]: time=2026-09-23T04:13:08.618Z level=INFO msg="Upload complete. (173ms)"6403lb # [ 112.563151] niks3-server[1119]: 2026/09/23 04:13:08 INFO Completed upload id=496404lb # [ 112.566830] niks3-server[1119]: 2026/09/23 04:13:08 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo6405worker1 # [ 112.536503] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38808 duration_s=0 paths=1 nar_bytes_in=4166406worker1 # [ 112.543092] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:388086407worker1 # [ 112.551505] nix-daemon[846]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)6408worker1 # [ 112.560124] nix-daemon[846]: reaped child process 1454, status = succeeded6409worker1 # [ 112.563191] nix-daemon[846]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)6410client # [ 112.546346] nix[1366]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...6411lb: must succeed: curl -sf localhost:9901/clusters6412lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6413lb: must succeed: curl -sf localhost:9901/clusters6414lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6415lb: must succeed: curl -sf localhost:9901/clusters6416lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6417lb: must succeed: curl -sf localhost:9901/clusters6418lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6419worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true6420worker2 # [ 114.789646] systemd-journald[284]: Received client request to rotate journal, rotating.6421worker2 # [ 114.809836] systemd-journald[284]: Vacuuming done, freed 0B of archived journals from /var/log/journal/e44a80e7de7f43a39e37b0ceb0968e04.6422worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.06 seconds)6423worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service6424worker1 # [ 114.707458] systemd[1]: Stopping Nix worker-protocol over gRPC...6425worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.03 seconds)6426client: must succeed: sleep 5.06427worker1 # [ 115.187210] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:11Z level=info event=draining6428lb # [ 115.894118] envoy[914]: conn peer=[2001:db8:1::1]:58406 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9560 tx=4370 ms=45726429client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6430worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='6431worker1 # [ 119.999892] nix-daemon[846]: reaped child process 1463, status = succeeded6432worker1 # [ 120.073430] nix-daemon[846]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6433worker1 # [ 120.085493] nix-daemon[846]: reaped child process 1507, status = succeeded6434lb # [ 120.121794] niks3-server[1119]: 2026/09/23 04:13:16 INFO Received uploads request method=POST path=/api/pending_closures6435worker1 # [ 120.100973] nix-grpc-daemon[1340]: time=2026-09-23T04:13:16.186Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6436worker1 # [ 120.102515] nix-grpc-daemon[1340]: time=2026-09-23T04:13:16.188Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"6437lb # [ 120.164557] niks3-server[1119]: 2026/09/23 04:13:16 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst6438lb # [ 120.172921] niks3-server[1119]: 2026/09/23 04:13:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6439lb # [ 120.179429] niks3-server[1119]: 2026/09/23 04:13:16 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls6440lb # [ 120.185504] niks3-server[1119]: 2026/09/23 04:13:16 INFO Signed narinfos id=50 count=16441worker1 # [ 120.156490] nix-grpc-daemon[1340]: time=2026-09-23T04:13:16.241Z level=INFO msg="Uploading 1 narinfos"6442lb # [ 120.190833] niks3-server[1119]: 2026/09/23 04:13:16 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6443lb # [ 120.200180] niks3-server[1119]: 2026/09/23 04:13:16 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6444lb # [ 120.204547] niks3-server[1119]: 2026/09/23 04:13:16 INFO Completed upload id=506445lb # [ 120.206847] niks3-server[1119]: 2026/09/23 04:13:16 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo6446worker1 # [ 120.176580] nix-grpc-daemon[1340]: time=2026-09-23T04:13:16.262Z level=INFO msg="Upload complete. (200ms)"6447worker1 # [ 120.183494] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38808 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=16448worker1 # [ 120.192321] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:16Z level=info event=signal_exit6449worker1 # [ 120.200891] nix-daemon[846]: reaped child process 1461, status = succeeded6450client # [ 120.204759] systemd[1]: rs.service: Deactivated successfully.6451client # [ 120.209142] systemd[1]: rs.service: Consumed 131ms CPU time over 8.112s wall clock time, 41.4M memory peak, 7.1K incoming IP traffic, 8.3K outgoing IP traffic.6452lb # [ 120.364235] niks3-server[1119]: 2026/09/23 04:13:16 INFO lead: released remote=192.168.1.3:518326453worker2 # [ 121.345730] nix-grpc-daemon[1453]: ts=2026-09-23T04:13:17Z level=info event=scheduler_restarting addr=lb:500516454worker1 # [ 121.340447] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:17Z level=info event=scheduler_restarting6455worker1 # [ 121.344647] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:17Z level=info event=scheduler_restarting addr=lb:500516456worker1 # [ 121.347911] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:17Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:542326457worker1 # [ 121.351441] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:17Z level=info event=worker_gone addr=192.168.1.4:500516458worker1 # [ 121.355206] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:17Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:542326459worker1 # [ 121.358982] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:17Z level=info event=worker_gone addr=192.168.1.3:500516460worker1 # [ 121.378214] nix-daemon[846]: reaped child process 1313, status = succeeded6461worker2 # [ 121.546324] nix-grpc-daemon[1453]: ts=2026-09-23T04:13:17Z level=info event=scheduler_connected addr=lb:500516462worker2 # [ 121.550912] nix-grpc-daemon[1453]: ts=2026-09-23T04:13:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6463worker1 # [ 121.550918] nix-grpc-daemon[1260]: ts=2026-09-23T04:13:17Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6464worker2 # [ 121.752495] nix-grpc-daemon[1453]: ts=2026-09-23T04:13:17Z level=info event=scheduler_connected addr=lb:500516465worker2 # [ 121.756605] nix-grpc-daemon[1453]: ts=2026-09-23T04:13:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6466lb # [ 121.881717] niks3-server[1119]: 2026/09/23 04:13:18 INFO lead: acquired remote=192.168.1.4:591626467worker2 # [ 121.863494] nix-grpc-daemon[1453]: ts=2026-09-23T04:13:17Z level=info event=scheduler_take_over6468worker2 # [ 121.957963] nix-grpc-daemon[1453]: ts=2026-09-23T04:13:18Z level=info event=scheduler_connected addr=lb:500516469worker2 # [ 121.962094] nix-grpc-daemon[1453]: ts=2026-09-23T04:13:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6470worker1 # [ 121.995156] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6471worker1 # [ 121.997935] systemd[1]: Stopped Nix worker-protocol over gRPC.6472worker1 # [ 122.001382] systemd[1]: nix-grpc-daemon.service: Consumed 1.042s CPU time over 27.919s wall clock time, 37.9M memory peak, 49.1K incoming IP traffic, 106.1K outgoing IP traffic.6473worker1 # [ 122.016162] systemd[1]: Starting Nix worker-protocol over gRPC...6474worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.11 seconds)6475client: waiting for success: ! systemctl is-active rs6476client: (finished: waiting for success: ! systemctl is-active rs, in 0.04 seconds)6477client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }6478worker1 # [ 122.103976] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:18Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6479client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.03 seconds)6480client: must fail: journalctl -u rs | grep -q 'reconnecting'6481worker1 # [ 122.108273] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:18Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6482worker1 # [ 122.111430] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:18Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16483worker1 # [ 122.116772] nix-grpc-daemon[1514]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6484worker1 # [ 122.120123] nix-grpc-daemon[1514]: E0000 00:00:1790136798.199808 1514 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6485client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.02 seconds)6486lb: must succeed: curl -sf localhost:9901/clusters6487worker1 # [ 122.123842] nix-grpc-daemon[1514]: E0000 00:00:1790136798.201383 1514 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6488worker1 # [ 122.126608] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:18Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6489worker1 # [ 122.129140] systemd[1]: Started Nix worker-protocol over gRPC.6490worker1 # [ 122.131157] nix-grpc-daemon[1514]: E0000 00:00:1790136798.216239 1555 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6491worker1 # [ 122.142724] nix-grpc-daemon[1514]: E0000 00:00:1790136798.228455 1555 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6492worker1 # [ 122.148054] nix-grpc-daemon[1514]: E0000 00:00:1790136798.231133 1555 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6493worker2 # [ 122.163367] nix-grpc-daemon[1453]: ts=2026-09-23T04:13:18Z level=info event=scheduler_connected addr=lb:500516494worker1 # [ 122.157577] nix-grpc-daemon[1514]: E0000 00:00:1790136798.243369 1555 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6495worker2 # [ 122.167230] nix-grpc-daemon[1453]: ts=2026-09-23T04:13:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6496worker1 # [ 122.169104] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:18Z level=info event=scheduler_connected addr=lb:500516497worker1 # [ 122.170827] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6498lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6499worker2 # [ 122.370357] nix-grpc-daemon[1453]: ts=2026-09-23T04:13:18Z level=info event=scheduler_connected addr=lb:500516500worker2 # [ 122.373561] nix-grpc-daemon[1453]: ts=2026-09-23T04:13:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6501worker1 # [ 122.371804] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:18Z level=info event=scheduler_connected addr=lb:500516502worker1 # [ 122.376371] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6503worker2 # [ 122.571926] nix-grpc-daemon[1453]: ts=2026-09-23T04:13:18Z level=info event=scheduler_connected addr=lb:500516504worker2 # [ 122.576206] nix-grpc-daemon[1453]: ts=2026-09-23T04:13:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6505worker1 # [ 122.577165] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:18Z level=info event=scheduler_connected addr=lb:500516506worker1 # [ 122.581736] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6507worker2 # [ 122.777890] nix-grpc-daemon[1453]: ts=2026-09-23T04:13:18Z level=info event=scheduler_connected addr=lb:500516508worker1 # [ 122.782775] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:18Z level=info event=scheduler_connected addr=lb:500516509worker2 # [ 122.792721] nix-grpc-daemon[1453]: ts=2026-09-23T04:13:18Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:376366510worker2 # [ 122.796670] nix-grpc-daemon[1453]: ts=2026-09-23T04:13:18Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06511worker2 # [ 122.805136] nix-grpc-daemon[1453]: ts=2026-09-23T04:13:18Z level=info event=oidc_jwks provider=mock keys=16512worker2 # [ 122.808920] nix-grpc-daemon[1453]: ts=2026-09-23T04:13:18Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:376366513worker2 # [ 122.812857] nix-grpc-daemon[1453]: ts=2026-09-23T04:13:18Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06514lb: must succeed: curl -sf localhost:9901/clusters6515lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6516worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6517worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6518(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 11.20 seconds)6519subtest: scheduler restart mid-build: build finishes, no double build6520client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid6521client # [ 123.296739] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid.6522client # Running as unit: mid.service; invocation ID: 53f316ae1c7249938e3e60a1285e70f46523client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid, in 0.06 seconds)6524worker1 # [ 123.462789] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:414626525worker1 # [ 123.475107] nix-daemon[846]: accepted connection from pid 1514, user nix-grpc-daemon (trusted)6526client # [ 123.472241] nix[1402]: don't know how to build these paths:6527client # [ 123.473225] nix[1402]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6528worker2 # [ 123.506639] nix-grpc-daemon[1453]: ts=2026-09-23T04:13:19Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06529worker2 # [ 123.510858] nix-grpc-daemon[1453]: ts=2026-09-23T04:13:19Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=16530worker1 # [ 123.542811] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414626531worker1 # [ 123.554984] nix-grpc-daemon[1514]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 61 ms (attempt 1/5)6532worker1 # [ 123.624494] nix-grpc-daemon[1514]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 19 ms (attempt 2/5)6533worker1 # [ 123.645183] nix-grpc-daemon[1514]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 249 ms (attempt 3/5)6534worker1 # [ 123.896352] nix-grpc-daemon[1514]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 668 ms (attempt 4/5)6535worker1 # [ 124.567606] nix-grpc-daemon[1514]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6536worker1 # [ 124.571849] nix-grpc-daemon[1514]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6537worker1 # [ 124.589693] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:414626538client # [ 124.584947] nix[1402]: copying 1 paths to 'lb:50051'...6539client # [ 124.588714] nix[1402]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...6540worker1 # [ 124.605589] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:414626541worker1 # [ 124.617937] nix-daemon[846]: accepted connection from pid 1514, user nix-grpc-daemon (trusted)6542worker1 # [ 124.640115] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:20Z level=info event=niks3_push_started pid=15836543worker1 # [ 124.739570] nix-grpc-daemon[1583]: time=2026-09-23T04:13:20.825Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6544worker1 # [ 124.824747] nix-daemon[846]: accepted connection from pid 1596, user nix-grpc-daemon (trusted)6545worker1 # [ 124.836761] nix-daemon[846]: reaped child process 1599, status = succeeded6546lb # [ 124.875796] niks3-server[1119]: 2026/09/23 04:13:21 INFO Received uploads request method=POST path=/api/pending_closures6547worker1 # [ 124.854547] nix-grpc-daemon[1583]: time=2026-09-23T04:13:20.940Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6548worker1 # [ 124.856346] nix-grpc-daemon[1583]: time=2026-09-23T04:13:20.942Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"6549lb # [ 124.912511] niks3-server[1119]: 2026/09/23 04:13:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6550lb # [ 124.922536] niks3-server[1119]: 2026/09/23 04:13:21 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls6551lb # [ 124.933262] niks3-server[1119]: 2026/09/23 04:13:21 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst6552worker1 # [ 124.915258] nix-grpc-daemon[1583]: time=2026-09-23T04:13:21.001Z level=INFO msg="Uploading 1 narinfos"6553lb # [ 124.944541] niks3-server[1119]: 2026/09/23 04:13:21 INFO Signed narinfos id=51 count=16554lb # [ 124.956373] niks3-server[1119]: 2026/09/23 04:13:21 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6555lb # [ 124.961290] niks3-server[1119]: 2026/09/23 04:13:21 INFO Completed upload id=516556worker1 # [ 124.931805] nix-grpc-daemon[1583]: time=2026-09-23T04:13:21.017Z level=INFO msg="Upload complete. (188ms)"6557lb # [ 124.966247] niks3-server[1119]: 2026/09/23 04:13:21 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo6558worker1 # [ 124.936198] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41462 duration_s=0 paths=1 nar_bytes_in=4246559worker1 # [ 124.942431] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414626560worker1 # [ 124.955245] nix-daemon[846]: accepted connection from pid 1514, user nix-grpc-daemon (trusted)6561worker1 # [ 124.958457] nix-daemon[846]: reaped child process 1581, status = succeeded6562worker1 # [ 124.962578] nix-daemon[846]: accepted connection from pid 1514, user nix-grpc-daemon (trusted)6563client # [ 124.946149] nix[1402]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6564lb # [ 125.894549] envoy[914]: conn peer=[2001:db8:1::1]:58414 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2956 tx=1498 ms=79456565lb # [ 125.898322] envoy[914]: conn peer=[2001:db8:1::3]:60776 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=4416 tx=1112 ms=273136566lb: must succeed: curl -sf localhost:9901/clusters6567lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6568lb: must succeed: curl -sf localhost:9901/clusters6569lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6570worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6571worker2 # [ 126.922602] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1453 (nix-grpc-daemon) on client request.6572lb # [ 126.960832] niks3-server[1119]: 2026/09/23 04:13:23 INFO lead: released remote=192.168.1.4:591626573worker2 # [ 126.940459] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6574worker1 # [ 126.936073] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:23Z level=info event=scheduler_disconnected addr=lb:50051 error="Stream removed (RST_STREAM (Received RST_STREAM with error code 2))"6575client # [ 126.920776] nix[1402]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6576worker2 # [ 126.957364] systemd[1]: Starting scheduler-token.service...6577worker2 # [ 126.963962] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6578worker2 # [ 126.965564] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6579worker2 # [ 126.966979] systemd[1]: nix-grpc-daemon.service: Consumed 236ms CPU time over 15.211s wall clock time, 13.5M memory peak, 11.1K incoming IP traffic, 22.2K outgoing IP traffic.6580worker2 # [ 127.001648] systemd[1]: scheduler-token.service: Deactivated successfully.6581worker2 # [ 127.003341] systemd[1]: Finished scheduler-token.service.6582worker2 # [ 127.004938] systemd[1]: scheduler-token.service: Consumed 17ms CPU time over 45ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6583worker2 # [ 127.007563] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6584worker2 # [ 127.011167] systemd[1]: Starting scheduler-token.service...6585worker2 # [ 127.045251] systemd[1]: scheduler-token.service: Deactivated successfully.6586worker2 # [ 127.046902] systemd[1]: Finished scheduler-token.service.6587worker2 # [ 127.048275] systemd[1]: scheduler-token.service: Consumed 16ms CPU time over 35ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6588worker2 # [ 127.053112] systemd[1]: Starting Nix worker-protocol over gRPC...6589worker2 # [ 127.134413] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:23Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6590lb # [ 127.156139] niks3-server[1119]: 2026/09/23 04:13:23 INFO lead: acquired remote=192.168.1.3:491286591worker2 # [ 127.136614] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:23Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6592worker1 # [ 127.130396] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:23Z level=info event=scheduler_take_over6593worker2 # [ 127.140282] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:23Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16594worker1 # [ 127.136219] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:23Z level=info event=scheduler_connected addr=lb:500516595worker2 # [ 127.145057] nix-grpc-daemon[1595]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6596worker2 # [ 127.146945] nix-grpc-daemon[1595]: E0000 00:00:1790136803.270811 1595 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6597worker2 # [ 127.152646] nix-grpc-daemon[1595]: E0000 00:00:1790136803.276400 1595 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6598worker2 # [ 127.157801] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:23Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6599worker2 # [ 127.161380] systemd[1]: Started Nix worker-protocol over gRPC.6600worker2 # [ 127.164697] nix-grpc-daemon[1595]: E0000 00:00:1790136803.290081 1638 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6601worker1 # [ 127.162233] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:23Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6602worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.29 seconds)6603client: must succeed: sleep 5.06604worker2 # [ 127.180068] nix-grpc-daemon[1595]: E0000 00:00:1790136803.305312 1638 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6605worker2 # [ 127.183192] nix-grpc-daemon[1595]: E0000 00:00:1790136803.305557 1638 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6606worker2 # [ 127.193333] nix-grpc-daemon[1595]: E0000 00:00:1790136803.319680 1638 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6607worker2 # [ 127.202680] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:23Z level=info event=scheduler_connected addr=lb:500516608worker2 # [ 127.205176] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:23Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6609worker1 # [ 127.364453] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:23Z level=info event=scheduler_connected addr=lb:500516610worker1 # [ 127.371547] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:23Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:414706611worker1 # [ 127.373284] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:23Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=16612worker2 # [ 127.406964] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:23Z level=info event=scheduler_connected addr=lb:500516613worker2 # [ 127.412539] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:23Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6614worker2 # [ 127.616141] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:23Z level=info event=scheduler_connected addr=lb:500516615worker2 # [ 127.624524] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:23Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6616worker2 # [ 127.826218] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:23Z level=info event=scheduler_connected addr=lb:500516617worker1 # [ 127.823905] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:23Z level=info event=oidc_jwks provider=mock keys=16618worker1 # [ 127.827147] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:23Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:414706619worker1 # [ 127.828728] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:23Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06620client: (finished: must succeed: sleep 5.0, in 5.04 seconds)6621client: waiting for success: ! systemctl is-active mid6622worker1 # [ 132.388643] nix-daemon[846]: reaped child process 1603, status = succeeded6623worker1 # [ 132.463769] nix-daemon[846]: accepted connection from pid 1640, user nix-grpc-daemon (trusted)6624worker1 # [ 132.476055] nix-daemon[846]: reaped child process 1643, status = succeeded6625lb # [ 132.513843] niks3-server[1119]: 2026/09/23 04:13:28 INFO Received uploads request method=POST path=/api/pending_closures6626worker1 # [ 132.493826] nix-grpc-daemon[1583]: time=2026-09-23T04:13:28.579Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6627worker1 # [ 132.496066] nix-grpc-daemon[1583]: time=2026-09-23T04:13:28.581Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6628lb # [ 132.558861] niks3-server[1119]: 2026/09/23 04:13:28 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6629lb # [ 132.562100] niks3-server[1119]: 2026/09/23 04:13:28 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6630lb # [ 132.571883] niks3-server[1119]: 2026/09/23 04:13:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6631lb # [ 132.576691] niks3-server[1119]: 2026/09/23 04:13:28 INFO Signed narinfos id=52 count=16632worker1 # [ 132.549834] nix-grpc-daemon[1583]: time=2026-09-23T04:13:28.635Z level=INFO msg="Uploading 1 narinfos"6633lb # [ 132.580803] niks3-server[1119]: 2026/09/23 04:13:28 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6634lb # [ 132.586918] niks3-server[1119]: 2026/09/23 04:13:28 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6635lb # [ 132.591190] niks3-server[1119]: 2026/09/23 04:13:28 INFO Completed upload id=526636lb # [ 132.592969] niks3-server[1119]: 2026/09/23 04:13:28 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo6637worker1 # [ 132.563314] nix-grpc-daemon[1583]: time=2026-09-23T04:13:28.649Z level=INFO msg="Upload complete. (182ms)"6638worker1 # [ 132.569634] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41462 duration_s=7 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16639worker1 # [ 132.584765] nix-daemon[846]: reaped child process 1601, status = succeeded6640client # [ 132.592860] systemd[1]: mid.service: Deactivated successfully.6641client # [ 132.596696] systemd[1]: mid.service: Consumed 144ms CPU time over 9.299s wall clock time, 41.4M memory peak, 7.8K incoming IP traffic, 8.8K outgoing IP traffic.6642client: (finished: waiting for success: ! systemctl is-active mid, in 1.09 seconds)6643client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6644client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.03 seconds)6645(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.24 seconds)6646subtest: interrupting the client stops the build on the worker6647client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr6648client # [ 133.507106] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr.6649client # Running as unit: intr.service; invocation ID: ed07ea6dd2c04015a97601539883d9166650client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr, in 0.03 seconds)6651client # [ 133.638290] nix[1438]: don't know how to build these paths:6652client # [ 133.639216] nix[1438]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6653worker1 # [ 133.654180] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:29Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:414626654worker1 # [ 133.665207] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:29Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06655worker1 # [ 133.669567] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:29Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=16656worker2 # [ 133.726883] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437846657worker2 # [ 133.738342] nix-daemon[870]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6658worker2 # [ 133.755469] nix-grpc-daemon[1595]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 65 ms (attempt 1/5)6659worker2 # [ 133.828433] nix-grpc-daemon[1595]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 55 ms (attempt 2/5)6660worker2 # [ 133.891163] nix-grpc-daemon[1595]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 42 ms (attempt 3/5)6661worker2 # [ 133.934972] nix-grpc-daemon[1595]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 616 ms (attempt 4/5)6662worker2 # [ 134.553735] nix-grpc-daemon[1595]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6663worker2 # [ 134.557542] nix-grpc-daemon[1595]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6664worker2 # [ 134.575532] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:437846665client # [ 134.561191] nix[1438]: copying 1 paths to 'lb:50051'...6666client # [ 134.565175] nix[1438]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6667worker2 # [ 134.589802] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:437846668worker2 # [ 134.600169] nix-daemon[870]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6669worker2 # [ 134.622185] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:30Z level=info event=niks3_push_started pid=16686670worker2 # [ 134.741202] nix-grpc-daemon[1668]: time=2026-09-23T04:13:30.867Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""6671worker2 # [ 134.837981] nix-daemon[870]: accepted connection from pid 1684, user nix-grpc-daemon (trusted)6672worker2 # [ 134.852169] nix-daemon[870]: reaped child process 1687, status = succeeded6673lb # [ 134.879595] niks3-server[1119]: 2026/09/23 04:13:31 INFO Received uploads request method=POST path=/api/pending_closures6674worker2 # [ 134.866443] nix-grpc-daemon[1668]: time=2026-09-23T04:13:30.992Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6675worker2 # [ 134.867996] nix-grpc-daemon[1668]: time=2026-09-23T04:13:30.994Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6676lb # [ 134.920386] niks3-server[1119]: 2026/09/23 04:13:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6677lb # [ 134.927834] niks3-server[1119]: 2026/09/23 04:13:31 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6678worker2 # [ 134.911342] nix-grpc-daemon[1668]: time=2026-09-23T04:13:31.037Z level=INFO msg="Uploading 1 narinfos"6679lb # [ 134.932843] niks3-server[1119]: 2026/09/23 04:13:31 INFO Signed narinfos id=53 count=16680lb # [ 134.950615] niks3-server[1119]: 2026/09/23 04:13:31 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6681lb # [ 134.956070] niks3-server[1119]: 2026/09/23 04:13:31 INFO Completed upload id=536682lb # [ 134.958349] niks3-server[1119]: 2026/09/23 04:13:31 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6683worker2 # [ 134.935985] nix-grpc-daemon[1668]: time=2026-09-23T04:13:31.062Z level=INFO msg="Upload complete. (191ms)"6684lb # [ 134.962477] niks3-server[1119]: 2026/09/23 04:13:31 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6685worker2 # [ 134.940716] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43784 duration_s=0 paths=1 nar_bytes_in=4246686worker2 # [ 134.947289] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437846687worker2 # [ 134.957579] nix-daemon[870]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6688worker2 # [ 134.960977] nix-daemon[870]: reaped child process 1666, status = succeeded6689worker2 # [ 134.967125] nix-daemon[870]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6690client # [ 134.942593] nix[1438]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6691lb # [ 135.895126] envoy[914]: conn peer=[2001:db8:1::4]:43446 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=6970 tx=540 ms=150496692lb # [ 135.898992] envoy[914]: conn peer=[2001:db8:1::1]:41634 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3074 tx=1607 ms=91316693client: must succeed: systemctl kill -s INT intr6694client # [ 136.964597] systemd[1]: intr.service: Sent signal SIGINT to main process 1438 (nix) on client request.6695worker2 # [ 136.996482] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:33Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6696client # [ 136.971668] nix[1438]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6697client # [ 136.978333] nix[1438]: error: interrupted by the user6698client: (finished: must succeed: systemctl kill -s INT intr, in 0.05 seconds)6699client # [ 137.009458] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6700client # [ 137.012232] systemd[1]: intr.service: Failed with result 'exit-code'.6701client # [ 137.015963] systemd[1]: intr.service: Consumed 130ms CPU time over 3.507s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic.6702worker2 # [ 137.185495] nix-daemon[870]: reaped child process 1689, status = succeeded6703worker2 # [ 137.186971] nix-daemon[1691]: unexpected Nix daemon error: error: interrupted by the user6704worker2 # [ 137.191613] nix-daemon[870]: reaped child process 1691, status = failed with exit code 16705(finished: subtest: interrupting the client stops the build on the worker, in 4.68 seconds)6706subtest: a deploy mid-build drains: the build finishes, the new generation takes over6707client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw6708client # [ 138.212573] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw.6709client # Running as unit: sw.service; invocation ID: 2d0dc8b4197a48aa8e65a4b074e54f666710client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw, in 0.07 seconds)6711worker2 # [ 138.382526] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:34Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:437846712client # [ 138.364563] nix[1459]: don't know how to build these paths:6713client # [ 138.365528] nix[1459]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6714worker1 # [ 138.390891] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:34Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06715worker1 # [ 138.396968] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:34Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=16716worker1 # [ 138.439565] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414626717worker1 # [ 138.453461] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:414626718client # [ 138.447189] nix[1459]: copying 1 paths to 'lb:50051'...6719client # [ 138.450107] nix[1459]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6720worker1 # [ 138.467750] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:414626721worker1 # [ 138.476846] nix-daemon[846]: accepted connection from pid 1514, user nix-grpc-daemon (trusted)6722worker1 # [ 138.580641] nix-daemon[846]: accepted connection from pid 1679, user nix-grpc-daemon (trusted)6723worker1 # [ 138.593424] nix-daemon[846]: reaped child process 1682, status = succeeded6724lb # [ 138.631711] niks3-server[1119]: 2026/09/23 04:13:34 INFO Received uploads request method=POST path=/api/pending_closures6725worker1 # [ 138.615682] nix-grpc-daemon[1583]: time=2026-09-23T04:13:34.700Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6726worker1 # [ 138.617290] nix-grpc-daemon[1583]: time=2026-09-23T04:13:34.700Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6727lb # [ 138.667356] niks3-server[1119]: 2026/09/23 04:13:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6728lb # [ 138.672348] niks3-server[1119]: 2026/09/23 04:13:34 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6729lb # [ 138.676939] niks3-server[1119]: 2026/09/23 04:13:34 INFO Signed narinfos id=54 count=16730worker1 # [ 138.651478] nix-grpc-daemon[1583]: time=2026-09-23T04:13:34.736Z level=INFO msg="Uploading 1 narinfos"6731lb # [ 138.688760] niks3-server[1119]: 2026/09/23 04:13:34 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6732lb # [ 138.698186] niks3-server[1119]: 2026/09/23 04:13:34 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6733lb # [ 138.704679] niks3-server[1119]: 2026/09/23 04:13:34 INFO Completed upload id=546734worker1 # [ 138.675189] nix-grpc-daemon[1583]: time=2026-09-23T04:13:34.760Z level=INFO msg="Upload complete. (184ms)"6735lb # [ 138.706351] niks3-server[1119]: 2026/09/23 04:13:34 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6736worker1 # [ 138.679265] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41462 duration_s=0 paths=1 nar_bytes_in=4166737worker1 # [ 138.684191] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414626738worker1 # [ 138.693653] nix-daemon[846]: accepted connection from pid 1514, user nix-grpc-daemon (trusted)6739worker1 # [ 138.701616] nix-daemon[846]: reaped child process 1677, status = succeeded6740worker1 # [ 138.706511] nix-daemon[846]: accepted connection from pid 1514, user nix-grpc-daemon (trusted)6741client # [ 138.689932] nix[1459]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6742worker1: must succeed: systemctl reload nix-grpc-daemon.service6743worker1 # [ 140.653156] systemd[1]: Reloading Nix worker-protocol over gRPC...6744worker1 # [ 140.678501] systemd[1]: Reloaded Nix worker-protocol over gRPC.6745worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.06 seconds)6746worker1: must succeed: pgrep -f 'read -t [9]0 x'6747worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.03 seconds)6748client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&26749worker1 # [ 140.870370] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:36Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:414626750client # don't know how to build these paths:6751client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6752worker2 # [ 140.885933] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:437846753worker1 # [ 140.900599] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:36Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06754worker1 # [ 140.902557] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:36Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=16755worker2 # [ 140.958144] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437846756worker2 # [ 140.970664] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:437846757client # copying 1 paths to 'lb:50051'...6758worker2 # [ 140.983513] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:437846759client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6760worker2 # [ 141.005214] nix-daemon[870]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6761worker2 # [ 141.110722] nix-daemon[870]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6762worker2 # [ 141.124168] nix-daemon[870]: reaped child process 1745, status = succeeded6763worker1 # [ 141.125350] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:37Z level=info event=draining6764lb # [ 141.152804] niks3-server[1119]: 2026/09/23 04:13:37 INFO Received uploads request method=POST path=/api/pending_closures6765worker2 # [ 141.139791] nix-grpc-daemon[1668]: time=2026-09-23T04:13:37.265Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6766worker2 # [ 141.142069] nix-grpc-daemon[1668]: time=2026-09-23T04:13:37.267Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6767lb # [ 141.180723] niks3-server[1119]: 2026/09/23 04:13:37 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6768lb # [ 141.193135] niks3-server[1119]: 2026/09/23 04:13:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6769worker2 # [ 141.176295] nix-grpc-daemon[1668]: time=2026-09-23T04:13:37.302Z level=INFO msg="Uploading 1 narinfos"6770lb # [ 141.197504] niks3-server[1119]: 2026/09/23 04:13:37 INFO Signed narinfos id=55 count=16771lb # [ 141.204866] niks3-server[1119]: 2026/09/23 04:13:37 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6772lb # [ 141.218105] niks3-server[1119]: 2026/09/23 04:13:37 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6773lb # [ 141.224704] niks3-server[1119]: 2026/09/23 04:13:37 INFO Completed upload id=556774worker2 # [ 141.203551] nix-grpc-daemon[1668]: time=2026-09-23T04:13:37.329Z level=INFO msg="Upload complete. (186ms)"6775lb # [ 141.226546] niks3-server[1119]: 2026/09/23 04:13:37 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6776worker2 # [ 141.207821] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43784 duration_s=0 paths=1 nar_bytes_in=5286777worker2 # [ 141.213741] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437846778worker2 # [ 141.223474] nix-daemon[870]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6779worker2 # [ 141.232254] nix-daemon[870]: reaped child process 1740, status = succeeded6780worker2 # [ 141.237172] nix-daemon[870]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6781client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6782client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6783client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6784client # farm-a> LOG-farm-a-during-drain6785worker2 # [ 142.454588] nix-daemon[870]: reaped child process 1749, status = succeeded6786worker2 # [ 142.531252] nix-daemon[870]: accepted connection from pid 1765, user nix-grpc-daemon (trusted)6787worker2 # [ 142.543782] nix-daemon[870]: reaped child process 1768, status = succeeded6788lb # [ 142.573549] niks3-server[1119]: 2026/09/23 04:13:38 INFO Received uploads request method=POST path=/api/pending_closures6789worker2 # [ 142.560457] nix-grpc-daemon[1668]: time=2026-09-23T04:13:38.686Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6790worker2 # [ 142.561878] nix-grpc-daemon[1668]: time=2026-09-23T04:13:38.688Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6791worker2 # [ 142.592216] nix-grpc-daemon[1668]: time=2026-09-23T04:13:38.718Z level=INFO msg="Uploading 1 narinfos"6792lb # [ 142.614431] niks3-server[1119]: 2026/09/23 04:13:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6793lb # [ 142.618125] niks3-server[1119]: 2026/09/23 04:13:38 INFO Signed narinfos id=56 count=16794lb # [ 142.630110] niks3-server[1119]: 2026/09/23 04:13:38 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6795lb # [ 142.638286] niks3-server[1119]: 2026/09/23 04:13:38 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6796lb # [ 142.642884] niks3-server[1119]: 2026/09/23 04:13:38 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6797lb # [ 142.650989] niks3-server[1119]: 2026/09/23 04:13:38 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6798lb # [ 142.657785] niks3-server[1119]: 2026/09/23 04:13:38 INFO Completed upload id=566799lb # [ 142.659104] niks3-server[1119]: 2026/09/23 04:13:38 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6800worker2 # [ 142.636634] nix-grpc-daemon[1668]: time=2026-09-23T04:13:38.762Z level=INFO msg="Upload complete. (188ms)"6801worker2 # [ 142.641523] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43784 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=3 outputs=16802worker1 # [ 142.638767] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:38Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06803worker1 # [ 142.640640] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:38Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=16804worker2 # [ 142.654743] nix-daemon[870]: reaped child process 1747, status = succeeded6805worker2 # [ 142.690354] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437846806worker2 # [ 142.705865] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:437846807worker2 # [ 142.718443] nix-grpc-daemon[1668]: time=2026-09-23T04:13:38.844Z level=INFO msg="All 1 paths already cached"6808client # copying 1 paths to 'lb:50051'...6809worker2 # [ 142.727314] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:437846810client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6811worker2 # [ 142.752178] nix-daemon[870]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6812worker2 # [ 142.854327] nix-daemon[870]: accepted connection from pid 1772, user nix-grpc-daemon (trusted)6813worker2 # [ 142.866422] nix-daemon[870]: reaped child process 1775, status = succeeded6814lb # [ 142.897723] niks3-server[1119]: 2026/09/23 04:13:39 INFO Received uploads request method=POST path=/api/pending_closures6815worker2 # [ 142.884461] nix-grpc-daemon[1668]: time=2026-09-23T04:13:39.010Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6816worker2 # [ 142.885920] nix-grpc-daemon[1668]: time=2026-09-23T04:13:39.012Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6817lb # [ 142.929928] niks3-server[1119]: 2026/09/23 04:13:39 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6818lb # [ 142.937445] niks3-server[1119]: 2026/09/23 04:13:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6819lb # [ 142.942111] niks3-server[1119]: 2026/09/23 04:13:39 INFO Signed narinfos id=57 count=16820worker2 # [ 142.923861] nix-grpc-daemon[1668]: time=2026-09-23T04:13:39.050Z level=INFO msg="Uploading 1 narinfos"6821lb # [ 142.951126] niks3-server[1119]: 2026/09/23 04:13:39 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6822lb # [ 142.963927] niks3-server[1119]: 2026/09/23 04:13:39 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6823lb # [ 142.970535] niks3-server[1119]: 2026/09/23 04:13:39 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6824lb # [ 142.974732] niks3-server[1119]: 2026/09/23 04:13:39 INFO Completed upload id=576825worker2 # [ 142.953385] nix-grpc-daemon[1668]: time=2026-09-23T04:13:39.079Z level=INFO msg="Upload complete. (188ms)"6826worker2 # [ 142.958145] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43784 duration_s=0 paths=1 nar_bytes_in=6406827worker2 # [ 142.963818] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437846828worker2 # [ 142.976107] nix-daemon[870]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6829worker2 # [ 142.983384] nix-daemon[870]: reaped child process 1770, status = succeeded6830worker2 # [ 142.990477] nix-daemon[870]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6831client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6832client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6833client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6834client # farm-b> LOG-farm-b-during-drain6835worker2 # [ 144.224887] nix-daemon[870]: reaped child process 1779, status = succeeded6836worker2 # [ 144.310064] nix-daemon[870]: accepted connection from pid 1796, user nix-grpc-daemon (trusted)6837worker2 # [ 144.321491] nix-daemon[870]: reaped child process 1799, status = succeeded6838lb # [ 144.351820] niks3-server[1119]: 2026/09/23 04:13:40 INFO Received uploads request method=POST path=/api/pending_closures6839worker2 # [ 144.338513] nix-grpc-daemon[1668]: time=2026-09-23T04:13:40.464Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6840worker2 # [ 144.340308] nix-grpc-daemon[1668]: time=2026-09-23T04:13:40.466Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6841lb # [ 144.392341] niks3-server[1119]: 2026/09/23 04:13:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6842worker2 # [ 144.381727] nix-grpc-daemon[1668]: time=2026-09-23T04:13:40.507Z level=INFO msg="Uploading 1 narinfos"6843lb # [ 144.403821] niks3-server[1119]: 2026/09/23 04:13:40 INFO Signed narinfos id=58 count=16844lb # [ 144.406370] niks3-server[1119]: 2026/09/23 04:13:40 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6845lb # [ 144.415544] niks3-server[1119]: 2026/09/23 04:13:40 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6846lb # [ 144.419802] niks3-server[1119]: 2026/09/23 04:13:40 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6847lb # [ 144.429650] niks3-server[1119]: 2026/09/23 04:13:40 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6848lb # [ 144.432394] niks3-server[1119]: 2026/09/23 04:13:40 INFO Completed upload id=586849lb # [ 144.435512] niks3-server[1119]: 2026/09/23 04:13:40 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6850worker2 # [ 144.412371] nix-grpc-daemon[1668]: time=2026-09-23T04:13:40.538Z level=INFO msg="Upload complete. (212ms)"6851worker1 # [ 144.416630] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:40Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06852worker2 # [ 144.418962] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43784 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=4 outputs=16853worker1 # [ 144.418485] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:40Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=16854worker2 # [ 144.434581] nix-daemon[870]: reaped child process 1777, status = succeeded6855worker2 # [ 144.468917] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437846856worker2 # [ 144.481088] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:437846857worker2 # [ 144.493433] nix-grpc-daemon[1668]: time=2026-09-23T04:13:40.619Z level=INFO msg="All 2 paths already cached"6858client # copying 1 paths to 'lb:50051'...6859worker2 # [ 144.502666] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:437846860client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6861worker2 # [ 144.517146] nix-daemon[870]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6862worker2 # [ 144.615373] nix-daemon[870]: accepted connection from pid 1803, user nix-grpc-daemon (trusted)6863worker2 # [ 144.629605] nix-daemon[870]: reaped child process 1806, status = succeeded6864lb # [ 144.657358] niks3-server[1119]: 2026/09/23 04:13:40 INFO Received uploads request method=POST path=/api/pending_closures6865worker2 # [ 144.643288] nix-grpc-daemon[1668]: time=2026-09-23T04:13:40.769Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6866worker2 # [ 144.644743] nix-grpc-daemon[1668]: time=2026-09-23T04:13:40.771Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6867lb # [ 144.687732] niks3-server[1119]: 2026/09/23 04:13:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6868lb # [ 144.693965] niks3-server[1119]: 2026/09/23 04:13:40 INFO Signed narinfos id=59 count=16869worker2 # [ 144.675623] nix-grpc-daemon[1668]: time=2026-09-23T04:13:40.801Z level=INFO msg="Uploading 1 narinfos"6870lb # [ 144.702838] niks3-server[1119]: 2026/09/23 04:13:40 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6871lb # [ 144.715386] niks3-server[1119]: 2026/09/23 04:13:40 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6872lb # [ 144.721238] niks3-server[1119]: 2026/09/23 04:13:40 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6873lb # [ 144.732429] niks3-server[1119]: 2026/09/23 04:13:40 INFO Completed upload id=596874worker2 # [ 144.711206] nix-grpc-daemon[1668]: time=2026-09-23T04:13:40.837Z level=INFO msg="Upload complete. (181ms)"6875lb # [ 144.735674] niks3-server[1119]: 2026/09/23 04:13:40 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6876worker2 # [ 144.716256] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43784 duration_s=0 paths=1 nar_bytes_in=7686877worker2 # [ 144.723258] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437846878worker2 # [ 144.732807] nix-daemon[870]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6879worker2 # [ 144.743146] nix-daemon[870]: reaped child process 1801, status = succeeded6880worker2 # [ 144.746715] nix-daemon[870]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6881client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6882client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6883client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6884client # farm-top> LOG-farm-top-during-drain6885lb # [ 145.895443] envoy[914]: conn peer=[2001:db8:1::1]:53102 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3006 tx=829 ms=33536886worker2 # [ 145.985725] nix-daemon[870]: reaped child process 1810, status = succeeded6887worker2 # [ 146.072090] nix-daemon[870]: accepted connection from pid 1826, user nix-grpc-daemon (trusted)6888worker2 # [ 146.085308] nix-daemon[870]: reaped child process 1829, status = succeeded6889lb # [ 146.114374] niks3-server[1119]: 2026/09/23 04:13:42 INFO Received uploads request method=POST path=/api/pending_closures6890worker2 # [ 146.102226] nix-grpc-daemon[1668]: time=2026-09-23T04:13:42.228Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6891worker2 # [ 146.105167] nix-grpc-daemon[1668]: time=2026-09-23T04:13:42.230Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6892lb # [ 146.156271] niks3-server[1119]: 2026/09/23 04:13:42 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6893lb # [ 146.163217] niks3-server[1119]: 2026/09/23 04:13:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6894lb # [ 146.167572] niks3-server[1119]: 2026/09/23 04:13:42 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6895lb # [ 146.172480] niks3-server[1119]: 2026/09/23 04:13:42 INFO Signed narinfos id=60 count=16896worker2 # [ 146.154380] nix-grpc-daemon[1668]: time=2026-09-23T04:13:42.280Z level=INFO msg="Uploading 1 narinfos"6897lb # [ 146.182125] niks3-server[1119]: 2026/09/23 04:13:42 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6898lb # [ 146.186930] niks3-server[1119]: 2026/09/23 04:13:42 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6899lb # [ 146.190699] niks3-server[1119]: 2026/09/23 04:13:42 INFO Completed upload id=606900lb # [ 146.193550] niks3-server[1119]: 2026/09/23 04:13:42 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6901worker2 # [ 146.169400] nix-grpc-daemon[1668]: time=2026-09-23T04:13:42.295Z level=INFO msg="Upload complete. (214ms)"6902worker2 # [ 146.175900] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43784 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=5 outputs=16903worker2 # [ 146.189440] nix-daemon[870]: reaped child process 1808, status = succeeded6904client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&2, in 5.49 seconds)6905client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath6906client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath, in 0.12 seconds)6907worker1: must succeed: pkill -f 'read -t [9]0 x'6908worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.04 seconds)6909client: waiting for success: ! systemctl is-active sw6910worker1 # [ 146.553456] nix-daemon[846]: reaped child process 1686, status = succeeded6911worker1 # [ 146.637419] nix-daemon[846]: accepted connection from pid 1734, user nix-grpc-daemon (trusted)6912worker1 # [ 146.650223] nix-daemon[846]: reaped child process 1737, status = succeeded6913lb # [ 146.687461] niks3-server[1119]: 2026/09/23 04:13:42 INFO Received uploads request method=POST path=/api/pending_closures6914worker1 # [ 146.666542] nix-grpc-daemon[1583]: time=2026-09-23T04:13:42.752Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6915worker1 # [ 146.668382] nix-grpc-daemon[1583]: time=2026-09-23T04:13:42.753Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6916lb # [ 146.728688] niks3-server[1119]: 2026/09/23 04:13:42 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6917lb # [ 146.737596] niks3-server[1119]: 2026/09/23 04:13:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6918lb # [ 146.742670] niks3-server[1119]: 2026/09/23 04:13:42 INFO Signed narinfos id=61 count=16919worker1 # [ 146.716535] nix-grpc-daemon[1583]: time=2026-09-23T04:13:42.802Z level=INFO msg="Uploading 1 narinfos"6920lb # [ 146.745721] niks3-server[1119]: 2026/09/23 04:13:42 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6921lb # [ 146.759074] niks3-server[1119]: 2026/09/23 04:13:42 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6922lb # [ 146.765623] niks3-server[1119]: 2026/09/23 04:13:42 INFO Completed upload id=616923lb # [ 146.768889] niks3-server[1119]: 2026/09/23 04:13:42 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6924worker1 # [ 146.738610] nix-grpc-daemon[1583]: time=2026-09-23T04:13:42.824Z level=INFO msg="Upload complete. (215ms)"6925lb # [ 146.772514] niks3-server[1119]: 2026/09/23 04:13:42 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6926worker1 # [ 146.747877] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41462 duration_s=8 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=2 outputs=16927worker1 # [ 146.761889] nix-daemon[846]: reaped child process 1684, status = succeeded6928client # [ 146.762607] systemd[1]: sw.service: Deactivated successfully.6929client # [ 146.767332] systemd[1]: sw.service: Consumed 135ms CPU time over 8.553s wall clock time, 43.4M memory peak, 7.1K incoming IP traffic, 8.3K outgoing IP traffic.6930worker1 # [ 147.128906] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:43Z level=info event=signal_exit6931client: (finished: waiting for success: ! systemctl is-active sw, in 1.06 seconds)6932client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6933client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds)6934worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6935lb # [ 148.160868] niks3-server[1119]: 2026/09/23 04:13:44 INFO lead: released remote=192.168.1.3:491286936worker2 # [ 149.141090] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:45Z level=info event=scheduler_restarting addr=lb:500516937worker1 # [ 149.134712] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:45Z level=info event=scheduler_restarting6938worker1 # [ 149.139769] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:45Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:414706939worker1 # [ 149.143998] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:45Z level=info event=worker_gone addr=192.168.1.4:500516940worker1 # [ 149.147765] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:45Z level=info event=scheduler_restarting addr=lb:500516941worker1 # [ 149.151006] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:45Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:414706942worker1 # [ 149.154665] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:45Z level=info event=worker_gone addr=192.168.1.3:500516943worker1 # [ 149.168951] nix-daemon[846]: reaped child process 1571, status = succeeded6944worker1 # [ 149.329184] nix-grpc-daemon[1514]: E0000 00:00:1790136825.414622 1560 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 22, started true, shutdown true6945worker2 # [ 149.341399] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:45Z level=info event=scheduler_connected addr=lb:500516946worker2 # [ 149.345225] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6947worker1 # [ 149.334362] nix-grpc-daemon[1514]: E0000 00:00:1790136825.414821 1560 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true6948worker1 # [ 149.351188] nix-grpc-daemon[1514]: ts=2026-09-23T04:13:45Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6949worker2 # [ 149.546312] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:45Z level=info event=scheduler_connected addr=lb:500516950worker2 # [ 149.550315] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6951worker2 # [ 149.751404] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:45Z level=info event=scheduler_connected addr=lb:500516952worker2 # [ 149.755329] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6953worker2 # [ 149.956586] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:46Z level=info event=scheduler_connected addr=lb:500516954worker2 # [ 149.960530] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6955worker2 # [ 150.162653] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:46Z level=info event=scheduler_connected addr=lb:500516956worker2 # [ 150.165786] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6957worker1 # [ 150.164880] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6958worker1 # [ 150.168880] systemd[1]: nix-grpc-daemon.service: Consumed 701ms CPU time over 28.151s wall clock time, 36.4M memory peak, 28.8K incoming IP traffic, 40K outgoing IP traffic.6959worker1 # [ 150.188347] systemd[1]: Starting Nix worker-protocol over gRPC...6960worker1 # [ 150.276623] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:46Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6961worker1 # [ 150.281527] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:46Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6962worker1 # [ 150.284242] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:46Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16963worker1 # [ 150.286215] nix-grpc-daemon[1751]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6964worker1 # [ 150.287524] nix-grpc-daemon[1751]: E0000 00:00:1790136826.369347 1751 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6965worker1 # [ 150.295702] nix-grpc-daemon[1751]: E0000 00:00:1790136826.376615 1751 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6966worker1 # [ 150.299377] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:46Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6967lb # [ 150.329924] niks3-server[1119]: 2026/09/23 04:13:46 INFO lead: acquired remote=192.168.1.3:489446968worker1 # [ 150.303119] systemd[1]: Started Nix worker-protocol over gRPC.6969worker1 # [ 150.304123] nix-grpc-daemon[1751]: E0000 00:00:1790136826.387061 1794 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6970worker1 # [ 150.310103] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:46Z level=info event=scheduler_take_over6971worker1 # [ 150.318074] nix-grpc-daemon[1751]: E0000 00:00:1790136826.402542 1794 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6972worker1 # [ 150.320214] nix-grpc-daemon[1751]: E0000 00:00:1790136826.402946 1794 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6973worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.87 seconds)6974lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6975lb: must succeed: curl -sf localhost:9901/clusters6976worker1 # [ 150.332340] nix-grpc-daemon[1751]: E0000 00:00:1790136826.418132 1794 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6977worker1 # [ 150.345570] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:46Z level=info event=scheduler_connected addr=lb:500516978worker1 # [ 150.347199] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6979worker2 # [ 150.363953] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:46Z level=info event=scheduler_connected addr=lb:500516980worker2 # [ 150.367838] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6981lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6982worker1 # [ 150.548431] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:46Z level=info event=scheduler_connected addr=lb:500516983worker1 # [ 150.552497] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6984worker2 # [ 150.569426] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:46Z level=info event=scheduler_connected addr=lb:500516985worker2 # [ 150.573335] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6986worker1 # [ 150.753410] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:46Z level=info event=scheduler_connected addr=lb:500516987worker1 # [ 150.757485] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6988worker2 # [ 150.774739] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:46Z level=info event=scheduler_connected addr=lb:500516989worker2 # [ 150.778506] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6990worker1 # [ 150.958371] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:47Z level=info event=scheduler_connected addr=lb:500516991worker1 # [ 150.962479] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6992worker2 # [ 150.979638] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:47Z level=info event=scheduler_connected addr=lb:500516993worker2 # [ 150.983512] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6994worker1 # [ 151.164516] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:47Z level=info event=scheduler_connected addr=lb:500516995worker1 # [ 151.171188] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6996worker2 # [ 151.185339] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:47Z level=info event=scheduler_connected addr=lb:500516997worker1 # [ 151.186144] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:47Z level=info event=oidc_jwks provider=mock keys=16998worker1 # [ 151.189972] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:47Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:524346999worker1 # [ 151.193929] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:47Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=07000worker1 # [ 151.370169] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:47Z level=info event=scheduler_connected addr=lb:500517001worker1 # [ 151.375094] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:47Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:524347002worker1 # [ 151.378817] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:47Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=07003lb: must succeed: curl -sf localhost:9901/clusters7004lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7005lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.10 seconds)7006lb: must succeed: curl -sf localhost:9901/clusters7007lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7008worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7009worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7010(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 13.32 seconds)7011subtest: queue gauges show a burst while every build blocks7012lb: must succeed: curl -sf localhost:9901/clusters7013lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7014lb: must succeed: curl -sf localhost:9901/clusters7015lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7016worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7017worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7018client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]'7019client # [ 151.630615] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]".7020client # Running as unit: burst.service; invocation ID: 9ac6461ac5134b94b1e10be872bff2777021client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.07 seconds)7022lb: must succeed: curl -sf localhost:9901/clusters7023lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)7024lb: must succeed: curl -sf localhost:9901/clusters7025lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7026worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7027worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7028worker1 # [ 151.796230] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:524367029worker1 # [ 151.807634] nix-daemon[846]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7030client # [ 151.803826] nix[1515]: don't know how to build these paths:7031client # [ 151.805235] nix[1515]: /nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv7032client # [ 151.806318] nix[1515]: /nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv7033client # [ 151.807308] nix[1515]: /nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv7034client # [ 151.808525] nix[1515]: /nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv7035worker1 # [ 151.832667] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:47Z level=debug event=want client=1 drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv cached=07036worker1 # [ 151.837804] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:47Z level=debug event=assigned drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv worker=192.168.1.4:50051 clients=17037worker1 # [ 151.842414] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:47Z level=debug event=want client=1 drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv cached=07038worker1 # [ 151.845994] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:47Z level=debug event=assigned drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv worker=192.168.1.3:50051 clients=17039worker1 # [ 151.849447] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:47Z level=debug event=want client=1 drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv cached=07040worker1 # [ 151.852507] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:47Z level=debug event=want client=1 drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv cached=07041worker1 # [ 151.883531] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524367042worker2 # [ 151.891946] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437847043worker1 # [ 151.894808] nix-grpc-daemon[1751]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 36 ms (attempt 1/5)7044worker2 # [ 151.905590] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:437847045client # [ 151.892295] nix[1515]: copying 1 paths to 'lb:50051'...7046worker2 # [ 151.917540] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:437847047client # [ 151.895155] nix[1515]: copying path '/nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv' to 'lb:50051'...7048worker2 # [ 151.931173] nix-daemon[870]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)7049worker1 # [ 151.938615] nix-grpc-daemon[1751]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 177 ms (attempt 2/5)7050worker2 # [ 152.033575] nix-daemon[870]: accepted connection from pid 1833, user nix-grpc-daemon (trusted)7051worker2 # [ 152.047172] nix-daemon[870]: reaped child process 1836, status = succeeded7052lb # [ 152.076250] niks3-server[1119]: 2026/09/23 04:13:48 INFO Received uploads request method=POST path=/api/pending_closures7053worker2 # [ 152.064612] nix-grpc-daemon[1668]: time=2026-09-23T04:13:48.190Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7054worker2 # [ 152.066512] nix-grpc-daemon[1668]: time=2026-09-23T04:13:48.192Z level=INFO msg="Uploading 2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv (416B)"7055lb # [ 152.106462] niks3-server[1119]: 2026/09/23 04:13:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign7056lb # [ 152.125658] niks3-server[1119]: 2026/09/23 04:13:48 INFO Registered completed upload object_key=nar/0raymp7y3ngiv7lj2qq6nkp63sms2510yhhlnr7hnz7ldnl7b3qc.nar.zst7057lb # [ 152.131832] niks3-server[1119]: 2026/09/23 04:13:48 INFO Registered completed upload object_key=2s9psgqm7awsyhlfyaclma7yr7qmrk0g.ls7058lb # [ 152.136280] niks3-server[1119]: 2026/09/23 04:13:48 INFO Signed narinfos id=62 count=17059worker2 # [ 152.115330] nix-grpc-daemon[1668]: time=2026-09-23T04:13:48.241Z level=INFO msg="Uploading 1 narinfos"7060lb # [ 152.147660] niks3-server[1119]: 2026/09/23 04:13:48 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete7061lb # [ 152.152145] niks3-server[1119]: 2026/09/23 04:13:48 INFO Completed upload id=627062worker2 # [ 152.131069] nix-grpc-daemon[1668]: time=2026-09-23T04:13:48.257Z level=INFO msg="Upload complete. (188ms)"7063worker1 # [ 152.123273] nix-grpc-daemon[1751]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 272 ms (attempt 3/5)7064lb # [ 152.156967] niks3-server[1119]: 2026/09/23 04:13:48 INFO Registered completed upload object_key=2s9psgqm7awsyhlfyaclma7yr7qmrk0g.narinfo7065worker2 # [ 152.135974] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43784 duration_s=0 paths=1 nar_bytes_in=4167066worker2 # [ 152.143271] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437847067worker2 # [ 152.151859] nix-daemon[870]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)7068worker2 # [ 152.162664] nix-daemon[870]: reaped child process 1831, status = succeeded7069worker2 # [ 152.167859] nix-daemon[870]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)7070client # [ 152.143551] nix[1515]: building '/nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv' on lb:50051...7071worker1 # [ 152.403758] nix-grpc-daemon[1751]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 431 ms (attempt 4/5)7072lb: must succeed: curl -sf localhost:9901/clusters7073lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7074lb: must succeed: curl -sf localhost:9901/clusters7075worker1 # [ 152.837193] nix-grpc-daemon[1751]: disabling binary cache 'https://cache.nixos.org' for 60 seconds7076worker1 # [ 152.840167] nix-grpc-daemon[1751]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org7077worker1 # [ 152.857392] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524367078lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7079worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7080client # [ 152.850908] nix[1515]: copying 1 paths to 'lb:50051'...7081client # [ 152.854122] nix[1515]: copying path '/nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv' to 'lb:50051'...7082worker1 # [ 152.870732] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524367083worker1 # [ 152.886401] nix-daemon[846]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7084worker1 # [ 152.912325] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:48Z level=info event=niks3_push_started pid=18267085worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)7086worker1 # [ 153.027339] nix-grpc-daemon[1826]: time=2026-09-23T04:13:49.112Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""7087worker1 # [ 153.115951] nix-daemon[846]: accepted connection from pid 1843, user nix-grpc-daemon (trusted)7088worker1 # [ 153.129345] nix-daemon[846]: reaped child process 1846, status = succeeded7089lb # [ 153.167355] niks3-server[1119]: 2026/09/23 04:13:49 INFO Received uploads request method=POST path=/api/pending_closures7090worker1 # [ 153.148810] nix-grpc-daemon[1826]: time=2026-09-23T04:13:49.234Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7091worker1 # [ 153.150352] nix-grpc-daemon[1826]: time=2026-09-23T04:13:49.236Z level=INFO msg="Uploading 885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv (416B)"7092lb # [ 153.210608] niks3-server[1119]: 2026/09/23 04:13:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign7093worker2 # [ 153.199831] nix-daemon[870]: reaped child process 1840, status = succeeded7094lb # [ 153.221824] niks3-server[1119]: 2026/09/23 04:13:49 INFO Signed narinfos id=63 count=17095worker1 # [ 153.195363] nix-grpc-daemon[1826]: time=2026-09-23T04:13:49.281Z level=INFO msg="Uploading 1 narinfos"7096lb # [ 153.226826] niks3-server[1119]: 2026/09/23 04:13:49 INFO Registered completed upload object_key=nar/0mmcg3cvna05ngbx6v5vspkmc5gj8lna2zc48rllhxr0g8i9mg6y.nar.zst7097lb # [ 153.233478] niks3-server[1119]: 2026/09/23 04:13:49 INFO Registered completed upload object_key=885n55y9srkrjs4p487lr255wvhla7hq.ls7098lb # [ 153.241760] niks3-server[1119]: 2026/09/23 04:13:49 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete7099lb # [ 153.249503] niks3-server[1119]: 2026/09/23 04:13:49 INFO Completed upload id=637100worker1 # [ 153.220322] nix-grpc-daemon[1826]: time=2026-09-23T04:13:49.305Z level=INFO msg="Upload complete. (190ms)"7101lb # [ 153.253325] niks3-server[1119]: 2026/09/23 04:13:49 INFO Registered completed upload object_key=885n55y9srkrjs4p487lr255wvhla7hq.narinfo7102worker1 # [ 153.224437] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52436 duration_s=0 paths=1 nar_bytes_in=4167103worker1 # [ 153.229322] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524367104worker1 # [ 153.239524] nix-daemon[846]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7105worker1 # [ 153.243760] nix-daemon[846]: reaped child process 1824, status = succeeded7106worker1 # [ 153.247894] nix-daemon[846]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7107client # [ 153.231485] nix[1515]: building '/nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv' on lb:50051...7108worker2 # [ 153.278863] nix-daemon[870]: accepted connection from pid 1861, user nix-grpc-daemon (trusted)7109worker2 # [ 153.291545] nix-daemon[870]: reaped child process 1864, status = succeeded7110lb # [ 153.320879] niks3-server[1119]: 2026/09/23 04:13:49 INFO Received uploads request method=POST path=/api/pending_closures7111worker2 # [ 153.308489] nix-grpc-daemon[1668]: time=2026-09-23T04:13:49.434Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7112worker2 # [ 153.310517] nix-grpc-daemon[1668]: time=2026-09-23T04:13:49.436Z level=INFO msg="Uploading vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq-slow-q3 (120B)"7113lb # [ 153.364176] niks3-server[1119]: 2026/09/23 04:13:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign7114lb # [ 153.369993] niks3-server[1119]: 2026/09/23 04:13:49 INFO Registered completed upload object_key=vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq.ls7115worker2 # [ 153.352695] nix-grpc-daemon[1668]: time=2026-09-23T04:13:49.478Z level=INFO msg="Uploading 1 narinfos"7116lb # [ 153.374490] niks3-server[1119]: 2026/09/23 04:13:49 INFO Signed narinfos id=64 count=17117lb # [ 153.376866] niks3-server[1119]: 2026/09/23 04:13:49 INFO Registered completed upload object_key=log/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv7118lb # [ 153.383779] niks3-server[1119]: 2026/09/23 04:13:49 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst7119lb # [ 153.387916] niks3-server[1119]: 2026/09/23 04:13:49 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete7120lb # [ 153.392712] niks3-server[1119]: 2026/09/23 04:13:49 INFO Completed upload id=647121worker2 # [ 153.371255] nix-grpc-daemon[1668]: time=2026-09-23T04:13:49.497Z level=INFO msg="Upload complete. (196ms)"7122lb # [ 153.395617] niks3-server[1119]: 2026/09/23 04:13:49 INFO Registered completed upload object_key=vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq.narinfo7123worker2 # [ 153.376516] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43784 duration_s=1 drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv assign_id=1 outputs=17124worker1 # [ 153.371333] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:49Z level=debug event=assigned drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv worker=192.168.1.4:50051 clients=17125worker2 # [ 153.388820] nix-daemon[870]: reaped child process 1838, status = succeeded7126worker2 # [ 153.428714] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437847127worker2 # [ 153.441514] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:437847128client # [ 153.427215] nix[1515]: copying 1 paths to 'lb:50051'...7129worker2 # [ 153.453081] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:437847130client # [ 153.430172] nix[1515]: copying path '/nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv' to 'lb:50051'...7131worker2 # [ 153.464344] nix-daemon[870]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)7132worker2 # [ 153.562439] nix-daemon[870]: accepted connection from pid 1868, user nix-grpc-daemon (trusted)7133worker2 # [ 153.576656] nix-daemon[870]: reaped child process 1871, status = succeeded7134lb # [ 153.605079] niks3-server[1119]: 2026/09/23 04:13:49 INFO Received uploads request method=POST path=/api/pending_closures7135worker2 # [ 153.591953] nix-grpc-daemon[1668]: time=2026-09-23T04:13:49.718Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7136worker2 # [ 153.593421] nix-grpc-daemon[1668]: time=2026-09-23T04:13:49.719Z level=INFO msg="Uploading 8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv (416B)"7137lb # [ 153.636267] niks3-server[1119]: 2026/09/23 04:13:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign7138lb # [ 153.644766] niks3-server[1119]: 2026/09/23 04:13:49 INFO Signed narinfos id=65 count=17139worker2 # [ 153.627586] nix-grpc-daemon[1668]: time=2026-09-23T04:13:49.753Z level=INFO msg="Uploading 1 narinfos"7140lb # [ 153.652290] niks3-server[1119]: 2026/09/23 04:13:49 INFO Registered completed upload object_key=8v867csm7d9sip4xd4nlg9jwvrpqqwv9.ls7141lb # [ 153.657832] niks3-server[1119]: 2026/09/23 04:13:49 INFO Registered completed upload object_key=nar/1xibay372y9by9a678y9h7bfzwahsxi0vq51acgywak31q6q2m87.nar.zst7142lb # [ 153.670129] niks3-server[1119]: 2026/09/23 04:13:49 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete7143lb # [ 153.675247] niks3-server[1119]: 2026/09/23 04:13:49 INFO Completed upload id=657144worker2 # [ 153.654143] nix-grpc-daemon[1668]: time=2026-09-23T04:13:49.780Z level=INFO msg="Upload complete. (178ms)"7145lb # [ 153.681526] niks3-server[1119]: 2026/09/23 04:13:49 INFO Registered completed upload object_key=8v867csm7d9sip4xd4nlg9jwvrpqqwv9.narinfo7146worker2 # [ 153.658626] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43784 duration_s=0 paths=1 nar_bytes_in=4167147worker2 # [ 153.665292] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437847148worker2 # [ 153.676484] nix-daemon[870]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)7149worker2 # [ 153.682675] nix-daemon[870]: reaped child process 1866, status = succeeded7150worker2 # [ 153.688119] nix-daemon[870]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)7151client # [ 153.663172] nix[1515]: building '/nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv' on lb:50051...7152worker1 # [ 155.482186] nix-daemon[846]: reaped child process 1850, status = succeeded7153worker2 # [ 155.533076] nix-daemon[870]: reaped child process 1875, status = succeeded7154worker1 # [ 155.569381] nix-daemon[846]: accepted connection from pid 1875, user nix-grpc-daemon (trusted)7155worker1 # [ 155.581935] nix-daemon[846]: reaped child process 1878, status = succeeded7156lb # [ 155.621731] niks3-server[1119]: 2026/09/23 04:13:51 INFO Received uploads request method=POST path=/api/pending_closures7157worker1 # [ 155.599410] nix-grpc-daemon[1826]: time=2026-09-23T04:13:51.684Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7158worker1 # [ 155.601142] nix-grpc-daemon[1826]: time=2026-09-23T04:13:51.686Z level=INFO msg="Uploading nv0kp5b571hsjibnd9v06981i024iima-slow-q2 (120B)"7159worker2 # [ 155.610079] nix-daemon[870]: accepted connection from pid 1900, user nix-grpc-daemon (trusted)7160worker2 # [ 155.623204] nix-daemon[870]: reaped child process 1903, status = succeeded7161lb # [ 155.652135] niks3-server[1119]: 2026/09/23 04:13:51 INFO Received uploads request method=POST path=/api/pending_closures7162worker2 # [ 155.643291] nix-grpc-daemon[1668]: time=2026-09-23T04:13:51.768Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7163lb # [ 155.665655] niks3-server[1119]: 2026/09/23 04:13:51 INFO Registered completed upload object_key=log/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv7164worker2 # [ 155.646949] nix-grpc-daemon[1668]: time=2026-09-23T04:13:51.768Z level=INFO msg="Uploading wmw8972gq2nf739xhg7is7lylxf8299y-slow-q0 (120B)"7165lb # [ 155.678833] niks3-server[1119]: 2026/09/23 04:13:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign7166lb # [ 155.684143] niks3-server[1119]: 2026/09/23 04:13:51 INFO Signed narinfos id=66 count=17167worker1 # [ 155.654735] nix-grpc-daemon[1826]: time=2026-09-23T04:13:51.739Z level=INFO msg="Uploading 1 narinfos"7168lb # [ 155.700072] niks3-server[1119]: 2026/09/23 04:13:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign7169lb # [ 155.704094] niks3-server[1119]: 2026/09/23 04:13:51 INFO Signed narinfos id=67 count=17170worker2 # [ 155.682333] nix-grpc-daemon[1668]: time=2026-09-23T04:13:51.808Z level=INFO msg="Uploading 1 narinfos"7171lb # [ 155.707863] niks3-server[1119]: 2026/09/23 04:13:51 INFO Registered completed upload object_key=nv0kp5b571hsjibnd9v06981i024iima.ls7172lb # [ 155.710247] niks3-server[1119]: 2026/09/23 04:13:51 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst7173lb # [ 155.714989] niks3-server[1119]: 2026/09/23 04:13:51 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst7174lb # [ 155.720705] niks3-server[1119]: 2026/09/23 04:13:51 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete7175lb # [ 155.723847] niks3-server[1119]: 2026/09/23 04:13:51 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete7176lb # [ 155.727061] niks3-server[1119]: 2026/09/23 04:13:51 INFO Completed upload id=677177worker2 # [ 155.704901] nix-grpc-daemon[1668]: time=2026-09-23T04:13:51.831Z level=INFO msg="Upload complete. (208ms)"7178lb # [ 155.731230] niks3-server[1119]: 2026/09/23 04:13:51 INFO Registered completed upload object_key=wmw8972gq2nf739xhg7is7lylxf8299y.ls7179lb # [ 155.735678] niks3-server[1119]: 2026/09/23 04:13:51 INFO Completed upload id=667180worker2 # [ 155.710342] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43784 duration_s=2 drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv assign_id=3 outputs=17181lb # [ 155.736950] niks3-server[1119]: 2026/09/23 04:13:51 INFO Registered completed upload object_key=log/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv7182lb # [ 155.738662] niks3-server[1119]: 2026/09/23 04:13:51 INFO Registered completed upload object_key=wmw8972gq2nf739xhg7is7lylxf8299y.narinfo7183worker1 # [ 155.703408] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:51Z level=debug event=assigned drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv worker=192.168.1.4:50051 clients=17184worker2 # [ 155.717368] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437847185lb # [ 155.742950] niks3-server[1119]: 2026/09/23 04:13:51 INFO Registered completed upload object_key=nv0kp5b571hsjibnd9v06981i024iima.narinfo7186worker1 # [ 155.712427] nix-grpc-daemon[1826]: time=2026-09-23T04:13:51.794Z level=INFO msg="Upload complete. (250ms)"7187worker1 # [ 155.716207] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52436 duration_s=2 drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv assign_id=2 outputs=17188worker2 # [ 155.728283] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:437847189worker2 # [ 155.731039] nix-daemon[870]: reaped child process 1873, status = succeeded7190worker2 # [ 155.734525] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:437847191client # [ 155.710854] nix[1515]: copying 1 paths to 'lb:50051'...7192client # [ 155.712766] nix[1515]: copying path '/nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv' to 'lb:50051'...7193worker1 # [ 155.730729] nix-daemon[846]: reaped child process 1848, status = succeeded7194worker2 # [ 155.744060] nix-daemon[870]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)7195worker2 # [ 155.830716] nix-daemon[870]: accepted connection from pid 1907, user nix-grpc-daemon (trusted)7196worker2 # [ 155.845118] nix-daemon[870]: reaped child process 1910, status = succeeded7197lb # [ 155.874485] niks3-server[1119]: 2026/09/23 04:13:52 INFO Received uploads request method=POST path=/api/pending_closures7198worker2 # [ 155.860723] nix-grpc-daemon[1668]: time=2026-09-23T04:13:51.987Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7199worker2 # [ 155.862665] nix-grpc-daemon[1668]: time=2026-09-23T04:13:51.988Z level=INFO msg="Uploading qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv (416B)"7200lb # [ 155.895832] envoy[914]: conn peer=[2001:db8:1::1]:53114 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9555 tx=4403 ms=53137201lb # [ 155.900232] envoy[914]: conn peer=[2001:db8:1::1]:53106 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2945 tx=1498 ms=83827202lb # [ 155.903762] envoy[914]: conn peer=[2001:db8:1::3]:46348 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2089 tx=652 ms=271787203lb # [ 155.915951] niks3-server[1119]: 2026/09/23 04:13:52 INFO Registered completed upload object_key=nar/1spxkiwzvlj9sgbjal4jf4p68h5kbylxymfww73s5va157qxgafi.nar.zst7204lb # [ 155.921791] niks3-server[1119]: 2026/09/23 04:13:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign7205worker2 # [ 155.906600] nix-grpc-daemon[1668]: time=2026-09-23T04:13:52.032Z level=INFO msg="Uploading 1 narinfos"7206lb # [ 155.928096] niks3-server[1119]: 2026/09/23 04:13:52 INFO Signed narinfos id=68 count=17207lb # [ 155.934169] niks3-server[1119]: 2026/09/23 04:13:52 INFO Registered completed upload object_key=qs0cgdr560r5cllf2djj9bb4gz0vrp4p.ls7208lb # [ 155.940558] niks3-server[1119]: 2026/09/23 04:13:52 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete7209lb # [ 155.945661] niks3-server[1119]: 2026/09/23 04:13:52 INFO Completed upload id=687210worker2 # [ 155.924377] nix-grpc-daemon[1668]: time=2026-09-23T04:13:52.050Z level=INFO msg="Upload complete. (175ms)"7211lb # [ 155.947463] niks3-server[1119]: 2026/09/23 04:13:52 INFO Registered completed upload object_key=qs0cgdr560r5cllf2djj9bb4gz0vrp4p.narinfo7212worker2 # [ 155.928489] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43784 duration_s=0 paths=1 nar_bytes_in=4167213worker2 # [ 155.933490] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437847214worker2 # [ 155.942110] nix-daemon[870]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)7215worker2 # [ 155.949580] nix-daemon[870]: reaped child process 1905, status = succeeded7216worker2 # [ 155.954306] nix-daemon[870]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)7217client # [ 155.929766] nix[1515]: building '/nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv' on lb:50051...7218worker2 # [ 157.795260] nix-daemon[870]: reaped child process 1914, status = succeeded7219worker2 # [ 157.873532] nix-daemon[870]: accepted connection from pid 1939, user nix-grpc-daemon (trusted)7220worker2 # [ 157.885658] nix-daemon[870]: reaped child process 1942, status = succeeded7221lb # [ 157.916702] niks3-server[1119]: 2026/09/23 04:13:54 INFO Received uploads request method=POST path=/api/pending_closures7222worker2 # [ 157.905568] nix-grpc-daemon[1668]: time=2026-09-23T04:13:54.031Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7223worker2 # [ 157.907176] nix-grpc-daemon[1668]: time=2026-09-23T04:13:54.033Z level=INFO msg="Uploading 106i40yblbkazayvn9ppswrq6kqs26sj-slow-q1 (120B)"7224lb # [ 157.958799] niks3-server[1119]: 2026/09/23 04:13:54 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst7225lb # [ 157.970262] niks3-server[1119]: 2026/09/23 04:13:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign7226lb # [ 157.976640] niks3-server[1119]: 2026/09/23 04:13:54 INFO Signed narinfos id=69 count=17227worker2 # [ 157.955648] nix-grpc-daemon[1668]: time=2026-09-23T04:13:54.081Z level=INFO msg="Uploading 1 narinfos"7228lb # [ 157.980754] niks3-server[1119]: 2026/09/23 04:13:54 INFO Registered completed upload object_key=log/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv7229lb # [ 157.990524] niks3-server[1119]: 2026/09/23 04:13:54 INFO Registered completed upload object_key=106i40yblbkazayvn9ppswrq6kqs26sj.ls7230lb # [ 157.999738] niks3-server[1119]: 2026/09/23 04:13:54 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete7231lb # [ 158.002800] niks3-server[1119]: 2026/09/23 04:13:54 INFO Completed upload id=697232worker2 # [ 157.981285] nix-grpc-daemon[1668]: time=2026-09-23T04:13:54.107Z level=INFO msg="Upload complete. (218ms)"7233lb # [ 158.005509] niks3-server[1119]: 2026/09/23 04:13:54 INFO Registered completed upload object_key=106i40yblbkazayvn9ppswrq6kqs26sj.narinfo7234worker2 # [ 157.987717] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43784 duration_s=2 drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv assign_id=4 outputs=17235worker2 # [ 158.002533] nix-daemon[870]: reaped child process 1912, status = succeeded7236client # [ 157.998128] systemd[1]: burst.service: Deactivated successfully.7237client # [ 158.001767] systemd[1]: burst.service: Consumed 175ms CPU time over 6.370s wall clock time, 55.9M memory peak, 14.3K incoming IP traffic, 19.1K outgoing IP traffic.7238client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }7239client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.05 seconds)7240lb: must succeed: curl -sf localhost:9901/clusters7241lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7242lb: must succeed: curl -sf localhost:9901/clusters7243lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7244worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7245worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7246(finished: subtest: queue gauges show a burst while every build blocks, in 7.46 seconds)7247subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does7248client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&17249worker2 # [ 159.122542] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:437847250worker2 # [ 159.130210] nix-grpc-daemon[1595]: ts=2026-09-23T04:13:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:437847251worker1 # [ 159.161832] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:55Z level=debug event=want client=2 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=07252worker1 # [ 159.166093] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:55Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=17253worker1 # [ 159.170408] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:55Z level=debug event=want client=2 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=07254worker1 # [ 159.174493] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:55Z level=debug event=want client=2 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=07255worker1 # [ 159.205401] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524367256worker1 # [ 159.217673] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524367257worker1 # [ 159.229436] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524367258worker1 # [ 159.240190] nix-daemon[846]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7259worker1 # [ 159.339721] nix-daemon[846]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)7260worker1 # [ 159.352523] nix-daemon[846]: reaped child process 1902, status = succeeded7261lb # [ 159.390820] niks3-server[1119]: 2026/09/23 04:13:55 INFO Received uploads request method=POST path=/api/pending_closures7262worker1 # [ 159.369961] nix-grpc-daemon[1826]: time=2026-09-23T04:13:55.455Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7263worker1 # [ 159.372099] nix-grpc-daemon[1826]: time=2026-09-23T04:13:55.457Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"7264lb # [ 159.420495] niks3-server[1119]: 2026/09/23 04:13:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign7265worker1 # [ 159.397308] nix-grpc-daemon[1826]: time=2026-09-23T04:13:55.482Z level=INFO msg="Uploading 1 narinfos"7266lb # [ 159.426956] niks3-server[1119]: 2026/09/23 04:13:55 INFO Signed narinfos id=70 count=17267lb # [ 159.435442] niks3-server[1119]: 2026/09/23 04:13:55 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls7268lb # [ 159.446398] niks3-server[1119]: 2026/09/23 04:13:55 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst7269lb # [ 159.454373] niks3-server[1119]: 2026/09/23 04:13:55 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete7270lb # [ 159.460501] niks3-server[1119]: 2026/09/23 04:13:55 INFO Completed upload id=707271worker1 # [ 159.431425] nix-grpc-daemon[1826]: time=2026-09-23T04:13:55.517Z level=INFO msg="Upload complete. (179ms)"7272lb # [ 159.466722] niks3-server[1119]: 2026/09/23 04:13:55 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo7273worker1 # [ 159.435884] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52436 duration_s=0 paths=1 nar_bytes_in=5047274worker1 # [ 159.442284] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524367275worker1 # [ 159.451489] nix-daemon[846]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7276worker1 # [ 159.460117] nix-daemon[846]: reaped child process 1897, status = succeeded7277worker1 # [ 159.463467] nix-daemon[846]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7278worker1 # [ 160.477919] nix-daemon[846]: reaped child process 1906, status = succeeded7279worker1 # [ 160.555384] nix-daemon[846]: accepted connection from pid 1922, user nix-grpc-daemon (trusted)7280worker1 # [ 160.567827] nix-daemon[846]: reaped child process 1925, status = succeeded7281lb # [ 160.606858] niks3-server[1119]: 2026/09/23 04:13:56 INFO Received uploads request method=POST path=/api/pending_closures7282worker1 # [ 160.585720] nix-grpc-daemon[1826]: time=2026-09-23T04:13:56.671Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7283worker1 # [ 160.587243] nix-grpc-daemon[1826]: time=2026-09-23T04:13:56.673Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"7284lb # [ 160.646949] niks3-server[1119]: 2026/09/23 04:13:56 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls7285lb # [ 160.654205] niks3-server[1119]: 2026/09/23 04:13:56 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst7286lb # [ 160.663259] niks3-server[1119]: 2026/09/23 04:13:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign7287lb # [ 160.669370] niks3-server[1119]: 2026/09/23 04:13:56 INFO Signed narinfos id=71 count=17288worker1 # [ 160.640480] nix-grpc-daemon[1826]: time=2026-09-23T04:13:56.725Z level=INFO msg="Uploading 1 narinfos"7289lb # [ 160.674902] niks3-server[1119]: 2026/09/23 04:13:56 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv7290lb # [ 160.684991] niks3-server[1119]: 2026/09/23 04:13:56 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete7291lb # [ 160.688982] niks3-server[1119]: 2026/09/23 04:13:56 INFO Completed upload id=717292worker1 # [ 160.660070] nix-grpc-daemon[1826]: time=2026-09-23T04:13:56.745Z level=INFO msg="Upload complete. (189ms)"7293lb # [ 160.692490] niks3-server[1119]: 2026/09/23 04:13:56 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo7294worker1 # [ 160.666878] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52436 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=5 outputs=17295worker1 # [ 160.677816] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:56Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=17296worker1 # [ 160.683122] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524367297worker1 # [ 160.686287] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:56Z level=debug event=want client=2 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=07298worker1 # [ 160.696288] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524367299worker1 # [ 160.698708] nix-daemon[846]: reaped child process 1904, status = succeeded7300worker1 # [ 160.702729] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524367301worker1 # [ 160.706695] nix-daemon[846]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7302worker1 # [ 160.793216] nix-daemon[846]: accepted connection from pid 1931, user nix-grpc-daemon (trusted)7303worker1 # [ 160.805545] nix-daemon[846]: reaped child process 1934, status = succeeded7304lb # [ 160.843989] niks3-server[1119]: 2026/09/23 04:13:57 INFO Received uploads request method=POST path=/api/pending_closures7305worker1 # [ 160.822598] nix-grpc-daemon[1826]: time=2026-09-23T04:13:56.908Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7306worker1 # [ 160.824145] nix-grpc-daemon[1826]: time=2026-09-23T04:13:56.909Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"7307lb # [ 160.877307] niks3-server[1119]: 2026/09/23 04:13:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign7308worker1 # [ 160.853196] nix-grpc-daemon[1826]: time=2026-09-23T04:13:56.938Z level=INFO msg="Uploading 1 narinfos"7309lb # [ 160.883338] niks3-server[1119]: 2026/09/23 04:13:57 INFO Signed narinfos id=72 count=17310lb # [ 160.890816] niks3-server[1119]: 2026/09/23 04:13:57 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst7311lb # [ 160.897132] niks3-server[1119]: 2026/09/23 04:13:57 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls7312lb # [ 160.906592] niks3-server[1119]: 2026/09/23 04:13:57 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete7313lb # [ 160.914877] niks3-server[1119]: 2026/09/23 04:13:57 INFO Completed upload id=727314worker1 # [ 160.885313] nix-grpc-daemon[1826]: time=2026-09-23T04:13:56.971Z level=INFO msg="Upload complete. (173ms)"7315lb # [ 160.916649] niks3-server[1119]: 2026/09/23 04:13:57 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo7316worker1 # [ 160.889412] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52436 duration_s=0 paths=1 nar_bytes_in=5047317worker1 # [ 160.894681] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524367318worker1 # [ 160.905110] nix-daemon[846]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7319worker1 # [ 160.909156] nix-daemon[846]: reaped child process 1929, status = succeeded7320worker1 # [ 160.913938] nix-daemon[846]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7321worker1 # [ 161.928486] nix-daemon[846]: reaped child process 1938, status = succeeded7322worker1 # [ 162.005673] nix-daemon[846]: accepted connection from pid 1954, user nix-grpc-daemon (trusted)7323worker1 # [ 162.018222] nix-daemon[846]: reaped child process 1957, status = succeeded7324lb # [ 162.056830] niks3-server[1119]: 2026/09/23 04:13:58 INFO Received uploads request method=POST path=/api/pending_closures7325worker1 # [ 162.035612] nix-grpc-daemon[1826]: time=2026-09-23T04:13:58.121Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7326worker1 # [ 162.037142] nix-grpc-daemon[1826]: time=2026-09-23T04:13:58.122Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"7327lb # [ 162.094970] niks3-server[1119]: 2026/09/23 04:13:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign7328lb # [ 162.099712] niks3-server[1119]: 2026/09/23 04:13:58 INFO Signed narinfos id=73 count=17329worker1 # [ 162.073562] nix-grpc-daemon[1826]: time=2026-09-23T04:13:58.159Z level=INFO msg="Uploading 1 narinfos"7330lb # [ 162.111180] niks3-server[1119]: 2026/09/23 04:13:58 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls7331lb # [ 162.116491] niks3-server[1119]: 2026/09/23 04:13:58 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst7332lb # [ 162.121756] niks3-server[1119]: 2026/09/23 04:13:58 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv7333lb # [ 162.131930] niks3-server[1119]: 2026/09/23 04:13:58 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete7334lb # [ 162.135539] niks3-server[1119]: 2026/09/23 04:13:58 INFO Completed upload id=737335worker1 # [ 162.107606] nix-grpc-daemon[1826]: time=2026-09-23T04:13:58.193Z level=INFO msg="Upload complete. (185ms)"7336lb # [ 162.138688] niks3-server[1119]: 2026/09/23 04:13:58 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo7337worker1 # [ 162.113545] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52436 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=6 outputs=17338worker1 # [ 162.122660] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:58Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=17339worker1 # [ 162.128233] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524367340worker1 # [ 162.131412] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:58Z level=debug event=want client=2 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=07341worker1 # [ 162.139735] nix-daemon[846]: reaped child process 1936, status = succeeded7342worker1 # [ 162.141360] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524367343worker1 # [ 162.147363] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524367344worker1 # [ 162.152073] nix-daemon[846]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7345worker1 # [ 162.238977] nix-daemon[846]: accepted connection from pid 1963, user nix-grpc-daemon (trusted)7346worker1 # [ 162.251760] nix-daemon[846]: reaped child process 1966, status = succeeded7347lb # [ 162.289428] niks3-server[1119]: 2026/09/23 04:13:58 INFO Received uploads request method=POST path=/api/pending_closures7348worker1 # [ 162.268116] nix-grpc-daemon[1826]: time=2026-09-23T04:13:58.353Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7349worker1 # [ 162.270136] nix-grpc-daemon[1826]: time=2026-09-23T04:13:58.355Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"7350lb # [ 162.318151] niks3-server[1119]: 2026/09/23 04:13:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign7351worker1 # [ 162.294465] nix-grpc-daemon[1826]: time=2026-09-23T04:13:58.380Z level=INFO msg="Uploading 1 narinfos"7352lb # [ 162.324475] niks3-server[1119]: 2026/09/23 04:13:58 INFO Signed narinfos id=74 count=17353lb # [ 162.331601] niks3-server[1119]: 2026/09/23 04:13:58 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst7354lb # [ 162.340201] niks3-server[1119]: 2026/09/23 04:13:58 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls7355lb # [ 162.350304] niks3-server[1119]: 2026/09/23 04:13:58 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete7356lb # [ 162.357063] niks3-server[1119]: 2026/09/23 04:13:58 INFO Completed upload id=747357worker1 # [ 162.327709] nix-grpc-daemon[1826]: time=2026-09-23T04:13:58.413Z level=INFO msg="Upload complete. (171ms)"7358lb # [ 162.359049] niks3-server[1119]: 2026/09/23 04:13:58 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo7359worker1 # [ 162.331174] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52436 duration_s=0 paths=1 nar_bytes_in=5047360worker1 # [ 162.336215] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524367361worker1 # [ 162.345794] nix-daemon[846]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7362worker1 # [ 162.352139] nix-daemon[846]: reaped child process 1961, status = succeeded7363worker1 # [ 162.355208] nix-daemon[846]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7364worker1 # [ 163.389732] nix-daemon[846]: reaped child process 1970, status = succeeded7365worker1 # [ 163.477580] nix-daemon[846]: accepted connection from pid 1986, user nix-grpc-daemon (trusted)7366worker1 # [ 163.488545] nix-daemon[846]: reaped child process 1989, status = succeeded7367lb # [ 163.527951] niks3-server[1119]: 2026/09/23 04:13:59 INFO Received uploads request method=POST path=/api/pending_closures7368worker1 # [ 163.509531] nix-grpc-daemon[1826]: time=2026-09-23T04:13:59.595Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7369worker1 # [ 163.511148] nix-grpc-daemon[1826]: time=2026-09-23T04:13:59.596Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"7370lb # [ 163.569859] niks3-server[1119]: 2026/09/23 04:13:59 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst7371lb # [ 163.583129] niks3-server[1119]: 2026/09/23 04:13:59 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv7372lb # [ 163.591518] niks3-server[1119]: 2026/09/23 04:13:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign7373lb # [ 163.596467] niks3-server[1119]: 2026/09/23 04:13:59 INFO Signed narinfos id=75 count=17374worker1 # [ 163.570173] nix-grpc-daemon[1826]: time=2026-09-23T04:13:59.655Z level=INFO msg="Uploading 1 narinfos"7375lb # [ 163.601506] niks3-server[1119]: 2026/09/23 04:13:59 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls7376lb # [ 163.612322] niks3-server[1119]: 2026/09/23 04:13:59 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete7377lb # [ 163.614921] niks3-server[1119]: 2026/09/23 04:13:59 INFO Completed upload id=757378worker1 # [ 163.585597] nix-grpc-daemon[1826]: time=2026-09-23T04:13:59.671Z level=INFO msg="Upload complete. (223ms)"7379lb # [ 163.616798] niks3-server[1119]: 2026/09/23 04:13:59 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo7380worker1 # [ 163.594453] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52436 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=7 outputs=17381worker1 # [ 163.603714] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:59Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=17382worker1 # [ 163.609190] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524367383worker1 # [ 163.612388] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:59Z level=debug event=want client=2 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=07384worker1 # [ 163.621418] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524367385worker1 # [ 163.624848] nix-daemon[846]: reaped child process 1968, status = succeeded7386worker1 # [ 163.630291] nix-grpc-daemon[1826]: time=2026-09-23T04:13:59.716Z level=INFO msg="All 1 paths already cached"7387worker1 # [ 163.635737] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524367388worker1 # [ 163.639514] nix-daemon[846]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7389worker1 # [ 163.726165] nix-daemon[846]: accepted connection from pid 1995, user nix-grpc-daemon (trusted)7390worker1 # [ 163.738771] nix-daemon[846]: reaped child process 1998, status = succeeded7391lb # [ 163.777155] niks3-server[1119]: 2026/09/23 04:13:59 INFO Received uploads request method=POST path=/api/pending_closures7392worker1 # [ 163.756579] nix-grpc-daemon[1826]: time=2026-09-23T04:13:59.842Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7393worker1 # [ 163.758621] nix-grpc-daemon[1826]: time=2026-09-23T04:13:59.843Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"7394lb # [ 163.810647] niks3-server[1119]: 2026/09/23 04:14:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign7395worker1 # [ 163.787737] nix-grpc-daemon[1826]: time=2026-09-23T04:13:59.873Z level=INFO msg="Uploading 1 narinfos"7396lb # [ 163.817491] niks3-server[1119]: 2026/09/23 04:14:00 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst7397lb # [ 163.821623] niks3-server[1119]: 2026/09/23 04:14:00 INFO Signed narinfos id=76 count=17398lb # [ 163.830598] niks3-server[1119]: 2026/09/23 04:14:00 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls7399lb # [ 163.840264] niks3-server[1119]: 2026/09/23 04:14:00 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete7400worker1 # [ 163.816250] nix-grpc-daemon[1826]: time=2026-09-23T04:13:59.902Z level=INFO msg="Upload complete. (171ms)"7401lb # [ 163.845614] niks3-server[1119]: 2026/09/23 04:14:00 INFO Completed upload id=767402worker1 # [ 163.818994] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52436 duration_s=0 paths=1 nar_bytes_in=6247403lb # [ 163.853309] niks3-server[1119]: 2026/09/23 04:14:00 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo7404worker1 # [ 163.823931] nix-grpc-daemon[1751]: ts=2026-09-23T04:13:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524367405worker1 # [ 163.832343] nix-daemon[846]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7406worker1 # [ 163.841874] nix-daemon[846]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7407worker1 # [ 163.843964] nix-daemon[846]: reaped child process 1993, status = succeeded7408worker1 # [ 165.079814] nix-daemon[846]: reaped child process 2002, status = succeeded7409worker1 # [ 165.164092] nix-daemon[846]: accepted connection from pid 2018, user nix-grpc-daemon (trusted)7410worker1 # [ 165.175736] nix-daemon[846]: reaped child process 2021, status = succeeded7411lb # [ 165.214580] niks3-server[1119]: 2026/09/23 04:14:01 INFO Received uploads request method=POST path=/api/pending_closures7412worker1 # [ 165.193561] nix-grpc-daemon[1826]: time=2026-09-23T04:14:01.279Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7413worker1 # [ 165.198573] nix-grpc-daemon[1826]: time=2026-09-23T04:14:01.281Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"7414lb # [ 165.260062] niks3-server[1119]: 2026/09/23 04:14:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign7415worker1 # [ 165.236698] nix-grpc-daemon[1826]: time=2026-09-23T04:14:01.322Z level=INFO msg="Uploading 1 narinfos"7416lb # [ 165.266564] niks3-server[1119]: 2026/09/23 04:14:01 INFO Signed narinfos id=77 count=17417lb # [ 165.275294] niks3-server[1119]: 2026/09/23 04:14:01 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv7418lb # [ 165.279975] niks3-server[1119]: 2026/09/23 04:14:01 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls7419lb # [ 165.287138] niks3-server[1119]: 2026/09/23 04:14:01 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst7420lb # [ 165.295139] niks3-server[1119]: 2026/09/23 04:14:01 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete7421lb # [ 165.300994] niks3-server[1119]: 2026/09/23 04:14:01 INFO Completed upload id=777422worker1 # [ 165.271677] nix-grpc-daemon[1826]: time=2026-09-23T04:14:01.357Z level=INFO msg="Upload complete. (221ms)"7423lb # [ 165.303356] niks3-server[1119]: 2026/09/23 04:14:01 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo7424worker1 # [ 165.279640] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52436 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=8 outputs=17425worker1 # [ 165.290183] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:01Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=17426worker1 # [ 165.294923] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524367427worker1 # [ 165.299343] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:01Z level=debug event=want client=2 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=07428worker1 # [ 165.308275] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524367429worker1 # [ 165.312503] nix-daemon[846]: reaped child process 2000, status = succeeded7430worker1 # [ 165.317077] nix-grpc-daemon[1826]: time=2026-09-23T04:14:01.402Z level=INFO msg="All 1 paths already cached"7431worker1 # [ 165.321625] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524367432worker1 # [ 165.326218] nix-daemon[846]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7433worker1 # [ 165.411904] nix-daemon[846]: accepted connection from pid 2027, user nix-grpc-daemon (trusted)7434worker1 # [ 165.424405] nix-daemon[846]: reaped child process 2030, status = succeeded7435lb # [ 165.462321] niks3-server[1119]: 2026/09/23 04:14:01 INFO Received uploads request method=POST path=/api/pending_closures7436worker1 # [ 165.440411] nix-grpc-daemon[1826]: time=2026-09-23T04:14:01.526Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7437worker1 # [ 165.442413] nix-grpc-daemon[1826]: time=2026-09-23T04:14:01.527Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"7438lb # [ 165.493528] niks3-server[1119]: 2026/09/23 04:14:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign7439lb # [ 165.499271] niks3-server[1119]: 2026/09/23 04:14:01 INFO Signed narinfos id=78 count=17440worker1 # [ 165.473264] nix-grpc-daemon[1826]: time=2026-09-23T04:14:01.558Z level=INFO msg="Uploading 1 narinfos"7441lb # [ 165.504262] niks3-server[1119]: 2026/09/23 04:14:01 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls7442lb # [ 165.514279] niks3-server[1119]: 2026/09/23 04:14:01 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst7443lb # [ 165.521699] niks3-server[1119]: 2026/09/23 04:14:01 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete7444lb # [ 165.528318] niks3-server[1119]: 2026/09/23 04:14:01 INFO Completed upload id=787445worker1 # [ 165.502079] nix-grpc-daemon[1826]: time=2026-09-23T04:14:01.587Z level=INFO msg="Upload complete. (171ms)"7446lb # [ 165.533822] niks3-server[1119]: 2026/09/23 04:14:01 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo7447worker1 # [ 165.506085] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52436 duration_s=0 paths=1 nar_bytes_in=6247448worker1 # [ 165.510994] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524367449worker1 # [ 165.520502] nix-daemon[846]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7450worker1 # [ 165.525761] nix-daemon[846]: reaped child process 2025, status = succeeded7451worker1 # [ 165.530127] nix-daemon[846]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7452lb # [ 165.896122] envoy[914]: conn peer=[2001:db8:1::1]:50586 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9729 tx=5209 ms=61997453worker1 # [ 166.766649] nix-daemon[846]: reaped child process 2034, status = succeeded7454worker1 # [ 166.851552] nix-daemon[846]: accepted connection from pid 2050, user nix-grpc-daemon (trusted)7455worker1 # [ 166.864038] nix-daemon[846]: reaped child process 2053, status = succeeded7456lb # [ 166.902899] niks3-server[1119]: 2026/09/23 04:14:03 INFO Received uploads request method=POST path=/api/pending_closures7457worker1 # [ 166.882582] nix-grpc-daemon[1826]: time=2026-09-23T04:14:02.968Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7458worker1 # [ 166.884085] nix-grpc-daemon[1826]: time=2026-09-23T04:14:02.969Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"7459lb # [ 166.945148] niks3-server[1119]: 2026/09/23 04:14:03 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst7460lb # [ 166.950711] niks3-server[1119]: 2026/09/23 04:14:03 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv7461lb # [ 166.958208] niks3-server[1119]: 2026/09/23 04:14:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign7462lb # [ 166.962747] niks3-server[1119]: 2026/09/23 04:14:03 INFO Signed narinfos id=79 count=17463worker1 # [ 166.936364] nix-grpc-daemon[1826]: time=2026-09-23T04:14:03.021Z level=INFO msg="Uploading 1 narinfos"7464lb # [ 166.976258] niks3-server[1119]: 2026/09/23 04:14:03 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls7465lb # [ 166.984521] niks3-server[1119]: 2026/09/23 04:14:03 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete7466lb # [ 166.988784] niks3-server[1119]: 2026/09/23 04:14:03 INFO Completed upload id=797467worker1 # [ 166.959590] nix-grpc-daemon[1826]: time=2026-09-23T04:14:03.045Z level=INFO msg="Upload complete. (221ms)"7468lb # [ 166.990787] niks3-server[1119]: 2026/09/23 04:14:03 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo7469worker1 # [ 166.967770] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52436 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=9 outputs=17470worker1 # [ 166.977722] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:03Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=17471worker1 # [ 166.983398] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524367472worker1 # [ 166.987455] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:03Z level=debug event=want client=2 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=07473worker1 # [ 166.997545] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524367474worker1 # [ 167.003695] nix-daemon[846]: reaped child process 2032, status = succeeded7475worker1 # [ 167.009531] nix-grpc-daemon[1826]: time=2026-09-23T04:14:03.095Z level=INFO msg="All 1 paths already cached"7476worker1 # [ 167.014244] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524367477worker1 # [ 167.019099] nix-daemon[846]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7478worker1 # [ 167.104737] nix-daemon[846]: accepted connection from pid 2059, user nix-grpc-daemon (trusted)7479worker1 # [ 167.118410] nix-daemon[846]: reaped child process 2062, status = succeeded7480lb # [ 167.154208] niks3-server[1119]: 2026/09/23 04:14:03 INFO Received uploads request method=POST path=/api/pending_closures7481worker1 # [ 167.132640] nix-grpc-daemon[1826]: time=2026-09-23T04:14:03.218Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7482worker1 # [ 167.134610] nix-grpc-daemon[1826]: time=2026-09-23T04:14:03.219Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"7483lb # [ 167.185755] niks3-server[1119]: 2026/09/23 04:14:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign7484worker1 # [ 167.162095] nix-grpc-daemon[1826]: time=2026-09-23T04:14:03.247Z level=INFO msg="Uploading 1 narinfos"7485lb # [ 167.193594] niks3-server[1119]: 2026/09/23 04:14:03 INFO Signed narinfos id=80 count=17486lb # [ 167.201126] niks3-server[1119]: 2026/09/23 04:14:03 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls7487lb # [ 167.207798] niks3-server[1119]: 2026/09/23 04:14:03 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst7488lb # [ 167.218066] niks3-server[1119]: 2026/09/23 04:14:03 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete7489lb # [ 167.223688] niks3-server[1119]: 2026/09/23 04:14:03 INFO Completed upload id=807490worker1 # [ 167.194408] nix-grpc-daemon[1826]: time=2026-09-23T04:14:03.280Z level=INFO msg="Upload complete. (170ms)"7491lb # [ 167.226547] niks3-server[1119]: 2026/09/23 04:14:03 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo7492worker1 # [ 167.198397] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52436 duration_s=0 paths=1 nar_bytes_in=6247493worker1 # [ 167.203277] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524367494worker1 # [ 167.213460] nix-daemon[846]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7495worker1 # [ 167.220646] nix-daemon[846]: reaped child process 2057, status = succeeded7496worker1 # [ 167.226192] nix-daemon[846]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7497worker1 # [ 168.467386] nix-daemon[846]: reaped child process 2066, status = succeeded7498worker1 # [ 168.545307] nix-daemon[846]: accepted connection from pid 2082, user nix-grpc-daemon (trusted)7499worker1 # [ 168.556510] nix-daemon[846]: reaped child process 2085, status = succeeded7500lb # [ 168.595772] niks3-server[1119]: 2026/09/23 04:14:04 INFO Received uploads request method=POST path=/api/pending_closures7501worker1 # [ 168.574707] nix-grpc-daemon[1826]: time=2026-09-23T04:14:04.660Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7502worker1 # [ 168.576202] nix-grpc-daemon[1826]: time=2026-09-23T04:14:04.661Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"7503lb # [ 168.639937] niks3-server[1119]: 2026/09/23 04:14:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign7504lb # [ 168.645632] niks3-server[1119]: 2026/09/23 04:14:04 INFO Signed narinfos id=81 count=17505worker1 # [ 168.619375] nix-grpc-daemon[1826]: time=2026-09-23T04:14:04.705Z level=INFO msg="Uploading 1 narinfos"7506lb # [ 168.651770] niks3-server[1119]: 2026/09/23 04:14:04 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls7507lb # [ 168.655385] niks3-server[1119]: 2026/09/23 04:14:04 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst7508lb # [ 168.665629] niks3-server[1119]: 2026/09/23 04:14:04 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv7509lb # [ 168.674454] niks3-server[1119]: 2026/09/23 04:14:04 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete7510lb # [ 168.679119] niks3-server[1119]: 2026/09/23 04:14:04 INFO Completed upload id=817511worker1 # [ 168.651337] nix-grpc-daemon[1826]: time=2026-09-23T04:14:04.737Z level=INFO msg="Upload complete. (218ms)"7512lb # [ 168.683699] niks3-server[1119]: 2026/09/23 04:14:04 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo7513worker1 # [ 168.663681] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52436 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=10 outputs=17514worker1 # [ 168.670923] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:04Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=17515worker1 # [ 168.676449] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524367516worker1 # [ 168.679602] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:04Z level=debug event=want client=2 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=07517worker1 # [ 168.690350] nix-daemon[846]: reaped child process 2064, status = succeeded7518worker1 # [ 168.692380] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524367519worker1 # [ 168.700176] nix-grpc-daemon[1826]: time=2026-09-23T04:14:04.785Z level=INFO msg="All 2 paths already cached"7520worker1 # [ 168.705229] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524367521worker1 # [ 168.710073] nix-daemon[846]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7522worker1 # [ 168.795470] nix-daemon[846]: accepted connection from pid 2091, user nix-grpc-daemon (trusted)7523worker1 # [ 168.808694] nix-daemon[846]: reaped child process 2094, status = succeeded7524lb # [ 168.846443] niks3-server[1119]: 2026/09/23 04:14:05 INFO Received uploads request method=POST path=/api/pending_closures7525worker1 # [ 168.825387] nix-grpc-daemon[1826]: time=2026-09-23T04:14:04.911Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7526worker1 # [ 168.827397] nix-grpc-daemon[1826]: time=2026-09-23T04:14:04.912Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"7527lb # [ 168.876988] niks3-server[1119]: 2026/09/23 04:14:05 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst7528lb # [ 168.886177] niks3-server[1119]: 2026/09/23 04:14:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign7529lb # [ 168.890677] niks3-server[1119]: 2026/09/23 04:14:05 INFO Signed narinfos id=82 count=17530worker1 # [ 168.864509] nix-grpc-daemon[1826]: time=2026-09-23T04:14:04.950Z level=INFO msg="Uploading 1 narinfos"7531lb # [ 168.903209] niks3-server[1119]: 2026/09/23 04:14:05 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls7532lb # [ 168.911672] niks3-server[1119]: 2026/09/23 04:14:05 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete7533lb # [ 168.917785] niks3-server[1119]: 2026/09/23 04:14:05 INFO Completed upload id=827534worker1 # [ 168.888646] nix-grpc-daemon[1826]: time=2026-09-23T04:14:04.974Z level=INFO msg="Upload complete. (173ms)"7535lb # [ 168.920511] niks3-server[1119]: 2026/09/23 04:14:05 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo7536worker1 # [ 168.892959] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52436 duration_s=0 paths=1 nar_bytes_in=7527537worker1 # [ 168.899104] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524367538worker1 # [ 168.907655] nix-daemon[846]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7539worker1 # [ 168.915205] nix-daemon[846]: reaped child process 2089, status = succeeded7540worker1 # [ 168.920179] nix-daemon[846]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7541worker1 # [ 170.157527] nix-daemon[846]: reaped child process 2098, status = succeeded7542worker1 # [ 170.237798] nix-daemon[846]: accepted connection from pid 2114, user nix-grpc-daemon (trusted)7543worker1 # [ 170.250233] nix-daemon[846]: reaped child process 2117, status = succeeded7544lb # [ 170.288088] niks3-server[1119]: 2026/09/23 04:14:06 INFO Received uploads request method=POST path=/api/pending_closures7545worker1 # [ 170.267052] nix-grpc-daemon[1826]: time=2026-09-23T04:14:06.352Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7546worker1 # [ 170.269749] nix-grpc-daemon[1826]: time=2026-09-23T04:14:06.354Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"7547lb # [ 170.325359] niks3-server[1119]: 2026/09/23 04:14:06 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv7548lb # [ 170.335968] niks3-server[1119]: 2026/09/23 04:14:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign7549worker1 # [ 170.312314] nix-grpc-daemon[1826]: time=2026-09-23T04:14:06.398Z level=INFO msg="Uploading 1 narinfos"7550lb # [ 170.342414] niks3-server[1119]: 2026/09/23 04:14:06 INFO Signed narinfos id=83 count=17551lb # [ 170.349263] niks3-server[1119]: 2026/09/23 04:14:06 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst7552lb # [ 170.356521] niks3-server[1119]: 2026/09/23 04:14:06 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls7553lb # [ 170.365798] niks3-server[1119]: 2026/09/23 04:14:06 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete7554lb # [ 170.368723] niks3-server[1119]: 2026/09/23 04:14:06 INFO Completed upload id=837555worker1 # [ 170.339505] nix-grpc-daemon[1826]: time=2026-09-23T04:14:06.425Z level=INFO msg="Upload complete. (211ms)"7556lb # [ 170.372599] niks3-server[1119]: 2026/09/23 04:14:06 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo7557worker1 # [ 170.344427] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52436 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=11 outputs=17558worker1 # [ 170.351475] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:06Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=17559worker1 # [ 170.356285] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524367560worker1 # [ 170.367370] nix-daemon[846]: reaped child process 2096, status = succeeded7561worker1 # [ 170.399587] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524367562worker1 # [ 170.413116] nix-grpc-daemon[1826]: time=2026-09-23T04:14:06.498Z level=INFO msg="All 2 paths already cached"7563worker1 # [ 170.421062] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524367564worker1 # [ 170.431157] nix-daemon[846]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7565worker1 # [ 170.527871] nix-daemon[846]: accepted connection from pid 2121, user nix-grpc-daemon (trusted)7566worker1 # [ 170.540280] nix-daemon[846]: reaped child process 2124, status = succeeded7567lb # [ 170.578786] niks3-server[1119]: 2026/09/23 04:14:06 INFO Received uploads request method=POST path=/api/pending_closures7568worker1 # [ 170.557136] nix-grpc-daemon[1826]: time=2026-09-23T04:14:06.642Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7569worker1 # [ 170.558607] nix-grpc-daemon[1826]: time=2026-09-23T04:14:06.644Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"7570lb # [ 170.611442] niks3-server[1119]: 2026/09/23 04:14:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign7571worker1 # [ 170.589153] nix-grpc-daemon[1826]: time=2026-09-23T04:14:06.674Z level=INFO msg="Uploading 1 narinfos"7572lb # [ 170.619191] niks3-server[1119]: 2026/09/23 04:14:06 INFO Signed narinfos id=84 count=17573lb # [ 170.622953] niks3-server[1119]: 2026/09/23 04:14:06 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst7574lb # [ 170.630439] niks3-server[1119]: 2026/09/23 04:14:06 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls7575lb # [ 170.643539] niks3-server[1119]: 2026/09/23 04:14:06 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete7576lb # [ 170.650923] niks3-server[1119]: 2026/09/23 04:14:06 INFO Completed upload id=847577worker1 # [ 170.620803] nix-grpc-daemon[1826]: time=2026-09-23T04:14:06.706Z level=INFO msg="Upload complete. (179ms)"7578lb # [ 170.653029] niks3-server[1119]: 2026/09/23 04:14:06 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo7579worker1 # [ 170.625722] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52436 duration_s=0 paths=1 nar_bytes_in=7527580worker1 # [ 170.632390] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524367581worker1 # [ 170.642198] nix-daemon[846]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7582worker1 # [ 170.650383] nix-daemon[846]: reaped child process 2119, status = succeeded7583worker1 # [ 170.655218] nix-daemon[846]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7584worker1 # [ 171.691950] nix-daemon[846]: reaped child process 2128, status = succeeded7585worker1 # [ 171.775688] nix-daemon[846]: accepted connection from pid 2144, user nix-grpc-daemon (trusted)7586worker1 # [ 171.788078] nix-daemon[846]: reaped child process 2147, status = succeeded7587lb # [ 171.824861] niks3-server[1119]: 2026/09/23 04:14:08 INFO Received uploads request method=POST path=/api/pending_closures7588worker1 # [ 171.806391] nix-grpc-daemon[1826]: time=2026-09-23T04:14:07.891Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7589worker1 # [ 171.809058] nix-grpc-daemon[1826]: time=2026-09-23T04:14:07.893Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"7590lb # [ 171.866815] niks3-server[1119]: 2026/09/23 04:14:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign7591lb # [ 171.877116] niks3-server[1119]: 2026/09/23 04:14:08 INFO Signed narinfos id=85 count=17592worker1 # [ 171.851332] nix-grpc-daemon[1826]: time=2026-09-23T04:14:07.936Z level=INFO msg="Uploading 1 narinfos"7593lb # [ 171.882338] niks3-server[1119]: 2026/09/23 04:14:08 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst7594lb # [ 171.889891] niks3-server[1119]: 2026/09/23 04:14:08 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls7595lb # [ 171.893792] niks3-server[1119]: 2026/09/23 04:14:08 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv7596lb # [ 171.905711] niks3-server[1119]: 2026/09/23 04:14:08 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete7597lb # [ 171.908056] niks3-server[1119]: 2026/09/23 04:14:08 INFO Completed upload id=857598worker1 # [ 171.878884] nix-grpc-daemon[1826]: time=2026-09-23T04:14:07.964Z level=INFO msg="Upload complete. (217ms)"7599lb # [ 171.910781] niks3-server[1119]: 2026/09/23 04:14:08 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo7600worker1 # [ 171.887476] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52436 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=12 outputs=17601worker1 # [ 171.894290] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:07Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=17602worker1 # [ 171.905656] nix-daemon[846]: reaped child process 2126, status = succeeded7603worker1 # [ 171.929765] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524367604worker1 # [ 171.941960] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524367605worker1 # [ 171.954253] nix-grpc-daemon[1826]: time=2026-09-23T04:14:08.039Z level=INFO msg="All 2 paths already cached"7606worker1 # [ 171.962537] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524367607worker1 # [ 171.972158] nix-daemon[846]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7608worker1 # [ 172.069096] nix-daemon[846]: accepted connection from pid 2151, user nix-grpc-daemon (trusted)7609worker1 # [ 172.082653] nix-daemon[846]: reaped child process 2154, status = succeeded7610lb # [ 172.119095] niks3-server[1119]: 2026/09/23 04:14:08 INFO Received uploads request method=POST path=/api/pending_closures7611worker1 # [ 172.098429] nix-grpc-daemon[1826]: time=2026-09-23T04:14:08.184Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7612worker1 # [ 172.099925] nix-grpc-daemon[1826]: time=2026-09-23T04:14:08.185Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"7613lb # [ 172.152274] niks3-server[1119]: 2026/09/23 04:14:08 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls7614lb # [ 172.155304] niks3-server[1119]: 2026/09/23 04:14:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/86/sign7615worker1 # [ 172.133198] nix-grpc-daemon[1826]: time=2026-09-23T04:14:08.218Z level=INFO msg="Uploading 1 narinfos"7616lb # [ 172.162113] niks3-server[1119]: 2026/09/23 04:14:08 INFO Signed narinfos id=86 count=17617lb # [ 172.168492] niks3-server[1119]: 2026/09/23 04:14:08 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst7618lb # [ 172.179423] niks3-server[1119]: 2026/09/23 04:14:08 INFO Received complete upload request method=POST path=/api/pending_closures/86/complete7619lb # [ 172.182725] niks3-server[1119]: 2026/09/23 04:14:08 INFO Completed upload id=867620worker1 # [ 172.154604] nix-grpc-daemon[1826]: time=2026-09-23T04:14:08.240Z level=INFO msg="Upload complete. (172ms)"7621lb # [ 172.184579] niks3-server[1119]: 2026/09/23 04:14:08 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo7622worker1 # [ 172.156982] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52436 duration_s=0 paths=1 nar_bytes_in=7527623worker1 # [ 172.161700] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524367624worker1 # [ 172.171518] nix-daemon[846]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7625worker1 # [ 172.176653] nix-daemon[846]: reaped child process 2149, status = succeeded7626worker1 # [ 172.179694] nix-daemon[846]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7627worker1 # [ 173.415438] nix-daemon[846]: reaped child process 2158, status = succeeded7628worker1 # [ 173.501754] nix-daemon[846]: accepted connection from pid 2174, user nix-grpc-daemon (trusted)7629worker1 # [ 173.513150] nix-daemon[846]: reaped child process 2177, status = succeeded7630lb # [ 173.551836] niks3-server[1119]: 2026/09/23 04:14:09 INFO Received uploads request method=POST path=/api/pending_closures7631worker1 # [ 173.532507] nix-grpc-daemon[1826]: time=2026-09-23T04:14:09.618Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7632worker1 # [ 173.534170] nix-grpc-daemon[1826]: time=2026-09-23T04:14:09.619Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"7633lb # [ 173.594359] niks3-server[1119]: 2026/09/23 04:14:09 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst7634lb # [ 173.599338] niks3-server[1119]: 2026/09/23 04:14:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/87/sign7635worker1 # [ 173.575602] nix-grpc-daemon[1826]: time=2026-09-23T04:14:09.661Z level=INFO msg="Uploading 1 narinfos"7636lb # [ 173.605348] niks3-server[1119]: 2026/09/23 04:14:09 INFO Signed narinfos id=87 count=17637lb # [ 173.607499] niks3-server[1119]: 2026/09/23 04:14:09 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv7638lb # [ 173.613812] niks3-server[1119]: 2026/09/23 04:14:09 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls7639lb # [ 173.618311] niks3-server[1119]: 2026/09/23 04:14:09 INFO Received complete upload request method=POST path=/api/pending_closures/87/complete7640lb # [ 173.623037] niks3-server[1119]: 2026/09/23 04:14:09 INFO Completed upload id=877641worker1 # [ 173.593724] nix-grpc-daemon[1826]: time=2026-09-23T04:14:09.679Z level=INFO msg="Upload complete. (206ms)"7642lb # [ 173.624891] niks3-server[1119]: 2026/09/23 04:14:09 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo7643worker1 # [ 173.601404] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52436 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=13 outputs=17644worker1 # [ 173.613418] nix-daemon[846]: reaped child process 2156, status = succeeded7645client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 14.69 seconds)7646worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7647worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7648client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]'7649client # [ 173.720771] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7650client # Running as unit: gpu.service; invocation ID: 24ae6ed435b64f3d9091313a43b8c1917651client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.06 seconds)7652lb: must succeed: curl -sf localhost:9901/clusters7653lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7654lb: must succeed: curl -sf localhost:9901/clusters7655lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7656worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7657worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)7658worker2 # [ 173.888484] nix-grpc-daemon[1595]: ts=2026-09-23T04:14:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:437847659client # [ 173.869907] nix[1589]: don't know how to build these paths:7660client # [ 173.871166] nix[1589]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv7661worker2 # [ 173.899491] nix-grpc-daemon[1595]: ts=2026-09-23T04:14:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:437847662worker1 # [ 173.905885] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:09Z level=debug event=want client=3 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=07663client # [ 173.889924] nix[1589]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}, waiting up to 30s for a worker7664lb: must succeed: curl -sf localhost:9901/clusters7665lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7666lb: must succeed: curl -sf localhost:9901/clusters7667lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7668worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7669worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)7670client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7671lb # [ 175.896473] envoy[914]: conn peer=[2001:db8:1::1]:37846 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27146 tx=12270 ms=145037672client # [ 203.892923] nix[1589]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}7673client # [ 203.900705] nix[1589]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed7674client # [ 203.934956] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7675client # [ 203.939523] systemd[1]: gpu.service: Failed with result 'exit-code'.7676client # [ 203.943290] systemd[1]: gpu.service: Consumed 167ms CPU time over 30.218s wall clock time, 35.6M memory peak, 5.6K incoming IP traffic, 5.2K outgoing IP traffic.7677client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.47 seconds)7678client: must succeed: journalctl -u gpu -o cat7679client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds)7680lb: must succeed: curl -sf localhost:9901/clusters7681lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7682lb: must succeed: curl -sf localhost:9901/clusters7683lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7684worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7685worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7686(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 45.60 seconds)7687subtest: low disk drains a worker and builds go to the other7688worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7689worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.91 seconds)7690worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7691lb # [ 205.899335] envoy[914]: conn peer=[2001:db8:1::1]:42246 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=929 tx=596 ms=300497692worker1 # [ 206.314236] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:42Z level=info event=unhealthy reason=min_free7693worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.09 seconds)7694client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&27695client # don't know how to build these paths:7696client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7697worker1 # [ 206.726811] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:42Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:524367698worker1 # [ 206.733129] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524367699worker1 # [ 206.754234] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:42Z level=debug event=want client=4 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=07700worker1 # [ 206.758913] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:42Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=17701worker2 # [ 206.809284] nix-grpc-daemon[1595]: ts=2026-09-23T04:14:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437847702worker2 # [ 206.820920] nix-grpc-daemon[1595]: ts=2026-09-23T04:14:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:437847703client # copying 1 paths to 'lb:50051'...7704client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...7705worker2 # [ 206.833889] nix-grpc-daemon[1595]: ts=2026-09-23T04:14:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:437847706worker2 # [ 206.845227] nix-daemon[870]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)7707worker2 # [ 206.948120] nix-daemon[870]: accepted connection from pid 1950, user nix-grpc-daemon (trusted)7708worker2 # [ 206.961804] nix-daemon[870]: reaped child process 1953, status = succeeded7709lb # [ 206.990662] niks3-server[1119]: 2026/09/23 04:14:43 INFO Received uploads request method=POST path=/api/pending_closures7710worker2 # [ 206.977404] nix-grpc-daemon[1668]: time=2026-09-23T04:14:43.103Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7711worker2 # [ 206.979336] nix-grpc-daemon[1668]: time=2026-09-23T04:14:43.105Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"7712lb # [ 207.019133] niks3-server[1119]: 2026/09/23 04:14:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/88/sign7713worker2 # [ 207.004571] nix-grpc-daemon[1668]: time=2026-09-23T04:14:43.130Z level=INFO msg="Uploading 1 narinfos"7714lb # [ 207.026167] niks3-server[1119]: 2026/09/23 04:14:43 INFO Signed narinfos id=88 count=17715lb # [ 207.035058] niks3-server[1119]: 2026/09/23 04:14:43 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls7716lb # [ 207.041063] niks3-server[1119]: 2026/09/23 04:14:43 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst7717lb # [ 207.050583] niks3-server[1119]: 2026/09/23 04:14:43 INFO Received complete upload request method=POST path=/api/pending_closures/88/complete7718lb # [ 207.056835] niks3-server[1119]: 2026/09/23 04:14:43 INFO Completed upload id=887719worker2 # [ 207.037770] nix-grpc-daemon[1668]: time=2026-09-23T04:14:43.164Z level=INFO msg="Upload complete. (180ms)"7720lb # [ 207.061761] niks3-server[1119]: 2026/09/23 04:14:43 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo7721worker2 # [ 207.041968] nix-grpc-daemon[1595]: ts=2026-09-23T04:14:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43784 duration_s=0 paths=1 nar_bytes_in=5127722worker2 # [ 207.047390] nix-grpc-daemon[1595]: ts=2026-09-23T04:14:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437847723worker2 # [ 207.057499] nix-daemon[870]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)7724worker2 # [ 207.059513] nix-daemon[870]: reaped child process 1948, status = succeeded7725worker2 # [ 207.064434] nix-daemon[870]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)7726client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...7727client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7728client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7729client # farm-a> LOG-farm-a-drain7730worker2 # [ 208.299504] nix-daemon[870]: reaped child process 1957, status = succeeded7731worker2 # [ 208.386891] nix-daemon[870]: accepted connection from pid 1973, user nix-grpc-daemon (trusted)7732worker2 # [ 208.399912] nix-daemon[870]: reaped child process 1976, status = succeeded7733lb # [ 208.429783] niks3-server[1119]: 2026/09/23 04:14:44 INFO Received uploads request method=POST path=/api/pending_closures7734worker2 # [ 208.420211] nix-grpc-daemon[1668]: time=2026-09-23T04:14:44.546Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7735worker2 # [ 208.422622] nix-grpc-daemon[1668]: time=2026-09-23T04:14:44.548Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"7736lb # [ 208.474257] niks3-server[1119]: 2026/09/23 04:14:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/89/sign7737lb # [ 208.485124] niks3-server[1119]: 2026/09/23 04:14:44 INFO Signed narinfos id=89 count=17738worker2 # [ 208.468267] nix-grpc-daemon[1668]: time=2026-09-23T04:14:44.594Z level=INFO msg="Uploading 1 narinfos"7739lb # [ 208.493111] niks3-server[1119]: 2026/09/23 04:14:44 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls7740lb # [ 208.497203] niks3-server[1119]: 2026/09/23 04:14:44 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7741lb # [ 208.500977] niks3-server[1119]: 2026/09/23 04:14:44 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7742lb # [ 208.513047] niks3-server[1119]: 2026/09/23 04:14:44 INFO Received complete upload request method=POST path=/api/pending_closures/89/complete7743lb # [ 208.516306] niks3-server[1119]: 2026/09/23 04:14:44 INFO Completed upload id=897744worker2 # [ 208.494840] nix-grpc-daemon[1668]: time=2026-09-23T04:14:44.621Z level=INFO msg="Upload complete. (222ms)"7745lb # [ 208.518745] niks3-server[1119]: 2026/09/23 04:14:44 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo7746worker2 # [ 208.500640] nix-grpc-daemon[1595]: ts=2026-09-23T04:14:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43784 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=14 outputs=17747worker2 # [ 208.514826] nix-daemon[870]: reaped child process 1955, status = succeeded7748worker1 # [ 208.541791] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:44Z level=debug event=want client=4 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=07749worker1 # [ 208.546539] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:44Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=17750worker2 # [ 208.593882] nix-grpc-daemon[1595]: ts=2026-09-23T04:14:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437847751worker2 # [ 208.606696] nix-grpc-daemon[1595]: ts=2026-09-23T04:14:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:437847752worker2 # [ 208.619618] nix-grpc-daemon[1668]: time=2026-09-23T04:14:44.745Z level=INFO msg="All 1 paths already cached"7753client # copying 1 paths to 'lb:50051'...7754client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...7755worker2 # [ 208.628121] nix-grpc-daemon[1595]: ts=2026-09-23T04:14:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:437847756worker2 # [ 208.636561] nix-daemon[870]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)7757worker2 # [ 208.744382] nix-daemon[870]: accepted connection from pid 1980, user nix-grpc-daemon (trusted)7758worker2 # [ 208.759107] nix-daemon[870]: reaped child process 1983, status = succeeded7759lb # [ 208.787123] niks3-server[1119]: 2026/09/23 04:14:44 INFO Received uploads request method=POST path=/api/pending_closures7760worker2 # [ 208.773289] nix-grpc-daemon[1668]: time=2026-09-23T04:14:44.899Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7761worker2 # [ 208.774862] nix-grpc-daemon[1668]: time=2026-09-23T04:14:44.901Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"7762lb # [ 208.818684] niks3-server[1119]: 2026/09/23 04:14:45 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls7763lb # [ 208.826120] niks3-server[1119]: 2026/09/23 04:14:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/90/sign7764lb # [ 208.832631] niks3-server[1119]: 2026/09/23 04:14:45 INFO Signed narinfos id=90 count=17765worker2 # [ 208.813923] nix-grpc-daemon[1668]: time=2026-09-23T04:14:44.940Z level=INFO msg="Uploading 1 narinfos"7766lb # [ 208.839255] niks3-server[1119]: 2026/09/23 04:14:45 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst7767lb # [ 208.853868] niks3-server[1119]: 2026/09/23 04:14:45 INFO Received complete upload request method=POST path=/api/pending_closures/90/complete7768lb # [ 208.859631] niks3-server[1119]: 2026/09/23 04:14:45 INFO Completed upload id=907769worker2 # [ 208.838098] nix-grpc-daemon[1668]: time=2026-09-23T04:14:44.964Z level=INFO msg="Upload complete. (187ms)"7770lb # [ 208.862763] niks3-server[1119]: 2026/09/23 04:14:45 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo7771worker2 # [ 208.842304] nix-grpc-daemon[1595]: ts=2026-09-23T04:14:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43784 duration_s=0 paths=1 nar_bytes_in=6247772worker2 # [ 208.848429] nix-grpc-daemon[1595]: ts=2026-09-23T04:14:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437847773worker2 # [ 208.857723] nix-daemon[870]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)7774worker2 # [ 208.867125] nix-daemon[870]: reaped child process 1978, status = succeeded7775worker2 # [ 208.871858] nix-daemon[870]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)7776client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...7777client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7778client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7779client # farm-b> LOG-farm-b-drain7780worker2 # [ 209.709562] nix-daemon[870]: reaped child process 1987, status = succeeded7781worker2 # [ 209.791417] nix-daemon[870]: accepted connection from pid 2003, user nix-grpc-daemon (trusted)7782worker2 # [ 209.803466] nix-daemon[870]: reaped child process 2006, status = succeeded7783lb # [ 209.835505] niks3-server[1119]: 2026/09/23 04:14:46 INFO Received uploads request method=POST path=/api/pending_closures7784worker2 # [ 209.821965] nix-grpc-daemon[1668]: time=2026-09-23T04:14:45.948Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7785worker2 # [ 209.823471] nix-grpc-daemon[1668]: time=2026-09-23T04:14:45.949Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"7786lb # [ 209.876388] niks3-server[1119]: 2026/09/23 04:14:46 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls7787lb # [ 209.881491] niks3-server[1119]: 2026/09/23 04:14:46 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst7788lb # [ 209.888818] niks3-server[1119]: 2026/09/23 04:14:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/91/sign7789worker2 # [ 209.873098] nix-grpc-daemon[1668]: time=2026-09-23T04:14:45.999Z level=INFO msg="Uploading 1 narinfos"7790lb # [ 209.894284] niks3-server[1119]: 2026/09/23 04:14:46 INFO Signed narinfos id=91 count=17791lb # [ 209.904308] niks3-server[1119]: 2026/09/23 04:14:46 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7792lb # [ 209.912979] niks3-server[1119]: 2026/09/23 04:14:46 INFO Received complete upload request method=POST path=/api/pending_closures/91/complete7793lb # [ 209.918860] niks3-server[1119]: 2026/09/23 04:14:46 INFO Completed upload id=917794worker2 # [ 209.897545] nix-grpc-daemon[1668]: time=2026-09-23T04:14:46.023Z level=INFO msg="Upload complete. (219ms)"7795lb # [ 209.921638] niks3-server[1119]: 2026/09/23 04:14:46 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo7796worker2 # [ 209.905159] nix-grpc-daemon[1595]: ts=2026-09-23T04:14:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43784 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=15 outputs=17797worker1 # [ 209.903783] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:45Z level=debug event=want client=4 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=07798worker2 # [ 209.913787] nix-grpc-daemon[1595]: ts=2026-09-23T04:14:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437847799worker1 # [ 209.908182] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:45Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=17800worker2 # [ 209.924561] nix-grpc-daemon[1595]: ts=2026-09-23T04:14:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:437847801worker2 # [ 209.932441] nix-daemon[870]: reaped child process 1985, status = succeeded7802client # copying 1 paths to 'lb:50051'...7803worker2 # [ 209.935796] nix-grpc-daemon[1668]: time=2026-09-23T04:14:46.060Z level=INFO msg="All 2 paths already cached"7804client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...7805worker2 # [ 209.939518] nix-grpc-daemon[1595]: ts=2026-09-23T04:14:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:437847806worker2 # [ 209.945180] nix-daemon[870]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)7807worker2 # [ 210.032754] nix-daemon[870]: accepted connection from pid 2010, user nix-grpc-daemon (trusted)7808worker2 # [ 210.045845] nix-daemon[870]: reaped child process 2013, status = succeeded7809lb # [ 210.075152] niks3-server[1119]: 2026/09/23 04:14:46 INFO Received uploads request method=POST path=/api/pending_closures7810worker2 # [ 210.062028] nix-grpc-daemon[1668]: time=2026-09-23T04:14:46.188Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7811worker2 # [ 210.063945] nix-grpc-daemon[1668]: time=2026-09-23T04:14:46.189Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"7812lb # [ 210.103970] niks3-server[1119]: 2026/09/23 04:14:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/92/sign7813worker2 # [ 210.087188] nix-grpc-daemon[1668]: time=2026-09-23T04:14:46.213Z level=INFO msg="Uploading 1 narinfos"7814lb # [ 210.108530] niks3-server[1119]: 2026/09/23 04:14:46 INFO Signed narinfos id=92 count=17815lb # [ 210.114556] niks3-server[1119]: 2026/09/23 04:14:46 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst7816lb # [ 210.123527] niks3-server[1119]: 2026/09/23 04:14:46 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls7817lb # [ 210.134193] niks3-server[1119]: 2026/09/23 04:14:46 INFO Received complete upload request method=POST path=/api/pending_closures/92/complete7818worker2 # [ 210.118329] nix-grpc-daemon[1668]: time=2026-09-23T04:14:46.244Z level=INFO msg="Upload complete. (167ms)"7819lb # [ 210.140213] niks3-server[1119]: 2026/09/23 04:14:46 INFO Completed upload id=927820worker2 # [ 210.122439] nix-grpc-daemon[1595]: ts=2026-09-23T04:14:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43784 duration_s=0 paths=1 nar_bytes_in=7527821lb # [ 210.145149] niks3-server[1119]: 2026/09/23 04:14:46 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo7822worker2 # [ 210.128219] nix-grpc-daemon[1595]: ts=2026-09-23T04:14:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437847823worker2 # [ 210.138324] nix-daemon[870]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)7824worker2 # [ 210.142100] nix-daemon[870]: reaped child process 2008, status = succeeded7825worker2 # [ 210.145867] nix-daemon[870]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)7826client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...7827client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7828client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7829client # farm-top> LOG-farm-top-drain7830worker2 # [ 210.780631] nix-daemon[870]: reaped child process 2017, status = succeeded7831worker2 # [ 210.868087] nix-daemon[870]: accepted connection from pid 2033, user nix-grpc-daemon (trusted)7832worker2 # [ 210.881283] nix-daemon[870]: reaped child process 2036, status = succeeded7833lb # [ 210.911516] niks3-server[1119]: 2026/09/23 04:14:47 INFO Received uploads request method=POST path=/api/pending_closures7834worker2 # [ 210.900581] nix-grpc-daemon[1668]: time=2026-09-23T04:14:47.026Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7835worker2 # [ 210.902145] nix-grpc-daemon[1668]: time=2026-09-23T04:14:47.028Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"7836lb # [ 210.957737] niks3-server[1119]: 2026/09/23 04:14:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/93/sign7837lb # [ 210.969992] niks3-server[1119]: 2026/09/23 04:14:47 INFO Signed narinfos id=93 count=17838worker2 # [ 210.951959] nix-grpc-daemon[1668]: time=2026-09-23T04:14:47.078Z level=INFO msg="Uploading 1 narinfos"7839lb # [ 210.979172] niks3-server[1119]: 2026/09/23 04:14:47 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7840lb # [ 210.986850] niks3-server[1119]: 2026/09/23 04:14:47 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7841lb # [ 210.990824] niks3-server[1119]: 2026/09/23 04:14:47 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7842lb # [ 211.002623] niks3-server[1119]: 2026/09/23 04:14:47 INFO Received complete upload request method=POST path=/api/pending_closures/93/complete7843lb # [ 211.009686] niks3-server[1119]: 2026/09/23 04:14:47 INFO Completed upload id=937844lb # [ 211.012838] niks3-server[1119]: 2026/09/23 04:14:47 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7845worker2 # [ 210.989801] nix-grpc-daemon[1668]: time=2026-09-23T04:14:47.116Z level=INFO msg="Upload complete. (237ms)"7846worker2 # [ 210.995939] nix-grpc-daemon[1595]: ts=2026-09-23T04:14:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43784 duration_s=0 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=16 outputs=17847worker2 # [ 211.011329] nix-daemon[870]: reaped child process 2015, status = succeeded7848client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&2, in 4.45 seconds)7849client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath7850client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath, in 0.12 seconds)7851worker1: must succeed: rm /nix/.rw-store/fill7852worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.12 seconds)7853worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7854worker1 # [ 212.317548] nix-grpc-daemon[1751]: ts=2026-09-23T04:14:48Z level=info event=healthy reason=min_free7855worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.09 seconds)7856(finished: subtest: low disk drains a worker and builds go to the other, in 7.85 seconds)7857(finished: run the VM test script, in 213.68 seconds)7858test script finished in 213.86s7859cleanup7860kill QemuMachine (pid 50)7861client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7862client # [2026-09-23T04:14:49Z INFO virtiofsd] Client disconnected, shutting down7863client # [2026-09-23T04:14:49Z INFO virtiofsd] Client disconnected, shutting down7864client # [2026-09-23T04:14:49Z INFO virtiofsd] Client disconnected, shutting down7865kill QemuMachine (pid 49)7866lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7867lb # [2026-09-23T04:14:49Z INFO virtiofsd] Client disconnected, shutting down7868lb # [2026-09-23T04:14:49Z INFO virtiofsd] Client disconnected, shutting down7869lb # [2026-09-23T04:14:49Z INFO virtiofsd] Client disconnected, shutting down7870kill QemuMachine (pid 51)7871worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7872worker1 # [2026-09-23T04:14:49Z INFO virtiofsd] Client disconnected, shutting down7873worker1 # [2026-09-23T04:14:49Z INFO virtiofsd] Client disconnected, shutting down7874worker1 # [2026-09-23T04:14:49Z INFO virtiofsd] Client disconnected, shutting down7875kill QemuMachine (pid 52)7876worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7877worker2 # [2026-09-23T04:14:49Z INFO virtiofsd] Client disconnected, shutting down7878worker2 # [2026-09-23T04:14:49Z INFO virtiofsd] Client disconnected, shutting down7879worker2 # [2026-09-23T04:14:49Z INFO virtiofsd] Client disconnected, shutting down7880(finished: cleanup, in 0.98 seconds)7881additionally exposed symbols:7882 client, lb, worker1, worker2,7883 vlan1,7884 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh