vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #476
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker2 # Disk image does not exist, creating the virtualisation disk image...13worker2: QEMU running (pid 52)14worker2 # Formatting '/build/vm-state-worker2/tmp.RbQxwjIkMG', fmt=raw size=107374182415worker2 # mke2fs 1.47.4 (6-Mar-2025)16client: QEMU running (pid 50)17worker2 # Discarding device blocks: 0/262144 done18client # Disk image does not exist, creating the virtualisation disk image...19worker1 # Disk image does not exist, creating the virtualisation disk image...20worker1: QEMU running (pid 51)21worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes22client # Formatting '/build/vm-state-client/tmp.O3cD3aVeRd', fmt=raw size=107374182423worker2 # Filesystem UUID: e1e42a58-9f38-4837-b42f-9fc35cac70cb24client # mke2fs 1.47.4 (6-Mar-2025)25lb: QEMU running (pid 49)26lb # Disk image does not exist, creating the virtualisation disk image...27client # Discarding device blocks: 0/262144 done28worker1 # Formatting '/build/vm-state-worker1/tmp.4r922dBfeR', fmt=raw size=107374182429worker2 # Superblock backups stored on blocks:30(finished: start all VMs, in 0.27 seconds)31worker1 # mke2fs 1.47.4 (6-Mar-2025)32worker2 # 32768, 98304, 163840, 22937633client: waiting for unit network-addresses-eth1.service34worker1 # Discarding device blocks: 0/262144 done35worker2 # 36client: waiting for the VM to finish booting37worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes38worker2 # Allocating group tables: 0/8 done39client # Creating filesystem with 262144 4k blocks and 65536 inodes40lb # Formatting '/build/vm-state-lb/tmp.1IlkwRqLfj', fmt=raw size=107374182441worker1 # Filesystem UUID: 700417bf-29b6-4ca5-b41f-45c0d097ed3042worker2 # Writing inode tables: 0/8 done43client # Filesystem UUID: 14543ea9-4997-4b63-bb74-261c080261ab44lb # mke2fs 1.47.4 (6-Mar-2025)45worker1 # Superblock backups stored on blocks:46worker2 # Creating journal (8192 blocks): done47client # Superblock backups stored on blocks:48lb # Discarding device blocks: 0/262144 done49worker1 # 32768, 98304, 163840, 22937650worker2 # Writing superblocks and filesystem accounting information: 0/8 done51client # 32768, 98304, 163840, 22937652lb # Creating filesystem with 262144 4k blocks and 65536 inodes53worker1 # 54worker2 # 55client # 56lb # Filesystem UUID: 963f18fa-c8a8-4f61-b20a-b59a7f6f833e57worker1 # Allocating group tables: 0/8 done58worker2 # Virtualisation disk image created.59client # Allocating group tables: 0/8 done60lb # Superblock backups stored on blocks:61worker1 # Writing inode tables: 0/8 done62worker2 # Starting virtiofs daemons...63client # Writing inode tables: 0/8 done64lb # 32768, 98304, 163840, 22937665worker1 # Creating journal (8192 blocks): done66worker2 # [2026-09-21T13:21:02Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)67client # Creating journal (8192 blocks): done68lb # 69worker1 # Writing superblocks and filesystem accounting information: 0/8 done70worker2 # [2026-09-21T13:21:02Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether71client # Writing superblocks and filesystem accounting information: 0/8 done72lb # Allocating group tables: 0/8 done73worker1 # 74worker2 # [2026-09-21T13:21:02Z INFO virtiofsd] Waiting for vhost-user socket connection...75client # 76lb # Writing inode tables: 0/8 done77worker1 # Virtualisation disk image created.78worker2 # [2026-09-21T13:21:02Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)79client # Virtualisation disk image created.80lb # Creating journal (8192 blocks): done81worker1 # Starting virtiofs daemons...82worker2 # [2026-09-21T13:21:02Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether83client # Starting virtiofs daemons...84lb # Writing superblocks and filesystem accounting information: 0/8 done85worker1 # [2026-09-21T13:21:02Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)86worker2 # [2026-09-21T13:21:02Z INFO virtiofsd] Waiting for vhost-user socket connection...87client # [2026-09-21T13:21:02Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)88lb # 89worker1 # [2026-09-21T13:21:02Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether90worker2 # [2026-09-21T13:21:02Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)91client # [2026-09-21T13:21:02Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether92lb # Virtualisation disk image created.93worker1 # [2026-09-21T13:21:02Z INFO virtiofsd] Waiting for vhost-user socket connection...94worker2 # [2026-09-21T13:21:02Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether95client # [2026-09-21T13:21:02Z INFO virtiofsd] Waiting for vhost-user socket connection...96lb # Starting virtiofs daemons...97worker1 # [2026-09-21T13:21:02Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)98worker2 # [2026-09-21T13:21:02Z INFO virtiofsd] Waiting for vhost-user socket connection...99client # [2026-09-21T13:21:02Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)100lb # [2026-09-21T13:21:02Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)101worker1 # [2026-09-21T13:21:02Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether102worker2 # [2026-09-21T13:21:02Z INFO virtiofsd] Client connected, servicing requests103client # [2026-09-21T13:21:02Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether104lb # [2026-09-21T13:21:02Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether105worker1 # [2026-09-21T13:21:02Z INFO virtiofsd] Waiting for vhost-user socket connection...106worker2 # [2026-09-21T13:21:02Z INFO virtiofsd] Client connected, servicing requests107client # [2026-09-21T13:21:02Z INFO virtiofsd] Waiting for vhost-user socket connection...108lb # [2026-09-21T13:21:02Z INFO virtiofsd] Waiting for vhost-user socket connection...109worker1 # [2026-09-21T13:21:02Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)110worker2 # [2026-09-21T13:21:02Z INFO virtiofsd] Client connected, servicing requests111client # [2026-09-21T13:21:02Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)112lb # [2026-09-21T13:21:02Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)113client # [2026-09-21T13:21:02Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether114lb # [2026-09-21T13:21:02Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether115client # [2026-09-21T13:21:02Z INFO virtiofsd] Waiting for vhost-user socket connection...116lb # [2026-09-21T13:21:02Z INFO virtiofsd] Waiting for vhost-user socket connection...117client # [2026-09-21T13:21:02Z INFO virtiofsd] Client connected, servicing requests118lb # [2026-09-21T13:21:02Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)119client # [2026-09-21T13:21:02Z INFO virtiofsd] Client connected, servicing requests120lb # [2026-09-21T13:21:02Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether121client # [2026-09-21T13:21:02Z INFO virtiofsd] Client connected, servicing requests122lb # [2026-09-21T13:21:02Z INFO virtiofsd] Waiting for vhost-user socket connection...123worker1 # [2026-09-21T13:21:02Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether124lb # [2026-09-21T13:21:02Z INFO virtiofsd] Client connected, servicing requests125worker1 # [2026-09-21T13:21:02Z INFO virtiofsd] Waiting for vhost-user socket connection...126lb # [2026-09-21T13:21:02Z INFO virtiofsd] Client connected, servicing requests127worker1 # [2026-09-21T13:21:02Z INFO virtiofsd] Client connected, servicing requests128lb # [2026-09-21T13:21:02Z INFO virtiofsd] Client connected, servicing requests129worker1 # [2026-09-21T13:21:02Z INFO virtiofsd] Client connected, servicing requests130worker1 # [2026-09-21T13:21:02Z INFO virtiofsd] Client connected, servicing requests131worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)132client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)133worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)134lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)135worker2 # 136worker2 # 137worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00138client # 139client # 140client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00141worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...142worker2 # 143worker2 # 144worker2 # 145worker2 # 146client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...147client # 148client # 149client # 150client # 151worker2 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00152client # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00153worker2 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...154worker2 # 155worker2 # 156client # Press Ctrl-B to configure iPXE (PCI 00:05.0)...157client # 158client # 159worker1 # 160worker1 # 161worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00162lb # 163lb # 164worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...165worker1 # 166worker1 # 167worker1 # 168worker1 # 169lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00170worker1 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00171lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...172lb # 173lb # 174lb # 175lb # 176lb # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00177worker1 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...178worker1 # 179worker1 # 180lb # Press Ctrl-B to configure iPXE (PCI 00:05.0)...181lb # 182lb # 183client # Booting from ROM...184worker2 # Booting from ROM...185worker1 # Booting from ROM...186lb # Booting from ROM...187client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026188client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/yvs21lk49a0hcgbp46d92zly2h4ldq9b-nixos-system-client-test/init regInfo=/nix/store/a6vg96kr3hi8fl5l6kj6lrhwpi4v114p-closure-info/registration console=ttyS0,115200n8 console=tty0189client # [ 0.000000] BIOS-provided physical RAM map:190worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026191client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable192client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved193client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved194client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable195worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/7fjvcj1pd83a7fajwxd09a6rd7z1931w-nixos-system-worker2-test/init regInfo=/nix/store/hhzafgaz92a41xax13b6mnwn58zkpysn-closure-info/registration console=ttyS0,115200n8 console=tty0196client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved197worker2 # [ 0.000000] BIOS-provided physical RAM map:198client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved199worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable200client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved201worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved202client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved203worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved204client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved205worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable206client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved207worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved208client # [ 0.000000] NX (Execute Disable) protection: active209worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved210client # [ 0.000000] APIC: Static calls initialized211client # [ 0.000000] SMBIOS 2.8 present.212worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved213worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved214client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014215client # [ 0.000000] DMI: Memory slots populated: 1/1216worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved217client # [ 0.000000] Hypervisor detected: KVM218worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved219client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000220worker2 # [ 0.000000] NX (Execute Disable) protection: active221client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00222worker2 # [ 0.000000] APIC: Static calls initialized223client # [ 0.000000] kvm-clock: using sched offset of 469558692 cycles224worker2 # [ 0.000000] SMBIOS 2.8 present.225worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026226client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns227worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014228client # [ 0.000005] tsc: Detected 2400.012 MHz processor229worker2 # [ 0.000000] DMI: Memory slots populated: 1/1230worker2 # [ 0.000000] Hypervisor detected: KVM231client # [ 0.000823] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000232worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000233client # [ 0.000851] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs234worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00235client # [ 0.000853] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT236worker2 # [ 0.000000] kvm-clock: using sched offset of 461339743 cycles237worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0vi3sqnwf3sq6d8qqjj2wmgml3cn6ib4-nixos-system-worker1-test/init regInfo=/nix/store/65qcknzvrc37g7icy125709s11grb1ji-closure-info/registration console=ttyS0,115200n8 console=tty0238client # [ 0.002740] found SMP MP-table at [mem 0x000f5450-0x000f545f]239worker1 # [ 0.000000] BIOS-provided physical RAM map:240worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns241client # [ 0.002753] Using GB pages for direct mapping242client # [ 0.002834] RAMDISK: [mem 0x5e370000-0x5ffcffff]243worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable244worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor245lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026246client # [ 0.002842] ACPI: Early table checksum verification disabled247worker2 # [ 0.000811] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000248worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved249client # [ 0.002844] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )250worker2 # [ 0.000839] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs251worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved252client # [ 0.002848] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)253worker2 # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT254worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable255worker2 # [ 0.002733] found SMP MP-table at [mem 0x000f5450-0x000f545f]256client # [ 0.002853] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)257worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved258worker2 # [ 0.002745] Using GB pages for direct mapping259lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/wk139pm69cqmz3k7k9y197wlnywfyb0l-nixos-system-lb-test/init regInfo=/nix/store/mlv4di21nd161n17qrvycifnnn1r6la1-closure-info/registration console=ttyS0,115200n8 console=tty0260worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved261worker2 # [ 0.002818] RAMDISK: [mem 0x5e36f000-0x5ffcffff]262client # [ 0.002860] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)263lb # [ 0.000000] BIOS-provided physical RAM map:264client # [ 0.002862] ACPI: FACS 0x000000005FFE0000 000040265worker2 # [ 0.002826] ACPI: Early table checksum verification disabled266worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved267lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable268worker2 # [ 0.002829] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )269worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved270client # [ 0.002863] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)271lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved272worker2 # [ 0.002833] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)273worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved274client # [ 0.002865] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)275lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved276worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved277lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable278worker2 # [ 0.002837] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)279client # [ 0.002867] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)280worker1 # [ 0.000000] NX (Execute Disable) protection: active281lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved282worker1 # [ 0.000000] APIC: Static calls initialized283worker2 # [ 0.002844] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)284worker1 # [ 0.000000] SMBIOS 2.8 present.285client # [ 0.002868] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)286worker2 # [ 0.002846] ACPI: FACS 0x000000005FFE0000 000040287lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved288client # [ 0.002869] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]289lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved290worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014291worker2 # [ 0.002848] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)292client # [ 0.002870] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]293worker1 # [ 0.000000] DMI: Memory slots populated: 1/1294lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved295worker1 # [ 0.000000] Hypervisor detected: KVM296worker2 # [ 0.002849] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)297client # [ 0.002871] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]298worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000299lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved300client # [ 0.002871] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]301worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00302worker2 # [ 0.002851] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)303lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved304worker1 # [ 0.000000] kvm-clock: using sched offset of 479142414 cycles305client # [ 0.002872] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]306lb # [ 0.000000] NX (Execute Disable) protection: active307worker2 # [ 0.002852] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)308lb # [ 0.000000] APIC: Static calls initialized309client # [ 0.002872] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]310lb # [ 0.000000] SMBIOS 2.8 present.311worker2 # [ 0.002854] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]312worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns313client # [ 0.002873] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]314worker1 # [ 0.000004] tsc: Detected 2400.012 MHz processor315worker2 # [ 0.002854] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]316lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014317client # [ 0.003099] No NUMA configuration found318worker1 # [ 0.000810] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000319worker2 # [ 0.002855] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]320lb # [ 0.000000] DMI: Memory slots populated: 1/1321client # [ 0.003099] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]322lb # [ 0.000000] Hypervisor detected: KVM323worker1 # [ 0.000838] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs324client # [ 0.003102] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]325worker2 # [ 0.002855] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]326lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000327client # [ 0.005679] Zone ranges:328worker1 # [ 0.000840] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT329lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00330worker2 # [ 0.002856] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]331client # [ 0.005680] DMA [mem 0x0000000000001000-0x0000000000ffffff]332worker1 # [ 0.002726] found SMP MP-table at [mem 0x000f5450-0x000f545f]333lb # [ 0.000001] kvm-clock: using sched offset of 502141860 cycles334worker2 # [ 0.002856] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]335client # [ 0.005682] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]336worker1 # [ 0.002738] Using GB pages for direct mapping337client # [ 0.005683] Normal empty338worker1 # [ 0.002889] RAMDISK: [mem 0x5e370000-0x5ffcffff]339client # [ 0.005684] Device empty340worker2 # [ 0.002857] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]341lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns342worker2 # [ 0.003084] No NUMA configuration found343worker1 # [ 0.002896] ACPI: Early table checksum verification disabled344client # [ 0.005684] Movable zone start for each node345lb # [ 0.000005] tsc: Detected 2400.012 MHz processor346client # [ 0.005685] Early memory node ranges347worker1 # [ 0.002899] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )348worker2 # [ 0.003085] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]349lb # [ 0.000813] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000350client # [ 0.005685] node 0: [mem 0x0000000000001000-0x000000000009efff]351worker2 # [ 0.003087] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]352worker1 # [ 0.002903] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)353worker2 # [ 0.005839] Zone ranges:354client # [ 0.005686] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]355lb # [ 0.000840] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs356worker2 # [ 0.005841] DMA [mem 0x0000000000001000-0x0000000000ffffff]357worker1 # [ 0.002907] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)358client # [ 0.005688] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]359lb # [ 0.000843] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT360lb # [ 0.002729] found SMP MP-table at [mem 0x000f5450-0x000f545f]361worker2 # [ 0.005842] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]362client # [ 0.005710] On node 0, zone DMA: 1 pages in unavailable ranges363worker2 # [ 0.005843] Normal empty364worker1 # [ 0.002914] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)365lb # [ 0.002739] Using GB pages for direct mapping366worker2 # [ 0.005844] Device empty367client # [ 0.005980] On node 0, zone DMA: 97 pages in unavailable ranges368worker1 # [ 0.002916] ACPI: FACS 0x000000005FFE0000 000040369lb # [ 0.002813] RAMDISK: [mem 0x5e36e000-0x5ffcffff]370worker2 # [ 0.005845] Movable zone start for each node371worker2 # [ 0.005846] Early memory node ranges372client # [ 0.033009] On node 0, zone DMA32: 40 pages in unavailable ranges373lb # [ 0.002821] ACPI: Early table checksum verification disabled374worker1 # [ 0.002918] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)375client # [ 0.033475] ACPI: PM-Timer IO Port: 0x608376worker2 # [ 0.005846] node 0: [mem 0x0000000000001000-0x000000000009efff]377lb # [ 0.002823] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )378client # [ 0.033488] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])379worker1 # [ 0.002919] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)380worker2 # [ 0.005847] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]381lb # [ 0.002827] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)382client # [ 0.033515] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23383worker2 # [ 0.005848] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]384worker1 # [ 0.002921] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)385client # [ 0.033517] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)386lb # [ 0.002831] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)387worker2 # [ 0.005868] On node 0, zone DMA: 1 pages in unavailable ranges388worker1 # [ 0.002923] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)389client # [ 0.033519] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)390lb # [ 0.002838] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)391worker2 # [ 0.006146] On node 0, zone DMA: 97 pages in unavailable ranges392client # [ 0.033520] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)393worker1 # [ 0.002924] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]394lb # [ 0.002841] ACPI: FACS 0x000000005FFE0000 000040395worker2 # [ 0.034378] On node 0, zone DMA32: 40 pages in unavailable ranges396worker2 # [ 0.034839] ACPI: PM-Timer IO Port: 0x608397client # [ 0.033521] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)398worker1 # [ 0.002925] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]399lb # [ 0.002842] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)400worker2 # [ 0.034852] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])401client # [ 0.033522] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)402worker1 # [ 0.002925] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]403lb # [ 0.002844] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)404worker2 # [ 0.034882] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23405client # [ 0.033525] ACPI: Using ACPI (MADT) for SMP configuration information406worker1 # [ 0.002926] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]407client # [ 0.033526] ACPI: HPET id: 0x8086a201 base: 0xfed00000408worker2 # [ 0.034885] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)409lb # [ 0.002845] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)410worker1 # [ 0.002926] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]411client # [ 0.033530] TSC deadline timer available412worker2 # [ 0.034887] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)413client # [ 0.033534] CPU topo: Max. logical packages: 1414worker1 # [ 0.002927] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]415lb # [ 0.002847] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)416client # [ 0.033535] CPU topo: Max. logical dies: 1417worker2 # [ 0.034888] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)418lb # [ 0.002848] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]419worker1 # [ 0.002927] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]420client # [ 0.033535] CPU topo: Max. dies per package: 1421worker2 # [ 0.034893] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)422worker1 # [ 0.003151] No NUMA configuration found423client # [ 0.033539] CPU topo: Max. threads per core: 1424lb # [ 0.002849] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]425client # [ 0.033539] CPU topo: Num. cores per package: 1426worker2 # [ 0.034894] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)427worker1 # [ 0.003152] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]428lb # [ 0.002850] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]429client # [ 0.033540] CPU topo: Num. threads per package: 1430worker1 # [ 0.003156] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]431worker2 # [ 0.034897] ACPI: Using ACPI (MADT) for SMP configuration information432worker1 # [ 0.005414] Zone ranges:433client # [ 0.033540] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs434lb # [ 0.002850] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]435worker2 # [ 0.034898] ACPI: HPET id: 0x8086a201 base: 0xfed00000436worker1 # [ 0.005416] DMA [mem 0x0000000000001000-0x0000000000ffffff]437worker2 # [ 0.034902] TSC deadline timer available438client # [ 0.033563] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()439lb # [ 0.002851] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]440worker2 # [ 0.034906] CPU topo: Max. logical packages: 1441worker1 # [ 0.005417] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]442worker1 # [ 0.005418] Normal empty443client # [ 0.033595] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]444worker2 # [ 0.034907] CPU topo: Max. logical dies: 1445lb # [ 0.002851] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]446worker1 # [ 0.005419] Device empty447worker2 # [ 0.034907] CPU topo: Max. dies per package: 1448worker1 # [ 0.005420] Movable zone start for each node449lb # [ 0.002852] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]450client # [ 0.033597] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]451worker2 # [ 0.034910] CPU topo: Max. threads per core: 1452worker1 # [ 0.005421] Early memory node ranges453lb # [ 0.003068] No NUMA configuration found454client # [ 0.033599] [mem 0x60000000-0xafffffff] available for PCI devices455worker2 # [ 0.034911] CPU topo: Num. cores per package: 1456worker1 # [ 0.005421] node 0: [mem 0x0000000000001000-0x000000000009efff]457lb # [ 0.003069] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]458client # [ 0.033600] Booting paravirtualized kernel on KVM459worker2 # [ 0.034911] CPU topo: Num. threads per package: 1460worker1 # [ 0.005422] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]461lb # [ 0.003071] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]462worker2 # [ 0.034912] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs463lb # [ 0.005372] Zone ranges:464client # [ 0.033604] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns465worker1 # [ 0.005423] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]466lb # [ 0.005372] DMA [mem 0x0000000000001000-0x0000000000ffffff]467worker2 # [ 0.034932] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()468client # [ 0.038088] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1469worker1 # [ 0.005443] On node 0, zone DMA: 1 pages in unavailable ranges470lb # [ 0.005374] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]471worker2 # [ 0.034963] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]472lb # [ 0.005375] Normal empty473worker1 # [ 0.005714] On node 0, zone DMA: 97 pages in unavailable ranges474client # [ 0.040262] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152475lb # [ 0.005376] Device empty476worker1 # [ 0.032666] On node 0, zone DMA32: 40 pages in unavailable ranges477worker2 # [ 0.034965] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]478client # [ 0.040309] kvm-guest: PV spinlocks disabled, single CPU479lb # [ 0.005376] Movable zone start for each node480worker1 # [ 0.033121] ACPI: PM-Timer IO Port: 0x608481lb # [ 0.005377] Early memory node ranges482worker2 # [ 0.034966] [mem 0x60000000-0xafffffff] available for PCI devices483worker1 # [ 0.033132] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])484lb # [ 0.005377] node 0: [mem 0x0000000000001000-0x000000000009efff]485worker2 # [ 0.034968] Booting paravirtualized kernel on KVM486worker1 # [ 0.033157] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23487lb # [ 0.005378] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]488worker2 # [ 0.034971] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns489worker1 # [ 0.033159] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)490lb # [ 0.005379] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]491worker2 # [ 0.039466] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1492lb # [ 0.005399] On node 0, zone DMA: 1 pages in unavailable ranges493worker1 # [ 0.033161] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)494client # [ 0.040311] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/yvs21lk49a0hcgbp46d92zly2h4ldq9b-nixos-system-client-test/init regInfo=/nix/store/a6vg96kr3hi8fl5l6kj6lrhwpi4v114p-closure-info/registration console=ttyS0,115200n8 console=tty0495worker2 # [ 0.041743] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152496worker1 # [ 0.033162] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)497lb # [ 0.005672] On node 0, zone DMA: 97 pages in unavailable ranges498worker2 # [ 0.041790] kvm-guest: PV spinlocks disabled, single CPU499lb # [ 0.033203] On node 0, zone DMA32: 40 pages in unavailable ranges500worker1 # [ 0.033163] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)501lb # [ 0.033657] ACPI: PM-Timer IO Port: 0x608502client # [ 0.040407] Unknown kernel command line parameters "regInfo=/nix/store/a6vg96kr3hi8fl5l6kj6lrhwpi4v114p-closure-info/registration", will be passed to user space.503worker1 # [ 0.033164] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)504client # [ 0.040420] random: crng init done505lb # [ 0.033669] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])506worker1 # [ 0.033166] ACPI: Using ACPI (MADT) for SMP configuration information507client # [ 0.040421] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes508lb # [ 0.033695] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23509worker1 # [ 0.033167] ACPI: HPET id: 0x8086a201 base: 0xfed00000510lb # [ 0.033697] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)511worker1 # [ 0.033170] TSC deadline timer available512client # [ 0.042559] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)513worker1 # [ 0.033174] CPU topo: Max. logical packages: 1514lb # [ 0.033699] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)515worker2 # [ 0.041792] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/7fjvcj1pd83a7fajwxd09a6rd7z1931w-nixos-system-worker2-test/init regInfo=/nix/store/hhzafgaz92a41xax13b6mnwn58zkpysn-closure-info/registration console=ttyS0,115200n8 console=tty0516client # [ 0.043625] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)517worker1 # [ 0.033175] CPU topo: Max. logical dies: 1518client # [ 0.043661] Fallback order for Node 0: 0519lb # [ 0.033700] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)520worker1 # [ 0.033175] CPU topo: Max. dies per package: 1521client # [ 0.043664] Built 1 zonelists, mobility grouping on. Total pages: 393078522worker1 # [ 0.033179] CPU topo: Max. threads per core: 1523lb # [ 0.033701] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)524client # [ 0.043666] Policy zone: DMA32525worker2 # [ 0.041890] Unknown kernel command line parameters "regInfo=/nix/store/hhzafgaz92a41xax13b6mnwn58zkpysn-closure-info/registration", will be passed to user space.526worker1 # [ 0.033179] CPU topo: Num. cores per package: 1527worker2 # [ 0.041902] random: crng init done528lb # [ 0.033702] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)529client # [ 0.046435] mem auto-init: stack:all(zero), heap alloc:on, heap free:off530worker1 # [ 0.033179] CPU topo: Num. threads per package: 1531worker2 # [ 0.041903] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes532lb # [ 0.033704] ACPI: Using ACPI (MADT) for SMP configuration information533worker1 # [ 0.033180] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs534client # [ 0.049681] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1535lb # [ 0.033705] ACPI: HPET id: 0x8086a201 base: 0xfed00000536client # [ 0.052983] allocated 3145728 bytes of page_ext537worker2 # [ 0.044146] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)538lb # [ 0.033709] TSC deadline timer available539worker1 # [ 0.033199] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()540client # [ 0.062892] ftrace: allocating 48787 entries in 192 pages541lb # [ 0.033712] CPU topo: Max. logical packages: 1542worker2 # [ 0.045265] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)543client # [ 0.062894] ftrace: allocated 192 pages with 2 groups544worker1 # [ 0.033228] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]545lb # [ 0.033713] CPU topo: Max. logical dies: 1546worker2 # [ 0.045298] Fallback order for Node 0: 0547client # [ 0.063774] Dynamic Preempt: lazy548lb # [ 0.033713] CPU topo: Max. dies per package: 1549worker1 # [ 0.033229] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]550client # [ 0.063958] rcu: Preemptible hierarchical RCU implementation.551worker2 # [ 0.045301] Built 1 zonelists, mobility grouping on. Total pages: 393078552lb # [ 0.033717] CPU topo: Max. threads per core: 1553client # [ 0.063959] rcu: RCU event tracing is enabled.554worker2 # [ 0.045302] Policy zone: DMA32555worker1 # [ 0.033230] [mem 0x60000000-0xafffffff] available for PCI devices556lb # [ 0.033717] CPU topo: Num. cores per package: 1557worker1 # [ 0.033232] Booting paravirtualized kernel on KVM558worker2 # [ 0.048188] mem auto-init: stack:all(zero), heap alloc:on, heap free:off559client # [ 0.063960] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.560lb # [ 0.033718] CPU topo: Num. threads per package: 1561client # [ 0.063961] Trampoline variant of Tasks RCU enabled.562lb # [ 0.033718] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs563worker2 # [ 0.051562] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1564worker1 # [ 0.033235] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns565client # [ 0.063962] Rude variant of Tasks RCU enabled.566worker2 # [ 0.055023] allocated 3145728 bytes of page_ext567lb # [ 0.033736] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()568client # [ 0.063962] Tracing variant of Tasks RCU enabled.569worker1 # [ 0.037722] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1570worker2 # [ 0.065013] ftrace: allocating 48787 entries in 192 pages571lb # [ 0.033766] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]572worker2 # [ 0.065015] ftrace: allocated 192 pages with 2 groups573client # [ 0.063963] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.574worker1 # [ 0.039904] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152575worker2 # [ 0.065874] Dynamic Preempt: lazy576worker1 # [ 0.039949] kvm-guest: PV spinlocks disabled, single CPU577lb # [ 0.033767] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]578client # [ 0.063963] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1579worker2 # [ 0.065987] rcu: Preemptible hierarchical RCU implementation.580lb # [ 0.033769] [mem 0x60000000-0xafffffff] available for PCI devices581worker2 # [ 0.065988] rcu: RCU event tracing is enabled.582client # [ 0.063985] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.583lb # [ 0.033770] Booting paravirtualized kernel on KVM584worker2 # [ 0.065989] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.585client # [ 0.063987] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.586worker2 # [ 0.065990] Trampoline variant of Tasks RCU enabled.587lb # [ 0.033773] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns588worker2 # [ 0.065991] Rude variant of Tasks RCU enabled.589client # [ 0.063987] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.590lb # [ 0.038241] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1591worker2 # [ 0.065991] Tracing variant of Tasks RCU enabled.592worker1 # [ 0.039950] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0vi3sqnwf3sq6d8qqjj2wmgml3cn6ib4-nixos-system-worker1-test/init regInfo=/nix/store/65qcknzvrc37g7icy125709s11grb1ji-closure-info/registration console=ttyS0,115200n8 console=tty0593client # [ 0.068406] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16594lb # [ 0.040444] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152595worker2 # [ 0.065992] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.596client # [ 0.068693] rcu: srcu_init: Setting srcu_struct sizes based on contention.597lb # [ 0.040488] kvm-guest: PV spinlocks disabled, single CPU598worker2 # [ 0.065992] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1599worker1 # [ 0.040041] Unknown kernel command line parameters "regInfo=/nix/store/65qcknzvrc37g7icy125709s11grb1ji-closure-info/registration", will be passed to user space.600worker1 # [ 0.040054] random: crng init done601client # [ 0.068699] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns602worker2 # [ 0.066008] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.603worker1 # [ 0.040055] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes604client # [ 0.068810] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)605worker2 # [ 0.066009] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.606client # [ 0.072346] Console: colour VGA+ 80x25607worker1 # [ 0.042220] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)608client # [ 0.072349] printk: legacy console [tty0] enabled609worker2 # [ 0.066010] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.610lb # [ 0.040489] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/wk139pm69cqmz3k7k9y197wlnywfyb0l-nixos-system-lb-test/init regInfo=/nix/store/mlv4di21nd161n17qrvycifnnn1r6la1-closure-info/registration console=ttyS0,115200n8 console=tty0611worker1 # [ 0.043567] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)612client # [ 0.103506] printk: legacy console [ttyS0] enabled613worker1 # [ 0.043597] Fallback order for Node 0: 0614worker2 # [ 0.070487] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16615client # [ 0.199991] ACPI: Core revision 20250807616worker1 # [ 0.043600] Built 1 zonelists, mobility grouping on. Total pages: 393078617worker2 # [ 0.070809] rcu: srcu_init: Setting srcu_struct sizes based on contention.618lb # [ 0.040621] Unknown kernel command line parameters "regInfo=/nix/store/mlv4di21nd161n17qrvycifnnn1r6la1-closure-info/registration", will be passed to user space.619worker1 # [ 0.043601] Policy zone: DMA32620lb # [ 0.040634] random: crng init done621worker1 # [ 0.046109] mem auto-init: stack:all(zero), heap alloc:on, heap free:off622worker2 # [ 0.070817] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns623client # [ 0.200873] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns624lb # [ 0.040635] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes625worker1 # [ 0.049244] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1626client # [ 0.202342] APIC: Switch to symmetric I/O mode setup627worker2 # [ 0.070925] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)628worker1 # [ 0.052563] allocated 3145728 bytes of page_ext629lb # [ 0.042816] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)630client # [ 0.203303] x2apic enabled631worker2 # [ 0.074504] Console: colour VGA+ 80x25632worker1 # [ 0.062513] ftrace: allocating 48787 entries in 192 pages633lb # [ 0.044179] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)634worker2 # [ 0.074507] printk: legacy console [tty0] enabled635worker1 # [ 0.062514] ftrace: allocated 192 pages with 2 groups636lb # [ 0.044209] Fallback order for Node 0: 0637worker2 # [ 0.104357] printk: legacy console [ttyS0] enabled638client # [ 0.204028] APIC: Switched APIC routing to: physical x2apic639worker1 # [ 0.063353] Dynamic Preempt: lazy640lb # [ 0.044212] Built 1 zonelists, mobility grouping on. Total pages: 393078641worker1 # [ 0.063516] rcu: Preemptible hierarchical RCU implementation.642worker2 # [ 0.201198] ACPI: Core revision 20250807643lb # [ 0.044213] Policy zone: DMA32644worker1 # [ 0.063517] rcu: RCU event tracing is enabled.645client # [ 0.205814] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1646lb # [ 0.046759] mem auto-init: stack:all(zero), heap alloc:on, heap free:off647worker1 # [ 0.063517] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.648worker2 # [ 0.202071] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns649lb # [ 0.049858] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1650worker1 # [ 0.063519] Trampoline variant of Tasks RCU enabled.651worker1 # [ 0.063519] Rude variant of Tasks RCU enabled.652worker2 # [ 0.203573] APIC: Switch to symmetric I/O mode setup653lb # [ 0.053234] allocated 3145728 bytes of page_ext654client # [ 0.206789] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns655worker1 # [ 0.063520] Tracing variant of Tasks RCU enabled.656lb # [ 0.063072] ftrace: allocating 48787 entries in 192 pages657worker2 # [ 0.204569] x2apic enabled658lb # [ 0.063074] ftrace: allocated 192 pages with 2 groups659worker1 # [ 0.063520] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.660lb # [ 0.063919] Dynamic Preempt: lazy661client # [ 0.208424] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)662worker2 # [ 0.205355] APIC: Switched APIC routing to: physical x2apic663worker1 # [ 0.063521] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1664lb # [ 0.064083] rcu: Preemptible hierarchical RCU implementation.665client # [ 0.209506] x86/cpu: User Mode Instruction Prevention (UMIP) activated666lb # [ 0.064084] rcu: RCU event tracing is enabled.667worker1 # [ 0.063542] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.668lb # [ 0.064085] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.669client # [ 0.210547] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127670worker2 # [ 0.207190] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1671lb # [ 0.064086] Trampoline variant of Tasks RCU enabled.672worker1 # [ 0.063544] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.673lb # [ 0.064087] Rude variant of Tasks RCU enabled.674client # [ 0.211421] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0675lb # [ 0.064087] Tracing variant of Tasks RCU enabled.676worker1 # [ 0.063544] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.677worker2 # [ 0.208169] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns678worker1 # [ 0.068191] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16679lb # [ 0.064088] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.680client # [ 0.212425] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto681worker2 # [ 0.209848] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)682worker1 # [ 0.068468] rcu: srcu_init: Setting srcu_struct sizes based on contention.683lb # [ 0.064088] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1684client # [ 0.213422] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl685lb # [ 0.064107] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.686worker2 # [ 0.210934] x86/cpu: User Mode Instruction Prevention (UMIP) activated687worker1 # [ 0.068474] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns688client # [ 0.214421] Transient Scheduler Attacks: Vulnerable: No microcode689lb # [ 0.064108] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.690worker2 # [ 0.211987] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127691worker1 # [ 0.068576] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)692worker1 # [ 0.072123] Console: colour VGA+ 80x25693client # [ 0.215421] Spectre V2 : Mitigation: Enhanced / Automatic IBRS694worker2 # [ 0.213528] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0695lb # [ 0.064109] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.696worker1 # [ 0.072126] printk: legacy console [tty0] enabled697client # [ 0.216422] Speculative Return Stack Overflow: Mitigation: Safe RET698worker1 # [ 0.101678] printk: legacy console [ttyS0] enabled699lb # [ 0.068773] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16700worker1 # [ 0.197136] ACPI: Core revision 20250807701lb # [ 0.069038] rcu: srcu_init: Setting srcu_struct sizes based on contention.702worker2 # [ 0.214596] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto703lb # [ 0.069044] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns704worker2 # [ 0.215845] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl705worker1 # [ 0.197989] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns706lb # [ 0.069143] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)707worker2 # [ 0.216845] Transient Scheduler Attacks: Vulnerable: No microcode708worker1 # [ 0.199588] APIC: Switch to symmetric I/O mode setup709lb # [ 0.072661] Console: colour VGA+ 80x25710worker1 # [ 0.200512] x2apic enabled711client # [ 0.217421] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization712lb # [ 0.072664] printk: legacy console [tty0] enabled713worker2 # [ 0.218558] Spectre V2 : Mitigation: Enhanced / Automatic IBRS714client # [ 0.219429] Spectre V2 : Enabling IBPB for BPF715lb # [ 0.102046] printk: legacy console [ttyS0] enabled716worker1 # [ 0.201233] APIC: Switched APIC routing to: physical x2apic717worker2 # [ 0.219512] Speculative Return Stack Overflow: Mitigation: Safe RET718lb # [ 0.197125] ACPI: Core revision 20250807719client # [ 0.220422] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier720client # [ 0.221422] active return thunk: srso_alias_return_thunk721worker1 # [ 0.203043] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1722lb # [ 0.198011] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns723lb # [ 0.199525] APIC: Switch to symmetric I/O mode setup724client # [ 0.222443] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'725worker1 # [ 0.204027] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns726lb # [ 0.200474] x2apic enabled727worker2 # [ 0.219844] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization728client # [ 0.223421] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'729lb # [ 0.201201] APIC: Switched APIC routing to: physical x2apic730worker1 # [ 0.205709] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)731worker2 # [ 0.221852] Spectre V2 : Enabling IBPB for BPF732client # [ 0.224421] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'733worker1 # [ 0.206794] x86/cpu: User Mode Instruction Prevention (UMIP) activated734client # [ 0.225421] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'735worker2 # [ 0.222845] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier736lb # [ 0.203015] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1737worker1 # [ 0.207831] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127738worker2 # [ 0.223845] active return thunk: srso_alias_return_thunk739client # [ 0.226421] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'740worker1 # [ 0.208707] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0741client # [ 0.227421] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'742lb # [ 0.203980] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns743worker2 # [ 0.225477] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'744worker2 # [ 0.226727] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'745client # [ 0.228421] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'746lb # [ 0.205651] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)747worker1 # [ 0.209711] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto748worker2 # [ 0.227577] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'749client # [ 0.230303] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256750lb # [ 0.206736] x86/cpu: User Mode Instruction Prevention (UMIP) activated751worker1 # [ 0.210707] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl752client # [ 0.231133] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64753worker2 # [ 0.228589] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'754lb # [ 0.207773] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127755worker1 # [ 0.211707] Transient Scheduler Attacks: Vulnerable: No microcode756client # [ 0.232124] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512757worker2 # [ 0.229592] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'758lb # [ 0.208649] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0759worker1 # [ 0.212706] Spectre V2 : Mitigation: Enhanced / Automatic IBRS760client # [ 0.233117] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024761worker2 # [ 0.230573] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'762worker1 # [ 0.213706] Speculative Return Stack Overflow: Mitigation: Safe RET763client # [ 0.233421] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8764lb # [ 0.209653] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto765worker2 # [ 0.231640] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'766client # [ 0.234421] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.767worker2 # [ 0.232778] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256768lb # [ 0.210649] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl769worker2 # [ 0.233844] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64770lb # [ 0.211649] Transient Scheduler Attacks: Vulnerable: No microcode771worker2 # [ 0.234844] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512772lb # [ 0.212649] Spectre V2 : Mitigation: Enhanced / Automatic IBRS773worker1 # [ 0.214706] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization774worker2 # [ 0.235844] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024775worker1 # [ 0.216714] Spectre V2 : Enabling IBPB for BPF776lb # [ 0.213649] Speculative Return Stack Overflow: Mitigation: Safe RET777worker2 # [ 0.236844] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8778worker1 # [ 0.217707] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier779worker1 # [ 0.218707] active return thunk: srso_alias_return_thunk780worker2 # [ 0.237844] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.781worker1 # [ 0.219727] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'782lb # [ 0.214648] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization783worker1 # [ 0.220707] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'784lb # [ 0.216656] Spectre V2 : Enabling IBPB for BPF785worker1 # [ 0.222493] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'786lb # [ 0.217650] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier787worker1 # [ 0.223501] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'788lb # [ 0.218650] active return thunk: srso_alias_return_thunk789worker1 # [ 0.224488] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'790lb # [ 0.220310] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'791worker1 # [ 0.225482] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'792lb # [ 0.221649] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'793worker1 # [ 0.226521] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'794lb # [ 0.222648] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'795worker1 # [ 0.227707] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256796lb # [ 0.223648] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'797worker1 # [ 0.228706] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64798lb # [ 0.224648] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'799worker1 # [ 0.229706] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512800lb # [ 0.225648] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'801worker1 # [ 0.230706] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024802lb # [ 0.226649] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'803worker1 # [ 0.231707] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8804lb # [ 0.227649] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256805worker1 # [ 0.232707] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.806lb # [ 0.228649] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64807lb # [ 0.229648] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512808lb # [ 0.230649] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024809lb # [ 0.231649] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8810lb # [ 0.233435] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.811client # [ 0.268104] Freeing SMP alternatives memory: 44K812client # [ 0.268424] pid_max: default: 32768 minimum: 301813client # [ 0.269527] LSM: initializing lsm=capability,landlock,yama,bpf,ima814client # [ 0.270533] landlock: Up and running.815client # [ 0.271422] Yama: becoming mindful.816client # [ 0.272578] LSM support for eBPF active817client # [ 0.273387] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)818worker2 # [ 0.272768] Freeing SMP alternatives memory: 44K819client # [ 0.274467] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)820worker2 # [ 0.273672] pid_max: default: 32768 minimum: 301821worker2 # [ 0.274516] LSM: initializing lsm=capability,landlock,yama,bpf,ima822worker2 # [ 0.275723] landlock: Up and running.823client # [ 0.277037] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)824worker2 # [ 0.276292] Yama: becoming mindful.825worker2 # [ 0.277059] LSM support for eBPF active826client # [ 0.278046] Performance Events: Fam17h+ core perfctr, AMD PMU driver.827worker2 # [ 0.277960] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)828client # [ 0.278426] ... version: 2829client # [ 0.279130] ... bit width: 48830worker2 # [ 0.278886] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)831client # [ 0.279424] ... generic counters: 6832client # [ 0.280147] ... generic bitmap: 000000000000003f833client # [ 0.280436] ... fixed-purpose counters: 0834client # [ 0.281175] ... fixed-purpose bitmap: 0000000000000000835worker2 # [ 0.282069] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)836client # [ 0.281423] ... value mask: 0000ffffffffffff837worker2 # [ 0.283422] Performance Events: Fam17h+ core perfctr, AMD PMU driver.838client # [ 0.282372] ... max period: 00007fffffffffff839worker2 # [ 0.283853] ... version: 2840client # [ 0.283155] ... global_ctrl mask: 000000000000003f841worker2 # [ 0.284629] ... bit width: 48842client # [ 0.283545] signal: max sigframe size: 3376843worker1 # [ 0.267501] Freeing SMP alternatives memory: 44K844worker2 # [ 0.285430] ... generic counters: 6845client # [ 0.284356] rcu: Hierarchical SRCU implementation.846worker1 # [ 0.268259] pid_max: default: 32768 minimum: 301847worker2 # [ 0.285847] ... generic bitmap: 000000000000003f848client # [ 0.285032] rcu: Max phase no-delay instances is 400.849worker2 # [ 0.286713] ... fixed-purpose counters: 0850worker1 # [ 0.268792] LSM: initializing lsm=capability,landlock,yama,bpf,ima851worker1 # [ 0.269822] landlock: Up and running.852worker2 # [ 0.287398] ... fixed-purpose bitmap: 0000000000000000853worker1 # [ 0.270707] Yama: becoming mindful.854worker2 # [ 0.287846] ... value mask: 0000ffffffffffff855worker1 # [ 0.271824] LSM support for eBPF active856worker2 # [ 0.288779] ... max period: 00007fffffffffff857lb # [ 0.268116] Freeing SMP alternatives memory: 44K858worker1 # [ 0.272625] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)859worker2 # [ 0.289574] ... global_ctrl mask: 000000000000003f860lb # [ 0.268652] pid_max: default: 32768 minimum: 301861worker2 # [ 0.289960] signal: max sigframe size: 3376862client # [ 0.289944] smp: Bringing up secondary CPUs ...863worker1 # [ 0.273746] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)864worker2 # [ 0.290786] rcu: Hierarchical SRCU implementation.865lb # [ 0.269737] LSM: initializing lsm=capability,landlock,yama,bpf,ima866client # [ 0.290438] smp: Brought up 1 node, 1 CPU867lb # [ 0.270756] landlock: Up and running.868worker2 # [ 0.291422] rcu: Max phase no-delay instances is 400.869client # [ 0.291145] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)870lb # [ 0.271366] Yama: becoming mindful.871worker1 # [ 0.276197] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)872lb # [ 0.272271] LSM support for eBPF active873lb # [ 0.272787] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)874client # [ 0.291649] Memory: 1454596K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)875worker1 # [ 0.277269] Performance Events: Fam17h+ core perfctr, AMD PMU driver.876worker1 # [ 0.277711] ... version: 2877client # [ 0.292673] devtmpfs: initialized878lb # [ 0.273686] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)879worker1 # [ 0.278428] ... bit width: 48880client # [ 0.293479] x86/mm: Memory block size: 128MB881worker1 # [ 0.278709] ... generic counters: 6882worker2 # [ 0.296426] smp: Bringing up secondary CPUs ...883worker1 # [ 0.279424] ... generic bitmap: 000000000000003f884worker2 # [ 0.296860] smp: Brought up 1 node, 1 CPU885lb # [ 0.276834] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)886worker1 # [ 0.279715] ... fixed-purpose counters: 0887client # [ 0.295459] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)888worker2 # [ 0.297535] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)889worker1 # [ 0.280418] ... fixed-purpose bitmap: 0000000000000000890lb # [ 0.278213] Performance Events: Fam17h+ core perfctr, AMD PMU driver.891client # [ 0.296447] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).892worker1 # [ 0.280709] ... value mask: 0000ffffffffffff893lb # [ 0.278657] ... version: 2894worker1 # [ 0.281575] ... max period: 00007fffffffffff895client # [ 0.297505] pinctrl core: initialized pinctrl subsystem896worker2 # [ 0.298012] Memory: 1454596K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109312K reserved, 0K cma-reserved)897lb # [ 0.279323] ... bit width: 48898worker1 # [ 0.282379] ... global_ctrl mask: 000000000000003f899worker2 # [ 0.299115] devtmpfs: initialized900lb # [ 0.279651] ... generic counters: 6901client # [ 0.298569] PM: RTC time: 13:21:02, date: 2026-09-21902worker1 # [ 0.282817] signal: max sigframe size: 3376903worker2 # [ 0.299973] x86/mm: Memory block size: 128MB904lb # [ 0.280361] ... generic bitmap: 000000000000003f905worker1 # [ 0.283639] rcu: Hierarchical SRCU implementation.906lb # [ 0.280651] ... fixed-purpose counters: 0907worker1 # [ 0.284303] rcu: Max phase no-delay instances is 400.908lb # [ 0.281346] ... fixed-purpose bitmap: 0000000000000000909worker2 # [ 0.301787] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)910lb # [ 0.281651] ... value mask: 0000ffffffffffff911client # [ 0.302191] NET: Registered PF_NETLINK/PF_ROUTE protocol family912lb # [ 0.282558] ... max period: 00007fffffffffff913worker2 # [ 0.302766] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).914lb # [ 0.283359] ... global_ctrl mask: 000000000000003f915worker2 # [ 0.303947] pinctrl core: initialized pinctrl subsystem916client # [ 0.302935] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations917lb # [ 0.283764] signal: max sigframe size: 3376918worker2 # [ 0.305088] PM: RTC time: 13:21:02, date: 2026-09-21919client # [ 0.303453] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations920lb # [ 0.284578] rcu: Hierarchical SRCU implementation.921worker1 # [ 0.289255] smp: Bringing up secondary CPUs ...922lb # [ 0.285272] rcu: Max phase no-delay instances is 400.923worker1 # [ 0.289723] smp: Brought up 1 node, 1 CPU924client # [ 0.304711] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations925client # [ 0.305435] audit: initializing netlink subsys (disabled)926worker1 # [ 0.290417] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)927worker2 # [ 0.308699] NET: Registered PF_NETLINK/PF_ROUTE protocol family928client # [ 0.306614] thermal_sys: Registered thermal governor 'fair_share'929client # [ 0.306616] thermal_sys: Registered thermal governor 'bang_bang'930worker1 # [ 0.290923] Memory: 1454608K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)931worker2 # [ 0.310127] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations932client # [ 0.307425] thermal_sys: Registered thermal governor 'step_wise'933worker1 # [ 0.291953] devtmpfs: initialized934lb # [ 0.290226] smp: Bringing up secondary CPUs ...935worker2 # [ 0.310872] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations936worker1 # [ 0.292693] x86/mm: Memory block size: 128MB937client # [ 0.308372] thermal_sys: Registered thermal governor 'user_space'938lb # [ 0.290665] smp: Brought up 1 node, 1 CPU939client # [ 0.309162] audit: type=2000 audit(1789996862.731:1): state=initialized audit_enabled=0 res=1940lb # [ 0.291371] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)941worker2 # [ 0.312144] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations942worker1 # [ 0.294274] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)943worker2 # [ 0.312858] audit: initializing netlink subsys (disabled)944client # [ 0.310426] thermal_sys: Registered thermal governor 'power_allocator'945client # [ 0.310441] cpuidle: using governor menu946worker1 # [ 0.294734] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).947worker2 # [ 0.314044] thermal_sys: Registered thermal governor 'fair_share'948lb # [ 0.291834] Memory: 1454588K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109316K reserved, 0K cma-reserved)949worker1 # [ 0.295788] pinctrl core: initialized pinctrl subsystem950lb # [ 0.292885] devtmpfs: initialized951worker2 # [ 0.314046] thermal_sys: Registered thermal governor 'bang_bang'952client # [ 0.312634] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5953lb # [ 0.293588] x86/mm: Memory block size: 128MB954worker1 # [ 0.296902] PM: RTC time: 13:21:02, date: 2026-09-21955worker2 # [ 0.314849] thermal_sys: Registered thermal governor 'step_wise'956client # [ 0.313684] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]957worker2 # [ 0.315809] thermal_sys: Registered thermal governor 'user_space'958client # [ 0.314427] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry959lb # [ 0.295240] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)960worker2 # [ 0.316608] audit: type=2000 audit(1789996862.743:1): state=initialized audit_enabled=0 res=1961client # [ 0.315436] PCI: Using configuration type 1 for base access962worker2 # [ 0.317850] thermal_sys: Registered thermal governor 'power_allocator'963lb # [ 0.295676] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).964worker1 # [ 0.300479] NET: Registered PF_NETLINK/PF_ROUTE protocol family965worker2 # [ 0.317864] cpuidle: using governor menu966lb # [ 0.296735] pinctrl core: initialized pinctrl subsystem967client # [ 0.316558] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.968worker1 # [ 0.301916] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations969lb # [ 0.297863] PM: RTC time: 13:21:02, date: 2026-09-21970worker2 # [ 0.320095] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5971worker1 # [ 0.302736] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations972worker2 # [ 0.321075] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]973worker1 # [ 0.303987] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations974worker2 # [ 0.321849] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry975worker1 # [ 0.304719] audit: initializing netlink subsys (disabled)976lb # [ 0.301423] NET: Registered PF_NETLINK/PF_ROUTE protocol family977worker2 # [ 0.322859] PCI: Using configuration type 1 for base access978worker1 # [ 0.305891] thermal_sys: Registered thermal governor 'fair_share'979lb # [ 0.302848] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations980worker2 # [ 0.323993] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.981worker1 # [ 0.305893] thermal_sys: Registered thermal governor 'bang_bang'982client # [ 0.323702] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages983lb # [ 0.303673] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations984worker1 # [ 0.306717] thermal_sys: Registered thermal governor 'step_wise'985client # [ 0.324425] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page986worker1 # [ 0.307643] thermal_sys: Registered thermal governor 'user_space'987lb # [ 0.304938] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations988worker1 # [ 0.308448] audit: type=2000 audit(1789996862.754:1): state=initialized audit_enabled=0 res=1989lb # [ 0.305661] audit: initializing netlink subsys (disabled)990worker1 # [ 0.309712] thermal_sys: Registered thermal governor 'power_allocator'991lb # [ 0.306806] thermal_sys: Registered thermal governor 'fair_share'992worker1 # [ 0.309727] cpuidle: using governor menu993worker2 # [ 0.329079] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages994lb # [ 0.306808] thermal_sys: Registered thermal governor 'bang_bang'995worker2 # [ 0.329848] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page996lb # [ 0.307652] thermal_sys: Registered thermal governor 'step_wise'997client # [ 0.329425] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages998worker1 # [ 0.311927] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5999lb # [ 0.308625] thermal_sys: Registered thermal governor 'user_space'1000client # [ 0.330425] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1001worker1 # [ 0.312941] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]1002lb # [ 0.309370] audit: type=2000 audit(1789996862.761:1): state=initialized audit_enabled=0 res=11003worker1 # [ 0.313712] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1004lb # [ 0.310654] thermal_sys: Registered thermal governor 'power_allocator'1005worker1 # [ 0.314720] PCI: Using configuration type 1 for base access1006lb # [ 0.310668] cpuidle: using governor menu1007worker2 # [ 0.334848] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1008worker1 # [ 0.315833] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1009lb # [ 0.312866] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.51010worker2 # [ 0.335848] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1011lb # [ 0.313883] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]1012lb # [ 0.314654] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1013client # [ 0.338998] ACPI: Added _OSI(Module Device)1014lb # [ 0.315664] PCI: Using configuration type 1 for base access1015client # [ 0.339426] ACPI: Added _OSI(Processor Device)1016client # [ 0.340200] ACPI: Added _OSI(Processor Aggregator Device)1017lb # [ 0.316783] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1018worker1 # [ 0.322858] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1019worker1 # [ 0.323710] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1020worker2 # [ 0.346260] ACPI: Added _OSI(Module Device)1021lb # [ 0.321931] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1022worker2 # [ 0.346849] ACPI: Added _OSI(Processor Device)1023lb # [ 0.322652] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1024worker2 # [ 0.347587] ACPI: Added _OSI(Processor Aggregator Device)1025client # [ 0.345987] ACPI: 1 ACPI AML tables successfully acquired and loaded1026worker1 # [ 0.328710] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1027worker1 # [ 0.329710] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1028client # [ 0.349339] ACPI: Interpreter enabled1029client # [ 0.349953] ACPI: PM: (supports S0 S3 S4 S5)1030lb # [ 0.327654] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1031lb # [ 0.328651] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1032client # [ 0.352426] ACPI: Using IOAPIC for interrupt routing1033client # [ 0.353365] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1034worker2 # [ 0.356436] ACPI: 1 ACPI AML tables successfully acquired and loaded1035worker2 # [ 0.357785] ACPI: Interpreter enabled1036client # [ 0.356424] PCI: Using E820 reservations for host bridge windows1037worker2 # [ 0.358364] ACPI: PM: (supports S0 S3 S4 S5)1038worker1 # [ 0.340104] ACPI: Added _OSI(Module Device)1039client # [ 0.357541] ACPI: Enabled 2 GPEs in block 00 to 3F1040worker1 # [ 0.340711] ACPI: Added _OSI(Processor Device)1041worker1 # [ 0.341441] ACPI: Added _OSI(Processor Aggregator Device)1042worker2 # [ 0.360848] ACPI: Using IOAPIC for interrupt routing1043lb # [ 0.338994] ACPI: Added _OSI(Module Device)1044worker2 # [ 0.361730] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1045lb # [ 0.339653] ACPI: Added _OSI(Processor Device)1046lb # [ 0.340414] ACPI: Added _OSI(Processor Aggregator Device)1047worker2 # [ 0.364848] PCI: Using E820 reservations for host bridge windows1048worker2 # [ 0.365979] ACPI: Enabled 2 GPEs in block 00 to 3F1049client # [ 0.366245] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1050worker1 # [ 0.348259] ACPI: 1 ACPI AML tables successfully acquired and loaded1051client # [ 0.368688] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1052worker1 # [ 0.352144] ACPI: Interpreter enabled1053client # [ 0.369505] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1054worker1 # [ 0.352727] ACPI: PM: (supports S0 S3 S4 S5)1055lb # [ 0.349268] ACPI: 1 ACPI AML tables successfully acquired and loaded1056client # [ 0.370545] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1057worker1 # [ 0.353419] ACPI: Using IOAPIC for interrupt routing1058client # [ 0.371777] PCI host bridge to bus 0000:001059worker2 # [ 0.374722] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1060client # [ 0.372428] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1061client # [ 0.373424] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1062lb # [ 0.353117] ACPI: Interpreter enabled1063worker1 # [ 0.355796] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1064lb # [ 0.353667] ACPI: PM: (supports S0 S3 S4 S5)1065worker2 # [ 0.376425] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1066client # [ 0.374424] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1067worker1 # [ 0.356710] PCI: Using E820 reservations for host bridge windows1068lb # [ 0.354331] ACPI: Using IOAPIC for interrupt routing1069worker2 # [ 0.376925] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1070client # [ 0.375425] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1071worker2 # [ 0.377982] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1072client # [ 0.376424] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1073worker1 # [ 0.359790] ACPI: Enabled 2 GPEs in block 00 to 3F1074worker2 # [ 0.379215] PCI host bridge to bus 0000:001075client # [ 0.377424] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1076lb # [ 0.356736] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1077worker2 # [ 0.379852] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1078client # [ 0.378424] pci_bus 0000:00: root bus resource [bus 00-ff]1079lb # [ 0.357652] PCI: Using E820 reservations for host bridge windows1080worker2 # [ 0.380847] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1081client # [ 0.379425] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1082worker2 # [ 0.381847] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1083client # [ 0.380885] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1084worker2 # [ 0.382847] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1085lb # [ 0.360764] ACPI: Enabled 2 GPEs in block 00 to 3F1086worker2 # [ 0.383847] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1087worker1 # [ 0.366870] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1088worker2 # [ 0.384847] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1089worker2 # [ 0.385847] pci_bus 0000:00: root bus resource [bus 00-ff]1090worker1 # [ 0.367715] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1091worker2 # [ 0.386914] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1092worker1 # [ 0.368794] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1093worker1 # [ 0.369830] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1094worker2 # [ 0.388323] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1095worker1 # [ 0.371049] PCI host bridge to bus 0000:001096lb # [ 0.368257] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1097worker1 # [ 0.371713] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1098client # [ 0.383509] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1099lb # [ 0.368658] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1100worker1 # [ 0.372710] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1101client # [ 0.384436] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1102lb # [ 0.369730] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1103worker1 # [ 0.373709] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1104client # [ 0.385439] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1105lb # [ 0.370774] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1106worker1 # [ 0.374710] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1107lb # [ 0.372032] PCI host bridge to bus 0000:001108client # [ 0.386601] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1109worker1 # [ 0.375709] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1110lb # [ 0.372656] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1111worker1 # [ 0.376709] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1112client # [ 0.388125] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1113lb # [ 0.373651] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1114worker1 # [ 0.377715] pci_bus 0000:00: root bus resource [bus 00-ff]1115worker2 # [ 0.390951] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1116lb # [ 0.374651] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1117worker2 # [ 0.391862] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1118worker1 # [ 0.378690] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1119lb # [ 0.375651] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1120client # [ 0.391462] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1121worker2 # [ 0.392804] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1122lb # [ 0.376651] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1123worker1 # [ 0.380200] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1124client # [ 0.392302] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1125lb # [ 0.377651] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1126worker2 # [ 0.394158] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1127client # [ 0.393151] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1128lb # [ 0.378652] pci_bus 0000:00: root bus resource [bus 00-ff]1129client # [ 0.393430] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1130worker2 # [ 0.395577] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1131lb # [ 0.379663] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1132client # [ 0.395169] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1133lb # [ 0.381082] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1134worker2 # [ 0.398890] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1135client # [ 0.397460] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1136worker2 # [ 0.399854] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1137client # [ 0.398364] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1138worker2 # [ 0.400852] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1139worker1 # [ 0.382792] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1140client # [ 0.399217] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1141worker1 # [ 0.383721] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1142worker2 # [ 0.401853] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1143worker1 # [ 0.384725] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1144client # [ 0.400840] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1145worker2 # [ 0.403391] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1146worker1 # [ 0.385929] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1147lb # [ 0.383740] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1148worker1 # [ 0.387411] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1149client # [ 0.403432] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1150worker2 # [ 0.405674] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1151lb # [ 0.384663] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1152client # [ 0.404270] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1153worker2 # [ 0.406457] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1154lb # [ 0.385612] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1155client # [ 0.405148] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1156worker2 # [ 0.406869] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1157lb # [ 0.386682] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1158worker1 # [ 0.390749] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1159client # [ 0.405995] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1160worker2 # [ 0.408426] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1161worker1 # [ 0.391618] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1162worker1 # [ 0.392415] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1163lb # [ 0.388580] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1164client # [ 0.408350] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1165worker1 # [ 0.392715] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1166client # [ 0.409144] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1167worker2 # [ 0.410856] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1168lb # [ 0.390689] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1169worker2 # [ 0.411693] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1170worker1 # [ 0.394303] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1171lb # [ 0.391538] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1172client # [ 0.409446] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1173worker2 # [ 0.412586] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1174client # [ 0.410430] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1175lb # [ 0.392406] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1176lb # [ 0.392657] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1177worker1 # [ 0.396749] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1178worker2 # [ 0.413429] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1179client # [ 0.411960] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1180worker1 # [ 0.397580] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1181lb # [ 0.394193] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1182worker1 # [ 0.398501] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1183client # [ 0.413465] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1184worker2 # [ 0.415792] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1185client # [ 0.414446] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1186worker2 # [ 0.416558] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1187lb # [ 0.395684] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1188worker1 # [ 0.400181] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1189lb # [ 0.396489] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1190client # [ 0.415988] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1191worker2 # [ 0.416869] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1192lb # [ 0.397407] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1193worker2 # [ 0.417854] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1194lb # [ 0.398245] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1195client # [ 0.417466] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1196worker1 # [ 0.403748] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1197worker2 # [ 0.419403] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1198client # [ 0.418446] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1199worker1 # [ 0.404603] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1200worker1 # [ 0.405476] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1201worker2 # [ 0.420859] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1202client # [ 0.420022] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1203lb # [ 0.400691] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1204worker2 # [ 0.421836] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1205lb # [ 0.401565] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1206worker1 # [ 0.406253] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1207client # [ 0.421436] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1208lb # [ 0.402416] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1209worker2 # [ 0.423344] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1210client # [ 0.422401] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1211lb # [ 0.403193] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1212worker1 # [ 0.408748] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1213client # [ 0.423828] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1214worker1 # [ 0.409661] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1215worker2 # [ 0.424860] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1216lb # [ 0.404688] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1217worker2 # [ 0.426676] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1218worker1 # [ 0.410493] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1219lb # [ 0.405509] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1220client # [ 0.425465] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1221worker1 # [ 0.411555] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1222client # [ 0.426446] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1223worker2 # [ 0.428285] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1224lb # [ 0.406357] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1225lb # [ 0.406657] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1226worker1 # [ 0.413033] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1227client # [ 0.427989] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1228worker2 # [ 0.429860] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1229lb # [ 0.408186] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1230worker2 # [ 0.430901] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1231worker1 # [ 0.414753] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1232client # [ 0.429467] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1233worker1 # [ 0.415731] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1234lb # [ 0.409460] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1235worker2 # [ 0.432380] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1236client # [ 0.430361] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1237lb # [ 0.410337] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1238client # [ 0.431163] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1239worker1 # [ 0.417290] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1240lb # [ 0.411269] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1241worker2 # [ 0.433860] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1242client # [ 0.431968] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1243worker2 # [ 0.434870] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1244lb # [ 0.412694] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1245worker1 # [ 0.418723] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1246client # [ 0.434161] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1247worker2 # [ 0.436418] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1248lb # [ 0.413673] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1249worker1 # [ 0.420497] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1250client # [ 0.434431] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1251client # [ 0.435401] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1252lb # [ 0.415226] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1253worker1 # [ 0.422151] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1254worker2 # [ 0.437898] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1255worker2 # [ 0.438767] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1256client # [ 0.436819] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1257lb # [ 0.416489] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1258worker1 # [ 0.423750] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1259worker2 # [ 0.439631] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1260lb # [ 0.417409] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1261client # [ 0.438146] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1262worker1 # [ 0.424730] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1263lb # [ 0.418179] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1264client # [ 0.438618] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1265worker2 # [ 0.441328] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1266worker1 # [ 0.426249] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1267lb # [ 0.419634] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1268worker2 # [ 0.442890] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1269client # [ 0.441192] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1270lb # [ 0.420470] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1271worker2 # [ 0.443781] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1272worker1 # [ 0.427722] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1273client # [ 0.441633] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1274worker2 # [ 0.444588] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1275worker1 # [ 0.429455] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1276lb # [ 0.422183] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1277client # [ 0.443087] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1278worker1 # [ 0.430274] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1279worker2 # [ 0.445445] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1280lb # [ 0.423669] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1281worker2 # [ 0.446561] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1282lb # [ 0.424596] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1283worker1 # [ 0.432528] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1284lb # [ 0.425430] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1285worker2 # [ 0.447073] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1286worker1 # [ 0.433374] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1287worker1 # [ 0.433731] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1288lb # [ 0.427054] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1289worker2 # [ 0.449297] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1290worker2 # [ 0.450093] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1291worker1 # [ 0.435260] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1292lb # [ 0.428780] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1293worker2 # [ 0.451509] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1294lb # [ 0.429658] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1295worker1 # [ 0.436717] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1296lb # [ 0.430667] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1297worker1 # [ 0.437652] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1298client # [ 0.443655] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1299worker1 # [ 0.438506] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1300lb # [ 0.432226] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1301client # [ 0.446085] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1302lb # [ 0.433208] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1303worker1 # [ 0.440140] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1304lb # [ 0.433847] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1305client # [ 0.446701] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1306worker1 # [ 0.441513] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1307lb # [ 0.435191] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1308client # [ 0.447743] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1309worker1 # [ 0.442597] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1310lb # [ 0.435841] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1311client # [ 0.448681] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1312lb # [ 0.437167] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1313worker1 # [ 0.444572] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1314client # [ 0.450482] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1315worker1 # [ 0.445600] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1316client # [ 0.451397] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1317worker2 # [ 0.452067] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1318worker1 # [ 0.447386] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1319worker2 # [ 0.454490] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1320client # [ 0.452573] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1321client # [ 0.454083] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1322worker2 # [ 0.455173] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1323worker2 # [ 0.456177] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1324worker2 # [ 0.457092] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1325client # [ 0.457919] ACPI: PCI: Interrupt link LNKA configured for IRQ 101326worker2 # [ 0.458864] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1327client # [ 0.458528] ACPI: PCI: Interrupt link LNKB configured for IRQ 101328worker2 # [ 0.459759] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1329client # [ 0.459527] ACPI: PCI: Interrupt link LNKC configured for IRQ 111330lb # [ 0.437877] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1331client # [ 0.460524] ACPI: PCI: Interrupt link LNKD configured for IRQ 111332worker2 # [ 0.460963] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1333client # [ 0.461526] ACPI: PCI: Interrupt link LNKE configured for IRQ 101334lb # [ 0.440204] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1335client # [ 0.462531] ACPI: PCI: Interrupt link LNKF configured for IRQ 101336worker2 # [ 0.462808] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1337lb # [ 0.440968] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1338client # [ 0.463523] ACPI: PCI: Interrupt link LNKG configured for IRQ 111339worker1 # [ 0.447962] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1340lb # [ 0.441936] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1341client # [ 0.464520] ACPI: PCI: Interrupt link LNKH configured for IRQ 111342worker1 # [ 0.450531] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1343client # [ 0.465463] ACPI: PCI: Interrupt link GSIA configured for IRQ 161344lb # [ 0.442893] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1345client # [ 0.466441] ACPI: PCI: Interrupt link GSIB configured for IRQ 171346worker2 # [ 0.466906] ACPI: PCI: Interrupt link LNKA configured for IRQ 101347worker1 # [ 0.451743] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1348client # [ 0.467435] ACPI: PCI: Interrupt link GSIC configured for IRQ 181349worker2 # [ 0.467953] ACPI: PCI: Interrupt link LNKB configured for IRQ 101350lb # [ 0.444571] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1351worker1 # [ 0.453006] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1352client # [ 0.468433] ACPI: PCI: Interrupt link GSID configured for IRQ 191353lb # [ 0.445249] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1354worker2 # [ 0.468948] ACPI: PCI: Interrupt link LNKC configured for IRQ 111355client # [ 0.469399] ACPI: PCI: Interrupt link GSIE configured for IRQ 201356worker2 # [ 0.469946] ACPI: PCI: Interrupt link LNKD configured for IRQ 111357worker1 # [ 0.453952] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1358lb # [ 0.445967] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1359client # [ 0.470178] ACPI: PCI: Interrupt link GSIF configured for IRQ 211360worker2 # [ 0.470944] ACPI: PCI: Interrupt link LNKE configured for IRQ 101361client # [ 0.470436] ACPI: PCI: Interrupt link GSIG configured for IRQ 221362lb # [ 0.447259] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1363worker2 # [ 0.471949] ACPI: PCI: Interrupt link LNKF configured for IRQ 101364worker1 # [ 0.455738] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1365client # [ 0.471412] ACPI: PCI: Interrupt link GSIH configured for IRQ 231366worker1 # [ 0.456541] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1367worker2 # [ 0.472957] ACPI: PCI: Interrupt link LNKG configured for IRQ 111368client # [ 0.473166] iommu: Default domain type: Translated1369worker2 # [ 0.473952] ACPI: PCI: Interrupt link LNKH configured for IRQ 111370worker1 # [ 0.457811] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1371client # [ 0.473431] iommu: DMA domain TLB invalidation policy: lazy mode1372worker2 # [ 0.474889] ACPI: PCI: Interrupt link GSIA configured for IRQ 161373client # [ 0.474638] ACPI: bus type USB registered1374lb # [ 0.450698] ACPI: PCI: Interrupt link LNKA configured for IRQ 101375worker2 # [ 0.475862] ACPI: PCI: Interrupt link GSIB configured for IRQ 171376worker1 # [ 0.459388] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1377client # [ 0.475350] usbcore: registered new interface driver usbfs1378lb # [ 0.451762] ACPI: PCI: Interrupt link LNKB configured for IRQ 101379worker2 # [ 0.476841] ACPI: PCI: Interrupt link GSIC configured for IRQ 181380client # [ 0.476076] usbcore: registered new interface driver hub1381lb # [ 0.452752] ACPI: PCI: Interrupt link LNKC configured for IRQ 111382worker2 # [ 0.477610] ACPI: PCI: Interrupt link GSID configured for IRQ 191383client # [ 0.476433] usbcore: registered new device driver usb1384lb # [ 0.453751] ACPI: PCI: Interrupt link LNKD configured for IRQ 111385worker2 # [ 0.477862] ACPI: PCI: Interrupt link GSIE configured for IRQ 201386client # [ 0.478141] NetLabel: Initializing1387lb # [ 0.454750] ACPI: PCI: Interrupt link LNKE configured for IRQ 101388worker2 # [ 0.478859] ACPI: PCI: Interrupt link GSIF configured for IRQ 211389worker1 # [ 0.462759] ACPI: PCI: Interrupt link LNKA configured for IRQ 101390client # [ 0.478430] NetLabel: domain hash size = 1281391worker2 # [ 0.479828] ACPI: PCI: Interrupt link GSIG configured for IRQ 221392lb # [ 0.455749] ACPI: PCI: Interrupt link LNKF configured for IRQ 101393client # [ 0.479127] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1394worker1 # [ 0.463819] ACPI: PCI: Interrupt link LNKB configured for IRQ 101395worker2 # [ 0.480624] ACPI: PCI: Interrupt link GSIH configured for IRQ 231396lb # [ 0.456754] ACPI: PCI: Interrupt link LNKG configured for IRQ 111397client # [ 0.479459] NetLabel: unlabeled traffic allowed by default1398worker1 # [ 0.464819] ACPI: PCI: Interrupt link LNKC configured for IRQ 111399client # [ 0.480348] PCI: Using ACPI for IRQ routing1400lb # [ 0.457751] ACPI: PCI: Interrupt link LNKH configured for IRQ 111401worker1 # [ 0.465816] ACPI: PCI: Interrupt link LNKD configured for IRQ 111402worker2 # [ 0.481884] iommu: Default domain type: Translated1403lb # [ 0.458686] ACPI: PCI: Interrupt link GSIA configured for IRQ 161404worker2 # [ 0.482651] iommu: DMA domain TLB invalidation policy: lazy mode1405worker1 # [ 0.466806] ACPI: PCI: Interrupt link LNKE configured for IRQ 101406lb # [ 0.459631] ACPI: PCI: Interrupt link GSIB configured for IRQ 171407worker2 # [ 0.483826] ACPI: bus type USB registered1408worker1 # [ 0.467810] ACPI: PCI: Interrupt link LNKF configured for IRQ 101409lb # [ 0.460386] ACPI: PCI: Interrupt link GSIC configured for IRQ 181410worker2 # [ 0.484388] usbcore: registered new interface driver usbfs1411lb # [ 0.461400] ACPI: PCI: Interrupt link GSID configured for IRQ 191412worker1 # [ 0.468807] ACPI: PCI: Interrupt link LNKG configured for IRQ 111413worker2 # [ 0.484865] usbcore: registered new interface driver hub1414lb # [ 0.461666] ACPI: PCI: Interrupt link GSIE configured for IRQ 201415worker2 # [ 0.485729] usbcore: registered new device driver usb1416worker1 # [ 0.469819] ACPI: PCI: Interrupt link LNKH configured for IRQ 111417lb # [ 0.462621] ACPI: PCI: Interrupt link GSIF configured for IRQ 211418worker1 # [ 0.470747] ACPI: PCI: Interrupt link GSIA configured for IRQ 161419worker2 # [ 0.487391] NetLabel: Initializing1420lb # [ 0.463403] ACPI: PCI: Interrupt link GSIG configured for IRQ 221421worker1 # [ 0.471724] ACPI: PCI: Interrupt link GSIB configured for IRQ 171422worker2 # [ 0.487848] NetLabel: domain hash size = 1281423lb # [ 0.463663] ACPI: PCI: Interrupt link GSIH configured for IRQ 231424worker1 # [ 0.472685] ACPI: PCI: Interrupt link GSIC configured for IRQ 181425worker2 # [ 0.488571] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1426worker1 # [ 0.473511] ACPI: PCI: Interrupt link GSID configured for IRQ 191427worker2 # [ 0.488882] NetLabel: unlabeled traffic allowed by default1428lb # [ 0.465661] iommu: Default domain type: Translated1429worker2 # [ 0.489788] PCI: Using ACPI for IRQ routing1430worker1 # [ 0.474482] ACPI: PCI: Interrupt link GSIE configured for IRQ 201431lb # [ 0.466453] iommu: DMA domain TLB invalidation policy: lazy mode1432worker1 # [ 0.474725] ACPI: PCI: Interrupt link GSIF configured for IRQ 211433lb # [ 0.467651] ACPI: bus type USB registered1434worker1 # [ 0.475722] ACPI: PCI: Interrupt link GSIG configured for IRQ 221435lb # [ 0.468400] usbcore: registered new interface driver usbfs1436worker1 # [ 0.476718] ACPI: PCI: Interrupt link GSIH configured for IRQ 231437lb # [ 0.468668] usbcore: registered new interface driver hub1438lb # [ 0.469484] usbcore: registered new device driver usb1439worker1 # [ 0.478815] iommu: Default domain type: Translated1440lb # [ 0.471125] NetLabel: Initializing1441lb # [ 0.471653] NetLabel: domain hash size = 1281442worker1 # [ 0.479722] iommu: DMA domain TLB invalidation policy: lazy mode1443lb # [ 0.472338] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1444worker1 # [ 0.480997] ACPI: bus type USB registered1445lb # [ 0.472692] NetLabel: unlabeled traffic allowed by default1446worker1 # [ 0.481764] usbcore: registered new interface driver usbfs1447lb # [ 0.473585] PCI: Using ACPI for IRQ routing1448worker1 # [ 0.482848] usbcore: registered new interface driver hub1449worker1 # [ 0.483718] usbcore: registered new device driver usb1450worker1 # [ 0.485492] NetLabel: Initializing1451worker1 # [ 0.485716] NetLabel: domain hash size = 1281452worker1 # [ 0.486472] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1453worker1 # [ 0.487496] NetLabel: unlabeled traffic allowed by default1454worker1 # [ 0.487721] PCI: Using ACPI for IRQ routing1455client # [ 0.524595] pci 0000:00:01.0: vgaarb: setting as boot VGA device1456client # [ 0.525420] pci 0000:00:01.0: vgaarb: bridge control possible1457client # [ 0.525420] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1458client # [ 0.525428] vgaarb: loaded1459client # [ 0.526203] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01460client # [ 0.526426] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1461client # [ 0.530508] clocksource: Switched to clocksource kvm-clock1462worker2 # [ 0.534088] pci 0000:00:01.0: vgaarb: setting as boot VGA device1463worker2 # [ 0.534843] pci 0000:00:01.0: vgaarb: bridge control possible1464client # [ 0.534112] VFS: Disk quotas dquot_6.6.01465worker2 # [ 0.534843] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1466worker2 # [ 0.534851] vgaarb: loaded1467client # [ 0.534800] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1468client # [ 0.536128] pnp: PnP ACPI init1469worker2 # [ 0.535598] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01470client # [ 0.536921] ACPI: IRQ 4 override to edge(!), high(!)1471worker2 # [ 0.535849] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1472client # [ 0.537964] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1473client # [ 0.539546] pnp: PnP ACPI: found 5 devices1474lb # [ 0.518267] pci 0000:00:01.0: vgaarb: setting as boot VGA device1475lb # [ 0.518647] pci 0000:00:01.0: vgaarb: bridge control possible1476lb # [ 0.518647] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1477lb # [ 0.518656] vgaarb: loaded1478worker2 # [ 0.540936] clocksource: Switched to clocksource kvm-clock1479lb # [ 0.519384] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01480lb # [ 0.519654] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1481worker2 # [ 0.544591] VFS: Disk quotas dquot_6.6.01482worker1 # [ 0.531990] pci 0000:00:01.0: vgaarb: setting as boot VGA device1483worker1 # [ 0.532705] pci 0000:00:01.0: vgaarb: bridge control possible1484worker2 # [ 0.545352] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1485client # [ 0.547189] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1486worker2 # [ 0.546714] pnp: PnP ACPI init1487client # [ 0.548648] clocksource: Switched to clocksource acpi_pm1488worker1 # [ 0.532705] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1489worker1 # [ 0.532713] vgaarb: loaded1490client # [ 0.549613] NET: Registered PF_INET protocol family1491worker2 # [ 0.547538] ACPI: IRQ 4 override to edge(!), high(!)1492worker1 # [ 0.533434] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01493worker2 # [ 0.548504] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1494lb # [ 0.525737] clocksource: Switched to clocksource kvm-clock1495client # [ 0.550839] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1496worker1 # [ 0.533711] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1497worker2 # [ 0.550060] pnp: PnP ACPI: found 5 devices1498lb # [ 0.527407] VFS: Disk quotas dquot_6.6.01499lb # [ 0.529945] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1500worker1 # [ 0.536796] clocksource: Switched to clocksource kvm-clock1501lb # [ 0.531323] pnp: PnP ACPI init1502lb # [ 0.532174] ACPI: IRQ 4 override to edge(!), high(!)1503lb # [ 0.533177] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1504worker1 # [ 0.540290] VFS: Disk quotas dquot_6.6.01505lb # [ 0.534725] pnp: PnP ACPI: found 5 devices1506worker1 # [ 0.540985] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1507worker2 # [ 0.557663] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1508worker1 # [ 0.542303] pnp: PnP ACPI init1509worker2 # [ 0.559099] clocksource: Switched to clocksource acpi_pm1510worker1 # [ 0.543067] ACPI: IRQ 4 override to edge(!), high(!)1511worker2 # [ 0.560112] NET: Registered PF_INET protocol family1512worker1 # [ 0.544024] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1513worker2 # [ 0.561348] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1514worker1 # [ 0.545587] pnp: PnP ACPI: found 5 devices1515client # [ 0.566953] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1516lb # [ 0.542257] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1517client # [ 0.568540] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1518lb # [ 0.543776] clocksource: Switched to clocksource acpi_pm1519lb # [ 0.544763] NET: Registered PF_INET protocol family1520client # [ 0.569926] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1521lb # [ 0.545983] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1522client # [ 0.571370] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1523worker1 # [ 0.553213] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1524client # [ 0.572683] TCP: Hash tables configured (established 16384 bind 16384)1525worker1 # [ 0.554688] clocksource: Switched to clocksource acpi_pm1526client # [ 0.573936] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1527worker1 # [ 0.555715] NET: Registered PF_INET protocol family1528client # [ 0.575353] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1529worker1 # [ 0.556930] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1530client # [ 0.576555] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1531client # [ 0.577846] NET: Registered PF_UNIX/PF_LOCAL protocol family1532client # [ 0.578837] NET: Registered PF_XDP protocol family1533client # [ 0.579676] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1534client # [ 0.580757] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1535worker2 # [ 0.578472] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1536client # [ 0.581782] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1537worker2 # [ 0.580114] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1538client # [ 0.582902] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1539worker2 # [ 0.581447] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1540client # [ 0.583998] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1541worker2 # [ 0.582827] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1542client # [ 0.585113] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1543worker2 # [ 0.584121] TCP: Hash tables configured (established 16384 bind 16384)1544client # [ 0.586886] ACPI: \_SB_.GSIA: Enabled at IRQ 161545worker2 # [ 0.585305] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1546lb # [ 0.562719] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1547worker2 # [ 0.586637] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1548client # [ 0.589236] ACPI: \_SB_.GSIB: Enabled at IRQ 171549lb # [ 0.564293] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1550worker2 # [ 0.587744] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1551worker2 # [ 0.589009] NET: Registered PF_UNIX/PF_LOCAL protocol family1552lb # [ 0.565646] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1553client # [ 0.591651] ACPI: \_SB_.GSIC: Enabled at IRQ 181554worker2 # [ 0.589986] NET: Registered PF_XDP protocol family1555lb # [ 0.566964] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1556worker2 # [ 0.590826] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1557worker1 # [ 0.573666] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1558lb # [ 0.568185] TCP: Hash tables configured (established 16384 bind 16384)1559client # [ 0.593692] ACPI: \_SB_.GSID: Enabled at IRQ 191560worker2 # [ 0.591798] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1561worker1 # [ 0.575192] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1562lb # [ 0.569266] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1563worker2 # [ 0.592762] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1564client # [ 0.595422] PCI: CLS 0 bytes, default 641565worker1 # [ 0.576497] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1566lb # [ 0.570522] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1567worker2 # [ 0.593800] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1568worker1 # [ 0.577810] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1569client # [ 0.596400] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1570lb # [ 0.571574] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1571worker2 # [ 0.594840] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1572worker1 # [ 0.579017] TCP: Hash tables configured (established 16384 bind 16384)1573lb # [ 0.572765] NET: Registered PF_UNIX/PF_LOCAL protocol family1574client # [ 0.598102] Trying to unpack rootfs image as initramfs...1575worker2 # [ 0.595927] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1576lb # [ 0.573689] NET: Registered PF_XDP protocol family1577worker1 # [ 0.580109] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1578worker2 # [ 0.597650] ACPI: \_SB_.GSIA: Enabled at IRQ 161579lb # [ 0.574536] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1580worker1 # [ 0.581414] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1581lb # [ 0.575534] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1582worker1 # [ 0.582523] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1583worker2 # [ 0.599987] ACPI: \_SB_.GSIB: Enabled at IRQ 171584lb # [ 0.576563] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1585worker1 # [ 0.583758] NET: Registered PF_UNIX/PF_LOCAL protocol family1586lb # [ 0.577687] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1587worker1 # [ 0.584738] NET: Registered PF_XDP protocol family1588worker2 # [ 0.602311] ACPI: \_SB_.GSIC: Enabled at IRQ 181589worker1 # [ 0.585622] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1590lb # [ 0.578816] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1591lb # [ 0.579931] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1592worker1 # [ 0.586627] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1593worker2 # [ 0.604322] ACPI: \_SB_.GSID: Enabled at IRQ 191594worker1 # [ 0.587651] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1595lb # [ 0.581599] ACPI: \_SB_.GSIA: Enabled at IRQ 161596worker1 # [ 0.588809] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1597worker2 # [ 0.606077] PCI: CLS 0 bytes, default 641598worker1 # [ 0.589995] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1599lb # [ 0.584151] ACPI: \_SB_.GSIB: Enabled at IRQ 171600worker2 # [ 0.606983] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1601worker1 # [ 0.591085] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1602worker2 # [ 0.608767] Trying to unpack rootfs image as initramfs...1603lb # [ 0.586031] ACPI: \_SB_.GSIC: Enabled at IRQ 181604worker1 # [ 0.592865] ACPI: \_SB_.GSIA: Enabled at IRQ 161605lb # [ 0.587992] ACPI: \_SB_.GSID: Enabled at IRQ 191606worker1 # [ 0.595497] ACPI: \_SB_.GSIB: Enabled at IRQ 171607lb # [ 0.589704] PCI: CLS 0 bytes, default 641608lb # [ 0.590721] Trying to unpack rootfs image as initramfs...1609worker1 # [ 0.597448] ACPI: \_SB_.GSIC: Enabled at IRQ 181610worker1 # [ 0.599380] ACPI: \_SB_.GSID: Enabled at IRQ 191611worker1 # [ 0.601037] PCI: CLS 0 bytes, default 641612worker1 # [ 0.601934] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1613worker1 # [ 0.603693] Trying to unpack rootfs image as initramfs...1614lb # [ 0.597650] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1615client # [ 0.641232] Initialise system trusted keyrings1616client # [ 0.644803] workingset: timestamp_bits=40 max_order=19 bucket_order=01617worker2 # [ 0.651703] Initialise system trusted keyrings1618worker2 # [ 0.656104] workingset: timestamp_bits=40 max_order=19 bucket_order=01619lb # [ 0.636429] Initialise system trusted keyrings1620client # [ 0.664020] Key type asymmetric registered1621lb # [ 0.639684] workingset: timestamp_bits=40 max_order=19 bucket_order=01622worker1 # [ 0.647059] Initialise system trusted keyrings1623client # [ 0.666757] Asymmetric key parser 'x509' registered1624client # [ 0.667637] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1625worker1 # [ 0.650307] workingset: timestamp_bits=40 max_order=19 bucket_order=01626client # [ 0.670861] io scheduler mq-deadline registered1627client # [ 0.671645] io scheduler kyber registered1628client # [ 0.674334] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1629client # [ 0.675668] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1630worker2 # [ 0.675485] Key type asymmetric registered1631client # [ 0.681517] Linux agpgart interface v0.1031632worker2 # [ 0.680062] Asymmetric key parser 'x509' registered1633client # [ 0.682324] ACPI: bus type drm_connector registered1634worker2 # [ 0.680896] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1635lb # [ 0.659976] Key type asymmetric registered1636client # [ 0.685171] usbcore: registered new interface driver usbserial_generic1637worker2 # [ 0.684199] io scheduler mq-deadline registered1638client # [ 0.686309] usbserial: USB Serial support registered for generic1639worker2 # [ 0.684982] io scheduler kyber registered1640lb # [ 0.662637] Asymmetric key parser 'x509' registered1641lb # [ 0.663526] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1642client # [ 0.688766] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1643worker2 # [ 0.687661] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1644client # [ 0.688766] Please enable it if your BIOS has the CPPC option.1645worker1 # [ 0.671640] Key type asymmetric registered1646client # [ 0.691015] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1647worker2 # [ 0.688983] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1648lb # [ 0.666774] io scheduler mq-deadline registered1649lb # [ 0.667586] io scheduler kyber registered1650worker1 # [ 0.674268] Asymmetric key parser 'x509' registered1651worker1 # [ 0.675107] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1652client # [ 0.694879] drop_monitor: Initializing network drop monitor service1653client # [ 0.696091] NET: Registered PF_INET6 protocol family1654worker2 # [ 0.694882] Linux agpgart interface v0.1031655worker1 # [ 0.678392] io scheduler mq-deadline registered1656lb # [ 0.671814] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1657worker2 # [ 0.695654] ACPI: bus type drm_connector registered1658worker1 # [ 0.679130] io scheduler kyber registered1659lb # [ 0.673079] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1660client # [ 0.699241] Segment Routing with IPv61661worker2 # [ 0.698503] usbcore: registered new interface driver usbserial_generic1662client # [ 0.701792] In-situ OAM (IOAM) with IPv61663worker1 # [ 0.682777] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1664worker2 # [ 0.699666] usbserial: USB Serial support registered for generic1665client # [ 0.702863] IPI shorthand broadcast: enabled1666worker1 # [ 0.684035] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1667lb # [ 0.678400] Linux agpgart interface v0.1031668lb # [ 0.679213] ACPI: bus type drm_connector registered1669worker1 # [ 0.688074] Linux agpgart interface v0.1031670worker2 # [ 0.704057] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1671worker2 # [ 0.704057] Please enable it if your BIOS has the CPPC option.1672worker1 # [ 0.688828] ACPI: bus type drm_connector registered1673worker2 # [ 0.706375] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1674lb # [ 0.683783] usbcore: registered new interface driver usbserial_generic1675lb # [ 0.684919] usbserial: USB Serial support registered for generic1676worker2 # [ 0.709191] drop_monitor: Initializing network drop monitor service1677client # [ 0.711321] sched_clock: Marking stable (575019328, 135721951)->(788108579, -77367300)1678worker1 # [ 0.692745] usbcore: registered new interface driver usbserial_generic1679worker2 # [ 0.710388] NET: Registered PF_INET6 protocol family1680worker1 # [ 0.693832] usbserial: USB Serial support registered for generic1681lb # [ 0.687638] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1682lb # [ 0.687638] Please enable it if your BIOS has the CPPC option.1683lb # [ 0.689978] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1684worker1 # [ 0.696267] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1685worker1 # [ 0.696267] Please enable it if your BIOS has the CPPC option.1686client # [ 0.716874] registered taskstats version 11687worker2 # [ 0.715374] Segment Routing with IPv61688worker1 # [ 0.698539] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1689client # [ 0.717868] Loading compiled-in X.509 certificates1690worker2 # [ 0.716135] In-situ OAM (IOAM) with IPv61691lb # [ 0.694782] drop_monitor: Initializing network drop monitor service1692lb # [ 0.696020] NET: Registered PF_INET6 protocol family1693worker2 # [ 0.719341] IPI shorthand broadcast: enabled1694worker1 # [ 0.702398] drop_monitor: Initializing network drop monitor service1695worker1 # [ 0.703581] NET: Registered PF_INET6 protocol family1696lb # [ 0.699136] Segment Routing with IPv61697lb # [ 0.701661] In-situ OAM (IOAM) with IPv61698worker1 # [ 0.708705] Segment Routing with IPv61699lb # [ 0.702734] IPI shorthand broadcast: enabled1700worker1 # [ 0.709468] In-situ OAM (IOAM) with IPv61701worker2 # [ 0.726661] sched_clock: Marking stable (591015177, 135026954)->(801732991, -75690860)1702worker1 # [ 0.712546] IPI shorthand broadcast: enabled1703worker2 # [ 0.732129] registered taskstats version 11704client # [ 0.734274] Demotion targets for Node 0: null1705worker2 # [ 0.733081] Loading compiled-in X.509 certificates1706lb # [ 0.710949] sched_clock: Marking stable (578015080, 132604307)->(786725496, -76106109)1707client # [ 0.737787] Key type .fscrypt registered1708client # [ 0.738511] Key type fscrypt-provisioning registered1709client # [ 0.739499] ima: No TPM chip found, activating TPM-bypass!1710worker1 # [ 0.720977] sched_clock: Marking stable (587014716, 133231753)->(794660187, -74413718)1711lb # [ 0.715867] registered taskstats version 11712client # [ 0.741753] ima: Allocated hash algorithm: sha11713lb # [ 0.716887] Loading compiled-in X.509 certificates1714client # [ 0.742575] ima: No architecture policies found1715worker1 # [ 0.724451] registered taskstats version 11716worker1 # [ 0.725434] Loading compiled-in X.509 certificates1717client # [ 0.746759] PM: Magic number: 10:797:3821718client # [ 0.747529] container PNP0A06:00: hash matches1719client # [ 0.748364] acpi PNP0A06:00: hash matches1720client # [ 0.752590] RAS: Correctable Errors collector initialized.1721worker2 # [ 0.752059] Demotion targets for Node 0: null1722worker2 # [ 0.752988] Key type .fscrypt registered1723worker2 # [ 0.753694] Key type fscrypt-provisioning registered1724worker2 # [ 0.754694] ima: No TPM chip found, activating TPM-bypass!1725client # [ 0.759231] clk: Disabling unused clocks1726lb # [ 0.735637] Demotion targets for Node 0: null1727worker2 # [ 0.759060] ima: Allocated hash algorithm: sha11728lb # [ 0.736504] Key type .fscrypt registered1729worker2 # [ 0.759812] ima: No architecture policies found1730worker1 # [ 0.744261] Demotion targets for Node 0: null1731client # [ 0.763770] PM: genpd: Disabling unused power domains1732worker1 # [ 0.745155] Key type .fscrypt registered1733worker2 # [ 0.762053] PM: Magic number: 10:797:3821734worker2 # [ 0.762758] container PNP0A06:00: hash matches1735lb # [ 0.739635] Key type fscrypt-provisioning registered1736worker2 # [ 0.763552] acpi PNP0A06:00: hash matches1737lb # [ 0.740518] ima: No TPM chip found, activating TPM-bypass!1738lb # [ 0.741478] ima: Allocated hash algorithm: sha11739worker1 # [ 0.748251] Key type fscrypt-provisioning registered1740lb # [ 0.742287] ima: No architecture policies found1741worker1 # [ 0.749177] ima: No TPM chip found, activating TPM-bypass!1742worker1 # [ 0.750100] ima: Allocated hash algorithm: sha11743worker2 # [ 0.766932] RAS: Correctable Errors collector initialized.1744worker1 # [ 0.750905] ima: No architecture policies found1745lb # [ 0.744632] PM: Magic number: 10:797:3821746lb # [ 0.745349] container PNP0A06:00: hash matches1747lb # [ 0.746143] acpi PNP0A06:00: hash matches1748worker1 # [ 0.753258] PM: Magic number: 10:797:3821749worker1 # [ 0.754006] container PNP0A06:00: hash matches1750worker1 # [ 0.754820] acpi PNP0A06:00: hash matches1751lb # [ 0.749484] RAS: Correctable Errors collector initialized.1752worker1 # [ 0.758078] RAS: Correctable Errors collector initialized.1753worker2 # [ 0.775620] clk: Disabling unused clocks1754worker2 # [ 0.780059] PM: genpd: Disabling unused power domains1755lb # [ 0.758250] clk: Disabling unused clocks1756worker1 # [ 0.766755] clk: Disabling unused clocks1757lb # [ 0.762636] PM: genpd: Disabling unused power domains1758worker1 # [ 0.771264] PM: genpd: Disabling unused power domains1759client # [ 0.927399] Freeing initrd memory: 29056K1760client # [ 0.930591] Freeing unused decrypted memory: 2028K1761client # [ 0.933254] Freeing unused kernel image (initmem) memory: 3652K1762client # [ 0.934382] Write protecting the kernel read-only data: 32768k1763client # [ 0.936303] Freeing unused kernel image (text/rodata gap) memory: 1184K1764client # [ 0.937845] Freeing unused kernel image (rodata/data gap) memory: 720K1765worker2 # [ 0.946585] Freeing initrd memory: 29060K1766lb # [ 0.924125] Freeing initrd memory: 29064K1767worker2 # [ 0.949909] Freeing unused decrypted memory: 2028K1768lb # [ 0.927333] Freeing unused decrypted memory: 2028K1769worker1 # [ 0.934305] Freeing initrd memory: 29056K1770worker2 # [ 0.952667] Freeing unused kernel image (initmem) memory: 3652K1771lb # [ 0.929963] Freeing unused kernel image (initmem) memory: 3652K1772worker2 # [ 0.953794] Write protecting the kernel read-only data: 32768k1773worker1 # [ 0.937477] Freeing unused decrypted memory: 2028K1774lb # [ 0.931074] Write protecting the kernel read-only data: 32768k1775worker2 # [ 0.955665] Freeing unused kernel image (text/rodata gap) memory: 1184K1776lb # [ 0.932957] Freeing unused kernel image (text/rodata gap) memory: 1184K1777worker1 # [ 0.940097] Freeing unused kernel image (initmem) memory: 3652K1778worker2 # [ 0.957254] Freeing unused kernel image (rodata/data gap) memory: 720K1779lb # [ 0.934478] Freeing unused kernel image (rodata/data gap) memory: 720K1780worker1 # [ 0.941139] Write protecting the kernel read-only data: 32768k1781worker1 # [ 0.943028] Freeing unused kernel image (text/rodata gap) memory: 1184K1782worker1 # [ 0.944584] Freeing unused kernel image (rodata/data gap) memory: 720K1783client # [ 0.988599] x86/mm: Checked W+X mappings: passed, no W+X pages found.1784client # [ 0.989714] Run /init as init process1785client # [ 1.000652] systemd[1]: Inserted module 'autofs4'1786worker2 # [ 1.008331] x86/mm: Checked W+X mappings: passed, no W+X pages found.1787lb # [ 0.985145] x86/mm: Checked W+X mappings: passed, no W+X pages found.1788worker2 # [ 1.009421] Run /init as init process1789lb # [ 0.986236] Run /init as init process1790worker1 # [ 0.995924] x86/mm: Checked W+X mappings: passed, no W+X pages found.1791worker1 # [ 0.997042] Run /init as init process1792client # [ 1.020463] fuse: init (API version 7.45)1793lb # [ 0.996794] systemd[1]: Inserted module 'autofs4'1794worker2 # [ 1.020390] systemd[1]: Inserted module 'autofs4'1795client # [ 1.027924] ACPI: \_SB_.GSIG: Enabled at IRQ 221796worker1 # [ 1.009857] systemd[1]: Inserted module 'autofs4'1797client # [ 1.030384] ACPI: \_SB_.GSIH: Enabled at IRQ 231798client # [ 1.033313] ACPI: \_SB_.GSIE: Enabled at IRQ 201799client # [ 1.035285] ACPI: \_SB_.GSIF: Enabled at IRQ 211800client # [ 1.040396] virtiofs virtio5: discovered new tag: nix-store1801lb # [ 1.016516] fuse: init (API version 7.45)1802worker2 # [ 1.040261] fuse: init (API version 7.45)1803client # [ 1.042062] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1804worker1 # [ 1.030301] fuse: init (API version 7.45)1805lb # [ 1.023837] ACPI: \_SB_.GSIG: Enabled at IRQ 221806worker2 # [ 1.047655] ACPI: \_SB_.GSIG: Enabled at IRQ 221807client # [ 1.050745] virtiofs virtio6: discovered new tag: shared1808lb # [ 1.026192] ACPI: \_SB_.GSIH: Enabled at IRQ 231809worker2 # [ 1.050389] ACPI: \_SB_.GSIH: Enabled at IRQ 231810client # [ 1.052387] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1811lb # [ 1.029220] ACPI: \_SB_.GSIE: Enabled at IRQ 201812worker2 # [ 1.053561] ACPI: \_SB_.GSIE: Enabled at IRQ 201813client # [ 1.056101] virtiofs virtio7: discovered new tag: xchg1814lb # [ 1.031211] ACPI: \_SB_.GSIF: Enabled at IRQ 211815worker1 # [ 1.038010] ACPI: \_SB_.GSIG: Enabled at IRQ 221816client # [ 1.057577] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1817worker2 # [ 1.055809] ACPI: \_SB_.GSIF: Enabled at IRQ 211818worker1 # [ 1.040914] ACPI: \_SB_.GSIH: Enabled at IRQ 231819lb # [ 1.036648] virtiofs virtio5: discovered new tag: nix-store1820worker2 # [ 1.060960] virtiofs virtio5: discovered new tag: nix-store1821worker1 # [ 1.044468] ACPI: \_SB_.GSIE: Enabled at IRQ 201822lb # [ 1.038215] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1823worker2 # [ 1.062539] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1824worker1 # [ 1.046854] ACPI: \_SB_.GSIF: Enabled at IRQ 211825lb # [ 1.044989] virtiofs virtio6: discovered new tag: shared1826worker1 # [ 1.052541] virtiofs virtio5: discovered new tag: nix-store1827lb # [ 1.046789] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1828worker2 # [ 1.070695] virtiofs virtio6: discovered new tag: shared1829worker1 # [ 1.054134] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1830worker2 # [ 1.072359] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1831lb # [ 1.049974] virtiofs virtio7: discovered new tag: xchg1832lb # [ 1.051471] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1833worker2 # [ 1.075806] virtiofs virtio7: discovered new tag: xchg1834worker2 # [ 1.077343] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1835worker1 # [ 1.061846] virtiofs virtio6: discovered new tag: shared1836worker1 # [ 1.063433] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1837client # [ 1.082933] systemd[1]: Successfully made /usr/ read-only.1838worker1 # [ 1.066858] virtiofs virtio7: discovered new tag: xchg1839worker1 # [ 1.068424] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1840lb # [ 1.075406] systemd[1]: Successfully made /usr/ read-only.1841worker2 # [ 1.098874] systemd[1]: Successfully made /usr/ read-only.1842worker1 # [ 1.091081] systemd[1]: Successfully made /usr/ read-only.1843client # [ 1.418703] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1844client # [ 1.423943] systemd[1]: Detected virtualization kvm.1845client # [ 1.424842] systemd[1]: Detected architecture x86-64.1846client # [ 1.425706] systemd[1]: Running in initrd.1847client # [ 1.426961] systemd[1]: Initializing machine ID from random generator.1848client # [ 1.428234] systemd[1]: Hostname set to <client>.1849lb # [ 1.412009] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1850worker2 # [ 1.435671] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1851lb # [ 1.424121] systemd[1]: Detected virtualization kvm.1852worker2 # [ 1.447790] systemd[1]: Detected virtualization kvm.1853lb # [ 1.426253] systemd[1]: Detected architecture x86-64.1854worker2 # [ 1.449939] systemd[1]: Detected architecture x86-64.1855lb # [ 1.428456] systemd[1]: Running in initrd.1856worker2 # [ 1.452145] systemd[1]: Running in initrd.1857worker1 # [ 1.427565] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1858lb # [ 1.430949] systemd[1]: Initializing machine ID from random generator.1859worker2 # [ 1.454625] systemd[1]: Initializing machine ID from random generator.1860worker1 # [ 1.439638] systemd[1]: Detected virtualization kvm.1861lb # [ 1.433839] systemd[1]: Hostname set to <lb>.1862worker2 # [ 1.457500] systemd[1]: Hostname set to <worker2>.1863worker1 # [ 1.441768] systemd[1]: Detected architecture x86-64.1864worker1 # [ 1.443928] systemd[1]: Running in initrd.1865worker1 # [ 1.446394] systemd[1]: Initializing machine ID from random generator.1866worker1 # [ 1.449294] systemd[1]: Hostname set to <worker1>.1867client # [ 1.623490] systemd[1]: bpf-restrict-fs: LSM BPF program attached1868client # [ 1.663188] systemd[1]: Queued start job for default target Initrd Default Target.1869client # [ 1.667119] systemd[1]: Created slice Slice /system/modprobe.1870client # [ 1.668359] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1871client # [ 1.669930] systemd[1]: Expecting device /dev/disk/by-label/nixos...1872client # [ 1.671128] systemd[1]: Reached target Path Units.1873client # [ 1.672053] systemd[1]: Reached target Slice Units.1874client # [ 1.672945] systemd[1]: Reached target Swaps.1875client # [ 1.673795] systemd[1]: Reached target Timer Units.1876client # [ 1.674794] systemd[1]: Listening on D-Bus System Message Bus Socket.1877client # [ 1.676004] systemd[1]: Listening on Journal Socket (/dev/log).1878client # [ 1.677229] systemd[1]: Listening on Journal Sockets.1879client # [ 1.678278] systemd[1]: Listening on udev Control Socket.1880client # [ 1.679350] systemd[1]: Listening on udev Kernel Socket.1881client # [ 1.680356] systemd[1]: Reached target Socket Units.1882worker2 # [ 1.678297] systemd[1]: bpf-restrict-fs: LSM BPF program attached1883lb # [ 1.658674] systemd[1]: bpf-restrict-fs: LSM BPF program attached1884client # [ 1.682333] systemd[1]: Starting Create List of Static Device Nodes...1885worker1 # [ 1.674290] systemd[1]: bpf-restrict-fs: LSM BPF program attached1886client # [ 1.701996] systemd[1]: Starting Load Kernel Module configfs...1887client # [ 1.713414] systemd[1]: Starting Journal Service...1888client # [ 1.721827] systemd[1]: Starting Load Kernel Modules...1889client # [ 1.724828] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1890client # [ 1.730933] systemd[1]: Starting Coldplug All udev Devices...1891worker2 # [ 1.731087] systemd[1]: Queued start job for default target Initrd Default Target.1892lb # [ 1.710586] systemd[1]: Queued start job for default target Initrd Default Target.1893worker2 # [ 1.735339] systemd[1]: Created slice Slice /system/modprobe.1894worker2 # [ 1.736652] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1895lb # [ 1.714883] systemd[1]: Created slice Slice /system/modprobe.1896worker2 # [ 1.738216] systemd[1]: Expecting device /dev/disk/by-label/nixos...1897worker2 # [ 1.739344] systemd[1]: Reached target Path Units.1898lb # [ 1.716065] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1899worker2 # [ 1.740216] systemd[1]: Reached target Slice Units.1900lb # [ 1.717494] systemd[1]: Expecting device /dev/disk/by-label/nixos...1901worker2 # [ 1.741117] systemd[1]: Reached target Swaps.1902lb # [ 1.718643] systemd[1]: Reached target Path Units.1903worker2 # [ 1.741902] systemd[1]: Reached target Timer Units.1904lb # [ 1.719517] systemd[1]: Reached target Slice Units.1905client # [ 1.744847] systemd[1]: Finished Create List of Static Device Nodes.1906worker2 # [ 1.742988] systemd[1]: Listening on D-Bus System Message Bus Socket.1907lb # [ 1.720422] systemd[1]: Reached target Swaps.1908worker2 # [ 1.744220] systemd[1]: Listening on Journal Socket (/dev/log).1909lb # [ 1.721208] systemd[1]: Reached target Timer Units.1910worker1 # [ 1.727597] systemd[1]: Queued start job for default target Initrd Default Target.1911worker2 # [ 1.745381] systemd[1]: Listening on Journal Sockets.1912lb # [ 1.722146] systemd[1]: Listening on D-Bus System Message Bus Socket.1913worker2 # [ 1.746418] systemd[1]: Listening on udev Control Socket.1914lb # [ 1.723352] systemd[1]: Listening on Journal Socket (/dev/log).1915worker2 # [ 1.747493] systemd[1]: Listening on udev Kernel Socket.1916lb # [ 1.724533] systemd[1]: Listening on Journal Sockets.1917worker1 # [ 1.731560] systemd[1]: Created slice Slice /system/modprobe.1918worker2 # [ 1.748467] systemd[1]: Reached target Socket Units.1919lb # [ 1.725500] systemd[1]: Listening on udev Control Socket.1920lb # [ 1.726494] systemd[1]: Listening on udev Kernel Socket.1921worker1 # [ 1.732741] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1922client # [ 1.752384] systemd[1]: modprobe@configfs.service: Deactivated successfully.1923lb # [ 1.727516] systemd[1]: Reached target Socket Units.1924worker1 # [ 1.734180] systemd[1]: Expecting device /dev/disk/by-label/nixos...1925worker1 # [ 1.735300] systemd[1]: Reached target Path Units.1926worker1 # [ 1.736165] systemd[1]: Reached target Slice Units.1927worker1 # [ 1.737082] systemd[1]: Reached target Swaps.1928worker1 # [ 1.737898] systemd[1]: Reached target Timer Units.1929worker1 # [ 1.738890] systemd[1]: Listening on D-Bus System Message Bus Socket.1930client # [ 1.758803] systemd[1]: Finished Load Kernel Module configfs.1931worker1 # [ 1.740137] systemd[1]: Listening on Journal Socket (/dev/log).1932worker1 # [ 1.741371] systemd[1]: Listening on Journal Sockets.1933worker1 # [ 1.742444] systemd[1]: Listening on udev Control Socket.1934worker1 # [ 1.743508] systemd[1]: Listening on udev Kernel Socket.1935worker1 # [ 1.744514] systemd[1]: Reached target Socket Units.1936client # [ 1.763213] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1937worker2 # [ 1.750238] systemd[1]: Starting Create List of Static Device Nodes...1938client # [ 1.765364] systemd-journald[66]: Collecting audit messages is disabled.1939lb # [ 1.729219] systemd[1]: Starting Create List of Static Device Nodes...1940worker2 # [ 1.769613] systemd[1]: Starting Load Kernel Module configfs...1941lb # [ 1.748222] systemd[1]: Starting Load Kernel Module configfs...1942client # [ 1.775815] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1943worker1 # [ 1.746308] systemd[1]: Starting Create List of Static Device Nodes...1944worker2 # [ 1.777976] systemd[1]: Starting Journal Service...1945lb # [ 1.756479] systemd[1]: Starting Journal Service...1946worker1 # [ 1.765457] systemd[1]: Starting Load Kernel Module configfs...1947client # [ 1.786776] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1948worker2 # [ 1.787138] systemd[1]: Starting Load Kernel Modules...1949lb # [ 1.764059] systemd[1]: Starting Load Kernel Modules...1950worker2 # [ 1.789151] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1951lb # [ 1.767692] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1952worker1 # [ 1.775867] systemd[1]: Starting Journal Service...1953client # [ 1.795779] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1954lb # [ 1.772687] systemd[1]: Starting Coldplug All udev Devices...1955worker2 # [ 1.797258] systemd[1]: Starting Coldplug All udev Devices...1956worker1 # [ 1.783323] systemd[1]: Starting Load Kernel Modules...1957worker1 # [ 1.786352] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1958lb # [ 1.786040] systemd[1]: Finished Create List of Static Device Nodes.1959worker1 # [ 1.792443] systemd[1]: Starting Coldplug All udev Devices...1960worker2 # [ 1.809443] systemd[1]: Finished Create List of Static Device Nodes.1961lb # [ 1.790251] systemd[1]: modprobe@configfs.service: Deactivated successfully.1962worker2 # [ 1.815651] systemd[1]: modprobe@configfs.service: Deactivated successfully.1963client # [ 1.818358] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1964worker1 # [ 1.805676] systemd[1]: Finished Create List of Static Device Nodes.1965lb # [ 1.799363] systemd[1]: Finished Load Kernel Module configfs.1966worker2 # [ 1.822578] systemd-journald[66]: Collecting audit messages is disabled.1967worker2 # [ 1.824177] systemd[1]: Finished Load Kernel Module configfs.1968client # [ 1.827978] systemd[1]: Starting Create Static Device Nodes in /dev...1969worker1 # [ 1.811824] systemd[1]: modprobe@configfs.service: Deactivated successfully.1970lb # [ 1.805065] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1971worker2 # [ 1.829454] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1972lb # [ 1.809077] systemd-journald[66]: Collecting audit messages is disabled.1973worker1 # [ 1.819614] systemd[1]: Finished Load Kernel Module configfs.1974worker1 # [ 1.820836] systemd-journald[66]: Collecting audit messages is disabled.1975client # [ 1.841846] systemd[1]: Finished Load Kernel Modules.1976lb # [ 1.817687] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1977worker2 # [ 1.841112] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1978worker1 # [ 1.825679] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1979lb # [ 1.821470] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1980client # [ 1.849457] systemd[1]: Starting Apply Kernel Variables...1981worker2 # [ 1.850151] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1982worker1 # [ 1.837314] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1983lb # [ 1.834800] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1984worker2 # [ 1.861362] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1985worker1 # [ 1.849019] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1986client # [ 1.871826] systemd[1]: Finished Create Static Device Nodes in /dev.1987client # [ 1.876964] systemd[1]: Reached target Preparation for Local File Systems.1988worker1 # [ 1.858400] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1989client # [ 1.882855] systemd[1]: Reached target Local File Systems.1990lb # [ 1.858168] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1991worker2 # [ 1.882127] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1992client # [ 1.753365] systemd-modules-load[67]: Inserted module 'dm_mod'[ 1.890408] systemd[1]: Starting Rule-based Manager for Device Events and Files...1993client # 1994lb # [ 1.866809] systemd[1]: Starting Create Static Device Nodes in /dev...1995client # [ 1.757318] systemd-modules-load[67]: Inserted module 'virtio_balloon'1996worker2 # [ 1.891228] systemd[1]: Starting Create Static Device Nodes in /dev...1997client # [ 1.758318] systemd-modules-load[67]: Inserted module 'virtio_gpu'1998worker1 # [ 1.879788] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1999client # [ 1.899976] systemd[1]: Started Journal Service.2000lb # [ 1.877685] systemd[1]: Finished Load Kernel Modules.
loading 2,069 hidden lines…
4070lb # [ 15.305203] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default4071lb # [ 15.306656] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder4072lb # [ 15.308704] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log4073lb # [ 15.314660] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default4074lb # [ 15.317457] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert4075lb # [ 15.320082] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port4076lb # [ 15.321669] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter4077lb # [ 15.323457] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata4078lb # [ 15.325378] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection4079lb # [ 15.327105] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter4080lb # [ 15.328961] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san4081lb # [ 15.334984] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service4082lb # [ 15.339214] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header4083lb # [ 15.341455] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed4084lb # [ 15.345107] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name4085lb # [ 15.347298] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls4086lb # [ 15.351327] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http14087lb # [ 15.353244] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default4088lb # [ 15.354961] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso4089lb # [ 15.356775] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable4090lb # [ 15.358660] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4091lb # [ 15.360569] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher4092lb # [ 15.362337] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy4093lb # [ 15.373237] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override4094lb # [ 15.376648] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation4095lb # [ 15.378526] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua4096lb # [ 15.380126] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy4097lb # [ 15.382494] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule4098lb # [ 15.384929] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd4099lb # [ 15.389322] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts4100lb # [ 15.391939] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action4101lb # [ 15.393898] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based4102lb # [ 15.395409] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest4103lb # [ 15.400082] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router4104lb # [ 15.424888] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router4105lb # [ 15.427535] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin4106lb # [ 15.431303] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config4107lb # [ 15.434394] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory4108lb # [ 15.436680] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory4109lb # [ 15.439304] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name4110lb # [ 15.441363] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle4111lb # [ 15.443247] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default4112lb # [ 15.445054] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state4113lb # [ 15.448977] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality4114lb # [ 15.455264] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit4115lb # [ 15.457591] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default4116lb # [ 15.459375] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor4117lb # [ 15.461676] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based4118lb # [ 15.465360] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default4119lb # [ 15.468135] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default4120lb # [ 15.469781] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities4121lb # [ 15.471531] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4122lb # [ 15.473779] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic4123lb # [ 15.475363] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian24124lb # [ 15.476886] envoy[916]: [2026-09-21 13:21:17.507][916][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme4125lb # [ 15.479684] envoy[916]: [2026-09-21 13:21:17.562][916][info][main] [source/server/server.cc:531] HTTP header map info:4126lb # [ 15.481096] envoy[916]: [2026-09-21 13:21:17.602][916][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id4127lb # [ 15.495295] envoy[916]: [2026-09-21 13:21:17.602][916][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:4128lb # [ 15.496694] envoy[916]: [2026-09-21 13:21:17.602][916][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id4129lb # [ 15.504105] envoy[916]: [2026-09-21 13:21:17.602][916][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status4130lb # [ 15.513490] envoy[916]: [2026-09-21 13:21:17.865][916][info][main] [source/server/server.cc:968] runtime: {}4131lb # [ 15.516544] envoy[916]: [2026-09-21 13:21:17.868][916][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.4132lb # [ 15.518575] envoy[916]: [2026-09-21 13:21:17.870][916][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99014133lb # [ 15.521164] envoy[916]: [2026-09-21 13:21:17.873][916][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99014134lb # [ 15.523638] envoy[916]: [2026-09-21 13:21:17.876][916][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration4135lb # [ 15.525601] envoy[916]: [2026-09-21 13:21:17.877][916][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)4136lb # [ 15.527391] envoy[916]: [2026-09-21 13:21:17.877][916][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)4137lb # [ 15.550952] envoy[916]: [2026-09-21 13:21:17.903][916][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)4138worker2 # [ 15.575684] systemd[1]: Reached target Network is Online.4139lb # [ 15.557880] envoy[916]: [2026-09-21 13:21:17.910][916][info][config] [source/server/configuration_impl.cc:164] loading stats configuration4140worker1 # [ 15.564290] systemd[1]: Starting Nix worker-protocol over gRPC...4141lb # [ 15.559689] envoy[916]: [2026-09-21 13:21:17.912][916][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.4142worker2 # [ 15.582647] systemd[1]: Starting scheduler-token.service...4143lb # [ 15.562565] envoy[916]: [2026-09-21 13:21:17.914][916][info][main] [source/server/server.cc:1077] starting main dispatch loop4144worker2 # [ 15.645271] systemd[1]: scheduler-token.service: Deactivated successfully.4145worker2 # [ 15.648248] systemd[1]: Finished scheduler-token.service.4146worker2 # [ 15.650479] systemd[1]: scheduler-token.service: Consumed 38ms CPU time over 65ms wall clock time, 3.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4147worker2 # [ 15.655423] systemd[1]: Starting Nix worker-protocol over gRPC...4148worker1 # [ 15.689924] nix-grpc-daemon[725]: ts=2026-09-21T13:21:18Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4149worker1 # [ 15.698990] nix-grpc-daemon[725]: ts=2026-09-21T13:21:18Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4150worker1 # [ 15.701759] nix-grpc-daemon[725]: ts=2026-09-21T13:21:18Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14151worker1 # [ 15.712504] nix-grpc-daemon[725]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4152worker1 # [ 15.714090] nix-grpc-daemon[725]: E0000 00:00:1789996878.871470 725 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4153worker1 # [ 15.720025] nix-grpc-daemon[725]: E0000 00:00:1789996878.879084 725 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4154worker1 # [ 15.723078] nix-grpc-daemon[725]: ts=2026-09-21T13:21:18Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4155worker1 # [ 15.727098] systemd[1]: Started Nix worker-protocol over gRPC.4156worker2 # [ 15.748207] nix-grpc-daemon[783]: ts=2026-09-21T13:21:18Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4157worker1 # [ 15.741077] nix-grpc-daemon[725]: E0000 00:00:1789996878.900096 767 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4158worker2 # [ 15.758434] nix-grpc-daemon[783]: ts=2026-09-21T13:21:18Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4159worker2 # [ 15.761043] nix-grpc-daemon[783]: ts=2026-09-21T13:21:18Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14160worker2 # [ 15.773391] nix-grpc-daemon[783]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4161lb # [ 15.750991] envoy[916]: [2026-09-21 13:21:18.103][916][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization4162worker2 # [ 15.774928] nix-grpc-daemon[783]: E0000 00:00:1789996878.950041 783 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4163worker1 # [ 15.760377] nix-grpc-daemon[725]: E0000 00:00:1789996878.919620 767 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4164lb # [ 15.754904] envoy[916]: [2026-09-21 13:21:18.103][916][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized4165worker1 # [ 15.764208] nix-grpc-daemon[725]: E0000 00:00:1789996878.921706 767 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4166worker2 # [ 15.780511] nix-grpc-daemon[783]: E0000 00:00:1789996878.957483 783 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4167lb # [ 15.760201] envoy[916]: [2026-09-21 13:21:18.103][916][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager4168worker2 # [ 15.784095] nix-grpc-daemon[783]: ts=2026-09-21T13:21:18Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4169worker2 # [ 15.787630] systemd[1]: Started Nix worker-protocol over gRPC.4170worker1 # [ 15.776025] nix-grpc-daemon[725]: E0000 00:00:1789996878.935244 767 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4171lb # [ 15.768382] envoy[916]: [2026-09-21 13:21:18.120][916][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers4172worker2 # [ 15.801062] nix-grpc-daemon[783]: E0000 00:00:1789996878.977835 825 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4173worker1 # [ 15.802622] nix-grpc-daemon[725]: ts=2026-09-21T13:21:18Z level=info event=scheduler_connected addr=lb:500514174worker1 # [ 15.805217] nix-grpc-daemon[725]: ts=2026-09-21T13:21:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4175worker2 # [ 15.819719] nix-grpc-daemon[783]: E0000 00:00:1789996878.996633 825 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4176worker2 # [ 15.824062] nix-grpc-daemon[783]: E0000 00:00:1789996878.998832 825 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4177worker2 # [ 15.835533] nix-grpc-daemon[783]: E0000 00:00:1789996879.012405 825 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4178worker2 # [ 15.845355] nix-grpc-daemon[783]: ts=2026-09-21T13:21:19Z level=info event=scheduler_connected addr=lb:500514179worker2 # [ 15.847167] nix-grpc-daemon[783]: ts=2026-09-21T13:21:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4180worker1 # [ 16.006636] nix-grpc-daemon[725]: ts=2026-09-21T13:21:19Z level=info event=scheduler_connected addr=lb:500514181worker1 # [ 16.010811] nix-grpc-daemon[725]: ts=2026-09-21T13:21:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4182worker2 # [ 16.049159] nix-grpc-daemon[783]: ts=2026-09-21T13:21:19Z level=info event=scheduler_connected addr=lb:500514183worker2 # [ 16.053332] nix-grpc-daemon[783]: ts=2026-09-21T13:21:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4184worker1 # [ 16.211515] nix-grpc-daemon[725]: ts=2026-09-21T13:21:19Z level=info event=scheduler_connected addr=lb:500514185worker1 # [ 16.215566] nix-grpc-daemon[725]: ts=2026-09-21T13:21:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4186worker2 # [ 16.254483] nix-grpc-daemon[783]: ts=2026-09-21T13:21:19Z level=info event=scheduler_connected addr=lb:500514187worker2 # [ 16.258624] nix-grpc-daemon[783]: ts=2026-09-21T13:21:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4188worker1 # [ 16.364917] dhcpcd[693]: eth0: leased 10.0.2.15 for 86400 seconds4189worker1 # [ 16.368434] dhcpcd[693]: eth0: adding route to 10.0.2.0/244190worker1 # [ 16.371295] dhcpcd[693]: eth0: adding default route via 10.0.2.24191worker1 # [ 16.416353] nix-grpc-daemon[725]: ts=2026-09-21T13:21:19Z level=info event=scheduler_connected addr=lb:500514192worker1 # [ 16.420284] nix-grpc-daemon[725]: ts=2026-09-21T13:21:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4193worker2 # [ 16.459746] nix-grpc-daemon[783]: ts=2026-09-21T13:21:19Z level=info event=scheduler_connected addr=lb:500514194worker2 # [ 16.463850] nix-grpc-daemon[783]: ts=2026-09-21T13:21:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4195worker1 # [ 16.467861] systemd[1]: Started DHCP Client.4196worker1 # [ 16.469989] systemd[1]: Reached target Multi-User System.4197worker1 # [ 16.471122] systemd[1]: Startup finished in 868ms (kernel) + 4.590s (initrd) + 11.011s (userspace) = 16.470s.4198worker1 # [ 16.620803] nix-grpc-daemon[725]: ts=2026-09-21T13:21:19Z level=info event=scheduler_connected addr=lb:500514199worker1 # [ 16.625195] nix-grpc-daemon[725]: ts=2026-09-21T13:21:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4200worker2 # [ 16.665177] nix-grpc-daemon[783]: ts=2026-09-21T13:21:19Z level=info event=scheduler_connected addr=lb:500514201worker2 # [ 16.669083] nix-grpc-daemon[783]: ts=2026-09-21T13:21:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4202worker1 # [ 16.827387] nix-grpc-daemon[725]: ts=2026-09-21T13:21:19Z level=info event=scheduler_connected addr=lb:500514203worker1 # [ 16.830975] nix-grpc-daemon[725]: ts=2026-09-21T13:21:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4204worker2 # [ 16.871444] nix-grpc-daemon[783]: ts=2026-09-21T13:21:20Z level=info event=scheduler_connected addr=lb:500514205worker2 # [ 16.874638] nix-grpc-daemon[783]: ts=2026-09-21T13:21:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4206worker1 # [ 17.028329] nix-grpc-daemon[725]: ts=2026-09-21T13:21:20Z level=info event=scheduler_connected addr=lb:500514207worker1 # [ 17.032502] nix-grpc-daemon[725]: ts=2026-09-21T13:21:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4208worker2 # [ 17.072703] nix-grpc-daemon[783]: ts=2026-09-21T13:21:20Z level=info event=scheduler_connected addr=lb:500514209worker2 # [ 17.076454] nix-grpc-daemon[783]: ts=2026-09-21T13:21:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4210worker1 # [ 17.233496] nix-grpc-daemon[725]: ts=2026-09-21T13:21:20Z level=info event=scheduler_connected addr=lb:500514211worker1 # [ 17.237552] nix-grpc-daemon[725]: ts=2026-09-21T13:21:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4212worker2 # [ 17.277613] nix-grpc-daemon[783]: ts=2026-09-21T13:21:20Z level=info event=scheduler_connected addr=lb:500514213worker2 # [ 17.281463] nix-grpc-daemon[783]: ts=2026-09-21T13:21:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4214worker1 # [ 17.438814] nix-grpc-daemon[725]: ts=2026-09-21T13:21:20Z level=info event=scheduler_connected addr=lb:500514215worker1 # [ 17.442864] nix-grpc-daemon[725]: ts=2026-09-21T13:21:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4216worker2 # [ 17.483176] nix-grpc-daemon[783]: ts=2026-09-21T13:21:20Z level=info event=scheduler_connected addr=lb:500514217worker2 # [ 17.487108] nix-grpc-daemon[783]: ts=2026-09-21T13:21:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4218worker1 # [ 17.643969] nix-grpc-daemon[725]: ts=2026-09-21T13:21:20Z level=info event=scheduler_connected addr=lb:500514219worker1 # [ 17.648240] nix-grpc-daemon[725]: ts=2026-09-21T13:21:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4220worker2 # [ 17.688611] nix-grpc-daemon[783]: ts=2026-09-21T13:21:20Z level=info event=scheduler_connected addr=lb:500514221worker2 # [ 17.692444] nix-grpc-daemon[783]: ts=2026-09-21T13:21:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4222worker1 # [ 17.850865] nix-grpc-daemon[725]: ts=2026-09-21T13:21:21Z level=info event=scheduler_connected addr=lb:500514223worker1 # [ 17.854167] nix-grpc-daemon[725]: ts=2026-09-21T13:21:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4224worker2 # [ 17.895071] nix-grpc-daemon[783]: ts=2026-09-21T13:21:21Z level=info event=scheduler_connected addr=lb:500514225worker2 # [ 17.898293] nix-grpc-daemon[783]: ts=2026-09-21T13:21:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4226worker1 # [ 18.051721] nix-grpc-daemon[725]: ts=2026-09-21T13:21:21Z level=info event=scheduler_connected addr=lb:500514227worker1 # [ 18.055853] nix-grpc-daemon[725]: ts=2026-09-21T13:21:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4228worker2 # [ 18.095810] nix-grpc-daemon[783]: ts=2026-09-21T13:21:21Z level=info event=scheduler_connected addr=lb:500514229worker2 # [ 18.099147] nix-grpc-daemon[783]: ts=2026-09-21T13:21:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4230worker1 # [ 18.256961] nix-grpc-daemon[725]: ts=2026-09-21T13:21:21Z level=info event=scheduler_connected addr=lb:500514231worker1 # [ 18.261176] nix-grpc-daemon[725]: ts=2026-09-21T13:21:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4232worker2 # [ 18.300264] nix-grpc-daemon[783]: ts=2026-09-21T13:21:21Z level=info event=scheduler_connected addr=lb:500514233worker2 # [ 18.304174] nix-grpc-daemon[783]: ts=2026-09-21T13:21:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4234worker1 # [ 18.462183] nix-grpc-daemon[725]: ts=2026-09-21T13:21:21Z level=info event=scheduler_connected addr=lb:500514235worker1 # [ 18.466324] nix-grpc-daemon[725]: ts=2026-09-21T13:21:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4236worker2 # [ 18.505383] nix-grpc-daemon[783]: ts=2026-09-21T13:21:21Z level=info event=scheduler_connected addr=lb:500514237worker2 # [ 18.509196] nix-grpc-daemon[783]: ts=2026-09-21T13:21:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4238worker1 # [ 18.667532] nix-grpc-daemon[725]: ts=2026-09-21T13:21:21Z level=info event=scheduler_connected addr=lb:500514239worker1 # [ 18.671793] nix-grpc-daemon[725]: ts=2026-09-21T13:21:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4240worker2 # [ 18.710547] nix-grpc-daemon[783]: ts=2026-09-21T13:21:21Z level=info event=scheduler_connected addr=lb:500514241worker2 # [ 18.714789] nix-grpc-daemon[783]: ts=2026-09-21T13:21:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4242worker1 # [ 18.874095] nix-grpc-daemon[725]: ts=2026-09-21T13:21:22Z level=info event=scheduler_connected addr=lb:500514243worker1 # [ 18.877335] nix-grpc-daemon[725]: ts=2026-09-21T13:21:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4244worker2 # [ 18.917335] nix-grpc-daemon[783]: ts=2026-09-21T13:21:22Z level=info event=scheduler_connected addr=lb:500514245worker2 # [ 18.920476] nix-grpc-daemon[783]: ts=2026-09-21T13:21:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4246worker1 # [ 19.075295] nix-grpc-daemon[725]: ts=2026-09-21T13:21:22Z level=info event=scheduler_connected addr=lb:500514247worker1 # [ 19.079420] nix-grpc-daemon[725]: ts=2026-09-21T13:21:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4248worker2 # [ 19.118865] nix-grpc-daemon[783]: ts=2026-09-21T13:21:22Z level=info event=scheduler_connected addr=lb:500514249worker2 # [ 19.122767] nix-grpc-daemon[783]: ts=2026-09-21T13:21:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4250worker1 # [ 19.280346] nix-grpc-daemon[725]: ts=2026-09-21T13:21:22Z level=info event=scheduler_connected addr=lb:500514251worker1 # [ 19.284400] nix-grpc-daemon[725]: ts=2026-09-21T13:21:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4252worker2 # [ 19.323816] nix-grpc-daemon[783]: ts=2026-09-21T13:21:22Z level=info event=scheduler_connected addr=lb:500514253lb # [ 19.305591] rustfs-bucket-start[972]: mb s3://farm4254worker2 # [ 19.327639] nix-grpc-daemon[783]: ts=2026-09-21T13:21:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4255lb # [ 19.318515] systemd[1]: Finished rustfs-bucket.service.4256lb # [ 19.326920] systemd[1]: Starting niks3 server...4257lb # [ 19.456744] postgres[988]: [988] ERROR: relation "goose_db_version" does not exist at character 364258lb # [ 19.457959] postgres[988]: [988] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4259lb # [ 19.481635] niks3-server[983]: 2026/09/21 13:21:21 OK 20241026095416_initial_model.sql (14.25ms)4260worker1 # [ 19.485770] nix-grpc-daemon[725]: ts=2026-09-21T13:21:22Z level=info event=scheduler_connected addr=lb:500514261lb # [ 19.485144] niks3-server[983]: 2026/09/21 13:21:21 OK 20251210153512_drop_unused_gin_index.sql (3.68ms)4262worker1 # [ 19.490559] nix-grpc-daemon[725]: ts=2026-09-21T13:21:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4263lb # [ 19.490353] niks3-server[983]: 2026/09/21 13:21:21 OK 20251218171726_add_pins.sql (5.25ms)4264lb # [ 19.494559] niks3-server[983]: 2026/09/21 13:21:21 OK 20260628120000_add_object_size_and_stats.sql (3.08ms)4265lb # [ 19.499215] niks3-server[983]: 2026/09/21 13:21:21 OK 20260905000000_add_claims.sql (3.26ms)4266lb # [ 19.503055] niks3-server[983]: 2026/09/21 13:21:21 OK 20260920000000_drop_claims.sql (3.8ms)4267lb # [ 19.504231] niks3-server[983]: 2026/09/21 13:21:21 goose: successfully migrated database to version: 202609200000004268lb # [ 19.508441] niks3-server[983]: 2026/09/21 13:21:21 OK 1_commit_pending_closure.sql (2.88ms)4269worker2 # [ 19.529182] nix-grpc-daemon[783]: ts=2026-09-21T13:21:22Z level=info event=scheduler_connected addr=lb:500514270lb # [ 19.511475] niks3-server[983]: 2026/09/21 13:21:21 OK 2_object_stats_trigger.sql (1.8ms)4271lb # [ 19.512633] niks3-server[983]: 2026/09/21 13:21:21 goose: up to current file version: 24272worker2 # [ 19.533218] nix-grpc-daemon[783]: ts=2026-09-21T13:21:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4273lb # [ 19.516100] niks3-server[983]: 2026/09/21 13:21:21 INFO Loaded signing key name=farm-test-1 path=/nix/store/hk0nhypl37jvkn3gr36v3y7qx593p39i-key4274lb # [ 19.547455] niks3-server[983]: 2026/09/21 13:21:21 INFO Read proxy enabled — serving cache objects from S34275lb # [ 19.549322] niks3-server[983]: 2026/09/21 13:21:21 INFO Using socket-activated listener address=0.0.0.0:57514276lb # [ 19.553082] systemd[1]: Started niks3 server.4277lb # [ 19.553705] niks3-server[983]: 2026/09/21 13:21:21 INFO systemd watchdog enabled interval=15s4278lb # [ 19.554836] niks3-server[983]: 2026/09/21 13:21:21 INFO Starting HTTP server address=0.0.0.0:57514279lb # [ 19.555980] systemd[1]: Reached target Multi-User System.4280lb # [ 19.557911] systemd[1]: Startup finished in 856ms (kernel) + 3.627s (initrd) + 15.073s (userspace) = 19.557s.4281lb # [ 19.563912] niks3-server[983]: 2026/09/21 13:21:21 INFO lead: acquired remote=192.168.1.4:463524282worker2 # [ 19.588186] nix-grpc-daemon[783]: ts=2026-09-21T13:21:22Z level=info event=scheduler_take_over4283worker1 # [ 19.691793] nix-grpc-daemon[725]: ts=2026-09-21T13:21:22Z level=info event=scheduler_connected addr=lb:500514284worker1 # [ 19.696420] nix-grpc-daemon[725]: ts=2026-09-21T13:21:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4285worker2 # [ 19.734624] nix-grpc-daemon[783]: ts=2026-09-21T13:21:22Z level=info event=scheduler_connected addr=lb:500514286worker2 # [ 19.738572] nix-grpc-daemon[783]: ts=2026-09-21T13:21:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4287worker1 # [ 19.898814] nix-grpc-daemon[725]: ts=2026-09-21T13:21:23Z level=info event=scheduler_connected addr=lb:500514288worker2 # [ 19.920882] nix-grpc-daemon[783]: ts=2026-09-21T13:21:23Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:346124289worker2 # [ 19.925657] nix-grpc-daemon[783]: ts=2026-09-21T13:21:23Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04290worker2 # [ 19.939360] nix-grpc-daemon[783]: ts=2026-09-21T13:21:23Z level=info event=scheduler_connected addr=lb:500514291worker2 # [ 19.953736] nix-grpc-daemon[783]: ts=2026-09-21T13:21:23Z level=info event=oidc_jwks provider=mock keys=14292worker2 # [ 19.957127] nix-grpc-daemon[783]: ts=2026-09-21T13:21:23Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:346124293worker2 # [ 19.961870] nix-grpc-daemon[783]: ts=2026-09-21T13:21:23Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04294lb: (finished: waiting for unit niks3.service, in 10.74 seconds)4295lb: waiting for TCP port 5751 on localhost4296lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!4297lb: (finished: waiting for TCP port 5751 on localhost, in 0.05 seconds)4298lb: waiting for unit envoy.service4299lb: (finished: waiting for unit envoy.service, in 0.02 seconds)4300lb: waiting for TCP port 50051 on localhost4301lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!4302lb # [ 20.202518] envoy[916]: conn peer=[::1]:41054 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=24303lb: (finished: waiting for TCP port 50051 on localhost, in 0.02 seconds)4304worker2 # [ 20.334700] systemd[1]: Starting scheduler-token.service...4305worker2 # [ 20.383683] systemd[1]: scheduler-token.service: Deactivated successfully.4306worker2 # [ 20.385399] systemd[1]: Finished scheduler-token.service.4307worker2 # [ 20.388308] systemd[1]: scheduler-token.service: Consumed 24ms CPU time over 52ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4308lb: waiting until exactly no endpoints of x86_64-linux are unhealthy4309lb: must succeed: curl -sf localhost:9901/clusters4310lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4311lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.06 seconds)4312subtest: one node schedules and both builders hold a WorkerSession on it4313lb: must succeed: curl -sf localhost:9901/clusters4314lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4315worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4316worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4317lb: must succeed: curl -sf localhost:9901/clusters4318lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4319lb: must succeed: curl -sf localhost:9901/clusters4320lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4321lb: must succeed: curl -sf localhost:9901/clusters4322lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4323lb: must succeed: curl -sf localhost:9901/clusters4324lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4325worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'4326worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.04 seconds)4327worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4328worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4329lb: must succeed: curl -sf localhost:9901/clusters4330lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4331lb: must succeed: curl -sf localhost:9901/clusters4332lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4333worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4334worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4335worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4336worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4337worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4338worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4339worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4340worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)4341(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.43 seconds)4342subtest: balancer auth: client cert, bearer token, nothing, unknown CN4343worker1 # [ 21.071156] nix-grpc-daemon[725]: ts=2026-09-21T13:21:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:594504344worker1 # [ 21.082733] systemd[1]: Started Nix Daemon.4345worker1 # [ 21.199641] nix-daemon[842]: accepted connection from pid 725, user nix-grpc-daemon (trusted)4346worker1 # [ 21.214185] nix-grpc-daemon[725]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 82 ms (attempt 1/5)4347worker1 # [ 21.301097] nix-grpc-daemon[725]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 115 ms (attempt 2/5)4348worker1 # [ 21.418711] nix-grpc-daemon[725]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 86 ms (attempt 3/5)4349worker1 # [ 21.506868] nix-grpc-daemon[725]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 490 ms (attempt 4/5)4350worker1 # [ 22.000959] nix-grpc-daemon[725]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4351worker1 # [ 22.010906] nix-grpc-daemon[725]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4352worker1 # [ 22.032622] nix-grpc-daemon[725]: ts=2026-09-21T13:21:25Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:594504353worker1 # [ 22.044157] nix-grpc-daemon[725]: ts=2026-09-21T13:21:25Z level=debug event=path_info_miss path=yvs21lk49a0hcgbp46d92zly2h4ldq9b-nixos-system-client-test error="error: path '/nix/store/yvs21lk49a0hcgbp46d92zly2h4ldq9b-nixos-system-client-test' is required, but there is no substituter that can build it"4354worker2 # [ 22.088965] nix-grpc-daemon[783]: ts=2026-09-21T13:21:25Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:346204355worker2 # [ 22.099755] systemd[1]: Started Nix Daemon.4356worker2 # [ 22.208810] nix-daemon[865]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4357worker2 # [ 22.236149] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 37 ms (attempt 1/5)4358worker2 # [ 22.276614] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 60 ms (attempt 2/5)4359worker2 # [ 22.338868] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 139 ms (attempt 3/5)4360worker2 # [ 22.479925] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 782 ms (attempt 4/5)4361worker2 # [ 23.266847] nix-grpc-daemon[783]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4362worker2 # [ 23.271133] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4363worker2 # [ 23.283562] nix-grpc-daemon[783]: ts=2026-09-21T13:21:26Z level=debug event=path_info_miss path=yvs21lk49a0hcgbp46d92zly2h4ldq9b-nixos-system-client-test error="error: path '/nix/store/yvs21lk49a0hcgbp46d92zly2h4ldq9b-nixos-system-client-test' is required, but there is no substituter that can build it"4364client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4365client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.05 seconds)4366client: must succeed: install -D /nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4367client: (finished: must succeed: install -D /nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.02 seconds)4368worker1 # [ 23.499956] nix-grpc-daemon[725]: ts=2026-09-21T13:21:26Z level=info event=oidc_jwks provider=mock keys=14369worker1 # [ 23.504599] nix-grpc-daemon[725]: ts=2026-09-21T13:21:26Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:594504370worker1 # [ 23.514259] nix-grpc-daemon[725]: ts=2026-09-21T13:21:26Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:594504371worker1 # [ 23.524114] nix-grpc-daemon[725]: ts=2026-09-21T13:21:26Z level=debug event=path_info_miss path=yvs21lk49a0hcgbp46d92zly2h4ldq9b-nixos-system-client-test error="error: path '/nix/store/yvs21lk49a0hcgbp46d92zly2h4ldq9b-nixos-system-client-test' is required, but there is no substituter that can build it"4372worker1 # [ 23.534099] nix-grpc-daemon[725]: ts=2026-09-21T13:21:26Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:594504373worker1 # [ 23.543636] nix-grpc-daemon[725]: ts=2026-09-21T13:21:26Z level=debug event=path_info_miss path=yvs21lk49a0hcgbp46d92zly2h4ldq9b-nixos-system-client-test error="error: path '/nix/store/yvs21lk49a0hcgbp46d92zly2h4ldq9b-nixos-system-client-test' is required, but there is no substituter that can build it"4374client: must succeed: rm -r /var/lib/nix-grpc-store4375client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.01 seconds)4376worker2 # [ 23.697151] nix-grpc-daemon[783]: ts=2026-09-21T13:21:26Z level=info event=denied method=QueryMissing cn=- role=none4377worker2 # [ 23.826776] nix-grpc-daemon[783]: ts=2026-09-21T13:21:27Z level=info event=denied method=QueryMissing cn=stranger role=none4378(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 2.97 seconds)4379subtest: DAG build is scheduled across workers and lands in the cache4380client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&24381worker1 # [ 23.944220] nix-grpc-daemon[725]: ts=2026-09-21T13:21:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:594504382client # don't know how to build these paths:4383client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4384worker2 # [ 23.968626] nix-grpc-daemon[783]: ts=2026-09-21T13:21:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:346204385worker2 # [ 23.999705] nix-grpc-daemon[783]: ts=2026-09-21T13:21:27Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04386worker2 # [ 24.003570] nix-grpc-daemon[783]: ts=2026-09-21T13:21:27Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.3:50051 clients=14387worker1 # [ 24.044205] nix-grpc-daemon[725]: ts=2026-09-21T13:21:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594504388worker1 # [ 24.058165] nix-grpc-daemon[725]: ts=2026-09-21T13:21:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:594504389client # copying 1 paths to 'lb:50051'...4390worker1 # [ 24.071374] nix-grpc-daemon[725]: ts=2026-09-21T13:21:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:594504391client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4392worker1 # [ 24.094976] nix-daemon[842]: accepted connection from pid 725, user nix-grpc-daemon (trusted)4393worker1 # [ 24.129463] nix-grpc-daemon[725]: ts=2026-09-21T13:21:27Z level=info event=niks3_push_started pid=8554394worker1 # [ 24.289514] nix-grpc-daemon[855]: time=2026-09-21T13:21:27.447Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""4395worker1 # [ 24.382109] nix-daemon[842]: accepted connection from pid 867, user nix-grpc-daemon (trusted)4396worker1 # [ 24.397163] nix-daemon[842]: reaped child process 870, status = succeeded4397worker1 # [ 24.481824] nix-daemon[842]: accepted connection from pid 872, user nix-grpc-daemon (trusted)4398worker1 # [ 24.497078] nix-daemon[842]: reaped child process 875, status = succeeded4399lb # [ 24.496543] niks3-server[983]: 2026/09/21 13:21:26 INFO Received uploads request method=POST path=/api/pending_closures4400worker1 # [ 24.517328] nix-grpc-daemon[855]: time=2026-09-21T13:21:27.676Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4401worker1 # [ 24.520041] nix-grpc-daemon[855]: time=2026-09-21T13:21:27.678Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4402lb # [ 24.549682] niks3-server[983]: 2026/09/21 13:21:26 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4403lb # [ 24.555116] niks3-server[983]: 2026/09/21 13:21:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4404lb # [ 24.574071] niks3-server[983]: 2026/09/21 13:21:26 INFO Signed narinfos id=1 count=14405worker1 # [ 24.583184] nix-grpc-daemon[855]: time=2026-09-21T13:21:27.741Z level=INFO msg="Uploading 1 narinfos"4406lb # [ 24.585829] niks3-server[983]: 2026/09/21 13:21:26 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4407lb # [ 24.591458] niks3-server[983]: 2026/09/21 13:21:26 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4408lb # [ 24.602984] niks3-server[983]: 2026/09/21 13:21:26 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo4409lb # [ 24.608265] niks3-server[983]: 2026/09/21 13:21:26 INFO Completed upload id=14410worker1 # [ 24.615763] nix-grpc-daemon[855]: time=2026-09-21T13:21:27.774Z level=INFO msg="Upload complete. (323ms)"4411worker1 # [ 24.622238] nix-grpc-daemon[725]: ts=2026-09-21T13:21:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59450 duration_s=0 paths=1 nar_bytes_in=5044412worker1 # [ 24.627949] nix-grpc-daemon[725]: ts=2026-09-21T13:21:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594504413worker1 # [ 24.641115] nix-daemon[842]: accepted connection from pid 725, user nix-grpc-daemon (trusted)4414worker1 # [ 24.649494] nix-daemon[842]: reaped child process 853, status = succeeded4415client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4416worker1 # [ 24.658431] nix-daemon[842]: accepted connection from pid 725, user nix-grpc-daemon (trusted)4417client # farm-a> node-a: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4418client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4419lb # [ 25.554944] envoy[916]: conn peer=[2001:db8:1::1]:49056 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=749 tx=423 ms=22424420lb # [ 25.559249] envoy[916]: conn peer=[2001:db8:1::1]:33356 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=624421lb # [ 25.562547] envoy[916]: conn peer=[2001:db8:1::1]:33360 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=84422lb # [ 25.565914] envoy[916]: conn peer=[2001:db8:1::1]:33374 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=84423client # farm-a> LOG-farm-a-t14424worker1 # [ 25.900688] nix-daemon[842]: reaped child process 879, status = succeeded4425worker1 # [ 25.985100] nix-daemon[842]: accepted connection from pid 897, user nix-grpc-daemon (trusted)4426worker1 # [ 25.998202] nix-daemon[842]: reaped child process 900, status = succeeded4427lb # [ 26.001272] niks3-server[983]: 2026/09/21 13:21:28 INFO Received uploads request method=POST path=/api/pending_closures4428worker1 # [ 26.017111] nix-grpc-daemon[855]: time=2026-09-21T13:21:29.176Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4429worker1 # [ 26.018588] nix-grpc-daemon[855]: time=2026-09-21T13:21:29.177Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4430lb # [ 26.044481] niks3-server[983]: 2026/09/21 13:21:28 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4431lb # [ 26.055555] niks3-server[983]: 2026/09/21 13:21:28 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4432lb # [ 26.065903] niks3-server[983]: 2026/09/21 13:21:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4433lb # [ 26.072834] niks3-server[983]: 2026/09/21 13:21:28 INFO Signed narinfos id=2 count=14434worker1 # [ 26.081948] nix-grpc-daemon[855]: time=2026-09-21T13:21:29.240Z level=INFO msg="Uploading 1 narinfos"4435lb # [ 26.076262] niks3-server[983]: 2026/09/21 13:21:28 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4436lb # [ 26.082243] niks3-server[983]: 2026/09/21 13:21:28 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4437lb # [ 26.092353] niks3-server[983]: 2026/09/21 13:21:28 INFO Completed upload id=24438lb # [ 26.094235] niks3-server[983]: 2026/09/21 13:21:28 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4439worker1 # [ 26.100809] nix-grpc-daemon[855]: time=2026-09-21T13:21:29.259Z level=INFO msg="Upload complete. (230ms)"4440worker1 # [ 26.107221] nix-grpc-daemon[725]: ts=2026-09-21T13:21:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59450 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14441worker2 # [ 26.131598] nix-grpc-daemon[783]: ts=2026-09-21T13:21:29Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04442worker1 # [ 26.122145] nix-daemon[842]: reaped child process 877, status = succeeded4443worker2 # [ 26.135940] nix-grpc-daemon[783]: ts=2026-09-21T13:21:29Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.3:50051 clients=14444worker1 # [ 26.161519] nix-grpc-daemon[725]: ts=2026-09-21T13:21:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594504445worker1 # [ 26.179539] nix-grpc-daemon[725]: ts=2026-09-21T13:21:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:594504446worker1 # [ 26.194306] nix-grpc-daemon[855]: time=2026-09-21T13:21:29.353Z level=INFO msg="All 1 paths already cached"4447client # copying 1 paths to 'lb:50051'...4448worker1 # [ 26.202630] nix-grpc-daemon[725]: ts=2026-09-21T13:21:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:594504449client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4450worker1 # [ 26.220273] nix-daemon[842]: accepted connection from pid 725, user nix-grpc-daemon (trusted)4451worker1 # [ 26.329179] nix-daemon[842]: accepted connection from pid 904, user nix-grpc-daemon (trusted)4452worker1 # [ 26.343813] nix-daemon[842]: reaped child process 907, status = succeeded4453worker1 # [ 26.425909] nix-daemon[842]: accepted connection from pid 909, user nix-grpc-daemon (trusted)4454worker1 # [ 26.439829] nix-daemon[842]: reaped child process 912, status = succeeded4455lb # [ 26.440948] niks3-server[983]: 2026/09/21 13:21:28 INFO Received uploads request method=POST path=/api/pending_closures4456worker1 # [ 26.456175] nix-grpc-daemon[855]: time=2026-09-21T13:21:29.615Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4457worker1 # [ 26.457667] nix-grpc-daemon[855]: time=2026-09-21T13:21:29.616Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4458lb # [ 26.476690] niks3-server[983]: 2026/09/21 13:21:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4459worker1 # [ 26.492104] nix-grpc-daemon[855]: time=2026-09-21T13:21:29.651Z level=INFO msg="Uploading 1 narinfos"4460lb # [ 26.486193] niks3-server[983]: 2026/09/21 13:21:28 INFO Signed narinfos id=3 count=14461lb # [ 26.491861] niks3-server[983]: 2026/09/21 13:21:28 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4462lb # [ 26.497764] niks3-server[983]: 2026/09/21 13:21:28 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4463lb # [ 26.512107] niks3-server[983]: 2026/09/21 13:21:28 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4464lb # [ 26.517573] niks3-server[983]: 2026/09/21 13:21:28 INFO Completed upload id=34465worker1 # [ 26.526299] nix-grpc-daemon[855]: time=2026-09-21T13:21:29.685Z level=INFO msg="Upload complete. (290ms)"4466lb # [ 26.523802] niks3-server[983]: 2026/09/21 13:21:28 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4467worker1 # [ 26.530470] nix-grpc-daemon[725]: ts=2026-09-21T13:21:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59450 duration_s=0 paths=1 nar_bytes_in=6244468worker1 # [ 26.536138] nix-grpc-daemon[725]: ts=2026-09-21T13:21:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594504469worker1 # [ 26.545877] nix-daemon[842]: accepted connection from pid 725, user nix-grpc-daemon (trusted)4470worker1 # [ 26.554412] nix-daemon[842]: reaped child process 902, status = succeeded4471worker1 # [ 26.559735] nix-daemon[842]: accepted connection from pid 725, user nix-grpc-daemon (trusted)4472client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4473client # farm-b> node-a: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4474client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4475client # farm-b> LOG-farm-b-t14476worker1 # [ 27.196562] nix-daemon[842]: reaped child process 916, status = succeeded4477worker1 # [ 27.285114] nix-daemon[842]: accepted connection from pid 932, user nix-grpc-daemon (trusted)4478worker1 # [ 27.297841] nix-daemon[842]: reaped child process 935, status = succeeded4479lb # [ 27.300556] niks3-server[983]: 2026/09/21 13:21:29 INFO Received uploads request method=POST path=/api/pending_closures4480worker1 # [ 27.316925] nix-grpc-daemon[855]: time=2026-09-21T13:21:30.475Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4481worker1 # [ 27.318570] nix-grpc-daemon[855]: time=2026-09-21T13:21:30.477Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4482lb # [ 27.339227] niks3-server[983]: 2026/09/21 13:21:29 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4483lb # [ 27.352717] niks3-server[983]: 2026/09/21 13:21:29 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4484lb # [ 27.361709] niks3-server[983]: 2026/09/21 13:21:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4485worker1 # [ 27.373041] nix-grpc-daemon[855]: time=2026-09-21T13:21:30.531Z level=INFO msg="Uploading 1 narinfos"4486lb # [ 27.366535] niks3-server[983]: 2026/09/21 13:21:29 INFO Signed narinfos id=4 count=14487lb # [ 27.372172] niks3-server[983]: 2026/09/21 13:21:29 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4488lb # [ 27.380024] niks3-server[983]: 2026/09/21 13:21:29 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4489lb # [ 27.384700] niks3-server[983]: 2026/09/21 13:21:29 INFO Completed upload id=44490lb # [ 27.387104] niks3-server[983]: 2026/09/21 13:21:29 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4491worker1 # [ 27.392318] nix-grpc-daemon[855]: time=2026-09-21T13:21:30.551Z level=INFO msg="Upload complete. (224ms)"4492worker1 # [ 27.399215] nix-grpc-daemon[725]: ts=2026-09-21T13:21:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59450 duration_s=0 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14493worker2 # [ 27.420891] nix-grpc-daemon[783]: ts=2026-09-21T13:21:30Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04494worker2 # [ 27.425325] nix-grpc-daemon[783]: ts=2026-09-21T13:21:30Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.3:50051 clients=14495worker1 # [ 27.414848] nix-daemon[842]: reaped child process 914, status = succeeded4496worker1 # [ 27.449917] nix-grpc-daemon[725]: ts=2026-09-21T13:21:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594504497worker1 # [ 27.462263] nix-grpc-daemon[725]: ts=2026-09-21T13:21:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:594504498worker1 # [ 27.476185] nix-grpc-daemon[855]: time=2026-09-21T13:21:30.635Z level=INFO msg="All 2 paths already cached"4499client # copying 1 paths to 'lb:50051'...4500worker1 # [ 27.484488] nix-grpc-daemon[725]: ts=2026-09-21T13:21:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:594504501client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4502worker1 # [ 27.504166] nix-daemon[842]: accepted connection from pid 725, user nix-grpc-daemon (trusted)4503worker1 # [ 27.607602] nix-daemon[842]: accepted connection from pid 939, user nix-grpc-daemon (trusted)4504worker1 # [ 27.622336] nix-daemon[842]: reaped child process 942, status = succeeded4505worker1 # [ 27.705233] nix-daemon[842]: accepted connection from pid 944, user nix-grpc-daemon (trusted)4506worker1 # [ 27.719348] nix-daemon[842]: reaped child process 947, status = succeeded4507lb # [ 27.720874] niks3-server[983]: 2026/09/21 13:21:30 INFO Received uploads request method=POST path=/api/pending_closures4508worker1 # [ 27.736227] nix-grpc-daemon[855]: time=2026-09-21T13:21:30.895Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4509worker1 # [ 27.737690] nix-grpc-daemon[855]: time=2026-09-21T13:21:30.895Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4510lb # [ 27.747765] niks3-server[983]: 2026/09/21 13:21:30 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4511lb # [ 27.759252] niks3-server[983]: 2026/09/21 13:21:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4512lb # [ 27.765868] niks3-server[983]: 2026/09/21 13:21:30 INFO Signed narinfos id=5 count=14513worker1 # [ 27.775078] nix-grpc-daemon[855]: time=2026-09-21T13:21:30.934Z level=INFO msg="Uploading 1 narinfos"4514lb # [ 27.770956] niks3-server[983]: 2026/09/21 13:21:30 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4515lb # [ 27.788180] niks3-server[983]: 2026/09/21 13:21:30 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4516lb # [ 27.793232] niks3-server[983]: 2026/09/21 13:21:30 INFO Completed upload id=54517worker1 # [ 27.799509] nix-grpc-daemon[855]: time=2026-09-21T13:21:30.958Z level=INFO msg="Upload complete. (282ms)"4518lb # [ 27.796668] niks3-server[983]: 2026/09/21 13:21:30 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4519worker1 # [ 27.803376] nix-grpc-daemon[725]: ts=2026-09-21T13:21:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59450 duration_s=0 paths=1 nar_bytes_in=7524520worker1 # [ 27.808858] nix-grpc-daemon[725]: ts=2026-09-21T13:21:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594504521worker1 # [ 27.820056] nix-daemon[842]: accepted connection from pid 725, user nix-grpc-daemon (trusted)4522worker1 # [ 27.824884] nix-daemon[842]: reaped child process 937, status = succeeded4523worker1 # [ 27.829385] nix-daemon[842]: accepted connection from pid 725, user nix-grpc-daemon (trusted)4524client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4525client # farm-top> node-a: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4526client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4527client # farm-top> LOG-farm-top-t14528worker1 # [ 29.067930] nix-daemon[842]: reaped child process 951, status = succeeded4529worker1 # [ 29.153938] nix-daemon[842]: accepted connection from pid 968, user nix-grpc-daemon (trusted)4530worker1 # [ 29.169758] nix-daemon[842]: reaped child process 971, status = succeeded4531lb # [ 29.170746] niks3-server[983]: 2026/09/21 13:21:31 INFO Received uploads request method=POST path=/api/pending_closures4532worker1 # [ 29.185862] nix-grpc-daemon[855]: time=2026-09-21T13:21:32.344Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4533worker1 # [ 29.187370] nix-grpc-daemon[855]: time=2026-09-21T13:21:32.346Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4534lb # [ 29.213826] niks3-server[983]: 2026/09/21 13:21:31 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4535lb # [ 29.221387] niks3-server[983]: 2026/09/21 13:21:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4536lb # [ 29.226522] niks3-server[983]: 2026/09/21 13:21:31 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4537worker1 # [ 29.238934] nix-grpc-daemon[855]: time=2026-09-21T13:21:32.398Z level=INFO msg="Uploading 1 narinfos"4538lb # [ 29.232985] niks3-server[983]: 2026/09/21 13:21:31 INFO Signed narinfos id=6 count=14539lb # [ 29.239123] niks3-server[983]: 2026/09/21 13:21:31 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4540lb # [ 29.249499] niks3-server[983]: 2026/09/21 13:21:31 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4541lb # [ 29.257563] niks3-server[983]: 2026/09/21 13:21:31 INFO Completed upload id=64542worker1 # [ 29.263947] nix-grpc-daemon[855]: time=2026-09-21T13:21:32.423Z level=INFO msg="Upload complete. (223ms)"4543lb # [ 29.260821] niks3-server[983]: 2026/09/21 13:21:31 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4544worker1 # [ 29.269718] nix-grpc-daemon[725]: ts=2026-09-21T13:21:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59450 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14545worker1 # [ 29.285301] nix-daemon[842]: reaped child process 949, status = succeeded4546client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2, in 5.48 seconds)4547client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath4548client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath, in 0.11 seconds)4549client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4550client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.01 seconds)4551client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4552client # copying 3 paths...4553client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4554client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4555client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4556client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.16 seconds)4557(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 5.83 seconds)4558subtest: repeat is answered Cached by the scheduler without building4559worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4560worker1 # finding garbage collector roots...4561worker1 # deleting specified paths...4562worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4563worker1 # deleting unused links...4564worker1 # note: hard linking is currently saving 0.0 KiB4565worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.16 seconds)4566worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4567worker2 # finding garbage collector roots...4568worker2 # deleting specified paths...4569worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4570worker2 # deleting unused links...4571worker2 # note: hard linking is currently saving 0.0 KiB4572worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.18 seconds)4573client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&24574worker1 # [ 30.123921] nix-grpc-daemon[725]: ts=2026-09-21T13:21:33Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:594504575client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4576client # /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4577worker2 # [ 30.158995] nix-grpc-daemon[783]: ts=2026-09-21T13:21:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:346204578worker2 # [ 30.186176] nix-grpc-daemon[783]: ts=2026-09-21T13:21:33Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14579client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2, in 0.21 seconds)4580client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath4581client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath, in 0.09 seconds)4582lb: must succeed: curl -sf localhost:9901/clusters4583lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4584lb: must succeed: curl -sf localhost:9901/clusters4585lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4586worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4587worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4588(finished: subtest: repeat is answered Cached by the scheduler without building, in 0.82 seconds)4589subtest: inputs already in the cache are not sent to the scheduler4590lb: must succeed: curl -sf localhost:9901/clusters4591lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4592lb: must succeed: curl -sf localhost:9901/clusters4593lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4594worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4595worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.04 seconds)4596client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&24597client # don't know how to build these paths:4598client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4599worker2 # [ 30.703208] nix-grpc-daemon[783]: ts=2026-09-21T13:21:33Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:346204600worker1 # [ 30.694796] nix-grpc-daemon[725]: ts=2026-09-21T13:21:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:594504601worker1 # [ 30.704292] nix-grpc-daemon[855]: time=2026-09-21T13:21:33.863Z level=INFO msg="All 2 paths already cached"4602worker2 # [ 30.730843] nix-grpc-daemon[783]: ts=2026-09-21T13:21:33Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04603worker1 # [ 30.718572] nix-grpc-daemon[725]: ts=2026-09-21T13:21:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594504604worker2 # [ 30.735431] nix-grpc-daemon[783]: ts=2026-09-21T13:21:33Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.3:50051 clients=14605worker1 # [ 30.730504] nix-grpc-daemon[725]: ts=2026-09-21T13:21:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:594504606worker1 # [ 30.743440] nix-grpc-daemon[855]: time=2026-09-21T13:21:33.902Z level=INFO msg="All 2 paths already cached"4607client # copying 1 paths to 'lb:50051'...4608client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4609worker1 # [ 30.750296] nix-grpc-daemon[725]: ts=2026-09-21T13:21:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:594504610worker1 # [ 30.761335] nix-daemon[842]: accepted connection from pid 725, user nix-grpc-daemon (trusted)4611worker1 # [ 30.868046] nix-daemon[842]: accepted connection from pid 988, user nix-grpc-daemon (trusted)4612worker1 # [ 30.883272] nix-daemon[842]: reaped child process 991, status = succeeded4613worker1 # [ 30.972289] nix-daemon[842]: accepted connection from pid 993, user nix-grpc-daemon (trusted)4614worker1 # [ 30.987558] nix-daemon[842]: reaped child process 996, status = succeeded4615lb # [ 30.990645] niks3-server[983]: 2026/09/21 13:21:33 INFO Received uploads request method=POST path=/api/pending_closures4616worker1 # [ 31.004965] nix-grpc-daemon[855]: time=2026-09-21T13:21:34.163Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4617worker1 # [ 31.006336] nix-grpc-daemon[855]: time=2026-09-21T13:21:34.163Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4618lb # [ 31.020153] niks3-server[983]: 2026/09/21 13:21:33 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4619lb # [ 31.030131] niks3-server[983]: 2026/09/21 13:21:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4620worker1 # [ 31.042692] nix-grpc-daemon[855]: time=2026-09-21T13:21:34.201Z level=INFO msg="Uploading 1 narinfos"4621lb # [ 31.036413] niks3-server[983]: 2026/09/21 13:21:33 INFO Signed narinfos id=7 count=14622lb # [ 31.049489] niks3-server[983]: 2026/09/21 13:21:33 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4623lb # [ 31.056356] niks3-server[983]: 2026/09/21 13:21:33 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4624worker1 # [ 31.068114] nix-grpc-daemon[855]: time=2026-09-21T13:21:34.227Z level=INFO msg="Upload complete. (293ms)"4625lb # [ 31.062158] niks3-server[983]: 2026/09/21 13:21:33 INFO Completed upload id=74626worker1 # [ 31.070596] nix-grpc-daemon[725]: ts=2026-09-21T13:21:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59450 duration_s=0 paths=1 nar_bytes_in=7524627lb # [ 31.067420] niks3-server[983]: 2026/09/21 13:21:33 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4628worker1 # [ 31.076340] nix-grpc-daemon[725]: ts=2026-09-21T13:21:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594504629worker1 # [ 31.084877] nix-daemon[842]: accepted connection from pid 725, user nix-grpc-daemon (trusted)4630worker1 # [ 31.094173] nix-daemon[842]: reaped child process 986, status = succeeded4631worker1 # [ 31.097469] nix-daemon[842]: accepted connection from pid 725, user nix-grpc-daemon (trusted)4632client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4633client # farm-top> node-a: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4634client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4635client # farm-top> LOG-farm-top-t1b4636worker1 # [ 32.140562] nix-daemon[842]: reaped child process 1000, status = succeeded4637worker1 # [ 32.224603] nix-daemon[842]: accepted connection from pid 1017, user nix-grpc-daemon (trusted)4638worker1 # [ 32.239188] nix-daemon[842]: reaped child process 1020, status = succeeded4639lb # [ 32.241854] niks3-server[983]: 2026/09/21 13:21:34 INFO Received uploads request method=POST path=/api/pending_closures4640worker1 # [ 32.256267] nix-grpc-daemon[855]: time=2026-09-21T13:21:35.415Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4641worker1 # [ 32.258051] nix-grpc-daemon[855]: time=2026-09-21T13:21:35.417Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4642lb # [ 32.280293] niks3-server[983]: 2026/09/21 13:21:34 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4643lb # [ 32.292234] niks3-server[983]: 2026/09/21 13:21:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4644worker1 # [ 32.304216] nix-grpc-daemon[855]: time=2026-09-21T13:21:35.463Z level=INFO msg="Uploading 1 narinfos"4645lb # [ 32.298593] niks3-server[983]: 2026/09/21 13:21:34 INFO Signed narinfos id=8 count=14646lb # [ 32.305315] niks3-server[983]: 2026/09/21 13:21:34 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4647lb # [ 32.309675] niks3-server[983]: 2026/09/21 13:21:34 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4648lb # [ 32.321223] niks3-server[983]: 2026/09/21 13:21:34 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4649lb # [ 32.325660] niks3-server[983]: 2026/09/21 13:21:34 INFO Completed upload id=84650lb # [ 32.328591] niks3-server[983]: 2026/09/21 13:21:34 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4651worker1 # [ 32.331923] nix-grpc-daemon[855]: time=2026-09-21T13:21:35.491Z level=INFO msg="Upload complete. (225ms)"4652worker1 # [ 32.338416] nix-grpc-daemon[725]: ts=2026-09-21T13:21:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59450 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14653worker1 # [ 32.353581] nix-daemon[842]: reaped child process 998, status = succeeded4654client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&2, in 1.81 seconds)4655client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath4656client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.11 seconds)4657lb: must succeed: curl -sf localhost:9901/clusters4658lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4659lb: must succeed: curl -sf localhost:9901/clusters4660lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4661worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4662worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.04 seconds)4663(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.13 seconds)4664subtest: two clients wanting the same drv build it once4665client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup4666client # [ 32.655690] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.4667client # Running as unit: dedup1.service; invocation ID: afaad5b148cc4149a5446ee7ba2229aa4668client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 0.06 seconds)4669client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup4670client # [ 32.789978] nix[886]: don't know how to build these paths:4671client # [ 32.790919] nix[886]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4672worker1 # [ 32.772234] nix-grpc-daemon[725]: ts=2026-09-21T13:21:35Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:594504673worker2 # [ 32.798477] nix-grpc-daemon[783]: ts=2026-09-21T13:21:35Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04674worker2 # [ 32.804081] nix-grpc-daemon[783]: ts=2026-09-21T13:21:35Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.4:50051 clients=14675worker2 # [ 32.809411] nix-grpc-daemon[783]: ts=2026-09-21T13:21:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:346204676worker2 # [ 32.820484] nix-grpc-daemon[783]: ts=2026-09-21T13:21:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:346204677client # [ 32.831978] nix[886]: copying 1 paths to 'lb:50051'...4678client # [ 32.832842] nix[886]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4679worker2 # [ 32.831607] nix-grpc-daemon[783]: ts=2026-09-21T13:21:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:346204680worker2 # [ 32.842779] nix-daemon[865]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4681worker2 # [ 32.864356] nix-grpc-daemon[783]: ts=2026-09-21T13:21:36Z level=info event=niks3_push_started pid=9184682worker2 # [ 32.988854] nix-grpc-daemon[918]: time=2026-09-21T13:21:36.165Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""4683worker2 # [ 33.080148] nix-daemon[865]: accepted connection from pid 931, user nix-grpc-daemon (trusted)4684worker2 # [ 33.092918] nix-daemon[865]: reaped child process 934, status = succeeded4685worker2 # [ 33.175286] nix-daemon[865]: accepted connection from pid 936, user nix-grpc-daemon (trusted)4686worker2 # [ 33.189219] nix-daemon[865]: reaped child process 939, status = succeeded4687lb # [ 33.174855] niks3-server[983]: 2026/09/21 13:21:35 INFO Received uploads request method=POST path=/api/pending_closures4688worker2 # [ 33.205841] nix-grpc-daemon[918]: time=2026-09-21T13:21:36.381Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4689worker2 # [ 33.207186] nix-grpc-daemon[918]: time=2026-09-21T13:21:36.381Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4690lb # [ 33.209472] niks3-server[983]: 2026/09/21 13:21:35 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4691lb # [ 33.218179] niks3-server[983]: 2026/09/21 13:21:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4692worker2 # [ 33.247412] nix-grpc-daemon[918]: time=2026-09-21T13:21:36.424Z level=INFO msg="Uploading 1 narinfos"4693lb # [ 33.226157] niks3-server[983]: 2026/09/21 13:21:35 INFO Signed narinfos id=9 count=14694lb # [ 33.233421] niks3-server[983]: 2026/09/21 13:21:35 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4695lb # [ 33.245580] niks3-server[983]: 2026/09/21 13:21:35 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4696lb # [ 33.253738] niks3-server[983]: 2026/09/21 13:21:35 INFO Completed upload id=94697worker2 # [ 33.275995] nix-grpc-daemon[918]: time=2026-09-21T13:21:36.452Z level=INFO msg="Upload complete. (283ms)"4698lb # [ 33.257585] niks3-server[983]: 2026/09/21 13:21:35 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4699worker2 # [ 33.277928] nix-grpc-daemon[783]: ts=2026-09-21T13:21:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34620 duration_s=0 paths=1 nar_bytes_in=4324700worker2 # [ 33.286333] nix-grpc-daemon[783]: ts=2026-09-21T13:21:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:346204701worker2 # [ 33.295411] nix-daemon[865]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4702worker2 # [ 33.297701] nix-daemon[865]: reaped child process 916, status = succeeded4703worker2 # [ 33.302851] nix-daemon[865]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4704client # [ 33.304215] nix[886]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4705client # [ 33.717804] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.4706client # Running as unit: dedup2.service; invocation ID: 1b5d372304f24d2c9b1ca9bb4a0e56a94707client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 1.07 seconds)4708client: must succeed: sleep 24709worker2 # [ 33.921918] nix-grpc-daemon[783]: ts=2026-09-21T13:21:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:346204710client # [ 33.933331] nix[902]: this derivation will be built:4711worker1 # [ 33.921617] nix-grpc-daemon[725]: ts=2026-09-21T13:21:37Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:594504712worker1 # [ 33.934385] nix-grpc-daemon[725]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' from 'http://lb:5751'...4713client # [ 33.966626] nix[902]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4714worker2 # [ 33.972195] nix-grpc-daemon[783]: ts=2026-09-21T13:21:37Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04715worker2 # [ 33.975467] nix-grpc-daemon[783]: ts=2026-09-21T13:21:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:346204716client # [ 33.979450] nix[902]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4717lb # [ 35.555747] envoy[916]: conn peer=[2001:db8:1::1]:33378 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9537 tx=4048 ms=53464718lb # [ 35.559729] envoy[916]: conn peer=[2001:db8:1::1]:33386 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=624719lb # [ 35.563722] envoy[916]: conn peer=[2001:db8:1::1]:33394 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4182 tx=1806 ms=16584720client: (finished: must succeed: sleep 2, in 2.05 seconds)4721client: waiting for success: ! systemctl is-active dedup1 dedup24722worker2 # [ 36.144504] nix-daemon[865]: reaped child process 943, status = succeeded4723worker2 # [ 36.219404] nix-daemon[865]: accepted connection from pid 972, user nix-grpc-daemon (trusted)4724worker2 # [ 36.233850] nix-daemon[865]: reaped child process 975, status = succeeded4725lb # [ 36.219150] niks3-server[983]: 2026/09/21 13:21:38 INFO Received uploads request method=POST path=/api/pending_closures4726worker2 # [ 36.248496] nix-grpc-daemon[918]: time=2026-09-21T13:21:39.425Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4727worker2 # [ 36.250064] nix-grpc-daemon[918]: time=2026-09-21T13:21:39.426Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4728lb # [ 36.258448] niks3-server[983]: 2026/09/21 13:21:38 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4729lb # [ 36.268217] niks3-server[983]: 2026/09/21 13:21:38 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4730lb # [ 36.278504] niks3-server[983]: 2026/09/21 13:21:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4731lb # [ 36.285763] niks3-server[983]: 2026/09/21 13:21:38 INFO Signed narinfos id=10 count=14732worker2 # [ 36.307609] nix-grpc-daemon[918]: time=2026-09-21T13:21:39.483Z level=INFO msg="Uploading 1 narinfos"4733lb # [ 36.289816] niks3-server[983]: 2026/09/21 13:21:38 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4734lb # [ 36.298865] niks3-server[983]: 2026/09/21 13:21:38 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4735lb # [ 36.303671] niks3-server[983]: 2026/09/21 13:21:38 INFO Completed upload id=104736worker2 # [ 36.324815] nix-grpc-daemon[918]: time=2026-09-21T13:21:39.501Z level=INFO msg="Upload complete. (213ms)"4737lb # [ 36.305859] niks3-server[983]: 2026/09/21 13:21:38 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4738worker2 # [ 36.331918] nix-grpc-daemon[783]: ts=2026-09-21T13:21:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34620 duration_s=2 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14739worker2 # [ 36.338232] nix-grpc-daemon[783]: ts=2026-09-21T13:21:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34620 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14740worker2 # [ 36.348958] nix-daemon[865]: reaped child process 941, status = succeeded4741client # [ 36.375990] systemd[1]: dedup1.service: Deactivated successfully.4742client # [ 36.379549] systemd[1]: dedup1.service: Consumed 114ms CPU time over 3.723s wall clock time, 41.5M memory peak, 7.1K incoming IP traffic, 8.5K outgoing IP traffic.4743client # [ 36.387397] systemd[1]: dedup2.service: Deactivated successfully.4744client # [ 36.390297] systemd[1]: dedup2.service: Consumed 141ms CPU time over 2.672s wall clock time, 35.4M memory peak, 6.4K incoming IP traffic, 5.9K outgoing IP traffic.4745client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.06 seconds)4746client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4747client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.02 seconds)4748worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4749worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4750worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4751worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4752worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4753worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4754worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4755worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4756(finished: subtest: two clients wanting the same drv build it once, in 4.67 seconds)4757subtest: build hook: nix-daemon with builders = grpc://lb4758client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log4759client # [ 37.396636] systemd[1]: Started Nix Daemon.4760client # [ 37.484213] nix-daemon[946]: accepted connection from pid 943, user root (trusted)4761worker1 # [ 37.585450] nix-grpc-daemon[725]: ts=2026-09-21T13:21:40Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:594504762worker2 # [ 37.605807] nix-grpc-daemon[783]: ts=2026-09-21T13:21:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:346204763worker2 # [ 37.621561] nix-grpc-daemon[783]: ts=2026-09-21T13:21:40Z level=debug event=want client=6 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=04764worker2 # [ 37.625787] nix-grpc-daemon[783]: ts=2026-09-21T13:21:40Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.3:50051 clients=14765worker1 # [ 37.650676] nix-grpc-daemon[725]: ts=2026-09-21T13:21:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594504766worker1 # [ 37.663120] nix-grpc-daemon[725]: ts=2026-09-21T13:21:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:594504767worker1 # [ 37.674710] nix-grpc-daemon[725]: ts=2026-09-21T13:21:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:594504768worker1 # [ 37.690100] nix-daemon[842]: accepted connection from pid 725, user nix-grpc-daemon (trusted)4769worker1 # [ 37.798838] nix-daemon[842]: accepted connection from pid 1043, user nix-grpc-daemon (trusted)4770worker1 # [ 37.816531] nix-daemon[842]: reaped child process 1046, status = succeeded4771worker1 # [ 37.906142] nix-daemon[842]: accepted connection from pid 1048, user nix-grpc-daemon (trusted)4772worker1 # [ 37.924521] nix-daemon[842]: reaped child process 1051, status = succeeded4773lb # [ 37.925910] niks3-server[983]: 2026/09/21 13:21:40 INFO Received uploads request method=POST path=/api/pending_closures4774worker1 # [ 37.944070] nix-grpc-daemon[855]: time=2026-09-21T13:21:41.103Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4775worker1 # [ 37.945711] nix-grpc-daemon[855]: time=2026-09-21T13:21:41.104Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"4776lb # [ 37.962517] niks3-server[983]: 2026/09/21 13:21:40 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst4777lb # [ 37.973137] niks3-server[983]: 2026/09/21 13:21:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4778lb # [ 37.981853] niks3-server[983]: 2026/09/21 13:21:40 INFO Signed narinfos id=11 count=14779worker1 # [ 37.991243] nix-grpc-daemon[855]: time=2026-09-21T13:21:41.150Z level=INFO msg="Uploading 1 narinfos"4780lb # [ 37.994736] niks3-server[983]: 2026/09/21 13:21:40 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls4781lb # [ 38.002476] niks3-server[983]: 2026/09/21 13:21:40 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4782lb # [ 38.011585] niks3-server[983]: 2026/09/21 13:21:40 INFO Completed upload id=114783worker1 # [ 38.017473] nix-grpc-daemon[855]: time=2026-09-21T13:21:41.176Z level=INFO msg="Upload complete. (315ms)"4784lb # [ 38.015039] niks3-server[983]: 2026/09/21 13:21:40 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo4785worker1 # [ 38.022190] nix-grpc-daemon[725]: ts=2026-09-21T13:21:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59450 duration_s=0 paths=1 nar_bytes_in=5124786worker1 # [ 38.028350] nix-grpc-daemon[725]: ts=2026-09-21T13:21:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594504787worker1 # [ 38.041112] nix-daemon[842]: accepted connection from pid 725, user nix-grpc-daemon (trusted)4788worker1 # [ 38.050511] nix-daemon[842]: reaped child process 1041, status = succeeded4789worker1 # [ 38.056295] nix-daemon[842]: accepted connection from pid 725, user nix-grpc-daemon (trusted)4790worker1 # [ 39.473386] nix-daemon[842]: reaped child process 1055, status = succeeded4791worker1 # [ 39.554376] nix-daemon[842]: accepted connection from pid 1072, user nix-grpc-daemon (trusted)4792worker1 # [ 39.570474] nix-daemon[842]: reaped child process 1075, status = succeeded4793lb # [ 39.571587] niks3-server[983]: 2026/09/21 13:21:41 INFO Received uploads request method=POST path=/api/pending_closures4794worker1 # [ 39.586160] nix-grpc-daemon[855]: time=2026-09-21T13:21:42.745Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4795worker1 # [ 39.589074] nix-grpc-daemon[855]: time=2026-09-21T13:21:42.746Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"4796lb # [ 39.611725] niks3-server[983]: 2026/09/21 13:21:41 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4797lb # [ 39.622577] niks3-server[983]: 2026/09/21 13:21:41 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv4798lb # [ 39.633116] niks3-server[983]: 2026/09/21 13:21:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4799worker1 # [ 39.644846] nix-grpc-daemon[855]: time=2026-09-21T13:21:42.803Z level=INFO msg="Uploading 1 narinfos"4800lb # [ 39.638960] niks3-server[983]: 2026/09/21 13:21:41 INFO Signed narinfos id=12 count=14801lb # [ 39.642761] niks3-server[983]: 2026/09/21 13:21:41 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls4802lb # [ 39.652153] niks3-server[983]: 2026/09/21 13:21:42 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4803lb # [ 39.656426] niks3-server[983]: 2026/09/21 13:21:42 INFO Completed upload id=124804lb # [ 39.658239] niks3-server[983]: 2026/09/21 13:21:42 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo4805worker1 # [ 39.664339] nix-grpc-daemon[855]: time=2026-09-21T13:21:42.823Z level=INFO msg="Upload complete. (198ms)"4806worker1 # [ 39.671258] nix-grpc-daemon[725]: ts=2026-09-21T13:21:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59450 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=6 outputs=14807worker1 # [ 39.678196] nix-grpc-daemon[725]: ts=2026-09-21T13:21:42Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:594504808worker1 # [ 39.689279] nix-daemon[842]: reaped child process 1053, status = succeeded4809worker1 # [ 39.695234] nix-grpc-daemon[725]: ts=2026-09-21T13:21:42Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:594504810worker1 # [ 39.699536] nix-grpc-daemon[725]: ts=2026-09-21T13:21:42Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:59450 duration_s=0 paths=1 nar_bytes_out=1284811worker2 # [ 39.919297] nix-grpc-daemon[783]: ts=2026-09-21T13:21:43Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:346204812worker1 # [ 39.912163] nix-grpc-daemon[725]: ts=2026-09-21T13:21:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:594504813worker1 # [ 39.921201] nix-grpc-daemon[855]: time=2026-09-21T13:21:43.080Z level=INFO msg="All 1 paths already cached"4814worker2 # [ 39.949262] nix-grpc-daemon[783]: ts=2026-09-21T13:21:43Z level=debug event=want client=7 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=04815worker2 # [ 39.953400] nix-grpc-daemon[783]: ts=2026-09-21T13:21:43Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.3:50051 clients=14816worker1 # [ 39.945763] nix-grpc-daemon[725]: ts=2026-09-21T13:21:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594504817worker1 # [ 39.957330] nix-grpc-daemon[725]: ts=2026-09-21T13:21:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:594504818worker1 # [ 39.969714] nix-grpc-daemon[855]: time=2026-09-21T13:21:43.128Z level=INFO msg="All 1 paths already cached"4819worker1 # [ 39.977115] nix-grpc-daemon[725]: ts=2026-09-21T13:21:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:594504820worker1 # [ 39.985831] nix-daemon[842]: accepted connection from pid 725, user nix-grpc-daemon (trusted)4821worker1 # [ 40.094575] nix-daemon[842]: accepted connection from pid 1079, user nix-grpc-daemon (trusted)4822worker1 # [ 40.110697] nix-daemon[842]: reaped child process 1082, status = succeeded4823worker1 # [ 40.199166] nix-daemon[842]: accepted connection from pid 1084, user nix-grpc-daemon (trusted)4824worker1 # [ 40.215619] nix-daemon[842]: reaped child process 1087, status = succeeded4825lb # [ 40.216209] niks3-server[983]: 2026/09/21 13:21:42 INFO Received uploads request method=POST path=/api/pending_closures4826worker1 # [ 40.230037] nix-grpc-daemon[855]: time=2026-09-21T13:21:43.389Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4827worker1 # [ 40.231450] nix-grpc-daemon[855]: time=2026-09-21T13:21:43.389Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"4828lb # [ 40.242998] niks3-server[983]: 2026/09/21 13:21:42 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst4829lb # [ 40.251233] niks3-server[983]: 2026/09/21 13:21:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4830lb # [ 40.255767] niks3-server[983]: 2026/09/21 13:21:42 INFO Signed narinfos id=13 count=14831worker1 # [ 40.264874] nix-grpc-daemon[855]: time=2026-09-21T13:21:43.423Z level=INFO msg="Uploading 1 narinfos"4832lb # [ 40.267349] niks3-server[983]: 2026/09/21 13:21:42 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls4833lb # [ 40.275154] niks3-server[983]: 2026/09/21 13:21:42 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4834lb # [ 40.282571] niks3-server[983]: 2026/09/21 13:21:42 INFO Completed upload id=134835worker1 # [ 40.288601] nix-grpc-daemon[855]: time=2026-09-21T13:21:43.447Z level=INFO msg="Upload complete. (288ms)"4836lb # [ 40.285171] niks3-server[983]: 2026/09/21 13:21:42 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo4837worker1 # [ 40.292445] nix-grpc-daemon[725]: ts=2026-09-21T13:21:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59450 duration_s=0 paths=1 nar_bytes_in=6244838worker1 # [ 40.297962] nix-grpc-daemon[725]: ts=2026-09-21T13:21:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594504839worker1 # [ 40.309650] nix-daemon[842]: accepted connection from pid 725, user nix-grpc-daemon (trusted)4840worker1 # [ 40.313373] nix-daemon[842]: reaped child process 1077, status = succeeded4841worker1 # [ 40.317737] nix-daemon[842]: accepted connection from pid 725, user nix-grpc-daemon (trusted)4842worker1 # [ 41.134435] nix-daemon[842]: reaped child process 1091, status = succeeded4843worker1 # [ 41.213602] nix-daemon[842]: accepted connection from pid 1107, user nix-grpc-daemon (trusted)4844worker1 # [ 41.229270] nix-daemon[842]: reaped child process 1110, status = succeeded4845lb # [ 41.229732] niks3-server[983]: 2026/09/21 13:21:43 INFO Received uploads request method=POST path=/api/pending_closures4846worker1 # [ 41.243842] nix-grpc-daemon[855]: time=2026-09-21T13:21:44.402Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4847worker1 # [ 41.246273] nix-grpc-daemon[855]: time=2026-09-21T13:21:44.404Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"4848lb # [ 41.266130] niks3-server[983]: 2026/09/21 13:21:43 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst4849lb # [ 41.273843] niks3-server[983]: 2026/09/21 13:21:43 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv4850lb # [ 41.283737] niks3-server[983]: 2026/09/21 13:21:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4851lb # [ 41.288861] niks3-server[983]: 2026/09/21 13:21:43 INFO Signed narinfos id=14 count=14852worker1 # [ 41.298102] nix-grpc-daemon[855]: time=2026-09-21T13:21:44.457Z level=INFO msg="Uploading 1 narinfos"4853lb # [ 41.296106] niks3-server[983]: 2026/09/21 13:21:43 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls4854lb # [ 41.306299] niks3-server[983]: 2026/09/21 13:21:43 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4855lb # [ 41.309856] niks3-server[983]: 2026/09/21 13:21:43 INFO Completed upload id=144856lb # [ 41.311829] niks3-server[983]: 2026/09/21 13:21:43 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo4857worker1 # [ 41.317337] nix-grpc-daemon[855]: time=2026-09-21T13:21:44.476Z level=INFO msg="Upload complete. (192ms)"4858worker1 # [ 41.322824] nix-grpc-daemon[725]: ts=2026-09-21T13:21:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59450 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=7 outputs=14859worker1 # [ 41.331339] nix-grpc-daemon[725]: ts=2026-09-21T13:21:44Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:594504860worker1 # [ 41.341270] nix-daemon[842]: reaped child process 1089, status = succeeded4861worker1 # [ 41.344736] nix-grpc-daemon[725]: ts=2026-09-21T13:21:44Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:594504862worker1 # [ 41.347818] nix-grpc-daemon[725]: ts=2026-09-21T13:21:44Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:59450 duration_s=0 paths=1 nar_bytes_out=1764863worker2 # [ 41.564255] nix-grpc-daemon[783]: ts=2026-09-21T13:21:44Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:346204864worker1 # [ 41.552381] nix-grpc-daemon[725]: ts=2026-09-21T13:21:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:594504865worker1 # [ 41.562748] nix-grpc-daemon[855]: time=2026-09-21T13:21:44.721Z level=INFO msg="All 2 paths already cached"4866worker2 # [ 41.587850] nix-grpc-daemon[783]: ts=2026-09-21T13:21:44Z level=debug event=want client=8 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=04867worker1 # [ 41.575329] nix-grpc-daemon[725]: ts=2026-09-21T13:21:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594504868worker2 # [ 41.592226] nix-grpc-daemon[783]: ts=2026-09-21T13:21:44Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.3:50051 clients=14869worker1 # [ 41.584506] nix-grpc-daemon[725]: ts=2026-09-21T13:21:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:594504870worker1 # [ 41.593675] nix-grpc-daemon[855]: time=2026-09-21T13:21:44.752Z level=INFO msg="All 2 paths already cached"4871worker1 # [ 41.598163] nix-grpc-daemon[725]: ts=2026-09-21T13:21:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:594504872worker1 # [ 41.605136] nix-daemon[842]: accepted connection from pid 725, user nix-grpc-daemon (trusted)4873worker1 # [ 41.695978] nix-daemon[842]: accepted connection from pid 1115, user nix-grpc-daemon (trusted)4874worker1 # [ 41.711637] nix-daemon[842]: reaped child process 1118, status = succeeded4875worker1 # [ 41.799441] nix-daemon[842]: accepted connection from pid 1120, user nix-grpc-daemon (trusted)4876worker1 # [ 41.815153] nix-daemon[842]: reaped child process 1123, status = succeeded4877lb # [ 41.817124] niks3-server[983]: 2026/09/21 13:21:44 INFO Received uploads request method=POST path=/api/pending_closures4878worker1 # [ 41.830914] nix-grpc-daemon[855]: time=2026-09-21T13:21:44.990Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4879worker1 # [ 41.832502] nix-grpc-daemon[855]: time=2026-09-21T13:21:44.991Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"4880lb # [ 41.846950] niks3-server[983]: 2026/09/21 13:21:44 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst4881lb # [ 41.862212] niks3-server[983]: 2026/09/21 13:21:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4882worker1 # [ 41.872751] nix-grpc-daemon[855]: time=2026-09-21T13:21:45.032Z level=INFO msg="Uploading 1 narinfos"4883lb # [ 41.867149] niks3-server[983]: 2026/09/21 13:21:44 INFO Signed narinfos id=15 count=14884lb # [ 41.872190] niks3-server[983]: 2026/09/21 13:21:44 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls4885lb # [ 41.885559] niks3-server[983]: 2026/09/21 13:21:44 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4886lb # [ 41.891829] niks3-server[983]: 2026/09/21 13:21:44 INFO Completed upload id=154887worker1 # [ 41.897631] nix-grpc-daemon[855]: time=2026-09-21T13:21:45.056Z level=INFO msg="Upload complete. (286ms)"4888lb # [ 41.894413] niks3-server[983]: 2026/09/21 13:21:44 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo4889worker1 # [ 41.901238] nix-grpc-daemon[725]: ts=2026-09-21T13:21:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59450 duration_s=0 paths=1 nar_bytes_in=7524890worker1 # [ 41.906240] nix-grpc-daemon[725]: ts=2026-09-21T13:21:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594504891worker1 # [ 41.918069] nix-daemon[842]: accepted connection from pid 725, user nix-grpc-daemon (trusted)4892worker1 # [ 41.922376] nix-daemon[842]: reaped child process 1113, status = succeeded4893worker1 # [ 41.927575] nix-daemon[842]: accepted connection from pid 725, user nix-grpc-daemon (trusted)4894worker1 # [ 42.743246] nix-daemon[842]: reaped child process 1127, status = succeeded4895worker1 # [ 42.825271] nix-daemon[842]: accepted connection from pid 1143, user nix-grpc-daemon (trusted)4896worker1 # [ 42.840682] nix-daemon[842]: reaped child process 1146, status = succeeded4897lb # [ 42.843386] niks3-server[983]: 2026/09/21 13:21:45 INFO Received uploads request method=POST path=/api/pending_closures4898worker1 # [ 42.858143] nix-grpc-daemon[855]: time=2026-09-21T13:21:46.017Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4899worker1 # [ 42.859645] nix-grpc-daemon[855]: time=2026-09-21T13:21:46.018Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"4900lb # [ 42.878275] niks3-server[983]: 2026/09/21 13:21:45 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv4901lb # [ 42.893569] niks3-server[983]: 2026/09/21 13:21:45 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst4902lb # [ 42.906120] niks3-server[983]: 2026/09/21 13:21:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4903worker1 # [ 42.918360] nix-grpc-daemon[855]: time=2026-09-21T13:21:46.077Z level=INFO msg="Uploading 1 narinfos"4904lb # [ 42.912103] niks3-server[983]: 2026/09/21 13:21:45 INFO Signed narinfos id=16 count=14905lb # [ 42.917497] niks3-server[983]: 2026/09/21 13:21:45 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls4906lb # [ 42.925508] niks3-server[983]: 2026/09/21 13:21:45 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4907lb # [ 42.929067] niks3-server[983]: 2026/09/21 13:21:45 INFO Completed upload id=164908worker1 # [ 42.935414] nix-grpc-daemon[855]: time=2026-09-21T13:21:46.094Z level=INFO msg="Upload complete. (199ms)"4909lb # [ 42.931403] niks3-server[983]: 2026/09/21 13:21:45 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo4910worker1 # [ 42.940820] nix-grpc-daemon[725]: ts=2026-09-21T13:21:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59450 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=8 outputs=14911worker1 # [ 42.955104] nix-daemon[842]: reaped child process 1125, status = succeeded4912worker2 # [ 43.002734] nix-grpc-daemon[783]: ts=2026-09-21T13:21:46Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:346204913worker2 # [ 43.027951] nix-grpc-daemon[783]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...4914worker2 # [ 43.042295] nix-grpc-daemon[783]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...4915worker2 # [ 43.054284] nix-grpc-daemon[783]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...4916worker1 # [ 43.069290] nix-grpc-daemon[725]: ts=2026-09-21T13:21:46Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:594504917worker1 # [ 43.073889] nix-grpc-daemon[725]: ts=2026-09-21T13:21:46Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:59450 duration_s=0 paths=1 nar_bytes_out=2324918client # [ 43.161578] nix-daemon[946]: reaped child process 948, status = succeeded4919client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log, in 5.89 seconds)4920client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top4921client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.02 seconds)4922client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4923client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.02 seconds)4924client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4925client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.02 seconds)4926(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 5.95 seconds)4927subtest: an input only one worker has still reaches the builder4928client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only4929client # this derivation will be built:4930client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv4931client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...4932client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only, in 0.23 seconds)4933client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure4934client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.09 seconds)4935worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4936worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.14 seconds)4937worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only4938worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)4939client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&24940client # [ 43.827154] nix-daemon[946]: accepted connection from pid 1030, user root (trusted)4941client # this derivation will be built:4942client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv4943worker1 # [ 43.926136] nix-grpc-daemon[725]: ts=2026-09-21T13:21:47Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:594504944client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...4945worker2 # [ 43.948526] nix-grpc-daemon[783]: ts=2026-09-21T13:21:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:346204946client # copying 1 paths...4947client # copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' to 'grpc://lb:50051'...4948worker2 # [ 43.960667] nix-grpc-daemon[783]: ts=2026-09-21T13:21:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:346204949worker2 # [ 43.971804] nix-daemon[865]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4950worker2 # [ 44.079219] nix-daemon[865]: accepted connection from pid 1002, user nix-grpc-daemon (trusted)4951worker2 # [ 44.094255] nix-daemon[865]: reaped child process 1005, status = succeeded4952lb # [ 44.078402] niks3-server[983]: 2026/09/21 13:21:46 INFO Received uploads request method=POST path=/api/pending_closures4953worker2 # [ 44.107355] nix-grpc-daemon[918]: time=2026-09-21T13:21:47.284Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4954worker2 # [ 44.109248] nix-grpc-daemon[918]: time=2026-09-21T13:21:47.285Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"4955lb # [ 44.105658] niks3-server[983]: 2026/09/21 13:21:46 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4956lb # [ 44.122061] niks3-server[983]: 2026/09/21 13:21:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4957worker2 # [ 44.152850] nix-grpc-daemon[918]: time=2026-09-21T13:21:47.329Z level=INFO msg="Uploading 1 narinfos"4958lb # [ 44.131425] niks3-server[983]: 2026/09/21 13:21:46 INFO Signed narinfos id=17 count=14959lb # [ 44.137224] niks3-server[983]: 2026/09/21 13:21:46 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls4960lb # [ 44.148640] niks3-server[983]: 2026/09/21 13:21:46 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4961lb # [ 44.152257] niks3-server[983]: 2026/09/21 13:21:46 INFO Completed upload id=174962lb # [ 44.154061] niks3-server[983]: 2026/09/21 13:21:46 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo4963worker2 # [ 44.174677] nix-grpc-daemon[918]: time=2026-09-21T13:21:47.351Z level=INFO msg="Upload complete. (191ms)"4964worker2 # [ 44.179714] nix-grpc-daemon[783]: ts=2026-09-21T13:21:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34620 duration_s=0 paths=1 nar_bytes_in=1204965worker2 # [ 44.191700] nix-grpc-daemon[783]: ts=2026-09-21T13:21:47Z level=debug event=want client=9 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=04966worker2 # [ 44.196555] nix-grpc-daemon[783]: ts=2026-09-21T13:21:47Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=14967worker2 # [ 44.204203] nix-daemon[865]: reaped child process 1000, status = succeeded4968worker2 # [ 44.235395] nix-grpc-daemon[783]: ts=2026-09-21T13:21:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:346204969worker2 # [ 44.247888] nix-grpc-daemon[783]: ts=2026-09-21T13:21:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:346204970client # copying 2 paths to 'lb:50051'...4971worker2 # [ 44.263281] nix-grpc-daemon[783]: ts=2026-09-21T13:21:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:346204972client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...4973client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...4974worker2 # [ 44.279168] nix-daemon[865]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4975worker2 # [ 44.378992] nix-daemon[865]: accepted connection from pid 1011, user nix-grpc-daemon (trusted)4976worker2 # [ 44.394043] nix-daemon[865]: reaped child process 1014, status = succeeded4977worker2 # [ 44.473668] nix-daemon[865]: accepted connection from pid 1016, user nix-grpc-daemon (trusted)4978worker2 # [ 44.489566] nix-daemon[865]: reaped child process 1019, status = succeeded4979lb # [ 44.473430] niks3-server[983]: 2026/09/21 13:21:46 INFO Received uploads request method=POST path=/api/pending_closures4980lb # [ 44.482923] niks3-server[983]: 2026/09/21 13:21:46 INFO Received uploads request method=POST path=/api/pending_closures4981worker2 # [ 44.512608] nix-grpc-daemon[918]: time=2026-09-21T13:21:47.689Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4982worker2 # [ 44.514072] nix-grpc-daemon[918]: time=2026-09-21T13:21:47.689Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"4983worker2 # [ 44.515614] nix-grpc-daemon[918]: time=2026-09-21T13:21:47.689Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"4984lb # [ 44.514805] niks3-server[983]: 2026/09/21 13:21:46 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst4985lb # [ 44.524599] niks3-server[983]: 2026/09/21 13:21:46 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst4986lb # [ 44.534543] niks3-server[983]: 2026/09/21 13:21:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4987lb # [ 44.537858] niks3-server[983]: 2026/09/21 13:21:46 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls4988lb # [ 44.539258] niks3-server[983]: 2026/09/21 13:21:46 INFO Signed narinfos id=18 count=14989lb # [ 44.542127] niks3-server[983]: 2026/09/21 13:21:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4990lb # [ 44.544049] niks3-server[983]: 2026/09/21 13:21:46 INFO Signed narinfos id=19 count=24991worker2 # [ 44.565548] nix-grpc-daemon[918]: time=2026-09-21T13:21:47.742Z level=INFO msg="Uploading 3 narinfos"4992lb # [ 44.548027] niks3-server[983]: 2026/09/21 13:21:46 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls4993lb # [ 44.559336] niks3-server[983]: 2026/09/21 13:21:46 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo4994lb # [ 44.564424] niks3-server[983]: 2026/09/21 13:21:46 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4995lb # [ 44.569221] niks3-server[983]: 2026/09/21 13:21:46 INFO Completed upload id=184996lb # [ 44.570240] niks3-server[983]: 2026/09/21 13:21:46 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4997lb # [ 44.573510] niks3-server[983]: 2026/09/21 13:21:46 INFO Completed upload id=194998worker2 # [ 44.594794] nix-grpc-daemon[918]: time=2026-09-21T13:21:47.771Z level=INFO msg="Upload complete. (301ms)"4999lb # [ 44.576278] niks3-server[983]: 2026/09/21 13:21:46 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5000lb # [ 44.580046] niks3-server[983]: 2026/09/21 13:21:46 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5001worker2 # [ 44.599473] nix-grpc-daemon[783]: ts=2026-09-21T13:21:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34620 duration_s=0 paths=2 nar_bytes_in=9765002worker2 # [ 44.605593] nix-grpc-daemon[783]: ts=2026-09-21T13:21:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:346205003worker2 # [ 44.616115] nix-daemon[865]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5004worker2 # [ 44.625424] nix-daemon[865]: reaped child process 1009, status = succeeded5005worker2 # [ 44.628341] nix-daemon[865]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5006client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5007client # uses-input-w1onlya> worker2: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5008client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5009worker2 # [ 45.464625] nix-daemon[865]: reaped child process 1023, status = succeeded5010worker2 # [ 45.546347] nix-daemon[865]: accepted connection from pid 1039, user nix-grpc-daemon (trusted)5011worker2 # [ 45.558912] nix-daemon[865]: reaped child process 1042, status = succeeded5012lb # [ 45.546414] niks3-server[983]: 2026/09/21 13:21:47 INFO Received uploads request method=POST path=/api/pending_closures5013worker2 # [ 45.575456] nix-grpc-daemon[918]: time=2026-09-21T13:21:48.752Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5014lb # [ 45.556311] envoy[916]: conn peer=[2001:db8:1::1]:41000 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2998 tx=1451 ms=35585015lb # [ 45.561384] envoy[916]: conn peer=[2001:db8:1::1]:41004 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1589 tx=1486 ms=24295016lb # [ 45.565136] envoy[916]: conn peer=[2001:db8:1::1]:41018 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3420 tx=1715 ms=21735017lb # [ 45.569436] envoy[916]: conn peer=[2001:db8:1::1]:41020 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=615018lb # [ 45.573291] envoy[916]: conn peer=[2001:db8:1::1]:41030 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3849 tx=1940 ms=14935019lb # [ 45.576843] envoy[916]: conn peer=[2001:db8:1::1]:41034 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=378 ms=545020lb # [ 45.580269] envoy[916]: conn peer=[2001:db8:1::1]:41036 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4326 tx=2276 ms=15375021lb # [ 45.585333] envoy[916]: conn peer=[2001:db8:1::1]:40244 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=195022lb # [ 45.602698] niks3-server[983]: 2026/09/21 13:21:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5023lb # [ 45.609126] niks3-server[983]: 2026/09/21 13:21:47 INFO Signed narinfos id=20 count=15024worker2 # [ 45.632343] nix-grpc-daemon[918]: time=2026-09-21T13:21:48.809Z level=INFO msg="Uploading 1 narinfos"5025lb # [ 45.615993] niks3-server[983]: 2026/09/21 13:21:47 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5026lb # [ 45.618897] niks3-server[983]: 2026/09/21 13:21:47 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5027lb # [ 45.623982] niks3-server[983]: 2026/09/21 13:21:47 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5028lb # [ 45.628534] niks3-server[983]: 2026/09/21 13:21:47 INFO Completed upload id=205029lb # [ 45.631246] niks3-server[983]: 2026/09/21 13:21:47 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5030worker2 # [ 45.651211] nix-grpc-daemon[918]: time=2026-09-21T13:21:48.828Z level=INFO msg="Upload complete. (215ms)"5031worker2 # [ 45.657441] nix-grpc-daemon[783]: ts=2026-09-21T13:21:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34620 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=9 outputs=15032worker2 # [ 45.672088] nix-daemon[865]: reaped child process 1021, status = succeeded5033client # copying 1 paths...5034worker1 # [ 45.689104] nix-grpc-daemon[725]: ts=2026-09-21T13:21:48Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:594505035worker1 # [ 45.701485] nix-grpc-daemon[725]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5036client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5037worker1 # [ 45.727119] nix-grpc-daemon[725]: ts=2026-09-21T13:21:48Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:594505038worker1 # [ 45.730889] nix-grpc-daemon[725]: ts=2026-09-21T13:21:48Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:59450 duration_s=0 paths=1 nar_bytes_out=1205039client # [ 45.844367] nix-daemon[946]: reaped child process 1033, status = succeeded5040client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.14 seconds)5041client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&25042client # [ 45.955425] nix-daemon[946]: accepted connection from pid 1052, user root (trusted)5043client # this derivation will be built:5044client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5045worker1 # [ 46.054343] nix-grpc-daemon[725]: ts=2026-09-21T13:21:49Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:594505046client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5047worker2 # [ 46.075977] nix-grpc-daemon[783]: ts=2026-09-21T13:21:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:346205048worker2 # [ 46.085408] nix-grpc-daemon[918]: time=2026-09-21T13:21:49.262Z level=INFO msg="All 1 paths already cached"5049worker2 # [ 46.098831] nix-grpc-daemon[783]: ts=2026-09-21T13:21:49Z level=debug event=want client=10 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05050worker2 # [ 46.103558] nix-grpc-daemon[783]: ts=2026-09-21T13:21:49Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15051worker1 # [ 46.091161] nix-grpc-daemon[725]: ts=2026-09-21T13:21:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594505052worker1 # [ 46.103195] nix-grpc-daemon[725]: ts=2026-09-21T13:21:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:594505053client # copying 1 paths to 'lb:50051'...5054client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5055worker1 # [ 46.119926] nix-grpc-daemon[725]: ts=2026-09-21T13:21:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:594505056worker1 # [ 46.128854] nix-daemon[842]: accepted connection from pid 725, user nix-grpc-daemon (trusted)5057worker1 # [ 46.147207] nix-grpc-daemon[725]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 36 ms (attempt 1/5)5058worker1 # [ 46.190595] nix-grpc-daemon[725]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 174 ms (attempt 2/5)5059worker1 # [ 46.366955] nix-grpc-daemon[725]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 238 ms (attempt 3/5)5060worker1 # [ 46.606942] nix-grpc-daemon[725]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 496 ms (attempt 4/5)5061worker1 # [ 47.105889] nix-grpc-daemon[725]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5062worker1 # [ 47.109845] nix-grpc-daemon[725]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5063worker1 # [ 47.115304] nix-grpc-daemon[725]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5064worker1 # [ 47.236625] nix-daemon[842]: accepted connection from pid 1161, user nix-grpc-daemon (trusted)5065worker1 # [ 47.250878] nix-daemon[842]: reaped child process 1164, status = succeeded5066worker1 # [ 47.338552] nix-daemon[842]: accepted connection from pid 1166, user nix-grpc-daemon (trusted)5067worker1 # [ 47.354943] nix-daemon[842]: reaped child process 1169, status = succeeded5068lb # [ 47.358751] niks3-server[983]: 2026/09/21 13:21:49 INFO Received uploads request method=POST path=/api/pending_closures5069worker1 # [ 47.371476] nix-grpc-daemon[855]: time=2026-09-21T13:21:50.530Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5070worker1 # [ 47.372894] nix-grpc-daemon[855]: time=2026-09-21T13:21:50.530Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5071lb # [ 47.379941] niks3-server[983]: 2026/09/21 13:21:49 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5072lb # [ 47.394728] niks3-server[983]: 2026/09/21 13:21:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5073worker1 # [ 47.405891] nix-grpc-daemon[855]: time=2026-09-21T13:21:50.565Z level=INFO msg="Uploading 1 narinfos"5074lb # [ 47.400186] niks3-server[983]: 2026/09/21 13:21:49 INFO Signed narinfos id=21 count=15075lb # [ 47.406268] niks3-server[983]: 2026/09/21 13:21:49 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5076lb # [ 47.420904] niks3-server[983]: 2026/09/21 13:21:49 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5077worker1 # [ 47.433209] nix-grpc-daemon[855]: time=2026-09-21T13:21:50.592Z level=INFO msg="Upload complete. (299ms)"5078lb # [ 47.427216] niks3-server[983]: 2026/09/21 13:21:49 INFO Completed upload id=215079worker1 # [ 47.435382] nix-grpc-daemon[725]: ts=2026-09-21T13:21:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59450 duration_s=1 paths=1 nar_bytes_in=5605080lb # [ 47.430261] niks3-server[983]: 2026/09/21 13:21:49 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5081worker1 # [ 47.439605] nix-grpc-daemon[725]: ts=2026-09-21T13:21:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594505082worker1 # [ 47.449845] nix-daemon[842]: accepted connection from pid 725, user nix-grpc-daemon (trusted)5083worker1 # [ 47.454185] nix-daemon[842]: reaped child process 1154, status = succeeded5084worker1 # [ 47.459387] nix-daemon[842]: accepted connection from pid 725, user nix-grpc-daemon (trusted)5085client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5086client # uses-input-w1onlyb> node-a: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5087client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5088worker1 # [ 48.475257] nix-daemon[842]: reaped child process 1173, status = succeeded5089worker1 # [ 48.555093] nix-daemon[842]: accepted connection from pid 1189, user nix-grpc-daemon (trusted)5090worker1 # [ 48.567366] nix-daemon[842]: reaped child process 1192, status = succeeded5091lb # [ 48.569491] niks3-server[983]: 2026/09/21 13:21:50 INFO Received uploads request method=POST path=/api/pending_closures5092worker1 # [ 48.583387] nix-grpc-daemon[855]: time=2026-09-21T13:21:51.742Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5093lb # [ 48.599675] niks3-server[983]: 2026/09/21 13:21:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5094worker1 # [ 48.613594] nix-grpc-daemon[855]: time=2026-09-21T13:21:51.772Z level=INFO msg="Uploading 1 narinfos"5095lb # [ 48.606939] niks3-server[983]: 2026/09/21 13:21:50 INFO Signed narinfos id=22 count=15096lb # [ 48.615327] niks3-server[983]: 2026/09/21 13:21:50 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5097lb # [ 48.621155] niks3-server[983]: 2026/09/21 13:21:50 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5098lb # [ 48.631326] niks3-server[983]: 2026/09/21 13:21:50 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5099lb # [ 48.637352] niks3-server[983]: 2026/09/21 13:21:50 INFO Completed upload id=225100worker1 # [ 48.645689] nix-grpc-daemon[855]: time=2026-09-21T13:21:51.804Z level=INFO msg="Upload complete. (179ms)"5101lb # [ 48.640581] niks3-server[983]: 2026/09/21 13:21:50 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5102client # copying 1 paths...5103worker1 # [ 48.652285] nix-grpc-daemon[725]: ts=2026-09-21T13:21:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59450 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=10 outputs=15104worker2 # [ 48.671907] nix-grpc-daemon[783]: ts=2026-09-21T13:21:51Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:346205105worker1 # [ 48.665226] nix-daemon[842]: reaped child process 1171, status = succeeded5106worker2 # [ 48.681848] nix-grpc-daemon[783]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5107client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5108worker2 # [ 48.703917] nix-grpc-daemon[783]: ts=2026-09-21T13:21:51Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:346205109worker2 # [ 48.709905] nix-grpc-daemon[783]: ts=2026-09-21T13:21:51Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34620 duration_s=0 paths=1 nar_bytes_out=1205110client # [ 48.814776] nix-daemon[946]: reaped child process 1055, status = succeeded5111client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.97 seconds)5112client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&25113client # [ 48.914792] nix-daemon[946]: accepted connection from pid 1074, user root (trusted)5114client # this derivation will be built:5115client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5116worker2 # [ 49.028222] nix-grpc-daemon[783]: ts=2026-09-21T13:21:52Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:346205117client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5118worker1 # [ 49.016592] nix-grpc-daemon[725]: ts=2026-09-21T13:21:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:594505119worker1 # [ 49.025672] nix-grpc-daemon[855]: time=2026-09-21T13:21:52.184Z level=INFO msg="All 1 paths already cached"5120worker2 # [ 49.054615] nix-grpc-daemon[783]: ts=2026-09-21T13:21:52Z level=debug event=want client=11 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05121worker2 # [ 49.061285] nix-grpc-daemon[783]: ts=2026-09-21T13:21:52Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15122worker2 # [ 49.066278] nix-grpc-daemon[783]: ts=2026-09-21T13:21:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:346205123worker2 # [ 49.077694] nix-grpc-daemon[783]: ts=2026-09-21T13:21:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:346205124worker2 # [ 49.090849] nix-grpc-daemon[918]: time=2026-09-21T13:21:52.267Z level=INFO msg="All 1 paths already cached"5125client # copying 1 paths to 'lb:50051'...5126client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5127worker2 # [ 49.098699] nix-grpc-daemon[783]: ts=2026-09-21T13:21:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:346205128worker2 # [ 49.109935] nix-daemon[865]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5129worker2 # [ 49.204792] nix-daemon[865]: accepted connection from pid 1051, user nix-grpc-daemon (trusted)5130worker2 # [ 49.218165] nix-daemon[865]: reaped child process 1054, status = succeeded5131worker2 # [ 49.298300] nix-daemon[865]: accepted connection from pid 1056, user nix-grpc-daemon (trusted)5132worker2 # [ 49.311991] nix-daemon[865]: reaped child process 1059, status = succeeded5133lb # [ 49.295533] niks3-server[983]: 2026/09/21 13:21:51 INFO Received uploads request method=POST path=/api/pending_closures5134worker2 # [ 49.325145] nix-grpc-daemon[918]: time=2026-09-21T13:21:52.502Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5135worker2 # [ 49.327123] nix-grpc-daemon[918]: time=2026-09-21T13:21:52.503Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5136lb # [ 49.327758] niks3-server[983]: 2026/09/21 13:21:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5137lb # [ 49.332850] niks3-server[983]: 2026/09/21 13:21:51 INFO Signed narinfos id=23 count=15138worker2 # [ 49.357172] nix-grpc-daemon[918]: time=2026-09-21T13:21:52.533Z level=INFO msg="Uploading 1 narinfos"5139lb # [ 49.338411] niks3-server[983]: 2026/09/21 13:21:51 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5140lb # [ 49.351801] niks3-server[983]: 2026/09/21 13:21:51 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5141lb # [ 49.356833] niks3-server[983]: 2026/09/21 13:21:51 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5142worker2 # [ 49.384451] nix-grpc-daemon[918]: time=2026-09-21T13:21:52.561Z level=INFO msg="Upload complete. (262ms)"5143lb # [ 49.363870] niks3-server[983]: 2026/09/21 13:21:51 INFO Completed upload id=235144worker2 # [ 49.386481] nix-grpc-daemon[783]: ts=2026-09-21T13:21:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34620 duration_s=0 paths=1 nar_bytes_in=5605145worker2 # [ 49.389927] nix-grpc-daemon[783]: ts=2026-09-21T13:21:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:346205146lb # [ 49.370248] niks3-server[983]: 2026/09/21 13:21:51 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5147worker2 # [ 49.394610] nix-daemon[865]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5148worker2 # [ 49.399380] nix-daemon[865]: reaped child process 1049, status = succeeded5149worker2 # [ 49.404213] nix-daemon[865]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5150client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5151client # uses-input-w1onlyc> worker2: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5152client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5153worker2 # [ 50.035293] nix-daemon[865]: reaped child process 1063, status = succeeded5154worker2 # [ 50.114233] nix-daemon[865]: accepted connection from pid 1079, user nix-grpc-daemon (trusted)5155worker2 # [ 50.126053] nix-daemon[865]: reaped child process 1082, status = succeeded5156lb # [ 50.112237] niks3-server[983]: 2026/09/21 13:21:52 INFO Received uploads request method=POST path=/api/pending_closures5157worker2 # [ 50.142258] nix-grpc-daemon[918]: time=2026-09-21T13:21:53.319Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5158lb # [ 50.142768] niks3-server[983]: 2026/09/21 13:21:52 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5159lb # [ 50.146773] niks3-server[983]: 2026/09/21 13:21:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5160lb # [ 50.148713] niks3-server[983]: 2026/09/21 13:21:52 INFO Signed narinfos id=24 count=15161worker2 # [ 50.171174] nix-grpc-daemon[918]: time=2026-09-21T13:21:53.347Z level=INFO msg="Uploading 1 narinfos"5162lb # [ 50.153800] niks3-server[983]: 2026/09/21 13:21:52 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5163lb # [ 50.158689] niks3-server[983]: 2026/09/21 13:21:52 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5164lb # [ 50.161886] niks3-server[983]: 2026/09/21 13:21:52 INFO Completed upload id=245165worker2 # [ 50.182905] nix-grpc-daemon[918]: time=2026-09-21T13:21:53.359Z level=INFO msg="Upload complete. (171ms)"5166lb # [ 50.165280] niks3-server[983]: 2026/09/21 13:21:52 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5167worker2 # [ 50.185227] nix-grpc-daemon[783]: ts=2026-09-21T13:21:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34620 duration_s=0 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=11 outputs=15168worker2 # [ 50.192190] nix-daemon[865]: reaped child process 1061, status = succeeded5169client # copying 1 paths...5170worker2 # [ 50.230403] nix-grpc-daemon[783]: ts=2026-09-21T13:21:53Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:346205171client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5172worker2 # [ 50.253489] nix-grpc-daemon[783]: ts=2026-09-21T13:21:53Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:346205173worker2 # [ 50.257904] nix-grpc-daemon[783]: ts=2026-09-21T13:21:53Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34620 duration_s=0 paths=1 nar_bytes_out=1205174client # [ 50.367361] nix-daemon[946]: reaped child process 1077, status = succeeded5175client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&2, in 1.55 seconds)5176(finished: subtest: an input only one worker has still reaches the builder, in 7.14 seconds)5177subtest: upload whose reference the worker lacks is completed from the cache5178client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache5179client # this derivation will be built:5180client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5181client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5182client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache, in 0.24 seconds)5183client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache5184client # this derivation will be built:5185client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5186client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5187client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache, in 0.23 seconds)5188client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5189client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.10 seconds)5190worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5191worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.13 seconds)5192worker2 # [ 51.120136] systemd[1]: Stopping Nix worker-protocol over gRPC...5193worker2 # [ 51.804629] nix-grpc-daemon[783]: ts=2026-09-21T13:21:54Z level=info event=signal_exit5194worker2 # [ 52.586748] nix-grpc-daemon[783]: ts=2026-09-21T13:21:55Z level=info event=scheduler_yield5195lb # [ 52.568896] niks3-server[983]: 2026/09/21 13:21:54 INFO lead: released remote=192.168.1.4:463525196worker2 # [ 52.591716] nix-grpc-daemon[783]: ts=2026-09-21T13:21:55Z level=info event=scheduler_restarting addr=lb:500515197worker1 # [ 52.581234] nix-grpc-daemon[725]: ts=2026-09-21T13:21:55Z level=info event=scheduler_restarting addr=lb:500515198worker2 # [ 52.597943] nix-grpc-daemon[783]: ts=2026-09-21T13:21:55Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:346125199worker2 # [ 52.601887] nix-grpc-daemon[783]: ts=2026-09-21T13:21:55Z level=info event=worker_gone addr=192.168.1.4:500515200worker2 # [ 52.605414] nix-grpc-daemon[783]: ts=2026-09-21T13:21:55Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:346125201worker2 # [ 52.609253] nix-grpc-daemon[783]: ts=2026-09-21T13:21:55Z level=info event=worker_gone addr=192.168.1.3:500515202worker2 # [ 52.792799] nix-grpc-daemon[783]: ts=2026-09-21T13:21:55Z level=info event=scheduler_connected addr=lb:500515203worker1 # [ 52.781466] nix-grpc-daemon[725]: ts=2026-09-21T13:21:55Z level=info event=scheduler_connected addr=lb:500515204worker2 # [ 52.797357] nix-grpc-daemon[783]: ts=2026-09-21T13:21:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5205worker1 # [ 52.785538] nix-grpc-daemon[725]: ts=2026-09-21T13:21:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5206worker2 # [ 52.999656] nix-grpc-daemon[783]: ts=2026-09-21T13:21:56Z level=info event=scheduler_connected addr=lb:500515207worker1 # [ 52.986748] nix-grpc-daemon[725]: ts=2026-09-21T13:21:56Z level=info event=scheduler_connected addr=lb:500515208worker2 # [ 53.004575] nix-grpc-daemon[783]: ts=2026-09-21T13:21:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5209worker1 # [ 52.990949] nix-grpc-daemon[725]: ts=2026-09-21T13:21:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5210worker2 # [ 53.201628] nix-grpc-daemon[783]: ts=2026-09-21T13:21:56Z level=info event=scheduler_connected addr=lb:500515211worker1 # [ 53.192109] nix-grpc-daemon[725]: ts=2026-09-21T13:21:56Z level=info event=scheduler_connected addr=lb:500515212worker2 # [ 53.206162] nix-grpc-daemon[783]: ts=2026-09-21T13:21:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5213worker1 # [ 53.196160] nix-grpc-daemon[725]: ts=2026-09-21T13:21:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5214worker2 # [ 53.406985] nix-grpc-daemon[783]: ts=2026-09-21T13:21:56Z level=info event=scheduler_connected addr=lb:500515215worker1 # [ 53.397343] nix-grpc-daemon[725]: ts=2026-09-21T13:21:56Z level=info event=scheduler_connected addr=lb:500515216worker2 # [ 53.411564] nix-grpc-daemon[783]: ts=2026-09-21T13:21:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5217worker1 # [ 53.401532] nix-grpc-daemon[725]: ts=2026-09-21T13:21:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5218worker2 # [ 53.592200] nix-grpc-daemon[783]: ts=2026-09-21T13:21:56Z level=info event=scheduler_restarting5219worker1 # [ 53.603899] nix-grpc-daemon[725]: ts=2026-09-21T13:21:56Z level=info event=scheduler_connected addr=lb:500515220worker1 # [ 53.607616] nix-grpc-daemon[725]: ts=2026-09-21T13:21:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5221worker2 # [ 53.624441] nix-grpc-daemon[783]: ts=2026-09-21T13:21:56Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5222worker2 # [ 53.649661] nix-daemon[865]: reaped child process 867, status = succeeded5223worker1 # [ 53.805068] nix-grpc-daemon[725]: ts=2026-09-21T13:21:56Z level=info event=scheduler_connected addr=lb:500515224worker1 # [ 53.809168] nix-grpc-daemon[725]: ts=2026-09-21T13:21:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5225worker2 # [ 53.898444] nix-grpc-daemon[783]: E0000 00:00:1789996917.075076 828 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5226worker2 # [ 53.903614] nix-grpc-daemon[783]: E0000 00:00:1789996917.075209 828 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5227worker1 # [ 54.010314] nix-grpc-daemon[725]: ts=2026-09-21T13:21:57Z level=info event=scheduler_connected addr=lb:500515228worker1 # [ 54.014474] nix-grpc-daemon[725]: ts=2026-09-21T13:21:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5229worker1 # [ 54.215734] nix-grpc-daemon[725]: ts=2026-09-21T13:21:57Z level=info event=scheduler_connected addr=lb:500515230worker1 # [ 54.219878] nix-grpc-daemon[725]: ts=2026-09-21T13:21:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5231worker1 # [ 54.420884] nix-grpc-daemon[725]: ts=2026-09-21T13:21:57Z level=info event=scheduler_connected addr=lb:500515232worker1 # [ 54.424968] nix-grpc-daemon[725]: ts=2026-09-21T13:21:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5233lb # [ 54.563250] niks3-server[983]: 2026/09/21 13:21:56 INFO lead: acquired remote=192.168.1.3:436125234worker1 # [ 54.569200] nix-grpc-daemon[725]: ts=2026-09-21T13:21:57Z level=info event=scheduler_take_over5235worker1 # [ 54.626730] nix-grpc-daemon[725]: ts=2026-09-21T13:21:57Z level=info event=scheduler_connected addr=lb:500515236worker1 # [ 54.630484] nix-grpc-daemon[725]: ts=2026-09-21T13:21:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5237worker1 # [ 54.828725] nix-grpc-daemon[725]: ts=2026-09-21T13:21:57Z level=info event=scheduler_connected addr=lb:500515238worker1 # [ 54.832819] nix-grpc-daemon[725]: ts=2026-09-21T13:21:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5239worker1 # [ 55.033952] nix-grpc-daemon[725]: ts=2026-09-21T13:21:58Z level=info event=scheduler_connected addr=lb:500515240worker1 # [ 55.045058] nix-grpc-daemon[725]: ts=2026-09-21T13:21:58Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:414945241worker1 # [ 55.049584] nix-grpc-daemon[725]: ts=2026-09-21T13:21:58Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05242worker2 # [ 55.319173] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5243worker2 # [ 55.323074] systemd[1]: Stopped Nix worker-protocol over gRPC.5244worker2 # [ 55.327488] systemd[1]: nix-grpc-daemon.service: Consumed 1.380s CPU time over 39.666s wall clock time, 58.7M memory peak, 53.4K incoming IP traffic, 141.6K outgoing IP traffic.5245worker2 # [ 55.339427] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5246worker2 # [ 55.343714] systemd[1]: Closed Nix worker-protocol over gRPC.5247worker2 # [ 55.347443] systemd[1]: nix-grpc-daemon.socket: Consumed 324us CPU time over 47.825s wall clock time, 256K memory peak, 64.4K incoming IP traffic, 54.5K outgoing IP traffic.5248lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5249lb: must succeed: curl -sf localhost:9901/clusters5250lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5251lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.05 seconds)5252client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25253worker1 # [ 55.534765] nix-grpc-daemon[725]: ts=2026-09-21T13:21:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:594505254client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5255worker1 # [ 55.540166] nix-grpc-daemon[725]: ts=2026-09-21T13:21:58Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:594505256client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.17 seconds)5257worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5258lb # [ 55.556766] envoy[916]: conn peer=[2001:db8:1::1]:40246 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4212 tx=1942 ms=18505259lb # [ 55.560463] envoy[916]: conn peer=[2001:db8:1::1]:40258 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=505260lb # [ 55.563925] envoy[916]: conn peer=[2001:db8:1::1]:40282 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=555261lb # [ 55.567352] envoy[916]: conn peer=[2001:db8:1::1]:40268 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3739 tx=2037 ms=26925262worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds)5263lb # [ 55.571393] envoy[916]: conn peer=[2001:db8:1::1]:40292 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3723 tx=1882 ms=12815264lb # [ 55.575059] envoy[916]: conn peer=[2001:db8:1::1]:40296 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=585265lb # [ 55.578456] envoy[916]: conn peer=[2001:db8:1::4]:42546 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=21606 tx=1321 ms=377745266lb # [ 55.582202] envoy[916]: conn peer=[2001:db8:1::1]:41114 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=165267worker2 # [ 55.636309] systemd[1]: Listening on Nix worker-protocol over gRPC.5268worker1 # [ 55.669857] systemd[1]: Stopping Nix worker-protocol over gRPC...5269worker1 # [ 55.748862] nix-grpc-daemon[725]: ts=2026-09-21T13:21:58Z level=info event=signal_exit5270worker2 # [ 56.315635] systemd[1]: Starting scheduler-token.service...5271worker2 # [ 56.377978] systemd[1]: scheduler-token.service: Deactivated successfully.5272worker2 # [ 56.381092] systemd[1]: Finished scheduler-token.service.5273worker2 # [ 56.384442] systemd[1]: scheduler-token.service: Consumed 34ms CPU time over 63ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5274worker2 # [ 56.391275] systemd[1]: Starting Nix worker-protocol over gRPC...5275worker2 # [ 56.470127] nix-grpc-daemon[1106]: ts=2026-09-21T13:21:59Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5276worker2 # [ 56.472128] nix-grpc-daemon[1106]: ts=2026-09-21T13:21:59Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5277worker2 # [ 56.475480] nix-grpc-daemon[1106]: ts=2026-09-21T13:21:59Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15278worker2 # [ 56.479599] nix-grpc-daemon[1106]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5279worker2 # [ 56.482171] nix-grpc-daemon[1106]: E0000 00:00:1789996919.656405 1106 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5280worker2 # [ 56.487552] nix-grpc-daemon[1106]: E0000 00:00:1789996919.661907 1106 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5281worker2 # [ 56.490833] nix-grpc-daemon[1106]: ts=2026-09-21T13:21:59Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5282worker2 # [ 56.492486] systemd[1]: Started Nix worker-protocol over gRPC.5283worker2 # [ 56.498057] nix-grpc-daemon[1106]: E0000 00:00:1789996919.675002 1147 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5284worker2 # [ 56.512054] nix-grpc-daemon[1106]: E0000 00:00:1789996919.688988 1147 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5285worker2 # [ 56.517054] nix-grpc-daemon[1106]: E0000 00:00:1789996919.691350 1147 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5286worker2 # [ 56.526107] nix-grpc-daemon[1106]: E0000 00:00:1789996919.703003 1147 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5287worker2 # [ 56.535179] nix-grpc-daemon[1106]: ts=2026-09-21T13:21:59Z level=info event=scheduler_connected addr=lb:500515288worker2 # [ 56.536709] nix-grpc-daemon[1106]: ts=2026-09-21T13:21:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5289worker1 # [ 56.572190] nix-grpc-daemon[725]: ts=2026-09-21T13:21:59Z level=info event=scheduler_yield5290lb # [ 56.568841] niks3-server[983]: 2026/09/21 13:21:58 INFO lead: released remote=192.168.1.3:436125291worker1 # [ 56.579638] nix-grpc-daemon[725]: ts=2026-09-21T13:21:59Z level=info event=scheduler_restarting addr=lb:500515292worker1 # [ 56.584585] nix-grpc-daemon[725]: ts=2026-09-21T13:21:59Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:414945293worker1 # [ 56.588106] nix-grpc-daemon[725]: ts=2026-09-21T13:21:59Z level=info event=worker_gone addr=192.168.1.3:500515294worker2 # [ 56.738091] nix-grpc-daemon[1106]: ts=2026-09-21T13:21:59Z level=info event=scheduler_connected addr=lb:500515295worker2 # [ 56.742168] nix-grpc-daemon[1106]: ts=2026-09-21T13:21:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5296worker1 # [ 56.782421] nix-grpc-daemon[725]: ts=2026-09-21T13:21:59Z level=info event=scheduler_connected addr=lb:500515297worker1 # [ 56.785616] nix-grpc-daemon[725]: ts=2026-09-21T13:21:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5298worker2 # [ 56.943382] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:00Z level=info event=scheduler_connected addr=lb:500515299worker2 # [ 56.947492] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5300worker1 # [ 56.983638] nix-grpc-daemon[725]: ts=2026-09-21T13:22:00Z level=info event=scheduler_connected addr=lb:500515301worker1 # [ 56.987776] nix-grpc-daemon[725]: ts=2026-09-21T13:22:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5302worker2 # [ 57.148805] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:00Z level=info event=scheduler_connected addr=lb:500515303worker2 # [ 57.152816] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5304worker1 # [ 57.188734] nix-grpc-daemon[725]: ts=2026-09-21T13:22:00Z level=info event=scheduler_connected addr=lb:500515305worker1 # [ 57.192755] nix-grpc-daemon[725]: ts=2026-09-21T13:22:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5306worker2 # [ 57.353897] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:00Z level=info event=scheduler_connected addr=lb:500515307worker2 # [ 57.357869] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5308worker1 # [ 57.393584] nix-grpc-daemon[725]: ts=2026-09-21T13:22:00Z level=info event=scheduler_connected addr=lb:500515309worker1 # [ 57.397535] nix-grpc-daemon[725]: ts=2026-09-21T13:22:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5310worker2 # [ 57.560437] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:00Z level=info event=scheduler_connected addr=lb:500515311worker2 # [ 57.564258] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5312worker1 # [ 57.577189] nix-grpc-daemon[725]: ts=2026-09-21T13:22:00Z level=info event=scheduler_restarting5313worker1 # [ 57.617278] nix-grpc-daemon[725]: ts=2026-09-21T13:22:00Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5314worker1 # [ 57.622371] nix-daemon[842]: reaped child process 844, status = succeeded5315worker2 # [ 57.761819] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:00Z level=info event=scheduler_connected addr=lb:500515316worker2 # [ 57.766298] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5317worker1 # [ 57.844137] nix-grpc-daemon[725]: E0000 00:00:1789996921.002199 770 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true5318worker1 # [ 57.848950] nix-grpc-daemon[725]: E0000 00:00:1789996921.002457 770 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5319worker2 # [ 57.967457] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:01Z level=info event=scheduler_connected addr=lb:500515320worker2 # [ 57.971935] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5321worker2 # [ 58.173162] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:01Z level=info event=scheduler_connected addr=lb:500515322worker2 # [ 58.177686] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5323worker2 # [ 58.379171] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:01Z level=info event=scheduler_connected addr=lb:500515324worker2 # [ 58.383620] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5325worker2 # [ 58.585529] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:01Z level=info event=scheduler_connected addr=lb:500515326worker2 # [ 58.589785] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5327worker2 # [ 58.791168] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:01Z level=info event=scheduler_connected addr=lb:500515328worker2 # [ 58.795672] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5329worker2 # [ 58.996745] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:02Z level=info event=scheduler_connected addr=lb:500515330worker2 # [ 59.001338] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5331worker2 # [ 59.202501] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:02Z level=info event=scheduler_connected addr=lb:500515332worker2 # [ 59.206985] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5333worker1 # [ 59.243507] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5334worker1 # [ 59.246441] systemd[1]: Stopped Nix worker-protocol over gRPC.5335worker1 # [ 59.250366] systemd[1]: nix-grpc-daemon.service: Consumed 2.550s CPU time over 43.679s wall clock time, 67.6M memory peak, 81.7K incoming IP traffic, 151.2K outgoing IP traffic.5336worker1 # [ 59.262412] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5337worker1 # [ 59.266568] systemd[1]: Closed Nix worker-protocol over gRPC.5338worker1 # [ 59.270166] systemd[1]: nix-grpc-daemon.socket: Consumed 347us CPU time over 50.726s wall clock time, 256K memory peak, 74.2K incoming IP traffic, 62.5K outgoing IP traffic.5339lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5340lb: must succeed: curl -sf localhost:9901/clusters5341lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5342lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.05 seconds)5343client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25344worker2 # [ 59.408149] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:02Z level=info event=scheduler_connected addr=lb:500515345worker2 # [ 59.412676] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5346worker2 # [ 59.484735] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:526645347worker2 # [ 59.497505] nix-daemon[865]: accepted connection from pid 1106, user nix-grpc-daemon (trusted)5348worker2 # [ 59.516653] nix-grpc-daemon[1106]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 93 ms (attempt 1/5)5349worker2 # [ 59.614473] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:02Z level=info event=scheduler_connected addr=lb:500515350worker2 # [ 59.619534] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5351worker2 # [ 59.625258] nix-grpc-daemon[1106]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 35 ms (attempt 2/5)5352worker2 # [ 59.668616] nix-grpc-daemon[1106]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 346 ms (attempt 3/5)5353worker2 # [ 59.820190] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:02Z level=info event=scheduler_connected addr=lb:500515354worker2 # [ 59.825402] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5355worker2 # [ 60.017159] nix-grpc-daemon[1106]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 590 ms (attempt 4/5)5356worker2 # [ 60.026234] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:03Z level=info event=scheduler_connected addr=lb:500515357worker2 # [ 60.030111] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5358worker2 # [ 60.231440] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:03Z level=info event=scheduler_connected addr=lb:500515359worker2 # [ 60.235435] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5360worker2 # [ 60.436821] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:03Z level=info event=scheduler_connected addr=lb:500515361worker2 # [ 60.440831] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5362worker2 # [ 60.609918] nix-grpc-daemon[1106]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5363worker2 # [ 60.615509] nix-grpc-daemon[1106]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5364client # copying 2 paths...5365client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5366worker2 # [ 60.639826] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:526645367client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5368worker2 # [ 60.644745] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:03Z level=info event=scheduler_connected addr=lb:500515369worker2 # [ 60.648852] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5370worker2 # [ 60.659353] nix-daemon[865]: accepted connection from pid 1106, user nix-grpc-daemon (trusted)5371worker2 # [ 60.684229] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:03Z level=info event=niks3_push_started pid=11665372worker2 # [ 60.797693] nix-grpc-daemon[1166]: time=2026-09-21T13:22:03.974Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""5373worker2 # [ 60.849811] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:04Z level=info event=scheduler_connected addr=lb:500515374worker2 # [ 60.852356] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5375worker2 # [ 60.898935] nix-daemon[865]: accepted connection from pid 1179, user nix-grpc-daemon (trusted)5376worker2 # [ 60.914231] nix-daemon[865]: reaped child process 1182, status = succeeded5377lb # [ 60.903412] niks3-server[983]: 2026/09/21 13:22:03 INFO Received uploads request method=POST path=/api/pending_closures5378lb # [ 60.913967] niks3-server[983]: 2026/09/21 13:22:03 INFO Received uploads request method=POST path=/api/pending_closures5379worker2 # [ 60.942164] nix-grpc-daemon[1166]: time=2026-09-21T13:22:04.119Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5380worker2 # [ 60.943706] nix-grpc-daemon[1166]: time=2026-09-21T13:22:04.120Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5381worker2 # [ 60.945495] nix-grpc-daemon[1166]: time=2026-09-21T13:22:04.122Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5382lb # [ 60.960242] niks3-server[983]: 2026/09/21 13:22:03 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5383lb # [ 60.976603] niks3-server[983]: 2026/09/21 13:22:03 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5384lb # [ 60.984897] niks3-server[983]: 2026/09/21 13:22:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5385lb # [ 60.989176] niks3-server[983]: 2026/09/21 13:22:03 INFO Signed narinfos id=25 count=15386lb # [ 60.991868] niks3-server[983]: 2026/09/21 13:22:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5387lb # [ 60.995147] niks3-server[983]: 2026/09/21 13:22:03 INFO Signed narinfos id=26 count=25388lb # [ 60.996301] niks3-server[983]: 2026/09/21 13:22:03 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5389worker2 # [ 61.016300] nix-grpc-daemon[1166]: time=2026-09-21T13:22:04.193Z level=INFO msg="Uploading 3 narinfos"5390lb # [ 61.009591] niks3-server[983]: 2026/09/21 13:22:03 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5391lb # [ 61.020439] niks3-server[983]: 2026/09/21 13:22:03 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5392lb # [ 61.023867] niks3-server[983]: 2026/09/21 13:22:03 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5393lb # [ 61.026335] niks3-server[983]: 2026/09/21 13:22:03 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5394lb # [ 61.028642] niks3-server[983]: 2026/09/21 13:22:03 INFO Completed upload id=255395lb # [ 61.030533] niks3-server[983]: 2026/09/21 13:22:03 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5396lb # [ 61.034121] niks3-server[983]: 2026/09/21 13:22:03 INFO Completed upload id=265397worker2 # [ 61.053221] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:04Z level=info event=scheduler_connected addr=lb:500515398lb # [ 61.036176] niks3-server[983]: 2026/09/21 13:22:03 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5399worker2 # [ 61.057797] nix-grpc-daemon[1166]: time=2026-09-21T13:22:04.233Z level=INFO msg="Upload complete. (255ms)"5400worker2 # [ 61.062264] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5401worker2 # [ 61.067227] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52664 duration_s=0 paths=2 nar_bytes_in=3125402worker2 # [ 61.079266] nix-daemon[865]: reaped child process 1164, status = succeeded5403client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.75 seconds)5404worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5405worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds)5406worker1 # [ 61.144693] systemd[1]: Listening on Nix worker-protocol over gRPC.5407worker1 # [ 61.156482] systemd[1]: Starting Nix worker-protocol over gRPC...5408worker1 # [ 61.246768] nix-grpc-daemon[1215]: ts=2026-09-21T13:22:04Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5409worker1 # [ 61.248927] nix-grpc-daemon[1215]: ts=2026-09-21T13:22:04Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5410worker2 # [ 61.262246] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:04Z level=info event=scheduler_connected addr=lb:500515411worker1 # [ 61.252457] nix-grpc-daemon[1215]: ts=2026-09-21T13:22:04Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15412worker2 # [ 61.266182] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5413worker1 # [ 61.257058] nix-grpc-daemon[1215]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5414worker1 # [ 61.258384] nix-grpc-daemon[1215]: E0000 00:00:1789996924.415633 1215 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5415worker1 # [ 61.263674] nix-grpc-daemon[1215]: E0000 00:00:1789996924.421143 1215 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5416worker1 # [ 61.268132] nix-grpc-daemon[1215]: ts=2026-09-21T13:22:04Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5417worker1 # [ 61.271388] systemd[1]: Started Nix worker-protocol over gRPC.5418lb # [ 61.271622] niks3-server[983]: 2026/09/21 13:22:03 INFO lead: acquired remote=192.168.1.3:411405419worker1 # [ 61.278085] nix-grpc-daemon[1215]: E0000 00:00:1789996924.436530 1256 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5420worker1 # [ 61.282091] nix-grpc-daemon[1215]: ts=2026-09-21T13:22:04Z level=info event=scheduler_take_over5421lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5422lb: must succeed: curl -sf localhost:9901/clusters5423worker1 # [ 61.293986] nix-grpc-daemon[1215]: E0000 00:00:1789996924.453260 1256 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5424worker1 # [ 61.298054] nix-grpc-daemon[1215]: E0000 00:00:1789996924.455614 1256 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5425worker1 # [ 61.308056] nix-grpc-daemon[1215]: E0000 00:00:1789996924.467266 1256 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5426worker1 # [ 61.317567] nix-grpc-daemon[1215]: ts=2026-09-21T13:22:04Z level=info event=scheduler_connected addr=lb:500515427worker1 # [ 61.320362] nix-grpc-daemon[1215]: ts=2026-09-21T13:22:04Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:415465428worker1 # [ 61.322027] nix-grpc-daemon[1215]: ts=2026-09-21T13:22:04Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05429lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5430lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.05 seconds)5431(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.97 seconds)5432subtest: scheduler down: the other node takes the lock and keeps it5433lb: must succeed: curl -sf localhost:9901/clusters5434lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5435lb: must succeed: curl -sf localhost:9901/clusters5436lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5437lb: must succeed: curl -sf localhost:9901/clusters5438lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5439lb: must succeed: curl -sf localhost:9901/clusters5440lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5441worker2 # [ 61.467225] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:04Z level=info event=scheduler_connected addr=lb:500515442worker1 # [ 61.460286] systemd[1]: Stopping Nix worker-protocol over gRPC...5443worker1 # [ 61.461907] nix-grpc-daemon[1215]: ts=2026-09-21T13:22:04Z level=info event=oidc_jwks provider=mock keys=15444worker1 # [ 61.463180] nix-grpc-daemon[1215]: ts=2026-09-21T13:22:04Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:415465445worker1 # [ 61.465462] nix-grpc-daemon[1215]: ts=2026-09-21T13:22:04Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05446worker1 # [ 62.263719] nix-grpc-daemon[1215]: ts=2026-09-21T13:22:05Z level=info event=signal_exit5447worker1 # [ 62.282985] nix-grpc-daemon[1215]: ts=2026-09-21T13:22:05Z level=info event=scheduler_yield5448lb # [ 62.280511] niks3-server[983]: 2026/09/21 13:22:04 INFO lead: released remote=192.168.1.3:411405449worker2 # [ 62.302911] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:05Z level=info event=scheduler_restarting addr=lb:500515450worker1 # [ 62.291160] nix-grpc-daemon[1215]: ts=2026-09-21T13:22:05Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:415465451worker1 # [ 62.297261] nix-grpc-daemon[1215]: ts=2026-09-21T13:22:05Z level=info event=worker_gone addr=192.168.1.4:500515452worker1 # [ 62.301671] nix-grpc-daemon[1215]: ts=2026-09-21T13:22:05Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:415465453worker1 # [ 62.305077] nix-grpc-daemon[1215]: ts=2026-09-21T13:22:05Z level=info event=worker_gone addr=192.168.1.3:500515454worker1 # [ 62.308132] nix-grpc-daemon[1215]: ts=2026-09-21T13:22:05Z level=info event=scheduler_restarting addr=lb:500515455worker2 # [ 62.507892] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:05Z level=info event=scheduler_connected addr=lb:500515456worker2 # [ 62.512431] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5457worker1 # [ 62.499994] nix-grpc-daemon[1215]: ts=2026-09-21T13:22:05Z level=info event=scheduler_connected addr=lb:500515458worker1 # [ 62.504428] nix-grpc-daemon[1215]: ts=2026-09-21T13:22:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5459worker2 # [ 62.713766] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:05Z level=info event=scheduler_connected addr=lb:500515460worker2 # [ 62.718241] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5461worker1 # [ 62.705802] nix-grpc-daemon[1215]: ts=2026-09-21T13:22:05Z level=info event=scheduler_connected addr=lb:500515462worker1 # [ 62.709951] nix-grpc-daemon[1215]: ts=2026-09-21T13:22:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5463worker2 # [ 62.919610] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:06Z level=info event=scheduler_connected addr=lb:500515464worker2 # [ 62.924131] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5465worker1 # [ 62.910944] nix-grpc-daemon[1215]: ts=2026-09-21T13:22:06Z level=info event=scheduler_connected addr=lb:500515466worker1 # [ 62.915147] nix-grpc-daemon[1215]: ts=2026-09-21T13:22:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5467worker2 # [ 63.125344] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:06Z level=info event=scheduler_connected addr=lb:500515468worker2 # [ 63.129864] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5469worker1 # [ 63.116482] nix-grpc-daemon[1215]: ts=2026-09-21T13:22:06Z level=info event=scheduler_connected addr=lb:500515470worker1 # [ 63.120533] nix-grpc-daemon[1215]: ts=2026-09-21T13:22:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5471worker1 # [ 63.287849] nix-grpc-daemon[1215]: ts=2026-09-21T13:22:06Z level=info event=scheduler_restarting5472worker1 # [ 63.296195] nix-grpc-daemon[1215]: E0000 00:00:1789996926.455116 1260 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5473worker1 # [ 63.300972] nix-grpc-daemon[1215]: E0000 00:00:1789996926.455320 1260 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5474worker2 # [ 63.332626] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:06Z level=info event=scheduler_connected addr=lb:500515475worker2 # [ 63.336326] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5476worker2 # [ 63.533368] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:06Z level=info event=scheduler_connected addr=lb:500515477worker2 # [ 63.537696] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5478worker2 # [ 63.739252] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:06Z level=info event=scheduler_connected addr=lb:500515479worker2 # [ 63.743723] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5480worker2 # [ 63.945373] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:07Z level=info event=scheduler_connected addr=lb:500515481worker2 # [ 63.949665] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5482worker2 # [ 64.151095] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:07Z level=info event=scheduler_connected addr=lb:500515483worker2 # [ 64.155532] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5484worker2 # [ 64.358387] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:07Z level=info event=scheduler_connected addr=lb:500515485worker2 # [ 64.362511] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5486worker2 # [ 64.559317] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:07Z level=info event=scheduler_connected addr=lb:500515487worker2 # [ 64.563222] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5488worker2 # [ 64.764353] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:07Z level=info event=scheduler_connected addr=lb:500515489worker2 # [ 64.768220] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5490worker2 # [ 64.969446] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:08Z level=info event=scheduler_connected addr=lb:500515491worker2 # [ 64.973271] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5492worker2 # [ 65.174580] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:08Z level=info event=scheduler_connected addr=lb:500515493worker2 # [ 65.178435] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5494worker1 # [ 65.277326] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5495worker1 # [ 65.281257] systemd[1]: Stopped Nix worker-protocol over gRPC.5496worker1 # [ 65.285346] systemd[1]: nix-grpc-daemon.service: Consumed 152ms CPU time over 4.128s wall clock time, 12.8M memory peak, 9.5K incoming IP traffic, 8.1K outgoing IP traffic.5497worker1 # [ 65.295987] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5498worker1 # [ 65.301380] systemd[1]: Closed Nix worker-protocol over gRPC.5499worker1 # [ 65.304718] systemd[1]: nix-grpc-daemon.socket: Consumed 507us CPU time over 4.161s wall clock time, 256K memory peak, 13K incoming IP traffic, 9.9K outgoing IP traffic.5500worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5501worker2 # [ 65.380737] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:08Z level=info event=scheduler_connected addr=lb:500515502worker2 # [ 65.383618] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5503worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.06 seconds)5504worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5505worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)5506lb # [ 65.557299] envoy[916]: conn peer=[2001:db8:1::3]:34542 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=9336 tx=2147 ms=418135507lb # [ 65.561295] envoy[916]: conn peer=[2001:db8:1::1]:41118 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1003 tx=282 ms=15895508worker2 # [ 65.584719] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:08Z level=info event=scheduler_connected addr=lb:500515509lb # [ 65.567414] envoy[916]: conn peer=[2001:db8:1::3]:38096 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1348 tx=349 ms=20095510worker2 # [ 65.588714] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5511worker2 # [ 65.790197] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:08Z level=info event=scheduler_connected addr=lb:500515512worker2 # [ 65.794137] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5513worker2 # [ 65.995200] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:09Z level=info event=scheduler_connected addr=lb:500515514worker2 # [ 65.999251] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5515worker2 # [ 66.200630] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:09Z level=info event=scheduler_connected addr=lb:500515516worker2 # [ 66.204777] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5517worker2 # [ 66.406497] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:09Z level=info event=scheduler_connected addr=lb:500515518worker2 # [ 66.410524] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5519worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5520worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5521worker2 # [ 66.499238] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:09Z level=info event=scheduler_take_over5522lb # [ 66.478767] niks3-server[983]: 2026/09/21 13:22:08 INFO lead: acquired remote=192.168.1.4:340305523worker2 # [ 66.611537] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:09Z level=info event=scheduler_connected addr=lb:500515524worker2 # [ 66.628144] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:09Z level=info event=oidc_jwks provider=mock keys=15525worker2 # [ 66.631961] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:09Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:554045526worker2 # [ 66.635939] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:09Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05527worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5528worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5529client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&25530worker2 # [ 67.651239] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:526645531client # don't know how to build these paths:5532client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5533worker2 # [ 67.660598] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:526645534worker2 # [ 67.684284] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:10Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05535worker2 # [ 67.688522] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:10Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.4:50051 clients=15536worker2 # [ 67.738290] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:526645537worker2 # [ 67.751456] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:526645538client # copying 1 paths to 'lb:50051'...5539client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5540worker2 # [ 67.764918] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:526645541worker2 # [ 67.773804] nix-daemon[865]: accepted connection from pid 1106, user nix-grpc-daemon (trusted)5542worker2 # [ 67.877416] nix-daemon[865]: accepted connection from pid 1206, user nix-grpc-daemon (trusted)5543worker2 # [ 67.890713] nix-daemon[865]: reaped child process 1209, status = succeeded5544worker2 # [ 67.974431] nix-daemon[865]: accepted connection from pid 1211, user nix-grpc-daemon (trusted)5545worker2 # [ 67.986788] nix-daemon[865]: reaped child process 1214, status = succeeded5546lb # [ 67.974554] niks3-server[983]: 2026/09/21 13:22:10 INFO Received uploads request method=POST path=/api/pending_closures5547worker2 # [ 68.006852] nix-grpc-daemon[1166]: time=2026-09-21T13:22:11.183Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5548worker2 # [ 68.008408] nix-grpc-daemon[1166]: time=2026-09-21T13:22:11.185Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5549lb # [ 68.006679] niks3-server[983]: 2026/09/21 13:22:10 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5550lb # [ 68.022674] niks3-server[983]: 2026/09/21 13:22:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5551lb # [ 68.027136] niks3-server[983]: 2026/09/21 13:22:10 INFO Signed narinfos id=27 count=15552worker2 # [ 68.050899] nix-grpc-daemon[1166]: time=2026-09-21T13:22:11.227Z level=INFO msg="Uploading 1 narinfos"5553lb # [ 68.037086] niks3-server[983]: 2026/09/21 13:22:10 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5554lb # [ 68.045754] niks3-server[983]: 2026/09/21 13:22:10 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5555lb # [ 68.052720] niks3-server[983]: 2026/09/21 13:22:10 INFO Completed upload id=275556worker2 # [ 68.073961] nix-grpc-daemon[1166]: time=2026-09-21T13:22:11.250Z level=INFO msg="Upload complete. (286ms)"5557lb # [ 68.060395] niks3-server[983]: 2026/09/21 13:22:10 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5558worker2 # [ 68.078358] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52664 duration_s=0 paths=1 nar_bytes_in=5205559worker2 # [ 68.084335] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:526645560worker2 # [ 68.094061] nix-daemon[865]: accepted connection from pid 1106, user nix-grpc-daemon (trusted)5561worker2 # [ 68.106490] nix-daemon[865]: reaped child process 1204, status = succeeded5562worker2 # [ 68.109260] nix-daemon[865]: accepted connection from pid 1106, user nix-grpc-daemon (trusted)5563client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5564client # farm-a> worker2: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5565client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5566client # farm-a> LOG-farm-a-on-standby5567worker2 # [ 69.342747] nix-daemon[865]: reaped child process 1218, status = succeeded5568worker2 # [ 69.431195] nix-daemon[865]: accepted connection from pid 1234, user nix-grpc-daemon (trusted)5569worker2 # [ 69.444295] nix-daemon[865]: reaped child process 1237, status = succeeded5570lb # [ 69.431483] niks3-server[983]: 2026/09/21 13:22:11 INFO Received uploads request method=POST path=/api/pending_closures5571worker2 # [ 69.464394] nix-grpc-daemon[1166]: time=2026-09-21T13:22:12.641Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5572worker2 # [ 69.465944] nix-grpc-daemon[1166]: time=2026-09-21T13:22:12.642Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5573lb # [ 69.475406] niks3-server[983]: 2026/09/21 13:22:11 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5574lb # [ 69.483276] niks3-server[983]: 2026/09/21 13:22:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5575lb # [ 69.492536] niks3-server[983]: 2026/09/21 13:22:11 INFO Signed narinfos id=28 count=15576worker2 # [ 69.513809] nix-grpc-daemon[1166]: time=2026-09-21T13:22:12.690Z level=INFO msg="Uploading 1 narinfos"5577lb # [ 69.495881] niks3-server[983]: 2026/09/21 13:22:11 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5578lb # [ 69.513343] niks3-server[983]: 2026/09/21 13:22:11 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5579lb # [ 69.519029] niks3-server[983]: 2026/09/21 13:22:11 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5580lb # [ 69.521355] niks3-server[983]: 2026/09/21 13:22:11 INFO Completed upload id=285581worker2 # [ 69.542616] nix-grpc-daemon[1166]: time=2026-09-21T13:22:12.719Z level=INFO msg="Upload complete. (224ms)"5582lb # [ 69.523814] niks3-server[983]: 2026/09/21 13:22:11 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5583worker2 # [ 69.549752] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52664 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15584worker2 # [ 69.561680] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:12Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05585worker2 # [ 69.566228] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:12Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.4:50051 clients=15586worker2 # [ 69.575828] nix-daemon[865]: reaped child process 1216, status = succeeded5587worker2 # [ 69.606609] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:526645588worker2 # [ 69.619338] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:526645589worker2 # [ 69.633353] nix-grpc-daemon[1166]: time=2026-09-21T13:22:12.810Z level=INFO msg="All 1 paths already cached"5590client # copying 1 paths to 'lb:50051'...5591client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5592worker2 # [ 69.641647] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:526645593worker2 # [ 69.649903] nix-daemon[865]: accepted connection from pid 1106, user nix-grpc-daemon (trusted)5594worker2 # [ 69.749479] nix-daemon[865]: accepted connection from pid 1243, user nix-grpc-daemon (trusted)5595worker2 # [ 69.762806] nix-daemon[865]: reaped child process 1246, status = succeeded5596worker2 # [ 69.847171] nix-daemon[865]: accepted connection from pid 1248, user nix-grpc-daemon (trusted)5597worker2 # [ 69.859575] nix-daemon[865]: reaped child process 1251, status = succeeded5598lb # [ 69.846753] niks3-server[983]: 2026/09/21 13:22:12 INFO Received uploads request method=POST path=/api/pending_closures5599worker2 # [ 69.876235] nix-grpc-daemon[1166]: time=2026-09-21T13:22:13.052Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5600worker2 # [ 69.877623] nix-grpc-daemon[1166]: time=2026-09-21T13:22:13.052Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5601lb # [ 69.875689] niks3-server[983]: 2026/09/21 13:22:12 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5602lb # [ 69.886148] niks3-server[983]: 2026/09/21 13:22:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5603lb # [ 69.894583] niks3-server[983]: 2026/09/21 13:22:12 INFO Signed narinfos id=29 count=15604worker2 # [ 69.918390] nix-grpc-daemon[1166]: time=2026-09-21T13:22:13.095Z level=INFO msg="Uploading 1 narinfos"5605lb # [ 69.898377] niks3-server[983]: 2026/09/21 13:22:12 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5606lb # [ 69.910669] niks3-server[983]: 2026/09/21 13:22:12 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5607lb # [ 69.919588] niks3-server[983]: 2026/09/21 13:22:12 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5608lb # [ 69.923165] niks3-server[983]: 2026/09/21 13:22:12 INFO Completed upload id=295609worker2 # [ 69.944257] nix-grpc-daemon[1166]: time=2026-09-21T13:22:13.121Z level=INFO msg="Upload complete. (281ms)"5610worker2 # [ 69.948179] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52664 duration_s=0 paths=1 nar_bytes_in=6405611worker2 # [ 69.953254] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:526645612worker2 # [ 69.963114] nix-daemon[865]: accepted connection from pid 1106, user nix-grpc-daemon (trusted)5613worker2 # [ 69.972843] nix-daemon[865]: reaped child process 1241, status = succeeded5614worker2 # [ 69.975742] nix-daemon[865]: accepted connection from pid 1106, user nix-grpc-daemon (trusted)5615client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5616client # farm-b> worker2: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5617client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5618client # farm-b> LOG-farm-b-on-standby5619worker2 # [ 71.415673] nix-daemon[865]: reaped child process 1255, status = succeeded5620worker2 # [ 71.499951] nix-daemon[865]: accepted connection from pid 1271, user nix-grpc-daemon (trusted)5621worker2 # [ 71.511555] nix-daemon[865]: reaped child process 1274, status = succeeded5622lb # [ 71.499408] niks3-server[983]: 2026/09/21 13:22:13 INFO Received uploads request method=POST path=/api/pending_closures5623worker2 # [ 71.528744] nix-grpc-daemon[1166]: time=2026-09-21T13:22:14.705Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5624worker2 # [ 71.530294] nix-grpc-daemon[1166]: time=2026-09-21T13:22:14.707Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"5625lb # [ 71.542850] niks3-server[983]: 2026/09/21 13:22:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5626worker2 # [ 71.567413] nix-grpc-daemon[1166]: time=2026-09-21T13:22:14.744Z level=INFO msg="Uploading 1 narinfos"5627lb # [ 71.547863] niks3-server[983]: 2026/09/21 13:22:13 INFO Signed narinfos id=30 count=15628lb # [ 71.552483] niks3-server[983]: 2026/09/21 13:22:13 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst5629lb # [ 71.559302] niks3-server[983]: 2026/09/21 13:22:13 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5630lb # [ 71.570808] niks3-server[983]: 2026/09/21 13:22:13 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls5631lb # [ 71.576612] niks3-server[983]: 2026/09/21 13:22:13 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5632lb # [ 71.585714] niks3-server[983]: 2026/09/21 13:22:13 INFO Completed upload id=305633worker2 # [ 71.606815] nix-grpc-daemon[1166]: time=2026-09-21T13:22:14.783Z level=INFO msg="Upload complete. (222ms)"5634lb # [ 71.587434] niks3-server[983]: 2026/09/21 13:22:13 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo5635worker2 # [ 71.619888] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52664 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=15636worker2 # [ 71.625620] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:14Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=05637worker2 # [ 71.629485] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:14Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.4:50051 clients=15638worker2 # [ 71.639277] nix-daemon[865]: reaped child process 1253, status = succeeded5639worker2 # [ 71.663981] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:526645640worker2 # [ 71.676973] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:526645641worker2 # [ 71.690967] nix-grpc-daemon[1166]: time=2026-09-21T13:22:14.867Z level=INFO msg="All 2 paths already cached"5642client # copying 1 paths to 'lb:50051'...5643client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...5644worker2 # [ 71.699373] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:526645645worker2 # [ 71.707694] nix-daemon[865]: accepted connection from pid 1106, user nix-grpc-daemon (trusted)5646worker2 # [ 71.805211] nix-daemon[865]: accepted connection from pid 1280, user nix-grpc-daemon (trusted)5647worker2 # [ 71.819199] nix-daemon[865]: reaped child process 1283, status = succeeded5648worker2 # [ 71.902895] nix-daemon[865]: accepted connection from pid 1285, user nix-grpc-daemon (trusted)5649worker2 # [ 71.916528] nix-daemon[865]: reaped child process 1288, status = succeeded5650lb # [ 71.901883] niks3-server[983]: 2026/09/21 13:22:14 INFO Received uploads request method=POST path=/api/pending_closures5651worker2 # [ 71.931517] nix-grpc-daemon[1166]: time=2026-09-21T13:22:15.108Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5652worker2 # [ 71.933482] nix-grpc-daemon[1166]: time=2026-09-21T13:22:15.109Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"5653lb # [ 71.929940] niks3-server[983]: 2026/09/21 13:22:14 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst5654lb # [ 71.943793] niks3-server[983]: 2026/09/21 13:22:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5655worker2 # [ 71.970804] nix-grpc-daemon[1166]: time=2026-09-21T13:22:15.147Z level=INFO msg="Uploading 1 narinfos"5656lb # [ 71.950620] niks3-server[983]: 2026/09/21 13:22:14 INFO Signed narinfos id=31 count=15657lb # [ 71.955107] niks3-server[983]: 2026/09/21 13:22:14 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls5658lb # [ 71.968862] niks3-server[983]: 2026/09/21 13:22:14 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5659lb # [ 71.978657] niks3-server[983]: 2026/09/21 13:22:14 INFO Completed upload id=315660worker2 # [ 71.999885] nix-grpc-daemon[1166]: time=2026-09-21T13:22:15.176Z level=INFO msg="Upload complete. (279ms)"5661lb # [ 71.981485] niks3-server[983]: 2026/09/21 13:22:14 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo5662worker2 # [ 72.003780] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52664 duration_s=0 paths=1 nar_bytes_in=7685663worker2 # [ 72.009278] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:526645664worker2 # [ 72.018509] nix-daemon[865]: accepted connection from pid 1106, user nix-grpc-daemon (trusted)5665worker2 # [ 72.025330] nix-daemon[865]: reaped child process 1278, status = succeeded5666worker2 # [ 72.030428] nix-daemon[865]: accepted connection from pid 1106, user nix-grpc-daemon (trusted)5667client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...5668client # farm-top> worker2: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5669client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5670client # farm-top> LOG-farm-top-on-standby5671worker2 # [ 72.671069] nix-daemon[865]: reaped child process 1292, status = succeeded5672worker2 # [ 72.747097] nix-daemon[865]: accepted connection from pid 1308, user nix-grpc-daemon (trusted)5673worker2 # [ 72.762600] nix-daemon[865]: reaped child process 1311, status = succeeded5674lb # [ 72.750767] niks3-server[983]: 2026/09/21 13:22:15 INFO Received uploads request method=POST path=/api/pending_closures5675worker2 # [ 72.780817] nix-grpc-daemon[1166]: time=2026-09-21T13:22:15.957Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5676worker2 # [ 72.782385] nix-grpc-daemon[1166]: time=2026-09-21T13:22:15.959Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"5677lb # [ 72.788802] niks3-server[983]: 2026/09/21 13:22:15 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5678lb # [ 72.796369] niks3-server[983]: 2026/09/21 13:22:15 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst5679lb # [ 72.805354] niks3-server[983]: 2026/09/21 13:22:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5680lb # [ 72.810972] niks3-server[983]: 2026/09/21 13:22:15 INFO Signed narinfos id=32 count=15681worker2 # [ 72.835091] nix-grpc-daemon[1166]: time=2026-09-21T13:22:16.011Z level=INFO msg="Uploading 1 narinfos"5682lb # [ 72.827548] niks3-server[983]: 2026/09/21 13:22:15 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5683lb # [ 72.829092] niks3-server[983]: 2026/09/21 13:22:15 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls5684lb # [ 72.833644] niks3-server[983]: 2026/09/21 13:22:15 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo5685lb # [ 72.837417] niks3-server[983]: 2026/09/21 13:22:15 INFO Completed upload id=325686worker2 # [ 72.858779] nix-grpc-daemon[1166]: time=2026-09-21T13:22:16.035Z level=INFO msg="Upload complete. (221ms)"5687worker2 # [ 72.867930] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52664 duration_s=0 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=15688worker2 # [ 72.881926] nix-daemon[865]: reaped child process 1290, status = succeeded5689client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&2, in 5.39 seconds)5690client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath5691client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath, in 0.11 seconds)5692worker1 # [ 73.040807] systemd[1]: Listening on Nix worker-protocol over gRPC.5693worker1 # [ 73.050452] systemd[1]: Starting Nix worker-protocol over gRPC...5694worker1 # [ 73.140255] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:16Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5695worker1 # [ 73.144837] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:16Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5696worker1 # [ 73.148125] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:16Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15697worker1 # [ 73.151728] nix-grpc-daemon[1282]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5698worker1 # [ 73.156706] nix-grpc-daemon[1282]: E0000 00:00:1789996936.306855 1282 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5699worker1 # [ 73.159077] nix-grpc-daemon[1282]: E0000 00:00:1789996936.310782 1282 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5700worker1 # [ 73.161184] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:16Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5701worker1 # [ 73.165753] systemd[1]: Started Nix worker-protocol over gRPC.5702worker1 # [ 73.169131] nix-grpc-daemon[1282]: E0000 00:00:1789996936.326880 1323 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5703lb: must succeed: curl -sf localhost:9901/clusters5704worker1 # [ 73.181110] nix-grpc-daemon[1282]: E0000 00:00:1789996936.340408 1323 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5705worker1 # [ 73.184161] nix-grpc-daemon[1282]: E0000 00:00:1789996936.342711 1323 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5706worker1 # [ 73.195590] nix-grpc-daemon[1282]: E0000 00:00:1789996936.354833 1323 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5707worker1 # [ 73.207074] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:16Z level=info event=scheduler_connected addr=lb:500515708worker2 # [ 73.224188] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:16Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:554045709worker2 # [ 73.228101] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:16Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05710lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5711worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5712worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5713lb: must succeed: curl -sf localhost:9901/clusters5714lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5715lb: must succeed: curl -sf localhost:9901/clusters5716lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5717worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5718worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5719(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.07 seconds)5720subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5721client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs5722client # [ 73.456745] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs.5723client # Running as unit: rs.service; invocation ID: e03ab497183a48aba9804d93dd47e2fe5724client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs, in 0.06 seconds)5725client # [ 73.595135] nix[1178]: don't know how to build these paths:5726worker2 # [ 73.592221] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:526645727client # [ 73.596061] nix[1178]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5728worker2 # [ 73.603976] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:16Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=05729worker2 # [ 73.609477] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:16Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.4:50051 clients=15730worker2 # [ 73.648140] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:526645731worker2 # [ 73.661169] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:526645732client # [ 73.673238] nix[1178]: copying 1 paths to 'lb:50051'...5733worker2 # [ 73.673151] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:526645734client # [ 73.676151] nix[1178]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...5735worker2 # [ 73.687287] nix-daemon[865]: accepted connection from pid 1106, user nix-grpc-daemon (trusted)5736worker2 # [ 73.795975] nix-daemon[865]: accepted connection from pid 1324, user nix-grpc-daemon (trusted)5737worker2 # [ 73.811451] nix-daemon[865]: reaped child process 1327, status = succeeded5738worker2 # [ 73.905399] nix-daemon[865]: accepted connection from pid 1329, user nix-grpc-daemon (trusted)5739worker2 # [ 73.920857] nix-daemon[865]: reaped child process 1332, status = succeeded5740lb # [ 73.907562] niks3-server[983]: 2026/09/21 13:22:16 INFO Received uploads request method=POST path=/api/pending_closures5741worker2 # [ 73.939777] nix-grpc-daemon[1166]: time=2026-09-21T13:22:17.116Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5742worker2 # [ 73.941638] nix-grpc-daemon[1166]: time=2026-09-21T13:22:17.118Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"5743lb # [ 73.942649] niks3-server[983]: 2026/09/21 13:22:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5744worker2 # [ 73.968299] nix-grpc-daemon[1166]: time=2026-09-21T13:22:17.145Z level=INFO msg="Uploading 1 narinfos"5745lb # [ 73.948703] niks3-server[983]: 2026/09/21 13:22:16 INFO Signed narinfos id=33 count=15746lb # [ 73.956455] niks3-server[983]: 2026/09/21 13:22:16 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst5747lb # [ 73.963828] niks3-server[983]: 2026/09/21 13:22:16 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls5748lb # [ 73.970125] niks3-server[983]: 2026/09/21 13:22:16 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5749lb # [ 73.974259] niks3-server[983]: 2026/09/21 13:22:16 INFO Completed upload id=335750worker2 # [ 73.995153] nix-grpc-daemon[1166]: time=2026-09-21T13:22:17.172Z level=INFO msg="Upload complete. (296ms)"5751lb # [ 73.976281] niks3-server[983]: 2026/09/21 13:22:16 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo5752worker2 # [ 73.997877] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52664 duration_s=0 paths=1 nar_bytes_in=4165753worker2 # [ 74.002235] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:526645754worker2 # [ 74.010324] nix-daemon[865]: accepted connection from pid 1106, user nix-grpc-daemon (trusted)5755worker2 # [ 74.015136] nix-daemon[865]: reaped child process 1322, status = succeeded5756worker2 # [ 74.019770] nix-daemon[865]: accepted connection from pid 1106, user nix-grpc-daemon (trusted)5757client # [ 74.020592] nix[1178]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...5758lb # [ 75.557717] envoy[916]: conn peer=[2001:db8:1::1]:43548 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9577 tx=4030 ms=52275759lb: must succeed: curl -sf localhost:9901/clusters5760lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5761lb: must succeed: curl -sf localhost:9901/clusters5762lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5763lb: must succeed: curl -sf localhost:9901/clusters5764lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5765lb: must succeed: curl -sf localhost:9901/clusters5766lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5767worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5768worker1 # [ 76.122820] systemd-journald[284]: Received client request to rotate journal, rotating.5769worker1 # [ 76.141307] systemd-journald[284]: Vacuuming done, freed 0B of archived journals from /var/log/journal/d392c89353ed4d61bc66666b4888cb2c.5770worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.06 seconds)5771worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service5772worker2 # [ 76.072258] systemd[1]: Stopping Nix worker-protocol over gRPC...5773worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)5774client: must succeed: sleep 5.05775worker2 # [ 76.497730] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:19Z level=info event=draining5776client: (finished: must succeed: sleep 5.0, in 5.03 seconds)5777worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5778worker2 # [ 81.467399] nix-daemon[865]: reaped child process 1336, status = succeeded5779worker2 # [ 81.548724] nix-daemon[865]: accepted connection from pid 1377, user nix-grpc-daemon (trusted)5780worker2 # [ 81.564259] nix-daemon[865]: reaped child process 1380, status = succeeded5781lb # [ 81.549694] niks3-server[983]: 2026/09/21 13:22:23 INFO Received uploads request method=POST path=/api/pending_closures5782worker2 # [ 81.579080] nix-grpc-daemon[1166]: time=2026-09-21T13:22:24.755Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5783worker2 # [ 81.581708] nix-grpc-daemon[1166]: time=2026-09-21T13:22:24.757Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"5784lb # [ 81.587162] niks3-server[983]: 2026/09/21 13:22:23 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5785lb # [ 81.599251] niks3-server[983]: 2026/09/21 13:22:23 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5786lb # [ 81.611350] niks3-server[983]: 2026/09/21 13:22:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5787worker2 # [ 81.639690] nix-grpc-daemon[1166]: time=2026-09-21T13:22:24.816Z level=INFO msg="Uploading 1 narinfos"5788lb # [ 81.618506] niks3-server[983]: 2026/09/21 13:22:23 INFO Signed narinfos id=34 count=15789lb # [ 81.624249] niks3-server[983]: 2026/09/21 13:22:23 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls5790lb # [ 81.634177] niks3-server[983]: 2026/09/21 13:22:23 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5791lb # [ 81.637427] niks3-server[983]: 2026/09/21 13:22:23 INFO Completed upload id=345792worker2 # [ 81.658522] nix-grpc-daemon[1166]: time=2026-09-21T13:22:24.835Z level=INFO msg="Upload complete. (224ms)"5793lb # [ 81.639388] niks3-server[983]: 2026/09/21 13:22:23 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo5794worker2 # [ 81.666854] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52664 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=15795worker2 # [ 81.680822] nix-daemon[865]: reaped child process 1334, status = succeeded5796client # [ 81.695761] systemd[1]: rs.service: Deactivated successfully.5797client # [ 81.699341] systemd[1]: rs.service: Consumed 127ms CPU time over 8.241s wall clock time, 41.4M memory peak, 7.1K incoming IP traffic, 8.4K outgoing IP traffic.5798worker2 # [ 82.499477] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:25Z level=info event=signal_exit5799worker2 # [ 82.503699] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:25Z level=info event=scheduler_yield5800lb # [ 82.484875] niks3-server[983]: 2026/09/21 13:22:24 INFO lead: released remote=192.168.1.4:340305801worker1 # [ 82.491828] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:25Z level=info event=scheduler_restarting addr=lb:500515802worker2 # [ 82.507581] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:25Z level=info event=scheduler_restarting addr=lb:500515803worker2 # [ 82.511753] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:25Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:554045804worker2 # [ 82.515482] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:25Z level=info event=worker_gone addr=192.168.1.3:500515805worker2 # [ 82.518569] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:25Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:554045806worker2 # [ 82.521878] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:25Z level=info event=worker_gone addr=192.168.1.4:500515807worker1 # [ 82.692287] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:25Z level=info event=scheduler_connected addr=lb:500515808worker2 # [ 82.710196] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:25Z level=info event=scheduler_connected addr=lb:500515809worker1 # [ 82.696797] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5810worker2 # [ 82.713495] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5811worker2 # [ 82.911391] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:26Z level=info event=scheduler_connected addr=lb:500515812worker1 # [ 82.897746] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:26Z level=info event=scheduler_connected addr=lb:500515813worker2 # [ 82.915563] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5814worker1 # [ 82.901850] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5815worker2 # [ 83.116726] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:26Z level=info event=scheduler_connected addr=lb:500515816worker1 # [ 83.102879] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:26Z level=info event=scheduler_connected addr=lb:500515817worker2 # [ 83.120828] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5818worker1 # [ 83.107222] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5819worker1 # [ 83.168132] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:26Z level=info event=scheduler_take_over5820lb # [ 83.162451] niks3-server[983]: 2026/09/21 13:22:25 INFO lead: acquired remote=192.168.1.3:347605821worker2 # [ 83.321925] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:26Z level=info event=scheduler_connected addr=lb:500515822worker1 # [ 83.308171] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:26Z level=info event=scheduler_connected addr=lb:500515823worker2 # [ 83.326128] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5824worker1 # [ 83.312391] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5825worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.13 seconds)5826client: waiting for success: ! systemctl is-active rs5827client: (finished: waiting for success: ! systemctl is-active rs, in 0.04 seconds)5828client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5829client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.03 seconds)5830client: must fail: journalctl -u rs | grep -q 'reconnecting'5831client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.01 seconds)5832lb: must succeed: curl -sf localhost:9901/clusters5833worker2 # [ 83.504221] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:26Z level=info event=scheduler_restarting5834worker1 # [ 83.514313] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:26Z level=info event=scheduler_connected addr=lb:500515835worker1 # [ 83.518240] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5836worker2 # [ 83.540461] nix-grpc-daemon[1106]: ts=2026-09-21T13:22:26Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5837worker2 # [ 83.545495] nix-daemon[865]: reaped child process 1158, status = succeeded5838lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5839worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5840worker2 # [ 83.554873] nix-grpc-daemon[1106]: E0000 00:00:1789996946.731681 1151 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true5841worker2 # [ 83.559610] nix-grpc-daemon[1106]: E0000 00:00:1789996946.731783 1151 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5842worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5843worker1 # [ 83.719455] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:26Z level=info event=scheduler_connected addr=lb:500515844worker1 # [ 83.723575] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5845worker1 # [ 83.924818] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:27Z level=info event=scheduler_connected addr=lb:500515846worker1 # [ 83.928959] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5847worker1 # [ 84.130055] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:27Z level=info event=scheduler_connected addr=lb:500515848worker1 # [ 84.134199] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5849worker1 # [ 84.334855] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:27Z level=info event=scheduler_connected addr=lb:500515850worker1 # [ 84.342131] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:27Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:545745851worker1 # [ 84.345729] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:27Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05852lb: must succeed: curl -sf localhost:9901/clusters5853lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5854worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5855worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5856worker2 # [ 85.327347] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5857worker2 # [ 85.330271] systemd[1]: Stopped Nix worker-protocol over gRPC.5858worker2 # [ 85.333535] systemd[1]: nix-grpc-daemon.service: Consumed 1.524s CPU time over 28.935s wall clock time, 39.6M memory peak, 63.1K incoming IP traffic, 171.2K outgoing IP traffic.5859worker2 # [ 85.347283] systemd[1]: Starting scheduler-token.service...5860worker2 # [ 85.393262] systemd[1]: scheduler-token.service: Deactivated successfully.5861worker2 # [ 85.394930] systemd[1]: Finished scheduler-token.service.5862worker2 # [ 85.397184] systemd[1]: scheduler-token.service: Consumed 23ms CPU time over 47ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5863worker2 # [ 85.400575] systemd[1]: Starting Nix worker-protocol over gRPC...5864worker2 # [ 85.478960] nix-grpc-daemon[1390]: ts=2026-09-21T13:22:28Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5865worker2 # [ 85.481031] nix-grpc-daemon[1390]: ts=2026-09-21T13:22:28Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5866worker2 # [ 85.484208] nix-grpc-daemon[1390]: ts=2026-09-21T13:22:28Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15867worker2 # [ 85.488603] nix-grpc-daemon[1390]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5868worker2 # [ 85.490022] nix-grpc-daemon[1390]: E0000 00:00:1789996948.665417 1390 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5869worker2 # [ 85.495292] nix-grpc-daemon[1390]: E0000 00:00:1789996948.670792 1390 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5870worker2 # [ 85.498678] nix-grpc-daemon[1390]: ts=2026-09-21T13:22:28Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5871worker2 # [ 85.502050] systemd[1]: Started Nix worker-protocol over gRPC.5872worker2 # [ 85.505065] nix-grpc-daemon[1390]: E0000 00:00:1789996948.681006 1431 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5873worker2 # [ 85.515951] nix-grpc-daemon[1390]: E0000 00:00:1789996948.692867 1431 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5874worker2 # [ 85.520057] nix-grpc-daemon[1390]: E0000 00:00:1789996948.695190 1431 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5875worker2 # [ 85.530200] nix-grpc-daemon[1390]: E0000 00:00:1789996948.707124 1431 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5876worker2 # [ 85.539788] nix-grpc-daemon[1390]: ts=2026-09-21T13:22:28Z level=info event=scheduler_connected addr=lb:500515877worker1 # [ 85.533827] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:28Z level=info event=oidc_jwks provider=mock keys=15878worker1 # [ 85.537834] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:28Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:545745879worker1 # [ 85.541765] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:28Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05880lb # [ 85.557875] envoy[916]: conn peer=[2001:db8:1::1]:34922 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2960 tx=1404 ms=80865881lb # [ 85.561743] envoy[916]: conn peer=[2001:db8:1::4]:41504 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=42440 tx=2237 ms=269975882lb: must succeed: curl -sf localhost:9901/clusters5883lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5884worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5885worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5886(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.37 seconds)5887subtest: scheduler restart mid-build: build finishes, no double build5888client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid5889client # [ 85.827706] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid.5890client # Running as unit: mid.service; invocation ID: 8c0694965e334c0394687a28eb3d885e5891client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid, in 0.06 seconds)5892worker1 # [ 85.959294] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:29Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:545765893worker1 # [ 85.970243] nix-daemon[842]: accepted connection from pid 1282, user nix-grpc-daemon (trusted)5894client # [ 85.999443] nix[1214]: don't know how to build these paths:5895client # [ 86.000343] nix[1214]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5896worker1 # [ 85.993233] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:29Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05897worker1 # [ 85.998097] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:29Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=15898worker1 # [ 86.041971] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545765899worker1 # [ 86.054567] nix-grpc-daemon[1282]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 2 ms (attempt 1/5)5900worker1 # [ 86.062777] nix-grpc-daemon[1282]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 47 ms (attempt 2/5)5901worker1 # [ 86.116999] nix-grpc-daemon[1282]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 50 ms (attempt 3/5)5902worker1 # [ 86.168872] nix-grpc-daemon[1282]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 611 ms (attempt 4/5)5903worker1 # [ 86.784266] nix-grpc-daemon[1282]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5904worker1 # [ 86.787582] nix-grpc-daemon[1282]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5905worker1 # [ 86.804489] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:545765906client # [ 86.831122] nix[1214]: copying 1 paths to 'lb:50051'...5907client # [ 86.831979] nix[1214]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...5908worker1 # [ 86.815510] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:545765909worker1 # [ 86.825336] nix-daemon[842]: accepted connection from pid 1282, user nix-grpc-daemon (trusted)5910worker1 # [ 86.847322] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:30Z level=info event=niks3_push_started pid=14075911worker1 # [ 86.962697] nix-grpc-daemon[1407]: time=2026-09-21T13:22:30.121Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""5912worker1 # [ 87.061274] nix-daemon[842]: accepted connection from pid 1423, user nix-grpc-daemon (trusted)5913worker1 # [ 87.074099] nix-daemon[842]: reaped child process 1426, status = succeeded5914worker1 # [ 87.163465] nix-daemon[842]: accepted connection from pid 1428, user nix-grpc-daemon (trusted)5915worker1 # [ 87.177982] nix-daemon[842]: reaped child process 1431, status = succeeded5916lb # [ 87.180218] niks3-server[983]: 2026/09/21 13:22:29 INFO Received uploads request method=POST path=/api/pending_closures5917worker1 # [ 87.194669] nix-grpc-daemon[1407]: time=2026-09-21T13:22:30.353Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5918worker1 # [ 87.196935] nix-grpc-daemon[1407]: time=2026-09-21T13:22:30.355Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"5919lb # [ 87.214162] niks3-server[983]: 2026/09/21 13:22:29 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst5920lb # [ 87.227369] niks3-server[983]: 2026/09/21 13:22:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5921lb # [ 87.235868] niks3-server[983]: 2026/09/21 13:22:29 INFO Signed narinfos id=35 count=15922worker1 # [ 87.244862] nix-grpc-daemon[1407]: time=2026-09-21T13:22:30.403Z level=INFO msg="Uploading 1 narinfos"5923lb # [ 87.245462] niks3-server[983]: 2026/09/21 13:22:29 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls5924lb # [ 87.255364] niks3-server[983]: 2026/09/21 13:22:29 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5925lb # [ 87.260056] niks3-server[983]: 2026/09/21 13:22:29 INFO Completed upload id=355926worker1 # [ 87.266176] nix-grpc-daemon[1407]: time=2026-09-21T13:22:30.425Z level=INFO msg="Upload complete. (299ms)"5927lb # [ 87.262407] niks3-server[983]: 2026/09/21 13:22:29 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo5928worker1 # [ 87.270506] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54576 duration_s=0 paths=1 nar_bytes_in=4245929worker1 # [ 87.275484] nix-grpc-daemon[1282]: ts=2026-09-21T13:22:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545765930worker1 # [ 87.288466] nix-daemon[842]: accepted connection from pid 1282, user nix-grpc-daemon (trusted)5931worker1 # [ 87.290757] nix-daemon[842]: reaped child process 1405, status = succeeded5932worker1 # [ 87.296061] nix-daemon[842]: accepted connection from pid 1282, user nix-grpc-daemon (trusted)5933client # [ 87.312307] nix[1214]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5934lb: must succeed: curl -sf localhost:9901/clusters5935lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5936lb: must succeed: curl -sf localhost:9901/clusters5937lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5938worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5939worker1 # [ 89.426307] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1282 (nix-grpc-daemon) on client request.5940worker1 # [ 89.431344] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5941lb # [ 89.433535] niks3-server[983]: 2026/09/21 13:22:31 INFO lead: released remote=192.168.1.3:347605942client # [ 89.459623] nix[1214]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5943worker2 # [ 89.458178] nix-grpc-daemon[1390]: ts=2026-09-21T13:22:32Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5944worker1 # [ 89.448719] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5945client # [ 89.464451] nix[1214]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5946worker1 # [ 89.450255] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5947worker1 # [ 89.456556] systemd[1]: nix-grpc-daemon.service: Consumed 470ms CPU time over 16.403s wall clock time, 35.4M memory peak, 17K incoming IP traffic, 31.5K outgoing IP traffic.5948worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.07 seconds)5949client: must succeed: sleep 5.05950worker1 # [ 89.475779] nix-daemon[842]: reaped child process 1433, status = succeeded5951worker1 # [ 89.477190] nix-daemon[842]: reaped child process 1397, status = succeeded5952worker1 # [ 89.485268] nix-daemon[1435]: unexpected Nix daemon error: error: interrupted by the user5953worker1 # [ 89.489596] nix-daemon[842]: reaped child process 1435, status = failed with exit code 15954worker2 # [ 89.658991] nix-grpc-daemon[1390]: ts=2026-09-21T13:22:32Z level=info event=scheduler_connected addr=lb:500515955worker1 # [ 89.648168] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.5956worker1 # [ 89.657184] systemd[1]: Starting Nix worker-protocol over gRPC...5957worker1 # [ 89.763827] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:32Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5958worker1 # [ 89.765889] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:32Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5959worker1 # [ 89.769420] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:32Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15960worker1 # [ 89.773885] nix-grpc-daemon[1472]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5961worker1 # [ 89.775343] nix-grpc-daemon[1472]: E0000 00:00:1789996952.932692 1472 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5962worker1 # [ 89.782066] nix-grpc-daemon[1472]: E0000 00:00:1789996952.938216 1472 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5963worker1 # [ 89.784161] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:32Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5964worker1 # [ 89.785727] systemd[1]: Started Nix worker-protocol over gRPC.5965worker1 # [ 89.791061] nix-grpc-daemon[1472]: E0000 00:00:1789996952.950231 1513 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5966lb # [ 89.786875] niks3-server[983]: 2026/09/21 13:22:32 INFO lead: acquired remote=192.168.1.3:529665967worker1 # [ 89.795149] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:32Z level=info event=scheduler_take_over5968worker1 # [ 89.802931] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:32Z level=info event=oidc_jwks provider=mock keys=15969worker1 # [ 89.805104] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:32Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:545825970worker1 # [ 89.806647] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:32Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05971worker1 # [ 89.813408] nix-grpc-daemon[1472]: E0000 00:00:1789996952.972641 1513 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5972worker1 # [ 89.818056] nix-grpc-daemon[1472]: E0000 00:00:1789996952.974903 1513 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5973worker1 # [ 89.827462] nix-grpc-daemon[1472]: E0000 00:00:1789996952.986703 1513 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5974worker1 # [ 89.836412] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:32Z level=info event=scheduler_connected addr=lb:500515975worker1 # [ 89.838183] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:32Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:545825976worker1 # [ 89.839577] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:32Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05977worker1 # [ 89.956278] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:33Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05978worker1 # [ 89.960318] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:33Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=15979worker2 # [ 90.029860] nix-grpc-daemon[1390]: ts=2026-09-21T13:22:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:343965980worker2 # [ 90.041287] nix-daemon[865]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)5981worker2 # [ 90.057776] nix-grpc-daemon[1390]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 17 ms (attempt 1/5)5982worker2 # [ 90.082181] nix-grpc-daemon[1390]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 155 ms (attempt 2/5)5983worker2 # [ 90.239723] nix-grpc-daemon[1390]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 388 ms (attempt 3/5)5984worker2 # [ 90.630645] nix-grpc-daemon[1390]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 471 ms (attempt 4/5)5985worker2 # [ 91.103345] nix-grpc-daemon[1390]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5986worker2 # [ 91.107224] nix-grpc-daemon[1390]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5987worker2 # [ 91.127436] nix-grpc-daemon[1390]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' from 'http://lb:5751'...5988worker2 # [ 91.153400] nix-daemon[865]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)5989client # [ 91.168277] nix[1214]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5990worker2 # [ 91.166994] nix-daemon[865]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)5991client: (finished: must succeed: sleep 5.0, in 5.03 seconds)5992client: waiting for success: ! systemctl is-active mid5993worker2 # [ 94.784265] nix-grpc-daemon[1390]: ts=2026-09-21T13:22:37Z level=info event=niks3_push_started pid=14965994worker2 # [ 94.799499] nix-daemon[865]: reaped child process 1471, status = succeeded5995worker2 # [ 94.882206] nix-grpc-daemon[1496]: time=2026-09-21T13:22:38.057Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""5996worker2 # [ 94.967751] nix-daemon[865]: accepted connection from pid 1509, user nix-grpc-daemon (trusted)5997worker2 # [ 94.980371] nix-daemon[865]: reaped child process 1512, status = succeeded5998lb # [ 94.968878] niks3-server[983]: 2026/09/21 13:22:37 INFO Received uploads request method=POST path=/api/pending_closures5999worker2 # [ 95.001248] nix-grpc-daemon[1496]: time=2026-09-21T13:22:38.178Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6000worker2 # [ 95.002860] nix-grpc-daemon[1496]: time=2026-09-21T13:22:38.179Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6001lb # [ 95.019701] niks3-server[983]: 2026/09/21 13:22:37 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6002lb # [ 95.026245] niks3-server[983]: 2026/09/21 13:22:37 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6003lb # [ 95.035263] niks3-server[983]: 2026/09/21 13:22:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign6004lb # [ 95.039845] niks3-server[983]: 2026/09/21 13:22:37 INFO Signed narinfos id=36 count=16005worker2 # [ 95.061699] nix-grpc-daemon[1496]: time=2026-09-21T13:22:38.237Z level=INFO msg="Uploading 1 narinfos"6006lb # [ 95.047559] niks3-server[983]: 2026/09/21 13:22:37 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6007lb # [ 95.050581] niks3-server[983]: 2026/09/21 13:22:37 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete6008lb # [ 95.055291] niks3-server[983]: 2026/09/21 13:22:37 INFO Completed upload id=366009worker2 # [ 95.076735] nix-grpc-daemon[1496]: time=2026-09-21T13:22:38.253Z level=INFO msg="Upload complete. (195ms)"6010worker2 # [ 95.081916] nix-grpc-daemon[1390]: ts=2026-09-21T13:22:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34396 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16011worker2 # [ 95.087629] nix-grpc-daemon[1496]: time=2026-09-21T13:22:38.263Z level=WARN msg="Failed to register uploaded object" key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo error="server returned 404: object is not pending upload\n"6012worker2 # [ 95.099309] nix-daemon[865]: reaped child process 1469, status = succeeded6013client # [ 95.117530] systemd[1]: mid.service: Deactivated successfully.6014client # [ 95.120658] systemd[1]: mid.service: Consumed 146ms CPU time over 9.290s wall clock time, 43.4M memory peak, 8.7K incoming IP traffic, 10.4K outgoing IP traffic.6015lb # [ 95.558306] envoy[916]: conn peer=[2001:db8:1::3]:35440 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2386 tx=618 ms=162416016lb # [ 95.562340] envoy[916]: conn peer=[2001:db8:1::1]:42440 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3921 tx=1876 ms=91266017client: (finished: waiting for success: ! systemctl is-active mid, in 1.08 seconds)6018client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6019client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.03 seconds)6020(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.02 seconds)6021subtest: interrupting the client stops the build on the worker6022client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr6023client # [ 95.825835] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr.6024client # Running as unit: intr.service; invocation ID: 57df3ea52e0c4b22a674bae7b3ae43836025client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr, in 0.03 seconds)6026worker1 # [ 95.935342] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:39Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:444086027worker1 # [ 95.946954] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6028client # [ 95.977217] nix[1250]: don't know how to build these paths:6029client # [ 95.978050] nix[1250]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6030worker1 # [ 95.969541] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:39Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06031worker1 # [ 95.974779] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:39Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.3:50051 clients=16032worker1 # [ 96.019177] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444086033worker1 # [ 96.031916] nix-grpc-daemon[1472]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 89 ms (attempt 1/5)6034worker1 # [ 96.129672] nix-grpc-daemon[1472]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 56 ms (attempt 2/5)6035worker1 # [ 96.187812] nix-grpc-daemon[1472]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 90 ms (attempt 3/5)6036worker1 # [ 96.280086] nix-grpc-daemon[1472]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 513 ms (attempt 4/5)6037worker1 # [ 96.795473] nix-grpc-daemon[1472]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6038worker1 # [ 96.799455] nix-grpc-daemon[1472]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6039worker1 # [ 96.817382] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:444086040client # [ 96.843690] nix[1250]: copying 1 paths to 'lb:50051'...6041client # [ 96.847185] nix[1250]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6042worker1 # [ 96.831587] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:444086043worker1 # [ 96.842137] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6044worker1 # [ 96.864639] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:40Z level=info event=niks3_push_started pid=15496045worker1 # [ 96.986668] nix-grpc-daemon[1549]: time=2026-09-21T13:22:40.144Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6046worker1 # [ 97.077977] nix-daemon[842]: accepted connection from pid 1565, user nix-grpc-daemon (trusted)6047worker1 # [ 97.091928] nix-daemon[842]: reaped child process 1568, status = succeeded6048worker1 # [ 97.182920] nix-daemon[842]: accepted connection from pid 1570, user nix-grpc-daemon (trusted)6049worker1 # [ 97.197427] nix-daemon[842]: reaped child process 1573, status = succeeded6050lb # [ 97.200366] niks3-server[983]: 2026/09/21 13:22:39 INFO Received uploads request method=POST path=/api/pending_closures6051worker1 # [ 97.217388] nix-grpc-daemon[1549]: time=2026-09-21T13:22:40.376Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6052worker1 # [ 97.219238] nix-grpc-daemon[1549]: time=2026-09-21T13:22:40.378Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6053lb # [ 97.241909] niks3-server[983]: 2026/09/21 13:22:39 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6054lb # [ 97.249806] niks3-server[983]: 2026/09/21 13:22:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign6055lb # [ 97.256078] niks3-server[983]: 2026/09/21 13:22:39 INFO Signed narinfos id=37 count=16056worker1 # [ 97.265528] nix-grpc-daemon[1549]: time=2026-09-21T13:22:40.424Z level=INFO msg="Uploading 1 narinfos"6057lb # [ 97.267153] niks3-server[983]: 2026/09/21 13:22:39 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6058lb # [ 97.277521] niks3-server[983]: 2026/09/21 13:22:39 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete6059lb # [ 97.283739] niks3-server[983]: 2026/09/21 13:22:39 INFO Completed upload id=376060worker1 # [ 97.290118] nix-grpc-daemon[1549]: time=2026-09-21T13:22:40.449Z level=INFO msg="Upload complete. (300ms)"6061lb # [ 97.286076] niks3-server[983]: 2026/09/21 13:22:39 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6062worker1 # [ 97.294628] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44408 duration_s=0 paths=1 nar_bytes_in=4246063worker1 # [ 97.300268] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444086064worker1 # [ 97.311113] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6065worker1 # [ 97.316046] nix-daemon[842]: reaped child process 1547, status = succeeded6066worker1 # [ 97.321079] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6067client # [ 97.337827] nix[1250]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6068client: must succeed: systemctl kill -s INT intr6069client # [ 99.276203] systemd[1]: intr.service: Sent signal SIGINT to main process 1250 (nix) on client request.6070worker1 # [ 99.261871] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:42Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6071client # [ 99.281730] nix[1250]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6072client # [ 99.287831] nix[1250]: error: interrupted by the user6073client: (finished: must succeed: systemctl kill -s INT intr, in 0.05 seconds)6074client # [ 99.317956] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6075client # [ 99.320777] systemd[1]: intr.service: Failed with result 'exit-code'.6076client # [ 99.324386] systemd[1]: intr.service: Consumed 117ms CPU time over 3.497s wall clock time, 41.5M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic.6077worker1 # [ 99.341391] nix-daemon[1577]: unexpected Nix daemon error: error: interrupted by the user6078worker1 # [ 99.343309] nix-daemon[842]: reaped child process 1575, status = succeeded6079worker1 # [ 99.347990] nix-daemon[842]: reaped child process 1577, status = failed with exit code 16080(finished: subtest: interrupting the client stops the build on the worker, in 4.68 seconds)6081subtest: a deploy mid-build drains: the build finishes, the new generation takes over6082client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw6083client # [ 100.530805] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw.6084client # Running as unit: sw.service; invocation ID: 17492383ddd54619ba1a022c06e82c7e6085client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw, in 0.06 seconds)6086worker2 # [ 100.662953] nix-grpc-daemon[1390]: ts=2026-09-21T13:22:43Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:343966087client # [ 100.670987] nix[1271]: don't know how to build these paths:6088client # [ 100.671902] nix[1271]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6089worker1 # [ 100.663813] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:43Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06090worker1 # [ 100.668411] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:43Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.4:50051 clients=16091worker2 # [ 100.722631] nix-grpc-daemon[1390]: ts=2026-09-21T13:22:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:343966092worker2 # [ 100.735898] nix-grpc-daemon[1390]: ts=2026-09-21T13:22:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:343966093client # [ 100.748238] nix[1271]: copying 1 paths to 'lb:50051'...6094worker2 # [ 100.747972] nix-grpc-daemon[1390]: ts=2026-09-21T13:22:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:343966095client # [ 100.750746] nix[1271]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6096worker2 # [ 100.760990] nix-daemon[865]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)6097worker2 # [ 100.861063] nix-daemon[865]: accepted connection from pid 1544, user nix-grpc-daemon (trusted)6098worker2 # [ 100.873746] nix-daemon[865]: reaped child process 1547, status = succeeded6099worker2 # [ 100.959178] nix-daemon[865]: accepted connection from pid 1549, user nix-grpc-daemon (trusted)6100worker2 # [ 100.972412] nix-daemon[865]: reaped child process 1552, status = succeeded6101lb # [ 100.958436] niks3-server[983]: 2026/09/21 13:22:43 INFO Received uploads request method=POST path=/api/pending_closures6102worker2 # [ 100.993294] nix-grpc-daemon[1496]: time=2026-09-21T13:22:44.169Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6103worker2 # [ 100.994791] nix-grpc-daemon[1496]: time=2026-09-21T13:22:44.169Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6104lb # [ 100.992652] niks3-server[983]: 2026/09/21 13:22:43 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6105lb # [ 101.001211] niks3-server[983]: 2026/09/21 13:22:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6106worker2 # [ 101.026289] nix-grpc-daemon[1496]: time=2026-09-21T13:22:44.203Z level=INFO msg="Uploading 1 narinfos"6107lb # [ 101.005744] niks3-server[983]: 2026/09/21 13:22:43 INFO Signed narinfos id=38 count=16108lb # [ 101.013581] niks3-server[983]: 2026/09/21 13:22:43 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6109lb # [ 101.020980] niks3-server[983]: 2026/09/21 13:22:43 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6110lb # [ 101.026174] niks3-server[983]: 2026/09/21 13:22:43 INFO Completed upload id=386111worker2 # [ 101.047252] nix-grpc-daemon[1496]: time=2026-09-21T13:22:44.224Z level=INFO msg="Upload complete. (274ms)"6112lb # [ 101.028275] niks3-server[983]: 2026/09/21 13:22:43 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6113worker2 # [ 101.049437] nix-grpc-daemon[1390]: ts=2026-09-21T13:22:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34396 duration_s=0 paths=1 nar_bytes_in=4166114worker2 # [ 101.053399] nix-grpc-daemon[1390]: ts=2026-09-21T13:22:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:343966115worker2 # [ 101.059722] nix-daemon[865]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)6116worker2 # [ 101.061988] nix-daemon[865]: reaped child process 1542, status = succeeded6117worker2 # [ 101.067151] nix-daemon[865]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)6118client # [ 101.068504] nix[1271]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6119worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&26120worker2 # Checking switch inhibitors... done6121worker2 # [ 103.274861] nixos[1588]: switching to system configuration /nix/store/gppmwq0spbv5rglkzygj0f7fzblbqs0p-nixos-system-worker2-test6122worker2 # [ 103.279238] systemd[1]: Stopped target Remote File Systems.6123worker2 # activating the configuration...6124worker2 # [ 103.280846] systemd[1]: Stopped target Local File Systems.6125worker2 # setting up /etc...6126worker2 # [ 103.706580] systemd[1]: Reload requested from client PID 1588 ('.switch-to-conf') (unit backdoor.service)...6127worker2 # [ 103.708374] systemd[1]: Reloading...6128worker2 # [ 105.072872] systemd[1]: Reloading finished in 1362 ms.6129worker2 # restarting sysinit-reactivation.target6130worker2 # [ 105.136478] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....6131worker2 # [ 105.241234] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..6132worker2 # [ 105.244735] systemd[1]: Reached target Reactivate sysinit units.6133worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service6134worker2 # [ 105.249911] systemd[1]: Reloading D-Bus System Message Bus...6135worker2 # [ 105.255290] dbus-broker-launch[516]: Caught SIGHUP, trigger reload.6136worker2 # [ 105.257202] systemd[1]: Reloading Nix worker-protocol over gRPC...6137worker2 # [ 105.269089] dbus-broker-launch[516]: Looking up NSS user entry for 'systemd-timesync'...6138worker2 # [ 105.279614] dbus-broker-launch[516]: NSS returned no entry for 'systemd-timesync'6139worker2 # [ 105.280870] dbus-broker-launch[516]: Invalid user-name in /nix/store/9f31zf0wixjfmyvwg6m53sif40vrkysg-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"6140worker2 # [ 105.288376] systemd[1]: Reloaded Nix worker-protocol over gRPC.6141worker2 # [ 105.299804] systemd[1]: Reloaded D-Bus System Message Bus.6142worker2 # [ 105.304061] systemd[1]: Reached target Remote File Systems.6143worker2 # [ 105.310731] systemd[1]: Reached target Local File Systems.6144worker2 # [ 105.315067] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore6145worker2 # [ 105.319946] systemd[1]: Starting Create SUID/SGID Wrappers...6146worker2 # [ 105.320793] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.6147worker2 # [ 105.323433] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc676148worker2 # [ 105.325493] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore6149worker2 # [ 105.326974] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6150worker2 # [ 105.332212] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6151worker2 # [ 105.333377] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os6152worker2 # [ 105.334655] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki6153worker2 # [ 105.501936] nix-grpc-daemon[1390]: ts=2026-09-21T13:22:48Z level=info event=draining6154worker2 # [ 105.579629] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.6155lb # [ 105.558958] envoy[916]: conn peer=[2001:db8:1::1]:38484 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2980 tx=852 ms=33556156worker2 # [ 105.584938] systemd[1]: Finished Create SUID/SGID Wrappers.6157worker2 # [ 105.586283] systemd[1]: Starting linger-users.service...6158worker2 # [ 105.614568] systemd[1]: linger-users.service: Deactivated successfully.6159worker2 # [ 105.619070] systemd[1]: Finished linger-users.service.6160worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service6161worker2 # [ 105.874070] nixos[1588]: finished switching to system configuration /nix/store/gppmwq0spbv5rglkzygj0f7fzblbqs0p-nixos-system-worker2-test6162worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 2.96 seconds)6163worker2: must succeed: pgrep -f 'read -t [9]0 x'6164worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.04 seconds)6165client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&26166worker2 # [ 106.057639] nix-grpc-daemon[1390]: ts=2026-09-21T13:22:49Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:343966167client # don't know how to build these paths:6168client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6169worker2 # [ 106.063234] nix-grpc-daemon[1390]: ts=2026-09-21T13:22:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:343966170worker1 # [ 106.068423] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:49Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06171worker1 # [ 106.072557] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:49Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.3:50051 clients=16172worker1 # [ 106.116460] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444086173worker1 # [ 106.129412] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:444086174client # copying 1 paths to 'lb:50051'...6175worker1 # [ 106.141951] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:444086176client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6177worker1 # [ 106.169598] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6178worker1 # [ 106.276336] nix-daemon[842]: accepted connection from pid 1634, user nix-grpc-daemon (trusted)6179worker1 # [ 106.290520] nix-daemon[842]: reaped child process 1637, status = succeeded6180worker1 # [ 106.379668] nix-daemon[842]: accepted connection from pid 1639, user nix-grpc-daemon (trusted)6181worker1 # [ 106.394276] nix-daemon[842]: reaped child process 1642, status = succeeded6182lb # [ 106.396430] niks3-server[983]: 2026/09/21 13:22:48 INFO Received uploads request method=POST path=/api/pending_closures6183worker1 # [ 106.413319] nix-grpc-daemon[1549]: time=2026-09-21T13:22:49.571Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6184worker1 # [ 106.414775] nix-grpc-daemon[1549]: time=2026-09-21T13:22:49.572Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6185lb # [ 106.422119] niks3-server[983]: 2026/09/21 13:22:48 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6186lb # [ 106.436659] niks3-server[983]: 2026/09/21 13:22:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6187lb # [ 106.442187] niks3-server[983]: 2026/09/21 13:22:48 INFO Signed narinfos id=39 count=16188worker1 # [ 106.451347] nix-grpc-daemon[1549]: time=2026-09-21T13:22:49.610Z level=INFO msg="Uploading 1 narinfos"6189lb # [ 106.450379] niks3-server[983]: 2026/09/21 13:22:48 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6190lb # [ 106.459223] niks3-server[983]: 2026/09/21 13:22:48 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6191lb # [ 106.465338] niks3-server[983]: 2026/09/21 13:22:48 INFO Completed upload id=396192worker1 # [ 106.471497] nix-grpc-daemon[1549]: time=2026-09-21T13:22:49.630Z level=WARN msg="Failed to register uploaded object" key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo error="server returned 404: object is not pending upload\n"6193worker1 # [ 106.477937] nix-grpc-daemon[1549]: time=2026-09-21T13:22:49.635Z level=INFO msg="Upload complete. (295ms)"6194worker1 # [ 106.481282] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44408 duration_s=0 paths=1 nar_bytes_in=5286195worker1 # [ 106.485582] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444086196worker1 # [ 106.493648] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6197worker1 # [ 106.501094] nix-daemon[842]: reaped child process 1631, status = succeeded6198worker1 # [ 106.504937] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6199client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6200client # farm-a> node-a: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6201client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6202client # farm-a> LOG-farm-a-during-drain6203worker1 # [ 107.335725] nix-daemon[842]: reaped child process 1646, status = succeeded6204worker1 # [ 107.412099] nix-daemon[842]: accepted connection from pid 1662, user nix-grpc-daemon (trusted)6205worker1 # [ 107.425473] nix-daemon[842]: reaped child process 1665, status = succeeded6206lb # [ 107.427619] niks3-server[983]: 2026/09/21 13:22:49 INFO Received uploads request method=POST path=/api/pending_closures6207worker1 # [ 107.442943] nix-grpc-daemon[1549]: time=2026-09-21T13:22:50.601Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6208worker1 # [ 107.444465] nix-grpc-daemon[1549]: time=2026-09-21T13:22:50.603Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6209lb # [ 107.468155] niks3-server[983]: 2026/09/21 13:22:49 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6210lb # [ 107.476446] niks3-server[983]: 2026/09/21 13:22:49 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6211lb # [ 107.488151] niks3-server[983]: 2026/09/21 13:22:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6212lb # [ 107.493967] niks3-server[983]: 2026/09/21 13:22:49 INFO Signed narinfos id=40 count=16213worker1 # [ 107.502769] nix-grpc-daemon[1549]: time=2026-09-21T13:22:50.662Z level=INFO msg="Uploading 1 narinfos"6214lb # [ 107.502739] niks3-server[983]: 2026/09/21 13:22:49 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6215lb # [ 107.511836] niks3-server[983]: 2026/09/21 13:22:49 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6216lb # [ 107.514563] niks3-server[983]: 2026/09/21 13:22:49 INFO Completed upload id=406217worker1 # [ 107.520828] nix-grpc-daemon[1549]: time=2026-09-21T13:22:50.680Z level=INFO msg="Upload complete. (209ms)"6218lb # [ 107.517438] niks3-server[983]: 2026/09/21 13:22:49 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6219worker1 # [ 107.528457] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44408 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16220worker1 # [ 107.537615] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:50Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06221worker1 # [ 107.541005] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:50Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.3:50051 clients=16222worker1 # [ 107.550891] nix-daemon[842]: reaped child process 1644, status = succeeded6223worker1 # [ 107.580617] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444086224worker1 # [ 107.598251] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:444086225worker1 # [ 107.611430] nix-grpc-daemon[1549]: time=2026-09-21T13:22:50.770Z level=INFO msg="All 1 paths already cached"6226client # copying 1 paths to 'lb:50051'...6227worker1 # [ 107.619754] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:444086228client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6229worker1 # [ 107.639167] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6230worker1 # [ 107.743482] nix-daemon[842]: accepted connection from pid 1671, user nix-grpc-daemon (trusted)6231worker1 # [ 107.758091] nix-daemon[842]: reaped child process 1674, status = succeeded6232worker1 # [ 107.846986] nix-daemon[842]: accepted connection from pid 1676, user nix-grpc-daemon (trusted)6233worker1 # [ 107.861142] nix-daemon[842]: reaped child process 1679, status = succeeded6234lb # [ 107.863660] niks3-server[983]: 2026/09/21 13:22:50 INFO Received uploads request method=POST path=/api/pending_closures6235worker1 # [ 107.878387] nix-grpc-daemon[1549]: time=2026-09-21T13:22:51.037Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6236worker1 # [ 107.879829] nix-grpc-daemon[1549]: time=2026-09-21T13:22:51.037Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6237lb # [ 107.893333] niks3-server[983]: 2026/09/21 13:22:50 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6238lb # [ 107.903820] niks3-server[983]: 2026/09/21 13:22:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6239lb # [ 107.909635] niks3-server[983]: 2026/09/21 13:22:50 INFO Signed narinfos id=41 count=16240worker1 # [ 107.919537] nix-grpc-daemon[1549]: time=2026-09-21T13:22:51.078Z level=INFO msg="Uploading 1 narinfos"6241lb # [ 107.919762] niks3-server[983]: 2026/09/21 13:22:50 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6242lb # [ 107.928713] niks3-server[983]: 2026/09/21 13:22:50 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6243lb # [ 107.935857] niks3-server[983]: 2026/09/21 13:22:50 INFO Completed upload id=416244worker1 # [ 107.941669] nix-grpc-daemon[1549]: time=2026-09-21T13:22:51.100Z level=INFO msg="Upload complete. (289ms)"6245lb # [ 107.937830] niks3-server[983]: 2026/09/21 13:22:50 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6246worker1 # [ 107.945743] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44408 duration_s=0 paths=1 nar_bytes_in=6406247worker1 # [ 107.951259] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444086248worker1 # [ 107.960828] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6249worker1 # [ 107.966845] nix-daemon[842]: reaped child process 1669, status = succeeded6250worker1 # [ 107.971266] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6251client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6252client # farm-b> node-a: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6253client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6254client # farm-b> LOG-farm-b-during-drain6255worker1 # [ 108.409434] nix-daemon[842]: reaped child process 1683, status = succeeded6256worker1 # [ 108.488099] nix-daemon[842]: accepted connection from pid 1699, user nix-grpc-daemon (trusted)6257worker1 # [ 108.500778] nix-daemon[842]: reaped child process 1702, status = succeeded6258lb # [ 108.503829] niks3-server[983]: 2026/09/21 13:22:50 INFO Received uploads request method=POST path=/api/pending_closures6259worker1 # [ 108.519090] nix-grpc-daemon[1549]: time=2026-09-21T13:22:51.678Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6260worker1 # [ 108.520590] nix-grpc-daemon[1549]: time=2026-09-21T13:22:51.679Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6261lb # [ 108.543932] niks3-server[983]: 2026/09/21 13:22:50 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6262lb # [ 108.554803] niks3-server[983]: 2026/09/21 13:22:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6263lb # [ 108.559300] niks3-server[983]: 2026/09/21 13:22:50 INFO Signed narinfos id=42 count=16264worker1 # [ 108.568829] nix-grpc-daemon[1549]: time=2026-09-21T13:22:51.727Z level=INFO msg="Uploading 1 narinfos"6265lb # [ 108.568316] niks3-server[983]: 2026/09/21 13:22:50 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6266lb # [ 108.583029] niks3-server[983]: 2026/09/21 13:22:50 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6267lb # [ 108.584592] niks3-server[983]: 2026/09/21 13:22:50 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6268lb # [ 108.587447] niks3-server[983]: 2026/09/21 13:22:50 INFO Completed upload id=426269lb # [ 108.589900] niks3-server[983]: 2026/09/21 13:22:50 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6270worker1 # [ 108.594909] nix-grpc-daemon[1549]: time=2026-09-21T13:22:51.754Z level=INFO msg="Upload complete. (217ms)"6271worker1 # [ 108.606151] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44408 duration_s=0 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16272worker1 # [ 108.612618] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:51Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06273worker1 # [ 108.616244] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:51Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.3:50051 clients=16274worker1 # [ 108.626249] nix-daemon[842]: reaped child process 1681, status = succeeded6275worker1 # [ 108.655810] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444086276worker1 # [ 108.668208] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:444086277worker1 # [ 108.681398] nix-grpc-daemon[1549]: time=2026-09-21T13:22:51.840Z level=INFO msg="All 2 paths already cached"6278client # copying 1 paths to 'lb:50051'...6279worker1 # [ 108.689731] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:444086280client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6281worker1 # [ 108.708358] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6282worker1 # [ 108.813544] nix-daemon[842]: accepted connection from pid 1708, user nix-grpc-daemon (trusted)6283worker1 # [ 108.828316] nix-daemon[842]: reaped child process 1711, status = succeeded6284worker1 # [ 108.917490] nix-daemon[842]: accepted connection from pid 1713, user nix-grpc-daemon (trusted)6285worker1 # [ 108.932645] nix-daemon[842]: reaped child process 1716, status = succeeded6286lb # [ 108.932911] niks3-server[983]: 2026/09/21 13:22:51 INFO Received uploads request method=POST path=/api/pending_closures6287worker1 # [ 108.947213] nix-grpc-daemon[1549]: time=2026-09-21T13:22:52.106Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6288worker1 # [ 108.948753] nix-grpc-daemon[1549]: time=2026-09-21T13:22:52.108Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6289lb # [ 108.964277] niks3-server[983]: 2026/09/21 13:22:51 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6290lb # [ 108.973764] niks3-server[983]: 2026/09/21 13:22:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6291lb # [ 108.978625] niks3-server[983]: 2026/09/21 13:22:51 INFO Signed narinfos id=43 count=16292worker1 # [ 108.989637] nix-grpc-daemon[1549]: time=2026-09-21T13:22:52.148Z level=INFO msg="Uploading 1 narinfos"6293lb # [ 108.991661] niks3-server[983]: 2026/09/21 13:22:51 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6294lb # [ 109.000149] niks3-server[983]: 2026/09/21 13:22:51 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6295lb # [ 109.008175] niks3-server[983]: 2026/09/21 13:22:51 INFO Completed upload id=436296worker1 # [ 109.014202] nix-grpc-daemon[1549]: time=2026-09-21T13:22:52.173Z level=INFO msg="Upload complete. (291ms)"6297lb # [ 109.011608] niks3-server[983]: 2026/09/21 13:22:51 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6298worker1 # [ 109.018362] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44408 duration_s=0 paths=1 nar_bytes_in=7686299worker1 # [ 109.023455] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444086300worker1 # [ 109.035111] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6301worker1 # [ 109.039761] nix-daemon[842]: reaped child process 1706, status = succeeded6302worker1 # [ 109.044834] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6303client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6304client # farm-top> node-a: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6305client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6306client # farm-top> LOG-farm-top-during-drain6307worker1 # [ 110.064152] nix-daemon[842]: reaped child process 1720, status = succeeded6308worker1 # [ 110.139886] nix-daemon[842]: accepted connection from pid 1736, user nix-grpc-daemon (trusted)6309worker1 # [ 110.154356] nix-daemon[842]: reaped child process 1739, status = succeeded6310lb # [ 110.157388] niks3-server[983]: 2026/09/21 13:22:52 INFO Received uploads request method=POST path=/api/pending_closures6311worker1 # [ 110.172577] nix-grpc-daemon[1549]: time=2026-09-21T13:22:53.331Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6312worker1 # [ 110.174210] nix-grpc-daemon[1549]: time=2026-09-21T13:22:53.333Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6313lb # [ 110.197463] niks3-server[983]: 2026/09/21 13:22:52 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6314lb # [ 110.209750] niks3-server[983]: 2026/09/21 13:22:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6315lb # [ 110.214927] niks3-server[983]: 2026/09/21 13:22:52 INFO Signed narinfos id=44 count=16316worker1 # [ 110.224401] nix-grpc-daemon[1549]: time=2026-09-21T13:22:53.383Z level=INFO msg="Uploading 1 narinfos"6317lb # [ 110.222362] niks3-server[983]: 2026/09/21 13:22:52 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6318lb # [ 110.227130] niks3-server[983]: 2026/09/21 13:22:52 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6319lb # [ 110.237390] niks3-server[983]: 2026/09/21 13:22:52 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6320lb # [ 110.242658] niks3-server[983]: 2026/09/21 13:22:52 INFO Completed upload id=446321lb # [ 110.243985] niks3-server[983]: 2026/09/21 13:22:52 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6322worker1 # [ 110.248952] nix-grpc-daemon[1549]: time=2026-09-21T13:22:53.408Z level=INFO msg="Upload complete. (197ms)"6323worker1 # [ 110.257257] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44408 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16324worker1 # [ 110.271242] nix-daemon[842]: reaped child process 1718, status = succeeded6325client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&2, in 4.39 seconds)6326client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath6327client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath, in 0.12 seconds)6328worker2: must succeed: pkill -f 'read -t [9]0 x'6329worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.04 seconds)6330client: waiting for success: ! systemctl is-active sw6331worker2 # [ 110.494913] nix-daemon[865]: reaped child process 1556, status = succeeded6332worker2 # [ 110.572406] nix-daemon[865]: accepted connection from pid 1729, user nix-grpc-daemon (trusted)6333worker2 # [ 110.585334] nix-daemon[865]: reaped child process 1732, status = succeeded6334lb # [ 110.574260] niks3-server[983]: 2026/09/21 13:22:52 INFO Received uploads request method=POST path=/api/pending_closures6335worker2 # [ 110.605557] nix-grpc-daemon[1496]: time=2026-09-21T13:22:53.782Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6336worker2 # [ 110.607169] nix-grpc-daemon[1496]: time=2026-09-21T13:22:53.784Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6337lb # [ 110.612156] niks3-server[983]: 2026/09/21 13:22:52 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6338lb # [ 110.617980] niks3-server[983]: 2026/09/21 13:22:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6339lb # [ 110.620720] niks3-server[983]: 2026/09/21 13:22:52 INFO Signed narinfos id=45 count=16340worker2 # [ 110.641686] nix-grpc-daemon[1496]: time=2026-09-21T13:22:53.818Z level=INFO msg="Uploading 1 narinfos"6341lb # [ 110.622670] niks3-server[983]: 2026/09/21 13:22:52 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6342lb # [ 110.632767] niks3-server[983]: 2026/09/21 13:22:52 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6343lb # [ 110.638354] niks3-server[983]: 2026/09/21 13:22:52 INFO Completed upload id=456344worker2 # [ 110.659276] nix-grpc-daemon[1496]: time=2026-09-21T13:22:53.836Z level=INFO msg="Upload complete. (173ms)"6345lb # [ 110.642389] niks3-server[983]: 2026/09/21 13:22:52 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6346worker2 # [ 110.663162] nix-grpc-daemon[1390]: ts=2026-09-21T13:22:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34396 duration_s=9 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16347worker2 # [ 110.666864] nix-grpc-daemon[1496]: time=2026-09-21T13:22:53.843Z level=WARN msg="Failed to register uploaded object" key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls error="server returned 404: object is not pending upload\n"6348worker2 # [ 110.672442] nix-daemon[865]: reaped child process 1554, status = succeeded6349client # [ 110.695859] systemd[1]: sw.service: Deactivated successfully.6350client # [ 110.700187] systemd[1]: sw.service: Consumed 123ms CPU time over 10.167s wall clock time, 41.5M memory peak, 6.9K incoming IP traffic, 8.2K outgoing IP traffic.6351worker2 # [ 111.504819] nix-grpc-daemon[1390]: ts=2026-09-21T13:22:54Z level=info event=signal_exit6352client: (finished: waiting for success: ! systemctl is-active sw, in 1.06 seconds)6353client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6354client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds)6355worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6356worker2 # [ 111.609463] systemd[1]: Starting scheduler-token.service...6357worker2 # [ 111.648097] systemd[1]: scheduler-token.service: Deactivated successfully.6358worker2 # [ 111.649417] systemd[1]: Finished scheduler-token.service.6359worker2 # [ 111.650632] systemd[1]: scheduler-token.service: Consumed 16ms CPU time over 39ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6360worker2 # [ 112.510723] nix-grpc-daemon[1390]: ts=2026-09-21T13:22:55Z level=info event=scheduler_restarting6361worker1 # [ 112.496839] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:55Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:545826362worker1 # [ 112.500583] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:55Z level=info event=worker_gone addr=192.168.1.4:500516363worker2 # [ 112.524950] nix-grpc-daemon[1390]: ts=2026-09-21T13:22:55Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6364worker2 # [ 112.543458] nix-daemon[865]: reaped child process 1462, status = succeeded6365worker2 # [ 113.380947] nix-grpc-daemon[1390]: E0000 00:00:1789996976.557657 1435 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true6366worker2 # [ 113.386092] nix-grpc-daemon[1390]: E0000 00:00:1789996976.557808 1435 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6367worker2 # [ 113.540393] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6368worker2 # [ 113.544552] systemd[1]: nix-grpc-daemon.service: Consumed 618ms CPU time over 28.142s wall clock time, 36.2M memory peak, 21K incoming IP traffic, 28.5K outgoing IP traffic.6369worker2 # [ 113.561946] systemd[1]: Starting Nix worker-protocol over gRPC...6370worker2 # [ 113.653400] nix-grpc-daemon[1749]: ts=2026-09-21T13:22:56Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6371worker2 # [ 113.655499] nix-grpc-daemon[1749]: ts=2026-09-21T13:22:56Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6372worker2 # [ 113.658756] nix-grpc-daemon[1749]: ts=2026-09-21T13:22:56Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16373worker2 # [ 113.663051] nix-grpc-daemon[1749]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6374worker2 # [ 113.666176] nix-grpc-daemon[1749]: E0000 00:00:1789996976.839980 1749 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6375worker2 # [ 113.671062] nix-grpc-daemon[1749]: E0000 00:00:1789996976.845449 1749 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6376worker2 # [ 113.674118] nix-grpc-daemon[1749]: ts=2026-09-21T13:22:56Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6377worker2 # [ 113.677901] systemd[1]: Started Nix worker-protocol over gRPC.6378worker2 # [ 113.682075] nix-grpc-daemon[1749]: E0000 00:00:1789996976.856988 1790 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6379worker2 # [ 113.693659] nix-grpc-daemon[1749]: E0000 00:00:1789996976.870620 1790 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6380worker2 # [ 113.697167] nix-grpc-daemon[1749]: E0000 00:00:1789996976.872915 1790 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6381worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.15 seconds)6382lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6383lb: must succeed: curl -sf localhost:9901/clusters6384worker2 # [ 113.709622] nix-grpc-daemon[1749]: E0000 00:00:1789996976.885926 1790 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6385worker2 # [ 113.719961] nix-grpc-daemon[1749]: ts=2026-09-21T13:22:56Z level=info event=scheduler_connected addr=lb:500516386worker1 # [ 113.708485] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:56Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:545826387worker1 # [ 113.713288] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:56Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06388lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6389lb: must succeed: curl -sf localhost:9901/clusters6390lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6391lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.10 seconds)6392lb: must succeed: curl -sf localhost:9901/clusters6393lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6394worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6395worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6396(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 14.39 seconds)6397subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6398client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16399worker1 # [ 115.006560] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:444086400worker1 # [ 115.012742] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:444086401worker1 # [ 115.053165] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:58Z level=debug event=want client=5 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06402worker1 # [ 115.057270] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:58Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16403worker1 # [ 115.061639] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:58Z level=debug event=want client=5 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06404worker1 # [ 115.066907] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:58Z level=debug event=want client=5 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06405worker1 # [ 115.110713] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444086406worker1 # [ 115.123421] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:444086407worker1 # [ 115.135129] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:444086408worker1 # [ 115.147412] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6409worker1 # [ 115.250458] nix-daemon[842]: accepted connection from pid 1748, user nix-grpc-daemon (trusted)6410worker1 # [ 115.265042] nix-daemon[842]: reaped child process 1751, status = succeeded6411worker1 # [ 115.348441] nix-daemon[842]: accepted connection from pid 1753, user nix-grpc-daemon (trusted)6412worker1 # [ 115.362766] nix-daemon[842]: reaped child process 1756, status = succeeded6413lb # [ 115.364684] niks3-server[983]: 2026/09/21 13:22:57 INFO Received uploads request method=POST path=/api/pending_closures6414worker1 # [ 115.384523] nix-grpc-daemon[1549]: time=2026-09-21T13:22:58.543Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6415worker1 # [ 115.386046] nix-grpc-daemon[1549]: time=2026-09-21T13:22:58.543Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6416lb # [ 115.409432] niks3-server[983]: 2026/09/21 13:22:57 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6417lb # [ 115.421286] niks3-server[983]: 2026/09/21 13:22:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6418worker1 # [ 115.433023] nix-grpc-daemon[1549]: time=2026-09-21T13:22:58.592Z level=INFO msg="Uploading 1 narinfos"6419lb # [ 115.427099] niks3-server[983]: 2026/09/21 13:22:57 INFO Signed narinfos id=46 count=16420lb # [ 115.433317] niks3-server[983]: 2026/09/21 13:22:57 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6421lb # [ 115.442964] niks3-server[983]: 2026/09/21 13:22:57 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6422lb # [ 115.445739] niks3-server[983]: 2026/09/21 13:22:57 INFO Completed upload id=466423worker1 # [ 115.451869] nix-grpc-daemon[1549]: time=2026-09-21T13:22:58.611Z level=INFO msg="Upload complete. (292ms)"6424lb # [ 115.448624] niks3-server[983]: 2026/09/21 13:22:57 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6425worker1 # [ 115.457111] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44408 duration_s=0 paths=1 nar_bytes_in=5046426worker1 # [ 115.462767] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444086427worker1 # [ 115.474485] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6428worker1 # [ 115.481142] nix-daemon[842]: reaped child process 1746, status = succeeded6429worker1 # [ 115.484736] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6430lb # [ 115.559348] envoy[916]: conn peer=[2001:db8:1::1]:47752 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9568 tx=4043 ms=42256431lb # [ 115.563547] envoy[916]: conn peer=[2001:db8:1::1]:38496 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2958 tx=1389 ms=100126432lb # [ 115.567704] envoy[916]: conn peer=[2001:db8:1::4]:47094 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=2366 tx=425 ms=269756433worker1 # [ 115.901368] nix-daemon[842]: reaped child process 1760, status = succeeded6434worker1 # [ 115.976392] nix-daemon[842]: accepted connection from pid 1776, user nix-grpc-daemon (trusted)6435worker1 # [ 115.991376] nix-daemon[842]: reaped child process 1779, status = succeeded6436lb # [ 115.990770] niks3-server[983]: 2026/09/21 13:22:58 INFO Received uploads request method=POST path=/api/pending_closures6437worker1 # [ 116.004820] nix-grpc-daemon[1549]: time=2026-09-21T13:22:59.164Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6438worker1 # [ 116.006427] nix-grpc-daemon[1549]: time=2026-09-21T13:22:59.165Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6439lb # [ 116.024940] niks3-server[983]: 2026/09/21 13:22:58 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6440lb # [ 116.040719] niks3-server[983]: 2026/09/21 13:22:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6441worker1 # [ 116.051172] nix-grpc-daemon[1549]: time=2026-09-21T13:22:59.210Z level=INFO msg="Uploading 1 narinfos"6442lb # [ 116.045539] niks3-server[983]: 2026/09/21 13:22:58 INFO Signed narinfos id=47 count=16443lb # [ 116.051479] niks3-server[983]: 2026/09/21 13:22:58 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6444lb # [ 116.061791] niks3-server[983]: 2026/09/21 13:22:58 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6445lb # [ 116.063780] niks3-server[983]: 2026/09/21 13:22:58 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6446lb # [ 116.069488] niks3-server[983]: 2026/09/21 13:22:58 INFO Completed upload id=476447worker1 # [ 116.075580] nix-grpc-daemon[1549]: time=2026-09-21T13:22:59.234Z level=INFO msg="Upload complete. (184ms)"6448lb # [ 116.071778] niks3-server[983]: 2026/09/21 13:22:58 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6449worker1 # [ 116.081787] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44408 duration_s=0 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=7 outputs=16450worker1 # [ 116.090232] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:59Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16451worker1 # [ 116.096115] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444086452worker1 # [ 116.099268] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:59Z level=debug event=want client=5 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06453worker1 # [ 116.108232] nix-daemon[842]: reaped child process 1758, status = succeeded6454worker1 # [ 116.130202] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:444086455worker1 # [ 116.139604] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:444086456worker1 # [ 116.146965] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6457worker1 # [ 116.237295] nix-daemon[842]: accepted connection from pid 1785, user nix-grpc-daemon (trusted)6458worker1 # [ 116.251722] nix-daemon[842]: reaped child process 1788, status = succeeded6459worker1 # [ 116.335872] nix-daemon[842]: accepted connection from pid 1790, user nix-grpc-daemon (trusted)6460worker1 # [ 116.351048] nix-daemon[842]: reaped child process 1793, status = succeeded6461lb # [ 116.350861] niks3-server[983]: 2026/09/21 13:22:58 INFO Received uploads request method=POST path=/api/pending_closures6462worker1 # [ 116.365378] nix-grpc-daemon[1549]: time=2026-09-21T13:22:59.524Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6463worker1 # [ 116.366774] nix-grpc-daemon[1549]: time=2026-09-21T13:22:59.524Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6464lb # [ 116.377640] niks3-server[983]: 2026/09/21 13:22:58 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6465lb # [ 116.385671] niks3-server[983]: 2026/09/21 13:22:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6466lb # [ 116.391873] niks3-server[983]: 2026/09/21 13:22:58 INFO Signed narinfos id=48 count=16467worker1 # [ 116.401144] nix-grpc-daemon[1549]: time=2026-09-21T13:22:59.559Z level=INFO msg="Uploading 1 narinfos"6468lb # [ 116.404702] niks3-server[983]: 2026/09/21 13:22:58 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6469lb # [ 116.411119] niks3-server[983]: 2026/09/21 13:22:58 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6470lb # [ 116.417505] niks3-server[983]: 2026/09/21 13:22:58 INFO Completed upload id=486471lb # [ 116.419724] niks3-server[983]: 2026/09/21 13:22:58 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6472worker1 # [ 116.425164] nix-grpc-daemon[1549]: time=2026-09-21T13:22:59.584Z level=INFO msg="Upload complete. (269ms)"6473worker1 # [ 116.428577] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44408 duration_s=0 paths=1 nar_bytes_in=5046474worker1 # [ 116.434287] nix-grpc-daemon[1472]: ts=2026-09-21T13:22:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444086475worker1 # [ 116.442473] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6476worker1 # [ 116.448422] nix-daemon[842]: reaped child process 1783, status = succeeded6477worker1 # [ 116.453478] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6478worker1 # [ 117.671612] nix-daemon[842]: reaped child process 1797, status = succeeded6479worker1 # [ 117.746647] nix-daemon[842]: accepted connection from pid 1813, user nix-grpc-daemon (trusted)6480worker1 # [ 117.760577] nix-daemon[842]: reaped child process 1816, status = succeeded6481lb # [ 117.762931] niks3-server[983]: 2026/09/21 13:23:00 INFO Received uploads request method=POST path=/api/pending_closures6482worker1 # [ 117.777220] nix-grpc-daemon[1549]: time=2026-09-21T13:23:00.936Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6483worker1 # [ 117.780064] nix-grpc-daemon[1549]: time=2026-09-21T13:23:00.938Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6484lb # [ 117.797343] niks3-server[983]: 2026/09/21 13:23:00 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6485lb # [ 117.811550] niks3-server[983]: 2026/09/21 13:23:00 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6486lb # [ 117.824304] niks3-server[983]: 2026/09/21 13:23:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6487worker1 # [ 117.836337] nix-grpc-daemon[1549]: time=2026-09-21T13:23:00.995Z level=INFO msg="Uploading 1 narinfos"6488lb # [ 117.830061] niks3-server[983]: 2026/09/21 13:23:00 INFO Signed narinfos id=49 count=16489lb # [ 117.835830] niks3-server[983]: 2026/09/21 13:23:00 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6490lb # [ 117.844514] niks3-server[983]: 2026/09/21 13:23:00 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6491lb # [ 117.847278] niks3-server[983]: 2026/09/21 13:23:00 INFO Completed upload id=496492worker1 # [ 117.853528] nix-grpc-daemon[1549]: time=2026-09-21T13:23:01.012Z level=INFO msg="Upload complete. (192ms)"6493lb # [ 117.850130] niks3-server[983]: 2026/09/21 13:23:00 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6494worker1 # [ 117.865732] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:01Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16495worker1 # [ 117.871128] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44408 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=8 outputs=16496worker1 # [ 117.875624] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:01Z level=debug event=want client=5 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06497worker1 # [ 117.884800] nix-daemon[842]: reaped child process 1795, status = succeeded6498worker1 # [ 117.904679] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444086499worker1 # [ 117.915592] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:444086500worker1 # [ 117.926125] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:444086501worker1 # [ 117.932833] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6502worker1 # [ 118.025434] nix-daemon[842]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)6503worker1 # [ 118.042049] nix-daemon[842]: reaped child process 1825, status = succeeded6504worker1 # [ 118.124261] nix-daemon[842]: accepted connection from pid 1827, user nix-grpc-daemon (trusted)6505worker1 # [ 118.139172] nix-daemon[842]: reaped child process 1830, status = succeeded6506lb # [ 118.139276] niks3-server[983]: 2026/09/21 13:23:00 INFO Received uploads request method=POST path=/api/pending_closures6507worker1 # [ 118.153548] nix-grpc-daemon[1549]: time=2026-09-21T13:23:01.312Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6508worker1 # [ 118.155129] nix-grpc-daemon[1549]: time=2026-09-21T13:23:01.314Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6509lb # [ 118.171775] niks3-server[983]: 2026/09/21 13:23:00 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6510lb # [ 118.181467] niks3-server[983]: 2026/09/21 13:23:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6511worker1 # [ 118.193096] nix-grpc-daemon[1549]: time=2026-09-21T13:23:01.352Z level=INFO msg="Uploading 1 narinfos"6512lb # [ 118.186892] niks3-server[983]: 2026/09/21 13:23:00 INFO Signed narinfos id=50 count=16513lb # [ 118.196876] niks3-server[983]: 2026/09/21 13:23:00 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6514lb # [ 118.208828] niks3-server[983]: 2026/09/21 13:23:00 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6515lb # [ 118.215798] niks3-server[983]: 2026/09/21 13:23:00 INFO Completed upload id=506516worker1 # [ 118.223976] nix-grpc-daemon[1549]: time=2026-09-21T13:23:01.383Z level=INFO msg="Upload complete. (280ms)"6517lb # [ 118.219145] niks3-server[983]: 2026/09/21 13:23:00 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6518worker1 # [ 118.227313] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44408 duration_s=0 paths=1 nar_bytes_in=5046519worker1 # [ 118.232406] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444086520worker1 # [ 118.246067] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6521worker1 # [ 118.249133] nix-daemon[842]: reaped child process 1820, status = succeeded6522worker1 # [ 118.254378] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6523worker1 # [ 119.472244] nix-daemon[842]: reaped child process 1834, status = succeeded6524worker1 # [ 119.549671] nix-daemon[842]: accepted connection from pid 1851, user nix-grpc-daemon (trusted)6525worker1 # [ 119.562072] nix-daemon[842]: reaped child process 1854, status = succeeded6526lb # [ 119.563614] niks3-server[983]: 2026/09/21 13:23:01 INFO Received uploads request method=POST path=/api/pending_closures6527worker1 # [ 119.574388] nix-grpc-daemon[1549]: time=2026-09-21T13:23:02.733Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6528worker1 # [ 119.575879] nix-grpc-daemon[1549]: time=2026-09-21T13:23:02.735Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6529lb # [ 119.584480] niks3-server[983]: 2026/09/21 13:23:01 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6530lb # [ 119.587553] niks3-server[983]: 2026/09/21 13:23:01 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv6531lb # [ 119.590595] niks3-server[983]: 2026/09/21 13:23:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6532lb # [ 119.592868] niks3-server[983]: 2026/09/21 13:23:01 INFO Signed narinfos id=51 count=16533worker1 # [ 119.599059] nix-grpc-daemon[1549]: time=2026-09-21T13:23:02.758Z level=INFO msg="Uploading 1 narinfos"6534lb # [ 119.596214] niks3-server[983]: 2026/09/21 13:23:01 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls6535lb # [ 119.601126] niks3-server[983]: 2026/09/21 13:23:01 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6536lb # [ 119.604086] niks3-server[983]: 2026/09/21 13:23:01 INFO Completed upload id=516537worker1 # [ 119.610130] nix-grpc-daemon[1549]: time=2026-09-21T13:23:02.769Z level=INFO msg="Upload complete. (148ms)"6538lb # [ 119.607246] niks3-server[983]: 2026/09/21 13:23:01 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo6539worker1 # [ 119.613363] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44408 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=9 outputs=16540worker1 # [ 119.618096] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:02Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=16541worker1 # [ 119.620368] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444086542worker1 # [ 119.623291] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:02Z level=debug event=want client=5 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=06543worker1 # [ 119.630294] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:444086544worker1 # [ 119.632785] nix-daemon[842]: reaped child process 1832, status = succeeded6545worker1 # [ 119.636353] nix-grpc-daemon[1549]: time=2026-09-21T13:23:02.795Z level=INFO msg="All 1 paths already cached"6546worker1 # [ 119.640256] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:444086547worker1 # [ 119.644256] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6548worker1 # [ 119.730554] nix-daemon[842]: accepted connection from pid 1860, user nix-grpc-daemon (trusted)6549worker1 # [ 119.745786] nix-daemon[842]: reaped child process 1863, status = succeeded6550worker1 # [ 119.829372] nix-daemon[842]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)6551worker1 # [ 119.843623] nix-daemon[842]: reaped child process 1868, status = succeeded6552lb # [ 119.844131] niks3-server[983]: 2026/09/21 13:23:02 INFO Received uploads request method=POST path=/api/pending_closures6553worker1 # [ 119.854831] nix-grpc-daemon[1549]: time=2026-09-21T13:23:03.013Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6554worker1 # [ 119.856269] nix-grpc-daemon[1549]: time=2026-09-21T13:23:03.013Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"6555lb # [ 119.857587] niks3-server[983]: 2026/09/21 13:23:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6556lb # [ 119.859957] niks3-server[983]: 2026/09/21 13:23:02 INFO Signed narinfos id=52 count=16557worker1 # [ 119.866167] nix-grpc-daemon[1549]: time=2026-09-21T13:23:03.025Z level=INFO msg="Uploading 1 narinfos"6558lb # [ 119.863301] niks3-server[983]: 2026/09/21 13:23:02 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls6559lb # [ 119.866486] niks3-server[983]: 2026/09/21 13:23:02 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst6560lb # [ 119.870644] niks3-server[983]: 2026/09/21 13:23:02 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6561lb # [ 119.873916] niks3-server[983]: 2026/09/21 13:23:02 INFO Completed upload id=526562worker1 # [ 119.879852] nix-grpc-daemon[1549]: time=2026-09-21T13:23:03.039Z level=INFO msg="Upload complete. (229ms)"6563worker1 # [ 119.882552] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44408 duration_s=0 paths=1 nar_bytes_in=6246564lb # [ 119.877394] niks3-server[983]: 2026/09/21 13:23:02 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo6565worker1 # [ 119.885843] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444086566worker1 # [ 119.892190] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6567worker1 # [ 119.896853] nix-daemon[842]: reaped child process 1858, status = succeeded6568worker1 # [ 119.901092] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6569worker1 # [ 121.144752] nix-daemon[842]: reaped child process 1872, status = succeeded6570worker1 # [ 121.226099] nix-daemon[842]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)6571worker1 # [ 121.239462] nix-daemon[842]: reaped child process 1891, status = succeeded6572lb # [ 121.240196] niks3-server[983]: 2026/09/21 13:23:03 INFO Received uploads request method=POST path=/api/pending_closures6573worker1 # [ 121.255356] nix-grpc-daemon[1549]: time=2026-09-21T13:23:04.414Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6574worker1 # [ 121.256854] nix-grpc-daemon[1549]: time=2026-09-21T13:23:04.416Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"6575lb # [ 121.287259] niks3-server[983]: 2026/09/21 13:23:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6576lb # [ 121.289935] niks3-server[983]: 2026/09/21 13:23:03 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst6577lb # [ 121.295608] niks3-server[983]: 2026/09/21 13:23:03 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv6578lb # [ 121.299339] niks3-server[983]: 2026/09/21 13:23:03 INFO Signed narinfos id=53 count=16579worker1 # [ 121.308398] nix-grpc-daemon[1549]: time=2026-09-21T13:23:04.467Z level=INFO msg="Uploading 1 narinfos"6580lb # [ 121.308858] niks3-server[983]: 2026/09/21 13:23:03 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6581lb # [ 121.311194] niks3-server[983]: 2026/09/21 13:23:03 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls6582lb # [ 121.314612] niks3-server[983]: 2026/09/21 13:23:03 INFO Completed upload id=536583worker1 # [ 121.320679] nix-grpc-daemon[1549]: time=2026-09-21T13:23:04.479Z level=INFO msg="Upload complete. (208ms)"6584lb # [ 121.317661] niks3-server[983]: 2026/09/21 13:23:03 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo6585worker1 # [ 121.325601] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44408 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=10 outputs=16586worker1 # [ 121.336083] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:04Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=16587worker1 # [ 121.338907] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:04Z level=debug event=want client=5 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=06588worker1 # [ 121.345344] nix-daemon[842]: reaped child process 1870, status = succeeded6589worker1 # [ 121.376392] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444086590worker1 # [ 121.387919] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:444086591worker1 # [ 121.402605] nix-grpc-daemon[1549]: time=2026-09-21T13:23:04.561Z level=INFO msg="All 1 paths already cached"6592worker1 # [ 121.409270] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:444086593worker1 # [ 121.419308] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6594worker1 # [ 121.513765] nix-daemon[842]: accepted connection from pid 1897, user nix-grpc-daemon (trusted)6595worker1 # [ 121.528221] nix-daemon[842]: reaped child process 1900, status = succeeded6596worker1 # [ 121.612135] nix-daemon[842]: accepted connection from pid 1902, user nix-grpc-daemon (trusted)6597worker1 # [ 121.626749] nix-daemon[842]: reaped child process 1905, status = succeeded6598lb # [ 121.627481] niks3-server[983]: 2026/09/21 13:23:03 INFO Received uploads request method=POST path=/api/pending_closures6599worker1 # [ 121.642212] nix-grpc-daemon[1549]: time=2026-09-21T13:23:04.801Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6600worker1 # [ 121.643767] nix-grpc-daemon[1549]: time=2026-09-21T13:23:04.803Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"6601lb # [ 121.661462] niks3-server[983]: 2026/09/21 13:23:04 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst6602lb # [ 121.668154] niks3-server[983]: 2026/09/21 13:23:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6603worker1 # [ 121.678972] nix-grpc-daemon[1549]: time=2026-09-21T13:23:04.838Z level=INFO msg="Uploading 1 narinfos"6604lb # [ 121.672969] niks3-server[983]: 2026/09/21 13:23:04 INFO Signed narinfos id=54 count=16605lb # [ 121.681865] niks3-server[983]: 2026/09/21 13:23:04 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls6606lb # [ 121.691318] niks3-server[983]: 2026/09/21 13:23:04 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6607lb # [ 121.693946] niks3-server[983]: 2026/09/21 13:23:04 INFO Completed upload id=546608worker1 # [ 121.699990] nix-grpc-daemon[1549]: time=2026-09-21T13:23:04.859Z level=INFO msg="Upload complete. (270ms)"6609lb # [ 121.696412] niks3-server[983]: 2026/09/21 13:23:04 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo6610worker1 # [ 121.702158] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44408 duration_s=0 paths=1 nar_bytes_in=6246611worker1 # [ 121.706567] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444086612worker1 # [ 121.717136] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6613worker1 # [ 121.720694] nix-daemon[842]: reaped child process 1895, status = succeeded6614worker1 # [ 121.725849] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6615worker1 # [ 122.742272] nix-daemon[842]: reaped child process 1909, status = succeeded6616worker1 # [ 122.817949] nix-daemon[842]: accepted connection from pid 1925, user nix-grpc-daemon (trusted)6617worker1 # [ 122.833121] nix-daemon[842]: reaped child process 1928, status = succeeded6618lb # [ 122.833795] niks3-server[983]: 2026/09/21 13:23:05 INFO Received uploads request method=POST path=/api/pending_closures6619worker1 # [ 122.848211] nix-grpc-daemon[1549]: time=2026-09-21T13:23:06.007Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6620worker1 # [ 122.850881] nix-grpc-daemon[1549]: time=2026-09-21T13:23:06.009Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"6621lb # [ 122.869955] niks3-server[983]: 2026/09/21 13:23:05 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst6622lb # [ 122.889093] niks3-server[983]: 2026/09/21 13:23:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6623lb # [ 122.894672] niks3-server[983]: 2026/09/21 13:23:05 INFO Signed narinfos id=55 count=16624worker1 # [ 122.903770] nix-grpc-daemon[1549]: time=2026-09-21T13:23:06.062Z level=INFO msg="Uploading 1 narinfos"6625lb # [ 122.899502] niks3-server[983]: 2026/09/21 13:23:05 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls6626lb # [ 122.902847] niks3-server[983]: 2026/09/21 13:23:05 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv6627lb # [ 122.912208] niks3-server[983]: 2026/09/21 13:23:05 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6628lb # [ 122.915127] niks3-server[983]: 2026/09/21 13:23:05 INFO Completed upload id=556629worker1 # [ 122.922436] nix-grpc-daemon[1549]: time=2026-09-21T13:23:06.081Z level=INFO msg="Upload complete. (189ms)"6630lb # [ 122.918110] niks3-server[983]: 2026/09/21 13:23:05 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo6631worker1 # [ 122.931709] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44408 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=11 outputs=16632worker1 # [ 122.940836] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:06Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=16633worker1 # [ 122.944860] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444086634worker1 # [ 122.949219] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:06Z level=debug event=want client=5 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=06635worker1 # [ 122.959085] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:444086636worker1 # [ 122.961350] nix-daemon[842]: reaped child process 1907, status = succeeded6637worker1 # [ 122.966202] nix-grpc-daemon[1549]: time=2026-09-21T13:23:06.125Z level=INFO msg="All 1 paths already cached"6638worker1 # [ 122.970547] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:444086639worker1 # [ 122.974569] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6640worker1 # [ 123.062268] nix-daemon[842]: accepted connection from pid 1934, user nix-grpc-daemon (trusted)6641worker1 # [ 123.076896] nix-daemon[842]: reaped child process 1937, status = succeeded6642worker1 # [ 123.161041] nix-daemon[842]: accepted connection from pid 1939, user nix-grpc-daemon (trusted)6643worker1 # [ 123.175449] nix-daemon[842]: reaped child process 1942, status = succeeded6644lb # [ 123.176303] niks3-server[983]: 2026/09/21 13:23:05 INFO Received uploads request method=POST path=/api/pending_closures6645worker1 # [ 123.190754] nix-grpc-daemon[1549]: time=2026-09-21T13:23:06.349Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6646worker1 # [ 123.192336] nix-grpc-daemon[1549]: time=2026-09-21T13:23:06.349Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"6647lb # [ 123.204581] niks3-server[983]: 2026/09/21 13:23:05 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst6648lb # [ 123.213443] niks3-server[983]: 2026/09/21 13:23:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6649lb # [ 123.217744] niks3-server[983]: 2026/09/21 13:23:05 INFO Signed narinfos id=56 count=16650worker1 # [ 123.229152] nix-grpc-daemon[1549]: time=2026-09-21T13:23:06.388Z level=INFO msg="Uploading 1 narinfos"6651lb # [ 123.227350] niks3-server[983]: 2026/09/21 13:23:05 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls6652lb # [ 123.238598] niks3-server[983]: 2026/09/21 13:23:05 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6653worker1 # [ 123.250859] nix-grpc-daemon[1549]: time=2026-09-21T13:23:06.410Z level=INFO msg="Upload complete. (270ms)"6654lb # [ 123.245068] niks3-server[983]: 2026/09/21 13:23:05 INFO Completed upload id=566655worker1 # [ 123.253087] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44408 duration_s=0 paths=1 nar_bytes_in=6246656lb # [ 123.248796] niks3-server[983]: 2026/09/21 13:23:05 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo6657worker1 # [ 123.258099] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444086658worker1 # [ 123.264169] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6659worker1 # [ 123.267258] nix-daemon[842]: reaped child process 1932, status = succeeded6660worker1 # [ 123.272773] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6661worker1 # [ 123.687527] nix-daemon[842]: reaped child process 1946, status = succeeded6662worker1 # [ 123.763438] nix-daemon[842]: accepted connection from pid 1962, user nix-grpc-daemon (trusted)6663worker1 # [ 123.778068] nix-daemon[842]: reaped child process 1965, status = succeeded6664lb # [ 123.778721] niks3-server[983]: 2026/09/21 13:23:06 INFO Received uploads request method=POST path=/api/pending_closures6665worker1 # [ 123.792591] nix-grpc-daemon[1549]: time=2026-09-21T13:23:06.951Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6666worker1 # [ 123.794195] nix-grpc-daemon[1549]: time=2026-09-21T13:23:06.953Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"6667lb # [ 123.810644] niks3-server[983]: 2026/09/21 13:23:06 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv6668lb # [ 123.826070] niks3-server[983]: 2026/09/21 13:23:06 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst6669lb # [ 123.840270] niks3-server[983]: 2026/09/21 13:23:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6670worker1 # [ 123.850963] nix-grpc-daemon[1549]: time=2026-09-21T13:23:07.009Z level=INFO msg="Uploading 1 narinfos"6671lb # [ 123.844924] niks3-server[983]: 2026/09/21 13:23:06 INFO Signed narinfos id=57 count=16672lb # [ 123.851513] niks3-server[983]: 2026/09/21 13:23:06 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls6673lb # [ 123.856283] niks3-server[983]: 2026/09/21 13:23:06 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6674lb # [ 123.860265] niks3-server[983]: 2026/09/21 13:23:06 INFO Completed upload id=576675lb # [ 123.861550] niks3-server[983]: 2026/09/21 13:23:06 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo6676worker1 # [ 123.866263] nix-grpc-daemon[1549]: time=2026-09-21T13:23:07.025Z level=INFO msg="Upload complete. (187ms)"6677worker1 # [ 123.873294] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44408 duration_s=0 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=12 outputs=16678worker1 # [ 123.882708] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:07Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=16679worker1 # [ 123.887401] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444086680worker1 # [ 123.890806] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:07Z level=debug event=want client=5 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=06681worker1 # [ 123.900329] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:444086682worker1 # [ 123.903902] nix-daemon[842]: reaped child process 1944, status = succeeded6683worker1 # [ 123.907778] nix-grpc-daemon[1549]: time=2026-09-21T13:23:07.067Z level=INFO msg="All 2 paths already cached"6684worker1 # [ 123.911922] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:444086685worker1 # [ 123.916279] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6686worker1 # [ 124.003289] nix-daemon[842]: accepted connection from pid 1971, user nix-grpc-daemon (trusted)6687worker1 # [ 124.018730] nix-daemon[842]: reaped child process 1974, status = succeeded6688worker1 # [ 124.102947] nix-daemon[842]: accepted connection from pid 1976, user nix-grpc-daemon (trusted)6689worker1 # [ 124.118131] nix-daemon[842]: reaped child process 1979, status = succeeded6690lb # [ 124.117994] niks3-server[983]: 2026/09/21 13:23:06 INFO Received uploads request method=POST path=/api/pending_closures6691worker1 # [ 124.132189] nix-grpc-daemon[1549]: time=2026-09-21T13:23:07.291Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6692worker1 # [ 124.133589] nix-grpc-daemon[1549]: time=2026-09-21T13:23:07.291Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"6693lb # [ 124.145308] niks3-server[983]: 2026/09/21 13:23:06 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst6694lb # [ 124.156564] niks3-server[983]: 2026/09/21 13:23:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6695lb # [ 124.160969] niks3-server[983]: 2026/09/21 13:23:06 INFO Signed narinfos id=58 count=16696worker1 # [ 124.171290] nix-grpc-daemon[1549]: time=2026-09-21T13:23:07.330Z level=INFO msg="Uploading 1 narinfos"6697lb # [ 124.170407] niks3-server[983]: 2026/09/21 13:23:06 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls6698lb # [ 124.182461] niks3-server[983]: 2026/09/21 13:23:06 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6699worker1 # [ 124.193531] nix-grpc-daemon[1549]: time=2026-09-21T13:23:07.352Z level=INFO msg="Upload complete. (271ms)"6700lb # [ 124.187712] niks3-server[983]: 2026/09/21 13:23:06 INFO Completed upload id=586701lb # [ 124.191945] niks3-server[983]: 2026/09/21 13:23:06 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo6702worker1 # [ 124.196749] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44408 duration_s=0 paths=1 nar_bytes_in=7526703worker1 # [ 124.202596] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444086704worker1 # [ 124.212067] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6705worker1 # [ 124.217275] nix-daemon[842]: reaped child process 1969, status = succeeded6706worker1 # [ 124.222560] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6707worker1 # [ 125.441203] nix-daemon[842]: reaped child process 1983, status = succeeded6708worker1 # [ 125.517347] nix-daemon[842]: accepted connection from pid 1999, user nix-grpc-daemon (trusted)6709worker1 # [ 125.532414] nix-daemon[842]: reaped child process 2002, status = succeeded6710lb # [ 125.533468] niks3-server[983]: 2026/09/21 13:23:07 INFO Received uploads request method=POST path=/api/pending_closures6711worker1 # [ 125.547784] nix-grpc-daemon[1549]: time=2026-09-21T13:23:08.706Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6712worker1 # [ 125.549368] nix-grpc-daemon[1549]: time=2026-09-21T13:23:08.708Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"6713lb # [ 125.566560] niks3-server[983]: 2026/09/21 13:23:07 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv6714lb # [ 125.583195] niks3-server[983]: 2026/09/21 13:23:07 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst6715lb # [ 125.593429] niks3-server[983]: 2026/09/21 13:23:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6716lb # [ 125.596579] niks3-server[983]: 2026/09/21 13:23:07 INFO Signed narinfos id=59 count=16717worker1 # [ 125.605229] nix-grpc-daemon[1549]: time=2026-09-21T13:23:08.764Z level=INFO msg="Uploading 1 narinfos"6718lb # [ 125.609283] niks3-server[983]: 2026/09/21 13:23:07 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6719lb # [ 125.616913] niks3-server[983]: 2026/09/21 13:23:07 INFO Completed upload id=596720lb # [ 125.618422] niks3-server[983]: 2026/09/21 13:23:07 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls6721worker1 # [ 125.623189] nix-grpc-daemon[1549]: time=2026-09-21T13:23:08.782Z level=INFO msg="Upload complete. (192ms)"6722lb # [ 125.620886] niks3-server[983]: 2026/09/21 13:23:07 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo6723worker1 # [ 125.632418] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44408 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=13 outputs=16724worker1 # [ 125.638331] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:08Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=16725worker1 # [ 125.650178] nix-daemon[842]: reaped child process 1981, status = succeeded6726worker1 # [ 125.673783] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444086727worker1 # [ 125.686938] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:444086728worker1 # [ 125.700878] nix-grpc-daemon[1549]: time=2026-09-21T13:23:08.859Z level=INFO msg="All 2 paths already cached"6729worker1 # [ 125.708264] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:444086730worker1 # [ 125.716903] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6731worker1 # [ 125.816588] nix-daemon[842]: accepted connection from pid 2006, user nix-grpc-daemon (trusted)6732worker1 # [ 125.832467] nix-daemon[842]: reaped child process 2009, status = succeeded6733worker1 # [ 125.914803] nix-daemon[842]: accepted connection from pid 2011, user nix-grpc-daemon (trusted)6734worker1 # [ 125.929672] nix-daemon[842]: reaped child process 2014, status = succeeded6735lb # [ 125.929819] niks3-server[983]: 2026/09/21 13:23:08 INFO Received uploads request method=POST path=/api/pending_closures6736worker1 # [ 125.944031] nix-grpc-daemon[1549]: time=2026-09-21T13:23:09.103Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6737worker1 # [ 125.945450] nix-grpc-daemon[1549]: time=2026-09-21T13:23:09.103Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"6738lb # [ 125.958303] niks3-server[983]: 2026/09/21 13:23:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6739worker1 # [ 125.971737] nix-grpc-daemon[1549]: time=2026-09-21T13:23:09.131Z level=INFO msg="Uploading 1 narinfos"6740lb # [ 125.965706] niks3-server[983]: 2026/09/21 13:23:08 INFO Signed narinfos id=60 count=16741lb # [ 125.970877] niks3-server[983]: 2026/09/21 13:23:08 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst6742lb # [ 125.977063] niks3-server[983]: 2026/09/21 13:23:08 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls6743lb # [ 125.992381] niks3-server[983]: 2026/09/21 13:23:08 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6744lb # [ 125.996629] niks3-server[983]: 2026/09/21 13:23:08 INFO Completed upload id=606745worker1 # [ 126.002797] nix-grpc-daemon[1549]: time=2026-09-21T13:23:09.162Z level=INFO msg="Upload complete. (273ms)"6746worker1 # [ 126.005061] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44408 duration_s=0 paths=1 nar_bytes_in=7526747lb # [ 125.999975] niks3-server[983]: 2026/09/21 13:23:08 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo6748worker1 # [ 126.008383] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444086749worker1 # [ 126.016637] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6750worker1 # [ 126.019571] nix-daemon[842]: reaped child process 2004, status = succeeded6751worker1 # [ 126.026056] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6752worker1 # [ 127.468620] nix-daemon[842]: reaped child process 2018, status = succeeded6753worker1 # [ 127.548231] nix-daemon[842]: accepted connection from pid 2034, user nix-grpc-daemon (trusted)6754worker1 # [ 127.563520] nix-daemon[842]: reaped child process 2037, status = succeeded6755lb # [ 127.563770] niks3-server[983]: 2026/09/21 13:23:09 INFO Received uploads request method=POST path=/api/pending_closures6756worker1 # [ 127.578476] nix-grpc-daemon[1549]: time=2026-09-21T13:23:10.737Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6757worker1 # [ 127.580070] nix-grpc-daemon[1549]: time=2026-09-21T13:23:10.739Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"6758lb # [ 127.599410] niks3-server[983]: 2026/09/21 13:23:09 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst6759lb # [ 127.614640] niks3-server[983]: 2026/09/21 13:23:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6760lb # [ 127.622650] niks3-server[983]: 2026/09/21 13:23:09 INFO Signed narinfos id=61 count=16761worker1 # [ 127.631211] nix-grpc-daemon[1549]: time=2026-09-21T13:23:10.790Z level=INFO msg="Uploading 1 narinfos"6762lb # [ 127.625525] niks3-server[983]: 2026/09/21 13:23:09 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls6763lb # [ 127.632271] niks3-server[983]: 2026/09/21 13:23:09 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv6764lb # [ 127.640947] niks3-server[983]: 2026/09/21 13:23:09 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6765lb # [ 127.644116] niks3-server[983]: 2026/09/21 13:23:09 INFO Completed upload id=616766lb # [ 127.645986] niks3-server[983]: 2026/09/21 13:23:09 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo6767worker1 # [ 127.651588] nix-grpc-daemon[1549]: time=2026-09-21T13:23:10.810Z level=INFO msg="Upload complete. (216ms)"6768worker1 # [ 127.661504] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44408 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=14 outputs=16769worker1 # [ 127.668293] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:10Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=16770worker1 # [ 127.678987] nix-daemon[842]: reaped child process 2016, status = succeeded6771worker1 # [ 127.702678] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444086772worker1 # [ 127.715472] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:444086773worker1 # [ 127.728459] nix-grpc-daemon[1549]: time=2026-09-21T13:23:10.887Z level=INFO msg="All 2 paths already cached"6774worker1 # [ 127.735824] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:444086775worker1 # [ 127.746306] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6776worker1 # [ 127.842817] nix-daemon[842]: accepted connection from pid 2041, user nix-grpc-daemon (trusted)6777worker1 # [ 127.857457] nix-daemon[842]: reaped child process 2044, status = succeeded6778worker1 # [ 127.942116] nix-daemon[842]: accepted connection from pid 2046, user nix-grpc-daemon (trusted)6779worker1 # [ 127.956410] nix-daemon[842]: reaped child process 2049, status = succeeded6780lb # [ 127.957430] niks3-server[983]: 2026/09/21 13:23:10 INFO Received uploads request method=POST path=/api/pending_closures6781worker1 # [ 127.971535] nix-grpc-daemon[1549]: time=2026-09-21T13:23:11.130Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6782worker1 # [ 127.973330] nix-grpc-daemon[1549]: time=2026-09-21T13:23:11.132Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"6783lb # [ 127.986181] niks3-server[983]: 2026/09/21 13:23:10 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst6784lb # [ 127.998244] niks3-server[983]: 2026/09/21 13:23:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6785lb # [ 128.002883] niks3-server[983]: 2026/09/21 13:23:10 INFO Signed narinfos id=62 count=16786worker1 # [ 128.013708] nix-grpc-daemon[1549]: time=2026-09-21T13:23:11.172Z level=INFO msg="Uploading 1 narinfos"6787lb # [ 128.009517] niks3-server[983]: 2026/09/21 13:23:10 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls6788lb # [ 128.020353] niks3-server[983]: 2026/09/21 13:23:10 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6789worker1 # [ 128.033245] nix-grpc-daemon[1549]: time=2026-09-21T13:23:11.192Z level=INFO msg="Upload complete. (276ms)"6790lb # [ 128.027278] niks3-server[983]: 2026/09/21 13:23:10 INFO Completed upload id=626791worker1 # [ 128.035489] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44408 duration_s=0 paths=1 nar_bytes_in=7526792lb # [ 128.030917] niks3-server[983]: 2026/09/21 13:23:10 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo6793worker1 # [ 128.039328] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444086794worker1 # [ 128.048231] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6795worker1 # [ 128.049614] nix-daemon[842]: reaped child process 2039, status = succeeded6796worker1 # [ 128.055181] nix-daemon[842]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6797worker1 # [ 128.472939] nix-daemon[842]: reaped child process 2053, status = succeeded6798worker1 # [ 128.548099] nix-daemon[842]: accepted connection from pid 2069, user nix-grpc-daemon (trusted)6799worker1 # [ 128.561771] nix-daemon[842]: reaped child process 2072, status = succeeded6800lb # [ 128.561638] niks3-server[983]: 2026/09/21 13:23:10 INFO Received uploads request method=POST path=/api/pending_closures6801worker1 # [ 128.575830] nix-grpc-daemon[1549]: time=2026-09-21T13:23:11.735Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6802worker1 # [ 128.577404] nix-grpc-daemon[1549]: time=2026-09-21T13:23:11.736Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"6803lb # [ 128.600509] niks3-server[983]: 2026/09/21 13:23:10 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst6804lb # [ 128.611158] niks3-server[983]: 2026/09/21 13:23:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6805worker1 # [ 128.623803] nix-grpc-daemon[1549]: time=2026-09-21T13:23:11.782Z level=INFO msg="Uploading 1 narinfos"6806lb # [ 128.617351] niks3-server[983]: 2026/09/21 13:23:10 INFO Signed narinfos id=63 count=16807lb # [ 128.626622] niks3-server[983]: 2026/09/21 13:23:10 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv6808lb # [ 128.631811] niks3-server[983]: 2026/09/21 13:23:10 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls6809lb # [ 128.639337] niks3-server[983]: 2026/09/21 13:23:10 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6810lb # [ 128.644191] niks3-server[983]: 2026/09/21 13:23:10 INFO Completed upload id=636811worker1 # [ 128.650516] nix-grpc-daemon[1549]: time=2026-09-21T13:23:11.809Z level=INFO msg="Upload complete. (189ms)"6812lb # [ 128.647886] niks3-server[983]: 2026/09/21 13:23:11 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo6813worker1 # [ 128.656740] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44408 duration_s=0 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=15 outputs=16814worker1 # [ 128.672479] nix-daemon[842]: reaped child process 2051, status = succeeded6815client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 13.84 seconds)6816worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6817worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6818client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]'6819client # [ 128.809317] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6820client # Running as unit: gpu.service; invocation ID: 3d3cd7e34dc14124974c987f8aeb7b006821client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.06 seconds)6822lb: must succeed: curl -sf localhost:9901/clusters6823lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6824lb: must succeed: curl -sf localhost:9901/clusters6825lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6826worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6827worker1 # [ 128.922306] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:12Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:444086828client # [ 128.945554] nix[1350]: don't know how to build these paths:6829client # [ 128.946464] nix[1350]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv6830worker2 # [ 128.949815] nix-grpc-daemon[1749]: ts=2026-09-21T13:23:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:363166831worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6832worker2 # [ 128.961604] nix-daemon[865]: accepted connection from pid 1749, user nix-grpc-daemon (trusted)6833worker2 # [ 128.977954] nix-grpc-daemon[1749]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 55 ms (attempt 1/5)6834worker2 # [ 129.040621] nix-grpc-daemon[1749]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 198 ms (attempt 2/5)6835worker2 # [ 129.240871] nix-grpc-daemon[1749]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 270 ms (attempt 3/5)6836worker2 # [ 129.518919] nix-grpc-daemon[1749]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 10 ms (attempt 4/5)6837worker2 # [ 129.530378] nix-grpc-daemon[1749]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6838worker2 # [ 129.533599] nix-grpc-daemon[1749]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6839worker1 # [ 129.545257] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:12Z level=debug event=want client=6 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=06840client # [ 129.566883] nix[1350]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, waiting up to 30s for a worker6841lb: must succeed: curl -sf localhost:9901/clusters6842lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6843lb: must succeed: curl -sf localhost:9901/clusters6844lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6845worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6846worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6847client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6848lb # [ 135.563240] envoy[916]: conn peer=[2001:db8:1::1]:59062 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27051 tx=11296 ms=136726849client # [ 159.567522] nix[1350]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}6850client # [ 159.572350] nix[1350]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed6851client # [ 159.587432] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6852client # [ 159.588817] systemd[1]: gpu.service: Failed with result 'exit-code'.6853client # [ 159.590899] systemd[1]: gpu.service: Consumed 124ms CPU time over 30.782s wall clock time, 37.7M memory peak, 5.4K incoming IP traffic, 5.1K outgoing IP traffic.6854client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 30.37 seconds)6855client: must succeed: journalctl -u gpu -o cat6856client: (finished: must succeed: journalctl -u gpu -o cat, in 0.01 seconds)6857lb: must succeed: curl -sf localhost:9901/clusters6858lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6859lb: must succeed: curl -sf localhost:9901/clusters6860lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6861worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6862worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)6863(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 45.64 seconds)6864subtest: low disk drains a worker and builds go to the other6865worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6866worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.03 seconds)6867worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6868worker1 # [ 161.814782] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:44Z level=info event=unhealthy reason=min_free6869worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.06 seconds)6870client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&26871worker1 # [ 162.717199] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:444086872client # don't know how to build these paths:6873client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6874worker1 # [ 162.724098] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:444086875worker1 # [ 162.739133] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:45Z level=debug event=want client=7 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=06876worker1 # [ 162.741396] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:45Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=16877worker2 # [ 162.799250] nix-grpc-daemon[1749]: ts=2026-09-21T13:23:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:363166878worker2 # [ 162.810195] nix-grpc-daemon[1749]: ts=2026-09-21T13:23:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:363166879client # copying 1 paths to 'lb:50051'...6880client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...6881worker2 # [ 162.818265] nix-grpc-daemon[1749]: ts=2026-09-21T13:23:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:363166882worker2 # [ 162.823935] nix-daemon[865]: accepted connection from pid 1749, user nix-grpc-daemon (trusted)6883worker2 # [ 162.835205] nix-grpc-daemon[1749]: ts=2026-09-21T13:23:46Z level=info event=niks3_push_started pid=18096884worker2 # [ 162.931292] nix-grpc-daemon[1809]: time=2026-09-21T13:23:46.107Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""6885worker2 # [ 163.037203] nix-daemon[865]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)6886worker2 # [ 163.052959] nix-daemon[865]: reaped child process 1825, status = succeeded6887worker2 # [ 163.146211] nix-daemon[865]: accepted connection from pid 1827, user nix-grpc-daemon (trusted)6888worker2 # [ 163.162229] nix-daemon[865]: reaped child process 1830, status = succeeded6889lb # [ 163.146384] niks3-server[983]: 2026/09/21 13:23:45 INFO Received uploads request method=POST path=/api/pending_closures6890worker2 # [ 163.173809] nix-grpc-daemon[1809]: time=2026-09-21T13:23:46.349Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6891worker2 # [ 163.175492] nix-grpc-daemon[1809]: time=2026-09-21T13:23:46.349Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"6892lb # [ 163.172355] niks3-server[983]: 2026/09/21 13:23:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6893lb # [ 163.174421] niks3-server[983]: 2026/09/21 13:23:45 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst6894lb # [ 163.177430] niks3-server[983]: 2026/09/21 13:23:45 INFO Signed narinfos id=64 count=16895worker2 # [ 163.198627] nix-grpc-daemon[1809]: time=2026-09-21T13:23:46.375Z level=INFO msg="Uploading 1 narinfos"6896lb # [ 163.183651] niks3-server[983]: 2026/09/21 13:23:45 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6897lb # [ 163.189707] niks3-server[983]: 2026/09/21 13:23:45 INFO Completed upload id=646898worker2 # [ 163.210457] nix-grpc-daemon[1809]: time=2026-09-21T13:23:46.387Z level=INFO msg="Upload complete. (275ms)"6899lb # [ 163.191318] niks3-server[983]: 2026/09/21 13:23:45 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls6900worker2 # [ 163.212688] nix-grpc-daemon[1749]: ts=2026-09-21T13:23:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36316 duration_s=0 paths=1 nar_bytes_in=5126901worker2 # [ 163.216242] nix-grpc-daemon[1749]: ts=2026-09-21T13:23:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:363166902lb # [ 163.196573] niks3-server[983]: 2026/09/21 13:23:45 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo6903worker2 # [ 163.222739] nix-daemon[865]: accepted connection from pid 1749, user nix-grpc-daemon (trusted)6904worker2 # [ 163.229144] nix-daemon[865]: reaped child process 1807, status = succeeded6905client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...6906worker2 # [ 163.232149] nix-daemon[865]: accepted connection from pid 1749, user nix-grpc-daemon (trusted)6907client # farm-a> worker2-next: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6908client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6909client # farm-a> LOG-farm-a-drain6910worker2 # [ 164.254409] nix-daemon[865]: reaped child process 1834, status = succeeded6911worker2 # [ 164.377123] nix-daemon[865]: accepted connection from pid 1850, user nix-grpc-daemon (trusted)6912worker2 # [ 164.394288] nix-daemon[865]: reaped child process 1853, status = succeeded6913lb # [ 164.384281] niks3-server[983]: 2026/09/21 13:23:46 INFO Received uploads request method=POST path=/api/pending_closures6914worker2 # [ 164.415242] nix-grpc-daemon[1809]: time=2026-09-21T13:23:47.591Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6915worker2 # [ 164.417957] nix-grpc-daemon[1809]: time=2026-09-21T13:23:47.594Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"6916lb # [ 164.423460] niks3-server[983]: 2026/09/21 13:23:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6917lb # [ 164.427350] niks3-server[983]: 2026/09/21 13:23:46 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6918lb # [ 164.432461] niks3-server[983]: 2026/09/21 13:23:46 INFO Signed narinfos id=65 count=16919worker2 # [ 164.455445] nix-grpc-daemon[1809]: time=2026-09-21T13:23:47.632Z level=INFO msg="Uploading 1 narinfos"6920lb # [ 164.436705] niks3-server[983]: 2026/09/21 13:23:46 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6921lb # [ 164.445411] niks3-server[983]: 2026/09/21 13:23:46 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6922lb # [ 164.449873] niks3-server[983]: 2026/09/21 13:23:46 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls6923lb # [ 164.463375] niks3-server[983]: 2026/09/21 13:23:46 INFO Completed upload id=656924worker2 # [ 164.485111] nix-grpc-daemon[1809]: time=2026-09-21T13:23:47.661Z level=INFO msg="Upload complete. (246ms)"6925lb # [ 164.469112] niks3-server[983]: 2026/09/21 13:23:46 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo6926worker2 # [ 164.490332] nix-grpc-daemon[1749]: ts=2026-09-21T13:23:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36316 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=16 outputs=16927worker1 # [ 164.488673] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:47Z level=debug event=want client=7 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=06928worker2 # [ 164.504398] nix-daemon[865]: reaped child process 1832, status = succeeded6929worker1 # [ 164.491546] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:47Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=16930worker2 # [ 164.553137] nix-grpc-daemon[1749]: ts=2026-09-21T13:23:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:363166931worker2 # [ 164.564188] nix-grpc-daemon[1749]: ts=2026-09-21T13:23:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:363166932worker2 # [ 164.574753] nix-grpc-daemon[1809]: time=2026-09-21T13:23:47.751Z level=INFO msg="All 1 paths already cached"6933client # copying 1 paths to 'lb:50051'...6934client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...6935worker2 # [ 164.584281] nix-grpc-daemon[1749]: ts=2026-09-21T13:23:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:363166936worker2 # [ 164.591106] nix-daemon[865]: accepted connection from pid 1749, user nix-grpc-daemon (trusted)6937worker2 # [ 164.704771] nix-daemon[865]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6938worker2 # [ 164.720324] nix-daemon[865]: reaped child process 1860, status = succeeded6939worker2 # [ 164.850607] nix-daemon[865]: accepted connection from pid 1862, user nix-grpc-daemon (trusted)6940worker2 # [ 164.867486] nix-daemon[865]: reaped child process 1865, status = succeeded6941lb # [ 164.855279] niks3-server[983]: 2026/09/21 13:23:47 INFO Received uploads request method=POST path=/api/pending_closures6942worker2 # [ 164.881860] nix-grpc-daemon[1809]: time=2026-09-21T13:23:48.058Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6943worker2 # [ 164.884483] nix-grpc-daemon[1809]: time=2026-09-21T13:23:48.060Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"6944lb # [ 164.876184] niks3-server[983]: 2026/09/21 13:23:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6945lb # [ 164.879063] niks3-server[983]: 2026/09/21 13:23:47 INFO Signed narinfos id=66 count=16946worker2 # [ 164.902407] nix-grpc-daemon[1809]: time=2026-09-21T13:23:48.079Z level=INFO msg="Uploading 1 narinfos"6947lb # [ 164.881430] niks3-server[983]: 2026/09/21 13:23:47 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst6948lb # [ 164.887289] niks3-server[983]: 2026/09/21 13:23:47 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls6949lb # [ 164.894435] niks3-server[983]: 2026/09/21 13:23:47 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6950lb # [ 164.898985] niks3-server[983]: 2026/09/21 13:23:47 INFO Completed upload id=666951worker2 # [ 164.920170] nix-grpc-daemon[1809]: time=2026-09-21T13:23:48.097Z level=INFO msg="Upload complete. (322ms)"6952worker2 # [ 164.924085] nix-grpc-daemon[1749]: ts=2026-09-21T13:23:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36316 duration_s=0 paths=1 nar_bytes_in=6246953lb # [ 164.905081] niks3-server[983]: 2026/09/21 13:23:47 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo6954worker2 # [ 164.928667] nix-grpc-daemon[1749]: ts=2026-09-21T13:23:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:363166955worker2 # [ 164.935726] nix-daemon[865]: accepted connection from pid 1749, user nix-grpc-daemon (trusted)6956worker2 # [ 164.938456] nix-daemon[865]: reaped child process 1855, status = succeeded6957client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...6958worker2 # [ 164.946247] nix-daemon[865]: accepted connection from pid 1749, user nix-grpc-daemon (trusted)6959client # farm-b> worker2-next: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6960lb # [ 165.566490] envoy[916]: conn peer=[2001:db8:1::1]:33496 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=912 tx=516 ms=306436961client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6962client # farm-b> LOG-farm-b-drain6963worker2 # [ 165.965625] nix-daemon[865]: reaped child process 1869, status = succeeded6964worker2 # [ 166.072116] nix-daemon[865]: accepted connection from pid 1885, user nix-grpc-daemon (trusted)6965worker2 # [ 166.090169] nix-daemon[865]: reaped child process 1888, status = succeeded6966lb # [ 166.076710] niks3-server[983]: 2026/09/21 13:23:48 INFO Received uploads request method=POST path=/api/pending_closures6967worker2 # [ 166.105916] nix-grpc-daemon[1809]: time=2026-09-21T13:23:49.282Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6968worker2 # [ 166.110458] nix-grpc-daemon[1809]: time=2026-09-21T13:23:49.285Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"6969lb # [ 166.104102] niks3-server[983]: 2026/09/21 13:23:48 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6970lb # [ 166.109455] niks3-server[983]: 2026/09/21 13:23:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6971lb # [ 166.112136] niks3-server[983]: 2026/09/21 13:23:48 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst6972lb # [ 166.114743] niks3-server[983]: 2026/09/21 13:23:48 INFO Signed narinfos id=67 count=16973worker2 # [ 166.138347] nix-grpc-daemon[1809]: time=2026-09-21T13:23:49.314Z level=INFO msg="Uploading 1 narinfos"6974lb # [ 166.122570] niks3-server[983]: 2026/09/21 13:23:48 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls6975lb # [ 166.127889] niks3-server[983]: 2026/09/21 13:23:48 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6976lb # [ 166.137562] niks3-server[983]: 2026/09/21 13:23:48 INFO Completed upload id=676977worker2 # [ 166.159270] nix-grpc-daemon[1809]: time=2026-09-21T13:23:49.336Z level=INFO msg="Upload complete. (202ms)"6978lb # [ 166.142825] niks3-server[983]: 2026/09/21 13:23:48 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo6979worker2 # [ 166.164315] nix-grpc-daemon[1749]: ts=2026-09-21T13:23:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36316 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=17 outputs=16980worker2 # [ 166.175238] nix-daemon[865]: reaped child process 1867, status = succeeded6981worker1 # [ 166.159205] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:49Z level=debug event=want client=7 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=06982worker1 # [ 166.162331] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:49Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=16983worker2 # [ 166.180306] nix-grpc-daemon[1749]: ts=2026-09-21T13:23:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:363166984worker2 # [ 166.192867] nix-grpc-daemon[1749]: ts=2026-09-21T13:23:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:363166985worker2 # [ 166.207343] nix-grpc-daemon[1809]: time=2026-09-21T13:23:49.384Z level=INFO msg="All 2 paths already cached"6986client # copying 1 paths to 'lb:50051'...6987client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...6988worker2 # [ 166.214817] nix-grpc-daemon[1749]: ts=2026-09-21T13:23:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:363166989worker2 # [ 166.223337] nix-daemon[865]: accepted connection from pid 1749, user nix-grpc-daemon (trusted)6990worker2 # [ 166.359691] nix-daemon[865]: accepted connection from pid 1892, user nix-grpc-daemon (trusted)6991worker2 # [ 166.378788] nix-daemon[865]: reaped child process 1895, status = succeeded6992worker2 # [ 166.517340] nix-daemon[865]: accepted connection from pid 1897, user nix-grpc-daemon (trusted)6993worker2 # [ 166.533579] nix-daemon[865]: reaped child process 1900, status = succeeded6994lb # [ 166.520962] niks3-server[983]: 2026/09/21 13:23:48 INFO Received uploads request method=POST path=/api/pending_closures6995worker2 # [ 166.549338] nix-grpc-daemon[1809]: time=2026-09-21T13:23:49.726Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6996worker2 # [ 166.551245] nix-grpc-daemon[1809]: time=2026-09-21T13:23:49.728Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"6997lb # [ 166.544190] niks3-server[983]: 2026/09/21 13:23:48 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst6998lb # [ 166.547473] niks3-server[983]: 2026/09/21 13:23:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6999lb # [ 166.549680] niks3-server[983]: 2026/09/21 13:23:48 INFO Signed narinfos id=68 count=17000worker2 # [ 166.572945] nix-grpc-daemon[1809]: time=2026-09-21T13:23:49.749Z level=INFO msg="Uploading 1 narinfos"7001lb # [ 166.554589] niks3-server[983]: 2026/09/21 13:23:48 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls7002lb # [ 166.562912] niks3-server[983]: 2026/09/21 13:23:48 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete7003lb # [ 166.566428] niks3-server[983]: 2026/09/21 13:23:48 INFO Completed upload id=687004worker2 # [ 166.587429] nix-grpc-daemon[1809]: time=2026-09-21T13:23:49.764Z level=INFO msg="Upload complete. (354ms)"7005worker2 # [ 166.591109] nix-grpc-daemon[1749]: ts=2026-09-21T13:23:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36316 duration_s=0 paths=1 nar_bytes_in=7527006lb # [ 166.572146] niks3-server[983]: 2026/09/21 13:23:48 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo7007worker2 # [ 166.594853] nix-grpc-daemon[1749]: ts=2026-09-21T13:23:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:363167008worker2 # [ 166.601366] nix-daemon[865]: accepted connection from pid 1749, user nix-grpc-daemon (trusted)7009worker2 # [ 166.609439] nix-daemon[865]: reaped child process 1890, status = succeeded7010client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...7011worker2 # [ 166.614867] nix-daemon[865]: accepted connection from pid 1749, user nix-grpc-daemon (trusted)7012client # farm-top> worker2-next: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7013client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7014client # farm-top> LOG-farm-top-drain7015worker2 # [ 167.441277] nix-daemon[865]: reaped child process 1904, status = succeeded7016worker2 # [ 167.519302] nix-daemon[865]: accepted connection from pid 1920, user nix-grpc-daemon (trusted)7017worker2 # [ 167.538631] nix-daemon[865]: reaped child process 1923, status = succeeded7018lb # [ 167.518980] niks3-server[983]: 2026/09/21 13:23:49 INFO Received uploads request method=POST path=/api/pending_closures7019worker2 # [ 167.545337] nix-grpc-daemon[1809]: time=2026-09-21T13:23:50.722Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7020worker2 # [ 167.548151] nix-grpc-daemon[1809]: time=2026-09-21T13:23:50.724Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"7021lb # [ 167.541489] niks3-server[983]: 2026/09/21 13:23:49 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7022lb # [ 167.546536] niks3-server[983]: 2026/09/21 13:23:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign7023lb # [ 167.549200] niks3-server[983]: 2026/09/21 13:23:49 INFO Signed narinfos id=69 count=17024worker2 # [ 167.570389] nix-grpc-daemon[1809]: time=2026-09-21T13:23:50.747Z level=INFO msg="Uploading 1 narinfos"7025lb # [ 167.552549] niks3-server[983]: 2026/09/21 13:23:49 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7026lb # [ 167.555173] niks3-server[983]: 2026/09/21 13:23:49 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7027lb # [ 167.561094] niks3-server[983]: 2026/09/21 13:23:49 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete7028lb # [ 167.563983] niks3-server[983]: 2026/09/21 13:23:49 INFO Completed upload id=697029worker2 # [ 167.585222] nix-grpc-daemon[1809]: time=2026-09-21T13:23:50.762Z level=INFO msg="Upload complete. (161ms)"7030lb # [ 167.566252] niks3-server[983]: 2026/09/21 13:23:49 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7031worker2 # [ 167.588327] nix-grpc-daemon[1749]: ts=2026-09-21T13:23:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36316 duration_s=0 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=18 outputs=17032worker2 # [ 167.597356] nix-daemon[865]: reaped child process 1902, status = succeeded7033client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&2, in 5.00 seconds)7034client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath7035client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath, in 0.10 seconds)7036worker1: must succeed: rm /nix/.rw-store/fill7037worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.11 seconds)7038worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7039worker1 # [ 168.818525] nix-grpc-daemon[1472]: ts=2026-09-21T13:23:51Z level=info event=healthy reason=min_free7040worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.06 seconds)7041(finished: subtest: low disk drains a worker and builds go to the other, in 8.39 seconds)7042(finished: run the VM test script, in 169.86 seconds)7043test script finished in 170.62s7044cleanup7045kill QemuMachine (pid 50)7046client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7047client # [2026-09-21T13:23:52Z INFO virtiofsd] Client disconnected, shutting down7048client # [2026-09-21T13:23:52Z INFO virtiofsd] Client disconnected, shutting down7049client # [2026-09-21T13:23:52Z INFO virtiofsd] Client disconnected, shutting down7050kill QemuMachine (pid 49)7051lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7052lb # [2026-09-21T13:23:52Z INFO virtiofsd] Client disconnected, shutting down7053lb # [2026-09-21T13:23:52Z INFO virtiofsd] Client disconnected, shutting down7054lb # [2026-09-21T13:23:52Z INFO virtiofsd] Client disconnected, shutting down7055kill QemuMachine (pid 51)7056worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7057worker1 # [2026-09-21T13:23:53Z INFO virtiofsd] Client disconnected, shutting down7058worker1 # [2026-09-21T13:23:53Z INFO virtiofsd] Client disconnected, shutting down7059worker1 # [2026-09-21T13:23:53Z INFO virtiofsd] Client disconnected, shutting down7060kill QemuMachine (pid 52)7061worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7062worker2 # [2026-09-21T13:23:53Z INFO virtiofsd] Client disconnected, shutting down7063worker2 # [2026-09-21T13:23:53Z INFO virtiofsd] Client disconnected, shutting down7064worker2 # [2026-09-21T13:23:53Z INFO virtiofsd] Client disconnected, shutting down7065(finished: cleanup, in 1.26 seconds)7066additionally exposed symbols:7067 client, lb, worker1, worker2,7068 vlan1,7069 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh