vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #349
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12lb: QEMU running (pid 50)13worker2 # Disk image does not exist, creating the virtualisation disk image...14lb # Disk image does not exist, creating the virtualisation disk image...15worker2: QEMU running (pid 52)16lb # Formatting '/build/vm-state-lb/tmp.V64cArk8Ry', fmt=raw size=107374182417worker2 # Formatting '/build/vm-state-worker2/tmp.O8d6sPcn3C', fmt=raw size=107374182418lb # mke2fs 1.47.4 (6-Mar-2025)19client: QEMU running (pid 49)20client # Disk image does not exist, creating the virtualisation disk image...21worker2 # mke2fs 1.47.4 (6-Mar-2025)22client # Formatting '/build/vm-state-client/tmp.2rXaR1Gvwl', fmt=raw size=107374182423worker2 # Discarding device blocks: 0/262144 done24client # mke2fs 1.47.4 (6-Mar-2025)25worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes26client # Discarding device blocks: 0/262144 done27worker2 # Filesystem UUID: bfa060b1-2c4f-43ac-b50a-4b3dbb87d6b328client # Creating filesystem with 262144 4k blocks and 65536 inodes29worker1 # Disk image does not exist, creating the virtualisation disk image...30worker2 # Superblock backups stored on blocks:31client # Filesystem UUID: e5b14b04-5293-4f4a-bcb1-4708d7d630a132worker1 # Formatting '/build/vm-state-worker1/tmp.r3hF5P8RCz', fmt=raw size=107374182433worker1 # mke2fs 1.47.4 (6-Mar-2025)34worker2 # 32768, 98304, 163840, 22937635lb # Discarding device blocks: 0/262144 done36client # Superblock backups stored on blocks:37client # 32768, 98304, 163840, 22937638client # 39worker1: QEMU running (pid 51)40lb # Creating filesystem with 262144 4k blocks and 65536 inodes41worker2 # 42client # Allocating group tables: 0/8 done43worker2 # Allocating group tables: 0/8 done44(finished: start all VMs, in 0.24 seconds)45worker1 # Discarding device blocks: 0/262144 done46worker2 # Writing inode tables: 0/8 done47client: waiting for unit network-addresses-eth1.service48worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes49worker1 # Filesystem UUID: b7567b98-e43f-4583-b7a6-726f0e1174f250worker1 # Superblock backups stored on blocks:51client: waiting for the VM to finish booting52lb # Filesystem UUID: df667e1d-bd02-4131-896f-225a366bd00853client # Writing inode tables: 0/8 done54worker1 # 32768, 98304, 163840, 22937655worker2 # Creating journal (8192 blocks): done56client # Creating journal (8192 blocks): done57lb # Superblock backups stored on blocks:58worker1 # 59worker2 # Writing superblocks and filesystem accounting information: 0/8 done60client # Writing superblocks and filesystem accounting information: 0/8 done61lb # 32768, 98304, 163840, 22937662worker1 # Allocating group tables: 0/8 done63worker2 # 64client # 65lb # 66worker1 # Writing inode tables: 0/8 done67worker2 # Virtualisation disk image created.68client # Virtualisation disk image created.69lb # Allocating group tables: 0/8 done70lb # Writing inode tables: 0/8 done71lb # Creating journal (8192 blocks): done72lb # Writing superblocks and filesystem accounting information: 0/8 done73lb # 74lb # Virtualisation disk image created.75worker1 # Creating journal (8192 blocks): done76worker1 # Writing superblocks and filesystem accounting information: 0/8 done77worker1 # 78worker1 # Virtualisation disk image created.79worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)80client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)81lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)82worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)83worker2 # 84worker2 # 85worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0086worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...87worker2 # 88worker2 # 89worker2 # 90worker2 # 91client # 92client # 93worker2 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB0094client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0095lb # 96lb # 97lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0098client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...99client # 100client # 101client # 102client # 103worker2 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...104worker2 # 105worker2 # 106client # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00107lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...108lb # 109lb # 110lb # 111lb # 112lb # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00113client # Press Ctrl-B to configure iPXE (PCI 00:08.0)...114client # 115client # 116lb # Press Ctrl-B to configure iPXE (PCI 00:08.0)...117lb # 118lb # 119worker2 # Booting from ROM...120worker1 # 121worker1 # 122worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00123client # Booting from ROM...124worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...125worker1 # 126worker1 # 127worker1 # 128worker1 # 129worker1 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00130lb # Booting from ROM...131worker1 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...132worker1 # 133worker1 # 134worker1 # Booting from ROM...135worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026136client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026137worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/w9yy8mh218zr280xr8f2zl80jncfh3bi-nixos-system-worker2-test/init regInfo=/nix/store/in851v4bx0gzn0rdcg0c8y3rkwa2f2vw-closure-info/registration console=ttyS0,115200n8 console=tty0138client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/vkg050c808kjr1y99c83mq9d036s55gg-nixos-system-client-test/init regInfo=/nix/store/6rc2f453w4kgksclyjy11bljxwr953rn-closure-info/registration console=ttyS0,115200n8 console=tty0139worker2 # [ 0.000000] BIOS-provided physical RAM map:140client # [ 0.000000] BIOS-provided physical RAM map:141worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable142client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable143worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved144client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved145worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved146client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved147worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable148client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable149lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026150worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved151client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved152worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved153client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved154worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved155client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved156worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved157client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved158lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0qln5mjy767qjp5wbmxjg8jay8k05dsm-nixos-system-lb-test/init regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration console=ttyS0,115200n8 console=tty0159worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved160client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved161lb # [ 0.000000] BIOS-provided physical RAM map:162worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved163client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved164lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable165worker2 # [ 0.000000] NX (Execute Disable) protection: active166client # [ 0.000000] NX (Execute Disable) protection: active167worker2 # [ 0.000000] APIC: Static calls initialized168client # [ 0.000000] APIC: Static calls initialized169lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved170worker2 # [ 0.000000] SMBIOS 2.8 present.171client # [ 0.000000] SMBIOS 2.8 present.172lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved173worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014174client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014175lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable176worker2 # [ 0.000000] DMI: Memory slots populated: 1/1177client # [ 0.000000] DMI: Memory slots populated: 1/1178worker2 # [ 0.000000] Hypervisor detected: KVM179client # [ 0.000000] Hypervisor detected: KVM180lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved181worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000182client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000183lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved184worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00185client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00186lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved187worker2 # [ 0.000000] kvm-clock: using sched offset of 482334291 cycles188client # [ 0.000000] kvm-clock: using sched offset of 460179839 cycles189lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved190client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns191worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns192lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved193client # [ 0.000005] tsc: Detected 2400.012 MHz processor194worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor195lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved196client # [ 0.000822] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000197worker2 # [ 0.000815] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000198lb # [ 0.000000] NX (Execute Disable) protection: active199lb # [ 0.000000] APIC: Static calls initialized200client # [ 0.000849] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs201worker2 # [ 0.000842] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs202lb # [ 0.000000] SMBIOS 2.8 present.203client # [ 0.000852] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT204worker2 # [ 0.000845] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT205lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014206client # [ 0.002751] found SMP MP-table at [mem 0x000f5450-0x000f545f]207worker2 # [ 0.002743] found SMP MP-table at [mem 0x000f5450-0x000f545f]208client # [ 0.002761] Using GB pages for direct mapping209lb # [ 0.000000] DMI: Memory slots populated: 1/1210worker2 # [ 0.002754] Using GB pages for direct mapping211lb # [ 0.000000] Hypervisor detected: KVM212client # [ 0.002837] RAMDISK: [mem 0x5e366000-0x5ffcffff]213worker2 # [ 0.002854] RAMDISK: [mem 0x5e367000-0x5ffcffff]214lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000215client # [ 0.002847] ACPI: Early table checksum verification disabled216worker2 # [ 0.002864] ACPI: Early table checksum verification disabled217lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00218client # [ 0.002850] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )219worker2 # [ 0.002867] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )220lb # [ 0.000001] kvm-clock: using sched offset of 493929390 cycles221client # [ 0.002854] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)222worker2 # [ 0.002871] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)223lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns224client # [ 0.002858] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)225worker2 # [ 0.002875] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)226lb # [ 0.000005] tsc: Detected 2400.012 MHz processor227lb # [ 0.000816] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000228client # [ 0.002865] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)229worker2 # [ 0.002882] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)230client # [ 0.002867] ACPI: FACS 0x000000005FFE0000 000040231worker2 # [ 0.002884] ACPI: FACS 0x000000005FFE0000 000040232lb # [ 0.000844] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs233client # [ 0.002869] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)234lb # [ 0.000847] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT235worker2 # [ 0.002885] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)236lb # [ 0.002756] found SMP MP-table at [mem 0x000f5450-0x000f545f]237client # [ 0.002870] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)238worker2 # [ 0.002887] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)239lb # [ 0.002769] Using GB pages for direct mapping240lb # [ 0.002880] RAMDISK: [mem 0x5e367000-0x5ffcffff]241client # [ 0.002872] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)242worker2 # [ 0.002888] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)243lb # [ 0.002889] ACPI: Early table checksum verification disabled244client # [ 0.002873] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)245worker2 # [ 0.002890] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)246lb # [ 0.002892] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )247client # [ 0.002875] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]248worker2 # [ 0.002891] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]249lb # [ 0.002896] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)250client # [ 0.002876] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]251worker2 # [ 0.002892] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]252lb # [ 0.002901] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)253client # [ 0.002876] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]254worker2 # [ 0.002893] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]255client # [ 0.002877] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]256lb # [ 0.002908] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)257worker2 # [ 0.002893] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]258lb # [ 0.002910] ACPI: FACS 0x000000005FFE0000 000040259client # [ 0.002877] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]260worker2 # [ 0.002894] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]261lb # [ 0.002912] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)262client # [ 0.002878] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]263worker2 # [ 0.002894] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]264client # [ 0.002878] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]265lb # [ 0.002913] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)266worker2 # [ 0.002895] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]267client # [ 0.003111] No NUMA configuration found268worker2 # [ 0.003129] No NUMA configuration found269lb # [ 0.002915] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)270client # [ 0.003112] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]271worker2 # [ 0.003130] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]272client # [ 0.003115] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]273lb # [ 0.002916] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)274client # [ 0.005410] Zone ranges:275worker2 # [ 0.003132] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]276worker2 # [ 0.005471] Zone ranges:277client # [ 0.005411] DMA [mem 0x0000000000001000-0x0000000000ffffff]278lb # [ 0.002917] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]279worker2 # [ 0.005472] DMA [mem 0x0000000000001000-0x0000000000ffffff]280client # [ 0.005413] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]281lb # [ 0.002919] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]282client # [ 0.005414] Normal empty283worker2 # [ 0.005473] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]284client # [ 0.005415] Device empty285worker2 # [ 0.005474] Normal empty286lb # [ 0.002919] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]287worker2 # [ 0.005475] Device empty288client # [ 0.005416] Movable zone start for each node289worker2 # [ 0.005476] Movable zone start for each node290client # [ 0.005417] Early memory node ranges291lb # [ 0.002920] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]292worker2 # [ 0.005477] Early memory node ranges293client # [ 0.005417] node 0: [mem 0x0000000000001000-0x000000000009efff]294lb # [ 0.002920] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]295worker2 # [ 0.005477] node 0: [mem 0x0000000000001000-0x000000000009efff]296client # [ 0.005418] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]297lb # [ 0.002921] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]298worker2 # [ 0.005478] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]299client # [ 0.005419] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]300lb # [ 0.002921] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]301worker2 # [ 0.005479] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]302client # [ 0.005439] On node 0, zone DMA: 1 pages in unavailable ranges303lb # [ 0.003156] No NUMA configuration found304worker2 # [ 0.005500] On node 0, zone DMA: 1 pages in unavailable ranges305client # [ 0.005712] On node 0, zone DMA: 97 pages in unavailable ranges306lb # [ 0.003157] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]307worker2 # [ 0.005765] On node 0, zone DMA: 97 pages in unavailable ranges308client # [ 0.032467] On node 0, zone DMA32: 40 pages in unavailable ranges309lb # [ 0.003159] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]310worker2 # [ 0.032570] On node 0, zone DMA32: 40 pages in unavailable ranges311client # [ 0.032927] ACPI: PM-Timer IO Port: 0x608312lb # [ 0.005687] Zone ranges:313worker2 # [ 0.033057] ACPI: PM-Timer IO Port: 0x608314client # [ 0.032939] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])315lb # [ 0.005688] DMA [mem 0x0000000000001000-0x0000000000ffffff]316worker2 # [ 0.033069] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])317lb # [ 0.005690] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]318client # [ 0.032964] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23319lb # [ 0.005691] Normal empty320worker2 # [ 0.033100] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23321lb # [ 0.005692] Device empty322client # [ 0.032967] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)323lb # [ 0.005692] Movable zone start for each node324worker2 # [ 0.033103] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)325client # [ 0.032969] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)326lb # [ 0.005693] Early memory node ranges327worker2 # [ 0.033105] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)328client # [ 0.032970] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)329lb # [ 0.005694] node 0: [mem 0x0000000000001000-0x000000000009efff]330worker2 # [ 0.033106] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)331worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026332lb # [ 0.005695] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]333client # [ 0.032971] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)334worker2 # [ 0.033107] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)335client # [ 0.032972] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)336lb # [ 0.005696] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]337worker2 # [ 0.033107] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)338lb # [ 0.005716] On node 0, zone DMA: 1 pages in unavailable ranges339client # [ 0.032975] ACPI: Using ACPI (MADT) for SMP configuration information340worker2 # [ 0.033110] ACPI: Using ACPI (MADT) for SMP configuration information341client # [ 0.032976] ACPI: HPET id: 0x8086a201 base: 0xfed00000342lb # [ 0.005983] On node 0, zone DMA: 97 pages in unavailable ranges343worker2 # [ 0.033111] ACPI: HPET id: 0x8086a201 base: 0xfed00000344client # [ 0.032979] TSC deadline timer available345lb # [ 0.032750] On node 0, zone DMA32: 40 pages in unavailable ranges346worker2 # [ 0.033114] TSC deadline timer available347client # [ 0.032983] CPU topo: Max. logical packages: 1348lb # [ 0.033209] ACPI: PM-Timer IO Port: 0x608349worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/zc33zxqpja89axzlybdb1162nkw7qird-nixos-system-worker1-test/init regInfo=/nix/store/bajb9bpm36z5w6gwdpxbnwgfcpm6qpx8-closure-info/registration console=ttyS0,115200n8 console=tty0350worker2 # [ 0.033119] CPU topo: Max. logical packages: 1351client # [ 0.032984] CPU topo: Max. logical dies: 1352worker1 # [ 0.000000] BIOS-provided physical RAM map:353worker2 # [ 0.033120] CPU topo: Max. logical dies: 1354lb # [ 0.033220] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])355client # [ 0.032984] CPU topo: Max. dies per package: 1356worker2 # [ 0.033120] CPU topo: Max. dies per package: 1357client # [ 0.032988] CPU topo: Max. threads per core: 1358worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable359lb # [ 0.033248] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23360worker2 # [ 0.033123] CPU topo: Max. threads per core: 1361client # [ 0.032988] CPU topo: Num. cores per package: 1362lb # [ 0.033250] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)363worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved364worker2 # [ 0.033124] CPU topo: Num. cores per package: 1365client # [ 0.032989] CPU topo: Num. threads per package: 1366worker2 # [ 0.033124] CPU topo: Num. threads per package: 1367lb # [ 0.033252] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)368worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved369client # [ 0.032989] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs370worker2 # [ 0.033125] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs371lb # [ 0.033253] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)372worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable373client # [ 0.033008] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()374worker2 # [ 0.033143] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()375lb # [ 0.033254] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)376worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved377client # [ 0.033040] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]378lb # [ 0.033255] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)379worker2 # [ 0.033176] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]380worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved381client # [ 0.033042] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]382lb # [ 0.033257] ACPI: Using ACPI (MADT) for SMP configuration information383worker2 # [ 0.033178] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]384worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved385client # [ 0.033043] [mem 0x60000000-0xafffffff] available for PCI devices386lb # [ 0.033258] ACPI: HPET id: 0x8086a201 base: 0xfed00000387worker2 # [ 0.033179] [mem 0x60000000-0xafffffff] available for PCI devices388client # [ 0.033045] Booting paravirtualized kernel on KVM389lb # [ 0.033262] TSC deadline timer available390worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved391worker2 # [ 0.033181] Booting paravirtualized kernel on KVM392lb # [ 0.033266] CPU topo: Max. logical packages: 1393worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved394client # [ 0.033048] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns395lb # [ 0.033267] CPU topo: Max. logical dies: 1396worker2 # [ 0.033184] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns397lb # [ 0.033268] CPU topo: Max. dies per package: 1398worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved399client # [ 0.037641] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1400lb # [ 0.033271] CPU topo: Max. threads per core: 1401worker1 # [ 0.000000] NX (Execute Disable) protection: active402worker2 # [ 0.037669] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1403client # [ 0.040005] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152404lb # [ 0.033272] CPU topo: Num. cores per package: 1405worker1 # [ 0.000000] APIC: Static calls initialized406worker2 # [ 0.040064] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152407client # [ 0.040047] kvm-guest: PV spinlocks disabled, single CPU408worker1 # [ 0.000000] SMBIOS 2.8 present.409lb # [ 0.033272] CPU topo: Num. threads per package: 1410worker2 # [ 0.040110] kvm-guest: PV spinlocks disabled, single CPU411lb # [ 0.033272] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs412worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014413lb # [ 0.033291] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()414worker1 # [ 0.000000] DMI: Memory slots populated: 1/1415worker1 # [ 0.000000] Hypervisor detected: KVM416lb # [ 0.033326] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]417worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000418lb # [ 0.033328] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]419worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00420client # [ 0.040048] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/vkg050c808kjr1y99c83mq9d036s55gg-nixos-system-client-test/init regInfo=/nix/store/6rc2f453w4kgksclyjy11bljxwr953rn-closure-info/registration console=ttyS0,115200n8 console=tty0421lb # [ 0.033330] [mem 0x60000000-0xafffffff] available for PCI devices422worker2 # [ 0.040112] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/w9yy8mh218zr280xr8f2zl80jncfh3bi-nixos-system-worker2-test/init regInfo=/nix/store/in851v4bx0gzn0rdcg0c8y3rkwa2f2vw-closure-info/registration console=ttyS0,115200n8 console=tty0423worker1 # [ 0.000001] kvm-clock: using sched offset of 506194069 cycles424lb # [ 0.033331] Booting paravirtualized kernel on KVM425client # [ 0.040148] Unknown kernel command line parameters "regInfo=/nix/store/6rc2f453w4kgksclyjy11bljxwr953rn-closure-info/registration", will be passed to user space.426worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns427client # [ 0.040161] random: crng init done428worker2 # [ 0.040218] Unknown kernel command line parameters "regInfo=/nix/store/in851v4bx0gzn0rdcg0c8y3rkwa2f2vw-closure-info/registration", will be passed to user space.429lb # [ 0.033334] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns430worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor431worker2 # [ 0.040231] random: crng init done432client # [ 0.040161] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes433worker1 # [ 0.000819] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000434lb # [ 0.037810] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1435worker2 # [ 0.040231] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes436client # [ 0.042313] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)437lb # [ 0.040009] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152438worker1 # [ 0.000846] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs439worker2 # [ 0.042421] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)440lb # [ 0.040051] kvm-guest: PV spinlocks disabled, single CPU441client # [ 0.043370] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)442worker1 # [ 0.000849] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT443client # [ 0.043402] Fallback order for Node 0: 0444worker2 # [ 0.043501] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)445worker1 # [ 0.002750] found SMP MP-table at [mem 0x000f5450-0x000f545f]446worker2 # [ 0.043534] Fallback order for Node 0: 0447client # [ 0.043405] Built 1 zonelists, mobility grouping on. Total pages: 393078448worker1 # [ 0.002760] Using GB pages for direct mapping449client # [ 0.043406] Policy zone: DMA32450worker2 # [ 0.043538] Built 1 zonelists, mobility grouping on. Total pages: 393078451worker1 # [ 0.002916] RAMDISK: [mem 0x5e367000-0x5ffcffff]452worker2 # [ 0.043539] Policy zone: DMA32453client # [ 0.045905] mem auto-init: stack:all(zero), heap alloc:on, heap free:off454worker1 # [ 0.002925] ACPI: Early table checksum verification disabled455worker2 # [ 0.046107] mem auto-init: stack:all(zero), heap alloc:on, heap free:off456client # [ 0.049181] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1457lb # [ 0.040053] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0qln5mjy767qjp5wbmxjg8jay8k05dsm-nixos-system-lb-test/init regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration console=ttyS0,115200n8 console=tty0458worker1 # [ 0.002927] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )459worker2 # [ 0.049354] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1460client # [ 0.052732] allocated 3145728 bytes of page_ext461worker2 # [ 0.052906] allocated 3145728 bytes of page_ext462client # [ 0.062911] ftrace: allocating 48731 entries in 192 pages463worker1 # [ 0.002931] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)464worker2 # [ 0.063111] ftrace: allocating 48731 entries in 192 pages465client # [ 0.062913] ftrace: allocated 192 pages with 2 groups466lb # [ 0.040146] Unknown kernel command line parameters "regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration", will be passed to user space.467client # [ 0.063763] Dynamic Preempt: lazy468worker2 # [ 0.063113] ftrace: allocated 192 pages with 2 groups469worker1 # [ 0.002935] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)470lb # [ 0.040159] random: crng init done471worker2 # [ 0.063965] Dynamic Preempt: lazy472client # [ 0.063903] rcu: Preemptible hierarchical RCU implementation.473lb # [ 0.040160] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes474worker2 # [ 0.064099] rcu: Preemptible hierarchical RCU implementation.475client # [ 0.063904] rcu: RCU event tracing is enabled.476worker1 # [ 0.002942] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)477worker2 # [ 0.064100] rcu: RCU event tracing is enabled.478worker1 # [ 0.002944] ACPI: FACS 0x000000005FFE0000 000040479client # [ 0.063905] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.480lb # [ 0.042299] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)481worker2 # [ 0.064100] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.482client # [ 0.063906] Trampoline variant of Tasks RCU enabled.483lb # [ 0.043382] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)484worker1 # [ 0.002946] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)485client # [ 0.063907] Rude variant of Tasks RCU enabled.486worker2 # [ 0.064102] Trampoline variant of Tasks RCU enabled.487lb # [ 0.043413] Fallback order for Node 0: 0488worker2 # [ 0.064102] Rude variant of Tasks RCU enabled.489client # [ 0.063907] Tracing variant of Tasks RCU enabled.490worker1 # [ 0.002947] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)491worker2 # [ 0.064102] Tracing variant of Tasks RCU enabled.492lb # [ 0.043416] Built 1 zonelists, mobility grouping on. Total pages: 393078493lb # [ 0.043417] Policy zone: DMA32494client # [ 0.063908] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.495worker1 # [ 0.002949] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)496worker2 # [ 0.064103] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.497client # [ 0.063909] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1498lb # [ 0.046154] mem auto-init: stack:all(zero), heap alloc:on, heap free:off499worker2 # [ 0.064104] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1500worker1 # [ 0.002950] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)501lb # [ 0.049254] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1502client # [ 0.063966] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.503lb # [ 0.052617] allocated 3145728 bytes of page_ext504worker1 # [ 0.002952] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]505worker2 # [ 0.064158] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.506lb # [ 0.062829] ftrace: allocating 48731 entries in 192 pages507client # [ 0.063968] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.508worker1 # [ 0.002953] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]509lb # [ 0.062831] ftrace: allocated 192 pages with 2 groups510worker2 # [ 0.064160] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.511lb # [ 0.063759] Dynamic Preempt: lazy512client # [ 0.063968] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.513worker1 # [ 0.002953] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]514lb # [ 0.063938] rcu: Preemptible hierarchical RCU implementation.515worker2 # [ 0.064161] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.516client # [ 0.068351] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16517lb # [ 0.063939] rcu: RCU event tracing is enabled.518worker1 # [ 0.002954] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]519worker2 # [ 0.068598] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16520client # [ 0.068642] rcu: srcu_init: Setting srcu_struct sizes based on contention.521lb # [ 0.063939] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.522worker1 # [ 0.002954] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]523worker2 # [ 0.068880] rcu: srcu_init: Setting srcu_struct sizes based on contention.524lb # [ 0.063941] Trampoline variant of Tasks RCU enabled.525worker1 # [ 0.002955] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]526client # [ 0.068648] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns527lb # [ 0.063941] Rude variant of Tasks RCU enabled.528worker2 # [ 0.068886] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns529lb # [ 0.063942] Tracing variant of Tasks RCU enabled.530worker1 # [ 0.002955] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]531client # [ 0.068763] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)532worker1 # [ 0.003201] No NUMA configuration found533lb # [ 0.063942] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.534client # [ 0.072308] Console: colour VGA+ 80x25535worker2 # [ 0.068991] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)536worker2 # [ 0.072575] Console: colour VGA+ 80x25537client # [ 0.072311] printk: legacy console [tty0] enabled538worker1 # [ 0.003202] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]539lb # [ 0.063943] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1540worker2 # [ 0.072578] printk: legacy console [tty0] enabled541client # [ 0.101784] printk: legacy console [ttyS0] enabled542worker1 # [ 0.003204] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]543worker1 # [ 0.005799] Zone ranges:544worker2 # [ 0.102067] printk: legacy console [ttyS0] enabled545lb # [ 0.063953] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.546client # [ 0.198970] ACPI: Core revision 20250807547worker1 # [ 0.005800] DMA [mem 0x0000000000001000-0x0000000000ffffff]548worker2 # [ 0.200017] ACPI: Core revision 20250807549lb # [ 0.063954] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.550worker1 # [ 0.005802] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]551worker1 # [ 0.005803] Normal empty552client # [ 0.199887] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns553worker1 # [ 0.005804] Device empty554lb # [ 0.063955] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.555worker2 # [ 0.200915] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns556client # [ 0.201427] APIC: Switch to symmetric I/O mode setup557worker1 # [ 0.005804] Movable zone start for each node558lb # [ 0.068357] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16559worker1 # [ 0.005805] Early memory node ranges560worker2 # [ 0.202414] APIC: Switch to symmetric I/O mode setup561client # [ 0.202357] x2apic enabled562lb # [ 0.068642] rcu: srcu_init: Setting srcu_struct sizes based on contention.563worker2 # [ 0.203357] x2apic enabled564worker1 # [ 0.005805] node 0: [mem 0x0000000000001000-0x000000000009efff]565client # [ 0.203087] APIC: Switched APIC routing to: physical x2apic566worker1 # [ 0.005807] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]567lb # [ 0.068649] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns568worker2 # [ 0.204104] APIC: Switched APIC routing to: physical x2apic569worker1 # [ 0.005808] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]570lb # [ 0.068753] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)571worker1 # [ 0.005827] On node 0, zone DMA: 1 pages in unavailable ranges572client # [ 0.205074] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1573lb # [ 0.072256] Console: colour VGA+ 80x25574worker2 # [ 0.205949] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1575worker1 # [ 0.006085] On node 0, zone DMA: 97 pages in unavailable ranges576lb # [ 0.072260] printk: legacy console [tty0] enabled577lb # [ 0.101855] printk: legacy console [ttyS0] enabled578worker1 # [ 0.032861] On node 0, zone DMA32: 40 pages in unavailable ranges579client # [ 0.206062] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns580worker1 # [ 0.033379] ACPI: PM-Timer IO Port: 0x608581worker2 # [ 0.206901] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns582lb # [ 0.199337] ACPI: Core revision 20250807583worker1 # [ 0.033391] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])584client # [ 0.207698] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)585worker2 # [ 0.208497] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)586worker1 # [ 0.033419] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23587lb # [ 0.200227] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns588client # [ 0.208783] x86/cpu: User Mode Instruction Prevention (UMIP) activated589worker1 # [ 0.033421] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)590worker2 # [ 0.209583] x86/cpu: User Mode Instruction Prevention (UMIP) activated591lb # [ 0.201810] APIC: Switch to symmetric I/O mode setup592worker1 # [ 0.033423] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)593client # [ 0.209815] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127594lb # [ 0.202755] x2apic enabled595worker2 # [ 0.210645] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127596worker1 # [ 0.033424] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)597client # [ 0.211393] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0598lb # [ 0.203504] APIC: Switched APIC routing to: physical x2apic599worker2 # [ 0.211494] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0600worker1 # [ 0.033425] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)601worker1 # [ 0.033426] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)602client # [ 0.211699] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto603worker2 # [ 0.212498] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto604lb # [ 0.205272] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1605worker1 # [ 0.033428] ACPI: Using ACPI (MADT) for SMP configuration information606worker1 # [ 0.033429] ACPI: HPET id: 0x8086a201 base: 0xfed00000607client # [ 0.213695] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl608worker2 # [ 0.213495] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl609worker1 # [ 0.033434] TSC deadline timer available610lb # [ 0.206229] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns611client # [ 0.214695] Transient Scheduler Attacks: Vulnerable: No microcode612worker1 # [ 0.033438] CPU topo: Max. logical packages: 1613worker2 # [ 0.214495] Transient Scheduler Attacks: Vulnerable: No microcode614worker1 # [ 0.033439] CPU topo: Max. logical dies: 1615client # [ 0.216396] Spectre V2 : Mitigation: Enhanced / Automatic IBRS616lb # [ 0.207922] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)617worker1 # [ 0.033439] CPU topo: Max. dies per package: 1618worker2 # [ 0.215494] Spectre V2 : Mitigation: Enhanced / Automatic IBRS619worker1 # [ 0.033443] CPU topo: Max. threads per core: 1620client # [ 0.216695] Speculative Return Stack Overflow: Mitigation: Safe RET621worker2 # [ 0.216494] Speculative Return Stack Overflow: Mitigation: Safe RET622lb # [ 0.209007] x86/cpu: User Mode Instruction Prevention (UMIP) activated623worker1 # [ 0.033443] CPU topo: Num. cores per package: 1624worker1 # [ 0.033444] CPU topo: Num. threads per package: 1625lb # [ 0.210051] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127626worker1 # [ 0.033444] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs627lb # [ 0.210919] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0628worker1 # [ 0.033461] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()629worker1 # [ 0.033493] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]630client # [ 0.217694] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization631worker2 # [ 0.217494] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization632lb # [ 0.211923] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto633client # [ 0.219703] Spectre V2 : Enabling IBPB for BPF634worker1 # [ 0.033495] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]635worker2 # [ 0.219502] Spectre V2 : Enabling IBPB for BPF636worker1 # [ 0.033496] [mem 0x60000000-0xafffffff] available for PCI devices637lb # [ 0.213920] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl638client # [ 0.220695] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier639worker1 # [ 0.033497] Booting paravirtualized kernel on KVM640worker2 # [ 0.220218] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier641lb # [ 0.214919] Transient Scheduler Attacks: Vulnerable: No microcode642client # [ 0.221695] active return thunk: srso_alias_return_thunk643worker2 # [ 0.221495] active return thunk: srso_alias_return_thunk644worker1 # [ 0.033500] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns645lb # [ 0.215919] Spectre V2 : Mitigation: Enhanced / Automatic IBRS646client # [ 0.222715] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'647worker2 # [ 0.222515] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'648worker1 # [ 0.037953] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1649lb # [ 0.216919] Speculative Return Stack Overflow: Mitigation: Safe RET650client # [ 0.223694] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'651worker1 # [ 0.040117] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152652worker2 # [ 0.223494] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'653worker1 # [ 0.040159] kvm-guest: PV spinlocks disabled, single CPU654client # [ 0.224694] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'655worker2 # [ 0.224494] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'656client # [ 0.225694] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'657worker2 # [ 0.225494] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'658client # [ 0.226694] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'659worker2 # [ 0.226494] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'660lb # [ 0.217919] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization661client # [ 0.227694] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'662lb # [ 0.219927] Spectre V2 : Enabling IBPB for BPF663worker2 # [ 0.227494] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'664worker1 # [ 0.040160] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/zc33zxqpja89axzlybdb1162nkw7qird-nixos-system-worker1-test/init regInfo=/nix/store/bajb9bpm36z5w6gwdpxbnwgfcpm6qpx8-closure-info/registration console=ttyS0,115200n8 console=tty0665client # [ 0.228694] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'666worker2 # [ 0.228494] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'667lb # [ 0.220920] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier668client # [ 0.229695] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256669worker2 # [ 0.229494] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256670lb # [ 0.221920] active return thunk: srso_alias_return_thunk671worker1 # [ 0.040256] Unknown kernel command line parameters "regInfo=/nix/store/bajb9bpm36z5w6gwdpxbnwgfcpm6qpx8-closure-info/registration", will be passed to user space.672worker1 # [ 0.040268] random: crng init done673client # [ 0.231396] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64674worker2 # [ 0.230494] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64675lb # [ 0.222940] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'676worker1 # [ 0.040269] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes677client # [ 0.232409] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512678worker2 # [ 0.231494] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512679lb # [ 0.223919] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'680worker1 # [ 0.042459] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)681client # [ 0.232694] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024682worker2 # [ 0.232494] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024683lb # [ 0.224919] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'684worker1 # [ 0.043536] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)685client # [ 0.233694] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8686worker2 # [ 0.233494] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8687worker1 # [ 0.043572] Fallback order for Node 0: 0688lb # [ 0.225919] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'689worker1 # [ 0.043575] Built 1 zonelists, mobility grouping on. Total pages: 393078690worker1 # [ 0.043576] Policy zone: DMA32691lb # [ 0.226919] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'692worker2 # [ 0.234494] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.693client # [ 0.234694] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.694worker1 # [ 0.046239] mem auto-init: stack:all(zero), heap alloc:on, heap free:off695lb # [ 0.227919] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'696worker1 # [ 0.049364] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1697worker1 # [ 0.052691] allocated 3145728 bytes of page_ext698lb # [ 0.228919] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'699worker1 # [ 0.062801] ftrace: allocating 48731 entries in 192 pages700lb # [ 0.229919] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256701worker1 # [ 0.062803] ftrace: allocated 192 pages with 2 groups702worker1 # [ 0.063654] Dynamic Preempt: lazy703lb # [ 0.231677] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64704worker1 # [ 0.063779] rcu: Preemptible hierarchical RCU implementation.705worker1 # [ 0.063780] rcu: RCU event tracing is enabled.706lb # [ 0.232749] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512707worker1 # [ 0.063780] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.708lb # [ 0.233687] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024709worker1 # [ 0.063782] Trampoline variant of Tasks RCU enabled.710lb # [ 0.234685] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8711worker1 # [ 0.063783] Rude variant of Tasks RCU enabled.712worker1 # [ 0.063783] Tracing variant of Tasks RCU enabled.713lb # [ 0.235704] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.714worker1 # [ 0.063784] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.715worker1 # [ 0.063784] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1716worker1 # [ 0.063854] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.717worker1 # [ 0.063857] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.718worker1 # [ 0.063858] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.719worker1 # [ 0.068418] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16720worker1 # [ 0.068708] rcu: srcu_init: Setting srcu_struct sizes based on contention.721worker1 # [ 0.068716] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns722worker1 # [ 0.068819] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)723worker1 # [ 0.072554] Console: colour VGA+ 80x25724worker1 # [ 0.072558] printk: legacy console [tty0] enabled725worker1 # [ 0.104597] printk: legacy console [ttyS0] enabled726worker1 # [ 0.207525] ACPI: Core revision 20250807727worker1 # [ 0.208420] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns728worker1 # [ 0.210138] APIC: Switch to symmetric I/O mode setup729worker1 # [ 0.211193] x2apic enabled730worker1 # [ 0.211980] APIC: Switched APIC routing to: physical x2apic731worker1 # [ 0.213923] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1732worker1 # [ 0.215042] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns733worker1 # [ 0.216864] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)734worker1 # [ 0.218950] x86/cpu: User Mode Instruction Prevention (UMIP) activated735worker1 # [ 0.220012] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127736worker2 # [ 0.269190] Freeing SMP alternatives memory: 44K737worker1 # [ 0.220862] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0738client # [ 0.269173] Freeing SMP alternatives memory: 44K739worker2 # [ 0.269497] pid_max: default: 32768 minimum: 301740client # [ 0.269698] pid_max: default: 32768 minimum: 301741worker1 # [ 0.221865] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto742worker2 # [ 0.270593] LSM: initializing lsm=capability,landlock,yama,bpf,ima743client # [ 0.270776] LSM: initializing lsm=capability,landlock,yama,bpf,ima744worker2 # [ 0.271599] landlock: Up and running.745client # [ 0.271798] landlock: Up and running.746worker2 # [ 0.272495] Yama: becoming mindful.747worker1 # [ 0.223861] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl748client # [ 0.272418] Yama: becoming mindful.749worker2 # [ 0.273295] LSM support for eBPF active750client # [ 0.273326] LSM support for eBPF active751worker1 # [ 0.224862] Transient Scheduler Attacks: Vulnerable: No microcode752worker2 # [ 0.274047] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)753worker1 # [ 0.225861] Spectre V2 : Mitigation: Enhanced / Automatic IBRS754client # [ 0.273792] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)755worker1 # [ 0.226861] Speculative Return Stack Overflow: Mitigation: Safe RET756worker2 # [ 0.275416] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)757client # [ 0.274731] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)758worker2 # [ 0.277841] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)759client # [ 0.277593] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)760lb # [ 0.271324] Freeing SMP alternatives memory: 44K761lb # [ 0.271921] pid_max: default: 32768 minimum: 301762worker1 # [ 0.227861] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization763worker2 # [ 0.279084] Performance Events: Fam17h+ core perfctr, AMD PMU driver.764client # [ 0.278221] Performance Events: Fam17h+ core perfctr, AMD PMU driver.765worker2 # [ 0.279503] ... version: 2766worker1 # [ 0.229871] Spectre V2 : Enabling IBPB for BPF767client # [ 0.278699] ... version: 2768lb # [ 0.273013] LSM: initializing lsm=capability,landlock,yama,bpf,ima769lb # [ 0.274007] landlock: Up and running.770client # [ 0.279368] ... bit width: 48771worker2 # [ 0.280190] ... bit width: 48772lb # [ 0.274617] Yama: becoming mindful.773worker2 # [ 0.280498] ... generic counters: 6774client # [ 0.279697] ... generic counters: 6775worker1 # [ 0.230862] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier776lb # [ 0.275525] LSM support for eBPF active777worker2 # [ 0.281163] ... generic bitmap: 000000000000003f778client # [ 0.280396] ... generic bitmap: 000000000000003f779worker1 # [ 0.231863] active return thunk: srso_alias_return_thunk780worker2 # [ 0.281496] ... fixed-purpose counters: 0781client # [ 0.280709] ... fixed-purpose counters: 0782lb # [ 0.276054] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)783worker2 # [ 0.282196] ... fixed-purpose bitmap: 0000000000000000784client # [ 0.281442] ... fixed-purpose bitmap: 0000000000000000785worker1 # [ 0.232885] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'786worker2 # [ 0.282496] ... value mask: 0000ffffffffffff787lb # [ 0.276967] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)788client # [ 0.281697] ... value mask: 0000ffffffffffff789worker1 # [ 0.233861] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'790worker2 # [ 0.283338] ... max period: 00007fffffffffff791client # [ 0.282554] ... max period: 00007fffffffffff792worker1 # [ 0.234861] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'793worker2 # [ 0.284139] ... global_ctrl mask: 000000000000003f794client # [ 0.283357] ... global_ctrl mask: 000000000000003f795worker2 # [ 0.284596] signal: max sigframe size: 3376796lb # [ 0.280046] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)797client # [ 0.283834] signal: max sigframe size: 3376798worker1 # [ 0.235861] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'799worker2 # [ 0.285379] rcu: Hierarchical SRCU implementation.800client # [ 0.284591] rcu: Hierarchical SRCU implementation.801worker1 # [ 0.236861] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'802lb # [ 0.281498] Performance Events: Fam17h+ core perfctr, AMD PMU driver.803worker2 # [ 0.286123] rcu: Max phase no-delay instances is 400.804client # [ 0.285256] rcu: Max phase no-delay instances is 400.805lb # [ 0.281928] ... version: 2806worker1 # [ 0.238861] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'807lb # [ 0.282622] ... bit width: 48808lb # [ 0.282922] ... generic counters: 6809worker1 # [ 0.239861] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'810lb # [ 0.283686] ... generic bitmap: 000000000000003f811worker1 # [ 0.240862] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256812lb # [ 0.283922] ... fixed-purpose counters: 0813lb # [ 0.284658] ... fixed-purpose bitmap: 0000000000000000814worker1 # [ 0.242861] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64815worker2 # [ 0.290997] smp: Bringing up secondary CPUs ...816client # [ 0.290330] smp: Bringing up secondary CPUs ...817lb # [ 0.284921] ... value mask: 0000ffffffffffff818worker1 # [ 0.243861] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512819client # [ 0.290711] smp: Brought up 1 node, 1 CPU820worker2 # [ 0.291511] smp: Brought up 1 node, 1 CPU821lb # [ 0.285769] ... max period: 00007fffffffffff822client # [ 0.291352] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)823worker1 # [ 0.244861] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024824worker2 # [ 0.292207] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)825lb # [ 0.286573] ... global_ctrl mask: 000000000000003f826lb # [ 0.287017] signal: max sigframe size: 3376827worker1 # [ 0.245861] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8828lb # [ 0.287810] rcu: Hierarchical SRCU implementation.829client # [ 0.291899] Memory: 1454552K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109348K reserved, 0K cma-reserved)830worker2 # [ 0.292650] Memory: 1454572K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)831lb # [ 0.288504] rcu: Max phase no-delay instances is 400.832worker1 # [ 0.246861] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.833client # [ 0.292951] devtmpfs: initialized834worker2 # [ 0.293752] devtmpfs: initialized835client # [ 0.293800] x86/mm: Memory block size: 128MB836worker2 # [ 0.294645] x86/mm: Memory block size: 128MB837client # [ 0.295569] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)838worker2 # [ 0.296557] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)839client # [ 0.296591] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).840worker2 # [ 0.297521] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).841lb # [ 0.293708] smp: Bringing up secondary CPUs ...842client # [ 0.297779] pinctrl core: initialized pinctrl subsystem843worker2 # [ 0.298576] pinctrl core: initialized pinctrl subsystem844lb # [ 0.294535] smp: Brought up 1 node, 1 CPU845client # [ 0.298910] PM: RTC time: 12:45:50, date: 2026-09-20846lb # [ 0.294923] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)847worker2 # [ 0.299714] PM: RTC time: 12:45:50, date: 2026-09-20848lb # [ 0.296116] Memory: 1454584K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)849lb # [ 0.297196] devtmpfs: initialized850client # [ 0.302487] NET: Registered PF_NETLINK/PF_ROUTE protocol family851worker2 # [ 0.303542] NET: Registered PF_NETLINK/PF_ROUTE protocol family852lb # [ 0.298015] x86/mm: Memory block size: 128MB853client # [ 0.303981] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations854worker2 # [ 0.305006] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations855client # [ 0.304718] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations856lb # [ 0.299797] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)857worker2 # [ 0.305520] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations858lb # [ 0.300825] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).859client # [ 0.306178] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations860worker2 # [ 0.306771] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations861lb # [ 0.302009] pinctrl core: initialized pinctrl subsystem862client # [ 0.306708] audit: initializing netlink subsys (disabled)863worker2 # [ 0.307507] audit: initializing netlink subsys (disabled)864lb # [ 0.303086] PM: RTC time: 12:45:50, date: 2026-09-20865client # [ 0.307819] thermal_sys: Registered thermal governor 'fair_share'866worker2 # [ 0.308648] thermal_sys: Registered thermal governor 'fair_share'867client # [ 0.307821] thermal_sys: Registered thermal governor 'bang_bang'868worker2 # [ 0.308650] thermal_sys: Registered thermal governor 'bang_bang'869client # [ 0.308697] thermal_sys: Registered thermal governor 'step_wise'870worker2 # [ 0.309497] thermal_sys: Registered thermal governor 'step_wise'871client # [ 0.309603] thermal_sys: Registered thermal governor 'user_space'872worker2 # [ 0.310450] thermal_sys: Registered thermal governor 'user_space'873client # [ 0.310389] audit: type=2000 audit(1789908351.413:1): state=initialized audit_enabled=0 res=1874lb # [ 0.307414] NET: Registered PF_NETLINK/PF_ROUTE protocol family875worker2 # [ 0.311273] audit: type=2000 audit(1789908351.412:1): state=initialized audit_enabled=0 res=1876client # [ 0.311700] thermal_sys: Registered thermal governor 'power_allocator'877client # [ 0.311723] cpuidle: using governor menu878worker2 # [ 0.312499] thermal_sys: Registered thermal governor 'power_allocator'879lb # [ 0.308558] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations880worker2 # [ 0.312514] cpuidle: using governor menu881client # [ 0.313903] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5882lb # [ 0.308951] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations883worker2 # [ 0.314738] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5884client # [ 0.314926] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]885lb # [ 0.310212] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations886worker2 # [ 0.315743] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]887client # [ 0.315700] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry888lb # [ 0.310933] audit: initializing netlink subsys (disabled)889client # [ 0.316709] PCI: Using configuration type 1 for base access890worker2 # [ 0.316502] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry891lb # [ 0.312120] thermal_sys: Registered thermal governor 'fair_share'892worker2 # [ 0.317512] PCI: Using configuration type 1 for base access893client # [ 0.317802] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.894lb # [ 0.312123] thermal_sys: Registered thermal governor 'bang_bang'895worker2 # [ 0.318637] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.896lb # [ 0.312922] thermal_sys: Registered thermal governor 'step_wise'897lb # [ 0.313873] thermal_sys: Registered thermal governor 'user_space'898lb # [ 0.314690] audit: type=2000 audit(1789908351.424:1): state=initialized audit_enabled=0 res=1899lb # [ 0.315924] thermal_sys: Registered thermal governor 'power_allocator'900lb # [ 0.315938] cpuidle: using governor menu901worker2 # [ 0.323773] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages902lb # [ 0.318190] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5903client # [ 0.324977] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages904worker2 # [ 0.324498] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page905client # [ 0.325698] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page906lb # [ 0.319148] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]907lb # [ 0.319924] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry908lb # [ 0.320933] PCI: Using configuration type 1 for base access909worker1 # [ 0.288032] Freeing SMP alternatives memory: 44K910worker1 # [ 0.288866] pid_max: default: 32768 minimum: 301911lb # [ 0.322057] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.912worker2 # [ 0.329509] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages913worker1 # [ 0.289979] LSM: initializing lsm=capability,landlock,yama,bpf,ima914client # [ 0.330698] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages915worker2 # [ 0.330496] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page916worker1 # [ 0.291007] landlock: Up and running.917client # [ 0.331698] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page918worker1 # [ 0.291862] Yama: becoming mindful.919worker1 # [ 0.293112] LSM support for eBPF active920worker1 # [ 0.293982] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)921lb # [ 0.327214] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages922lb # [ 0.327923] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page923worker1 # [ 0.294908] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)924worker1 # [ 0.298257] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)925client # [ 0.340304] ACPI: Added _OSI(Module Device)926client # [ 0.340699] ACPI: Added _OSI(Processor Device)927worker1 # [ 0.299485] Performance Events: Fam17h+ core perfctr, AMD PMU driver.928worker2 # [ 0.340842] ACPI: Added _OSI(Module Device)929lb # [ 0.332923] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages930client # [ 0.341422] ACPI: Added _OSI(Processor Aggregator Device)931worker2 # [ 0.341498] ACPI: Added _OSI(Processor Device)932worker1 # [ 0.299871] ... version: 2933lb # [ 0.333921] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page934worker1 # [ 0.300675] ... bit width: 48935worker2 # [ 0.342184] ACPI: Added _OSI(Processor Aggregator Device)936worker1 # [ 0.301435] ... generic counters: 6937worker1 # [ 0.301863] ... generic bitmap: 000000000000003f938worker1 # [ 0.302824] ... fixed-purpose counters: 0939worker1 # [ 0.303463] ... fixed-purpose bitmap: 0000000000000000940worker1 # [ 0.303863] ... value mask: 0000ffffffffffff941worker1 # [ 0.304843] ... max period: 00007fffffffffff942worker1 # [ 0.305631] ... global_ctrl mask: 000000000000003f943client # [ 0.348252] ACPI: 1 ACPI AML tables successfully acquired and loaded944worker1 # [ 0.305985] signal: max sigframe size: 3376945worker1 # [ 0.306877] rcu: Hierarchical SRCU implementation.946worker1 # [ 0.307708] rcu: Max phase no-delay instances is 400.947worker2 # [ 0.350733] ACPI: 1 ACPI AML tables successfully acquired and loaded948lb # [ 0.344312] ACPI: Added _OSI(Module Device)949client # [ 0.351618] ACPI: Interpreter enabled950lb # [ 0.344923] ACPI: Added _OSI(Processor Device)951client # [ 0.352194] ACPI: PM: (supports S0 S3 S4 S5)952lb # [ 0.345745] ACPI: Added _OSI(Processor Aggregator Device)953client # [ 0.352698] ACPI: Using IOAPIC for interrupt routing954worker2 # [ 0.354917] ACPI: Interpreter enabled955client # [ 0.353551] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug956worker2 # [ 0.355512] ACPI: PM: (supports S0 S3 S4 S5)957worker1 # [ 0.313123] smp: Bringing up secondary CPUs ...958worker2 # [ 0.356241] ACPI: Using IOAPIC for interrupt routing959worker1 # [ 0.313878] smp: Brought up 1 node, 1 CPU960worker2 # [ 0.356551] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug961worker1 # [ 0.314622] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)962worker2 # [ 0.357497] PCI: Using E820 reservations for host bridge windows963client # [ 0.358698] PCI: Using E820 reservations for host bridge windows964worker1 # [ 0.315078] Memory: 1454560K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)965client # [ 0.359804] ACPI: Enabled 2 GPEs in block 00 to 3F966worker1 # [ 0.316134] devtmpfs: initialized967worker2 # [ 0.360607] ACPI: Enabled 2 GPEs in block 00 to 3F968worker1 # [ 0.317019] x86/mm: Memory block size: 128MB969lb # [ 0.355470] ACPI: 1 ACPI AML tables successfully acquired and loaded970worker1 # [ 0.318878] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)971worker1 # [ 0.319893] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).972lb # [ 0.359348] ACPI: Interpreter enabled973worker1 # [ 0.320933] pinctrl core: initialized pinctrl subsystem974lb # [ 0.359938] ACPI: PM: (supports S0 S3 S4 S5)975lb # [ 0.360615] ACPI: Using IOAPIC for interrupt routing976worker1 # [ 0.322198] PM: RTC time: 12:45:50, date: 2026-09-20977client # [ 0.368405] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])978lb # [ 0.360972] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug979worker2 # [ 0.369267] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])980lb # [ 0.361923] PCI: Using E820 reservations for host bridge windows981client # [ 0.369362] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]982worker2 # [ 0.370261] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]983worker1 # [ 0.325629] NET: Registered PF_NETLINK/PF_ROUTE protocol family984client # [ 0.370775] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]985lb # [ 0.365044] ACPI: Enabled 2 GPEs in block 00 to 3F986worker1 # [ 0.327172] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations987worker2 # [ 0.372529] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]988client # [ 0.371820] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]989client # [ 0.373056] PCI host bridge to bus 0000:00990worker1 # [ 0.327886] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations991worker2 # [ 0.373619] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]992client # [ 0.373701] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]993worker2 # [ 0.374876] PCI host bridge to bus 0000:00994worker1 # [ 0.329151] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations995client # [ 0.374697] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]996worker2 # [ 0.375501] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]997worker1 # [ 0.329874] audit: initializing netlink subsys (disabled)998worker2 # [ 0.376497] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]999client # [ 0.375698] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1000worker1 # [ 0.331126] thermal_sys: Registered thermal governor 'fair_share'1001worker2 # [ 0.377497] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1002client # [ 0.376697] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1003worker1 # [ 0.331128] thermal_sys: Registered thermal governor 'bang_bang'1004worker2 # [ 0.378497] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1005client # [ 0.377697] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1006worker1 # [ 0.331865] thermal_sys: Registered thermal governor 'step_wise'1007worker2 # [ 0.379496] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1008worker1 # [ 0.332837] thermal_sys: Registered thermal governor 'user_space'1009client # [ 0.378697] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1010lb # [ 0.373726] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1011client # [ 0.379697] pci_bus 0000:00: root bus resource [bus 00-ff]1012worker2 # [ 0.380501] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1013worker1 # [ 0.333613] audit: type=2000 audit(1789908351.479:1): state=initialized audit_enabled=0 res=11014lb # [ 0.374654] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1015worker2 # [ 0.381525] pci_bus 0000:00: root bus resource [bus 00-ff]1016client # [ 0.380644] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1017worker1 # [ 0.334866] thermal_sys: Registered thermal governor 'power_allocator'1018worker1 # [ 0.334880] cpuidle: using governor menu1019lb # [ 0.376162] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1020worker2 # [ 0.382499] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1021client # [ 0.382128] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1022lb # [ 0.377044] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1023worker1 # [ 0.337130] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.51024worker2 # [ 0.383931] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1025lb # [ 0.378662] PCI host bridge to bus 0000:001026worker1 # [ 0.338113] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]1027lb # [ 0.378926] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1028worker1 # [ 0.338867] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1029lb # [ 0.379922] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1030worker1 # [ 0.339876] PCI: Using configuration type 1 for base access1031lb # [ 0.380921] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1032lb # [ 0.381921] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1033worker1 # [ 0.341018] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1034lb # [ 0.382923] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1035client # [ 0.384762] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1036lb # [ 0.383922] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1037client # [ 0.385720] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1038lb # [ 0.384922] pci_bus 0000:00: root bus resource [bus 00-ff]1039worker2 # [ 0.386563] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1040client # [ 0.386840] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1041worker2 # [ 0.387509] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1042lb # [ 0.385849] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1043worker2 # [ 0.388507] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1044worker1 # [ 0.345988] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1045client # [ 0.388063] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1046lb # [ 0.387357] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1047worker1 # [ 0.346865] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1048worker2 # [ 0.389745] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1049client # [ 0.390392] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1050worker2 # [ 0.391230] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1051client # [ 0.392705] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1052worker1 # [ 0.351865] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1053client # [ 0.393578] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1054worker2 # [ 0.394529] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1055worker1 # [ 0.352863] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1056client # [ 0.394431] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1057worker2 # [ 0.395419] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1058client # [ 0.394703] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1059lb # [ 0.390052] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1060worker2 # [ 0.396290] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1061lb # [ 0.390935] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1062worker2 # [ 0.397405] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1063client # [ 0.396257] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1064lb # [ 0.391879] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1065worker2 # [ 0.399177] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1066lb # [ 0.392984] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1067client # [ 0.398705] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1068client # [ 0.399582] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1069lb # [ 0.394655] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1070worker2 # [ 0.401532] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1071client # [ 0.400477] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1072worker2 # [ 0.402414] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1073worker2 # [ 0.403295] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1074client # [ 0.402075] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1075worker1 # [ 0.363252] ACPI: Added _OSI(Module Device)1076lb # [ 0.397956] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1077worker1 # [ 0.363866] ACPI: Added _OSI(Processor Device)1078lb # [ 0.398928] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1079worker2 # [ 0.405018] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1080worker1 # [ 0.364740] ACPI: Added _OSI(Processor Aggregator Device)1081lb # [ 0.399927] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1082lb # [ 0.400927] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1083client # [ 0.405536] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1084client # [ 0.406332] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1085lb # [ 0.402499] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1086worker2 # [ 0.407504] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1087client # [ 0.406719] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1088worker2 # [ 0.408504] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1089worker2 # [ 0.409486] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1090client # [ 0.408239] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1091lb # [ 0.405902] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1092lb # [ 0.406533] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1093worker2 # [ 0.410903] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1094client # [ 0.409705] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1095lb # [ 0.406943] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1096worker1 # [ 0.374094] ACPI: 1 ACPI AML tables successfully acquired and loaded1097client # [ 0.410580] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1098worker2 # [ 0.412518] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1099client # [ 0.411430] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1100lb # [ 0.408489] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1101worker2 # [ 0.413364] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1102worker2 # [ 0.414297] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1103client # [ 0.412240] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1104worker1 # [ 0.378271] ACPI: Interpreter enabled1105worker1 # [ 0.378881] ACPI: PM: (supports S0 S3 S4 S5)1106lb # [ 0.411173] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1107worker2 # [ 0.416513] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1108worker1 # [ 0.379629] ACPI: Using IOAPIC for interrupt routing1109client # [ 0.414720] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1110lb # [ 0.412561] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1111client # [ 0.415553] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1112lb # [ 0.412966] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1113client # [ 0.416410] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1114worker2 # [ 0.418504] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1115worker2 # [ 0.419359] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1116worker1 # [ 0.381933] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1117lb # [ 0.414463] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1118client # [ 0.417329] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1119worker2 # [ 0.420218] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1120worker1 # [ 0.382865] PCI: Using E820 reservations for host bridge windows1121worker2 # [ 0.421099] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1122lb # [ 0.416788] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1123lb # [ 0.417545] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1124worker1 # [ 0.385952] ACPI: Enabled 2 GPEs in block 00 to 3F1125client # [ 0.419685] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1126lb # [ 0.417944] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1127client # [ 0.420704] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1128client # [ 0.421645] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1129worker2 # [ 0.423478] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1130worker2 # [ 0.424505] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1131lb # [ 0.420035] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1132client # [ 0.423055] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1133worker2 # [ 0.425453] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1134lb # [ 0.422834] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1135worker2 # [ 0.426996] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1136client # [ 0.424713] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1137lb # [ 0.423562] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1138client # [ 0.425536] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1139worker1 # [ 0.393498] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1140lb # [ 0.423943] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1141client # [ 0.426406] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1142worker2 # [ 0.428504] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1143client # [ 0.426703] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1144worker1 # [ 0.393870] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1145worker2 # [ 0.429401] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1146lb # [ 0.425490] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1147worker1 # [ 0.394955] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1148worker2 # [ 0.430223] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1149worker2 # [ 0.430507] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1150client # [ 0.428178] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1151worker1 # [ 0.395988] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1152worker1 # [ 0.397222] PCI host bridge to bus 0000:001153client # [ 0.429717] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1154lb # [ 0.428174] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1155worker1 # [ 0.397868] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1156worker2 # [ 0.432770] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1157client # [ 0.430635] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1158lb # [ 0.429533] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1159worker1 # [ 0.398864] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1160lb # [ 0.429944] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1161worker2 # [ 0.434509] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1162client # [ 0.432034] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1163worker1 # [ 0.399864] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1164worker2 # [ 0.435456] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1165lb # [ 0.431533] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1166worker1 # [ 0.400864] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1167client # [ 0.433704] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1168worker2 # [ 0.436853] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1169worker1 # [ 0.401864] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1170client # [ 0.434543] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1171lb # [ 0.433629] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1172worker1 # [ 0.402863] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1173client # [ 0.435429] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1174lb # [ 0.433929] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1175worker1 # [ 0.403868] pci_bus 0000:00: root bus resource [bus 00-ff]1176worker2 # [ 0.438524] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1177lb # [ 0.434869] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1178worker2 # [ 0.439334] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1179worker1 # [ 0.404887] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1180client # [ 0.437017] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1181lb # [ 0.435737] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1182worker2 # [ 0.440538] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1183worker1 # [ 0.406363] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1184lb # [ 0.437477] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1185client # [ 0.438704] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1186worker2 # [ 0.442072] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1187client # [ 0.439554] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1188client # [ 0.440454] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1189lb # [ 0.438955] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1190worker2 # [ 0.443505] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1191lb # [ 0.439909] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1192worker2 # [ 0.444351] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1193client # [ 0.441274] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1194worker2 # [ 0.445218] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1195lb # [ 0.441287] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1196client # [ 0.442435] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1197worker2 # [ 0.446087] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1198worker1 # [ 0.408930] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1199lb # [ 0.442929] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1200client # [ 0.442895] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1201worker1 # [ 0.409878] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1202worker2 # [ 0.447164] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1203worker1 # [ 0.410885] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1204lb # [ 0.443756] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1205client # [ 0.445373] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1206lb # [ 0.444944] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1207worker2 # [ 0.448310] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1208worker1 # [ 0.412087] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1209worker2 # [ 0.450180] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1210lb # [ 0.446493] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1211worker1 # [ 0.413585] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1212lb # [ 0.448570] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1213lb # [ 0.448929] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1214worker1 # [ 0.416893] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1215lb # [ 0.449876] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1216worker1 # [ 0.417828] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1217worker1 # [ 0.418668] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1218lb # [ 0.451315] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1219worker1 # [ 0.419784] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1220lb # [ 0.453413] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1221worker1 # [ 0.421685] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1222client # [ 0.445906] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1223lb # [ 0.454174] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1224client # [ 0.448341] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1225lb # [ 0.455674] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1226worker2 # [ 0.450715] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1227worker1 # [ 0.423912] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1228client # [ 0.449016] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1229worker1 # [ 0.424802] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1230worker2 # [ 0.453175] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1231client # [ 0.450326] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1232worker1 # [ 0.425681] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1233worker2 # [ 0.453755] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1234client # [ 0.450915] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1235worker1 # [ 0.427403] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1236worker2 # [ 0.455112] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1237client # [ 0.451983] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1238worker2 # [ 0.456552] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1239client # [ 0.452965] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1240worker1 # [ 0.429892] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1241worker2 # [ 0.457792] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1242worker1 # [ 0.430871] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1243client # [ 0.454726] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1244worker2 # [ 0.458768] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1245worker1 # [ 0.431890] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1246client # [ 0.455559] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1247client # [ 0.456774] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1248lb # [ 0.456124] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1249worker2 # [ 0.460540] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1250worker1 # [ 0.433468] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1251worker2 # [ 0.461358] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1252lb # [ 0.458662] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1253client # [ 0.458290] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1254worker1 # [ 0.434892] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1255lb # [ 0.459158] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1256worker2 # [ 0.462585] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1257worker1 # [ 0.435829] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1258worker2 # [ 0.464129] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1259lb # [ 0.460922] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1260client # [ 0.461664] ACPI: PCI: Interrupt link LNKA configured for IRQ 101261worker1 # [ 0.436674] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1262lb # [ 0.462105] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1263client # [ 0.462529] ACPI: PCI: Interrupt link LNKB configured for IRQ 101264worker1 # [ 0.438423] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1265client # [ 0.463517] ACPI: PCI: Interrupt link LNKC configured for IRQ 111266lb # [ 0.463220] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1267worker2 # [ 0.467546] ACPI: PCI: Interrupt link LNKA configured for IRQ 101268client # [ 0.464510] ACPI: PCI: Interrupt link LNKD configured for IRQ 111269lb # [ 0.464195] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1270worker2 # [ 0.468611] ACPI: PCI: Interrupt link LNKB configured for IRQ 101271client # [ 0.465510] ACPI: PCI: Interrupt link LNKE configured for IRQ 101272worker1 # [ 0.439892] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1273worker2 # [ 0.469596] ACPI: PCI: Interrupt link LNKC configured for IRQ 111274client # [ 0.466500] ACPI: PCI: Interrupt link LNKF configured for IRQ 101275worker1 # [ 0.440801] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1276lb # [ 0.465950] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1277worker2 # [ 0.470591] ACPI: PCI: Interrupt link LNKD configured for IRQ 111278client # [ 0.467515] ACPI: PCI: Interrupt link LNKG configured for IRQ 111279worker1 # [ 0.441739] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1280lb # [ 0.466761] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1281worker2 # [ 0.471603] ACPI: PCI: Interrupt link LNKE configured for IRQ 101282client # [ 0.468764] ACPI: PCI: Interrupt link LNKH configured for IRQ 111283worker2 # [ 0.472610] ACPI: PCI: Interrupt link LNKF configured for IRQ 101284client # [ 0.469737] ACPI: PCI: Interrupt link GSIA configured for IRQ 161285worker1 # [ 0.443394] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1286lb # [ 0.467943] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1287client # [ 0.470675] ACPI: PCI: Interrupt link GSIB configured for IRQ 171288worker2 # [ 0.473597] ACPI: PCI: Interrupt link LNKG configured for IRQ 111289client # [ 0.471425] ACPI: PCI: Interrupt link GSIC configured for IRQ 181290lb # [ 0.469715] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1291worker2 # [ 0.474596] ACPI: PCI: Interrupt link LNKH configured for IRQ 111292client # [ 0.471709] ACPI: PCI: Interrupt link GSID configured for IRQ 191293worker2 # [ 0.475533] ACPI: PCI: Interrupt link GSIA configured for IRQ 161294client # [ 0.472662] ACPI: PCI: Interrupt link GSIE configured for IRQ 201295worker1 # [ 0.445898] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1296worker2 # [ 0.476512] ACPI: PCI: Interrupt link GSIB configured for IRQ 171297client # [ 0.473427] ACPI: PCI: Interrupt link GSIF configured for IRQ 211298worker1 # [ 0.446846] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1299worker2 # [ 0.477460] ACPI: PCI: Interrupt link GSIC configured for IRQ 181300client # [ 0.473713] ACPI: PCI: Interrupt link GSIG configured for IRQ 221301lb # [ 0.473533] ACPI: PCI: Interrupt link LNKA configured for IRQ 101302worker2 # [ 0.478240] ACPI: PCI: Interrupt link GSID configured for IRQ 191303worker1 # [ 0.447766] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1304client # [ 0.474658] ACPI: PCI: Interrupt link GSIH configured for IRQ 231305worker2 # [ 0.478511] ACPI: PCI: Interrupt link GSIE configured for IRQ 201306lb # [ 0.474030] ACPI: PCI: Interrupt link LNKB configured for IRQ 101307worker2 # [ 0.479484] ACPI: PCI: Interrupt link GSIF configured for IRQ 211308lb # [ 0.475030] ACPI: PCI: Interrupt link LNKC configured for IRQ 111309client # [ 0.476515] iommu: Default domain type: Translated1310worker1 # [ 0.449709] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1311worker2 # [ 0.480307] ACPI: PCI: Interrupt link GSIG configured for IRQ 221312lb # [ 0.476025] ACPI: PCI: Interrupt link LNKD configured for IRQ 111313client # [ 0.477301] iommu: DMA domain TLB invalidation policy: lazy mode1314worker2 # [ 0.481397] ACPI: PCI: Interrupt link GSIH configured for IRQ 231315lb # [ 0.477023] ACPI: PCI: Interrupt link LNKE configured for IRQ 101316client # [ 0.477975] ACPI: bus type USB registered1317client # [ 0.478733] usbcore: registered new interface driver usbfs1318lb # [ 0.478021] ACPI: PCI: Interrupt link LNKF configured for IRQ 101319worker1 # [ 0.451894] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1320worker2 # [ 0.483411] iommu: Default domain type: Translated1321client # [ 0.479647] usbcore: registered new interface driver hub1322lb # [ 0.479024] ACPI: PCI: Interrupt link LNKG configured for IRQ 111323client # [ 0.480334] usbcore: registered new device driver usb1324worker2 # [ 0.484226] iommu: DMA domain TLB invalidation policy: lazy mode1325worker1 # [ 0.452811] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1326lb # [ 0.480048] ACPI: PCI: Interrupt link LNKH configured for IRQ 111327worker2 # [ 0.484893] ACPI: bus type USB registered1328worker1 # [ 0.453889] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1329client # [ 0.481625] NetLabel: Initializing1330lb # [ 0.480958] ACPI: PCI: Interrupt link GSIA configured for IRQ 161331worker2 # [ 0.485555] usbcore: registered new interface driver usbfs1332client # [ 0.482104] NetLabel: domain hash size = 1281333lb # [ 0.481929] ACPI: PCI: Interrupt link GSIB configured for IRQ 171334worker1 # [ 0.454871] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1335worker2 # [ 0.486445] usbcore: registered new interface driver hub1336client # [ 0.482697] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1337lb # [ 0.482838] ACPI: PCI: Interrupt link GSIC configured for IRQ 181338worker2 # [ 0.487139] usbcore: registered new device driver usb1339client # [ 0.483612] NetLabel: unlabeled traffic allowed by default1340lb # [ 0.483648] ACPI: PCI: Interrupt link GSID configured for IRQ 191341worker1 # [ 0.456471] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1342client # [ 0.484356] PCI: Using ACPI for IRQ routing1343lb # [ 0.483936] ACPI: PCI: Interrupt link GSIE configured for IRQ 201344worker2 # [ 0.488421] NetLabel: Initializing1345lb # [ 0.484850] ACPI: PCI: Interrupt link GSIF configured for IRQ 211346worker2 # [ 0.488943] NetLabel: domain hash size = 1281347worker1 # [ 0.457877] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1348worker2 # [ 0.489497] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1349lb # [ 0.485623] ACPI: PCI: Interrupt link GSIG configured for IRQ 221350worker1 # [ 0.458887] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1351lb # [ 0.485945] ACPI: PCI: Interrupt link GSIH configured for IRQ 231352worker2 # [ 0.490532] NetLabel: unlabeled traffic allowed by default1353worker2 # [ 0.491514] PCI: Using ACPI for IRQ routing1354worker1 # [ 0.460428] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1355lb # [ 0.488008] iommu: Default domain type: Translated1356lb # [ 0.488809] iommu: DMA domain TLB invalidation policy: lazy mode1357lb # [ 0.489935] ACPI: bus type USB registered1358worker1 # [ 0.462649] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1359lb # [ 0.490659] usbcore: registered new interface driver usbfs1360lb # [ 0.490941] usbcore: registered new interface driver hub1361worker1 # [ 0.463614] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1362lb # [ 0.491809] usbcore: registered new device driver usb1363worker1 # [ 0.463886] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1364lb # [ 0.493708] NetLabel: Initializing1365worker1 # [ 0.465512] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1366lb # [ 0.493928] NetLabel: domain hash size = 1281367lb # [ 0.494732] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1368lb # [ 0.495730] NetLabel: unlabeled traffic allowed by default1369lb # [ 0.496606] PCI: Using ACPI for IRQ routing1370worker1 # [ 0.466901] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1371worker1 # [ 0.467863] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1372worker1 # [ 0.468887] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1373worker1 # [ 0.470471] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1374worker1 # [ 0.471675] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1375worker1 # [ 0.472794] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1376worker1 # [ 0.474544] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1377worker1 # [ 0.475062] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1378worker1 # [ 0.478487] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1379client # [ 0.528975] pci 0000:00:01.0: vgaarb: setting as boot VGA device1380worker1 # [ 0.479148] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1381client # [ 0.529693] pci 0000:00:01.0: vgaarb: bridge control possible1382client # [ 0.529693] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1383worker1 # [ 0.480900] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1384client # [ 0.529702] vgaarb: loaded1385client # [ 0.530425] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01386worker2 # [ 0.535620] pci 0000:00:01.0: vgaarb: setting as boot VGA device1387worker1 # [ 0.482142] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1388client # [ 0.530699] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1389worker2 # [ 0.536493] pci 0000:00:01.0: vgaarb: bridge control possible1390worker1 # [ 0.483217] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1391worker2 # [ 0.536493] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1392worker2 # [ 0.536504] vgaarb: loaded1393worker1 # [ 0.484158] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1394worker2 # [ 0.537305] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01395worker2 # [ 0.538115] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1396client # [ 0.534807] clocksource: Switched to clocksource kvm-clock1397worker1 # [ 0.485922] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1398worker1 # [ 0.486854] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1399worker2 # [ 0.541625] clocksource: Switched to clocksource kvm-clock1400client # [ 0.538372] VFS: Disk quotas dquot_6.6.01401worker1 # [ 0.488046] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1402client # [ 0.539114] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1403worker2 # [ 0.543380] VFS: Disk quotas dquot_6.6.01404lb # [ 0.541036] pci 0000:00:01.0: vgaarb: setting as boot VGA device1405client # [ 0.540477] pnp: PnP ACPI init1406worker1 # [ 0.489923] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1407lb # [ 0.541918] pci 0000:00:01.0: vgaarb: bridge control possible1408client # [ 0.541298] ACPI: IRQ 4 override to edge(!), high(!)1409lb # [ 0.541918] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1410client # [ 0.542289] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1411lb # [ 0.541927] vgaarb: loaded1412worker2 # [ 0.546159] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1413client # [ 0.543806] pnp: PnP ACPI: found 6 devices1414lb # [ 0.542672] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01415worker2 # [ 0.547593] pnp: PnP ACPI init1416lb # [ 0.542923] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1417worker2 # [ 0.548445] ACPI: IRQ 4 override to edge(!), high(!)1418worker1 # [ 0.494445] ACPI: PCI: Interrupt link LNKA configured for IRQ 101419worker1 # [ 0.494977] ACPI: PCI: Interrupt link LNKB configured for IRQ 101420worker2 # [ 0.549412] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1421worker1 # [ 0.495970] ACPI: PCI: Interrupt link LNKC configured for IRQ 111422worker2 # [ 0.551030] pnp: PnP ACPI: found 6 devices1423worker1 # [ 0.497003] ACPI: PCI: Interrupt link LNKD configured for IRQ 111424worker1 # [ 0.497962] ACPI: PCI: Interrupt link LNKE configured for IRQ 101425lb # [ 0.547015] clocksource: Switched to clocksource kvm-clock1426worker1 # [ 0.498965] ACPI: PCI: Interrupt link LNKF configured for IRQ 101427worker1 # [ 0.499967] ACPI: PCI: Interrupt link LNKG configured for IRQ 111428client # [ 0.551444] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1429client # [ 0.552858] clocksource: Switched to clocksource acpi_pm1430worker1 # [ 0.500966] ACPI: PCI: Interrupt link LNKH configured for IRQ 111431lb # [ 0.550630] VFS: Disk quotas dquot_6.6.01432client # [ 0.553805] NET: Registered PF_INET protocol family1433worker1 # [ 0.501901] ACPI: PCI: Interrupt link GSIA configured for IRQ 161434lb # [ 0.551336] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1435lb # [ 0.552647] pnp: PnP ACPI init1436client # [ 0.555003] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1437worker1 # [ 0.502909] ACPI: PCI: Interrupt link GSIB configured for IRQ 171438worker2 # [ 0.558696] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1439lb # [ 0.553437] ACPI: IRQ 4 override to edge(!), high(!)1440worker1 # [ 0.503889] ACPI: PCI: Interrupt link GSIC configured for IRQ 181441worker2 # [ 0.560096] clocksource: Switched to clocksource acpi_pm1442worker1 # [ 0.504876] ACPI: PCI: Interrupt link GSID configured for IRQ 191443lb # [ 0.554369] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1444worker2 # [ 0.561115] NET: Registered PF_INET protocol family1445worker1 # [ 0.505878] ACPI: PCI: Interrupt link GSIE configured for IRQ 201446lb # [ 0.555864] pnp: PnP ACPI: found 6 devices1447worker2 # [ 0.562302] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1448worker1 # [ 0.506876] ACPI: PCI: Interrupt link GSIF configured for IRQ 211449worker1 # [ 0.507945] ACPI: PCI: Interrupt link GSIG configured for IRQ 221450worker1 # [ 0.509081] ACPI: PCI: Interrupt link GSIH configured for IRQ 231451worker1 # [ 0.510907] iommu: Default domain type: Translated1452worker1 # [ 0.511818] iommu: DMA domain TLB invalidation policy: lazy mode1453worker1 # [ 0.512963] ACPI: bus type USB registered1454lb # [ 0.563479] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1455worker1 # [ 0.513751] usbcore: registered new interface driver usbfs1456lb # [ 0.564920] clocksource: Switched to clocksource acpi_pm1457worker1 # [ 0.514632] usbcore: registered new interface driver hub1458lb # [ 0.565918] NET: Registered PF_INET protocol family1459worker1 # [ 0.514875] usbcore: registered new device driver usb1460lb # [ 0.567098] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1461worker1 # [ 0.516648] NetLabel: Initializing1462worker1 # [ 0.516871] NetLabel: domain hash size = 1281463client # [ 0.571568] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1464worker1 # [ 0.517635] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1465worker1 # [ 0.518597] NetLabel: unlabeled traffic allowed by default1466client # [ 0.573070] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1467worker1 # [ 0.518877] PCI: Using ACPI for IRQ routing1468client # [ 0.574398] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1469client # [ 0.575682] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1470worker2 # [ 0.579316] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1471client # [ 0.576882] TCP: Hash tables configured (established 16384 bind 16384)1472worker2 # [ 0.580896] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1473client # [ 0.577952] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1474worker2 # [ 0.582200] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1475client # [ 0.579268] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1476client # [ 0.580319] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1477worker2 # [ 0.583578] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1478client # [ 0.581486] NET: Registered PF_UNIX/PF_LOCAL protocol family1479worker2 # [ 0.584831] TCP: Hash tables configured (established 16384 bind 16384)1480client # [ 0.582433] NET: Registered PF_XDP protocol family1481worker2 # [ 0.585956] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1482client # [ 0.583232] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1483worker2 # [ 0.587214] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1484client # [ 0.584194] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1485worker2 # [ 0.588335] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1486client # [ 0.585173] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1487worker2 # [ 0.589557] NET: Registered PF_UNIX/PF_LOCAL protocol family1488client # [ 0.586233] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1489worker2 # [ 0.590681] NET: Registered PF_XDP protocol family1490lb # [ 0.583903] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1491client # [ 0.587311] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1492worker2 # [ 0.591464] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1493client # [ 0.588348] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1494lb # [ 0.585399] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1495worker2 # [ 0.592456] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1496client # [ 0.589987] ACPI: \_SB_.GSIA: Enabled at IRQ 161497lb # [ 0.586696] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1498worker2 # [ 0.593457] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1499lb # [ 0.587997] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1500worker2 # [ 0.594517] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1501client # [ 0.592177] ACPI: \_SB_.GSIB: Enabled at IRQ 171502lb # [ 0.589237] TCP: Hash tables configured (established 16384 bind 16384)1503worker2 # [ 0.595773] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1504lb # [ 0.590362] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1505worker2 # [ 0.596821] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1506client # [ 0.594364] ACPI: \_SB_.GSIC: Enabled at IRQ 181507lb # [ 0.591580] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1508worker2 # [ 0.598557] ACPI: \_SB_.GSIA: Enabled at IRQ 161509lb # [ 0.592672] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1510client # [ 0.596411] ACPI: \_SB_.GSID: Enabled at IRQ 191511lb # [ 0.593841] NET: Registered PF_UNIX/PF_LOCAL protocol family1512lb # [ 0.594779] NET: Registered PF_XDP protocol family1513worker2 # [ 0.601099] ACPI: \_SB_.GSIB: Enabled at IRQ 171514client # [ 0.598109] PCI: CLS 0 bytes, default 641515lb # [ 0.595590] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1516lb # [ 0.596568] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1517client # [ 0.598992] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1518worker2 # [ 0.603423] ACPI: \_SB_.GSIC: Enabled at IRQ 181519lb # [ 0.597528] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1520client # [ 0.600669] Trying to unpack rootfs image as initramfs...1521lb # [ 0.598629] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1522worker2 # [ 0.605486] ACPI: \_SB_.GSID: Enabled at IRQ 191523lb # [ 0.599708] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1524worker2 # [ 0.607255] PCI: CLS 0 bytes, default 641525lb # [ 0.600810] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1526lb # [ 0.602571] ACPI: \_SB_.GSIA: Enabled at IRQ 161527worker2 # [ 0.608189] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1528worker2 # [ 0.609952] Trying to unpack rootfs image as initramfs...1529lb # [ 0.605080] ACPI: \_SB_.GSIB: Enabled at IRQ 171530lb # [ 0.607135] ACPI: \_SB_.GSIC: Enabled at IRQ 181531lb # [ 0.609092] ACPI: \_SB_.GSID: Enabled at IRQ 191532lb # [ 0.610786] PCI: CLS 0 bytes, default 641533lb # [ 0.611656] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1534lb # [ 0.613331] Trying to unpack rootfs image as initramfs...1535worker1 # [ 0.564490] pci 0000:00:01.0: vgaarb: setting as boot VGA device1536worker1 # [ 0.564860] pci 0000:00:01.0: vgaarb: bridge control possible1537worker1 # [ 0.564860] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1538worker1 # [ 0.564869] vgaarb: loaded1539worker1 # [ 0.565617] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01540worker1 # [ 0.565866] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1541worker1 # [ 0.569957] clocksource: Switched to clocksource kvm-clock1542worker1 # [ 0.573558] VFS: Disk quotas dquot_6.6.01543worker1 # [ 0.574328] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1544worker1 # [ 0.575762] pnp: PnP ACPI init1545worker1 # [ 0.576576] ACPI: IRQ 4 override to edge(!), high(!)1546worker1 # [ 0.577757] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1547worker1 # [ 0.579410] pnp: PnP ACPI: found 6 devices1548worker1 # [ 0.587029] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1549client # [ 0.647499] Initialise system trusted keyrings1550worker1 # [ 0.588584] clocksource: Switched to clocksource acpi_pm1551client # [ 0.648458] workingset: timestamp_bits=40 max_order=19 bucket_order=01552worker1 # [ 0.589592] NET: Registered PF_INET protocol family1553worker1 # [ 0.590834] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1554worker2 # [ 0.656143] Initialise system trusted keyrings1555worker2 # [ 0.659700] workingset: timestamp_bits=40 max_order=19 bucket_order=01556lb # [ 0.658871] Initialise system trusted keyrings1557lb # [ 0.662351] workingset: timestamp_bits=40 max_order=19 bucket_order=01558worker1 # [ 0.607508] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1559worker1 # [ 0.609123] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1560worker1 # [ 0.610497] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1561worker1 # [ 0.611909] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1562client # [ 0.672313] Key type asymmetric registered1563worker1 # [ 0.613243] TCP: Hash tables configured (established 16384 bind 16384)1564client # [ 0.673024] Asymmetric key parser 'x509' registered1565client # [ 0.673878] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1566worker1 # [ 0.614523] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1567worker1 # [ 0.615902] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1568worker1 # [ 0.617074] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1569client # [ 0.677217] io scheduler mq-deadline registered1570worker2 # [ 0.681120] Key type asymmetric registered1571client # [ 0.677969] io scheduler kyber registered1572worker1 # [ 0.618427] NET: Registered PF_UNIX/PF_LOCAL protocol family1573worker1 # [ 0.619437] NET: Registered PF_XDP protocol family1574worker2 # [ 0.682638] Asymmetric key parser 'x509' registered1575worker1 # [ 0.620327] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1576worker2 # [ 0.683531] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1577worker1 # [ 0.621366] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1578client # [ 0.681603] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1579worker1 # [ 0.622407] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1580worker1 # [ 0.623550] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1581client # [ 0.682893] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1582worker2 # [ 0.687804] io scheduler mq-deadline registered1583worker1 # [ 0.624692] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1584worker1 # [ 0.625858] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1585lb # [ 0.683602] Key type asymmetric registered1586worker1 # [ 0.627639] ACPI: \_SB_.GSIA: Enabled at IRQ 161587worker2 # [ 0.690630] io scheduler kyber registered1588lb # [ 0.684307] Asymmetric key parser 'x509' registered1589client # [ 0.688225] Linux agpgart interface v0.1031590lb # [ 0.685152] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1591worker2 # [ 0.692061] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1592client # [ 0.688968] ACPI: bus type drm_connector registered1593worker1 # [ 0.629964] ACPI: \_SB_.GSIB: Enabled at IRQ 171594worker2 # [ 0.693383] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1595worker1 # [ 0.631971] ACPI: \_SB_.GSIC: Enabled at IRQ 181596client # [ 0.692234] usbcore: registered new interface driver usbserial_generic1597lb # [ 0.690424] io scheduler mq-deadline registered1598worker1 # [ 0.634021] ACPI: \_SB_.GSID: Enabled at IRQ 191599client # [ 0.693362] usbserial: USB Serial support registered for generic1600lb # [ 0.691146] io scheduler kyber registered1601worker1 # [ 0.635730] PCI: CLS 0 bytes, default 641602worker2 # [ 0.699293] Linux agpgart interface v0.1031603worker2 # [ 0.700145] ACPI: bus type drm_connector registered1604worker1 # [ 0.636722] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1605client # [ 0.697088] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1606lb # [ 0.694851] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1607worker1 # [ 0.638506] Trying to unpack rootfs image as initramfs...1608client # [ 0.697088] Please enable it if your BIOS has the CPPC option.1609lb # [ 0.696108] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1610client # [ 0.699378] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1611worker2 # [ 0.703123] usbcore: registered new interface driver usbserial_generic1612worker2 # [ 0.704172] usbserial: USB Serial support registered for generic1613lb # [ 0.700069] Linux agpgart interface v0.1031614client # [ 0.703215] drop_monitor: Initializing network drop monitor service1615lb # [ 0.700854] ACPI: bus type drm_connector registered1616client # [ 0.704405] NET: Registered PF_INET6 protocol family1617worker2 # [ 0.707652] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1618worker2 # [ 0.707652] Please enable it if your BIOS has the CPPC option.1619worker2 # [ 0.709756] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1620lb # [ 0.703838] usbcore: registered new interface driver usbserial_generic1621client # [ 0.707563] Segment Routing with IPv61622lb # [ 0.704884] usbserial: USB Serial support registered for generic1623client # [ 0.710097] In-situ OAM (IOAM) with IPv61624worker2 # [ 0.713791] drop_monitor: Initializing network drop monitor service1625lb # [ 0.707308] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1626client # [ 0.711165] IPI shorthand broadcast: enabled1627lb # [ 0.707308] Please enable it if your BIOS has the CPPC option.1628worker2 # [ 0.714916] NET: Registered PF_INET6 protocol family1629lb # [ 0.709460] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1630worker2 # [ 0.717760] Segment Routing with IPv61631worker2 # [ 0.718396] In-situ OAM (IOAM) with IPv61632worker2 # [ 0.719439] IPI shorthand broadcast: enabled1633lb # [ 0.713432] drop_monitor: Initializing network drop monitor service1634lb # [ 0.714564] NET: Registered PF_INET6 protocol family1635client # [ 0.719374] sched_clock: Marking stable (585015639, 134046044)->(792498307, -73436624)1636lb # [ 0.718064] Segment Routing with IPv61637lb # [ 0.720320] In-situ OAM (IOAM) with IPv61638client # [ 0.724261] registered taskstats version 11639worker2 # [ 0.727524] sched_clock: Marking stable (591013380, 135607062)->(797182393, -70561951)1640client # [ 0.725192] Loading compiled-in X.509 certificates1641lb # [ 0.722448] IPI shorthand broadcast: enabled1642worker2 # [ 0.730862] registered taskstats version 11643worker2 # [ 0.731893] Loading compiled-in X.509 certificates1644lb # [ 0.730730] sched_clock: Marking stable (595014777, 135269624)->(798502468, -68218067)1645lb # [ 0.735392] registered taskstats version 11646lb # [ 0.736353] Loading compiled-in X.509 certificates1647worker1 # [ 0.683587] Initialise system trusted keyrings1648client # [ 0.744079] Demotion targets for Node 0: null1649client # [ 0.745000] Key type .fscrypt registered1650client # [ 0.745707] Key type fscrypt-provisioning registered1651client # [ 0.746736] ima: No TPM chip found, activating TPM-bypass!1652worker1 # [ 0.687212] workingset: timestamp_bits=40 max_order=19 bucket_order=01653worker2 # [ 0.751637] Demotion targets for Node 0: null1654worker2 # [ 0.752555] Key type .fscrypt registered1655worker2 # [ 0.753295] Key type fscrypt-provisioning registered1656worker2 # [ 0.754261] ima: No TPM chip found, activating TPM-bypass!1657client # [ 0.751078] ima: Allocated hash algorithm: sha11658client # [ 0.751931] ima: No architecture policies found1659client # [ 0.754083] PM: Magic number: 10:31:7851660worker2 # [ 0.758652] ima: Allocated hash algorithm: sha11661lb # [ 0.752865] Demotion targets for Node 0: null1662client # [ 0.755805] RAS: Correctable Errors collector initialized.1663worker2 # [ 0.759457] ima: No architecture policies found1664worker2 # [ 0.760422] PM: Magic number: 10:31:7851665lb # [ 0.756345] Key type .fscrypt registered1666lb # [ 0.757028] Key type fscrypt-provisioning registered1667lb # [ 0.757982] ima: No TPM chip found, activating TPM-bypass!1668worker2 # [ 0.764240] RAS: Correctable Errors collector initialized.1669lb # [ 0.760298] ima: Allocated hash algorithm: sha11670lb # [ 0.761203] ima: No architecture policies found1671client # [ 0.765841] clk: Disabling unused clocks1672lb # [ 0.765310] PM: Magic number: 10:31:7851673client # [ 0.768082] PM: genpd: Disabling unused power domains1674worker1 # [ 0.709562] Key type asymmetric registered1675worker2 # [ 0.773014] clk: Disabling unused clocks1676worker1 # [ 0.710361] Asymmetric key parser 'x509' registered1677lb # [ 0.766932] RAS: Correctable Errors collector initialized.1678worker2 # [ 0.775656] PM: genpd: Disabling unused power domains1679worker1 # [ 0.713247] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1680worker1 # [ 0.716269] io scheduler mq-deadline registered1681worker1 # [ 0.717101] io scheduler kyber registered1682lb # [ 0.774950] clk: Disabling unused clocks1683lb # [ 0.777309] PM: genpd: Disabling unused power domains1684worker1 # [ 0.720691] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1685worker1 # [ 0.722055] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1686worker1 # [ 0.727576] Linux agpgart interface v0.1031687worker1 # [ 0.728440] ACPI: bus type drm_connector registered1688worker1 # [ 0.731675] usbcore: registered new interface driver usbserial_generic1689worker1 # [ 0.732774] usbserial: USB Serial support registered for generic1690worker1 # [ 0.736186] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1691worker1 # [ 0.736186] Please enable it if your BIOS has the CPPC option.1692worker1 # [ 0.738493] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1693worker1 # [ 0.742304] drop_monitor: Initializing network drop monitor service1694worker1 # [ 0.743519] NET: Registered PF_INET6 protocol family1695worker1 # [ 0.746634] Segment Routing with IPv61696worker1 # [ 0.749192] In-situ OAM (IOAM) with IPv61697worker1 # [ 0.750270] IPI shorthand broadcast: enabled1698worker1 # [ 0.758390] sched_clock: Marking stable (615013605, 143144669)->(840447097, -82288823)1699worker1 # [ 0.762416] registered taskstats version 11700worker1 # [ 0.763434] Loading compiled-in X.509 certificates1701worker1 # [ 0.781871] Demotion targets for Node 0: null1702worker1 # [ 0.785212] Key type .fscrypt registered1703worker1 # [ 0.785938] Key type fscrypt-provisioning registered1704worker1 # [ 0.786913] ima: No TPM chip found, activating TPM-bypass!1705worker1 # [ 0.789177] ima: Allocated hash algorithm: sha11706worker1 # [ 0.790034] ima: No architecture policies found1707worker1 # [ 0.794184] PM: Magic number: 10:31:7851708worker1 # [ 0.795940] RAS: Correctable Errors collector initialized.1709worker1 # [ 0.804864] clk: Disabling unused clocks1710worker1 # [ 0.807180] PM: genpd: Disabling unused power domains1711client # [ 0.929825] Freeing initrd memory: 29096K1712client # [ 0.933077] Freeing unused decrypted memory: 2028K1713client # [ 0.935738] Freeing unused kernel image (initmem) memory: 3644K1714client # [ 0.936875] Write protecting the kernel read-only data: 32768k1715worker2 # [ 0.940819] Freeing initrd memory: 29092K1716client # [ 0.938758] Freeing unused kernel image (text/rodata gap) memory: 1208K1717client # [ 0.940329] Freeing unused kernel image (rodata/data gap) memory: 752K1718worker2 # [ 0.944194] Freeing unused decrypted memory: 2028K1719lb # [ 0.939853] Freeing initrd memory: 29092K1720worker2 # [ 0.947057] Freeing unused kernel image (initmem) memory: 3644K1721worker2 # [ 0.948197] Write protecting the kernel read-only data: 32768k1722lb # [ 0.943112] Freeing unused decrypted memory: 2028K1723worker2 # [ 0.950197] Freeing unused kernel image (text/rodata gap) memory: 1208K1724worker2 # [ 0.951805] Freeing unused kernel image (rodata/data gap) memory: 752K1725lb # [ 0.945766] Freeing unused kernel image (initmem) memory: 3644K1726lb # [ 0.946815] Write protecting the kernel read-only data: 32768k1727lb # [ 0.948664] Freeing unused kernel image (text/rodata gap) memory: 1208K1728lb # [ 0.950152] Freeing unused kernel image (rodata/data gap) memory: 752K1729client # [ 0.991608] x86/mm: Checked W+X mappings: passed, no W+X pages found.1730client # [ 0.992689] Run /init as init process1731worker2 # [ 1.003126] x86/mm: Checked W+X mappings: passed, no W+X pages found.1732worker2 # [ 1.004261] Run /init as init process1733client # [ 1.002976] systemd[1]: Inserted module 'autofs4'1734lb # [ 1.001257] x86/mm: Checked W+X mappings: passed, no W+X pages found.1735lb # [ 1.002332] Run /init as init process1736worker2 # [ 1.015005] systemd[1]: Inserted module 'autofs4'1737lb # [ 1.012595] systemd[1]: Inserted module 'autofs4'1738client # [ 1.019601] fuse: init (API version 7.45)1739client # [ 1.025530] ACPI: \_SB_.GSIG: Enabled at IRQ 221740worker1 # [ 0.968119] Freeing initrd memory: 29092K1741client # [ 1.028209] ACPI: \_SB_.GSIH: Enabled at IRQ 231742worker2 # [ 1.032030] fuse: init (API version 7.45)1743worker1 # [ 0.971426] Freeing unused decrypted memory: 2028K1744client # [ 1.031120] ACPI: \_SB_.GSIE: Enabled at IRQ 201745lb # [ 1.028887] fuse: init (API version 7.45)1746client # [ 1.033276] ACPI: \_SB_.GSIF: Enabled at IRQ 211747worker1 # [ 0.974130] Freeing unused kernel image (initmem) memory: 3644K1748worker2 # [ 1.038024] ACPI: \_SB_.GSIG: Enabled at IRQ 221749worker1 # [ 0.975325] Write protecting the kernel read-only data: 32768k1750worker1 # [ 0.977297] Freeing unused kernel image (text/rodata gap) memory: 1208K1751worker2 # [ 1.040512] ACPI: \_SB_.GSIH: Enabled at IRQ 231752lb # [ 1.034751] ACPI: \_SB_.GSIG: Enabled at IRQ 221753worker1 # [ 0.978984] Freeing unused kernel image (rodata/data gap) memory: 752K1754lb # [ 1.037216] ACPI: \_SB_.GSIH: Enabled at IRQ 231755worker2 # [ 1.043667] ACPI: \_SB_.GSIE: Enabled at IRQ 201756worker2 # [ 1.045847] ACPI: \_SB_.GSIF: Enabled at IRQ 211757lb # [ 1.040424] ACPI: \_SB_.GSIE: Enabled at IRQ 201758lb # [ 1.042762] ACPI: \_SB_.GSIF: Enabled at IRQ 211759client # [ 1.068108] systemd[1]: Successfully made /usr/ read-only.1760worker2 # [ 1.077701] systemd[1]: Successfully made /usr/ read-only.1761lb # [ 1.075434] systemd[1]: Successfully made /usr/ read-only.1762worker1 # [ 1.030348] x86/mm: Checked W+X mappings: passed, no W+X pages found.1763worker1 # [ 1.031667] Run /init as init process1764worker1 # [ 1.041865] systemd[1]: Inserted module 'autofs4'1765worker1 # [ 1.058339] fuse: init (API version 7.45)1766worker1 # [ 1.064232] ACPI: \_SB_.GSIG: Enabled at IRQ 221767worker1 # [ 1.066670] ACPI: \_SB_.GSIH: Enabled at IRQ 231768worker1 # [ 1.069853] ACPI: \_SB_.GSIE: Enabled at IRQ 201769worker1 # [ 1.072252] ACPI: \_SB_.GSIF: Enabled at IRQ 211770worker1 # [ 1.106021] systemd[1]: Successfully made /usr/ read-only.1771client # [ 1.404737] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1772client # [ 1.413215] systemd[1]: Detected virtualization kvm.1773client # [ 1.414265] systemd[1]: Detected architecture x86-64.1774client # [ 1.415362] systemd[1]: Running in initrd.1775client # [ 1.416613] systemd[1]: Initializing machine ID from random generator.1776client # [ 1.418000] systemd[1]: Hostname set to <client>.1777worker2 # [ 1.414339] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1778worker2 # [ 1.427475] systemd[1]: Detected virtualization kvm.1779worker2 # [ 1.429758] systemd[1]: Detected architecture x86-64.1780lb # [ 1.412544] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1781worker2 # [ 1.431979] systemd[1]: Running in initrd.1782lb # [ 1.426120] systemd[1]: Detected virtualization kvm.1783worker2 # [ 1.434462] systemd[1]: Initializing machine ID from random generator.1784lb # [ 1.428830] systemd[1]: Detected architecture x86-64.1785worker2 # [ 1.437412] systemd[1]: Hostname set to <worker2>.1786lb # [ 1.431477] systemd[1]: Running in initrd.1787lb # [ 1.434600] systemd[1]: Initializing machine ID from random generator.1788lb # [ 1.437929] systemd[1]: Hostname set to <lb>.1789worker1 # [ 1.442793] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1790worker1 # [ 1.455195] systemd[1]: Detected virtualization kvm.1791worker1 # [ 1.457348] systemd[1]: Detected architecture x86-64.1792worker1 # [ 1.459531] systemd[1]: Running in initrd.1793worker1 # [ 1.461985] systemd[1]: Initializing machine ID from random generator.1794worker1 # [ 1.464870] systemd[1]: Hostname set to <worker1>.1795client # [ 1.626729] systemd[1]: bpf-restrict-fs: LSM BPF program attached1796worker2 # [ 1.660769] systemd[1]: bpf-restrict-fs: LSM BPF program attached1797lb # [ 1.660179] systemd[1]: bpf-restrict-fs: LSM BPF program attached1798client # [ 1.679773] systemd[1]: Queued start job for default target Initrd Default Target.1799client # [ 1.684437] systemd[1]: Created slice Slice /system/modprobe.1800client # [ 1.685657] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1801client # [ 1.687137] systemd[1]: Expecting device /dev/disk/by-label/nixos...1802client # [ 1.688299] systemd[1]: Reached target Path Units.1803client # [ 1.689202] systemd[1]: Reached target Slice Units.1804client # [ 1.690126] systemd[1]: Reached target Swaps.1805client # [ 1.690925] systemd[1]: Reached target Timer Units.1806client # [ 1.691899] systemd[1]: Listening on D-Bus System Message Bus Socket.1807client # [ 1.693165] systemd[1]: Listening on Journal Socket (/dev/log).1808client # [ 1.694365] systemd[1]: Listening on Journal Sockets.1809client # [ 1.695429] systemd[1]: Listening on udev Control Socket.1810client # [ 1.696506] systemd[1]: Listening on udev Kernel Socket.1811client # [ 1.697502] systemd[1]: Reached target Socket Units.1812client # [ 1.699319] systemd[1]: Starting Create List of Static Device Nodes...1813client # [ 1.703135] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1814client # [ 1.711152] systemd[1]: Starting Load Kernel Module configfs...1815worker2 # [ 1.715454] systemd[1]: Queued start job for default target Initrd Default Target.1816worker2 # [ 1.720063] systemd[1]: Created slice Slice /system/modprobe.1817lb # [ 1.714933] systemd[1]: Queued start job for default target Initrd Default Target.1818worker2 # [ 1.721357] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1819worker2 # [ 1.722770] systemd[1]: Expecting device /dev/disk/by-label/nixos...1820worker2 # [ 1.723804] systemd[1]: Reached target Path Units.1821worker2 # [ 1.724693] systemd[1]: Reached target Slice Units.1822worker2 # [ 1.725555] systemd[1]: Reached target Swaps.1823lb # [ 1.719625] systemd[1]: Created slice Slice /system/modprobe.1824worker2 # [ 1.726438] systemd[1]: Reached target Timer Units.1825lb # [ 1.720829] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1826worker2 # [ 1.727458] systemd[1]: Listening on D-Bus System Message Bus Socket.1827client # [ 1.724694] systemd[1]: Starting Journal Service...1828lb # [ 1.722291] systemd[1]: Expecting device /dev/disk/by-label/nixos...1829worker2 # [ 1.728720] systemd[1]: Listening on Journal Socket (/dev/log).1830lb # [ 1.723424] systemd[1]: Reached target Path Units.1831worker2 # [ 1.729840] systemd[1]: Listening on Journal Sockets.1832lb # [ 1.724308] systemd[1]: Reached target Slice Units.1833worker2 # [ 1.730831] systemd[1]: Listening on udev Control Socket.1834lb # [ 1.725199] systemd[1]: Reached target Swaps.1835worker2 # [ 1.731870] systemd[1]: Listening on udev Kernel Socket.1836lb # [ 1.726047] systemd[1]: Reached target Timer Units.1837worker2 # [ 1.732846] systemd[1]: Reached target Socket Units.1838lb # [ 1.727061] systemd[1]: Listening on D-Bus System Message Bus Socket.1839lb # [ 1.728346] systemd[1]: Listening on Journal Socket (/dev/log).1840worker2 # [ 1.734661] systemd[1]: Starting Create List of Static Device Nodes...1841lb # [ 1.729536] systemd[1]: Listening on Journal Sockets.1842lb # [ 1.730578] systemd[1]: Listening on udev Control Socket.1843lb # [ 1.731699] systemd[1]: Listening on udev Kernel Socket.1844worker2 # [ 1.738479] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1845lb # [ 1.732721] systemd[1]: Reached target Socket Units.1846lb # [ 1.734602] systemd[1]: Starting Create List of Static Device Nodes...1847client # [ 1.739212] systemd[1]: Starting Load Kernel Modules...1848lb # [ 1.739355] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1849worker2 # [ 1.747721] systemd[1]: Starting Load Kernel Module configfs...1850worker1 # [ 1.684495] systemd[1]: bpf-restrict-fs: LSM BPF program attached1851lb # [ 1.747482] systemd[1]: Starting Load Kernel Module configfs...1852worker2 # [ 1.757290] systemd[1]: Starting Journal Service...1853client # [ 1.743577] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1854client # [ 1.759913] netfs: FS-Cache loaded1855lb # [ 1.759999] systemd[1]: Starting Journal Service...1856worker2 # [ 1.767825] systemd[1]: Starting Load Kernel Modules...1857client # [ 1.765191] 9pnet: Installing 9P2000 support1858lb # [ 1.774370] systemd[1]: Starting Load Kernel Modules...1859client # [ 1.781323] systemd[1]: Starting Coldplug All udev Devices...1860worker2 # [ 1.773707] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1861worker2 # [ 1.796143] netfs: FS-Cache loaded1862lb # [ 1.780363] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1863client # [ 1.795716] systemd[1]: Finished Create List of Static Device Nodes.1864worker2 # [ 1.801705] 9pnet: Installing 9P2000 support1865client # [ 1.798911] systemd-journald[67]: Collecting audit messages is disabled.1866worker1 # [ 1.739716] systemd[1]: Queued start job for default target Initrd Default Target.1867lb # [ 1.797628] netfs: FS-Cache loaded1868client # [ 1.801659] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1869worker1 # [ 1.743478] systemd[1]: Created slice Slice /system/modprobe.1870worker1 # [ 1.744975] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1871lb # [ 1.802441] 9pnet: Installing 9P2000 support1872worker1 # [ 1.746400] systemd[1]: Expecting device /dev/disk/by-label/nixos...1873worker1 # [ 1.747550] systemd[1]: Reached target Path Units.1874worker2 # [ 1.810628] systemd[1]: Starting Coldplug All udev Devices...1875worker1 # [ 1.748441] systemd[1]: Reached target Slice Units.1876worker1 # [ 1.749341] systemd[1]: Reached target Swaps.1877worker1 # [ 1.750184] systemd[1]: Reached target Timer Units.1878client # [ 1.810512] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1879worker1 # [ 1.751215] systemd[1]: Listening on D-Bus System Message Bus Socket.1880worker1 # [ 1.752433] systemd[1]: Listening on Journal Socket (/dev/log).1881worker1 # [ 1.753634] systemd[1]: Listening on Journal Sockets.1882worker1 # [ 1.754707] systemd[1]: Listening on udev Control Socket.1883worker1 # [ 1.755770] systemd[1]: Listening on udev Kernel Socket.1884worker1 # [ 1.756781] systemd[1]: Reached target Socket Units.1885client # [ 1.817514] systemd[1]: modprobe@configfs.service: Deactivated successfully.1886worker1 # [ 1.758719] systemd[1]: Starting Create List of Static Device Nodes...1887lb # [ 1.816415] systemd[1]: Starting Coldplug All udev Devices...1888client # [ 1.822511] systemd[1]: Finished Load Kernel Module configfs.1889worker1 # [ 1.763230] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1890worker2 # [ 1.829786] systemd[1]: Finished Create List of Static Device Nodes.1891client # [ 1.828511] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1892worker1 # [ 1.770232] systemd[1]: Starting Load Kernel Module configfs...1893lb # [ 1.829762] systemd[1]: Finished Create List of Static Device Nodes.1894worker2 # [ 1.836022] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1895client # [ 1.833830] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1896worker2 # [ 1.843153] systemd-journald[67]: Collecting audit messages is disabled.1897worker2 # [ 1.844811] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1898client # [ 1.841142] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1899lb # [ 1.838485] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1900client # [ 1.843271] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1901worker1 # [ 1.785715] systemd[1]: Starting Journal Service...1902lb # [ 1.845357] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1903worker2 # [ 1.852105] systemd[1]: modprobe@configfs.service: Deactivated successfully.1904worker2 # [ 1.857122] systemd[1]: Finished Load Kernel Module configfs.1905lb # [ 1.852821] systemd[1]: modprobe@configfs.service: Deactivated successfully.1906lb # [ 1.854102] systemd-journald[67]: Collecting audit messages is disabled.1907worker1 # [ 1.800228] systemd[1]: Starting Load Kernel Modules...1908lb # [ 1.858727] systemd[1]: Finished Load Kernel Module configfs.1909worker2 # [ 1.864121] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1910worker2 # [ 1.869121] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1911lb # [ 1.863810] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1912lb # [ 1.868732] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1913worker2 # [ 1.876892] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1914client # [ 1.875688] systemd[1]: Finished Load Kernel Modules.1915worker1 # [ 1.804253] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1916worker2 # [ 1.880650] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1917worker1 # [ 1.820345] netfs: FS-Cache loaded1918lb # [ 1.876316] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1919lb # [ 1.878356] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1920client # [ 1.884311] systemd[1]: Starting Apply Kernel Variables...1921worker1 # [ 1.825672] 9pnet: Installing 9P2000 support1922client # [ 1.894134] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1923worker1 # [ 1.843455] systemd[1]: Starting Coldplug All udev Devices...1924client # [ 1.903260] systemd[1]: Starting Create Static Device Nodes in /dev...1925worker2 # [ 1.910231] systemd[1]: Finished Load Kernel Modules.1926lb # [ 1.908894] systemd[1]: Finished Load Kernel Modules.1927worker2 # [ 1.916757] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1928worker1 # [ 1.856002] systemd-journald[67]: Collecting audit messages is disabled.1929worker1 # [ 1.859254] systemd[1]: Finished Create List of Static Device Nodes.1930client # [ 1.784168] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.919309] systemd[1]: Started Journal Service.1931client # 1932lb # [ 1.917511] systemd[1]: Starting Apply Kernel Variables...1933worker2 # [ 1.924989] systemd[1]: Starting Apply Kernel Variables...1934client # [ 1.789232] systemd-modules-load[68]: Inserted module 'virtio_balloon'1935client # [ 1.790298] systemd-modules-load[68]: Inserted module 'virtio_gpu'1936worker1 # [ 1.865085] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1937client # [ 1.795934] systemd[1]: Finished Apply Kernel Variables.1938worker1 # [ 1.871483] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1939lb # [ 1.929445] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1940worker2 # [ 1.936714] systemd[1]: Starting Create Static Device Nodes in /dev...1941worker1 # [ 1.878573] systemd[1]: modprobe@configfs.service: Deactivated successfully.1942client # [ 1.807096] systemd[1]: Finished Create Static Device Nodes in /dev.1943client # [ 1.808118] systemd[1]: Reached target Preparation for Local File Systems.1944client # [ 1.809081] systemd[1]: Reached target Local File Systems.1945lb # [ 1.940009] systemd[1]: Starting Create Static Device Nodes in /dev...1946worker1 # [ 1.885201] systemd[1]: Finished Load Kernel Module configfs.1947client # [ 1.812041] systemd[1]: Starting Create System Files and Directories...1948worker1 # [ 1.889760] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1949client # [ 1.819087] systemd[1]: Starting Rule-based Manager for Device Events and Files...1950worker1 # [ 1.895383] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1951worker2 # [ 1.961761] systemd[1]: Started Journal Service.1952worker2 # [ 1.830255] systemd-modules-load[68]: Inserted module 'dm_mod'1953worker2 # [ 1.831650] systemd-modules-load[68]: Inserted module 'virtio_balloon'1954worker1 # [ 1.905441] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1955lb # [ 1.963630] systemd[1]: Finished Apply Kernel Variables.1956worker1 # [ 1.907308] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1957worker2 # [ 1.835498] systemd-modules-load[68]: Inserted module 'virtio_gpu'1958worker2 # [ 1.841099] systemd[1]: Finished Apply Kernel Variables.1959lb # [ 1.838575] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.974989] systemd[1]: Started Journal Service.1960worker2 # [ 1.847976] systemd[1]: Finished Create Static Device Nodes in /dev.1961lb # 1962client # [ 1.846230] systemd[1]: Finished Create System Files and Directories.1963lb # [ 1.843314] systemd-modules-load[68]: Inserted module 'virtio_balloon'1964lb # [ 1.844336] systemd-modules-load[68]: Inserted module 'virtio_gpu'1965worker2 # [ 1.851103] systemd[1]: Reached target Preparation for Local File Systems.1966worker2 # [ 1.852098] systemd[1]: Reached target Local File Systems.1967worker2 # [ 1.855066] systemd[1]: Starting Create System Files and Directories...1968lb # [ 1.850110] systemd[1]: Finished Create Static Device Nodes in /dev.1969worker2 # [ 1.862081] systemd[1]: Starting Rule-based Manager for Device Events and Files...1970lb # [ 1.857101] systemd[1]: Reached target Preparation for Local File Systems.1971lb # [ 1.858154] systemd[1]: Reached target Local File Systems.1972lb # [ 1.861067] systemd[1]: Starting Create System Files and Directories...1973lb # [ 1.865329] systemd[1]: Starting Rule-based Manager for Device Events and Files...1974worker1 # [ 1.951391] systemd[1]: Finished Load Kernel Modules.1975client # [ 1.877337] systemd-udevd[83]: Using default interface naming scheme 'v261'.1976worker2 # [ 1.886112] systemd[1]: Finished Create System Files and Directories.1977worker1 # [ 1.960261] systemd[1]: Starting Apply Kernel Variables...1978lb # [ 1.895246] systemd[1]: Finished Create System Files and Directories.1979worker1 # [ 1.974309] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1980client # [ 1.900597] systemd[1]: Started Rule-based Manager for Device Events and Files.1981worker2 # [ 1.911280] systemd-udevd[83]: Using default interface naming scheme 'v261'.1982worker1 # [ 1.989245] systemd[1]: Starting Create Static Device Nodes in /dev...1983lb # [ 1.917991] systemd-udevd[83]: Using default interface naming scheme 'v261'.1984worker1 # [ 2.003768] systemd[1]: Started Journal Service.1985worker1 # [ 1.862259] systemd-modules-load[68]: Inserted module 'dm_mod'1986worker2 # [ 1.936355] systemd[1]: Started Rule-based Manager for Device Events and Files.1987worker1 # [ 1.866568] systemd-modules-load[68]: Inserted module 'virtio_balloon'1988worker1 # [ 1.868339] systemd-modules-load[68]: Inserted module 'virtio_gpu'1989worker1 # [ 1.878316] systemd[1]: Finished Apply Kernel Variables.1990lb # [ 1.942942] systemd[1]: Started Rule-based Manager for Device Events and Files.1991worker1 # [ 1.894122] systemd[1]: Finished Create Static Device Nodes in /dev.1992worker1 # [ 1.896624] systemd[1]: Reached target Preparation for Local File Systems.1993worker1 # [ 1.900072] systemd[1]: Reached target Local File Systems.1994client # [ 1.970085] systemd[1]: Finished Coldplug All udev Devices.1995client # [ 1.971121] systemd[1]: Reached target System Initialization.1996client # [ 1.971902] systemd[1]: Reached target Basic System.1997worker1 # [ 1.903079] systemd[1]: Starting Create System Files and Directories...1998worker1 # [ 1.910081] systemd[1]: Starting Rule-based Manager for Device Events and Files...1999worker2 # [ 1.985356] systemd[1]: Finished Coldplug All udev Devices.2000worker2 # [ 1.987790] systemd[1]: Reached target System Initialization.
loading 1,984 hidden lines…
3985lb # [ 16.892880] envoy[910]: [2026-09-20 12:46:07.964][910][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log3986lb # [ 16.899042] envoy[910]: [2026-09-20 12:46:07.964][910][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name3987lb # [ 16.901179] envoy[910]: [2026-09-20 12:46:07.964][910][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff3988lb # [ 16.903380] envoy[910]: [2026-09-20 12:46:07.964][910][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation3989lb # [ 16.905441] envoy[910]: [2026-09-20 12:46:07.964][910][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default3990lb # [ 16.907205] envoy[910]: [2026-09-20 12:46:07.964][910][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource3991lb # [ 16.910493] envoy[910]: [2026-09-20 12:46:07.964][910][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm3992lb # [ 16.918215] envoy[910]: [2026-09-20 12:46:07.964][910][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v83993lb # [ 16.919987] envoy[910]: [2026-09-20 12:46:07.964][910][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router3994lb # [ 16.922725] envoy[910]: [2026-09-20 12:46:07.964][910][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router3995lb # [ 16.924294] envoy[910]: [2026-09-20 12:46:07.964][910][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts3996lb # [ 16.927248] envoy[910]: [2026-09-20 12:46:07.964][910][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher3997lb # [ 16.929836] envoy[910]: [2026-09-20 12:46:07.964][910][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header3998lb # [ 16.932405] envoy[910]: [2026-09-20 12:46:07.964][910][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name3999lb # [ 16.935082] envoy[910]: [2026-09-20 12:46:07.964][910][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua4000lb # [ 16.936728] envoy[910]: [2026-09-20 12:46:07.964][910][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port4001lb # [ 16.943682] envoy[910]: [2026-09-20 12:46:07.964][910][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert4002lb # [ 16.945676] envoy[910]: [2026-09-20 12:46:07.964][910][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service4003lb # [ 16.949905] envoy[910]: [2026-09-20 12:46:07.964][910][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata4004lb # [ 16.952523] envoy[910]: [2026-09-20 12:46:07.964][910][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config4005client # [ 16.959672] systemd[1]: Started DHCP Client.4006lb # [ 16.955739] envoy[910]: [2026-09-20 12:46:07.964][910][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator4007client # [ 16.962323] systemd[1]: Reached target Multi-User System.4008lb # [ 16.958258] envoy[910]: [2026-09-20 12:46:07.964][910][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities4009client # [ 16.963273] systemd[1]: Startup finished in 862ms (kernel) + 4.070s (initrd) + 12.030s (userspace) = 16.963s.4010lb # [ 16.960314] envoy[910]: [2026-09-20 12:46:07.969][910][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats4011lb # [ 16.963098] envoy[910]: [2026-09-20 12:46:07.969][910][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http14012lb # [ 16.965069] envoy[910]: [2026-09-20 12:46:07.969][910][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state4013lb # [ 16.967735] envoy[910]: [2026-09-20 12:46:07.969][910][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable4014lb # [ 16.969627] envoy[910]: [2026-09-20 12:46:07.969][910][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json4015lb # [ 16.971156] envoy[910]: [2026-09-20 12:46:07.969][910][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher4016lb # [ 16.974113] envoy[910]: [2026-09-20 12:46:07.969][910][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy4017lb # [ 16.976688] envoy[910]: [2026-09-20 12:46:07.969][910][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router4018lb # [ 16.978771] envoy[910]: [2026-09-20 12:46:07.969][910][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection4019lb # [ 16.983248] envoy[910]: [2026-09-20 12:46:07.969][910][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso4020lb # [ 16.985122] envoy[910]: [2026-09-20 12:46:07.969][910][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default4021lb # [ 16.986799] envoy[910]: [2026-09-20 12:46:07.969][910][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file4022lb # [ 16.988452] envoy[910]: [2026-09-20 12:46:07.969][910][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder4023lb # [ 16.991983] envoy[910]: [2026-09-20 12:46:07.969][910][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action4024lb # [ 16.993939] envoy[910]: [2026-09-20 12:46:07.969][910][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator4025lb # [ 16.996136] envoy[910]: [2026-09-20 12:46:07.969][910][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor4026lb # [ 16.998497] envoy[910]: [2026-09-20 12:46:07.972][910][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default4027lb # [ 17.000059] envoy[910]: [2026-09-20 12:46:07.972][910][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default4028lb # [ 17.001549] envoy[910]: [2026-09-20 12:46:07.972][910][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind4029lb # [ 17.003180] envoy[910]: [2026-09-20 12:46:07.972][910][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift4030lb # [ 17.006798] envoy[910]: [2026-09-20 12:46:07.972][910][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default4031lb # [ 17.008929] envoy[910]: [2026-09-20 12:46:07.972][910][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher4032lb # [ 17.010846] envoy[910]: [2026-09-20 12:46:07.972][910][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls4033lb # [ 17.015221] envoy[910]: [2026-09-20 12:46:07.978][910][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value4034lb # [ 17.018376] envoy[910]: [2026-09-20 12:46:07.978][910][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation4035lb # [ 17.020637] envoy[910]: [2026-09-20 12:46:07.978][910][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options4036lb # [ 17.024695] envoy[910]: [2026-09-20 12:46:07.978][910][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input4037lb # [ 17.026864] envoy[910]: [2026-09-20 12:46:07.978][910][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed4038lb # [ 17.029335] envoy[910]: [2026-09-20 12:46:07.978][910][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4039lb # [ 17.031828] envoy[910]: [2026-09-20 12:46:07.978][910][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case4040lb # [ 17.034320] envoy[910]: [2026-09-20 12:46:07.978][910][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor4041lb # [ 17.037103] envoy[910]: [2026-09-20 12:46:07.978][910][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection4042lb # [ 17.038769] envoy[910]: [2026-09-20 12:46:07.978][910][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian24043lb # [ 17.040303] envoy[910]: [2026-09-20 12:46:07.979][910][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo4044lb # [ 17.041730] envoy[910]: [2026-09-20 12:46:07.979][910][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest4045lb # [ 17.046547] envoy[910]: [2026-09-20 12:46:07.979][910][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re24046lb # [ 17.048504] envoy[910]: [2026-09-20 12:46:07.979][910][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params4047lb # [ 17.058076] envoy[910]: [2026-09-20 12:46:07.979][910][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector4048lb # [ 17.063755] envoy[910]: [2026-09-20 12:46:07.979][910][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override4049lb # [ 17.066703] envoy[910]: [2026-09-20 12:46:07.979][910][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr4050lb # [ 17.070101] envoy[910]: [2026-09-20 12:46:07.979][910][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based4051lb # [ 17.073705] envoy[910]: [2026-09-20 12:46:07.979][910][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit4052lb # [ 17.075983] envoy[910]: [2026-09-20 12:46:07.979][910][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response4053lb # [ 17.077951] envoy[910]: [2026-09-20 12:46:07.979][910][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter4054lb # [ 17.079747] envoy[910]: [2026-09-20 12:46:07.979][910][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default4055lb # [ 17.081565] envoy[910]: [2026-09-20 12:46:07.979][910][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple4056lb # [ 17.083659] envoy[910]: [2026-09-20 12:46:07.979][910][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme4057lb # [ 17.086654] envoy[910]: [2026-09-20 12:46:07.979][910][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap4058lb # [ 17.090918] envoy[910]: [2026-09-20 12:46:07.979][910][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy4059lb # [ 17.093423] envoy[910]: [2026-09-20 12:46:07.979][910][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated4060lb # [ 17.096087] envoy[910]: [2026-09-20 12:46:07.979][910][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle4061lb # [ 17.098101] envoy[910]: [2026-09-20 12:46:07.979][910][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default4062lb # [ 17.099653] envoy[910]: [2026-09-20 12:46:07.979][910][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query4063lb # [ 17.102646] envoy[910]: [2026-09-20 12:46:07.980][910][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4064lb # [ 17.104693] envoy[910]: [2026-09-20 12:46:07.980][910][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory4065lb # [ 17.107520] envoy[910]: [2026-09-20 12:46:07.980][910][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid4066lb # [ 17.109089] envoy[910]: [2026-09-20 12:46:07.980][910][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain4067lb # [ 17.111096] envoy[910]: [2026-09-20 12:46:07.980][910][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default4068lb # [ 17.113567] envoy[910]: [2026-09-20 12:46:07.980][910][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple4069lb # [ 17.115727] envoy[910]: [2026-09-20 12:46:07.980][910][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule4070lb # [ 17.118312] envoy[910]: [2026-09-20 12:46:07.980][910][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4071lb # [ 17.120274] envoy[910]: [2026-09-20 12:46:07.980][910][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin4072lb # [ 17.123282] envoy[910]: [2026-09-20 12:46:07.980][910][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed4073lb # [ 17.125151] envoy[910]: [2026-09-20 12:46:07.980][910][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd4074lb # [ 17.129908] envoy[910]: [2026-09-20 12:46:07.980][910][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy4075lb # [ 17.141512] envoy[910]: [2026-09-20 12:46:07.980][910][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port4076lb # [ 17.143525] envoy[910]: [2026-09-20 12:46:07.980][910][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche4077lb # [ 17.145466] envoy[910]: [2026-09-20 12:46:07.980][910][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default4078lb # [ 17.147504] envoy[910]: [2026-09-20 12:46:07.980][910][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter4079lb # [ 17.149493] envoy[910]: [2026-09-20 12:46:07.980][910][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal4080lb # [ 17.151270] envoy[910]: [2026-09-20 12:46:07.980][910][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe4081lb # [ 17.153814] envoy[910]: [2026-09-20 12:46:07.980][910][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls4082lb # [ 17.159651] envoy[910]: [2026-09-20 12:46:07.980][910][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth24083lb # [ 17.161867] envoy[910]: [2026-09-20 12:46:07.980][910][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override4084lb # [ 17.164227] envoy[910]: [2026-09-20 12:46:07.980][910][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic4085lb # [ 17.165849] envoy[910]: [2026-09-20 12:46:07.980][910][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction4086lb # [ 17.169973] envoy[910]: [2026-09-20 12:46:07.980][910][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san4087lb # [ 17.176093] envoy[910]: [2026-09-20 12:46:07.980][910][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based4088lb # [ 17.178480] envoy[910]: [2026-09-20 12:46:07.980][910][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router4089lb # [ 17.203963] envoy[910]: [2026-09-20 12:46:08.103][910][info][main] [source/server/server.cc:531] HTTP header map info:4090lb # [ 17.205356] envoy[910]: [2026-09-20 12:46:08.158][910][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id4091lb # [ 17.218440] envoy[910]: [2026-09-20 12:46:08.158][910][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:4092lb # [ 17.220082] envoy[910]: [2026-09-20 12:46:08.158][910][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id4093lb # [ 17.227750] envoy[910]: [2026-09-20 12:46:08.158][910][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status4094lb # [ 17.267743] envoy[910]: [2026-09-20 12:46:08.362][910][info][main] [source/server/server.cc:968] runtime: {}4095lb # [ 17.275342] envoy[910]: [2026-09-20 12:46:08.370][910][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.4096lb # [ 17.277558] envoy[910]: [2026-09-20 12:46:08.372][910][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99014097lb # [ 17.280932] envoy[910]: [2026-09-20 12:46:08.375][910][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99014098lb # [ 17.283615] envoy[910]: [2026-09-20 12:46:08.378][910][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration4099lb # [ 17.285309] envoy[910]: [2026-09-20 12:46:08.379][910][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)4100lb # [ 17.286861] envoy[910]: [2026-09-20 12:46:08.379][910][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)4101lb # [ 17.335081] envoy[910]: [2026-09-20 12:46:08.429][910][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)4102worker2 # [ 17.343819] systemd[1]: Reached target Network is Online.4103worker1 # [ 17.275801] systemd[1]: Starting Nix worker-protocol over gRPC...4104worker2 # [ 17.351783] systemd[1]: Starting scheduler-token.service...4105lb # [ 17.356217] envoy[910]: [2026-09-20 12:46:08.450][910][info][config] [source/server/configuration_impl.cc:164] loading stats configuration4106lb # [ 17.358744] envoy[910]: [2026-09-20 12:46:08.453][910][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.4107lb # [ 17.361701] envoy[910]: [2026-09-20 12:46:08.453][910][info][main] [source/server/server.cc:1077] starting main dispatch loop4108worker2 # [ 17.452739] systemd[1]: scheduler-token.service: Deactivated successfully.4109worker2 # [ 17.454570] systemd[1]: Finished scheduler-token.service.4110worker2 # [ 17.455973] systemd[1]: scheduler-token.service: Consumed 29ms CPU time over 102ms wall clock time, 3.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4111worker2 # [ 17.461072] systemd[1]: Starting Nix worker-protocol over gRPC...4112worker1 # [ 17.464622] dhcpcd[755]: eth0: leased 10.0.2.15 for 86400 seconds4113worker1 # [ 17.467680] dhcpcd[755]: eth0: adding route to 10.0.2.0/244114worker1 # [ 17.471418] dhcpcd[755]: eth0: adding default route via 10.0.2.24115worker1 # [ 17.510988] nix-grpc-daemon[795]: ts=2026-09-20T12:46:07Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4116worker1 # [ 17.522105] nix-grpc-daemon[795]: ts=2026-09-20T12:46:08Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4117worker1 # [ 17.529511] nix-grpc-daemon[795]: ts=2026-09-20T12:46:08Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14118worker1 # [ 17.579932] nix-grpc-daemon[795]: ts=2026-09-20T12:46:08Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4119worker1 # [ 17.584709] systemd[1]: Started Nix worker-protocol over gRPC.4120worker2 # [ 17.656454] nix-grpc-daemon[791]: ts=2026-09-20T12:46:08Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4121worker2 # [ 17.665615] nix-grpc-daemon[791]: ts=2026-09-20T12:46:08Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4122worker2 # [ 17.670806] nix-grpc-daemon[791]: ts=2026-09-20T12:46:08Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14123worker1 # [ 17.626508] systemd[1]: Started DHCP Client.4124worker1 # [ 17.629244] systemd[1]: Reached target Multi-User System.4125worker1 # [ 17.630155] systemd[1]: Startup finished in 892ms (kernel) + 4.315s (initrd) + 12.421s (userspace) = 17.628s.4126worker1 # [ 17.637299] nix-grpc-daemon[795]: ts=2026-09-20T12:46:08Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4127lb # [ 17.704562] envoy[910]: [2026-09-20 12:46:08.799][910][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization4128worker2 # [ 17.716270] nix-grpc-daemon[791]: ts=2026-09-20T12:46:08Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4129lb # [ 17.709398] envoy[910]: [2026-09-20 12:46:08.799][910][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized4130worker2 # [ 17.719062] systemd[1]: Started Nix worker-protocol over gRPC.4131lb # [ 17.713420] envoy[910]: [2026-09-20 12:46:08.799][910][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager4132lb # [ 17.718196] envoy[910]: [2026-09-20 12:46:08.803][910][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers4133worker2 # [ 17.774429] nix-grpc-daemon[791]: ts=2026-09-20T12:46:08Z level=info event=scheduler_connected addr=lb:500514134worker2 # [ 17.787094] nix-grpc-daemon[791]: ts=2026-09-20T12:46:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4135worker1 # [ 17.838406] nix-grpc-daemon[795]: ts=2026-09-20T12:46:08Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4136worker2 # [ 17.989796] nix-grpc-daemon[791]: ts=2026-09-20T12:46:08Z level=info event=scheduler_connected addr=lb:500514137worker2 # [ 17.993740] nix-grpc-daemon[791]: ts=2026-09-20T12:46:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4138worker2 # [ 18.194721] nix-grpc-daemon[791]: ts=2026-09-20T12:46:09Z level=info event=scheduler_connected addr=lb:500514139worker2 # [ 18.198632] nix-grpc-daemon[791]: ts=2026-09-20T12:46:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4140worker1 # [ 18.238893] nix-grpc-daemon[795]: ts=2026-09-20T12:46:08Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4141worker2 # [ 18.399796] nix-grpc-daemon[791]: ts=2026-09-20T12:46:09Z level=info event=scheduler_connected addr=lb:500514142worker2 # [ 18.403733] nix-grpc-daemon[791]: ts=2026-09-20T12:46:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4143worker2 # [ 18.604979] nix-grpc-daemon[791]: ts=2026-09-20T12:46:09Z level=info event=scheduler_connected addr=lb:500514144worker2 # [ 18.609303] nix-grpc-daemon[791]: ts=2026-09-20T12:46:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4145worker2 # [ 18.811583] nix-grpc-daemon[791]: ts=2026-09-20T12:46:09Z level=info event=scheduler_connected addr=lb:500514146worker2 # [ 18.815611] nix-grpc-daemon[791]: ts=2026-09-20T12:46:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4147worker2 # [ 19.016857] nix-grpc-daemon[791]: ts=2026-09-20T12:46:09Z level=info event=scheduler_connected addr=lb:500514148worker2 # [ 19.020997] nix-grpc-daemon[791]: ts=2026-09-20T12:46:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4149worker1 # [ 19.042855] nix-grpc-daemon[795]: ts=2026-09-20T12:46:09Z level=info event=scheduler_connected addr=lb:500514150worker1 # [ 19.047573] nix-grpc-daemon[795]: ts=2026-09-20T12:46:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4151worker2 # [ 19.222355] nix-grpc-daemon[791]: ts=2026-09-20T12:46:10Z level=info event=scheduler_connected addr=lb:500514152worker2 # [ 19.226588] nix-grpc-daemon[791]: ts=2026-09-20T12:46:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4153worker1 # [ 19.248755] nix-grpc-daemon[795]: ts=2026-09-20T12:46:09Z level=info event=scheduler_connected addr=lb:500514154worker1 # [ 19.252817] nix-grpc-daemon[795]: ts=2026-09-20T12:46:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4155worker2 # [ 19.427763] nix-grpc-daemon[791]: ts=2026-09-20T12:46:10Z level=info event=scheduler_connected addr=lb:500514156worker2 # [ 19.431786] nix-grpc-daemon[791]: ts=2026-09-20T12:46:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4157worker1 # [ 19.453341] nix-grpc-daemon[795]: ts=2026-09-20T12:46:09Z level=info event=scheduler_connected addr=lb:500514158worker1 # [ 19.455489] nix-grpc-daemon[795]: ts=2026-09-20T12:46:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4159worker2 # [ 19.633255] nix-grpc-daemon[791]: ts=2026-09-20T12:46:10Z level=info event=scheduler_connected addr=lb:500514160worker2 # [ 19.637275] nix-grpc-daemon[791]: ts=2026-09-20T12:46:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4161worker1 # [ 19.656148] nix-grpc-daemon[795]: ts=2026-09-20T12:46:10Z level=info event=scheduler_connected addr=lb:500514162worker1 # [ 19.660185] nix-grpc-daemon[795]: ts=2026-09-20T12:46:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4163worker2 # [ 19.840680] nix-grpc-daemon[791]: ts=2026-09-20T12:46:10Z level=info event=scheduler_connected addr=lb:500514164worker2 # [ 19.844348] nix-grpc-daemon[791]: ts=2026-09-20T12:46:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4165worker1 # [ 19.860653] nix-grpc-daemon[795]: ts=2026-09-20T12:46:10Z level=info event=scheduler_connected addr=lb:500514166worker1 # [ 19.864712] nix-grpc-daemon[795]: ts=2026-09-20T12:46:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4167worker2 # [ 20.041299] nix-grpc-daemon[791]: ts=2026-09-20T12:46:10Z level=info event=scheduler_connected addr=lb:500514168worker2 # [ 20.043175] nix-grpc-daemon[791]: ts=2026-09-20T12:46:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4169worker1 # [ 20.065540] nix-grpc-daemon[795]: ts=2026-09-20T12:46:10Z level=info event=scheduler_connected addr=lb:500514170worker1 # [ 20.067324] nix-grpc-daemon[795]: ts=2026-09-20T12:46:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4171worker2 # [ 20.244492] nix-grpc-daemon[791]: ts=2026-09-20T12:46:11Z level=info event=scheduler_connected addr=lb:500514172worker2 # [ 20.248223] nix-grpc-daemon[791]: ts=2026-09-20T12:46:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4173worker1 # [ 20.269375] nix-grpc-daemon[795]: ts=2026-09-20T12:46:10Z level=info event=scheduler_connected addr=lb:500514174worker1 # [ 20.273082] nix-grpc-daemon[795]: ts=2026-09-20T12:46:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4175worker2 # [ 20.449668] nix-grpc-daemon[791]: ts=2026-09-20T12:46:11Z level=info event=scheduler_connected addr=lb:500514176worker2 # [ 20.454114] nix-grpc-daemon[791]: ts=2026-09-20T12:46:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4177worker1 # [ 20.473944] nix-grpc-daemon[795]: ts=2026-09-20T12:46:10Z level=info event=scheduler_connected addr=lb:500514178worker1 # [ 20.478206] nix-grpc-daemon[795]: ts=2026-09-20T12:46:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4179worker2 # [ 20.655182] nix-grpc-daemon[791]: ts=2026-09-20T12:46:11Z level=info event=scheduler_connected addr=lb:500514180worker2 # [ 20.659369] nix-grpc-daemon[791]: ts=2026-09-20T12:46:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4181worker1 # [ 20.679306] nix-grpc-daemon[795]: ts=2026-09-20T12:46:11Z level=info event=scheduler_connected addr=lb:500514182worker1 # [ 20.683556] nix-grpc-daemon[795]: ts=2026-09-20T12:46:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4183worker2 # [ 20.861563] nix-grpc-daemon[791]: ts=2026-09-20T12:46:11Z level=info event=scheduler_connected addr=lb:500514184worker2 # [ 20.865189] nix-grpc-daemon[791]: ts=2026-09-20T12:46:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4185worker1 # [ 20.884419] nix-grpc-daemon[795]: ts=2026-09-20T12:46:11Z level=info event=scheduler_connected addr=lb:500514186worker1 # [ 20.888490] nix-grpc-daemon[795]: ts=2026-09-20T12:46:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4187worker2 # [ 21.066463] nix-grpc-daemon[791]: ts=2026-09-20T12:46:11Z level=info event=scheduler_connected addr=lb:500514188worker2 # [ 21.070576] nix-grpc-daemon[791]: ts=2026-09-20T12:46:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4189worker1 # [ 21.090940] nix-grpc-daemon[795]: ts=2026-09-20T12:46:11Z level=info event=scheduler_connected addr=lb:500514190worker1 # [ 21.094263] nix-grpc-daemon[795]: ts=2026-09-20T12:46:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4191worker2 # [ 21.271849] nix-grpc-daemon[791]: ts=2026-09-20T12:46:12Z level=info event=scheduler_connected addr=lb:500514192worker2 # [ 21.276080] nix-grpc-daemon[791]: ts=2026-09-20T12:46:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4193worker1 # [ 21.291829] nix-grpc-daemon[795]: ts=2026-09-20T12:46:11Z level=info event=scheduler_connected addr=lb:500514194worker1 # [ 21.295877] nix-grpc-daemon[795]: ts=2026-09-20T12:46:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4195worker2 # [ 21.477646] nix-grpc-daemon[791]: ts=2026-09-20T12:46:12Z level=info event=scheduler_connected addr=lb:500514196worker2 # [ 21.481761] nix-grpc-daemon[791]: ts=2026-09-20T12:46:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4197worker1 # [ 21.496785] nix-grpc-daemon[795]: ts=2026-09-20T12:46:11Z level=info event=scheduler_connected addr=lb:500514198worker1 # [ 21.500730] nix-grpc-daemon[795]: ts=2026-09-20T12:46:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4199worker2 # [ 21.683359] nix-grpc-daemon[791]: ts=2026-09-20T12:46:12Z level=info event=scheduler_connected addr=lb:500514200worker2 # [ 21.687312] nix-grpc-daemon[791]: ts=2026-09-20T12:46:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4201worker1 # [ 21.701243] nix-grpc-daemon[795]: ts=2026-09-20T12:46:12Z level=info event=scheduler_connected addr=lb:500514202worker1 # [ 21.703336] nix-grpc-daemon[795]: ts=2026-09-20T12:46:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4203worker2 # [ 21.889716] nix-grpc-daemon[791]: ts=2026-09-20T12:46:12Z level=info event=scheduler_connected addr=lb:500514204worker2 # [ 21.892959] nix-grpc-daemon[791]: ts=2026-09-20T12:46:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4205worker1 # [ 21.904460] nix-grpc-daemon[795]: ts=2026-09-20T12:46:12Z level=info event=scheduler_connected addr=lb:500514206worker1 # [ 21.908327] nix-grpc-daemon[795]: ts=2026-09-20T12:46:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4207worker2 # [ 22.091280] nix-grpc-daemon[791]: ts=2026-09-20T12:46:13Z level=info event=scheduler_connected addr=lb:500514208worker2 # [ 22.095430] nix-grpc-daemon[791]: ts=2026-09-20T12:46:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4209worker1 # [ 22.110494] nix-grpc-daemon[795]: ts=2026-09-20T12:46:12Z level=info event=scheduler_connected addr=lb:500514210worker1 # [ 22.113652] nix-grpc-daemon[795]: ts=2026-09-20T12:46:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4211worker2 # [ 22.296546] nix-grpc-daemon[791]: ts=2026-09-20T12:46:13Z level=info event=scheduler_connected addr=lb:500514212worker2 # [ 22.300618] nix-grpc-daemon[791]: ts=2026-09-20T12:46:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4213worker1 # [ 22.311291] nix-grpc-daemon[795]: ts=2026-09-20T12:46:12Z level=info event=scheduler_connected addr=lb:500514214worker1 # [ 22.315329] nix-grpc-daemon[795]: ts=2026-09-20T12:46:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4215worker2 # [ 22.501917] nix-grpc-daemon[791]: ts=2026-09-20T12:46:13Z level=info event=scheduler_connected addr=lb:500514216worker2 # [ 22.505863] nix-grpc-daemon[791]: ts=2026-09-20T12:46:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4217worker1 # [ 22.515982] nix-grpc-daemon[795]: ts=2026-09-20T12:46:13Z level=info event=scheduler_connected addr=lb:500514218worker1 # [ 22.520059] nix-grpc-daemon[795]: ts=2026-09-20T12:46:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4219worker2 # [ 22.707103] nix-grpc-daemon[791]: ts=2026-09-20T12:46:13Z level=info event=scheduler_connected addr=lb:500514220worker2 # [ 22.711268] nix-grpc-daemon[791]: ts=2026-09-20T12:46:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4221worker1 # [ 22.720707] nix-grpc-daemon[795]: ts=2026-09-20T12:46:13Z level=info event=scheduler_connected addr=lb:500514222worker1 # [ 22.724302] nix-grpc-daemon[795]: ts=2026-09-20T12:46:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4223worker2 # [ 22.913891] nix-grpc-daemon[791]: ts=2026-09-20T12:46:13Z level=info event=scheduler_connected addr=lb:500514224worker2 # [ 22.917243] nix-grpc-daemon[791]: ts=2026-09-20T12:46:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4225worker1 # [ 22.925221] nix-grpc-daemon[795]: ts=2026-09-20T12:46:13Z level=info event=scheduler_connected addr=lb:500514226worker1 # [ 22.929747] nix-grpc-daemon[795]: ts=2026-09-20T12:46:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4227worker2 # [ 23.114932] nix-grpc-daemon[791]: ts=2026-09-20T12:46:14Z level=info event=scheduler_connected addr=lb:500514228worker2 # [ 23.119752] nix-grpc-daemon[791]: ts=2026-09-20T12:46:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4229worker1 # [ 23.131420] nix-grpc-daemon[795]: ts=2026-09-20T12:46:13Z level=info event=scheduler_connected addr=lb:500514230worker1 # [ 23.135311] nix-grpc-daemon[795]: ts=2026-09-20T12:46:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4231worker2 # [ 23.321213] nix-grpc-daemon[791]: ts=2026-09-20T12:46:14Z level=info event=scheduler_connected addr=lb:500514232worker2 # [ 23.325198] nix-grpc-daemon[791]: ts=2026-09-20T12:46:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4233worker1 # [ 23.335712] nix-grpc-daemon[795]: ts=2026-09-20T12:46:13Z level=info event=scheduler_connected addr=lb:500514234worker1 # [ 23.338780] nix-grpc-daemon[795]: ts=2026-09-20T12:46:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4235worker2 # [ 23.526492] nix-grpc-daemon[791]: ts=2026-09-20T12:46:14Z level=info event=scheduler_connected addr=lb:500514236worker2 # [ 23.530826] nix-grpc-daemon[791]: ts=2026-09-20T12:46:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4237worker1 # [ 23.539607] nix-grpc-daemon[795]: ts=2026-09-20T12:46:14Z level=info event=scheduler_connected addr=lb:500514238worker1 # [ 23.542929] nix-grpc-daemon[795]: ts=2026-09-20T12:46:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4239worker2 # [ 23.731869] nix-grpc-daemon[791]: ts=2026-09-20T12:46:14Z level=info event=scheduler_connected addr=lb:500514240worker2 # [ 23.735986] nix-grpc-daemon[791]: ts=2026-09-20T12:46:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4241worker1 # [ 23.743612] nix-grpc-daemon[795]: ts=2026-09-20T12:46:14Z level=info event=scheduler_connected addr=lb:500514242worker1 # [ 23.747572] nix-grpc-daemon[795]: ts=2026-09-20T12:46:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4243worker2 # [ 23.939592] nix-grpc-daemon[791]: ts=2026-09-20T12:46:14Z level=info event=scheduler_connected addr=lb:500514244worker2 # [ 23.943434] nix-grpc-daemon[791]: ts=2026-09-20T12:46:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4245worker1 # [ 23.948647] nix-grpc-daemon[795]: ts=2026-09-20T12:46:14Z level=info event=scheduler_connected addr=lb:500514246worker1 # [ 23.952584] nix-grpc-daemon[795]: ts=2026-09-20T12:46:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4247worker2 # [ 24.140385] nix-grpc-daemon[791]: ts=2026-09-20T12:46:15Z level=info event=scheduler_connected addr=lb:500514248worker2 # [ 24.144768] nix-grpc-daemon[791]: ts=2026-09-20T12:46:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4249worker1 # [ 24.154948] nix-grpc-daemon[795]: ts=2026-09-20T12:46:14Z level=info event=scheduler_connected addr=lb:500514250worker1 # [ 24.158195] nix-grpc-daemon[795]: ts=2026-09-20T12:46:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4251worker2 # [ 24.342508] nix-grpc-daemon[791]: ts=2026-09-20T12:46:15Z level=info event=scheduler_connected addr=lb:500514252worker2 # [ 24.346566] nix-grpc-daemon[791]: ts=2026-09-20T12:46:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4253worker1 # [ 24.355270] nix-grpc-daemon[795]: ts=2026-09-20T12:46:14Z level=info event=scheduler_connected addr=lb:500514254worker1 # [ 24.356980] nix-grpc-daemon[795]: ts=2026-09-20T12:46:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4255worker2 # [ 24.547796] nix-grpc-daemon[791]: ts=2026-09-20T12:46:15Z level=info event=scheduler_connected addr=lb:500514256worker2 # [ 24.554843] nix-grpc-daemon[791]: ts=2026-09-20T12:46:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4257worker1 # [ 24.557530] nix-grpc-daemon[795]: ts=2026-09-20T12:46:15Z level=info event=scheduler_connected addr=lb:500514258worker1 # [ 24.559387] nix-grpc-daemon[795]: ts=2026-09-20T12:46:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4259worker2 # [ 24.755754] nix-grpc-daemon[791]: ts=2026-09-20T12:46:15Z level=info event=scheduler_connected addr=lb:500514260worker2 # [ 24.759672] nix-grpc-daemon[791]: ts=2026-09-20T12:46:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4261worker1 # [ 24.760719] nix-grpc-daemon[795]: ts=2026-09-20T12:46:15Z level=info event=scheduler_connected addr=lb:500514262worker1 # [ 24.765478] nix-grpc-daemon[795]: ts=2026-09-20T12:46:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4263worker2 # [ 24.961381] nix-grpc-daemon[791]: ts=2026-09-20T12:46:15Z level=info event=scheduler_connected addr=lb:500514264worker2 # [ 24.963380] nix-grpc-daemon[791]: ts=2026-09-20T12:46:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4265worker1 # [ 24.966287] nix-grpc-daemon[795]: ts=2026-09-20T12:46:15Z level=info event=scheduler_connected addr=lb:500514266worker1 # [ 24.970422] nix-grpc-daemon[795]: ts=2026-09-20T12:46:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4267worker2 # [ 25.164293] nix-grpc-daemon[791]: ts=2026-09-20T12:46:16Z level=info event=scheduler_connected addr=lb:500514268worker2 # [ 25.166427] nix-grpc-daemon[791]: ts=2026-09-20T12:46:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4269worker1 # [ 25.172899] nix-grpc-daemon[795]: ts=2026-09-20T12:46:15Z level=info event=scheduler_connected addr=lb:500514270worker1 # [ 25.175980] nix-grpc-daemon[795]: ts=2026-09-20T12:46:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4271worker2 # [ 25.367398] nix-grpc-daemon[791]: ts=2026-09-20T12:46:16Z level=info event=scheduler_connected addr=lb:500514272worker2 # [ 25.369448] nix-grpc-daemon[791]: ts=2026-09-20T12:46:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4273worker1 # [ 25.373449] nix-grpc-daemon[795]: ts=2026-09-20T12:46:15Z level=info event=scheduler_connected addr=lb:500514274worker1 # [ 25.377528] nix-grpc-daemon[795]: ts=2026-09-20T12:46:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4275worker2 # [ 25.570154] nix-grpc-daemon[791]: ts=2026-09-20T12:46:16Z level=info event=scheduler_connected addr=lb:500514276worker2 # [ 25.571940] nix-grpc-daemon[791]: ts=2026-09-20T12:46:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4277worker1 # [ 25.578574] nix-grpc-daemon[795]: ts=2026-09-20T12:46:16Z level=info event=scheduler_connected addr=lb:500514278worker1 # [ 25.582748] nix-grpc-daemon[795]: ts=2026-09-20T12:46:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4279worker2 # [ 25.773221] nix-grpc-daemon[791]: ts=2026-09-20T12:46:16Z level=info event=scheduler_connected addr=lb:500514280worker2 # [ 25.779664] nix-grpc-daemon[791]: ts=2026-09-20T12:46:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4281worker1 # [ 25.783527] nix-grpc-daemon[795]: ts=2026-09-20T12:46:16Z level=info event=scheduler_connected addr=lb:500514282worker1 # [ 25.787544] nix-grpc-daemon[795]: ts=2026-09-20T12:46:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4283worker2 # [ 25.981715] nix-grpc-daemon[791]: ts=2026-09-20T12:46:16Z level=info event=scheduler_connected addr=lb:500514284worker2 # [ 25.985566] nix-grpc-daemon[791]: ts=2026-09-20T12:46:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4285worker1 # [ 25.988555] nix-grpc-daemon[795]: ts=2026-09-20T12:46:16Z level=info event=scheduler_connected addr=lb:500514286worker1 # [ 25.992791] nix-grpc-daemon[795]: ts=2026-09-20T12:46:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4287worker2 # [ 26.187418] nix-grpc-daemon[791]: ts=2026-09-20T12:46:17Z level=info event=scheduler_connected addr=lb:500514288worker2 # [ 26.192561] nix-grpc-daemon[791]: ts=2026-09-20T12:46:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4289worker1 # [ 26.193659] nix-grpc-daemon[795]: ts=2026-09-20T12:46:16Z level=info event=scheduler_connected addr=lb:500514290worker1 # [ 26.195472] nix-grpc-daemon[795]: ts=2026-09-20T12:46:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4291worker2 # [ 26.393324] nix-grpc-daemon[791]: ts=2026-09-20T12:46:17Z level=info event=scheduler_connected addr=lb:500514292worker2 # [ 26.397411] nix-grpc-daemon[791]: ts=2026-09-20T12:46:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4293worker1 # [ 26.396543] nix-grpc-daemon[795]: ts=2026-09-20T12:46:16Z level=info event=scheduler_connected addr=lb:500514294worker1 # [ 26.400389] nix-grpc-daemon[795]: ts=2026-09-20T12:46:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4295worker2 # [ 26.597898] nix-grpc-daemon[791]: ts=2026-09-20T12:46:17Z level=info event=scheduler_connected addr=lb:500514296worker2 # [ 26.601849] nix-grpc-daemon[791]: ts=2026-09-20T12:46:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4297worker1 # [ 26.600756] nix-grpc-daemon[795]: ts=2026-09-20T12:46:17Z level=info event=scheduler_connected addr=lb:500514298worker1 # [ 26.602771] nix-grpc-daemon[795]: ts=2026-09-20T12:46:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4299worker2 # [ 26.802238] nix-grpc-daemon[791]: ts=2026-09-20T12:46:17Z level=info event=scheduler_connected addr=lb:500514300worker2 # [ 26.803943] nix-grpc-daemon[791]: ts=2026-09-20T12:46:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4301worker1 # [ 26.803369] nix-grpc-daemon[795]: ts=2026-09-20T12:46:17Z level=info event=scheduler_connected addr=lb:500514302worker1 # [ 26.805115] nix-grpc-daemon[795]: ts=2026-09-20T12:46:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4303worker2 # [ 27.006729] nix-grpc-daemon[791]: ts=2026-09-20T12:46:17Z level=info event=scheduler_connected addr=lb:500514304worker2 # [ 27.009887] nix-grpc-daemon[791]: ts=2026-09-20T12:46:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4305worker1 # [ 27.006140] nix-grpc-daemon[795]: ts=2026-09-20T12:46:17Z level=info event=scheduler_connected addr=lb:500514306worker1 # [ 27.010103] nix-grpc-daemon[795]: ts=2026-09-20T12:46:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4307worker2 # [ 27.207591] nix-grpc-daemon[791]: ts=2026-09-20T12:46:18Z level=info event=scheduler_connected addr=lb:500514308worker2 # [ 27.211483] nix-grpc-daemon[791]: ts=2026-09-20T12:46:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4309worker1 # [ 27.211832] nix-grpc-daemon[795]: ts=2026-09-20T12:46:17Z level=info event=scheduler_connected addr=lb:500514310worker1 # [ 27.215470] nix-grpc-daemon[795]: ts=2026-09-20T12:46:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4311worker2 # [ 27.412571] nix-grpc-daemon[791]: ts=2026-09-20T12:46:18Z level=info event=scheduler_connected addr=lb:500514312worker2 # [ 27.416568] nix-grpc-daemon[791]: ts=2026-09-20T12:46:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4313worker1 # [ 27.413614] nix-grpc-daemon[795]: ts=2026-09-20T12:46:17Z level=info event=scheduler_connected addr=lb:500514314worker1 # [ 27.417475] nix-grpc-daemon[795]: ts=2026-09-20T12:46:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4315worker2 # [ 27.617736] nix-grpc-daemon[791]: ts=2026-09-20T12:46:18Z level=info event=scheduler_connected addr=lb:500514316worker2 # [ 27.621767] nix-grpc-daemon[791]: ts=2026-09-20T12:46:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4317worker1 # [ 27.618440] nix-grpc-daemon[795]: ts=2026-09-20T12:46:18Z level=info event=scheduler_connected addr=lb:500514318worker1 # [ 27.622779] nix-grpc-daemon[795]: ts=2026-09-20T12:46:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4319worker2 # [ 27.822900] nix-grpc-daemon[791]: ts=2026-09-20T12:46:18Z level=info event=scheduler_connected addr=lb:500514320worker2 # [ 27.827150] nix-grpc-daemon[791]: ts=2026-09-20T12:46:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4321worker1 # [ 27.823554] nix-grpc-daemon[795]: ts=2026-09-20T12:46:18Z level=info event=scheduler_connected addr=lb:500514322worker1 # [ 27.827506] nix-grpc-daemon[795]: ts=2026-09-20T12:46:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4323worker2 # [ 28.029456] nix-grpc-daemon[791]: ts=2026-09-20T12:46:18Z level=info event=scheduler_connected addr=lb:500514324worker2 # [ 28.032622] nix-grpc-daemon[791]: ts=2026-09-20T12:46:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4325worker1 # [ 28.028409] nix-grpc-daemon[795]: ts=2026-09-20T12:46:18Z level=info event=scheduler_connected addr=lb:500514326worker1 # [ 28.032238] nix-grpc-daemon[795]: ts=2026-09-20T12:46:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4327worker2 # [ 28.230290] nix-grpc-daemon[791]: ts=2026-09-20T12:46:19Z level=info event=scheduler_connected addr=lb:500514328worker2 # [ 28.232504] nix-grpc-daemon[791]: ts=2026-09-20T12:46:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4329worker1 # [ 28.233715] nix-grpc-daemon[795]: ts=2026-09-20T12:46:18Z level=info event=scheduler_connected addr=lb:500514330worker1 # [ 28.237435] nix-grpc-daemon[795]: ts=2026-09-20T12:46:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4331worker2 # [ 28.433119] nix-grpc-daemon[791]: ts=2026-09-20T12:46:19Z level=info event=scheduler_connected addr=lb:500514332worker2 # [ 28.434878] nix-grpc-daemon[791]: ts=2026-09-20T12:46:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4333worker1 # [ 28.438114] nix-grpc-daemon[795]: ts=2026-09-20T12:46:18Z level=info event=scheduler_connected addr=lb:500514334worker1 # [ 28.442064] nix-grpc-daemon[795]: ts=2026-09-20T12:46:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4335worker2 # [ 28.636136] nix-grpc-daemon[791]: ts=2026-09-20T12:46:19Z level=info event=scheduler_connected addr=lb:500514336worker2 # [ 28.640122] nix-grpc-daemon[791]: ts=2026-09-20T12:46:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4337worker1 # [ 28.642805] nix-grpc-daemon[795]: ts=2026-09-20T12:46:19Z level=info event=scheduler_connected addr=lb:500514338worker1 # [ 28.647613] nix-grpc-daemon[795]: ts=2026-09-20T12:46:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4339worker2 # [ 28.841166] nix-grpc-daemon[791]: ts=2026-09-20T12:46:19Z level=info event=scheduler_connected addr=lb:500514340worker2 # [ 28.845463] nix-grpc-daemon[791]: ts=2026-09-20T12:46:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4341worker1 # [ 28.848591] nix-grpc-daemon[795]: ts=2026-09-20T12:46:19Z level=info event=scheduler_connected addr=lb:500514342worker1 # [ 28.852718] nix-grpc-daemon[795]: ts=2026-09-20T12:46:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4343worker2 # [ 29.047580] nix-grpc-daemon[791]: ts=2026-09-20T12:46:19Z level=info event=scheduler_connected addr=lb:500514344worker2 # [ 29.050096] nix-grpc-daemon[791]: ts=2026-09-20T12:46:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4345worker1 # [ 29.053822] nix-grpc-daemon[795]: ts=2026-09-20T12:46:19Z level=info event=scheduler_connected addr=lb:500514346worker1 # [ 29.057845] nix-grpc-daemon[795]: ts=2026-09-20T12:46:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4347worker2 # [ 29.251389] nix-grpc-daemon[791]: ts=2026-09-20T12:46:20Z level=info event=scheduler_connected addr=lb:500514348worker2 # [ 29.255235] nix-grpc-daemon[791]: ts=2026-09-20T12:46:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4349worker1 # [ 29.260527] nix-grpc-daemon[795]: ts=2026-09-20T12:46:19Z level=info event=scheduler_connected addr=lb:500514350worker1 # [ 29.263776] nix-grpc-daemon[795]: ts=2026-09-20T12:46:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4351worker2 # [ 29.456111] nix-grpc-daemon[791]: ts=2026-09-20T12:46:20Z level=info event=scheduler_connected addr=lb:500514352worker2 # [ 29.459537] nix-grpc-daemon[791]: ts=2026-09-20T12:46:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4353worker1 # [ 29.461106] nix-grpc-daemon[795]: ts=2026-09-20T12:46:19Z level=info event=scheduler_connected addr=lb:500514354worker1 # [ 29.465153] nix-grpc-daemon[795]: ts=2026-09-20T12:46:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4355worker2 # [ 29.659963] nix-grpc-daemon[791]: ts=2026-09-20T12:46:20Z level=info event=scheduler_connected addr=lb:500514356worker2 # [ 29.662193] nix-grpc-daemon[791]: ts=2026-09-20T12:46:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4357worker1 # [ 29.665779] nix-grpc-daemon[795]: ts=2026-09-20T12:46:20Z level=info event=scheduler_connected addr=lb:500514358worker1 # [ 29.669631] nix-grpc-daemon[795]: ts=2026-09-20T12:46:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4359worker2 # [ 29.863383] nix-grpc-daemon[791]: ts=2026-09-20T12:46:20Z level=info event=scheduler_connected addr=lb:500514360worker2 # [ 29.867889] nix-grpc-daemon[791]: ts=2026-09-20T12:46:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4361worker1 # [ 29.870955] nix-grpc-daemon[795]: ts=2026-09-20T12:46:20Z level=info event=scheduler_connected addr=lb:500514362worker1 # [ 29.875316] nix-grpc-daemon[795]: ts=2026-09-20T12:46:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4363lb # [ 29.996416] rustfs-bucket-start[1041]: mb s3://farm4364lb # [ 30.001690] systemd[1]: Finished rustfs-bucket.service.4365lb # [ 30.008132] systemd[1]: Starting niks3 server...4366worker2 # [ 30.069780] nix-grpc-daemon[791]: ts=2026-09-20T12:46:20Z level=info event=scheduler_connected addr=lb:500514367worker2 # [ 30.073410] nix-grpc-daemon[791]: ts=2026-09-20T12:46:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4368worker1 # [ 30.075980] nix-grpc-daemon[795]: ts=2026-09-20T12:46:20Z level=info event=scheduler_connected addr=lb:500514369worker1 # [ 30.080271] nix-grpc-daemon[795]: ts=2026-09-20T12:46:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4370lb # [ 30.228811] postgres[1057]: [1057] ERROR: relation "goose_db_version" does not exist at character 364371lb # [ 30.230206] postgres[1057]: [1057] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4372worker2 # [ 30.274969] nix-grpc-daemon[791]: ts=2026-09-20T12:46:21Z level=info event=scheduler_connected addr=lb:500514373lb # [ 30.271563] niks3-server[1052]: 2026/09/20 12:46:21 OK 20241026095416_initial_model.sql (22.11ms)4374lb # [ 30.276296] niks3-server[1052]: 2026/09/20 12:46:21 OK 20251210153512_drop_unused_gin_index.sql (2.46ms)4375worker2 # [ 30.279568] nix-grpc-daemon[791]: ts=2026-09-20T12:46:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4376lb # [ 30.281603] niks3-server[1052]: 2026/09/20 12:46:21 OK 20251218171726_add_pins.sql (5.29ms)4377lb # [ 30.287289] niks3-server[1052]: 2026/09/20 12:46:21 OK 20260628120000_add_object_size_and_stats.sql (5.61ms)4378lb # [ 30.291986] niks3-server[1052]: 2026/09/20 12:46:21 OK 20260905000000_add_claims.sql (4.7ms)4379lb # [ 30.298144] niks3-server[1052]: 2026/09/20 12:46:21 OK 20260920000000_drop_claims.sql (5.66ms)4380lb # [ 30.299489] niks3-server[1052]: 2026/09/20 12:46:21 goose: successfully migrated database to version: 202609200000004381lb # [ 30.305930] niks3-server[1052]: 2026/09/20 12:46:21 OK 1_commit_pending_closure.sql (8.17ms)4382lb # [ 30.309084] niks3-server[1052]: 2026/09/20 12:46:21 OK 2_object_stats_trigger.sql (3.09ms)4383lb # [ 30.310597] niks3-server[1052]: 2026/09/20 12:46:21 goose: up to current file version: 24384lb # [ 30.318523] niks3-server[1052]: 2026/09/20 12:46:21 INFO Loaded signing key name=farm-test-1 path=/nix/store/c745bsy529madd2s6hz4z5is2436ni04-key4385worker1 # [ 30.281734] nix-grpc-daemon[795]: ts=2026-09-20T12:46:20Z level=info event=scheduler_connected addr=lb:500514386worker1 # [ 30.285888] nix-grpc-daemon[795]: ts=2026-09-20T12:46:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4387lb # [ 30.401593] niks3-server[1052]: 2026/09/20 12:46:21 INFO Read proxy enabled — serving cache objects from S34388lb # [ 30.403448] niks3-server[1052]: 2026/09/20 12:46:21 INFO Using socket-activated listener address=0.0.0.0:57514389lb # [ 30.409531] systemd[1]: Started niks3 server.4390lb # [ 30.410338] niks3-server[1052]: 2026/09/20 12:46:21 INFO systemd watchdog enabled interval=15s4391lb # [ 30.411622] systemd[1]: Reached target Multi-User System.4392lb # [ 30.413456] systemd[1]: Startup finished in 870ms (kernel) + 4.059s (initrd) + 25.483s (userspace) = 30.413s.4393lb # [ 30.415975] niks3-server[1052]: 2026/09/20 12:46:21 INFO Starting HTTP server address=0.0.0.0:57514394lb # [ 30.423592] niks3-server[1052]: 2026/09/20 12:46:21 INFO lead: acquired remote=192.168.1.4:589324395worker2 # [ 30.437457] nix-grpc-daemon[791]: ts=2026-09-20T12:46:21Z level=info event=scheduler_take_over4396worker2 # [ 30.480502] nix-grpc-daemon[791]: ts=2026-09-20T12:46:21Z level=info event=scheduler_connected addr=lb:500514397worker2 # [ 30.484320] nix-grpc-daemon[791]: ts=2026-09-20T12:46:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4398worker1 # [ 30.486988] nix-grpc-daemon[795]: ts=2026-09-20T12:46:20Z level=info event=scheduler_connected addr=lb:500514399worker1 # [ 30.491466] nix-grpc-daemon[795]: ts=2026-09-20T12:46:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4400worker2 # [ 30.685513] nix-grpc-daemon[791]: ts=2026-09-20T12:46:21Z level=info event=scheduler_connected addr=lb:500514401worker2 # [ 30.689987] nix-grpc-daemon[791]: ts=2026-09-20T12:46:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4402worker1 # [ 30.692223] nix-grpc-daemon[795]: ts=2026-09-20T12:46:21Z level=info event=scheduler_connected addr=lb:500514403worker1 # [ 30.696163] nix-grpc-daemon[795]: ts=2026-09-20T12:46:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4404worker2 # [ 30.890852] nix-grpc-daemon[791]: ts=2026-09-20T12:46:21Z level=info event=scheduler_connected addr=lb:500514405worker2 # [ 30.923835] nix-grpc-daemon[791]: ts=2026-09-20T12:46:21Z level=info event=oidc_jwks provider=mock keys=14406worker2 # [ 30.928437] nix-grpc-daemon[791]: ts=2026-09-20T12:46:21Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:470944407worker2 # [ 30.932950] nix-grpc-daemon[791]: ts=2026-09-20T12:46:21Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04408worker1 # [ 30.897129] nix-grpc-daemon[795]: ts=2026-09-20T12:46:21Z level=info event=scheduler_connected addr=lb:500514409worker2 # [ 30.969843] nix-grpc-daemon[791]: ts=2026-09-20T12:46:21Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:470944410worker2 # [ 30.973572] nix-grpc-daemon[791]: ts=2026-09-20T12:46:21Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04411lb: (finished: waiting for unit niks3.service, in 19.94 seconds)4412lb: waiting for TCP port 5751 on localhost4413lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!4414lb: (finished: waiting for TCP port 5751 on localhost, in 0.08 seconds)4415lb: waiting for unit envoy.service4416lb: (finished: waiting for unit envoy.service, in 0.05 seconds)4417lb: waiting for TCP port 50051 on localhost4418lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!4419lb # [ 31.606315] envoy[910]: conn peer=[::1]:48372 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=34420lb: (finished: waiting for TCP port 50051 on localhost, in 0.04 seconds)4421worker2 # [ 31.750126] systemd[1]: Starting scheduler-token.service...4422worker2 # [ 31.820234] systemd[1]: scheduler-token.service: Deactivated successfully.4423worker2 # [ 31.821958] systemd[1]: Finished scheduler-token.service.4424worker2 # [ 31.823409] systemd[1]: scheduler-token.service: Consumed 21ms CPU time over 73ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4425lb: waiting until exactly no endpoints of x86_64-linux are unhealthy4426??? Warning (UserWarning): retry(): The 'timeout_seconds' argument is deprecated. Use 'timeout' instead.4427 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 1284428lb: must succeed: curl -sf localhost:9901/clusters4429lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4430lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.06 seconds)4431subtest: one node schedules and both builders hold a WorkerSession on it4432lb: must succeed: curl -sf localhost:9901/clusters4433lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4434worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4435worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4436lb: must succeed: curl -sf localhost:9901/clusters4437lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4438lb: must succeed: curl -sf localhost:9901/clusters4439lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4440worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'4441worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.05 seconds)4442(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.24 seconds)4443subtest: balancer auth: client cert, bearer token, nothing, unknown CN4444worker2 # [ 32.455957] nix-grpc-daemon[791]: ts=2026-09-20T12:46:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:471044445worker2 # [ 32.468894] systemd[1]: Started Nix Daemon.4446worker2 # [ 32.680909] nix-daemon[860]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4447worker2 # [ 32.730952] nix-grpc-daemon[791]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 38 ms (attempt 1/5)4448worker2 # [ 32.771968] nix-grpc-daemon[791]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 23 ms (attempt 2/5)4449worker2 # [ 32.797645] nix-grpc-daemon[791]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 226 ms (attempt 3/5)4450worker2 # [ 33.024477] nix-grpc-daemon[791]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 114 ms (attempt 4/5)4451worker2 # [ 33.144302] nix-grpc-daemon[791]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4452worker2 # [ 33.175751] nix-grpc-daemon[791]: ts=2026-09-20T12:46:24Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:471044453worker2 # [ 33.192919] nix-grpc-daemon[791]: ts=2026-09-20T12:46:24Z level=debug event=path_info_miss path=vkg050c808kjr1y99c83mq9d036s55gg-nixos-system-client-test error="error: path '/nix/store/vkg050c808kjr1y99c83mq9d036s55gg-nixos-system-client-test' is required, but there is no substituter that can build it"4454worker1 # [ 33.138612] nix-grpc-daemon[795]: ts=2026-09-20T12:46:23Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:376004455worker1 # [ 33.144588] systemd[1]: Started Nix Daemon.4456worker1 # [ 33.347648] nix-daemon[908]: accepted connection from pid 795, user nix-grpc-daemon (trusted)4457worker1 # [ 33.379198] nix-grpc-daemon[795]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 16 ms (attempt 1/5)4458worker1 # [ 33.397605] nix-grpc-daemon[795]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 147 ms (attempt 2/5)4459worker1 # [ 33.547456] nix-grpc-daemon[795]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 80 ms (attempt 3/5)4460worker1 # [ 33.633187] nix-grpc-daemon[795]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 551 ms (attempt 4/5)4461worker1 # [ 34.196976] nix-grpc-daemon[795]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4462worker1 # [ 34.216383] nix-grpc-daemon[795]: ts=2026-09-20T12:46:24Z level=debug event=path_info_miss path=vkg050c808kjr1y99c83mq9d036s55gg-nixos-system-client-test error="error: path '/nix/store/vkg050c808kjr1y99c83mq9d036s55gg-nixos-system-client-test' is required, but there is no substituter that can build it"4463client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4464client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.07 seconds)4465client: must succeed: install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4466client: (finished: must succeed: install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds)4467worker2 # [ 34.647359] nix-grpc-daemon[791]: ts=2026-09-20T12:46:25Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:471044468worker1 # [ 34.604078] nix-grpc-daemon[795]: ts=2026-09-20T12:46:25Z level=info event=oidc_jwks provider=mock keys=14469worker1 # [ 34.606497] nix-grpc-daemon[795]: ts=2026-09-20T12:46:25Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:376004470worker1 # [ 34.612443] nix-grpc-daemon[795]: ts=2026-09-20T12:46:25Z level=debug event=path_info_miss path=vkg050c808kjr1y99c83mq9d036s55gg-nixos-system-client-test error="error: path '/nix/store/vkg050c808kjr1y99c83mq9d036s55gg-nixos-system-client-test' is required, but there is no substituter that can build it"4471worker2 # [ 34.687113] nix-grpc-daemon[791]: ts=2026-09-20T12:46:25Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:471044472worker2 # [ 34.696390] nix-grpc-daemon[791]: ts=2026-09-20T12:46:25Z level=debug event=path_info_miss path=vkg050c808kjr1y99c83mq9d036s55gg-nixos-system-client-test error="error: path '/nix/store/vkg050c808kjr1y99c83mq9d036s55gg-nixos-system-client-test' is required, but there is no substituter that can build it"4473client: must succeed: rm -r /var/lib/nix-grpc-store4474client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.02 seconds)4475worker1 # [ 34.858387] nix-grpc-daemon[795]: ts=2026-09-20T12:46:25Z level=info event=denied method=QueryMissing cn=- role=none4476worker2 # [ 35.148350] nix-grpc-daemon[791]: ts=2026-09-20T12:46:26Z level=info event=denied method=QueryMissing cn=stranger role=none4477(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.03 seconds)4478subtest: DAG build is scheduled across workers and lands in the cache4479client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24480worker2 # [ 35.388310] nix-grpc-daemon[791]: ts=2026-09-20T12:46:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:471044481client # don't know how to build these paths:4482client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4483worker2 # [ 35.402158] nix-grpc-daemon[791]: ts=2026-09-20T12:46:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471044484worker2 # [ 35.441624] nix-grpc-daemon[791]: ts=2026-09-20T12:46:26Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04485worker2 # [ 35.446768] nix-grpc-daemon[791]: ts=2026-09-20T12:46:26Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.4:50051 clients=14486worker2 # [ 35.491188] nix-grpc-daemon[791]: ts=2026-09-20T12:46:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471044487worker2 # [ 35.509941] nix-grpc-daemon[791]: ts=2026-09-20T12:46:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471044488client # copying 1 paths to 'lb:50051'...4489worker2 # [ 35.523268] nix-grpc-daemon[791]: ts=2026-09-20T12:46:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:471044490client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4491worker2 # [ 35.548925] nix-daemon[860]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4492worker2 # [ 35.577074] nix-grpc-daemon[791]: ts=2026-09-20T12:46:26Z level=info event=niks3_push_started pid=8724493worker2 # [ 35.828527] nix-grpc-daemon[872]: time=2026-09-20T12:46:26.733Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""4494worker2 # [ 36.040129] nix-daemon[860]: accepted connection from pid 884, user nix-grpc-daemon (trusted)4495worker2 # [ 36.059315] nix-daemon[860]: reaped child process 887, status = succeeded4496worker2 # [ 36.211624] nix-daemon[860]: accepted connection from pid 890, user nix-grpc-daemon (trusted)4497worker2 # [ 36.223760] nix-daemon[860]: reaped child process 893, status = succeeded4498lb # [ 36.232636] niks3-server[1052]: 2026/09/20 12:46:27 INFO Received uploads request method=POST path=/api/pending_closures4499worker2 # [ 36.266777] nix-grpc-daemon[872]: time=2026-09-20T12:46:27.176Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4500worker2 # [ 36.268948] nix-grpc-daemon[872]: time=2026-09-20T12:46:27.178Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4501lb # [ 36.307309] niks3-server[1052]: 2026/09/20 12:46:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4502lb # [ 36.310678] niks3-server[1052]: 2026/09/20 12:46:27 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4503lb # [ 36.326854] niks3-server[1052]: 2026/09/20 12:46:27 INFO Signed narinfos id=1 count=14504lb # [ 36.328705] niks3-server[1052]: 2026/09/20 12:46:27 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4505worker2 # [ 36.334771] nix-grpc-daemon[872]: time=2026-09-20T12:46:27.243Z level=INFO msg="Uploading 1 narinfos"4506lb # [ 36.338908] niks3-server[1052]: 2026/09/20 12:46:27 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4507lb # [ 36.348561] niks3-server[1052]: 2026/09/20 12:46:27 INFO Completed upload id=14508worker2 # [ 36.354653] nix-grpc-daemon[872]: time=2026-09-20T12:46:27.263Z level=INFO msg="Upload complete. (509ms)"4509lb # [ 36.354753] niks3-server[1052]: 2026/09/20 12:46:27 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo4510worker2 # [ 36.359934] nix-grpc-daemon[791]: ts=2026-09-20T12:46:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47104 duration_s=0 paths=1 nar_bytes_in=5044511worker2 # [ 36.367227] nix-grpc-daemon[791]: ts=2026-09-20T12:46:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471044512worker2 # [ 36.379117] nix-daemon[860]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4513worker2 # [ 36.386468] nix-daemon[860]: reaped child process 870, status = succeeded4514worker2 # [ 36.395504] nix-daemon[860]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4515client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4516client # farm-a> worker2: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4517lb # [ 37.342555] envoy[910]: conn peer=[2001:db8:1::1]:60474 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=749 tx=423 ms=18954518lb # [ 37.346460] envoy[910]: conn peer=[2001:db8:1::1]:47540 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=614519lb # [ 37.350112] envoy[910]: conn peer=[2001:db8:1::1]:47548 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=104520lb # [ 37.353689] envoy[910]: conn peer=[2001:db8:1::1]:47554 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=104521client # farm-a> LOG-farm-a-t14522worker2 # [ 38.013430] nix-daemon[860]: reaped child process 897, status = succeeded4523worker2 # [ 38.153178] nix-daemon[860]: accepted connection from pid 912, user nix-grpc-daemon (trusted)4524worker2 # [ 38.165868] nix-daemon[860]: reaped child process 915, status = succeeded4525lb # [ 38.175507] niks3-server[1052]: 2026/09/20 12:46:29 INFO Received uploads request method=POST path=/api/pending_closures4526worker2 # [ 38.193533] nix-grpc-daemon[872]: time=2026-09-20T12:46:29.102Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4527worker2 # [ 38.195338] nix-grpc-daemon[872]: time=2026-09-20T12:46:29.104Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4528lb # [ 38.223760] niks3-server[1052]: 2026/09/20 12:46:29 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4529lb # [ 38.231951] niks3-server[1052]: 2026/09/20 12:46:29 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4530lb # [ 38.242590] niks3-server[1052]: 2026/09/20 12:46:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4531worker2 # [ 38.253619] nix-grpc-daemon[872]: time=2026-09-20T12:46:29.162Z level=INFO msg="Uploading 1 narinfos"4532lb # [ 38.247143] niks3-server[1052]: 2026/09/20 12:46:29 INFO Signed narinfos id=2 count=14533lb # [ 38.252528] niks3-server[1052]: 2026/09/20 12:46:29 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4534lb # [ 38.261168] niks3-server[1052]: 2026/09/20 12:46:29 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4535lb # [ 38.263881] niks3-server[1052]: 2026/09/20 12:46:29 INFO Completed upload id=24536worker2 # [ 38.270231] nix-grpc-daemon[872]: time=2026-09-20T12:46:29.179Z level=INFO msg="Upload complete. (263ms)"4537lb # [ 38.266250] niks3-server[1052]: 2026/09/20 12:46:29 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4538worker2 # [ 38.280343] nix-grpc-daemon[791]: ts=2026-09-20T12:46:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47104 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14539worker2 # [ 38.295160] nix-grpc-daemon[791]: ts=2026-09-20T12:46:29Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04540worker2 # [ 38.297621] nix-grpc-daemon[791]: ts=2026-09-20T12:46:29Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.4:50051 clients=14541worker2 # [ 38.301721] nix-daemon[860]: reaped child process 895, status = succeeded4542worker2 # [ 38.342828] nix-grpc-daemon[791]: ts=2026-09-20T12:46:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471044543worker2 # [ 38.355297] nix-grpc-daemon[791]: ts=2026-09-20T12:46:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471044544worker2 # [ 38.381551] nix-grpc-daemon[872]: time=2026-09-20T12:46:29.290Z level=INFO msg="All 1 paths already cached"4545client # copying 1 paths to 'lb:50051'...4546worker2 # [ 38.389609] nix-grpc-daemon[791]: ts=2026-09-20T12:46:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:471044547client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4548worker2 # [ 38.414908] nix-daemon[860]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4549worker2 # [ 38.592173] nix-daemon[860]: accepted connection from pid 921, user nix-grpc-daemon (trusted)4550worker2 # [ 38.605509] nix-daemon[860]: reaped child process 924, status = succeeded4551worker2 # [ 38.766318] nix-daemon[860]: accepted connection from pid 927, user nix-grpc-daemon (trusted)4552worker2 # [ 38.780991] nix-daemon[860]: reaped child process 930, status = succeeded4553lb # [ 38.790170] niks3-server[1052]: 2026/09/20 12:46:29 INFO Received uploads request method=POST path=/api/pending_closures4554worker2 # [ 38.810609] nix-grpc-daemon[872]: time=2026-09-20T12:46:29.719Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4555worker2 # [ 38.812023] nix-grpc-daemon[872]: time=2026-09-20T12:46:29.719Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4556lb # [ 38.824910] niks3-server[1052]: 2026/09/20 12:46:29 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4557lb # [ 38.835436] niks3-server[1052]: 2026/09/20 12:46:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4558lb # [ 38.840974] niks3-server[1052]: 2026/09/20 12:46:29 INFO Signed narinfos id=3 count=14559worker2 # [ 38.852396] nix-grpc-daemon[872]: time=2026-09-20T12:46:29.761Z level=INFO msg="Uploading 1 narinfos"4560lb # [ 38.853237] niks3-server[1052]: 2026/09/20 12:46:29 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4561lb # [ 38.867363] niks3-server[1052]: 2026/09/20 12:46:29 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4562lb # [ 38.871407] niks3-server[1052]: 2026/09/20 12:46:29 INFO Completed upload id=34563worker2 # [ 38.877376] nix-grpc-daemon[872]: time=2026-09-20T12:46:29.786Z level=INFO msg="Upload complete. (446ms)"4564lb # [ 38.873264] niks3-server[1052]: 2026/09/20 12:46:29 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4565worker2 # [ 38.882152] nix-grpc-daemon[791]: ts=2026-09-20T12:46:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47104 duration_s=0 paths=1 nar_bytes_in=6244566worker2 # [ 38.886734] nix-grpc-daemon[791]: ts=2026-09-20T12:46:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471044567worker2 # [ 38.896216] nix-daemon[860]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4568worker2 # [ 38.906136] nix-daemon[860]: reaped child process 919, status = succeeded4569worker2 # [ 38.909278] nix-daemon[860]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4570client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4571client # farm-b> worker2: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4572client # farm-b> LOG-farm-b-t14573worker2 # [ 40.537562] nix-daemon[860]: reaped child process 934, status = succeeded4574worker2 # [ 40.697338] nix-daemon[860]: accepted connection from pid 950, user nix-grpc-daemon (trusted)4575worker2 # [ 40.711699] nix-daemon[860]: reaped child process 953, status = succeeded4576lb # [ 40.720728] niks3-server[1052]: 2026/09/20 12:46:31 INFO Received uploads request method=POST path=/api/pending_closures4577worker2 # [ 40.732857] nix-grpc-daemon[872]: time=2026-09-20T12:46:31.641Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4578worker2 # [ 40.735071] nix-grpc-daemon[872]: time=2026-09-20T12:46:31.643Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4579lb # [ 40.742150] niks3-server[1052]: 2026/09/20 12:46:31 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4580lb # [ 40.746274] niks3-server[1052]: 2026/09/20 12:46:31 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4581lb # [ 40.749507] niks3-server[1052]: 2026/09/20 12:46:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4582lb # [ 40.752107] niks3-server[1052]: 2026/09/20 12:46:31 INFO Signed narinfos id=4 count=14583worker2 # [ 40.758371] nix-grpc-daemon[872]: time=2026-09-20T12:46:31.667Z level=INFO msg="Uploading 1 narinfos"4584lb # [ 40.757758] niks3-server[1052]: 2026/09/20 12:46:31 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4585lb # [ 40.763666] niks3-server[1052]: 2026/09/20 12:46:31 INFO Completed upload id=44586worker2 # [ 40.769751] nix-grpc-daemon[872]: time=2026-09-20T12:46:31.678Z level=INFO msg="Upload complete. (251ms)"4587lb # [ 40.764900] niks3-server[1052]: 2026/09/20 12:46:31 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4588lb # [ 40.767886] niks3-server[1052]: 2026/09/20 12:46:31 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4589worker2 # [ 40.772519] nix-grpc-daemon[791]: ts=2026-09-20T12:46:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47104 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14590worker2 # [ 40.779520] nix-grpc-daemon[791]: ts=2026-09-20T12:46:31Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04591worker2 # [ 40.781591] nix-grpc-daemon[791]: ts=2026-09-20T12:46:31Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.4:50051 clients=14592worker2 # [ 40.786137] nix-daemon[860]: reaped child process 932, status = succeeded4593worker2 # [ 40.822838] nix-grpc-daemon[791]: ts=2026-09-20T12:46:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471044594worker2 # [ 40.836492] nix-grpc-daemon[791]: ts=2026-09-20T12:46:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471044595worker2 # [ 40.845752] nix-grpc-daemon[872]: time=2026-09-20T12:46:31.754Z level=INFO msg="All 2 paths already cached"4596client # copying 1 paths to 'lb:50051'...4597worker2 # [ 40.852847] nix-grpc-daemon[791]: ts=2026-09-20T12:46:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:471044598client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4599worker2 # [ 40.874194] nix-daemon[860]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4600worker2 # [ 41.038757] nix-daemon[860]: accepted connection from pid 959, user nix-grpc-daemon (trusted)4601worker2 # [ 41.055810] nix-daemon[860]: reaped child process 962, status = succeeded4602worker2 # [ 41.212970] nix-daemon[860]: accepted connection from pid 964, user nix-grpc-daemon (trusted)4603worker2 # [ 41.227145] nix-daemon[860]: reaped child process 967, status = succeeded4604lb # [ 41.230329] niks3-server[1052]: 2026/09/20 12:46:32 INFO Received uploads request method=POST path=/api/pending_closures4605worker2 # [ 41.241167] nix-grpc-daemon[872]: time=2026-09-20T12:46:32.150Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4606worker2 # [ 41.242543] nix-grpc-daemon[872]: time=2026-09-20T12:46:32.150Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4607lb # [ 41.244476] niks3-server[1052]: 2026/09/20 12:46:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4608worker2 # [ 41.254599] nix-grpc-daemon[872]: time=2026-09-20T12:46:32.163Z level=INFO msg="Uploading 1 narinfos"4609lb # [ 41.248717] niks3-server[1052]: 2026/09/20 12:46:32 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4610lb # [ 41.250656] niks3-server[1052]: 2026/09/20 12:46:32 INFO Signed narinfos id=5 count=14611lb # [ 41.253557] niks3-server[1052]: 2026/09/20 12:46:32 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4612lb # [ 41.258165] niks3-server[1052]: 2026/09/20 12:46:32 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4613lb # [ 41.264196] niks3-server[1052]: 2026/09/20 12:46:32 INFO Completed upload id=54614worker2 # [ 41.270231] nix-grpc-daemon[872]: time=2026-09-20T12:46:32.179Z level=INFO msg="Upload complete. (383ms)"4615lb # [ 41.265589] niks3-server[1052]: 2026/09/20 12:46:32 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4616worker2 # [ 41.273096] nix-grpc-daemon[791]: ts=2026-09-20T12:46:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47104 duration_s=0 paths=1 nar_bytes_in=7524617worker2 # [ 41.275806] nix-grpc-daemon[791]: ts=2026-09-20T12:46:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471044618worker2 # [ 41.280844] nix-daemon[860]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4619worker2 # [ 41.285395] nix-daemon[860]: reaped child process 957, status = succeeded4620client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4621client # farm-top> worker2: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4622worker2 # [ 41.291145] nix-daemon[860]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4623worker2 # [ 42.304837] nix-daemon[860]: reaped child process 971, status = succeeded4624client # farm-top> LOG-farm-top-t14625worker2 # [ 42.459125] nix-daemon[860]: accepted connection from pid 986, user nix-grpc-daemon (trusted)4626worker2 # [ 42.471602] nix-daemon[860]: reaped child process 990, status = succeeded4627lb # [ 42.479642] niks3-server[1052]: 2026/09/20 12:46:33 INFO Received uploads request method=POST path=/api/pending_closures4628worker2 # [ 42.491306] nix-grpc-daemon[872]: time=2026-09-20T12:46:33.398Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4629worker2 # [ 42.492736] nix-grpc-daemon[872]: time=2026-09-20T12:46:33.399Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4630lb # [ 42.495097] niks3-server[1052]: 2026/09/20 12:46:33 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4631lb # [ 42.498805] niks3-server[1052]: 2026/09/20 12:46:33 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4632lb # [ 42.502828] niks3-server[1052]: 2026/09/20 12:46:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4633lb # [ 42.505142] niks3-server[1052]: 2026/09/20 12:46:33 INFO Signed narinfos id=6 count=14634worker2 # [ 42.512543] nix-grpc-daemon[872]: time=2026-09-20T12:46:33.421Z level=INFO msg="Uploading 1 narinfos"4635lb # [ 42.509379] niks3-server[1052]: 2026/09/20 12:46:33 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4636lb # [ 42.513698] niks3-server[1052]: 2026/09/20 12:46:33 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4637lb # [ 42.517044] niks3-server[1052]: 2026/09/20 12:46:33 INFO Completed upload id=64638worker2 # [ 42.524513] nix-grpc-daemon[872]: time=2026-09-20T12:46:33.433Z level=INFO msg="Upload complete. (227ms)"4639lb # [ 42.520419] niks3-server[1052]: 2026/09/20 12:46:33 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4640worker2 # [ 42.527260] nix-grpc-daemon[791]: ts=2026-09-20T12:46:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47104 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14641worker2 # [ 42.534543] nix-daemon[860]: reaped child process 969, status = succeeded4642client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 7.39 seconds)4643client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4644client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.18 seconds)4645client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4646client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.02 seconds)4647client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4648client # copying 3 paths...4649client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4650client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4651client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4652client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.25 seconds)4653(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 7.95 seconds)4654subtest: repeat is answered Cached by the scheduler without building4655worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4656worker1 # finding garbage collector roots...4657worker1 # deleting specified paths...4658worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4659worker1 # deleting unused links...4660worker1 # note: hard linking is currently saving 0.0 KiB4661worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.27 seconds)4662worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4663worker2 # finding garbage collector roots...4664worker2 # deleting specified paths...4665worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4666worker2 # deleting unused links...4667worker2 # note: hard linking is currently saving 0.0 KiB4668worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.25 seconds)4669client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24670worker2 # [ 43.871390] nix-grpc-daemon[791]: ts=2026-09-20T12:46:34Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:471044671client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4672client # /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4673worker2 # [ 43.891157] nix-grpc-daemon[791]: ts=2026-09-20T12:46:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471044674worker2 # [ 43.911931] nix-grpc-daemon[872]: time=2026-09-20T12:46:34.820Z level=INFO msg="All 2 paths already cached"4675worker2 # [ 43.926464] nix-grpc-daemon[791]: ts=2026-09-20T12:46:34Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14676client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 0.32 seconds)4677client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4678client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.19 seconds)4679lb: must succeed: curl -sf localhost:9901/clusters4680lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4681lb: must succeed: curl -sf localhost:9901/clusters4682lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4683worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4684worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4685(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.26 seconds)4686subtest: inputs already in the cache are not sent to the scheduler4687lb: must succeed: curl -sf localhost:9901/clusters4688lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4689lb: must succeed: curl -sf localhost:9901/clusters4690lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4691worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4692worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4693client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&24694worker2 # [ 44.728619] nix-grpc-daemon[791]: ts=2026-09-20T12:46:35Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:471044695client # don't know how to build these paths:4696client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4697worker1 # [ 44.665636] nix-grpc-daemon[795]: ts=2026-09-20T12:46:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376004698worker2 # [ 44.753346] nix-grpc-daemon[791]: ts=2026-09-20T12:46:35Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04699worker2 # [ 44.755446] nix-grpc-daemon[791]: ts=2026-09-20T12:46:35Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.4:50051 clients=14700worker2 # [ 44.798058] nix-grpc-daemon[791]: ts=2026-09-20T12:46:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471044701worker2 # [ 44.807200] nix-grpc-daemon[791]: ts=2026-09-20T12:46:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471044702worker2 # [ 44.817796] nix-grpc-daemon[872]: time=2026-09-20T12:46:35.727Z level=INFO msg="All 2 paths already cached"4703client # copying 1 paths to 'lb:50051'...4704worker2 # [ 44.823496] nix-grpc-daemon[791]: ts=2026-09-20T12:46:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:471044705client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4706worker2 # [ 44.828988] nix-daemon[860]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4707worker2 # [ 44.990816] nix-daemon[860]: accepted connection from pid 1019, user nix-grpc-daemon (trusted)4708worker2 # [ 45.005613] nix-daemon[860]: reaped child process 1022, status = succeeded4709worker2 # [ 45.172560] nix-daemon[860]: accepted connection from pid 1024, user nix-grpc-daemon (trusted)4710worker2 # [ 45.184894] nix-daemon[860]: reaped child process 1027, status = succeeded4711lb # [ 45.195040] niks3-server[1052]: 2026/09/20 12:46:36 INFO Received uploads request method=POST path=/api/pending_closures4712worker2 # [ 45.205851] nix-grpc-daemon[872]: time=2026-09-20T12:46:36.114Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4713worker2 # [ 45.207460] nix-grpc-daemon[872]: time=2026-09-20T12:46:36.116Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4714lb # [ 45.212382] niks3-server[1052]: 2026/09/20 12:46:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4715lb # [ 45.214611] niks3-server[1052]: 2026/09/20 12:46:36 INFO Signed narinfos id=7 count=14716worker2 # [ 45.222301] nix-grpc-daemon[872]: time=2026-09-20T12:46:36.131Z level=INFO msg="Uploading 1 narinfos"4717lb # [ 45.218943] niks3-server[1052]: 2026/09/20 12:46:36 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4718lb # [ 45.223708] niks3-server[1052]: 2026/09/20 12:46:36 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4719lb # [ 45.226816] niks3-server[1052]: 2026/09/20 12:46:36 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4720lb # [ 45.230772] niks3-server[1052]: 2026/09/20 12:46:36 INFO Completed upload id=74721worker2 # [ 45.236729] nix-grpc-daemon[872]: time=2026-09-20T12:46:36.145Z level=INFO msg="Upload complete. (401ms)"4722lb # [ 45.232416] niks3-server[1052]: 2026/09/20 12:46:36 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4723worker2 # [ 45.238922] nix-grpc-daemon[791]: ts=2026-09-20T12:46:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47104 duration_s=0 paths=1 nar_bytes_in=7524724worker2 # [ 45.243208] nix-grpc-daemon[791]: ts=2026-09-20T12:46:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471044725worker2 # [ 45.248981] nix-daemon[860]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4726worker2 # [ 45.251143] nix-daemon[860]: reaped child process 1017, status = succeeded4727worker2 # [ 45.256754] nix-daemon[860]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4728client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4729client # farm-top> worker2: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4730client # farm-top> LOG-farm-top-t1b4731worker2 # [ 46.076625] nix-daemon[860]: reaped child process 1031, status = succeeded4732worker2 # [ 46.221828] nix-daemon[860]: accepted connection from pid 1046, user nix-grpc-daemon (trusted)4733worker2 # [ 46.234655] nix-daemon[860]: reaped child process 1049, status = succeeded4734lb # [ 46.240521] niks3-server[1052]: 2026/09/20 12:46:37 INFO Received uploads request method=POST path=/api/pending_closures4735worker2 # [ 46.258312] nix-grpc-daemon[872]: time=2026-09-20T12:46:37.167Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4736worker2 # [ 46.260169] nix-grpc-daemon[872]: time=2026-09-20T12:46:37.168Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4737lb # [ 46.277730] niks3-server[1052]: 2026/09/20 12:46:37 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4738lb # [ 46.290164] niks3-server[1052]: 2026/09/20 12:46:37 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4739lb # [ 46.302153] niks3-server[1052]: 2026/09/20 12:46:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4740worker2 # [ 46.313886] nix-grpc-daemon[872]: time=2026-09-20T12:46:37.223Z level=INFO msg="Uploading 1 narinfos"4741lb # [ 46.307844] niks3-server[1052]: 2026/09/20 12:46:37 INFO Signed narinfos id=8 count=14742lb # [ 46.313164] niks3-server[1052]: 2026/09/20 12:46:37 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4743lb # [ 46.321222] niks3-server[1052]: 2026/09/20 12:46:37 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4744lb # [ 46.327581] niks3-server[1052]: 2026/09/20 12:46:37 INFO Completed upload id=84745worker2 # [ 46.333740] nix-grpc-daemon[872]: time=2026-09-20T12:46:37.242Z level=INFO msg="Upload complete. (269ms)"4746lb # [ 46.330243] niks3-server[1052]: 2026/09/20 12:46:37 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4747worker2 # [ 46.336832] nix-grpc-daemon[791]: ts=2026-09-20T12:46:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47104 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14748worker2 # [ 46.343862] nix-daemon[860]: reaped child process 1029, status = succeeded4749client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&2, in 1.86 seconds)4750client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath4751client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.19 seconds)4752lb: must succeed: curl -sf localhost:9901/clusters4753lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4754lb: must succeed: curl -sf localhost:9901/clusters4755lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4756worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4757worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.03 seconds)4758(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.33 seconds)4759subtest: two clients wanting the same drv build it once4760client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4761client # [ 46.762760] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4762client # Running as unit: dedup1.service; invocation ID: e2d0f888235c46c5b22865caea6700ef4763client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 0.07 seconds)4764client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4765worker1 # [ 46.899284] nix-grpc-daemon[795]: ts=2026-09-20T12:46:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:376004766client # [ 46.976361] nix[894]: don't know how to build these paths:4767client # [ 46.977273] nix[894]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4768worker2 # [ 46.985717] nix-grpc-daemon[791]: ts=2026-09-20T12:46:37Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04769worker2 # [ 46.990639] nix-grpc-daemon[791]: ts=2026-09-20T12:46:37Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.3:50051 clients=14770worker1 # [ 46.922534] nix-grpc-daemon[795]: ts=2026-09-20T12:46:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376004771worker1 # [ 46.937490] nix-grpc-daemon[795]: ts=2026-09-20T12:46:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376004772client # [ 47.017107] nix[894]: copying 1 paths to 'lb:50051'...4773worker1 # [ 46.948218] nix-grpc-daemon[795]: ts=2026-09-20T12:46:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376004774client # [ 47.018513] nix[894]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4775worker1 # [ 46.956625] nix-daemon[908]: accepted connection from pid 795, user nix-grpc-daemon (trusted)4776worker1 # [ 46.968871] nix-grpc-daemon[795]: ts=2026-09-20T12:46:37Z level=info event=niks3_push_started pid=9344777worker1 # [ 47.235723] nix-grpc-daemon[934]: time=2026-09-20T12:46:37.718Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""4778lb # [ 47.343507] envoy[910]: conn peer=[2001:db8:1::1]:47558 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9524 tx=3842 ms=71804779lb # [ 47.347701] envoy[910]: conn peer=[2001:db8:1::1]:53062 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=694780lb # [ 47.352108] envoy[910]: conn peer=[2001:db8:1::1]:53064 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4152 tx=1685 ms=16504781worker1 # [ 47.446498] nix-daemon[908]: accepted connection from pid 946, user nix-grpc-daemon (trusted)4782worker1 # [ 47.464104] nix-daemon[908]: reaped child process 949, status = succeeded4783worker1 # [ 47.626654] nix-daemon[908]: accepted connection from pid 951, user nix-grpc-daemon (trusted)4784worker1 # [ 47.639974] nix-daemon[908]: reaped child process 954, status = succeeded4785lb # [ 47.721222] niks3-server[1052]: 2026/09/20 12:46:38 INFO Received uploads request method=POST path=/api/pending_closures4786worker1 # [ 47.661354] nix-grpc-daemon[934]: time=2026-09-20T12:46:38.147Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4787worker1 # [ 47.664147] nix-grpc-daemon[934]: time=2026-09-20T12:46:38.150Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4788lb # [ 47.744860] niks3-server[1052]: 2026/09/20 12:46:38 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4789lb # [ 47.748843] niks3-server[1052]: 2026/09/20 12:46:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4790lb # [ 47.750779] niks3-server[1052]: 2026/09/20 12:46:38 INFO Signed narinfos id=9 count=14791worker1 # [ 47.686526] nix-grpc-daemon[934]: time=2026-09-20T12:46:38.172Z level=INFO msg="Uploading 1 narinfos"4792lb # [ 47.756716] niks3-server[1052]: 2026/09/20 12:46:38 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4793lb # [ 47.761149] niks3-server[1052]: 2026/09/20 12:46:38 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4794lb # [ 47.764056] niks3-server[1052]: 2026/09/20 12:46:38 INFO Completed upload id=94795worker1 # [ 47.701031] nix-grpc-daemon[934]: time=2026-09-20T12:46:38.187Z level=INFO msg="Upload complete. (439ms)"4796lb # [ 47.766525] niks3-server[1052]: 2026/09/20 12:46:38 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4797worker1 # [ 47.703168] nix-grpc-daemon[795]: ts=2026-09-20T12:46:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37600 duration_s=0 paths=1 nar_bytes_in=4324798worker1 # [ 47.709356] nix-grpc-daemon[795]: ts=2026-09-20T12:46:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376004799worker1 # [ 47.712112] nix-daemon[908]: reaped child process 932, status = succeeded4800worker1 # [ 47.714948] nix-daemon[908]: accepted connection from pid 795, user nix-grpc-daemon (trusted)4801worker1 # [ 47.721858] nix-daemon[908]: accepted connection from pid 795, user nix-grpc-daemon (trusted)4802client # [ 47.789555] nix[894]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4803client # [ 47.824311] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4804client # Running as unit: dedup2.service; invocation ID: 2faecafb819c4a59a2d449833c0333674805client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 1.06 seconds)4806worker1 # [ 47.962830] nix-grpc-daemon[795]: ts=2026-09-20T12:46:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:376004807client # [ 48.043821] nix[910]: this derivation will be built:4808worker2 # [ 48.048966] nix-grpc-daemon[791]: ts=2026-09-20T12:46:38Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:471044809worker2 # [ 48.056692] nix-grpc-daemon[791]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' from 'http://lb:5751'...4810client # [ 48.073915] nix[910]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4811worker2 # [ 48.081966] nix-grpc-daemon[791]: ts=2026-09-20T12:46:38Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04812worker1 # [ 48.021069] nix-grpc-daemon[795]: ts=2026-09-20T12:46:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376004813client # [ 48.094856] nix[910]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4814client: must succeed: sleep 24815client: (finished: must succeed: sleep 2, in 2.03 seconds)4816worker1 # [ 51.135889] nix-daemon[908]: reaped child process 958, status = succeeded4817??? Warning (UserWarning): wait_until_succeeds(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4818 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394819client: waiting for success: ! systemctl is-active dedup1 dedup24820??? Warning (UserWarning): execute(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4821 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394822worker1 # [ 51.290984] nix-daemon[908]: accepted connection from pid 987, user nix-grpc-daemon (trusted)4823worker1 # [ 51.303427] nix-daemon[908]: reaped child process 990, status = succeeded4824lb # [ 51.384799] niks3-server[1052]: 2026/09/20 12:46:42 INFO Received uploads request method=POST path=/api/pending_closures4825worker1 # [ 51.329704] nix-grpc-daemon[934]: time=2026-09-20T12:46:41.815Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4826worker1 # [ 51.331232] nix-grpc-daemon[934]: time=2026-09-20T12:46:41.817Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4827lb # [ 51.432575] niks3-server[1052]: 2026/09/20 12:46:42 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4828lb # [ 51.438588] niks3-server[1052]: 2026/09/20 12:46:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4829worker1 # [ 51.380109] nix-grpc-daemon[934]: time=2026-09-20T12:46:41.865Z level=INFO msg="Uploading 1 narinfos"4830lb # [ 51.444324] niks3-server[1052]: 2026/09/20 12:46:42 INFO Signed narinfos id=10 count=14831lb # [ 51.453158] niks3-server[1052]: 2026/09/20 12:46:42 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4832lb # [ 51.460063] niks3-server[1052]: 2026/09/20 12:46:42 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4833lb # [ 51.468468] niks3-server[1052]: 2026/09/20 12:46:42 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4834lb # [ 51.473440] niks3-server[1052]: 2026/09/20 12:46:42 INFO Completed upload id=104835worker1 # [ 51.409441] nix-grpc-daemon[934]: time=2026-09-20T12:46:41.895Z level=INFO msg="Upload complete. (275ms)"4836lb # [ 51.475736] niks3-server[1052]: 2026/09/20 12:46:42 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4837worker1 # [ 51.415517] nix-grpc-daemon[795]: ts=2026-09-20T12:46:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37600 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14838worker1 # [ 51.421421] nix-grpc-daemon[795]: ts=2026-09-20T12:46:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37600 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14839worker1 # [ 51.431595] nix-daemon[908]: reaped child process 956, status = succeeded4840client # [ 51.523174] systemd[1]: dedup1.service: Deactivated successfully.4841client # [ 51.527502] systemd[1]: dedup1.service: Consumed 105ms CPU time over 4.763s wall clock time, 39.3M memory peak, 8.5K incoming IP traffic, 9K outgoing IP traffic.4842client # [ 51.566448] systemd[1]: dedup2.service: Deactivated successfully.4843client # [ 51.567926] systemd[1]: dedup2.service: Consumed 114ms CPU time over 3.743s wall clock time, 33.2M memory peak, 8K incoming IP traffic, 6.6K outgoing IP traffic.4844client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.10 seconds)4845client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4846client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)4847worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4848worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.06 seconds)4849worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4850worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4851worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4852worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4853worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4854worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4855(finished: subtest: two clients wanting the same drv build it once, in 5.90 seconds)4856subtest: build hook: nix-daemon with builders = grpc://lb4857client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log4858client # [ 52.798905] systemd[1]: Started Nix Daemon.4859client # [ 52.964149] nix-daemon[952]: accepted connection from pid 949, user root (trusted)4860worker1 # [ 53.066659] nix-grpc-daemon[795]: ts=2026-09-20T12:46:43Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:376004861worker2 # [ 53.143928] nix-grpc-daemon[791]: ts=2026-09-20T12:46:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471044862worker2 # [ 53.159121] nix-grpc-daemon[791]: ts=2026-09-20T12:46:44Z level=debug event=want client=6 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=04863worker2 # [ 53.163893] nix-grpc-daemon[791]: ts=2026-09-20T12:46:44Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.4:50051 clients=14864worker2 # [ 53.201901] nix-grpc-daemon[791]: ts=2026-09-20T12:46:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471044865worker2 # [ 53.212358] nix-grpc-daemon[791]: ts=2026-09-20T12:46:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471044866worker2 # [ 53.222130] nix-grpc-daemon[791]: ts=2026-09-20T12:46:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:471044867worker2 # [ 53.233355] nix-daemon[860]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4868worker2 # [ 53.389204] nix-daemon[860]: accepted connection from pid 1086, user nix-grpc-daemon (trusted)4869worker2 # [ 53.403450] nix-daemon[860]: reaped child process 1089, status = succeeded4870worker2 # [ 53.566673] nix-daemon[860]: accepted connection from pid 1091, user nix-grpc-daemon (trusted)4871worker2 # [ 53.578959] nix-daemon[860]: reaped child process 1094, status = succeeded4872lb # [ 53.591257] niks3-server[1052]: 2026/09/20 12:46:44 INFO Received uploads request method=POST path=/api/pending_closures4873worker2 # [ 53.605552] nix-grpc-daemon[872]: time=2026-09-20T12:46:44.514Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4874worker2 # [ 53.606957] nix-grpc-daemon[872]: time=2026-09-20T12:46:44.514Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"4875lb # [ 53.623129] niks3-server[1052]: 2026/09/20 12:46:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4876worker2 # [ 53.635761] nix-grpc-daemon[872]: time=2026-09-20T12:46:44.544Z level=INFO msg="Uploading 1 narinfos"4877lb # [ 53.629799] niks3-server[1052]: 2026/09/20 12:46:44 INFO Signed narinfos id=11 count=14878lb # [ 53.637291] niks3-server[1052]: 2026/09/20 12:46:44 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls4879lb # [ 53.641419] niks3-server[1052]: 2026/09/20 12:46:44 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst4880lb # [ 53.656171] niks3-server[1052]: 2026/09/20 12:46:44 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4881lb # [ 53.660533] niks3-server[1052]: 2026/09/20 12:46:44 INFO Completed upload id=114882worker2 # [ 53.669732] nix-grpc-daemon[872]: time=2026-09-20T12:46:44.578Z level=INFO msg="Upload complete. (424ms)"4883lb # [ 53.664947] niks3-server[1052]: 2026/09/20 12:46:44 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo4884worker2 # [ 53.673490] nix-grpc-daemon[791]: ts=2026-09-20T12:46:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47104 duration_s=0 paths=1 nar_bytes_in=5124885worker2 # [ 53.679291] nix-grpc-daemon[791]: ts=2026-09-20T12:46:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471044886worker2 # [ 53.690165] nix-daemon[860]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4887worker2 # [ 53.694392] nix-daemon[860]: reaped child process 1084, status = succeeded4888worker2 # [ 53.699430] nix-daemon[860]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4889worker2 # [ 54.712836] nix-daemon[860]: reaped child process 1098, status = succeeded4890worker2 # [ 54.870567] nix-daemon[860]: accepted connection from pid 1113, user nix-grpc-daemon (trusted)4891worker2 # [ 54.882278] nix-daemon[860]: reaped child process 1116, status = succeeded4892lb # [ 54.892298] niks3-server[1052]: 2026/09/20 12:46:45 INFO Received uploads request method=POST path=/api/pending_closures4893worker2 # [ 54.903104] nix-grpc-daemon[872]: time=2026-09-20T12:46:45.812Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4894worker2 # [ 54.905542] nix-grpc-daemon[872]: time=2026-09-20T12:46:45.813Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"4895lb # [ 54.918168] niks3-server[1052]: 2026/09/20 12:46:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4896worker2 # [ 54.926901] nix-grpc-daemon[872]: time=2026-09-20T12:46:45.836Z level=INFO msg="Uploading 1 narinfos"4897lb # [ 54.920750] niks3-server[1052]: 2026/09/20 12:46:46 INFO Signed narinfos id=12 count=14898lb # [ 54.924506] niks3-server[1052]: 2026/09/20 12:46:46 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv4899lb # [ 54.926396] niks3-server[1052]: 2026/09/20 12:46:46 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4900lb # [ 54.932377] niks3-server[1052]: 2026/09/20 12:46:46 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4901lb # [ 54.934439] niks3-server[1052]: 2026/09/20 12:46:46 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls4902lb # [ 54.937497] niks3-server[1052]: 2026/09/20 12:46:46 INFO Completed upload id=124903worker2 # [ 54.943357] nix-grpc-daemon[872]: time=2026-09-20T12:46:45.852Z level=INFO msg="Upload complete. (237ms)"4904lb # [ 54.939282] niks3-server[1052]: 2026/09/20 12:46:46 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo4905worker2 # [ 54.945860] nix-grpc-daemon[791]: ts=2026-09-20T12:46:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47104 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=6 outputs=14906worker2 # [ 54.953519] nix-daemon[860]: reaped child process 1096, status = succeeded4907worker1 # [ 54.884060] nix-grpc-daemon[795]: ts=2026-09-20T12:46:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:376004908worker1 # [ 54.891412] nix-grpc-daemon[795]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...4909worker1 # [ 54.920480] nix-grpc-daemon[795]: ts=2026-09-20T12:46:45Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:376004910worker1 # [ 54.923412] nix-grpc-daemon[795]: ts=2026-09-20T12:46:45Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37600 duration_s=0 paths=1 nar_bytes_out=1284911worker1 # [ 55.192762] nix-grpc-daemon[795]: ts=2026-09-20T12:46:45Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:376004912worker2 # [ 55.266540] nix-grpc-daemon[791]: ts=2026-09-20T12:46:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471044913worker2 # [ 55.277690] nix-grpc-daemon[872]: time=2026-09-20T12:46:46.186Z level=INFO msg="All 1 paths already cached"4914worker2 # [ 55.291723] nix-grpc-daemon[791]: ts=2026-09-20T12:46:46Z level=debug event=want client=7 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=04915worker2 # [ 55.297289] nix-grpc-daemon[791]: ts=2026-09-20T12:46:46Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.4:50051 clients=14916worker2 # [ 55.302951] nix-grpc-daemon[791]: ts=2026-09-20T12:46:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471044917worker2 # [ 55.314447] nix-grpc-daemon[791]: ts=2026-09-20T12:46:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471044918worker2 # [ 55.326422] nix-grpc-daemon[872]: time=2026-09-20T12:46:46.235Z level=INFO msg="All 1 paths already cached"4919worker2 # [ 55.333355] nix-grpc-daemon[791]: ts=2026-09-20T12:46:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:471044920worker2 # [ 55.343379] nix-daemon[860]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4921worker2 # [ 55.510090] nix-daemon[860]: accepted connection from pid 1122, user nix-grpc-daemon (trusted)4922worker2 # [ 55.524385] nix-daemon[860]: reaped child process 1125, status = succeeded4923worker2 # [ 55.695098] nix-daemon[860]: accepted connection from pid 1127, user nix-grpc-daemon (trusted)4924worker2 # [ 55.708134] nix-daemon[860]: reaped child process 1130, status = succeeded4925lb # [ 55.714959] niks3-server[1052]: 2026/09/20 12:46:46 INFO Received uploads request method=POST path=/api/pending_closures4926worker2 # [ 55.728538] nix-grpc-daemon[872]: time=2026-09-20T12:46:46.637Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4927worker2 # [ 55.730088] nix-grpc-daemon[872]: time=2026-09-20T12:46:46.639Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"4928lb # [ 55.749800] niks3-server[1052]: 2026/09/20 12:46:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4929lb # [ 55.754709] niks3-server[1052]: 2026/09/20 12:46:46 INFO Signed narinfos id=13 count=14930worker2 # [ 55.764110] nix-grpc-daemon[872]: time=2026-09-20T12:46:46.673Z level=INFO msg="Uploading 1 narinfos"4931lb # [ 55.758102] niks3-server[1052]: 2026/09/20 12:46:46 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst4932lb # [ 55.767543] niks3-server[1052]: 2026/09/20 12:46:46 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls4933lb # [ 55.778936] niks3-server[1052]: 2026/09/20 12:46:46 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4934lb # [ 55.785318] niks3-server[1052]: 2026/09/20 12:46:46 INFO Completed upload id=134935worker2 # [ 55.791637] nix-grpc-daemon[872]: time=2026-09-20T12:46:46.700Z level=INFO msg="Upload complete. (435ms)"4936lb # [ 55.788250] niks3-server[1052]: 2026/09/20 12:46:46 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo4937worker2 # [ 55.796152] nix-grpc-daemon[791]: ts=2026-09-20T12:46:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47104 duration_s=0 paths=1 nar_bytes_in=6244938worker2 # [ 55.801135] nix-grpc-daemon[791]: ts=2026-09-20T12:46:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471044939worker2 # [ 55.809723] nix-daemon[860]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4940worker2 # [ 55.815530] nix-daemon[860]: reaped child process 1120, status = succeeded4941worker2 # [ 55.820800] nix-daemon[860]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4942worker2 # [ 57.238076] nix-daemon[860]: reaped child process 1134, status = succeeded4943lb # [ 57.344370] envoy[910]: conn peer=[2001:db8:1::1]:53070 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2968 tx=1354 ms=45324944lb # [ 57.348261] envoy[910]: conn peer=[2001:db8:1::1]:53084 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1589 tx=1487 ms=35164945lb # [ 57.351878] envoy[910]: conn peer=[2001:db8:1::1]:53530 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3383 tx=1757 ms=19094946lb # [ 57.356404] envoy[910]: conn peer=[2001:db8:1::1]:53532 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=584947worker2 # [ 57.394238] nix-daemon[860]: accepted connection from pid 1149, user nix-grpc-daemon (trusted)4948worker2 # [ 57.407777] nix-daemon[860]: reaped child process 1152, status = succeeded4949lb # [ 57.418818] niks3-server[1052]: 2026/09/20 12:46:48 INFO Received uploads request method=POST path=/api/pending_closures4950worker2 # [ 57.471992] nix-grpc-daemon[872]: time=2026-09-20T12:46:48.380Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4951worker2 # [ 57.474807] nix-grpc-daemon[872]: time=2026-09-20T12:46:48.382Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"4952lb # [ 57.489204] niks3-server[1052]: 2026/09/20 12:46:48 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv4953lb # [ 57.505142] niks3-server[1052]: 2026/09/20 12:46:48 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst4954lb # [ 57.512664] niks3-server[1052]: 2026/09/20 12:46:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4955lb # [ 57.517813] niks3-server[1052]: 2026/09/20 12:46:48 INFO Signed narinfos id=14 count=14956worker2 # [ 57.527206] nix-grpc-daemon[872]: time=2026-09-20T12:46:48.435Z level=INFO msg="Uploading 1 narinfos"4957lb # [ 57.522911] niks3-server[1052]: 2026/09/20 12:46:48 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls4958lb # [ 57.532129] niks3-server[1052]: 2026/09/20 12:46:48 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4959lb # [ 57.537791] niks3-server[1052]: 2026/09/20 12:46:48 INFO Completed upload id=144960worker2 # [ 57.544213] nix-grpc-daemon[872]: time=2026-09-20T12:46:48.453Z level=INFO msg="Upload complete. (314ms)"4961lb # [ 57.540170] niks3-server[1052]: 2026/09/20 12:46:48 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo4962worker2 # [ 57.552500] nix-grpc-daemon[791]: ts=2026-09-20T12:46:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47104 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=7 outputs=14963worker2 # [ 57.561571] nix-grpc-daemon[791]: ts=2026-09-20T12:46:48Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:471044964worker2 # [ 57.572941] nix-daemon[860]: reaped child process 1132, status = succeeded4965worker1 # [ 57.504378] nix-grpc-daemon[795]: ts=2026-09-20T12:46:47Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:376004966worker1 # [ 57.513197] nix-grpc-daemon[795]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...4967worker1 # [ 57.524778] nix-grpc-daemon[795]: ts=2026-09-20T12:46:48Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37600 duration_s=0 paths=1 nar_bytes_out=1764968worker2 # [ 57.861800] nix-grpc-daemon[791]: ts=2026-09-20T12:46:48Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:471044969worker2 # [ 57.867962] nix-grpc-daemon[791]: ts=2026-09-20T12:46:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471044970worker2 # [ 57.873569] nix-grpc-daemon[872]: time=2026-09-20T12:46:48.781Z level=INFO msg="All 2 paths already cached"4971worker2 # [ 57.884509] nix-grpc-daemon[791]: ts=2026-09-20T12:46:48Z level=debug event=want client=8 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=04972worker2 # [ 57.889932] nix-grpc-daemon[791]: ts=2026-09-20T12:46:48Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.4:50051 clients=14973worker2 # [ 57.928786] nix-grpc-daemon[791]: ts=2026-09-20T12:46:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471044974worker2 # [ 57.941731] nix-grpc-daemon[791]: ts=2026-09-20T12:46:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471044975worker2 # [ 57.954398] nix-grpc-daemon[872]: time=2026-09-20T12:46:48.863Z level=INFO msg="All 2 paths already cached"4976worker2 # [ 57.962171] nix-grpc-daemon[791]: ts=2026-09-20T12:46:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:471044977worker2 # [ 57.970939] nix-daemon[860]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4978worker2 # [ 58.147117] nix-daemon[860]: accepted connection from pid 1158, user nix-grpc-daemon (trusted)4979worker2 # [ 58.160606] nix-daemon[860]: reaped child process 1161, status = succeeded4980worker2 # [ 58.318760] nix-daemon[860]: accepted connection from pid 1163, user nix-grpc-daemon (trusted)4981worker2 # [ 58.332842] nix-daemon[860]: reaped child process 1166, status = succeeded4982lb # [ 58.341692] niks3-server[1052]: 2026/09/20 12:46:49 INFO Received uploads request method=POST path=/api/pending_closures4983worker2 # [ 58.355455] nix-grpc-daemon[872]: time=2026-09-20T12:46:49.264Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4984worker2 # [ 58.357333] nix-grpc-daemon[872]: time=2026-09-20T12:46:49.266Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"4985lb # [ 58.370899] niks3-server[1052]: 2026/09/20 12:46:49 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst4986lb # [ 58.379860] niks3-server[1052]: 2026/09/20 12:46:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4987lb # [ 58.384344] niks3-server[1052]: 2026/09/20 12:46:49 INFO Signed narinfos id=15 count=14988worker2 # [ 58.393461] nix-grpc-daemon[872]: time=2026-09-20T12:46:49.302Z level=INFO msg="Uploading 1 narinfos"4989lb # [ 58.389993] niks3-server[1052]: 2026/09/20 12:46:49 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls4990lb # [ 58.401639] niks3-server[1052]: 2026/09/20 12:46:49 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4991worker2 # [ 58.413246] nix-grpc-daemon[872]: time=2026-09-20T12:46:49.322Z level=INFO msg="Upload complete. (428ms)"4992lb # [ 58.407323] niks3-server[1052]: 2026/09/20 12:46:49 INFO Completed upload id=154993worker2 # [ 58.416085] nix-grpc-daemon[791]: ts=2026-09-20T12:46:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47104 duration_s=0 paths=1 nar_bytes_in=7524994lb # [ 58.410579] niks3-server[1052]: 2026/09/20 12:46:49 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo4995worker2 # [ 58.420495] nix-grpc-daemon[791]: ts=2026-09-20T12:46:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471044996worker2 # [ 58.427220] nix-daemon[860]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4997worker2 # [ 58.429764] nix-daemon[860]: reaped child process 1156, status = succeeded4998worker2 # [ 58.435661] nix-daemon[860]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4999worker2 # [ 59.658936] nix-daemon[860]: reaped child process 1170, status = succeeded5000worker2 # [ 59.797363] nix-daemon[860]: accepted connection from pid 1185, user nix-grpc-daemon (trusted)5001worker2 # [ 59.809923] nix-daemon[860]: reaped child process 1188, status = succeeded5002lb # [ 59.820423] niks3-server[1052]: 2026/09/20 12:46:50 INFO Received uploads request method=POST path=/api/pending_closures5003worker2 # [ 59.837210] nix-grpc-daemon[872]: time=2026-09-20T12:46:50.746Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5004worker2 # [ 59.839349] nix-grpc-daemon[872]: time=2026-09-20T12:46:50.748Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"5005lb # [ 59.869222] niks3-server[1052]: 2026/09/20 12:46:50 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst5006lb # [ 59.878991] niks3-server[1052]: 2026/09/20 12:46:50 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv5007lb # [ 59.887677] niks3-server[1052]: 2026/09/20 12:46:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign5008lb # [ 59.892348] niks3-server[1052]: 2026/09/20 12:46:50 INFO Signed narinfos id=16 count=15009worker2 # [ 59.901695] nix-grpc-daemon[872]: time=2026-09-20T12:46:50.810Z level=INFO msg="Uploading 1 narinfos"5010lb # [ 59.898342] niks3-server[1052]: 2026/09/20 12:46:50 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls5011lb # [ 59.905558] niks3-server[1052]: 2026/09/20 12:46:50 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete5012lb # [ 59.908982] niks3-server[1052]: 2026/09/20 12:46:51 INFO Completed upload id=165013worker2 # [ 59.915472] nix-grpc-daemon[872]: time=2026-09-20T12:46:50.824Z level=INFO msg="Upload complete. (271ms)"5014lb # [ 59.911132] niks3-server[1052]: 2026/09/20 12:46:51 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo5015worker2 # [ 59.923591] nix-grpc-daemon[791]: ts=2026-09-20T12:46:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47104 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=8 outputs=15016worker1 # [ 59.858327] nix-grpc-daemon[795]: ts=2026-09-20T12:46:50Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:376005017worker2 # [ 59.935587] nix-daemon[860]: reaped child process 1168, status = succeeded5018worker1 # [ 59.866462] nix-grpc-daemon[795]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...5019worker2 # [ 59.958997] nix-grpc-daemon[791]: ts=2026-09-20T12:46:50Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:471045020worker2 # [ 59.965695] nix-grpc-daemon[791]: ts=2026-09-20T12:46:50Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:47104 duration_s=0 paths=1 nar_bytes_out=2325021client # [ 60.060153] nix-daemon[952]: reaped child process 954, status = succeeded5022client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.46 seconds)5023client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5024client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.02 seconds)5025client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5026client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.03 seconds)5027client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5028client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)5029(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.54 seconds)5030subtest: an input only one worker has still reaches the builder5031client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only5032client # this derivation will be built:5033client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5034client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5035client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only, in 0.42 seconds)5036client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5037client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.17 seconds)5038worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5039worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.22 seconds)5040worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5041worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.03 seconds)5042client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&25043client # [ 61.162061] nix-daemon[952]: accepted connection from pid 1035, user root (trusted)5044client # this derivation will be built:5045client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5046worker2 # [ 61.322445] nix-grpc-daemon[791]: ts=2026-09-20T12:46:52Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:471045047client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...5048worker1 # [ 61.255185] nix-grpc-daemon[795]: ts=2026-09-20T12:46:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376005049worker1 # [ 61.432743] nix-daemon[908]: accepted connection from pid 1010, user nix-grpc-daemon (trusted)5050worker1 # [ 61.444313] nix-daemon[908]: reaped child process 1013, status = succeeded5051lb # [ 61.524675] niks3-server[1052]: 2026/09/20 12:46:52 INFO Received uploads request method=POST path=/api/pending_closures5052worker1 # [ 62.213644] nix-grpc-daemon[934]: time=2026-09-20T12:46:52.699Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5053worker1 # [ 62.218075] nix-grpc-daemon[934]: time=2026-09-20T12:46:52.699Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5054lb # [ 62.304763] niks3-server[1052]: 2026/09/20 12:46:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign5055lb # [ 62.312348] niks3-server[1052]: 2026/09/20 12:46:53 INFO Signed narinfos id=17 count=15056worker1 # [ 62.248528] nix-grpc-daemon[934]: time=2026-09-20T12:46:52.734Z level=INFO msg="Uploading 1 narinfos"5057lb # [ 62.327909] niks3-server[1052]: 2026/09/20 12:46:53 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete5058lb # [ 62.338900] niks3-server[1052]: 2026/09/20 12:46:53 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5059lb # [ 62.357611] niks3-server[1052]: 2026/09/20 12:46:53 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5060lb # [ 62.371704] niks3-server[1052]: 2026/09/20 12:46:53 INFO Completed upload id=175061worker1 # [ 62.307509] nix-grpc-daemon[934]: time=2026-09-20T12:46:52.793Z level=INFO msg="Upload complete. (1.047s)"5062lb # [ 62.378547] niks3-server[1052]: 2026/09/20 12:46:53 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5063worker2 # [ 62.398870] nix-grpc-daemon[791]: ts=2026-09-20T12:46:53Z level=debug event=want client=9 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05064worker2 # [ 62.403807] nix-grpc-daemon[791]: ts=2026-09-20T12:46:53Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=15065worker1 # [ 62.373802] nix-grpc-daemon[795]: ts=2026-09-20T12:46:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376005066worker1 # [ 62.385752] nix-grpc-daemon[795]: ts=2026-09-20T12:46:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376005067client # copying 2 paths to 'lb:50051'...5068client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5069worker1 # [ 62.399212] nix-grpc-daemon[795]: ts=2026-09-20T12:46:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376005070client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5071worker1 # [ 62.413181] nix-daemon[908]: accepted connection from pid 795, user nix-grpc-daemon (trusted)5072worker1 # [ 62.596773] nix-daemon[908]: accepted connection from pid 1017, user nix-grpc-daemon (trusted)5073worker1 # [ 62.609788] nix-daemon[908]: reaped child process 1020, status = succeeded5074worker1 # [ 62.756783] nix-daemon[908]: accepted connection from pid 1022, user nix-grpc-daemon (trusted)5075worker1 # [ 62.769758] nix-daemon[908]: reaped child process 1025, status = succeeded5076lb # [ 62.845488] niks3-server[1052]: 2026/09/20 12:46:53 INFO Received uploads request method=POST path=/api/pending_closures5077lb # [ 62.849913] niks3-server[1052]: 2026/09/20 12:46:53 INFO Received uploads request method=POST path=/api/pending_closures5078worker1 # [ 62.788989] nix-grpc-daemon[934]: time=2026-09-20T12:46:53.275Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5079worker1 # [ 62.790568] nix-grpc-daemon[934]: time=2026-09-20T12:46:53.276Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5080worker1 # [ 62.793111] nix-grpc-daemon[934]: time=2026-09-20T12:46:53.278Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5081lb # [ 62.867943] niks3-server[1052]: 2026/09/20 12:46:53 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5082lb # [ 62.871923] niks3-server[1052]: 2026/09/20 12:46:53 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5083lb # [ 62.878477] niks3-server[1052]: 2026/09/20 12:46:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5084lb # [ 62.881936] niks3-server[1052]: 2026/09/20 12:46:53 INFO Signed narinfos id=18 count=15085lb # [ 62.883715] niks3-server[1052]: 2026/09/20 12:46:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5086lb # [ 62.885849] niks3-server[1052]: 2026/09/20 12:46:53 INFO Signed narinfos id=19 count=25087lb # [ 62.887280] niks3-server[1052]: 2026/09/20 12:46:53 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5088worker1 # [ 62.823337] nix-grpc-daemon[934]: time=2026-09-20T12:46:53.309Z level=INFO msg="Uploading 3 narinfos"5089lb # [ 62.889647] niks3-server[1052]: 2026/09/20 12:46:53 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5090lb # [ 62.904060] niks3-server[1052]: 2026/09/20 12:46:53 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5091lb # [ 62.910152] niks3-server[1052]: 2026/09/20 12:46:54 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5092lb # [ 62.912787] niks3-server[1052]: 2026/09/20 12:46:54 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5093lb # [ 62.916149] niks3-server[1052]: 2026/09/20 12:46:54 INFO Completed upload id=185094lb # [ 62.918147] niks3-server[1052]: 2026/09/20 12:46:54 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5095lb # [ 62.919823] niks3-server[1052]: 2026/09/20 12:46:54 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5096lb # [ 62.922277] niks3-server[1052]: 2026/09/20 12:46:54 INFO Completed upload id=195097worker1 # [ 62.859349] nix-grpc-daemon[934]: time=2026-09-20T12:46:53.345Z level=INFO msg="Upload complete. (430ms)"5098worker1 # [ 62.864127] nix-grpc-daemon[795]: ts=2026-09-20T12:46:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37600 duration_s=0 paths=2 nar_bytes_in=9765099worker1 # [ 62.868623] nix-grpc-daemon[795]: ts=2026-09-20T12:46:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376005100worker1 # [ 62.876290] nix-daemon[908]: accepted connection from pid 795, user nix-grpc-daemon (trusted)5101worker1 # [ 62.882200] nix-daemon[908]: reaped child process 1015, status = succeeded5102client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5103worker1 # [ 62.886602] nix-daemon[908]: accepted connection from pid 795, user nix-grpc-daemon (trusted)5104client # uses-input-w1onlya> node-a: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5105worker1 # [ 63.911523] nix-daemon[908]: reaped child process 1029, status = succeeded5106worker1 # [ 64.073383] nix-daemon[908]: accepted connection from pid 1044, user nix-grpc-daemon (trusted)5107worker1 # [ 64.085963] nix-daemon[908]: reaped child process 1047, status = succeeded5108lb # [ 64.167967] niks3-server[1052]: 2026/09/20 12:46:55 INFO Received uploads request method=POST path=/api/pending_closures5109worker1 # [ 64.121469] nix-grpc-daemon[934]: time=2026-09-20T12:46:54.607Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5110lb # [ 64.209149] niks3-server[1052]: 2026/09/20 12:46:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5111lb # [ 64.216484] niks3-server[1052]: 2026/09/20 12:46:55 INFO Signed narinfos id=20 count=15112worker1 # [ 64.156296] nix-grpc-daemon[934]: time=2026-09-20T12:46:54.642Z level=INFO msg="Uploading 1 narinfos"5113lb # [ 64.220899] niks3-server[1052]: 2026/09/20 12:46:55 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5114lb # [ 64.234750] niks3-server[1052]: 2026/09/20 12:46:55 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5115lb # [ 64.240353] niks3-server[1052]: 2026/09/20 12:46:55 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5116lb # [ 64.245717] niks3-server[1052]: 2026/09/20 12:46:55 INFO Completed upload id=205117worker1 # [ 64.181193] nix-grpc-daemon[934]: time=2026-09-20T12:46:54.667Z level=INFO msg="Upload complete. (287ms)"5118lb # [ 64.250591] niks3-server[1052]: 2026/09/20 12:46:55 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5119worker1 # [ 64.186556] nix-grpc-daemon[795]: ts=2026-09-20T12:46:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37600 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=9 outputs=15120worker1 # [ 64.198687] nix-daemon[908]: reaped child process 1027, status = succeeded5121client # copying 1 paths...5122worker2 # [ 64.299799] nix-grpc-daemon[791]: ts=2026-09-20T12:46:55Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:471045123worker2 # [ 64.315695] nix-grpc-daemon[791]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5124client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5125worker2 # [ 64.340687] nix-grpc-daemon[791]: ts=2026-09-20T12:46:55Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:471045126worker2 # [ 64.342350] nix-grpc-daemon[791]: ts=2026-09-20T12:46:55Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:47104 duration_s=0 paths=1 nar_bytes_out=1205127client # [ 64.443927] nix-daemon[952]: reaped child process 1038, status = succeeded5128client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&2, in 3.47 seconds)5129client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&25130client # [ 64.600179] nix-daemon[952]: accepted connection from pid 1057, user root (trusted)5131client # this derivation will be built:5132client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5133worker1 # [ 64.806401] nix-grpc-daemon[795]: ts=2026-09-20T12:46:55Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:376005134client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...5135worker1 # [ 64.812815] nix-grpc-daemon[795]: ts=2026-09-20T12:46:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376005136worker1 # [ 64.818089] nix-grpc-daemon[934]: time=2026-09-20T12:46:55.303Z level=INFO msg="All 1 paths already cached"5137worker2 # [ 64.898773] nix-grpc-daemon[791]: ts=2026-09-20T12:46:55Z level=debug event=want client=10 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05138worker2 # [ 64.904635] nix-grpc-daemon[791]: ts=2026-09-20T12:46:55Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=15139worker2 # [ 64.910432] nix-grpc-daemon[791]: ts=2026-09-20T12:46:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471045140worker2 # [ 64.921972] nix-grpc-daemon[791]: ts=2026-09-20T12:46:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471045141client # copying 1 paths to 'lb:50051'...5142client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5143worker2 # [ 64.939645] nix-grpc-daemon[791]: ts=2026-09-20T12:46:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:471045144worker2 # [ 64.948758] nix-daemon[860]: accepted connection from pid 791, user nix-grpc-daemon (trusted)5145worker2 # [ 64.966929] nix-grpc-daemon[791]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 81 ms (attempt 1/5)5146worker2 # [ 65.053199] nix-grpc-daemon[791]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 90 ms (attempt 2/5)5147worker2 # [ 65.149356] nix-grpc-daemon[791]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 195 ms (attempt 3/5)5148worker2 # [ 65.352307] nix-grpc-daemon[791]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 615 ms (attempt 4/5)5149worker2 # [ 65.968395] nix-grpc-daemon[791]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5150worker2 # [ 65.973793] nix-grpc-daemon[791]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5151worker2 # [ 66.175611] nix-daemon[860]: accepted connection from pid 1204, user nix-grpc-daemon (trusted)5152worker2 # [ 66.189766] nix-daemon[860]: reaped child process 1207, status = succeeded5153worker2 # [ 66.349277] nix-daemon[860]: accepted connection from pid 1209, user nix-grpc-daemon (trusted)5154worker2 # [ 66.362526] nix-daemon[860]: reaped child process 1212, status = succeeded5155lb # [ 66.373213] niks3-server[1052]: 2026/09/20 12:46:57 INFO Received uploads request method=POST path=/api/pending_closures5156worker2 # [ 66.387267] nix-grpc-daemon[872]: time=2026-09-20T12:46:57.295Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5157worker2 # [ 66.388736] nix-grpc-daemon[872]: time=2026-09-20T12:46:57.296Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5158lb # [ 66.407455] niks3-server[1052]: 2026/09/20 12:46:57 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5159lb # [ 66.413954] niks3-server[1052]: 2026/09/20 12:46:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5160lb # [ 66.419655] niks3-server[1052]: 2026/09/20 12:46:57 INFO Signed narinfos id=21 count=15161worker2 # [ 66.428822] nix-grpc-daemon[872]: time=2026-09-20T12:46:57.337Z level=INFO msg="Uploading 1 narinfos"5162lb # [ 66.424136] niks3-server[1052]: 2026/09/20 12:46:57 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5163lb # [ 66.435149] niks3-server[1052]: 2026/09/20 12:46:57 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5164lb # [ 66.441835] niks3-server[1052]: 2026/09/20 12:46:57 INFO Completed upload id=215165worker2 # [ 66.447858] nix-grpc-daemon[872]: time=2026-09-20T12:46:57.357Z level=INFO msg="Upload complete. (452ms)"5166worker2 # [ 66.451853] nix-grpc-daemon[791]: ts=2026-09-20T12:46:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47104 duration_s=1 paths=1 nar_bytes_in=5605167lb # [ 66.447553] niks3-server[1052]: 2026/09/20 12:46:57 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5168worker2 # [ 66.458426] nix-grpc-daemon[791]: ts=2026-09-20T12:46:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471045169worker2 # [ 66.467138] nix-daemon[860]: accepted connection from pid 791, user nix-grpc-daemon (trusted)5170worker2 # [ 66.474928] nix-daemon[860]: reaped child process 1198, status = succeeded5171worker2 # [ 66.481208] nix-grpc-daemon[791]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...5172worker2 # [ 66.490206] nix-daemon[860]: accepted connection from pid 791, user nix-grpc-daemon (trusted)5173client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5174client # uses-input-w1onlyb> worker2: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5175lb # [ 67.344924] envoy[910]: conn peer=[2001:db8:1::1]:53540 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3893 tx=1877 ms=23845176lb # [ 67.349428] envoy[910]: conn peer=[2001:db8:1::1]:53544 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=379 ms=725177lb # [ 67.353242] envoy[910]: conn peer=[2001:db8:1::1]:53552 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4315 tx=2223 ms=21545178lb # [ 67.357140] envoy[910]: conn peer=[2001:db8:1::1]:53558 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=555179lb # [ 67.360937] envoy[910]: conn peer=[2001:db8:1::1]:53566 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3823 tx=1881 ms=30715180lb # [ 67.364977] envoy[910]: conn peer=[2001:db8:1::1]:48510 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=515181worker2 # [ 67.503176] nix-daemon[860]: reaped child process 1217, status = succeeded5182worker2 # [ 67.650350] nix-daemon[860]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)5183worker2 # [ 67.661731] nix-daemon[860]: reaped child process 1235, status = succeeded5184lb # [ 67.674648] niks3-server[1052]: 2026/09/20 12:46:58 INFO Received uploads request method=POST path=/api/pending_closures5185worker2 # [ 67.686939] nix-grpc-daemon[872]: time=2026-09-20T12:46:58.595Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5186lb # [ 67.702498] niks3-server[1052]: 2026/09/20 12:46:58 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5187lb # [ 67.711927] niks3-server[1052]: 2026/09/20 12:46:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5188worker2 # [ 67.723720] nix-grpc-daemon[872]: time=2026-09-20T12:46:58.632Z level=INFO msg="Uploading 1 narinfos"5189lb # [ 67.717678] niks3-server[1052]: 2026/09/20 12:46:58 INFO Signed narinfos id=22 count=15190lb # [ 67.723975] niks3-server[1052]: 2026/09/20 12:46:58 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5191lb # [ 67.735311] niks3-server[1052]: 2026/09/20 12:46:58 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5192worker2 # [ 67.747846] nix-grpc-daemon[872]: time=2026-09-20T12:46:58.657Z level=INFO msg="Upload complete. (250ms)"5193lb # [ 67.741867] niks3-server[1052]: 2026/09/20 12:46:58 INFO Completed upload id=225194worker2 # [ 67.750438] nix-grpc-daemon[791]: ts=2026-09-20T12:46:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47104 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=10 outputs=15195lb # [ 67.745104] niks3-server[1052]: 2026/09/20 12:46:58 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5196client # copying 1 paths...5197worker2 # [ 67.758504] nix-daemon[860]: reaped child process 1214, status = succeeded5198worker1 # [ 67.689591] nix-grpc-daemon[795]: ts=2026-09-20T12:46:58Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:376005199worker1 # [ 67.696241] nix-grpc-daemon[795]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5200client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5201worker2 # [ 67.792377] nix-grpc-daemon[791]: ts=2026-09-20T12:46:58Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:471045202worker2 # [ 67.794178] nix-grpc-daemon[791]: ts=2026-09-20T12:46:58Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:47104 duration_s=0 paths=1 nar_bytes_out=1205203client # [ 67.860664] nix-daemon[952]: reaped child process 1060, status = succeeded5204client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.41 seconds)5205client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&25206client # [ 68.027081] nix-daemon[952]: accepted connection from pid 1079, user root (trusted)5207client # this derivation will be built:5208client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5209worker1 # [ 68.129258] nix-grpc-daemon[795]: ts=2026-09-20T12:46:58Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:376005210client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...5211worker1 # [ 68.136477] nix-grpc-daemon[795]: ts=2026-09-20T12:46:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376005212worker1 # [ 68.144927] nix-grpc-daemon[934]: time=2026-09-20T12:46:58.630Z level=INFO msg="All 1 paths already cached"5213worker2 # [ 68.227695] nix-grpc-daemon[791]: ts=2026-09-20T12:46:59Z level=debug event=want client=11 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05214worker2 # [ 68.232557] nix-grpc-daemon[791]: ts=2026-09-20T12:46:59Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=15215worker1 # [ 68.201570] nix-grpc-daemon[795]: ts=2026-09-20T12:46:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376005216worker1 # [ 68.213938] nix-grpc-daemon[795]: ts=2026-09-20T12:46:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376005217worker1 # [ 68.226228] nix-grpc-daemon[934]: time=2026-09-20T12:46:58.712Z level=INFO msg="All 1 paths already cached"5218client # copying 1 paths to 'lb:50051'...5219client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5220worker1 # [ 68.234549] nix-grpc-daemon[795]: ts=2026-09-20T12:46:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376005221worker1 # [ 68.245915] nix-daemon[908]: accepted connection from pid 795, user nix-grpc-daemon (trusted)5222worker1 # [ 68.410791] nix-daemon[908]: accepted connection from pid 1052, user nix-grpc-daemon (trusted)5223worker1 # [ 68.424674] nix-daemon[908]: reaped child process 1055, status = succeeded5224worker1 # [ 68.578828] nix-daemon[908]: accepted connection from pid 1057, user nix-grpc-daemon (trusted)5225worker1 # [ 68.590675] nix-daemon[908]: reaped child process 1060, status = succeeded5226lb # [ 68.671886] niks3-server[1052]: 2026/09/20 12:46:59 INFO Received uploads request method=POST path=/api/pending_closures5227worker1 # [ 68.615559] nix-grpc-daemon[934]: time=2026-09-20T12:46:59.101Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5228worker1 # [ 68.616983] nix-grpc-daemon[934]: time=2026-09-20T12:46:59.101Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5229lb # [ 68.699443] niks3-server[1052]: 2026/09/20 12:46:59 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5230lb # [ 68.707712] niks3-server[1052]: 2026/09/20 12:46:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5231lb # [ 68.712892] niks3-server[1052]: 2026/09/20 12:46:59 INFO Signed narinfos id=23 count=15232worker1 # [ 68.651995] nix-grpc-daemon[934]: time=2026-09-20T12:46:59.138Z level=INFO msg="Uploading 1 narinfos"5233lb # [ 68.728389] niks3-server[1052]: 2026/09/20 12:46:59 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5234lb # [ 68.814612] niks3-server[1052]: 2026/09/20 12:46:59 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5235lb # [ 68.821652] niks3-server[1052]: 2026/09/20 12:46:59 INFO Completed upload id=235236worker1 # [ 68.757541] nix-grpc-daemon[934]: time=2026-09-20T12:46:59.243Z level=INFO msg="Upload complete. (499ms)"5237lb # [ 68.827333] niks3-server[1052]: 2026/09/20 12:46:59 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5238worker1 # [ 68.763785] nix-grpc-daemon[795]: ts=2026-09-20T12:46:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37600 duration_s=0 paths=1 nar_bytes_in=5605239worker1 # [ 68.769779] nix-grpc-daemon[795]: ts=2026-09-20T12:46:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376005240worker1 # [ 68.779587] nix-daemon[908]: accepted connection from pid 795, user nix-grpc-daemon (trusted)5241worker1 # [ 68.790528] nix-daemon[908]: reaped child process 1050, status = succeeded5242client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5243worker1 # [ 68.802526] nix-daemon[908]: accepted connection from pid 795, user nix-grpc-daemon (trusted)5244client # uses-input-w1onlyc> node-a: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5245worker1 # [ 70.223551] nix-daemon[908]: reaped child process 1064, status = succeeded5246worker1 # [ 70.369577] nix-daemon[908]: accepted connection from pid 1079, user nix-grpc-daemon (trusted)5247worker1 # [ 70.381689] nix-daemon[908]: reaped child process 1082, status = succeeded5248lb # [ 70.462792] niks3-server[1052]: 2026/09/20 12:47:01 INFO Received uploads request method=POST path=/api/pending_closures5249worker1 # [ 70.408062] nix-grpc-daemon[934]: time=2026-09-20T12:47:00.894Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5250lb # [ 70.493775] niks3-server[1052]: 2026/09/20 12:47:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5251lb # [ 70.501072] niks3-server[1052]: 2026/09/20 12:47:01 INFO Signed narinfos id=24 count=15252worker1 # [ 70.439809] nix-grpc-daemon[934]: time=2026-09-20T12:47:00.926Z level=INFO msg="Uploading 1 narinfos"5253lb # [ 70.516843] niks3-server[1052]: 2026/09/20 12:47:01 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5254lb # [ 70.524634] niks3-server[1052]: 2026/09/20 12:47:01 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5255lb # [ 70.542860] niks3-server[1052]: 2026/09/20 12:47:01 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5256lb # [ 70.549498] niks3-server[1052]: 2026/09/20 12:47:01 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5257lb # [ 70.560096] niks3-server[1052]: 2026/09/20 12:47:01 INFO Completed upload id=245258worker1 # [ 70.498961] nix-grpc-daemon[934]: time=2026-09-20T12:47:00.985Z level=INFO msg="Upload complete. (282ms)"5259worker1 # [ 70.504615] nix-grpc-daemon[795]: ts=2026-09-20T12:47:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37600 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=11 outputs=15260worker1 # [ 70.518157] nix-daemon[908]: reaped child process 1062, status = succeeded5261client # copying 1 paths...5262worker1 # [ 70.550131] nix-grpc-daemon[795]: ts=2026-09-20T12:47:01Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:376005263client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5264worker2 # [ 70.639842] nix-grpc-daemon[791]: ts=2026-09-20T12:47:01Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:471045265worker2 # [ 70.651445] nix-grpc-daemon[791]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5266worker2 # [ 70.665124] nix-grpc-daemon[791]: ts=2026-09-20T12:47:01Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:47104 duration_s=0 paths=1 nar_bytes_out=1205267client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.91 seconds)5268(finished: subtest: an input only one worker has still reaches the builder, in 10.63 seconds)5269subtest: upload whose reference the worker lacks is completed from the cache5270client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache5271client # [ 71.082252] nix-daemon[952]: reaped child process 1082, status = succeeded5272client # this derivation will be built:5273client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5274client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5275client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache, in 0.59 seconds)5276client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache5277client # this derivation will be built:5278client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5279client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5280client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache, in 0.38 seconds)5281client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5282client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.18 seconds)5283worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5284worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.20 seconds)5285worker2 # [ 72.207722] systemd[1]: Stopping Nix worker-protocol over gRPC...5286worker2 # [ 72.739864] nix-grpc-daemon[791]: ts=2026-09-20T12:47:03Z level=info event=signal_exit5287worker2 # [ 72.986599] nix-grpc-daemon[791]: ts=2026-09-20T12:47:03Z level=info event=scheduler_yield5288worker1 # [ 72.919807] nix-grpc-daemon[795]: ts=2026-09-20T12:47:03Z level=info event=scheduler_restarting addr=lb:500515289lb # [ 72.984691] niks3-server[1052]: 2026/09/20 12:47:04 INFO lead: released remote=192.168.1.4:589325290worker2 # [ 72.996724] nix-grpc-daemon[791]: ts=2026-09-20T12:47:03Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:470945291worker2 # [ 73.001558] nix-grpc-daemon[791]: ts=2026-09-20T12:47:03Z level=info event=worker_gone addr=192.168.1.3:500515292worker2 # [ 73.004617] nix-grpc-daemon[791]: ts=2026-09-20T12:47:03Z level=info event=scheduler_restarting addr=lb:500515293worker2 # [ 73.007674] nix-grpc-daemon[791]: ts=2026-09-20T12:47:03Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:470945294worker2 # [ 73.010922] nix-grpc-daemon[791]: ts=2026-09-20T12:47:03Z level=info event=worker_gone addr=192.168.1.4:500515295worker1 # [ 73.120375] nix-grpc-daemon[795]: ts=2026-09-20T12:47:03Z level=info event=scheduler_connected addr=lb:500515296worker1 # [ 73.124580] nix-grpc-daemon[795]: ts=2026-09-20T12:47:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5297worker2 # [ 73.201745] nix-grpc-daemon[791]: ts=2026-09-20T12:47:04Z level=info event=scheduler_connected addr=lb:500515298worker2 # [ 73.205672] nix-grpc-daemon[791]: ts=2026-09-20T12:47:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5299worker1 # [ 73.325386] nix-grpc-daemon[795]: ts=2026-09-20T12:47:03Z level=info event=scheduler_connected addr=lb:500515300worker1 # [ 73.329318] nix-grpc-daemon[795]: ts=2026-09-20T12:47:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5301worker2 # [ 73.406323] nix-grpc-daemon[791]: ts=2026-09-20T12:47:04Z level=info event=scheduler_connected addr=lb:500515302worker2 # [ 73.408199] nix-grpc-daemon[791]: ts=2026-09-20T12:47:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5303worker1 # [ 73.530070] nix-grpc-daemon[795]: ts=2026-09-20T12:47:04Z level=info event=scheduler_connected addr=lb:500515304worker1 # [ 73.534070] nix-grpc-daemon[795]: ts=2026-09-20T12:47:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5305worker2 # [ 73.608965] nix-grpc-daemon[791]: ts=2026-09-20T12:47:04Z level=info event=scheduler_connected addr=lb:500515306worker2 # [ 73.611119] nix-grpc-daemon[791]: ts=2026-09-20T12:47:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5307worker1 # [ 73.735214] nix-grpc-daemon[795]: ts=2026-09-20T12:47:04Z level=info event=scheduler_connected addr=lb:500515308worker2 # [ 73.812728] nix-grpc-daemon[791]: ts=2026-09-20T12:47:04Z level=info event=scheduler_connected addr=lb:500515309worker1 # [ 73.740348] nix-grpc-daemon[795]: ts=2026-09-20T12:47:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5310worker2 # [ 73.814130] nix-grpc-daemon[791]: ts=2026-09-20T12:47:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5311worker2 # [ 73.991118] nix-grpc-daemon[791]: ts=2026-09-20T12:47:04Z level=info event=scheduler_restarting5312worker1 # [ 73.942629] nix-grpc-daemon[795]: ts=2026-09-20T12:47:04Z level=info event=scheduler_connected addr=lb:500515313worker1 # [ 73.945771] nix-grpc-daemon[795]: ts=2026-09-20T12:47:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5314worker2 # [ 74.027928] nix-grpc-daemon[791]: ts=2026-09-20T12:47:04Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5315worker2 # [ 74.083731] nix-daemon[860]: reaped child process 862, status = succeeded5316worker1 # [ 74.143322] nix-grpc-daemon[795]: ts=2026-09-20T12:47:04Z level=info event=scheduler_connected addr=lb:500515317worker1 # [ 74.147175] nix-grpc-daemon[795]: ts=2026-09-20T12:47:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5318worker1 # [ 74.348079] nix-grpc-daemon[795]: ts=2026-09-20T12:47:04Z level=info event=scheduler_connected addr=lb:500515319worker1 # [ 74.352108] nix-grpc-daemon[795]: ts=2026-09-20T12:47:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5320worker1 # [ 74.552870] nix-grpc-daemon[795]: ts=2026-09-20T12:47:05Z level=info event=scheduler_connected addr=lb:500515321worker1 # [ 74.556797] nix-grpc-daemon[795]: ts=2026-09-20T12:47:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5322worker2 # [ 74.659472] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5323worker2 # [ 74.664541] systemd[1]: Stopped Nix worker-protocol over gRPC.5324worker2 # [ 74.667404] systemd[1]: nix-grpc-daemon.service: Consumed 2.668s CPU time over 57.197s wall clock time, 60.9M memory peak, 92.9K incoming IP traffic, 227.4K outgoing IP traffic.5325worker2 # [ 74.678298] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5326worker2 # [ 74.683164] systemd[1]: Closed Nix worker-protocol over gRPC.5327worker2 # [ 74.686792] systemd[1]: nix-grpc-daemon.socket: Consumed 336us CPU time over 1min 5.048s wall clock time, 256K memory peak, 89.3K incoming IP traffic, 70.2K outgoing IP traffic.5328lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5329lb: must succeed: curl -sf localhost:9901/clusters5330lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5331lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.07 seconds)5332client: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25333worker1 # [ 74.757310] nix-grpc-daemon[795]: ts=2026-09-20T12:47:05Z level=info event=scheduler_connected addr=lb:500515334worker1 # [ 74.761448] nix-grpc-daemon[795]: ts=2026-09-20T12:47:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5335worker1 # [ 74.897267] nix-grpc-daemon[795]: ts=2026-09-20T12:47:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:376005336worker1 # [ 74.904200] nix-grpc-daemon[795]: ts=2026-09-20T12:47:05Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:376005337client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5338client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.24 seconds)5339worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5340worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.03 seconds)5341worker1 # [ 74.962879] nix-grpc-daemon[795]: ts=2026-09-20T12:47:05Z level=info event=scheduler_connected addr=lb:500515342worker1 # [ 74.966784] nix-grpc-daemon[795]: ts=2026-09-20T12:47:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5343worker2 # [ 75.085814] systemd[1]: Listening on Nix worker-protocol over gRPC.5344worker1 # [ 75.076484] systemd[1]: Stopping Nix worker-protocol over gRPC...5345worker1 # [ 75.167191] nix-grpc-daemon[795]: ts=2026-09-20T12:47:05Z level=info event=scheduler_connected addr=lb:500515346worker1 # [ 75.169267] nix-grpc-daemon[795]: ts=2026-09-20T12:47:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5347worker1 # [ 75.358050] nix-grpc-daemon[795]: ts=2026-09-20T12:47:05Z level=info event=scheduler_take_over5348lb # [ 75.422957] niks3-server[1052]: 2026/09/20 12:47:06 INFO lead: acquired remote=192.168.1.3:507965349worker1 # [ 75.369658] nix-grpc-daemon[795]: ts=2026-09-20T12:47:05Z level=info event=scheduler_connected addr=lb:500515350worker1 # [ 75.371331] nix-grpc-daemon[795]: ts=2026-09-20T12:47:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5351worker1 # [ 75.572338] nix-grpc-daemon[795]: ts=2026-09-20T12:47:06Z level=info event=scheduler_connected addr=lb:500515352worker1 # [ 75.576345] nix-grpc-daemon[795]: ts=2026-09-20T12:47:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5353worker1 # [ 75.605556] nix-grpc-daemon[795]: ts=2026-09-20T12:47:06Z level=info event=signal_exit5354worker1 # [ 75.776533] nix-grpc-daemon[795]: ts=2026-09-20T12:47:06Z level=info event=scheduler_connected addr=lb:500515355worker1 # [ 75.778472] nix-grpc-daemon[795]: ts=2026-09-20T12:47:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5356worker2 # [ 75.873996] systemd[1]: Starting scheduler-token.service...5357worker1 # [ 75.867899] nix-grpc-daemon[795]: ts=2026-09-20T12:47:06Z level=info event=scheduler_yield5358lb # [ 75.934219] niks3-server[1052]: 2026/09/20 12:47:07 INFO lead: released remote=192.168.1.3:507965359worker2 # [ 75.959701] systemd[1]: scheduler-token.service: Deactivated successfully.5360worker2 # [ 75.961533] systemd[1]: Finished scheduler-token.service.5361worker2 # [ 75.963177] systemd[1]: scheduler-token.service: Consumed 31ms CPU time over 88ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5362worker2 # [ 75.968380] systemd[1]: Starting Nix worker-protocol over gRPC...5363worker1 # [ 75.979285] nix-grpc-daemon[795]: ts=2026-09-20T12:47:06Z level=info event=scheduler_connected addr=lb:500515364worker1 # [ 75.980995] nix-grpc-daemon[795]: ts=2026-09-20T12:47:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5365worker2 # [ 76.119893] nix-grpc-daemon[1262]: ts=2026-09-20T12:47:07Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5366worker2 # [ 76.121786] nix-grpc-daemon[1262]: ts=2026-09-20T12:47:07Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5367worker2 # [ 76.125147] nix-grpc-daemon[1262]: ts=2026-09-20T12:47:07Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15368worker2 # [ 76.132297] nix-grpc-daemon[1262]: ts=2026-09-20T12:47:07Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5369worker2 # [ 76.133907] systemd[1]: Started Nix worker-protocol over gRPC.5370worker2 # [ 76.143661] nix-grpc-daemon[1262]: ts=2026-09-20T12:47:07Z level=info event=scheduler_take_over5371lb # [ 76.137738] niks3-server[1052]: 2026/09/20 12:47:07 INFO lead: acquired remote=192.168.1.4:535225372worker2 # [ 76.155470] nix-grpc-daemon[1262]: ts=2026-09-20T12:47:07Z level=info event=scheduler_connected addr=lb:500515373worker2 # [ 76.162080] nix-grpc-daemon[1262]: ts=2026-09-20T12:47:07Z level=info event=oidc_jwks provider=mock keys=15374worker2 # [ 76.163497] nix-grpc-daemon[1262]: ts=2026-09-20T12:47:07Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:480565375worker2 # [ 76.165167] nix-grpc-daemon[1262]: ts=2026-09-20T12:47:07Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05376worker1 # [ 76.181681] nix-grpc-daemon[795]: ts=2026-09-20T12:47:06Z level=info event=scheduler_connected addr=lb:500515377worker2 # [ 76.254100] nix-grpc-daemon[1262]: ts=2026-09-20T12:47:07Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:480565378worker2 # [ 76.257374] nix-grpc-daemon[1262]: ts=2026-09-20T12:47:07Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05379worker1 # [ 76.868621] nix-grpc-daemon[795]: ts=2026-09-20T12:47:07Z level=info event=scheduler_restarting5380worker2 # [ 76.947275] nix-grpc-daemon[1262]: ts=2026-09-20T12:47:07Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:480565381worker2 # [ 76.948845] nix-grpc-daemon[1262]: ts=2026-09-20T12:47:07Z level=info event=worker_gone addr=192.168.1.3:500515382worker1 # [ 76.887392] nix-grpc-daemon[795]: ts=2026-09-20T12:47:07Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5383worker1 # [ 76.908536] nix-daemon[908]: reaped child process 911, status = succeeded5384lb # [ 77.344570] envoy[910]: conn peer=[2001:db8:1::1]:48522 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3700 tx=1918 ms=29285385lb # [ 77.346430] envoy[910]: conn peer=[2001:db8:1::1]:48532 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=135386lb # [ 77.347809] envoy[910]: conn peer=[2001:db8:1::1]:48550 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=329 ms=805387lb # [ 77.349221] envoy[910]: conn peer=[2001:db8:1::1]:48546 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3701 tx=1804 ms=25185388lb # [ 77.350674] envoy[910]: conn peer=[2001:db8:1::4]:45302 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=58842 tx=3160 ms=562645389lb # [ 77.352133] envoy[910]: conn peer=[2001:db8:1::1]:45982 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=215390lb # [ 77.353578] envoy[910]: conn peer=192.168.1.3:48052 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=15919 tx=2923 ms=582005391worker1 # [ 77.748133] nix-grpc-daemon[795]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5392worker1 # [ 77.751714] nix-grpc-daemon[795]: E0000 00:00:1789908428.233144 885 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true5393worker1 # [ 77.756482] nix-grpc-daemon[795]: E0000 00:00:1789908428.233696 885 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5394worker1 # [ 78.537652] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5395worker1 # [ 78.542595] systemd[1]: Stopped Nix worker-protocol over gRPC.5396worker1 # [ 78.545372] systemd[1]: nix-grpc-daemon.service: Consumed 1.456s CPU time over 1min 1.262s wall clock time, 47.5M memory peak, 53.6K incoming IP traffic, 99.2K outgoing IP traffic.5397worker1 # [ 78.556476] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5398worker1 # [ 78.560675] systemd[1]: Closed Nix worker-protocol over gRPC.5399worker1 # [ 78.564226] systemd[1]: nix-grpc-daemon.socket: Consumed 296us CPU time over 1min 8.606s wall clock time, 256K memory peak, 60.8K incoming IP traffic, 54.5K outgoing IP traffic.5400lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5401lb: must succeed: curl -sf localhost:9901/clusters5402lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5403lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.05 seconds)5404client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25405worker2 # [ 78.922352] nix-grpc-daemon[1262]: ts=2026-09-20T12:47:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:480725406worker2 # [ 78.934763] nix-daemon[860]: accepted connection from pid 1262, user nix-grpc-daemon (trusted)5407worker2 # [ 78.951409] nix-grpc-daemon[1262]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 22 ms (attempt 1/5)5408worker2 # [ 78.980149] nix-grpc-daemon[1262]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 137 ms (attempt 2/5)5409worker2 # [ 79.123717] nix-grpc-daemon[1262]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 94 ms (attempt 3/5)5410worker2 # [ 79.224852] nix-grpc-daemon[1262]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 182 ms (attempt 4/5)5411worker2 # [ 79.413452] nix-grpc-daemon[1262]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5412client # copying 2 paths...5413client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5414worker2 # [ 79.437179] nix-grpc-daemon[1262]: ts=2026-09-20T12:47:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:480725415client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5416worker2 # [ 79.447692] nix-daemon[860]: accepted connection from pid 1262, user nix-grpc-daemon (trusted)5417worker2 # [ 79.472659] nix-grpc-daemon[1262]: ts=2026-09-20T12:47:10Z level=info event=niks3_push_started pid=13195418worker2 # [ 79.664629] nix-grpc-daemon[1319]: time=2026-09-20T12:47:10.573Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""5419worker2 # [ 79.825394] nix-daemon[860]: accepted connection from pid 1332, user nix-grpc-daemon (trusted)5420worker2 # [ 79.836931] nix-daemon[860]: reaped child process 1335, status = succeeded5421lb # [ 79.844477] niks3-server[1052]: 2026/09/20 12:47:10 INFO Received uploads request method=POST path=/api/pending_closures5422lb # [ 79.853326] niks3-server[1052]: 2026/09/20 12:47:10 INFO Received uploads request method=POST path=/api/pending_closures5423worker2 # [ 79.866732] nix-grpc-daemon[1319]: time=2026-09-20T12:47:10.775Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5424worker2 # [ 79.868264] nix-grpc-daemon[1319]: time=2026-09-20T12:47:10.777Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5425worker2 # [ 79.870173] nix-grpc-daemon[1319]: time=2026-09-20T12:47:10.779Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5426lb # [ 79.900457] niks3-server[1052]: 2026/09/20 12:47:10 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5427lb # [ 79.919059] niks3-server[1052]: 2026/09/20 12:47:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5428lb # [ 79.921085] niks3-server[1052]: 2026/09/20 12:47:11 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5429lb # [ 79.927143] niks3-server[1052]: 2026/09/20 12:47:11 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5430lb # [ 79.936490] niks3-server[1052]: 2026/09/20 12:47:11 INFO Signed narinfos id=25 count=15431lb # [ 79.937879] niks3-server[1052]: 2026/09/20 12:47:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5432lb # [ 79.940027] niks3-server[1052]: 2026/09/20 12:47:11 INFO Signed narinfos id=26 count=25433worker2 # [ 79.948247] nix-grpc-daemon[1319]: time=2026-09-20T12:47:10.856Z level=INFO msg="Uploading 3 narinfos"5434lb # [ 79.946852] niks3-server[1052]: 2026/09/20 12:47:11 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5435lb # [ 79.958971] niks3-server[1052]: 2026/09/20 12:47:11 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5436lb # [ 79.963057] niks3-server[1052]: 2026/09/20 12:47:11 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5437lb # [ 79.970105] niks3-server[1052]: 2026/09/20 12:47:11 INFO Completed upload id=255438lb # [ 79.972344] niks3-server[1052]: 2026/09/20 12:47:11 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5439lb # [ 79.974883] niks3-server[1052]: 2026/09/20 12:47:11 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5440lb # [ 79.978038] niks3-server[1052]: 2026/09/20 12:47:11 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5441lb # [ 79.980299] niks3-server[1052]: 2026/09/20 12:47:11 INFO Completed upload id=265442worker2 # [ 79.986828] nix-grpc-daemon[1319]: time=2026-09-20T12:47:10.895Z level=INFO msg="Upload complete. (318ms)"5443worker2 # [ 79.992650] nix-grpc-daemon[1262]: ts=2026-09-20T12:47:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48072 duration_s=0 paths=2 nar_bytes_in=3125444worker2 # [ 80.005450] nix-daemon[860]: reaped child process 1317, status = succeeded5445client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.33 seconds)5446worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5447worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.03 seconds)5448worker1 # [ 80.045675] systemd[1]: Listening on Nix worker-protocol over gRPC.5449worker1 # [ 80.052858] systemd[1]: Starting Nix worker-protocol over gRPC...5450worker1 # [ 80.229777] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:10Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5451worker1 # [ 80.232240] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:10Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5452worker1 # [ 80.235770] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:10Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15453worker1 # [ 80.242847] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:10Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5454worker1 # [ 80.246508] systemd[1]: Started Nix worker-protocol over gRPC.5455lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5456lb: must succeed: curl -sf localhost:9901/clusters5457worker1 # [ 80.262028] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:10Z level=info event=scheduler_connected addr=lb:500515458worker2 # [ 80.336299] nix-grpc-daemon[1262]: ts=2026-09-20T12:47:11Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:480565459worker2 # [ 80.340476] nix-grpc-daemon[1262]: ts=2026-09-20T12:47:11Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05460lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5461lb: must succeed: curl -sf localhost:9901/clusters5462lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5463lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.14 seconds)5464(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.68 seconds)5465subtest: scheduler down: the other node takes the lock and keeps it5466lb: must succeed: curl -sf localhost:9901/clusters5467lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5468lb: must succeed: curl -sf localhost:9901/clusters5469lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5470lb: must succeed: curl -sf localhost:9901/clusters5471lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5472lb: must succeed: curl -sf localhost:9901/clusters5473lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5474worker2 # [ 81.668398] systemd[1]: Stopping Nix worker-protocol over gRPC...5475worker2 # [ 82.140342] nix-grpc-daemon[1262]: ts=2026-09-20T12:47:13Z level=info event=signal_exit5476worker2 # [ 82.149703] nix-grpc-daemon[1262]: ts=2026-09-20T12:47:13Z level=info event=scheduler_yield5477lb # [ 82.146982] niks3-server[1052]: 2026/09/20 12:47:13 INFO lead: released remote=192.168.1.4:535225478worker1 # [ 82.084756] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:12Z level=info event=scheduler_restarting addr=lb:500515479worker2 # [ 82.155420] nix-grpc-daemon[1262]: ts=2026-09-20T12:47:13Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:480565480worker2 # [ 82.160691] nix-grpc-daemon[1262]: ts=2026-09-20T12:47:13Z level=info event=worker_gone addr=192.168.1.3:500515481worker2 # [ 82.164877] nix-grpc-daemon[1262]: ts=2026-09-20T12:47:13Z level=info event=scheduler_restarting addr=lb:500515482worker2 # [ 82.170312] nix-grpc-daemon[1262]: ts=2026-09-20T12:47:13Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:480565483worker2 # [ 82.173632] nix-grpc-daemon[1262]: ts=2026-09-20T12:47:13Z level=info event=worker_gone addr=192.168.1.4:500515484worker1 # [ 82.284575] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:12Z level=info event=scheduler_connected addr=lb:500515485worker1 # [ 82.288712] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5486worker2 # [ 82.361698] nix-grpc-daemon[1262]: ts=2026-09-20T12:47:13Z level=info event=scheduler_connected addr=lb:500515487worker2 # [ 82.365532] nix-grpc-daemon[1262]: ts=2026-09-20T12:47:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5488worker1 # [ 82.489604] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:12Z level=info event=scheduler_connected addr=lb:500515489worker2 # [ 82.564543] nix-grpc-daemon[1262]: ts=2026-09-20T12:47:13Z level=info event=scheduler_connected addr=lb:500515490worker1 # [ 82.493709] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5491worker2 # [ 82.567720] nix-grpc-daemon[1262]: ts=2026-09-20T12:47:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5492worker1 # [ 82.694594] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:13Z level=info event=scheduler_connected addr=lb:500515493worker2 # [ 82.766459] nix-grpc-daemon[1262]: ts=2026-09-20T12:47:13Z level=info event=scheduler_connected addr=lb:500515494worker1 # [ 82.698713] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5495worker2 # [ 82.769676] nix-grpc-daemon[1262]: ts=2026-09-20T12:47:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5496worker2 # [ 82.968189] nix-grpc-daemon[1262]: ts=2026-09-20T12:47:13Z level=info event=scheduler_connected addr=lb:500515497worker1 # [ 82.899536] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:13Z level=info event=scheduler_connected addr=lb:500515498worker2 # [ 82.972802] nix-grpc-daemon[1262]: ts=2026-09-20T12:47:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5499worker1 # [ 82.903573] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5500worker2 # [ 83.154842] nix-grpc-daemon[1262]: ts=2026-09-20T12:47:14Z level=info event=scheduler_restarting5501worker2 # [ 83.167130] nix-grpc-daemon[1262]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5502worker2 # [ 83.171244] nix-grpc-daemon[1262]: E0000 00:00:1789908434.075753 1305 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5503worker1 # [ 83.105736] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:13Z level=info event=scheduler_connected addr=lb:500515504worker1 # [ 83.109064] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5505worker2 # [ 83.180619] nix-grpc-daemon[1262]: E0000 00:00:1789908434.079954 1305 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5506worker2 # [ 83.195567] nix-grpc-daemon[1262]: ts=2026-09-20T12:47:14Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5507worker2 # [ 83.200596] nix-daemon[860]: reaped child process 1311, status = succeeded5508worker1 # [ 83.306796] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:13Z level=info event=scheduler_connected addr=lb:500515509worker1 # [ 83.310616] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5510worker1 # [ 83.511443] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:13Z level=info event=scheduler_connected addr=lb:500515511worker1 # [ 83.515314] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5512worker1 # [ 83.716095] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:14Z level=info event=scheduler_connected addr=lb:500515513worker1 # [ 83.720091] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5514worker1 # [ 83.920963] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:14Z level=info event=scheduler_connected addr=lb:500515515worker1 # [ 83.924870] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5516worker2 # [ 84.161995] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5517worker2 # [ 84.164961] systemd[1]: Stopped Nix worker-protocol over gRPC.5518worker2 # [ 84.169646] systemd[1]: nix-grpc-daemon.service: Consumed 332ms CPU time over 8.192s wall clock time, 30.6M memory peak, 19.4K incoming IP traffic, 26.1K outgoing IP traffic.5519worker2 # [ 84.182382] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5520worker2 # [ 84.186970] systemd[1]: Closed Nix worker-protocol over gRPC.5521worker2 # [ 84.190905] systemd[1]: nix-grpc-daemon.socket: Consumed 306us CPU time over 9.103s wall clock time, 256K memory peak, 19.9K incoming IP traffic, 14.5K outgoing IP traffic.5522worker1 # [ 84.126519] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:14Z level=info event=scheduler_connected addr=lb:500515523worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5524worker1 # [ 84.130238] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5525worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.07 seconds)5526worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5527worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5528worker1 # [ 84.330849] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:14Z level=info event=scheduler_connected addr=lb:500515529worker1 # [ 84.334748] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5530worker1 # [ 84.535686] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:15Z level=info event=scheduler_connected addr=lb:500515531worker1 # [ 84.539686] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5532worker1 # [ 84.740524] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:15Z level=info event=scheduler_connected addr=lb:500515533worker1 # [ 84.744462] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5534worker1 # [ 84.945363] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:15Z level=info event=scheduler_connected addr=lb:500515535worker1 # [ 84.949393] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5536worker1 # [ 85.151134] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:15Z level=info event=scheduler_connected addr=lb:500515537worker1 # [ 85.154946] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5538worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5539worker1 # [ 85.253855] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:15Z level=info event=scheduler_take_over5540lb # [ 85.318584] niks3-server[1052]: 2026/09/20 12:47:16 INFO lead: acquired remote=192.168.1.3:444345541worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5542worker1 # [ 85.355907] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:15Z level=info event=scheduler_connected addr=lb:500515543worker1 # [ 85.359959] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5544worker1 # [ 85.560563] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:16Z level=info event=scheduler_connected addr=lb:500515545worker1 # [ 85.568181] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:16Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:531845546worker1 # [ 85.572345] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:16Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05547worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5548worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5549client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&25550worker1 # [ 86.601618] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:531885551worker1 # [ 86.612889] nix-daemon[908]: accepted connection from pid 1105, user nix-grpc-daemon (trusted)5552client # don't know how to build these paths:5553client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5554worker1 # [ 86.627093] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:531885555worker1 # [ 86.638153] nix-grpc-daemon[1105]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 81 ms (attempt 1/5)5556worker1 # [ 86.724950] nix-grpc-daemon[1105]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 28 ms (attempt 2/5)5557worker1 # [ 86.758708] nix-grpc-daemon[1105]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 286 ms (attempt 3/5)5558worker1 # [ 87.050753] nix-grpc-daemon[1105]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 20 ms (attempt 4/5)5559worker1 # [ 87.076963] nix-grpc-daemon[1105]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5560worker1 # [ 87.102809] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:17Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05561worker1 # [ 87.104885] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:17Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=15562worker1 # [ 87.149561] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531885563worker1 # [ 87.158134] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:531885564client # copying 1 paths to 'lb:50051'...5565worker1 # [ 87.166644] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:531885566client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5567worker1 # [ 87.173072] nix-daemon[908]: accepted connection from pid 1105, user nix-grpc-daemon (trusted)5568worker1 # [ 87.221064] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:17Z level=info event=niks3_push_started pid=11785569lb # [ 87.345147] envoy[910]: conn peer=[2001:db8:1::1]:45984 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1008 tx=282 ms=10845570lb # [ 87.348798] envoy[910]: conn peer=[2001:db8:1::4]:57910 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=4478 tx=349 ms=70295571worker1 # [ 87.444133] nix-grpc-daemon[1178]: time=2026-09-20T12:47:17.929Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""5572worker1 # [ 87.623809] nix-daemon[908]: accepted connection from pid 1191, user nix-grpc-daemon (trusted)5573worker1 # [ 87.634232] nix-daemon[908]: reaped child process 1194, status = succeeded5574worker1 # [ 87.803191] nix-daemon[908]: accepted connection from pid 1196, user nix-grpc-daemon (trusted)5575worker1 # [ 87.814731] nix-daemon[908]: reaped child process 1199, status = succeeded5576lb # [ 87.898928] niks3-server[1052]: 2026/09/20 12:47:18 INFO Received uploads request method=POST path=/api/pending_closures5577worker1 # [ 87.849688] nix-grpc-daemon[1178]: time=2026-09-20T12:47:18.334Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5578worker1 # [ 87.851103] nix-grpc-daemon[1178]: time=2026-09-20T12:47:18.335Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5579lb # [ 87.946177] niks3-server[1052]: 2026/09/20 12:47:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5580lb # [ 87.953439] niks3-server[1052]: 2026/09/20 12:47:19 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5581lb # [ 87.966409] niks3-server[1052]: 2026/09/20 12:47:19 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5582worker1 # [ 87.909759] nix-grpc-daemon[1178]: time=2026-09-20T12:47:18.395Z level=INFO msg="Uploading 1 narinfos"5583lb # [ 87.973976] niks3-server[1052]: 2026/09/20 12:47:19 INFO Signed narinfos id=27 count=15584lb # [ 87.984184] niks3-server[1052]: 2026/09/20 12:47:19 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5585lb # [ 87.987792] niks3-server[1052]: 2026/09/20 12:47:19 INFO Completed upload id=275586worker1 # [ 87.923354] nix-grpc-daemon[1178]: time=2026-09-20T12:47:18.409Z level=INFO msg="Upload complete. (475ms)"5587worker1 # [ 87.925566] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53188 duration_s=0 paths=1 nar_bytes_in=5205588lb # [ 87.993102] niks3-server[1052]: 2026/09/20 12:47:19 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5589worker1 # [ 87.930553] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531885590worker1 # [ 87.940107] nix-daemon[908]: accepted connection from pid 1105, user nix-grpc-daemon (trusted)5591worker1 # [ 87.944552] nix-daemon[908]: reaped child process 1176, status = succeeded5592worker1 # [ 87.949469] nix-daemon[908]: accepted connection from pid 1105, user nix-grpc-daemon (trusted)5593client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5594client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5595client # farm-a> LOG-farm-a-on-standby5596worker1 # [ 89.376912] nix-daemon[908]: reaped child process 1203, status = succeeded5597worker1 # [ 89.534861] nix-daemon[908]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5598worker1 # [ 89.545635] nix-daemon[908]: reaped child process 1221, status = succeeded5599lb # [ 89.628683] niks3-server[1052]: 2026/09/20 12:47:20 INFO Received uploads request method=POST path=/api/pending_closures5600worker1 # [ 89.577163] nix-grpc-daemon[1178]: time=2026-09-20T12:47:20.063Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5601worker1 # [ 89.580058] nix-grpc-daemon[1178]: time=2026-09-20T12:47:20.065Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5602lb # [ 89.666357] niks3-server[1052]: 2026/09/20 12:47:20 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5603lb # [ 89.681563] niks3-server[1052]: 2026/09/20 12:47:20 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5604lb # [ 89.691365] niks3-server[1052]: 2026/09/20 12:47:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5605lb # [ 89.694583] niks3-server[1052]: 2026/09/20 12:47:20 INFO Signed narinfos id=28 count=15606worker1 # [ 89.633796] nix-grpc-daemon[1178]: time=2026-09-20T12:47:20.119Z level=INFO msg="Uploading 1 narinfos"5607lb # [ 89.702526] niks3-server[1052]: 2026/09/20 12:47:20 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5608lb # [ 89.709712] niks3-server[1052]: 2026/09/20 12:47:20 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5609lb # [ 89.715093] niks3-server[1052]: 2026/09/20 12:47:20 INFO Completed upload id=285610lb # [ 89.716733] niks3-server[1052]: 2026/09/20 12:47:20 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5611worker1 # [ 89.651171] nix-grpc-daemon[1178]: time=2026-09-20T12:47:20.137Z level=INFO msg="Upload complete. (293ms)"5612worker1 # [ 89.664256] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53188 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15613worker1 # [ 89.679232] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:20Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05614worker1 # [ 89.681706] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:20Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=15615worker1 # [ 89.684279] nix-daemon[908]: reaped child process 1201, status = succeeded5616worker1 # [ 89.724605] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531885617worker1 # [ 89.736784] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:531885618worker1 # [ 89.750299] nix-grpc-daemon[1178]: time=2026-09-20T12:47:20.236Z level=INFO msg="All 1 paths already cached"5619client # copying 1 paths to 'lb:50051'...5620client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5621worker1 # [ 89.758332] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:531885622worker1 # [ 89.765897] nix-daemon[908]: accepted connection from pid 1105, user nix-grpc-daemon (trusted)5623worker1 # [ 89.944734] nix-daemon[908]: accepted connection from pid 1227, user nix-grpc-daemon (trusted)5624worker1 # [ 89.957265] nix-daemon[908]: reaped child process 1230, status = succeeded5625worker1 # [ 90.100595] nix-daemon[908]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)5626worker1 # [ 90.112276] nix-daemon[908]: reaped child process 1235, status = succeeded5627lb # [ 90.194597] niks3-server[1052]: 2026/09/20 12:47:21 INFO Received uploads request method=POST path=/api/pending_closures5628worker1 # [ 90.139313] nix-grpc-daemon[1178]: time=2026-09-20T12:47:20.625Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5629worker1 # [ 90.140857] nix-grpc-daemon[1178]: time=2026-09-20T12:47:20.627Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5630lb # [ 90.228091] niks3-server[1052]: 2026/09/20 12:47:21 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5631lb # [ 90.237205] niks3-server[1052]: 2026/09/20 12:47:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5632worker1 # [ 90.179098] nix-grpc-daemon[1178]: time=2026-09-20T12:47:20.665Z level=INFO msg="Uploading 1 narinfos"5633lb # [ 90.243289] niks3-server[1052]: 2026/09/20 12:47:21 INFO Signed narinfos id=29 count=15634lb # [ 90.250498] niks3-server[1052]: 2026/09/20 12:47:21 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5635lb # [ 90.259986] niks3-server[1052]: 2026/09/20 12:47:21 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5636worker1 # [ 90.202050] nix-grpc-daemon[1178]: time=2026-09-20T12:47:20.688Z level=INFO msg="Upload complete. (423ms)"5637lb # [ 90.266769] niks3-server[1052]: 2026/09/20 12:47:21 INFO Completed upload id=295638lb # [ 90.269034] niks3-server[1052]: 2026/09/20 12:47:21 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5639worker1 # [ 90.205074] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53188 duration_s=0 paths=1 nar_bytes_in=6405640worker1 # [ 90.207630] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531885641worker1 # [ 90.214332] nix-daemon[908]: accepted connection from pid 1105, user nix-grpc-daemon (trusted)5642worker1 # [ 90.216750] nix-daemon[908]: reaped child process 1225, status = succeeded5643client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5644worker1 # [ 90.223096] nix-daemon[908]: accepted connection from pid 1105, user nix-grpc-daemon (trusted)5645client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5646client # farm-b> LOG-farm-b-on-standby5647worker1 # [ 91.236470] nix-daemon[908]: reaped child process 1239, status = succeeded5648worker1 # [ 91.392554] nix-daemon[908]: accepted connection from pid 1254, user nix-grpc-daemon (trusted)5649worker1 # [ 91.405150] nix-daemon[908]: reaped child process 1257, status = succeeded5650lb # [ 91.488389] niks3-server[1052]: 2026/09/20 12:47:22 INFO Received uploads request method=POST path=/api/pending_closures5651worker1 # [ 91.434883] nix-grpc-daemon[1178]: time=2026-09-20T12:47:21.921Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5652worker1 # [ 91.436344] nix-grpc-daemon[1178]: time=2026-09-20T12:47:21.922Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"5653lb # [ 91.524680] niks3-server[1052]: 2026/09/20 12:47:22 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5654lb # [ 91.535275] niks3-server[1052]: 2026/09/20 12:47:22 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst5655lb # [ 91.547944] niks3-server[1052]: 2026/09/20 12:47:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5656lb # [ 91.554462] niks3-server[1052]: 2026/09/20 12:47:22 INFO Signed narinfos id=30 count=15657worker1 # [ 91.493203] nix-grpc-daemon[1178]: time=2026-09-20T12:47:21.979Z level=INFO msg="Uploading 1 narinfos"5658lb # [ 91.557186] niks3-server[1052]: 2026/09/20 12:47:22 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls5659lb # [ 91.567210] niks3-server[1052]: 2026/09/20 12:47:22 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5660lb # [ 91.573131] niks3-server[1052]: 2026/09/20 12:47:22 INFO Completed upload id=305661worker1 # [ 91.509089] nix-grpc-daemon[1178]: time=2026-09-20T12:47:21.995Z level=INFO msg="Upload complete. (279ms)"5662lb # [ 91.575987] niks3-server[1052]: 2026/09/20 12:47:22 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo5663worker1 # [ 91.519706] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53188 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=15664worker1 # [ 91.529177] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:22Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=05665worker1 # [ 91.532308] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:22Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=15666worker1 # [ 91.540981] nix-daemon[908]: reaped child process 1237, status = succeeded5667worker1 # [ 91.571853] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531885668worker1 # [ 91.584104] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:531885669worker1 # [ 91.597238] nix-grpc-daemon[1178]: time=2026-09-20T12:47:22.083Z level=INFO msg="All 2 paths already cached"5670client # copying 1 paths to 'lb:50051'...5671client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...5672worker1 # [ 91.604747] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:531885673worker1 # [ 91.615224] nix-daemon[908]: accepted connection from pid 1105, user nix-grpc-daemon (trusted)5674worker1 # [ 91.781653] nix-daemon[908]: accepted connection from pid 1263, user nix-grpc-daemon (trusted)5675worker1 # [ 91.794217] nix-daemon[908]: reaped child process 1266, status = succeeded5676worker1 # [ 91.965794] nix-daemon[908]: accepted connection from pid 1268, user nix-grpc-daemon (trusted)5677worker1 # [ 91.978117] nix-daemon[908]: reaped child process 1271, status = succeeded5678lb # [ 92.060208] niks3-server[1052]: 2026/09/20 12:47:23 INFO Received uploads request method=POST path=/api/pending_closures5679worker1 # [ 92.000278] nix-grpc-daemon[1178]: time=2026-09-20T12:47:22.486Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5680worker1 # [ 92.001749] nix-grpc-daemon[1178]: time=2026-09-20T12:47:22.488Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"5681lb # [ 92.078736] niks3-server[1052]: 2026/09/20 12:47:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5682lb # [ 92.081191] niks3-server[1052]: 2026/09/20 12:47:23 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst5683lb # [ 92.083977] niks3-server[1052]: 2026/09/20 12:47:23 INFO Signed narinfos id=31 count=15684worker1 # [ 92.019693] nix-grpc-daemon[1178]: time=2026-09-20T12:47:22.506Z level=INFO msg="Uploading 1 narinfos"5685lb # [ 92.085701] niks3-server[1052]: 2026/09/20 12:47:23 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls5686lb # [ 92.095073] niks3-server[1052]: 2026/09/20 12:47:23 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5687lb # [ 92.100026] niks3-server[1052]: 2026/09/20 12:47:23 INFO Completed upload id=315688worker1 # [ 92.035578] nix-grpc-daemon[1178]: time=2026-09-20T12:47:22.521Z level=INFO msg="Upload complete. (410ms)"5689lb # [ 92.101742] niks3-server[1052]: 2026/09/20 12:47:23 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo5690worker1 # [ 92.037065] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53188 duration_s=0 paths=1 nar_bytes_in=7685691worker1 # [ 92.042130] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531885692worker1 # [ 92.047526] nix-daemon[908]: accepted connection from pid 1105, user nix-grpc-daemon (trusted)5693worker1 # [ 92.049766] nix-daemon[908]: reaped child process 1261, status = succeeded5694worker1 # [ 92.055046] nix-daemon[908]: accepted connection from pid 1105, user nix-grpc-daemon (trusted)5695client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...5696client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5697client # farm-top> LOG-farm-top-on-standby5698worker1 # [ 93.069181] nix-daemon[908]: reaped child process 1275, status = succeeded5699worker1 # [ 93.224673] nix-daemon[908]: accepted connection from pid 1290, user nix-grpc-daemon (trusted)5700worker1 # [ 93.236692] nix-daemon[908]: reaped child process 1293, status = succeeded5701lb # [ 93.318902] niks3-server[1052]: 2026/09/20 12:47:24 INFO Received uploads request method=POST path=/api/pending_closures5702worker1 # [ 93.263759] nix-grpc-daemon[1178]: time=2026-09-20T12:47:23.749Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5703worker1 # [ 93.265269] nix-grpc-daemon[1178]: time=2026-09-20T12:47:23.751Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"5704lb # [ 93.352583] niks3-server[1052]: 2026/09/20 12:47:24 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5705lb # [ 93.364268] niks3-server[1052]: 2026/09/20 12:47:24 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst5706lb # [ 93.378077] niks3-server[1052]: 2026/09/20 12:47:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5707lb # [ 93.382972] niks3-server[1052]: 2026/09/20 12:47:24 INFO Signed narinfos id=32 count=15708worker1 # [ 93.319068] nix-grpc-daemon[1178]: time=2026-09-20T12:47:23.805Z level=INFO msg="Uploading 1 narinfos"5709lb # [ 93.385852] niks3-server[1052]: 2026/09/20 12:47:24 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls5710lb # [ 93.395262] niks3-server[1052]: 2026/09/20 12:47:24 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5711lb # [ 93.398498] niks3-server[1052]: 2026/09/20 12:47:24 INFO Completed upload id=325712worker1 # [ 93.334360] nix-grpc-daemon[1178]: time=2026-09-20T12:47:23.820Z level=INFO msg="Upload complete. (272ms)"5713lb # [ 93.401041] niks3-server[1052]: 2026/09/20 12:47:24 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo5714worker1 # [ 93.342198] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53188 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=15715worker1 # [ 93.353834] nix-daemon[908]: reaped child process 1273, status = succeeded5716client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&2, in 7.02 seconds)5717client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath5718client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath, in 0.23 seconds)5719worker2 # [ 93.725907] systemd[1]: Listening on Nix worker-protocol over gRPC.5720worker2 # [ 93.728807] systemd[1]: Starting scheduler-token.service...5721worker2 # [ 93.795875] systemd[1]: scheduler-token.service: Deactivated successfully.5722worker2 # [ 93.798196] systemd[1]: Finished scheduler-token.service.5723worker2 # [ 93.798937] systemd[1]: scheduler-token.service: Consumed 18ms CPU time over 68ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5724worker2 # [ 93.804421] systemd[1]: Starting Nix worker-protocol over gRPC...5725worker2 # [ 93.946949] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:24Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5726worker2 # [ 93.948754] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:24Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5727worker2 # [ 93.951884] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:24Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15728worker2 # [ 93.958804] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:24Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5729worker2 # [ 93.960398] systemd[1]: Started Nix worker-protocol over gRPC.5730lb: must succeed: curl -sf localhost:9901/clusters5731worker2 # [ 93.978749] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:24Z level=info event=scheduler_connected addr=lb:500515732worker1 # [ 93.917175] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:24Z level=info event=oidc_jwks provider=mock keys=15733worker1 # [ 93.920707] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:24Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:531845734worker1 # [ 93.924575] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:24Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05735lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5736worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5737worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5738lb: must succeed: curl -sf localhost:9901/clusters5739lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5740lb: must succeed: curl -sf localhost:9901/clusters5741lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5742worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5743worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5744(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.80 seconds)5745subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5746client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs5747client # [ 94.312990] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs.5748client # Running as unit: rs.service; invocation ID: 8d9f0dfff7f549179ffa9ffa6d384b3f5749client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs, in 0.07 seconds)5750worker1 # [ 94.461585] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:531885751client # [ 94.534479] nix[1182]: don't know how to build these paths:5752client # [ 94.535543] nix[1182]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5753worker1 # [ 94.472048] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:24Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=05754worker1 # [ 94.474240] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:24Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=15755worker1 # [ 94.515981] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531885756worker1 # [ 94.524387] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:531885757client # [ 94.600311] nix[1182]: copying 1 paths to 'lb:50051'...5758worker1 # [ 94.532049] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:531885759client # [ 94.603152] nix[1182]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...5760worker1 # [ 94.538563] nix-daemon[908]: accepted connection from pid 1105, user nix-grpc-daemon (trusted)5761worker1 # [ 94.702501] nix-daemon[908]: accepted connection from pid 1306, user nix-grpc-daemon (trusted)5762worker1 # [ 94.715171] nix-daemon[908]: reaped child process 1309, status = succeeded5763worker1 # [ 94.889522] nix-daemon[908]: accepted connection from pid 1311, user nix-grpc-daemon (trusted)5764worker1 # [ 94.901650] nix-daemon[908]: reaped child process 1314, status = succeeded5765lb # [ 94.982792] niks3-server[1052]: 2026/09/20 12:47:26 INFO Received uploads request method=POST path=/api/pending_closures5766worker1 # [ 94.929289] nix-grpc-daemon[1178]: time=2026-09-20T12:47:25.415Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5767worker1 # [ 94.930781] nix-grpc-daemon[1178]: time=2026-09-20T12:47:25.415Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"5768lb # [ 95.003615] niks3-server[1052]: 2026/09/20 12:47:26 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst5769lb # [ 95.009059] niks3-server[1052]: 2026/09/20 12:47:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5770lb # [ 95.012217] niks3-server[1052]: 2026/09/20 12:47:26 INFO Signed narinfos id=33 count=15771worker1 # [ 94.949342] nix-grpc-daemon[1178]: time=2026-09-20T12:47:25.435Z level=INFO msg="Uploading 1 narinfos"5772lb # [ 95.018030] niks3-server[1052]: 2026/09/20 12:47:26 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls5773lb # [ 95.022211] niks3-server[1052]: 2026/09/20 12:47:26 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5774lb # [ 95.027370] niks3-server[1052]: 2026/09/20 12:47:26 INFO Completed upload id=335775worker1 # [ 94.964470] nix-grpc-daemon[1178]: time=2026-09-20T12:47:25.450Z level=INFO msg="Upload complete. (421ms)"5776lb # [ 95.029552] niks3-server[1052]: 2026/09/20 12:47:26 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo5777worker1 # [ 94.966481] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53188 duration_s=0 paths=1 nar_bytes_in=4165778worker1 # [ 94.970114] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531885779worker1 # [ 94.976630] nix-daemon[908]: accepted connection from pid 1105, user nix-grpc-daemon (trusted)5780worker1 # [ 94.978826] nix-daemon[908]: reaped child process 1304, status = succeeded5781worker1 # [ 94.983885] nix-daemon[908]: accepted connection from pid 1105, user nix-grpc-daemon (trusted)5782client # [ 95.052085] nix[1182]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...5783lb: must succeed: curl -sf localhost:9901/clusters5784lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5785lb: must succeed: curl -sf localhost:9901/clusters5786lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5787lb: must succeed: curl -sf localhost:9901/clusters5788lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5789lb: must succeed: curl -sf localhost:9901/clusters5790lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5791worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5792worker2 # [ 97.150089] systemd-journald[288]: Received client request to rotate journal, rotating.5793worker2 # [ 97.168538] systemd-journald[288]: Vacuuming done, freed 0B of archived journals from /var/log/journal/b86f42fc155e493f91b324030cfa4cf4.5794worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.07 seconds)5795worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5796worker1 # [ 97.015572] systemd[1]: Stopping Nix worker-protocol over gRPC...5797worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)5798??? Warning (UserWarning): sleep(): The 'secs' argument is deprecated. Use 'duration' instead.5799 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 5735800client: must succeed: sleep 5.05801worker1 # [ 97.248398] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:27Z level=info event=draining5802lb # [ 97.345285] envoy[910]: conn peer=[2001:db8:1::1]:41236 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9577 tx=3815 ms=67565803client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5804worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5805worker1 # [ 102.404240] nix-daemon[908]: reaped child process 1318, status = succeeded5806worker1 # [ 102.561697] nix-daemon[908]: accepted connection from pid 1358, user nix-grpc-daemon (trusted)5807worker1 # [ 102.572299] nix-daemon[908]: reaped child process 1361, status = succeeded5808lb # [ 102.655142] niks3-server[1052]: 2026/09/20 12:47:33 INFO Received uploads request method=POST path=/api/pending_closures5809worker1 # [ 102.595971] nix-grpc-daemon[1178]: time=2026-09-20T12:47:33.082Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5810worker1 # [ 102.597592] nix-grpc-daemon[1178]: time=2026-09-20T12:47:33.083Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"5811lb # [ 102.695513] niks3-server[1052]: 2026/09/20 12:47:33 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5812lb # [ 102.703913] niks3-server[1052]: 2026/09/20 12:47:33 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5813lb # [ 102.714962] niks3-server[1052]: 2026/09/20 12:47:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5814lb # [ 102.719435] niks3-server[1052]: 2026/09/20 12:47:33 INFO Signed narinfos id=34 count=15815worker1 # [ 102.657597] nix-grpc-daemon[1178]: time=2026-09-20T12:47:33.143Z level=INFO msg="Uploading 1 narinfos"5816lb # [ 102.726951] niks3-server[1052]: 2026/09/20 12:47:33 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls5817lb # [ 102.734795] niks3-server[1052]: 2026/09/20 12:47:33 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5818lb # [ 102.739090] niks3-server[1052]: 2026/09/20 12:47:33 INFO Completed upload id=345819worker1 # [ 102.674838] nix-grpc-daemon[1178]: time=2026-09-20T12:47:33.161Z level=INFO msg="Upload complete. (278ms)"5820lb # [ 102.740739] niks3-server[1052]: 2026/09/20 12:47:33 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo5821worker1 # [ 102.677181] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53188 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=15822worker1 # [ 102.684785] nix-daemon[908]: reaped child process 1316, status = succeeded5823client # [ 102.784824] systemd[1]: rs.service: Deactivated successfully.5824client # [ 102.788701] systemd[1]: rs.service: Consumed 123ms CPU time over 8.474s wall clock time, 39.1M memory peak, 8.6K incoming IP traffic, 9K outgoing IP traffic.5825worker1 # [ 103.248767] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:33Z level=info event=signal_exit5826worker1 # [ 103.473500] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:33Z level=info event=scheduler_yield5827lb # [ 103.539522] niks3-server[1052]: 2026/09/20 12:47:34 INFO lead: released remote=192.168.1.3:444345828worker1 # [ 103.475580] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:33Z level=info event=scheduler_restarting addr=lb:500515829worker2 # [ 103.546884] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:34Z level=info event=scheduler_restarting addr=lb:500515830worker1 # [ 103.477641] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:33Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:531845831worker1 # [ 103.479251] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:33Z level=info event=worker_gone addr=192.168.1.4:500515832worker1 # [ 103.480439] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:33Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:531845833worker1 # [ 103.481692] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:33Z level=info event=worker_gone addr=192.168.1.3:500515834worker1 # [ 103.676633] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:34Z level=info event=scheduler_connected addr=lb:500515835worker2 # [ 103.751480] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:34Z level=info event=scheduler_connected addr=lb:500515836worker1 # [ 103.681264] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5837worker2 # [ 103.755685] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5838worker1 # [ 103.880891] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:34Z level=info event=scheduler_connected addr=lb:500515839worker1 # [ 103.882959] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5840worker2 # [ 103.956291] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:34Z level=info event=scheduler_connected addr=lb:500515841worker2 # [ 103.958343] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5842worker2 # [ 103.968477] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:34Z level=info event=scheduler_take_over5843lb # [ 103.962746] niks3-server[1052]: 2026/09/20 12:47:35 INFO lead: acquired remote=192.168.1.4:601805844worker1 # [ 104.083452] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:34Z level=info event=scheduler_connected addr=lb:500515845worker1 # [ 104.085604] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5846worker2 # [ 104.158834] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:35Z level=info event=scheduler_connected addr=lb:500515847worker2 # [ 104.160524] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5848worker1 # [ 104.286156] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:34Z level=info event=scheduler_connected addr=lb:500515849worker1 # [ 104.288389] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5850worker2 # [ 104.361799] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:35Z level=info event=scheduler_connected addr=lb:500515851worker2 # [ 104.365759] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5852worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.15 seconds)5853worker1 # [ 104.474501] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:34Z level=info event=scheduler_restarting5854client: waiting for success: ! systemctl is-active rs5855worker1 # [ 104.490574] nix-daemon[908]: reaped child process 1169, status = succeeded5856worker1 # [ 104.495819] nix-grpc-daemon[1105]: ts=2026-09-20T12:47:34Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5857worker2 # [ 104.568221] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:35Z level=info event=scheduler_connected addr=lb:500515858worker2 # [ 104.571726] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5859client: (finished: waiting for success: ! systemctl is-active rs, in 0.04 seconds)5860client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5861client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.03 seconds)5862client: must fail: journalctl -u rs | grep -q 'reconnecting'5863worker1 # [ 104.575342] nix-grpc-daemon[1105]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5864worker1 # [ 104.577303] nix-grpc-daemon[1105]: E0000 00:00:1789908455.061609 1148 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5865worker1 # [ 104.579716] nix-grpc-daemon[1105]: E0000 00:00:1789908455.063689 1148 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5866client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5867lb: must succeed: curl -sf localhost:9901/clusters5868lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5869worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5870worker2 # [ 104.769488] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:35Z level=info event=scheduler_connected addr=lb:500515871worker2 # [ 104.773653] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5872worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5873worker2 # [ 104.974222] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:35Z level=info event=scheduler_connected addr=lb:500515874worker2 # [ 104.978141] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5875worker2 # [ 105.178642] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:36Z level=info event=scheduler_connected addr=lb:500515876worker2 # [ 105.180976] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5877worker2 # [ 105.381586] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:36Z level=info event=scheduler_connected addr=lb:500515878worker2 # [ 105.388074] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:36Z level=info event=oidc_jwks provider=mock keys=15879worker2 # [ 105.389597] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:36Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:332405880worker2 # [ 105.391219] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:36Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05881lb: must succeed: curl -sf localhost:9901/clusters5882lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5883worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5884worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5885worker1 # [ 106.055865] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5886worker1 # [ 106.058723] systemd[1]: Stopped Nix worker-protocol over gRPC.5887worker1 # [ 106.061346] systemd[1]: nix-grpc-daemon.service: Consumed 1.243s CPU time over 26.001s wall clock time, 33.8M memory peak, 49.1K incoming IP traffic, 91.6K outgoing IP traffic.5888worker1 # [ 106.076181] systemd[1]: Starting Nix worker-protocol over gRPC...5889worker1 # [ 106.221953] nix-grpc-daemon[1368]: ts=2026-09-20T12:47:36Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5890worker1 # [ 106.224245] nix-grpc-daemon[1368]: ts=2026-09-20T12:47:36Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5891worker1 # [ 106.227770] nix-grpc-daemon[1368]: ts=2026-09-20T12:47:36Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15892worker1 # [ 106.234681] nix-grpc-daemon[1368]: ts=2026-09-20T12:47:36Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5893worker1 # [ 106.236232] systemd[1]: Started Nix worker-protocol over gRPC.5894worker1 # [ 106.249185] nix-grpc-daemon[1368]: ts=2026-09-20T12:47:36Z level=info event=scheduler_connected addr=lb:500515895worker2 # [ 106.322754] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:37Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:332405896worker2 # [ 106.326478] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:37Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05897lb: must succeed: curl -sf localhost:9901/clusters5898lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5899worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5900worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5901(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.78 seconds)5902subtest: scheduler restart mid-build: build finishes, no double build5903client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid5904client # [ 107.100782] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid.5905client # Running as unit: mid.service; invocation ID: 70f1c7609e2b484d85e1062e092a9c1b5906client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid, in 0.07 seconds)5907worker2 # [ 107.327859] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:332565908worker2 # [ 107.339100] nix-daemon[860]: accepted connection from pid 1359, user nix-grpc-daemon (trusted)5909client # [ 107.349393] nix[1218]: don't know how to build these paths:5910lb # [ 107.344823] envoy[910]: conn peer=[2001:db8:1::1]:55620 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2960 tx=1323 ms=82305911client # [ 107.350324] nix[1218]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5912lb # [ 107.346684] envoy[910]: conn peer=[2001:db8:1::3]:38160 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=5568 tx=1292 ms=242345913worker2 # [ 107.360530] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:38Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05914worker2 # [ 107.365571] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:38Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=15915worker2 # [ 107.404158] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:332565916worker2 # [ 107.415449] nix-grpc-daemon[1359]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 54 ms (attempt 1/5)5917worker2 # [ 107.476323] nix-grpc-daemon[1359]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 34 ms (attempt 2/5)5918worker2 # [ 107.510681] nix-grpc-daemon[1359]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 342 ms (attempt 3/5)5919worker2 # [ 107.858429] nix-grpc-daemon[1359]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 29 ms (attempt 4/5)5920worker2 # [ 107.893255] nix-grpc-daemon[1359]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5921worker2 # [ 107.911473] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:332565922client # [ 107.920122] nix[1218]: copying 1 paths to 'lb:50051'...5923client # [ 107.920954] nix[1218]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...5924worker2 # [ 107.922811] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:332565925worker2 # [ 107.933262] nix-daemon[860]: accepted connection from pid 1359, user nix-grpc-daemon (trusted)5926worker2 # [ 107.950992] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:38Z level=info event=niks3_push_started pid=14805927worker2 # [ 108.129063] nix-grpc-daemon[1480]: time=2026-09-20T12:47:39.037Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""5928worker2 # [ 108.288658] nix-daemon[860]: accepted connection from pid 1493, user nix-grpc-daemon (trusted)5929worker2 # [ 108.300435] nix-daemon[860]: reaped child process 1496, status = succeeded5930worker2 # [ 108.465966] nix-daemon[860]: accepted connection from pid 1498, user nix-grpc-daemon (trusted)5931worker2 # [ 108.478445] nix-daemon[860]: reaped child process 1505, status = succeeded5932lb # [ 108.487965] niks3-server[1052]: 2026/09/20 12:47:39 INFO Received uploads request method=POST path=/api/pending_closures5933worker2 # [ 108.504096] nix-grpc-daemon[1480]: time=2026-09-20T12:47:39.412Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5934worker2 # [ 108.505548] nix-grpc-daemon[1480]: time=2026-09-20T12:47:39.412Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"5935lb # [ 108.522998] niks3-server[1052]: 2026/09/20 12:47:39 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst5936lb # [ 108.529777] niks3-server[1052]: 2026/09/20 12:47:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5937lb # [ 108.534364] niks3-server[1052]: 2026/09/20 12:47:39 INFO Signed narinfos id=35 count=15938worker2 # [ 108.543428] nix-grpc-daemon[1480]: time=2026-09-20T12:47:39.452Z level=INFO msg="Uploading 1 narinfos"5939lb # [ 108.546555] niks3-server[1052]: 2026/09/20 12:47:39 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5940worker2 # [ 108.561211] nix-grpc-daemon[1480]: time=2026-09-20T12:47:39.470Z level=INFO msg="Upload complete. (428ms)"5941lb # [ 108.555645] niks3-server[1052]: 2026/09/20 12:47:39 INFO Completed upload id=355942worker2 # [ 108.564175] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33256 duration_s=0 paths=1 nar_bytes_in=4245943worker2 # [ 108.566220] nix-grpc-daemon[1480]: time=2026-09-20T12:47:39.473Z level=WARN msg="Failed to register uploaded object" key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo error="server returned 404: object is not pending upload\n"5944worker2 # [ 108.569562] nix-grpc-daemon[1359]: ts=2026-09-20T12:47:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:332565945lb # [ 108.566278] niks3-server[1052]: 2026/09/20 12:47:39 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls5946worker2 # [ 108.574455] nix-daemon[860]: accepted connection from pid 1359, user nix-grpc-daemon (trusted)5947worker2 # [ 108.579622] nix-daemon[860]: reaped child process 1478, status = succeeded5948worker2 # [ 108.584283] nix-daemon[860]: accepted connection from pid 1359, user nix-grpc-daemon (trusted)5949client # [ 108.582540] nix[1218]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5950lb: must succeed: curl -sf localhost:9901/clusters5951lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5952lb: must succeed: curl -sf localhost:9901/clusters5953lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5954worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5955worker2 # [ 110.863288] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1359 (nix-grpc-daemon) on client request.5956worker2 # [ 110.865288] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5957lb # [ 110.868881] niks3-server[1052]: 2026/09/20 12:47:41 INFO lead: released remote=192.168.1.4:601805958worker2 # [ 110.885926] systemd[1]: Starting scheduler-token.service...5959worker1 # [ 110.811460] nix-grpc-daemon[1368]: ts=2026-09-20T12:47:41Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5960client # [ 110.881456] nix[1218]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5961client # [ 110.885789] nix[1218]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5962worker2 # [ 110.901567] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5963worker2 # [ 110.903000] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5964worker2 # [ 110.904878] systemd[1]: nix-grpc-daemon.service: Consumed 431ms CPU time over 17.095s wall clock time, 31.2M memory peak, 17K incoming IP traffic, 31.3K outgoing IP traffic.5965worker2 # [ 110.913182] nix-daemon[860]: reaped child process 1507, status = succeeded5966worker2 # [ 110.914660] nix-daemon[860]: reaped child process 1471, status = succeeded5967worker2 # [ 110.923272] nix-daemon[1509]: unexpected Nix daemon error: error: interrupted by the user5968worker2 # [ 110.928687] nix-daemon[860]: reaped child process 1509, status = failed with exit code 15969worker2 # [ 110.973356] systemd[1]: scheduler-token.service: Deactivated successfully.5970worker2 # [ 110.974396] systemd[1]: Finished scheduler-token.service.5971worker2 # [ 110.975661] systemd[1]: scheduler-token.service: Consumed 18ms CPU time over 87ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5972worker2 # [ 110.978686] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.5973worker2 # [ 110.983516] systemd[1]: Starting scheduler-token.service...5974worker2 # [ 111.046293] systemd[1]: scheduler-token.service: Deactivated successfully.5975worker2 # [ 111.048307] systemd[1]: Finished scheduler-token.service.5976worker2 # [ 111.049443] systemd[1]: scheduler-token.service: Consumed 18ms CPU time over 62ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5977worker2 # [ 111.054157] systemd[1]: Starting Nix worker-protocol over gRPC...5978worker1 # [ 111.011836] nix-grpc-daemon[1368]: ts=2026-09-20T12:47:41Z level=info event=scheduler_connected addr=lb:500515979worker2 # [ 111.208393] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:42Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5980worker2 # [ 111.210207] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:42Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5981worker2 # [ 111.213615] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:42Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15982worker2 # [ 111.222056] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:42Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5983worker2 # [ 111.225142] systemd[1]: Started Nix worker-protocol over gRPC.5984worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.41 seconds)5985client: must succeed: sleep 5.05986worker2 # [ 111.235912] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:42Z level=info event=scheduler_take_over5987lb # [ 111.230113] niks3-server[1052]: 2026/09/20 12:47:42 INFO lead: acquired remote=192.168.1.4:533925988worker2 # [ 111.247050] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:42Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:332585989worker2 # [ 111.248812] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:42Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05990worker2 # [ 111.251840] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:42Z level=info event=scheduler_connected addr=lb:500515991worker2 # [ 111.256743] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:42Z level=info event=oidc_jwks provider=mock keys=15992worker2 # [ 111.258086] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:42Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:332585993worker2 # [ 111.259756] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:42Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05994worker2 # [ 111.393212] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:42Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05995worker2 # [ 111.398106] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:42Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=15996worker1 # [ 111.381147] nix-grpc-daemon[1368]: ts=2026-09-20T12:47:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446685997worker1 # [ 111.393192] nix-daemon[908]: accepted connection from pid 1368, user nix-grpc-daemon (trusted)5998worker1 # [ 111.411352] nix-grpc-daemon[1368]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 58 ms (attempt 1/5)5999worker1 # [ 111.475354] nix-grpc-daemon[1368]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 106 ms (attempt 2/5)6000worker1 # [ 111.582330] nix-grpc-daemon[1368]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 239 ms (attempt 3/5)6001worker1 # [ 111.828268] nix-grpc-daemon[1368]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 611 ms (attempt 4/5)6002worker1 # [ 112.446295] nix-grpc-daemon[1368]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6003worker1 # [ 112.462195] nix-grpc-daemon[1368]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' from 'http://lb:5751'...6004worker1 # [ 112.485339] nix-daemon[908]: accepted connection from pid 1368, user nix-grpc-daemon (trusted)6005worker1 # [ 112.498481] nix-daemon[908]: accepted connection from pid 1368, user nix-grpc-daemon (trusted)6006client # [ 112.566609] nix[1218]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6007client: (finished: must succeed: sleep 5.0, in 5.05 seconds)6008client: waiting for success: ! systemctl is-active mid6009worker1 # [ 116.517539] nix-grpc-daemon[1368]: ts=2026-09-20T12:47:47Z level=info event=niks3_push_started pid=14696010worker1 # [ 116.528177] nix-daemon[908]: reaped child process 1445, status = succeeded6011worker1 # [ 116.713464] nix-grpc-daemon[1469]: time=2026-09-20T12:47:47.199Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""6012worker1 # [ 116.881621] nix-daemon[908]: accepted connection from pid 1482, user nix-grpc-daemon (trusted)6013worker1 # [ 116.893560] nix-daemon[908]: reaped child process 1485, status = succeeded6014lb # [ 116.976387] niks3-server[1052]: 2026/09/20 12:47:48 INFO Received uploads request method=POST path=/api/pending_closures6015worker1 # [ 116.921211] nix-grpc-daemon[1469]: time=2026-09-20T12:47:47.407Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6016worker1 # [ 116.922778] nix-grpc-daemon[1469]: time=2026-09-20T12:47:47.409Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6017lb # [ 117.019392] niks3-server[1052]: 2026/09/20 12:47:48 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6018lb # [ 117.026906] niks3-server[1052]: 2026/09/20 12:47:48 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6019lb # [ 117.039560] niks3-server[1052]: 2026/09/20 12:47:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign6020worker1 # [ 116.980189] nix-grpc-daemon[1469]: time=2026-09-20T12:47:47.466Z level=INFO msg="Uploading 1 narinfos"6021lb # [ 117.044803] niks3-server[1052]: 2026/09/20 12:47:48 INFO Signed narinfos id=36 count=16022lb # [ 117.048334] niks3-server[1052]: 2026/09/20 12:47:48 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6023lb # [ 117.058515] niks3-server[1052]: 2026/09/20 12:47:48 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete6024lb # [ 117.066943] niks3-server[1052]: 2026/09/20 12:47:48 INFO Completed upload id=366025worker1 # [ 117.002809] nix-grpc-daemon[1469]: time=2026-09-20T12:47:47.489Z level=INFO msg="Upload complete. (285ms)"6026lb # [ 117.068692] niks3-server[1052]: 2026/09/20 12:47:48 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo6027worker1 # [ 117.007532] nix-grpc-daemon[1368]: ts=2026-09-20T12:47:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44668 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16028worker1 # [ 117.020679] nix-daemon[908]: reaped child process 1443, status = succeeded6029client # [ 117.117300] systemd[1]: mid.service: Deactivated successfully.6030client # [ 117.121252] systemd[1]: mid.service: Consumed 136ms CPU time over 10.019s wall clock time, 37.3M memory peak, 9.7K incoming IP traffic, 10.6K outgoing IP traffic.6031lb # [ 117.346651] envoy[910]: conn peer=[2001:db8:1::4]:39518 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=8646 tx=595 ms=169106032lb # [ 117.350507] envoy[910]: conn peer=[2001:db8:1::1]:44922 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3835 tx=1661 ms=97716033client: (finished: waiting for success: ! systemctl is-active mid, in 1.10 seconds)6034client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6035client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.03 seconds)6036(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.61 seconds)6037subtest: interrupting the client stops the build on the worker6038client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr6039client # [ 117.683261] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr.6040client # Running as unit: intr.service; invocation ID: 60bec60d7d7f4575863f333151b55f926041client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr, in 0.04 seconds)6042worker2 # [ 117.907465] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:48Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:545146043worker2 # [ 117.918784] nix-daemon[860]: accepted connection from pid 1551, user nix-grpc-daemon (trusted)6044client # [ 117.929879] nix[1254]: don't know how to build these paths:6045client # [ 117.930882] nix[1254]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6046worker2 # [ 117.937267] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:48Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06047worker2 # [ 117.939280] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:48Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=16048worker2 # [ 117.984166] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545146049worker2 # [ 117.995873] nix-grpc-daemon[1551]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 61 ms (attempt 1/5)6050worker2 # [ 118.062799] nix-grpc-daemon[1551]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 75 ms (attempt 2/5)6051worker2 # [ 118.143775] nix-grpc-daemon[1551]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 134 ms (attempt 3/5)6052worker2 # [ 118.283779] nix-grpc-daemon[1551]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 772 ms (attempt 4/5)6053worker2 # [ 119.063982] nix-grpc-daemon[1551]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6054worker2 # [ 119.080708] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:545146055client # [ 119.089425] nix[1254]: copying 1 paths to 'lb:50051'...6056client # [ 119.090843] nix[1254]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6057worker2 # [ 119.092317] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:545146058worker2 # [ 119.105223] nix-daemon[860]: accepted connection from pid 1551, user nix-grpc-daemon (trusted)6059worker2 # [ 119.123628] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:50Z level=info event=niks3_push_started pid=16276060worker2 # [ 119.290533] nix-grpc-daemon[1627]: time=2026-09-20T12:47:50.198Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""6061worker2 # [ 119.431781] nix-daemon[860]: accepted connection from pid 1640, user nix-grpc-daemon (trusted)6062worker2 # [ 119.442822] nix-daemon[860]: reaped child process 1643, status = succeeded6063worker2 # [ 119.597455] nix-daemon[860]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)6064worker2 # [ 119.609838] nix-daemon[860]: reaped child process 1648, status = succeeded6065lb # [ 119.619187] niks3-server[1052]: 2026/09/20 12:47:50 INFO Received uploads request method=POST path=/api/pending_closures6066worker2 # [ 119.635973] nix-grpc-daemon[1627]: time=2026-09-20T12:47:50.544Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6067worker2 # [ 119.637374] nix-grpc-daemon[1627]: time=2026-09-20T12:47:50.544Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6068lb # [ 119.652272] niks3-server[1052]: 2026/09/20 12:47:50 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6069lb # [ 119.663259] niks3-server[1052]: 2026/09/20 12:47:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign6070lb # [ 119.667993] niks3-server[1052]: 2026/09/20 12:47:50 INFO Signed narinfos id=37 count=16071worker2 # [ 119.677394] nix-grpc-daemon[1627]: time=2026-09-20T12:47:50.586Z level=INFO msg="Uploading 1 narinfos"6072lb # [ 119.680176] niks3-server[1052]: 2026/09/20 12:47:50 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6073lb # [ 119.684453] niks3-server[1052]: 2026/09/20 12:47:50 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete6074lb # [ 119.691039] niks3-server[1052]: 2026/09/20 12:47:50 INFO Completed upload id=376075worker2 # [ 119.697086] nix-grpc-daemon[1627]: time=2026-09-20T12:47:50.606Z level=INFO msg="Upload complete. (403ms)"6076lb # [ 119.692913] niks3-server[1052]: 2026/09/20 12:47:50 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6077worker2 # [ 119.699482] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54514 duration_s=0 paths=1 nar_bytes_in=4246078worker2 # [ 119.704923] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545146079worker2 # [ 119.714068] nix-daemon[860]: accepted connection from pid 1551, user nix-grpc-daemon (trusted)6080worker2 # [ 119.716467] nix-daemon[860]: reaped child process 1625, status = succeeded6081worker2 # [ 119.721897] nix-daemon[860]: accepted connection from pid 1551, user nix-grpc-daemon (trusted)6082client # [ 119.719629] nix[1254]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6083client: must succeed: systemctl kill -s INT intr6084client # [ 121.271642] systemd[1]: intr.service: Sent signal SIGINT to main process 1254 (nix) on client request.6085client # [ 121.278895] nix[1254]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6086client: (finished: must succeed: systemctl kill -s INT intr, in 0.06 seconds)6087client # [ 121.285832] nix[1254]: error: interrupted by the user6088client # [ 121.314400] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6089client # [ 121.315930] systemd[1]: intr.service: Failed with result 'exit-code'.6090client # [ 121.318078] systemd[1]: intr.service: Consumed 114ms CPU time over 3.634s wall clock time, 39.1M memory peak, 7.6K incoming IP traffic, 8.7K outgoing IP traffic.6091worker2 # [ 121.339607] nix-daemon[1652]: unexpected Nix daemon error: error: interrupted by the user6092worker2 # [ 121.342523] nix-daemon[860]: reaped child process 1650, status = succeeded6093worker2 # [ 121.347531] nix-daemon[860]: reaped child process 1652, status = failed with exit code 16094(finished: subtest: interrupting the client stops the build on the worker, in 3.74 seconds)6095subtest: a deploy mid-build drains: the build finishes, the new generation takes over6096client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw6097client # [ 121.434550] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw.6098client # Running as unit: sw.service; invocation ID: 3ac204d8332b483eb0f410faeb8bb8986099client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw, in 0.05 seconds)6100worker1 # [ 121.577762] nix-grpc-daemon[1368]: ts=2026-09-20T12:47:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:446686101client # [ 121.652963] nix[1274]: don't know how to build these paths:6102client # [ 121.653853] nix[1274]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6103worker2 # [ 121.663696] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:52Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06104worker2 # [ 121.667749] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:52Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=16105worker1 # [ 121.637841] nix-grpc-daemon[1368]: ts=2026-09-20T12:47:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446686106worker1 # [ 121.651170] nix-grpc-daemon[1368]: ts=2026-09-20T12:47:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446686107client # [ 121.730285] nix[1274]: copying 1 paths to 'lb:50051'...6108worker1 # [ 121.662952] nix-grpc-daemon[1368]: ts=2026-09-20T12:47:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:446686109client # [ 121.732708] nix[1274]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6110worker1 # [ 121.675592] nix-daemon[908]: accepted connection from pid 1368, user nix-grpc-daemon (trusted)6111worker1 # [ 121.865332] nix-daemon[908]: accepted connection from pid 1513, user nix-grpc-daemon (trusted)6112worker1 # [ 121.876989] nix-daemon[908]: reaped child process 1516, status = succeeded6113worker1 # [ 122.045512] nix-daemon[908]: accepted connection from pid 1518, user nix-grpc-daemon (trusted)6114worker1 # [ 122.055795] nix-daemon[908]: reaped child process 1521, status = succeeded6115lb # [ 122.137796] niks3-server[1052]: 2026/09/20 12:47:53 INFO Received uploads request method=POST path=/api/pending_closures6116worker1 # [ 122.078584] nix-grpc-daemon[1469]: time=2026-09-20T12:47:52.564Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6117worker1 # [ 122.080143] nix-grpc-daemon[1469]: time=2026-09-20T12:47:52.566Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6118lb # [ 122.155320] niks3-server[1052]: 2026/09/20 12:47:53 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6119lb # [ 122.157667] niks3-server[1052]: 2026/09/20 12:47:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6120lb # [ 122.160531] niks3-server[1052]: 2026/09/20 12:47:53 INFO Signed narinfos id=38 count=16121worker1 # [ 122.096414] nix-grpc-daemon[1469]: time=2026-09-20T12:47:52.582Z level=INFO msg="Uploading 1 narinfos"6122lb # [ 122.163782] niks3-server[1052]: 2026/09/20 12:47:53 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6123lb # [ 122.168058] niks3-server[1052]: 2026/09/20 12:47:53 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6124lb # [ 122.173696] niks3-server[1052]: 2026/09/20 12:47:53 INFO Completed upload id=386125worker1 # [ 122.109520] nix-grpc-daemon[1469]: time=2026-09-20T12:47:52.595Z level=INFO msg="Upload complete. (422ms)"6126worker1 # [ 122.111435] nix-grpc-daemon[1368]: ts=2026-09-20T12:47:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44668 duration_s=0 paths=1 nar_bytes_in=4166127lb # [ 122.177120] niks3-server[1052]: 2026/09/20 12:47:53 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6128worker1 # [ 122.115281] nix-grpc-daemon[1368]: ts=2026-09-20T12:47:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446686129worker1 # [ 122.121428] nix-daemon[908]: accepted connection from pid 1368, user nix-grpc-daemon (trusted)6130worker1 # [ 122.123616] nix-daemon[908]: reaped child process 1511, status = succeeded6131worker1 # [ 122.128905] nix-daemon[908]: accepted connection from pid 1368, user nix-grpc-daemon (trusted)6132client # [ 122.196855] nix[1274]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6133worker1: must succeed: systemctl reload nix-grpc-daemon.service6134worker1 # [ 123.903256] systemd[1]: Reloading Nix worker-protocol over gRPC...6135worker1 # [ 123.945734] systemd[1]: Reloaded Nix worker-protocol over gRPC.6136worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.09 seconds)6137worker1: must succeed: pgrep -f 'read -t [9]0 x'6138worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)6139client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&26140worker2 # [ 124.289454] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:545146141client # don't know how to build these paths:6142client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6143worker1 # [ 124.229058] nix-grpc-daemon[1368]: ts=2026-09-20T12:47:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446686144worker1 # [ 124.241076] nix-grpc-daemon[1368]: ts=2026-09-20T12:47:54Z level=info event=draining6145worker2 # [ 124.318825] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:55Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06146worker2 # [ 124.324673] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:55Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=16147worker2 # [ 124.363180] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545146148worker2 # [ 124.383264] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:545146149client # copying 1 paths to 'lb:50051'...6150worker2 # [ 124.397322] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:545146151client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6152worker2 # [ 124.424925] nix-daemon[860]: accepted connection from pid 1551, user nix-grpc-daemon (trusted)6153worker2 # [ 124.608131] nix-daemon[860]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6154worker2 # [ 124.621730] nix-daemon[860]: reaped child process 1705, status = succeeded6155worker2 # [ 124.797690] nix-daemon[860]: accepted connection from pid 1707, user nix-grpc-daemon (trusted)6156worker2 # [ 124.810666] nix-daemon[860]: reaped child process 1710, status = succeeded6157lb # [ 124.823408] niks3-server[1052]: 2026/09/20 12:47:55 INFO Received uploads request method=POST path=/api/pending_closures6158worker2 # [ 124.840177] nix-grpc-daemon[1627]: time=2026-09-20T12:47:55.749Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6159worker2 # [ 124.842214] nix-grpc-daemon[1627]: time=2026-09-20T12:47:55.750Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6160lb # [ 124.857643] niks3-server[1052]: 2026/09/20 12:47:55 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6161lb # [ 124.864442] niks3-server[1052]: 2026/09/20 12:47:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6162lb # [ 124.869646] niks3-server[1052]: 2026/09/20 12:47:55 INFO Signed narinfos id=39 count=16163worker2 # [ 124.878839] nix-grpc-daemon[1627]: time=2026-09-20T12:47:55.787Z level=INFO msg="Uploading 1 narinfos"6164lb # [ 124.876213] niks3-server[1052]: 2026/09/20 12:47:55 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6165lb # [ 124.886843] niks3-server[1052]: 2026/09/20 12:47:55 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6166lb # [ 124.893666] niks3-server[1052]: 2026/09/20 12:47:55 INFO Completed upload id=396167worker2 # [ 124.899724] nix-grpc-daemon[1627]: time=2026-09-20T12:47:55.808Z level=INFO msg="Upload complete. (462ms)"6168lb # [ 124.896441] niks3-server[1052]: 2026/09/20 12:47:55 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6169worker2 # [ 124.903475] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54514 duration_s=0 paths=1 nar_bytes_in=5286170worker2 # [ 124.910326] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545146171worker2 # [ 124.920681] nix-daemon[860]: accepted connection from pid 1551, user nix-grpc-daemon (trusted)6172worker2 # [ 124.929174] nix-daemon[860]: reaped child process 1700, status = succeeded6173worker2 # [ 124.934454] nix-daemon[860]: accepted connection from pid 1551, user nix-grpc-daemon (trusted)6174client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6175client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6176client # farm-a> LOG-farm-a-during-drain6177worker2 # [ 126.151467] nix-daemon[860]: reaped child process 1714, status = succeeded6178worker2 # [ 126.313282] nix-daemon[860]: accepted connection from pid 1729, user nix-grpc-daemon (trusted)6179worker2 # [ 126.327152] nix-daemon[860]: reaped child process 1732, status = succeeded6180lb # [ 126.339621] niks3-server[1052]: 2026/09/20 12:47:57 INFO Received uploads request method=POST path=/api/pending_closures6181worker2 # [ 126.355081] nix-grpc-daemon[1627]: time=2026-09-20T12:47:57.264Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6182worker2 # [ 126.356638] nix-grpc-daemon[1627]: time=2026-09-20T12:47:57.265Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6183lb # [ 126.373646] niks3-server[1052]: 2026/09/20 12:47:57 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6184lb # [ 126.391232] niks3-server[1052]: 2026/09/20 12:47:57 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6185lb # [ 126.399485] niks3-server[1052]: 2026/09/20 12:47:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6186lb # [ 126.404068] niks3-server[1052]: 2026/09/20 12:47:57 INFO Signed narinfos id=40 count=16187worker2 # [ 126.411029] nix-grpc-daemon[1627]: time=2026-09-20T12:47:57.319Z level=INFO msg="Uploading 1 narinfos"6188lb # [ 126.408513] niks3-server[1052]: 2026/09/20 12:47:57 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6189lb # [ 126.417721] niks3-server[1052]: 2026/09/20 12:47:57 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6190lb # [ 126.420874] niks3-server[1052]: 2026/09/20 12:47:57 INFO Completed upload id=406191worker2 # [ 126.428264] nix-grpc-daemon[1627]: time=2026-09-20T12:47:57.337Z level=INFO msg="Upload complete. (285ms)"6192lb # [ 126.423301] niks3-server[1052]: 2026/09/20 12:47:57 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6193worker2 # [ 126.437705] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54514 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16194worker2 # [ 126.444753] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:57Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06195worker2 # [ 126.448277] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:57Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=16196worker2 # [ 126.457162] nix-daemon[860]: reaped child process 1712, status = succeeded6197worker2 # [ 126.487642] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545146198worker2 # [ 126.500293] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:545146199worker2 # [ 126.514284] nix-grpc-daemon[1627]: time=2026-09-20T12:47:57.423Z level=INFO msg="All 1 paths already cached"6200client # copying 1 paths to 'lb:50051'...6201client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6202worker2 # [ 126.521508] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:545146203worker2 # [ 126.531166] nix-daemon[860]: accepted connection from pid 1551, user nix-grpc-daemon (trusted)6204worker2 # [ 126.681568] nix-daemon[860]: accepted connection from pid 1738, user nix-grpc-daemon (trusted)6205worker2 # [ 126.695417] nix-daemon[860]: reaped child process 1741, status = succeeded6206worker2 # [ 126.843611] nix-daemon[860]: accepted connection from pid 1743, user nix-grpc-daemon (trusted)6207worker2 # [ 126.856719] nix-daemon[860]: reaped child process 1746, status = succeeded6208lb # [ 126.867275] niks3-server[1052]: 2026/09/20 12:47:57 INFO Received uploads request method=POST path=/api/pending_closures6209worker2 # [ 126.881592] nix-grpc-daemon[1627]: time=2026-09-20T12:47:57.790Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6210worker2 # [ 126.882972] nix-grpc-daemon[1627]: time=2026-09-20T12:47:57.790Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6211lb # [ 126.894924] niks3-server[1052]: 2026/09/20 12:47:57 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6212lb # [ 126.905071] niks3-server[1052]: 2026/09/20 12:47:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6213lb # [ 126.911500] niks3-server[1052]: 2026/09/20 12:47:58 INFO Signed narinfos id=41 count=16214worker2 # [ 126.920886] nix-grpc-daemon[1627]: time=2026-09-20T12:47:57.829Z level=INFO msg="Uploading 1 narinfos"6215lb # [ 126.914959] niks3-server[1052]: 2026/09/20 12:47:58 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6216lb # [ 126.928501] niks3-server[1052]: 2026/09/20 12:47:58 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6217worker2 # [ 126.941393] nix-grpc-daemon[1627]: time=2026-09-20T12:47:57.850Z level=INFO msg="Upload complete. (399ms)"6218lb # [ 126.935331] niks3-server[1052]: 2026/09/20 12:47:58 INFO Completed upload id=416219lb # [ 126.938338] niks3-server[1052]: 2026/09/20 12:47:58 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6220worker2 # [ 126.943584] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54514 duration_s=0 paths=1 nar_bytes_in=6406221worker2 # [ 126.948994] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545146222worker2 # [ 126.958702] nix-daemon[860]: accepted connection from pid 1551, user nix-grpc-daemon (trusted)6223worker2 # [ 126.962313] nix-daemon[860]: reaped child process 1736, status = succeeded6224worker2 # [ 126.967362] nix-daemon[860]: accepted connection from pid 1551, user nix-grpc-daemon (trusted)6225client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6226client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6227lb # [ 127.345829] envoy[910]: conn peer=[2001:db8:1::1]:54144 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2924 tx=733 ms=34016228worker2 # [ 128.180909] nix-daemon[860]: reaped child process 1750, status = succeeded6229client # farm-b> LOG-farm-b-during-drain6230worker2 # [ 128.335695] nix-daemon[860]: accepted connection from pid 1765, user nix-grpc-daemon (trusted)6231worker2 # [ 128.347443] nix-daemon[860]: reaped child process 1768, status = succeeded6232lb # [ 128.359934] niks3-server[1052]: 2026/09/20 12:47:59 INFO Received uploads request method=POST path=/api/pending_closures6233worker2 # [ 128.374180] nix-grpc-daemon[1627]: time=2026-09-20T12:47:59.283Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6234worker2 # [ 128.375787] nix-grpc-daemon[1627]: time=2026-09-20T12:47:59.285Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6235lb # [ 128.394803] niks3-server[1052]: 2026/09/20 12:47:59 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6236lb # [ 128.400901] niks3-server[1052]: 2026/09/20 12:47:59 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6237lb # [ 128.414149] niks3-server[1052]: 2026/09/20 12:47:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6238worker2 # [ 128.424626] nix-grpc-daemon[1627]: time=2026-09-20T12:47:59.333Z level=INFO msg="Uploading 1 narinfos"6239lb # [ 128.418930] niks3-server[1052]: 2026/09/20 12:47:59 INFO Signed narinfos id=42 count=16240lb # [ 128.425537] niks3-server[1052]: 2026/09/20 12:47:59 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6241lb # [ 128.434473] niks3-server[1052]: 2026/09/20 12:47:59 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6242worker2 # [ 128.447279] nix-grpc-daemon[1627]: time=2026-09-20T12:47:59.356Z level=INFO msg="Upload complete. (273ms)"6243lb # [ 128.441254] niks3-server[1052]: 2026/09/20 12:47:59 INFO Completed upload id=426244lb # [ 128.444270] niks3-server[1052]: 2026/09/20 12:47:59 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6245worker2 # [ 128.450678] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54514 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16246worker2 # [ 128.457213] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:59Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06247worker2 # [ 128.459186] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:59Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=16248worker2 # [ 128.463961] nix-daemon[860]: reaped child process 1748, status = succeeded6249worker2 # [ 128.500763] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545146250worker2 # [ 128.507569] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:545146251worker2 # [ 128.514164] nix-grpc-daemon[1627]: time=2026-09-20T12:47:59.423Z level=INFO msg="All 2 paths already cached"6252client # copying 1 paths to 'lb:50051'...6253worker2 # [ 128.519164] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:545146254client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6255worker2 # [ 128.535566] nix-daemon[860]: accepted connection from pid 1551, user nix-grpc-daemon (trusted)6256worker2 # [ 128.687079] nix-daemon[860]: accepted connection from pid 1774, user nix-grpc-daemon (trusted)6257worker2 # [ 128.700756] nix-daemon[860]: reaped child process 1777, status = succeeded6258worker2 # [ 128.869171] nix-daemon[860]: accepted connection from pid 1779, user nix-grpc-daemon (trusted)6259worker2 # [ 128.883532] nix-daemon[860]: reaped child process 1782, status = succeeded6260lb # [ 128.892088] niks3-server[1052]: 2026/09/20 12:47:59 INFO Received uploads request method=POST path=/api/pending_closures6261worker2 # [ 128.905699] nix-grpc-daemon[1627]: time=2026-09-20T12:47:59.814Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6262worker2 # [ 128.907225] nix-grpc-daemon[1627]: time=2026-09-20T12:47:59.816Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6263lb # [ 128.926094] niks3-server[1052]: 2026/09/20 12:48:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6264lb # [ 128.934141] niks3-server[1052]: 2026/09/20 12:48:00 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6265worker2 # [ 128.945678] nix-grpc-daemon[1627]: time=2026-09-20T12:47:59.854Z level=INFO msg="Uploading 1 narinfos"6266lb # [ 128.939639] niks3-server[1052]: 2026/09/20 12:48:00 INFO Signed narinfos id=43 count=16267lb # [ 128.955025] niks3-server[1052]: 2026/09/20 12:48:00 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6268lb # [ 128.960195] niks3-server[1052]: 2026/09/20 12:48:00 INFO Completed upload id=436269worker2 # [ 128.966209] nix-grpc-daemon[1627]: time=2026-09-20T12:47:59.875Z level=INFO msg="Upload complete. (423ms)"6270lb # [ 128.961839] niks3-server[1052]: 2026/09/20 12:48:00 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6271worker2 # [ 128.968548] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54514 duration_s=0 paths=1 nar_bytes_in=7686272worker2 # [ 128.972872] nix-grpc-daemon[1551]: ts=2026-09-20T12:47:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545146273lb # [ 128.967428] niks3-server[1052]: 2026/09/20 12:48:00 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6274worker2 # [ 128.980173] nix-daemon[860]: accepted connection from pid 1551, user nix-grpc-daemon (trusted)6275worker2 # [ 128.982656] nix-daemon[860]: reaped child process 1772, status = succeeded6276worker2 # [ 128.988530] nix-daemon[860]: accepted connection from pid 1551, user nix-grpc-daemon (trusted)6277client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6278client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6279worker2 # [ 130.403626] nix-daemon[860]: reaped child process 1786, status = succeeded6280client # farm-top> LOG-farm-top-during-drain6281worker2 # [ 130.557614] nix-daemon[860]: accepted connection from pid 1801, user nix-grpc-daemon (trusted)6282worker2 # [ 130.572236] nix-daemon[860]: reaped child process 1804, status = succeeded6283lb # [ 130.583767] niks3-server[1052]: 2026/09/20 12:48:01 INFO Received uploads request method=POST path=/api/pending_closures6284worker2 # [ 130.601126] nix-grpc-daemon[1627]: time=2026-09-20T12:48:01.510Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6285worker2 # [ 130.604072] nix-grpc-daemon[1627]: time=2026-09-20T12:48:01.511Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6286lb # [ 130.621931] niks3-server[1052]: 2026/09/20 12:48:01 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6287lb # [ 130.631480] niks3-server[1052]: 2026/09/20 12:48:01 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6288lb # [ 130.641408] niks3-server[1052]: 2026/09/20 12:48:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6289lb # [ 130.647053] niks3-server[1052]: 2026/09/20 12:48:01 INFO Signed narinfos id=44 count=16290worker2 # [ 130.655747] nix-grpc-daemon[1627]: time=2026-09-20T12:48:01.564Z level=INFO msg="Uploading 1 narinfos"6291lb # [ 130.652370] niks3-server[1052]: 2026/09/20 12:48:01 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6292lb # [ 130.660775] niks3-server[1052]: 2026/09/20 12:48:01 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6293lb # [ 130.668504] niks3-server[1052]: 2026/09/20 12:48:01 INFO Completed upload id=446294worker2 # [ 130.674779] nix-grpc-daemon[1627]: time=2026-09-20T12:48:01.583Z level=INFO msg="Upload complete. (277ms)"6295lb # [ 130.670433] niks3-server[1052]: 2026/09/20 12:48:01 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6296worker2 # [ 130.682513] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54514 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16297worker2 # [ 130.695805] nix-daemon[860]: reaped child process 1784, status = succeeded6298client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&2, in 6.66 seconds)6299client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath6300client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath, in 0.22 seconds)6301worker1: must succeed: pkill -f 'read -t [9]0 x'6302worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6303client: waiting for success: ! systemctl is-active sw6304worker1 # [ 130.949875] nix-daemon[908]: reaped child process 1525, status = succeeded6305worker1 # [ 131.108362] nix-daemon[908]: accepted connection from pid 1566, user nix-grpc-daemon (trusted)6306worker1 # [ 131.118932] nix-daemon[908]: reaped child process 1569, status = succeeded6307lb # [ 131.201582] niks3-server[1052]: 2026/09/20 12:48:02 INFO Received uploads request method=POST path=/api/pending_closures6308worker1 # [ 131.141536] nix-grpc-daemon[1469]: time=2026-09-20T12:48:01.627Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6309worker1 # [ 131.143286] nix-grpc-daemon[1469]: time=2026-09-20T12:48:01.629Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6310lb # [ 131.221941] niks3-server[1052]: 2026/09/20 12:48:02 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6311lb # [ 131.223986] niks3-server[1052]: 2026/09/20 12:48:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6312lb # [ 131.226141] niks3-server[1052]: 2026/09/20 12:48:02 INFO Signed narinfos id=45 count=16313worker1 # [ 131.162213] nix-grpc-daemon[1469]: time=2026-09-20T12:48:01.648Z level=INFO msg="Uploading 1 narinfos"6314lb # [ 131.228933] niks3-server[1052]: 2026/09/20 12:48:02 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6315lb # [ 131.232662] niks3-server[1052]: 2026/09/20 12:48:02 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6316lb # [ 131.237050] niks3-server[1052]: 2026/09/20 12:48:02 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6317lb # [ 131.239963] niks3-server[1052]: 2026/09/20 12:48:02 INFO Completed upload id=456318worker1 # [ 131.177032] nix-grpc-daemon[1469]: time=2026-09-20T12:48:01.663Z level=INFO msg="Upload complete. (233ms)"6319lb # [ 131.241731] niks3-server[1052]: 2026/09/20 12:48:02 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6320worker1 # [ 131.179952] nix-grpc-daemon[1368]: ts=2026-09-20T12:48:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44668 duration_s=9 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16321worker1 # [ 131.186443] nix-daemon[908]: reaped child process 1523, status = succeeded6322client # [ 131.293309] systemd[1]: sw.service: Deactivated successfully.6323client # [ 131.297128] systemd[1]: sw.service: Consumed 121ms CPU time over 9.861s wall clock time, 39.1M memory peak, 8.4K incoming IP traffic, 8.9K outgoing IP traffic.6324worker1 # [ 131.242842] nix-grpc-daemon[1368]: ts=2026-09-20T12:48:01Z level=info event=signal_exit6325client: (finished: waiting for success: ! systemctl is-active sw, in 1.09 seconds)6326client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6327client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.04 seconds)6328worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6329worker1 # [ 132.466433] nix-grpc-daemon[1368]: ts=2026-09-20T12:48:02Z level=info event=scheduler_restarting6330worker2 # [ 132.541470] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:03Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:332586331worker1 # [ 132.473541] nix-grpc-daemon[1368]: ts=2026-09-20T12:48:02Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6332worker2 # [ 132.546107] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:03Z level=info event=worker_gone addr=192.168.1.3:500516333worker1 # [ 132.484895] nix-daemon[908]: reaped child process 1437, status = succeeded6334worker1 # [ 133.099313] nix-grpc-daemon[1368]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6335worker1 # [ 133.100759] nix-grpc-daemon[1368]: E0000 00:00:1789908483.585564 1411 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true6336worker1 # [ 133.102956] nix-grpc-daemon[1368]: E0000 00:00:1789908483.585682 1411 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6337worker1 # [ 134.282600] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6338worker1 # [ 134.286762] systemd[1]: nix-grpc-daemon.service: Consumed 607ms CPU time over 28.209s wall clock time, 30.2M memory peak, 20.9K incoming IP traffic, 27.6K outgoing IP traffic.6339worker1 # [ 134.303954] systemd[1]: Starting Nix worker-protocol over gRPC...6340worker1 # [ 134.462926] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:04Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6341worker1 # [ 134.464786] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:04Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6342worker1 # [ 134.468567] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:04Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16343worker1 # [ 134.475326] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:04Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6344worker1 # [ 134.476981] systemd[1]: Started Nix worker-protocol over gRPC.6345worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.42 seconds)6346lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6347lb: must succeed: curl -sf localhost:9901/clusters6348worker1 # [ 134.492524] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:04Z level=info event=scheduler_connected addr=lb:500516349worker2 # [ 134.567216] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:05Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:332586350worker2 # [ 134.571394] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:05Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06351lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6352lb: must succeed: curl -sf localhost:9901/clusters6353lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6354lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.14 seconds)6355lb: must succeed: curl -sf localhost:9901/clusters6356lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6357worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6358worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6359(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 14.43 seconds)6360subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6361client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16362worker2 # [ 136.051651] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:545146363worker2 # [ 136.058900] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:545146364worker2 # [ 136.102749] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:07Z level=debug event=want client=5 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06365worker2 # [ 136.107268] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:07Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16366worker2 # [ 136.111490] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:07Z level=debug event=want client=5 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06367worker2 # [ 136.115283] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:07Z level=debug event=want client=5 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06368worker1 # [ 136.081787] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602526369worker1 # [ 136.092634] nix-daemon[908]: accepted connection from pid 1583, user nix-grpc-daemon (trusted)6370worker1 # [ 136.109157] nix-grpc-daemon[1583]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 74 ms (attempt 1/5)6371worker1 # [ 136.188967] nix-grpc-daemon[1583]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 12 ms (attempt 2/5)6372worker1 # [ 136.206863] nix-grpc-daemon[1583]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 300 ms (attempt 3/5)6373worker1 # [ 136.512800] nix-grpc-daemon[1583]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 236 ms (attempt 4/5)6374worker1 # [ 136.754934] nix-grpc-daemon[1583]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6375worker1 # [ 136.771584] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602526376worker1 # [ 136.782829] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:602526377worker1 # [ 136.791308] nix-daemon[908]: accepted connection from pid 1583, user nix-grpc-daemon (trusted)6378worker1 # [ 136.812768] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:07Z level=info event=niks3_push_started pid=16396379worker1 # [ 137.010464] nix-grpc-daemon[1639]: time=2026-09-20T12:48:07.496Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""6380worker1 # [ 137.173538] nix-daemon[908]: accepted connection from pid 1652, user nix-grpc-daemon (trusted)6381worker1 # [ 137.184196] nix-daemon[908]: reaped child process 1655, status = succeeded6382lb # [ 137.346591] envoy[910]: conn peer=[2001:db8:1::1]:45842 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9641 tx=3893 ms=64116383lb # [ 137.350696] envoy[910]: conn peer=[2001:db8:1::1]:54160 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2932 tx=1328 ms=96156384lb # [ 137.354354] envoy[910]: conn peer=[2001:db8:1::3]:33176 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1114 tx=448 ms=262286385worker1 # [ 137.340798] nix-daemon[908]: accepted connection from pid 1657, user nix-grpc-daemon (trusted)6386worker1 # [ 137.352651] nix-daemon[908]: reaped child process 1660, status = succeeded6387lb # [ 137.434744] niks3-server[1052]: 2026/09/20 12:48:08 INFO Received uploads request method=POST path=/api/pending_closures6388worker1 # [ 137.380067] nix-grpc-daemon[1639]: time=2026-09-20T12:48:07.865Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6389worker1 # [ 137.381434] nix-grpc-daemon[1639]: time=2026-09-20T12:48:07.865Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6390lb # [ 137.470890] niks3-server[1052]: 2026/09/20 12:48:08 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6391lb # [ 137.480744] niks3-server[1052]: 2026/09/20 12:48:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6392lb # [ 137.486523] niks3-server[1052]: 2026/09/20 12:48:08 INFO Signed narinfos id=46 count=16393worker1 # [ 137.425461] nix-grpc-daemon[1639]: time=2026-09-20T12:48:07.911Z level=INFO msg="Uploading 1 narinfos"6394lb # [ 137.499520] niks3-server[1052]: 2026/09/20 12:48:08 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6395lb # [ 137.505946] niks3-server[1052]: 2026/09/20 12:48:08 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6396worker1 # [ 137.447995] nix-grpc-daemon[1639]: time=2026-09-20T12:48:07.934Z level=INFO msg="Upload complete. (433ms)"6397lb # [ 137.512235] niks3-server[1052]: 2026/09/20 12:48:08 INFO Completed upload id=466398worker1 # [ 137.450258] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60252 duration_s=0 paths=1 nar_bytes_in=5046399lb # [ 137.516445] niks3-server[1052]: 2026/09/20 12:48:08 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6400worker1 # [ 137.456509] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602526401worker1 # [ 137.460881] nix-daemon[908]: accepted connection from pid 1583, user nix-grpc-daemon (trusted)6402worker1 # [ 137.463189] nix-daemon[908]: reaped child process 1637, status = succeeded6403worker1 # [ 137.468534] nix-daemon[908]: accepted connection from pid 1583, user nix-grpc-daemon (trusted)6404worker1 # [ 138.483126] nix-daemon[908]: reaped child process 1664, status = succeeded6405worker1 # [ 138.624279] nix-daemon[908]: accepted connection from pid 1679, user nix-grpc-daemon (trusted)6406worker1 # [ 138.635905] nix-daemon[908]: reaped child process 1682, status = succeeded6407lb # [ 138.717662] niks3-server[1052]: 2026/09/20 12:48:09 INFO Received uploads request method=POST path=/api/pending_closures6408worker1 # [ 138.661505] nix-grpc-daemon[1639]: time=2026-09-20T12:48:09.147Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6409worker1 # [ 138.663320] nix-grpc-daemon[1639]: time=2026-09-20T12:48:09.149Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6410lb # [ 138.754982] niks3-server[1052]: 2026/09/20 12:48:09 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6411lb # [ 138.766989] niks3-server[1052]: 2026/09/20 12:48:09 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6412lb # [ 138.776081] niks3-server[1052]: 2026/09/20 12:48:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6413lb # [ 138.781487] niks3-server[1052]: 2026/09/20 12:48:09 INFO Signed narinfos id=47 count=16414worker1 # [ 138.720424] nix-grpc-daemon[1639]: time=2026-09-20T12:48:09.206Z level=INFO msg="Uploading 1 narinfos"6415lb # [ 138.786761] niks3-server[1052]: 2026/09/20 12:48:09 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6416lb # [ 138.796702] niks3-server[1052]: 2026/09/20 12:48:09 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6417lb # [ 138.803603] niks3-server[1052]: 2026/09/20 12:48:09 INFO Completed upload id=476418worker1 # [ 138.739393] nix-grpc-daemon[1639]: time=2026-09-20T12:48:09.225Z level=INFO msg="Upload complete. (264ms)"6419lb # [ 138.805192] niks3-server[1052]: 2026/09/20 12:48:09 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6420worker1 # [ 138.742301] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60252 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=7 outputs=16421worker2 # [ 138.814550] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:09Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16422worker1 # [ 138.749395] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602526423worker2 # [ 138.820057] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:09Z level=debug event=want client=5 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06424worker1 # [ 138.751400] nix-daemon[908]: reaped child process 1662, status = succeeded6425worker1 # [ 138.756769] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602526426worker1 # [ 138.762368] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:602526427worker1 # [ 138.766262] nix-daemon[908]: accepted connection from pid 1583, user nix-grpc-daemon (trusted)6428worker1 # [ 138.921222] nix-daemon[908]: accepted connection from pid 1686, user nix-grpc-daemon (trusted)6429worker1 # [ 138.933563] nix-daemon[908]: reaped child process 1689, status = succeeded6430worker1 # [ 139.087916] nix-daemon[908]: accepted connection from pid 1691, user nix-grpc-daemon (trusted)6431worker1 # [ 139.099458] nix-daemon[908]: reaped child process 1694, status = succeeded6432lb # [ 139.180191] niks3-server[1052]: 2026/09/20 12:48:10 INFO Received uploads request method=POST path=/api/pending_closures6433worker1 # [ 139.123528] nix-grpc-daemon[1639]: time=2026-09-20T12:48:09.609Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6434worker1 # [ 139.125280] nix-grpc-daemon[1639]: time=2026-09-20T12:48:09.611Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6435lb # [ 139.210890] niks3-server[1052]: 2026/09/20 12:48:10 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6436lb # [ 139.224280] niks3-server[1052]: 2026/09/20 12:48:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6437lb # [ 139.231935] niks3-server[1052]: 2026/09/20 12:48:10 INFO Signed narinfos id=48 count=16438worker1 # [ 139.170929] nix-grpc-daemon[1639]: time=2026-09-20T12:48:09.657Z level=INFO msg="Uploading 1 narinfos"6439lb # [ 139.235666] niks3-server[1052]: 2026/09/20 12:48:10 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6440lb # [ 139.250110] niks3-server[1052]: 2026/09/20 12:48:10 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6441worker1 # [ 139.191209] nix-grpc-daemon[1639]: time=2026-09-20T12:48:09.677Z level=INFO msg="Upload complete. (419ms)"6442lb # [ 139.255521] niks3-server[1052]: 2026/09/20 12:48:10 INFO Completed upload id=486443worker1 # [ 139.193478] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60252 duration_s=0 paths=1 nar_bytes_in=5046444lb # [ 139.260731] niks3-server[1052]: 2026/09/20 12:48:10 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6445worker1 # [ 139.198518] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602526446worker1 # [ 139.209242] nix-daemon[908]: accepted connection from pid 1583, user nix-grpc-daemon (trusted)6447worker1 # [ 139.216272] nix-daemon[908]: reaped child process 1684, status = succeeded6448worker1 # [ 139.221408] nix-daemon[908]: accepted connection from pid 1583, user nix-grpc-daemon (trusted)6449worker1 # [ 140.247685] nix-daemon[908]: reaped child process 1698, status = succeeded6450worker1 # [ 140.419560] nix-daemon[908]: accepted connection from pid 1713, user nix-grpc-daemon (trusted)6451worker1 # [ 140.431294] nix-daemon[908]: reaped child process 1716, status = succeeded6452lb # [ 140.513973] niks3-server[1052]: 2026/09/20 12:48:11 INFO Received uploads request method=POST path=/api/pending_closures6453worker1 # [ 140.460373] nix-grpc-daemon[1639]: time=2026-09-20T12:48:10.946Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6454worker1 # [ 140.461834] nix-grpc-daemon[1639]: time=2026-09-20T12:48:10.946Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6455lb # [ 140.551371] niks3-server[1052]: 2026/09/20 12:48:11 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6456lb # [ 140.557551] niks3-server[1052]: 2026/09/20 12:48:11 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6457lb # [ 140.565783] niks3-server[1052]: 2026/09/20 12:48:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6458worker1 # [ 140.507075] nix-grpc-daemon[1639]: time=2026-09-20T12:48:10.993Z level=INFO msg="Uploading 1 narinfos"6459lb # [ 140.571785] niks3-server[1052]: 2026/09/20 12:48:11 INFO Signed narinfos id=49 count=16460lb # [ 140.575349] niks3-server[1052]: 2026/09/20 12:48:11 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6461lb # [ 140.581196] niks3-server[1052]: 2026/09/20 12:48:11 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6462lb # [ 140.585415] niks3-server[1052]: 2026/09/20 12:48:11 INFO Completed upload id=496463worker1 # [ 140.521293] nix-grpc-daemon[1639]: time=2026-09-20T12:48:11.007Z level=INFO msg="Upload complete. (291ms)"6464lb # [ 140.587448] niks3-server[1052]: 2026/09/20 12:48:11 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6465worker1 # [ 140.525773] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60252 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=8 outputs=16466worker2 # [ 140.597795] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:11Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16467worker1 # [ 140.533283] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602526468worker2 # [ 140.607891] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:11Z level=debug event=want client=5 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06469worker1 # [ 140.543520] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602526470worker1 # [ 140.546681] nix-daemon[908]: reaped child process 1696, status = succeeded6471worker1 # [ 140.550193] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:602526472worker1 # [ 140.554152] nix-daemon[908]: accepted connection from pid 1583, user nix-grpc-daemon (trusted)6473worker1 # [ 140.711252] nix-daemon[908]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6474worker1 # [ 140.723090] nix-daemon[908]: reaped child process 1723, status = succeeded6475worker1 # [ 140.891256] nix-daemon[908]: accepted connection from pid 1725, user nix-grpc-daemon (trusted)6476worker1 # [ 140.902941] nix-daemon[908]: reaped child process 1728, status = succeeded6477lb # [ 140.985470] niks3-server[1052]: 2026/09/20 12:48:12 INFO Received uploads request method=POST path=/api/pending_closures6478worker1 # [ 140.929217] nix-grpc-daemon[1639]: time=2026-09-20T12:48:11.415Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6479worker1 # [ 140.930755] nix-grpc-daemon[1639]: time=2026-09-20T12:48:11.417Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6480lb # [ 141.015894] niks3-server[1052]: 2026/09/20 12:48:12 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6481lb # [ 141.025531] niks3-server[1052]: 2026/09/20 12:48:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6482lb # [ 141.030744] niks3-server[1052]: 2026/09/20 12:48:12 INFO Signed narinfos id=50 count=16483worker1 # [ 140.969300] nix-grpc-daemon[1639]: time=2026-09-20T12:48:11.455Z level=INFO msg="Uploading 1 narinfos"6484lb # [ 141.037106] niks3-server[1052]: 2026/09/20 12:48:12 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6485lb # [ 141.048101] niks3-server[1052]: 2026/09/20 12:48:12 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6486worker1 # [ 140.988686] nix-grpc-daemon[1639]: time=2026-09-20T12:48:11.475Z level=INFO msg="Upload complete. (428ms)"6487lb # [ 141.053043] niks3-server[1052]: 2026/09/20 12:48:12 INFO Completed upload id=506488worker1 # [ 140.990751] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60252 duration_s=0 paths=1 nar_bytes_in=5046489lb # [ 141.058423] niks3-server[1052]: 2026/09/20 12:48:12 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6490worker1 # [ 140.997698] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602526491worker1 # [ 141.007542] nix-daemon[908]: accepted connection from pid 1583, user nix-grpc-daemon (trusted)6492worker1 # [ 141.009749] nix-daemon[908]: reaped child process 1718, status = succeeded6493worker1 # [ 141.014623] nix-daemon[908]: accepted connection from pid 1583, user nix-grpc-daemon (trusted)6494worker1 # [ 141.828376] nix-daemon[908]: reaped child process 1732, status = succeeded6495worker1 # [ 141.999861] nix-daemon[908]: accepted connection from pid 1747, user nix-grpc-daemon (trusted)6496worker1 # [ 142.011833] nix-daemon[908]: reaped child process 1750, status = succeeded6497lb # [ 142.097901] niks3-server[1052]: 2026/09/20 12:48:13 INFO Received uploads request method=POST path=/api/pending_closures6498worker1 # [ 142.042173] nix-grpc-daemon[1639]: time=2026-09-20T12:48:12.527Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6499worker1 # [ 142.043513] nix-grpc-daemon[1639]: time=2026-09-20T12:48:12.527Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6500lb # [ 142.132154] niks3-server[1052]: 2026/09/20 12:48:13 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6501lb # [ 142.139169] niks3-server[1052]: 2026/09/20 12:48:13 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv6502lb # [ 142.149719] niks3-server[1052]: 2026/09/20 12:48:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6503worker1 # [ 142.090438] nix-grpc-daemon[1639]: time=2026-09-20T12:48:12.576Z level=INFO msg="Uploading 1 narinfos"6504lb # [ 142.155167] niks3-server[1052]: 2026/09/20 12:48:13 INFO Signed narinfos id=51 count=16505lb # [ 142.160483] niks3-server[1052]: 2026/09/20 12:48:13 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls6506lb # [ 142.168991] niks3-server[1052]: 2026/09/20 12:48:13 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6507lb # [ 142.173391] niks3-server[1052]: 2026/09/20 12:48:13 INFO Completed upload id=516508worker1 # [ 142.111443] nix-grpc-daemon[1639]: time=2026-09-20T12:48:12.597Z level=INFO msg="Upload complete. (290ms)"6509lb # [ 142.177358] niks3-server[1052]: 2026/09/20 12:48:13 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo6510worker1 # [ 142.116294] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60252 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=9 outputs=16511worker2 # [ 142.187645] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:13Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=16512worker1 # [ 142.125216] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602526513worker2 # [ 142.197359] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:13Z level=debug event=want client=5 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=06514worker1 # [ 142.137557] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602526515worker1 # [ 142.142080] nix-daemon[908]: reaped child process 1730, status = succeeded6516worker1 # [ 142.148808] nix-grpc-daemon[1639]: time=2026-09-20T12:48:12.635Z level=INFO msg="All 1 paths already cached"6517worker1 # [ 142.154757] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:602526518worker1 # [ 142.161423] nix-daemon[908]: accepted connection from pid 1583, user nix-grpc-daemon (trusted)6519worker1 # [ 142.323290] nix-daemon[908]: accepted connection from pid 1754, user nix-grpc-daemon (trusted)6520worker1 # [ 142.335459] nix-daemon[908]: reaped child process 1757, status = succeeded6521worker1 # [ 142.506041] nix-daemon[908]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)6522worker1 # [ 142.517510] nix-daemon[908]: reaped child process 1762, status = succeeded6523lb # [ 142.599148] niks3-server[1052]: 2026/09/20 12:48:13 INFO Received uploads request method=POST path=/api/pending_closures6524worker1 # [ 142.543172] nix-grpc-daemon[1639]: time=2026-09-20T12:48:13.029Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6525worker1 # [ 142.544642] nix-grpc-daemon[1639]: time=2026-09-20T12:48:13.031Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"6526lb # [ 142.627172] niks3-server[1052]: 2026/09/20 12:48:13 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst6527lb # [ 142.640675] niks3-server[1052]: 2026/09/20 12:48:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6528lb # [ 142.647637] niks3-server[1052]: 2026/09/20 12:48:13 INFO Signed narinfos id=52 count=16529worker1 # [ 142.586534] nix-grpc-daemon[1639]: time=2026-09-20T12:48:13.072Z level=INFO msg="Uploading 1 narinfos"6530lb # [ 142.651349] niks3-server[1052]: 2026/09/20 12:48:13 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls6531lb # [ 142.669958] niks3-server[1052]: 2026/09/20 12:48:13 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6532lb # [ 142.674622] niks3-server[1052]: 2026/09/20 12:48:13 INFO Completed upload id=526533lb # [ 142.677265] niks3-server[1052]: 2026/09/20 12:48:13 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo6534worker1 # [ 142.612634] nix-grpc-daemon[1639]: time=2026-09-20T12:48:13.098Z level=INFO msg="Upload complete. (444ms)"6535worker1 # [ 142.616891] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60252 duration_s=0 paths=1 nar_bytes_in=6246536worker1 # [ 142.623320] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602526537worker1 # [ 142.633245] nix-daemon[908]: accepted connection from pid 1583, user nix-grpc-daemon (trusted)6538worker1 # [ 142.642343] nix-daemon[908]: reaped child process 1752, status = succeeded6539worker1 # [ 142.648212] nix-daemon[908]: accepted connection from pid 1583, user nix-grpc-daemon (trusted)6540worker1 # [ 143.861611] nix-daemon[908]: reaped child process 1766, status = succeeded6541worker1 # [ 144.003277] nix-daemon[908]: accepted connection from pid 1781, user nix-grpc-daemon (trusted)6542worker1 # [ 144.014165] nix-daemon[908]: reaped child process 1784, status = succeeded6543lb # [ 144.091042] niks3-server[1052]: 2026/09/20 12:48:15 INFO Received uploads request method=POST path=/api/pending_closures6544worker1 # [ 144.031586] nix-grpc-daemon[1639]: time=2026-09-20T12:48:14.517Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6545worker1 # [ 144.033237] nix-grpc-daemon[1639]: time=2026-09-20T12:48:14.519Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"6546lb # [ 144.114261] niks3-server[1052]: 2026/09/20 12:48:15 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv6547lb # [ 144.116497] niks3-server[1052]: 2026/09/20 12:48:15 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst6548lb # [ 144.121055] niks3-server[1052]: 2026/09/20 12:48:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6549lb # [ 144.123786] niks3-server[1052]: 2026/09/20 12:48:15 INFO Signed narinfos id=53 count=16550worker1 # [ 144.059686] nix-grpc-daemon[1639]: time=2026-09-20T12:48:14.545Z level=INFO msg="Uploading 1 narinfos"6551lb # [ 144.126909] niks3-server[1052]: 2026/09/20 12:48:15 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls6552lb # [ 144.131171] niks3-server[1052]: 2026/09/20 12:48:15 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6553lb # [ 144.133744] niks3-server[1052]: 2026/09/20 12:48:15 INFO Completed upload id=536554worker1 # [ 144.070568] nix-grpc-daemon[1639]: time=2026-09-20T12:48:14.556Z level=INFO msg="Upload complete. (215ms)"6555lb # [ 144.136227] niks3-server[1052]: 2026/09/20 12:48:15 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo6556worker1 # [ 144.072872] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60252 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=10 outputs=16557worker1 # [ 144.079915] nix-daemon[908]: reaped child process 1764, status = succeeded6558worker2 # [ 144.146606] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:15Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=16559worker1 # [ 144.082445] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602526560worker1 # [ 144.088578] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602526561worker2 # [ 144.157228] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:15Z level=debug event=want client=5 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=06562worker1 # [ 144.094477] nix-grpc-daemon[1639]: time=2026-09-20T12:48:14.580Z level=INFO msg="All 1 paths already cached"6563worker1 # [ 144.098452] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:602526564worker1 # [ 144.103511] nix-daemon[908]: accepted connection from pid 1583, user nix-grpc-daemon (trusted)6565worker1 # [ 144.271279] nix-daemon[908]: accepted connection from pid 1788, user nix-grpc-daemon (trusted)6566worker1 # [ 144.283132] nix-daemon[908]: reaped child process 1791, status = succeeded6567worker1 # [ 144.433983] nix-daemon[908]: accepted connection from pid 1793, user nix-grpc-daemon (trusted)6568worker1 # [ 144.445528] nix-daemon[908]: reaped child process 1796, status = succeeded6569lb # [ 144.528335] niks3-server[1052]: 2026/09/20 12:48:15 INFO Received uploads request method=POST path=/api/pending_closures6570worker1 # [ 144.471888] nix-grpc-daemon[1639]: time=2026-09-20T12:48:14.957Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6571worker1 # [ 144.473282] nix-grpc-daemon[1639]: time=2026-09-20T12:48:14.958Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"6572lb # [ 144.555283] niks3-server[1052]: 2026/09/20 12:48:15 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst6573lb # [ 144.564253] niks3-server[1052]: 2026/09/20 12:48:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6574lb # [ 144.569490] niks3-server[1052]: 2026/09/20 12:48:15 INFO Signed narinfos id=54 count=16575worker1 # [ 144.508346] nix-grpc-daemon[1639]: time=2026-09-20T12:48:14.994Z level=INFO msg="Uploading 1 narinfos"6576lb # [ 144.580954] niks3-server[1052]: 2026/09/20 12:48:15 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls6577lb # [ 144.588411] niks3-server[1052]: 2026/09/20 12:48:15 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6578worker1 # [ 144.530022] nix-grpc-daemon[1639]: time=2026-09-20T12:48:15.016Z level=INFO msg="Upload complete. (421ms)"6579lb # [ 144.594527] niks3-server[1052]: 2026/09/20 12:48:15 INFO Completed upload id=546580lb # [ 144.598335] niks3-server[1052]: 2026/09/20 12:48:15 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo6581worker1 # [ 144.534080] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60252 duration_s=0 paths=1 nar_bytes_in=6246582worker1 # [ 144.538396] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602526583worker1 # [ 144.549682] nix-daemon[908]: accepted connection from pid 1583, user nix-grpc-daemon (trusted)6584worker1 # [ 144.551300] nix-daemon[908]: reaped child process 1786, status = succeeded6585worker1 # [ 144.556251] nix-daemon[908]: accepted connection from pid 1583, user nix-grpc-daemon (trusted)6586worker1 # [ 145.780739] nix-daemon[908]: reaped child process 1800, status = succeeded6587worker1 # [ 145.953632] nix-daemon[908]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6588worker1 # [ 145.965485] nix-daemon[908]: reaped child process 1818, status = succeeded6589lb # [ 146.044965] niks3-server[1052]: 2026/09/20 12:48:17 INFO Received uploads request method=POST path=/api/pending_closures6590worker1 # [ 145.991192] nix-grpc-daemon[1639]: time=2026-09-20T12:48:16.477Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6591worker1 # [ 145.992679] nix-grpc-daemon[1639]: time=2026-09-20T12:48:16.479Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"6592lb # [ 146.080152] niks3-server[1052]: 2026/09/20 12:48:17 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst6593lb # [ 146.093747] niks3-server[1052]: 2026/09/20 12:48:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6594lb # [ 146.102253] niks3-server[1052]: 2026/09/20 12:48:17 INFO Signed narinfos id=55 count=16595worker1 # [ 146.040931] nix-grpc-daemon[1639]: time=2026-09-20T12:48:16.527Z level=INFO msg="Uploading 1 narinfos"6596lb # [ 146.108592] niks3-server[1052]: 2026/09/20 12:48:17 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls6597lb # [ 146.113656] niks3-server[1052]: 2026/09/20 12:48:17 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv6598lb # [ 146.122294] niks3-server[1052]: 2026/09/20 12:48:17 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6599lb # [ 146.127297] niks3-server[1052]: 2026/09/20 12:48:17 INFO Completed upload id=556600worker1 # [ 146.063196] nix-grpc-daemon[1639]: time=2026-09-20T12:48:16.549Z level=INFO msg="Upload complete. (298ms)"6601lb # [ 146.129881] niks3-server[1052]: 2026/09/20 12:48:17 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo6602worker1 # [ 146.070381] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60252 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=11 outputs=16603worker2 # [ 146.141965] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:17Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=16604worker1 # [ 146.077157] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602526605worker2 # [ 146.149524] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:17Z level=debug event=want client=5 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=06606worker1 # [ 146.087505] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602526607worker1 # [ 146.093740] nix-daemon[908]: reaped child process 1798, status = succeeded6608worker1 # [ 146.098559] nix-grpc-daemon[1639]: time=2026-09-20T12:48:16.584Z level=INFO msg="All 1 paths already cached"6609worker1 # [ 146.102882] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:602526610worker1 # [ 146.108648] nix-daemon[908]: accepted connection from pid 1583, user nix-grpc-daemon (trusted)6611worker1 # [ 146.267057] nix-daemon[908]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)6612worker1 # [ 146.278720] nix-daemon[908]: reaped child process 1825, status = succeeded6613worker1 # [ 146.455143] nix-daemon[908]: accepted connection from pid 1827, user nix-grpc-daemon (trusted)6614worker1 # [ 146.466776] nix-daemon[908]: reaped child process 1830, status = succeeded6615lb # [ 146.550633] niks3-server[1052]: 2026/09/20 12:48:17 INFO Received uploads request method=POST path=/api/pending_closures6616worker1 # [ 146.494201] nix-grpc-daemon[1639]: time=2026-09-20T12:48:16.980Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6617worker1 # [ 146.495679] nix-grpc-daemon[1639]: time=2026-09-20T12:48:16.982Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"6618lb # [ 146.578386] niks3-server[1052]: 2026/09/20 12:48:17 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst6619lb # [ 146.590113] niks3-server[1052]: 2026/09/20 12:48:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6620worker1 # [ 146.532293] nix-grpc-daemon[1639]: time=2026-09-20T12:48:17.018Z level=INFO msg="Uploading 1 narinfos"6621lb # [ 146.596718] niks3-server[1052]: 2026/09/20 12:48:17 INFO Signed narinfos id=56 count=16622lb # [ 146.601054] niks3-server[1052]: 2026/09/20 12:48:17 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls6623lb # [ 146.613751] niks3-server[1052]: 2026/09/20 12:48:17 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6624worker1 # [ 146.554036] nix-grpc-daemon[1639]: time=2026-09-20T12:48:17.040Z level=INFO msg="Upload complete. (440ms)"6625lb # [ 146.618623] niks3-server[1052]: 2026/09/20 12:48:17 INFO Completed upload id=566626worker1 # [ 146.556232] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60252 duration_s=0 paths=1 nar_bytes_in=6246627lb # [ 146.622452] niks3-server[1052]: 2026/09/20 12:48:17 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo6628worker1 # [ 146.562343] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602526629worker1 # [ 146.571731] nix-daemon[908]: accepted connection from pid 1583, user nix-grpc-daemon (trusted)6630worker1 # [ 146.575163] nix-daemon[908]: reaped child process 1820, status = succeeded6631worker1 # [ 146.580169] nix-daemon[908]: accepted connection from pid 1583, user nix-grpc-daemon (trusted)6632worker1 # [ 147.794428] nix-daemon[908]: reaped child process 1834, status = succeeded6633worker1 # [ 147.941823] nix-daemon[908]: accepted connection from pid 1849, user nix-grpc-daemon (trusted)6634worker1 # [ 147.953756] nix-daemon[908]: reaped child process 1852, status = succeeded6635lb # [ 148.031762] niks3-server[1052]: 2026/09/20 12:48:19 INFO Received uploads request method=POST path=/api/pending_closures6636worker1 # [ 147.976346] nix-grpc-daemon[1639]: time=2026-09-20T12:48:18.462Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6637worker1 # [ 147.978515] nix-grpc-daemon[1639]: time=2026-09-20T12:48:18.464Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"6638lb # [ 148.063626] niks3-server[1052]: 2026/09/20 12:48:19 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv6639lb # [ 148.078199] niks3-server[1052]: 2026/09/20 12:48:19 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst6640lb # [ 148.087511] niks3-server[1052]: 2026/09/20 12:48:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6641lb # [ 148.092868] niks3-server[1052]: 2026/09/20 12:48:19 INFO Signed narinfos id=57 count=16642worker1 # [ 148.031269] nix-grpc-daemon[1639]: time=2026-09-20T12:48:18.517Z level=INFO msg="Uploading 1 narinfos"6643lb # [ 148.098056] niks3-server[1052]: 2026/09/20 12:48:19 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls6644lb # [ 148.106834] niks3-server[1052]: 2026/09/20 12:48:19 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6645lb # [ 148.111325] niks3-server[1052]: 2026/09/20 12:48:19 INFO Completed upload id=576646lb # [ 148.114332] niks3-server[1052]: 2026/09/20 12:48:19 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo6647worker1 # [ 148.049112] nix-grpc-daemon[1639]: time=2026-09-20T12:48:18.535Z level=INFO msg="Upload complete. (262ms)"6648worker1 # [ 148.053767] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60252 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=12 outputs=16649worker2 # [ 148.128417] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:19Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=16650worker1 # [ 148.061784] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602526651worker2 # [ 148.134895] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:19Z level=debug event=want client=5 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=06652worker1 # [ 148.072349] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602526653worker1 # [ 148.080542] nix-daemon[908]: reaped child process 1832, status = succeeded6654worker1 # [ 148.083144] nix-grpc-daemon[1639]: time=2026-09-20T12:48:18.569Z level=INFO msg="All 2 paths already cached"6655worker1 # [ 148.085945] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:602526656worker1 # [ 148.090737] nix-daemon[908]: accepted connection from pid 1583, user nix-grpc-daemon (trusted)6657worker1 # [ 148.249620] nix-daemon[908]: accepted connection from pid 1856, user nix-grpc-daemon (trusted)6658worker1 # [ 148.261811] nix-daemon[908]: reaped child process 1859, status = succeeded6659worker1 # [ 148.437088] nix-daemon[908]: accepted connection from pid 1861, user nix-grpc-daemon (trusted)6660worker1 # [ 148.447578] nix-daemon[908]: reaped child process 1864, status = succeeded6661lb # [ 148.528888] niks3-server[1052]: 2026/09/20 12:48:19 INFO Received uploads request method=POST path=/api/pending_closures6662worker1 # [ 148.472427] nix-grpc-daemon[1639]: time=2026-09-20T12:48:18.958Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6663worker1 # [ 148.474029] nix-grpc-daemon[1639]: time=2026-09-20T12:48:18.960Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"6664lb # [ 148.557425] niks3-server[1052]: 2026/09/20 12:48:19 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst6665lb # [ 148.568629] niks3-server[1052]: 2026/09/20 12:48:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6666worker1 # [ 148.510693] nix-grpc-daemon[1639]: time=2026-09-20T12:48:18.997Z level=INFO msg="Uploading 1 narinfos"6667lb # [ 148.575175] niks3-server[1052]: 2026/09/20 12:48:19 INFO Signed narinfos id=58 count=16668lb # [ 148.581538] niks3-server[1052]: 2026/09/20 12:48:19 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls6669lb # [ 148.593918] niks3-server[1052]: 2026/09/20 12:48:19 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6670worker1 # [ 148.534181] nix-grpc-daemon[1639]: time=2026-09-20T12:48:19.020Z level=INFO msg="Upload complete. (438ms)"6671lb # [ 148.598514] niks3-server[1052]: 2026/09/20 12:48:19 INFO Completed upload id=586672worker1 # [ 148.536211] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60252 duration_s=0 paths=1 nar_bytes_in=7526673lb # [ 148.602269] niks3-server[1052]: 2026/09/20 12:48:19 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo6674worker1 # [ 148.542497] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602526675worker1 # [ 148.546589] nix-daemon[908]: accepted connection from pid 1583, user nix-grpc-daemon (trusted)6676worker1 # [ 148.548884] nix-daemon[908]: reaped child process 1854, status = succeeded6677worker1 # [ 148.554098] nix-daemon[908]: accepted connection from pid 1583, user nix-grpc-daemon (trusted)6678worker1 # [ 149.179336] nix-daemon[908]: reaped child process 1868, status = succeeded6679worker1 # [ 149.337351] nix-daemon[908]: accepted connection from pid 1883, user nix-grpc-daemon (trusted)6680worker1 # [ 149.349342] nix-daemon[908]: reaped child process 1886, status = succeeded6681lb # [ 149.433944] niks3-server[1052]: 2026/09/20 12:48:20 INFO Received uploads request method=POST path=/api/pending_closures6682worker1 # [ 149.380530] nix-grpc-daemon[1639]: time=2026-09-20T12:48:19.866Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6683worker1 # [ 149.382341] nix-grpc-daemon[1639]: time=2026-09-20T12:48:19.868Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"6684lb # [ 149.481175] niks3-server[1052]: 2026/09/20 12:48:20 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst6685lb # [ 149.492274] niks3-server[1052]: 2026/09/20 12:48:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6686lb # [ 149.496636] niks3-server[1052]: 2026/09/20 12:48:20 INFO Signed narinfos id=59 count=16687worker1 # [ 149.434859] nix-grpc-daemon[1639]: time=2026-09-20T12:48:19.921Z level=INFO msg="Uploading 1 narinfos"6688lb # [ 149.499391] niks3-server[1052]: 2026/09/20 12:48:20 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv6689lb # [ 149.505667] niks3-server[1052]: 2026/09/20 12:48:20 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls6690lb # [ 149.510975] niks3-server[1052]: 2026/09/20 12:48:20 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6691lb # [ 149.513607] niks3-server[1052]: 2026/09/20 12:48:20 INFO Completed upload id=596692worker1 # [ 149.449578] nix-grpc-daemon[1639]: time=2026-09-20T12:48:19.935Z level=INFO msg="Upload complete. (289ms)"6693lb # [ 149.516261] niks3-server[1052]: 2026/09/20 12:48:20 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo6694worker2 # [ 149.526664] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:20Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=16695worker1 # [ 149.454972] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60252 duration_s=0 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=13 outputs=16696worker1 # [ 149.468730] nix-daemon[908]: reaped child process 1866, status = succeeded6697worker1 # [ 149.499754] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602526698worker1 # [ 149.512597] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602526699worker1 # [ 149.525697] nix-grpc-daemon[1639]: time=2026-09-20T12:48:20.011Z level=INFO msg="All 2 paths already cached"6700worker1 # [ 149.532647] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:602526701worker1 # [ 149.542171] nix-daemon[908]: accepted connection from pid 1583, user nix-grpc-daemon (trusted)6702worker1 # [ 149.709550] nix-daemon[908]: accepted connection from pid 1890, user nix-grpc-daemon (trusted)6703worker1 # [ 149.722177] nix-daemon[908]: reaped child process 1893, status = succeeded6704worker1 # [ 149.892772] nix-daemon[908]: accepted connection from pid 1895, user nix-grpc-daemon (trusted)6705worker1 # [ 149.904540] nix-daemon[908]: reaped child process 1898, status = succeeded6706lb # [ 149.981825] niks3-server[1052]: 2026/09/20 12:48:21 INFO Received uploads request method=POST path=/api/pending_closures6707worker1 # [ 149.925755] nix-grpc-daemon[1639]: time=2026-09-20T12:48:20.411Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6708worker1 # [ 149.927118] nix-grpc-daemon[1639]: time=2026-09-20T12:48:20.411Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"6709lb # [ 150.005249] niks3-server[1052]: 2026/09/20 12:48:21 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst6710lb # [ 150.017756] niks3-server[1052]: 2026/09/20 12:48:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6711worker1 # [ 149.958308] nix-grpc-daemon[1639]: time=2026-09-20T12:48:20.444Z level=INFO msg="Uploading 1 narinfos"6712lb # [ 150.023906] niks3-server[1052]: 2026/09/20 12:48:21 INFO Signed narinfos id=60 count=16713lb # [ 150.028897] niks3-server[1052]: 2026/09/20 12:48:21 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls6714lb # [ 150.038967] niks3-server[1052]: 2026/09/20 12:48:21 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6715worker1 # [ 149.980350] nix-grpc-daemon[1639]: time=2026-09-20T12:48:20.466Z level=INFO msg="Upload complete. (427ms)"6716lb # [ 150.044464] niks3-server[1052]: 2026/09/20 12:48:21 INFO Completed upload id=606717worker1 # [ 149.982591] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60252 duration_s=0 paths=1 nar_bytes_in=7526718lb # [ 150.050108] niks3-server[1052]: 2026/09/20 12:48:21 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo6719worker1 # [ 149.990632] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602526720worker1 # [ 149.995355] nix-daemon[908]: accepted connection from pid 1583, user nix-grpc-daemon (trusted)6721worker1 # [ 149.997620] nix-daemon[908]: reaped child process 1888, status = succeeded6722worker1 # [ 150.003026] nix-daemon[908]: accepted connection from pid 1583, user nix-grpc-daemon (trusted)6723worker1 # [ 151.229206] nix-daemon[908]: reaped child process 1902, status = succeeded6724worker1 # [ 151.390771] nix-daemon[908]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)6725worker1 # [ 151.401481] nix-daemon[908]: reaped child process 1920, status = succeeded6726lb # [ 151.484405] niks3-server[1052]: 2026/09/20 12:48:22 INFO Received uploads request method=POST path=/api/pending_closures6727worker1 # [ 151.428366] nix-grpc-daemon[1639]: time=2026-09-20T12:48:21.914Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6728worker1 # [ 151.429876] nix-grpc-daemon[1639]: time=2026-09-20T12:48:21.916Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"6729lb # [ 151.524584] niks3-server[1052]: 2026/09/20 12:48:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6730worker1 # [ 151.472381] nix-grpc-daemon[1639]: time=2026-09-20T12:48:21.958Z level=INFO msg="Uploading 1 narinfos"6731lb # [ 151.538196] niks3-server[1052]: 2026/09/20 12:48:22 INFO Signed narinfos id=61 count=16732lb # [ 151.540895] niks3-server[1052]: 2026/09/20 12:48:22 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst6733lb # [ 151.547617] niks3-server[1052]: 2026/09/20 12:48:22 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls6734lb # [ 151.552381] niks3-server[1052]: 2026/09/20 12:48:22 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv6735lb # [ 151.560985] niks3-server[1052]: 2026/09/20 12:48:22 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6736lb # [ 151.563855] niks3-server[1052]: 2026/09/20 12:48:22 INFO Completed upload id=616737worker1 # [ 151.499779] nix-grpc-daemon[1639]: time=2026-09-20T12:48:21.986Z level=INFO msg="Upload complete. (288ms)"6738lb # [ 151.566622] niks3-server[1052]: 2026/09/20 12:48:22 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo6739worker1 # [ 151.504801] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60252 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=14 outputs=16740worker2 # [ 151.576712] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:22Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=16741worker1 # [ 151.518128] nix-daemon[908]: reaped child process 1900, status = succeeded6742worker1 # [ 151.549566] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602526743worker1 # [ 151.561973] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602526744worker1 # [ 151.574434] nix-grpc-daemon[1639]: time=2026-09-20T12:48:22.060Z level=INFO msg="All 2 paths already cached"6745worker1 # [ 151.581677] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:602526746worker1 # [ 151.589952] nix-daemon[908]: accepted connection from pid 1583, user nix-grpc-daemon (trusted)6747worker1 # [ 151.758416] nix-daemon[908]: accepted connection from pid 1924, user nix-grpc-daemon (trusted)6748worker1 # [ 151.770343] nix-daemon[908]: reaped child process 1927, status = succeeded6749worker1 # [ 151.934298] nix-daemon[908]: accepted connection from pid 1929, user nix-grpc-daemon (trusted)6750worker1 # [ 151.945960] nix-daemon[908]: reaped child process 1932, status = succeeded6751lb # [ 152.027639] niks3-server[1052]: 2026/09/20 12:48:23 INFO Received uploads request method=POST path=/api/pending_closures6752worker1 # [ 151.972226] nix-grpc-daemon[1639]: time=2026-09-20T12:48:22.458Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6753worker1 # [ 151.973639] nix-grpc-daemon[1639]: time=2026-09-20T12:48:22.458Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"6754lb # [ 152.052661] niks3-server[1052]: 2026/09/20 12:48:23 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst6755lb # [ 152.065094] niks3-server[1052]: 2026/09/20 12:48:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6756worker1 # [ 152.005509] nix-grpc-daemon[1639]: time=2026-09-20T12:48:22.491Z level=INFO msg="Uploading 1 narinfos"6757lb # [ 152.071153] niks3-server[1052]: 2026/09/20 12:48:23 INFO Signed narinfos id=62 count=16758lb # [ 152.076659] niks3-server[1052]: 2026/09/20 12:48:23 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls6759lb # [ 152.087381] niks3-server[1052]: 2026/09/20 12:48:23 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6760worker1 # [ 152.029302] nix-grpc-daemon[1639]: time=2026-09-20T12:48:22.515Z level=INFO msg="Upload complete. (426ms)"6761lb # [ 152.093419] niks3-server[1052]: 2026/09/20 12:48:23 INFO Completed upload id=626762worker1 # [ 152.031679] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60252 duration_s=0 paths=1 nar_bytes_in=7526763lb # [ 152.096612] niks3-server[1052]: 2026/09/20 12:48:23 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo6764worker1 # [ 152.036366] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602526765worker1 # [ 152.041603] nix-daemon[908]: accepted connection from pid 1583, user nix-grpc-daemon (trusted)6766worker1 # [ 152.043862] nix-daemon[908]: reaped child process 1922, status = succeeded6767worker1 # [ 152.049062] nix-daemon[908]: accepted connection from pid 1583, user nix-grpc-daemon (trusted)6768worker1 # [ 152.869366] nix-daemon[908]: reaped child process 1936, status = succeeded6769worker1 # [ 153.016858] nix-daemon[908]: accepted connection from pid 1951, user nix-grpc-daemon (trusted)6770worker1 # [ 153.028778] nix-daemon[908]: reaped child process 1954, status = succeeded6771lb # [ 153.105870] niks3-server[1052]: 2026/09/20 12:48:24 INFO Received uploads request method=POST path=/api/pending_closures6772worker1 # [ 153.046040] nix-grpc-daemon[1639]: time=2026-09-20T12:48:23.532Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6773worker1 # [ 153.047687] nix-grpc-daemon[1639]: time=2026-09-20T12:48:23.533Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"6774lb # [ 153.130957] niks3-server[1052]: 2026/09/20 12:48:24 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv6775lb # [ 153.133600] niks3-server[1052]: 2026/09/20 12:48:24 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst6776lb # [ 153.137084] niks3-server[1052]: 2026/09/20 12:48:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6777lb # [ 153.139086] niks3-server[1052]: 2026/09/20 12:48:24 INFO Signed narinfos id=63 count=16778worker1 # [ 153.076287] nix-grpc-daemon[1639]: time=2026-09-20T12:48:23.562Z level=INFO msg="Uploading 1 narinfos"6779lb # [ 153.141648] niks3-server[1052]: 2026/09/20 12:48:24 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls6780lb # [ 153.147329] niks3-server[1052]: 2026/09/20 12:48:24 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6781lb # [ 153.150069] niks3-server[1052]: 2026/09/20 12:48:24 INFO Completed upload id=636782worker1 # [ 153.086988] nix-grpc-daemon[1639]: time=2026-09-20T12:48:23.573Z level=INFO msg="Upload complete. (231ms)"6783lb # [ 153.151993] niks3-server[1052]: 2026/09/20 12:48:24 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo6784worker1 # [ 153.089365] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60252 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=15 outputs=16785worker1 # [ 153.095926] nix-daemon[908]: reaped child process 1934, status = succeeded6786client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 17.41 seconds)6787worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6788worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6789client: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&16790worker1 # [ 153.442558] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:602526791worker1 # [ 153.450351] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602526792worker2 # [ 153.543873] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:24Z level=debug event=want client=6 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=06793lb # [ 157.346743] envoy[910]: conn peer=[2001:db8:1::1]:47532 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27046 tx=10656 ms=171306794client: (finished: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&1, in 30.29 seconds)6795(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 47.76 seconds)6796subtest: low disk drains a worker and builds go to the other6797worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6798worker1 # [ 184.493307] nix-grpc-daemon[1583]: ts=2026-09-20T12:48:54Z level=info event=unhealthy reason=min_free6799worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.00 seconds)6800worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6801worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 0.05 seconds)6802client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&26803worker2 # [ 184.843616] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:545146804client # don't know how to build these paths:6805client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6806worker2 # [ 184.851779] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:545146807worker2 # [ 184.870422] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:55Z level=debug event=want client=7 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=06808worker2 # [ 184.876435] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:55Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=16809worker2 # [ 184.881254] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545146810worker2 # [ 184.890133] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:545146811client # copying 1 paths to 'lb:50051'...6812client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...6813worker2 # [ 184.899441] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:545146814worker2 # [ 184.910368] nix-daemon[860]: accepted connection from pid 1551, user nix-grpc-daemon (trusted)6815worker2 # [ 185.084221] nix-daemon[860]: accepted connection from pid 1829, user nix-grpc-daemon (trusted)6816worker2 # [ 185.098313] nix-daemon[860]: reaped child process 1832, status = succeeded6817worker2 # [ 185.264861] nix-daemon[860]: accepted connection from pid 1834, user nix-grpc-daemon (trusted)6818worker2 # [ 185.278198] nix-daemon[860]: reaped child process 1837, status = succeeded6819lb # [ 185.287767] niks3-server[1052]: 2026/09/20 12:48:56 INFO Received uploads request method=POST path=/api/pending_closures6820worker2 # [ 185.304665] nix-grpc-daemon[1627]: time=2026-09-20T12:48:56.213Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6821worker2 # [ 185.306287] nix-grpc-daemon[1627]: time=2026-09-20T12:48:56.213Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"6822lb # [ 185.318986] niks3-server[1052]: 2026/09/20 12:48:56 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst6823lb # [ 185.332478] niks3-server[1052]: 2026/09/20 12:48:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6824worker2 # [ 185.347759] nix-grpc-daemon[1627]: time=2026-09-20T12:48:56.256Z level=INFO msg="Uploading 1 narinfos"6825lb # [ 185.341350] niks3-server[1052]: 2026/09/20 12:48:56 INFO Signed narinfos id=64 count=16826lb # [ 185.348789] niks3-server[1052]: 2026/09/20 12:48:56 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls6827lb # [ 185.359322] niks3-server[1052]: 2026/09/20 12:48:56 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6828lb # [ 185.365396] niks3-server[1052]: 2026/09/20 12:48:56 INFO Completed upload id=646829worker2 # [ 185.371543] nix-grpc-daemon[1627]: time=2026-09-20T12:48:56.280Z level=INFO msg="Upload complete. (451ms)"6830lb # [ 185.367157] niks3-server[1052]: 2026/09/20 12:48:56 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo6831worker2 # [ 185.374310] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54514 duration_s=0 paths=1 nar_bytes_in=5126832worker2 # [ 185.377988] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545146833worker2 # [ 185.385532] nix-daemon[860]: accepted connection from pid 1551, user nix-grpc-daemon (trusted)6834worker2 # [ 185.391160] nix-daemon[860]: reaped child process 1827, status = succeeded6835worker2 # [ 185.396957] nix-daemon[860]: accepted connection from pid 1551, user nix-grpc-daemon (trusted)6836client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...6837client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6838client # farm-a> LOG-farm-a-drain6839worker2 # [ 187.026665] nix-daemon[860]: reaped child process 1841, status = succeeded6840worker2 # [ 187.189210] nix-daemon[860]: accepted connection from pid 1856, user nix-grpc-daemon (trusted)6841worker2 # [ 187.203506] nix-daemon[860]: reaped child process 1859, status = succeeded6842lb # [ 187.215179] niks3-server[1052]: 2026/09/20 12:48:58 INFO Received uploads request method=POST path=/api/pending_closures6843worker2 # [ 187.228171] nix-grpc-daemon[1627]: time=2026-09-20T12:48:58.137Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6844worker2 # [ 187.233248] nix-grpc-daemon[1627]: time=2026-09-20T12:48:58.138Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"6845lb # [ 187.248419] niks3-server[1052]: 2026/09/20 12:48:58 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6846lb # [ 187.266111] niks3-server[1052]: 2026/09/20 12:48:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6847lb # [ 187.271317] niks3-server[1052]: 2026/09/20 12:48:58 INFO Signed narinfos id=65 count=16848worker2 # [ 187.280571] nix-grpc-daemon[1627]: time=2026-09-20T12:48:58.189Z level=INFO msg="Uploading 1 narinfos"6849lb # [ 187.278808] niks3-server[1052]: 2026/09/20 12:48:58 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6850lb # [ 187.283531] niks3-server[1052]: 2026/09/20 12:48:58 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls6851lb # [ 187.292695] niks3-server[1052]: 2026/09/20 12:48:58 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6852lb # [ 187.297752] niks3-server[1052]: 2026/09/20 12:48:58 INFO Completed upload id=656853worker2 # [ 187.303959] nix-grpc-daemon[1627]: time=2026-09-20T12:48:58.213Z level=INFO msg="Upload complete. (297ms)"6854lb # [ 187.299537] niks3-server[1052]: 2026/09/20 12:48:58 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo6855worker2 # [ 187.312538] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54514 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=16 outputs=16856worker2 # [ 187.322101] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:58Z level=debug event=want client=7 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=06857worker2 # [ 187.326269] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:58Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=16858worker2 # [ 187.335692] nix-daemon[860]: reaped child process 1839, status = succeeded6859lb # [ 187.349730] envoy[910]: conn peer=[2001:db8:1::1]:42840 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=895 tx=498 ms=300466860worker2 # [ 187.366247] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545146861worker2 # [ 187.378324] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:545146862worker2 # [ 187.392354] nix-grpc-daemon[1627]: time=2026-09-20T12:48:58.301Z level=INFO msg="All 1 paths already cached"6863client # copying 1 paths to 'lb:50051'...6864client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...6865worker2 # [ 187.399349] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:545146866worker2 # [ 187.409172] nix-daemon[860]: accepted connection from pid 1551, user nix-grpc-daemon (trusted)6867worker2 # [ 187.558352] nix-daemon[860]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)6868worker2 # [ 187.572188] nix-daemon[860]: reaped child process 1868, status = succeeded6869worker2 # [ 187.725524] nix-daemon[860]: accepted connection from pid 1870, user nix-grpc-daemon (trusted)6870worker2 # [ 187.738731] nix-daemon[860]: reaped child process 1873, status = succeeded6871lb # [ 187.743278] niks3-server[1052]: 2026/09/20 12:48:58 INFO Received uploads request method=POST path=/api/pending_closures6872worker2 # [ 187.757196] nix-grpc-daemon[1627]: time=2026-09-20T12:48:58.666Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6873worker2 # [ 187.758576] nix-grpc-daemon[1627]: time=2026-09-20T12:48:58.666Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"6874lb # [ 187.768596] niks3-server[1052]: 2026/09/20 12:48:58 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst6875lb # [ 187.779130] niks3-server[1052]: 2026/09/20 12:48:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6876worker2 # [ 187.789690] nix-grpc-daemon[1627]: time=2026-09-20T12:48:58.698Z level=INFO msg="Uploading 1 narinfos"6877lb # [ 187.783887] niks3-server[1052]: 2026/09/20 12:48:58 INFO Signed narinfos id=66 count=16878lb # [ 187.789951] niks3-server[1052]: 2026/09/20 12:48:58 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls6879lb # [ 187.800745] niks3-server[1052]: 2026/09/20 12:48:58 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6880lb # [ 187.807841] niks3-server[1052]: 2026/09/20 12:48:58 INFO Completed upload id=666881worker2 # [ 187.816675] nix-grpc-daemon[1627]: time=2026-09-20T12:48:58.725Z level=INFO msg="Upload complete. (396ms)"6882worker2 # [ 187.820079] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54514 duration_s=0 paths=1 nar_bytes_in=6246883lb # [ 187.815709] niks3-server[1052]: 2026/09/20 12:48:58 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo6884worker2 # [ 187.822607] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545146885worker2 # [ 187.827585] nix-daemon[860]: accepted connection from pid 1551, user nix-grpc-daemon (trusted)6886worker2 # [ 187.832510] nix-daemon[860]: reaped child process 1863, status = succeeded6887client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...6888client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6889worker2 # [ 187.838707] nix-daemon[860]: accepted connection from pid 1551, user nix-grpc-daemon (trusted)6890client # farm-b> LOG-farm-b-drain6891worker2 # [ 188.855256] nix-daemon[860]: reaped child process 1877, status = succeeded6892worker2 # [ 188.990419] nix-daemon[860]: accepted connection from pid 1892, user nix-grpc-daemon (trusted)6893worker2 # [ 189.003718] nix-daemon[860]: reaped child process 1895, status = succeeded6894lb # [ 189.011079] niks3-server[1052]: 2026/09/20 12:49:00 INFO Received uploads request method=POST path=/api/pending_closures6895worker2 # [ 189.021619] nix-grpc-daemon[1627]: time=2026-09-20T12:48:59.930Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6896worker2 # [ 189.023168] nix-grpc-daemon[1627]: time=2026-09-20T12:48:59.932Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"6897lb # [ 189.031278] niks3-server[1052]: 2026/09/20 12:49:00 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst6898lb # [ 189.037510] niks3-server[1052]: 2026/09/20 12:49:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6899lb # [ 189.040205] niks3-server[1052]: 2026/09/20 12:49:00 INFO Signed narinfos id=67 count=16900worker2 # [ 189.046567] nix-grpc-daemon[1627]: time=2026-09-20T12:48:59.955Z level=INFO msg="Uploading 1 narinfos"6901lb # [ 189.043046] niks3-server[1052]: 2026/09/20 12:49:00 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6902lb # [ 189.046324] niks3-server[1052]: 2026/09/20 12:49:00 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls6903lb # [ 189.051141] niks3-server[1052]: 2026/09/20 12:49:00 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6904lb # [ 189.054182] niks3-server[1052]: 2026/09/20 12:49:00 INFO Completed upload id=676905worker2 # [ 189.060251] nix-grpc-daemon[1627]: time=2026-09-20T12:48:59.969Z level=INFO msg="Upload complete. (214ms)"6906lb # [ 189.056920] niks3-server[1052]: 2026/09/20 12:49:00 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo6907worker2 # [ 189.063279] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54514 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=17 outputs=16908worker2 # [ 189.069460] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:59Z level=debug event=want client=7 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=06909worker2 # [ 189.071383] nix-grpc-daemon[1551]: ts=2026-09-20T12:48:59Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=16910worker2 # [ 189.076202] nix-daemon[860]: reaped child process 1875, status = succeeded6911worker2 # [ 189.112502] nix-grpc-daemon[1551]: ts=2026-09-20T12:49:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545146912worker2 # [ 189.120084] nix-grpc-daemon[1551]: ts=2026-09-20T12:49:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:545146913worker2 # [ 189.129191] nix-grpc-daemon[1627]: time=2026-09-20T12:49:00.038Z level=INFO msg="All 2 paths already cached"6914client # copying 1 paths to 'lb:50051'...6915client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...6916worker2 # [ 189.136113] nix-grpc-daemon[1551]: ts=2026-09-20T12:49:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:545146917worker2 # [ 189.144148] nix-daemon[860]: accepted connection from pid 1551, user nix-grpc-daemon (trusted)6918worker2 # [ 189.291671] nix-daemon[860]: accepted connection from pid 1901, user nix-grpc-daemon (trusted)6919worker2 # [ 189.305369] nix-daemon[860]: reaped child process 1904, status = succeeded6920worker2 # [ 189.435793] nix-daemon[860]: accepted connection from pid 1906, user nix-grpc-daemon (trusted)6921worker2 # [ 189.448208] nix-daemon[860]: reaped child process 1909, status = succeeded6922lb # [ 189.449201] niks3-server[1052]: 2026/09/20 12:49:00 INFO Received uploads request method=POST path=/api/pending_closures6923worker2 # [ 189.459744] nix-grpc-daemon[1627]: time=2026-09-20T12:49:00.368Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6924worker2 # [ 189.461230] nix-grpc-daemon[1627]: time=2026-09-20T12:49:00.370Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"6925lb # [ 189.465775] niks3-server[1052]: 2026/09/20 12:49:00 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst6926lb # [ 189.468520] niks3-server[1052]: 2026/09/20 12:49:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6927lb # [ 189.470656] niks3-server[1052]: 2026/09/20 12:49:00 INFO Signed narinfos id=68 count=16928worker2 # [ 189.476607] nix-grpc-daemon[1627]: time=2026-09-20T12:49:00.385Z level=INFO msg="Uploading 1 narinfos"6929lb # [ 189.473474] niks3-server[1052]: 2026/09/20 12:49:00 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls6930lb # [ 189.478780] niks3-server[1052]: 2026/09/20 12:49:00 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6931lb # [ 189.481364] niks3-server[1052]: 2026/09/20 12:49:00 INFO Completed upload id=686932worker2 # [ 189.487450] nix-grpc-daemon[1627]: time=2026-09-20T12:49:00.396Z level=INFO msg="Upload complete. (338ms)"6933lb # [ 189.483851] niks3-server[1052]: 2026/09/20 12:49:00 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo6934worker2 # [ 189.489632] nix-grpc-daemon[1551]: ts=2026-09-20T12:49:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54514 duration_s=0 paths=1 nar_bytes_in=7526935worker2 # [ 189.492360] nix-grpc-daemon[1551]: ts=2026-09-20T12:49:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545146936worker2 # [ 189.499069] nix-daemon[860]: accepted connection from pid 1551, user nix-grpc-daemon (trusted)6937worker2 # [ 189.502340] nix-daemon[860]: reaped child process 1899, status = succeeded6938worker2 # [ 189.507404] nix-daemon[860]: accepted connection from pid 1551, user nix-grpc-daemon (trusted)6939client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...6940client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6941client # farm-top> LOG-farm-top-drain6942worker2 # [ 190.723422] nix-daemon[860]: reaped child process 1913, status = succeeded6943worker2 # [ 190.864284] nix-daemon[860]: accepted connection from pid 1928, user nix-grpc-daemon (trusted)6944worker2 # [ 190.877852] nix-daemon[860]: reaped child process 1931, status = succeeded6945lb # [ 190.887909] niks3-server[1052]: 2026/09/20 12:49:01 INFO Received uploads request method=POST path=/api/pending_closures6946worker2 # [ 190.903717] nix-grpc-daemon[1627]: time=2026-09-20T12:49:01.811Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6947worker2 # [ 190.905184] nix-grpc-daemon[1627]: time=2026-09-20T12:49:01.811Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"6948lb # [ 190.924776] niks3-server[1052]: 2026/09/20 12:49:02 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6949lb # [ 190.930868] niks3-server[1052]: 2026/09/20 12:49:02 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst6950lb # [ 190.941545] niks3-server[1052]: 2026/09/20 12:49:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6951lb # [ 190.947415] niks3-server[1052]: 2026/09/20 12:49:02 INFO Signed narinfos id=69 count=16952worker2 # [ 190.955983] nix-grpc-daemon[1627]: time=2026-09-20T12:49:01.864Z level=INFO msg="Uploading 1 narinfos"6953lb # [ 190.951204] niks3-server[1052]: 2026/09/20 12:49:02 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls6954lb # [ 190.960331] niks3-server[1052]: 2026/09/20 12:49:02 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6955lb # [ 190.963784] niks3-server[1052]: 2026/09/20 12:49:02 INFO Completed upload id=696956worker2 # [ 190.969910] nix-grpc-daemon[1627]: time=2026-09-20T12:49:01.879Z level=INFO msg="Upload complete. (255ms)"6957lb # [ 190.965648] niks3-server[1052]: 2026/09/20 12:49:02 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo6958worker2 # [ 190.977170] nix-grpc-daemon[1551]: ts=2026-09-20T12:49:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54514 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=18 outputs=16959worker2 # [ 190.987807] nix-daemon[860]: reaped child process 1911, status = succeeded6960client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&2, in 6.38 seconds)6961client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath6962client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath, in 0.19 seconds)6963worker1: must succeed: rm /nix/.rw-store/fill6964worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.13 seconds)6965worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6966worker1 # [ 191.496638] nix-grpc-daemon[1583]: ts=2026-09-20T12:49:01Z level=info event=healthy reason=min_free6967worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.12 seconds)6968(finished: subtest: low disk drains a worker and builds go to the other, in 8.96 seconds)6969(finished: run the VM test script, in 193.44 seconds)6970test script finished in 193.65s6971cleanup6972kill QemuMachine (pid 49)6973client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6974kill QemuMachine (pid 50)6975lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6976kill QemuMachine (pid 51)6977worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6978kill QemuMachine (pid 52)6979worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6980(finished: cleanup, in 0.98 seconds)6981additionally exposed symbols:6982 client, lb, worker1, worker2,6983 vlan1,6984 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh