vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #461
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12lb # Disk image does not exist, creating the virtualisation disk image...13lb # Formatting '/build/vm-state-lb/tmp.hmdGJAXPCZ', fmt=raw size=107374182414lb # mke2fs 1.47.4 (6-Mar-2025)15lb # Discarding device blocks: 0/262144 done16lb # Creating filesystem with 262144 4k blocks and 65536 inodes17lb: QEMU running (pid 50)18lb # Filesystem UUID: 8034f23f-92c6-439b-9e98-2ed282ad632819lb # Superblock backups stored on blocks:20lb # 32768, 98304, 163840, 22937621lb # 22lb # Allocating group tables: 0/8 done23lb # Writing inode tables: 0/8 done24lb # Creating journal (8192 blocks): done25client # Disk image does not exist, creating the virtualisation disk image...26lb # Writing superblocks and filesystem accounting information: 0/8 done27client # Formatting '/build/vm-state-client/tmp.tdU2PEqY5F', fmt=raw size=107374182428worker2 # Disk image does not exist, creating the virtualisation disk image...29lb # 30client # mke2fs 1.47.4 (6-Mar-2025)31worker2 # Formatting '/build/vm-state-worker2/tmp.Sr8M9mBFFE', fmt=raw size=107374182432lb # Virtualisation disk image created.33client # Discarding device blocks: 0/262144 done34worker2 # mke2fs 1.47.4 (6-Mar-2025)35client: QEMU running (pid 49)36worker2: QEMU running (pid 52)37client # Creating filesystem with 262144 4k blocks and 65536 inodes38worker2 # Discarding device blocks: 0/262144 done39client # Filesystem UUID: 4843435d-512a-4262-9b96-59c7cec2b9d340worker1: QEMU running (pid 51)41worker1 # Disk image does not exist, creating the virtualisation disk image...42worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes43worker1 # Formatting '/build/vm-state-worker1/tmp.S34kmvDf8P', fmt=raw size=107374182444worker2 # Filesystem UUID: 8d9a1b2f-7222-4d63-a6fc-5891cd56e1d545worker1 # mke2fs 1.47.4 (6-Mar-2025)46client # Superblock backups stored on blocks:47(finished: start all VMs, in 0.23 seconds)48worker2 # Superblock backups stored on blocks:49worker1 # Discarding device blocks: 0/262144 done50client # 32768, 98304, 163840, 22937651client: waiting for unit network-addresses-eth1.service52worker2 # 32768, 98304, 163840, 22937653worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes54client # 55client: waiting for the VM to finish booting56worker2 # 57worker1 # Filesystem UUID: 4ebca16a-f079-42f9-b75a-dae294b086c058worker2 # Allocating group tables: 0/8 done59worker1 # Superblock backups stored on blocks:60worker2 # Writing inode tables: 0/8 done61worker1 # 32768, 98304, 163840, 22937662worker2 # Creating journal (8192 blocks): done63worker1 # 64worker2 # Writing superblocks and filesystem accounting information: 0/8 done65worker1 # Allocating group tables: 0/8 done66worker2 # 67worker1 # Writing inode tables: 0/8 done68worker2 # Virtualisation disk image created.69worker1 # Creating journal (8192 blocks): done70client # Allocating group tables: 0/8 done71worker1 # Writing superblocks and filesystem accounting information: 0/8 done72client # Writing inode tables: 0/8 done73worker1 # 74client # Creating journal (8192 blocks): done75worker1 # Virtualisation disk image created.76client # Writing superblocks and filesystem accounting information: 0/8 done77client # 78client # Virtualisation disk image created.79lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)80worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)81worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)82client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)83lb # 84lb # 85worker1 # 86worker1 # 87worker2 # 88lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0089worker2 # 90worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0091worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0092lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...93lb # 94lb # 95lb # 96lb # 97worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...98worker1 # 99worker1 # 100worker1 # 101worker1 # 102worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...103worker2 # 104worker2 # 105worker2 # 106lb # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00107worker1 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00108worker2 # 109worker2 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00110client # 111client # 112worker1 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...113worker1 # 114worker1 # 115lb # Press Ctrl-B to configure iPXE (PCI 00:08.0)...116lb # 117lb # 118client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00119worker2 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...120worker2 # 121worker2 # 122client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...123client # 124client # 125client # 126client # 127client # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00128client # Press Ctrl-B to configure iPXE (PCI 00:08.0)...129client # 130client # 131worker1 # Booting from ROM...132lb # Booting from ROM...133worker2 # Booting from ROM...134client # Booting from ROM...135worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026136worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/fnfbcvqql2xcyc51k8r10rs3v3ib96pz-nixos-system-worker1-test/init regInfo=/nix/store/vnl2mh93z06amjxnmszsz1l1da1alz7a-closure-info/registration console=ttyS0,115200n8 console=tty0137worker1 # [ 0.000000] BIOS-provided physical RAM map:138worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable139worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved140worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved141worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable142worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved143worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved144worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved145worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved146worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved147worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved148client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026149worker1 # [ 0.000000] NX (Execute Disable) protection: active150worker1 # [ 0.000000] APIC: Static calls initialized151worker1 # [ 0.000000] SMBIOS 2.8 present.152worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014153worker1 # [ 0.000000] DMI: Memory slots populated: 1/1154worker1 # [ 0.000000] Hypervisor detected: KVM155client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/nnbvhsmcqrm6cxydqr22aq61iwmb9zhz-nixos-system-client-test/init regInfo=/nix/store/pw255kh8y8mmc1mgwwf78whzqmn3csfd-closure-info/registration console=ttyS0,115200n8 console=tty0156worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000157client # [ 0.000000] BIOS-provided physical RAM map:158worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00159client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable160worker1 # [ 0.000000] kvm-clock: using sched offset of 458219899 cycles161client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved162worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns163client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved164worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor165worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026166client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable167worker1 # [ 0.000810] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000168client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved169lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026170worker1 # [ 0.000837] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs171client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved172worker1 # [ 0.000839] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT173client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved174worker1 # [ 0.002728] found SMP MP-table at [mem 0x000f5450-0x000f545f]175worker1 # [ 0.002740] Using GB pages for direct mapping176client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved177worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/p8vpdp57y1j7m0hjvw8iyzfl88ilbjqv-nixos-system-worker2-test/init regInfo=/nix/store/7b1kb6gx1wfajv8rghh7szk1radgk2zq-closure-info/registration console=ttyS0,115200n8 console=tty0178worker1 # [ 0.002910] RAMDISK: [mem 0x5e367000-0x5ffcffff]179worker2 # [ 0.000000] BIOS-provided physical RAM map:180client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved181worker1 # [ 0.002917] ACPI: Early table checksum verification disabled182lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/m097agrx0jinhy67cn4afbshq893g59l-nixos-system-lb-test/init regInfo=/nix/store/6ryicbywj6r050v0hx7xqbv6gfi18m0f-closure-info/registration console=ttyS0,115200n8 console=tty0183worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable184client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved185lb # [ 0.000000] BIOS-provided physical RAM map:186worker1 # [ 0.002920] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )187worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved188client # [ 0.000000] NX (Execute Disable) protection: active189lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable190worker1 # [ 0.002924] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)191client # [ 0.000000] APIC: Static calls initialized192worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved193client # [ 0.000000] SMBIOS 2.8 present.194lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved195worker1 # [ 0.002928] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)196worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable197lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved198client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014199worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved200worker1 # [ 0.002935] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)201client # [ 0.000000] DMI: Memory slots populated: 1/1202lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable203worker1 # [ 0.002937] ACPI: FACS 0x000000005FFE0000 000040204client # [ 0.000000] Hypervisor detected: KVM205worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved206lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved207client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000208worker1 # [ 0.002939] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)209worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved210lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved211client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00212worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved213worker1 # [ 0.002940] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)214lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved215client # [ 0.000000] kvm-clock: using sched offset of 475828491 cycles216worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved217worker1 # [ 0.002942] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)218lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved219client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns220worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved221client # [ 0.000005] tsc: Detected 2400.012 MHz processor222lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved223worker1 # [ 0.002943] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)224worker2 # [ 0.000000] NX (Execute Disable) protection: active225client # [ 0.000815] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000226worker2 # [ 0.000000] APIC: Static calls initialized227lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved228worker2 # [ 0.000000] SMBIOS 2.8 present.229worker1 # [ 0.002944] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]230lb # [ 0.000000] NX (Execute Disable) protection: active231client # [ 0.000842] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs232lb # [ 0.000000] APIC: Static calls initialized233worker1 # [ 0.002946] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]234lb # [ 0.000000] SMBIOS 2.8 present.235client # [ 0.000845] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT236worker1 # [ 0.002946] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]237worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014238client # [ 0.002738] found SMP MP-table at [mem 0x000f5450-0x000f545f]239worker2 # [ 0.000000] DMI: Memory slots populated: 1/1240worker2 # [ 0.000000] Hypervisor detected: KVM241lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014242worker1 # [ 0.002947] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]243client # [ 0.002750] Using GB pages for direct mapping244worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000245lb # [ 0.000000] DMI: Memory slots populated: 1/1246client # [ 0.002849] RAMDISK: [mem 0x5e366000-0x5ffcffff]247lb # [ 0.000000] Hypervisor detected: KVM248worker1 # [ 0.002947] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]249worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00250client # [ 0.002856] ACPI: Early table checksum verification disabled251lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000252worker2 # [ 0.000000] kvm-clock: using sched offset of 470692826 cycles253worker1 # [ 0.002948] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]254client # [ 0.002859] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )255lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00256worker1 # [ 0.002948] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]257lb # [ 0.000001] kvm-clock: using sched offset of 491476352 cycles258worker1 # [ 0.003172] No NUMA configuration found259worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns260client # [ 0.002863] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)261worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor262worker1 # [ 0.003173] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]263client # [ 0.002867] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)264lb # [ 0.000003] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns265worker2 # [ 0.000816] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000266worker1 # [ 0.003175] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]267lb # [ 0.000007] tsc: Detected 2400.012 MHz processor268worker1 # [ 0.005295] Zone ranges:269client # [ 0.002874] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)270worker2 # [ 0.000843] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs271lb # [ 0.000984] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000272worker1 # [ 0.005296] DMA [mem 0x0000000000001000-0x0000000000ffffff]273client # [ 0.002875] ACPI: FACS 0x000000005FFE0000 000040274worker2 # [ 0.000846] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT275worker1 # [ 0.005297] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]276lb # [ 0.001025] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs277client # [ 0.002877] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)278worker2 # [ 0.002741] found SMP MP-table at [mem 0x000f5450-0x000f545f]279worker1 # [ 0.005299] Normal empty280worker1 # [ 0.005299] Device empty281lb # [ 0.001029] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT282worker2 # [ 0.002753] Using GB pages for direct mapping283worker1 # [ 0.005300] Movable zone start for each node284client # [ 0.002878] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)285worker2 # [ 0.002876] RAMDISK: [mem 0x5e367000-0x5ffcffff]286lb # [ 0.003052] found SMP MP-table at [mem 0x000f5450-0x000f545f]287worker1 # [ 0.005301] Early memory node ranges288lb # [ 0.003075] Using GB pages for direct mapping289worker2 # [ 0.002886] ACPI: Early table checksum verification disabled290worker1 # [ 0.005301] node 0: [mem 0x0000000000001000-0x000000000009efff]291client # [ 0.002880] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)292lb # [ 0.003168] RAMDISK: [mem 0x5e367000-0x5ffcffff]293worker2 # [ 0.002888] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )294worker1 # [ 0.005302] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]295lb # [ 0.003180] ACPI: Early table checksum verification disabled296client # [ 0.002882] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)297worker2 # [ 0.002893] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)298worker1 # [ 0.005303] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]299lb # [ 0.003184] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )300client # [ 0.002883] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]301worker1 # [ 0.005321] On node 0, zone DMA: 1 pages in unavailable ranges302worker2 # [ 0.002897] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)303lb # [ 0.003189] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)304client # [ 0.002884] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]305worker1 # [ 0.005572] On node 0, zone DMA: 97 pages in unavailable ranges306worker2 # [ 0.002904] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)307client # [ 0.002884] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]308lb # [ 0.003194] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)309worker1 # [ 0.031283] On node 0, zone DMA32: 40 pages in unavailable ranges310worker2 # [ 0.002906] ACPI: FACS 0x000000005FFE0000 000040311worker1 # [ 0.031729] ACPI: PM-Timer IO Port: 0x608312client # [ 0.002885] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]313lb # [ 0.003202] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)314worker1 # [ 0.031740] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])315worker2 # [ 0.002908] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)316client # [ 0.002885] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]317lb # [ 0.003204] ACPI: FACS 0x000000005FFE0000 000040318worker1 # [ 0.031763] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23319worker2 # [ 0.002909] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)320client # [ 0.002886] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]321lb # [ 0.003205] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)322worker1 # [ 0.031766] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)323client # [ 0.002886] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]324worker2 # [ 0.002911] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)325worker1 # [ 0.031768] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)326lb # [ 0.003207] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)327client # [ 0.003106] No NUMA configuration found328worker1 # [ 0.031769] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)329worker2 # [ 0.002912] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)330lb # [ 0.003209] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)331client # [ 0.003107] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]332worker1 # [ 0.031770] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)333worker2 # [ 0.002914] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]334client # [ 0.003109] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]335lb # [ 0.003210] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)336client # [ 0.005320] Zone ranges:337worker1 # [ 0.031770] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)338worker2 # [ 0.002915] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]339client # [ 0.005320] DMA [mem 0x0000000000001000-0x0000000000ffffff]340lb # [ 0.003211] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]341worker1 # [ 0.031773] ACPI: Using ACPI (MADT) for SMP configuration information342worker2 # [ 0.002915] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]343client # [ 0.005322] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]344lb # [ 0.003213] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]345client # [ 0.005323] Normal empty346worker1 # [ 0.031774] ACPI: HPET id: 0x8086a201 base: 0xfed00000347client # [ 0.005324] Device empty348worker2 # [ 0.002916] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]349worker1 # [ 0.031777] TSC deadline timer available350lb # [ 0.003213] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]351client # [ 0.005325] Movable zone start for each node352worker1 # [ 0.031781] CPU topo: Max. logical packages: 1353worker2 # [ 0.002916] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]354client # [ 0.005325] Early memory node ranges355lb # [ 0.003214] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]356worker1 # [ 0.031782] CPU topo: Max. logical dies: 1357client # [ 0.005326] node 0: [mem 0x0000000000001000-0x000000000009efff]358worker1 # [ 0.031782] CPU topo: Max. dies per package: 1359worker2 # [ 0.002917] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]360lb # [ 0.003214] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]361worker1 # [ 0.031792] CPU topo: Max. threads per core: 1362client # [ 0.005327] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]363worker2 # [ 0.002917] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]364worker1 # [ 0.031793] CPU topo: Num. cores per package: 1365lb # [ 0.003215] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]366worker2 # [ 0.003153] No NUMA configuration found367client # [ 0.005328] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]368worker1 # [ 0.031794] CPU topo: Num. threads per package: 1369lb # [ 0.003215] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]370worker2 # [ 0.003154] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]371client # [ 0.005351] On node 0, zone DMA: 1 pages in unavailable ranges372lb # [ 0.003468] No NUMA configuration found373worker1 # [ 0.031794] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs374worker2 # [ 0.003156] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]375client # [ 0.005605] On node 0, zone DMA: 97 pages in unavailable ranges376worker2 # [ 0.005432] Zone ranges:377lb # [ 0.003469] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]378worker1 # [ 0.031813] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()379client # [ 0.031445] On node 0, zone DMA32: 40 pages in unavailable ranges380worker2 # [ 0.005433] DMA [mem 0x0000000000001000-0x0000000000ffffff]381lb # [ 0.003472] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]382client # [ 0.031895] ACPI: PM-Timer IO Port: 0x608383lb # [ 0.005755] Zone ranges:384worker1 # [ 0.031843] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]385worker2 # [ 0.005434] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]386client # [ 0.031906] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])387lb # [ 0.005756] DMA [mem 0x0000000000001000-0x0000000000ffffff]388worker1 # [ 0.031845] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]389worker2 # [ 0.005436] Normal empty390worker2 # [ 0.005436] Device empty391client # [ 0.031932] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23392lb # [ 0.005757] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]393worker2 # [ 0.005437] Movable zone start for each node394worker1 # [ 0.031846] [mem 0x60000000-0xafffffff] available for PCI devices395lb # [ 0.005758] Normal empty396worker2 # [ 0.005438] Early memory node ranges397worker1 # [ 0.031848] Booting paravirtualized kernel on KVM398lb # [ 0.005759] Device empty399client # [ 0.031934] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)400lb # [ 0.005760] Movable zone start for each node401worker2 # [ 0.005438] node 0: [mem 0x0000000000001000-0x000000000009efff]402client # [ 0.031936] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)403lb # [ 0.005761] Early memory node ranges404worker1 # [ 0.031850] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns405worker2 # [ 0.005439] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]406lb # [ 0.005761] node 0: [mem 0x0000000000001000-0x000000000009efff]407client # [ 0.031937] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)408worker2 # [ 0.005440] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]409worker1 # [ 0.036323] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1410lb # [ 0.005762] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]411client # [ 0.031938] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)412worker2 # [ 0.005460] On node 0, zone DMA: 1 pages in unavailable ranges413worker1 # [ 0.038436] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152414lb # [ 0.005764] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]415client # [ 0.031938] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)416worker2 # [ 0.005731] On node 0, zone DMA: 97 pages in unavailable ranges417worker1 # [ 0.038476] kvm-guest: PV spinlocks disabled, single CPU418lb # [ 0.005784] On node 0, zone DMA: 1 pages in unavailable ranges419client # [ 0.031941] ACPI: Using ACPI (MADT) for SMP configuration information420worker2 # [ 0.033984] On node 0, zone DMA32: 40 pages in unavailable ranges421lb # [ 0.006051] On node 0, zone DMA: 97 pages in unavailable ranges422worker2 # [ 0.034527] ACPI: PM-Timer IO Port: 0x608423client # [ 0.031942] ACPI: HPET id: 0x8086a201 base: 0xfed00000424client # [ 0.031945] TSC deadline timer available425lb # [ 0.033651] On node 0, zone DMA32: 40 pages in unavailable ranges426worker2 # [ 0.034547] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])427client # [ 0.031949] CPU topo: Max. logical packages: 1428lb # [ 0.034146] ACPI: PM-Timer IO Port: 0x608429worker2 # [ 0.034587] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23430client # [ 0.031950] CPU topo: Max. logical dies: 1431lb # [ 0.034165] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])432client # [ 0.031950] CPU topo: Max. dies per package: 1433worker2 # [ 0.034591] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)434client # [ 0.031953] CPU topo: Max. threads per core: 1435lb # [ 0.034196] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23436worker1 # [ 0.038477] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/fnfbcvqql2xcyc51k8r10rs3v3ib96pz-nixos-system-worker1-test/init regInfo=/nix/store/vnl2mh93z06amjxnmszsz1l1da1alz7a-closure-info/registration console=ttyS0,115200n8 console=tty0437worker2 # [ 0.034593] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)438client # [ 0.031954] CPU topo: Num. cores per package: 1439lb # [ 0.034199] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)440client # [ 0.031955] CPU topo: Num. threads per package: 1441worker2 # [ 0.034596] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)442lb # [ 0.034202] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)443worker1 # [ 0.038569] Unknown kernel command line parameters "regInfo=/nix/store/vnl2mh93z06amjxnmszsz1l1da1alz7a-closure-info/registration", will be passed to user space.444client # [ 0.031955] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs445worker1 # [ 0.038582] random: crng init done446worker2 # [ 0.034597] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)447lb # [ 0.034204] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)448client # [ 0.031973] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()449worker2 # [ 0.034598] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)450worker1 # [ 0.038583] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes451lb # [ 0.034205] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)452client # [ 0.032008] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]453worker2 # [ 0.034602] ACPI: Using ACPI (MADT) for SMP configuration information454worker1 # [ 0.040666] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)455lb # [ 0.034206] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)456worker2 # [ 0.034603] ACPI: HPET id: 0x8086a201 base: 0xfed00000457client # [ 0.032010] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]458lb # [ 0.034209] ACPI: Using ACPI (MADT) for SMP configuration information459worker2 # [ 0.034613] TSC deadline timer available460worker1 # [ 0.041874] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)461client # [ 0.032011] [mem 0x60000000-0xafffffff] available for PCI devices462worker1 # [ 0.041906] Fallback order for Node 0: 0463worker2 # [ 0.034618] CPU topo: Max. logical packages: 1464lb # [ 0.034210] ACPI: HPET id: 0x8086a201 base: 0xfed00000465client # [ 0.032013] Booting paravirtualized kernel on KVM466worker2 # [ 0.034619] CPU topo: Max. logical dies: 1467lb # [ 0.034219] TSC deadline timer available468worker1 # [ 0.041909] Built 1 zonelists, mobility grouping on. Total pages: 393078469lb # [ 0.034224] CPU topo: Max. logical packages: 1470worker2 # [ 0.034620] CPU topo: Max. dies per package: 1471worker1 # [ 0.041910] Policy zone: DMA32472lb # [ 0.034225] CPU topo: Max. logical dies: 1473worker2 # [ 0.034625] CPU topo: Max. threads per core: 1474client # [ 0.032016] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns475worker1 # [ 0.044324] mem auto-init: stack:all(zero), heap alloc:on, heap free:off476lb # [ 0.034226] CPU topo: Max. dies per package: 1477worker2 # [ 0.034626] CPU topo: Num. cores per package: 1478client # [ 0.036464] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1479lb # [ 0.034230] CPU topo: Max. threads per core: 1480worker1 # [ 0.047396] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1481worker2 # [ 0.034626] CPU topo: Num. threads per package: 1482lb # [ 0.034231] CPU topo: Num. cores per package: 1483worker1 # [ 0.050604] allocated 3145728 bytes of page_ext484client # [ 0.038570] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152485worker2 # [ 0.034626] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs486lb # [ 0.034231] CPU topo: Num. threads per package: 1487worker1 # [ 0.060678] ftrace: allocating 48731 entries in 192 pages488client # [ 0.038614] kvm-guest: PV spinlocks disabled, single CPU489worker2 # [ 0.034661] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()490worker1 # [ 0.060680] ftrace: allocated 192 pages with 2 groups491lb # [ 0.034231] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs492worker1 # [ 0.061521] Dynamic Preempt: lazy493worker2 # [ 0.034711] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]494lb # [ 0.034265] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()495worker1 # [ 0.061699] rcu: Preemptible hierarchical RCU implementation.496worker2 # [ 0.034713] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]497worker1 # [ 0.061699] rcu: RCU event tracing is enabled.498lb # [ 0.034315] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]499worker2 # [ 0.034715] [mem 0x60000000-0xafffffff] available for PCI devices500worker1 # [ 0.061700] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.501lb # [ 0.034316] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]502worker2 # [ 0.034717] Booting paravirtualized kernel on KVM503worker1 # [ 0.061702] Trampoline variant of Tasks RCU enabled.504client # [ 0.038615] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/nnbvhsmcqrm6cxydqr22aq61iwmb9zhz-nixos-system-client-test/init regInfo=/nix/store/pw255kh8y8mmc1mgwwf78whzqmn3csfd-closure-info/registration console=ttyS0,115200n8 console=tty0505lb # [ 0.034318] [mem 0x60000000-0xafffffff] available for PCI devices506worker1 # [ 0.061703] Rude variant of Tasks RCU enabled.507lb # [ 0.034321] Booting paravirtualized kernel on KVM508worker2 # [ 0.034722] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns509worker1 # [ 0.061703] Tracing variant of Tasks RCU enabled.510client # [ 0.038711] Unknown kernel command line parameters "regInfo=/nix/store/pw255kh8y8mmc1mgwwf78whzqmn3csfd-closure-info/registration", will be passed to user space.511worker2 # [ 0.039262] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1512lb # [ 0.034327] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns513worker1 # [ 0.061704] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.514client # [ 0.038724] random: crng init done515worker2 # [ 0.041372] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152516worker1 # [ 0.061704] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1517lb # [ 0.038850] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1518client # [ 0.038725] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes519worker2 # [ 0.041425] kvm-guest: PV spinlocks disabled, single CPU520lb # [ 0.041087] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152521worker1 # [ 0.061713] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.522client # [ 0.040800] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)523lb # [ 0.041132] kvm-guest: PV spinlocks disabled, single CPU524worker1 # [ 0.061715] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.525client # [ 0.042004] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)526client # [ 0.042036] Fallback order for Node 0: 0527worker1 # [ 0.061716] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.528client # [ 0.042038] Built 1 zonelists, mobility grouping on. Total pages: 393078529client # [ 0.042039] Policy zone: DMA32530worker1 # [ 0.066222] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16531worker2 # [ 0.041428] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/p8vpdp57y1j7m0hjvw8iyzfl88ilbjqv-nixos-system-worker2-test/init regInfo=/nix/store/7b1kb6gx1wfajv8rghh7szk1radgk2zq-closure-info/registration console=ttyS0,115200n8 console=tty0532client # [ 0.044469] mem auto-init: stack:all(zero), heap alloc:on, heap free:off533worker1 # [ 0.066479] rcu: srcu_init: Setting srcu_struct sizes based on contention.534lb # [ 0.041134] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/m097agrx0jinhy67cn4afbshq893g59l-nixos-system-lb-test/init regInfo=/nix/store/6ryicbywj6r050v0hx7xqbv6gfi18m0f-closure-info/registration console=ttyS0,115200n8 console=tty0535client # [ 0.047554] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1536worker2 # [ 0.041546] Unknown kernel command line parameters "regInfo=/nix/store/7b1kb6gx1wfajv8rghh7szk1radgk2zq-closure-info/registration", will be passed to user space.537client # [ 0.050739] allocated 3145728 bytes of page_ext538worker1 # [ 0.066485] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns539worker2 # [ 0.041564] random: crng init done540client # [ 0.060563] ftrace: allocating 48731 entries in 192 pages541lb # [ 0.041254] Unknown kernel command line parameters "regInfo=/nix/store/6ryicbywj6r050v0hx7xqbv6gfi18m0f-closure-info/registration", will be passed to user space.542worker1 # [ 0.066588] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)543client # [ 0.060565] ftrace: allocated 192 pages with 2 groups544worker2 # [ 0.041565] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes545lb # [ 0.041275] random: crng init done546worker1 # [ 0.070109] Console: colour VGA+ 80x25547client # [ 0.061386] Dynamic Preempt: lazy548worker1 # [ 0.070111] printk: legacy console [tty0] enabled549worker2 # [ 0.044006] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)550lb # [ 0.041276] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes551client # [ 0.061539] rcu: Preemptible hierarchical RCU implementation.552worker1 # [ 0.099594] printk: legacy console [ttyS0] enabled553client # [ 0.061539] rcu: RCU event tracing is enabled.554lb # [ 0.043535] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)555worker2 # [ 0.045148] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)556worker1 # [ 0.196493] ACPI: Core revision 20250807557worker2 # [ 0.045203] Fallback order for Node 0: 0558client # [ 0.061540] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.559lb # [ 0.044730] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)560client # [ 0.061541] Trampoline variant of Tasks RCU enabled.561worker2 # [ 0.045207] Built 1 zonelists, mobility grouping on. Total pages: 393078562lb # [ 0.044770] Fallback order for Node 0: 0563worker2 # [ 0.045209] Policy zone: DMA32564client # [ 0.061542] Rude variant of Tasks RCU enabled.565worker1 # [ 0.197344] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns566lb # [ 0.044774] Built 1 zonelists, mobility grouping on. Total pages: 393078567client # [ 0.061542] Tracing variant of Tasks RCU enabled.568worker2 # [ 0.047678] mem auto-init: stack:all(zero), heap alloc:on, heap free:off569lb # [ 0.044775] Policy zone: DMA32570worker1 # [ 0.198829] APIC: Switch to symmetric I/O mode setup571worker2 # [ 0.052100] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1572client # [ 0.061543] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.573lb # [ 0.047332] mem auto-init: stack:all(zero), heap alloc:on, heap free:off574worker1 # [ 0.199754] x2apic enabled575worker2 # [ 0.055576] allocated 3145728 bytes of page_ext576client # [ 0.061543] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1577lb # [ 0.051178] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1578worker2 # [ 0.069879] ftrace: allocating 48731 entries in 192 pages579worker1 # [ 0.200489] APIC: Switched APIC routing to: physical x2apic580lb # [ 0.054624] allocated 3145728 bytes of page_ext581worker2 # [ 0.069883] ftrace: allocated 192 pages with 2 groups582client # [ 0.061558] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.583lb # [ 0.068630] ftrace: allocating 48731 entries in 192 pages584worker2 # [ 0.070913] Dynamic Preempt: lazy585lb # [ 0.068633] ftrace: allocated 192 pages with 2 groups586worker2 # [ 0.071060] rcu: Preemptible hierarchical RCU implementation.587client # [ 0.061559] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.588lb # [ 0.069603] Dynamic Preempt: lazy589worker1 # [ 0.202306] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1590worker2 # [ 0.071061] rcu: RCU event tracing is enabled.591lb # [ 0.069742] rcu: Preemptible hierarchical RCU implementation.592client # [ 0.061560] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.593worker2 # [ 0.071061] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.594lb # [ 0.069743] rcu: RCU event tracing is enabled.595client # [ 0.066025] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16596worker2 # [ 0.071063] Trampoline variant of Tasks RCU enabled.597worker1 # [ 0.203257] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns598lb # [ 0.069743] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.599worker2 # [ 0.071064] Rude variant of Tasks RCU enabled.600client # [ 0.066306] rcu: srcu_init: Setting srcu_struct sizes based on contention.601lb # [ 0.069746] Trampoline variant of Tasks RCU enabled.602worker2 # [ 0.071064] Tracing variant of Tasks RCU enabled.603worker1 # [ 0.204871] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)604lb # [ 0.069746] Rude variant of Tasks RCU enabled.605client # [ 0.066313] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns606worker2 # [ 0.071065] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.607lb # [ 0.069747] Tracing variant of Tasks RCU enabled.608worker1 # [ 0.205954] x86/cpu: User Mode Instruction Prevention (UMIP) activated609worker2 # [ 0.071066] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1610lb # [ 0.069748] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.611client # [ 0.066415] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)612client # [ 0.069933] Console: colour VGA+ 80x25613worker1 # [ 0.206988] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127614worker2 # [ 0.071116] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.615lb # [ 0.069749] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1616client # [ 0.069936] printk: legacy console [tty0] enabled617client # [ 0.099362] printk: legacy console [ttyS0] enabled618worker1 # [ 0.207869] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0619worker2 # [ 0.071118] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.620lb # [ 0.069765] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.621client # [ 0.195769] ACPI: Core revision 20250807622lb # [ 0.069768] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.623worker2 # [ 0.071120] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.624worker1 # [ 0.208872] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto625worker2 # [ 0.075780] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16626lb # [ 0.069769] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.627client # [ 0.196637] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns628worker2 # [ 0.076082] rcu: srcu_init: Setting srcu_struct sizes based on contention.629lb # [ 0.074509] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16630worker1 # [ 0.209869] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl631client # [ 0.198116] APIC: Switch to symmetric I/O mode setup632lb # [ 0.074800] rcu: srcu_init: Setting srcu_struct sizes based on contention.633client # [ 0.199074] x2apic enabled634worker2 # [ 0.076091] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns635worker1 # [ 0.210869] Transient Scheduler Attacks: Vulnerable: No microcode636lb # [ 0.074807] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns637client # [ 0.199804] APIC: Switched APIC routing to: physical x2apic638worker2 # [ 0.076209] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)639worker1 # [ 0.211868] Spectre V2 : Mitigation: Enhanced / Automatic IBRS640worker2 # [ 0.079779] Console: colour VGA+ 80x25641worker1 # [ 0.212868] Speculative Return Stack Overflow: Mitigation: Safe RET642worker2 # [ 0.079784] printk: legacy console [tty0] enabled643lb # [ 0.074922] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)644lb # [ 0.078455] Console: colour VGA+ 80x25645worker2 # [ 0.109196] printk: legacy console [ttyS0] enabled646client # [ 0.201569] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1647lb # [ 0.078459] printk: legacy console [tty0] enabled648worker2 # [ 0.204633] ACPI: Core revision 20250807649lb # [ 0.107913] printk: legacy console [ttyS0] enabled650lb # [ 0.203325] ACPI: Core revision 20250807651client # [ 0.202574] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns652worker2 # [ 0.205543] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns653worker1 # [ 0.213868] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization654client # [ 0.204292] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)655worker2 # [ 0.207142] APIC: Switch to symmetric I/O mode setup656lb # [ 0.204218] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns657worker1 # [ 0.215875] Spectre V2 : Enabling IBPB for BPF658worker2 # [ 0.208079] x2apic enabled659lb # [ 0.205708] APIC: Switch to symmetric I/O mode setup660client # [ 0.205376] x86/cpu: User Mode Instruction Prevention (UMIP) activated661worker2 # [ 0.208862] APIC: Switched APIC routing to: physical x2apic662lb # [ 0.206680] x2apic enabled663worker1 # [ 0.216584] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier664client # [ 0.206410] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127665worker1 # [ 0.217869] active return thunk: srso_alias_return_thunk666lb # [ 0.207473] APIC: Switched APIC routing to: physical x2apic667client # [ 0.207290] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0668worker2 # [ 0.210639] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1669worker1 # [ 0.218889] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'670lb # [ 0.209241] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1671client # [ 0.208294] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto672worker1 # [ 0.219868] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'673worker2 # [ 0.211669] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns674worker1 # [ 0.220868] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'675client # [ 0.210290] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl676lb # [ 0.210203] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns677worker2 # [ 0.213299] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)678worker1 # [ 0.221868] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'679client # [ 0.211290] Transient Scheduler Attacks: Vulnerable: No microcode680lb # [ 0.211788] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)681worker2 # [ 0.214386] x86/cpu: User Mode Instruction Prevention (UMIP) activated682worker1 # [ 0.222868] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'683client # [ 0.212289] Spectre V2 : Mitigation: Enhanced / Automatic IBRS684lb # [ 0.212873] x86/cpu: User Mode Instruction Prevention (UMIP) activated685worker1 # [ 0.223868] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'686worker2 # [ 0.215430] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127687client # [ 0.213289] Speculative Return Stack Overflow: Mitigation: Safe RET688lb # [ 0.213931] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127689worker2 # [ 0.216295] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0690worker1 # [ 0.224868] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'691lb # [ 0.214784] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0692worker1 # [ 0.225869] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256693worker2 # [ 0.217299] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto694worker1 # [ 0.226868] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64695lb # [ 0.215789] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto696client # [ 0.214289] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization697worker2 # [ 0.219295] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl698worker1 # [ 0.227868] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512699client # [ 0.216297] Spectre V2 : Enabling IBPB for BPF700lb # [ 0.216784] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl701worker2 # [ 0.220295] Transient Scheduler Attacks: Vulnerable: No microcode702worker1 # [ 0.228868] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024703worker2 # [ 0.221295] Spectre V2 : Mitigation: Enhanced / Automatic IBRS704worker1 # [ 0.229868] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8705lb # [ 0.217784] Transient Scheduler Attacks: Vulnerable: No microcode706client # [ 0.217290] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier707lb # [ 0.218784] Spectre V2 : Mitigation: Enhanced / Automatic IBRS708worker2 # [ 0.222295] Speculative Return Stack Overflow: Mitigation: Safe RET709client # [ 0.218290] active return thunk: srso_alias_return_thunk710worker1 # [ 0.230868] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.711lb # [ 0.219784] Speculative Return Stack Overflow: Mitigation: Safe RET712client # [ 0.219311] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'713client # [ 0.220290] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'714client # [ 0.221290] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'715client # [ 0.222290] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'716worker2 # [ 0.223294] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization717worker2 # [ 0.225308] Spectre V2 : Enabling IBPB for BPF718client # [ 0.223289] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'719lb # [ 0.220786] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization720client # [ 0.224289] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'721lb # [ 0.222796] Spectre V2 : Enabling IBPB for BPF722worker2 # [ 0.226884] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier723client # [ 0.225289] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'724worker2 # [ 0.228297] active return thunk: srso_alias_return_thunk725lb # [ 0.223785] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier726client # [ 0.226290] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256727worker2 # [ 0.229318] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'728lb # [ 0.224786] active return thunk: srso_alias_return_thunk729client # [ 0.228013] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64730worker2 # [ 0.230295] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'731lb # [ 0.225807] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'732client # [ 0.229017] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512733worker2 # [ 0.231296] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'734client # [ 0.230031] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024735lb # [ 0.227728] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'736worker2 # [ 0.232295] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'737lb # [ 0.228784] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'738client # [ 0.231010] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8739worker2 # [ 0.233294] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'740lb # [ 0.229731] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'741client # [ 0.231290] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.742worker2 # [ 0.234295] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'743lb # [ 0.230784] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'744worker2 # [ 0.235295] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'745lb # [ 0.231784] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'746worker2 # [ 0.237296] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256747lb # [ 0.232784] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'748worker2 # [ 0.238295] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64749lb # [ 0.233784] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256750worker2 # [ 0.239295] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512751lb # [ 0.234784] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64752worker2 # [ 0.240294] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024753lb # [ 0.235784] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512754worker2 # [ 0.241294] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8755lb # [ 0.236783] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024756lb # [ 0.237784] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8757worker2 # [ 0.242294] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.758lb # [ 0.238784] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.759worker1 # [ 0.266068] Freeing SMP alternatives memory: 44K760worker1 # [ 0.266871] pid_max: default: 32768 minimum: 301761worker1 # [ 0.267695] LSM: initializing lsm=capability,landlock,yama,bpf,ima762worker1 # [ 0.268729] landlock: Up and running.763worker1 # [ 0.269328] Yama: becoming mindful.764worker1 # [ 0.270087] LSM support for eBPF active765worker1 # [ 0.270979] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)766worker1 # [ 0.271903] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)767worker1 # [ 0.274697] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)768worker1 # [ 0.275435] Performance Events: Fam17h+ core perfctr, AMD PMU driver.769worker1 # [ 0.275874] ... version: 2770worker1 # [ 0.276594] ... bit width: 48771worker1 # [ 0.276871] ... generic counters: 6772client # [ 0.266834] Freeing SMP alternatives memory: 44K773client # [ 0.267293] pid_max: default: 32768 minimum: 301774worker1 # [ 0.277597] ... generic bitmap: 000000000000003f775worker1 # [ 0.277882] ... fixed-purpose counters: 0776client # [ 0.268376] LSM: initializing lsm=capability,landlock,yama,bpf,ima777worker1 # [ 0.278591] ... fixed-purpose bitmap: 0000000000000000778client # [ 0.269399] landlock: Up and running.779client # [ 0.270290] Yama: becoming mindful.780worker1 # [ 0.278871] ... value mask: 0000ffffffffffff781client # [ 0.271092] LSM support for eBPF active782worker1 # [ 0.279763] ... max period: 00007fffffffffff783worker1 # [ 0.280545] ... global_ctrl mask: 000000000000003f784client # [ 0.271880] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)785worker1 # [ 0.280994] signal: max sigframe size: 3376786worker1 # [ 0.281805] rcu: Hierarchical SRCU implementation.787client # [ 0.272326] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)788worker1 # [ 0.282473] rcu: Max phase no-delay instances is 400.789client # [ 0.275688] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)790client # [ 0.276869] Performance Events: Fam17h+ core perfctr, AMD PMU driver.791client # [ 0.277299] ... version: 2792client # [ 0.278058] ... bit width: 48793worker1 # [ 0.287363] smp: Bringing up secondary CPUs ...794client # [ 0.278292] ... generic counters: 6795worker1 # [ 0.287886] smp: Brought up 1 node, 1 CPU796client # [ 0.278975] ... generic bitmap: 000000000000003f797worker1 # [ 0.288547] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)798client # [ 0.279292] ... fixed-purpose counters: 0799client # [ 0.280016] ... fixed-purpose bitmap: 0000000000000000800worker2 # [ 0.285434] Freeing SMP alternatives memory: 44K801worker1 # [ 0.289037] Memory: 1454568K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)802client # [ 0.280292] ... value mask: 0000ffffffffffff803worker2 # [ 0.286293] pid_max: default: 32768 minimum: 301804lb # [ 0.280643] Freeing SMP alternatives memory: 44K805worker1 # [ 0.290095] devtmpfs: initialized806lb # [ 0.281323] pid_max: default: 32768 minimum: 301807client # [ 0.281155] ... max period: 00007fffffffffff808worker2 # [ 0.286921] LSM: initializing lsm=capability,landlock,yama,bpf,ima809worker1 # [ 0.290856] x86/mm: Memory block size: 128MB810client # [ 0.281950] ... global_ctrl mask: 000000000000003f811lb # [ 0.281874] LSM: initializing lsm=capability,landlock,yama,bpf,ima812worker2 # [ 0.288124] landlock: Up and running.813worker2 # [ 0.288717] Yama: becoming mindful.814lb # [ 0.282906] landlock: Up and running.815client # [ 0.282388] signal: max sigframe size: 3376816lb # [ 0.283784] Yama: becoming mindful.817worker2 # [ 0.289556] LSM support for eBPF active818client # [ 0.283344] rcu: Hierarchical SRCU implementation.819worker1 # [ 0.292440] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)820lb # [ 0.284866] LSM support for eBPF active821client # [ 0.284077] rcu: Max phase no-delay instances is 400.822worker2 # [ 0.290408] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)823worker1 # [ 0.292898] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).824lb # [ 0.285610] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)825worker2 # [ 0.291329] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)826worker1 # [ 0.293956] pinctrl core: initialized pinctrl subsystem827lb # [ 0.286820] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)828worker1 # [ 0.295057] PM: RTC time: 21:08:13, date: 2026-09-20829worker2 # [ 0.293900] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)830lb # [ 0.289241] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)831client # [ 0.288784] smp: Bringing up secondary CPUs ...832worker2 # [ 0.294829] Performance Events: Fam17h+ core perfctr, AMD PMU driver.833client # [ 0.289307] smp: Brought up 1 node, 1 CPU834worker2 # [ 0.295300] ... version: 2835worker1 # [ 0.298531] NET: Registered PF_NETLINK/PF_ROUTE protocol family836lb # [ 0.290361] Performance Events: Fam17h+ core perfctr, AMD PMU driver.837client # [ 0.290008] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)838worker2 # [ 0.295987] ... bit width: 48839lb # [ 0.290795] ... version: 2840worker2 # [ 0.296298] ... generic counters: 6841lb # [ 0.291501] ... bit width: 48842worker1 # [ 0.299343] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations843worker2 # [ 0.296995] ... generic bitmap: 000000000000003f844lb # [ 0.291786] ... generic counters: 6845client # [ 0.290451] Memory: 1454572K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109348K reserved, 0K cma-reserved)846worker1 # [ 0.299891] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations847worker2 # [ 0.297310] ... fixed-purpose counters: 0848lb # [ 0.292471] ... generic bitmap: 000000000000003f849client # [ 0.291545] devtmpfs: initialized850lb # [ 0.292786] ... fixed-purpose counters: 0851worker2 # [ 0.298021] ... fixed-purpose bitmap: 0000000000000000852worker1 # [ 0.301135] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations853client # [ 0.292318] x86/mm: Memory block size: 128MB854worker2 # [ 0.298297] ... value mask: 0000ffffffffffff855lb # [ 0.293483] ... fixed-purpose bitmap: 0000000000000000856worker1 # [ 0.301881] audit: initializing netlink subsys (disabled)857lb # [ 0.293786] ... value mask: 0000ffffffffffff858worker2 # [ 0.299168] ... max period: 00007fffffffffff859worker1 # [ 0.302991] thermal_sys: Registered thermal governor 'fair_share'860worker2 # [ 0.299989] ... global_ctrl mask: 000000000000003f861client # [ 0.294088] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)862lb # [ 0.294677] ... max period: 00007fffffffffff863worker2 # [ 0.300427] signal: max sigframe size: 3376864lb # [ 0.295425] ... global_ctrl mask: 000000000000003f865worker1 # [ 0.302994] thermal_sys: Registered thermal governor 'bang_bang'866lb # [ 0.295901] signal: max sigframe size: 3376867client # [ 0.295140] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).868worker2 # [ 0.301249] rcu: Hierarchical SRCU implementation.869worker1 # [ 0.303872] thermal_sys: Registered thermal governor 'step_wise'870lb # [ 0.296658] rcu: Hierarchical SRCU implementation.871worker2 # [ 0.301846] rcu: Max phase no-delay instances is 400.872client # [ 0.296381] pinctrl core: initialized pinctrl subsystem873worker1 # [ 0.304808] thermal_sys: Registered thermal governor 'user_space'874lb # [ 0.297359] rcu: Max phase no-delay instances is 400.875client # [ 0.297474] PM: RTC time: 21:08:13, date: 2026-09-20876worker1 # [ 0.305600] audit: type=2000 audit(1789938493.836:1): state=initialized audit_enabled=0 res=1877worker1 # [ 0.306873] thermal_sys: Registered thermal governor 'power_allocator'878worker1 # [ 0.306887] cpuidle: using governor menu879client # [ 0.300984] NET: Registered PF_NETLINK/PF_ROUTE protocol family880worker1 # [ 0.309078] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5881worker2 # [ 0.307039] smp: Bringing up secondary CPUs ...882worker2 # [ 0.307316] smp: Brought up 1 node, 1 CPU883lb # [ 0.302577] smp: Bringing up secondary CPUs ...884client # [ 0.301799] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations885worker1 # [ 0.310091] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]886lb # [ 0.302805] smp: Brought up 1 node, 1 CPU887worker2 # [ 0.307970] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)888worker1 # [ 0.310874] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry889client # [ 0.302314] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations890lb # [ 0.303515] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)891worker1 # [ 0.311883] PCI: Using configuration type 1 for base access892client # [ 0.303567] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations893worker2 # [ 0.308474] Memory: 1454560K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109348K reserved, 0K cma-reserved)894worker1 # [ 0.312982] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.895client # [ 0.304302] audit: initializing netlink subsys (disabled)896worker2 # [ 0.309553] devtmpfs: initialized897lb # [ 0.303976] Memory: 1454544K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)898lb # [ 0.305037] devtmpfs: initialized899worker2 # [ 0.310371] x86/mm: Memory block size: 128MB900client # [ 0.305419] thermal_sys: Registered thermal governor 'fair_share'901lb # [ 0.305831] x86/mm: Memory block size: 128MB902client # [ 0.305421] thermal_sys: Registered thermal governor 'bang_bang'903client # [ 0.306293] thermal_sys: Registered thermal governor 'step_wise'904worker2 # [ 0.312119] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)905client # [ 0.307260] thermal_sys: Registered thermal governor 'user_space'906lb # [ 0.307578] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)907worker2 # [ 0.313233] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).908client # [ 0.308052] audit: type=2000 audit(1789938493.852:1): state=initialized audit_enabled=0 res=1909lb # [ 0.308677] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).910worker2 # [ 0.314418] pinctrl core: initialized pinctrl subsystem911client # [ 0.309295] thermal_sys: Registered thermal governor 'power_allocator'912lb # [ 0.309911] pinctrl core: initialized pinctrl subsystem913worker2 # [ 0.315503] PM: RTC time: 21:08:13, date: 2026-09-20914client # [ 0.309307] cpuidle: using governor menu915worker1 # [ 0.320099] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages916lb # [ 0.311043] PM: RTC time: 21:08:13, date: 2026-09-20917worker1 # [ 0.320872] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page918client # [ 0.311526] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5919worker2 # [ 0.318904] NET: Registered PF_NETLINK/PF_ROUTE protocol family920client # [ 0.312520] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]921lb # [ 0.314405] NET: Registered PF_NETLINK/PF_ROUTE protocol family922client # [ 0.313295] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry923worker2 # [ 0.319801] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations924client # [ 0.314304] PCI: Using configuration type 1 for base access925lb # [ 0.315278] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations926worker2 # [ 0.320324] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations927worker1 # [ 0.325872] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages928client # [ 0.315432] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.929lb # [ 0.315814] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations930worker1 # [ 0.326872] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page931worker2 # [ 0.321563] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations932worker2 # [ 0.322307] audit: initializing netlink subsys (disabled)933lb # [ 0.317181] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations934lb # [ 0.317797] audit: initializing netlink subsys (disabled)935worker2 # [ 0.323457] thermal_sys: Registered thermal governor 'fair_share'936worker2 # [ 0.323460] thermal_sys: Registered thermal governor 'bang_bang'937lb # [ 0.318977] thermal_sys: Registered thermal governor 'fair_share'938worker2 # [ 0.324298] thermal_sys: Registered thermal governor 'step_wise'939lb # [ 0.318979] thermal_sys: Registered thermal governor 'bang_bang'940client # [ 0.320418] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages941worker2 # [ 0.325288] thermal_sys: Registered thermal governor 'user_space'942lb # [ 0.319788] thermal_sys: Registered thermal governor 'step_wise'943client # [ 0.321294] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page944lb # [ 0.320735] thermal_sys: Registered thermal governor 'user_space'945worker2 # [ 0.326055] audit: type=2000 audit(1789938493.869:1): state=initialized audit_enabled=0 res=1946lb # [ 0.321528] audit: type=2000 audit(1789938493.867:1): state=initialized audit_enabled=0 res=1947worker2 # [ 0.327300] thermal_sys: Registered thermal governor 'power_allocator'948worker1 # [ 0.335535] ACPI: Added _OSI(Module Device)949worker2 # [ 0.327319] cpuidle: using governor menu950lb # [ 0.322789] thermal_sys: Registered thermal governor 'power_allocator'951worker1 # [ 0.335873] ACPI: Added _OSI(Processor Device)952lb # [ 0.322805] cpuidle: using governor menu953worker1 # [ 0.336585] ACPI: Added _OSI(Processor Aggregator Device)954worker2 # [ 0.329561] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5955client # [ 0.326293] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages956lb # [ 0.325032] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5957client # [ 0.327292] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page958worker2 # [ 0.330524] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]959lb # [ 0.326024] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]960worker2 # [ 0.331302] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry961worker2 # [ 0.332312] PCI: Using configuration type 1 for base access962lb # [ 0.326789] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry963lb # [ 0.327801] PCI: Using configuration type 1 for base access964worker2 # [ 0.333483] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.965lb # [ 0.328971] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.966worker1 # [ 0.343425] ACPI: 1 ACPI AML tables successfully acquired and loaded967worker1 # [ 0.346775] ACPI: Interpreter enabled968worker1 # [ 0.347347] ACPI: PM: (supports S0 S3 S4 S5)969client # [ 0.337633] ACPI: Added _OSI(Module Device)970lb # [ 0.334019] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages971client # [ 0.338294] ACPI: Added _OSI(Processor Device)972worker1 # [ 0.347871] ACPI: Using IOAPIC for interrupt routing973worker2 # [ 0.340545] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages974lb # [ 0.334788] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page975client # [ 0.338982] ACPI: Added _OSI(Processor Aggregator Device)976worker2 # [ 0.341298] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page977worker1 # [ 0.348718] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug978lb # [ 0.339788] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages979worker2 # [ 0.346298] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages980worker1 # [ 0.353871] PCI: Using E820 reservations for host bridge windows981lb # [ 0.340786] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page982worker1 # [ 0.354992] ACPI: Enabled 2 GPEs in block 00 to 3F983worker2 # [ 0.347298] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page984client # [ 0.347529] ACPI: 1 ACPI AML tables successfully acquired and loaded985client # [ 0.351690] ACPI: Interpreter enabled986client # [ 0.352307] ACPI: PM: (supports S0 S3 S4 S5)987client # [ 0.353011] ACPI: Using IOAPIC for interrupt routing988worker2 # [ 0.355930] ACPI: Added _OSI(Module Device)989worker1 # [ 0.363656] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])990lb # [ 0.351182] ACPI: Added _OSI(Module Device)991worker2 # [ 0.356299] ACPI: Added _OSI(Processor Device)992lb # [ 0.351788] ACPI: Added _OSI(Processor Device)993worker2 # [ 0.357087] ACPI: Added _OSI(Processor Aggregator Device)994lb # [ 0.352569] ACPI: Added _OSI(Processor Aggregator Device)995worker1 # [ 0.365186] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]996client # [ 0.355347] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug997worker1 # [ 0.365964] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]998client # [ 0.356293] PCI: Using E820 reservations for host bridge windows999worker1 # [ 0.366994] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1000worker1 # [ 0.368222] PCI host bridge to bus 0000:001001client # [ 0.359339] ACPI: Enabled 2 GPEs in block 00 to 3F1002worker1 # [ 0.368876] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1003worker1 # [ 0.369871] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1004worker2 # [ 0.363910] ACPI: 1 ACPI AML tables successfully acquired and loaded1005worker1 # [ 0.370871] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1006worker1 # [ 0.371871] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1007lb # [ 0.361330] ACPI: 1 ACPI AML tables successfully acquired and loaded1008worker1 # [ 0.372871] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1009worker2 # [ 0.367222] ACPI: Interpreter enabled1010worker1 # [ 0.373871] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1011worker2 # [ 0.367765] ACPI: PM: (supports S0 S3 S4 S5)1012worker1 # [ 0.374871] pci_bus 0000:00: root bus resource [bus 00-ff]1013worker2 # [ 0.368297] ACPI: Using IOAPIC for interrupt routing1014client # [ 0.366857] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1015worker1 # [ 0.375873] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1016worker2 # [ 0.369178] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1017lb # [ 0.365240] ACPI: Interpreter enabled1018client # [ 0.367299] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1019lb # [ 0.365804] ACPI: PM: (supports S0 S3 S4 S5)1020client # [ 0.368370] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1021worker1 # [ 0.377306] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1022lb # [ 0.366512] ACPI: Using IOAPIC for interrupt routing1023client # [ 0.369424] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1024lb # [ 0.366842] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1025client # [ 0.370629] PCI host bridge to bus 0000:001026lb # [ 0.367787] PCI: Using E820 reservations for host bridge windows1027client # [ 0.371296] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1028worker2 # [ 0.374298] PCI: Using E820 reservations for host bridge windows1029client # [ 0.372292] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1030worker2 # [ 0.375376] ACPI: Enabled 2 GPEs in block 00 to 3F1031client # [ 0.373292] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1032lb # [ 0.370899] ACPI: Enabled 2 GPEs in block 00 to 3F1033client # [ 0.374292] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1034worker1 # [ 0.379897] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1035client # [ 0.375292] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1036worker1 # [ 0.380884] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1037client # [ 0.376292] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1038worker1 # [ 0.381876] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1039client # [ 0.377292] pci_bus 0000:00: root bus resource [bus 00-ff]1040client # [ 0.378231] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1041worker1 # [ 0.383093] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1042client # [ 0.379701] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1043worker2 # [ 0.383998] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1044worker1 # [ 0.384652] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1045lb # [ 0.379586] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1046worker2 # [ 0.385034] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1047lb # [ 0.380574] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1048worker2 # [ 0.386378] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1049worker1 # [ 0.387909] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1050worker1 # [ 0.388755] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1051lb # [ 0.382475] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1052worker2 # [ 0.387421] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1053worker1 # [ 0.389630] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1054worker2 # [ 0.388663] PCI host bridge to bus 0000:001055lb # [ 0.382918] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1056worker1 # [ 0.389877] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1057worker2 # [ 0.389302] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1058lb # [ 0.384168] PCI host bridge to bus 0000:001059client # [ 0.382344] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1060worker2 # [ 0.390298] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1061lb # [ 0.384792] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1062client # [ 0.383305] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1063worker1 # [ 0.391596] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1064lb # [ 0.385787] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1065worker2 # [ 0.391297] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1066client # [ 0.384218] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1067lb # [ 0.386787] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1068worker2 # [ 0.392297] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1069client # [ 0.385219] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1070lb # [ 0.387787] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1071worker2 # [ 0.393297] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1072worker1 # [ 0.393902] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1073worker1 # [ 0.394727] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1074lb # [ 0.388786] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1075worker2 # [ 0.394297] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1076client # [ 0.387184] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1077worker1 # [ 0.395593] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1078worker2 # [ 0.395298] pci_bus 0000:00: root bus resource [bus 00-ff]1079lb # [ 0.389787] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1080lb # [ 0.390787] pci_bus 0000:00: root bus resource [bus 00-ff]1081worker2 # [ 0.396273] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1082worker1 # [ 0.396457] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1083client # [ 0.390321] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1084lb # [ 0.391821] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1085worker2 # [ 0.397790] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1086client # [ 0.391143] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1087client # [ 0.392021] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1088lb # [ 0.393311] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1089client # [ 0.392298] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1090worker1 # [ 0.398916] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1091worker1 # [ 0.400622] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1092client # [ 0.393827] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1093worker1 # [ 0.400908] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1094worker1 # [ 0.402420] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1095client # [ 0.396320] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1096client # [ 0.397130] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1097worker1 # [ 0.403901] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1098worker2 # [ 0.400341] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1099client # [ 0.397991] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1100worker1 # [ 0.404760] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1101worker2 # [ 0.401425] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1102lb # [ 0.395788] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1103worker1 # [ 0.405621] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1104client # [ 0.398892] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1105lb # [ 0.396799] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1106worker2 # [ 0.402365] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1107lb # [ 0.397749] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1108worker2 # [ 0.403663] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1109worker1 # [ 0.406442] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1110lb # [ 0.398964] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1111client # [ 0.401320] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1112worker2 # [ 0.405016] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1113worker1 # [ 0.408899] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1114lb # [ 0.400512] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1115client # [ 0.402316] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1116worker1 # [ 0.409743] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1117client # [ 0.403228] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1118worker1 # [ 0.410622] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1119worker2 # [ 0.408332] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1120client # [ 0.404646] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1121lb # [ 0.403796] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1122worker2 # [ 0.409213] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1123worker1 # [ 0.411456] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1124lb # [ 0.404644] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1125worker2 # [ 0.410127] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1126lb # [ 0.405540] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1127client # [ 0.407143] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1128worker2 # [ 0.411239] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1129client # [ 0.407932] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1130lb # [ 0.405793] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1131worker1 # [ 0.413900] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1132client # [ 0.408314] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1133worker2 # [ 0.412839] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1134worker1 # [ 0.414878] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1135lb # [ 0.407357] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1136worker1 # [ 0.415833] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1137client # [ 0.409857] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1138worker2 # [ 0.415257] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1139worker1 # [ 0.417312] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1140lb # [ 0.409699] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1141worker2 # [ 0.415948] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1142lb # [ 0.410433] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1143client # [ 0.412058] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1144worker2 # [ 0.416320] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1145lb # [ 0.410809] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1146worker1 # [ 0.418901] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1147client # [ 0.412299] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1148worker1 # [ 0.419757] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1149client # [ 0.413230] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1150worker2 # [ 0.417861] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1151lb # [ 0.412343] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1152worker1 # [ 0.420615] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1153worker1 # [ 0.420877] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1154client # [ 0.414682] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1155worker2 # [ 0.421173] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1156worker1 # [ 0.422395] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1157worker2 # [ 0.421949] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1158lb # [ 0.415078] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1159client # [ 0.417321] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1160worker2 # [ 0.422319] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1161lb # [ 0.416527] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1162worker1 # [ 0.424857] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1163client # [ 0.418300] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1164worker1 # [ 0.425614] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1165lb # [ 0.416810] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1166client # [ 0.419233] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1167worker2 # [ 0.423856] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1168worker1 # [ 0.426415] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1169lb # [ 0.418349] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1170client # [ 0.420711] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1171worker2 # [ 0.426234] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1172worker1 # [ 0.427906] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1173worker2 # [ 0.426992] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1174lb # [ 0.420589] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1175worker1 # [ 0.428739] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1176client # [ 0.423086] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1177lb # [ 0.421439] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1178client # [ 0.423934] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1179lb # [ 0.421808] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1180worker2 # [ 0.427319] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1181worker1 # [ 0.429591] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1182client # [ 0.424314] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1183client # [ 0.425298] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1184worker2 # [ 0.428885] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1185worker1 # [ 0.430437] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1186lb # [ 0.424028] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1187client # [ 0.427151] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1188worker1 # [ 0.431879] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1189worker2 # [ 0.431073] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1190worker1 # [ 0.432716] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1191lb # [ 0.426577] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1192worker2 # [ 0.431304] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1193worker1 # [ 0.433620] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1194client # [ 0.429323] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1195lb # [ 0.427533] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1196worker2 # [ 0.432297] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1197client # [ 0.430232] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1198lb # [ 0.427809] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1199worker1 # [ 0.434447] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1200worker2 # [ 0.433940] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1201client # [ 0.431649] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1202lb # [ 0.429356] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1203worker1 # [ 0.435598] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1204client # [ 0.433326] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1205worker1 # [ 0.436080] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1206worker2 # [ 0.437335] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1207worker1 # [ 0.438541] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1208client # [ 0.434172] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1209lb # [ 0.431823] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1210worker2 # [ 0.438208] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1211client # [ 0.435314] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1212lb # [ 0.433514] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1213worker2 # [ 0.438995] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1214lb # [ 0.433808] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1215client # [ 0.436828] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1216worker2 # [ 0.439887] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1217lb # [ 0.435422] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1218client # [ 0.438300] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1219worker2 # [ 0.441333] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1220client # [ 0.439164] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1221worker2 # [ 0.442279] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1222lb # [ 0.437549] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1223client # [ 0.440023] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1224lb # [ 0.437794] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1225worker2 # [ 0.443085] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1226client # [ 0.440907] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1227lb # [ 0.438797] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1228worker2 # [ 0.444305] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1229lb # [ 0.439793] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1230client # [ 0.442730] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1231worker2 # [ 0.445842] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1232worker1 # [ 0.439074] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1233lb # [ 0.441338] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1234client # [ 0.443490] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1235worker1 # [ 0.441489] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1236worker2 # [ 0.447310] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1237client # [ 0.445013] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1238lb # [ 0.442800] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1239worker1 # [ 0.442129] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1240worker2 # [ 0.448275] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1241lb # [ 0.443809] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1242worker1 # [ 0.443456] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1243worker2 # [ 0.449756] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1244lb # [ 0.445366] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1245worker1 # [ 0.444164] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1246worker1 # [ 0.445188] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1247worker2 # [ 0.452083] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1248lb # [ 0.446820] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1249worker1 # [ 0.446125] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1250worker2 # [ 0.452995] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1251lb # [ 0.447794] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1252worker2 # [ 0.453319] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1253worker1 # [ 0.447858] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1254lb # [ 0.448808] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1255worker1 # [ 0.448476] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1256worker2 # [ 0.454859] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1257lb # [ 0.450336] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1258worker1 # [ 0.449211] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1259worker1 # [ 0.450474] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1260worker2 # [ 0.456942] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1261client # [ 0.445481] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1262lb # [ 0.451794] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1263worker2 # [ 0.457304] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1264lb # [ 0.452634] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1265client # [ 0.448073] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1266worker2 # [ 0.458275] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1267lb # [ 0.453576] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1268client # [ 0.448524] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1269worker1 # [ 0.454243] ACPI: PCI: Interrupt link LNKA configured for IRQ 101270worker2 # [ 0.460363] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1271worker1 # [ 0.454978] ACPI: PCI: Interrupt link LNKB configured for IRQ 101272client # [ 0.450002] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1273lb # [ 0.455275] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1274worker1 # [ 0.455970] ACPI: PCI: Interrupt link LNKC configured for IRQ 111275worker2 # [ 0.462091] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1276client # [ 0.450547] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1277lb # [ 0.457646] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1278worker1 # [ 0.456969] ACPI: PCI: Interrupt link LNKD configured for IRQ 111279worker2 # [ 0.462500] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1280worker1 # [ 0.457967] ACPI: PCI: Interrupt link LNKE configured for IRQ 101281client # [ 0.451605] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1282lb # [ 0.458689] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1283worker2 # [ 0.463976] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1284worker1 # [ 0.458967] ACPI: PCI: Interrupt link LNKF configured for IRQ 101285client # [ 0.452536] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1286lb # [ 0.460488] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1287worker1 # [ 0.459970] ACPI: PCI: Interrupt link LNKG configured for IRQ 111288worker1 # [ 0.461143] ACPI: PCI: Interrupt link LNKH configured for IRQ 111289client # [ 0.454342] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1290worker1 # [ 0.461912] ACPI: PCI: Interrupt link GSIA configured for IRQ 161291client # [ 0.455287] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1292worker1 # [ 0.462885] ACPI: PCI: Interrupt link GSIB configured for IRQ 171293worker1 # [ 0.463892] ACPI: PCI: Interrupt link GSIC configured for IRQ 181294client # [ 0.457214] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1295worker1 # [ 0.464886] ACPI: PCI: Interrupt link GSID configured for IRQ 191296worker1 # [ 0.465883] ACPI: PCI: Interrupt link GSIE configured for IRQ 201297client # [ 0.459089] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1298worker1 # [ 0.466883] ACPI: PCI: Interrupt link GSIF configured for IRQ 211299worker1 # [ 0.467882] ACPI: PCI: Interrupt link GSIG configured for IRQ 221300worker1 # [ 0.468886] ACPI: PCI: Interrupt link GSIH configured for IRQ 231301client # [ 0.462912] ACPI: PCI: Interrupt link LNKA configured for IRQ 101302worker1 # [ 0.470947] iommu: Default domain type: Translated1303worker2 # [ 0.464493] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1304client # [ 0.463400] ACPI: PCI: Interrupt link LNKB configured for IRQ 101305worker1 # [ 0.471738] iommu: DMA domain TLB invalidation policy: lazy mode1306lb # [ 0.461018] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1307client # [ 0.464392] ACPI: PCI: Interrupt link LNKC configured for IRQ 111308worker1 # [ 0.472823] ACPI: bus type USB registered1309worker2 # [ 0.467065] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1310worker1 # [ 0.473398] usbcore: registered new interface driver usbfs1311client # [ 0.465392] ACPI: PCI: Interrupt link LNKD configured for IRQ 111312lb # [ 0.463539] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1313worker2 # [ 0.467683] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1314worker1 # [ 0.473883] usbcore: registered new interface driver hub1315client # [ 0.466398] ACPI: PCI: Interrupt link LNKE configured for IRQ 101316worker1 # [ 0.474747] usbcore: registered new device driver usb1317lb # [ 0.464027] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1318client # [ 0.467394] ACPI: PCI: Interrupt link LNKF configured for IRQ 101319worker2 # [ 0.469005] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1320worker1 # [ 0.476399] NetLabel: Initializing1321client # [ 0.468390] ACPI: PCI: Interrupt link LNKG configured for IRQ 111322lb # [ 0.465474] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1323worker1 # [ 0.476872] NetLabel: domain hash size = 1281324worker2 # [ 0.469534] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1325client # [ 0.469390] ACPI: PCI: Interrupt link LNKH configured for IRQ 111326worker1 # [ 0.477622] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1327lb # [ 0.466077] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1328client # [ 0.470329] ACPI: PCI: Interrupt link GSIA configured for IRQ 161329worker2 # [ 0.470592] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1330worker1 # [ 0.477907] NetLabel: unlabeled traffic allowed by default1331client # [ 0.471264] ACPI: PCI: Interrupt link GSIB configured for IRQ 171332worker1 # [ 0.478783] PCI: Using ACPI for IRQ routing1333lb # [ 0.467077] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1334worker2 # [ 0.471568] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1335client # [ 0.472039] ACPI: PCI: Interrupt link GSIC configured for IRQ 181336lb # [ 0.468054] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1337client # [ 0.472307] ACPI: PCI: Interrupt link GSID configured for IRQ 191338worker2 # [ 0.473325] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1339client # [ 0.473307] ACPI: PCI: Interrupt link GSIE configured for IRQ 201340lb # [ 0.469786] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1341client # [ 0.474286] ACPI: PCI: Interrupt link GSIF configured for IRQ 211342worker2 # [ 0.474198] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1343client # [ 0.475304] ACPI: PCI: Interrupt link GSIG configured for IRQ 221344lb # [ 0.470647] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1345client # [ 0.476298] ACPI: PCI: Interrupt link GSIH configured for IRQ 231346worker2 # [ 0.475384] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1347lb # [ 0.472115] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1348client # [ 0.478315] iommu: Default domain type: Translated1349worker2 # [ 0.477115] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1350client # [ 0.479130] iommu: DMA domain TLB invalidation policy: lazy mode1351client # [ 0.480352] ACPI: bus type USB registered1352lb # [ 0.473943] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1353client # [ 0.481159] usbcore: registered new interface driver usbfs1354client # [ 0.482037] usbcore: registered new interface driver hub1355worker2 # [ 0.481010] ACPI: PCI: Interrupt link LNKA configured for IRQ 101356client # [ 0.482302] usbcore: registered new device driver usb1357worker2 # [ 0.481407] ACPI: PCI: Interrupt link LNKB configured for IRQ 101358client # [ 0.483973] NetLabel: Initializing1359worker2 # [ 0.482399] ACPI: PCI: Interrupt link LNKC configured for IRQ 111360client # [ 0.484300] NetLabel: domain hash size = 1281361worker2 # [ 0.483400] ACPI: PCI: Interrupt link LNKD configured for IRQ 111362client # [ 0.485103] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1363worker2 # [ 0.484395] ACPI: PCI: Interrupt link LNKE configured for IRQ 101364client # [ 0.486069] NetLabel: unlabeled traffic allowed by default1365lb # [ 0.477837] ACPI: PCI: Interrupt link LNKA configured for IRQ 101366worker2 # [ 0.485404] ACPI: PCI: Interrupt link LNKF configured for IRQ 101367client # [ 0.486304] PCI: Using ACPI for IRQ routing1368lb # [ 0.478896] ACPI: PCI: Interrupt link LNKB configured for IRQ 101369worker2 # [ 0.486404] ACPI: PCI: Interrupt link LNKG configured for IRQ 111370lb # [ 0.479883] ACPI: PCI: Interrupt link LNKC configured for IRQ 111371worker2 # [ 0.487395] ACPI: PCI: Interrupt link LNKH configured for IRQ 111372lb # [ 0.480911] ACPI: PCI: Interrupt link LNKD configured for IRQ 111373worker2 # [ 0.488334] ACPI: PCI: Interrupt link GSIA configured for IRQ 161374lb # [ 0.481884] ACPI: PCI: Interrupt link LNKE configured for IRQ 101375worker2 # [ 0.489312] ACPI: PCI: Interrupt link GSIB configured for IRQ 171376lb # [ 0.482882] ACPI: PCI: Interrupt link LNKF configured for IRQ 101377worker2 # [ 0.490312] ACPI: PCI: Interrupt link GSIC configured for IRQ 181378lb # [ 0.483896] ACPI: PCI: Interrupt link LNKG configured for IRQ 111379lb # [ 0.484886] ACPI: PCI: Interrupt link LNKH configured for IRQ 111380lb # [ 0.485823] ACPI: PCI: Interrupt link GSIA configured for IRQ 161381lb # [ 0.486801] ACPI: PCI: Interrupt link GSIB configured for IRQ 171382worker2 # [ 0.491310] ACPI: PCI: Interrupt link GSID configured for IRQ 191383lb # [ 0.487800] ACPI: PCI: Interrupt link GSIC configured for IRQ 181384worker2 # [ 0.492320] ACPI: PCI: Interrupt link GSIE configured for IRQ 201385lb # [ 0.489564] ACPI: PCI: Interrupt link GSID configured for IRQ 191386worker2 # [ 0.493310] ACPI: PCI: Interrupt link GSIF configured for IRQ 211387lb # [ 0.490565] ACPI: PCI: Interrupt link GSIE configured for IRQ 201388worker2 # [ 0.494299] ACPI: PCI: Interrupt link GSIG configured for IRQ 221389lb # [ 0.491542] ACPI: PCI: Interrupt link GSIF configured for IRQ 211390worker2 # [ 0.495300] ACPI: PCI: Interrupt link GSIH configured for IRQ 231391lb # [ 0.491803] ACPI: PCI: Interrupt link GSIG configured for IRQ 221392lb # [ 0.492818] ACPI: PCI: Interrupt link GSIH configured for IRQ 231393worker2 # [ 0.497416] iommu: Default domain type: Translated1394worker2 # [ 0.498245] iommu: DMA domain TLB invalidation policy: lazy mode1395lb # [ 0.494933] iommu: Default domain type: Translated1396worker2 # [ 0.499377] ACPI: bus type USB registered1397lb # [ 0.495746] iommu: DMA domain TLB invalidation policy: lazy mode1398worker2 # [ 0.500116] usbcore: registered new interface driver usbfs1399lb # [ 0.496991] ACPI: bus type USB registered1400worker2 # [ 0.501042] usbcore: registered new interface driver hub1401worker2 # [ 0.501309] usbcore: registered new device driver usb1402lb # [ 0.497756] usbcore: registered new interface driver usbfs1403lb # [ 0.498577] usbcore: registered new interface driver hub1404worker2 # [ 0.503326] NetLabel: Initializing1405lb # [ 0.499551] usbcore: registered new device driver usb1406worker2 # [ 0.504007] NetLabel: domain hash size = 1281407worker2 # [ 0.504297] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1408lb # [ 0.500836] NetLabel: Initializing1409worker2 # [ 0.505345] NetLabel: unlabeled traffic allowed by default1410lb # [ 0.501515] NetLabel: domain hash size = 1281411worker2 # [ 0.506366] PCI: Using ACPI for IRQ routing1412worker1 # [ 0.523009] pci 0000:00:01.0: vgaarb: setting as boot VGA device1413lb # [ 0.502393] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1414worker1 # [ 0.523867] pci 0000:00:01.0: vgaarb: bridge control possible1415lb # [ 0.502831] NetLabel: unlabeled traffic allowed by default1416lb # [ 0.503763] PCI: Using ACPI for IRQ routing1417worker1 # [ 0.523867] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1418worker1 # [ 0.523875] vgaarb: loaded1419worker1 # [ 0.524639] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01420worker1 # [ 0.524873] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1421worker1 # [ 0.528960] clocksource: Switched to clocksource kvm-clock1422worker1 # [ 0.532402] VFS: Disk quotas dquot_6.6.01423worker1 # [ 0.533110] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1424worker1 # [ 0.534494] pnp: PnP ACPI init1425worker1 # [ 0.535290] ACPI: IRQ 4 override to edge(!), high(!)1426worker1 # [ 0.536298] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1427worker1 # [ 0.537916] pnp: PnP ACPI: found 6 devices1428client # [ 0.530501] pci 0000:00:01.0: vgaarb: setting as boot VGA device1429client # [ 0.531288] pci 0000:00:01.0: vgaarb: bridge control possible1430client # [ 0.531288] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1431client # [ 0.531296] vgaarb: loaded1432client # [ 0.532033] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01433client # [ 0.532293] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1434worker1 # [ 0.545467] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1435client # [ 0.536378] clocksource: Switched to clocksource kvm-clock1436worker1 # [ 0.547181] clocksource: Switched to clocksource acpi_pm1437worker1 # [ 0.548111] NET: Registered PF_INET protocol family1438worker1 # [ 0.549272] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1439client # [ 0.539777] VFS: Disk quotas dquot_6.6.01440client # [ 0.540539] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1441client # [ 0.541884] pnp: PnP ACPI init1442client # [ 0.542693] ACPI: IRQ 4 override to edge(!), high(!)1443client # [ 0.543712] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1444client # [ 0.545293] pnp: PnP ACPI: found 6 devices1445worker2 # [ 0.549676] pci 0000:00:01.0: vgaarb: setting as boot VGA device1446client # [ 0.552871] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1447worker2 # [ 0.550293] pci 0000:00:01.0: vgaarb: bridge control possible1448client # [ 0.554313] clocksource: Switched to clocksource acpi_pm1449worker2 # [ 0.550293] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1450lb # [ 0.547304] pci 0000:00:01.0: vgaarb: setting as boot VGA device1451client # [ 0.555273] NET: Registered PF_INET protocol family1452worker2 # [ 0.550301] vgaarb: loaded1453worker1 # [ 0.566071] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1454lb # [ 0.547782] pci 0000:00:01.0: vgaarb: bridge control possible1455worker2 # [ 0.551026] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01456client # [ 0.556393] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1457worker1 # [ 0.567575] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1458lb # [ 0.547782] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1459worker2 # [ 0.551299] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1460lb # [ 0.547791] vgaarb: loaded1461worker1 # [ 0.568850] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1462lb # [ 0.548550] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01463lb # [ 0.548790] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1464worker1 # [ 0.570323] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1465worker1 # [ 0.571600] TCP: Hash tables configured (established 16384 bind 16384)1466worker2 # [ 0.554401] clocksource: Switched to clocksource kvm-clock1467worker1 # [ 0.572750] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1468worker2 # [ 0.556075] VFS: Disk quotas dquot_6.6.01469worker1 # [ 0.574059] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1470worker1 # [ 0.575174] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1471lb # [ 0.552887] clocksource: Switched to clocksource kvm-clock1472worker1 # [ 0.576456] NET: Registered PF_UNIX/PF_LOCAL protocol family1473worker2 # [ 0.558682] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1474worker1 # [ 0.577451] NET: Registered PF_XDP protocol family1475worker2 # [ 0.560038] pnp: PnP ACPI init1476worker1 # [ 0.578275] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1477worker2 # [ 0.560834] ACPI: IRQ 4 override to edge(!), high(!)1478lb # [ 0.556451] VFS: Disk quotas dquot_6.6.01479worker1 # [ 0.579314] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1480worker2 # [ 0.561856] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1481lb # [ 0.557179] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1482worker1 # [ 0.580324] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1483lb # [ 0.558554] pnp: PnP ACPI init1484worker2 # [ 0.563420] pnp: PnP ACPI: found 6 devices1485worker1 # [ 0.581458] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1486lb # [ 0.559389] ACPI: IRQ 4 override to edge(!), high(!)1487worker1 # [ 0.582573] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1488lb # [ 0.560408] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1489worker1 # [ 0.583688] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1490client # [ 0.572799] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1491lb # [ 0.561959] pnp: PnP ACPI: found 6 devices1492worker1 # [ 0.585409] ACPI: \_SB_.GSIA: Enabled at IRQ 161493client # [ 0.574321] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1494client # [ 0.575636] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1495worker1 # [ 0.587635] ACPI: \_SB_.GSIB: Enabled at IRQ 171496client # [ 0.577024] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1497client # [ 0.578299] TCP: Hash tables configured (established 16384 bind 16384)1498worker2 # [ 0.571041] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1499worker2 # [ 0.572540] clocksource: Switched to clocksource acpi_pm1500worker1 # [ 0.590506] ACPI: \_SB_.GSIC: Enabled at IRQ 181501client # [ 0.579417] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1502worker2 # [ 0.573501] NET: Registered PF_INET protocol family1503client # [ 0.580705] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1504worker1 # [ 0.592425] ACPI: \_SB_.GSID: Enabled at IRQ 191505client # [ 0.581816] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1506worker2 # [ 0.574666] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1507lb # [ 0.569569] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1508client # [ 0.583044] NET: Registered PF_UNIX/PF_LOCAL protocol family1509lb # [ 0.570989] clocksource: Switched to clocksource acpi_pm1510worker1 # [ 0.594036] PCI: CLS 0 bytes, default 641511client # [ 0.583972] NET: Registered PF_XDP protocol family1512lb # [ 0.571937] NET: Registered PF_INET protocol family1513client # [ 0.584820] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1514worker1 # [ 0.594987] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1515lb # [ 0.573100] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1516client # [ 0.585830] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1517worker1 # [ 0.596726] Trying to unpack rootfs image as initramfs...1518client # [ 0.586832] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1519client # [ 0.588010] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1520client # [ 0.589167] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1521client # [ 0.590313] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1522client # [ 0.592087] ACPI: \_SB_.GSIA: Enabled at IRQ 161523client # [ 0.594530] ACPI: \_SB_.GSIB: Enabled at IRQ 171524client # [ 0.596450] ACPI: \_SB_.GSIC: Enabled at IRQ 181525client # [ 0.598380] ACPI: \_SB_.GSID: Enabled at IRQ 191526worker2 # [ 0.591598] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1527client # [ 0.599959] PCI: CLS 0 bytes, default 641528worker2 # [ 0.593185] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1529client # [ 0.600900] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1530worker2 # [ 0.594567] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1531lb # [ 0.589769] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1532client # [ 0.602694] Trying to unpack rootfs image as initramfs...1533worker2 # [ 0.595919] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1534lb # [ 0.591436] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1535worker2 # [ 0.597165] TCP: Hash tables configured (established 16384 bind 16384)1536lb # [ 0.592779] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1537worker2 # [ 0.598340] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1538lb # [ 0.594173] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1539worker2 # [ 0.599640] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1540lb # [ 0.595424] TCP: Hash tables configured (established 16384 bind 16384)1541worker2 # [ 0.600757] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1542lb # [ 0.596570] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1543worker2 # [ 0.602005] NET: Registered PF_UNIX/PF_LOCAL protocol family1544worker2 # [ 0.602998] NET: Registered PF_XDP protocol family1545lb # [ 0.597850] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1546worker2 # [ 0.603860] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1547lb # [ 0.598945] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1548worker2 # [ 0.604894] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1549lb # [ 0.600230] NET: Registered PF_UNIX/PF_LOCAL protocol family1550worker2 # [ 0.605904] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1551lb # [ 0.601299] NET: Registered PF_XDP protocol family1552worker2 # [ 0.607048] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1553lb # [ 0.602180] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1554lb # [ 0.603225] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1555worker2 # [ 0.608172] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1556lb # [ 0.604274] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1557worker2 # [ 0.609392] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1558lb # [ 0.605417] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1559worker2 # [ 0.611150] ACPI: \_SB_.GSIA: Enabled at IRQ 161560lb # [ 0.606548] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1561lb # [ 0.607701] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1562worker2 # [ 0.613400] ACPI: \_SB_.GSIB: Enabled at IRQ 171563lb # [ 0.609455] ACPI: \_SB_.GSIA: Enabled at IRQ 161564worker2 # [ 0.615299] ACPI: \_SB_.GSIC: Enabled at IRQ 181565lb # [ 0.611552] ACPI: \_SB_.GSIB: Enabled at IRQ 171566worker2 # [ 0.617340] ACPI: \_SB_.GSID: Enabled at IRQ 191567lb # [ 0.613647] ACPI: \_SB_.GSIC: Enabled at IRQ 181568worker2 # [ 0.619033] PCI: CLS 0 bytes, default 641569worker2 # [ 0.619928] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1570lb # [ 0.615874] ACPI: \_SB_.GSID: Enabled at IRQ 191571worker2 # [ 0.621727] Trying to unpack rootfs image as initramfs...1572lb # [ 0.617583] PCI: CLS 0 bytes, default 641573worker1 # [ 0.640483] Initialise system trusted keyrings1574lb # [ 0.618548] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1575lb # [ 0.620411] Trying to unpack rootfs image as initramfs...1576worker1 # [ 0.643452] workingset: timestamp_bits=40 max_order=19 bucket_order=01577client # [ 0.645662] Initialise system trusted keyrings1578client # [ 0.649046] workingset: timestamp_bits=40 max_order=19 bucket_order=01579worker1 # [ 0.666485] Key type asymmetric registered1580worker1 # [ 0.667214] Asymmetric key parser 'x509' registered1581worker1 # [ 0.668132] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1582worker1 # [ 0.671564] io scheduler mq-deadline registered1583worker1 # [ 0.672345] io scheduler kyber registered1584worker1 # [ 0.676597] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1585worker1 # [ 0.677929] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1586client # [ 0.672253] Key type asymmetric registered1587worker1 # [ 0.683175] Linux agpgart interface v0.1031588client # [ 0.673005] Asymmetric key parser 'x509' registered1589worker1 # [ 0.683955] ACPI: bus type drm_connector registered1590client # [ 0.673892] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1591worker2 # [ 0.668694] Initialise system trusted keyrings1592worker1 # [ 0.686834] usbcore: registered new interface driver usbserial_generic1593worker2 # [ 0.669692] workingset: timestamp_bits=40 max_order=19 bucket_order=01594worker1 # [ 0.687904] usbserial: USB Serial support registered for generic1595lb # [ 0.665410] Initialise system trusted keyrings1596client # [ 0.679131] io scheduler mq-deadline registered1597client # [ 0.679907] io scheduler kyber registered1598worker1 # [ 0.691421] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1599lb # [ 0.669052] workingset: timestamp_bits=40 max_order=19 bucket_order=01600worker1 # [ 0.691421] Please enable it if your BIOS has the CPPC option.1601client # [ 0.682554] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1602worker1 # [ 0.693659] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1603client # [ 0.683888] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1604worker1 # [ 0.697559] drop_monitor: Initializing network drop monitor service1605worker1 # [ 0.698785] NET: Registered PF_INET6 protocol family1606client # [ 0.689791] Linux agpgart interface v0.1031607client # [ 0.690616] ACPI: bus type drm_connector registered1608worker1 # [ 0.701857] Segment Routing with IPv61609worker1 # [ 0.704439] In-situ OAM (IOAM) with IPv61610worker1 # [ 0.705494] IPI shorthand broadcast: enabled1611client # [ 0.695122] usbcore: registered new interface driver usbserial_generic1612client # [ 0.696249] usbserial: USB Serial support registered for generic1613worker2 # [ 0.693359] Key type asymmetric registered1614client # [ 0.700006] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1615worker2 # [ 0.694087] Asymmetric key parser 'x509' registered1616client # [ 0.700006] Please enable it if your BIOS has the CPPC option.1617lb # [ 0.690347] Key type asymmetric registered1618worker2 # [ 0.695003] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1619client # [ 0.702256] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1620lb # [ 0.691098] Asymmetric key parser 'x509' registered1621worker1 # [ 0.713762] sched_clock: Marking stable (579013620, 134383148)->(786945865, -73549097)1622lb # [ 0.691940] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1623client # [ 0.705123] drop_monitor: Initializing network drop monitor service1624client # [ 0.706300] NET: Registered PF_INET6 protocol family1625worker2 # [ 0.699412] io scheduler mq-deadline registered1626worker2 # [ 0.700166] io scheduler kyber registered1627worker1 # [ 0.718585] registered taskstats version 11628worker1 # [ 0.719558] Loading compiled-in X.509 certificates1629client # [ 0.709438] Segment Routing with IPv61630lb # [ 0.697169] io scheduler mq-deadline registered1631lb # [ 0.697943] io scheduler kyber registered1632worker2 # [ 0.703814] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1633client # [ 0.712019] In-situ OAM (IOAM) with IPv61634worker2 # [ 0.705144] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1635client # [ 0.713117] IPI shorthand broadcast: enabled1636lb # [ 0.701615] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1637lb # [ 0.702916] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1638worker2 # [ 0.709058] Linux agpgart interface v0.1031639worker2 # [ 0.709893] ACPI: bus type drm_connector registered1640lb # [ 0.706799] Linux agpgart interface v0.1031641lb # [ 0.707590] ACPI: bus type drm_connector registered1642client # [ 0.721249] sched_clock: Marking stable (587014484, 133976071)->(791668832, -70678277)1643worker2 # [ 0.714763] usbcore: registered new interface driver usbserial_generic1644worker2 # [ 0.715846] usbserial: USB Serial support registered for generic1645lb # [ 0.712722] usbcore: registered new interface driver usbserial_generic1646client # [ 0.726165] registered taskstats version 11647lb # [ 0.713847] usbserial: USB Serial support registered for generic1648worker2 # [ 0.719238] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1649client # [ 0.727132] Loading compiled-in X.509 certificates1650worker1 # [ 0.738446] Demotion targets for Node 0: null1651worker2 # [ 0.719238] Please enable it if your BIOS has the CPPC option.1652lb # [ 0.714871] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1653lb # [ 0.714871] Please enable it if your BIOS has the CPPC option.1654worker1 # [ 0.739511] Key type .fscrypt registered1655worker2 # [ 0.721438] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1656worker1 # [ 0.740145] Key type fscrypt-provisioning registered1657lb # [ 0.717062] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1658worker1 # [ 0.741108] ima: No TPM chip found, activating TPM-bypass!1659worker2 # [ 0.725406] drop_monitor: Initializing network drop monitor service1660lb # [ 0.721149] drop_monitor: Initializing network drop monitor service1661worker2 # [ 0.726602] NET: Registered PF_INET6 protocol family1662lb # [ 0.722360] NET: Registered PF_INET6 protocol family1663worker1 # [ 0.746425] ima: Allocated hash algorithm: sha11664worker2 # [ 0.729363] Segment Routing with IPv61665worker1 # [ 0.747364] ima: No architecture policies found1666worker2 # [ 0.730042] In-situ OAM (IOAM) with IPv61667lb # [ 0.727512] Segment Routing with IPv61668worker2 # [ 0.732517] IPI shorthand broadcast: enabled1669worker1 # [ 0.750416] PM: Magic number: 10:814:1461670worker1 # [ 0.752169] RAS: Correctable Errors collector initialized.1671lb # [ 0.730021] In-situ OAM (IOAM) with IPv61672lb # [ 0.731107] IPI shorthand broadcast: enabled1673client # [ 0.743491] Demotion targets for Node 0: null1674client # [ 0.747049] Key type .fscrypt registered1675client # [ 0.747775] Key type fscrypt-provisioning registered1676worker2 # [ 0.740707] sched_clock: Marking stable (607015457, 133203614)->(820315139, -80096068)1677client # [ 0.748752] ima: No TPM chip found, activating TPM-bypass!1678worker1 # [ 0.761152] clk: Disabling unused clocks1679client # [ 0.751002] ima: Allocated hash algorithm: sha11680client # [ 0.751828] ima: No architecture policies found1681lb # [ 0.739295] sched_clock: Marking stable (606013852, 132976825)->(821672552, -82681875)1682worker2 # [ 0.745452] registered taskstats version 11683worker2 # [ 0.746462] Loading compiled-in X.509 certificates1684worker1 # [ 0.764419] PM: genpd: Disabling unused power domains1685lb # [ 0.743092] registered taskstats version 11686client # [ 0.756006] PM: Magic number: 10:814:1461687lb # [ 0.744079] Loading compiled-in X.509 certificates1688client # [ 0.757660] RAS: Correctable Errors collector initialized.1689client # [ 0.766669] clk: Disabling unused clocks1690client # [ 0.769007] PM: genpd: Disabling unused power domains1691worker2 # [ 0.765231] Demotion targets for Node 0: null1692worker2 # [ 0.766115] Key type .fscrypt registered1693worker2 # [ 0.766821] Key type fscrypt-provisioning registered1694worker2 # [ 0.767809] ima: No TPM chip found, activating TPM-bypass!1695lb # [ 0.764008] Demotion targets for Node 0: null1696lb # [ 0.764933] Key type .fscrypt registered1697lb # [ 0.765634] Key type fscrypt-provisioning registered1698lb # [ 0.766654] ima: No TPM chip found, activating TPM-bypass!1699worker2 # [ 0.772235] ima: Allocated hash algorithm: sha11700worker2 # [ 0.773025] ima: No architecture policies found1701worker2 # [ 0.774006] PM: Magic number: 10:814:1461702lb # [ 0.770014] ima: Allocated hash algorithm: sha11703lb # [ 0.770834] ima: No architecture policies found1704worker2 # [ 0.777885] RAS: Correctable Errors collector initialized.1705lb # [ 0.775010] PM: Magic number: 10:814:1461706lb # [ 0.776763] RAS: Correctable Errors collector initialized.1707worker2 # [ 0.786595] clk: Disabling unused clocks1708lb # [ 0.783626] clk: Disabling unused clocks1709worker2 # [ 0.789238] PM: genpd: Disabling unused power domains1710lb # [ 0.788016] PM: genpd: Disabling unused power domains1711client # [ 0.926108] Freeing initrd memory: 29096K1712client # [ 0.929226] Freeing unused decrypted memory: 2028K1713client # [ 0.931803] Freeing unused kernel image (initmem) memory: 3644K1714client # [ 0.932919] Write protecting the kernel read-only data: 32768k1715client # [ 0.934796] Freeing unused kernel image (text/rodata gap) memory: 1208K1716client # [ 0.936351] Freeing unused kernel image (rodata/data gap) memory: 752K1717worker1 # [ 0.953984] Freeing initrd memory: 29092K1718worker1 # [ 0.957472] Freeing unused decrypted memory: 2028K1719worker1 # [ 0.960376] Freeing unused kernel image (initmem) memory: 3644K1720worker1 # [ 0.961557] Write protecting the kernel read-only data: 32768k1721worker1 # [ 0.963706] Freeing unused kernel image (text/rodata gap) memory: 1208K1722worker1 # [ 0.965360] Freeing unused kernel image (rodata/data gap) memory: 752K1723worker2 # [ 0.950081] Freeing initrd memory: 29092K1724worker2 # [ 0.953489] Freeing unused decrypted memory: 2028K1725lb # [ 0.951154] Freeing initrd memory: 29092K1726worker2 # [ 0.956470] Freeing unused kernel image (initmem) memory: 3644K1727worker2 # [ 0.957602] Write protecting the kernel read-only data: 32768k1728lb # [ 0.954459] Freeing unused decrypted memory: 2028K1729worker2 # [ 0.959604] Freeing unused kernel image (text/rodata gap) memory: 1208K1730worker2 # [ 0.961170] Freeing unused kernel image (rodata/data gap) memory: 752K1731lb # [ 0.957311] Freeing unused kernel image (initmem) memory: 3644K1732lb # [ 0.958452] Write protecting the kernel read-only data: 32768k1733lb # [ 0.960482] Freeing unused kernel image (text/rodata gap) memory: 1208K1734lb # [ 0.962092] Freeing unused kernel image (rodata/data gap) memory: 752K1735client # [ 0.987586] x86/mm: Checked W+X mappings: passed, no W+X pages found.1736client # [ 0.988655] Run /init as init process1737client # [ 0.998653] systemd[1]: Inserted module 'autofs4'1738worker1 # [ 1.016883] x86/mm: Checked W+X mappings: passed, no W+X pages found.1739worker1 # [ 1.018080] Run /init as init process1740client # [ 1.014873] fuse: init (API version 7.45)1741worker1 # [ 1.029377] systemd[1]: Inserted module 'autofs4'1742worker2 # [ 1.012337] x86/mm: Checked W+X mappings: passed, no W+X pages found.1743worker2 # [ 1.013443] Run /init as init process1744client # [ 1.020712] ACPI: \_SB_.GSIG: Enabled at IRQ 221745client # [ 1.023148] ACPI: \_SB_.GSIH: Enabled at IRQ 231746lb # [ 1.013333] x86/mm: Checked W+X mappings: passed, no W+X pages found.1747client # [ 1.026100] ACPI: \_SB_.GSIE: Enabled at IRQ 201748lb # [ 1.014438] Run /init as init process1749client # [ 1.028243] ACPI: \_SB_.GSIF: Enabled at IRQ 211750worker2 # [ 1.023736] systemd[1]: Inserted module 'autofs4'1751worker1 # [ 1.047428] fuse: init (API version 7.45)1752lb # [ 1.024801] systemd[1]: Inserted module 'autofs4'1753worker1 # [ 1.053584] ACPI: \_SB_.GSIG: Enabled at IRQ 221754worker1 # [ 1.056353] ACPI: \_SB_.GSIH: Enabled at IRQ 231755worker2 # [ 1.040236] fuse: init (API version 7.45)1756worker1 # [ 1.059592] ACPI: \_SB_.GSIE: Enabled at IRQ 201757worker1 # [ 1.062000] ACPI: \_SB_.GSIF: Enabled at IRQ 211758client # [ 1.052641] systemd[1]: Successfully made /usr/ read-only.1759worker2 # [ 1.046153] ACPI: \_SB_.GSIG: Enabled at IRQ 221760lb # [ 1.041358] fuse: init (API version 7.45)1761worker2 # [ 1.048535] ACPI: \_SB_.GSIH: Enabled at IRQ 231762worker2 # [ 1.051672] ACPI: \_SB_.GSIE: Enabled at IRQ 201763lb # [ 1.047264] ACPI: \_SB_.GSIG: Enabled at IRQ 221764worker2 # [ 1.053868] ACPI: \_SB_.GSIF: Enabled at IRQ 211765lb # [ 1.049680] ACPI: \_SB_.GSIH: Enabled at IRQ 231766lb # [ 1.053417] ACPI: \_SB_.GSIE: Enabled at IRQ 201767lb # [ 1.055834] ACPI: \_SB_.GSIF: Enabled at IRQ 211768worker1 # [ 1.092325] systemd[1]: Successfully made /usr/ read-only.1769worker2 # [ 1.082174] systemd[1]: Successfully made /usr/ read-only.1770lb # [ 1.080855] systemd[1]: Successfully made /usr/ read-only.1771client # [ 1.387829] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1772client # [ 1.392951] systemd[1]: Detected virtualization kvm.1773client # [ 1.393851] systemd[1]: Detected architecture x86-64.1774client # [ 1.394746] systemd[1]: Running in initrd.1775client # [ 1.395856] systemd[1]: Initializing machine ID from random generator.1776client # [ 1.397103] systemd[1]: Hostname set to <client>.1777worker1 # [ 1.429219] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1778worker1 # [ 1.441173] systemd[1]: Detected virtualization kvm.1779worker1 # [ 1.443454] systemd[1]: Detected architecture x86-64.1780worker1 # [ 1.445570] systemd[1]: Running in initrd.1781worker2 # [ 1.418825] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1782worker1 # [ 1.448149] systemd[1]: Initializing machine ID from random generator.1783worker2 # [ 1.430986] systemd[1]: Detected virtualization kvm.1784lb # [ 1.417528] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1785worker1 # [ 1.451044] systemd[1]: Hostname set to <worker1>.1786worker2 # [ 1.433200] systemd[1]: Detected architecture x86-64.1787lb # [ 1.429639] systemd[1]: Detected virtualization kvm.1788worker2 # [ 1.435403] systemd[1]: Running in initrd.1789lb # [ 1.431768] systemd[1]: Detected architecture x86-64.1790worker2 # [ 1.437947] systemd[1]: Initializing machine ID from random generator.1791lb # [ 1.434040] systemd[1]: Running in initrd.1792worker2 # [ 1.440932] systemd[1]: Hostname set to <worker2>.1793lb # [ 1.436648] systemd[1]: Initializing machine ID from random generator.1794lb # [ 1.439502] systemd[1]: Hostname set to <lb>.1795client # [ 1.587076] systemd[1]: bpf-restrict-fs: LSM BPF program attached1796client # [ 1.624863] systemd[1]: Queued start job for default target Initrd Default Target.1797client # [ 1.629258] systemd[1]: Created slice Slice /system/modprobe.1798client # [ 1.630444] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1799client # [ 1.631901] systemd[1]: Expecting device /dev/disk/by-label/nixos...1800client # [ 1.633125] systemd[1]: Reached target Path Units.1801client # [ 1.634046] systemd[1]: Reached target Slice Units.1802client # [ 1.634946] systemd[1]: Reached target Swaps.1803client # [ 1.635790] systemd[1]: Reached target Timer Units.1804client # [ 1.636792] systemd[1]: Listening on D-Bus System Message Bus Socket.1805client # [ 1.638068] systemd[1]: Listening on Journal Socket (/dev/log).1806client # [ 1.639258] systemd[1]: Listening on Journal Sockets.1807client # [ 1.640267] systemd[1]: Listening on udev Control Socket.1808client # [ 1.641278] systemd[1]: Listening on udev Kernel Socket.1809client # [ 1.642241] systemd[1]: Reached target Socket Units.1810client # [ 1.644054] systemd[1]: Starting Create List of Static Device Nodes...1811client # [ 1.647800] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1812worker1 # [ 1.659970] systemd[1]: bpf-restrict-fs: LSM BPF program attached1813client # [ 1.654539] systemd[1]: Starting Load Kernel Module configfs...1814worker2 # [ 1.652653] systemd[1]: bpf-restrict-fs: LSM BPF program attached1815client # [ 1.670208] systemd[1]: Starting Journal Service...1816lb # [ 1.664738] systemd[1]: bpf-restrict-fs: LSM BPF program attached1817client # [ 1.684071] systemd[1]: Starting Load Kernel Modules...1818worker1 # [ 1.700974] systemd[1]: Queued start job for default target Initrd Default Target.1819worker1 # [ 1.705805] systemd[1]: Created slice Slice /system/modprobe.1820worker1 # [ 1.707039] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1821worker1 # [ 1.708541] systemd[1]: Expecting device /dev/disk/by-label/nixos...1822worker1 # [ 1.709694] systemd[1]: Reached target Path Units.1823worker2 # [ 1.691534] systemd[1]: Queued start job for default target Initrd Default Target.1824worker1 # [ 1.710594] systemd[1]: Reached target Slice Units.1825worker1 # [ 1.711523] systemd[1]: Reached target Swaps.1826client # [ 1.689078] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1827worker1 # [ 1.712323] systemd[1]: Reached target Timer Units.1828worker2 # [ 1.695550] systemd[1]: Created slice Slice /system/modprobe.1829worker1 # [ 1.713311] systemd[1]: Listening on D-Bus System Message Bus Socket.1830worker1 # [ 1.714626] systemd[1]: Listening on Journal Socket (/dev/log).1831worker2 # [ 1.696741] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1832client # [ 1.704875] netfs: FS-Cache loaded1833worker1 # [ 1.715794] systemd[1]: Listening on Journal Sockets.1834worker2 # [ 1.698134] systemd[1]: Expecting device /dev/disk/by-label/nixos...1835worker1 # [ 1.716811] systemd[1]: Listening on udev Control Socket.1836worker2 # [ 1.699263] systemd[1]: Reached target Path Units.1837worker1 # [ 1.717845] systemd[1]: Listening on udev Kernel Socket.1838worker2 # [ 1.700118] systemd[1]: Reached target Slice Units.1839worker2 # [ 1.701020] systemd[1]: Reached target Swaps.1840worker1 # [ 1.718823] systemd[1]: Reached target Socket Units.1841worker2 # [ 1.701849] systemd[1]: Reached target Timer Units.1842client # [ 1.709504] 9pnet: Installing 9P2000 support1843worker2 # [ 1.702864] systemd[1]: Listening on D-Bus System Message Bus Socket.1844worker1 # [ 1.720720] systemd[1]: Starting Create List of Static Device Nodes...1845worker2 # [ 1.704087] systemd[1]: Listening on Journal Socket (/dev/log).1846worker2 # [ 1.705327] systemd[1]: Listening on Journal Sockets.1847worker2 # [ 1.706401] systemd[1]: Listening on udev Control Socket.1848worker1 # [ 1.724474] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1849worker2 # [ 1.707460] systemd[1]: Listening on udev Kernel Socket.1850worker2 # [ 1.708492] systemd[1]: Reached target Socket Units.1851worker2 # [ 1.710292] systemd[1]: Starting Create List of Static Device Nodes...1852client # [ 1.717726] systemd[1]: Starting Coldplug All udev Devices...1853worker2 # [ 1.714049] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1854worker1 # [ 1.733594] systemd[1]: Starting Load Kernel Module configfs...1855worker2 # [ 1.720931] systemd[1]: Starting Load Kernel Module configfs...1856client # [ 1.731091] systemd[1]: Finished Create List of Static Device Nodes.1857lb # [ 1.719626] systemd[1]: Queued start job for default target Initrd Default Target.1858lb # [ 1.724386] systemd[1]: Created slice Slice /system/modprobe.1859client # [ 1.736564] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1860lb # [ 1.725609] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1861worker1 # [ 1.749640] systemd[1]: Starting Journal Service...1862lb # [ 1.727071] systemd[1]: Expecting device /dev/disk/by-label/nixos...1863lb # [ 1.728209] systemd[1]: Reached target Path Units.1864lb # [ 1.729130] systemd[1]: Reached target Slice Units.1865lb # [ 1.730031] systemd[1]: Reached target Swaps.1866worker2 # [ 1.735437] systemd[1]: Starting Journal Service...1867lb # [ 1.730837] systemd[1]: Reached target Timer Units.1868lb # [ 1.731845] systemd[1]: Listening on D-Bus System Message Bus Socket.1869client # [ 1.744410] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1870lb # [ 1.733185] systemd[1]: Listening on Journal Socket (/dev/log).1871lb # [ 1.734411] systemd[1]: Listening on Journal Sockets.1872lb # [ 1.735451] systemd[1]: Listening on udev Control Socket.1873lb # [ 1.736518] systemd[1]: Listening on udev Kernel Socket.1874lb # [ 1.737518] systemd[1]: Reached target Socket Units.1875worker1 # [ 1.760624] systemd[1]: Starting Load Kernel Modules...1876client # [ 1.750411] systemd[1]: modprobe@configfs.service: Deactivated successfully.1877lb # [ 1.739392] systemd[1]: Starting Create List of Static Device Nodes...1878client # [ 1.755164] systemd-journald[67]: Collecting audit messages is disabled.1879lb # [ 1.744129] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1880client # [ 1.756699] systemd[1]: Finished Load Kernel Module configfs.1881worker2 # [ 1.750322] systemd[1]: Starting Load Kernel Modules...1882lb # [ 1.751177] systemd[1]: Starting Load Kernel Module configfs...1883client # [ 1.763430] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1884client # [ 1.766533] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1885worker1 # [ 1.766489] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1886worker1 # [ 1.784285] netfs: FS-Cache loaded1887client # [ 1.774058] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1888worker2 # [ 1.755359] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1889lb # [ 1.763709] systemd[1]: Starting Journal Service...1890client # [ 1.775518] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1891worker1 # [ 1.789519] 9pnet: Installing 9P2000 support1892worker2 # [ 1.773686] netfs: FS-Cache loaded1893worker2 # [ 1.778513] 9pnet: Installing 9P2000 support1894worker1 # [ 1.799177] systemd[1]: Starting Coldplug All udev Devices...1895lb # [ 1.778080] systemd[1]: Starting Load Kernel Modules...1896worker2 # [ 1.791467] systemd[1]: Starting Coldplug All udev Devices...1897client # [ 1.803571] systemd[1]: Finished Load Kernel Modules.1898worker1 # [ 1.815504] systemd[1]: Finished Create List of Static Device Nodes.1899lb # [ 1.785077] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1900worker1 # [ 1.821018] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1901client # [ 1.812188] systemd[1]: Starting Apply Kernel Variables...1902lb # [ 1.801556] netfs: FS-Cache loaded1903worker2 # [ 1.806372] systemd[1]: Finished Create List of Static Device Nodes.1904worker1 # [ 1.828850] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1905lb # [ 1.806516] 9pnet: Installing 9P2000 support1906worker2 # [ 1.812365] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1907worker1 # [ 1.831370] systemd-journald[67]: Collecting audit messages is disabled.1908client # [ 1.823164] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1909worker1 # [ 1.835838] systemd[1]: modprobe@configfs.service: Deactivated successfully.1910worker2 # [ 1.819286] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1911lb # [ 1.815269] systemd[1]: Starting Coldplug All udev Devices...1912worker2 # [ 1.822136] systemd-journald[67]: Collecting audit messages is disabled.1913client # [ 1.831240] systemd[1]: Starting Create Static Device Nodes in /dev...1914worker1 # [ 1.842781] systemd[1]: Finished Load Kernel Module configfs.1915worker2 # [ 1.827675] systemd[1]: modprobe@configfs.service: Deactivated successfully.1916worker1 # [ 1.847919] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1917worker2 # [ 1.832583] systemd[1]: Finished Load Kernel Module configfs.1918lb # [ 1.829096] systemd[1]: Finished Create List of Static Device Nodes.1919worker1 # [ 1.853585] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1920worker2 # [ 1.837723] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1921lb # [ 1.835604] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1922worker1 # [ 1.859322] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1923worker2 # [ 1.841878] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1924client # [ 1.851280] systemd[1]: Finished Apply Kernel Variables.1925worker1 # [ 1.863415] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1926lb # [ 1.843420] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1927worker2 # [ 1.850390] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1928worker2 # [ 1.852292] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1929lb # [ 1.850429] systemd[1]: modprobe@configfs.service: Deactivated successfully.1930client # [ 1.865584] systemd[1]: Finished Create Static Device Nodes in /dev.1931lb # [ 1.853629] systemd-journald[67]: Collecting audit messages is disabled.1932lb # [ 1.856082] systemd[1]: Finished Load Kernel Module configfs.1933client # [ 1.734486] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.869608] systemd[1]: Started Journal Service.1934client # 1935client # [ 1.736503] systemd-modules-load[68]: Inserted module 'virtio_balloon'1936client # [ 1.737586] systemd-modules-load[68]: Inserted module 'virtio_gpu'1937lb # [ 1.862465] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1938lb # [ 1.865221] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1939client # [ 1.747112] systemd[1]: Reached target Preparation for Local File Systems.1940client # [ 1.749209] systemd[1]: Reached target Local File Systems.1941worker1 # [ 1.895011] systemd[1]: Finished Load Kernel Modules.1942client # [ 1.751984] systemd[1]: Starting Create System Files and Directories...1943lb # [ 1.874065] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1944worker2 # [ 1.881419] systemd[1]: Finished Load Kernel Modules.1945lb # [ 1.877022] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1946worker1 # [ 1.902621] systemd[1]: Starting Apply Kernel Variables...1947client # [ 1.759067] systemd[1]: Starting Rule-based Manager for Device Events and Files...1948worker2 # [ 1.889475] systemd[1]: Starting Apply Kernel Variables...1949worker1 # [ 1.912551] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1950worker1 # [ 1.923488] systemd[1]: Starting Create Static Device Nodes in /dev...1951worker2 # [ 1.908570] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1952client # [ 1.782403] systemd[1]: Finished Create System Files and Directories.1953lb # [ 1.907547] systemd[1]: Finished Load Kernel Modules.1954lb # [ 1.914177] systemd[1]: Starting Apply Kernel Variables...1955worker2 # [ 1.919425] systemd[1]: Starting Create Static Device Nodes in /dev...1956worker1 # [ 1.943526] systemd[1]: Finished Apply Kernel Variables.1957client # [ 1.803389] systemd-udevd[83]: Using default interface naming scheme 'v261'.1958lb # [ 1.924943] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1959worker2 # [ 1.935355] systemd[1]: Finished Apply Kernel Variables.1960worker1 # [ 1.953516] systemd[1]: Started Journal Service.1961worker1 # [ 1.820216] systemd-modules-load[68]: Inserted module 'dm_mod'1962worker1 # [ 1.823675] systemd-modules-load[68]: Inserted module 'virtio_balloon'1963worker1 # [ 1.825457] systemd-modules-load[68]: Inserted module 'virtio_gpu'1964lb # [ 1.937193] systemd[1]: Starting Create Static Device Nodes in /dev...1965worker2 # [ 1.944365] systemd[1]: Started Journal Service.1966worker2 # [ 1.812256] systemd-modules-load[68]: Inserted module 'dm_mod'1967worker1 # [ 1.831143] systemd[1]: Finished Create Static Device Nodes in /dev.1968worker2 # [ 1.815526] systemd-modules-load[68]: Inserted module 'virtio_balloon'1969worker2 # [ 1.817969] systemd-modules-load[68]: Inserted module 'virtio_gpu'1970lb # [ 1.949135] systemd[1]: Finished Apply Kernel Variables.1971client # [ 1.827322] systemd[1]: Started Rule-based Manager for Device Events and Files.1972worker1 # [ 1.837842] systemd[1]: Reached target Preparation for Local File Systems.1973worker1 # [ 1.838879] systemd[1]: Reached target Local File Systems.1974worker1 # [ 1.841918] systemd[1]: Starting Create System Files and Directories...1975worker1 # [ 1.847227] systemd[1]: Starting Rule-based Manager for Device Events and Files...1976worker2 # [ 1.833146] systemd[1]: Finished Create Static Device Nodes in /dev.1977worker2 # [ 1.834237] systemd[1]: Reached target Preparation for Local File Systems.1978worker2 # [ 1.835187] systemd[1]: Reached target Local File Systems.1979lb # [ 1.966527] systemd[1]: Finished Create Static Device Nodes in /dev.1980worker2 # [ 1.839107] systemd[1]: Starting Create System Files and Directories...1981lb # [ 1.970272] systemd[1]: Started Journal Service.1982lb # [ 1.836750] systemd-modules-load[68]: Inserted module 'dm_mod'1983lb # [ 1.838919] systemd-modules-load[68]: Inserted module 'virtio_balloon'1984worker2 # [ 1.844141] systemd[1]: Starting Rule-based Manager for Device Events and Files...1985lb # [ 1.841282] systemd-modules-load[68]: Inserted module 'virtio_gpu'1986lb # [ 1.850111] systemd[1]: Reached target Preparation for Local File Systems.1987lb # [ 1.851143] systemd[1]: Reached target Local File Systems.1988worker1 # [ 1.873578] systemd[1]: Finished Create System Files and Directories.1989lb # [ 1.854069] systemd[1]: Starting Create System Files and Directories...1990lb # [ 1.858799] systemd[1]: Starting Rule-based Manager for Device Events and Files...1991client # [ 1.871956] systemd[1]: Finished Coldplug All udev Devices.1992client # [ 1.872927] systemd[1]: Reached target System Initialization.1993client # [ 1.873720] systemd[1]: Reached target Basic System.1994worker2 # [ 1.873287] systemd[1]: Finished Create System Files and Directories.1995worker1 # [ 1.896335] systemd-udevd[83]: Using default interface naming scheme 'v261'.1996lb # [ 1.882364] systemd[1]: Finished Create System Files and Directories.1997worker2 # [ 1.895215] systemd-udevd[84]: Using default interface naming scheme 'v261'.1998worker1 # [ 1.921073] systemd[1]: Started Rule-based Manager for Device Events and Files.1999lb # [ 1.906754] systemd-udevd[83]: Using default interface naming scheme 'v261'.2000worker2 # [ 1.919911] systemd[1]: Started Rule-based Manager for Device Events and Files.
loading 1,928 hidden lines…
3929worker2 # [ 14.056835] dhcpcd[692]: eth0: soliciting an IPv6 router3930worker2 # [ 14.059386] dhcpcd[692]: eth0: Router Advertisement from fe80::23931worker2 # [ 14.061443] dhcpcd[692]: eth0: adding address fec0::5054:ff:fe12:3456/643932worker2 # [ 14.063597] dhcpcd[692]: eth0: adding route to fec0::/643933worker2 # [ 14.065474] dhcpcd[692]: eth0: adding default route via fe80::23934lb # [ 14.434462] postgresql-pre-start[795]: performing post-bootstrap initialization ... ok3935lb # [ 14.574386] postgresql-pre-start[795]: syncing data to disk ... ok3936lb # [ 14.576182] postgresql-pre-start[795]: initdb: warning: enabling "trust" authentication for local connections3937lb # [ 14.577455] postgresql-pre-start[795]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.3938lb # [ 14.579458] postgresql-pre-start[795]: Success. You can now start the database server using:3939lb # [ 14.580627] postgresql-pre-start[795]: pg_ctl -D /var/lib/postgresql/18 -l logfile start3940lb # [ 14.693939] postgres[865]: [865] LOG: starting PostgreSQL 18.6 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit3941lb # [ 14.698780] postgres[865]: [865] LOG: listening on IPv6 address "::1", port 54323942lb # [ 14.700206] postgres[865]: [865] LOG: listening on IPv4 address "127.0.0.1", port 54323943lb # [ 14.702921] postgres[865]: [865] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"3944lb # [ 14.719729] postgres[876]: [876] LOG: database system was shut down at 2026-09-20 21:08:28 GMT3945lb # [ 14.727636] postgres[865]: [865] LOG: database system is ready to accept connections3946lb # [ 14.734299] systemd[1]: Started PostgreSQL Server.3947lb # [ 14.741570] systemd[1]: Starting PostgreSQL Setup Scripts...3948lb # [ 14.932394] postgresql-setup-start[890]: CREATE DATABASE3949lb # [ 14.980380] postgresql-setup-start[895]: CREATE ROLE3950lb # [ 15.003086] postgresql-setup-start[897]: ALTER DATABASE3951lb # [ 15.007890] systemd[1]: Finished PostgreSQL Setup Scripts.3952lb # [ 15.009722] systemd[1]: Reached target PostgreSQL.3953worker2 # [ 17.317357] dhcpcd[692]: eth0: leased 10.0.2.15 for 86400 seconds3954worker2 # [ 17.319894] dhcpcd[692]: eth0: adding route to 10.0.2.0/243955worker2 # [ 17.322174] dhcpcd[692]: eth0: adding default route via 10.0.2.23956lb # [ 17.391183] dhcpcd[781]: eth0: leased 10.0.2.15 for 86400 seconds3957lb # [ 17.393873] dhcpcd[781]: eth0: adding route to 10.0.2.0/243958lb # [ 17.396162] dhcpcd[781]: eth0: adding default route via 10.0.2.23959worker2 # [ 17.448646] systemd[1]: Started DHCP Client.3960worker2 # [ 17.450564] systemd[1]: Reached target Multi-User System.3961worker2 # [ 17.451746] systemd[1]: Startup finished in 883ms (kernel) + 5.141s (initrd) + 11.426s (userspace) = 17.451s.3962lb # [ 17.513240] systemd[1]: Started DHCP Client.3963lb # [ 17.515322] systemd[1]: Reached target Network is Online.3964lb # [ 17.519129] systemd[1]: Started Envoy reverse proxy.3965lb # [ 17.522554] systemd[1]: Started mock-oidc.service.3966lb # [ 17.661496] mock-oidc-server[974]: Mock OIDC Server running3967lb # [ 17.662739] mock-oidc-server[974]: OIDC Address: 192.168.1.2:80803968lb # [ 17.663827] mock-oidc-server[974]: Issue Address: 0.0.0.0:80813969lb # [ 17.665246] mock-oidc-server[974]: Issuer: http://192.168.1.2:8080/oidc3970lb # [ 17.666430] mock-oidc-server[974]: JWKS: http://192.168.1.2:8080/oidc/.well-known/jwks.json3971lb # [ 17.667651] mock-oidc-server[974]: Discovery: http://192.168.1.2:8080/oidc/.well-known/openid-configuration3972lb # [ 17.669506] mock-oidc-server[974]: Issue tokens: http://0.0.0.0:8081/issue?sub=...3973lb # [ 18.156517] envoy[973]: [2026-09-20 21:08:32.218][973][info][main] [source/server/server.cc:467] initializing epoch 0 (base id=0, hot restart version=11.104)3974lb # [ 18.158424] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:469] statically linked extensions:3975lb # [ 18.160245] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input3976lb # [ 18.162316] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory3977client # [ 18.173792] dhcpcd[566]: eth0: leased 10.0.2.15 for 86400 seconds3978client # [ 18.177474] dhcpcd[566]: eth0: adding route to 10.0.2.0/243979client # [ 18.179716] dhcpcd[566]: eth0: adding default route via 10.0.2.23980lb # [ 18.165430] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality3981lb # [ 18.179842] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log3982lb # [ 18.186669] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name3983lb # [ 18.188872] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff3984worker1 # [ 18.210716] dhcpcd[692]: eth0: leased 10.0.2.15 for 86400 seconds3985lb # [ 18.191046] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation3986worker1 # [ 18.213714] dhcpcd[692]: eth0: adding route to 10.0.2.0/243987lb # [ 18.193056] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default3988worker1 # [ 18.215874] dhcpcd[692]: eth0: adding default route via 10.0.2.23989lb # [ 18.194870] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource3990lb # [ 18.197697] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm3991lb # [ 18.203797] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v83992lb # [ 18.205721] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router3993lb # [ 18.208609] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router3994lb # [ 18.210406] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts3995lb # [ 18.213385] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher3996lb # [ 18.216086] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header3997lb # [ 18.218517] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name3998lb # [ 18.220819] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua3999lb # [ 18.222371] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port4000lb # [ 18.229350] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert4001lb # [ 18.231234] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service4002lb # [ 18.235745] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata4003lb # [ 18.237859] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config4004lb # [ 18.241181] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator4005lb # [ 18.243478] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities4006lb # [ 18.245239] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats4007lb # [ 18.247693] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http14008lb # [ 18.249984] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state4009lb # [ 18.252761] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable4010lb # [ 18.255126] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json4011lb # [ 18.256676] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher4012lb # [ 18.258939] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy4013lb # [ 18.261883] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router4014lb # [ 18.263596] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection4015lb # [ 18.267897] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso4016lb # [ 18.269809] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default4017lb # [ 18.271551] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file4018lb # [ 18.273262] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder4019lb # [ 18.275377] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action4020lb # [ 18.277344] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator4021lb # [ 18.279653] envoy[973]: [2026-09-20 21:08:32.219][973][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor4022lb # [ 18.282343] envoy[973]: [2026-09-20 21:08:32.223][973][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default4023lb # [ 18.284174] envoy[973]: [2026-09-20 21:08:32.223][973][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default4024lb # [ 18.285988] envoy[973]: [2026-09-20 21:08:32.223][973][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind4025lb # [ 18.287738] envoy[973]: [2026-09-20 21:08:32.223][973][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift4026client # [ 18.302111] systemd[1]: Started DHCP Client.4027lb # [ 18.290970] envoy[973]: [2026-09-20 21:08:32.223][973][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default4028client # [ 18.304063] systemd[1]: Reached target Multi-User System.4029lb # [ 18.292815] envoy[973]: [2026-09-20 21:08:32.223][973][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher4030client # [ 18.304903] systemd[1]: Startup finished in 858ms (kernel) + 5.017s (initrd) + 12.429s (userspace) = 18.304s.4031lb # [ 18.294683] envoy[973]: [2026-09-20 21:08:32.223][973][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls4032lb # [ 18.298789] envoy[973]: [2026-09-20 21:08:32.223][973][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value4033lb # [ 18.302114] envoy[973]: [2026-09-20 21:08:32.223][973][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation4034lb # [ 18.304148] envoy[973]: [2026-09-20 21:08:32.223][973][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options4035lb # [ 18.308114] envoy[973]: [2026-09-20 21:08:32.223][973][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input4036lb # [ 18.309961] envoy[973]: [2026-09-20 21:08:32.223][973][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed4037lb # [ 18.312265] envoy[973]: [2026-09-20 21:08:32.223][973][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4038lb # [ 18.314681] envoy[973]: [2026-09-20 21:08:32.223][973][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case4039lb # [ 18.316931] envoy[973]: [2026-09-20 21:08:32.223][973][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor4040worker1 # [ 18.341407] systemd[1]: Started DHCP Client.4041lb # [ 18.319351] envoy[973]: [2026-09-20 21:08:32.223][973][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection4042worker1 # [ 18.343220] systemd[1]: Reached target Multi-User System.4043lb # [ 18.321091] envoy[973]: [2026-09-20 21:08:32.223][973][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian24044worker1 # [ 18.344282] systemd[1]: Startup finished in 887ms (kernel) + 4.160s (initrd) + 13.295s (userspace) = 18.343s.4045lb # [ 18.322674] envoy[973]: [2026-09-20 21:08:32.225][973][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo4046lb # [ 18.324271] envoy[973]: [2026-09-20 21:08:32.225][973][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest4047lb # [ 18.329134] envoy[973]: [2026-09-20 21:08:32.225][973][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re24048lb # [ 18.330888] envoy[973]: [2026-09-20 21:08:32.225][973][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params4049lb # [ 18.340185] envoy[973]: [2026-09-20 21:08:32.225][973][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector4050lb # [ 18.345827] envoy[973]: [2026-09-20 21:08:32.228][973][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override4051lb # [ 18.349104] envoy[973]: [2026-09-20 21:08:32.228][973][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr4052lb # [ 18.350802] envoy[973]: [2026-09-20 21:08:32.228][973][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based4053lb # [ 18.354447] envoy[973]: [2026-09-20 21:08:32.228][973][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit4054lb # [ 18.356902] envoy[973]: [2026-09-20 21:08:32.228][973][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response4055lb # [ 18.358905] envoy[973]: [2026-09-20 21:08:32.228][973][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter4056lb # [ 18.360640] envoy[973]: [2026-09-20 21:08:32.228][973][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default4057lb # [ 18.362475] envoy[973]: [2026-09-20 21:08:32.228][973][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple4058lb # [ 18.364576] envoy[973]: [2026-09-20 21:08:32.228][973][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme4059lb # [ 18.367500] envoy[973]: [2026-09-20 21:08:32.228][973][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap4060lb # [ 18.372036] envoy[973]: [2026-09-20 21:08:32.228][973][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy4061lb # [ 18.374745] envoy[973]: [2026-09-20 21:08:32.228][973][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated4062lb # [ 18.376600] envoy[973]: [2026-09-20 21:08:32.228][973][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle4063lb # [ 18.378686] envoy[973]: [2026-09-20 21:08:32.228][973][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default4064lb # [ 18.380334] envoy[973]: [2026-09-20 21:08:32.235][973][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query4065lb # [ 18.383235] envoy[973]: [2026-09-20 21:08:32.235][973][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4066lb # [ 18.385435] envoy[973]: [2026-09-20 21:08:32.235][973][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory4067lb # [ 18.387886] envoy[973]: [2026-09-20 21:08:32.235][973][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid4068lb # [ 18.389508] envoy[973]: [2026-09-20 21:08:32.235][973][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain4069lb # [ 18.391204] envoy[973]: [2026-09-20 21:08:32.235][973][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default4070lb # [ 18.393071] envoy[973]: [2026-09-20 21:08:32.235][973][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple4071lb # [ 18.395241] envoy[973]: [2026-09-20 21:08:32.235][973][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule4072lb # [ 18.397781] envoy[973]: [2026-09-20 21:08:32.235][973][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4073lb # [ 18.399775] envoy[973]: [2026-09-20 21:08:32.235][973][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin4074lb # [ 18.402850] envoy[973]: [2026-09-20 21:08:32.235][973][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed4075lb # [ 18.404634] envoy[973]: [2026-09-20 21:08:32.235][973][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd4076lb # [ 18.409367] envoy[973]: [2026-09-20 21:08:32.235][973][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy4077lb # [ 18.420770] envoy[973]: [2026-09-20 21:08:32.235][973][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port4078lb # [ 18.422745] envoy[973]: [2026-09-20 21:08:32.235][973][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche4079lb # [ 18.424653] envoy[973]: [2026-09-20 21:08:32.235][973][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default4080lb # [ 18.426649] envoy[973]: [2026-09-20 21:08:32.235][973][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter4081lb # [ 18.429089] envoy[973]: [2026-09-20 21:08:32.235][973][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal4082lb # [ 18.430678] envoy[973]: [2026-09-20 21:08:32.235][973][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe4083lb # [ 18.433027] envoy[973]: [2026-09-20 21:08:32.235][973][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls4084lb # [ 18.438259] envoy[973]: [2026-09-20 21:08:32.235][973][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth24085lb # [ 18.440343] envoy[973]: [2026-09-20 21:08:32.235][973][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override4086lb # [ 18.442706] envoy[973]: [2026-09-20 21:08:32.235][973][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic4087lb # [ 18.444416] envoy[973]: [2026-09-20 21:08:32.235][973][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction4088lb # [ 18.448426] envoy[973]: [2026-09-20 21:08:32.236][973][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san4089lb # [ 18.454518] envoy[973]: [2026-09-20 21:08:32.236][973][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based4090lb # [ 18.456512] envoy[973]: [2026-09-20 21:08:32.236][973][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router4091lb # [ 18.482210] envoy[973]: [2026-09-20 21:08:32.349][973][info][main] [source/server/server.cc:531] HTTP header map info:4092lb # [ 18.483535] envoy[973]: [2026-09-20 21:08:32.394][973][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id4093lb # [ 18.496561] envoy[973]: [2026-09-20 21:08:32.394][973][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:4094lb # [ 18.497921] envoy[973]: [2026-09-20 21:08:32.394][973][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id4095lb # [ 18.505552] envoy[973]: [2026-09-20 21:08:32.394][973][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status4096lb # [ 18.544516] envoy[973]: [2026-09-20 21:08:32.607][973][info][main] [source/server/server.cc:968] runtime: {}4097lb # [ 18.552083] envoy[973]: [2026-09-20 21:08:32.615][973][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.4098lb # [ 18.554415] envoy[973]: [2026-09-20 21:08:32.617][973][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99014099lb # [ 18.557426] envoy[973]: [2026-09-20 21:08:32.620][973][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99014100lb # [ 18.560767] envoy[973]: [2026-09-20 21:08:32.624][973][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration4101lb # [ 18.562426] envoy[973]: [2026-09-20 21:08:32.625][973][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)4102lb # [ 18.563943] envoy[973]: [2026-09-20 21:08:32.625][973][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)4103lb # [ 18.616587] envoy[973]: [2026-09-20 21:08:32.680][973][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)4104worker1 # [ 18.640733] systemd[1]: Starting Nix worker-protocol over gRPC...4105worker2 # [ 18.625634] systemd[1]: Reached target Network is Online.4106worker2 # [ 18.632859] systemd[1]: Starting scheduler-token.service...4107lb # [ 18.640216] envoy[973]: [2026-09-20 21:08:32.703][973][info][config] [source/server/configuration_impl.cc:164] loading stats configuration4108lb # [ 18.642846] envoy[973]: [2026-09-20 21:08:32.706][973][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.4109lb # [ 18.645878] envoy[973]: [2026-09-20 21:08:32.706][973][info][main] [source/server/server.cc:1077] starting main dispatch loop4110worker2 # [ 18.754746] systemd[1]: scheduler-token.service: Deactivated successfully.4111worker2 # [ 18.755974] systemd[1]: Finished scheduler-token.service.4112worker2 # [ 18.757278] systemd[1]: scheduler-token.service: Consumed 36ms CPU time over 122ms wall clock time, 3.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4113worker2 # [ 18.762668] systemd[1]: Starting Nix worker-protocol over gRPC...4114worker1 # [ 18.870934] nix-grpc-daemon[791]: ts=2026-09-20T21:08:32Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4115worker1 # [ 18.881257] nix-grpc-daemon[791]: ts=2026-09-20T21:08:32Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4116worker1 # [ 18.886894] nix-grpc-daemon[791]: ts=2026-09-20T21:08:32Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14117worker1 # [ 18.940987] nix-grpc-daemon[791]: ts=2026-09-20T21:08:32Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4118worker1 # [ 18.944311] systemd[1]: Started Nix worker-protocol over gRPC.4119worker2 # [ 18.964166] nix-grpc-daemon[791]: ts=2026-09-20T21:08:32Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4120worker2 # [ 18.974292] nix-grpc-daemon[791]: ts=2026-09-20T21:08:32Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4121worker2 # [ 18.979502] nix-grpc-daemon[791]: ts=2026-09-20T21:08:32Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14122lb # [ 18.984997] envoy[973]: [2026-09-20 21:08:33.048][973][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization4123lb # [ 18.989515] envoy[973]: [2026-09-20 21:08:33.052][973][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized4124lb # [ 18.993853] envoy[973]: [2026-09-20 21:08:33.052][973][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager4125lb # [ 18.998421] envoy[973]: [2026-09-20 21:08:33.052][973][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers4126worker1 # [ 19.043389] nix-grpc-daemon[791]: ts=2026-09-20T21:08:32Z level=info event=scheduler_connected addr=lb:500514127worker2 # [ 19.031089] nix-grpc-daemon[791]: ts=2026-09-20T21:08:32Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4128worker2 # [ 19.034489] systemd[1]: Started Nix worker-protocol over gRPC.4129worker1 # [ 19.055681] nix-grpc-daemon[791]: ts=2026-09-20T21:08:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4130worker2 # [ 19.118748] nix-grpc-daemon[791]: ts=2026-09-20T21:08:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4131worker1 # [ 19.256933] nix-grpc-daemon[791]: ts=2026-09-20T21:08:32Z level=info event=scheduler_connected addr=lb:500514132worker1 # [ 19.260830] nix-grpc-daemon[791]: ts=2026-09-20T21:08:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4133worker2 # [ 19.320759] nix-grpc-daemon[791]: ts=2026-09-20T21:08:33Z level=info event=scheduler_connected addr=lb:500514134worker2 # [ 19.322595] nix-grpc-daemon[791]: ts=2026-09-20T21:08:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4135worker1 # [ 19.461616] nix-grpc-daemon[791]: ts=2026-09-20T21:08:32Z level=info event=scheduler_connected addr=lb:500514136worker1 # [ 19.465497] nix-grpc-daemon[791]: ts=2026-09-20T21:08:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4137worker2 # [ 19.523650] nix-grpc-daemon[791]: ts=2026-09-20T21:08:33Z level=info event=scheduler_connected addr=lb:500514138worker2 # [ 19.526937] nix-grpc-daemon[791]: ts=2026-09-20T21:08:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4139worker1 # [ 19.666661] nix-grpc-daemon[791]: ts=2026-09-20T21:08:32Z level=info event=scheduler_connected addr=lb:500514140worker1 # [ 19.670527] nix-grpc-daemon[791]: ts=2026-09-20T21:08:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4141worker2 # [ 19.727717] nix-grpc-daemon[791]: ts=2026-09-20T21:08:33Z level=info event=scheduler_connected addr=lb:500514142worker2 # [ 19.729967] nix-grpc-daemon[791]: ts=2026-09-20T21:08:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4143worker1 # [ 19.873230] nix-grpc-daemon[791]: ts=2026-09-20T21:08:33Z level=info event=scheduler_connected addr=lb:500514144worker1 # [ 19.876424] nix-grpc-daemon[791]: ts=2026-09-20T21:08:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4145worker2 # [ 19.930720] nix-grpc-daemon[791]: ts=2026-09-20T21:08:33Z level=info event=scheduler_connected addr=lb:500514146worker2 # [ 19.932512] nix-grpc-daemon[791]: ts=2026-09-20T21:08:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4147worker1 # [ 20.073906] nix-grpc-daemon[791]: ts=2026-09-20T21:08:33Z level=info event=scheduler_connected addr=lb:500514148worker1 # [ 20.077721] nix-grpc-daemon[791]: ts=2026-09-20T21:08:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4149worker2 # [ 20.133901] nix-grpc-daemon[791]: ts=2026-09-20T21:08:33Z level=info event=scheduler_connected addr=lb:500514150worker2 # [ 20.135577] nix-grpc-daemon[791]: ts=2026-09-20T21:08:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4151worker1 # [ 20.278548] nix-grpc-daemon[791]: ts=2026-09-20T21:08:33Z level=info event=scheduler_connected addr=lb:500514152worker1 # [ 20.282468] nix-grpc-daemon[791]: ts=2026-09-20T21:08:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4153worker2 # [ 20.336952] nix-grpc-daemon[791]: ts=2026-09-20T21:08:34Z level=info event=scheduler_connected addr=lb:500514154worker2 # [ 20.340857] nix-grpc-daemon[791]: ts=2026-09-20T21:08:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4155worker1 # [ 20.483640] nix-grpc-daemon[791]: ts=2026-09-20T21:08:33Z level=info event=scheduler_connected addr=lb:500514156worker1 # [ 20.487688] nix-grpc-daemon[791]: ts=2026-09-20T21:08:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4157worker2 # [ 20.542675] nix-grpc-daemon[791]: ts=2026-09-20T21:08:34Z level=info event=scheduler_connected addr=lb:500514158worker2 # [ 20.545865] nix-grpc-daemon[791]: ts=2026-09-20T21:08:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4159worker1 # [ 20.688183] nix-grpc-daemon[791]: ts=2026-09-20T21:08:33Z level=info event=scheduler_connected addr=lb:500514160worker1 # [ 20.691344] nix-grpc-daemon[791]: ts=2026-09-20T21:08:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4161worker2 # [ 20.744104] nix-grpc-daemon[791]: ts=2026-09-20T21:08:34Z level=info event=scheduler_connected addr=lb:500514162worker2 # [ 20.747841] nix-grpc-daemon[791]: ts=2026-09-20T21:08:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4163worker1 # [ 20.892667] nix-grpc-daemon[791]: ts=2026-09-20T21:08:34Z level=info event=scheduler_connected addr=lb:500514164worker1 # [ 20.894514] nix-grpc-daemon[791]: ts=2026-09-20T21:08:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4165worker2 # [ 20.949323] nix-grpc-daemon[791]: ts=2026-09-20T21:08:34Z level=info event=scheduler_connected addr=lb:500514166worker2 # [ 20.953168] nix-grpc-daemon[791]: ts=2026-09-20T21:08:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4167worker1 # [ 21.095427] nix-grpc-daemon[791]: ts=2026-09-20T21:08:34Z level=info event=scheduler_connected addr=lb:500514168worker1 # [ 21.097362] nix-grpc-daemon[791]: ts=2026-09-20T21:08:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4169worker2 # [ 21.154695] nix-grpc-daemon[791]: ts=2026-09-20T21:08:34Z level=info event=scheduler_connected addr=lb:500514170worker2 # [ 21.157708] nix-grpc-daemon[791]: ts=2026-09-20T21:08:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4171worker1 # [ 21.298679] nix-grpc-daemon[791]: ts=2026-09-20T21:08:34Z level=info event=scheduler_connected addr=lb:500514172worker1 # [ 21.303108] nix-grpc-daemon[791]: ts=2026-09-20T21:08:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4173worker2 # [ 21.356273] nix-grpc-daemon[791]: ts=2026-09-20T21:08:35Z level=info event=scheduler_connected addr=lb:500514174worker2 # [ 21.357933] nix-grpc-daemon[791]: ts=2026-09-20T21:08:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4175worker1 # [ 21.503843] nix-grpc-daemon[791]: ts=2026-09-20T21:08:34Z level=info event=scheduler_connected addr=lb:500514176worker1 # [ 21.508273] nix-grpc-daemon[791]: ts=2026-09-20T21:08:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4177worker2 # [ 21.558717] nix-grpc-daemon[791]: ts=2026-09-20T21:08:35Z level=info event=scheduler_connected addr=lb:500514178worker2 # [ 21.560708] nix-grpc-daemon[791]: ts=2026-09-20T21:08:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4179worker1 # [ 21.709507] nix-grpc-daemon[791]: ts=2026-09-20T21:08:35Z level=info event=scheduler_connected addr=lb:500514180worker1 # [ 21.713720] nix-grpc-daemon[791]: ts=2026-09-20T21:08:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4181worker2 # [ 21.762584] nix-grpc-daemon[791]: ts=2026-09-20T21:08:35Z level=info event=scheduler_connected addr=lb:500514182worker2 # [ 21.767134] nix-grpc-daemon[791]: ts=2026-09-20T21:08:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4183worker1 # [ 21.916812] nix-grpc-daemon[791]: ts=2026-09-20T21:08:35Z level=info event=scheduler_connected addr=lb:500514184worker1 # [ 21.920175] nix-grpc-daemon[791]: ts=2026-09-20T21:08:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4185worker2 # [ 21.968171] nix-grpc-daemon[791]: ts=2026-09-20T21:08:35Z level=info event=scheduler_connected addr=lb:500514186worker2 # [ 21.972382] nix-grpc-daemon[791]: ts=2026-09-20T21:08:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4187worker1 # [ 22.118603] nix-grpc-daemon[791]: ts=2026-09-20T21:08:35Z level=info event=scheduler_connected addr=lb:500514188worker1 # [ 22.122654] nix-grpc-daemon[791]: ts=2026-09-20T21:08:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4189worker2 # [ 22.174444] nix-grpc-daemon[791]: ts=2026-09-20T21:08:35Z level=info event=scheduler_connected addr=lb:500514190worker2 # [ 22.178263] nix-grpc-daemon[791]: ts=2026-09-20T21:08:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4191worker1 # [ 22.323072] nix-grpc-daemon[791]: ts=2026-09-20T21:08:35Z level=info event=scheduler_connected addr=lb:500514192worker1 # [ 22.325274] nix-grpc-daemon[791]: ts=2026-09-20T21:08:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4193worker2 # [ 22.379955] nix-grpc-daemon[791]: ts=2026-09-20T21:08:36Z level=info event=scheduler_connected addr=lb:500514194worker2 # [ 22.384081] nix-grpc-daemon[791]: ts=2026-09-20T21:08:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4195worker1 # [ 22.527059] nix-grpc-daemon[791]: ts=2026-09-20T21:08:35Z level=info event=scheduler_connected addr=lb:500514196worker1 # [ 22.531306] nix-grpc-daemon[791]: ts=2026-09-20T21:08:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4197worker2 # [ 22.585775] nix-grpc-daemon[791]: ts=2026-09-20T21:08:36Z level=info event=scheduler_connected addr=lb:500514198worker2 # [ 22.590136] nix-grpc-daemon[791]: ts=2026-09-20T21:08:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4199worker1 # [ 22.732845] nix-grpc-daemon[791]: ts=2026-09-20T21:08:36Z level=info event=scheduler_connected addr=lb:500514200worker1 # [ 22.737316] nix-grpc-daemon[791]: ts=2026-09-20T21:08:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4201worker2 # [ 22.791522] nix-grpc-daemon[791]: ts=2026-09-20T21:08:36Z level=info event=scheduler_connected addr=lb:500514202worker2 # [ 22.795869] nix-grpc-daemon[791]: ts=2026-09-20T21:08:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4203worker1 # [ 22.941882] nix-grpc-daemon[791]: ts=2026-09-20T21:08:36Z level=info event=scheduler_connected addr=lb:500514204worker1 # [ 22.946138] nix-grpc-daemon[791]: ts=2026-09-20T21:08:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4205worker2 # [ 22.997416] nix-grpc-daemon[791]: ts=2026-09-20T21:08:36Z level=info event=scheduler_connected addr=lb:500514206worker2 # [ 23.001447] nix-grpc-daemon[791]: ts=2026-09-20T21:08:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4207worker1 # [ 23.146914] nix-grpc-daemon[791]: ts=2026-09-20T21:08:36Z level=info event=scheduler_connected addr=lb:500514208worker1 # [ 23.151329] nix-grpc-daemon[791]: ts=2026-09-20T21:08:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4209worker2 # [ 23.203769] nix-grpc-daemon[791]: ts=2026-09-20T21:08:36Z level=info event=scheduler_connected addr=lb:500514210worker2 # [ 23.208370] nix-grpc-daemon[791]: ts=2026-09-20T21:08:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4211worker1 # [ 23.352890] nix-grpc-daemon[791]: ts=2026-09-20T21:08:36Z level=info event=scheduler_connected addr=lb:500514212worker1 # [ 23.357204] nix-grpc-daemon[791]: ts=2026-09-20T21:08:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4213worker2 # [ 23.409775] nix-grpc-daemon[791]: ts=2026-09-20T21:08:37Z level=info event=scheduler_connected addr=lb:500514214worker2 # [ 23.412292] nix-grpc-daemon[791]: ts=2026-09-20T21:08:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4215worker1 # [ 23.558377] nix-grpc-daemon[791]: ts=2026-09-20T21:08:36Z level=info event=scheduler_connected addr=lb:500514216worker1 # [ 23.563589] nix-grpc-daemon[791]: ts=2026-09-20T21:08:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4217worker2 # [ 23.613252] nix-grpc-daemon[791]: ts=2026-09-20T21:08:37Z level=info event=scheduler_connected addr=lb:500514218worker2 # [ 23.615760] nix-grpc-daemon[791]: ts=2026-09-20T21:08:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4219worker1 # [ 23.765109] nix-grpc-daemon[791]: ts=2026-09-20T21:08:37Z level=info event=scheduler_connected addr=lb:500514220worker1 # [ 23.769536] nix-grpc-daemon[791]: ts=2026-09-20T21:08:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4221worker2 # [ 23.816891] nix-grpc-daemon[791]: ts=2026-09-20T21:08:37Z level=info event=scheduler_connected addr=lb:500514222worker2 # [ 23.818751] nix-grpc-daemon[791]: ts=2026-09-20T21:08:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4223worker1 # [ 23.972604] nix-grpc-daemon[791]: ts=2026-09-20T21:08:37Z level=info event=scheduler_connected addr=lb:500514224worker1 # [ 23.976489] nix-grpc-daemon[791]: ts=2026-09-20T21:08:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4225worker2 # [ 24.019650] nix-grpc-daemon[791]: ts=2026-09-20T21:08:37Z level=info event=scheduler_connected addr=lb:500514226worker2 # [ 24.021510] nix-grpc-daemon[791]: ts=2026-09-20T21:08:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4227worker1 # [ 24.177693] nix-grpc-daemon[791]: ts=2026-09-20T21:08:37Z level=info event=scheduler_connected addr=lb:500514228worker1 # [ 24.182075] nix-grpc-daemon[791]: ts=2026-09-20T21:08:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4229worker2 # [ 24.223186] nix-grpc-daemon[791]: ts=2026-09-20T21:08:37Z level=info event=scheduler_connected addr=lb:500514230worker2 # [ 24.225137] nix-grpc-daemon[791]: ts=2026-09-20T21:08:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4231worker1 # [ 24.383424] nix-grpc-daemon[791]: ts=2026-09-20T21:08:37Z level=info event=scheduler_connected addr=lb:500514232worker1 # [ 24.387965] nix-grpc-daemon[791]: ts=2026-09-20T21:08:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4233worker2 # [ 24.426227] nix-grpc-daemon[791]: ts=2026-09-20T21:08:38Z level=info event=scheduler_connected addr=lb:500514234worker2 # [ 24.430356] nix-grpc-daemon[791]: ts=2026-09-20T21:08:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4235worker1 # [ 24.589445] nix-grpc-daemon[791]: ts=2026-09-20T21:08:37Z level=info event=scheduler_connected addr=lb:500514236worker1 # [ 24.593547] nix-grpc-daemon[791]: ts=2026-09-20T21:08:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4237worker2 # [ 24.632071] nix-grpc-daemon[791]: ts=2026-09-20T21:08:38Z level=info event=scheduler_connected addr=lb:500514238worker2 # [ 24.636109] nix-grpc-daemon[791]: ts=2026-09-20T21:08:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4239lb # [ 24.699453] rustfs-bucket-start[1056]: mb s3://farm4240lb # [ 24.708153] systemd[1]: Finished rustfs-bucket.service.4241lb # [ 24.718452] systemd[1]: Starting niks3 server...4242worker1 # [ 24.794610] nix-grpc-daemon[791]: ts=2026-09-20T21:08:38Z level=info event=scheduler_connected addr=lb:500514243worker1 # [ 24.799093] nix-grpc-daemon[791]: ts=2026-09-20T21:08:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4244worker2 # [ 24.837210] nix-grpc-daemon[791]: ts=2026-09-20T21:08:38Z level=info event=scheduler_connected addr=lb:500514245worker2 # [ 24.841393] nix-grpc-daemon[791]: ts=2026-09-20T21:08:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4246worker1 # [ 25.001648] nix-grpc-daemon[791]: ts=2026-09-20T21:08:38Z level=info event=scheduler_connected addr=lb:500514247worker1 # [ 25.004875] nix-grpc-daemon[791]: ts=2026-09-20T21:08:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4248lb # [ 25.000529] postgres[1070]: [1070] ERROR: relation "goose_db_version" does not exist at character 364249lb # [ 25.001746] postgres[1070]: [1070] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4250worker2 # [ 25.042345] nix-grpc-daemon[791]: ts=2026-09-20T21:08:38Z level=info event=scheduler_connected addr=lb:500514251worker2 # [ 25.046244] nix-grpc-daemon[791]: ts=2026-09-20T21:08:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4252lb # [ 25.046971] niks3-server[1065]: 2026/09/20 21:08:39 OK 20241026095416_initial_model.sql (21.92ms)4253lb # [ 25.051112] niks3-server[1065]: 2026/09/20 21:08:39 OK 20251210153512_drop_unused_gin_index.sql (4.03ms)4254lb # [ 25.055976] niks3-server[1065]: 2026/09/20 21:08:39 OK 20251218171726_add_pins.sql (4.87ms)4255lb # [ 25.061775] niks3-server[1065]: 2026/09/20 21:08:39 OK 20260628120000_add_object_size_and_stats.sql (5.85ms)4256lb # [ 25.066247] niks3-server[1065]: 2026/09/20 21:08:39 OK 20260905000000_add_claims.sql (3.04ms)4257lb # [ 25.070702] niks3-server[1065]: 2026/09/20 21:08:39 OK 20260920000000_drop_claims.sql (4.34ms)4258lb # [ 25.071822] niks3-server[1065]: 2026/09/20 21:08:39 goose: successfully migrated database to version: 202609200000004259lb # [ 25.077467] niks3-server[1065]: 2026/09/20 21:08:39 OK 1_commit_pending_closure.sql (6.84ms)4260lb # [ 25.079993] niks3-server[1065]: 2026/09/20 21:08:39 OK 2_object_stats_trigger.sql (1.4ms)4261lb # [ 25.081162] niks3-server[1065]: 2026/09/20 21:08:39 goose: up to current file version: 24262lb # [ 25.087155] niks3-server[1065]: 2026/09/20 21:08:39 INFO Loaded signing key name=farm-test-1 path=/nix/store/c745bsy529madd2s6hz4z5is2436ni04-key4263lb # [ 25.162210] niks3-server[1065]: 2026/09/20 21:08:39 INFO Read proxy enabled — serving cache objects from S34264lb # [ 25.164159] niks3-server[1065]: 2026/09/20 21:08:39 INFO Using socket-activated listener address=0.0.0.0:57514265lb # [ 25.169606] systemd[1]: Started niks3 server.4266lb # [ 25.170285] niks3-server[1065]: 2026/09/20 21:08:39 INFO systemd watchdog enabled interval=15s4267lb # [ 25.172914] systemd[1]: Reached target Multi-User System.4268lb # [ 25.175082] systemd[1]: Startup finished in 885ms (kernel) + 4.099s (initrd) + 20.190s (userspace) = 25.174s.4269lb # [ 25.176430] niks3-server[1065]: 2026/09/20 21:08:39 INFO Starting HTTP server address=0.0.0.0:57514270worker1 # [ 25.202442] nix-grpc-daemon[791]: ts=2026-09-20T21:08:38Z level=info event=scheduler_connected addr=lb:500514271lb # [ 25.184171] niks3-server[1065]: 2026/09/20 21:08:39 INFO lead: acquired remote=192.168.1.4:566684272worker1 # [ 25.206482] nix-grpc-daemon[791]: ts=2026-09-20T21:08:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4273worker2 # [ 25.197474] nix-grpc-daemon[791]: ts=2026-09-20T21:08:38Z level=info event=scheduler_take_over4274worker2 # [ 25.248662] nix-grpc-daemon[791]: ts=2026-09-20T21:08:39Z level=info event=scheduler_connected addr=lb:500514275lb: (finished: waiting for unit niks3.service, in 13.14 seconds)4276lb: waiting for TCP port 5751 on localhost4277worker2 # [ 25.252086] nix-grpc-daemon[791]: ts=2026-09-20T21:08:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4278lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!4279lb: (finished: waiting for TCP port 5751 on localhost, in 0.06 seconds)4280lb: waiting for unit envoy.service4281lb: (finished: waiting for unit envoy.service, in 0.05 seconds)4282lb: waiting for TCP port 50051 on localhost4283worker1 # [ 25.404128] nix-grpc-daemon[791]: ts=2026-09-20T21:08:38Z level=info event=scheduler_connected addr=lb:500514284worker1 # [ 25.406111] nix-grpc-daemon[791]: ts=2026-09-20T21:08:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4285lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!4286lb # [ 25.394720] envoy[973]: conn peer=[::1]:42570 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=34287lb: (finished: waiting for TCP port 50051 on localhost, in 0.03 seconds)4288worker2 # [ 25.449625] nix-grpc-daemon[791]: ts=2026-09-20T21:08:39Z level=info event=scheduler_connected addr=lb:500514289worker2 # [ 25.454406] nix-grpc-daemon[791]: ts=2026-09-20T21:08:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4290worker2 # [ 25.519642] systemd[1]: Starting scheduler-token.service...4291worker2 # [ 25.579993] systemd[1]: scheduler-token.service: Deactivated successfully.4292worker2 # [ 25.581327] systemd[1]: Finished scheduler-token.service.4293worker2 # [ 25.582910] systemd[1]: scheduler-token.service: Consumed 21ms CPU time over 60ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4294lb: waiting until exactly no endpoints of x86_64-linux are unhealthy4295lb: must succeed: curl -sf localhost:9901/clusters4296worker1 # [ 25.606814] nix-grpc-daemon[791]: ts=2026-09-20T21:08:38Z level=info event=scheduler_connected addr=lb:500514297worker1 # [ 25.608677] nix-grpc-daemon[791]: ts=2026-09-20T21:08:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4298worker2 # [ 25.654503] nix-grpc-daemon[791]: ts=2026-09-20T21:08:39Z level=info event=scheduler_connected addr=lb:500514299worker2 # [ 25.656295] nix-grpc-daemon[791]: ts=2026-09-20T21:08:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4300lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4301lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.07 seconds)4302subtest: one node schedules and both builders hold a WorkerSession on it4303lb: must succeed: curl -sf localhost:9901/clusters4304lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4305worker1 # [ 25.809583] nix-grpc-daemon[791]: ts=2026-09-20T21:08:39Z level=info event=scheduler_connected addr=lb:500514306worker1 # [ 25.813145] nix-grpc-daemon[791]: ts=2026-09-20T21:08:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4307worker2 # [ 25.857711] nix-grpc-daemon[791]: ts=2026-09-20T21:08:39Z level=info event=scheduler_connected addr=lb:500514308worker2 # [ 25.861788] nix-grpc-daemon[791]: ts=2026-09-20T21:08:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4309worker1 # [ 26.014197] nix-grpc-daemon[791]: ts=2026-09-20T21:08:39Z level=info event=scheduler_connected addr=lb:500514310worker1 # [ 26.015944] nix-grpc-daemon[791]: ts=2026-09-20T21:08:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4311worker2 # [ 26.062687] nix-grpc-daemon[791]: ts=2026-09-20T21:08:39Z level=info event=scheduler_connected addr=lb:500514312worker2 # [ 26.066507] nix-grpc-daemon[791]: ts=2026-09-20T21:08:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4313worker1 # [ 26.216824] nix-grpc-daemon[791]: ts=2026-09-20T21:08:39Z level=info event=scheduler_connected addr=lb:500514314worker2 # [ 26.210259] nix-grpc-daemon[791]: ts=2026-09-20T21:08:39Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:333484315worker2 # [ 26.212849] nix-grpc-daemon[791]: ts=2026-09-20T21:08:39Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04316worker2 # [ 26.269416] nix-grpc-daemon[791]: ts=2026-09-20T21:08:40Z level=info event=scheduler_connected addr=lb:500514317worker2 # [ 26.286406] nix-grpc-daemon[791]: ts=2026-09-20T21:08:40Z level=info event=oidc_jwks provider=mock keys=14318worker2 # [ 26.290716] nix-grpc-daemon[791]: ts=2026-09-20T21:08:40Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:333484319worker2 # [ 26.294914] nix-grpc-daemon[791]: ts=2026-09-20T21:08:40Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04320lb: must succeed: curl -sf localhost:9901/clusters4321lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4322worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4323worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4324lb: must succeed: curl -sf localhost:9901/clusters4325lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4326lb: must succeed: curl -sf localhost:9901/clusters4327lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4328lb: must succeed: curl -sf localhost:9901/clusters4329lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4330lb: must succeed: curl -sf localhost:9901/clusters4331lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4332worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'4333worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.05 seconds)4334worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4335worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4336lb: must succeed: curl -sf localhost:9901/clusters4337lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4338lb: must succeed: curl -sf localhost:9901/clusters4339lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4340worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4341worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)4342worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4343worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4344worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4345worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4346worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4347worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4348(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 1.83 seconds)4349subtest: balancer auth: client cert, bearer token, nothing, unknown CN4350worker1 # [ 27.901589] nix-grpc-daemon[791]: ts=2026-09-20T21:08:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:521024351worker1 # [ 27.908316] systemd[1]: Started Nix Daemon.4352worker1 # [ 28.104955] nix-daemon[851]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4353worker1 # [ 28.155225] nix-grpc-daemon[791]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 14 ms (attempt 1/5)4354worker1 # [ 28.169252] nix-grpc-daemon[791]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 136 ms (attempt 2/5)4355worker1 # [ 28.308079] nix-grpc-daemon[791]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 201 ms (attempt 3/5)4356worker1 # [ 28.515225] nix-grpc-daemon[791]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 478 ms (attempt 4/5)4357worker1 # [ 29.001394] nix-grpc-daemon[791]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4358worker1 # [ 29.028310] nix-grpc-daemon[791]: ts=2026-09-20T21:08:42Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:521024359worker1 # [ 29.039398] nix-grpc-daemon[791]: ts=2026-09-20T21:08:42Z level=debug event=path_info_miss path=nnbvhsmcqrm6cxydqr22aq61iwmb9zhz-nixos-system-client-test error="error: path '/nix/store/nnbvhsmcqrm6cxydqr22aq61iwmb9zhz-nixos-system-client-test' is required, but there is no substituter that can build it"4360worker1 # [ 29.044579] nix-grpc-daemon[791]: ts=2026-09-20T21:08:42Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:521024361worker1 # [ 29.049234] nix-grpc-daemon[791]: ts=2026-09-20T21:08:42Z level=debug event=path_info_miss path=nnbvhsmcqrm6cxydqr22aq61iwmb9zhz-nixos-system-client-test error="error: path '/nix/store/nnbvhsmcqrm6cxydqr22aq61iwmb9zhz-nixos-system-client-test' is required, but there is no substituter that can build it"4362client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4363client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.08 seconds)4364client: must succeed: install -D /nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4365client: (finished: must succeed: install -D /nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds)4366worker1 # [ 29.417422] nix-grpc-daemon[791]: ts=2026-09-20T21:08:42Z level=info event=oidc_jwks provider=mock keys=14367worker1 # [ 29.422429] nix-grpc-daemon[791]: ts=2026-09-20T21:08:42Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:521024368worker2 # [ 29.424184] nix-grpc-daemon[791]: ts=2026-09-20T21:08:43Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:333504369worker2 # [ 29.430943] systemd[1]: Started Nix Daemon.4370worker2 # [ 29.631108] nix-daemon[871]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4371worker2 # [ 29.658155] nix-grpc-daemon[791]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 44 ms (attempt 1/5)4372worker2 # [ 29.704669] nix-grpc-daemon[791]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 171 ms (attempt 2/5)4373worker2 # [ 29.878502] nix-grpc-daemon[791]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 176 ms (attempt 3/5)4374worker2 # [ 30.060520] nix-grpc-daemon[791]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 111 ms (attempt 4/5)4375worker2 # [ 30.183127] nix-grpc-daemon[791]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4376worker2 # [ 30.200626] nix-grpc-daemon[791]: ts=2026-09-20T21:08:43Z level=debug event=path_info_miss path=nnbvhsmcqrm6cxydqr22aq61iwmb9zhz-nixos-system-client-test error="error: path '/nix/store/nnbvhsmcqrm6cxydqr22aq61iwmb9zhz-nixos-system-client-test' is required, but there is no substituter that can build it"4377worker2 # [ 30.211266] nix-grpc-daemon[791]: ts=2026-09-20T21:08:43Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:333504378worker2 # [ 30.220088] nix-grpc-daemon[791]: ts=2026-09-20T21:08:43Z level=debug event=path_info_miss path=nnbvhsmcqrm6cxydqr22aq61iwmb9zhz-nixos-system-client-test error="error: path '/nix/store/nnbvhsmcqrm6cxydqr22aq61iwmb9zhz-nixos-system-client-test' is required, but there is no substituter that can build it"4379client: must succeed: rm -r /var/lib/nix-grpc-store4380client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.03 seconds)4381worker2 # [ 30.447472] nix-grpc-daemon[791]: ts=2026-09-20T21:08:44Z level=info event=denied method=QueryMissing cn=- role=none4382worker2 # [ 30.641331] nix-grpc-daemon[791]: ts=2026-09-20T21:08:44Z level=info event=denied method=QueryMissing cn=stranger role=none4383(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.17 seconds)4384subtest: DAG build is scheduled across workers and lands in the cache4385client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24386worker2 # [ 30.870956] nix-grpc-daemon[791]: ts=2026-09-20T21:08:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:333504387client # don't know how to build these paths:4388client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4389worker1 # [ 30.907701] nix-grpc-daemon[791]: ts=2026-09-20T21:08:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:521024390worker2 # [ 30.915190] nix-grpc-daemon[791]: ts=2026-09-20T21:08:44Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04391worker2 # [ 30.920371] nix-grpc-daemon[791]: ts=2026-09-20T21:08:44Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.3:50051 clients=14392worker1 # [ 30.987722] nix-grpc-daemon[791]: ts=2026-09-20T21:08:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:521024393worker1 # [ 31.006880] nix-grpc-daemon[791]: ts=2026-09-20T21:08:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:521024394client # copying 1 paths to 'lb:50051'...4395worker1 # [ 31.016341] nix-grpc-daemon[791]: ts=2026-09-20T21:08:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:521024396client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4397worker1 # [ 31.041037] nix-daemon[851]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4398worker1 # [ 31.057531] nix-grpc-daemon[791]: ts=2026-09-20T21:08:44Z level=info event=niks3_push_started pid=8614399worker1 # [ 31.328155] nix-grpc-daemon[861]: time=2026-09-20T21:08:44.629Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.key ca=""4400worker1 # [ 31.501096] nix-daemon[851]: accepted connection from pid 874, user nix-grpc-daemon (trusted)4401worker1 # [ 31.516070] nix-daemon[851]: reaped child process 877, status = succeeded4402worker1 # [ 31.676622] nix-daemon[851]: accepted connection from pid 879, user nix-grpc-daemon (trusted)4403worker1 # [ 31.689162] nix-daemon[851]: reaped child process 882, status = succeeded4404lb # [ 31.680731] niks3-server[1065]: 2026/09/20 21:08:45 INFO Received uploads request method=POST path=/api/pending_closures4405worker1 # [ 31.721665] nix-grpc-daemon[861]: time=2026-09-20T21:08:45.026Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4406worker1 # [ 31.723770] nix-grpc-daemon[861]: time=2026-09-20T21:08:45.028Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4407lb # [ 31.738335] niks3-server[1065]: 2026/09/20 21:08:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4408lb # [ 31.752714] niks3-server[1065]: 2026/09/20 21:08:45 INFO Signed narinfos id=1 count=14409worker1 # [ 31.777227] nix-grpc-daemon[861]: time=2026-09-20T21:08:45.081Z level=INFO msg="Uploading 1 narinfos"4410lb # [ 31.771269] niks3-server[1065]: 2026/09/20 21:08:45 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4411lb # [ 31.777909] niks3-server[1065]: 2026/09/20 21:08:45 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4412lb # [ 31.784255] niks3-server[1065]: 2026/09/20 21:08:45 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4413lb # [ 31.789719] niks3-server[1065]: 2026/09/20 21:08:45 INFO Completed upload id=14414worker1 # [ 31.811615] nix-grpc-daemon[861]: time=2026-09-20T21:08:45.116Z level=INFO msg="Upload complete. (466ms)"4415lb # [ 31.791711] niks3-server[1065]: 2026/09/20 21:08:45 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo4416worker1 # [ 31.816725] nix-grpc-daemon[791]: ts=2026-09-20T21:08:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52102 duration_s=0 paths=1 nar_bytes_in=5044417worker1 # [ 31.823335] nix-grpc-daemon[791]: ts=2026-09-20T21:08:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:521024418worker1 # [ 31.834126] nix-daemon[851]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4419worker1 # [ 31.842185] nix-daemon[851]: reaped child process 859, status = succeeded4420worker1 # [ 31.848219] nix-daemon[851]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4421client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4422client # farm-a> node-a: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4423client # farm-a> LOG-farm-a-t14424worker1 # [ 32.867410] nix-daemon[851]: reaped child process 886, status = succeeded4425worker1 # [ 33.018645] nix-daemon[851]: accepted connection from pid 901, user nix-grpc-daemon (trusted)4426worker1 # [ 33.033088] nix-daemon[851]: reaped child process 904, status = succeeded4427lb # [ 33.026661] niks3-server[1065]: 2026/09/20 21:08:47 INFO Received uploads request method=POST path=/api/pending_closures4428worker1 # [ 33.065946] nix-grpc-daemon[861]: time=2026-09-20T21:08:46.370Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4429worker1 # [ 33.067574] nix-grpc-daemon[861]: time=2026-09-20T21:08:46.372Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4430lb # [ 33.065382] niks3-server[1065]: 2026/09/20 21:08:47 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4431lb # [ 33.069195] niks3-server[1065]: 2026/09/20 21:08:47 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4432lb # [ 33.071944] niks3-server[1065]: 2026/09/20 21:08:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4433lb # [ 33.074492] niks3-server[1065]: 2026/09/20 21:08:47 INFO Signed narinfos id=2 count=14434worker1 # [ 33.096372] nix-grpc-daemon[861]: time=2026-09-20T21:08:46.400Z level=INFO msg="Uploading 1 narinfos"4435lb # [ 33.077534] niks3-server[1065]: 2026/09/20 21:08:47 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4436lb # [ 33.082259] niks3-server[1065]: 2026/09/20 21:08:47 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4437lb # [ 33.086619] niks3-server[1065]: 2026/09/20 21:08:47 INFO Completed upload id=24438worker1 # [ 33.108121] nix-grpc-daemon[861]: time=2026-09-20T21:08:46.412Z level=INFO msg="Upload complete. (250ms)"4439lb # [ 33.088240] niks3-server[1065]: 2026/09/20 21:08:47 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4440worker1 # [ 33.111358] nix-grpc-daemon[791]: ts=2026-09-20T21:08:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52102 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14441worker1 # [ 33.117897] nix-daemon[851]: reaped child process 884, status = succeeded4442worker2 # [ 33.116614] nix-grpc-daemon[791]: ts=2026-09-20T21:08:46Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04443worker2 # [ 33.120571] nix-grpc-daemon[791]: ts=2026-09-20T21:08:46Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.3:50051 clients=14444worker1 # [ 33.175448] nix-grpc-daemon[791]: ts=2026-09-20T21:08:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:521024445worker1 # [ 33.182365] nix-grpc-daemon[791]: ts=2026-09-20T21:08:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:521024446worker1 # [ 33.192282] nix-grpc-daemon[861]: time=2026-09-20T21:08:46.496Z level=INFO msg="All 1 paths already cached"4447client # copying 1 paths to 'lb:50051'...4448worker1 # [ 33.205183] nix-grpc-daemon[791]: ts=2026-09-20T21:08:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:521024449client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4450worker1 # [ 33.225160] nix-daemon[851]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4451worker1 # [ 33.405865] nix-daemon[851]: accepted connection from pid 908, user nix-grpc-daemon (trusted)4452worker1 # [ 33.424346] nix-daemon[851]: reaped child process 911, status = succeeded4453worker1 # [ 33.578615] nix-daemon[851]: accepted connection from pid 913, user nix-grpc-daemon (trusted)4454worker1 # [ 33.592288] nix-daemon[851]: reaped child process 917, status = succeeded4455lb # [ 33.586333] niks3-server[1065]: 2026/09/20 21:08:47 INFO Received uploads request method=POST path=/api/pending_closures4456worker1 # [ 33.621336] nix-grpc-daemon[861]: time=2026-09-20T21:08:46.926Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4457worker1 # [ 33.623313] nix-grpc-daemon[861]: time=2026-09-20T21:08:46.927Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4458lb # [ 33.613082] niks3-server[1065]: 2026/09/20 21:08:47 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4459lb # [ 33.617704] niks3-server[1065]: 2026/09/20 21:08:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4460lb # [ 33.621101] niks3-server[1065]: 2026/09/20 21:08:47 INFO Signed narinfos id=3 count=14461worker1 # [ 33.642701] nix-grpc-daemon[861]: time=2026-09-20T21:08:46.947Z level=INFO msg="Uploading 1 narinfos"4462lb # [ 33.625836] niks3-server[1065]: 2026/09/20 21:08:47 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4463lb # [ 33.633974] niks3-server[1065]: 2026/09/20 21:08:47 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4464lb # [ 33.638785] niks3-server[1065]: 2026/09/20 21:08:47 INFO Completed upload id=34465worker1 # [ 33.660373] nix-grpc-daemon[861]: time=2026-09-20T21:08:46.964Z level=INFO msg="Upload complete. (421ms)"4466lb # [ 33.641551] niks3-server[1065]: 2026/09/20 21:08:47 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4467worker1 # [ 33.662680] nix-grpc-daemon[791]: ts=2026-09-20T21:08:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52102 duration_s=0 paths=1 nar_bytes_in=6244468worker1 # [ 33.665670] nix-grpc-daemon[791]: ts=2026-09-20T21:08:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:521024469worker1 # [ 33.672788] nix-daemon[851]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4470worker1 # [ 33.675338] nix-daemon[851]: reaped child process 906, status = succeeded4471worker1 # [ 33.680660] nix-daemon[851]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4472client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4473client # farm-b> node-a: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4474client # farm-b> LOG-farm-b-t14475worker1 # [ 35.297411] nix-daemon[851]: reaped child process 921, status = succeeded4476worker1 # [ 35.441134] nix-daemon[851]: accepted connection from pid 936, user nix-grpc-daemon (trusted)4477worker1 # [ 35.460631] nix-daemon[851]: reaped child process 939, status = succeeded4478lb # [ 35.449458] niks3-server[1065]: 2026/09/20 21:08:49 INFO Received uploads request method=POST path=/api/pending_closures4479worker1 # [ 35.478665] nix-grpc-daemon[861]: time=2026-09-20T21:08:48.783Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4480worker1 # [ 35.481162] nix-grpc-daemon[861]: time=2026-09-20T21:08:48.784Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4481lb # [ 35.473195] niks3-server[1065]: 2026/09/20 21:08:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4482lb # [ 35.475354] niks3-server[1065]: 2026/09/20 21:08:49 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4483lb # [ 35.480574] niks3-server[1065]: 2026/09/20 21:08:49 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4484lb # [ 35.483597] niks3-server[1065]: 2026/09/20 21:08:49 INFO Signed narinfos id=4 count=14485worker1 # [ 35.507632] nix-grpc-daemon[861]: time=2026-09-20T21:08:48.812Z level=INFO msg="Uploading 1 narinfos"4486lb # [ 35.487757] niks3-server[1065]: 2026/09/20 21:08:49 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4487lb # [ 35.493058] niks3-server[1065]: 2026/09/20 21:08:49 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4488lb # [ 35.508754] niks3-server[1065]: 2026/09/20 21:08:49 INFO Completed upload id=44489worker1 # [ 35.530523] nix-grpc-daemon[861]: time=2026-09-20T21:08:48.835Z level=INFO msg="Upload complete. (242ms)"4490lb # [ 35.512178] niks3-server[1065]: 2026/09/20 21:08:49 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4491worker1 # [ 35.534334] nix-grpc-daemon[791]: ts=2026-09-20T21:08:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52102 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14492worker1 # [ 35.542470] nix-daemon[851]: reaped child process 919, status = succeeded4493worker2 # [ 35.527105] nix-grpc-daemon[791]: ts=2026-09-20T21:08:49Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04494worker2 # [ 35.531261] nix-grpc-daemon[791]: ts=2026-09-20T21:08:49Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.3:50051 clients=14495worker1 # [ 35.587211] nix-grpc-daemon[791]: ts=2026-09-20T21:08:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:521024496worker1 # [ 35.597941] nix-grpc-daemon[791]: ts=2026-09-20T21:08:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:521024497worker1 # [ 35.621528] nix-grpc-daemon[861]: time=2026-09-20T21:08:48.926Z level=INFO msg="All 2 paths already cached"4498client # copying 1 paths to 'lb:50051'...4499client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4500worker1 # [ 35.628762] nix-grpc-daemon[791]: ts=2026-09-20T21:08:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:521024501worker1 # [ 35.638275] nix-daemon[851]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4502worker1 # [ 35.806309] nix-daemon[851]: accepted connection from pid 943, user nix-grpc-daemon (trusted)4503worker1 # [ 35.820789] nix-daemon[851]: reaped child process 946, status = succeeded4504worker1 # [ 35.984491] nix-daemon[851]: accepted connection from pid 948, user nix-grpc-daemon (trusted)4505worker1 # [ 35.997305] nix-daemon[851]: reaped child process 951, status = succeeded4506lb # [ 35.991651] niks3-server[1065]: 2026/09/20 21:08:50 INFO Received uploads request method=POST path=/api/pending_closures4507worker1 # [ 36.119922] nix-grpc-daemon[861]: time=2026-09-20T21:08:49.424Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4508worker1 # [ 36.125102] nix-grpc-daemon[861]: time=2026-09-20T21:08:49.428Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4509lb # [ 36.114511] niks3-server[1065]: 2026/09/20 21:08:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4510lb # [ 36.117151] niks3-server[1065]: 2026/09/20 21:08:50 INFO Signed narinfos id=5 count=14511worker1 # [ 36.140492] nix-grpc-daemon[861]: time=2026-09-20T21:08:49.445Z level=INFO msg="Uploading 1 narinfos"4512lb # [ 36.132448] niks3-server[1065]: 2026/09/20 21:08:50 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4513lb # [ 36.170078] niks3-server[1065]: 2026/09/20 21:08:50 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4514lb # [ 36.175290] niks3-server[1065]: 2026/09/20 21:08:50 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4515lb # [ 36.177558] niks3-server[1065]: 2026/09/20 21:08:50 INFO Completed upload id=54516worker1 # [ 36.199243] nix-grpc-daemon[861]: time=2026-09-20T21:08:49.503Z level=INFO msg="Upload complete. (545ms)"4517lb # [ 36.179466] niks3-server[1065]: 2026/09/20 21:08:50 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4518worker1 # [ 36.201910] nix-grpc-daemon[791]: ts=2026-09-20T21:08:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52102 duration_s=0 paths=1 nar_bytes_in=7524519worker1 # [ 36.205494] nix-grpc-daemon[791]: ts=2026-09-20T21:08:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:521024520worker1 # [ 36.213059] nix-daemon[851]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4521worker1 # [ 36.219161] nix-daemon[851]: reaped child process 941, status = succeeded4522client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4523worker1 # [ 36.222910] nix-daemon[851]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4524client # farm-top> node-a: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4525client # farm-top> LOG-farm-top-t14526worker1 # [ 37.638739] nix-daemon[851]: reaped child process 955, status = succeeded4527worker1 # [ 37.766770] nix-daemon[851]: accepted connection from pid 971, user nix-grpc-daemon (trusted)4528worker1 # [ 37.779712] nix-daemon[851]: reaped child process 974, status = succeeded4529lb # [ 37.770301] niks3-server[1065]: 2026/09/20 21:08:51 INFO Received uploads request method=POST path=/api/pending_closures4530worker1 # [ 37.797340] nix-grpc-daemon[861]: time=2026-09-20T21:08:51.101Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4531worker1 # [ 37.798759] nix-grpc-daemon[861]: time=2026-09-20T21:08:51.103Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4532lb # [ 37.810121] niks3-server[1065]: 2026/09/20 21:08:51 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4533lb # [ 37.821317] niks3-server[1065]: 2026/09/20 21:08:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4534lb # [ 37.829610] niks3-server[1065]: 2026/09/20 21:08:51 INFO Signed narinfos id=6 count=14535worker1 # [ 37.853889] nix-grpc-daemon[861]: time=2026-09-20T21:08:51.158Z level=INFO msg="Uploading 1 narinfos"4536lb # [ 37.834892] niks3-server[1065]: 2026/09/20 21:08:51 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4537lb # [ 37.840586] niks3-server[1065]: 2026/09/20 21:08:51 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4538lb # [ 37.849676] niks3-server[1065]: 2026/09/20 21:08:51 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4539lb # [ 37.852958] niks3-server[1065]: 2026/09/20 21:08:51 INFO Completed upload id=64540worker1 # [ 37.874517] nix-grpc-daemon[861]: time=2026-09-20T21:08:51.179Z level=INFO msg="Upload complete. (244ms)"4541lb # [ 37.855285] niks3-server[1065]: 2026/09/20 21:08:51 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4542worker1 # [ 37.877373] nix-grpc-daemon[791]: ts=2026-09-20T21:08:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52102 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14543worker1 # [ 37.883965] nix-daemon[851]: reaped child process 953, status = succeeded4544client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 7.24 seconds)4545client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4546client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.17 seconds)4547client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4548client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.02 seconds)4549client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4550client # copying 3 paths...4551client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4552client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4553client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4554lb # [ 38.627067] envoy[973]: conn peer=[2001:db8:1::1]:48944 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=715 tx=364 ms=12004555client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.48 seconds)4556(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 7.97 seconds)4557subtest: repeat is answered Cached by the scheduler without building4558worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4559lb # [ 38.630821] envoy[973]: conn peer=[2001:db8:1::1]:48958 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2613 tx=406 ms=8464560lb # [ 38.634244] envoy[973]: conn peer=[2001:db8:1::1]:48968 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=74561lb # [ 38.637459] envoy[973]: conn peer=[2001:db8:1::1]:48982 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=84562lb # [ 38.640860] envoy[973]: conn peer=[2001:db8:1::1]:48992 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9541 tx=3854 ms=70464563worker1 # finding garbage collector roots...4564worker1 # deleting specified paths...4565worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4566worker1 # deleting unused links...4567worker1 # note: hard linking is currently saving 0.0 KiB4568worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.25 seconds)4569worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4570worker2 # finding garbage collector roots...4571worker2 # deleting specified paths...4572worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4573worker2 # deleting unused links...4574worker2 # note: hard linking is currently saving 0.0 KiB4575worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.29 seconds)4576client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24577worker2 # [ 39.419506] nix-grpc-daemon[791]: ts=2026-09-20T21:08:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:333504578client # don't know how to build these paths:4579client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4580worker2 # [ 39.427755] nix-grpc-daemon[791]: ts=2026-09-20T21:08:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:333504581worker2 # [ 39.457450] nix-grpc-daemon[791]: ts=2026-09-20T21:08:53Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14582client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 0.30 seconds)4583client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4584client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.17 seconds)4585lb: must succeed: curl -sf localhost:9901/clusters4586lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4587lb: must succeed: curl -sf localhost:9901/clusters4588lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4589worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4590worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4591(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.28 seconds)4592subtest: inputs already in the cache are not sent to the scheduler4593lb: must succeed: curl -sf localhost:9901/clusters4594lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4595lb: must succeed: curl -sf localhost:9901/clusters4596lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4597worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4598worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4599client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&24600client # don't know how to build these paths:4601client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4602worker2 # [ 40.290971] nix-grpc-daemon[791]: ts=2026-09-20T21:08:54Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:333504603worker2 # [ 40.298698] nix-grpc-daemon[791]: ts=2026-09-20T21:08:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:333504604worker2 # [ 40.308599] nix-grpc-daemon[791]: ts=2026-09-20T21:08:54Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04605worker2 # [ 40.312574] nix-grpc-daemon[791]: ts=2026-09-20T21:08:54Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.3:50051 clients=14606worker1 # [ 40.369206] nix-grpc-daemon[791]: ts=2026-09-20T21:08:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:521024607worker1 # [ 40.377691] nix-grpc-daemon[791]: ts=2026-09-20T21:08:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:521024608worker1 # [ 40.390795] nix-grpc-daemon[861]: time=2026-09-20T21:08:53.695Z level=INFO msg="All 2 paths already cached"4609client # copying 1 paths to 'lb:50051'...4610worker1 # [ 40.396766] nix-grpc-daemon[791]: ts=2026-09-20T21:08:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:521024611client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4612worker1 # [ 40.403888] nix-daemon[851]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4613worker1 # [ 40.557683] nix-daemon[851]: accepted connection from pid 992, user nix-grpc-daemon (trusted)4614worker1 # [ 40.573766] nix-daemon[851]: reaped child process 995, status = succeeded4615worker1 # [ 40.725740] nix-daemon[851]: accepted connection from pid 997, user nix-grpc-daemon (trusted)4616worker1 # [ 40.737919] nix-daemon[851]: reaped child process 1000, status = succeeded4617lb # [ 40.727295] niks3-server[1065]: 2026/09/20 21:08:54 INFO Received uploads request method=POST path=/api/pending_closures4618worker1 # [ 40.768150] nix-grpc-daemon[861]: time=2026-09-20T21:08:54.072Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4619worker1 # [ 40.769650] nix-grpc-daemon[861]: time=2026-09-20T21:08:54.074Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4620lb # [ 40.762150] niks3-server[1065]: 2026/09/20 21:08:54 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4621lb # [ 40.764565] niks3-server[1065]: 2026/09/20 21:08:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4622lb # [ 40.767428] niks3-server[1065]: 2026/09/20 21:08:54 INFO Signed narinfos id=7 count=14623worker1 # [ 40.789088] nix-grpc-daemon[861]: time=2026-09-20T21:08:54.093Z level=INFO msg="Uploading 1 narinfos"4624lb # [ 40.770368] niks3-server[1065]: 2026/09/20 21:08:54 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4625lb # [ 40.775507] niks3-server[1065]: 2026/09/20 21:08:54 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4626lb # [ 40.779769] niks3-server[1065]: 2026/09/20 21:08:54 INFO Completed upload id=74627worker1 # [ 40.803046] nix-grpc-daemon[861]: time=2026-09-20T21:08:54.107Z level=INFO msg="Upload complete. (393ms)"4628lb # [ 40.781907] niks3-server[1065]: 2026/09/20 21:08:54 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4629worker1 # [ 40.805762] nix-grpc-daemon[791]: ts=2026-09-20T21:08:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52102 duration_s=0 paths=1 nar_bytes_in=7524630worker1 # [ 40.809101] nix-grpc-daemon[791]: ts=2026-09-20T21:08:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:521024631worker1 # [ 40.815098] nix-daemon[851]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4632worker1 # [ 40.820885] nix-daemon[851]: reaped child process 990, status = succeeded4633client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4634worker1 # [ 40.825981] nix-daemon[851]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4635client # farm-top> node-a: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4636client # farm-top> LOG-farm-top-t1b4637worker1 # [ 42.249460] nix-daemon[851]: reaped child process 1004, status = succeeded4638worker1 # [ 42.402597] nix-daemon[851]: accepted connection from pid 1019, user nix-grpc-daemon (trusted)4639worker1 # [ 42.414921] nix-daemon[851]: reaped child process 1022, status = succeeded4640lb # [ 42.406553] niks3-server[1065]: 2026/09/20 21:08:56 INFO Received uploads request method=POST path=/api/pending_closures4641worker1 # [ 42.437117] nix-grpc-daemon[861]: time=2026-09-20T21:08:55.741Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4642worker1 # [ 42.438620] nix-grpc-daemon[861]: time=2026-09-20T21:08:55.743Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4643lb # [ 42.452396] niks3-server[1065]: 2026/09/20 21:08:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4644lb # [ 42.458350] niks3-server[1065]: 2026/09/20 21:08:56 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4645lb # [ 42.463706] niks3-server[1065]: 2026/09/20 21:08:56 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4646lb # [ 42.469630] niks3-server[1065]: 2026/09/20 21:08:56 INFO Signed narinfos id=8 count=14647lb # [ 42.471919] niks3-server[1065]: 2026/09/20 21:08:56 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4648worker1 # [ 42.493598] nix-grpc-daemon[861]: time=2026-09-20T21:08:55.797Z level=INFO msg="Uploading 1 narinfos"4649lb # [ 42.478675] niks3-server[1065]: 2026/09/20 21:08:56 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4650lb # [ 42.482896] niks3-server[1065]: 2026/09/20 21:08:56 INFO Completed upload id=84651worker1 # [ 42.504684] nix-grpc-daemon[861]: time=2026-09-20T21:08:55.809Z level=INFO msg="Upload complete. (267ms)"4652lb # [ 42.484984] niks3-server[1065]: 2026/09/20 21:08:56 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4653worker1 # [ 42.509501] nix-grpc-daemon[791]: ts=2026-09-20T21:08:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52102 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14654worker1 # [ 42.521528] nix-daemon[851]: reaped child process 1002, status = succeeded4655client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.44 seconds)4656client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath4657client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.18 seconds)4658lb: must succeed: curl -sf localhost:9901/clusters4659lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4660lb: must succeed: curl -sf localhost:9901/clusters4661lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4662worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4663worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4664(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.96 seconds)4665subtest: two clients wanting the same drv build it once4666client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4667client # [ 42.936779] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4668client # Running as unit: dedup1.service; invocation ID: 419933c134ae4a368c12ed6373114edc4669client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 0.07 seconds)4670client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4671client # [ 43.173884] nix[892]: don't know how to build these paths:4672client # [ 43.174812] nix[892]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4673worker2 # [ 43.165068] nix-grpc-daemon[791]: ts=2026-09-20T21:08:56Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:333504674worker2 # [ 43.176101] nix-grpc-daemon[791]: ts=2026-09-20T21:08:56Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04675worker2 # [ 43.181815] nix-grpc-daemon[791]: ts=2026-09-20T21:08:56Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.4:50051 clients=14676worker2 # [ 43.187308] nix-grpc-daemon[791]: ts=2026-09-20T21:08:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333504677worker2 # [ 43.207256] nix-grpc-daemon[791]: ts=2026-09-20T21:08:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:333504678client # [ 43.224813] nix[892]: copying 1 paths to 'lb:50051'...4679client # [ 43.225634] nix[892]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4680worker2 # [ 43.219260] nix-grpc-daemon[791]: ts=2026-09-20T21:08:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:333504681worker2 # [ 43.231474] nix-daemon[871]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4682worker2 # [ 43.320369] nix-grpc-daemon[791]: ts=2026-09-20T21:08:57Z level=info event=niks3_push_started pid=9224683worker2 # [ 43.637082] nix-grpc-daemon[922]: time=2026-09-20T21:08:57.387Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.key ca=""4684worker2 # [ 43.849828] nix-daemon[871]: accepted connection from pid 935, user nix-grpc-daemon (trusted)4685worker2 # [ 43.865136] nix-daemon[871]: reaped child process 938, status = succeeded4686client # [ 44.018585] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4687client # Running as unit: dedup2.service; invocation ID: 8e32034517c3408d92b0b0a3a8908e754688client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 1.08 seconds)4689worker2 # [ 44.042886] nix-daemon[871]: accepted connection from pid 941, user nix-grpc-daemon (trusted)4690worker2 # [ 44.056093] nix-daemon[871]: reaped child process 944, status = succeeded4691lb # [ 44.068820] niks3-server[1065]: 2026/09/20 21:08:58 INFO Received uploads request method=POST path=/api/pending_closures4692worker2 # [ 44.083791] nix-grpc-daemon[922]: time=2026-09-20T21:08:57.838Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4693worker2 # [ 44.086145] nix-grpc-daemon[922]: time=2026-09-20T21:08:57.841Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4694lb # [ 44.106213] niks3-server[1065]: 2026/09/20 21:08:58 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4695lb # [ 44.117471] niks3-server[1065]: 2026/09/20 21:08:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4696worker2 # [ 44.132096] nix-grpc-daemon[922]: time=2026-09-20T21:08:57.887Z level=INFO msg="Uploading 1 narinfos"4697lb # [ 44.127906] niks3-server[1065]: 2026/09/20 21:08:58 INFO Signed narinfos id=9 count=14698lb # [ 44.135110] niks3-server[1065]: 2026/09/20 21:08:58 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4699lb # [ 44.144087] niks3-server[1065]: 2026/09/20 21:08:58 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4700lb # [ 44.151470] niks3-server[1065]: 2026/09/20 21:08:58 INFO Completed upload id=94701worker2 # [ 44.157800] nix-grpc-daemon[922]: time=2026-09-20T21:08:57.912Z level=INFO msg="Upload complete. (497ms)"4702lb # [ 44.153728] niks3-server[1065]: 2026/09/20 21:08:58 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4703worker2 # [ 44.159491] nix-grpc-daemon[791]: ts=2026-09-20T21:08:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33350 duration_s=0 paths=1 nar_bytes_in=4324704worker2 # [ 44.164607] nix-grpc-daemon[791]: ts=2026-09-20T21:08:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333504705worker2 # [ 44.171585] nix-daemon[871]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4706worker2 # [ 44.174277] nix-daemon[871]: reaped child process 920, status = succeeded4707client # [ 44.184881] nix[892]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4708worker2 # [ 44.179359] nix-daemon[871]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4709client # [ 44.296578] nix[909]: don't know how to build these paths:4710client # [ 44.297866] nix[909]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4711worker1 # [ 44.305652] nix-grpc-daemon[791]: ts=2026-09-20T21:08:57Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:521024712worker2 # [ 44.299812] nix-grpc-daemon[791]: ts=2026-09-20T21:08:58Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04713worker2 # [ 44.309757] nix-grpc-daemon[791]: ts=2026-09-20T21:08:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333504714client # [ 44.321396] nix[909]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4715client: must succeed: sleep 24716client: (finished: must succeed: sleep 2, in 2.06 seconds)4717client: waiting for success: ! systemctl is-active dedup1 dedup24718lb # [ 48.626770] envoy[973]: conn peer=[2001:db8:1::1]:42868 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=514719lb # [ 48.628752] envoy[973]: conn peer=[2001:db8:1::1]:42870 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4139 tx=1685 ms=22114720worker2 # [ 48.801697] nix-daemon[871]: reaped child process 952, status = succeeded4721worker2 # [ 49.004110] nix-daemon[871]: accepted connection from pid 983, user nix-grpc-daemon (trusted)4722worker2 # [ 49.026407] nix-daemon[871]: reaped child process 986, status = succeeded4723lb # [ 49.035383] niks3-server[1065]: 2026/09/20 21:09:03 INFO Received uploads request method=POST path=/api/pending_closures4724worker2 # [ 49.047797] nix-grpc-daemon[922]: time=2026-09-20T21:09:02.802Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4725worker2 # [ 49.049887] nix-grpc-daemon[922]: time=2026-09-20T21:09:02.804Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4726lb # [ 49.064907] niks3-server[1065]: 2026/09/20 21:09:03 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4727lb # [ 49.071775] niks3-server[1065]: 2026/09/20 21:09:03 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4728lb # [ 49.077725] niks3-server[1065]: 2026/09/20 21:09:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4729lb # [ 49.079872] niks3-server[1065]: 2026/09/20 21:09:03 INFO Signed narinfos id=10 count=14730worker2 # [ 49.088050] nix-grpc-daemon[922]: time=2026-09-20T21:09:02.842Z level=INFO msg="Uploading 1 narinfos"4731lb # [ 49.089148] niks3-server[1065]: 2026/09/20 21:09:03 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4732lb # [ 49.097514] niks3-server[1065]: 2026/09/20 21:09:03 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4733lb # [ 49.104855] niks3-server[1065]: 2026/09/20 21:09:03 INFO Completed upload id=104734worker2 # [ 49.110056] nix-grpc-daemon[922]: time=2026-09-20T21:09:02.865Z level=INFO msg="Upload complete. (315ms)"4735lb # [ 49.108461] niks3-server[1065]: 2026/09/20 21:09:03 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4736worker2 # [ 49.114884] nix-grpc-daemon[791]: ts=2026-09-20T21:09:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33350 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14737worker2 # [ 49.119536] nix-grpc-daemon[791]: ts=2026-09-20T21:09:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33350 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14738worker2 # [ 49.127773] nix-daemon[871]: reaped child process 950, status = succeeded4739client # [ 49.157353] systemd[1]: dedup2.service: Deactivated successfully.4740client # [ 49.160204] systemd[1]: dedup2.service: Consumed 141ms CPU time over 5.141s wall clock time, 35.1M memory peak, 5.9K incoming IP traffic, 5.6K outgoing IP traffic.4741client # [ 49.195878] systemd[1]: dedup1.service: Deactivated successfully.4742client # [ 49.198530] systemd[1]: dedup1.service: Consumed 118ms CPU time over 6.261s wall clock time, 41.5M memory peak, 6.9K incoming IP traffic, 8.2K outgoing IP traffic.4743client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.10 seconds)4744client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4745client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)4746worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4747worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.04 seconds)4748worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4749worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.04 seconds)4750worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4751worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4752worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4753worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4754(finished: subtest: two clients wanting the same drv build it once, in 7.07 seconds)4755subtest: build hook: nix-daemon with builders = grpc://lb4756client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log4757client # [ 50.162715] systemd[1]: Started Nix Daemon.4758client # [ 50.340565] nix-daemon[951]: accepted connection from pid 948, user root (trusted)4759worker2 # [ 50.541561] nix-grpc-daemon[791]: ts=2026-09-20T21:09:04Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:333504760worker2 # [ 50.544924] nix-grpc-daemon[791]: ts=2026-09-20T21:09:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:333504761worker2 # [ 50.557836] nix-grpc-daemon[791]: ts=2026-09-20T21:09:04Z level=debug event=want client=6 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=04762worker2 # [ 50.560339] nix-grpc-daemon[791]: ts=2026-09-20T21:09:04Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.3:50051 clients=14763worker1 # [ 50.621266] nix-grpc-daemon[791]: ts=2026-09-20T21:09:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:521024764worker1 # [ 50.630406] nix-grpc-daemon[791]: ts=2026-09-20T21:09:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:521024765worker1 # [ 50.639062] nix-grpc-daemon[791]: ts=2026-09-20T21:09:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:521024766worker1 # [ 50.647099] nix-daemon[851]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4767worker1 # [ 50.877659] nix-daemon[851]: accepted connection from pid 1052, user nix-grpc-daemon (trusted)4768worker1 # [ 50.895440] nix-daemon[851]: reaped child process 1055, status = succeeded4769worker1 # [ 51.070157] nix-daemon[851]: accepted connection from pid 1057, user nix-grpc-daemon (trusted)4770worker1 # [ 51.085519] nix-daemon[851]: reaped child process 1060, status = succeeded4771lb # [ 51.081446] niks3-server[1065]: 2026/09/20 21:09:05 INFO Received uploads request method=POST path=/api/pending_closures4772worker1 # [ 51.110427] nix-grpc-daemon[861]: time=2026-09-20T21:09:04.414Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4773worker1 # [ 51.112333] nix-grpc-daemon[861]: time=2026-09-20T21:09:04.414Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"4774lb # [ 51.100156] niks3-server[1065]: 2026/09/20 21:09:05 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst4775lb # [ 51.106723] niks3-server[1065]: 2026/09/20 21:09:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4776lb # [ 51.110260] niks3-server[1065]: 2026/09/20 21:09:05 INFO Signed narinfos id=11 count=14777worker1 # [ 51.131793] nix-grpc-daemon[861]: time=2026-09-20T21:09:04.436Z level=INFO msg="Uploading 1 narinfos"4778lb # [ 51.113733] niks3-server[1065]: 2026/09/20 21:09:05 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls4779lb # [ 51.119686] niks3-server[1065]: 2026/09/20 21:09:05 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4780lb # [ 51.126824] niks3-server[1065]: 2026/09/20 21:09:05 INFO Completed upload id=114781worker1 # [ 51.148525] nix-grpc-daemon[861]: time=2026-09-20T21:09:04.453Z level=INFO msg="Upload complete. (492ms)"4782worker1 # [ 51.151071] nix-grpc-daemon[791]: ts=2026-09-20T21:09:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52102 duration_s=0 paths=1 nar_bytes_in=5124783lb # [ 51.128880] niks3-server[1065]: 2026/09/20 21:09:05 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo4784worker1 # [ 51.159598] nix-grpc-daemon[791]: ts=2026-09-20T21:09:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:521024785worker1 # [ 51.164448] nix-daemon[851]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4786worker1 # [ 51.167140] nix-daemon[851]: reaped child process 1050, status = succeeded4787worker1 # [ 51.173495] nix-daemon[851]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4788worker1 # [ 52.802986] nix-daemon[851]: reaped child process 1064, status = succeeded4789worker1 # [ 52.947279] nix-daemon[851]: accepted connection from pid 1079, user nix-grpc-daemon (trusted)4790worker1 # [ 52.961226] nix-daemon[851]: reaped child process 1082, status = succeeded4791lb # [ 52.955964] niks3-server[1065]: 2026/09/20 21:09:07 INFO Received uploads request method=POST path=/api/pending_closures4792worker1 # [ 52.985694] nix-grpc-daemon[861]: time=2026-09-20T21:09:06.290Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4793worker1 # [ 52.987295] nix-grpc-daemon[861]: time=2026-09-20T21:09:06.291Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"4794lb # [ 52.987081] niks3-server[1065]: 2026/09/20 21:09:07 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv4795lb # [ 53.003287] niks3-server[1065]: 2026/09/20 21:09:07 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4796lb # [ 53.010344] niks3-server[1065]: 2026/09/20 21:09:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4797lb # [ 53.013832] niks3-server[1065]: 2026/09/20 21:09:07 INFO Signed narinfos id=12 count=14798worker1 # [ 53.037990] nix-grpc-daemon[861]: time=2026-09-20T21:09:06.342Z level=INFO msg="Uploading 1 narinfos"4799lb # [ 53.022246] niks3-server[1065]: 2026/09/20 21:09:07 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls4800lb # [ 53.030262] niks3-server[1065]: 2026/09/20 21:09:07 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4801lb # [ 53.035346] niks3-server[1065]: 2026/09/20 21:09:07 INFO Completed upload id=124802worker1 # [ 53.056998] nix-grpc-daemon[861]: time=2026-09-20T21:09:06.361Z level=INFO msg="Upload complete. (273ms)"4803lb # [ 53.036978] niks3-server[1065]: 2026/09/20 21:09:07 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo4804worker1 # [ 53.062609] nix-grpc-daemon[791]: ts=2026-09-20T21:09:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52102 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=6 outputs=14805worker1 # [ 53.072219] nix-grpc-daemon[791]: ts=2026-09-20T21:09:06Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:521024806worker1 # [ 53.081674] nix-daemon[851]: reaped child process 1062, status = succeeded4807worker1 # [ 53.084417] nix-grpc-daemon[791]: ts=2026-09-20T21:09:06Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:521024808worker1 # [ 53.089209] nix-grpc-daemon[791]: ts=2026-09-20T21:09:06Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:52102 duration_s=0 paths=1 nar_bytes_out=1284809worker2 # [ 53.326530] nix-grpc-daemon[791]: ts=2026-09-20T21:09:07Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:333504810worker1 # [ 53.345793] nix-grpc-daemon[791]: ts=2026-09-20T21:09:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:521024811worker1 # [ 53.355144] nix-grpc-daemon[861]: time=2026-09-20T21:09:06.659Z level=INFO msg="All 1 paths already cached"4812worker2 # [ 53.354250] nix-grpc-daemon[791]: ts=2026-09-20T21:09:07Z level=debug event=want client=7 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=04813worker2 # [ 53.358787] nix-grpc-daemon[791]: ts=2026-09-20T21:09:07Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.3:50051 clients=14814worker1 # [ 53.377729] nix-grpc-daemon[791]: ts=2026-09-20T21:09:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:521024815worker1 # [ 53.389784] nix-grpc-daemon[791]: ts=2026-09-20T21:09:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:521024816worker1 # [ 53.402467] nix-grpc-daemon[861]: time=2026-09-20T21:09:06.707Z level=INFO msg="All 1 paths already cached"4817worker1 # [ 53.409325] nix-grpc-daemon[791]: ts=2026-09-20T21:09:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:521024818worker1 # [ 53.417165] nix-daemon[851]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4819worker1 # [ 53.589244] nix-daemon[851]: accepted connection from pid 1086, user nix-grpc-daemon (trusted)4820worker1 # [ 53.605197] nix-daemon[851]: reaped child process 1089, status = succeeded4821worker1 # [ 53.773964] nix-daemon[851]: accepted connection from pid 1091, user nix-grpc-daemon (trusted)4822worker1 # [ 53.787205] nix-daemon[851]: reaped child process 1094, status = succeeded4823lb # [ 53.779519] niks3-server[1065]: 2026/09/20 21:09:07 INFO Received uploads request method=POST path=/api/pending_closures4824worker1 # [ 53.808658] nix-grpc-daemon[861]: time=2026-09-20T21:09:07.113Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4825worker1 # [ 53.810510] nix-grpc-daemon[861]: time=2026-09-20T21:09:07.114Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"4826lb # [ 53.809152] niks3-server[1065]: 2026/09/20 21:09:07 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst4827lb # [ 53.820524] niks3-server[1065]: 2026/09/20 21:09:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4828worker1 # [ 53.847270] nix-grpc-daemon[861]: time=2026-09-20T21:09:07.151Z level=INFO msg="Uploading 1 narinfos"4829lb # [ 53.825671] niks3-server[1065]: 2026/09/20 21:09:07 INFO Signed narinfos id=13 count=14830lb # [ 53.833113] niks3-server[1065]: 2026/09/20 21:09:07 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls4831lb # [ 53.842131] niks3-server[1065]: 2026/09/20 21:09:07 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4832worker1 # [ 53.869125] nix-grpc-daemon[861]: time=2026-09-20T21:09:07.173Z level=INFO msg="Upload complete. (436ms)"4833lb # [ 53.847683] niks3-server[1065]: 2026/09/20 21:09:07 INFO Completed upload id=134834worker1 # [ 53.872074] nix-grpc-daemon[791]: ts=2026-09-20T21:09:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52102 duration_s=0 paths=1 nar_bytes_in=6244835worker1 # [ 53.874131] nix-grpc-daemon[861]: time=2026-09-20T21:09:07.175Z level=WARN msg="Failed to register uploaded object" key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo error="server returned 404: object is not pending upload\n"4836worker1 # [ 53.876996] nix-grpc-daemon[791]: ts=2026-09-20T21:09:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:521024837worker1 # [ 53.890121] nix-daemon[851]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4838worker1 # [ 53.893409] nix-daemon[851]: reaped child process 1084, status = succeeded4839worker1 # [ 53.899145] nix-daemon[851]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4840worker1 # [ 54.923309] nix-daemon[851]: reaped child process 1098, status = succeeded4841worker1 # [ 55.073593] nix-daemon[851]: accepted connection from pid 1113, user nix-grpc-daemon (trusted)4842worker1 # [ 55.088704] nix-daemon[851]: reaped child process 1116, status = succeeded4843lb # [ 55.075981] niks3-server[1065]: 2026/09/20 21:09:09 INFO Received uploads request method=POST path=/api/pending_closures4844worker1 # [ 55.105451] nix-grpc-daemon[861]: time=2026-09-20T21:09:08.409Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4845worker1 # [ 55.106959] nix-grpc-daemon[861]: time=2026-09-20T21:09:08.411Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"4846lb # [ 55.102859] niks3-server[1065]: 2026/09/20 21:09:09 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv4847lb # [ 55.119841] niks3-server[1065]: 2026/09/20 21:09:09 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst4848lb # [ 55.129059] niks3-server[1065]: 2026/09/20 21:09:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4849lb # [ 55.133798] niks3-server[1065]: 2026/09/20 21:09:09 INFO Signed narinfos id=14 count=14850worker1 # [ 55.157970] nix-grpc-daemon[861]: time=2026-09-20T21:09:08.462Z level=INFO msg="Uploading 1 narinfos"4851lb # [ 55.139080] niks3-server[1065]: 2026/09/20 21:09:09 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls4852lb # [ 55.145415] niks3-server[1065]: 2026/09/20 21:09:09 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4853lb # [ 55.148310] niks3-server[1065]: 2026/09/20 21:09:09 INFO Completed upload id=144854worker1 # [ 55.170144] nix-grpc-daemon[861]: time=2026-09-20T21:09:08.474Z level=INFO msg="Upload complete. (262ms)"4855lb # [ 55.150135] niks3-server[1065]: 2026/09/20 21:09:09 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo4856worker1 # [ 55.175390] nix-grpc-daemon[791]: ts=2026-09-20T21:09:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52102 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=7 outputs=14857worker1 # [ 55.182281] nix-grpc-daemon[791]: ts=2026-09-20T21:09:08Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:521024858worker1 # [ 55.191300] nix-grpc-daemon[791]: ts=2026-09-20T21:09:08Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:521024859worker1 # [ 55.193045] nix-grpc-daemon[791]: ts=2026-09-20T21:09:08Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:52102 duration_s=0 paths=1 nar_bytes_out=1764860worker1 # [ 55.195596] nix-daemon[851]: reaped child process 1096, status = succeeded4861worker1 # [ 55.447197] nix-grpc-daemon[791]: ts=2026-09-20T21:09:08Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:521024862worker1 # [ 55.451862] nix-grpc-daemon[791]: ts=2026-09-20T21:09:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:521024863worker1 # [ 55.457517] nix-grpc-daemon[861]: time=2026-09-20T21:09:08.762Z level=INFO msg="All 2 paths already cached"4864worker2 # [ 55.454506] nix-grpc-daemon[791]: ts=2026-09-20T21:09:09Z level=debug event=want client=8 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=04865worker2 # [ 55.459960] nix-grpc-daemon[791]: ts=2026-09-20T21:09:09Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.3:50051 clients=14866worker1 # [ 55.478454] nix-grpc-daemon[791]: ts=2026-09-20T21:09:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:521024867worker1 # [ 55.490178] nix-grpc-daemon[791]: ts=2026-09-20T21:09:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:521024868worker1 # [ 55.502316] nix-grpc-daemon[861]: time=2026-09-20T21:09:08.806Z level=INFO msg="All 2 paths already cached"4869worker1 # [ 55.510195] nix-grpc-daemon[791]: ts=2026-09-20T21:09:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:521024870worker1 # [ 55.522204] nix-daemon[851]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4871worker1 # [ 55.697494] nix-daemon[851]: accepted connection from pid 1120, user nix-grpc-daemon (trusted)4872worker1 # [ 55.710994] nix-daemon[851]: reaped child process 1123, status = succeeded4873worker1 # [ 55.896532] nix-daemon[851]: accepted connection from pid 1125, user nix-grpc-daemon (trusted)4874worker1 # [ 55.910512] nix-daemon[851]: reaped child process 1128, status = succeeded4875lb # [ 55.904970] niks3-server[1065]: 2026/09/20 21:09:09 INFO Received uploads request method=POST path=/api/pending_closures4876worker1 # [ 55.934924] nix-grpc-daemon[861]: time=2026-09-20T21:09:09.239Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4877worker1 # [ 55.936364] nix-grpc-daemon[861]: time=2026-09-20T21:09:09.239Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"4878lb # [ 55.932533] niks3-server[1065]: 2026/09/20 21:09:09 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst4879lb # [ 55.943229] niks3-server[1065]: 2026/09/20 21:09:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4880worker1 # [ 55.970123] nix-grpc-daemon[861]: time=2026-09-20T21:09:09.274Z level=INFO msg="Uploading 1 narinfos"4881lb # [ 55.948143] niks3-server[1065]: 2026/09/20 21:09:10 INFO Signed narinfos id=15 count=14882lb # [ 55.956178] niks3-server[1065]: 2026/09/20 21:09:10 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls4883lb # [ 55.966549] niks3-server[1065]: 2026/09/20 21:09:10 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4884lb # [ 55.972575] niks3-server[1065]: 2026/09/20 21:09:10 INFO Completed upload id=154885worker1 # [ 55.996660] nix-grpc-daemon[861]: time=2026-09-20T21:09:09.301Z level=INFO msg="Upload complete. (462ms)"4886lb # [ 55.975770] niks3-server[1065]: 2026/09/20 21:09:10 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo4887worker1 # [ 55.999202] nix-grpc-daemon[791]: ts=2026-09-20T21:09:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52102 duration_s=0 paths=1 nar_bytes_in=7524888worker1 # [ 56.004489] nix-grpc-daemon[791]: ts=2026-09-20T21:09:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:521024889worker1 # [ 56.012381] nix-daemon[851]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4890worker1 # [ 56.020183] nix-daemon[851]: reaped child process 1118, status = succeeded4891worker1 # [ 56.023409] nix-daemon[851]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4892worker1 # [ 56.837623] nix-daemon[851]: reaped child process 1132, status = succeeded4893worker1 # [ 56.997114] nix-daemon[851]: accepted connection from pid 1147, user nix-grpc-daemon (trusted)4894worker1 # [ 57.009502] nix-daemon[851]: reaped child process 1150, status = succeeded4895lb # [ 57.007272] niks3-server[1065]: 2026/09/20 21:09:11 INFO Received uploads request method=POST path=/api/pending_closures4896worker1 # [ 57.035660] nix-grpc-daemon[861]: time=2026-09-20T21:09:10.340Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4897worker1 # [ 57.037267] nix-grpc-daemon[861]: time=2026-09-20T21:09:10.341Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"4898lb # [ 57.045622] niks3-server[1065]: 2026/09/20 21:09:11 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv4899lb # [ 57.064227] niks3-server[1065]: 2026/09/20 21:09:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4900lb # [ 57.069233] niks3-server[1065]: 2026/09/20 21:09:11 INFO Signed narinfos id=16 count=14901worker1 # [ 57.094071] nix-grpc-daemon[861]: time=2026-09-20T21:09:10.398Z level=INFO msg="Uploading 1 narinfos"4902lb # [ 57.074757] niks3-server[1065]: 2026/09/20 21:09:11 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst4903lb # [ 57.083978] niks3-server[1065]: 2026/09/20 21:09:11 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls4904lb # [ 57.091293] niks3-server[1065]: 2026/09/20 21:09:11 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4905lb # [ 57.095407] niks3-server[1065]: 2026/09/20 21:09:11 INFO Completed upload id=164906lb # [ 57.097152] niks3-server[1065]: 2026/09/20 21:09:11 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo4907worker1 # [ 57.118547] nix-grpc-daemon[861]: time=2026-09-20T21:09:10.423Z level=INFO msg="Upload complete. (288ms)"4908worker1 # [ 57.124332] nix-grpc-daemon[791]: ts=2026-09-20T21:09:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52102 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=8 outputs=14909worker1 # [ 57.132278] nix-grpc-daemon[791]: ts=2026-09-20T21:09:10Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:521024910worker1 # [ 57.143401] nix-daemon[851]: reaped child process 1130, status = succeeded4911worker1 # [ 57.146128] nix-grpc-daemon[791]: ts=2026-09-20T21:09:10Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:521024912worker1 # [ 57.149942] nix-grpc-daemon[791]: ts=2026-09-20T21:09:10Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:52102 duration_s=0 paths=1 nar_bytes_out=2324913client # [ 57.251715] nix-daemon[951]: reaped child process 953, status = succeeded4914client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.31 seconds)4915client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top4916client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.03 seconds)4917client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4918client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.04 seconds)4919client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4920client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4921(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.41 seconds)4922subtest: an input only one worker has still reaches the builder4923client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only4924client # this derivation will be built:4925client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv4926client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...4927client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only, in 0.42 seconds)4928client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure4929client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.20 seconds)4930worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4931worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.24 seconds)4932worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only4933worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.03 seconds)4934client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&24935client # [ 58.465475] nix-daemon[951]: accepted connection from pid 1033, user root (trusted)4936client # this derivation will be built:4937client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv4938lb # [ 58.627777] envoy[973]: conn peer=[2001:db8:1::1]:54288 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1440 tx=1085 ms=48484939lb # [ 58.631790] envoy[973]: conn peer=[2001:db8:1::1]:54272 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2972 tx=1373 ms=60124940lb # [ 58.635442] envoy[973]: conn peer=[2001:db8:1::1]:58886 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=544941lb # [ 58.638931] envoy[973]: conn peer=[2001:db8:1::1]:54296 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3408 tx=1635 ms=25824942lb # [ 58.642502] envoy[973]: conn peer=[2001:db8:1::1]:58892 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3837 tx=1837 ms=19004943lb # [ 58.646123] envoy[973]: conn peer=[2001:db8:1::1]:58902 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=378 ms=494944lb # [ 58.649569] envoy[973]: conn peer=[2001:db8:1::1]:58928 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=554945lb # [ 58.653263] envoy[973]: conn peer=[2001:db8:1::1]:58918 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4314 tx=2084 ms=17544946client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux'...4947worker2 # [ 58.705531] nix-grpc-daemon[791]: ts=2026-09-20T21:09:12Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:333504948worker2 # [ 58.711551] nix-grpc-daemon[791]: ts=2026-09-20T21:09:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:333504949client # copying 1 paths...4950client # copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' to 'grpc://lb:50051'...4951worker2 # [ 58.722991] nix-grpc-daemon[791]: ts=2026-09-20T21:09:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:333504952worker2 # [ 58.735334] nix-daemon[871]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4953worker2 # [ 58.912444] nix-daemon[871]: accepted connection from pid 1008, user nix-grpc-daemon (trusted)4954worker2 # [ 58.926270] nix-daemon[871]: reaped child process 1011, status = succeeded4955lb # [ 58.938511] niks3-server[1065]: 2026/09/20 21:09:13 INFO Received uploads request method=POST path=/api/pending_closures4956worker2 # [ 58.951329] nix-grpc-daemon[922]: time=2026-09-20T21:09:12.706Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4957worker2 # [ 58.953422] nix-grpc-daemon[922]: time=2026-09-20T21:09:12.707Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"4958lb # [ 58.968676] niks3-server[1065]: 2026/09/20 21:09:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4959lb # [ 58.974371] niks3-server[1065]: 2026/09/20 21:09:13 INFO Signed narinfos id=17 count=14960worker2 # [ 58.982561] nix-grpc-daemon[922]: time=2026-09-20T21:09:12.737Z level=INFO msg="Uploading 1 narinfos"4961lb # [ 58.979701] niks3-server[1065]: 2026/09/20 21:09:13 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4962lb # [ 58.995340] niks3-server[1065]: 2026/09/20 21:09:13 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4963lb # [ 59.001883] niks3-server[1065]: 2026/09/20 21:09:13 INFO Completed upload id=174964worker2 # [ 59.009412] nix-grpc-daemon[922]: time=2026-09-20T21:09:12.764Z level=INFO msg="Upload complete. (264ms)"4965lb # [ 59.004790] niks3-server[1065]: 2026/09/20 21:09:13 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls4966worker2 # [ 59.011381] nix-grpc-daemon[791]: ts=2026-09-20T21:09:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33350 duration_s=0 paths=1 nar_bytes_in=1204967lb # [ 59.010803] niks3-server[1065]: 2026/09/20 21:09:13 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo4968worker2 # [ 59.018184] nix-daemon[871]: reaped child process 1006, status = succeeded4969worker2 # [ 59.026233] nix-grpc-daemon[791]: ts=2026-09-20T21:09:12Z level=debug event=want client=9 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=04970worker2 # [ 59.028374] nix-grpc-daemon[791]: ts=2026-09-20T21:09:12Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=14971worker2 # [ 59.072614] nix-grpc-daemon[791]: ts=2026-09-20T21:09:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333504972worker2 # [ 59.082095] nix-grpc-daemon[791]: ts=2026-09-20T21:09:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:333504973client # copying 2 paths to 'lb:50051'...4974client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...4975worker2 # [ 59.092954] nix-grpc-daemon[791]: ts=2026-09-20T21:09:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:333504976client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...4977worker2 # [ 59.100856] nix-daemon[871]: accepted connection from pid 791, user nix-grpc-daemon (trusted)4978worker2 # [ 59.284280] nix-daemon[871]: accepted connection from pid 1017, user nix-grpc-daemon (trusted)4979worker2 # [ 59.299752] nix-daemon[871]: reaped child process 1020, status = succeeded4980worker2 # [ 59.483451] nix-daemon[871]: accepted connection from pid 1022, user nix-grpc-daemon (trusted)4981worker2 # [ 59.497390] nix-daemon[871]: reaped child process 1025, status = succeeded4982lb # [ 59.505524] niks3-server[1065]: 2026/09/20 21:09:13 INFO Received uploads request method=POST path=/api/pending_closures4983lb # [ 59.518272] niks3-server[1065]: 2026/09/20 21:09:13 INFO Received uploads request method=POST path=/api/pending_closures4984worker2 # [ 59.532941] nix-grpc-daemon[922]: time=2026-09-20T21:09:13.287Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4985worker2 # [ 59.534444] nix-grpc-daemon[922]: time=2026-09-20T21:09:13.287Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"4986worker2 # [ 59.536025] nix-grpc-daemon[922]: time=2026-09-20T21:09:13.287Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"4987lb # [ 59.552433] niks3-server[1065]: 2026/09/20 21:09:13 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst4988lb # [ 59.558214] niks3-server[1065]: 2026/09/20 21:09:13 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst4989lb # [ 59.562887] niks3-server[1065]: 2026/09/20 21:09:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4990lb # [ 59.565057] niks3-server[1065]: 2026/09/20 21:09:13 INFO Signed narinfos id=18 count=14991lb # [ 59.566288] niks3-server[1065]: 2026/09/20 21:09:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4992lb # [ 59.569563] niks3-server[1065]: 2026/09/20 21:09:13 INFO Signed narinfos id=19 count=24993lb # [ 59.571113] niks3-server[1065]: 2026/09/20 21:09:13 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls4994worker2 # [ 59.575133] nix-grpc-daemon[922]: time=2026-09-20T21:09:13.329Z level=INFO msg="Uploading 3 narinfos"4995lb # [ 59.574257] niks3-server[1065]: 2026/09/20 21:09:13 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls4996lb # [ 59.588107] niks3-server[1065]: 2026/09/20 21:09:13 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4997lb # [ 59.589745] niks3-server[1065]: 2026/09/20 21:09:13 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo4998lb # [ 59.592741] niks3-server[1065]: 2026/09/20 21:09:13 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo4999lb # [ 59.594686] niks3-server[1065]: 2026/09/20 21:09:13 INFO Completed upload id=185000lb # [ 59.596327] niks3-server[1065]: 2026/09/20 21:09:13 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5001lb # [ 59.705118] niks3-server[1065]: 2026/09/20 21:09:13 INFO Completed upload id=195002worker2 # [ 59.709970] nix-grpc-daemon[922]: time=2026-09-20T21:09:13.464Z level=INFO msg="Upload complete. (601ms)"5003lb # [ 59.711328] niks3-server[1065]: 2026/09/20 21:09:13 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5004worker2 # [ 59.715783] nix-grpc-daemon[791]: ts=2026-09-20T21:09:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33350 duration_s=0 paths=2 nar_bytes_in=9765005worker2 # [ 59.722847] nix-grpc-daemon[791]: ts=2026-09-20T21:09:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333505006worker2 # [ 59.737122] nix-daemon[871]: accepted connection from pid 791, user nix-grpc-daemon (trusted)5007worker2 # [ 59.745823] nix-daemon[871]: reaped child process 1015, status = succeeded5008client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5009worker2 # [ 59.757868] nix-daemon[871]: accepted connection from pid 791, user nix-grpc-daemon (trusted)5010client # uses-input-w1onlya> worker2: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5011worker2 # [ 60.988620] nix-daemon[871]: reaped child process 1029, status = succeeded5012worker2 # [ 61.118335] nix-daemon[871]: accepted connection from pid 1044, user nix-grpc-daemon (trusted)5013worker2 # [ 61.129358] nix-daemon[871]: reaped child process 1047, status = succeeded5014lb # [ 61.134463] niks3-server[1065]: 2026/09/20 21:09:15 INFO Received uploads request method=POST path=/api/pending_closures5015worker2 # [ 61.147844] nix-grpc-daemon[922]: time=2026-09-20T21:09:14.902Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5016lb # [ 61.166242] niks3-server[1065]: 2026/09/20 21:09:15 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5017lb # [ 61.173621] niks3-server[1065]: 2026/09/20 21:09:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5018lb # [ 61.179600] niks3-server[1065]: 2026/09/20 21:09:15 INFO Signed narinfos id=20 count=15019worker2 # [ 61.187685] nix-grpc-daemon[922]: time=2026-09-20T21:09:14.942Z level=INFO msg="Uploading 1 narinfos"5020lb # [ 61.184600] niks3-server[1065]: 2026/09/20 21:09:15 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5021lb # [ 61.198528] niks3-server[1065]: 2026/09/20 21:09:15 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5022lb # [ 61.203915] niks3-server[1065]: 2026/09/20 21:09:15 INFO Completed upload id=205023worker2 # [ 61.208853] nix-grpc-daemon[922]: time=2026-09-20T21:09:14.963Z level=INFO msg="Upload complete. (237ms)"5024lb # [ 61.206227] niks3-server[1065]: 2026/09/20 21:09:15 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5025worker2 # [ 61.214467] nix-grpc-daemon[791]: ts=2026-09-20T21:09:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33350 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=9 outputs=15026worker2 # [ 61.225710] nix-daemon[871]: reaped child process 1027, status = succeeded5027client # copying 1 paths...5028worker1 # [ 61.278634] nix-grpc-daemon[791]: ts=2026-09-20T21:09:14Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:521025029worker1 # [ 61.301316] nix-grpc-daemon[791]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5030client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5031worker2 # [ 61.329210] nix-grpc-daemon[791]: ts=2026-09-20T21:09:15Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:333505032worker2 # [ 61.335144] nix-grpc-daemon[791]: ts=2026-09-20T21:09:15Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:33350 duration_s=0 paths=1 nar_bytes_out=1205033client # [ 61.446872] nix-daemon[951]: reaped child process 1036, status = succeeded5034client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&2, in 3.21 seconds)5035client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&25036client # [ 61.633083] nix-daemon[951]: accepted connection from pid 1055, user root (trusted)5037client # this derivation will be built:5038client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5039worker2 # [ 61.834642] nix-grpc-daemon[791]: ts=2026-09-20T21:09:15Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:333505040client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux'...5041worker1 # [ 61.855691] nix-grpc-daemon[791]: ts=2026-09-20T21:09:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:521025042worker1 # [ 61.868610] nix-grpc-daemon[861]: time=2026-09-20T21:09:15.172Z level=INFO msg="All 1 paths already cached"5043worker2 # [ 61.866907] nix-grpc-daemon[791]: ts=2026-09-20T21:09:15Z level=debug event=want client=10 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05044worker2 # [ 61.871777] nix-grpc-daemon[791]: ts=2026-09-20T21:09:15Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15045worker1 # [ 61.890874] nix-grpc-daemon[791]: ts=2026-09-20T21:09:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:521025046worker1 # [ 61.903607] nix-grpc-daemon[791]: ts=2026-09-20T21:09:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:521025047client # copying 1 paths to 'lb:50051'...5048client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5049worker1 # [ 61.921945] nix-grpc-daemon[791]: ts=2026-09-20T21:09:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:521025050worker1 # [ 61.935724] nix-daemon[851]: accepted connection from pid 791, user nix-grpc-daemon (trusted)5051worker1 # [ 61.956312] nix-grpc-daemon[791]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 14 ms (attempt 1/5)5052worker1 # [ 61.975205] nix-grpc-daemon[791]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 161 ms (attempt 2/5)5053worker1 # [ 62.142788] nix-grpc-daemon[791]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 111 ms (attempt 3/5)5054worker1 # [ 62.259723] nix-grpc-daemon[791]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 107 ms (attempt 4/5)5055worker1 # [ 62.373979] nix-grpc-daemon[791]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5056worker1 # [ 62.380352] nix-grpc-daemon[791]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5057worker1 # [ 62.581696] nix-daemon[851]: accepted connection from pid 1164, user nix-grpc-daemon (trusted)5058worker1 # [ 62.597690] nix-daemon[851]: reaped child process 1167, status = succeeded5059worker1 # [ 62.779510] nix-daemon[851]: accepted connection from pid 1169, user nix-grpc-daemon (trusted)5060worker1 # [ 62.793597] nix-daemon[851]: reaped child process 1172, status = succeeded5061lb # [ 62.788802] niks3-server[1065]: 2026/09/20 21:09:16 INFO Received uploads request method=POST path=/api/pending_closures5062worker1 # [ 62.820445] nix-grpc-daemon[861]: time=2026-09-20T21:09:16.125Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5063worker1 # [ 62.822773] nix-grpc-daemon[861]: time=2026-09-20T21:09:16.126Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5064lb # [ 62.823596] niks3-server[1065]: 2026/09/20 21:09:16 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5065lb # [ 62.830108] niks3-server[1065]: 2026/09/20 21:09:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5066lb # [ 62.835647] niks3-server[1065]: 2026/09/20 21:09:16 INFO Signed narinfos id=21 count=15067worker1 # [ 62.860587] nix-grpc-daemon[861]: time=2026-09-20T21:09:16.164Z level=INFO msg="Uploading 1 narinfos"5068lb # [ 62.842621] niks3-server[1065]: 2026/09/20 21:09:16 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5069lb # [ 62.851959] niks3-server[1065]: 2026/09/20 21:09:16 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5070lb # [ 62.859883] niks3-server[1065]: 2026/09/20 21:09:16 INFO Completed upload id=215071worker1 # [ 62.881951] nix-grpc-daemon[861]: time=2026-09-20T21:09:16.186Z level=INFO msg="Upload complete. (479ms)"5072lb # [ 62.862864] niks3-server[1065]: 2026/09/20 21:09:16 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5073worker1 # [ 62.885635] nix-grpc-daemon[791]: ts=2026-09-20T21:09:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52102 duration_s=0 paths=1 nar_bytes_in=5605074worker1 # [ 62.890604] nix-grpc-daemon[791]: ts=2026-09-20T21:09:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:521025075worker1 # [ 62.901474] nix-daemon[851]: accepted connection from pid 791, user nix-grpc-daemon (trusted)5076worker1 # [ 62.908247] nix-daemon[851]: reaped child process 1158, status = succeeded5077client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5078client # uses-input-w1onlyb> node-a: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5079worker1 # [ 62.910910] nix-daemon[851]: accepted connection from pid 791, user nix-grpc-daemon (trusted)5080worker1 # [ 64.127075] nix-daemon[851]: reaped child process 1176, status = succeeded5081worker1 # [ 64.269749] nix-daemon[851]: accepted connection from pid 1191, user nix-grpc-daemon (trusted)5082worker1 # [ 64.284525] nix-daemon[851]: reaped child process 1194, status = succeeded5083lb # [ 64.273491] niks3-server[1065]: 2026/09/20 21:09:18 INFO Received uploads request method=POST path=/api/pending_closures5084worker1 # [ 64.300131] nix-grpc-daemon[861]: time=2026-09-20T21:09:17.604Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5085lb # [ 64.290863] niks3-server[1065]: 2026/09/20 21:09:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5086lb # [ 64.292711] niks3-server[1065]: 2026/09/20 21:09:18 INFO Signed narinfos id=22 count=15087worker1 # [ 64.315872] nix-grpc-daemon[861]: time=2026-09-20T21:09:17.620Z level=INFO msg="Uploading 1 narinfos"5088lb # [ 64.296687] niks3-server[1065]: 2026/09/20 21:09:18 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5089lb # [ 64.299352] niks3-server[1065]: 2026/09/20 21:09:18 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5090lb # [ 64.304300] niks3-server[1065]: 2026/09/20 21:09:18 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5091lb # [ 64.306962] niks3-server[1065]: 2026/09/20 21:09:18 INFO Completed upload id=225092worker1 # [ 64.328560] nix-grpc-daemon[861]: time=2026-09-20T21:09:17.633Z level=INFO msg="Upload complete. (210ms)"5093client # copying 1 paths...5094worker1 # [ 64.331249] nix-grpc-daemon[791]: ts=2026-09-20T21:09:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52102 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=10 outputs=15095lb # [ 64.311277] niks3-server[1065]: 2026/09/20 21:09:18 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5096worker2 # [ 64.319433] nix-grpc-daemon[791]: ts=2026-09-20T21:09:18Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:333505097worker1 # [ 64.338546] nix-daemon[851]: reaped child process 1174, status = succeeded5098worker2 # [ 64.326803] nix-grpc-daemon[791]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5099client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5100worker2 # [ 64.347836] nix-grpc-daemon[791]: ts=2026-09-20T21:09:18Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:333505101worker2 # [ 64.350068] nix-grpc-daemon[791]: ts=2026-09-20T21:09:18Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:33350 duration_s=0 paths=1 nar_bytes_out=1205102client # [ 64.448304] nix-daemon[951]: reaped child process 1058, status = succeeded5103client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.99 seconds)5104client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&25105client # [ 64.594726] nix-daemon[951]: accepted connection from pid 1077, user root (trusted)5106client # this derivation will be built:5107client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5108worker1 # [ 64.753241] nix-grpc-daemon[791]: ts=2026-09-20T21:09:18Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:521025109client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux'...5110worker2 # [ 64.739059] nix-grpc-daemon[791]: ts=2026-09-20T21:09:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:333505111worker2 # [ 64.747870] nix-grpc-daemon[922]: time=2026-09-20T21:09:18.502Z level=INFO msg="All 1 paths already cached"5112worker2 # [ 64.759973] nix-grpc-daemon[791]: ts=2026-09-20T21:09:18Z level=debug event=want client=11 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05113worker2 # [ 64.765438] nix-grpc-daemon[791]: ts=2026-09-20T21:09:18Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15114worker2 # [ 64.804255] nix-grpc-daemon[791]: ts=2026-09-20T21:09:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333505115worker2 # [ 64.814455] nix-grpc-daemon[791]: ts=2026-09-20T21:09:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:333505116worker2 # [ 64.829476] nix-grpc-daemon[922]: time=2026-09-20T21:09:18.584Z level=INFO msg="All 1 paths already cached"5117client # copying 1 paths to 'lb:50051'...5118worker2 # [ 64.836961] nix-grpc-daemon[791]: ts=2026-09-20T21:09:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:333505119client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5120worker2 # [ 64.844413] nix-daemon[871]: accepted connection from pid 791, user nix-grpc-daemon (trusted)5121worker2 # [ 65.005107] nix-daemon[871]: accepted connection from pid 1056, user nix-grpc-daemon (trusted)5122worker2 # [ 65.019243] nix-daemon[871]: reaped child process 1059, status = succeeded5123worker2 # [ 65.181347] nix-daemon[871]: accepted connection from pid 1061, user nix-grpc-daemon (trusted)5124worker2 # [ 65.194148] nix-daemon[871]: reaped child process 1064, status = succeeded5125lb # [ 65.205250] niks3-server[1065]: 2026/09/20 21:09:19 INFO Received uploads request method=POST path=/api/pending_closures5126worker2 # [ 65.260567] nix-grpc-daemon[922]: time=2026-09-20T21:09:19.015Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5127worker2 # [ 65.262068] nix-grpc-daemon[922]: time=2026-09-20T21:09:19.015Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5128lb # [ 65.280480] niks3-server[1065]: 2026/09/20 21:09:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5129worker2 # [ 65.291668] nix-grpc-daemon[922]: time=2026-09-20T21:09:19.046Z level=INFO msg="Uploading 1 narinfos"5130lb # [ 65.286925] niks3-server[1065]: 2026/09/20 21:09:19 INFO Signed narinfos id=23 count=15131lb # [ 65.296958] niks3-server[1065]: 2026/09/20 21:09:19 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5132lb # [ 65.307359] niks3-server[1065]: 2026/09/20 21:09:19 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5133lb # [ 65.312504] niks3-server[1065]: 2026/09/20 21:09:19 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5134worker2 # [ 65.330375] nix-grpc-daemon[922]: time=2026-09-20T21:09:19.085Z level=INFO msg="Upload complete. (479ms)"5135lb # [ 65.325420] niks3-server[1065]: 2026/09/20 21:09:19 INFO Completed upload id=235136worker2 # [ 65.332937] nix-grpc-daemon[791]: ts=2026-09-20T21:09:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33350 duration_s=0 paths=1 nar_bytes_in=5605137worker2 # [ 65.336446] nix-grpc-daemon[791]: ts=2026-09-20T21:09:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333505138lb # [ 65.332304] niks3-server[1065]: 2026/09/20 21:09:19 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5139worker2 # [ 65.341845] nix-daemon[871]: accepted connection from pid 791, user nix-grpc-daemon (trusted)5140worker2 # [ 65.347058] nix-daemon[871]: reaped child process 1054, status = succeeded5141client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5142client # uses-input-w1onlyc> worker2: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5143worker2 # [ 65.351994] nix-daemon[871]: accepted connection from pid 791, user nix-grpc-daemon (trusted)5144worker2 # [ 66.364516] nix-daemon[871]: reaped child process 1068, status = succeeded5145worker2 # [ 66.519208] nix-daemon[871]: accepted connection from pid 1083, user nix-grpc-daemon (trusted)5146worker2 # [ 66.532703] nix-daemon[871]: reaped child process 1086, status = succeeded5147lb # [ 66.546325] niks3-server[1065]: 2026/09/20 21:09:20 INFO Received uploads request method=POST path=/api/pending_closures5148worker2 # [ 66.560391] nix-grpc-daemon[922]: time=2026-09-20T21:09:20.315Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5149lb # [ 66.577167] niks3-server[1065]: 2026/09/20 21:09:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5150worker2 # [ 66.588037] nix-grpc-daemon[922]: time=2026-09-20T21:09:20.343Z level=INFO msg="Uploading 1 narinfos"5151lb # [ 66.584315] niks3-server[1065]: 2026/09/20 21:09:20 INFO Signed narinfos id=24 count=15152lb # [ 66.589568] niks3-server[1065]: 2026/09/20 21:09:20 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5153lb # [ 66.595131] niks3-server[1065]: 2026/09/20 21:09:20 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5154lb # [ 66.608887] niks3-server[1065]: 2026/09/20 21:09:20 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5155lb # [ 66.613915] niks3-server[1065]: 2026/09/20 21:09:20 INFO Completed upload id=245156worker2 # [ 66.621480] nix-grpc-daemon[922]: time=2026-09-20T21:09:20.376Z level=INFO msg="Upload complete. (263ms)"5157lb # [ 66.616690] niks3-server[1065]: 2026/09/20 21:09:20 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5158worker2 # [ 66.626741] nix-grpc-daemon[791]: ts=2026-09-20T21:09:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33350 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=11 outputs=15159worker2 # [ 66.638284] nix-daemon[871]: reaped child process 1066, status = succeeded5160client # copying 1 paths...5161worker2 # [ 66.672155] nix-grpc-daemon[791]: ts=2026-09-20T21:09:20Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:333505162client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5163worker1 # [ 66.709575] nix-grpc-daemon[791]: ts=2026-09-20T21:09:20Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:521025164worker1 # [ 66.721240] nix-grpc-daemon[791]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5165worker1 # [ 66.735148] nix-grpc-daemon[791]: ts=2026-09-20T21:09:20Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:52102 duration_s=0 paths=1 nar_bytes_out=1205166client # [ 66.818884] nix-daemon[951]: reaped child process 1080, status = succeeded5167client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.37 seconds)5168(finished: subtest: an input only one worker has still reaches the builder, in 9.47 seconds)5169subtest: upload whose reference the worker lacks is completed from the cache5170client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache5171client # this derivation will be built:5172client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5173client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5174client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache, in 0.35 seconds)5175client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache5176client # this derivation will be built:5177client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5178client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5179client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache, in 0.40 seconds)5180client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5181client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.19 seconds)5182worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5183worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.24 seconds)5184worker2 # [ 68.056990] systemd[1]: Stopping Nix worker-protocol over gRPC...5185worker2 # [ 68.061611] nix-grpc-daemon[791]: ts=2026-09-20T21:09:21Z level=info event=signal_exit5186worker2 # [ 68.298854] nix-grpc-daemon[791]: ts=2026-09-20T21:09:22Z level=info event=scheduler_yield5187worker2 # [ 68.304359] nix-grpc-daemon[791]: ts=2026-09-20T21:09:22Z level=info event=scheduler_restarting addr=lb:500515188lb # [ 68.300373] niks3-server[1065]: 2026/09/20 21:09:22 INFO lead: released remote=192.168.1.4:566685189worker2 # [ 68.311717] nix-grpc-daemon[791]: ts=2026-09-20T21:09:22Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:333485190worker1 # [ 68.329314] nix-grpc-daemon[791]: ts=2026-09-20T21:09:21Z level=info event=scheduler_restarting addr=lb:500515191worker2 # [ 68.316672] nix-grpc-daemon[791]: ts=2026-09-20T21:09:22Z level=info event=worker_gone addr=192.168.1.4:500515192worker2 # [ 68.319775] nix-grpc-daemon[791]: ts=2026-09-20T21:09:22Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:333485193worker2 # [ 68.323084] nix-grpc-daemon[791]: ts=2026-09-20T21:09:22Z level=info event=worker_gone addr=192.168.1.3:500515194worker2 # [ 68.505403] nix-grpc-daemon[791]: ts=2026-09-20T21:09:22Z level=info event=scheduler_connected addr=lb:500515195worker2 # [ 68.509289] nix-grpc-daemon[791]: ts=2026-09-20T21:09:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5196worker1 # [ 68.529634] nix-grpc-daemon[791]: ts=2026-09-20T21:09:21Z level=info event=scheduler_connected addr=lb:500515197worker1 # [ 68.533639] nix-grpc-daemon[791]: ts=2026-09-20T21:09:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5198lb # [ 68.627542] envoy[973]: conn peer=[2001:db8:1::1]:58938 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4244 tx=1883 ms=26845199lb # [ 68.631211] envoy[973]: conn peer=[2001:db8:1::1]:58946 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=635200lb # [ 68.634594] envoy[973]: conn peer=[2001:db8:1::1]:58952 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3697 tx=1910 ms=25665201lb # [ 68.638102] envoy[973]: conn peer=[2001:db8:1::1]:51278 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=495202lb # [ 68.641695] envoy[973]: conn peer=[2001:db8:1::1]:51308 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=329 ms=805203lb # [ 68.645147] envoy[973]: conn peer=[2001:db8:1::1]:51294 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3699 tx=1803 ms=20335204worker2 # [ 68.710556] nix-grpc-daemon[791]: ts=2026-09-20T21:09:22Z level=info event=scheduler_connected addr=lb:500515205worker2 # [ 68.714484] nix-grpc-daemon[791]: ts=2026-09-20T21:09:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5206worker1 # [ 68.734538] nix-grpc-daemon[791]: ts=2026-09-20T21:09:22Z level=info event=scheduler_connected addr=lb:500515207worker1 # [ 68.738372] nix-grpc-daemon[791]: ts=2026-09-20T21:09:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5208worker2 # [ 68.915567] nix-grpc-daemon[791]: ts=2026-09-20T21:09:22Z level=info event=scheduler_connected addr=lb:500515209worker2 # [ 68.919757] nix-grpc-daemon[791]: ts=2026-09-20T21:09:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5210worker1 # [ 68.939470] nix-grpc-daemon[791]: ts=2026-09-20T21:09:22Z level=info event=scheduler_connected addr=lb:500515211worker1 # [ 68.944125] nix-grpc-daemon[791]: ts=2026-09-20T21:09:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5212worker2 # [ 69.122120] nix-grpc-daemon[791]: ts=2026-09-20T21:09:22Z level=info event=scheduler_connected addr=lb:500515213worker2 # [ 69.125928] nix-grpc-daemon[791]: ts=2026-09-20T21:09:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5214worker1 # [ 69.144760] nix-grpc-daemon[791]: ts=2026-09-20T21:09:22Z level=info event=scheduler_connected addr=lb:500515215worker1 # [ 69.148553] nix-grpc-daemon[791]: ts=2026-09-20T21:09:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5216worker2 # [ 69.303849] nix-grpc-daemon[791]: ts=2026-09-20T21:09:23Z level=info event=scheduler_restarting5217worker1 # [ 69.350904] nix-grpc-daemon[791]: ts=2026-09-20T21:09:22Z level=info event=scheduler_connected addr=lb:500515218worker2 # [ 69.337888] nix-grpc-daemon[791]: ts=2026-09-20T21:09:23Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5219worker1 # [ 69.354659] nix-grpc-daemon[791]: ts=2026-09-20T21:09:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5220worker2 # [ 69.346735] nix-daemon[871]: reaped child process 873, status = succeeded5221worker1 # [ 69.551449] nix-grpc-daemon[791]: ts=2026-09-20T21:09:22Z level=info event=scheduler_connected addr=lb:500515222worker1 # [ 69.553503] nix-grpc-daemon[791]: ts=2026-09-20T21:09:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5223worker1 # [ 69.754160] nix-grpc-daemon[791]: ts=2026-09-20T21:09:23Z level=info event=scheduler_connected addr=lb:500515224worker1 # [ 69.756290] nix-grpc-daemon[791]: ts=2026-09-20T21:09:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5225worker1 # [ 69.957220] nix-grpc-daemon[791]: ts=2026-09-20T21:09:23Z level=info event=scheduler_connected addr=lb:500515226worker1 # [ 69.960907] nix-grpc-daemon[791]: ts=2026-09-20T21:09:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5227worker2 # [ 69.998098] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5228worker2 # [ 70.000274] systemd[1]: Stopped Nix worker-protocol over gRPC.5229worker2 # [ 70.002500] systemd[1]: nix-grpc-daemon.service: Consumed 1.492s CPU time over 51.237s wall clock time, 52.9M memory peak, 56.8K incoming IP traffic, 157.7K outgoing IP traffic.5230worker2 # [ 70.008700] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5231worker2 # [ 70.010658] systemd[1]: Closed Nix worker-protocol over gRPC.5232worker2 # [ 70.011889] systemd[1]: nix-grpc-daemon.socket: Consumed 308us CPU time over 59.276s wall clock time, 256K memory peak, 70.3K incoming IP traffic, 60.1K outgoing IP traffic.5233lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5234lb: must succeed: curl -sf localhost:9901/clusters5235lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5236lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.07 seconds)5237client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25238worker1 # [ 70.161683] nix-grpc-daemon[791]: ts=2026-09-20T21:09:23Z level=info event=scheduler_connected addr=lb:500515239worker1 # [ 70.165169] nix-grpc-daemon[791]: ts=2026-09-20T21:09:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5240lb # [ 70.184146] niks3-server[1065]: 2026/09/20 21:09:24 INFO lead: acquired remote=192.168.1.3:365185241worker1 # [ 70.209511] nix-grpc-daemon[791]: ts=2026-09-20T21:09:23Z level=info event=scheduler_take_over5242worker1 # [ 70.344486] nix-grpc-daemon[791]: ts=2026-09-20T21:09:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:521025243client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5244worker1 # [ 70.346743] nix-grpc-daemon[791]: ts=2026-09-20T21:09:23Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:521025245worker1 # [ 70.366086] nix-grpc-daemon[791]: ts=2026-09-20T21:09:23Z level=info event=scheduler_connected addr=lb:500515246client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.26 seconds)5247worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5248worker1 # [ 70.367611] nix-grpc-daemon[791]: ts=2026-09-20T21:09:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5249worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.01 seconds)5250worker2 # [ 70.391723] systemd[1]: Listening on Nix worker-protocol over gRPC.5251worker1 # [ 70.454410] systemd[1]: Stopping Nix worker-protocol over gRPC...5252worker1 # [ 70.568765] nix-grpc-daemon[791]: ts=2026-09-20T21:09:23Z level=info event=scheduler_connected addr=lb:500515253worker1 # [ 70.572628] nix-grpc-daemon[791]: ts=2026-09-20T21:09:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5254worker1 # [ 70.773584] nix-grpc-daemon[791]: ts=2026-09-20T21:09:24Z level=info event=scheduler_connected addr=lb:500515255worker1 # [ 70.777546] nix-grpc-daemon[791]: ts=2026-09-20T21:09:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5256worker1 # [ 70.969640] nix-grpc-daemon[791]: ts=2026-09-20T21:09:24Z level=info event=signal_exit5257worker1 # [ 70.978227] nix-grpc-daemon[791]: ts=2026-09-20T21:09:24Z level=info event=scheduler_connected addr=lb:500515258worker1 # [ 70.981985] nix-grpc-daemon[791]: ts=2026-09-20T21:09:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5259worker1 # [ 71.182933] nix-grpc-daemon[791]: ts=2026-09-20T21:09:24Z level=info event=scheduler_connected addr=lb:500515260worker1 # [ 71.186771] nix-grpc-daemon[791]: ts=2026-09-20T21:09:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5261worker2 # [ 71.189741] systemd[1]: Starting scheduler-token.service...5262worker1 # [ 71.227370] nix-grpc-daemon[791]: ts=2026-09-20T21:09:24Z level=info event=scheduler_yield5263lb # [ 71.207828] niks3-server[1065]: 2026/09/20 21:09:25 INFO lead: released remote=192.168.1.3:365185264worker2 # [ 71.289757] systemd[1]: scheduler-token.service: Deactivated successfully.5265worker2 # [ 71.292148] systemd[1]: Finished scheduler-token.service.5266worker2 # [ 71.292969] systemd[1]: scheduler-token.service: Consumed 38ms CPU time over 102ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5267worker2 # [ 71.298586] systemd[1]: Starting Nix worker-protocol over gRPC...5268worker1 # [ 71.387685] nix-grpc-daemon[791]: ts=2026-09-20T21:09:24Z level=info event=scheduler_connected addr=lb:500515269worker1 # [ 71.389847] nix-grpc-daemon[791]: ts=2026-09-20T21:09:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5270worker2 # [ 71.466485] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:25Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5271worker2 # [ 71.469041] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:25Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5272worker2 # [ 71.472916] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:25Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15273worker2 # [ 71.480760] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:25Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5274worker2 # [ 71.482366] systemd[1]: Started Nix worker-protocol over gRPC.5275worker2 # [ 71.493464] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:25Z level=info event=scheduler_take_over5276lb # [ 71.488745] niks3-server[1065]: 2026/09/20 21:09:25 INFO lead: acquired remote=192.168.1.4:381505277worker2 # [ 71.514254] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:25Z level=info event=scheduler_connected addr=lb:500515278worker2 # [ 71.515953] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5279worker1 # [ 71.590979] nix-grpc-daemon[791]: ts=2026-09-20T21:09:24Z level=info event=scheduler_connected addr=lb:500515280worker1 # [ 71.594902] nix-grpc-daemon[791]: ts=2026-09-20T21:09:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5281worker2 # [ 71.717661] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:25Z level=info event=scheduler_connected addr=lb:500515282worker2 # [ 71.721625] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5283worker1 # [ 71.795765] nix-grpc-daemon[791]: ts=2026-09-20T21:09:25Z level=info event=scheduler_connected addr=lb:500515284worker1 # [ 71.799636] nix-grpc-daemon[791]: ts=2026-09-20T21:09:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5285worker2 # [ 71.922844] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:25Z level=info event=scheduler_connected addr=lb:500515286worker2 # [ 71.926747] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5287worker1 # [ 72.000682] nix-grpc-daemon[791]: ts=2026-09-20T21:09:25Z level=info event=scheduler_connected addr=lb:500515288worker1 # [ 72.004816] nix-grpc-daemon[791]: ts=2026-09-20T21:09:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5289worker2 # [ 72.128061] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:25Z level=info event=scheduler_connected addr=lb:500515290worker2 # [ 72.131912] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5291worker1 # [ 72.205132] nix-grpc-daemon[791]: ts=2026-09-20T21:09:25Z level=info event=scheduler_connected addr=lb:500515292worker1 # [ 72.206913] nix-grpc-daemon[791]: ts=2026-09-20T21:09:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5293worker1 # [ 72.227791] nix-grpc-daemon[791]: ts=2026-09-20T21:09:25Z level=info event=scheduler_restarting5294worker1 # [ 72.244484] nix-daemon[851]: reaped child process 853, status = succeeded5295worker2 # [ 72.332581] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:26Z level=info event=scheduler_connected addr=lb:500515296worker2 # [ 72.334461] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5297worker1 # [ 72.424492] nix-grpc-daemon[791]: ts=2026-09-20T21:09:25Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5298worker2 # [ 72.535841] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:26Z level=info event=scheduler_connected addr=lb:500515299worker2 # [ 72.545081] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:26Z level=info event=oidc_jwks provider=mock keys=15300worker2 # [ 72.547195] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:26Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:499805301worker2 # [ 72.549147] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:26Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05302worker1 # [ 73.109485] nix-grpc-daemon[791]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5303worker1 # [ 73.111273] nix-grpc-daemon[791]: E0000 00:00:1789938566.413546 834 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true5304worker1 # [ 73.113564] nix-grpc-daemon[791]: E0000 00:00:1789938566.414120 834 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5305worker1 # [ 73.966625] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5306worker1 # [ 73.968690] systemd[1]: Stopped Nix worker-protocol over gRPC.5307worker1 # [ 73.970284] systemd[1]: nix-grpc-daemon.service: Consumed 2.694s CPU time over 55.326s wall clock time, 57.8M memory peak, 83K incoming IP traffic, 155.7K outgoing IP traffic.5308worker1 # [ 73.975072] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5309worker1 # [ 73.976730] systemd[1]: Closed Nix worker-protocol over gRPC.5310worker1 # [ 73.978138] systemd[1]: nix-grpc-daemon.socket: Consumed 329us CPU time over 1min 3.757s wall clock time, 256K memory peak, 74.2K incoming IP traffic, 60.7K outgoing IP traffic.5311lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5312lb: must succeed: curl -sf localhost:9901/clusters5313lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5314lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.08 seconds)5315client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25316worker2 # [ 74.290860] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334625317worker2 # [ 74.301609] nix-daemon[871]: accepted connection from pid 1110, user nix-grpc-daemon (trusted)5318worker2 # [ 74.319598] nix-grpc-daemon[1110]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 11 ms (attempt 1/5)5319worker2 # [ 74.336377] nix-grpc-daemon[1110]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 16 ms (attempt 2/5)5320worker2 # [ 74.358399] nix-grpc-daemon[1110]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 258 ms (attempt 3/5)5321worker2 # [ 74.622558] nix-grpc-daemon[1110]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 793 ms (attempt 4/5)5322worker2 # [ 75.417167] nix-grpc-daemon[1110]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5323client # copying 2 paths...5324client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5325worker2 # [ 75.442242] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334625326client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5327worker2 # [ 75.452979] nix-daemon[871]: accepted connection from pid 1110, user nix-grpc-daemon (trusted)5328worker2 # [ 75.478234] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:29Z level=info event=niks3_push_started pid=11665329worker2 # [ 75.703040] nix-grpc-daemon[1166]: time=2026-09-20T21:09:29.457Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.key ca=""5330worker2 # [ 75.894638] nix-daemon[871]: accepted connection from pid 1179, user nix-grpc-daemon (trusted)5331worker2 # [ 75.906252] nix-daemon[871]: reaped child process 1182, status = succeeded5332lb # [ 75.920423] niks3-server[1065]: 2026/09/20 21:09:29 INFO Received uploads request method=POST path=/api/pending_closures5333lb # [ 75.931998] niks3-server[1065]: 2026/09/20 21:09:29 INFO Received uploads request method=POST path=/api/pending_closures5334worker2 # [ 75.944518] nix-grpc-daemon[1166]: time=2026-09-20T21:09:29.699Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5335worker2 # [ 75.946128] nix-grpc-daemon[1166]: time=2026-09-20T21:09:29.701Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5336worker2 # [ 75.948134] nix-grpc-daemon[1166]: time=2026-09-20T21:09:29.703Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5337lb # [ 75.975979] niks3-server[1065]: 2026/09/20 21:09:30 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5338lb # [ 75.987718] niks3-server[1065]: 2026/09/20 21:09:30 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5339lb # [ 75.990220] niks3-server[1065]: 2026/09/20 21:09:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5340lb # [ 75.994377] niks3-server[1065]: 2026/09/20 21:09:30 INFO Signed narinfos id=25 count=15341lb # [ 75.995966] niks3-server[1065]: 2026/09/20 21:09:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5342worker2 # [ 76.004198] nix-grpc-daemon[1166]: time=2026-09-20T21:09:29.758Z level=INFO msg="Uploading 3 narinfos"5343lb # [ 76.000082] niks3-server[1065]: 2026/09/20 21:09:30 INFO Signed narinfos id=26 count=25344lb # [ 76.014342] niks3-server[1065]: 2026/09/20 21:09:30 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5345lb # [ 76.022913] niks3-server[1065]: 2026/09/20 21:09:30 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5346lb # [ 76.026698] niks3-server[1065]: 2026/09/20 21:09:30 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5347lb # [ 76.031090] niks3-server[1065]: 2026/09/20 21:09:30 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5348lb # [ 76.033163] niks3-server[1065]: 2026/09/20 21:09:30 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5349lb # [ 76.037946] niks3-server[1065]: 2026/09/20 21:09:30 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5350lb # [ 76.040453] niks3-server[1065]: 2026/09/20 21:09:30 INFO Completed upload id=255351lb # [ 76.041759] niks3-server[1065]: 2026/09/20 21:09:30 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5352lb # [ 76.044730] niks3-server[1065]: 2026/09/20 21:09:30 INFO Completed upload id=265353worker2 # [ 76.050174] nix-grpc-daemon[1166]: time=2026-09-20T21:09:29.805Z level=INFO msg="Upload complete. (342ms)"5354worker2 # [ 76.055151] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33462 duration_s=0 paths=2 nar_bytes_in=3125355worker2 # [ 76.067941] nix-daemon[871]: reaped child process 1164, status = succeeded5356client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 2.05 seconds)5357worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5358worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.03 seconds)5359worker1 # [ 76.210761] systemd[1]: Listening on Nix worker-protocol over gRPC.5360worker1 # [ 76.218345] systemd[1]: Starting Nix worker-protocol over gRPC...5361worker1 # [ 76.392849] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:29Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5362worker1 # [ 76.395295] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:29Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5363worker1 # [ 76.399533] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:29Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15364worker1 # [ 76.407087] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:29Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5365worker1 # [ 76.410344] systemd[1]: Started Nix worker-protocol over gRPC.5366lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5367lb: must succeed: curl -sf localhost:9901/clusters5368worker1 # [ 76.436099] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:29Z level=info event=scheduler_connected addr=lb:500515369worker2 # [ 76.422253] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:30Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:499805370worker2 # [ 76.426621] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:30Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05371lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5372lb: must succeed: curl -sf localhost:9901/clusters5373lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5374lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.13 seconds)5375(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.71 seconds)5376subtest: scheduler down: the other node takes the lock and keeps it5377lb: must succeed: curl -sf localhost:9901/clusters5378lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5379lb: must succeed: curl -sf localhost:9901/clusters5380lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5381lb: must succeed: curl -sf localhost:9901/clusters5382lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5383lb: must succeed: curl -sf localhost:9901/clusters5384lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5385worker2 # [ 77.752591] systemd[1]: Stopping Nix worker-protocol over gRPC...5386worker2 # [ 78.484464] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:32Z level=info event=signal_exit5387lb # [ 78.627687] envoy[973]: conn peer=[2001:db8:1::4]:40994 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=33804 tx=1787 ms=502195388lb # [ 78.631690] envoy[973]: conn peer=[2001:db8:1::1]:51314 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=519 ms=95389lb # [ 78.635149] envoy[973]: conn peer=192.168.1.3:48174 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=12862 tx=2699 ms=533825390lb # [ 78.638658] envoy[973]: conn peer=[2001:db8:1::1]:33226 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1005 tx=282 ms=17815391lb # [ 78.695531] niks3-server[1065]: 2026/09/20 21:09:32 INFO lead: released remote=192.168.1.4:381505392worker2 # [ 78.699109] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:32Z level=info event=scheduler_yield5393worker1 # [ 78.720923] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:32Z level=info event=scheduler_restarting addr=lb:500515394worker2 # [ 78.704936] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:32Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:499805395worker2 # [ 78.710643] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:32Z level=info event=worker_gone addr=192.168.1.3:500515396worker2 # [ 78.713830] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:32Z level=info event=scheduler_restarting addr=lb:500515397worker2 # [ 78.716970] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:32Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:499805398worker2 # [ 78.720472] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:32Z level=info event=worker_gone addr=192.168.1.4:500515399worker1 # [ 78.921220] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:32Z level=info event=scheduler_connected addr=lb:500515400worker1 # [ 78.925290] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5401worker2 # [ 78.911897] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:32Z level=info event=scheduler_connected addr=lb:500515402worker2 # [ 78.915329] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5403worker1 # [ 79.125642] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:32Z level=info event=scheduler_connected addr=lb:500515404worker1 # [ 79.127458] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5405worker2 # [ 79.112785] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:32Z level=info event=scheduler_connected addr=lb:500515406worker2 # [ 79.114560] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5407worker1 # [ 79.328137] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:32Z level=info event=scheduler_connected addr=lb:500515408worker1 # [ 79.329967] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5409worker2 # [ 79.315888] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:33Z level=info event=scheduler_connected addr=lb:500515410worker2 # [ 79.319879] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5411worker1 # [ 79.530888] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:32Z level=info event=scheduler_connected addr=lb:500515412worker1 # [ 79.533147] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5413worker2 # [ 79.521086] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:33Z level=info event=scheduler_connected addr=lb:500515414worker2 # [ 79.522465] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5415worker2 # [ 79.703335] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:33Z level=info event=scheduler_restarting5416worker1 # [ 79.734894] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:33Z level=info event=scheduler_connected addr=lb:500515417worker1 # [ 79.736616] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5418worker2 # [ 79.734539] nix-daemon[871]: reaped child process 1159, status = succeeded5419worker2 # [ 79.736933] nix-grpc-daemon[1110]: ts=2026-09-20T21:09:33Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5420worker1 # [ 79.936243] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:33Z level=info event=scheduler_connected addr=lb:500515421worker1 # [ 79.940312] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5422worker1 # [ 80.140724] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:33Z level=info event=scheduler_connected addr=lb:500515423worker1 # [ 80.142715] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5424worker1 # [ 80.343663] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:33Z level=info event=scheduler_connected addr=lb:500515425worker1 # [ 80.347775] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5426worker2 # [ 80.519273] nix-grpc-daemon[1110]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5427worker2 # [ 80.524447] nix-grpc-daemon[1110]: E0000 00:00:1789938574.273650 1153 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5428worker2 # [ 80.529534] nix-grpc-daemon[1110]: E0000 00:00:1789938574.274705 1153 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5429worker1 # [ 80.548817] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:33Z level=info event=scheduler_connected addr=lb:500515430worker1 # [ 80.553265] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5431worker1 # [ 80.754351] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:34Z level=info event=scheduler_connected addr=lb:500515432worker1 # [ 80.756116] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5433worker1 # [ 80.957062] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:34Z level=info event=scheduler_connected addr=lb:500515434worker1 # [ 80.959603] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5435worker1 # [ 81.160572] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:34Z level=info event=scheduler_connected addr=lb:500515436worker1 # [ 81.164635] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5437worker1 # [ 81.365492] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:34Z level=info event=scheduler_connected addr=lb:500515438worker1 # [ 81.370273] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5439lb # [ 81.396361] niks3-server[1065]: 2026/09/20 21:09:35 INFO lead: acquired remote=192.168.1.3:424425440worker1 # [ 81.421758] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:34Z level=info event=scheduler_take_over5441worker2 # [ 81.523701] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5442worker2 # [ 81.524974] systemd[1]: Stopped Nix worker-protocol over gRPC.5443worker2 # [ 81.525865] systemd[1]: nix-grpc-daemon.service: Consumed 359ms CPU time over 10.222s wall clock time, 33.2M memory peak, 20.3K incoming IP traffic, 31.4K outgoing IP traffic.5444worker2 # [ 81.530822] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5445worker2 # [ 81.532809] systemd[1]: Closed Nix worker-protocol over gRPC.5446worker2 # [ 81.534486] systemd[1]: nix-grpc-daemon.socket: Consumed 324us CPU time over 11.145s wall clock time, 256K memory peak, 20.7K incoming IP traffic, 15.4K outgoing IP traffic.5447worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5448worker1 # [ 81.570546] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:34Z level=info event=scheduler_connected addr=lb:500515449worker1 # [ 81.572989] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5450worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.05 seconds)5451worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5452worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5453worker1 # [ 81.774219] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:35Z level=info event=scheduler_connected addr=lb:500515454worker1 # [ 81.778353] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5455worker1 # [ 81.979157] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:35Z level=info event=scheduler_connected addr=lb:500515456worker1 # [ 81.989660] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:35Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:369205457worker1 # [ 81.994255] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:35Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05458worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5459worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5460client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&25461worker1 # [ 82.872628] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:36Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:557225462worker1 # [ 82.885287] nix-daemon[851]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5463client # don't know how to build these paths:5464client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5465worker1 # [ 82.902687] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:557225466worker1 # [ 82.915484] nix-grpc-daemon[1218]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 33 ms (attempt 1/5)5467worker1 # [ 82.954281] nix-grpc-daemon[1218]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 194 ms (attempt 2/5)5468worker1 # [ 83.148652] nix-grpc-daemon[1218]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 52 ms (attempt 3/5)5469worker1 # [ 83.203555] nix-grpc-daemon[1218]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 506 ms (attempt 4/5)5470worker1 # [ 83.713894] nix-grpc-daemon[1218]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5471worker1 # [ 83.743646] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:37Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05472worker1 # [ 83.749132] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:37Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=15473worker1 # [ 83.795539] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:557225474worker1 # [ 83.809225] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:557225475client # copying 1 paths to 'lb:50051'...5476client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5477worker1 # [ 83.821570] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:557225478worker1 # [ 83.831286] nix-daemon[851]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5479worker1 # [ 83.849849] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:37Z level=info event=niks3_push_started pid=12885480worker1 # [ 84.053598] nix-grpc-daemon[1288]: time=2026-09-20T21:09:37.356Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.key ca=""5481worker1 # [ 84.217331] nix-daemon[851]: accepted connection from pid 1301, user nix-grpc-daemon (trusted)5482worker1 # [ 84.230607] nix-daemon[851]: reaped child process 1304, status = succeeded5483worker1 # [ 84.404170] nix-daemon[851]: accepted connection from pid 1306, user nix-grpc-daemon (trusted)5484worker1 # [ 84.419405] nix-daemon[851]: reaped child process 1309, status = succeeded5485lb # [ 84.412951] niks3-server[1065]: 2026/09/20 21:09:38 INFO Received uploads request method=POST path=/api/pending_closures5486worker1 # [ 84.446706] nix-grpc-daemon[1288]: time=2026-09-20T21:09:37.751Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5487worker1 # [ 84.450395] nix-grpc-daemon[1288]: time=2026-09-20T21:09:37.753Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5488lb # [ 84.449508] niks3-server[1065]: 2026/09/20 21:09:38 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5489lb # [ 84.462494] niks3-server[1065]: 2026/09/20 21:09:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5490worker1 # [ 84.493450] nix-grpc-daemon[1288]: time=2026-09-20T21:09:37.797Z level=INFO msg="Uploading 1 narinfos"5491lb # [ 84.472538] niks3-server[1065]: 2026/09/20 21:09:38 INFO Signed narinfos id=27 count=15492lb # [ 84.476705] niks3-server[1065]: 2026/09/20 21:09:38 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5493lb # [ 84.486603] niks3-server[1065]: 2026/09/20 21:09:38 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5494lb # [ 84.496715] niks3-server[1065]: 2026/09/20 21:09:38 INFO Completed upload id=275495worker1 # [ 84.518431] nix-grpc-daemon[1288]: time=2026-09-20T21:09:37.823Z level=INFO msg="Upload complete. (465ms)"5496lb # [ 84.498537] niks3-server[1065]: 2026/09/20 21:09:38 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5497worker1 # [ 84.522371] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55722 duration_s=0 paths=1 nar_bytes_in=5205498worker1 # [ 84.525250] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:557225499worker1 # [ 84.532568] nix-daemon[851]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5500worker1 # [ 84.535359] nix-daemon[851]: reaped child process 1286, status = succeeded5501worker1 # [ 84.540390] nix-daemon[851]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5502client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5503client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5504client # farm-a> LOG-farm-a-on-standby5505worker1 # [ 85.758058] nix-daemon[851]: reaped child process 1313, status = succeeded5506worker1 # [ 85.926432] nix-daemon[851]: accepted connection from pid 1328, user nix-grpc-daemon (trusted)5507worker1 # [ 85.941872] nix-daemon[851]: reaped child process 1331, status = succeeded5508lb # [ 85.933278] niks3-server[1065]: 2026/09/20 21:09:39 INFO Received uploads request method=POST path=/api/pending_closures5509worker1 # [ 85.972710] nix-grpc-daemon[1288]: time=2026-09-20T21:09:39.277Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5510worker1 # [ 85.975241] nix-grpc-daemon[1288]: time=2026-09-20T21:09:39.278Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5511lb # [ 85.977385] niks3-server[1065]: 2026/09/20 21:09:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5512lb # [ 85.989393] niks3-server[1065]: 2026/09/20 21:09:40 INFO Signed narinfos id=28 count=15513worker1 # [ 86.011344] nix-grpc-daemon[1288]: time=2026-09-20T21:09:39.315Z level=INFO msg="Uploading 1 narinfos"5514lb # [ 85.991284] niks3-server[1065]: 2026/09/20 21:09:40 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5515lb # [ 85.997750] niks3-server[1065]: 2026/09/20 21:09:40 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5516lb # [ 86.000106] niks3-server[1065]: 2026/09/20 21:09:40 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5517lb # [ 86.007264] niks3-server[1065]: 2026/09/20 21:09:40 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5518lb # [ 86.014120] niks3-server[1065]: 2026/09/20 21:09:40 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5519lb # [ 86.023728] niks3-server[1065]: 2026/09/20 21:09:40 INFO Completed upload id=285520worker1 # [ 86.045542] nix-grpc-daemon[1288]: time=2026-09-20T21:09:39.350Z level=INFO msg="Upload complete. (297ms)"5521worker1 # [ 86.052542] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55722 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15522worker1 # [ 86.070337] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:39Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05523worker1 # [ 86.072421] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:39Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=15524worker1 # [ 86.075050] nix-daemon[851]: reaped child process 1311, status = succeeded5525worker1 # [ 86.114557] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:557225526worker1 # [ 86.127464] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:557225527client # copying 1 paths to 'lb:50051'...5528worker1 # [ 86.143128] nix-grpc-daemon[1288]: time=2026-09-20T21:09:39.446Z level=INFO msg="All 1 paths already cached"5529client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5530worker1 # [ 86.147202] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:557225531worker1 # [ 86.152534] nix-daemon[851]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5532worker1 # [ 86.286634] nix-daemon[851]: accepted connection from pid 1337, user nix-grpc-daemon (trusted)5533worker1 # [ 86.300524] nix-daemon[851]: reaped child process 1340, status = succeeded5534worker1 # [ 86.467169] nix-daemon[851]: accepted connection from pid 1342, user nix-grpc-daemon (trusted)5535worker1 # [ 86.481439] nix-daemon[851]: reaped child process 1345, status = succeeded5536lb # [ 86.475534] niks3-server[1065]: 2026/09/20 21:09:40 INFO Received uploads request method=POST path=/api/pending_closures5537worker1 # [ 86.508591] nix-grpc-daemon[1288]: time=2026-09-20T21:09:39.812Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5538worker1 # [ 86.510036] nix-grpc-daemon[1288]: time=2026-09-20T21:09:39.812Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5539lb # [ 86.496307] niks3-server[1065]: 2026/09/20 21:09:40 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5540lb # [ 86.500936] niks3-server[1065]: 2026/09/20 21:09:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5541lb # [ 86.503564] niks3-server[1065]: 2026/09/20 21:09:40 INFO Signed narinfos id=29 count=15542worker1 # [ 86.525356] nix-grpc-daemon[1288]: time=2026-09-20T21:09:39.829Z level=INFO msg="Uploading 1 narinfos"5543lb # [ 86.505508] niks3-server[1065]: 2026/09/20 21:09:40 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5544lb # [ 86.509979] niks3-server[1065]: 2026/09/20 21:09:40 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5545lb # [ 86.513903] niks3-server[1065]: 2026/09/20 21:09:40 INFO Completed upload id=295546worker1 # [ 86.536539] nix-grpc-daemon[1288]: time=2026-09-20T21:09:39.841Z level=INFO msg="Upload complete. (377ms)"5547lb # [ 86.515503] niks3-server[1065]: 2026/09/20 21:09:40 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5548worker1 # [ 86.538643] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55722 duration_s=0 paths=1 nar_bytes_in=6405549worker1 # [ 86.542184] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:557225550worker1 # [ 86.546668] nix-daemon[851]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5551worker1 # [ 86.552534] nix-daemon[851]: reaped child process 1335, status = succeeded5552client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5553client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5554worker1 # [ 86.556829] nix-daemon[851]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5555client # farm-b> LOG-farm-b-on-standby5556worker1 # [ 87.182670] nix-daemon[851]: reaped child process 1349, status = succeeded5557worker1 # [ 87.344260] nix-daemon[851]: accepted connection from pid 1364, user nix-grpc-daemon (trusted)5558worker1 # [ 87.357336] nix-daemon[851]: reaped child process 1367, status = succeeded5559lb # [ 87.353355] niks3-server[1065]: 2026/09/20 21:09:41 INFO Received uploads request method=POST path=/api/pending_closures5560worker1 # [ 87.456816] nix-grpc-daemon[1288]: time=2026-09-20T21:09:40.761Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5561worker1 # [ 87.459295] nix-grpc-daemon[1288]: time=2026-09-20T21:09:40.763Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"5562lb # [ 87.465168] niks3-server[1065]: 2026/09/20 21:09:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5563lb # [ 87.473351] niks3-server[1065]: 2026/09/20 21:09:41 INFO Signed narinfos id=30 count=15564worker1 # [ 87.494665] nix-grpc-daemon[1288]: time=2026-09-20T21:09:40.798Z level=INFO msg="Uploading 1 narinfos"5565lb # [ 87.490553] niks3-server[1065]: 2026/09/20 21:09:41 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5566lb # [ 87.502565] niks3-server[1065]: 2026/09/20 21:09:41 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst5567lb # [ 87.507632] niks3-server[1065]: 2026/09/20 21:09:41 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5568lb # [ 87.512527] niks3-server[1065]: 2026/09/20 21:09:41 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls5569lb # [ 87.517100] niks3-server[1065]: 2026/09/20 21:09:41 INFO Completed upload id=305570worker1 # [ 87.538883] nix-grpc-daemon[1288]: time=2026-09-20T21:09:40.843Z level=INFO msg="Upload complete. (375ms)"5571lb # [ 87.521610] niks3-server[1065]: 2026/09/20 21:09:41 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo5572worker1 # [ 87.544584] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55722 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=15573worker1 # [ 87.557337] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:40Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=05574worker1 # [ 87.562183] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:40Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=15575worker1 # [ 87.574188] nix-daemon[851]: reaped child process 1347, status = succeeded5576worker1 # [ 87.602422] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:557225577worker1 # [ 87.614297] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:557225578worker1 # [ 87.642580] nix-grpc-daemon[1288]: time=2026-09-20T21:09:40.947Z level=INFO msg="All 2 paths already cached"5579client # copying 1 paths to 'lb:50051'...5580client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...5581worker1 # [ 87.651383] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:557225582worker1 # [ 87.659755] nix-daemon[851]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5583worker1 # [ 87.840622] nix-daemon[851]: accepted connection from pid 1373, user nix-grpc-daemon (trusted)5584worker1 # [ 87.853940] nix-daemon[851]: reaped child process 1376, status = succeeded5585worker1 # [ 88.015204] nix-daemon[851]: accepted connection from pid 1378, user nix-grpc-daemon (trusted)5586worker1 # [ 88.028198] nix-daemon[851]: reaped child process 1381, status = succeeded5587lb # [ 88.022490] niks3-server[1065]: 2026/09/20 21:09:42 INFO Received uploads request method=POST path=/api/pending_closures5588worker1 # [ 88.049587] nix-grpc-daemon[1288]: time=2026-09-20T21:09:41.354Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5589worker1 # [ 88.051475] nix-grpc-daemon[1288]: time=2026-09-20T21:09:41.355Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"5590lb # [ 88.042255] niks3-server[1065]: 2026/09/20 21:09:42 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst5591lb # [ 88.045412] niks3-server[1065]: 2026/09/20 21:09:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5592lb # [ 88.047692] niks3-server[1065]: 2026/09/20 21:09:42 INFO Signed narinfos id=31 count=15593worker1 # [ 88.071165] nix-grpc-daemon[1288]: time=2026-09-20T21:09:41.375Z level=INFO msg="Uploading 1 narinfos"5594lb # [ 88.053742] niks3-server[1065]: 2026/09/20 21:09:42 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls5595lb # [ 88.059439] niks3-server[1065]: 2026/09/20 21:09:42 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5596lb # [ 88.064599] niks3-server[1065]: 2026/09/20 21:09:42 INFO Completed upload id=315597worker1 # [ 88.086211] nix-grpc-daemon[1288]: time=2026-09-20T21:09:41.390Z level=INFO msg="Upload complete. (412ms)"5598worker1 # [ 88.088239] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55722 duration_s=0 paths=1 nar_bytes_in=7685599lb # [ 88.067350] niks3-server[1065]: 2026/09/20 21:09:42 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo5600worker1 # [ 88.092262] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:557225601worker1 # [ 88.097175] nix-daemon[851]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5602worker1 # [ 88.103165] nix-daemon[851]: reaped child process 1371, status = succeeded5603client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...5604worker1 # [ 88.106220] nix-daemon[851]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5605client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5606lb # [ 88.627866] envoy[973]: conn peer=[2001:db8:1::4]:33808 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=8603 tx=535 ms=82145607client # farm-top> LOG-farm-top-on-standby5608worker1 # [ 89.125681] nix-daemon[851]: reaped child process 1385, status = succeeded5609worker1 # [ 89.259783] nix-daemon[851]: accepted connection from pid 1400, user nix-grpc-daemon (trusted)5610worker1 # [ 89.274720] nix-daemon[851]: reaped child process 1403, status = succeeded5611lb # [ 89.264802] niks3-server[1065]: 2026/09/20 21:09:43 INFO Received uploads request method=POST path=/api/pending_closures5612worker1 # [ 89.295986] nix-grpc-daemon[1288]: time=2026-09-20T21:09:42.600Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5613worker1 # [ 89.297646] nix-grpc-daemon[1288]: time=2026-09-20T21:09:42.602Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"5614lb # [ 89.298143] niks3-server[1065]: 2026/09/20 21:09:43 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst5615lb # [ 89.312213] niks3-server[1065]: 2026/09/20 21:09:43 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5616lb # [ 89.318598] niks3-server[1065]: 2026/09/20 21:09:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5617worker1 # [ 89.344250] nix-grpc-daemon[1288]: time=2026-09-20T21:09:42.648Z level=INFO msg="Uploading 1 narinfos"5618lb # [ 89.322564] niks3-server[1065]: 2026/09/20 21:09:43 INFO Signed narinfos id=32 count=15619lb # [ 89.327735] niks3-server[1065]: 2026/09/20 21:09:43 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls5620lb # [ 89.337091] niks3-server[1065]: 2026/09/20 21:09:43 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5621lb # [ 89.343455] niks3-server[1065]: 2026/09/20 21:09:43 INFO Completed upload id=325622worker1 # [ 89.364803] nix-grpc-daemon[1288]: time=2026-09-20T21:09:42.669Z level=INFO msg="Upload complete. (248ms)"5623lb # [ 89.344946] niks3-server[1065]: 2026/09/20 21:09:43 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo5624worker1 # [ 89.367656] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55722 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=15625worker1 # [ 89.376219] nix-daemon[851]: reaped child process 1383, status = succeeded5626client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&2, in 6.73 seconds)5627client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath5628client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath, in 0.21 seconds)5629worker2 # [ 89.671704] systemd[1]: Listening on Nix worker-protocol over gRPC.5630worker2 # [ 89.678328] systemd[1]: Starting scheduler-token.service...5631worker2 # [ 89.744818] systemd[1]: scheduler-token.service: Deactivated successfully.5632worker2 # [ 89.745871] systemd[1]: Finished scheduler-token.service.5633worker2 # [ 89.747412] systemd[1]: scheduler-token.service: Consumed 20ms CPU time over 67ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5634worker2 # [ 89.752894] systemd[1]: Starting Nix worker-protocol over gRPC...5635worker2 # [ 89.932596] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:43Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5636worker2 # [ 89.934994] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:43Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5637worker2 # [ 89.938915] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:43Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15638worker2 # [ 89.947066] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:43Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5639worker2 # [ 89.954713] systemd[1]: Started Nix worker-protocol over gRPC.5640lb: must succeed: curl -sf localhost:9901/clusters5641worker2 # [ 89.977508] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:43Z level=info event=scheduler_connected addr=lb:500515642worker1 # [ 90.001190] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:43Z level=info event=oidc_jwks provider=mock keys=15643worker1 # [ 90.006120] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:43Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:369205644worker1 # [ 90.010280] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:43Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05645lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5646worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5647worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5648lb: must succeed: curl -sf localhost:9901/clusters5649lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5650lb: must succeed: curl -sf localhost:9901/clusters5651lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5652worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5653worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5654(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.69 seconds)5655subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5656client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs5657client # [ 90.277897] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs.5658client # Running as unit: rs.service; invocation ID: 96dbdf110ef6486784f7b6bab008848f5659client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs, in 0.06 seconds)5660worker2 # [ 90.540119] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:422685661worker2 # [ 90.547231] nix-daemon[871]: accepted connection from pid 1206, user nix-grpc-daemon (trusted)5662client # [ 90.561891] nix[1181]: don't know how to build these paths:5663client # [ 90.564357] nix[1181]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5664worker1 # [ 90.581619] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:43Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=05665worker1 # [ 90.584754] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:43Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=15666worker1 # [ 90.626619] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:557225667worker1 # [ 90.637302] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:557225668client # [ 90.635217] nix[1181]: copying 1 paths to 'lb:50051'...5669client # [ 90.636307] nix[1181]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...5670worker1 # [ 90.646654] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:557225671worker1 # [ 90.654881] nix-daemon[851]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5672worker1 # [ 90.811275] nix-daemon[851]: accepted connection from pid 1416, user nix-grpc-daemon (trusted)5673worker1 # [ 90.825099] nix-daemon[851]: reaped child process 1419, status = succeeded5674worker1 # [ 91.030215] nix-daemon[851]: accepted connection from pid 1421, user nix-grpc-daemon (trusted)5675worker1 # [ 91.044546] nix-daemon[851]: reaped child process 1424, status = succeeded5676lb # [ 91.034071] niks3-server[1065]: 2026/09/20 21:09:45 INFO Received uploads request method=POST path=/api/pending_closures5677worker1 # [ 91.124599] nix-grpc-daemon[1288]: time=2026-09-20T21:09:44.428Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5678worker1 # [ 91.126232] nix-grpc-daemon[1288]: time=2026-09-20T21:09:44.428Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"5679lb # [ 91.120108] niks3-server[1065]: 2026/09/20 21:09:45 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst5680lb # [ 91.132840] niks3-server[1065]: 2026/09/20 21:09:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5681worker1 # [ 91.158591] nix-grpc-daemon[1288]: time=2026-09-20T21:09:44.463Z level=INFO msg="Uploading 1 narinfos"5682lb # [ 91.136855] niks3-server[1065]: 2026/09/20 21:09:45 INFO Signed narinfos id=33 count=15683lb # [ 91.143226] niks3-server[1065]: 2026/09/20 21:09:45 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls5684lb # [ 91.149269] niks3-server[1065]: 2026/09/20 21:09:45 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5685lb # [ 91.152528] niks3-server[1065]: 2026/09/20 21:09:45 INFO Completed upload id=335686worker1 # [ 91.174382] nix-grpc-daemon[1288]: time=2026-09-20T21:09:44.478Z level=INFO msg="Upload complete. (512ms)"5687lb # [ 91.155187] niks3-server[1065]: 2026/09/20 21:09:45 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo5688worker1 # [ 91.179259] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55722 duration_s=0 paths=1 nar_bytes_in=4165689worker1 # [ 91.185629] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:557225690worker1 # [ 91.197149] nix-daemon[851]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5691worker1 # [ 91.210302] nix-daemon[851]: reaped child process 1414, status = succeeded5692client # [ 91.204693] nix[1181]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...5693worker1 # [ 91.216625] nix-daemon[851]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5694lb: must succeed: curl -sf localhost:9901/clusters5695lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5696lb: must succeed: curl -sf localhost:9901/clusters5697lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5698lb: must succeed: curl -sf localhost:9901/clusters5699lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5700lb: must succeed: curl -sf localhost:9901/clusters5701lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5702worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5703worker2 # [ 93.020568] systemd-journald[287]: Received client request to rotate journal, rotating.5704worker2 # [ 93.035272] systemd-journald[287]: Vacuuming done, freed 0B of archived journals from /var/log/journal/e739f1c805aa4a4aa64affb82e69947f.5705worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.06 seconds)5706worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5707worker1 # [ 92.970619] systemd[1]: Stopping Nix worker-protocol over gRPC...5708worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)5709client: must succeed: sleep 5.05710worker1 # [ 93.415081] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:46Z level=info event=draining5711client: (finished: must succeed: sleep 5.0, in 5.03 seconds)5712worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5713worker1 # [ 98.243751] nix-daemon[851]: reaped child process 1428, status = succeeded5714worker1 # [ 98.392114] nix-daemon[851]: accepted connection from pid 1468, user nix-grpc-daemon (trusted)5715worker1 # [ 98.407426] nix-daemon[851]: reaped child process 1471, status = succeeded5716lb # [ 98.404102] niks3-server[1065]: 2026/09/20 21:09:52 INFO Received uploads request method=POST path=/api/pending_closures5717worker1 # [ 98.435099] nix-grpc-daemon[1288]: time=2026-09-20T21:09:51.739Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5718worker1 # [ 98.438199] nix-grpc-daemon[1288]: time=2026-09-20T21:09:51.741Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"5719lb # [ 98.448144] niks3-server[1065]: 2026/09/20 21:09:52 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5720lb # [ 98.467896] niks3-server[1065]: 2026/09/20 21:09:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5721worker1 # [ 98.494849] nix-grpc-daemon[1288]: time=2026-09-20T21:09:51.799Z level=INFO msg="Uploading 1 narinfos"5722lb # [ 98.474855] niks3-server[1065]: 2026/09/20 21:09:52 INFO Signed narinfos id=34 count=15723lb # [ 98.482061] niks3-server[1065]: 2026/09/20 21:09:52 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls5724lb # [ 98.491314] niks3-server[1065]: 2026/09/20 21:09:52 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5725lb # [ 98.494678] niks3-server[1065]: 2026/09/20 21:09:52 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5726lb # [ 98.500110] niks3-server[1065]: 2026/09/20 21:09:52 INFO Completed upload id=345727lb # [ 98.501795] niks3-server[1065]: 2026/09/20 21:09:52 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo5728worker1 # [ 98.522210] nix-grpc-daemon[1288]: time=2026-09-20T21:09:51.826Z level=INFO msg="Upload complete. (288ms)"5729worker1 # [ 98.531738] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55722 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=15730worker1 # [ 98.546411] nix-daemon[851]: reaped child process 1426, status = succeeded5731client # [ 98.555585] systemd[1]: rs.service: Deactivated successfully.5732client # [ 98.559424] systemd[1]: rs.service: Consumed 177ms CPU time over 8.280s wall clock time, 43.4M memory peak, 6.8K incoming IP traffic, 8.1K outgoing IP traffic.5733lb # [ 98.628686] envoy[973]: conn peer=[2001:db8:1::1]:38212 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9555 tx=3886 ms=65195734lb # [ 98.632262] envoy[973]: conn peer=[2001:db8:1::1]:38222 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2930 tx=1330 ms=79945735worker1 # [ 99.415214] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:52Z level=info event=signal_exit5736worker1 # [ 99.639984] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:52Z level=info event=scheduler_yield5737worker1 # [ 99.642238] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:52Z level=info event=scheduler_restarting addr=lb:500515738worker1 # [ 99.645384] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:52Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:369205739worker2 # [ 99.627413] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:53Z level=info event=scheduler_restarting addr=lb:500515740lb # [ 99.624284] niks3-server[1065]: 2026/09/20 21:09:53 INFO lead: released remote=192.168.1.3:424425741worker1 # [ 99.647104] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:52Z level=info event=worker_gone addr=192.168.1.3:500515742worker1 # [ 99.648835] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:52Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:369205743worker1 # [ 99.650268] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:52Z level=info event=worker_gone addr=192.168.1.4:500515744worker1 # [ 99.843509] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:53Z level=info event=scheduler_connected addr=lb:500515745worker2 # [ 99.827482] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:53Z level=info event=scheduler_connected addr=lb:500515746worker2 # [ 99.832450] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:53Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5747worker1 # [ 99.849304] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:53Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5748lb # [ 99.953876] niks3-server[1065]: 2026/09/20 21:09:54 INFO lead: acquired remote=192.168.1.4:444105749worker2 # [ 99.959338] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:53Z level=info event=scheduler_take_over5750worker1 # [ 100.050286] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:53Z level=info event=scheduler_connected addr=lb:500515751worker2 # [ 100.033856] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:53Z level=info event=scheduler_connected addr=lb:500515752worker1 # [ 100.054269] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5753worker2 # [ 100.037632] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5754worker2 # [ 100.238891] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:53Z level=info event=scheduler_connected addr=lb:500515755worker1 # [ 100.255409] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:53Z level=info event=scheduler_connected addr=lb:500515756worker2 # [ 100.242728] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5757worker1 # [ 100.259821] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5758worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.15 seconds)5759client: waiting for success: ! systemctl is-active rs5760client: (finished: waiting for success: ! systemctl is-active rs, in 0.05 seconds)5761client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5762worker2 # [ 100.443963] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:54Z level=info event=scheduler_connected addr=lb:500515763worker1 # [ 100.461183] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:53Z level=info event=scheduler_connected addr=lb:500515764worker2 # [ 100.448148] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5765worker1 # [ 100.465387] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5766client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.04 seconds)5767client: must fail: journalctl -u rs | grep -q 'reconnecting'5768client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5769lb: must succeed: curl -sf localhost:9901/clusters5770lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5771worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5772worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5773worker1 # [ 100.646132] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:53Z level=info event=scheduler_restarting5774worker2 # [ 100.649870] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:54Z level=info event=scheduler_connected addr=lb:500515775worker2 # [ 100.653841] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5776worker1 # [ 100.675501] nix-daemon[851]: reaped child process 1279, status = succeeded5777worker1 # [ 100.682782] nix-grpc-daemon[1218]: ts=2026-09-20T21:09:53Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5778worker2 # [ 100.854928] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:54Z level=info event=scheduler_connected addr=lb:500515779worker2 # [ 100.858764] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5780worker1 # [ 101.007817] nix-grpc-daemon[1218]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5781worker1 # [ 101.011312] nix-grpc-daemon[1218]: E0000 00:00:1789938594.312000 1261 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5782worker1 # [ 101.016231] nix-grpc-daemon[1218]: E0000 00:00:1789938594.312296 1261 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5783worker2 # [ 101.059356] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:54Z level=info event=scheduler_connected addr=lb:500515784worker2 # [ 101.061123] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5785worker2 # [ 101.261843] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:55Z level=info event=scheduler_connected addr=lb:500515786worker2 # [ 101.263807] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5787worker2 # [ 101.464624] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:55Z level=info event=scheduler_connected addr=lb:500515788worker2 # [ 101.466540] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5789lb: must succeed: curl -sf localhost:9901/clusters5790worker2 # [ 101.668415] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:55Z level=info event=scheduler_connected addr=lb:500515791lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5792worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5793worker2 # [ 101.682772] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:55Z level=info event=oidc_jwks provider=mock keys=15794worker2 # [ 101.685674] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:55Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:545505795worker2 # [ 101.689427] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:55Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05796worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5797worker1 # [ 102.129375] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5798worker1 # [ 102.131250] systemd[1]: Stopped Nix worker-protocol over gRPC.5799worker1 # [ 102.132048] systemd[1]: nix-grpc-daemon.service: Consumed 1.438s CPU time over 25.911s wall clock time, 36.1M memory peak, 49.5K incoming IP traffic, 96.7K outgoing IP traffic.5800worker1 # [ 102.138829] systemd[1]: Starting Nix worker-protocol over gRPC...5801worker1 # [ 102.300061] nix-grpc-daemon[1478]: ts=2026-09-20T21:09:55Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5802worker1 # [ 102.302087] nix-grpc-daemon[1478]: ts=2026-09-20T21:09:55Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5803worker1 # [ 102.306254] nix-grpc-daemon[1478]: ts=2026-09-20T21:09:55Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15804worker1 # [ 102.314909] nix-grpc-daemon[1478]: ts=2026-09-20T21:09:55Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5805worker1 # [ 102.320970] systemd[1]: Started Nix worker-protocol over gRPC.5806worker1 # [ 102.343281] nix-grpc-daemon[1478]: ts=2026-09-20T21:09:55Z level=info event=scheduler_connected addr=lb:500515807worker2 # [ 102.328691] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:56Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:545505808worker2 # [ 102.332806] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:56Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05809lb: must succeed: curl -sf localhost:9901/clusters5810lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5811worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5812worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5813(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.65 seconds)5814subtest: scheduler restart mid-build: build finishes, no double build5815client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid5816client # [ 102.915253] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid.5817client # Running as unit: mid.service; invocation ID: 9e0517ca12cd499a89be1b52572ba6485818client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid, in 0.04 seconds)5819worker1 # [ 103.142162] nix-grpc-daemon[1478]: ts=2026-09-20T21:09:56Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:373865820worker1 # [ 103.154113] nix-daemon[851]: accepted connection from pid 1478, user nix-grpc-daemon (trusted)5821client # [ 103.156304] nix[1217]: don't know how to build these paths:5822client # [ 103.157334] nix[1217]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5823worker2 # [ 103.162579] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:56Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05824worker2 # [ 103.166760] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:56Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=15825worker2 # [ 103.204862] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:422685826worker2 # [ 103.216138] nix-grpc-daemon[1206]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 3 ms (attempt 1/5)5827worker2 # [ 103.224726] nix-grpc-daemon[1206]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 171 ms (attempt 2/5)5828worker2 # [ 103.401530] nix-grpc-daemon[1206]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 16 ms (attempt 3/5)5829worker2 # [ 103.423721] nix-grpc-daemon[1206]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 24 ms (attempt 4/5)5830worker2 # [ 103.453770] nix-grpc-daemon[1206]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5831worker2 # [ 103.472338] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:422685832client # [ 103.489250] nix[1217]: copying 1 paths to 'lb:50051'...5833worker2 # [ 103.484104] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:422685834client # [ 103.492144] nix[1217]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...5835worker2 # [ 103.497699] nix-daemon[871]: accepted connection from pid 1206, user nix-grpc-daemon (trusted)5836worker2 # [ 103.520846] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:57Z level=info event=niks3_push_started pid=13275837worker2 # [ 103.714510] nix-grpc-daemon[1327]: time=2026-09-20T21:09:57.467Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.key ca=""5838worker2 # [ 103.849827] nix-daemon[871]: accepted connection from pid 1340, user nix-grpc-daemon (trusted)5839worker2 # [ 103.862075] nix-daemon[871]: reaped child process 1343, status = succeeded5840worker2 # [ 104.026224] nix-daemon[871]: accepted connection from pid 1345, user nix-grpc-daemon (trusted)5841worker2 # [ 104.039194] nix-daemon[871]: reaped child process 1348, status = succeeded5842lb # [ 104.050573] niks3-server[1065]: 2026/09/20 21:09:58 INFO Received uploads request method=POST path=/api/pending_closures5843worker2 # [ 104.064040] nix-grpc-daemon[1327]: time=2026-09-20T21:09:57.819Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5844worker2 # [ 104.066745] nix-grpc-daemon[1327]: time=2026-09-20T21:09:57.820Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"5845lb # [ 104.086235] niks3-server[1065]: 2026/09/20 21:09:58 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst5846lb # [ 104.091971] niks3-server[1065]: 2026/09/20 21:09:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5847lb # [ 104.099895] niks3-server[1065]: 2026/09/20 21:09:58 INFO Signed narinfos id=35 count=15848worker2 # [ 104.108118] nix-grpc-daemon[1327]: time=2026-09-20T21:09:57.862Z level=INFO msg="Uploading 1 narinfos"5849lb # [ 104.105289] niks3-server[1065]: 2026/09/20 21:09:58 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls5850lb # [ 104.116785] niks3-server[1065]: 2026/09/20 21:09:58 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5851worker2 # [ 104.127617] nix-grpc-daemon[1327]: time=2026-09-20T21:09:57.882Z level=INFO msg="Upload complete. (413ms)"5852lb # [ 104.122971] niks3-server[1065]: 2026/09/20 21:09:58 INFO Completed upload id=355853lb # [ 104.125254] niks3-server[1065]: 2026/09/20 21:09:58 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo5854worker2 # [ 104.131073] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42268 duration_s=0 paths=1 nar_bytes_in=4245855worker2 # [ 104.133882] nix-grpc-daemon[1206]: ts=2026-09-20T21:09:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:422685856worker2 # [ 104.138410] nix-daemon[871]: accepted connection from pid 1206, user nix-grpc-daemon (trusted)5857worker2 # [ 104.143362] nix-daemon[871]: reaped child process 1325, status = succeeded5858worker2 # [ 104.148075] nix-daemon[871]: accepted connection from pid 1206, user nix-grpc-daemon (trusted)5859client # [ 104.153805] nix[1217]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5860lb: must succeed: curl -sf localhost:9901/clusters5861lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5862lb: must succeed: curl -sf localhost:9901/clusters5863lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5864worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5865worker2 # [ 105.452352] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1206 (nix-grpc-daemon) on client request.5866worker2 # [ 105.455500] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5867lb # [ 105.461755] niks3-server[1065]: 2026/09/20 21:09:59 INFO lead: released remote=192.168.1.4:444105868worker1 # [ 105.487979] nix-grpc-daemon[1478]: ts=2026-09-20T21:09:58Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5869client # [ 105.478875] nix[1217]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5870client # [ 105.483723] nix[1217]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5871worker2 # [ 105.480953] systemd[1]: Starting scheduler-token.service...5872worker2 # [ 105.494228] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5873worker2 # [ 105.495788] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5874worker2 # [ 105.497034] systemd[1]: nix-grpc-daemon.service: Consumed 467ms CPU time over 15.736s wall clock time, 33.5M memory peak, 17.2K incoming IP traffic, 38.9K outgoing IP traffic.5875worker2 # [ 105.506619] nix-daemon[871]: reaped child process 1354, status = succeeded5876worker2 # [ 105.508474] nix-daemon[871]: reaped child process 1262, status = succeeded5877worker2 # [ 105.516862] nix-daemon[1356]: unexpected Nix daemon error: error: interrupted by the user5878worker2 # [ 105.525384] nix-daemon[871]: reaped child process 1356, status = failed with exit code 15879worker2 # [ 105.566141] systemd[1]: scheduler-token.service: Deactivated successfully.5880worker2 # [ 105.567437] systemd[1]: Finished scheduler-token.service.5881worker2 # [ 105.569786] systemd[1]: scheduler-token.service: Consumed 19ms CPU time over 85ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5882worker2 # [ 105.572197] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.5883worker2 # [ 105.577243] systemd[1]: Starting scheduler-token.service...5884worker2 # [ 105.634586] systemd[1]: scheduler-token.service: Deactivated successfully.5885worker2 # [ 105.637432] systemd[1]: Finished scheduler-token.service.5886worker2 # [ 105.638382] systemd[1]: scheduler-token.service: Consumed 19ms CPU time over 59ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5887worker2 # [ 105.642903] systemd[1]: Starting Nix worker-protocol over gRPC...5888worker1 # [ 105.688586] nix-grpc-daemon[1478]: ts=2026-09-20T21:09:58Z level=info event=scheduler_connected addr=lb:500515889worker2 # [ 105.794393] nix-grpc-daemon[1394]: ts=2026-09-20T21:09:59Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5890worker2 # [ 105.796764] nix-grpc-daemon[1394]: ts=2026-09-20T21:09:59Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5891worker2 # [ 105.800539] nix-grpc-daemon[1394]: ts=2026-09-20T21:09:59Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15892worker2 # [ 105.807568] nix-grpc-daemon[1394]: ts=2026-09-20T21:09:59Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5893worker2 # [ 105.809398] systemd[1]: Started Nix worker-protocol over gRPC.5894lb # [ 105.817070] niks3-server[1065]: 2026/09/20 21:09:59 INFO lead: acquired remote=192.168.1.4:492485895worker2 # [ 105.824072] nix-grpc-daemon[1394]: ts=2026-09-20T21:09:59Z level=info event=scheduler_take_over5896worker2 # [ 105.831109] nix-grpc-daemon[1394]: ts=2026-09-20T21:09:59Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:333425897worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.42 seconds)5898client: must succeed: sleep 5.05899worker2 # [ 105.834329] nix-grpc-daemon[1394]: ts=2026-09-20T21:09:59Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05900worker2 # [ 105.848326] nix-grpc-daemon[1394]: ts=2026-09-20T21:09:59Z level=info event=scheduler_connected addr=lb:500515901worker2 # [ 105.853438] nix-grpc-daemon[1394]: ts=2026-09-20T21:09:59Z level=info event=oidc_jwks provider=mock keys=15902worker2 # [ 105.854968] nix-grpc-daemon[1394]: ts=2026-09-20T21:09:59Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:333425903worker2 # [ 105.856787] nix-grpc-daemon[1394]: ts=2026-09-20T21:09:59Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05904worker2 # [ 105.986134] nix-grpc-daemon[1394]: ts=2026-09-20T21:09:59Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05905worker2 # [ 105.991311] nix-grpc-daemon[1394]: ts=2026-09-20T21:09:59Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=15906worker1 # [ 106.047893] nix-grpc-daemon[1478]: ts=2026-09-20T21:09:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:373865907worker1 # [ 106.054108] nix-grpc-daemon[1478]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 64 ms (attempt 1/5)5908worker1 # [ 106.120594] nix-grpc-daemon[1478]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 169 ms (attempt 2/5)5909worker1 # [ 106.292531] nix-grpc-daemon[1478]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 383 ms (attempt 3/5)5910worker1 # [ 106.681971] nix-grpc-daemon[1478]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 50 ms (attempt 4/5)5911worker1 # [ 106.735611] nix-grpc-daemon[1478]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5912worker1 # [ 106.754689] nix-grpc-daemon[1478]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' from 'http://lb:5751'...5913worker1 # [ 106.779092] nix-daemon[851]: accepted connection from pid 1478, user nix-grpc-daemon (trusted)5914worker1 # [ 106.792472] nix-daemon[851]: accepted connection from pid 1478, user nix-grpc-daemon (trusted)5915client # [ 106.782453] nix[1217]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5916lb # [ 108.628531] envoy[973]: conn peer=[2001:db8:1::3]:47872 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=5369 tx=1293 ms=242345917lb # [ 108.630627] envoy[973]: conn peer=[2001:db8:1::4]:44680 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=9471 tx=674 ms=154975918client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5919client: waiting for success: ! systemctl is-active mid5920worker1 # [ 111.210635] nix-grpc-daemon[1478]: ts=2026-09-20T21:10:04Z level=info event=niks3_push_started pid=15755921worker1 # [ 111.222943] nix-daemon[851]: reaped child process 1551, status = succeeded5922worker1 # [ 111.401194] nix-grpc-daemon[1575]: time=2026-09-20T21:10:04.705Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.key ca=""5923worker1 # [ 111.557895] nix-daemon[851]: accepted connection from pid 1588, user nix-grpc-daemon (trusted)5924worker1 # [ 111.574799] nix-daemon[851]: reaped child process 1591, status = succeeded5925lb # [ 111.570948] niks3-server[1065]: 2026/09/20 21:10:05 INFO Received uploads request method=POST path=/api/pending_closures5926worker1 # [ 111.601304] nix-grpc-daemon[1575]: time=2026-09-20T21:10:04.905Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5927worker1 # [ 111.602968] nix-grpc-daemon[1575]: time=2026-09-20T21:10:04.907Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"5928lb # [ 111.613601] niks3-server[1065]: 2026/09/20 21:10:05 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5929lb # [ 111.629261] niks3-server[1065]: 2026/09/20 21:10:05 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5930lb # [ 111.636585] niks3-server[1065]: 2026/09/20 21:10:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5931worker1 # [ 111.662106] nix-grpc-daemon[1575]: time=2026-09-20T21:10:04.966Z level=INFO msg="Uploading 1 narinfos"5932lb # [ 111.640870] niks3-server[1065]: 2026/09/20 21:10:05 INFO Signed narinfos id=36 count=15933lb # [ 111.651431] niks3-server[1065]: 2026/09/20 21:10:05 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls5934lb # [ 111.653549] niks3-server[1065]: 2026/09/20 21:10:05 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5935lb # [ 111.660259] niks3-server[1065]: 2026/09/20 21:10:05 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo5936lb # [ 111.669073] niks3-server[1065]: 2026/09/20 21:10:05 INFO Completed upload id=365937worker1 # [ 111.692316] nix-grpc-daemon[1575]: time=2026-09-20T21:10:04.996Z level=INFO msg="Upload complete. (287ms)"5938worker1 # [ 111.697125] nix-grpc-daemon[1478]: ts=2026-09-20T21:10:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37386 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=15939worker1 # [ 111.709554] nix-daemon[851]: reaped child process 1549, status = succeeded5940client # [ 111.728240] systemd[1]: mid.service: Deactivated successfully.5941client # [ 111.732211] systemd[1]: mid.service: Consumed 150ms CPU time over 8.816s wall clock time, 41.4M memory peak, 8.1K incoming IP traffic, 10K outgoing IP traffic.5942client: (finished: waiting for success: ! systemctl is-active mid, in 1.12 seconds)5943client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5944client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.04 seconds)5945(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 9.41 seconds)5946subtest: interrupting the client stops the build on the worker5947client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr5948client # [ 112.326440] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr.5949client # Running as unit: intr.service; invocation ID: 70eb242296e648d19bf934449f70f4625950client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr, in 0.04 seconds)5951worker1 # [ 112.563795] nix-grpc-daemon[1478]: ts=2026-09-20T21:10:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:373865952client # [ 112.561128] nix[1253]: don't know how to build these paths:5953client # [ 112.562390] nix[1253]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv5954worker2 # [ 112.570101] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:06Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=05955worker2 # [ 112.574361] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:06Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=15956worker2 # [ 112.617950] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333525957worker2 # [ 112.629174] nix-daemon[871]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)5958worker2 # [ 112.645968] nix-grpc-daemon[1394]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 43 ms (attempt 1/5)5959worker2 # [ 112.694825] nix-grpc-daemon[1394]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 133 ms (attempt 2/5)5960worker2 # [ 112.833911] nix-grpc-daemon[1394]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 162 ms (attempt 3/5)5961worker2 # [ 113.002343] nix-grpc-daemon[1394]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 735 ms (attempt 4/5)5962worker2 # [ 113.738978] nix-grpc-daemon[1394]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5963worker2 # [ 113.757111] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:333525964client # [ 113.774376] nix[1253]: copying 1 paths to 'lb:50051'...5965client # [ 113.776806] nix[1253]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...5966worker2 # [ 113.770951] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:333525967worker2 # [ 113.781211] nix-daemon[871]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)5968worker2 # [ 113.802841] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:07Z level=info event=niks3_push_started pid=14705969worker2 # [ 113.978085] nix-grpc-daemon[1470]: time=2026-09-20T21:10:07.731Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.key ca=""5970worker2 # [ 114.123095] nix-daemon[871]: accepted connection from pid 1483, user nix-grpc-daemon (trusted)5971worker2 # [ 114.137233] nix-daemon[871]: reaped child process 1486, status = succeeded5972worker2 # [ 114.286528] nix-daemon[871]: accepted connection from pid 1488, user nix-grpc-daemon (trusted)5973worker2 # [ 114.299877] nix-daemon[871]: reaped child process 1491, status = succeeded5974lb # [ 114.307910] niks3-server[1065]: 2026/09/20 21:10:08 INFO Received uploads request method=POST path=/api/pending_closures5975worker2 # [ 114.325244] nix-grpc-daemon[1470]: time=2026-09-20T21:10:08.080Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5976worker2 # [ 114.328484] nix-grpc-daemon[1470]: time=2026-09-20T21:10:08.081Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"5977lb # [ 114.349904] niks3-server[1065]: 2026/09/20 21:10:08 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst5978lb # [ 114.356286] niks3-server[1065]: 2026/09/20 21:10:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5979worker2 # [ 114.368941] nix-grpc-daemon[1470]: time=2026-09-20T21:10:08.123Z level=INFO msg="Uploading 1 narinfos"5980lb # [ 114.363713] niks3-server[1065]: 2026/09/20 21:10:08 INFO Signed narinfos id=37 count=15981lb # [ 114.374308] niks3-server[1065]: 2026/09/20 21:10:08 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls5982lb # [ 114.380707] niks3-server[1065]: 2026/09/20 21:10:08 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5983lb # [ 114.386928] niks3-server[1065]: 2026/09/20 21:10:08 INFO Completed upload id=375984lb # [ 114.389569] niks3-server[1065]: 2026/09/20 21:10:08 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo5985worker2 # [ 114.394269] nix-grpc-daemon[1470]: time=2026-09-20T21:10:08.149Z level=INFO msg="Upload complete. (416ms)"5986worker2 # [ 114.399261] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33352 duration_s=0 paths=1 nar_bytes_in=4245987worker2 # [ 114.404246] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333525988worker2 # [ 114.412586] nix-daemon[871]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)5989worker2 # [ 114.418722] nix-daemon[871]: reaped child process 1468, status = succeeded5990client # [ 114.428908] nix[1253]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...5991worker2 # [ 114.423267] nix-daemon[871]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)5992client: must succeed: systemctl kill -s INT intr5993client # [ 115.837418] systemd[1]: intr.service: Sent signal SIGINT to main process 1253 (nix) on client request.5994worker2 # [ 115.836981] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:09Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv5995client # [ 115.845359] nix[1253]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5996client: (finished: must succeed: systemctl kill -s INT intr, in 0.06 seconds)5997client # [ 115.852616] nix[1253]: error: interrupted by the user5998client # [ 115.881903] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5999client # [ 115.883710] systemd[1]: intr.service: Failed with result 'exit-code'.6000client # [ 115.885830] systemd[1]: intr.service: Consumed 143ms CPU time over 3.557s wall clock time, 41.4M memory peak, 6.1K incoming IP traffic, 8K outgoing IP traffic.6001worker2 # [ 116.042326] nix-daemon[871]: reaped child process 1493, status = succeeded6002worker2 # [ 116.044426] nix-daemon[1495]: unexpected Nix daemon error: error: interrupted by the user6003worker2 # [ 116.048286] nix-daemon[871]: reaped child process 1495, status = failed with exit code 16004(finished: subtest: interrupting the client stops the build on the worker, in 4.78 seconds)6005subtest: a deploy mid-build drains: the build finishes, the new generation takes over6006client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw6007client # [ 117.129950] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw.6008client # Running as unit: sw.service; invocation ID: ff8b7f54ce6c4e17a526adca6667d1116009client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw, in 0.07 seconds)6010worker1 # [ 117.362776] nix-grpc-daemon[1478]: ts=2026-09-20T21:10:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:373866011client # [ 117.356917] nix[1274]: don't know how to build these paths:6012client # [ 117.357862] nix[1274]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6013worker2 # [ 117.360916] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:11Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06014worker2 # [ 117.365679] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:11Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=16015worker1 # [ 117.426557] nix-grpc-daemon[1478]: ts=2026-09-20T21:10:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:373866016worker1 # [ 117.440714] nix-grpc-daemon[1478]: ts=2026-09-20T21:10:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:373866017client # [ 117.441278] nix[1274]: copying 1 paths to 'lb:50051'...6018client # [ 117.445129] nix[1274]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6019worker1 # [ 117.455725] nix-grpc-daemon[1478]: ts=2026-09-20T21:10:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:373866020worker1 # [ 117.464975] nix-daemon[851]: accepted connection from pid 1478, user nix-grpc-daemon (trusted)6021worker1 # [ 117.659836] nix-daemon[851]: accepted connection from pid 1623, user nix-grpc-daemon (trusted)6022worker1 # [ 117.675611] nix-daemon[851]: reaped child process 1626, status = succeeded6023worker1 # [ 117.858088] nix-daemon[851]: accepted connection from pid 1628, user nix-grpc-daemon (trusted)6024worker1 # [ 117.871345] nix-daemon[851]: reaped child process 1631, status = succeeded6025lb # [ 117.868804] niks3-server[1065]: 2026/09/20 21:10:11 INFO Received uploads request method=POST path=/api/pending_closures6026worker1 # [ 117.930727] nix-grpc-daemon[1575]: time=2026-09-20T21:10:11.234Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6027worker1 # [ 117.933100] nix-grpc-daemon[1575]: time=2026-09-20T21:10:11.237Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6028lb # [ 117.933125] niks3-server[1065]: 2026/09/20 21:10:11 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6029lb # [ 117.945900] niks3-server[1065]: 2026/09/20 21:10:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6030lb # [ 117.950526] niks3-server[1065]: 2026/09/20 21:10:12 INFO Signed narinfos id=38 count=16031worker1 # [ 117.971957] nix-grpc-daemon[1575]: time=2026-09-20T21:10:11.276Z level=INFO msg="Uploading 1 narinfos"6032lb # [ 117.955793] niks3-server[1065]: 2026/09/20 21:10:12 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6033lb # [ 117.965711] niks3-server[1065]: 2026/09/20 21:10:12 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6034lb # [ 117.969721] niks3-server[1065]: 2026/09/20 21:10:12 INFO Completed upload id=386035worker1 # [ 117.991357] nix-grpc-daemon[1575]: time=2026-09-20T21:10:11.295Z level=INFO msg="Upload complete. (512ms)"6036lb # [ 117.971648] niks3-server[1065]: 2026/09/20 21:10:12 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6037worker1 # [ 117.997165] nix-grpc-daemon[1478]: ts=2026-09-20T21:10:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37386 duration_s=0 paths=1 nar_bytes_in=4166038worker1 # [ 118.002885] nix-grpc-daemon[1478]: ts=2026-09-20T21:10:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:373866039worker1 # [ 118.016109] nix-daemon[851]: accepted connection from pid 1478, user nix-grpc-daemon (trusted)6040worker1 # [ 118.023082] nix-daemon[851]: reaped child process 1621, status = succeeded6041client # [ 118.020473] nix[1274]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6042worker1 # [ 118.032568] nix-daemon[851]: accepted connection from pid 1478, user nix-grpc-daemon (trusted)6043lb # [ 118.629514] envoy[973]: conn peer=[2001:db8:1::1]:47014 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3861 tx=1661 ms=85756044lb # [ 118.634112] envoy[973]: conn peer=[2001:db8:1::1]:47018 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2937 tx=757 ms=33116045worker1: must succeed: systemctl reload nix-grpc-daemon.service6046worker1 # [ 119.687323] systemd[1]: Reloading Nix worker-protocol over gRPC...6047worker1 # [ 119.733206] systemd[1]: Reloaded Nix worker-protocol over gRPC.6048worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.10 seconds)6049worker1: must succeed: pgrep -f 'read -t [9]0 x'6050worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)6051client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&26052client # don't know how to build these paths:6053client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6054worker2 # [ 119.988624] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:333526055worker1 # [ 120.011278] nix-grpc-daemon[1478]: ts=2026-09-20T21:10:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:373866056worker2 # [ 120.013910] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:13Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06057worker2 # [ 120.019428] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:13Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=16058worker2 # [ 120.057885] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333526059worker2 # [ 120.070619] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:333526060client # copying 1 paths to 'lb:50051'...6061worker2 # [ 120.077405] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:333526062client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6063worker2 # [ 120.101366] nix-daemon[871]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6064worker2 # [ 120.286306] nix-daemon[871]: accepted connection from pid 1549, user nix-grpc-daemon (trusted)6065worker2 # [ 120.298837] nix-daemon[871]: reaped child process 1552, status = succeeded6066worker1 # [ 120.325096] nix-grpc-daemon[1478]: ts=2026-09-20T21:10:13Z level=info event=draining6067worker2 # [ 120.488169] nix-daemon[871]: accepted connection from pid 1554, user nix-grpc-daemon (trusted)6068worker2 # [ 120.501338] nix-daemon[871]: reaped child process 1557, status = succeeded6069lb # [ 120.514591] niks3-server[1065]: 2026/09/20 21:10:14 INFO Received uploads request method=POST path=/api/pending_closures6070worker2 # [ 120.535670] nix-grpc-daemon[1470]: time=2026-09-20T21:10:14.290Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6071worker2 # [ 120.537735] nix-grpc-daemon[1470]: time=2026-09-20T21:10:14.292Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6072lb # [ 120.562202] niks3-server[1065]: 2026/09/20 21:10:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6073lb # [ 120.573627] niks3-server[1065]: 2026/09/20 21:10:14 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6074worker2 # [ 120.584751] nix-grpc-daemon[1470]: time=2026-09-20T21:10:14.339Z level=INFO msg="Uploading 1 narinfos"6075lb # [ 120.582138] niks3-server[1065]: 2026/09/20 21:10:14 INFO Signed narinfos id=39 count=16076lb # [ 120.599123] niks3-server[1065]: 2026/09/20 21:10:14 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6077lb # [ 120.607365] niks3-server[1065]: 2026/09/20 21:10:14 INFO Completed upload id=396078worker2 # [ 120.612576] nix-grpc-daemon[1470]: time=2026-09-20T21:10:14.367Z level=INFO msg="Upload complete. (499ms)"6079lb # [ 120.612449] niks3-server[1065]: 2026/09/20 21:10:14 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6080worker2 # [ 120.618170] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33352 duration_s=0 paths=1 nar_bytes_in=5286081lb # [ 120.617636] niks3-server[1065]: 2026/09/20 21:10:14 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6082worker2 # [ 120.624264] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333526083worker2 # [ 120.636270] nix-daemon[871]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6084worker2 # [ 120.644781] nix-daemon[871]: reaped child process 1547, status = succeeded6085client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6086worker2 # [ 120.652988] nix-daemon[871]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6087client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6088client # farm-a> LOG-farm-a-during-drain6089worker2 # [ 121.882347] nix-daemon[871]: reaped child process 1561, status = succeeded6090worker2 # [ 122.044714] nix-daemon[871]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)6091worker2 # [ 122.057557] nix-daemon[871]: reaped child process 1579, status = succeeded6092lb # [ 122.069616] niks3-server[1065]: 2026/09/20 21:10:16 INFO Received uploads request method=POST path=/api/pending_closures6093worker2 # [ 122.171472] nix-grpc-daemon[1470]: time=2026-09-20T21:10:15.925Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6094worker2 # [ 122.181942] nix-grpc-daemon[1470]: time=2026-09-20T21:10:15.930Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6095lb # [ 122.195145] niks3-server[1065]: 2026/09/20 21:10:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6096lb # [ 122.197180] niks3-server[1065]: 2026/09/20 21:10:16 INFO Signed narinfos id=40 count=16097worker2 # [ 122.204148] nix-grpc-daemon[1470]: time=2026-09-20T21:10:15.958Z level=INFO msg="Uploading 1 narinfos"6098lb # [ 122.200572] niks3-server[1065]: 2026/09/20 21:10:16 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6099lb # [ 122.208586] niks3-server[1065]: 2026/09/20 21:10:16 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6100lb # [ 122.211737] niks3-server[1065]: 2026/09/20 21:10:16 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6101lb # [ 122.215822] niks3-server[1065]: 2026/09/20 21:10:16 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6102lb # [ 122.218144] niks3-server[1065]: 2026/09/20 21:10:16 INFO Completed upload id=406103worker2 # [ 122.223333] nix-grpc-daemon[1470]: time=2026-09-20T21:10:15.978Z level=INFO msg="Upload complete. (358ms)"6104lb # [ 122.220138] niks3-server[1065]: 2026/09/20 21:10:16 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6105worker2 # [ 122.232712] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33352 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16106worker2 # [ 122.241951] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:15Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06107worker2 # [ 122.245898] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:15Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=16108worker2 # [ 122.254518] nix-daemon[871]: reaped child process 1559, status = succeeded6109worker2 # [ 122.285884] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333526110worker2 # [ 122.298695] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:333526111worker2 # [ 122.312790] nix-grpc-daemon[1470]: time=2026-09-20T21:10:16.067Z level=INFO msg="All 1 paths already cached"6112client # copying 1 paths to 'lb:50051'...6113client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6114worker2 # [ 122.321075] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:333526115worker2 # [ 122.335161] nix-daemon[871]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6116worker2 # [ 122.488708] nix-daemon[871]: accepted connection from pid 1585, user nix-grpc-daemon (trusted)6117worker2 # [ 122.501325] nix-daemon[871]: reaped child process 1588, status = succeeded6118worker2 # [ 122.655853] nix-daemon[871]: accepted connection from pid 1590, user nix-grpc-daemon (trusted)6119worker2 # [ 122.667846] nix-daemon[871]: reaped child process 1593, status = succeeded6120lb # [ 122.679392] niks3-server[1065]: 2026/09/20 21:10:16 INFO Received uploads request method=POST path=/api/pending_closures6121worker2 # [ 122.699420] nix-grpc-daemon[1470]: time=2026-09-20T21:10:16.454Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6122worker2 # [ 122.700857] nix-grpc-daemon[1470]: time=2026-09-20T21:10:16.454Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6123lb # [ 122.713225] niks3-server[1065]: 2026/09/20 21:10:16 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6124lb # [ 122.724923] niks3-server[1065]: 2026/09/20 21:10:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6125worker2 # [ 122.734490] nix-grpc-daemon[1470]: time=2026-09-20T21:10:16.489Z level=INFO msg="Uploading 1 narinfos"6126lb # [ 122.729569] niks3-server[1065]: 2026/09/20 21:10:16 INFO Signed narinfos id=41 count=16127lb # [ 122.735970] niks3-server[1065]: 2026/09/20 21:10:16 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6128lb # [ 122.745391] niks3-server[1065]: 2026/09/20 21:10:16 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6129lb # [ 122.750441] niks3-server[1065]: 2026/09/20 21:10:16 INFO Completed upload id=416130worker2 # [ 122.755567] nix-grpc-daemon[1470]: time=2026-09-20T21:10:16.510Z level=INFO msg="Upload complete. (408ms)"6131lb # [ 122.754510] niks3-server[1065]: 2026/09/20 21:10:16 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6132worker2 # [ 122.759348] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33352 duration_s=0 paths=1 nar_bytes_in=6406133worker2 # [ 122.764280] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333526134worker2 # [ 122.774075] nix-daemon[871]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6135worker2 # [ 122.780154] nix-daemon[871]: reaped child process 1583, status = succeeded6136worker2 # [ 122.783400] nix-daemon[871]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6137client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6138client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6139client # farm-b> LOG-farm-b-during-drain6140worker2 # [ 124.009760] nix-daemon[871]: reaped child process 1597, status = succeeded6141worker2 # [ 124.178076] nix-daemon[871]: accepted connection from pid 1612, user nix-grpc-daemon (trusted)6142worker2 # [ 124.192718] nix-daemon[871]: reaped child process 1615, status = succeeded6143lb # [ 124.200789] niks3-server[1065]: 2026/09/20 21:10:18 INFO Received uploads request method=POST path=/api/pending_closures6144worker2 # [ 124.211336] nix-grpc-daemon[1470]: time=2026-09-20T21:10:17.966Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6145worker2 # [ 124.213924] nix-grpc-daemon[1470]: time=2026-09-20T21:10:17.968Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6146lb # [ 124.221182] niks3-server[1065]: 2026/09/20 21:10:18 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6147lb # [ 124.223749] niks3-server[1065]: 2026/09/20 21:10:18 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6148lb # [ 124.228239] niks3-server[1065]: 2026/09/20 21:10:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6149lb # [ 124.230343] niks3-server[1065]: 2026/09/20 21:10:18 INFO Signed narinfos id=42 count=16150worker2 # [ 124.236711] nix-grpc-daemon[1470]: time=2026-09-20T21:10:17.991Z level=INFO msg="Uploading 1 narinfos"6151lb # [ 124.233156] niks3-server[1065]: 2026/09/20 21:10:18 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6152lb # [ 124.239216] niks3-server[1065]: 2026/09/20 21:10:18 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6153lb # [ 124.242021] niks3-server[1065]: 2026/09/20 21:10:18 INFO Completed upload id=426154worker2 # [ 124.248425] nix-grpc-daemon[1470]: time=2026-09-20T21:10:18.003Z level=INFO msg="Upload complete. (256ms)"6155lb # [ 124.244161] niks3-server[1065]: 2026/09/20 21:10:18 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6156worker2 # [ 124.252264] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33352 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16157worker2 # [ 124.258388] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:18Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06158worker2 # [ 124.260610] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:18Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=16159worker2 # [ 124.265736] nix-daemon[871]: reaped child process 1595, status = succeeded6160worker2 # [ 124.302314] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333526161worker2 # [ 124.310321] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:333526162worker2 # [ 124.317991] nix-grpc-daemon[1470]: time=2026-09-20T21:10:18.073Z level=INFO msg="All 2 paths already cached"6163client # copying 1 paths to 'lb:50051'...6164client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6165worker2 # [ 124.322725] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:333526166worker2 # [ 124.329587] nix-daemon[871]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6167worker2 # [ 124.487264] nix-daemon[871]: accepted connection from pid 1621, user nix-grpc-daemon (trusted)6168worker2 # [ 124.501934] nix-daemon[871]: reaped child process 1624, status = succeeded6169worker2 # [ 124.678093] nix-daemon[871]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)6170worker2 # [ 124.689504] nix-daemon[871]: reaped child process 1629, status = succeeded6171lb # [ 124.701590] niks3-server[1065]: 2026/09/20 21:10:18 INFO Received uploads request method=POST path=/api/pending_closures6172worker2 # [ 124.741171] nix-grpc-daemon[1470]: time=2026-09-20T21:10:18.496Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6173worker2 # [ 124.743138] nix-grpc-daemon[1470]: time=2026-09-20T21:10:18.498Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6174lb # [ 124.767610] niks3-server[1065]: 2026/09/20 21:10:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6175lb # [ 124.773656] niks3-server[1065]: 2026/09/20 21:10:18 INFO Signed narinfos id=43 count=16176worker2 # [ 124.781918] nix-grpc-daemon[1470]: time=2026-09-20T21:10:18.536Z level=INFO msg="Uploading 1 narinfos"6177lb # [ 124.780681] niks3-server[1065]: 2026/09/20 21:10:18 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6178lb # [ 124.795189] niks3-server[1065]: 2026/09/20 21:10:18 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6179lb # [ 124.800200] niks3-server[1065]: 2026/09/20 21:10:18 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6180lb # [ 124.856259] niks3-server[1065]: 2026/09/20 21:10:18 INFO Completed upload id=436181worker2 # [ 124.861587] nix-grpc-daemon[1470]: time=2026-09-20T21:10:18.616Z level=INFO msg="Upload complete. (524ms)"6182worker2 # [ 124.866656] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33352 duration_s=0 paths=1 nar_bytes_in=7686183worker2 # [ 124.874356] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333526184lb # [ 124.872621] niks3-server[1065]: 2026/09/20 21:10:18 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6185worker2 # [ 124.886187] nix-daemon[871]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6186worker2 # [ 124.897244] nix-daemon[871]: reaped child process 1619, status = succeeded6187client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6188client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6189worker2 # [ 124.908980] nix-daemon[871]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6190client # farm-top> LOG-farm-top-during-drain6191worker2 # [ 126.139800] nix-daemon[871]: reaped child process 1633, status = succeeded6192worker2 # [ 126.310497] nix-daemon[871]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6193worker2 # [ 126.322318] nix-daemon[871]: reaped child process 1651, status = succeeded6194lb # [ 126.334988] niks3-server[1065]: 2026/09/20 21:10:20 INFO Received uploads request method=POST path=/api/pending_closures6195worker2 # [ 126.356649] nix-grpc-daemon[1470]: time=2026-09-20T21:10:20.111Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6196worker2 # [ 126.358226] nix-grpc-daemon[1470]: time=2026-09-20T21:10:20.113Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6197lb # [ 126.385073] niks3-server[1065]: 2026/09/20 21:10:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6198lb # [ 126.388725] niks3-server[1065]: 2026/09/20 21:10:20 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6199lb # [ 126.393309] niks3-server[1065]: 2026/09/20 21:10:20 INFO Signed narinfos id=44 count=16200worker2 # [ 126.400915] nix-grpc-daemon[1470]: time=2026-09-20T21:10:20.155Z level=INFO msg="Uploading 1 narinfos"6201lb # [ 126.396092] niks3-server[1065]: 2026/09/20 21:10:20 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6202lb # [ 126.406553] niks3-server[1065]: 2026/09/20 21:10:20 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6203lb # [ 126.409871] niks3-server[1065]: 2026/09/20 21:10:20 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6204lb # [ 126.415757] niks3-server[1065]: 2026/09/20 21:10:20 INFO Completed upload id=446205worker2 # [ 126.420983] nix-grpc-daemon[1470]: time=2026-09-20T21:10:20.176Z level=INFO msg="Upload complete. (299ms)"6206lb # [ 126.418077] niks3-server[1065]: 2026/09/20 21:10:20 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6207worker2 # [ 126.429463] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33352 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16208worker2 # [ 126.440442] nix-daemon[871]: reaped child process 1631, status = succeeded6209client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&2, in 6.70 seconds)6210client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath6211client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath, in 0.18 seconds)6212worker1: must succeed: pkill -f 'read -t [9]0 x'6213worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6214client: waiting for success: ! systemctl is-active sw6215worker1 # [ 126.874431] nix-daemon[851]: reaped child process 1635, status = succeeded6216worker1 # [ 127.027789] nix-daemon[851]: accepted connection from pid 1676, user nix-grpc-daemon (trusted)6217worker1 # [ 127.040314] nix-daemon[851]: reaped child process 1679, status = succeeded6218lb # [ 127.034279] niks3-server[1065]: 2026/09/20 21:10:21 INFO Received uploads request method=POST path=/api/pending_closures6219worker1 # [ 127.105379] nix-grpc-daemon[1575]: time=2026-09-20T21:10:20.409Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6220worker1 # [ 127.107654] nix-grpc-daemon[1575]: time=2026-09-20T21:10:20.411Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6221lb # [ 127.107575] niks3-server[1065]: 2026/09/20 21:10:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6222lb # [ 127.112422] niks3-server[1065]: 2026/09/20 21:10:21 INFO Signed narinfos id=45 count=16223worker1 # [ 127.135202] nix-grpc-daemon[1575]: time=2026-09-20T21:10:20.439Z level=INFO msg="Uploading 1 narinfos"6224lb # [ 127.115298] niks3-server[1065]: 2026/09/20 21:10:21 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6225lb # [ 127.118862] niks3-server[1065]: 2026/09/20 21:10:21 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6226lb # [ 127.128286] niks3-server[1065]: 2026/09/20 21:10:21 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6227lb # [ 127.136087] niks3-server[1065]: 2026/09/20 21:10:21 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6228lb # [ 127.140257] niks3-server[1065]: 2026/09/20 21:10:21 INFO Completed upload id=456229worker1 # [ 127.161588] nix-grpc-daemon[1575]: time=2026-09-20T21:10:20.466Z level=INFO msg="Upload complete. (308ms)"6230lb # [ 127.143049] niks3-server[1065]: 2026/09/20 21:10:21 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6231worker1 # [ 127.170657] nix-grpc-daemon[1478]: ts=2026-09-20T21:10:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37386 duration_s=9 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16232worker1 # [ 127.185577] nix-daemon[851]: reaped child process 1633, status = succeeded6233client # [ 127.198859] systemd[1]: sw.service: Deactivated successfully.6234client # [ 127.203152] systemd[1]: sw.service: Consumed 135ms CPU time over 10.071s wall clock time, 39.4M memory peak, 6.9K incoming IP traffic, 8.2K outgoing IP traffic.6235worker1 # [ 127.328042] nix-grpc-daemon[1478]: ts=2026-09-20T21:10:20Z level=info event=signal_exit6236client: (finished: waiting for success: ! systemctl is-active sw, in 1.09 seconds)6237client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6238client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.04 seconds)6239worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6240worker2 # [ 128.341486] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:22Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:333426241worker2 # [ 128.345267] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:22Z level=info event=worker_gone addr=192.168.1.3:500516242worker1 # [ 128.364333] nix-grpc-daemon[1478]: ts=2026-09-20T21:10:21Z level=info event=scheduler_restarting6243worker1 # [ 128.372303] nix-grpc-daemon[1478]: ts=2026-09-20T21:10:21Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6244worker1 # [ 128.398558] nix-daemon[851]: reaped child process 1531, status = succeeded6245lb # [ 128.629269] envoy[973]: conn peer=[2001:db8:1::1]:45580 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9611 tx=3834 ms=64536246lb # [ 128.633205] envoy[973]: conn peer=[2001:db8:1::1]:45564 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2949 tx=1326 ms=98226247lb # [ 128.636647] envoy[973]: conn peer=[2001:db8:1::3]:34900 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1114 tx=425 ms=260186248worker1 # [ 129.181132] nix-grpc-daemon[1478]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6249worker1 # [ 129.184643] nix-grpc-daemon[1478]: E0000 00:00:1789938622.485217 1521 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true6250worker1 # [ 129.189542] nix-grpc-daemon[1478]: E0000 00:00:1789938622.485692 1521 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6251worker1 # [ 130.364564] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6252worker1 # [ 130.368528] systemd[1]: nix-grpc-daemon.service: Consumed 707ms CPU time over 28.225s wall clock time, 32.6M memory peak, 21.3K incoming IP traffic, 27.9K outgoing IP traffic.6253worker1 # [ 130.386299] systemd[1]: Starting Nix worker-protocol over gRPC...6254worker1 # [ 130.553253] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:23Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6255worker1 # [ 130.555210] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:23Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6256worker1 # [ 130.558777] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:23Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16257worker1 # [ 130.566059] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:23Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6258worker1 # [ 130.570199] systemd[1]: Started Nix worker-protocol over gRPC.6259worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.74 seconds)6260lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6261lb: must succeed: curl -sf localhost:9901/clusters6262worker1 # [ 130.598980] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:23Z level=info event=scheduler_connected addr=lb:500516263worker2 # [ 130.584359] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:24Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:333426264worker2 # [ 130.586634] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:24Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06265lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6266lb: must succeed: curl -sf localhost:9901/clusters6267lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6268lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.11 seconds)6269lb: must succeed: curl -sf localhost:9901/clusters6270lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6271worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6272worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6273(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 14.73 seconds)6274subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6275client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16276worker1 # [ 132.035947] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:342766277worker1 # [ 132.048303] nix-daemon[851]: accepted connection from pid 1693, user nix-grpc-daemon (trusted)6278worker1 # [ 132.062106] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:342766279worker1 # [ 132.073380] nix-grpc-daemon[1693]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 26 ms (attempt 1/5)6280worker1 # [ 132.104946] nix-grpc-daemon[1693]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 140 ms (attempt 2/5)6281worker1 # [ 132.247834] nix-grpc-daemon[1693]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 168 ms (attempt 3/5)6282worker1 # [ 132.422114] nix-grpc-daemon[1693]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 224 ms (attempt 4/5)6283worker1 # [ 132.653537] nix-grpc-daemon[1693]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6284worker2 # [ 132.684774] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:26Z level=debug event=want client=5 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06285worker2 # [ 132.689112] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:26Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16286worker2 # [ 132.693437] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:26Z level=debug event=want client=5 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06287worker2 # [ 132.697319] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:26Z level=debug event=want client=5 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06288worker1 # [ 132.745373] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:342766289worker1 # [ 132.753526] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:342766290worker1 # [ 132.760479] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:342766291worker1 # [ 132.767069] nix-daemon[851]: accepted connection from pid 1693, user nix-grpc-daemon (trusted)6292worker1 # [ 132.809384] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:26Z level=info event=niks3_push_started pid=17496293worker1 # [ 133.010438] nix-grpc-daemon[1749]: time=2026-09-20T21:10:26.313Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.key ca=""6294worker1 # [ 133.180271] nix-daemon[851]: accepted connection from pid 1762, user nix-grpc-daemon (trusted)6295worker1 # [ 133.192608] nix-daemon[851]: reaped child process 1765, status = succeeded6296worker1 # [ 133.383115] nix-daemon[851]: accepted connection from pid 1767, user nix-grpc-daemon (trusted)6297worker1 # [ 133.396605] nix-daemon[851]: reaped child process 1770, status = succeeded6298lb # [ 133.390841] niks3-server[1065]: 2026/09/20 21:10:27 INFO Received uploads request method=POST path=/api/pending_closures6299worker1 # [ 133.419365] nix-grpc-daemon[1749]: time=2026-09-20T21:10:26.724Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6300worker1 # [ 133.420848] nix-grpc-daemon[1749]: time=2026-09-20T21:10:26.725Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6301lb # [ 133.417283] niks3-server[1065]: 2026/09/20 21:10:27 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6302lb # [ 133.420232] niks3-server[1065]: 2026/09/20 21:10:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6303lb # [ 133.422162] niks3-server[1065]: 2026/09/20 21:10:27 INFO Signed narinfos id=46 count=16304worker1 # [ 133.445661] nix-grpc-daemon[1749]: time=2026-09-20T21:10:26.750Z level=INFO msg="Uploading 1 narinfos"6305lb # [ 133.426849] niks3-server[1065]: 2026/09/20 21:10:27 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6306lb # [ 133.432321] niks3-server[1065]: 2026/09/20 21:10:27 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6307lb # [ 133.436890] niks3-server[1065]: 2026/09/20 21:10:27 INFO Completed upload id=466308worker1 # [ 133.458508] nix-grpc-daemon[1749]: time=2026-09-20T21:10:26.763Z level=INFO msg="Upload complete. (444ms)"6309lb # [ 133.438733] niks3-server[1065]: 2026/09/20 21:10:27 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6310worker1 # [ 133.460627] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34276 duration_s=0 paths=1 nar_bytes_in=5046311worker1 # [ 133.466132] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:342766312worker1 # [ 133.472353] nix-daemon[851]: accepted connection from pid 1693, user nix-grpc-daemon (trusted)6313worker1 # [ 133.474781] nix-daemon[851]: reaped child process 1747, status = succeeded6314worker1 # [ 133.480533] nix-daemon[851]: accepted connection from pid 1693, user nix-grpc-daemon (trusted)6315worker1 # [ 134.910678] nix-daemon[851]: reaped child process 1774, status = succeeded6316worker1 # [ 135.084618] nix-daemon[851]: accepted connection from pid 1789, user nix-grpc-daemon (trusted)6317worker1 # [ 135.095994] nix-daemon[851]: reaped child process 1792, status = succeeded6318lb # [ 135.092189] niks3-server[1065]: 2026/09/20 21:10:29 INFO Received uploads request method=POST path=/api/pending_closures6319worker1 # [ 135.139148] nix-grpc-daemon[1749]: time=2026-09-20T21:10:28.443Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6320worker1 # [ 135.140729] nix-grpc-daemon[1749]: time=2026-09-20T21:10:28.445Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6321lb # [ 135.141202] niks3-server[1065]: 2026/09/20 21:10:29 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6322lb # [ 135.147075] niks3-server[1065]: 2026/09/20 21:10:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6323lb # [ 135.149464] niks3-server[1065]: 2026/09/20 21:10:29 INFO Signed narinfos id=47 count=16324worker1 # [ 135.171735] nix-grpc-daemon[1749]: time=2026-09-20T21:10:28.475Z level=INFO msg="Uploading 1 narinfos"6325lb # [ 135.157580] niks3-server[1065]: 2026/09/20 21:10:29 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6326lb # [ 135.163688] niks3-server[1065]: 2026/09/20 21:10:29 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6327lb # [ 135.165976] niks3-server[1065]: 2026/09/20 21:10:29 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6328lb # [ 135.171264] niks3-server[1065]: 2026/09/20 21:10:29 INFO Completed upload id=476329worker1 # [ 135.192843] nix-grpc-daemon[1749]: time=2026-09-20T21:10:28.497Z level=INFO msg="Upload complete. (302ms)"6330lb # [ 135.173785] niks3-server[1065]: 2026/09/20 21:10:29 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6331worker2 # [ 135.182416] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:28Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16332worker1 # [ 135.197598] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34276 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=7 outputs=16333worker2 # [ 135.187782] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:28Z level=debug event=want client=5 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06334worker1 # [ 135.205155] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:342766335worker1 # [ 135.215295] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:342766336worker1 # [ 135.218057] nix-daemon[851]: reaped child process 1772, status = succeeded6337worker1 # [ 135.221270] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:342766338worker1 # [ 135.225450] nix-daemon[851]: accepted connection from pid 1693, user nix-grpc-daemon (trusted)6339worker1 # [ 135.394521] nix-daemon[851]: accepted connection from pid 1796, user nix-grpc-daemon (trusted)6340worker1 # [ 135.407818] nix-daemon[851]: reaped child process 1799, status = succeeded6341worker1 # [ 135.587326] nix-daemon[851]: accepted connection from pid 1801, user nix-grpc-daemon (trusted)6342worker1 # [ 135.600138] nix-daemon[851]: reaped child process 1804, status = succeeded6343lb # [ 135.594957] niks3-server[1065]: 2026/09/20 21:10:29 INFO Received uploads request method=POST path=/api/pending_closures6344worker1 # [ 135.626656] nix-grpc-daemon[1749]: time=2026-09-20T21:10:28.931Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6345worker1 # [ 135.628201] nix-grpc-daemon[1749]: time=2026-09-20T21:10:28.932Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6346lb # [ 135.618700] niks3-server[1065]: 2026/09/20 21:10:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6347lb # [ 135.621316] niks3-server[1065]: 2026/09/20 21:10:29 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6348lb # [ 135.623505] niks3-server[1065]: 2026/09/20 21:10:29 INFO Signed narinfos id=48 count=16349worker1 # [ 135.645178] nix-grpc-daemon[1749]: time=2026-09-20T21:10:28.949Z level=INFO msg="Uploading 1 narinfos"6350lb # [ 135.627954] niks3-server[1065]: 2026/09/20 21:10:29 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6351lb # [ 135.631329] niks3-server[1065]: 2026/09/20 21:10:29 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6352lb # [ 135.635520] niks3-server[1065]: 2026/09/20 21:10:29 INFO Completed upload id=486353worker1 # [ 135.658318] nix-grpc-daemon[1749]: time=2026-09-20T21:10:28.963Z level=INFO msg="Upload complete. (426ms)"6354lb # [ 135.637671] niks3-server[1065]: 2026/09/20 21:10:29 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6355worker1 # [ 135.660371] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34276 duration_s=0 paths=1 nar_bytes_in=5046356worker1 # [ 135.663959] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:342766357worker1 # [ 135.670566] nix-daemon[851]: accepted connection from pid 1693, user nix-grpc-daemon (trusted)6358worker1 # [ 135.672892] nix-daemon[851]: reaped child process 1794, status = succeeded6359worker1 # [ 135.678354] nix-daemon[851]: accepted connection from pid 1693, user nix-grpc-daemon (trusted)6360worker1 # [ 136.893507] nix-daemon[851]: reaped child process 1808, status = succeeded6361worker1 # [ 137.029965] nix-daemon[851]: accepted connection from pid 1823, user nix-grpc-daemon (trusted)6362worker1 # [ 137.042413] nix-daemon[851]: reaped child process 1826, status = succeeded6363lb # [ 137.033623] niks3-server[1065]: 2026/09/20 21:10:31 INFO Received uploads request method=POST path=/api/pending_closures6364worker1 # [ 137.060746] nix-grpc-daemon[1749]: time=2026-09-20T21:10:30.364Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6365worker1 # [ 137.062532] nix-grpc-daemon[1749]: time=2026-09-20T21:10:30.364Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6366lb # [ 137.052080] niks3-server[1065]: 2026/09/20 21:10:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6367lb # [ 137.054107] niks3-server[1065]: 2026/09/20 21:10:31 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6368lb # [ 137.056549] niks3-server[1065]: 2026/09/20 21:10:31 INFO Signed narinfos id=49 count=16369worker1 # [ 137.078146] nix-grpc-daemon[1749]: time=2026-09-20T21:10:30.382Z level=INFO msg="Uploading 1 narinfos"6370lb # [ 137.058727] niks3-server[1065]: 2026/09/20 21:10:31 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6371lb # [ 137.064074] niks3-server[1065]: 2026/09/20 21:10:31 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6372lb # [ 137.066934] niks3-server[1065]: 2026/09/20 21:10:31 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6373lb # [ 137.069754] niks3-server[1065]: 2026/09/20 21:10:31 INFO Completed upload id=496374worker1 # [ 137.092536] nix-grpc-daemon[1749]: time=2026-09-20T21:10:30.397Z level=INFO msg="Upload complete. (207ms)"6375lb # [ 137.071888] niks3-server[1065]: 2026/09/20 21:10:31 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6376worker1 # [ 137.095610] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34276 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=8 outputs=16377worker2 # [ 137.081284] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:30Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16378worker1 # [ 137.103954] nix-daemon[851]: reaped child process 1806, status = succeeded6379worker2 # [ 137.091225] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:30Z level=debug event=want client=5 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06380worker1 # [ 137.140129] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:342766381worker1 # [ 137.147066] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:342766382worker1 # [ 137.154607] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:342766383worker1 # [ 137.159336] nix-daemon[851]: accepted connection from pid 1693, user nix-grpc-daemon (trusted)6384worker1 # [ 137.345381] nix-daemon[851]: accepted connection from pid 1830, user nix-grpc-daemon (trusted)6385worker1 # [ 137.358167] nix-daemon[851]: reaped child process 1833, status = succeeded6386worker1 # [ 137.518109] nix-daemon[851]: accepted connection from pid 1835, user nix-grpc-daemon (trusted)6387worker1 # [ 137.530885] nix-daemon[851]: reaped child process 1838, status = succeeded6388lb # [ 137.523353] niks3-server[1065]: 2026/09/20 21:10:31 INFO Received uploads request method=POST path=/api/pending_closures6389worker1 # [ 137.671166] nix-grpc-daemon[1749]: time=2026-09-20T21:10:30.975Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6390worker1 # [ 137.676568] nix-grpc-daemon[1749]: time=2026-09-20T21:10:30.979Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6391lb # [ 137.665924] niks3-server[1065]: 2026/09/20 21:10:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6392lb # [ 137.671106] niks3-server[1065]: 2026/09/20 21:10:31 INFO Signed narinfos id=50 count=16393worker1 # [ 137.695591] nix-grpc-daemon[1749]: time=2026-09-20T21:10:30.999Z level=INFO msg="Uploading 1 narinfos"6394lb # [ 137.678432] niks3-server[1065]: 2026/09/20 21:10:31 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6395lb # [ 137.682727] niks3-server[1065]: 2026/09/20 21:10:31 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6396lb # [ 137.688644] niks3-server[1065]: 2026/09/20 21:10:31 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6397lb # [ 137.691343] niks3-server[1065]: 2026/09/20 21:10:31 INFO Completed upload id=506398worker1 # [ 137.713184] nix-grpc-daemon[1749]: time=2026-09-20T21:10:31.017Z level=INFO msg="Upload complete. (546ms)"6399lb # [ 137.694130] niks3-server[1065]: 2026/09/20 21:10:31 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6400worker1 # [ 137.718193] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34276 duration_s=0 paths=1 nar_bytes_in=5046401worker1 # [ 137.723602] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:342766402worker1 # [ 137.734423] nix-daemon[851]: accepted connection from pid 1693, user nix-grpc-daemon (trusted)6403worker1 # [ 137.747120] nix-daemon[851]: reaped child process 1828, status = succeeded6404worker1 # [ 137.754763] nix-daemon[851]: accepted connection from pid 1693, user nix-grpc-daemon (trusted)6405worker1 # [ 138.573706] nix-daemon[851]: reaped child process 1842, status = succeeded6406worker1 # [ 138.725447] nix-daemon[851]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6407worker1 # [ 138.739535] nix-daemon[851]: reaped child process 1860, status = succeeded6408lb # [ 138.733766] niks3-server[1065]: 2026/09/20 21:10:32 INFO Received uploads request method=POST path=/api/pending_closures6409worker1 # [ 138.766175] nix-grpc-daemon[1749]: time=2026-09-20T21:10:32.070Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6410worker1 # [ 138.767921] nix-grpc-daemon[1749]: time=2026-09-20T21:10:32.072Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6411lb # [ 138.775135] niks3-server[1065]: 2026/09/20 21:10:32 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6412lb # [ 138.781983] niks3-server[1065]: 2026/09/20 21:10:32 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv6413lb # [ 138.790639] niks3-server[1065]: 2026/09/20 21:10:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6414worker1 # [ 138.817450] nix-grpc-daemon[1749]: time=2026-09-20T21:10:32.121Z level=INFO msg="Uploading 1 narinfos"6415lb # [ 138.795595] niks3-server[1065]: 2026/09/20 21:10:32 INFO Signed narinfos id=51 count=16416lb # [ 138.802744] niks3-server[1065]: 2026/09/20 21:10:32 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls6417lb # [ 138.810174] niks3-server[1065]: 2026/09/20 21:10:32 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6418lb # [ 138.814733] niks3-server[1065]: 2026/09/20 21:10:32 INFO Completed upload id=516419worker1 # [ 138.836175] nix-grpc-daemon[1749]: time=2026-09-20T21:10:32.140Z level=INFO msg="Upload complete. (271ms)"6420lb # [ 138.817572] niks3-server[1065]: 2026/09/20 21:10:32 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo6421worker1 # [ 138.839314] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34276 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=9 outputs=16422worker2 # [ 138.823314] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:32Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=16423worker1 # [ 138.844385] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:342766424worker2 # [ 138.829062] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:32Z level=debug event=want client=5 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=06425worker1 # [ 138.849990] nix-daemon[851]: reaped child process 1840, status = succeeded6426worker1 # [ 138.853957] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:342766427worker1 # [ 138.861960] nix-grpc-daemon[1749]: time=2026-09-20T21:10:32.166Z level=INFO msg="All 1 paths already cached"6428worker1 # [ 138.865696] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:342766429worker1 # [ 138.869909] nix-daemon[851]: accepted connection from pid 1693, user nix-grpc-daemon (trusted)6430worker1 # [ 139.015413] nix-daemon[851]: accepted connection from pid 1864, user nix-grpc-daemon (trusted)6431worker1 # [ 139.029499] nix-daemon[851]: reaped child process 1867, status = succeeded6432worker1 # [ 139.187419] nix-daemon[851]: accepted connection from pid 1869, user nix-grpc-daemon (trusted)6433worker1 # [ 139.200709] nix-daemon[851]: reaped child process 1872, status = succeeded6434lb # [ 139.194456] niks3-server[1065]: 2026/09/20 21:10:33 INFO Received uploads request method=POST path=/api/pending_closures6435worker1 # [ 139.225395] nix-grpc-daemon[1749]: time=2026-09-20T21:10:32.530Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6436worker1 # [ 139.227083] nix-grpc-daemon[1749]: time=2026-09-20T21:10:32.531Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"6437lb # [ 139.223857] niks3-server[1065]: 2026/09/20 21:10:33 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst6438lb # [ 139.238121] niks3-server[1065]: 2026/09/20 21:10:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6439lb # [ 139.244091] niks3-server[1065]: 2026/09/20 21:10:33 INFO Signed narinfos id=52 count=16440worker1 # [ 139.268549] nix-grpc-daemon[1749]: time=2026-09-20T21:10:32.573Z level=INFO msg="Uploading 1 narinfos"6441lb # [ 139.247085] niks3-server[1065]: 2026/09/20 21:10:33 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls6442lb # [ 139.258298] niks3-server[1065]: 2026/09/20 21:10:33 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6443lb # [ 139.263487] niks3-server[1065]: 2026/09/20 21:10:33 INFO Completed upload id=526444lb # [ 139.265136] niks3-server[1065]: 2026/09/20 21:10:33 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo6445worker1 # [ 139.285384] nix-grpc-daemon[1749]: time=2026-09-20T21:10:32.589Z level=INFO msg="Upload complete. (408ms)"6446worker1 # [ 139.291100] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34276 duration_s=0 paths=1 nar_bytes_in=6246447worker1 # [ 139.295285] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:342766448worker1 # [ 139.304312] nix-daemon[851]: accepted connection from pid 1693, user nix-grpc-daemon (trusted)6449worker1 # [ 139.307058] nix-daemon[851]: reaped child process 1862, status = succeeded6450worker1 # [ 139.313988] nix-daemon[851]: accepted connection from pid 1693, user nix-grpc-daemon (trusted)6451worker1 # [ 139.935318] nix-daemon[851]: reaped child process 1876, status = succeeded6452worker1 # [ 140.103690] nix-daemon[851]: accepted connection from pid 1891, user nix-grpc-daemon (trusted)6453worker1 # [ 140.117177] nix-daemon[851]: reaped child process 1894, status = succeeded6454lb # [ 140.114560] niks3-server[1065]: 2026/09/20 21:10:34 INFO Received uploads request method=POST path=/api/pending_closures6455worker1 # [ 140.147344] nix-grpc-daemon[1749]: time=2026-09-20T21:10:33.450Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6456worker1 # [ 140.148690] nix-grpc-daemon[1749]: time=2026-09-20T21:10:33.451Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"6457lb # [ 140.151245] niks3-server[1065]: 2026/09/20 21:10:34 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv6458lb # [ 140.164447] niks3-server[1065]: 2026/09/20 21:10:34 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst6459lb # [ 140.176947] niks3-server[1065]: 2026/09/20 21:10:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6460lb # [ 140.181499] niks3-server[1065]: 2026/09/20 21:10:34 INFO Signed narinfos id=53 count=16461worker1 # [ 140.203775] nix-grpc-daemon[1749]: time=2026-09-20T21:10:33.507Z level=INFO msg="Uploading 1 narinfos"6462lb # [ 140.188146] niks3-server[1065]: 2026/09/20 21:10:34 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls6463lb # [ 140.198244] niks3-server[1065]: 2026/09/20 21:10:34 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6464lb # [ 140.203223] niks3-server[1065]: 2026/09/20 21:10:34 INFO Completed upload id=536465worker1 # [ 140.224983] nix-grpc-daemon[1749]: time=2026-09-20T21:10:33.529Z level=INFO msg="Upload complete. (304ms)"6466lb # [ 140.204931] niks3-server[1065]: 2026/09/20 21:10:34 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo6467worker1 # [ 140.231325] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34276 duration_s=0 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=10 outputs=16468worker2 # [ 140.217256] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:33Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=16469worker1 # [ 140.238464] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:342766470worker2 # [ 140.224138] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:33Z level=debug event=want client=5 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=06471worker1 # [ 140.250554] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:342766472worker1 # [ 140.257940] nix-daemon[851]: reaped child process 1874, status = succeeded6473worker1 # [ 140.261304] nix-grpc-daemon[1749]: time=2026-09-20T21:10:33.565Z level=INFO msg="All 1 paths already cached"6474worker1 # [ 140.266878] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:342766475worker1 # [ 140.273888] nix-daemon[851]: accepted connection from pid 1693, user nix-grpc-daemon (trusted)6476worker1 # [ 140.452609] nix-daemon[851]: accepted connection from pid 1898, user nix-grpc-daemon (trusted)6477worker1 # [ 140.465603] nix-daemon[851]: reaped child process 1901, status = succeeded6478worker1 # [ 140.638544] nix-daemon[851]: accepted connection from pid 1903, user nix-grpc-daemon (trusted)6479worker1 # [ 140.650906] nix-daemon[851]: reaped child process 1906, status = succeeded6480lb # [ 140.646204] niks3-server[1065]: 2026/09/20 21:10:34 INFO Received uploads request method=POST path=/api/pending_closures6481worker1 # [ 140.685282] nix-grpc-daemon[1749]: time=2026-09-20T21:10:33.989Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6482worker1 # [ 140.686700] nix-grpc-daemon[1749]: time=2026-09-20T21:10:33.989Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"6483lb # [ 140.684197] niks3-server[1065]: 2026/09/20 21:10:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6484lb # [ 140.689092] niks3-server[1065]: 2026/09/20 21:10:34 INFO Signed narinfos id=54 count=16485worker1 # [ 140.713765] nix-grpc-daemon[1749]: time=2026-09-20T21:10:34.018Z level=INFO msg="Uploading 1 narinfos"6486lb # [ 140.699131] niks3-server[1065]: 2026/09/20 21:10:34 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst6487lb # [ 140.712539] niks3-server[1065]: 2026/09/20 21:10:34 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls6488lb # [ 140.715538] niks3-server[1065]: 2026/09/20 21:10:34 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6489lb # [ 140.721284] niks3-server[1065]: 2026/09/20 21:10:34 INFO Completed upload id=546490worker1 # [ 140.743135] nix-grpc-daemon[1749]: time=2026-09-20T21:10:34.047Z level=INFO msg="Upload complete. (462ms)"6491lb # [ 140.726709] niks3-server[1065]: 2026/09/20 21:10:34 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo6492worker1 # [ 140.747207] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34276 duration_s=0 paths=1 nar_bytes_in=6246493worker1 # [ 140.753315] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:342766494worker1 # [ 140.763138] nix-daemon[851]: accepted connection from pid 1693, user nix-grpc-daemon (trusted)6495worker1 # [ 140.775668] nix-daemon[851]: accepted connection from pid 1693, user nix-grpc-daemon (trusted)6496worker1 # [ 140.777872] nix-daemon[851]: reaped child process 1896, status = succeeded6497worker1 # [ 141.597953] nix-daemon[851]: reaped child process 1910, status = succeeded6498worker1 # [ 141.745600] nix-daemon[851]: accepted connection from pid 1925, user nix-grpc-daemon (trusted)6499worker1 # [ 141.757935] nix-daemon[851]: reaped child process 1928, status = succeeded6500lb # [ 141.753293] niks3-server[1065]: 2026/09/20 21:10:35 INFO Received uploads request method=POST path=/api/pending_closures6501worker1 # [ 141.782627] nix-grpc-daemon[1749]: time=2026-09-20T21:10:35.087Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6502worker1 # [ 141.785107] nix-grpc-daemon[1749]: time=2026-09-20T21:10:35.088Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"6503lb # [ 141.799160] niks3-server[1065]: 2026/09/20 21:10:35 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst6504lb # [ 141.813580] niks3-server[1065]: 2026/09/20 21:10:35 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv6505lb # [ 141.818826] niks3-server[1065]: 2026/09/20 21:10:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6506lb # [ 141.822945] niks3-server[1065]: 2026/09/20 21:10:35 INFO Signed narinfos id=55 count=16507worker1 # [ 141.847146] nix-grpc-daemon[1749]: time=2026-09-20T21:10:35.151Z level=INFO msg="Uploading 1 narinfos"6508lb # [ 141.832314] niks3-server[1065]: 2026/09/20 21:10:35 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls6509lb # [ 141.839394] niks3-server[1065]: 2026/09/20 21:10:35 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6510lb # [ 141.842381] niks3-server[1065]: 2026/09/20 21:10:35 INFO Completed upload id=556511lb # [ 141.845279] niks3-server[1065]: 2026/09/20 21:10:35 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo6512worker1 # [ 141.865887] nix-grpc-daemon[1749]: time=2026-09-20T21:10:35.170Z level=INFO msg="Upload complete. (282ms)"6513worker2 # [ 141.856154] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:35Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=16514worker1 # [ 141.875717] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34276 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=11 outputs=16515worker1 # [ 141.880559] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:342766516worker2 # [ 141.865867] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:35Z level=debug event=want client=5 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=06517worker1 # [ 141.892773] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:342766518worker1 # [ 141.895109] nix-daemon[851]: reaped child process 1908, status = succeeded6519worker1 # [ 141.899210] nix-grpc-daemon[1749]: time=2026-09-20T21:10:35.203Z level=INFO msg="All 1 paths already cached"6520worker1 # [ 141.903328] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:342766521worker1 # [ 141.907589] nix-daemon[851]: accepted connection from pid 1693, user nix-grpc-daemon (trusted)6522worker1 # [ 142.080272] nix-daemon[851]: accepted connection from pid 1932, user nix-grpc-daemon (trusted)6523worker1 # [ 142.094552] nix-daemon[851]: reaped child process 1935, status = succeeded6524worker1 # [ 142.238611] nix-daemon[851]: accepted connection from pid 1937, user nix-grpc-daemon (trusted)6525worker1 # [ 142.251463] nix-daemon[851]: reaped child process 1940, status = succeeded6526lb # [ 142.241599] niks3-server[1065]: 2026/09/20 21:10:36 INFO Received uploads request method=POST path=/api/pending_closures6527worker1 # [ 142.268455] nix-grpc-daemon[1749]: time=2026-09-20T21:10:35.573Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6528worker1 # [ 142.270388] nix-grpc-daemon[1749]: time=2026-09-20T21:10:35.574Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"6529lb # [ 142.257844] niks3-server[1065]: 2026/09/20 21:10:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6530lb # [ 142.260493] niks3-server[1065]: 2026/09/20 21:10:36 INFO Signed narinfos id=56 count=16531worker1 # [ 142.282236] nix-grpc-daemon[1749]: time=2026-09-20T21:10:35.586Z level=INFO msg="Uploading 1 narinfos"6532lb # [ 142.262574] niks3-server[1065]: 2026/09/20 21:10:36 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst6533lb # [ 142.265508] niks3-server[1065]: 2026/09/20 21:10:36 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls6534lb # [ 142.269835] niks3-server[1065]: 2026/09/20 21:10:36 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6535lb # [ 142.272980] niks3-server[1065]: 2026/09/20 21:10:36 INFO Completed upload id=566536worker1 # [ 142.295752] nix-grpc-daemon[1749]: time=2026-09-20T21:10:35.600Z level=INFO msg="Upload complete. (380ms)"6537lb # [ 142.275159] niks3-server[1065]: 2026/09/20 21:10:36 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo6538worker1 # [ 142.297745] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34276 duration_s=0 paths=1 nar_bytes_in=6246539worker1 # [ 142.301903] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:342766540worker1 # [ 142.307589] nix-daemon[851]: accepted connection from pid 1693, user nix-grpc-daemon (trusted)6541worker1 # [ 142.311237] nix-daemon[851]: reaped child process 1930, status = succeeded6542worker1 # [ 142.315865] nix-daemon[851]: accepted connection from pid 1693, user nix-grpc-daemon (trusted)6543worker1 # [ 143.329693] nix-daemon[851]: reaped child process 1944, status = succeeded6544worker1 # [ 143.467266] nix-daemon[851]: accepted connection from pid 1959, user nix-grpc-daemon (trusted)6545worker1 # [ 143.480336] nix-daemon[851]: reaped child process 1962, status = succeeded6546lb # [ 143.470958] niks3-server[1065]: 2026/09/20 21:10:37 INFO Received uploads request method=POST path=/api/pending_closures6547worker1 # [ 143.502839] nix-grpc-daemon[1749]: time=2026-09-20T21:10:36.807Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6548worker1 # [ 143.505424] nix-grpc-daemon[1749]: time=2026-09-20T21:10:36.809Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"6549lb # [ 143.502213] niks3-server[1065]: 2026/09/20 21:10:37 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv6550lb # [ 143.517271] niks3-server[1065]: 2026/09/20 21:10:37 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst6551lb # [ 143.527671] niks3-server[1065]: 2026/09/20 21:10:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6552lb # [ 143.532189] niks3-server[1065]: 2026/09/20 21:10:37 INFO Signed narinfos id=57 count=16553worker1 # [ 143.556445] nix-grpc-daemon[1749]: time=2026-09-20T21:10:36.860Z level=INFO msg="Uploading 1 narinfos"6554lb # [ 143.537333] niks3-server[1065]: 2026/09/20 21:10:37 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls6555lb # [ 143.546852] niks3-server[1065]: 2026/09/20 21:10:37 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6556lb # [ 143.552819] niks3-server[1065]: 2026/09/20 21:10:37 INFO Completed upload id=576557lb # [ 143.554048] niks3-server[1065]: 2026/09/20 21:10:37 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo6558worker1 # [ 143.574521] nix-grpc-daemon[1749]: time=2026-09-20T21:10:36.879Z level=INFO msg="Upload complete. (249ms)"6559worker1 # [ 143.579798] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34276 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=12 outputs=16560worker2 # [ 143.565973] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:37Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=16561worker1 # [ 143.588073] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:342766562worker2 # [ 143.573586] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:37Z level=debug event=want client=5 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=06563worker1 # [ 143.597703] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:342766564worker1 # [ 143.605341] nix-daemon[851]: reaped child process 1942, status = succeeded6565worker1 # [ 143.608057] nix-grpc-daemon[1749]: time=2026-09-20T21:10:36.912Z level=INFO msg="All 2 paths already cached"6566worker1 # [ 143.611911] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:342766567worker1 # [ 143.616522] nix-daemon[851]: accepted connection from pid 1693, user nix-grpc-daemon (trusted)6568worker1 # [ 143.755416] nix-daemon[851]: accepted connection from pid 1966, user nix-grpc-daemon (trusted)6569worker1 # [ 143.767995] nix-daemon[851]: reaped child process 1969, status = succeeded6570worker1 # [ 143.909394] nix-daemon[851]: accepted connection from pid 1971, user nix-grpc-daemon (trusted)6571worker1 # [ 143.922142] nix-daemon[851]: reaped child process 1974, status = succeeded6572lb # [ 143.910917] niks3-server[1065]: 2026/09/20 21:10:37 INFO Received uploads request method=POST path=/api/pending_closures6573worker1 # [ 143.939478] nix-grpc-daemon[1749]: time=2026-09-20T21:10:37.244Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6574worker1 # [ 143.940941] nix-grpc-daemon[1749]: time=2026-09-20T21:10:37.245Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"6575lb # [ 143.937508] niks3-server[1065]: 2026/09/20 21:10:38 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst6576lb # [ 143.948690] niks3-server[1065]: 2026/09/20 21:10:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6577worker1 # [ 143.975533] nix-grpc-daemon[1749]: time=2026-09-20T21:10:37.280Z level=INFO msg="Uploading 1 narinfos"6578lb # [ 143.954127] niks3-server[1065]: 2026/09/20 21:10:38 INFO Signed narinfos id=58 count=16579lb # [ 143.959979] niks3-server[1065]: 2026/09/20 21:10:38 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls6580lb # [ 143.970986] niks3-server[1065]: 2026/09/20 21:10:38 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6581worker1 # [ 143.998661] nix-grpc-daemon[1749]: time=2026-09-20T21:10:37.303Z level=INFO msg="Upload complete. (377ms)"6582lb # [ 143.977282] niks3-server[1065]: 2026/09/20 21:10:38 INFO Completed upload id=586583lb # [ 143.980893] niks3-server[1065]: 2026/09/20 21:10:38 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo6584worker1 # [ 144.001643] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34276 duration_s=0 paths=1 nar_bytes_in=7526585worker1 # [ 144.006965] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:342766586worker1 # [ 144.015582] nix-daemon[851]: accepted connection from pid 1693, user nix-grpc-daemon (trusted)6587worker1 # [ 144.020142] nix-daemon[851]: reaped child process 1964, status = succeeded6588worker1 # [ 144.025054] nix-daemon[851]: accepted connection from pid 1693, user nix-grpc-daemon (trusted)6589worker1 # [ 144.639031] nix-daemon[851]: reaped child process 1978, status = succeeded6590worker1 # [ 144.780681] nix-daemon[851]: accepted connection from pid 1993, user nix-grpc-daemon (trusted)6591worker1 # [ 144.794842] nix-daemon[851]: reaped child process 1996, status = succeeded6592lb # [ 144.786297] niks3-server[1065]: 2026/09/20 21:10:38 INFO Received uploads request method=POST path=/api/pending_closures6593worker1 # [ 144.831324] nix-grpc-daemon[1749]: time=2026-09-20T21:10:38.134Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6594worker1 # [ 144.832893] nix-grpc-daemon[1749]: time=2026-09-20T21:10:38.134Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"6595lb # [ 144.833649] niks3-server[1065]: 2026/09/20 21:10:38 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv6596lb # [ 144.844240] niks3-server[1065]: 2026/09/20 21:10:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6597lb # [ 144.849646] niks3-server[1065]: 2026/09/20 21:10:38 INFO Signed narinfos id=59 count=16598worker1 # [ 144.874712] nix-grpc-daemon[1749]: time=2026-09-20T21:10:38.179Z level=INFO msg="Uploading 1 narinfos"6599lb # [ 144.855296] niks3-server[1065]: 2026/09/20 21:10:38 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst6600lb # [ 144.859979] niks3-server[1065]: 2026/09/20 21:10:38 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls6601lb # [ 144.870883] niks3-server[1065]: 2026/09/20 21:10:38 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6602lb # [ 144.875623] niks3-server[1065]: 2026/09/20 21:10:38 INFO Completed upload id=596603lb # [ 144.878218] niks3-server[1065]: 2026/09/20 21:10:38 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo6604worker1 # [ 144.897502] nix-grpc-daemon[1749]: time=2026-09-20T21:10:38.201Z level=INFO msg="Upload complete. (266ms)"6605worker2 # [ 144.886870] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:38Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=16606worker1 # [ 144.905529] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34276 duration_s=0 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=13 outputs=16607worker1 # [ 144.918266] nix-daemon[851]: reaped child process 1976, status = succeeded6608worker1 # [ 144.948086] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:342766609worker1 # [ 144.960379] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:342766610worker1 # [ 144.973730] nix-grpc-daemon[1749]: time=2026-09-20T21:10:38.278Z level=INFO msg="All 2 paths already cached"6611worker1 # [ 144.981128] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:342766612worker1 # [ 144.992137] nix-daemon[851]: accepted connection from pid 1693, user nix-grpc-daemon (trusted)6613worker1 # [ 145.183508] nix-daemon[851]: accepted connection from pid 2000, user nix-grpc-daemon (trusted)6614worker1 # [ 145.196103] nix-daemon[851]: reaped child process 2003, status = succeeded6615worker1 # [ 145.370275] nix-daemon[851]: accepted connection from pid 2005, user nix-grpc-daemon (trusted)6616worker1 # [ 145.383673] nix-daemon[851]: reaped child process 2008, status = succeeded6617lb # [ 145.379373] niks3-server[1065]: 2026/09/20 21:10:39 INFO Received uploads request method=POST path=/api/pending_closures6618worker1 # [ 145.409697] nix-grpc-daemon[1749]: time=2026-09-20T21:10:38.714Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6619worker1 # [ 145.411459] nix-grpc-daemon[1749]: time=2026-09-20T21:10:38.715Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"6620lb # [ 145.407154] niks3-server[1065]: 2026/09/20 21:10:39 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst6621lb # [ 145.420332] niks3-server[1065]: 2026/09/20 21:10:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6622lb # [ 145.425367] niks3-server[1065]: 2026/09/20 21:10:39 INFO Signed narinfos id=60 count=16623worker1 # [ 145.449683] nix-grpc-daemon[1749]: time=2026-09-20T21:10:38.754Z level=INFO msg="Uploading 1 narinfos"6624lb # [ 145.431882] niks3-server[1065]: 2026/09/20 21:10:39 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls6625lb # [ 145.444113] niks3-server[1065]: 2026/09/20 21:10:39 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6626lb # [ 145.448783] niks3-server[1065]: 2026/09/20 21:10:39 INFO Completed upload id=606627worker1 # [ 145.472784] nix-grpc-daemon[1749]: time=2026-09-20T21:10:38.777Z level=INFO msg="Upload complete. (469ms)"6628lb # [ 145.451692] niks3-server[1065]: 2026/09/20 21:10:39 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo6629worker1 # [ 145.477377] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34276 duration_s=0 paths=1 nar_bytes_in=7526630worker1 # [ 145.481839] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:342766631worker1 # [ 145.492112] nix-daemon[851]: accepted connection from pid 1693, user nix-grpc-daemon (trusted)6632worker1 # [ 145.496655] nix-daemon[851]: reaped child process 1998, status = succeeded6633worker1 # [ 145.501904] nix-daemon[851]: accepted connection from pid 1693, user nix-grpc-daemon (trusted)6634worker1 # [ 146.522802] nix-daemon[851]: reaped child process 2012, status = succeeded6635worker1 # [ 146.695082] nix-daemon[851]: accepted connection from pid 2027, user nix-grpc-daemon (trusted)6636worker1 # [ 146.707747] nix-daemon[851]: reaped child process 2030, status = succeeded6637lb # [ 146.704348] niks3-server[1065]: 2026/09/20 21:10:40 INFO Received uploads request method=POST path=/api/pending_closures6638worker1 # [ 146.735165] nix-grpc-daemon[1749]: time=2026-09-20T21:10:40.038Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6639worker1 # [ 146.736524] nix-grpc-daemon[1749]: time=2026-09-20T21:10:40.038Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"6640lb # [ 146.743499] niks3-server[1065]: 2026/09/20 21:10:40 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst6641lb # [ 146.749751] niks3-server[1065]: 2026/09/20 21:10:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6642lb # [ 146.754755] niks3-server[1065]: 2026/09/20 21:10:40 INFO Signed narinfos id=61 count=16643worker1 # [ 146.779691] nix-grpc-daemon[1749]: time=2026-09-20T21:10:40.084Z level=INFO msg="Uploading 1 narinfos"6644lb # [ 146.762928] niks3-server[1065]: 2026/09/20 21:10:40 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls6645lb # [ 146.770667] niks3-server[1065]: 2026/09/20 21:10:40 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv6646lb # [ 146.778626] niks3-server[1065]: 2026/09/20 21:10:40 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6647lb # [ 146.783420] niks3-server[1065]: 2026/09/20 21:10:40 INFO Completed upload id=616648worker1 # [ 146.805254] nix-grpc-daemon[1749]: time=2026-09-20T21:10:40.109Z level=INFO msg="Upload complete. (294ms)"6649lb # [ 146.785789] niks3-server[1065]: 2026/09/20 21:10:40 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo6650worker2 # [ 146.794598] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:40Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=16651worker1 # [ 146.813484] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34276 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=14 outputs=16652worker1 # [ 146.825513] nix-daemon[851]: reaped child process 2010, status = succeeded6653worker1 # [ 146.854752] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:342766654worker1 # [ 146.866871] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:342766655worker1 # [ 146.879742] nix-grpc-daemon[1749]: time=2026-09-20T21:10:40.184Z level=INFO msg="All 2 paths already cached"6656worker1 # [ 146.887086] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:342766657worker1 # [ 146.897203] nix-daemon[851]: accepted connection from pid 1693, user nix-grpc-daemon (trusted)6658worker1 # [ 147.077262] nix-daemon[851]: accepted connection from pid 2034, user nix-grpc-daemon (trusted)6659worker1 # [ 147.089762] nix-daemon[851]: reaped child process 2037, status = succeeded6660worker1 # [ 147.260148] nix-daemon[851]: accepted connection from pid 2039, user nix-grpc-daemon (trusted)6661worker1 # [ 147.272601] nix-daemon[851]: reaped child process 2042, status = succeeded6662lb # [ 147.266978] niks3-server[1065]: 2026/09/20 21:10:41 INFO Received uploads request method=POST path=/api/pending_closures6663worker1 # [ 147.296931] nix-grpc-daemon[1749]: time=2026-09-20T21:10:40.601Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6664worker1 # [ 147.298462] nix-grpc-daemon[1749]: time=2026-09-20T21:10:40.603Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"6665lb # [ 147.295646] niks3-server[1065]: 2026/09/20 21:10:41 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst6666lb # [ 147.306931] niks3-server[1065]: 2026/09/20 21:10:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6667lb # [ 147.312084] niks3-server[1065]: 2026/09/20 21:10:41 INFO Signed narinfos id=62 count=16668worker1 # [ 147.336683] nix-grpc-daemon[1749]: time=2026-09-20T21:10:40.641Z level=INFO msg="Uploading 1 narinfos"6669lb # [ 147.318210] niks3-server[1065]: 2026/09/20 21:10:41 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls6670lb # [ 147.328651] niks3-server[1065]: 2026/09/20 21:10:41 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6671lb # [ 147.332759] niks3-server[1065]: 2026/09/20 21:10:41 INFO Completed upload id=626672worker1 # [ 147.354364] nix-grpc-daemon[1749]: time=2026-09-20T21:10:40.659Z level=INFO msg="Upload complete. (446ms)"6673lb # [ 147.334553] niks3-server[1065]: 2026/09/20 21:10:41 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo6674worker1 # [ 147.356939] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34276 duration_s=0 paths=1 nar_bytes_in=7526675worker1 # [ 147.362307] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:342766676worker1 # [ 147.371487] nix-daemon[851]: accepted connection from pid 1693, user nix-grpc-daemon (trusted)6677worker1 # [ 147.375452] nix-daemon[851]: reaped child process 2032, status = succeeded6678worker1 # [ 147.380931] nix-daemon[851]: accepted connection from pid 1693, user nix-grpc-daemon (trusted)6679worker1 # [ 148.205298] nix-daemon[851]: reaped child process 2046, status = succeeded6680worker1 # [ 148.394268] nix-daemon[851]: accepted connection from pid 2061, user nix-grpc-daemon (trusted)6681worker1 # [ 148.407627] nix-daemon[851]: reaped child process 2064, status = succeeded6682lb # [ 148.404899] niks3-server[1065]: 2026/09/20 21:10:42 INFO Received uploads request method=POST path=/api/pending_closures6683worker1 # [ 148.434624] nix-grpc-daemon[1749]: time=2026-09-20T21:10:41.739Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6684worker1 # [ 148.436299] nix-grpc-daemon[1749]: time=2026-09-20T21:10:41.740Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"6685lb # [ 148.441325] niks3-server[1065]: 2026/09/20 21:10:42 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst6686lb # [ 148.451207] niks3-server[1065]: 2026/09/20 21:10:42 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv6687lb # [ 148.460837] niks3-server[1065]: 2026/09/20 21:10:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6688worker1 # [ 148.487104] nix-grpc-daemon[1749]: time=2026-09-20T21:10:41.791Z level=INFO msg="Uploading 1 narinfos"6689lb # [ 148.465423] niks3-server[1065]: 2026/09/20 21:10:42 INFO Signed narinfos id=63 count=16690lb # [ 148.472102] niks3-server[1065]: 2026/09/20 21:10:42 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls6691lb # [ 148.482414] niks3-server[1065]: 2026/09/20 21:10:42 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6692lb # [ 148.489367] niks3-server[1065]: 2026/09/20 21:10:42 INFO Completed upload id=636693worker1 # [ 148.510994] nix-grpc-daemon[1749]: time=2026-09-20T21:10:41.815Z level=INFO msg="Upload complete. (322ms)"6694lb # [ 148.491221] niks3-server[1065]: 2026/09/20 21:10:42 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo6695worker1 # [ 148.517333] nix-grpc-daemon[1693]: ts=2026-09-20T21:10:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34276 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=15 outputs=16696worker1 # [ 148.529983] nix-daemon[851]: reaped child process 2044, status = succeeded6697client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 16.75 seconds)6698worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6699worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6700client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]'6701lb # [ 148.630188] envoy[973]: conn peer=[2001:db8:1::1]:49006 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27072 tx=10745 ms=165036702client # [ 148.675138] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6703client # Running as unit: gpu.service; invocation ID: d99e8848b74a4667813593b57b007a766704client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.07 seconds)6705lb: must succeed: curl -sf localhost:9901/clusters6706lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6707lb: must succeed: curl -sf localhost:9901/clusters6708lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6709worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6710worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6711client # [ 148.889557] nix[1353]: don't know how to build these paths:6712worker2 # [ 148.881768] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:42Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:333526713client # [ 148.890479] nix[1353]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv6714worker2 # [ 148.884297] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:333526715client # [ 148.907832] nix[1353]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, waiting up to 30s for a worker6716worker2 # [ 148.900751] nix-grpc-daemon[1394]: ts=2026-09-20T21:10:42Z level=debug event=want client=6 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=06717lb: must succeed: curl -sf localhost:9901/clusters6718lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6719lb: must succeed: curl -sf localhost:9901/clusters6720lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6721worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6722worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6723client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6724client # [ 178.908953] nix[1353]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}6725client # [ 178.915769] nix[1353]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed6726client # [ 178.943631] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6727client # [ 178.945914] systemd[1]: gpu.service: Failed with result 'exit-code'.6728client # [ 178.948298] systemd[1]: gpu.service: Consumed 142ms CPU time over 30.272s wall clock time, 35.4M memory peak, 5.1K incoming IP traffic, 5K outgoing IP traffic.6729client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.57 seconds)6730client: must succeed: journalctl -u gpu -o cat6731client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds)6732lb: must succeed: curl -sf localhost:9901/clusters6733lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6734lb: must succeed: curl -sf localhost:9901/clusters6735lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6736worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6737worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6738(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 48.04 seconds)6739subtest: low disk drains a worker and builds go to the other6740worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6741worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.00 seconds)6742worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6743worker1 # [ 181.587339] nix-grpc-daemon[1693]: ts=2026-09-20T21:11:14Z level=info event=unhealthy reason=min_free6744worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.11 seconds)6745client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&26746client # don't know how to build these paths:6747client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6748worker2 # [ 182.193986] nix-grpc-daemon[1394]: ts=2026-09-20T21:11:15Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:333526749worker2 # [ 182.200064] nix-grpc-daemon[1394]: ts=2026-09-20T21:11:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:333526750worker2 # [ 182.223118] nix-grpc-daemon[1394]: ts=2026-09-20T21:11:15Z level=debug event=want client=7 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=06751worker2 # [ 182.227777] nix-grpc-daemon[1394]: ts=2026-09-20T21:11:15Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=16752worker2 # [ 182.272285] nix-grpc-daemon[1394]: ts=2026-09-20T21:11:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333526753worker2 # [ 182.283079] nix-grpc-daemon[1394]: ts=2026-09-20T21:11:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:333526754client # copying 1 paths to 'lb:50051'...6755client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...6756worker2 # [ 182.293666] nix-grpc-daemon[1394]: ts=2026-09-20T21:11:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:333526757worker2 # [ 182.305084] nix-daemon[871]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6758worker2 # [ 182.479819] nix-daemon[871]: accepted connection from pid 1685, user nix-grpc-daemon (trusted)6759worker2 # [ 182.494163] nix-daemon[871]: reaped child process 1688, status = succeeded6760worker2 # [ 182.674227] nix-daemon[871]: accepted connection from pid 1690, user nix-grpc-daemon (trusted)6761worker2 # [ 182.687418] nix-daemon[871]: reaped child process 1693, status = succeeded6762lb # [ 182.699547] niks3-server[1065]: 2026/09/20 21:11:16 INFO Received uploads request method=POST path=/api/pending_closures6763worker2 # [ 182.717309] nix-grpc-daemon[1470]: time=2026-09-20T21:11:16.471Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6764worker2 # [ 182.718975] nix-grpc-daemon[1470]: time=2026-09-20T21:11:16.471Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"6765lb # [ 182.735312] niks3-server[1065]: 2026/09/20 21:11:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6766worker2 # [ 182.750192] nix-grpc-daemon[1470]: time=2026-09-20T21:11:16.505Z level=INFO msg="Uploading 1 narinfos"6767lb # [ 182.745340] niks3-server[1065]: 2026/09/20 21:11:16 INFO Signed narinfos id=64 count=16768lb # [ 182.754671] niks3-server[1065]: 2026/09/20 21:11:16 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst6769lb # [ 182.760087] niks3-server[1065]: 2026/09/20 21:11:16 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls6770lb # [ 182.769687] niks3-server[1065]: 2026/09/20 21:11:16 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6771lb # [ 182.777251] niks3-server[1065]: 2026/09/20 21:11:16 INFO Completed upload id=646772worker2 # [ 182.782520] nix-grpc-daemon[1470]: time=2026-09-20T21:11:16.537Z level=INFO msg="Upload complete. (465ms)"6773lb # [ 182.779536] niks3-server[1065]: 2026/09/20 21:11:16 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo6774worker2 # [ 182.788154] nix-grpc-daemon[1394]: ts=2026-09-20T21:11:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33352 duration_s=0 paths=1 nar_bytes_in=5126775worker2 # [ 182.793642] nix-grpc-daemon[1394]: ts=2026-09-20T21:11:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333526776worker2 # [ 182.802622] nix-daemon[871]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6777worker2 # [ 182.806513] nix-daemon[871]: reaped child process 1683, status = succeeded6778client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...6779worker2 # [ 182.811856] nix-daemon[871]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6780client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6781client # farm-a> LOG-farm-a-drain6782worker2 # [ 183.625963] nix-daemon[871]: reaped child process 1697, status = succeeded6783worker2 # [ 183.758904] nix-daemon[871]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)6784worker2 # [ 183.771528] nix-daemon[871]: reaped child process 1715, status = succeeded6785lb # [ 183.776574] niks3-server[1065]: 2026/09/20 21:11:17 INFO Received uploads request method=POST path=/api/pending_closures6786worker2 # [ 183.788402] nix-grpc-daemon[1470]: time=2026-09-20T21:11:17.543Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6787worker2 # [ 183.790340] nix-grpc-daemon[1470]: time=2026-09-20T21:11:17.545Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"6788lb # [ 183.804674] niks3-server[1065]: 2026/09/20 21:11:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6789lb # [ 183.807547] niks3-server[1065]: 2026/09/20 21:11:17 INFO Signed narinfos id=65 count=16790worker2 # [ 183.812664] nix-grpc-daemon[1470]: time=2026-09-20T21:11:17.567Z level=INFO msg="Uploading 1 narinfos"6791lb # [ 183.810221] niks3-server[1065]: 2026/09/20 21:11:17 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6792lb # [ 183.817820] niks3-server[1065]: 2026/09/20 21:11:17 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6793lb # [ 183.820112] niks3-server[1065]: 2026/09/20 21:11:17 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6794lb # [ 183.830135] niks3-server[1065]: 2026/09/20 21:11:17 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls6795lb # [ 183.840548] niks3-server[1065]: 2026/09/20 21:11:17 INFO Completed upload id=656796worker2 # [ 183.846649] nix-grpc-daemon[1470]: time=2026-09-20T21:11:17.601Z level=INFO msg="Upload complete. (228ms)"6797worker2 # [ 183.850937] nix-grpc-daemon[1394]: ts=2026-09-20T21:11:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33352 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=16 outputs=16798lb # [ 183.850626] niks3-server[1065]: 2026/09/20 21:11:17 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo6799worker2 # [ 183.860669] nix-grpc-daemon[1394]: ts=2026-09-20T21:11:17Z level=debug event=want client=7 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=06800worker2 # [ 183.862736] nix-grpc-daemon[1394]: ts=2026-09-20T21:11:17Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=16801worker2 # [ 183.864991] nix-daemon[871]: reaped child process 1695, status = succeeded6802worker2 # [ 183.904216] nix-grpc-daemon[1394]: ts=2026-09-20T21:11:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333526803worker2 # [ 183.910728] nix-grpc-daemon[1394]: ts=2026-09-20T21:11:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:333526804worker2 # [ 183.919659] nix-grpc-daemon[1470]: time=2026-09-20T21:11:17.674Z level=INFO msg="All 1 paths already cached"6805client # copying 1 paths to 'lb:50051'...6806worker2 # [ 183.925542] nix-grpc-daemon[1394]: ts=2026-09-20T21:11:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:333526807client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...6808worker2 # [ 183.936583] nix-daemon[871]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6809worker2 # [ 184.084301] nix-daemon[871]: accepted connection from pid 1721, user nix-grpc-daemon (trusted)6810worker2 # [ 184.098678] nix-daemon[871]: reaped child process 1724, status = succeeded6811worker2 # [ 184.250272] nix-daemon[871]: accepted connection from pid 1726, user nix-grpc-daemon (trusted)6812worker2 # [ 184.262713] nix-daemon[871]: reaped child process 1729, status = succeeded6813lb # [ 184.272307] niks3-server[1065]: 2026/09/20 21:11:18 INFO Received uploads request method=POST path=/api/pending_closures6814worker2 # [ 184.286895] nix-grpc-daemon[1470]: time=2026-09-20T21:11:18.041Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6815worker2 # [ 184.288293] nix-grpc-daemon[1470]: time=2026-09-20T21:11:18.041Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"6816lb # [ 184.299635] niks3-server[1065]: 2026/09/20 21:11:18 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst6817lb # [ 184.309491] niks3-server[1065]: 2026/09/20 21:11:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6818lb # [ 184.315335] niks3-server[1065]: 2026/09/20 21:11:18 INFO Signed narinfos id=66 count=16819worker2 # [ 184.323280] nix-grpc-daemon[1470]: time=2026-09-20T21:11:18.078Z level=INFO msg="Uploading 1 narinfos"6820lb # [ 184.318629] niks3-server[1065]: 2026/09/20 21:11:18 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls6821lb # [ 184.331970] niks3-server[1065]: 2026/09/20 21:11:18 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6822lb # [ 184.336684] niks3-server[1065]: 2026/09/20 21:11:18 INFO Completed upload id=666823worker2 # [ 184.343849] nix-grpc-daemon[1470]: time=2026-09-20T21:11:18.099Z level=INFO msg="Upload complete. (399ms)"6824lb # [ 184.339573] niks3-server[1065]: 2026/09/20 21:11:18 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo6825worker2 # [ 184.347078] nix-grpc-daemon[1394]: ts=2026-09-20T21:11:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33352 duration_s=0 paths=1 nar_bytes_in=6246826worker2 # [ 184.350765] nix-grpc-daemon[1394]: ts=2026-09-20T21:11:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333526827worker2 # [ 184.359984] nix-daemon[871]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6828worker2 # [ 184.362346] nix-daemon[871]: reaped child process 1719, status = succeeded6829worker2 # [ 184.367835] nix-daemon[871]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6830client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...6831client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6832client # farm-b> LOG-farm-b-drain6833worker2 # [ 185.182507] nix-daemon[871]: reaped child process 1733, status = succeeded6834worker2 # [ 185.295722] nix-daemon[871]: accepted connection from pid 1748, user nix-grpc-daemon (trusted)6835worker2 # [ 185.308313] nix-daemon[871]: reaped child process 1751, status = succeeded6836lb # [ 185.313846] niks3-server[1065]: 2026/09/20 21:11:19 INFO Received uploads request method=POST path=/api/pending_closures6837worker2 # [ 185.322773] nix-grpc-daemon[1470]: time=2026-09-20T21:11:19.077Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6838worker2 # [ 185.325061] nix-grpc-daemon[1470]: time=2026-09-20T21:11:19.079Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"6839lb # [ 185.335369] niks3-server[1065]: 2026/09/20 21:11:19 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6840lb # [ 185.338058] niks3-server[1065]: 2026/09/20 21:11:19 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst6841lb # [ 185.347328] niks3-server[1065]: 2026/09/20 21:11:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6842lb # [ 185.352415] niks3-server[1065]: 2026/09/20 21:11:19 INFO Signed narinfos id=67 count=16843worker2 # [ 185.359877] nix-grpc-daemon[1470]: time=2026-09-20T21:11:19.114Z level=INFO msg="Uploading 1 narinfos"6844lb # [ 185.356252] niks3-server[1065]: 2026/09/20 21:11:19 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls6845lb # [ 185.363602] niks3-server[1065]: 2026/09/20 21:11:19 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6846lb # [ 185.367609] niks3-server[1065]: 2026/09/20 21:11:19 INFO Completed upload id=676847worker2 # [ 185.372547] nix-grpc-daemon[1470]: time=2026-09-20T21:11:19.127Z level=INFO msg="Upload complete. (198ms)"6848lb # [ 185.369188] niks3-server[1065]: 2026/09/20 21:11:19 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo6849worker2 # [ 185.374990] nix-grpc-daemon[1394]: ts=2026-09-20T21:11:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33352 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=17 outputs=16850worker2 # [ 185.382118] nix-grpc-daemon[1394]: ts=2026-09-20T21:11:19Z level=debug event=want client=7 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=06851worker2 # [ 185.384560] nix-grpc-daemon[1394]: ts=2026-09-20T21:11:19Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=16852worker2 # [ 185.386620] nix-grpc-daemon[1394]: ts=2026-09-20T21:11:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333526853worker2 # [ 185.391579] nix-daemon[871]: reaped child process 1731, status = succeeded6854worker2 # [ 185.393705] nix-grpc-daemon[1394]: ts=2026-09-20T21:11:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:333526855worker2 # [ 185.424701] nix-grpc-daemon[1470]: time=2026-09-20T21:11:19.179Z level=INFO msg="All 2 paths already cached"6856client # copying 1 paths to 'lb:50051'...6857client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...6858worker2 # [ 185.431095] nix-grpc-daemon[1394]: ts=2026-09-20T21:11:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:333526859worker2 # [ 185.437914] nix-daemon[871]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6860worker2 # [ 185.597078] nix-daemon[871]: accepted connection from pid 1757, user nix-grpc-daemon (trusted)6861worker2 # [ 185.611359] nix-daemon[871]: reaped child process 1760, status = succeeded6862worker2 # [ 185.760772] nix-daemon[871]: accepted connection from pid 1762, user nix-grpc-daemon (trusted)6863worker2 # [ 185.773490] nix-daemon[871]: reaped child process 1765, status = succeeded6864lb # [ 185.785129] niks3-server[1065]: 2026/09/20 21:11:19 INFO Received uploads request method=POST path=/api/pending_closures6865worker2 # [ 185.804350] nix-grpc-daemon[1470]: time=2026-09-20T21:11:19.559Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6866worker2 # [ 185.805724] nix-grpc-daemon[1470]: time=2026-09-20T21:11:19.559Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"6867lb # [ 185.818576] niks3-server[1065]: 2026/09/20 21:11:19 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst6868lb # [ 185.827167] niks3-server[1065]: 2026/09/20 21:11:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6869worker2 # [ 185.836606] nix-grpc-daemon[1470]: time=2026-09-20T21:11:19.591Z level=INFO msg="Uploading 1 narinfos"6870lb # [ 185.832146] niks3-server[1065]: 2026/09/20 21:11:19 INFO Signed narinfos id=68 count=16871lb # [ 185.840153] niks3-server[1065]: 2026/09/20 21:11:19 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls6872lb # [ 185.850085] niks3-server[1065]: 2026/09/20 21:11:19 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6873worker2 # [ 185.862324] nix-grpc-daemon[1470]: time=2026-09-20T21:11:19.617Z level=INFO msg="Upload complete. (417ms)"6874lb # [ 185.857648] niks3-server[1065]: 2026/09/20 21:11:19 INFO Completed upload id=686875lb # [ 185.861222] niks3-server[1065]: 2026/09/20 21:11:19 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo6876worker2 # [ 185.865338] nix-grpc-daemon[1394]: ts=2026-09-20T21:11:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33352 duration_s=0 paths=1 nar_bytes_in=7526877worker2 # [ 185.870374] nix-grpc-daemon[1394]: ts=2026-09-20T21:11:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333526878worker2 # [ 185.878477] nix-daemon[871]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6879worker2 # [ 185.888206] nix-daemon[871]: reaped child process 1755, status = succeeded6880worker2 # [ 185.891694] nix-daemon[871]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6881client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...6882client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6883client # farm-top> LOG-farm-top-drain6884worker2 # [ 186.712579] nix-daemon[871]: reaped child process 1769, status = succeeded6885worker2 # [ 186.858775] nix-daemon[871]: accepted connection from pid 1784, user nix-grpc-daemon (trusted)6886worker2 # [ 186.870394] nix-daemon[871]: reaped child process 1787, status = succeeded6887lb # [ 186.880517] niks3-server[1065]: 2026/09/20 21:11:20 INFO Received uploads request method=POST path=/api/pending_closures6888worker2 # [ 186.891518] nix-grpc-daemon[1470]: time=2026-09-20T21:11:20.646Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6889worker2 # [ 186.893425] nix-grpc-daemon[1470]: time=2026-09-20T21:11:20.648Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"6890lb # [ 186.907692] niks3-server[1065]: 2026/09/20 21:11:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6891lb # [ 186.909777] niks3-server[1065]: 2026/09/20 21:11:20 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst6892lb # [ 186.912580] niks3-server[1065]: 2026/09/20 21:11:20 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6893lb # [ 186.915332] niks3-server[1065]: 2026/09/20 21:11:20 INFO Signed narinfos id=69 count=16894worker2 # [ 186.920200] nix-grpc-daemon[1470]: time=2026-09-20T21:11:20.675Z level=INFO msg="Uploading 1 narinfos"6895lb # [ 186.918814] niks3-server[1065]: 2026/09/20 21:11:20 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls6896lb # [ 186.923973] niks3-server[1065]: 2026/09/20 21:11:20 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6897lb # [ 186.926895] niks3-server[1065]: 2026/09/20 21:11:20 INFO Completed upload id=696898worker2 # [ 186.933025] nix-grpc-daemon[1470]: time=2026-09-20T21:11:20.688Z level=INFO msg="Upload complete. (235ms)"6899lb # [ 186.929183] niks3-server[1065]: 2026/09/20 21:11:20 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo6900worker2 # [ 186.935365] nix-grpc-daemon[1394]: ts=2026-09-20T21:11:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33352 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=18 outputs=16901worker2 # [ 186.942229] nix-daemon[871]: reaped child process 1767, status = succeeded6902client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&2, in 5.06 seconds)6903client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath6904client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath, in 0.22 seconds)6905worker1: must succeed: rm /nix/.rw-store/fill6906worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.13 seconds)6907worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6908worker1 # [ 187.593659] nix-grpc-daemon[1693]: ts=2026-09-20T21:11:20Z level=info event=healthy reason=min_free6909worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.10 seconds)6910(finished: subtest: low disk drains a worker and builds go to the other, in 8.70 seconds)6911(finished: run the VM test script, in 189.45 seconds)6912lb # [ 188.632067] envoy[973]: conn peer=[2001:db8:1::1]:54356 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=895 tx=497 ms=300406913lb # [ 188.636674] envoy[973]: conn peer=[2001:db8:1::1]:49528 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9499 tx=3814 ms=47976914test script finished in 189.66s6915cleanup6916kill QemuMachine (pid 49)6917client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6918kill QemuMachine (pid 50)6919lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6920kill QemuMachine (pid 51)6921worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6922kill QemuMachine (pid 52)6923worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6924(finished: cleanup, in 0.93 seconds)6925additionally exposed symbols:6926 client, lb, worker1, worker2,6927 vlan1,6928 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh