vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #350
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker1 # Disk image does not exist, creating the virtualisation disk image...13client # Disk image does not exist, creating the virtualisation disk image...14worker1 # Formatting '/build/vm-state-worker1/tmp.c5SPC1O51q', fmt=raw size=107374182415worker1: QEMU running (pid 52)16client: QEMU running (pid 50)17client # Formatting '/build/vm-state-client/tmp.OMmbt67Hnl', fmt=raw size=107374182418lb: QEMU running (pid 49)19lb # Disk image does not exist, creating the virtualisation disk image...20worker1 # mke2fs 1.47.4 (6-Mar-2025)21lb # Formatting '/build/vm-state-lb/tmp.z0DB5ugeIN', fmt=raw size=107374182422worker1 # Discarding device blocks: 0/262144 done23lb # mke2fs 1.47.4 (6-Mar-2025)24worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes25lb # Discarding device blocks: 0/262144 done26worker1 # Filesystem UUID: e2ccd39c-c835-4fe3-bfb8-0e3a7654fa6627lb # Creating filesystem with 262144 4k blocks and 65536 inodes28worker1 # Superblock backups stored on blocks:29lb # Filesystem UUID: 96a883d5-3f46-4b0a-9ea3-c67a275bf74f30worker1 # 32768, 98304, 163840, 22937631lb # Superblock backups stored on blocks:32worker1 # 33lb # 32768, 98304, 163840, 22937634lb # 35worker1 # Allocating group tables: 0/8 done36client # mke2fs 1.47.4 (6-Mar-2025)37lb # Allocating group tables: 0/8 done38worker1 # Writing inode tables: 0/8 done39lb # Writing inode tables: 0/8 done40worker1 # Creating journal (8192 blocks): done41lb # Creating journal (8192 blocks): done42lb # Writing superblocks and filesystem accounting information: 0/8 done43client # Discarding device blocks: 0/262144 done44lb # 45client # Creating filesystem with 262144 4k blocks and 65536 inodes46lb # Virtualisation disk image created.47client # Filesystem UUID: 1fc8b0dc-7d15-4f14-97ab-187b448783db48worker1 # Writing superblocks and filesystem accounting information: 0/8 done49client # Superblock backups stored on blocks:50client # 32768, 98304, 163840, 22937651worker1 # 52worker2 # Disk image does not exist, creating the virtualisation disk image...53worker2: QEMU running (pid 51)54client # 55worker1 # Virtualisation disk image created.56client # Allocating group tables: 0/8 done57worker2 # Formatting '/build/vm-state-worker2/tmp.KKzeQSRws7', fmt=raw size=107374182458client # Writing inode tables: 0/8 done59worker2 # mke2fs 1.47.4 (6-Mar-2025)60client # Creating journal (8192 blocks): done61(finished: start all VMs, in 0.60 seconds)62client # Writing superblocks and filesystem accounting information: 0/8 done63client: waiting for unit network-addresses-eth1.service64client # 65client: waiting for the VM to finish booting66client # Virtualisation disk image created.67worker2 # Discarding device blocks: 0/262144 done68worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes69worker2 # Filesystem UUID: b52f8484-0f63-46b3-a2ab-1e3b571e1be370worker2 # Superblock backups stored on blocks:71worker2 # 32768, 98304, 163840, 22937672worker2 # 73worker2 # Allocating group tables: 0/8 done74worker2 # Writing inode tables: 0/8 done75worker2 # Creating journal (8192 blocks): done76worker2 # Writing superblocks and filesystem accounting information: 0/8 done77worker2 # 78worker2 # Virtualisation disk image created.79worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)80client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)81worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)82lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)83client # 84client # 85worker1 # 86worker1 # 87client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0088worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0089client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...90client # 91client # 92client # 93client # 94worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...95worker1 # 96worker1 # 97worker2 # 98worker1 # 99worker2 # 100worker1 # 101lb # 102lb # 103client # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00104worker1 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00105worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00106lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00107lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...108lb # 109lb # 110lb # 111lb # 112client # Press Ctrl-B to configure iPXE (PCI 00:08.0)...113client # 114client # 115worker1 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...116worker1 # 117worker1 # 118worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...119worker2 # 120worker2 # 121worker2 # 122worker2 # 123lb # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00124worker2 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00125lb # Press Ctrl-B to configure iPXE (PCI 00:08.0)...126lb # 127lb # 128worker2 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...129worker2 # 130worker2 # 131client # Booting from ROM...132worker1 # Booting from ROM...133lb # Booting from ROM...134worker2 # Booting from ROM...135client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026136client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/8yjya2ac0ki0lpwdcag7baz48ybnsw0d-nixos-system-client-test/init regInfo=/nix/store/4zrrnd9gljfrai49b1csncr8npfgb1qj-closure-info/registration console=ttyS0,115200n8 console=tty0137client # [ 0.000000] BIOS-provided physical RAM map:138client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable139worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026140client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved141client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved142client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable143client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved144worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/28k15c24ynj80n50y8xz4mn2ffrxv0di-nixos-system-worker1-test/init regInfo=/nix/store/vzq6in1clav5d3ah71k4fp2bq0gi7dng-closure-info/registration console=ttyS0,115200n8 console=tty0145client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved146worker1 # [ 0.000000] BIOS-provided physical RAM map:147client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved148worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable149client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved150worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved151client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved152worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved153client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved154worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable155client # [ 0.000000] NX (Execute Disable) protection: active156client # [ 0.000000] APIC: Static calls initialized157worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved158client # [ 0.000000] SMBIOS 2.8 present.159worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved160client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014161worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved162client # [ 0.000000] DMI: Memory slots populated: 1/1163client # [ 0.000000] Hypervisor detected: KVM164lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026165worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved166client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000167worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved168client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00169worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved170client # [ 0.000001] kvm-clock: using sched offset of 459484442 cycles171worker1 # [ 0.000000] NX (Execute Disable) protection: active172worker1 # [ 0.000000] APIC: Static calls initialized173worker1 # [ 0.000000] SMBIOS 2.8 present.174client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns175client # [ 0.000006] tsc: Detected 2400.012 MHz processor176lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0qln5mjy767qjp5wbmxjg8jay8k05dsm-nixos-system-lb-test/init regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration console=ttyS0,115200n8 console=tty0177lb # [ 0.000000] BIOS-provided physical RAM map:178worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014179client # [ 0.000817] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000180worker1 # [ 0.000000] DMI: Memory slots populated: 1/1181lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable182worker1 # [ 0.000000] Hypervisor detected: KVM183client # [ 0.000845] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs184worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000185lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved186client # [ 0.000848] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT187worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00188lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved189client # [ 0.002748] found SMP MP-table at [mem 0x000f5450-0x000f545f]190worker1 # [ 0.000000] kvm-clock: using sched offset of 465113345 cycles191client # [ 0.002762] Using GB pages for direct mapping192lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable193client # [ 0.002870] RAMDISK: [mem 0x5e366000-0x5ffcffff]194worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns195lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved196client # [ 0.002878] ACPI: Early table checksum verification disabled197worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor198lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved199client # [ 0.002882] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )200worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026201worker1 # [ 0.000815] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000202lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved203client # [ 0.002887] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)204worker1 # [ 0.000842] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs205lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved206client # [ 0.002891] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)207worker1 # [ 0.000845] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT208lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved209worker1 # [ 0.002743] found SMP MP-table at [mem 0x000f5450-0x000f545f]210client # [ 0.002899] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)211lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved212worker1 # [ 0.002755] Using GB pages for direct mapping213client # [ 0.002901] ACPI: FACS 0x000000005FFE0000 000040214lb # [ 0.000000] NX (Execute Disable) protection: active215worker1 # [ 0.002865] RAMDISK: [mem 0x5e367000-0x5ffcffff]216worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/98jrmc7dvx1iq9by65a311c4s2kfa5k4-nixos-system-worker2-test/init regInfo=/nix/store/9va08d8nrkb3f5vykhsq9nzd2mlr2dd9-closure-info/registration console=ttyS0,115200n8 console=tty0217lb # [ 0.000000] APIC: Static calls initialized218worker2 # [ 0.000000] BIOS-provided physical RAM map:219worker1 # [ 0.002872] ACPI: Early table checksum verification disabled220client # [ 0.002902] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)221lb # [ 0.000000] SMBIOS 2.8 present.222worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable223worker1 # [ 0.002875] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )224client # [ 0.002904] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)225worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved226lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014227worker1 # [ 0.002880] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)228client # [ 0.002905] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)229lb # [ 0.000000] DMI: Memory slots populated: 1/1230worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved231lb # [ 0.000000] Hypervisor detected: KVM232worker1 # [ 0.002884] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)233client # [ 0.002907] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)234worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable235lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000236worker1 # [ 0.002891] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)237lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00238client # [ 0.002908] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]239worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved240worker1 # [ 0.002893] ACPI: FACS 0x000000005FFE0000 000040241lb # [ 0.000001] kvm-clock: using sched offset of 470521256 cycles242worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved243client # [ 0.002909] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]244worker1 # [ 0.002894] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)245client # [ 0.002910] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]246worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved247lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns248worker1 # [ 0.002896] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)249lb # [ 0.000005] tsc: Detected 2400.012 MHz processor250client # [ 0.002910] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]251worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved252lb # [ 0.000820] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000253worker1 # [ 0.002897] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)254worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved255client # [ 0.002911] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]256worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved257client # [ 0.002911] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]258lb # [ 0.000850] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs259worker1 # [ 0.002899] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)260worker2 # [ 0.000000] NX (Execute Disable) protection: active261client # [ 0.002912] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]262lb # [ 0.000853] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT263worker1 # [ 0.002900] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]264worker2 # [ 0.000000] APIC: Static calls initialized265client # [ 0.003145] No NUMA configuration found266worker2 # [ 0.000000] SMBIOS 2.8 present.267lb # [ 0.002772] found SMP MP-table at [mem 0x000f5450-0x000f545f]268worker1 # [ 0.002901] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]269client # [ 0.003146] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]270lb # [ 0.002787] Using GB pages for direct mapping271worker1 # [ 0.002902] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]272lb # [ 0.002918] RAMDISK: [mem 0x5e367000-0x5ffcffff]273client # [ 0.003149] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]274worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014275client # [ 0.005461] Zone ranges:276worker2 # [ 0.000000] DMI: Memory slots populated: 1/1277lb # [ 0.002930] ACPI: Early table checksum verification disabled278worker1 # [ 0.002902] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]279worker2 # [ 0.000000] Hypervisor detected: KVM280client # [ 0.005462] DMA [mem 0x0000000000001000-0x0000000000ffffff]281lb # [ 0.002934] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )282worker1 # [ 0.002903] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]283worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000284client # [ 0.005463] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]285client # [ 0.005465] Normal empty286worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00287client # [ 0.005465] Device empty288worker1 # [ 0.002903] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]289lb # [ 0.002938] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)290worker2 # [ 0.000000] kvm-clock: using sched offset of 482650387 cycles291client # [ 0.005466] Movable zone start for each node292worker1 # [ 0.002904] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]293client # [ 0.005467] Early memory node ranges294lb # [ 0.002943] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)295worker1 # [ 0.003131] No NUMA configuration found296client # [ 0.005467] node 0: [mem 0x0000000000001000-0x000000000009efff]297worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns298worker1 # [ 0.003132] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]299lb # [ 0.002950] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)300worker2 # [ 0.000006] tsc: Detected 2400.012 MHz processor301client # [ 0.005469] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]302worker1 # [ 0.003135] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]303lb # [ 0.002952] ACPI: FACS 0x000000005FFE0000 000040304worker1 # [ 0.005729] Zone ranges:305worker2 # [ 0.000819] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000306client # [ 0.005470] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]307lb # [ 0.002954] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)308worker1 # [ 0.005730] DMA [mem 0x0000000000001000-0x0000000000ffffff]309worker2 # [ 0.000851] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs310client # [ 0.005490] On node 0, zone DMA: 1 pages in unavailable ranges311worker1 # [ 0.005732] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]312client # [ 0.005753] On node 0, zone DMA: 97 pages in unavailable ranges313worker1 # [ 0.005733] Normal empty314worker2 # [ 0.000854] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT315lb # [ 0.002955] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)316worker1 # [ 0.005734] Device empty317client # [ 0.033185] On node 0, zone DMA32: 40 pages in unavailable ranges318worker2 # [ 0.002775] found SMP MP-table at [mem 0x000f5450-0x000f545f]319worker1 # [ 0.005734] Movable zone start for each node320lb # [ 0.002957] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)321client # [ 0.033658] ACPI: PM-Timer IO Port: 0x608322worker1 # [ 0.005735] Early memory node ranges323worker2 # [ 0.002790] Using GB pages for direct mapping324client # [ 0.033671] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])325worker2 # [ 0.002941] RAMDISK: [mem 0x5e367000-0x5ffcffff]326worker1 # [ 0.005736] node 0: [mem 0x0000000000001000-0x000000000009efff]327lb # [ 0.002959] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)328client # [ 0.033699] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23329worker2 # [ 0.002950] ACPI: Early table checksum verification disabled330worker1 # [ 0.005737] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]331lb # [ 0.002960] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]332worker2 # [ 0.002954] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )333client # [ 0.033702] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)334worker1 # [ 0.005738] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]335lb # [ 0.002961] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]336client # [ 0.033703] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)337worker2 # [ 0.002959] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)338worker1 # [ 0.005757] On node 0, zone DMA: 1 pages in unavailable ranges339lb # [ 0.002961] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]340client # [ 0.033704] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)341worker1 # [ 0.006022] On node 0, zone DMA: 97 pages in unavailable ranges342worker2 # [ 0.002963] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)343lb # [ 0.002962] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]344worker1 # [ 0.033186] On node 0, zone DMA32: 40 pages in unavailable ranges345client # [ 0.033706] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)346worker1 # [ 0.033653] ACPI: PM-Timer IO Port: 0x608347worker2 # [ 0.002971] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)348lb # [ 0.002962] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]349client # [ 0.033706] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)350worker1 # [ 0.033665] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])351worker2 # [ 0.002973] ACPI: FACS 0x000000005FFE0000 000040352lb # [ 0.002963] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]353client # [ 0.033709] ACPI: Using ACPI (MADT) for SMP configuration information354worker1 # [ 0.033690] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23355client # [ 0.033710] ACPI: HPET id: 0x8086a201 base: 0xfed00000356lb # [ 0.002963] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]357worker2 # [ 0.002975] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)358client # [ 0.033714] TSC deadline timer available359lb # [ 0.003207] No NUMA configuration found360worker1 # [ 0.033693] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)361client # [ 0.033718] CPU topo: Max. logical packages: 1362worker2 # [ 0.002976] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)363worker1 # [ 0.033694] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)364lb # [ 0.003208] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]365client # [ 0.033719] CPU topo: Max. logical dies: 1366lb # [ 0.003211] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]367client # [ 0.033719] CPU topo: Max. dies per package: 1368worker1 # [ 0.033696] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)369worker2 # [ 0.002978] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)370lb # [ 0.005712] Zone ranges:371client # [ 0.033723] CPU topo: Max. threads per core: 1372worker1 # [ 0.033697] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)373lb # [ 0.005713] DMA [mem 0x0000000000001000-0x0000000000ffffff]374worker2 # [ 0.002980] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)375client # [ 0.033723] CPU topo: Num. cores per package: 1376worker1 # [ 0.033697] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)377client # [ 0.033724] CPU topo: Num. threads per package: 1378lb # [ 0.005715] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]379worker2 # [ 0.002981] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]380lb # [ 0.005716] Normal empty381worker1 # [ 0.033700] ACPI: Using ACPI (MADT) for SMP configuration information382lb # [ 0.005717] Device empty383client # [ 0.033724] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs384worker2 # [ 0.002982] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]385lb # [ 0.005717] Movable zone start for each node386worker1 # [ 0.033701] ACPI: HPET id: 0x8086a201 base: 0xfed00000387client # [ 0.033744] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()388lb # [ 0.005718] Early memory node ranges389worker1 # [ 0.033705] TSC deadline timer available390worker2 # [ 0.002982] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]391worker1 # [ 0.033710] CPU topo: Max. logical packages: 1392lb # [ 0.005718] node 0: [mem 0x0000000000001000-0x000000000009efff]393client # [ 0.033777] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]394worker2 # [ 0.002983] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]395worker1 # [ 0.033711] CPU topo: Max. logical dies: 1396lb # [ 0.005719] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]397worker1 # [ 0.033711] CPU topo: Max. dies per package: 1398client # [ 0.033779] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]399worker2 # [ 0.002983] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]400worker1 # [ 0.033717] CPU topo: Max. threads per core: 1401lb # [ 0.005721] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]402client # [ 0.033780] [mem 0x60000000-0xafffffff] available for PCI devices403worker1 # [ 0.033718] CPU topo: Num. cores per package: 1404worker2 # [ 0.002984] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]405client # [ 0.033782] Booting paravirtualized kernel on KVM406lb # [ 0.005741] On node 0, zone DMA: 1 pages in unavailable ranges407worker1 # [ 0.033718] CPU topo: Num. threads per package: 1408worker2 # [ 0.002984] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]409lb # [ 0.006004] On node 0, zone DMA: 97 pages in unavailable ranges410worker2 # [ 0.003228] No NUMA configuration found411worker1 # [ 0.033718] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs412client # [ 0.033786] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns413lb # [ 0.032860] On node 0, zone DMA32: 40 pages in unavailable ranges414worker1 # [ 0.033737] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()415worker2 # [ 0.003229] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]416lb # [ 0.033347] ACPI: PM-Timer IO Port: 0x608417client # [ 0.038274] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1418lb # [ 0.033362] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])419worker2 # [ 0.003232] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]420worker1 # [ 0.033773] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]421worker2 # [ 0.005647] Zone ranges:422client # [ 0.040447] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152423client # [ 0.040496] kvm-guest: PV spinlocks disabled, single CPU424lb # [ 0.033391] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23425worker2 # [ 0.005648] DMA [mem 0x0000000000001000-0x0000000000ffffff]426worker1 # [ 0.033775] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]427lb # [ 0.033394] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)428worker1 # [ 0.033776] [mem 0x60000000-0xafffffff] available for PCI devices429worker2 # [ 0.005649] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]430worker2 # [ 0.005650] Normal empty431worker1 # [ 0.033778] Booting paravirtualized kernel on KVM432worker2 # [ 0.005651] Device empty433lb # [ 0.033396] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)434worker2 # [ 0.005652] Movable zone start for each node435lb # [ 0.033397] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)436worker2 # [ 0.005653] Early memory node ranges437worker1 # [ 0.033781] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns438lb # [ 0.033398] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)439worker2 # [ 0.005653] node 0: [mem 0x0000000000001000-0x000000000009efff]440worker1 # [ 0.038252] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1441client # [ 0.040498] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/8yjya2ac0ki0lpwdcag7baz48ybnsw0d-nixos-system-client-test/init regInfo=/nix/store/4zrrnd9gljfrai49b1csncr8npfgb1qj-closure-info/registration console=ttyS0,115200n8 console=tty0442worker2 # [ 0.005654] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]443lb # [ 0.033399] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)444worker1 # [ 0.040427] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152445lb # [ 0.033401] ACPI: Using ACPI (MADT) for SMP configuration information446worker2 # [ 0.005655] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]447worker1 # [ 0.040475] kvm-guest: PV spinlocks disabled, single CPU448client # [ 0.040593] Unknown kernel command line parameters "regInfo=/nix/store/4zrrnd9gljfrai49b1csncr8npfgb1qj-closure-info/registration", will be passed to user space.449lb # [ 0.033402] ACPI: HPET id: 0x8086a201 base: 0xfed00000450worker2 # [ 0.005675] On node 0, zone DMA: 1 pages in unavailable ranges451client # [ 0.040606] random: crng init done452lb # [ 0.033408] TSC deadline timer available453worker2 # [ 0.005934] On node 0, zone DMA: 97 pages in unavailable ranges454lb # [ 0.033412] CPU topo: Max. logical packages: 1455client # [ 0.040607] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes456worker2 # [ 0.031879] On node 0, zone DMA32: 40 pages in unavailable ranges457lb # [ 0.033413] CPU topo: Max. logical dies: 1458worker2 # [ 0.032370] ACPI: PM-Timer IO Port: 0x608459lb # [ 0.033413] CPU topo: Max. dies per package: 1460client # [ 0.042726] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)461lb # [ 0.033417] CPU topo: Max. threads per core: 1462worker2 # [ 0.032385] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])463client # [ 0.043762] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)464worker1 # [ 0.040476] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/28k15c24ynj80n50y8xz4mn2ffrxv0di-nixos-system-worker1-test/init regInfo=/nix/store/vzq6in1clav5d3ah71k4fp2bq0gi7dng-closure-info/registration console=ttyS0,115200n8 console=tty0465lb # [ 0.033417] CPU topo: Num. cores per package: 1466client # [ 0.043798] Fallback order for Node 0: 0467worker2 # [ 0.032414] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23468lb # [ 0.033418] CPU topo: Num. threads per package: 1469client # [ 0.043802] Built 1 zonelists, mobility grouping on. Total pages: 393078470worker2 # [ 0.032416] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)471lb # [ 0.033418] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs472client # [ 0.043803] Policy zone: DMA32473worker1 # [ 0.040572] Unknown kernel command line parameters "regInfo=/nix/store/vzq6in1clav5d3ah71k4fp2bq0gi7dng-closure-info/registration", will be passed to user space.474worker2 # [ 0.032418] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)475worker1 # [ 0.040586] random: crng init done476lb # [ 0.033447] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()477client # [ 0.046475] mem auto-init: stack:all(zero), heap alloc:on, heap free:off478worker2 # [ 0.032419] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)479client # [ 0.049750] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1480lb # [ 0.033483] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]481worker1 # [ 0.040587] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes482client # [ 0.053049] allocated 3145728 bytes of page_ext483worker2 # [ 0.032420] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)484lb # [ 0.033485] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]485worker1 # [ 0.042724] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)486client # [ 0.064234] ftrace: allocating 48731 entries in 192 pages487worker2 # [ 0.032421] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)488lb # [ 0.033487] [mem 0x60000000-0xafffffff] available for PCI devices489client # [ 0.064238] ftrace: allocated 192 pages with 2 groups490worker1 # [ 0.043779] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)491client # [ 0.065334] Dynamic Preempt: lazy492worker2 # [ 0.032424] ACPI: Using ACPI (MADT) for SMP configuration information493lb # [ 0.033488] Booting paravirtualized kernel on KVM494worker1 # [ 0.043811] Fallback order for Node 0: 0495worker2 # [ 0.032425] ACPI: HPET id: 0x8086a201 base: 0xfed00000496client # [ 0.065559] rcu: Preemptible hierarchical RCU implementation.497worker1 # [ 0.043813] Built 1 zonelists, mobility grouping on. Total pages: 393078498worker2 # [ 0.032430] TSC deadline timer available499client # [ 0.065560] rcu: RCU event tracing is enabled.500lb # [ 0.033492] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns501worker1 # [ 0.043815] Policy zone: DMA32502worker2 # [ 0.032434] CPU topo: Max. logical packages: 1503client # [ 0.065561] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.504worker1 # [ 0.047178] mem auto-init: stack:all(zero), heap alloc:on, heap free:off505worker2 # [ 0.032435] CPU topo: Max. logical dies: 1506lb # [ 0.037986] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1507client # [ 0.065562] Trampoline variant of Tasks RCU enabled.508worker2 # [ 0.032436] CPU topo: Max. dies per package: 1509worker1 # [ 0.050575] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1510lb # [ 0.040097] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152511client # [ 0.065563] Rude variant of Tasks RCU enabled.512worker2 # [ 0.032439] CPU topo: Max. threads per core: 1513worker1 # [ 0.054388] allocated 3145728 bytes of page_ext514lb # [ 0.040152] kvm-guest: PV spinlocks disabled, single CPU515client # [ 0.065563] Tracing variant of Tasks RCU enabled.516worker2 # [ 0.032439] CPU topo: Num. cores per package: 1517worker1 # [ 0.064655] ftrace: allocating 48731 entries in 192 pages518worker2 # [ 0.032440] CPU topo: Num. threads per package: 1519client # [ 0.065564] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.520worker1 # [ 0.064658] ftrace: allocated 192 pages with 2 groups521worker1 # [ 0.065538] Dynamic Preempt: lazy522worker2 # [ 0.032440] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs523client # [ 0.065565] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1524worker1 # [ 0.065707] rcu: Preemptible hierarchical RCU implementation.525worker2 # [ 0.032466] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()526worker1 # [ 0.065708] rcu: RCU event tracing is enabled.527client # [ 0.065594] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.528worker2 # [ 0.032504] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]529worker1 # [ 0.065709] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.530lb # [ 0.040153] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0qln5mjy767qjp5wbmxjg8jay8k05dsm-nixos-system-lb-test/init regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration console=ttyS0,115200n8 console=tty0531client # [ 0.065597] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.532worker1 # [ 0.065710] Trampoline variant of Tasks RCU enabled.533worker2 # [ 0.032505] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]534worker1 # [ 0.065711] Rude variant of Tasks RCU enabled.535client # [ 0.065598] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.536worker2 # [ 0.032507] [mem 0x60000000-0xafffffff] available for PCI devices537worker1 # [ 0.065711] Tracing variant of Tasks RCU enabled.538lb # [ 0.040252] Unknown kernel command line parameters "regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration", will be passed to user space.539client # [ 0.071094] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16540worker2 # [ 0.032509] Booting paravirtualized kernel on KVM541lb # [ 0.040265] random: crng init done542worker1 # [ 0.065712] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.543client # [ 0.071394] rcu: srcu_init: Setting srcu_struct sizes based on contention.544lb # [ 0.040266] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes545worker1 # [ 0.065713] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1546worker2 # [ 0.032513] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns547client # [ 0.071400] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns548lb # [ 0.042367] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)549worker2 # [ 0.037062] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1550worker1 # [ 0.065732] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.551lb # [ 0.043407] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)552worker2 # [ 0.039184] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152553client # [ 0.071509] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)554worker1 # [ 0.065734] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.555lb # [ 0.043446] Fallback order for Node 0: 0556worker2 # [ 0.039238] kvm-guest: PV spinlocks disabled, single CPU557client # [ 0.075009] Console: colour VGA+ 80x25558client # [ 0.075012] printk: legacy console [tty0] enabled559lb # [ 0.043449] Built 1 zonelists, mobility grouping on. Total pages: 393078560worker1 # [ 0.065735] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.561lb # [ 0.043451] Policy zone: DMA32562client # [ 0.104496] printk: legacy console [ttyS0] enabled563worker1 # [ 0.070135] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16564lb # [ 0.046049] mem auto-init: stack:all(zero), heap alloc:on, heap free:off565client # [ 0.201997] ACPI: Core revision 20250807566worker1 # [ 0.070413] rcu: srcu_init: Setting srcu_struct sizes based on contention.567lb # [ 0.049198] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1568lb # [ 0.052457] allocated 3145728 bytes of page_ext569worker1 # [ 0.070420] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns570client # [ 0.202861] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns571lb # [ 0.062770] ftrace: allocating 48731 entries in 192 pages572worker2 # [ 0.039240] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/98jrmc7dvx1iq9by65a311c4s2kfa5k4-nixos-system-worker2-test/init regInfo=/nix/store/9va08d8nrkb3f5vykhsq9nzd2mlr2dd9-closure-info/registration console=ttyS0,115200n8 console=tty0573client # [ 0.204424] APIC: Switch to symmetric I/O mode setup574lb # [ 0.062772] ftrace: allocated 192 pages with 2 groups575worker1 # [ 0.070537] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)576lb # [ 0.063625] Dynamic Preempt: lazy577worker1 # [ 0.074049] Console: colour VGA+ 80x25578client # [ 0.205350] x2apic enabled579worker2 # [ 0.039338] Unknown kernel command line parameters "regInfo=/nix/store/9va08d8nrkb3f5vykhsq9nzd2mlr2dd9-closure-info/registration", will be passed to user space.580worker1 # [ 0.074052] printk: legacy console [tty0] enabled581lb # [ 0.063807] rcu: Preemptible hierarchical RCU implementation.582worker2 # [ 0.039351] random: crng init done583worker1 # [ 0.103600] printk: legacy console [ttyS0] enabled584lb # [ 0.063807] rcu: RCU event tracing is enabled.585client # [ 0.206093] APIC: Switched APIC routing to: physical x2apic586worker2 # [ 0.039352] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes587worker1 # [ 0.200238] ACPI: Core revision 20250807588lb # [ 0.063808] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.589lb # [ 0.063810] Trampoline variant of Tasks RCU enabled.590worker2 # [ 0.041450] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)591lb # [ 0.063810] Rude variant of Tasks RCU enabled.592client # [ 0.207965] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1593worker1 # [ 0.201105] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns594lb # [ 0.063811] Tracing variant of Tasks RCU enabled.595worker2 # [ 0.042497] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)596worker2 # [ 0.042537] Fallback order for Node 0: 0597worker1 # [ 0.202629] APIC: Switch to symmetric I/O mode setup598lb # [ 0.063811] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.599client # [ 0.208935] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns600worker2 # [ 0.042540] Built 1 zonelists, mobility grouping on. Total pages: 393078601worker1 # [ 0.203548] x2apic enabled602worker2 # [ 0.042541] Policy zone: DMA32603lb # [ 0.063812] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1604worker1 # [ 0.204275] APIC: Switched APIC routing to: physical x2apic605worker2 # [ 0.045185] mem auto-init: stack:all(zero), heap alloc:on, heap free:off606client # [ 0.210570] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)607lb # [ 0.063833] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.608worker2 # [ 0.048420] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1609client # [ 0.211654] x86/cpu: User Mode Instruction Prevention (UMIP) activated610worker2 # [ 0.051665] allocated 3145728 bytes of page_ext611lb # [ 0.063835] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.612worker2 # [ 0.062021] ftrace: allocating 48731 entries in 192 pages613worker1 # [ 0.206019] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1614client # [ 0.212686] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127615lb # [ 0.063836] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.616worker2 # [ 0.062024] ftrace: allocated 192 pages with 2 groups617worker2 # [ 0.062892] Dynamic Preempt: lazy618lb # [ 0.068211] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16619client # [ 0.213568] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0620worker1 # [ 0.206977] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns621worker2 # [ 0.063005] rcu: Preemptible hierarchical RCU implementation.622lb # [ 0.068482] rcu: srcu_init: Setting srcu_struct sizes based on contention.623worker2 # [ 0.063006] rcu: RCU event tracing is enabled.624worker1 # [ 0.208552] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)625client # [ 0.214571] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto626worker2 # [ 0.063006] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.627lb # [ 0.068489] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns628worker2 # [ 0.063008] Trampoline variant of Tasks RCU enabled.629worker1 # [ 0.210639] x86/cpu: User Mode Instruction Prevention (UMIP) activated630worker2 # [ 0.063009] Rude variant of Tasks RCU enabled.631client # [ 0.216568] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl632lb # [ 0.068593] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)633worker2 # [ 0.063009] Tracing variant of Tasks RCU enabled.634lb # [ 0.072201] Console: colour VGA+ 80x25635worker1 # [ 0.211673] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127636client # [ 0.217568] Transient Scheduler Attacks: Vulnerable: No microcode637lb # [ 0.072205] printk: legacy console [tty0] enabled638worker2 # [ 0.063010] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.639lb # [ 0.102299] printk: legacy console [ttyS0] enabled640worker1 # [ 0.212550] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0641client # [ 0.218567] Spectre V2 : Mitigation: Enhanced / Automatic IBRS642worker2 # [ 0.063011] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1643lb # [ 0.200234] ACPI: Core revision 20250807644client # [ 0.219568] Speculative Return Stack Overflow: Mitigation: Safe RET645worker2 # [ 0.063025] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.646worker1 # [ 0.213553] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto647lb # [ 0.201111] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns648worker2 # [ 0.063027] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.649worker1 # [ 0.214549] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl650lb # [ 0.202700] APIC: Switch to symmetric I/O mode setup651worker2 # [ 0.063028] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.652worker1 # [ 0.216549] Transient Scheduler Attacks: Vulnerable: No microcode653lb # [ 0.203655] x2apic enabled654worker2 # [ 0.067811] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16655worker1 # [ 0.217549] Spectre V2 : Mitigation: Enhanced / Automatic IBRS656client # [ 0.220568] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization657lb # [ 0.204420] APIC: Switched APIC routing to: physical x2apic658worker2 # [ 0.068127] rcu: srcu_init: Setting srcu_struct sizes based on contention.659worker1 # [ 0.218549] Speculative Return Stack Overflow: Mitigation: Safe RET660client # [ 0.222575] Spectre V2 : Enabling IBPB for BPF661worker2 # [ 0.068132] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns662lb # [ 0.206309] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1663client # [ 0.223568] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier664worker2 # [ 0.068250] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)665client # [ 0.224568] active return thunk: srso_alias_return_thunk666worker2 # [ 0.071796] Console: colour VGA+ 80x25667worker2 # [ 0.071799] printk: legacy console [tty0] enabled668lb # [ 0.207266] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns669worker2 # [ 0.101425] printk: legacy console [ttyS0] enabled670client # [ 0.225588] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'671worker1 # [ 0.219549] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization672worker2 # [ 0.199015] ACPI: Core revision 20250807673lb # [ 0.208926] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)674client # [ 0.226568] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'675worker1 # [ 0.221556] Spectre V2 : Enabling IBPB for BPF676lb # [ 0.210012] x86/cpu: User Mode Instruction Prevention (UMIP) activated677client # [ 0.227567] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'678worker1 # [ 0.223061] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier679worker2 # [ 0.199899] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns680lb # [ 0.211047] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127681client # [ 0.228568] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'682worker1 # [ 0.224508] active return thunk: srso_alias_return_thunk683worker2 # [ 0.201479] APIC: Switch to symmetric I/O mode setup684client # [ 0.229568] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'685lb # [ 0.211923] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0686worker2 # [ 0.202419] x2apic enabled687worker1 # [ 0.225172] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'688client # [ 0.230567] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'689worker2 # [ 0.203160] APIC: Switched APIC routing to: physical x2apic690worker1 # [ 0.226417] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'691lb # [ 0.212927] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto692client # [ 0.231567] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'693worker1 # [ 0.227274] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'694lb # [ 0.214922] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl695worker2 # [ 0.204994] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1696client # [ 0.232568] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256697worker1 # [ 0.228294] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'698lb # [ 0.215923] Transient Scheduler Attacks: Vulnerable: No microcode699client # [ 0.233567] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64700worker1 # [ 0.229264] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'701worker2 # [ 0.205957] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns702lb # [ 0.216922] Spectre V2 : Mitigation: Enhanced / Automatic IBRS703client # [ 0.234568] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512704worker1 # [ 0.230253] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'705lb # [ 0.217923] Speculative Return Stack Overflow: Mitigation: Safe RET706client # [ 0.235567] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024707worker2 # [ 0.207607] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)708worker1 # [ 0.230549] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'709client # [ 0.236567] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8710worker2 # [ 0.208703] x86/cpu: User Mode Instruction Prevention (UMIP) activated711worker1 # [ 0.232449] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256712worker1 # [ 0.233271] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64713worker2 # [ 0.209750] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127714client # [ 0.237567] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.715worker1 # [ 0.234261] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512716worker2 # [ 0.211305] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0717lb # [ 0.218922] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization718worker1 # [ 0.235273] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024719lb # [ 0.220930] Spectre V2 : Enabling IBPB for BPF720worker2 # [ 0.212394] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto721worker1 # [ 0.236280] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8722lb # [ 0.221923] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier723worker2 # [ 0.213605] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl724worker1 # [ 0.236549] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.725lb # [ 0.222923] active return thunk: srso_alias_return_thunk726worker2 # [ 0.214606] Transient Scheduler Attacks: Vulnerable: No microcode727lb # [ 0.223944] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'728worker2 # [ 0.216376] Spectre V2 : Mitigation: Enhanced / Automatic IBRS729lb # [ 0.224922] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'730worker2 # [ 0.217337] Speculative Return Stack Overflow: Mitigation: Safe RET731lb # [ 0.226700] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'732lb # [ 0.227727] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'733lb # [ 0.228743] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'734lb # [ 0.229922] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'735worker2 # [ 0.218604] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization736worker2 # [ 0.221613] Spectre V2 : Enabling IBPB for BPF737lb # [ 0.230922] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'738lb # [ 0.231922] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256739worker2 # [ 0.222364] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier740lb # [ 0.232922] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64741worker2 # [ 0.223605] active return thunk: srso_alias_return_thunk742lb # [ 0.233922] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512743worker2 # [ 0.224627] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'744lb # [ 0.234923] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024745worker2 # [ 0.225604] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'746lb # [ 0.235922] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8747worker2 # [ 0.226605] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'748worker2 # [ 0.227604] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'749lb # [ 0.236922] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.750worker2 # [ 0.228605] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'751worker2 # [ 0.229604] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'752worker2 # [ 0.231465] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'753worker2 # [ 0.232605] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256754worker2 # [ 0.233604] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64755worker2 # [ 0.234604] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512756worker2 # [ 0.235604] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024757worker2 # [ 0.236605] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8758worker2 # [ 0.237605] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.759client # [ 0.272410] Freeing SMP alternatives memory: 44K760client # [ 0.273152] pid_max: default: 32768 minimum: 301761client # [ 0.273661] LSM: initializing lsm=capability,landlock,yama,bpf,ima762client # [ 0.274692] landlock: Up and running.763client # [ 0.275568] Yama: becoming mindful.764client # [ 0.276780] LSM support for eBPF active765worker1 # [ 0.271858] Freeing SMP alternatives memory: 44K766worker1 # [ 0.272552] pid_max: default: 32768 minimum: 301767client # [ 0.277897] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)768worker1 # [ 0.273398] LSM: initializing lsm=capability,landlock,yama,bpf,ima769worker1 # [ 0.274626] landlock: Up and running.770client # [ 0.278611] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)771worker1 # [ 0.275237] Yama: becoming mindful.772worker1 # [ 0.275763] LSM support for eBPF active773client # [ 0.281450] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)774worker1 # [ 0.276660] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)775worker1 # [ 0.277586] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)776client # [ 0.282101] Performance Events: Fam17h+ core perfctr, AMD PMU driver.777client # [ 0.282572] ... version: 2778client # [ 0.283321] ... bit width: 48779worker1 # [ 0.280437] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)780client # [ 0.283570] ... generic counters: 6781client # [ 0.284310] ... generic bitmap: 000000000000003f782client # [ 0.284615] ... fixed-purpose counters: 0783worker1 # [ 0.281374] Performance Events: Fam17h+ core perfctr, AMD PMU driver.784worker1 # [ 0.282363] ... version: 2785client # [ 0.285343] ... fixed-purpose bitmap: 0000000000000000786worker1 # [ 0.282552] ... bit width: 48787client # [ 0.285570] ... value mask: 0000ffffffffffff788worker1 # [ 0.283257] ... generic counters: 6789client # [ 0.286493] ... max period: 00007fffffffffff790worker1 # [ 0.283597] ... generic bitmap: 000000000000003f791client # [ 0.287243] ... global_ctrl mask: 000000000000003f792lb # [ 0.272173] Freeing SMP alternatives memory: 44K793worker1 # [ 0.284492] ... fixed-purpose counters: 0794client # [ 0.287678] signal: max sigframe size: 3376795lb # [ 0.272925] pid_max: default: 32768 minimum: 301796worker1 # [ 0.285109] ... fixed-purpose bitmap: 0000000000000000797client # [ 0.288467] rcu: Hierarchical SRCU implementation.798lb # [ 0.273747] LSM: initializing lsm=capability,landlock,yama,bpf,ima799worker1 # [ 0.285552] ... value mask: 0000ffffffffffff800client # [ 0.289184] rcu: Max phase no-delay instances is 400.801lb # [ 0.274771] landlock: Up and running.802worker1 # [ 0.286418] ... max period: 00007fffffffffff803lb # [ 0.275378] Yama: becoming mindful.804worker1 # [ 0.287208] ... global_ctrl mask: 000000000000003f805lb # [ 0.276135] LSM support for eBPF active806worker1 # [ 0.287662] signal: max sigframe size: 3376807worker1 # [ 0.288474] rcu: Hierarchical SRCU implementation.808lb # [ 0.277051] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)809worker1 # [ 0.289169] rcu: Max phase no-delay instances is 400.810lb # [ 0.277960] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)811client # [ 0.294120] smp: Bringing up secondary CPUs ...812client # [ 0.294584] smp: Brought up 1 node, 1 CPU813lb # [ 0.280885] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)814client # [ 0.295287] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)815lb # [ 0.281505] Performance Events: Fam17h+ core perfctr, AMD PMU driver.816worker1 # [ 0.294073] smp: Bringing up secondary CPUs ...817client # [ 0.295728] Memory: 1454552K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109348K reserved, 0K cma-reserved)818lb # [ 0.281927] ... version: 2819worker1 # [ 0.294565] smp: Brought up 1 node, 1 CPU820lb # [ 0.282612] ... bit width: 48821worker2 # [ 0.274085] Freeing SMP alternatives memory: 44K822client # [ 0.296857] devtmpfs: initialized823lb # [ 0.282925] ... generic counters: 6824worker1 # [ 0.295267] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)825client # [ 0.297582] x86/mm: Memory block size: 128MB826worker2 # [ 0.274609] pid_max: default: 32768 minimum: 301827lb # [ 0.283646] ... generic bitmap: 000000000000003f828lb # [ 0.283936] ... fixed-purpose counters: 0829worker2 # [ 0.275716] LSM: initializing lsm=capability,landlock,yama,bpf,ima830worker1 # [ 0.295694] Memory: 1454560K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)831lb # [ 0.284644] ... fixed-purpose bitmap: 0000000000000000832client # [ 0.299316] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)833worker2 # [ 0.276746] landlock: Up and running.834worker2 # [ 0.278101] Yama: becoming mindful.835lb # [ 0.284925] ... value mask: 0000ffffffffffff836worker1 # [ 0.296792] devtmpfs: initialized837client # [ 0.300461] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).838lb # [ 0.285791] ... max period: 00007fffffffffff839worker1 # [ 0.297602] x86/mm: Memory block size: 128MB840worker2 # [ 0.278864] LSM support for eBPF active841client # [ 0.301655] pinctrl core: initialized pinctrl subsystem842lb # [ 0.286592] ... global_ctrl mask: 000000000000003f843worker2 # [ 0.279756] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)844client # [ 0.302734] PM: RTC time: 12:45:53, date: 2026-09-20845lb # [ 0.287050] signal: max sigframe size: 3376846worker1 # [ 0.299411] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)847lb # [ 0.287819] rcu: Hierarchical SRCU implementation.848worker2 # [ 0.280652] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)849lb # [ 0.288529] rcu: Max phase no-delay instances is 400.850worker1 # [ 0.300456] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).851worker1 # [ 0.301644] pinctrl core: initialized pinctrl subsystem852client # [ 0.306236] NET: Registered PF_NETLINK/PF_ROUTE protocol family853worker1 # [ 0.302764] PM: RTC time: 12:45:53, date: 2026-09-20854worker2 # [ 0.284211] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)855client # [ 0.307056] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations856worker2 # [ 0.285269] Performance Events: Fam17h+ core perfctr, AMD PMU driver.857client # [ 0.307591] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations858worker2 # [ 0.285611] ... version: 2859lb # [ 0.294052] smp: Bringing up secondary CPUs ...860worker1 # [ 0.306293] NET: Registered PF_NETLINK/PF_ROUTE protocol family861worker2 # [ 0.286480] ... bit width: 48862client # [ 0.308854] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations863lb # [ 0.294813] smp: Brought up 1 node, 1 CPU864worker2 # [ 0.287202] ... generic counters: 6865client # [ 0.309581] audit: initializing netlink subsys (disabled)866worker1 # [ 0.307046] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations867lb # [ 0.295407] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)868worker2 # [ 0.287623] ... generic bitmap: 000000000000003f869client # [ 0.310572] audit: type=2000 audit(1789908353.943:1): state=initialized audit_enabled=0 res=1870worker1 # [ 0.307574] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations871worker2 # [ 0.288607] ... fixed-purpose counters: 0872lb # [ 0.296145] Memory: 1454556K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)873client # [ 0.311691] thermal_sys: Registered thermal governor 'fair_share'874worker2 # [ 0.289411] ... fixed-purpose bitmap: 0000000000000000875worker1 # [ 0.308841] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations876lb # [ 0.297206] devtmpfs: initialized877client # [ 0.311693] thermal_sys: Registered thermal governor 'bang_bang'878worker2 # [ 0.290383] ... value mask: 0000ffffffffffff879worker1 # [ 0.309561] audit: initializing netlink subsys (disabled)880lb # [ 0.298019] x86/mm: Memory block size: 128MB881client # [ 0.312571] thermal_sys: Registered thermal governor 'step_wise'882worker2 # [ 0.291387] ... max period: 00007fffffffffff883client # [ 0.313563] thermal_sys: Registered thermal governor 'user_space'884worker1 # [ 0.310554] audit: type=2000 audit(1789908353.951:1): state=initialized audit_enabled=0 res=1885worker2 # [ 0.292307] ... global_ctrl mask: 000000000000003f886worker1 # [ 0.311691] thermal_sys: Registered thermal governor 'fair_share'887client # [ 0.314308] thermal_sys: Registered thermal governor 'power_allocator'888lb # [ 0.299874] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)889worker2 # [ 0.292760] signal: max sigframe size: 3376890client # [ 0.314583] cpuidle: using governor menu891worker1 # [ 0.311693] thermal_sys: Registered thermal governor 'bang_bang'892worker2 # [ 0.293717] rcu: Hierarchical SRCU implementation.893lb # [ 0.300811] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).894worker1 # [ 0.312552] thermal_sys: Registered thermal governor 'step_wise'895client # [ 0.316756] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5896worker2 # [ 0.294614] rcu: Max phase no-delay instances is 400.897lb # [ 0.302016] pinctrl core: initialized pinctrl subsystem898worker1 # [ 0.313501] thermal_sys: Registered thermal governor 'user_space'899lb # [ 0.303212] PM: RTC time: 12:45:53, date: 2026-09-20900client # [ 0.317802] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]901worker1 # [ 0.314293] thermal_sys: Registered thermal governor 'power_allocator'902worker1 # [ 0.314567] cpuidle: using governor menu903client # [ 0.318573] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry904client # [ 0.319587] PCI: Using configuration type 1 for base access905worker1 # [ 0.316830] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5906client # [ 0.320729] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.907lb # [ 0.306921] NET: Registered PF_NETLINK/PF_ROUTE protocol family908worker1 # [ 0.317829] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]909worker2 # [ 0.300682] smp: Bringing up secondary CPUs ...910worker2 # [ 0.301469] smp: Brought up 1 node, 1 CPU911worker1 # [ 0.318555] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry912lb # [ 0.308573] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations913worker1 # [ 0.319564] PCI: Using configuration type 1 for base access914worker2 # [ 0.302209] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)915lb # [ 0.308961] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations916worker1 # [ 0.320678] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.917lb # [ 0.310231] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations918worker2 # [ 0.302887] Memory: 1454560K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109348K reserved, 0K cma-reserved)919lb # [ 0.310938] audit: initializing netlink subsys (disabled)920worker2 # [ 0.303965] devtmpfs: initialized921client # [ 0.327589] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages922lb # [ 0.312056] thermal_sys: Registered thermal governor 'fair_share'923worker2 # [ 0.304933] x86/mm: Memory block size: 128MB924lb # [ 0.312059] thermal_sys: Registered thermal governor 'bang_bang'925client # [ 0.328571] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page926lb # [ 0.312926] thermal_sys: Registered thermal governor 'step_wise'927worker1 # [ 0.325824] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages928lb # [ 0.313858] thermal_sys: Registered thermal governor 'user_space'929worker1 # [ 0.326553] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page930worker2 # [ 0.307541] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)931lb # [ 0.314717] audit: type=2000 audit(1789908353.963:1): state=initialized audit_enabled=0 res=1932lb # [ 0.316928] thermal_sys: Registered thermal governor 'power_allocator'933worker2 # [ 0.308648] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).934client # [ 0.333571] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages935lb # [ 0.316944] cpuidle: using governor menu936worker2 # [ 0.309706] pinctrl core: initialized pinctrl subsystem937client # [ 0.334572] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page938lb # [ 0.319151] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5939worker2 # [ 0.310898] PM: RTC time: 12:45:53, date: 2026-09-20940worker1 # [ 0.331554] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages941lb # [ 0.320165] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]942worker1 # [ 0.332552] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page943lb # [ 0.320927] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry944lb # [ 0.321948] PCI: Using configuration type 1 for base access945worker2 # [ 0.314587] NET: Registered PF_NETLINK/PF_ROUTE protocol family946lb # [ 0.323113] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.947worker2 # [ 0.315909] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations948worker2 # [ 0.316653] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations949worker2 # [ 0.317907] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations950worker2 # [ 0.318617] audit: initializing netlink subsys (disabled)951client # [ 0.345814] ACPI: Added _OSI(Module Device)952worker2 # [ 0.319823] thermal_sys: Registered thermal governor 'fair_share'953client # [ 0.346573] ACPI: Added _OSI(Processor Device)954client # [ 0.347357] ACPI: Added _OSI(Processor Aggregator Device)955worker1 # [ 0.342954] ACPI: Added _OSI(Module Device)956worker2 # [ 0.319825] thermal_sys: Registered thermal governor 'bang_bang'957worker1 # [ 0.343554] ACPI: Added _OSI(Processor Device)958lb # [ 0.330232] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages959worker2 # [ 0.320611] audit: type=2000 audit(1789908353.983:1): state=initialized audit_enabled=0 res=1960worker1 # [ 0.344251] ACPI: Added _OSI(Processor Aggregator Device)961lb # [ 0.330926] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page962worker2 # [ 0.322608] thermal_sys: Registered thermal governor 'step_wise'963worker2 # [ 0.322610] thermal_sys: Registered thermal governor 'user_space'964worker2 # [ 0.323607] thermal_sys: Registered thermal governor 'power_allocator'965worker2 # [ 0.324627] cpuidle: using governor menu966client # [ 0.353350] ACPI: 1 ACPI AML tables successfully acquired and loaded967lb # [ 0.335926] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages968worker2 # [ 0.326899] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5969lb # [ 0.336926] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page970worker2 # [ 0.327879] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]971worker2 # [ 0.328610] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry972client # [ 0.357246] ACPI: Interpreter enabled973worker1 # [ 0.352441] ACPI: 1 ACPI AML tables successfully acquired and loaded974worker2 # [ 0.329620] PCI: Using configuration type 1 for base access975client # [ 0.357586] ACPI: PM: (supports S0 S3 S4 S5)976client # [ 0.358304] ACPI: Using IOAPIC for interrupt routing977worker2 # [ 0.330733] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.978worker1 # [ 0.356226] ACPI: Interpreter enabled979worker1 # [ 0.356569] ACPI: PM: (supports S0 S3 S4 S5)980client # [ 0.360619] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug981worker1 # [ 0.357284] ACPI: Using IOAPIC for interrupt routing982lb # [ 0.345576] ACPI: Added _OSI(Module Device)983client # [ 0.361571] PCI: Using E820 reservations for host bridge windows984lb # [ 0.345927] ACPI: Added _OSI(Processor Device)985lb # [ 0.346667] ACPI: Added _OSI(Processor Aggregator Device)986worker1 # [ 0.359614] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug987client # [ 0.364686] ACPI: Enabled 2 GPEs in block 00 to 3F988worker2 # [ 0.337863] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages989worker1 # [ 0.360553] PCI: Using E820 reservations for host bridge windows990worker2 # [ 0.338608] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page991worker1 # [ 0.363661] ACPI: Enabled 2 GPEs in block 00 to 3F992lb # [ 0.353510] ACPI: 1 ACPI AML tables successfully acquired and loaded993worker2 # [ 0.343608] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages994worker2 # [ 0.344610] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page995client # [ 0.372708] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])996lb # [ 0.356902] ACPI: Interpreter enabled997client # [ 0.373577] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]998lb # [ 0.357384] ACPI: PM: (supports S0 S3 S4 S5)999client # [ 0.374651] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1000lb # [ 0.357926] ACPI: Using IOAPIC for interrupt routing1001client # [ 0.375693] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1002worker1 # [ 0.372375] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1003lb # [ 0.358794] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1004client # [ 0.376949] PCI host bridge to bus 0000:001005worker1 # [ 0.373341] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1006client # [ 0.377575] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1007worker1 # [ 0.374635] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1008client # [ 0.378571] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1009worker1 # [ 0.375674] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1010client # [ 0.379570] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1011worker1 # [ 0.376896] PCI host bridge to bus 0000:001012lb # [ 0.363925] PCI: Using E820 reservations for host bridge windows1013client # [ 0.380571] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1014worker2 # [ 0.355341] ACPI: Added _OSI(Module Device)1015lb # [ 0.365022] ACPI: Enabled 2 GPEs in block 00 to 3F1016worker1 # [ 0.377558] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1017worker2 # [ 0.355608] ACPI: Added _OSI(Processor Device)1018client # [ 0.381570] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1019worker1 # [ 0.378554] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1020worker2 # [ 0.356331] ACPI: Added _OSI(Processor Aggregator Device)1021client # [ 0.382573] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1022worker1 # [ 0.379553] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1023client # [ 0.383572] pci_bus 0000:00: root bus resource [bus 00-ff]1024worker1 # [ 0.380552] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1025client # [ 0.384572] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1026worker1 # [ 0.381552] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1027client # [ 0.386044] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1028worker1 # [ 0.382552] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1029worker2 # [ 0.361167] ACPI: 1 ACPI AML tables successfully acquired and loaded1030worker1 # [ 0.383552] pci_bus 0000:00: root bus resource [bus 00-ff]1031worker1 # [ 0.384573] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1032lb # [ 0.373801] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1033worker1 # [ 0.386026] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1034lb # [ 0.375537] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1035lb # [ 0.376002] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1036worker2 # [ 0.366860] ACPI: Interpreter enabled1037worker2 # [ 0.367488] ACPI: PM: (supports S0 S3 S4 S5)1038lb # [ 0.377054] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1039worker2 # [ 0.368111] ACPI: Using IOAPIC for interrupt routing1040client # [ 0.388646] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1041lb # [ 0.378296] PCI host bridge to bus 0000:001042worker2 # [ 0.368659] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1043client # [ 0.389582] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1044lb # [ 0.378930] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1045worker2 # [ 0.369608] PCI: Using E820 reservations for host bridge windows1046lb # [ 0.379925] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1047client # [ 0.390519] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1048lb # [ 0.380926] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1049client # [ 0.391735] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1050worker1 # [ 0.388723] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1051worker2 # [ 0.372707] ACPI: Enabled 2 GPEs in block 00 to 3F1052lb # [ 0.381926] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1053worker1 # [ 0.389564] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1054lb # [ 0.382925] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1055client # [ 0.393468] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1056worker1 # [ 0.390530] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1057lb # [ 0.383925] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1058worker1 # [ 0.391804] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1059lb # [ 0.384926] pci_bus 0000:00: root bus resource [bus 00-ff]1060client # [ 0.396599] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1061lb # [ 0.385956] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1062worker1 # [ 0.393301] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1063client # [ 0.397577] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1064client # [ 0.398583] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1065lb # [ 0.387400] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1066client # [ 0.399576] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1067worker1 # [ 0.395587] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1068worker2 # [ 0.381300] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1069worker1 # [ 0.396483] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1070client # [ 0.401109] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1071worker2 # [ 0.381615] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1072worker1 # [ 0.397322] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1073worker1 # [ 0.398392] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1074client # [ 0.402609] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1075worker2 # [ 0.383980] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1076client # [ 0.403485] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1077worker1 # [ 0.400083] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1078worker2 # [ 0.384732] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1079client # [ 0.404385] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1080worker2 # [ 0.386014] PCI host bridge to bus 0000:001081worker2 # [ 0.386612] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1082lb # [ 0.390048] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1083worker1 # [ 0.401581] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1084client # [ 0.406115] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1085lb # [ 0.390939] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1086worker2 # [ 0.387609] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1087worker1 # [ 0.402414] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1088lb # [ 0.391937] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1089worker1 # [ 0.403269] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1090worker2 # [ 0.388607] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1091worker2 # [ 0.389606] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1092client # [ 0.408596] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1093lb # [ 0.393274] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1094worker1 # [ 0.404096] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1095worker2 # [ 0.390607] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1096client # [ 0.409418] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1097client # [ 0.410298] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1098worker2 # [ 0.391607] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1099lb # [ 0.394703] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1100worker2 # [ 0.392607] pci_bus 0000:00: root bus resource [bus 00-ff]1101worker1 # [ 0.406743] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1102client # [ 0.411137] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1103worker2 # [ 0.393633] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1104worker1 # [ 0.407559] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1105lb # [ 0.396933] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1106worker1 # [ 0.408522] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1107worker2 # [ 0.395060] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1108lb # [ 0.397813] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1109client # [ 0.412596] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1110client # [ 0.413431] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1111lb # [ 0.398692] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1112worker1 # [ 0.409934] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1113lb # [ 0.399820] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1114client # [ 0.414383] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1115worker1 # [ 0.411579] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1116lb # [ 0.401551] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1117client # [ 0.416112] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1118worker1 # [ 0.412494] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1119worker1 # [ 0.413256] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1120client # [ 0.417603] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1121lb # [ 0.402932] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1122client # [ 0.418508] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1123worker1 # [ 0.414087] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1124lb # [ 0.403835] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1125worker2 # [ 0.397738] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1126client # [ 0.419324] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1127lb # [ 0.404706] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1128worker2 # [ 0.398619] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1129worker1 # [ 0.415584] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1130client # [ 0.420173] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1131worker1 # [ 0.416427] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1132worker2 # [ 0.399655] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1133lb # [ 0.406387] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1134worker1 # [ 0.417280] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1135worker2 # [ 0.400976] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1136worker1 # [ 0.418121] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1137client # [ 0.422807] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1138lb # [ 0.408935] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1139worker2 # [ 0.402385] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1140client # [ 0.423577] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1141lb # [ 0.409924] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1142client # [ 0.424506] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1143worker1 # [ 0.420506] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1144lb # [ 0.410910] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1145worker1 # [ 0.421298] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1146worker2 # [ 0.405640] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1147client # [ 0.426029] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1148lb # [ 0.412366] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1149worker1 # [ 0.422312] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1150worker2 # [ 0.406532] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1151worker2 # [ 0.407319] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1152client # [ 0.427604] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1153worker1 # [ 0.423168] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1154lb # [ 0.413933] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1155worker2 # [ 0.407615] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1156client # [ 0.428525] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1157lb # [ 0.414838] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1158client # [ 0.429337] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1159worker1 # [ 0.424559] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1160worker2 # [ 0.409161] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1161worker1 # [ 0.425425] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1162lb # [ 0.415773] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1163client # [ 0.429576] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1164worker1 # [ 0.426286] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1165worker1 # [ 0.426557] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1166worker2 # [ 0.411466] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1167lb # [ 0.417483] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1168client # [ 0.431129] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1169worker2 # [ 0.412210] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1170client # [ 0.432409] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1171lb # [ 0.418932] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1172worker2 # [ 0.412630] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1173worker1 # [ 0.428036] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1174client # [ 0.433335] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1175lb # [ 0.419736] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1176worker1 # [ 0.429395] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1177worker2 # [ 0.414201] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1178lb # [ 0.420763] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1179worker1 # [ 0.430326] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1180client # [ 0.435088] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1181lb # [ 0.422351] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1182worker1 # [ 0.431914] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1183client # [ 0.436607] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1184worker2 # [ 0.416614] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1185worker2 # [ 0.417499] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1186client # [ 0.437577] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1187worker1 # [ 0.433589] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1188lb # [ 0.423936] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1189worker1 # [ 0.434454] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1190worker2 # [ 0.418448] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1191client # [ 0.438592] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1192lb # [ 0.424932] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1193worker1 # [ 0.435245] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1194client # [ 0.440121] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1195worker2 # [ 0.420041] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1196lb # [ 0.425889] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1197worker1 # [ 0.436368] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1198client # [ 0.441577] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1199lb # [ 0.427364] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1200worker2 # [ 0.421658] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1201client # [ 0.442438] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1202worker1 # [ 0.438580] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1203worker2 # [ 0.422522] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1204client # [ 0.443301] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1205lb # [ 0.428933] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1206worker1 # [ 0.439416] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1207lb # [ 0.429764] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1208worker1 # [ 0.440264] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1209worker2 # [ 0.423341] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1210client # [ 0.444145] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1211lb # [ 0.430688] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1212worker1 # [ 0.441098] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1213lb # [ 0.431771] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1214worker2 # [ 0.424223] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1215client # [ 0.445226] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1216worker1 # [ 0.442199] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1217lb # [ 0.433398] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1218client # [ 0.445771] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1219worker2 # [ 0.425639] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1220worker1 # [ 0.442753] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1221client # [ 0.447102] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1222lb # [ 0.434745] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1223worker2 # [ 0.426487] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1224worker1 # [ 0.444065] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1225worker2 # [ 0.427336] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1226lb # [ 0.435690] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1227worker2 # [ 0.428192] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1228lb # [ 0.437383] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1229lb # [ 0.439622] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1230worker2 # [ 0.430615] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1231lb # [ 0.439933] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1232worker2 # [ 0.431515] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1233lb # [ 0.440948] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1234worker2 # [ 0.432369] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1235lb # [ 0.442501] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1236worker2 # [ 0.434146] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1237client # [ 0.447762] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1238lb # [ 0.443933] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1239worker1 # [ 0.444747] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1240lb # [ 0.444791] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1241client # [ 0.450121] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1242worker2 # [ 0.435645] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1243lb # [ 0.445707] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1244worker1 # [ 0.447043] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1245worker2 # [ 0.436519] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1246client # [ 0.450827] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1247worker1 # [ 0.447781] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1248lb # [ 0.447321] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1249worker2 # [ 0.437358] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1250client # [ 0.452094] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1251worker2 # [ 0.437615] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1252worker1 # [ 0.449050] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1253lb # [ 0.448804] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1254client # [ 0.453090] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1255worker1 # [ 0.449818] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1256worker2 # [ 0.439149] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1257lb # [ 0.449793] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1258client # [ 0.453857] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1259worker1 # [ 0.450839] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1260lb # [ 0.451477] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1261client # [ 0.454841] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1262worker2 # [ 0.440636] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1263worker1 # [ 0.451811] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1264client # [ 0.456398] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1265worker2 # [ 0.441639] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1266worker1 # [ 0.453306] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1267client # [ 0.457257] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1268worker1 # [ 0.453558] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1269worker2 # [ 0.443239] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1270client # [ 0.457879] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1271worker1 # [ 0.454831] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1272client # [ 0.459169] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1273worker1 # [ 0.456119] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1274worker2 # [ 0.446618] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1275worker2 # [ 0.447587] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1276worker2 # [ 0.448480] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1277client # [ 0.462534] ACPI: PCI: Interrupt link LNKA configured for IRQ 101278worker1 # [ 0.459739] ACPI: PCI: Interrupt link LNKA configured for IRQ 101279client # [ 0.463473] ACPI: PCI: Interrupt link LNKB configured for IRQ 101280worker1 # [ 0.460655] ACPI: PCI: Interrupt link LNKB configured for IRQ 101281client # [ 0.464431] ACPI: PCI: Interrupt link LNKC configured for IRQ 111282worker1 # [ 0.461651] ACPI: PCI: Interrupt link LNKC configured for IRQ 111283worker2 # [ 0.450315] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1284lb # [ 0.452128] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1285client # [ 0.465360] ACPI: PCI: Interrupt link LNKD configured for IRQ 111286worker1 # [ 0.462650] ACPI: PCI: Interrupt link LNKD configured for IRQ 111287lb # [ 0.454529] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1288client # [ 0.466361] ACPI: PCI: Interrupt link LNKE configured for IRQ 101289worker1 # [ 0.463673] ACPI: PCI: Interrupt link LNKE configured for IRQ 101290worker2 # [ 0.453437] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1291client # [ 0.467387] ACPI: PCI: Interrupt link LNKF configured for IRQ 101292worker1 # [ 0.464647] ACPI: PCI: Interrupt link LNKF configured for IRQ 101293lb # [ 0.455200] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1294worker1 # [ 0.465636] ACPI: PCI: Interrupt link LNKG configured for IRQ 111295client # [ 0.468357] ACPI: PCI: Interrupt link LNKG configured for IRQ 111296worker2 # [ 0.454315] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1297lb # [ 0.456449] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1298worker1 # [ 0.466635] ACPI: PCI: Interrupt link LNKH configured for IRQ 111299client # [ 0.469423] ACPI: PCI: Interrupt link LNKH configured for IRQ 111300worker2 # [ 0.454639] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1301client # [ 0.470323] ACPI: PCI: Interrupt link GSIA configured for IRQ 161302worker1 # [ 0.467573] ACPI: PCI: Interrupt link GSIA configured for IRQ 161303lb # [ 0.457213] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1304client # [ 0.470585] ACPI: PCI: Interrupt link GSIB configured for IRQ 171305worker1 # [ 0.468582] ACPI: PCI: Interrupt link GSIB configured for IRQ 171306client # [ 0.471545] ACPI: PCI: Interrupt link GSIC configured for IRQ 181307worker2 # [ 0.456323] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1308lb # [ 0.458235] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1309worker1 # [ 0.469573] ACPI: PCI: Interrupt link GSIC configured for IRQ 181310client # [ 0.472278] ACPI: PCI: Interrupt link GSID configured for IRQ 191311worker1 # [ 0.470570] ACPI: PCI: Interrupt link GSID configured for IRQ 191312lb # [ 0.459174] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1313client # [ 0.472582] ACPI: PCI: Interrupt link GSIE configured for IRQ 201314worker2 # [ 0.458486] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1315worker1 # [ 0.471521] ACPI: PCI: Interrupt link GSIE configured for IRQ 201316client # [ 0.473486] ACPI: PCI: Interrupt link GSIF configured for IRQ 211317worker1 # [ 0.472271] ACPI: PCI: Interrupt link GSIF configured for IRQ 211318lb # [ 0.460857] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1319worker2 # [ 0.459587] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1320client # [ 0.474304] ACPI: PCI: Interrupt link GSIG configured for IRQ 221321worker1 # [ 0.472567] ACPI: PCI: Interrupt link GSIG configured for IRQ 221322lb # [ 0.461580] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1323client # [ 0.474582] ACPI: PCI: Interrupt link GSIH configured for IRQ 231324worker1 # [ 0.473511] ACPI: PCI: Interrupt link GSIH configured for IRQ 231325worker2 # [ 0.461454] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1326client # [ 0.476580] iommu: Default domain type: Translated1327lb # [ 0.462282] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1328worker1 # [ 0.475438] iommu: Default domain type: Translated1329client # [ 0.477393] iommu: DMA domain TLB invalidation policy: lazy mode1330worker1 # [ 0.476116] iommu: DMA domain TLB invalidation policy: lazy mode1331lb # [ 0.463474] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1332client # [ 0.478590] ACPI: bus type USB registered1333worker1 # [ 0.476824] ACPI: bus type USB registered1334client # [ 0.479370] usbcore: registered new interface driver usbfs1335worker1 # [ 0.477533] usbcore: registered new interface driver usbfs1336client # [ 0.480288] usbcore: registered new interface driver hub1337worker1 # [ 0.478196] usbcore: registered new interface driver hub1338client # [ 0.480580] usbcore: registered new device driver usb1339worker1 # [ 0.478563] usbcore: registered new device driver usb1340lb # [ 0.466983] ACPI: PCI: Interrupt link LNKA configured for IRQ 101341worker1 # [ 0.480309] NetLabel: Initializing1342lb # [ 0.468031] ACPI: PCI: Interrupt link LNKB configured for IRQ 101343client # [ 0.482853] NetLabel: Initializing1344worker1 # [ 0.480558] NetLabel: domain hash size = 1281345client # [ 0.483460] NetLabel: domain hash size = 1281346lb # [ 0.469025] ACPI: PCI: Interrupt link LNKC configured for IRQ 111347worker1 # [ 0.481268] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1348client # [ 0.484098] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1349lb # [ 0.470034] ACPI: PCI: Interrupt link LNKD configured for IRQ 111350worker1 # [ 0.481595] NetLabel: unlabeled traffic allowed by default1351client # [ 0.484607] NetLabel: unlabeled traffic allowed by default1352worker1 # [ 0.482493] PCI: Using ACPI for IRQ routing1353lb # [ 0.471022] ACPI: PCI: Interrupt link LNKE configured for IRQ 101354client # [ 0.485516] PCI: Using ACPI for IRQ routing1355lb # [ 0.472026] ACPI: PCI: Interrupt link LNKF configured for IRQ 101356lb # [ 0.473027] ACPI: PCI: Interrupt link LNKG configured for IRQ 111357lb # [ 0.474027] ACPI: PCI: Interrupt link LNKH configured for IRQ 111358lb # [ 0.474970] ACPI: PCI: Interrupt link GSIA configured for IRQ 161359lb # [ 0.475939] ACPI: PCI: Interrupt link GSIB configured for IRQ 171360worker2 # [ 0.462651] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1361lb # [ 0.476904] ACPI: PCI: Interrupt link GSIC configured for IRQ 181362lb # [ 0.477739] ACPI: PCI: Interrupt link GSID configured for IRQ 191363lb # [ 0.478724] ACPI: PCI: Interrupt link GSIE configured for IRQ 201364worker2 # [ 0.466354] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1365lb # [ 0.479745] ACPI: PCI: Interrupt link GSIF configured for IRQ 211366lb # [ 0.480841] ACPI: PCI: Interrupt link GSIG configured for IRQ 221367worker2 # [ 0.467758] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1368lb # [ 0.481702] ACPI: PCI: Interrupt link GSIH configured for IRQ 231369lb # [ 0.483803] iommu: Default domain type: Translated1370worker2 # [ 0.469636] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1371lb # [ 0.484531] iommu: DMA domain TLB invalidation policy: lazy mode1372lb # [ 0.485203] ACPI: bus type USB registered1373worker2 # [ 0.470981] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1374lb # [ 0.485991] usbcore: registered new interface driver usbfs1375lb # [ 0.486941] usbcore: registered new interface driver hub1376worker2 # [ 0.471998] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1377lb # [ 0.487883] usbcore: registered new device driver usb1378lb # [ 0.489481] NetLabel: Initializing1379worker2 # [ 0.472953] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1380lb # [ 0.489926] NetLabel: domain hash size = 1281381worker2 # [ 0.475656] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1382lb # [ 0.490680] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1383worker2 # [ 0.476618] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1384lb # [ 0.490969] NetLabel: unlabeled traffic allowed by default1385lb # [ 0.491939] PCI: Using ACPI for IRQ routing1386worker2 # [ 0.478048] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1387worker2 # [ 0.480212] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1388worker2 # [ 0.485443] ACPI: PCI: Interrupt link LNKA configured for IRQ 101389worker2 # [ 0.486592] ACPI: PCI: Interrupt link LNKB configured for IRQ 101390worker2 # [ 0.487716] ACPI: PCI: Interrupt link LNKC configured for IRQ 111391worker2 # [ 0.488708] ACPI: PCI: Interrupt link LNKD configured for IRQ 111392worker2 # [ 0.489704] ACPI: PCI: Interrupt link LNKE configured for IRQ 101393worker2 # [ 0.490704] ACPI: PCI: Interrupt link LNKF configured for IRQ 101394worker1 # [ 0.527106] pci 0000:00:01.0: vgaarb: setting as boot VGA device1395worker2 # [ 0.491708] ACPI: PCI: Interrupt link LNKG configured for IRQ 111396worker1 # [ 0.527548] pci 0000:00:01.0: vgaarb: bridge control possible1397client # [ 0.529686] pci 0000:00:01.0: vgaarb: setting as boot VGA device1398worker2 # [ 0.492702] ACPI: PCI: Interrupt link LNKH configured for IRQ 111399client # [ 0.530566] pci 0000:00:01.0: vgaarb: bridge control possible1400worker1 # [ 0.527548] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1401worker2 # [ 0.493642] ACPI: PCI: Interrupt link GSIA configured for IRQ 161402worker1 # [ 0.527556] vgaarb: loaded1403worker2 # [ 0.494591] ACPI: PCI: Interrupt link GSIB configured for IRQ 171404client # [ 0.530566] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1405client # [ 0.530575] vgaarb: loaded1406worker1 # [ 0.528305] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01407worker2 # [ 0.495335] ACPI: PCI: Interrupt link GSIC configured for IRQ 181408worker1 # [ 0.528552] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1409client # [ 0.531305] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01410worker2 # [ 0.495630] ACPI: PCI: Interrupt link GSID configured for IRQ 191411client # [ 0.531570] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1412worker2 # [ 0.496619] ACPI: PCI: Interrupt link GSIE configured for IRQ 201413worker2 # [ 0.497623] ACPI: PCI: Interrupt link GSIF configured for IRQ 211414worker2 # [ 0.498620] ACPI: PCI: Interrupt link GSIG configured for IRQ 221415worker1 # [ 0.531662] clocksource: Switched to clocksource kvm-clock1416worker2 # [ 0.499658] ACPI: PCI: Interrupt link GSIH configured for IRQ 231417client # [ 0.535659] clocksource: Switched to clocksource kvm-clock1418worker2 # [ 0.501772] iommu: Default domain type: Translated1419worker1 # [ 0.535161] VFS: Disk quotas dquot_6.6.01420worker2 # [ 0.502614] iommu: DMA domain TLB invalidation policy: lazy mode1421worker1 # [ 0.535842] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1422worker2 # [ 0.503879] ACPI: bus type USB registered1423worker1 # [ 0.537401] pnp: PnP ACPI init1424client # [ 0.539358] VFS: Disk quotas dquot_6.6.01425worker2 # [ 0.504665] usbcore: registered new interface driver usbfs1426worker1 # [ 0.538205] ACPI: IRQ 4 override to edge(!), high(!)1427client # [ 0.540142] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1428worker2 # [ 0.505582] usbcore: registered new interface driver hub1429client # [ 0.541550] pnp: PnP ACPI init1430worker1 # [ 0.539182] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1431worker2 # [ 0.506356] usbcore: registered new device driver usb1432client # [ 0.542374] ACPI: IRQ 4 override to edge(!), high(!)1433worker1 # [ 0.540673] pnp: PnP ACPI: found 6 devices1434worker2 # [ 0.507587] NetLabel: Initializing1435client # [ 0.543404] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1436worker2 # [ 0.508104] NetLabel: domain hash size = 1281437client # [ 0.544969] pnp: PnP ACPI: found 6 devices1438worker2 # [ 0.508607] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1439worker2 # [ 0.509614] NetLabel: unlabeled traffic allowed by default1440worker2 # [ 0.510571] PCI: Using ACPI for IRQ routing1441worker1 # [ 0.548396] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1442lb # [ 0.537169] pci 0000:00:01.0: vgaarb: setting as boot VGA device1443worker1 # [ 0.549892] clocksource: Switched to clocksource acpi_pm1444lb # [ 0.537921] pci 0000:00:01.0: vgaarb: bridge control possible1445worker1 # [ 0.550819] NET: Registered PF_INET protocol family1446client # [ 0.552575] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1447lb # [ 0.537921] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1448client # [ 0.554027] clocksource: Switched to clocksource acpi_pm1449worker1 # [ 0.552002] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1450lb # [ 0.537930] vgaarb: loaded1451client # [ 0.554969] NET: Registered PF_INET protocol family1452lb # [ 0.538713] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01453lb # [ 0.539563] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1454client # [ 0.556143] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1455lb # [ 0.543024] clocksource: Switched to clocksource kvm-clock1456lb # [ 0.546806] VFS: Disk quotas dquot_6.6.01457lb # [ 0.547561] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1458lb # [ 0.548954] pnp: PnP ACPI init1459lb # [ 0.549792] ACPI: IRQ 4 override to edge(!), high(!)1460lb # [ 0.550873] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1461lb # [ 0.552427] pnp: PnP ACPI: found 6 devices1462worker1 # [ 0.569096] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1463worker1 # [ 0.570675] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1464client # [ 0.573317] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1465worker1 # [ 0.572059] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1466client # [ 0.574784] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1467worker1 # [ 0.573445] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1468client # [ 0.576150] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1469worker1 # [ 0.574741] TCP: Hash tables configured (established 16384 bind 16384)1470client # [ 0.577603] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1471lb # [ 0.560074] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1472worker1 # [ 0.576030] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1473lb # [ 0.561585] clocksource: Switched to clocksource acpi_pm1474client # [ 0.578958] TCP: Hash tables configured (established 16384 bind 16384)1475worker1 # [ 0.577329] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1476lb # [ 0.562555] NET: Registered PF_INET protocol family1477worker1 # [ 0.578389] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1478client # [ 0.580192] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1479worker1 # [ 0.579607] NET: Registered PF_UNIX/PF_LOCAL protocol family1480lb # [ 0.563763] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1481client # [ 0.581524] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1482worker1 # [ 0.580567] NET: Registered PF_XDP protocol family1483client # [ 0.582579] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1484worker1 # [ 0.581433] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1485client # [ 0.583853] NET: Registered PF_UNIX/PF_LOCAL protocol family1486worker1 # [ 0.582487] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1487client # [ 0.584863] NET: Registered PF_XDP protocol family1488worker1 # [ 0.583531] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1489client # [ 0.585742] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1490worker1 # [ 0.584697] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1491client # [ 0.586790] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1492worker1 # [ 0.585806] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1493client # [ 0.587859] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1494worker1 # [ 0.586883] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1495client # [ 0.589072] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1496worker1 # [ 0.588550] ACPI: \_SB_.GSIA: Enabled at IRQ 161497client # [ 0.590230] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1498worker2 # [ 0.554365] pci 0000:00:01.0: vgaarb: setting as boot VGA device1499client # [ 0.591377] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1500worker2 # [ 0.554603] pci 0000:00:01.0: vgaarb: bridge control possible1501worker1 # [ 0.590753] ACPI: \_SB_.GSIB: Enabled at IRQ 171502client # [ 0.593046] ACPI: \_SB_.GSIA: Enabled at IRQ 161503worker2 # [ 0.554603] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1504worker2 # [ 0.554613] vgaarb: loaded1505worker1 # [ 0.592718] ACPI: \_SB_.GSIC: Enabled at IRQ 181506worker2 # [ 0.555375] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01507client # [ 0.595321] ACPI: \_SB_.GSIB: Enabled at IRQ 171508worker2 # [ 0.555609] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1509worker1 # [ 0.594669] ACPI: \_SB_.GSID: Enabled at IRQ 191510client # [ 0.597414] ACPI: \_SB_.GSIC: Enabled at IRQ 181511worker1 # [ 0.596697] PCI: CLS 0 bytes, default 641512lb # [ 0.580370] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1513client # [ 0.599481] ACPI: \_SB_.GSID: Enabled at IRQ 191514lb # [ 0.581925] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1515worker1 # [ 0.597608] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1516client # [ 0.601197] PCI: CLS 0 bytes, default 641517lb # [ 0.583230] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1518worker1 # [ 0.599411] Trying to unpack rootfs image as initramfs...1519worker2 # [ 0.560722] clocksource: Switched to clocksource kvm-clock1520lb # [ 0.584613] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1521client # [ 0.602050] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1522lb # [ 0.585839] TCP: Hash tables configured (established 16384 bind 16384)1523client # [ 0.603820] Trying to unpack rootfs image as initramfs...1524lb # [ 0.587004] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1525worker2 # [ 0.564727] VFS: Disk quotas dquot_6.6.01526lb # [ 0.588379] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1527worker2 # [ 0.565461] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1528lb # [ 0.589514] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1529worker2 # [ 0.566792] pnp: PnP ACPI init1530lb # [ 0.590742] NET: Registered PF_UNIX/PF_LOCAL protocol family1531worker2 # [ 0.567626] ACPI: IRQ 4 override to edge(!), high(!)1532lb # [ 0.591747] NET: Registered PF_XDP protocol family1533lb # [ 0.592631] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1534worker2 # [ 0.568606] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1535lb # [ 0.593687] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1536worker2 # [ 0.570287] pnp: PnP ACPI: found 6 devices1537lb # [ 0.594741] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1538lb # [ 0.595887] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1539lb # [ 0.597036] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1540lb # [ 0.598177] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1541lb # [ 0.599991] ACPI: \_SB_.GSIA: Enabled at IRQ 161542worker2 # [ 0.578127] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1543lb # [ 0.602946] ACPI: \_SB_.GSIB: Enabled at IRQ 171544worker2 # [ 0.579614] clocksource: Switched to clocksource acpi_pm1545worker2 # [ 0.580601] NET: Registered PF_INET protocol family1546lb # [ 0.605009] ACPI: \_SB_.GSIC: Enabled at IRQ 181547worker2 # [ 0.581805] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1548lb # [ 0.606997] ACPI: \_SB_.GSID: Enabled at IRQ 191549lb # [ 0.608727] PCI: CLS 0 bytes, default 641550lb # [ 0.609676] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1551lb # [ 0.611404] Trying to unpack rootfs image as initramfs...1552worker2 # [ 0.599077] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1553worker2 # [ 0.600663] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1554worker2 # [ 0.601974] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1555worker2 # [ 0.603407] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1556worker2 # [ 0.604745] TCP: Hash tables configured (established 16384 bind 16384)1557worker1 # [ 0.645588] Initialise system trusted keyrings1558worker2 # [ 0.606023] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1559client # [ 0.648376] Initialise system trusted keyrings1560worker1 # [ 0.646671] workingset: timestamp_bits=40 max_order=19 bucket_order=01561worker2 # [ 0.607467] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1562worker2 # [ 0.608550] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1563worker2 # [ 0.609776] NET: Registered PF_UNIX/PF_LOCAL protocol family1564client # [ 0.651342] workingset: timestamp_bits=40 max_order=19 bucket_order=01565worker2 # [ 0.610751] NET: Registered PF_XDP protocol family1566worker2 # [ 0.611589] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1567worker2 # [ 0.612663] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1568worker2 # [ 0.613701] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1569worker2 # [ 0.614832] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1570worker2 # [ 0.615925] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1571worker2 # [ 0.617091] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1572worker2 # [ 0.618896] ACPI: \_SB_.GSIA: Enabled at IRQ 161573worker2 # [ 0.621369] ACPI: \_SB_.GSIB: Enabled at IRQ 171574worker2 # [ 0.623400] ACPI: \_SB_.GSIC: Enabled at IRQ 181575worker2 # [ 0.625496] ACPI: \_SB_.GSID: Enabled at IRQ 191576worker2 # [ 0.627258] PCI: CLS 0 bytes, default 641577worker2 # [ 0.628169] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1578worker2 # [ 0.629952] Trying to unpack rootfs image as initramfs...1579worker1 # [ 0.670245] Key type asymmetric registered1580worker1 # [ 0.670941] Asymmetric key parser 'x509' registered1581lb # [ 0.655816] Initialise system trusted keyrings1582worker1 # [ 0.671768] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1583client # [ 0.675360] Key type asymmetric registered1584client # [ 0.676091] Asymmetric key parser 'x509' registered1585lb # [ 0.659611] workingset: timestamp_bits=40 max_order=19 bucket_order=01586client # [ 0.676995] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1587worker1 # [ 0.676315] io scheduler mq-deadline registered1588worker1 # [ 0.677053] io scheduler kyber registered1589client # [ 0.681376] io scheduler mq-deadline registered1590client # [ 0.682349] io scheduler kyber registered1591worker1 # [ 0.680776] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1592worker1 # [ 0.682018] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1593client # [ 0.685751] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1594client # [ 0.687005] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1595worker1 # [ 0.685961] Linux agpgart interface v0.1031596worker1 # [ 0.686703] ACPI: bus type drm_connector registered1597client # [ 0.691021] Linux agpgart interface v0.1031598client # [ 0.691772] ACPI: bus type drm_connector registered1599worker1 # [ 0.691238] usbcore: registered new interface driver usbserial_generic1600worker1 # [ 0.692263] usbserial: USB Serial support registered for generic1601client # [ 0.696322] usbcore: registered new interface driver usbserial_generic1602client # [ 0.697356] usbserial: USB Serial support registered for generic1603worker1 # [ 0.695157] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1604worker1 # [ 0.695157] Please enable it if your BIOS has the CPPC option.1605lb # [ 0.681505] Key type asymmetric registered1606worker1 # [ 0.697313] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1607lb # [ 0.682198] Asymmetric key parser 'x509' registered1608client # [ 0.700191] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1609lb # [ 0.683065] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1610client # [ 0.700191] Please enable it if your BIOS has the CPPC option.1611worker1 # [ 0.700272] drop_monitor: Initializing network drop monitor service1612client # [ 0.702293] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1613worker1 # [ 0.701400] NET: Registered PF_INET6 protocol family1614client # [ 0.705373] drop_monitor: Initializing network drop monitor service1615lb # [ 0.688633] io scheduler mq-deadline registered1616worker1 # [ 0.704595] Segment Routing with IPv61617client # [ 0.706502] NET: Registered PF_INET6 protocol family1618lb # [ 0.689383] io scheduler kyber registered1619worker1 # [ 0.707180] In-situ OAM (IOAM) with IPv61620client # [ 0.709654] Segment Routing with IPv61621lb # [ 0.692036] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1622worker1 # [ 0.708210] IPI shorthand broadcast: enabled1623lb # [ 0.693303] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1624client # [ 0.712229] In-situ OAM (IOAM) with IPv61625client # [ 0.713259] IPI shorthand broadcast: enabled1626lb # [ 0.698609] Linux agpgart interface v0.1031627lb # [ 0.699346] ACPI: bus type drm_connector registered1628worker2 # [ 0.675582] Initialise system trusted keyrings1629worker1 # [ 0.716321] sched_clock: Marking stable (582013683, 134128780)->(786712717, -70570254)1630worker2 # [ 0.678552] workingset: timestamp_bits=40 max_order=19 bucket_order=01631lb # [ 0.702671] usbcore: registered new interface driver usbserial_generic1632lb # [ 0.703718] usbserial: USB Serial support registered for generic1633client # [ 0.721414] sched_clock: Marking stable (586013306, 135166644)->(796537857, -75357907)1634worker1 # [ 0.721335] registered taskstats version 11635worker1 # [ 0.722263] Loading compiled-in X.509 certificates1636lb # [ 0.706475] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1637lb # [ 0.706475] Please enable it if your BIOS has the CPPC option.1638client # [ 0.726437] registered taskstats version 11639lb # [ 0.708626] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1640client # [ 0.727471] Loading compiled-in X.509 certificates1641lb # [ 0.711612] drop_monitor: Initializing network drop monitor service1642lb # [ 0.712854] NET: Registered PF_INET6 protocol family1643lb # [ 0.718578] Segment Routing with IPv61644lb # [ 0.719247] In-situ OAM (IOAM) with IPv61645lb # [ 0.720357] IPI shorthand broadcast: enabled1646worker1 # [ 0.738789] Demotion targets for Node 0: null1647worker2 # [ 0.699833] Key type asymmetric registered1648worker1 # [ 0.741212] Key type .fscrypt registered1649worker1 # [ 0.741906] Key type fscrypt-provisioning registered1650worker2 # [ 0.702495] Asymmetric key parser 'x509' registered1651worker1 # [ 0.742845] ima: No TPM chip found, activating TPM-bypass!1652worker2 # [ 0.703308] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1653client # [ 0.746197] Demotion targets for Node 0: null1654lb # [ 0.728837] sched_clock: Marking stable (592015986, 136432938)->(801860961, -73412037)1655client # [ 0.747098] Key type .fscrypt registered1656client # [ 0.747761] Key type fscrypt-provisioning registered1657worker2 # [ 0.706611] io scheduler mq-deadline registered1658worker2 # [ 0.707343] io scheduler kyber registered1659client # [ 0.748695] ima: No TPM chip found, activating TPM-bypass!1660worker1 # [ 0.747160] ima: Allocated hash algorithm: sha11661worker1 # [ 0.747987] ima: No architecture policies found1662lb # [ 0.733714] registered taskstats version 11663worker2 # [ 0.710136] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1664worker1 # [ 0.750166] PM: Magic number: 10:31:7851665lb # [ 0.734670] Loading compiled-in X.509 certificates1666client # [ 0.753198] ima: Allocated hash algorithm: sha11667worker2 # [ 0.711408] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1668worker1 # [ 0.751730] RAS: Correctable Errors collector initialized.1669client # [ 0.753956] ima: No architecture policies found1670client # [ 0.754868] PM: Magic number: 10:31:7851671worker2 # [ 0.716921] Linux agpgart interface v0.1031672client # [ 0.758536] RAS: Correctable Errors collector initialized.1673worker2 # [ 0.717800] ACPI: bus type drm_connector registered1674worker1 # [ 0.759734] clk: Disabling unused clocks1675worker2 # [ 0.721001] usbcore: registered new interface driver usbserial_generic1676worker2 # [ 0.722099] usbserial: USB Serial support registered for generic1677worker1 # [ 0.764163] PM: genpd: Disabling unused power domains1678client # [ 0.767227] clk: Disabling unused clocks1679worker2 # [ 0.725500] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1680client # [ 0.767890] PM: genpd: Disabling unused power domains1681worker2 # [ 0.725500] Please enable it if your BIOS has the CPPC option.1682worker2 # [ 0.727662] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1683lb # [ 0.753462] Demotion targets for Node 0: null1684lb # [ 0.754315] Key type .fscrypt registered1685lb # [ 0.754996] Key type fscrypt-provisioning registered1686worker2 # [ 0.731671] drop_monitor: Initializing network drop monitor service1687lb # [ 0.755906] ima: No TPM chip found, activating TPM-bypass!1688worker2 # [ 0.732820] NET: Registered PF_INET6 protocol family1689lb # [ 0.758461] ima: Allocated hash algorithm: sha11690lb # [ 0.759229] ima: No architecture policies found1691worker2 # [ 0.736841] Segment Routing with IPv61692worker2 # [ 0.737510] In-situ OAM (IOAM) with IPv61693worker2 # [ 0.738574] IPI shorthand broadcast: enabled1694lb # [ 0.763471] PM: Magic number: 10:31:7851695lb # [ 0.765070] RAS: Correctable Errors collector initialized.1696worker2 # [ 0.746886] sched_clock: Marking stable (611019716, 135470852)->(833357203, -86866635)1697lb # [ 0.773155] clk: Disabling unused clocks1698worker2 # [ 0.752626] registered taskstats version 11699lb # [ 0.777463] PM: genpd: Disabling unused power domains1700worker2 # [ 0.753648] Loading compiled-in X.509 certificates1701worker2 # [ 0.773525] Demotion targets for Node 0: null1702worker2 # [ 0.774599] Key type .fscrypt registered1703worker2 # [ 0.775288] Key type fscrypt-provisioning registered1704worker2 # [ 0.777645] ima: No TPM chip found, activating TPM-bypass!1705worker2 # [ 0.778650] ima: Allocated hash algorithm: sha11706worker2 # [ 0.781545] ima: No architecture policies found1707worker2 # [ 0.784503] PM: Magic number: 10:31:7851708worker2 # [ 0.786312] RAS: Correctable Errors collector initialized.1709worker2 # [ 0.796435] clk: Disabling unused clocks1710worker2 # [ 0.798507] PM: genpd: Disabling unused power domains1711client # [ 0.936491] Freeing initrd memory: 29096K1712client # [ 0.939816] Freeing unused decrypted memory: 2028K1713client # [ 0.942601] Freeing unused kernel image (initmem) memory: 3644K1714client # [ 0.943696] Write protecting the kernel read-only data: 32768k1715client # [ 0.945677] Freeing unused kernel image (text/rodata gap) memory: 1208K1716client # [ 0.947296] Freeing unused kernel image (rodata/data gap) memory: 752K1717worker1 # [ 0.954514] Freeing initrd memory: 29092K1718worker1 # [ 0.958119] Freeing unused decrypted memory: 2028K1719worker1 # [ 0.961183] Freeing unused kernel image (initmem) memory: 3644K1720worker1 # [ 0.962334] Write protecting the kernel read-only data: 32768k1721worker1 # [ 0.964408] Freeing unused kernel image (text/rodata gap) memory: 1208K1722worker1 # [ 0.966054] Freeing unused kernel image (rodata/data gap) memory: 752K1723lb # [ 0.952807] Freeing initrd memory: 29092K1724lb # [ 0.955970] Freeing unused decrypted memory: 2028K1725lb # [ 0.958641] Freeing unused kernel image (initmem) memory: 3644K1726lb # [ 0.959736] Write protecting the kernel read-only data: 32768k1727lb # [ 0.961724] Freeing unused kernel image (text/rodata gap) memory: 1208K1728lb # [ 0.963292] Freeing unused kernel image (rodata/data gap) memory: 752K1729client # [ 0.998428] x86/mm: Checked W+X mappings: passed, no W+X pages found.1730client # [ 0.999578] Run /init as init process1731client # [ 1.009748] systemd[1]: Inserted module 'autofs4'1732worker1 # [ 1.017383] x86/mm: Checked W+X mappings: passed, no W+X pages found.1733worker1 # [ 1.018560] Run /init as init process1734client # [ 1.026213] fuse: init (API version 7.45)1735client # [ 1.032035] ACPI: \_SB_.GSIG: Enabled at IRQ 221736lb # [ 1.014400] x86/mm: Checked W+X mappings: passed, no W+X pages found.1737worker1 # [ 1.030398] systemd[1]: Inserted module 'autofs4'1738worker2 # [ 0.990783] Freeing initrd memory: 29092K1739lb # [ 1.015482] Run /init as init process1740client # [ 1.034386] ACPI: \_SB_.GSIH: Enabled at IRQ 231741worker2 # [ 0.994177] Freeing unused decrypted memory: 2028K1742client # [ 1.038138] ACPI: \_SB_.GSIE: Enabled at IRQ 201743worker2 # [ 0.997137] Freeing unused kernel image (initmem) memory: 3644K1744worker2 # [ 0.998354] Write protecting the kernel read-only data: 32768k1745client # [ 1.041053] ACPI: \_SB_.GSIF: Enabled at IRQ 211746worker2 # [ 1.000435] Freeing unused kernel image (text/rodata gap) memory: 1208K1747lb # [ 1.026026] systemd[1]: Inserted module 'autofs4'1748worker2 # [ 1.002074] Freeing unused kernel image (rodata/data gap) memory: 752K1749worker1 # [ 1.047753] fuse: init (API version 7.45)1750worker1 # [ 1.054433] ACPI: \_SB_.GSIG: Enabled at IRQ 221751worker1 # [ 1.057250] ACPI: \_SB_.GSIH: Enabled at IRQ 231752lb # [ 1.042530] fuse: init (API version 7.45)1753worker1 # [ 1.060563] ACPI: \_SB_.GSIE: Enabled at IRQ 201754worker1 # [ 1.063071] ACPI: \_SB_.GSIF: Enabled at IRQ 211755lb # [ 1.048550] ACPI: \_SB_.GSIG: Enabled at IRQ 221756lb # [ 1.051266] ACPI: \_SB_.GSIH: Enabled at IRQ 231757lb # [ 1.054598] ACPI: \_SB_.GSIE: Enabled at IRQ 201758client # [ 1.073564] systemd[1]: Successfully made /usr/ read-only.1759lb # [ 1.057038] ACPI: \_SB_.GSIF: Enabled at IRQ 211760worker1 # [ 1.091096] systemd[1]: Successfully made /usr/ read-only.1761worker2 # [ 1.054055] x86/mm: Checked W+X mappings: passed, no W+X pages found.1762worker2 # [ 1.055261] Run /init as init process1763lb # [ 1.085251] systemd[1]: Successfully made /usr/ read-only.1764worker2 # [ 1.066793] systemd[1]: Inserted module 'autofs4'1765worker2 # [ 1.084178] fuse: init (API version 7.45)1766worker2 # [ 1.090511] ACPI: \_SB_.GSIG: Enabled at IRQ 221767worker2 # [ 1.092989] ACPI: \_SB_.GSIH: Enabled at IRQ 231768worker2 # [ 1.096620] ACPI: \_SB_.GSIE: Enabled at IRQ 201769worker2 # [ 1.099291] ACPI: \_SB_.GSIF: Enabled at IRQ 211770worker2 # [ 1.126660] systemd[1]: Successfully made /usr/ read-only.1771client # [ 1.409113] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1772client # [ 1.414197] systemd[1]: Detected virtualization kvm.1773client # [ 1.415024] systemd[1]: Detected architecture x86-64.1774client # [ 1.415911] systemd[1]: Running in initrd.1775client # [ 1.417138] systemd[1]: Initializing machine ID from random generator.1776client # [ 1.418446] systemd[1]: Hostname set to <client>.1777worker1 # [ 1.427822] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1778worker1 # [ 1.440066] systemd[1]: Detected virtualization kvm.1779worker1 # [ 1.442266] systemd[1]: Detected architecture x86-64.1780worker1 # [ 1.444383] systemd[1]: Running in initrd.1781lb # [ 1.421887] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1782worker1 # [ 1.446964] systemd[1]: Initializing machine ID from random generator.1783lb # [ 1.433760] systemd[1]: Detected virtualization kvm.1784worker1 # [ 1.449839] systemd[1]: Hostname set to <worker1>.1785lb # [ 1.435844] systemd[1]: Detected architecture x86-64.1786lb # [ 1.437959] systemd[1]: Running in initrd.1787lb # [ 1.440423] systemd[1]: Initializing machine ID from random generator.1788lb # [ 1.443217] systemd[1]: Hostname set to <lb>.1789worker2 # [ 1.463300] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1790worker2 # [ 1.475357] systemd[1]: Detected virtualization kvm.1791worker2 # [ 1.477531] systemd[1]: Detected architecture x86-64.1792worker2 # [ 1.479639] systemd[1]: Running in initrd.1793worker2 # [ 1.482171] systemd[1]: Initializing machine ID from random generator.1794worker2 # [ 1.485097] systemd[1]: Hostname set to <worker2>.1795client # [ 1.625128] systemd[1]: bpf-restrict-fs: LSM BPF program attached1796worker2 # [ 1.589447] systemd[1]: bpf-restrict-fs: LSM BPF program attached1797worker1 # [ 1.667336] systemd[1]: bpf-restrict-fs: LSM BPF program attached1798worker2 # [ 1.629018] systemd[1]: Queued start job for default target Initrd Default Target.1799worker2 # [ 1.632831] systemd[1]: Created slice Slice /system/modprobe.1800worker2 # [ 1.634035] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1801worker2 # [ 1.635568] systemd[1]: Expecting device /dev/disk/by-label/nixos...1802worker2 # [ 1.636741] systemd[1]: Reached target Path Units.1803worker2 # [ 1.637636] systemd[1]: Reached target Slice Units.1804worker2 # [ 1.638495] systemd[1]: Reached target Swaps.1805worker2 # [ 1.639272] systemd[1]: Reached target Timer Units.1806client # [ 1.680900] systemd[1]: Queued start job for default target Initrd Default Target.1807worker2 # [ 1.640279] systemd[1]: Listening on D-Bus System Message Bus Socket.1808worker2 # [ 1.641582] systemd[1]: Listening on Journal Socket (/dev/log).1809worker2 # [ 1.642801] systemd[1]: Listening on Journal Sockets.1810worker2 # [ 1.643864] systemd[1]: Listening on udev Control Socket.1811client # [ 1.685613] systemd[1]: Created slice Slice /system/modprobe.1812worker2 # [ 1.644957] systemd[1]: Listening on udev Kernel Socket.1813client # [ 1.686797] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1814worker2 # [ 1.645953] systemd[1]: Reached target Socket Units.1815client # [ 1.688248] systemd[1]: Expecting device /dev/disk/by-label/nixos...1816lb # [ 1.669499] systemd[1]: bpf-restrict-fs: LSM BPF program attached1817client # [ 1.689387] systemd[1]: Reached target Path Units.1818worker2 # [ 1.647910] systemd[1]: Starting Create List of Static Device Nodes...1819client # [ 1.690260] systemd[1]: Reached target Slice Units.1820client # [ 1.691142] systemd[1]: Reached target Swaps.1821client # [ 1.691950] systemd[1]: Reached target Timer Units.1822client # [ 1.692982] systemd[1]: Listening on D-Bus System Message Bus Socket.1823client # [ 1.694286] systemd[1]: Listening on Journal Socket (/dev/log).1824worker2 # [ 1.653568] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1825client # [ 1.695514] systemd[1]: Listening on Journal Sockets.1826client # [ 1.696552] systemd[1]: Listening on udev Control Socket.1827client # [ 1.697631] systemd[1]: Listening on udev Kernel Socket.1828client # [ 1.698664] systemd[1]: Reached target Socket Units.1829worker2 # [ 1.658952] systemd[1]: Starting Load Kernel Module configfs...1830client # [ 1.700733] systemd[1]: Starting Create List of Static Device Nodes...1831client # [ 1.705278] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1832client # [ 1.711302] systemd[1]: Starting Load Kernel Module configfs...1833worker2 # [ 1.675567] systemd[1]: Starting Journal Service...1834worker1 # [ 1.718058] systemd[1]: Queued start job for default target Initrd Default Target.1835worker1 # [ 1.722695] systemd[1]: Created slice Slice /system/modprobe.1836worker1 # [ 1.723957] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1837worker1 # [ 1.725445] systemd[1]: Expecting device /dev/disk/by-label/nixos...1838client # [ 1.728391] systemd[1]: Starting Journal Service...1839worker1 # [ 1.726627] systemd[1]: Reached target Path Units.1840worker1 # [ 1.727503] systemd[1]: Reached target Slice Units.1841worker2 # [ 1.688567] systemd[1]: Starting Load Kernel Modules...1842worker1 # [ 1.728440] systemd[1]: Reached target Swaps.1843worker1 # [ 1.729279] systemd[1]: Reached target Timer Units.1844worker1 # [ 1.730349] systemd[1]: Listening on D-Bus System Message Bus Socket.1845worker1 # [ 1.731625] systemd[1]: Listening on Journal Socket (/dev/log).1846worker1 # [ 1.732937] systemd[1]: Listening on Journal Sockets.1847worker1 # [ 1.733961] systemd[1]: Listening on udev Control Socket.1848worker1 # [ 1.735020] systemd[1]: Listening on udev Kernel Socket.1849lb # [ 1.719679] systemd[1]: Queued start job for default target Initrd Default Target.1850worker1 # [ 1.736031] systemd[1]: Reached target Socket Units.1851worker1 # [ 1.738070] systemd[1]: Starting Create List of Static Device Nodes...1852client # [ 1.740273] systemd[1]: Starting Load Kernel Modules...1853lb # [ 1.723748] systemd[1]: Created slice Slice /system/modprobe.1854lb # [ 1.724920] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1855lb # [ 1.726363] systemd[1]: Expecting device /dev/disk/by-label/nixos...1856worker1 # [ 1.742119] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1857lb # [ 1.727534] systemd[1]: Reached target Path Units.1858lb # [ 1.728381] systemd[1]: Reached target Slice Units.1859lb # [ 1.729297] systemd[1]: Reached target Swaps.1860lb # [ 1.730117] systemd[1]: Reached target Timer Units.1861lb # [ 1.731121] systemd[1]: Listening on D-Bus System Message Bus Socket.1862worker2 # [ 1.695579] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1863lb # [ 1.732318] systemd[1]: Listening on Journal Socket (/dev/log).1864lb # [ 1.733491] systemd[1]: Listening on Journal Sockets.1865lb # [ 1.734514] systemd[1]: Listening on udev Control Socket.1866worker2 # [ 1.711820] netfs: FS-Cache loaded1867lb # [ 1.735543] systemd[1]: Listening on udev Kernel Socket.1868worker1 # [ 1.751363] systemd[1]: Starting Load Kernel Module configfs...1869lb # [ 1.736476] systemd[1]: Reached target Socket Units.1870lb # [ 1.738298] systemd[1]: Starting Create List of Static Device Nodes...1871worker2 # [ 1.716634] 9pnet: Installing 9P2000 support1872client # [ 1.747283] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1873lb # [ 1.742310] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1874client # [ 1.765851] netfs: FS-Cache loaded1875lb # [ 1.749078] systemd[1]: Starting Load Kernel Module configfs...1876client # [ 1.770969] 9pnet: Installing 9P2000 support1877worker1 # [ 1.769251] systemd[1]: Starting Journal Service...1878worker2 # [ 1.730584] systemd[1]: Starting Coldplug All udev Devices...1879lb # [ 1.763637] systemd[1]: Starting Journal Service...1880worker1 # [ 1.783253] systemd[1]: Starting Load Kernel Modules...1881client # [ 1.786295] systemd[1]: Starting Coldplug All udev Devices...1882worker2 # [ 1.744869] systemd[1]: Finished Create List of Static Device Nodes.1883worker2 # [ 1.749422] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1884lb # [ 1.777537] systemd[1]: Starting Load Kernel Modules...1885worker2 # [ 1.757944] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1886client # [ 1.802336] systemd[1]: Finished Create List of Static Device Nodes.1887worker1 # [ 1.789257] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1888worker2 # [ 1.763524] systemd-journald[67]: Collecting audit messages is disabled.1889worker2 # [ 1.765939] systemd[1]: modprobe@configfs.service: Deactivated successfully.1890client # [ 1.807870] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1891worker1 # [ 1.806639] netfs: FS-Cache loaded1892lb # [ 1.782531] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1893worker2 # [ 1.772981] systemd[1]: Finished Load Kernel Module configfs.1894worker1 # [ 1.813431] 9pnet: Installing 9P2000 support1895lb # [ 1.800351] netfs: FS-Cache loaded1896client # [ 1.817626] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1897worker2 # [ 1.777132] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1898worker2 # [ 1.779830] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1899lb # [ 1.805165] 9pnet: Installing 9P2000 support1900client # [ 1.823646] systemd[1]: modprobe@configfs.service: Deactivated successfully.1901client # [ 1.826249] systemd-journald[67]: Collecting audit messages is disabled.1902worker2 # [ 1.787381] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1903client # [ 1.832552] systemd[1]: Finished Load Kernel Module configfs.1904worker2 # [ 1.791572] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1905worker1 # [ 1.832459] systemd[1]: Starting Coldplug All udev Devices...1906lb # [ 1.817571] systemd[1]: Starting Coldplug All udev Devices...1907client # [ 1.836688] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1908client # [ 1.839321] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1909worker1 # [ 1.843814] systemd-journald[67]: Collecting audit messages is disabled.1910client # [ 1.849386] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1911worker1 # [ 1.848218] systemd[1]: Finished Create List of Static Device Nodes.1912lb # [ 1.833933] systemd[1]: Finished Create List of Static Device Nodes.1913client # [ 1.851350] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1914worker1 # [ 1.853971] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1915lb # [ 1.840119] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1916worker2 # [ 1.819079] systemd[1]: Finished Load Kernel Modules.1917worker1 # [ 1.861551] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1918lb # [ 1.848509] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1919worker1 # [ 1.867539] systemd[1]: modprobe@configfs.service: Deactivated successfully.1920worker2 # [ 1.828703] systemd[1]: Starting Apply Kernel Variables...1921lb # [ 1.853993] systemd[1]: modprobe@configfs.service: Deactivated successfully.1922lb # [ 1.855563] systemd-journald[67]: Collecting audit messages is disabled.1923worker1 # [ 1.875550] systemd[1]: Finished Load Kernel Module configfs.1924lb # [ 1.862845] systemd[1]: Finished Load Kernel Module configfs.1925worker2 # [ 1.840071] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1926worker1 # [ 1.879657] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1927client # [ 1.882770] systemd[1]: Finished Load Kernel Modules.1928worker1 # [ 1.882161] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1929lb # [ 1.867034] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1930lb # [ 1.871727] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1931worker2 # [ 1.850063] systemd[1]: Starting Create Static Device Nodes in /dev...1932worker1 # [ 1.891330] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1933client # [ 1.894563] systemd[1]: Starting Apply Kernel Variables...1934worker1 # [ 1.893222] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1935lb # [ 1.879756] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1936lb # [ 1.882632] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1937client # [ 1.906321] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1938worker2 # [ 1.869553] systemd[1]: Finished Apply Kernel Variables.1939client # [ 1.918424] systemd[1]: Starting Create Static Device Nodes in /dev...1940worker2 # [ 1.883205] systemd[1]: Started Journal Service.1941worker1 # [ 1.923788] systemd[1]: Finished Load Kernel Modules.1942worker2 # [ 1.750220] systemd-modules-load[68]: Inserted module 'dm_mod'1943worker2 # [ 1.753460] systemd-modules-load[68]: Inserted module 'virtio_balloon'1944worker2 # [ 1.754583] systemd-modules-load[68]: Inserted module 'virtio_gpu'1945worker2 # [ 1.756265] systemd[1]: Finished Create Static Device Nodes in /dev.1946worker2 # [ 1.758346] systemd[1]: Reached target Preparation for Local File Systems.1947worker1 # [ 1.933381] systemd[1]: Starting Apply Kernel Variables...1948lb # [ 1.918578] systemd[1]: Finished Load Kernel Modules.1949worker2 # [ 1.760345] systemd[1]: Reached target Local File Systems.1950worker2 # [ 1.762258] systemd[1]: Starting Create System Files and Directories...1951client # [ 1.940560] systemd[1]: Finished Apply Kernel Variables.1952lb # [ 1.927691] systemd[1]: Starting Apply Kernel Variables...1953worker2 # [ 1.769053] systemd[1]: Starting Rule-based Manager for Device Events and Files...1954client # [ 1.950746] systemd[1]: Started Journal Service.1955client # [ 1.816771] systemd-modules-load[68]: Inserted module 'dm_mod'1956worker1 # [ 1.951224] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1957client # [ 1.818958] systemd-modules-load[68]: Inserted module 'virtio_balloon'1958client # [ 1.820317] systemd-modules-load[68]: Inserted module 'virtio_gpu'1959lb # [ 1.940535] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1960worker1 # [ 1.960470] systemd[1]: Starting Create Static Device Nodes in /dev...1961lb # [ 1.949064] systemd[1]: Starting Create Static Device Nodes in /dev...1962client # [ 1.831991] systemd[1]: Finished Create Static Device Nodes in /dev.1963client # [ 1.834690] systemd[1]: Reached target Preparation for Local File Systems.1964client # [ 1.837084] systemd[1]: Reached target Local File Systems.1965worker1 # [ 1.836256] systemd-modules-load[68]: Inserted module 'dm_mod'1966client # [ 1.840064] systemd[1]: Starting Create System Files and Directories...1967worker1 # [ 1.839262] systemd-modules-load[68]: Inserted module 'virtio_balloon'1968worker1 # [ 1.975430] systemd[1]: Started Journal Service.1969worker1 # [ 1.842324] systemd-modules-load[68]: Inserted module 'virtio_gpu'1970worker2 # [ 1.801460] systemd[1]: Finished Create System Files and Directories.1971client # [ 1.845077] systemd[1]: Starting Rule-based Manager for Device Events and Files...1972worker1 # [ 1.849099] systemd[1]: Finished Apply Kernel Variables.1973lb # [ 1.972568] systemd[1]: Finished Apply Kernel Variables.1974lb # [ 1.979247] systemd[1]: Started Journal Service.1975lb # [ 1.843841] systemd-modules-load[68]: Inserted module 'dm_mod'1976worker2 # [ 1.823613] systemd-udevd[76]: Using default interface naming scheme 'v261'.1977lb # [ 1.847436] systemd-modules-load[68]: Inserted module 'virtio_balloon'1978worker1 # [ 1.866208] systemd[1]: Finished Create Static Device Nodes in /dev.1979worker1 # [ 1.867607] systemd[1]: Reached target Preparation for Local File Systems.1980lb # [ 1.851579] systemd-modules-load[68]: Inserted module 'virtio_gpu'1981worker1 # [ 1.869902] systemd[1]: Reached target Local File Systems.1982client # [ 1.873212] systemd[1]: Finished Create System Files and Directories.1983worker1 # [ 1.873064] systemd[1]: Starting Create System Files and Directories...1984lb # [ 1.860253] systemd[1]: Finished Create Static Device Nodes in /dev.1985worker1 # [ 1.878073] systemd[1]: Starting Rule-based Manager for Device Events and Files...1986lb # [ 1.861976] systemd[1]: Reached target Preparation for Local File Systems.1987lb # [ 1.865110] systemd[1]: Reached target Local File Systems.1988lb # [ 1.868072] systemd[1]: Starting Create System Files and Directories...1989worker2 # [ 1.848623] systemd[1]: Started Rule-based Manager for Device Events and Files.1990lb # [ 1.873081] systemd[1]: Starting Rule-based Manager for Device Events and Files...1991client # [ 1.897807] systemd-udevd[83]: Using default interface naming scheme 'v261'.1992worker1 # [ 1.909072] systemd[1]: Finished Create System Files and Directories.1993lb # [ 1.900708] systemd[1]: Finished Create System Files and Directories.1994client # [ 1.922082] systemd[1]: Started Rule-based Manager for Device Events and Files.1995worker1 # [ 1.931394] systemd-udevd[83]: Using default interface naming scheme 'v261'.1996worker2 # [ 1.901090] systemd[1]: Finished Coldplug All udev Devices.1997worker2 # [ 1.902228] systemd[1]: Reached target System Initialization.1998worker2 # [ 1.902991] systemd[1]: Reached target Basic System.1999lb # [ 1.927275] systemd-udevd[83]: Using default interface naming scheme 'v261'.2000worker1 # [ 1.957075] systemd[1]: Started Rule-based Manager for Device Events and Files.
loading 2,118 hidden lines…
4119worker2 # [ 17.394753] systemd[1]: Finished scheduler-token.service.4120worker2 # [ 17.395879] systemd[1]: scheduler-token.service: Consumed 32ms CPU time over 104ms wall clock time, 3.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4121worker2 # [ 17.401435] systemd[1]: Starting Nix worker-protocol over gRPC...4122worker1 # [ 17.554908] nix-grpc-daemon[854]: ts=2026-09-20T12:46:10Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4123worker1 # [ 17.564474] nix-grpc-daemon[854]: ts=2026-09-20T12:46:10Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4124worker1 # [ 17.569490] nix-grpc-daemon[854]: ts=2026-09-20T12:46:10Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14125worker2 # [ 17.561476] nix-grpc-daemon[787]: ts=2026-09-20T12:46:11Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4126worker2 # [ 17.569475] nix-grpc-daemon[787]: ts=2026-09-20T12:46:11Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4127worker2 # [ 17.573751] nix-grpc-daemon[787]: ts=2026-09-20T12:46:11Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14128worker1 # [ 17.615732] nix-grpc-daemon[854]: ts=2026-09-20T12:46:11Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4129worker1 # [ 17.618965] systemd[1]: Started Nix worker-protocol over gRPC.4130worker2 # [ 17.612667] nix-grpc-daemon[787]: ts=2026-09-20T12:46:11Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4131worker2 # [ 17.615423] systemd[1]: Started Nix worker-protocol over gRPC.4132worker1 # [ 17.666348] nix-grpc-daemon[854]: ts=2026-09-20T12:46:11Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4133lb # [ 17.649594] envoy[973]: [2026-09-20 12:46:11.447][973][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization4134lb # [ 17.652993] envoy[973]: [2026-09-20 12:46:11.447][973][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized4135lb # [ 17.656582] envoy[973]: [2026-09-20 12:46:11.447][973][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager4136lb # [ 17.659641] envoy[973]: [2026-09-20 12:46:11.457][973][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers4137worker2 # [ 17.668845] nix-grpc-daemon[787]: ts=2026-09-20T12:46:11Z level=info event=scheduler_connected addr=lb:500514138worker2 # [ 17.679645] nix-grpc-daemon[787]: ts=2026-09-20T12:46:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4139worker1 # [ 17.867373] nix-grpc-daemon[854]: ts=2026-09-20T12:46:11Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4140worker2 # [ 17.882395] nix-grpc-daemon[787]: ts=2026-09-20T12:46:11Z level=info event=scheduler_connected addr=lb:500514141worker2 # [ 17.886251] nix-grpc-daemon[787]: ts=2026-09-20T12:46:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4142worker2 # [ 18.087264] nix-grpc-daemon[787]: ts=2026-09-20T12:46:12Z level=info event=scheduler_connected addr=lb:500514143worker2 # [ 18.090989] nix-grpc-daemon[787]: ts=2026-09-20T12:46:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4144worker1 # [ 18.268057] nix-grpc-daemon[854]: ts=2026-09-20T12:46:11Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4145worker2 # [ 18.292488] nix-grpc-daemon[787]: ts=2026-09-20T12:46:12Z level=info event=scheduler_connected addr=lb:500514146worker2 # [ 18.296408] nix-grpc-daemon[787]: ts=2026-09-20T12:46:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4147worker2 # [ 18.497478] nix-grpc-daemon[787]: ts=2026-09-20T12:46:12Z level=info event=scheduler_connected addr=lb:500514148worker2 # [ 18.501259] nix-grpc-daemon[787]: ts=2026-09-20T12:46:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4149worker2 # [ 18.702544] nix-grpc-daemon[787]: ts=2026-09-20T12:46:12Z level=info event=scheduler_connected addr=lb:500514150worker2 # [ 18.705712] nix-grpc-daemon[787]: ts=2026-09-20T12:46:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4151worker2 # [ 18.906392] nix-grpc-daemon[787]: ts=2026-09-20T12:46:12Z level=info event=scheduler_connected addr=lb:500514152worker2 # [ 18.908192] nix-grpc-daemon[787]: ts=2026-09-20T12:46:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4153worker1 # [ 19.072363] nix-grpc-daemon[854]: ts=2026-09-20T12:46:12Z level=info event=scheduler_connected addr=lb:500514154worker1 # [ 19.076380] nix-grpc-daemon[854]: ts=2026-09-20T12:46:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4155worker2 # [ 19.109342] nix-grpc-daemon[787]: ts=2026-09-20T12:46:13Z level=info event=scheduler_connected addr=lb:500514156worker2 # [ 19.113315] nix-grpc-daemon[787]: ts=2026-09-20T12:46:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4157worker1 # [ 19.277367] nix-grpc-daemon[854]: ts=2026-09-20T12:46:12Z level=info event=scheduler_connected addr=lb:500514158worker1 # [ 19.279259] nix-grpc-daemon[854]: ts=2026-09-20T12:46:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4159worker2 # [ 19.314270] nix-grpc-daemon[787]: ts=2026-09-20T12:46:13Z level=info event=scheduler_connected addr=lb:500514160worker2 # [ 19.318202] nix-grpc-daemon[787]: ts=2026-09-20T12:46:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4161worker1 # [ 19.480821] nix-grpc-daemon[854]: ts=2026-09-20T12:46:12Z level=info event=scheduler_connected addr=lb:500514162worker1 # [ 19.485527] nix-grpc-daemon[854]: ts=2026-09-20T12:46:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4163worker2 # [ 19.519386] nix-grpc-daemon[787]: ts=2026-09-20T12:46:13Z level=info event=scheduler_connected addr=lb:500514164worker2 # [ 19.523301] nix-grpc-daemon[787]: ts=2026-09-20T12:46:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4165worker1 # [ 19.685760] nix-grpc-daemon[854]: ts=2026-09-20T12:46:13Z level=info event=scheduler_connected addr=lb:500514166worker1 # [ 19.687890] nix-grpc-daemon[854]: ts=2026-09-20T12:46:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4167worker2 # [ 19.725709] nix-grpc-daemon[787]: ts=2026-09-20T12:46:13Z level=info event=scheduler_connected addr=lb:500514168worker2 # [ 19.728935] nix-grpc-daemon[787]: ts=2026-09-20T12:46:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4169worker1 # [ 19.888545] nix-grpc-daemon[854]: ts=2026-09-20T12:46:13Z level=info event=scheduler_connected addr=lb:500514170worker1 # [ 19.890689] nix-grpc-daemon[854]: ts=2026-09-20T12:46:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4171worker2 # [ 19.926442] nix-grpc-daemon[787]: ts=2026-09-20T12:46:13Z level=info event=scheduler_connected addr=lb:500514172worker2 # [ 19.929810] nix-grpc-daemon[787]: ts=2026-09-20T12:46:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4173worker1 # [ 20.092099] nix-grpc-daemon[854]: ts=2026-09-20T12:46:13Z level=info event=scheduler_connected addr=lb:500514174worker1 # [ 20.094113] nix-grpc-daemon[854]: ts=2026-09-20T12:46:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4175worker2 # [ 20.130762] nix-grpc-daemon[787]: ts=2026-09-20T12:46:14Z level=info event=scheduler_connected addr=lb:500514176worker2 # [ 20.134773] nix-grpc-daemon[787]: ts=2026-09-20T12:46:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4177worker1 # [ 20.295188] nix-grpc-daemon[854]: ts=2026-09-20T12:46:13Z level=info event=scheduler_connected addr=lb:500514178worker1 # [ 20.297404] nix-grpc-daemon[854]: ts=2026-09-20T12:46:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4179worker2 # [ 20.335509] nix-grpc-daemon[787]: ts=2026-09-20T12:46:14Z level=info event=scheduler_connected addr=lb:500514180worker2 # [ 20.337347] nix-grpc-daemon[787]: ts=2026-09-20T12:46:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4181worker1 # [ 20.498563] nix-grpc-daemon[854]: ts=2026-09-20T12:46:13Z level=info event=scheduler_connected addr=lb:500514182worker1 # [ 20.502785] nix-grpc-daemon[854]: ts=2026-09-20T12:46:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4183worker2 # [ 20.538045] nix-grpc-daemon[787]: ts=2026-09-20T12:46:14Z level=info event=scheduler_connected addr=lb:500514184worker2 # [ 20.539828] nix-grpc-daemon[787]: ts=2026-09-20T12:46:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4185worker1 # [ 20.703697] nix-grpc-daemon[854]: ts=2026-09-20T12:46:14Z level=info event=scheduler_connected addr=lb:500514186worker1 # [ 20.707561] nix-grpc-daemon[854]: ts=2026-09-20T12:46:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4187worker2 # [ 20.740823] nix-grpc-daemon[787]: ts=2026-09-20T12:46:14Z level=info event=scheduler_connected addr=lb:500514188worker2 # [ 20.742525] nix-grpc-daemon[787]: ts=2026-09-20T12:46:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4189worker1 # [ 20.908231] nix-grpc-daemon[854]: ts=2026-09-20T12:46:14Z level=info event=scheduler_connected addr=lb:500514190worker1 # [ 20.910035] nix-grpc-daemon[854]: ts=2026-09-20T12:46:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4191worker2 # [ 20.943878] nix-grpc-daemon[787]: ts=2026-09-20T12:46:14Z level=info event=scheduler_connected addr=lb:500514192worker2 # [ 20.947713] nix-grpc-daemon[787]: ts=2026-09-20T12:46:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4193worker1 # [ 21.112283] nix-grpc-daemon[854]: ts=2026-09-20T12:46:14Z level=info event=scheduler_connected addr=lb:500514194worker1 # [ 21.115836] nix-grpc-daemon[854]: ts=2026-09-20T12:46:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4195worker2 # [ 21.148295] nix-grpc-daemon[787]: ts=2026-09-20T12:46:15Z level=info event=scheduler_connected addr=lb:500514196worker2 # [ 21.150105] nix-grpc-daemon[787]: ts=2026-09-20T12:46:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4197worker1 # [ 21.316813] nix-grpc-daemon[854]: ts=2026-09-20T12:46:14Z level=info event=scheduler_connected addr=lb:500514198worker1 # [ 21.320795] nix-grpc-daemon[854]: ts=2026-09-20T12:46:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4199worker2 # [ 21.351535] nix-grpc-daemon[787]: ts=2026-09-20T12:46:15Z level=info event=scheduler_connected addr=lb:500514200worker2 # [ 21.355500] nix-grpc-daemon[787]: ts=2026-09-20T12:46:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4201worker1 # [ 21.521442] nix-grpc-daemon[854]: ts=2026-09-20T12:46:14Z level=info event=scheduler_connected addr=lb:500514202worker1 # [ 21.523356] nix-grpc-daemon[854]: ts=2026-09-20T12:46:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4203worker2 # [ 21.556649] nix-grpc-daemon[787]: ts=2026-09-20T12:46:15Z level=info event=scheduler_connected addr=lb:500514204worker2 # [ 21.560544] nix-grpc-daemon[787]: ts=2026-09-20T12:46:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4205worker1 # [ 21.724457] nix-grpc-daemon[854]: ts=2026-09-20T12:46:15Z level=info event=scheduler_connected addr=lb:500514206worker1 # [ 21.728374] nix-grpc-daemon[854]: ts=2026-09-20T12:46:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4207worker2 # [ 21.762753] nix-grpc-daemon[787]: ts=2026-09-20T12:46:15Z level=info event=scheduler_connected addr=lb:500514208worker2 # [ 21.766393] nix-grpc-daemon[787]: ts=2026-09-20T12:46:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4209worker1 # [ 21.929519] nix-grpc-daemon[854]: ts=2026-09-20T12:46:15Z level=info event=scheduler_connected addr=lb:500514210worker1 # [ 21.933981] nix-grpc-daemon[854]: ts=2026-09-20T12:46:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4211worker2 # [ 21.967526] nix-grpc-daemon[787]: ts=2026-09-20T12:46:15Z level=info event=scheduler_connected addr=lb:500514212worker2 # [ 21.971423] nix-grpc-daemon[787]: ts=2026-09-20T12:46:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4213worker1 # [ 22.135723] nix-grpc-daemon[854]: ts=2026-09-20T12:46:15Z level=info event=scheduler_connected addr=lb:500514214worker1 # [ 22.139569] nix-grpc-daemon[854]: ts=2026-09-20T12:46:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4215worker2 # [ 22.172534] nix-grpc-daemon[787]: ts=2026-09-20T12:46:16Z level=info event=scheduler_connected addr=lb:500514216worker2 # [ 22.176417] nix-grpc-daemon[787]: ts=2026-09-20T12:46:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4217worker1 # [ 22.340511] nix-grpc-daemon[854]: ts=2026-09-20T12:46:15Z level=info event=scheduler_connected addr=lb:500514218worker1 # [ 22.344708] nix-grpc-daemon[854]: ts=2026-09-20T12:46:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4219worker2 # [ 22.377505] nix-grpc-daemon[787]: ts=2026-09-20T12:46:16Z level=info event=scheduler_connected addr=lb:500514220worker2 # [ 22.381435] nix-grpc-daemon[787]: ts=2026-09-20T12:46:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4221worker1 # [ 22.545534] nix-grpc-daemon[854]: ts=2026-09-20T12:46:15Z level=info event=scheduler_connected addr=lb:500514222worker1 # [ 22.548958] nix-grpc-daemon[854]: ts=2026-09-20T12:46:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4223worker2 # [ 22.581941] nix-grpc-daemon[787]: ts=2026-09-20T12:46:16Z level=info event=scheduler_connected addr=lb:500514224worker2 # [ 22.583774] nix-grpc-daemon[787]: ts=2026-09-20T12:46:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4225worker1 # [ 22.749983] nix-grpc-daemon[854]: ts=2026-09-20T12:46:16Z level=info event=scheduler_connected addr=lb:500514226worker1 # [ 22.754938] nix-grpc-daemon[854]: ts=2026-09-20T12:46:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4227worker2 # [ 22.784967] nix-grpc-daemon[787]: ts=2026-09-20T12:46:16Z level=info event=scheduler_connected addr=lb:500514228worker2 # [ 22.786735] nix-grpc-daemon[787]: ts=2026-09-20T12:46:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4229worker1 # [ 22.955814] nix-grpc-daemon[854]: ts=2026-09-20T12:46:16Z level=info event=scheduler_connected addr=lb:500514230worker1 # [ 22.959864] nix-grpc-daemon[854]: ts=2026-09-20T12:46:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4231worker2 # [ 22.988169] nix-grpc-daemon[787]: ts=2026-09-20T12:46:16Z level=info event=scheduler_connected addr=lb:500514232worker2 # [ 22.992153] nix-grpc-daemon[787]: ts=2026-09-20T12:46:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4233worker1 # [ 23.162356] nix-grpc-daemon[854]: ts=2026-09-20T12:46:16Z level=info event=scheduler_connected addr=lb:500514234worker1 # [ 23.165601] nix-grpc-daemon[854]: ts=2026-09-20T12:46:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4235worker2 # [ 23.193554] nix-grpc-daemon[787]: ts=2026-09-20T12:46:17Z level=info event=scheduler_connected addr=lb:500514236worker2 # [ 23.197438] nix-grpc-daemon[787]: ts=2026-09-20T12:46:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4237worker1 # [ 23.363330] nix-grpc-daemon[854]: ts=2026-09-20T12:46:16Z level=info event=scheduler_connected addr=lb:500514238worker1 # [ 23.368299] nix-grpc-daemon[854]: ts=2026-09-20T12:46:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4239worker2 # [ 23.398637] nix-grpc-daemon[787]: ts=2026-09-20T12:46:17Z level=info event=scheduler_connected addr=lb:500514240worker2 # [ 23.402821] nix-grpc-daemon[787]: ts=2026-09-20T12:46:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4241worker1 # [ 23.567967] nix-grpc-daemon[854]: ts=2026-09-20T12:46:16Z level=info event=scheduler_connected addr=lb:500514242worker1 # [ 23.572601] nix-grpc-daemon[854]: ts=2026-09-20T12:46:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4243worker2 # [ 23.603383] nix-grpc-daemon[787]: ts=2026-09-20T12:46:17Z level=info event=scheduler_connected addr=lb:500514244worker2 # [ 23.605451] nix-grpc-daemon[787]: ts=2026-09-20T12:46:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4245worker1 # [ 23.773451] nix-grpc-daemon[854]: ts=2026-09-20T12:46:17Z level=info event=scheduler_connected addr=lb:500514246worker1 # [ 23.777491] nix-grpc-daemon[854]: ts=2026-09-20T12:46:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4247worker2 # [ 23.806523] nix-grpc-daemon[787]: ts=2026-09-20T12:46:17Z level=info event=scheduler_connected addr=lb:500514248worker2 # [ 23.808271] nix-grpc-daemon[787]: ts=2026-09-20T12:46:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4249worker1 # [ 23.978568] nix-grpc-daemon[854]: ts=2026-09-20T12:46:17Z level=info event=scheduler_connected addr=lb:500514250worker1 # [ 23.982536] nix-grpc-daemon[854]: ts=2026-09-20T12:46:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4251worker2 # [ 24.009209] nix-grpc-daemon[787]: ts=2026-09-20T12:46:17Z level=info event=scheduler_connected addr=lb:500514252worker2 # [ 24.010993] nix-grpc-daemon[787]: ts=2026-09-20T12:46:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4253worker1 # [ 24.185091] nix-grpc-daemon[854]: ts=2026-09-20T12:46:17Z level=info event=scheduler_connected addr=lb:500514254worker1 # [ 24.188355] nix-grpc-daemon[854]: ts=2026-09-20T12:46:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4255worker2 # [ 24.211675] nix-grpc-daemon[787]: ts=2026-09-20T12:46:18Z level=info event=scheduler_connected addr=lb:500514256worker2 # [ 24.213452] nix-grpc-daemon[787]: ts=2026-09-20T12:46:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4257worker1 # [ 24.385997] nix-grpc-daemon[854]: ts=2026-09-20T12:46:17Z level=info event=scheduler_connected addr=lb:500514258worker1 # [ 24.389041] nix-grpc-daemon[854]: ts=2026-09-20T12:46:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4259worker2 # [ 24.414639] nix-grpc-daemon[787]: ts=2026-09-20T12:46:18Z level=info event=scheduler_connected addr=lb:500514260worker2 # [ 24.418555] nix-grpc-daemon[787]: ts=2026-09-20T12:46:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4261worker1 # [ 24.589952] nix-grpc-daemon[854]: ts=2026-09-20T12:46:18Z level=info event=scheduler_connected addr=lb:500514262worker1 # [ 24.593833] nix-grpc-daemon[854]: ts=2026-09-20T12:46:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4263worker2 # [ 24.620144] nix-grpc-daemon[787]: ts=2026-09-20T12:46:18Z level=info event=scheduler_connected addr=lb:500514264worker2 # [ 24.624110] nix-grpc-daemon[787]: ts=2026-09-20T12:46:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4265worker1 # [ 24.794564] nix-grpc-daemon[854]: ts=2026-09-20T12:46:18Z level=info event=scheduler_connected addr=lb:500514266worker1 # [ 24.796624] nix-grpc-daemon[854]: ts=2026-09-20T12:46:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4267worker2 # [ 24.826576] nix-grpc-daemon[787]: ts=2026-09-20T12:46:18Z level=info event=scheduler_connected addr=lb:500514268worker2 # [ 24.829828] nix-grpc-daemon[787]: ts=2026-09-20T12:46:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4269worker1 # [ 24.997195] nix-grpc-daemon[854]: ts=2026-09-20T12:46:18Z level=info event=scheduler_connected addr=lb:500514270worker1 # [ 24.998993] nix-grpc-daemon[854]: ts=2026-09-20T12:46:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4271worker2 # [ 25.027984] nix-grpc-daemon[787]: ts=2026-09-20T12:46:19Z level=info event=scheduler_connected addr=lb:500514272worker2 # [ 25.031828] nix-grpc-daemon[787]: ts=2026-09-20T12:46:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4273worker1 # [ 25.200545] nix-grpc-daemon[854]: ts=2026-09-20T12:46:18Z level=info event=scheduler_connected addr=lb:500514274worker1 # [ 25.201892] nix-grpc-daemon[854]: ts=2026-09-20T12:46:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4275worker2 # [ 25.233164] nix-grpc-daemon[787]: ts=2026-09-20T12:46:19Z level=info event=scheduler_connected addr=lb:500514276worker2 # [ 25.237620] nix-grpc-daemon[787]: ts=2026-09-20T12:46:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4277worker1 # [ 25.401679] nix-grpc-daemon[854]: ts=2026-09-20T12:46:18Z level=info event=scheduler_connected addr=lb:500514278worker1 # [ 25.405693] nix-grpc-daemon[854]: ts=2026-09-20T12:46:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4279worker2 # [ 25.438612] nix-grpc-daemon[787]: ts=2026-09-20T12:46:19Z level=info event=scheduler_connected addr=lb:500514280worker2 # [ 25.442855] nix-grpc-daemon[787]: ts=2026-09-20T12:46:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4281worker1 # [ 25.606779] nix-grpc-daemon[854]: ts=2026-09-20T12:46:19Z level=info event=scheduler_connected addr=lb:500514282worker1 # [ 25.610704] nix-grpc-daemon[854]: ts=2026-09-20T12:46:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4283worker2 # [ 25.644171] nix-grpc-daemon[787]: ts=2026-09-20T12:46:19Z level=info event=scheduler_connected addr=lb:500514284worker2 # [ 25.648254] nix-grpc-daemon[787]: ts=2026-09-20T12:46:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4285worker1 # [ 25.811685] nix-grpc-daemon[854]: ts=2026-09-20T12:46:19Z level=info event=scheduler_connected addr=lb:500514286worker1 # [ 25.815616] nix-grpc-daemon[854]: ts=2026-09-20T12:46:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4287worker2 # [ 25.849328] nix-grpc-daemon[787]: ts=2026-09-20T12:46:19Z level=info event=scheduler_connected addr=lb:500514288worker2 # [ 25.851331] nix-grpc-daemon[787]: ts=2026-09-20T12:46:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4289worker1 # [ 26.016611] nix-grpc-daemon[854]: ts=2026-09-20T12:46:19Z level=info event=scheduler_connected addr=lb:500514290worker1 # [ 26.020743] nix-grpc-daemon[854]: ts=2026-09-20T12:46:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4291worker2 # [ 26.052277] nix-grpc-daemon[787]: ts=2026-09-20T12:46:20Z level=info event=scheduler_connected addr=lb:500514292worker2 # [ 26.054146] nix-grpc-daemon[787]: ts=2026-09-20T12:46:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4293worker1 # [ 26.223197] nix-grpc-daemon[854]: ts=2026-09-20T12:46:19Z level=info event=scheduler_connected addr=lb:500514294worker1 # [ 26.226428] nix-grpc-daemon[854]: ts=2026-09-20T12:46:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4295worker2 # [ 26.255686] nix-grpc-daemon[787]: ts=2026-09-20T12:46:20Z level=info event=scheduler_connected addr=lb:500514296worker2 # [ 26.259970] nix-grpc-daemon[787]: ts=2026-09-20T12:46:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4297worker1 # [ 26.424229] nix-grpc-daemon[854]: ts=2026-09-20T12:46:19Z level=info event=scheduler_connected addr=lb:500514298worker1 # [ 26.428208] nix-grpc-daemon[854]: ts=2026-09-20T12:46:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4299worker2 # [ 26.460939] nix-grpc-daemon[787]: ts=2026-09-20T12:46:20Z level=info event=scheduler_connected addr=lb:500514300worker2 # [ 26.464854] nix-grpc-daemon[787]: ts=2026-09-20T12:46:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4301worker1 # [ 26.628551] nix-grpc-daemon[854]: ts=2026-09-20T12:46:20Z level=info event=scheduler_connected addr=lb:500514302worker1 # [ 26.631447] nix-grpc-daemon[854]: ts=2026-09-20T12:46:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4303worker2 # [ 26.665764] nix-grpc-daemon[787]: ts=2026-09-20T12:46:20Z level=info event=scheduler_connected addr=lb:500514304worker2 # [ 26.669648] nix-grpc-daemon[787]: ts=2026-09-20T12:46:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4305worker1 # [ 26.832592] nix-grpc-daemon[854]: ts=2026-09-20T12:46:20Z level=info event=scheduler_connected addr=lb:500514306worker1 # [ 26.836594] nix-grpc-daemon[854]: ts=2026-09-20T12:46:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4307worker2 # [ 26.871494] nix-grpc-daemon[787]: ts=2026-09-20T12:46:20Z level=info event=scheduler_connected addr=lb:500514308worker2 # [ 26.875278] nix-grpc-daemon[787]: ts=2026-09-20T12:46:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4309worker1 # [ 27.037615] nix-grpc-daemon[854]: ts=2026-09-20T12:46:20Z level=info event=scheduler_connected addr=lb:500514310worker1 # [ 27.041669] nix-grpc-daemon[854]: ts=2026-09-20T12:46:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4311worker2 # [ 27.076338] nix-grpc-daemon[787]: ts=2026-09-20T12:46:21Z level=info event=scheduler_connected addr=lb:500514312worker2 # [ 27.080198] nix-grpc-daemon[787]: ts=2026-09-20T12:46:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4313worker1 # [ 27.244117] nix-grpc-daemon[854]: ts=2026-09-20T12:46:20Z level=info event=scheduler_connected addr=lb:500514314worker1 # [ 27.247333] nix-grpc-daemon[854]: ts=2026-09-20T12:46:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4315worker2 # [ 27.281360] nix-grpc-daemon[787]: ts=2026-09-20T12:46:21Z level=info event=scheduler_connected addr=lb:500514316worker2 # [ 27.285318] nix-grpc-daemon[787]: ts=2026-09-20T12:46:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4317worker1 # [ 27.445360] nix-grpc-daemon[854]: ts=2026-09-20T12:46:20Z level=info event=scheduler_connected addr=lb:500514318worker1 # [ 27.449473] nix-grpc-daemon[854]: ts=2026-09-20T12:46:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4319worker2 # [ 27.485830] nix-grpc-daemon[787]: ts=2026-09-20T12:46:21Z level=info event=scheduler_connected addr=lb:500514320worker2 # [ 27.487956] nix-grpc-daemon[787]: ts=2026-09-20T12:46:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4321worker1 # [ 27.650557] nix-grpc-daemon[854]: ts=2026-09-20T12:46:21Z level=info event=scheduler_connected addr=lb:500514322worker1 # [ 27.654480] nix-grpc-daemon[854]: ts=2026-09-20T12:46:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4323worker2 # [ 27.688357] nix-grpc-daemon[787]: ts=2026-09-20T12:46:21Z level=info event=scheduler_connected addr=lb:500514324worker2 # [ 27.690121] nix-grpc-daemon[787]: ts=2026-09-20T12:46:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4325worker1 # [ 27.855374] nix-grpc-daemon[854]: ts=2026-09-20T12:46:21Z level=info event=scheduler_connected addr=lb:500514326worker1 # [ 27.859593] nix-grpc-daemon[854]: ts=2026-09-20T12:46:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4327worker2 # [ 27.892391] nix-grpc-daemon[787]: ts=2026-09-20T12:46:21Z level=info event=scheduler_connected addr=lb:500514328worker2 # [ 27.895995] nix-grpc-daemon[787]: ts=2026-09-20T12:46:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4329worker1 # [ 28.060659] nix-grpc-daemon[854]: ts=2026-09-20T12:46:21Z level=info event=scheduler_connected addr=lb:500514330worker1 # [ 28.064841] nix-grpc-daemon[854]: ts=2026-09-20T12:46:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4331worker2 # [ 28.097219] nix-grpc-daemon[787]: ts=2026-09-20T12:46:22Z level=info event=scheduler_connected addr=lb:500514332worker2 # [ 28.101125] nix-grpc-daemon[787]: ts=2026-09-20T12:46:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4333worker1 # [ 28.267379] nix-grpc-daemon[854]: ts=2026-09-20T12:46:21Z level=info event=scheduler_connected addr=lb:500514334worker1 # [ 28.270588] nix-grpc-daemon[854]: ts=2026-09-20T12:46:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4335worker2 # [ 28.302167] nix-grpc-daemon[787]: ts=2026-09-20T12:46:22Z level=info event=scheduler_connected addr=lb:500514336worker2 # [ 28.306094] nix-grpc-daemon[787]: ts=2026-09-20T12:46:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4337worker1 # [ 28.468239] nix-grpc-daemon[854]: ts=2026-09-20T12:46:21Z level=info event=scheduler_connected addr=lb:500514338worker1 # [ 28.472266] nix-grpc-daemon[854]: ts=2026-09-20T12:46:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4339worker2 # [ 28.507360] nix-grpc-daemon[787]: ts=2026-09-20T12:46:22Z level=info event=scheduler_connected addr=lb:500514340worker2 # [ 28.511163] nix-grpc-daemon[787]: ts=2026-09-20T12:46:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4341worker1 # [ 28.673205] nix-grpc-daemon[854]: ts=2026-09-20T12:46:22Z level=info event=scheduler_connected addr=lb:500514342worker1 # [ 28.677360] nix-grpc-daemon[854]: ts=2026-09-20T12:46:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4343worker2 # [ 28.711919] nix-grpc-daemon[787]: ts=2026-09-20T12:46:22Z level=info event=scheduler_connected addr=lb:500514344worker2 # [ 28.715800] nix-grpc-daemon[787]: ts=2026-09-20T12:46:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4345worker1 # [ 28.878491] nix-grpc-daemon[854]: ts=2026-09-20T12:46:22Z level=info event=scheduler_connected addr=lb:500514346worker1 # [ 28.882532] nix-grpc-daemon[854]: ts=2026-09-20T12:46:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4347worker2 # [ 28.917808] nix-grpc-daemon[787]: ts=2026-09-20T12:46:22Z level=info event=scheduler_connected addr=lb:500514348worker2 # [ 28.921616] nix-grpc-daemon[787]: ts=2026-09-20T12:46:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4349worker1 # [ 29.083553] nix-grpc-daemon[854]: ts=2026-09-20T12:46:22Z level=info event=scheduler_connected addr=lb:500514350worker1 # [ 29.087568] nix-grpc-daemon[854]: ts=2026-09-20T12:46:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4351worker2 # [ 29.122737] nix-grpc-daemon[787]: ts=2026-09-20T12:46:23Z level=info event=scheduler_connected addr=lb:500514352worker2 # [ 29.126703] nix-grpc-daemon[787]: ts=2026-09-20T12:46:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4353worker1 # [ 29.290335] nix-grpc-daemon[854]: ts=2026-09-20T12:46:22Z level=info event=scheduler_connected addr=lb:500514354worker1 # [ 29.293531] nix-grpc-daemon[854]: ts=2026-09-20T12:46:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4355worker2 # [ 29.328076] nix-grpc-daemon[787]: ts=2026-09-20T12:46:23Z level=info event=scheduler_connected addr=lb:500514356worker2 # [ 29.331949] nix-grpc-daemon[787]: ts=2026-09-20T12:46:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4357worker1 # [ 29.490956] nix-grpc-daemon[854]: ts=2026-09-20T12:46:22Z level=info event=scheduler_connected addr=lb:500514358worker1 # [ 29.495069] nix-grpc-daemon[854]: ts=2026-09-20T12:46:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4359worker2 # [ 29.532718] nix-grpc-daemon[787]: ts=2026-09-20T12:46:23Z level=info event=scheduler_connected addr=lb:500514360worker2 # [ 29.535966] nix-grpc-daemon[787]: ts=2026-09-20T12:46:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4361worker1 # [ 29.696092] nix-grpc-daemon[854]: ts=2026-09-20T12:46:23Z level=info event=scheduler_connected addr=lb:500514362worker1 # [ 29.699788] nix-grpc-daemon[854]: ts=2026-09-20T12:46:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4363worker2 # [ 29.736887] nix-grpc-daemon[787]: ts=2026-09-20T12:46:23Z level=info event=scheduler_connected addr=lb:500514364worker2 # [ 29.740799] nix-grpc-daemon[787]: ts=2026-09-20T12:46:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4365worker1 # [ 29.900391] nix-grpc-daemon[854]: ts=2026-09-20T12:46:23Z level=info event=scheduler_connected addr=lb:500514366worker1 # [ 29.904248] nix-grpc-daemon[854]: ts=2026-09-20T12:46:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4367worker2 # [ 29.942878] nix-grpc-daemon[787]: ts=2026-09-20T12:46:23Z level=info event=scheduler_connected addr=lb:500514368worker2 # [ 29.946740] nix-grpc-daemon[787]: ts=2026-09-20T12:46:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4369worker1 # [ 30.105342] nix-grpc-daemon[854]: ts=2026-09-20T12:46:23Z level=info event=scheduler_connected addr=lb:500514370worker1 # [ 30.109510] nix-grpc-daemon[854]: ts=2026-09-20T12:46:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4371worker2 # [ 30.147855] nix-grpc-daemon[787]: ts=2026-09-20T12:46:24Z level=info event=scheduler_connected addr=lb:500514372worker2 # [ 30.151831] nix-grpc-daemon[787]: ts=2026-09-20T12:46:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4373worker1 # [ 30.312085] nix-grpc-daemon[854]: ts=2026-09-20T12:46:23Z level=info event=scheduler_connected addr=lb:500514374worker1 # [ 30.315365] nix-grpc-daemon[854]: ts=2026-09-20T12:46:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4375worker2 # [ 30.352946] nix-grpc-daemon[787]: ts=2026-09-20T12:46:24Z level=info event=scheduler_connected addr=lb:500514376worker2 # [ 30.356928] nix-grpc-daemon[787]: ts=2026-09-20T12:46:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4377worker1 # [ 30.513133] nix-grpc-daemon[854]: ts=2026-09-20T12:46:23Z level=info event=scheduler_connected addr=lb:500514378worker1 # [ 30.517298] nix-grpc-daemon[854]: ts=2026-09-20T12:46:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4379worker2 # [ 30.558148] nix-grpc-daemon[787]: ts=2026-09-20T12:46:24Z level=info event=scheduler_connected addr=lb:500514380worker2 # [ 30.562223] nix-grpc-daemon[787]: ts=2026-09-20T12:46:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4381worker1 # [ 30.718142] nix-grpc-daemon[854]: ts=2026-09-20T12:46:24Z level=info event=scheduler_connected addr=lb:500514382worker1 # [ 30.722218] nix-grpc-daemon[854]: ts=2026-09-20T12:46:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4383lb # [ 30.748722] rustfs-bucket-start[1121]: mb s3://farm4384lb # [ 30.754446] systemd[1]: Finished rustfs-bucket.service.4385lb # [ 30.760871] systemd[1]: Starting niks3 server...4386worker2 # [ 30.763361] nix-grpc-daemon[787]: ts=2026-09-20T12:46:24Z level=info event=scheduler_connected addr=lb:500514387worker2 # [ 30.767396] nix-grpc-daemon[787]: ts=2026-09-20T12:46:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4388worker1 # [ 30.923134] nix-grpc-daemon[854]: ts=2026-09-20T12:46:24Z level=info event=scheduler_connected addr=lb:500514389worker1 # [ 30.926959] nix-grpc-daemon[854]: ts=2026-09-20T12:46:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4390lb # [ 30.975889] postgres[1136]: [1136] ERROR: relation "goose_db_version" does not exist at character 364391lb # [ 30.977064] postgres[1136]: [1136] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4392worker2 # [ 30.969449] nix-grpc-daemon[787]: ts=2026-09-20T12:46:24Z level=info event=scheduler_connected addr=lb:500514393worker2 # [ 30.972934] nix-grpc-daemon[787]: ts=2026-09-20T12:46:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4394lb # [ 31.015915] niks3-server[1131]: 2026/09/20 12:46:24 OK 20241026095416_initial_model.sql (21.05ms)4395lb # [ 31.019865] niks3-server[1131]: 2026/09/20 12:46:24 OK 20251210153512_drop_unused_gin_index.sql (3.97ms)4396lb # [ 31.024950] niks3-server[1131]: 2026/09/20 12:46:24 OK 20251218171726_add_pins.sql (5.07ms)4397lb # [ 31.030441] niks3-server[1131]: 2026/09/20 12:46:24 OK 20260628120000_add_object_size_and_stats.sql (5.42ms)4398lb # [ 31.034740] niks3-server[1131]: 2026/09/20 12:46:24 OK 20260905000000_add_claims.sql (4.29ms)4399lb # [ 31.039246] niks3-server[1131]: 2026/09/20 12:46:24 OK 20260920000000_drop_claims.sql (4.41ms)4400lb # [ 31.040410] niks3-server[1131]: 2026/09/20 12:46:24 goose: successfully migrated database to version: 202609200000004401lb # [ 31.046148] niks3-server[1131]: 2026/09/20 12:46:24 OK 1_commit_pending_closure.sql (6.83ms)4402lb # [ 31.048622] niks3-server[1131]: 2026/09/20 12:46:24 OK 2_object_stats_trigger.sql (2.45ms)4403lb # [ 31.049779] niks3-server[1131]: 2026/09/20 12:46:24 goose: up to current file version: 24404lb # [ 31.056055] niks3-server[1131]: 2026/09/20 12:46:24 INFO Loaded signing key name=farm-test-1 path=/nix/store/c745bsy529madd2s6hz4z5is2436ni04-key4405worker1 # [ 31.127896] nix-grpc-daemon[854]: ts=2026-09-20T12:46:24Z level=info event=scheduler_connected addr=lb:500514406worker1 # [ 31.131970] nix-grpc-daemon[854]: ts=2026-09-20T12:46:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4407lb # [ 31.130083] niks3-server[1131]: 2026/09/20 12:46:24 INFO Read proxy enabled — serving cache objects from S34408lb # [ 31.132241] niks3-server[1131]: 2026/09/20 12:46:24 INFO Using socket-activated listener address=0.0.0.0:57514409lb # [ 31.138057] niks3-server[1131]: 2026/09/20 12:46:24 INFO systemd watchdog enabled interval=15s4410lb # [ 31.139955] systemd[1]: Started niks3 server.4411lb # [ 31.141609] systemd[1]: Reached target Multi-User System.4412lb # [ 31.142827] niks3-server[1131]: 2026/09/20 12:46:24 INFO Starting HTTP server address=0.0.0.0:57514413lb # [ 31.145068] systemd[1]: Startup finished in 882ms (kernel) + 4.125s (initrd) + 26.136s (userspace) = 31.144s.4414lb # [ 31.150040] niks3-server[1131]: 2026/09/20 12:46:24 INFO lead: acquired remote=192.168.1.4:485144415worker2 # [ 31.134168] nix-grpc-daemon[787]: ts=2026-09-20T12:46:25Z level=info event=scheduler_take_over4416worker2 # [ 31.173937] nix-grpc-daemon[787]: ts=2026-09-20T12:46:25Z level=info event=scheduler_connected addr=lb:500514417worker2 # [ 31.177714] nix-grpc-daemon[787]: ts=2026-09-20T12:46:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4418worker1 # [ 31.334165] nix-grpc-daemon[854]: ts=2026-09-20T12:46:24Z level=info event=scheduler_connected addr=lb:500514419worker1 # [ 31.335983] nix-grpc-daemon[854]: ts=2026-09-20T12:46:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4420worker2 # [ 31.378321] nix-grpc-daemon[787]: ts=2026-09-20T12:46:25Z level=info event=scheduler_connected addr=lb:500514421worker2 # [ 31.380440] nix-grpc-daemon[787]: ts=2026-09-20T12:46:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4422worker1 # [ 31.534748] nix-grpc-daemon[854]: ts=2026-09-20T12:46:24Z level=info event=scheduler_connected addr=lb:500514423worker1 # [ 31.536802] nix-grpc-daemon[854]: ts=2026-09-20T12:46:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4424worker2 # [ 31.581776] nix-grpc-daemon[787]: ts=2026-09-20T12:46:25Z level=info event=scheduler_connected addr=lb:500514425worker2 # [ 31.585924] nix-grpc-daemon[787]: ts=2026-09-20T12:46:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4426lb: (finished: waiting for unit niks3.service, in 20.79 seconds)4427lb: waiting for TCP port 5751 on localhost4428worker1 # [ 31.737544] nix-grpc-daemon[854]: ts=2026-09-20T12:46:25Z level=info event=scheduler_connected addr=lb:500514429worker2 # [ 31.708150] nix-grpc-daemon[787]: ts=2026-09-20T12:46:25Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:496324430worker2 # [ 31.712709] nix-grpc-daemon[787]: ts=2026-09-20T12:46:25Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04431lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!4432lb: (finished: waiting for TCP port 5751 on localhost, in 0.08 seconds)4433lb: waiting for unit envoy.service4434worker2 # [ 31.786811] nix-grpc-daemon[787]: ts=2026-09-20T12:46:25Z level=info event=scheduler_connected addr=lb:500514435lb: (finished: waiting for unit envoy.service, in 0.05 seconds)4436lb: waiting for TCP port 50051 on localhost4437worker2 # [ 31.808569] nix-grpc-daemon[787]: ts=2026-09-20T12:46:25Z level=info event=oidc_jwks provider=mock keys=14438worker2 # [ 31.812948] nix-grpc-daemon[787]: ts=2026-09-20T12:46:25Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:496324439worker2 # [ 31.816986] nix-grpc-daemon[787]: ts=2026-09-20T12:46:25Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04440lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!4441lb # [ 31.850880] envoy[973]: conn peer=[::1]:53280 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=24442lb: (finished: waiting for TCP port 50051 on localhost, in 0.04 seconds)4443worker2 # [ 31.935415] systemd[1]: Starting scheduler-token.service...4444worker2 # [ 31.997961] systemd[1]: scheduler-token.service: Deactivated successfully.4445worker2 # [ 31.999852] systemd[1]: Finished scheduler-token.service.4446worker2 # [ 32.001278] systemd[1]: scheduler-token.service: Consumed 19ms CPU time over 65ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4447lb: waiting until exactly no endpoints of x86_64-linux are unhealthy4448??? Warning (UserWarning): retry(): The 'timeout_seconds' argument is deprecated. Use 'timeout' instead.4449 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 1284450lb: must succeed: curl -sf localhost:9901/clusters4451lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4452lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.09 seconds)4453subtest: one node schedules and both builders hold a WorkerSession on it4454lb: must succeed: curl -sf localhost:9901/clusters4455lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4456worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4457worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4458lb: must succeed: curl -sf localhost:9901/clusters4459lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4460lb: must succeed: curl -sf localhost:9901/clusters4461lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4462worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'4463worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.04 seconds)4464(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.23 seconds)4465subtest: balancer auth: client cert, bearer token, nothing, unknown CN4466worker1 # [ 32.685552] nix-grpc-daemon[854]: ts=2026-09-20T12:46:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:396464467worker1 # [ 32.699993] systemd[1]: Started Nix Daemon.4468worker1 # [ 32.914899] nix-daemon[912]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4469worker1 # [ 32.970274] nix-grpc-daemon[854]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 20 ms (attempt 1/5)4470worker1 # [ 32.993317] nix-grpc-daemon[854]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 41 ms (attempt 2/5)4471worker1 # [ 33.037126] nix-grpc-daemon[854]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 109 ms (attempt 3/5)4472worker1 # [ 33.146729] nix-grpc-daemon[854]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 521 ms (attempt 4/5)4473worker1 # [ 33.678781] nix-grpc-daemon[854]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4474worker2 # [ 33.669838] nix-grpc-daemon[787]: ts=2026-09-20T12:46:27Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:598684475worker2 # [ 33.677440] systemd[1]: Started Nix Daemon.4476worker2 # [ 33.863347] nix-daemon[857]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4477worker2 # [ 33.890632] nix-grpc-daemon[787]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 51 ms (attempt 1/5)4478worker2 # [ 33.941883] nix-grpc-daemon[787]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 70 ms (attempt 2/5)4479worker2 # [ 34.014741] nix-grpc-daemon[787]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 126 ms (attempt 3/5)4480worker2 # [ 34.146599] nix-grpc-daemon[787]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 732 ms (attempt 4/5)4481worker2 # [ 34.892609] nix-grpc-daemon[787]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4482worker2 # [ 34.912416] nix-grpc-daemon[787]: ts=2026-09-20T12:46:28Z level=debug event=path_info_miss path=8yjya2ac0ki0lpwdcag7baz48ybnsw0d-nixos-system-client-test error="error: path '/nix/store/8yjya2ac0ki0lpwdcag7baz48ybnsw0d-nixos-system-client-test' is required, but there is no substituter that can build it"4483worker1 # [ 34.963117] nix-grpc-daemon[854]: ts=2026-09-20T12:46:28Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:396464484worker1 # [ 34.983194] nix-grpc-daemon[854]: ts=2026-09-20T12:46:28Z level=debug event=path_info_miss path=8yjya2ac0ki0lpwdcag7baz48ybnsw0d-nixos-system-client-test error="error: path '/nix/store/8yjya2ac0ki0lpwdcag7baz48ybnsw0d-nixos-system-client-test' is required, but there is no substituter that can build it"4485client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4486client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.08 seconds)4487client: must succeed: install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4488client: (finished: must succeed: install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.03 seconds)4489worker1 # [ 35.336190] nix-grpc-daemon[854]: ts=2026-09-20T12:46:28Z level=info event=oidc_jwks provider=mock keys=14490worker1 # [ 35.337868] nix-grpc-daemon[854]: ts=2026-09-20T12:46:28Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:396464491worker1 # [ 35.346866] nix-grpc-daemon[854]: ts=2026-09-20T12:46:28Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:396464492worker1 # [ 35.352300] nix-grpc-daemon[854]: ts=2026-09-20T12:46:28Z level=debug event=path_info_miss path=8yjya2ac0ki0lpwdcag7baz48ybnsw0d-nixos-system-client-test error="error: path '/nix/store/8yjya2ac0ki0lpwdcag7baz48ybnsw0d-nixos-system-client-test' is required, but there is no substituter that can build it"4493worker2 # [ 35.315924] nix-grpc-daemon[787]: ts=2026-09-20T12:46:29Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:598684494worker2 # [ 35.322095] nix-grpc-daemon[787]: ts=2026-09-20T12:46:29Z level=debug event=path_info_miss path=8yjya2ac0ki0lpwdcag7baz48ybnsw0d-nixos-system-client-test error="error: path '/nix/store/8yjya2ac0ki0lpwdcag7baz48ybnsw0d-nixos-system-client-test' is required, but there is no substituter that can build it"4495client: must succeed: rm -r /var/lib/nix-grpc-store4496client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.04 seconds)4497worker1 # [ 36.667602] nix-grpc-daemon[854]: ts=2026-09-20T12:46:30Z level=info event=denied method=QueryMissing cn=- role=none4498worker1 # [ 36.886164] nix-grpc-daemon[854]: ts=2026-09-20T12:46:30Z level=info event=denied method=QueryMissing cn=stranger role=none4499(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 4.55 seconds)4500subtest: DAG build is scheduled across workers and lands in the cache4501client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24502worker2 # [ 37.207641] nix-grpc-daemon[787]: ts=2026-09-20T12:46:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:598684503client # don't know how to build these paths:4504client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4505worker2 # [ 37.233151] nix-grpc-daemon[787]: ts=2026-09-20T12:46:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:598684506worker2 # [ 37.274357] nix-grpc-daemon[787]: ts=2026-09-20T12:46:31Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04507worker2 # [ 37.279311] nix-grpc-daemon[787]: ts=2026-09-20T12:46:31Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.3:50051 clients=14508lb # [ 37.312338] envoy[973]: conn peer=[2001:db8:1::1]:56128 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=23654509lb # [ 37.316175] envoy[973]: conn peer=[2001:db8:1::1]:45496 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=386 ms=474510lb # [ 37.319513] envoy[973]: conn peer=[2001:db8:1::1]:45510 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=94511lb # [ 37.323212] envoy[973]: conn peer=[2001:db8:1::1]:45514 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=134512worker1 # [ 37.370371] nix-grpc-daemon[854]: ts=2026-09-20T12:46:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:396464513worker1 # [ 37.394364] nix-grpc-daemon[854]: ts=2026-09-20T12:46:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:396464514client # copying 1 paths to 'lb:50051'...4515worker1 # [ 37.408537] nix-grpc-daemon[854]: ts=2026-09-20T12:46:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:396464516client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4517worker1 # [ 37.431363] nix-daemon[912]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4518worker1 # [ 37.458351] nix-grpc-daemon[854]: ts=2026-09-20T12:46:30Z level=info event=niks3_push_started pid=9234519worker1 # [ 37.748519] nix-grpc-daemon[923]: time=2026-09-20T12:46:31.170Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""4520worker1 # [ 37.945622] nix-daemon[912]: accepted connection from pid 935, user nix-grpc-daemon (trusted)4521worker1 # [ 37.960669] nix-daemon[912]: reaped child process 938, status = succeeded4522worker1 # [ 38.129213] nix-daemon[912]: accepted connection from pid 941, user nix-grpc-daemon (trusted)4523worker1 # [ 38.143686] nix-daemon[912]: reaped child process 944, status = succeeded4524lb # [ 38.143491] niks3-server[1131]: 2026/09/20 12:46:31 INFO Received uploads request method=POST path=/api/pending_closures4525worker1 # [ 38.183964] nix-grpc-daemon[923]: time=2026-09-20T12:46:31.610Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4526worker1 # [ 38.186359] nix-grpc-daemon[923]: time=2026-09-20T12:46:31.612Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4527lb # [ 38.212402] niks3-server[1131]: 2026/09/20 12:46:32 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4528lb # [ 38.216227] niks3-server[1131]: 2026/09/20 12:46:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4529lb # [ 38.225942] niks3-server[1131]: 2026/09/20 12:46:32 INFO Signed narinfos id=1 count=14530worker1 # [ 38.246424] nix-grpc-daemon[923]: time=2026-09-20T12:46:31.672Z level=INFO msg="Uploading 1 narinfos"4531lb # [ 38.239935] niks3-server[1131]: 2026/09/20 12:46:32 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4532lb # [ 38.247875] niks3-server[1131]: 2026/09/20 12:46:32 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4533lb # [ 38.261361] niks3-server[1131]: 2026/09/20 12:46:32 INFO Completed upload id=14534worker1 # [ 38.279879] nix-grpc-daemon[923]: time=2026-09-20T12:46:31.705Z level=INFO msg="Upload complete. (514ms)"4535worker1 # [ 38.285518] nix-grpc-daemon[854]: ts=2026-09-20T12:46:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39646 duration_s=0 paths=1 nar_bytes_in=5044536lb # [ 38.275359] niks3-server[1131]: 2026/09/20 12:46:32 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo4537worker1 # [ 38.292380] nix-grpc-daemon[854]: ts=2026-09-20T12:46:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:396464538worker1 # [ 38.305134] nix-daemon[912]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4539worker1 # [ 38.317187] nix-daemon[912]: reaped child process 921, status = succeeded4540worker1 # [ 38.321730] nix-daemon[912]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4541client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4542client # farm-a> node-a: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4543client # farm-a> LOG-farm-a-t14544worker1 # [ 39.339525] nix-daemon[912]: reaped child process 948, status = succeeded4545worker1 # [ 39.505417] nix-daemon[912]: accepted connection from pid 963, user nix-grpc-daemon (trusted)4546worker1 # [ 39.520953] nix-daemon[912]: reaped child process 967, status = succeeded4547lb # [ 39.519588] niks3-server[1131]: 2026/09/20 12:46:33 INFO Received uploads request method=POST path=/api/pending_closures4548worker1 # [ 39.575540] nix-grpc-daemon[923]: time=2026-09-20T12:46:33.001Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4549worker1 # [ 39.577395] nix-grpc-daemon[923]: time=2026-09-20T12:46:33.003Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4550lb # [ 39.607276] niks3-server[1131]: 2026/09/20 12:46:33 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4551lb # [ 39.615131] niks3-server[1131]: 2026/09/20 12:46:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4552lb # [ 39.619910] niks3-server[1131]: 2026/09/20 12:46:33 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4553lb # [ 39.624037] niks3-server[1131]: 2026/09/20 12:46:33 INFO Signed narinfos id=2 count=14554worker1 # [ 39.645268] nix-grpc-daemon[923]: time=2026-09-20T12:46:33.070Z level=INFO msg="Uploading 1 narinfos"4555lb # [ 39.630737] niks3-server[1131]: 2026/09/20 12:46:33 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4556lb # [ 39.641243] niks3-server[1131]: 2026/09/20 12:46:33 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4557lb # [ 39.644977] niks3-server[1131]: 2026/09/20 12:46:33 INFO Completed upload id=24558lb # [ 39.647249] niks3-server[1131]: 2026/09/20 12:46:33 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4559worker1 # [ 39.664613] nix-grpc-daemon[923]: time=2026-09-20T12:46:33.090Z level=INFO msg="Upload complete. (334ms)"4560worker1 # [ 39.672343] nix-grpc-daemon[854]: ts=2026-09-20T12:46:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39646 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14561worker1 # [ 39.685631] nix-daemon[912]: reaped child process 946, status = succeeded4562worker2 # [ 39.647951] nix-grpc-daemon[787]: ts=2026-09-20T12:46:33Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04563worker2 # [ 39.652148] nix-grpc-daemon[787]: ts=2026-09-20T12:46:33Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.3:50051 clients=14564worker1 # [ 39.733201] nix-grpc-daemon[854]: ts=2026-09-20T12:46:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:396464565worker1 # [ 39.746265] nix-grpc-daemon[854]: ts=2026-09-20T12:46:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:396464566worker1 # [ 39.760903] nix-grpc-daemon[923]: time=2026-09-20T12:46:33.186Z level=INFO msg="All 1 paths already cached"4567client # copying 1 paths to 'lb:50051'...4568client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4569worker1 # [ 39.769172] nix-grpc-daemon[854]: ts=2026-09-20T12:46:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:396464570worker1 # [ 39.777813] nix-daemon[912]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4571worker1 # [ 39.956360] nix-daemon[912]: accepted connection from pid 971, user nix-grpc-daemon (trusted)4572worker1 # [ 39.972411] nix-daemon[912]: reaped child process 974, status = succeeded4573worker1 # [ 40.145597] nix-daemon[912]: accepted connection from pid 976, user nix-grpc-daemon (trusted)4574worker1 # [ 40.160795] nix-daemon[912]: reaped child process 979, status = succeeded4575lb # [ 40.157584] niks3-server[1131]: 2026/09/20 12:46:33 INFO Received uploads request method=POST path=/api/pending_closures4576worker1 # [ 40.180692] nix-grpc-daemon[923]: time=2026-09-20T12:46:33.606Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4577worker1 # [ 40.182358] nix-grpc-daemon[923]: time=2026-09-20T12:46:33.608Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4578lb # [ 40.175735] niks3-server[1131]: 2026/09/20 12:46:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4579lb # [ 40.178989] niks3-server[1131]: 2026/09/20 12:46:33 INFO Signed narinfos id=3 count=14580worker1 # [ 40.197283] nix-grpc-daemon[923]: time=2026-09-20T12:46:33.623Z level=INFO msg="Uploading 1 narinfos"4581lb # [ 40.185096] niks3-server[1131]: 2026/09/20 12:46:33 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4582lb # [ 40.235377] niks3-server[1131]: 2026/09/20 12:46:34 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4583lb # [ 40.240276] niks3-server[1131]: 2026/09/20 12:46:34 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4584lb # [ 40.243792] niks3-server[1131]: 2026/09/20 12:46:34 INFO Completed upload id=34585lb # [ 40.246057] niks3-server[1131]: 2026/09/20 12:46:34 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4586worker1 # [ 40.263336] nix-grpc-daemon[923]: time=2026-09-20T12:46:33.689Z level=INFO msg="Upload complete. (470ms)"4587worker1 # [ 40.268316] nix-grpc-daemon[854]: ts=2026-09-20T12:46:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39646 duration_s=0 paths=1 nar_bytes_in=6244588worker1 # [ 40.275392] nix-grpc-daemon[854]: ts=2026-09-20T12:46:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:396464589worker1 # [ 40.288116] nix-daemon[912]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4590worker1 # [ 40.302282] nix-daemon[912]: reaped child process 969, status = succeeded4591client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4592worker1 # [ 40.305835] nix-daemon[912]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4593client # farm-b> node-a: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4594client # farm-b> LOG-farm-b-t14595worker1 # [ 41.123126] nix-daemon[912]: reaped child process 983, status = succeeded4596worker1 # [ 41.277857] nix-daemon[912]: accepted connection from pid 998, user nix-grpc-daemon (trusted)4597worker1 # [ 41.292562] nix-daemon[912]: reaped child process 1001, status = succeeded4598lb # [ 41.292224] niks3-server[1131]: 2026/09/20 12:46:35 INFO Received uploads request method=POST path=/api/pending_closures4599worker1 # [ 41.380555] nix-grpc-daemon[923]: time=2026-09-20T12:46:34.804Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4600worker1 # [ 41.382033] nix-grpc-daemon[923]: time=2026-09-20T12:46:34.805Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4601lb # [ 41.381716] niks3-server[1131]: 2026/09/20 12:46:35 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4602lb # [ 41.400154] niks3-server[1131]: 2026/09/20 12:46:35 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4603lb # [ 41.404266] niks3-server[1131]: 2026/09/20 12:46:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4604lb # [ 41.411783] niks3-server[1131]: 2026/09/20 12:46:35 INFO Signed narinfos id=4 count=14605worker1 # [ 41.430087] nix-grpc-daemon[923]: time=2026-09-20T12:46:34.855Z level=INFO msg="Uploading 1 narinfos"4606lb # [ 41.417167] niks3-server[1131]: 2026/09/20 12:46:35 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4607lb # [ 41.425870] niks3-server[1131]: 2026/09/20 12:46:35 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4608lb # [ 41.432919] niks3-server[1131]: 2026/09/20 12:46:35 INFO Completed upload id=44609lb # [ 41.434207] niks3-server[1131]: 2026/09/20 12:46:35 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4610worker1 # [ 41.451336] nix-grpc-daemon[923]: time=2026-09-20T12:46:34.877Z level=INFO msg="Upload complete. (336ms)"4611worker1 # [ 41.458248] nix-grpc-daemon[854]: ts=2026-09-20T12:46:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39646 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14612worker2 # [ 41.423920] nix-grpc-daemon[787]: ts=2026-09-20T12:46:35Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04613worker2 # [ 41.428839] nix-grpc-daemon[787]: ts=2026-09-20T12:46:35Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.3:50051 clients=14614worker1 # [ 41.472912] nix-daemon[912]: reaped child process 981, status = succeeded4615worker1 # [ 41.515216] nix-grpc-daemon[854]: ts=2026-09-20T12:46:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:396464616worker1 # [ 41.527965] nix-grpc-daemon[854]: ts=2026-09-20T12:46:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:396464617worker1 # [ 41.581121] nix-grpc-daemon[923]: time=2026-09-20T12:46:35.006Z level=INFO msg="All 2 paths already cached"4618client # copying 1 paths to 'lb:50051'...4619worker1 # [ 41.589832] nix-grpc-daemon[854]: ts=2026-09-20T12:46:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:396464620client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4621worker1 # [ 41.609133] nix-daemon[912]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4622worker1 # [ 41.767702] nix-daemon[912]: accepted connection from pid 1005, user nix-grpc-daemon (trusted)4623worker1 # [ 41.782571] nix-daemon[912]: reaped child process 1008, status = succeeded4624worker1 # [ 41.950469] nix-daemon[912]: accepted connection from pid 1010, user nix-grpc-daemon (trusted)4625worker1 # [ 41.965475] nix-daemon[912]: reaped child process 1013, status = succeeded4626lb # [ 41.962752] niks3-server[1131]: 2026/09/20 12:46:35 INFO Received uploads request method=POST path=/api/pending_closures4627worker1 # [ 41.989565] nix-grpc-daemon[923]: time=2026-09-20T12:46:35.415Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4628worker1 # [ 41.990996] nix-grpc-daemon[923]: time=2026-09-20T12:46:35.415Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4629lb # [ 41.989220] niks3-server[1131]: 2026/09/20 12:46:35 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4630lb # [ 41.999929] niks3-server[1131]: 2026/09/20 12:46:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4631worker1 # [ 42.022924] nix-grpc-daemon[923]: time=2026-09-20T12:46:35.448Z level=INFO msg="Uploading 1 narinfos"4632lb # [ 42.005203] niks3-server[1131]: 2026/09/20 12:46:35 INFO Signed narinfos id=5 count=14633lb # [ 42.011554] niks3-server[1131]: 2026/09/20 12:46:35 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4634lb # [ 42.024353] niks3-server[1131]: 2026/09/20 12:46:35 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4635lb # [ 42.029931] niks3-server[1131]: 2026/09/20 12:46:35 INFO Completed upload id=54636worker1 # [ 42.048247] nix-grpc-daemon[923]: time=2026-09-20T12:46:35.474Z level=INFO msg="Upload complete. (426ms)"4637lb # [ 42.033293] niks3-server[1131]: 2026/09/20 12:46:35 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4638worker1 # [ 42.051862] nix-grpc-daemon[854]: ts=2026-09-20T12:46:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39646 duration_s=0 paths=1 nar_bytes_in=7524639worker1 # [ 42.059112] nix-grpc-daemon[854]: ts=2026-09-20T12:46:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:396464640worker1 # [ 42.069119] nix-daemon[912]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4641worker1 # [ 42.071581] nix-daemon[912]: reaped child process 1003, status = succeeded4642client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4643worker1 # [ 42.078236] nix-daemon[912]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4644client # farm-top> node-a: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4645client # farm-top> LOG-farm-top-t14646worker1 # [ 43.292530] nix-daemon[912]: reaped child process 1017, status = succeeded4647worker1 # [ 43.420093] nix-daemon[912]: accepted connection from pid 1033, user nix-grpc-daemon (trusted)4648worker1 # [ 43.433542] nix-daemon[912]: reaped child process 1036, status = succeeded4649lb # [ 43.429264] niks3-server[1131]: 2026/09/20 12:46:37 INFO Received uploads request method=POST path=/api/pending_closures4650worker1 # [ 43.459346] nix-grpc-daemon[923]: time=2026-09-20T12:46:36.884Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4651worker1 # [ 43.460727] nix-grpc-daemon[923]: time=2026-09-20T12:46:36.884Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4652lb # [ 43.467656] niks3-server[1131]: 2026/09/20 12:46:37 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4653lb # [ 43.475969] niks3-server[1131]: 2026/09/20 12:46:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4654lb # [ 43.478369] niks3-server[1131]: 2026/09/20 12:46:37 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4655lb # [ 43.481088] niks3-server[1131]: 2026/09/20 12:46:37 INFO Signed narinfos id=6 count=14656worker1 # [ 43.499366] nix-grpc-daemon[923]: time=2026-09-20T12:46:36.925Z level=INFO msg="Uploading 1 narinfos"4657lb # [ 43.484224] niks3-server[1131]: 2026/09/20 12:46:37 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4658lb # [ 43.490239] niks3-server[1131]: 2026/09/20 12:46:37 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4659lb # [ 43.493488] niks3-server[1131]: 2026/09/20 12:46:37 INFO Completed upload id=64660worker1 # [ 43.511749] nix-grpc-daemon[923]: time=2026-09-20T12:46:36.937Z level=INFO msg="Upload complete. (227ms)"4661lb # [ 43.496089] niks3-server[1131]: 2026/09/20 12:46:37 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4662worker1 # [ 43.516823] nix-grpc-daemon[854]: ts=2026-09-20T12:46:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39646 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14663worker1 # [ 43.526522] nix-daemon[912]: reaped child process 1015, status = succeeded4664client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 6.65 seconds)4665client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4666client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.19 seconds)4667client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4668client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.02 seconds)4669client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4670client # copying 3 paths...4671client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4672client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4673client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4674client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.29 seconds)4675(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 7.24 seconds)4676subtest: repeat is answered Cached by the scheduler without building4677worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4678worker1 # finding garbage collector roots...4679worker1 # deleting specified paths...4680worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4681worker1 # deleting unused links...4682worker1 # note: hard linking is currently saving 0.0 KiB4683worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.23 seconds)4684worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4685worker2 # finding garbage collector roots...4686worker2 # deleting specified paths...4687worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4688worker2 # deleting unused links...4689worker2 # note: hard linking is currently saving 0.0 KiB4690worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.30 seconds)4691client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24692worker1 # [ 44.907341] nix-grpc-daemon[854]: ts=2026-09-20T12:46:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:396464693client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4694client # /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4695worker2 # [ 44.879961] nix-grpc-daemon[787]: ts=2026-09-20T12:46:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:598684696worker2 # [ 44.900176] nix-grpc-daemon[787]: ts=2026-09-20T12:46:38Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14697client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 0.29 seconds)4698client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4699client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.19 seconds)4700lb: must succeed: curl -sf localhost:9901/clusters4701lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4702lb: must succeed: curl -sf localhost:9901/clusters4703lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4704worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4705worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4706(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.26 seconds)4707subtest: inputs already in the cache are not sent to the scheduler4708lb: must succeed: curl -sf localhost:9901/clusters4709lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4710lb: must succeed: curl -sf localhost:9901/clusters4711lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4712worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4713worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.03 seconds)4714client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&24715worker2 # [ 45.685439] nix-grpc-daemon[787]: ts=2026-09-20T12:46:39Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:598684716client # don't know how to build these paths:4717client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4718worker1 # [ 45.733115] nix-grpc-daemon[854]: ts=2026-09-20T12:46:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:396464719worker1 # [ 45.740246] nix-grpc-daemon[923]: time=2026-09-20T12:46:39.166Z level=INFO msg="All 2 paths already cached"4720worker2 # [ 45.707882] nix-grpc-daemon[787]: ts=2026-09-20T12:46:39Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04721worker2 # [ 45.709764] nix-grpc-daemon[787]: ts=2026-09-20T12:46:39Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.3:50051 clients=14722worker1 # [ 45.793830] nix-grpc-daemon[854]: ts=2026-09-20T12:46:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:396464723worker1 # [ 45.803333] nix-grpc-daemon[854]: ts=2026-09-20T12:46:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:396464724worker1 # [ 45.811713] nix-grpc-daemon[923]: time=2026-09-20T12:46:39.237Z level=INFO msg="All 2 paths already cached"4725client # copying 1 paths to 'lb:50051'...4726worker1 # [ 45.817612] nix-grpc-daemon[854]: ts=2026-09-20T12:46:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:396464727client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4728worker1 # [ 45.824912] nix-daemon[912]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4729worker1 # [ 45.993751] nix-daemon[912]: accepted connection from pid 1053, user nix-grpc-daemon (trusted)4730worker1 # [ 46.009381] nix-daemon[912]: reaped child process 1056, status = succeeded4731worker1 # [ 46.164677] nix-daemon[912]: accepted connection from pid 1058, user nix-grpc-daemon (trusted)4732worker1 # [ 46.179706] nix-daemon[912]: reaped child process 1061, status = succeeded4733lb # [ 46.175652] niks3-server[1131]: 2026/09/20 12:46:39 INFO Received uploads request method=POST path=/api/pending_closures4734worker1 # [ 46.199514] nix-grpc-daemon[923]: time=2026-09-20T12:46:39.625Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4735worker1 # [ 46.201202] nix-grpc-daemon[923]: time=2026-09-20T12:46:39.627Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4736lb # [ 46.194514] niks3-server[1131]: 2026/09/20 12:46:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4737lb # [ 46.197772] niks3-server[1131]: 2026/09/20 12:46:39 INFO Signed narinfos id=7 count=14738worker1 # [ 46.217260] nix-grpc-daemon[923]: time=2026-09-20T12:46:39.643Z level=INFO msg="Uploading 1 narinfos"4739lb # [ 46.199069] niks3-server[1131]: 2026/09/20 12:46:39 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4740lb # [ 46.204639] niks3-server[1131]: 2026/09/20 12:46:40 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4741lb # [ 46.207372] niks3-server[1131]: 2026/09/20 12:46:40 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4742lb # [ 46.211340] niks3-server[1131]: 2026/09/20 12:46:40 INFO Completed upload id=74743worker1 # [ 46.229323] nix-grpc-daemon[923]: time=2026-09-20T12:46:39.655Z level=INFO msg="Upload complete. (399ms)"4744lb # [ 46.212849] niks3-server[1131]: 2026/09/20 12:46:40 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4745worker1 # [ 46.231818] nix-grpc-daemon[854]: ts=2026-09-20T12:46:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39646 duration_s=0 paths=1 nar_bytes_in=7524746worker1 # [ 46.235703] nix-grpc-daemon[854]: ts=2026-09-20T12:46:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:396464747worker1 # [ 46.242565] nix-daemon[912]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4748worker1 # [ 46.245515] nix-daemon[912]: reaped child process 1051, status = succeeded4749worker1 # [ 46.250790] nix-daemon[912]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4750client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4751client # farm-top> node-a: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4752lb # [ 47.312096] envoy[973]: conn peer=[2001:db8:1::1]:45526 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9537 tx=3804 ms=64404753lb # [ 47.313972] envoy[973]: conn peer=[2001:db8:1::1]:46888 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=479 ms=474754client # farm-top> LOG-farm-top-t1b4755worker1 # [ 47.868642] nix-daemon[912]: reaped child process 1065, status = succeeded4756worker1 # [ 48.007136] nix-daemon[912]: accepted connection from pid 1080, user nix-grpc-daemon (trusted)4757worker1 # [ 48.021701] nix-daemon[912]: reaped child process 1083, status = succeeded4758lb # [ 48.019079] niks3-server[1131]: 2026/09/20 12:46:41 INFO Received uploads request method=POST path=/api/pending_closures4759worker1 # [ 48.043914] nix-grpc-daemon[923]: time=2026-09-20T12:46:41.469Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4760worker1 # [ 48.046053] nix-grpc-daemon[923]: time=2026-09-20T12:46:41.471Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4761lb # [ 48.059217] niks3-server[1131]: 2026/09/20 12:46:41 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4762lb # [ 48.064367] niks3-server[1131]: 2026/09/20 12:46:41 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4763lb # [ 48.079250] niks3-server[1131]: 2026/09/20 12:46:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4764lb # [ 48.086630] niks3-server[1131]: 2026/09/20 12:46:41 INFO Signed narinfos id=8 count=14765worker1 # [ 48.107782] nix-grpc-daemon[923]: time=2026-09-20T12:46:41.533Z level=INFO msg="Uploading 1 narinfos"4766lb # [ 48.089611] niks3-server[1131]: 2026/09/20 12:46:41 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4767lb # [ 48.101179] niks3-server[1131]: 2026/09/20 12:46:41 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4768lb # [ 48.105500] niks3-server[1131]: 2026/09/20 12:46:41 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4769lb # [ 48.108434] niks3-server[1131]: 2026/09/20 12:46:41 INFO Completed upload id=84770worker1 # [ 48.127953] nix-grpc-daemon[923]: time=2026-09-20T12:46:41.553Z level=INFO msg="Upload complete. (268ms)"4771worker1 # [ 48.132257] nix-grpc-daemon[854]: ts=2026-09-20T12:46:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39646 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14772worker1 # [ 48.140649] nix-daemon[912]: reaped child process 1063, status = succeeded4773client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.63 seconds)4774client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath4775client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.19 seconds)4776lb: must succeed: curl -sf localhost:9901/clusters4777lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4778lb: must succeed: curl -sf localhost:9901/clusters4779lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4780worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4781worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4782(finished: subtest: inputs already in the cache are not sent to the scheduler, in 3.11 seconds)4783subtest: two clients wanting the same drv build it once4784client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4785client # [ 48.577246] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4786client # Running as unit: dedup1.service; invocation ID: ff57cf7c7e4245729b3c3458f621dce74787client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 0.06 seconds)4788client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4789client # [ 48.779602] nix[890]: don't know how to build these paths:4790client # [ 48.780513] nix[890]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4791worker2 # [ 48.737256] nix-grpc-daemon[787]: ts=2026-09-20T12:46:42Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:598684792worker2 # [ 48.748938] nix-grpc-daemon[787]: ts=2026-09-20T12:46:42Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04793worker2 # [ 48.754837] nix-grpc-daemon[787]: ts=2026-09-20T12:46:42Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.4:50051 clients=14794worker2 # [ 48.759637] nix-grpc-daemon[787]: ts=2026-09-20T12:46:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:598684795worker2 # [ 48.775067] nix-grpc-daemon[787]: ts=2026-09-20T12:46:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:598684796client # [ 48.827133] nix[890]: copying 1 paths to 'lb:50051'...4797client # [ 48.827941] nix[890]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4798worker2 # [ 48.786371] nix-grpc-daemon[787]: ts=2026-09-20T12:46:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:598684799worker2 # [ 48.795895] nix-daemon[857]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4800worker2 # [ 48.820554] nix-grpc-daemon[787]: ts=2026-09-20T12:46:42Z level=info event=niks3_push_started pid=9074801worker2 # [ 49.066637] nix-grpc-daemon[907]: time=2026-09-20T12:46:43.035Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""4802worker2 # [ 49.241785] nix-daemon[857]: accepted connection from pid 919, user nix-grpc-daemon (trusted)4803worker2 # [ 49.257349] nix-daemon[857]: reaped child process 922, status = succeeded4804worker2 # [ 49.390082] nix-daemon[857]: accepted connection from pid 924, user nix-grpc-daemon (trusted)4805worker2 # [ 49.403693] nix-daemon[857]: reaped child process 927, status = succeeded4806lb # [ 49.441294] niks3-server[1131]: 2026/09/20 12:46:43 INFO Received uploads request method=POST path=/api/pending_closures4807worker2 # [ 49.428085] nix-grpc-daemon[907]: time=2026-09-20T12:46:43.401Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4808worker2 # [ 49.430215] nix-grpc-daemon[907]: time=2026-09-20T12:46:43.403Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4809lb # [ 49.476899] niks3-server[1131]: 2026/09/20 12:46:43 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4810lb # [ 49.487939] niks3-server[1131]: 2026/09/20 12:46:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4811worker2 # [ 49.472388] nix-grpc-daemon[907]: time=2026-09-20T12:46:43.445Z level=INFO msg="Uploading 1 narinfos"4812lb # [ 49.494775] niks3-server[1131]: 2026/09/20 12:46:43 INFO Signed narinfos id=9 count=14813lb # [ 49.504602] niks3-server[1131]: 2026/09/20 12:46:43 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4814lb # [ 49.513073] niks3-server[1131]: 2026/09/20 12:46:43 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4815worker2 # [ 49.497522] nix-grpc-daemon[907]: time=2026-09-20T12:46:43.470Z level=INFO msg="Upload complete. (417ms)"4816lb # [ 49.520323] niks3-server[1131]: 2026/09/20 12:46:43 INFO Completed upload id=94817worker2 # [ 49.499327] nix-grpc-daemon[787]: ts=2026-09-20T12:46:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59868 duration_s=0 paths=1 nar_bytes_in=4324818lb # [ 49.523105] niks3-server[1131]: 2026/09/20 12:46:43 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4819worker2 # [ 49.506273] nix-daemon[857]: reaped child process 905, status = succeeded4820worker2 # [ 49.507836] nix-grpc-daemon[787]: ts=2026-09-20T12:46:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:598684821worker2 # [ 49.512330] nix-daemon[857]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4822client # [ 49.559929] nix[890]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4823worker2 # [ 49.519052] nix-daemon[857]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4824client # [ 49.637130] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4825client # Running as unit: dedup2.service; invocation ID: b84e59ed2ee249c69153094d22de8b954826client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 1.06 seconds)4827client # [ 49.857801] nix[905]: don't know how to build these paths:4828client # [ 49.858721] nix[905]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4829worker1 # [ 49.855496] nix-grpc-daemon[854]: ts=2026-09-20T12:46:43Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:396464830worker2 # [ 49.824976] nix-grpc-daemon[787]: ts=2026-09-20T12:46:43Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04831client # [ 49.872807] nix[905]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4832worker2 # [ 49.830671] nix-grpc-daemon[787]: ts=2026-09-20T12:46:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:598684833client: must succeed: sleep 24834client: (finished: must succeed: sleep 2, in 2.05 seconds)4835??? Warning (UserWarning): wait_until_succeeds(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4836 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394837client: waiting for success: ! systemctl is-active dedup1 dedup24838??? Warning (UserWarning): execute(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4839 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394840worker2 # [ 53.142653] nix-daemon[857]: reaped child process 931, status = succeeded4841worker2 # [ 53.297894] nix-daemon[857]: accepted connection from pid 962, user nix-grpc-daemon (trusted)4842worker2 # [ 53.310266] nix-daemon[857]: reaped child process 965, status = succeeded4843lb # [ 53.349482] niks3-server[1131]: 2026/09/20 12:46:47 INFO Received uploads request method=POST path=/api/pending_closures4844worker2 # [ 53.337597] nix-grpc-daemon[907]: time=2026-09-20T12:46:47.310Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4845worker2 # [ 53.339283] nix-grpc-daemon[907]: time=2026-09-20T12:46:47.312Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4846lb # [ 53.394505] niks3-server[1131]: 2026/09/20 12:46:47 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4847lb # [ 53.405657] niks3-server[1131]: 2026/09/20 12:46:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4848lb # [ 53.411146] niks3-server[1131]: 2026/09/20 12:46:47 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4849lb # [ 53.417486] niks3-server[1131]: 2026/09/20 12:46:47 INFO Signed narinfos id=10 count=14850worker2 # [ 53.399359] nix-grpc-daemon[907]: time=2026-09-20T12:46:47.371Z level=INFO msg="Uploading 1 narinfos"4851lb # [ 53.424360] niks3-server[1131]: 2026/09/20 12:46:47 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4852lb # [ 53.435224] niks3-server[1131]: 2026/09/20 12:46:47 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4853lb # [ 53.440935] niks3-server[1131]: 2026/09/20 12:46:47 INFO Completed upload id=104854worker2 # [ 53.418201] nix-grpc-daemon[907]: time=2026-09-20T12:46:47.391Z level=INFO msg="Upload complete. (288ms)"4855lb # [ 53.443024] niks3-server[1131]: 2026/09/20 12:46:47 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4856worker2 # [ 53.422902] nix-grpc-daemon[787]: ts=2026-09-20T12:46:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59868 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14857worker2 # [ 53.425939] nix-grpc-daemon[787]: ts=2026-09-20T12:46:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59868 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14858worker2 # [ 53.433851] nix-daemon[857]: reaped child process 929, status = succeeded4859client # [ 53.510186] systemd[1]: dedup2.service: Deactivated successfully.4860client # [ 53.515695] systemd[1]: dedup2.service: Consumed 122ms CPU time over 3.872s wall clock time, 31.2M memory peak, 7.4K incoming IP traffic, 6.4K outgoing IP traffic.4861client # [ 53.556598] systemd[1]: dedup1.service: Deactivated successfully.4862client # [ 53.558388] systemd[1]: dedup1.service: Consumed 115ms CPU time over 4.979s wall clock time, 39.3M memory peak, 8.7K incoming IP traffic, 9K outgoing IP traffic.4863client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.12 seconds)4864client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4865client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.05 seconds)4866worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4867worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.06 seconds)4868worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4869worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4870worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4871worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4872worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4873worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4874(finished: subtest: two clients wanting the same drv build it once, in 5.96 seconds)4875subtest: build hook: nix-daemon with builders = grpc://lb4876client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log4877client # [ 54.684945] systemd[1]: Started Nix Daemon.4878client # [ 54.850263] nix-daemon[946]: accepted connection from pid 943, user root (trusted)4879worker2 # [ 54.983469] nix-grpc-daemon[787]: ts=2026-09-20T12:46:48Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:598684880worker1 # [ 55.030683] nix-grpc-daemon[854]: ts=2026-09-20T12:46:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:396464881worker2 # [ 55.006452] nix-grpc-daemon[787]: ts=2026-09-20T12:46:48Z level=debug event=want client=6 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=04882worker2 # [ 55.010768] nix-grpc-daemon[787]: ts=2026-09-20T12:46:48Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.3:50051 clients=14883worker1 # [ 55.091717] nix-grpc-daemon[854]: ts=2026-09-20T12:46:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:396464884worker1 # [ 55.101642] nix-grpc-daemon[854]: ts=2026-09-20T12:46:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:396464885worker1 # [ 55.109962] nix-grpc-daemon[854]: ts=2026-09-20T12:46:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:396464886worker1 # [ 55.114940] nix-daemon[912]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4887worker1 # [ 55.289285] nix-daemon[912]: accepted connection from pid 1108, user nix-grpc-daemon (trusted)4888worker1 # [ 55.305206] nix-daemon[912]: reaped child process 1111, status = succeeded4889worker1 # [ 55.476877] nix-daemon[912]: accepted connection from pid 1113, user nix-grpc-daemon (trusted)4890worker1 # [ 55.491870] nix-daemon[912]: reaped child process 1116, status = succeeded4891lb # [ 55.491443] niks3-server[1131]: 2026/09/20 12:46:49 INFO Received uploads request method=POST path=/api/pending_closures4892worker1 # [ 55.518234] nix-grpc-daemon[923]: time=2026-09-20T12:46:48.943Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4893worker1 # [ 55.519670] nix-grpc-daemon[923]: time=2026-09-20T12:46:48.943Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"4894lb # [ 55.521688] niks3-server[1131]: 2026/09/20 12:46:49 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst4895lb # [ 55.536750] niks3-server[1131]: 2026/09/20 12:46:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4896lb # [ 55.544897] niks3-server[1131]: 2026/09/20 12:46:49 INFO Signed narinfos id=11 count=14897worker1 # [ 55.566138] nix-grpc-daemon[923]: time=2026-09-20T12:46:48.991Z level=INFO msg="Uploading 1 narinfos"4898lb # [ 55.549810] niks3-server[1131]: 2026/09/20 12:46:49 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls4899lb # [ 55.559283] niks3-server[1131]: 2026/09/20 12:46:49 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4900lb # [ 55.566936] niks3-server[1131]: 2026/09/20 12:46:49 INFO Completed upload id=114901worker1 # [ 55.585175] nix-grpc-daemon[923]: time=2026-09-20T12:46:49.011Z level=INFO msg="Upload complete. (462ms)"4902lb # [ 55.568173] niks3-server[1131]: 2026/09/20 12:46:49 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo4903worker1 # [ 55.590095] nix-grpc-daemon[854]: ts=2026-09-20T12:46:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39646 duration_s=0 paths=1 nar_bytes_in=5124904worker1 # [ 55.594714] nix-grpc-daemon[854]: ts=2026-09-20T12:46:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:396464905worker1 # [ 55.602990] nix-daemon[912]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4906worker1 # [ 55.613127] nix-daemon[912]: reaped child process 1106, status = succeeded4907worker1 # [ 55.616101] nix-daemon[912]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4908worker1 # [ 56.830422] nix-daemon[912]: reaped child process 1120, status = succeeded4909worker1 # [ 56.978690] nix-daemon[912]: accepted connection from pid 1135, user nix-grpc-daemon (trusted)4910worker1 # [ 56.993258] nix-daemon[912]: reaped child process 1138, status = succeeded4911lb # [ 56.989587] niks3-server[1131]: 2026/09/20 12:46:50 INFO Received uploads request method=POST path=/api/pending_closures4912worker1 # [ 57.018369] nix-grpc-daemon[923]: time=2026-09-20T12:46:50.443Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4913worker1 # [ 57.019809] nix-grpc-daemon[923]: time=2026-09-20T12:46:50.443Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"4914lb # [ 57.026547] niks3-server[1131]: 2026/09/20 12:46:50 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv4915lb # [ 57.044755] niks3-server[1131]: 2026/09/20 12:46:50 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4916lb # [ 57.053277] niks3-server[1131]: 2026/09/20 12:46:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4917lb # [ 57.057429] niks3-server[1131]: 2026/09/20 12:46:50 INFO Signed narinfos id=12 count=14918worker1 # [ 57.078215] nix-grpc-daemon[923]: time=2026-09-20T12:46:50.504Z level=INFO msg="Uploading 1 narinfos"4919lb # [ 57.068405] niks3-server[1131]: 2026/09/20 12:46:50 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4920lb # [ 57.070358] niks3-server[1131]: 2026/09/20 12:46:50 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls4921lb # [ 57.074815] niks3-server[1131]: 2026/09/20 12:46:50 INFO Completed upload id=124922worker1 # [ 57.093163] nix-grpc-daemon[923]: time=2026-09-20T12:46:50.519Z level=INFO msg="Upload complete. (270ms)"4923lb # [ 57.076417] niks3-server[1131]: 2026/09/20 12:46:50 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo4924worker1 # [ 57.097960] nix-grpc-daemon[854]: ts=2026-09-20T12:46:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39646 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=6 outputs=14925worker1 # [ 57.104865] nix-grpc-daemon[854]: ts=2026-09-20T12:46:50Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:396464926worker1 # [ 57.114187] nix-daemon[912]: reaped child process 1118, status = succeeded4927worker1 # [ 57.117290] nix-grpc-daemon[854]: ts=2026-09-20T12:46:50Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:396464928worker1 # [ 57.119877] nix-grpc-daemon[854]: ts=2026-09-20T12:46:50Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:39646 duration_s=0 paths=1 nar_bytes_out=1284929lb # [ 57.312970] envoy[973]: conn peer=[2001:db8:1::1]:46894 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4182 tx=1701 ms=24194930lb # [ 57.316699] envoy[973]: conn peer=[2001:db8:1::1]:46914 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1427 tx=1085 ms=36304931lb # [ 57.320269] envoy[973]: conn peer=[2001:db8:1::1]:46904 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2985 tx=1372 ms=47534932lb # [ 57.323886] envoy[973]: conn peer=[2001:db8:1::1]:32768 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=524933lb # [ 57.327416] envoy[973]: conn peer=[2001:db8:1::1]:60984 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3383 tx=1642 ms=21494934worker1 # [ 57.376350] nix-grpc-daemon[854]: ts=2026-09-20T12:46:50Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:396464935worker2 # [ 57.339082] nix-grpc-daemon[787]: ts=2026-09-20T12:46:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:598684936worker2 # [ 57.361846] nix-grpc-daemon[787]: ts=2026-09-20T12:46:51Z level=debug event=want client=7 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=04937worker2 # [ 57.364155] nix-grpc-daemon[787]: ts=2026-09-20T12:46:51Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.3:50051 clients=14938worker1 # [ 57.407517] nix-grpc-daemon[854]: ts=2026-09-20T12:46:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:396464939worker1 # [ 57.420704] nix-grpc-daemon[854]: ts=2026-09-20T12:46:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:396464940worker1 # [ 57.429513] nix-grpc-daemon[923]: time=2026-09-20T12:46:50.855Z level=INFO msg="All 1 paths already cached"4941worker1 # [ 57.433993] nix-grpc-daemon[854]: ts=2026-09-20T12:46:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:396464942worker1 # [ 57.439196] nix-daemon[912]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4943worker1 # [ 57.607115] nix-daemon[912]: accepted connection from pid 1142, user nix-grpc-daemon (trusted)4944worker1 # [ 57.628873] nix-daemon[912]: reaped child process 1145, status = succeeded4945worker1 # [ 57.800775] nix-daemon[912]: accepted connection from pid 1147, user nix-grpc-daemon (trusted)4946worker1 # [ 57.814737] nix-daemon[912]: reaped child process 1150, status = succeeded4947lb # [ 57.810329] niks3-server[1131]: 2026/09/20 12:46:51 INFO Received uploads request method=POST path=/api/pending_closures4948worker1 # [ 57.838671] nix-grpc-daemon[923]: time=2026-09-20T12:46:51.264Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4949worker1 # [ 57.840345] nix-grpc-daemon[923]: time=2026-09-20T12:46:51.266Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"4950lb # [ 57.843284] niks3-server[1131]: 2026/09/20 12:46:51 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst4951lb # [ 57.854271] niks3-server[1131]: 2026/09/20 12:46:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4952worker1 # [ 57.876778] nix-grpc-daemon[923]: time=2026-09-20T12:46:51.302Z level=INFO msg="Uploading 1 narinfos"4953lb # [ 57.858669] niks3-server[1131]: 2026/09/20 12:46:51 INFO Signed narinfos id=13 count=14954lb # [ 57.865125] niks3-server[1131]: 2026/09/20 12:46:51 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls4955lb # [ 57.875459] niks3-server[1131]: 2026/09/20 12:46:51 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4956lb # [ 57.881912] niks3-server[1131]: 2026/09/20 12:46:51 INFO Completed upload id=134957worker1 # [ 57.900166] nix-grpc-daemon[923]: time=2026-09-20T12:46:51.326Z level=INFO msg="Upload complete. (455ms)"4958lb # [ 57.886480] niks3-server[1131]: 2026/09/20 12:46:51 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo4959worker1 # [ 57.903869] nix-grpc-daemon[854]: ts=2026-09-20T12:46:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39646 duration_s=0 paths=1 nar_bytes_in=6244960worker1 # [ 57.910197] nix-grpc-daemon[854]: ts=2026-09-20T12:46:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:396464961worker1 # [ 57.918581] nix-daemon[912]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4962worker1 # [ 57.924843] nix-daemon[912]: reaped child process 1140, status = succeeded4963worker1 # [ 57.930606] nix-daemon[912]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4964worker1 # [ 59.357860] nix-daemon[912]: reaped child process 1154, status = succeeded4965worker1 # [ 59.517647] nix-daemon[912]: accepted connection from pid 1169, user nix-grpc-daemon (trusted)4966worker1 # [ 59.531989] nix-daemon[912]: reaped child process 1172, status = succeeded4967lb # [ 59.525901] niks3-server[1131]: 2026/09/20 12:46:53 INFO Received uploads request method=POST path=/api/pending_closures4968worker1 # [ 59.812844] nix-grpc-daemon[923]: time=2026-09-20T12:46:53.237Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4969worker1 # [ 59.818265] nix-grpc-daemon[923]: time=2026-09-20T12:46:53.237Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"4970lb # [ 59.833393] niks3-server[1131]: 2026/09/20 12:46:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4971lb # [ 59.838741] niks3-server[1131]: 2026/09/20 12:46:53 INFO Signed narinfos id=14 count=14972worker1 # [ 59.858339] nix-grpc-daemon[923]: time=2026-09-20T12:46:53.283Z level=INFO msg="Uploading 1 narinfos"4973lb # [ 59.843781] niks3-server[1131]: 2026/09/20 12:46:53 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv4974lb # [ 59.858191] niks3-server[1131]: 2026/09/20 12:46:53 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4975lb # [ 59.863128] niks3-server[1131]: 2026/09/20 12:46:53 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst4976lb # [ 59.885427] niks3-server[1131]: 2026/09/20 12:46:53 INFO Completed upload id=144977worker1 # [ 59.906261] nix-grpc-daemon[923]: time=2026-09-20T12:46:53.332Z level=INFO msg="Upload complete. (567ms)"4978worker1 # [ 59.913324] nix-grpc-daemon[854]: ts=2026-09-20T12:46:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39646 duration_s=2 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=7 outputs=14979lb # [ 59.899890] niks3-server[1131]: 2026/09/20 12:46:53 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls4980worker1 # [ 59.920588] nix-grpc-daemon[854]: ts=2026-09-20T12:46:53Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:396464981worker1 # [ 59.933758] nix-daemon[912]: reaped child process 1152, status = succeeded4982lb # [ 59.915204] niks3-server[1131]: 2026/09/20 12:46:53 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo4983worker1 # [ 59.945169] nix-grpc-daemon[854]: ts=2026-09-20T12:46:53Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:396464984worker1 # [ 59.950413] nix-grpc-daemon[854]: ts=2026-09-20T12:46:53Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:39646 duration_s=0 paths=1 nar_bytes_out=1764985worker1 # [ 60.212558] nix-grpc-daemon[854]: ts=2026-09-20T12:46:53Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:396464986worker2 # [ 60.175472] nix-grpc-daemon[787]: ts=2026-09-20T12:46:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:598684987worker1 # [ 60.237180] nix-grpc-daemon[854]: ts=2026-09-20T12:46:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:396464988worker2 # [ 60.194663] nix-grpc-daemon[787]: ts=2026-09-20T12:46:54Z level=debug event=want client=8 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=04989worker2 # [ 60.199220] nix-grpc-daemon[787]: ts=2026-09-20T12:46:54Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.3:50051 clients=14990worker1 # [ 60.243814] nix-grpc-daemon[854]: ts=2026-09-20T12:46:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:396464991worker1 # [ 60.250602] nix-grpc-daemon[923]: time=2026-09-20T12:46:53.676Z level=INFO msg="All 2 paths already cached"4992worker1 # [ 60.254429] nix-grpc-daemon[854]: ts=2026-09-20T12:46:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:396464993worker1 # [ 60.260978] nix-daemon[912]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4994worker1 # [ 60.410825] nix-daemon[912]: accepted connection from pid 1176, user nix-grpc-daemon (trusted)4995worker1 # [ 60.425831] nix-daemon[912]: reaped child process 1179, status = succeeded4996worker1 # [ 60.579915] nix-daemon[912]: accepted connection from pid 1181, user nix-grpc-daemon (trusted)4997worker1 # [ 60.593699] nix-daemon[912]: reaped child process 1184, status = succeeded4998lb # [ 60.590847] niks3-server[1131]: 2026/09/20 12:46:54 INFO Received uploads request method=POST path=/api/pending_closures4999worker1 # [ 60.617249] nix-grpc-daemon[923]: time=2026-09-20T12:46:54.043Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5000worker1 # [ 60.618733] nix-grpc-daemon[923]: time=2026-09-20T12:46:54.044Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"5001lb # [ 60.619979] niks3-server[1131]: 2026/09/20 12:46:54 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst5002lb # [ 60.632138] niks3-server[1131]: 2026/09/20 12:46:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign5003lb # [ 60.637594] niks3-server[1131]: 2026/09/20 12:46:54 INFO Signed narinfos id=15 count=15004worker1 # [ 60.657793] nix-grpc-daemon[923]: time=2026-09-20T12:46:54.083Z level=INFO msg="Uploading 1 narinfos"5005lb # [ 60.640725] niks3-server[1131]: 2026/09/20 12:46:54 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls5006lb # [ 60.646150] niks3-server[1131]: 2026/09/20 12:46:54 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete5007lb # [ 60.650238] niks3-server[1131]: 2026/09/20 12:46:54 INFO Completed upload id=155008worker1 # [ 60.668453] nix-grpc-daemon[923]: time=2026-09-20T12:46:54.094Z level=INFO msg="Upload complete. (401ms)"5009lb # [ 60.651944] niks3-server[1131]: 2026/09/20 12:46:54 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo5010worker1 # [ 60.670602] nix-grpc-daemon[854]: ts=2026-09-20T12:46:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39646 duration_s=0 paths=1 nar_bytes_in=7525011worker1 # [ 60.674539] nix-grpc-daemon[854]: ts=2026-09-20T12:46:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:396465012worker1 # [ 60.681375] nix-daemon[912]: accepted connection from pid 854, user nix-grpc-daemon (trusted)5013worker1 # [ 60.683805] nix-daemon[912]: reaped child process 1174, status = succeeded5014worker1 # [ 60.689358] nix-daemon[912]: accepted connection from pid 854, user nix-grpc-daemon (trusted)5015worker1 # [ 62.112467] nix-daemon[912]: reaped child process 1188, status = succeeded5016worker1 # [ 62.270963] nix-daemon[912]: accepted connection from pid 1203, user nix-grpc-daemon (trusted)5017worker1 # [ 62.284449] nix-daemon[912]: reaped child process 1206, status = succeeded5018lb # [ 62.284786] niks3-server[1131]: 2026/09/20 12:46:56 INFO Received uploads request method=POST path=/api/pending_closures5019worker1 # [ 62.312100] nix-grpc-daemon[923]: time=2026-09-20T12:46:55.737Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5020worker1 # [ 62.313569] nix-grpc-daemon[923]: time=2026-09-20T12:46:55.739Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"5021lb # [ 62.333117] niks3-server[1131]: 2026/09/20 12:46:56 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst5022lb # [ 62.344368] niks3-server[1131]: 2026/09/20 12:46:56 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv5023lb # [ 62.351075] niks3-server[1131]: 2026/09/20 12:46:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign5024lb # [ 62.356945] niks3-server[1131]: 2026/09/20 12:46:56 INFO Signed narinfos id=16 count=15025worker1 # [ 62.374777] nix-grpc-daemon[923]: time=2026-09-20T12:46:55.800Z level=INFO msg="Uploading 1 narinfos"5026lb # [ 62.362479] niks3-server[1131]: 2026/09/20 12:46:56 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls5027lb # [ 62.371916] niks3-server[1131]: 2026/09/20 12:46:56 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete5028lb # [ 62.375143] niks3-server[1131]: 2026/09/20 12:46:56 INFO Completed upload id=165029worker1 # [ 62.393521] nix-grpc-daemon[923]: time=2026-09-20T12:46:55.819Z level=INFO msg="Upload complete. (296ms)"5030lb # [ 62.377628] niks3-server[1131]: 2026/09/20 12:46:56 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo5031worker1 # [ 62.399658] nix-grpc-daemon[854]: ts=2026-09-20T12:46:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39646 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=8 outputs=15032worker2 # [ 62.363096] nix-grpc-daemon[787]: ts=2026-09-20T12:46:56Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:598685033worker2 # [ 62.372196] nix-grpc-daemon[787]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...5034worker1 # [ 62.413610] nix-daemon[912]: reaped child process 1186, status = succeeded5035worker2 # [ 62.382533] nix-grpc-daemon[787]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...5036worker2 # [ 62.388598] nix-grpc-daemon[787]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...5037worker1 # [ 62.451155] nix-grpc-daemon[854]: ts=2026-09-20T12:46:55Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:396465038worker1 # [ 62.455920] nix-grpc-daemon[854]: ts=2026-09-20T12:46:55Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:39646 duration_s=0 paths=1 nar_bytes_out=2325039client # [ 62.523828] nix-daemon[946]: reaped child process 948, status = succeeded5040client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log, in 8.04 seconds)5041client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5042client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.02 seconds)5043client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5044client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.04 seconds)5045client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5046client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)5047(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 8.13 seconds)5048subtest: an input only one worker has still reaches the builder5049client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only5050client # this derivation will be built:5051client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5052client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5053client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only, in 0.40 seconds)5054client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5055client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.17 seconds)5056worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5057worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.23 seconds)5058worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5059worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.03 seconds)5060client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&25061client # [ 63.640325] nix-daemon[946]: accepted connection from pid 1026, user root (trusted)5062client # this derivation will be built:5063client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5064client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...5065worker2 # [ 63.770286] nix-grpc-daemon[787]: ts=2026-09-20T12:46:57Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:598685066worker2 # [ 63.776168] nix-grpc-daemon[787]: ts=2026-09-20T12:46:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:598685067client # copying 1 paths...5068client # copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' to 'grpc://lb:50051'...5069worker2 # [ 63.787493] nix-grpc-daemon[787]: ts=2026-09-20T12:46:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:598685070worker2 # [ 63.798836] nix-daemon[857]: accepted connection from pid 787, user nix-grpc-daemon (trusted)5071worker2 # [ 63.973802] nix-daemon[857]: accepted connection from pid 991, user nix-grpc-daemon (trusted)5072worker2 # [ 63.987162] nix-daemon[857]: reaped child process 994, status = succeeded5073lb # [ 64.027315] niks3-server[1131]: 2026/09/20 12:46:57 INFO Received uploads request method=POST path=/api/pending_closures5074worker2 # [ 64.019684] nix-grpc-daemon[907]: time=2026-09-20T12:46:57.992Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5075worker2 # [ 64.021134] nix-grpc-daemon[907]: time=2026-09-20T12:46:57.992Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5076lb # [ 64.065386] niks3-server[1131]: 2026/09/20 12:46:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign5077lb # [ 64.070698] niks3-server[1131]: 2026/09/20 12:46:57 INFO Signed narinfos id=17 count=15078worker2 # [ 64.051154] nix-grpc-daemon[907]: time=2026-09-20T12:46:58.023Z level=INFO msg="Uploading 1 narinfos"5079lb # [ 64.074153] niks3-server[1131]: 2026/09/20 12:46:57 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5080lb # [ 64.086518] niks3-server[1131]: 2026/09/20 12:46:57 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5081lb # [ 64.094212] niks3-server[1131]: 2026/09/20 12:46:57 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete5082lb # [ 64.104973] niks3-server[1131]: 2026/09/20 12:46:57 INFO Completed upload id=175083worker2 # [ 64.082246] nix-grpc-daemon[907]: time=2026-09-20T12:46:58.055Z level=INFO msg="Upload complete. (272ms)"5084lb # [ 64.109816] niks3-server[1131]: 2026/09/20 12:46:57 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5085worker2 # [ 64.086334] nix-grpc-daemon[787]: ts=2026-09-20T12:46:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59868 duration_s=0 paths=1 nar_bytes_in=1205086worker2 # [ 64.099608] nix-grpc-daemon[787]: ts=2026-09-20T12:46:58Z level=debug event=want client=9 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05087worker2 # [ 64.104190] nix-grpc-daemon[787]: ts=2026-09-20T12:46:58Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=15088worker2 # [ 64.111218] nix-daemon[857]: reaped child process 988, status = succeeded5089worker2 # [ 64.143434] nix-grpc-daemon[787]: ts=2026-09-20T12:46:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:598685090worker2 # [ 64.153795] nix-grpc-daemon[787]: ts=2026-09-20T12:46:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:598685091client # copying 2 paths to 'lb:50051'...5092client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5093worker2 # [ 64.166676] nix-grpc-daemon[787]: ts=2026-09-20T12:46:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:598685094client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5095worker2 # [ 64.180335] nix-daemon[857]: accepted connection from pid 787, user nix-grpc-daemon (trusted)5096worker2 # [ 64.334665] nix-daemon[857]: accepted connection from pid 1000, user nix-grpc-daemon (trusted)5097worker2 # [ 64.348248] nix-daemon[857]: reaped child process 1003, status = succeeded5098worker2 # [ 64.498665] nix-daemon[857]: accepted connection from pid 1005, user nix-grpc-daemon (trusted)5099worker2 # [ 64.510923] nix-daemon[857]: reaped child process 1008, status = succeeded5100lb # [ 64.547219] niks3-server[1131]: 2026/09/20 12:46:58 INFO Received uploads request method=POST path=/api/pending_closures5101lb # [ 64.579598] niks3-server[1131]: 2026/09/20 12:46:58 INFO Received uploads request method=POST path=/api/pending_closures5102worker2 # [ 64.570490] nix-grpc-daemon[907]: time=2026-09-20T12:46:58.542Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5103worker2 # [ 64.571920] nix-grpc-daemon[907]: time=2026-09-20T12:46:58.543Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5104worker2 # [ 64.573573] nix-grpc-daemon[907]: time=2026-09-20T12:46:58.543Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5105lb # [ 64.618740] niks3-server[1131]: 2026/09/20 12:46:58 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5106lb # [ 64.629147] niks3-server[1131]: 2026/09/20 12:46:58 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5107lb # [ 64.645503] niks3-server[1131]: 2026/09/20 12:46:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5108lb # [ 64.650577] niks3-server[1131]: 2026/09/20 12:46:58 INFO Signed narinfos id=19 count=25109lb # [ 64.651986] niks3-server[1131]: 2026/09/20 12:46:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5110lb # [ 64.654235] niks3-server[1131]: 2026/09/20 12:46:58 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5111lb # [ 64.655893] niks3-server[1131]: 2026/09/20 12:46:58 INFO Signed narinfos id=18 count=15112worker2 # [ 64.633622] nix-grpc-daemon[907]: time=2026-09-20T12:46:58.606Z level=INFO msg="Uploading 3 narinfos"5113lb # [ 64.657803] niks3-server[1131]: 2026/09/20 12:46:58 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5114lb # [ 64.669477] niks3-server[1131]: 2026/09/20 12:46:58 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5115lb # [ 64.674489] niks3-server[1131]: 2026/09/20 12:46:58 INFO Completed upload id=185116lb # [ 64.675812] niks3-server[1131]: 2026/09/20 12:46:58 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5117lb # [ 64.678350] niks3-server[1131]: 2026/09/20 12:46:58 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5118lb # [ 64.680823] niks3-server[1131]: 2026/09/20 12:46:58 INFO Completed upload id=195119worker2 # [ 64.659202] nix-grpc-daemon[907]: time=2026-09-20T12:46:58.632Z level=INFO msg="Upload complete. (465ms)"5120lb # [ 64.683153] niks3-server[1131]: 2026/09/20 12:46:58 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5121lb # [ 64.685592] niks3-server[1131]: 2026/09/20 12:46:58 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5122worker2 # [ 64.661699] nix-grpc-daemon[787]: ts=2026-09-20T12:46:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59868 duration_s=0 paths=2 nar_bytes_in=9765123worker2 # [ 64.668259] nix-grpc-daemon[787]: ts=2026-09-20T12:46:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:598685124worker2 # [ 64.679111] nix-daemon[857]: accepted connection from pid 787, user nix-grpc-daemon (trusted)5125worker2 # [ 64.688389] nix-daemon[857]: reaped child process 998, status = succeeded5126client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5127worker2 # [ 64.692717] nix-daemon[857]: accepted connection from pid 787, user nix-grpc-daemon (trusted)5128client # uses-input-w1onlya> worker2: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5129worker2 # [ 65.705790] nix-daemon[857]: reaped child process 1012, status = succeeded5130worker2 # [ 65.843237] nix-daemon[857]: accepted connection from pid 1027, user nix-grpc-daemon (trusted)5131worker2 # [ 65.855774] nix-daemon[857]: reaped child process 1030, status = succeeded5132lb # [ 65.891095] niks3-server[1131]: 2026/09/20 12:46:59 INFO Received uploads request method=POST path=/api/pending_closures5133worker2 # [ 65.872843] nix-grpc-daemon[907]: time=2026-09-20T12:46:59.845Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5134lb # [ 65.907448] niks3-server[1131]: 2026/09/20 12:46:59 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5135lb # [ 65.909714] niks3-server[1131]: 2026/09/20 12:46:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5136worker2 # [ 65.890025] nix-grpc-daemon[907]: time=2026-09-20T12:46:59.863Z level=INFO msg="Uploading 1 narinfos"5137lb # [ 65.912654] niks3-server[1131]: 2026/09/20 12:46:59 INFO Signed narinfos id=20 count=15138lb # [ 65.915547] niks3-server[1131]: 2026/09/20 12:46:59 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5139lb # [ 65.920821] niks3-server[1131]: 2026/09/20 12:46:59 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5140lb # [ 65.924842] niks3-server[1131]: 2026/09/20 12:46:59 INFO Completed upload id=205141worker2 # [ 65.903593] nix-grpc-daemon[907]: time=2026-09-20T12:46:59.876Z level=INFO msg="Upload complete. (204ms)"5142lb # [ 65.926932] niks3-server[1131]: 2026/09/20 12:46:59 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5143worker2 # [ 65.905900] nix-grpc-daemon[787]: ts=2026-09-20T12:46:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59868 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=9 outputs=15144worker2 # [ 65.912834] nix-daemon[857]: reaped child process 1010, status = succeeded5145client # copying 1 paths...5146worker2 # [ 65.950174] nix-grpc-daemon[787]: ts=2026-09-20T12:46:59Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:598685147client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5148worker2 # [ 65.964221] nix-grpc-daemon[787]: ts=2026-09-20T12:46:59Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:598685149worker2 # [ 65.967820] nix-grpc-daemon[787]: ts=2026-09-20T12:46:59Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:59868 duration_s=0 paths=1 nar_bytes_out=1205150client # [ 66.098288] nix-daemon[946]: reaped child process 1029, status = succeeded5151client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.66 seconds)5152client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&25153client # [ 66.272469] nix-daemon[946]: accepted connection from pid 1047, user root (trusted)5154client # this derivation will be built:5155client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5156client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...5157worker1 # [ 66.454928] nix-grpc-daemon[854]: ts=2026-09-20T12:46:59Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:396465158worker2 # [ 66.416358] nix-grpc-daemon[787]: ts=2026-09-20T12:47:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:598685159worker2 # [ 66.424565] nix-grpc-daemon[907]: time=2026-09-20T12:47:00.397Z level=INFO msg="All 1 paths already cached"5160worker2 # [ 66.436714] nix-grpc-daemon[787]: ts=2026-09-20T12:47:00Z level=debug event=want client=10 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05161worker2 # [ 66.441443] nix-grpc-daemon[787]: ts=2026-09-20T12:47:00Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15162worker1 # [ 66.484777] nix-grpc-daemon[854]: ts=2026-09-20T12:46:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:396465163worker1 # [ 66.496995] nix-grpc-daemon[854]: ts=2026-09-20T12:46:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:396465164client # copying 1 paths to 'lb:50051'...5165client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5166worker1 # [ 66.515957] nix-grpc-daemon[854]: ts=2026-09-20T12:46:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:396465167worker1 # [ 66.526959] nix-daemon[912]: accepted connection from pid 854, user nix-grpc-daemon (trusted)5168worker1 # [ 66.543191] nix-grpc-daemon[854]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 17 ms (attempt 1/5)5169worker1 # [ 66.565967] nix-grpc-daemon[854]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 165 ms (attempt 2/5)5170worker1 # [ 66.736920] nix-grpc-daemon[854]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 7 ms (attempt 3/5)5171worker1 # [ 66.749874] nix-grpc-daemon[854]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 313 ms (attempt 4/5)5172worker1 # [ 67.069916] nix-grpc-daemon[854]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5173worker1 # [ 67.073394] nix-grpc-daemon[854]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5174worker1 # [ 67.246878] nix-daemon[912]: accepted connection from pid 1219, user nix-grpc-daemon (trusted)5175worker1 # [ 67.261964] nix-daemon[912]: reaped child process 1222, status = succeeded5176lb # [ 67.312959] envoy[973]: conn peer=[2001:db8:1::1]:32776 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3893 tx=1851 ms=26285177lb # [ 67.316821] envoy[973]: conn peer=[2001:db8:1::1]:32786 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=378 ms=645178lb # [ 67.320436] envoy[973]: conn peer=[2001:db8:1::1]:32798 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4315 tx=2220 ms=22545179lb # [ 67.323948] envoy[973]: conn peer=[2001:db8:1::1]:32806 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=165180lb # [ 67.327373] envoy[973]: conn peer=[2001:db8:1::1]:56686 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4221 tx=1753 ms=22515181lb # [ 67.330775] envoy[973]: conn peer=[2001:db8:1::1]:56690 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=515182worker1 # [ 67.419502] nix-daemon[912]: accepted connection from pid 1224, user nix-grpc-daemon (trusted)5183worker1 # [ 67.433478] nix-daemon[912]: reaped child process 1227, status = succeeded5184lb # [ 67.431961] niks3-server[1131]: 2026/09/20 12:47:01 INFO Received uploads request method=POST path=/api/pending_closures5185worker1 # [ 67.458451] nix-grpc-daemon[923]: time=2026-09-20T12:47:00.884Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5186worker1 # [ 67.460466] nix-grpc-daemon[923]: time=2026-09-20T12:47:00.886Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5187lb # [ 67.464153] niks3-server[1131]: 2026/09/20 12:47:01 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5188lb # [ 67.470123] niks3-server[1131]: 2026/09/20 12:47:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5189lb # [ 67.474241] niks3-server[1131]: 2026/09/20 12:47:01 INFO Signed narinfos id=21 count=15190worker1 # [ 67.495322] nix-grpc-daemon[923]: time=2026-09-20T12:47:00.921Z level=INFO msg="Uploading 1 narinfos"5191lb # [ 67.484107] niks3-server[1131]: 2026/09/20 12:47:01 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5192lb # [ 67.493200] niks3-server[1131]: 2026/09/20 12:47:01 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5193lb # [ 67.500254] niks3-server[1131]: 2026/09/20 12:47:01 INFO Completed upload id=215194worker1 # [ 67.520946] nix-grpc-daemon[923]: time=2026-09-20T12:47:00.946Z level=INFO msg="Upload complete. (429ms)"5195worker1 # [ 67.524344] nix-grpc-daemon[854]: ts=2026-09-20T12:47:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39646 duration_s=1 paths=1 nar_bytes_in=5605196lb # [ 67.509620] niks3-server[1131]: 2026/09/20 12:47:01 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5197worker1 # [ 67.526858] nix-grpc-daemon[854]: ts=2026-09-20T12:47:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:396465198worker1 # [ 67.537733] nix-daemon[912]: accepted connection from pid 854, user nix-grpc-daemon (trusted)5199worker1 # [ 67.542386] nix-daemon[912]: reaped child process 1213, status = succeeded5200worker1 # [ 67.547514] nix-daemon[912]: accepted connection from pid 854, user nix-grpc-daemon (trusted)5201client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5202client # uses-input-w1onlyb> node-a: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5203worker1 # [ 68.777114] nix-daemon[912]: reaped child process 1231, status = succeeded5204worker1 # [ 68.936246] nix-daemon[912]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5205worker1 # [ 68.949661] nix-daemon[912]: reaped child process 1249, status = succeeded5206lb # [ 68.948613] niks3-server[1131]: 2026/09/20 12:47:02 INFO Received uploads request method=POST path=/api/pending_closures5207worker1 # [ 68.974856] nix-grpc-daemon[923]: time=2026-09-20T12:47:02.400Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5208lb # [ 68.982923] niks3-server[1131]: 2026/09/20 12:47:02 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5209lb # [ 68.989391] niks3-server[1131]: 2026/09/20 12:47:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5210lb # [ 68.994676] niks3-server[1131]: 2026/09/20 12:47:02 INFO Signed narinfos id=22 count=15211worker1 # [ 69.015951] nix-grpc-daemon[923]: time=2026-09-20T12:47:02.442Z level=INFO msg="Uploading 1 narinfos"5212lb # [ 69.010888] niks3-server[1131]: 2026/09/20 12:47:02 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5213lb # [ 69.015763] niks3-server[1131]: 2026/09/20 12:47:02 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5214worker1 # [ 69.038727] nix-grpc-daemon[923]: time=2026-09-20T12:47:02.464Z level=INFO msg="Upload complete. (283ms)"5215lb # [ 69.021114] niks3-server[1131]: 2026/09/20 12:47:02 INFO Completed upload id=225216worker1 # [ 69.041271] nix-grpc-daemon[854]: ts=2026-09-20T12:47:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39646 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=10 outputs=15217lb # [ 69.024530] niks3-server[1131]: 2026/09/20 12:47:02 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5218worker1 # [ 69.048638] nix-daemon[912]: reaped child process 1229, status = succeeded5219client # copying 1 paths...5220worker1 # [ 69.088071] nix-grpc-daemon[854]: ts=2026-09-20T12:47:02Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:396465221client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5222worker2 # [ 69.069531] nix-grpc-daemon[787]: ts=2026-09-20T12:47:03Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:598685223worker2 # [ 69.082429] nix-grpc-daemon[787]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5224worker2 # [ 69.096480] nix-grpc-daemon[787]: ts=2026-09-20T12:47:03Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:59868 duration_s=0 paths=1 nar_bytes_out=1205225client # [ 69.244638] nix-daemon[946]: reaped child process 1050, status = succeeded5226client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.14 seconds)5227client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&25228client # [ 69.394230] nix-daemon[946]: accepted connection from pid 1068, user root (trusted)5229client # this derivation will be built:5230client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5231worker1 # [ 69.542063] nix-grpc-daemon[854]: ts=2026-09-20T12:47:02Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:396465232client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...5233worker1 # [ 69.548832] nix-grpc-daemon[854]: ts=2026-09-20T12:47:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:396465234worker1 # [ 69.576621] nix-grpc-daemon[923]: time=2026-09-20T12:47:03.002Z level=INFO msg="All 1 paths already cached"5235worker2 # [ 69.549254] nix-grpc-daemon[787]: ts=2026-09-20T12:47:03Z level=debug event=want client=11 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05236worker2 # [ 69.553519] nix-grpc-daemon[787]: ts=2026-09-20T12:47:03Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15237worker2 # [ 69.602609] nix-grpc-daemon[787]: ts=2026-09-20T12:47:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:598685238worker2 # [ 69.613277] nix-grpc-daemon[787]: ts=2026-09-20T12:47:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:598685239worker2 # [ 69.623913] nix-grpc-daemon[907]: time=2026-09-20T12:47:03.596Z level=INFO msg="All 1 paths already cached"5240client # copying 1 paths to 'lb:50051'...5241client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5242worker2 # [ 69.631468] nix-grpc-daemon[787]: ts=2026-09-20T12:47:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:598685243worker2 # [ 69.640954] nix-daemon[857]: accepted connection from pid 787, user nix-grpc-daemon (trusted)5244worker2 # [ 69.800942] nix-daemon[857]: accepted connection from pid 1039, user nix-grpc-daemon (trusted)5245worker2 # [ 69.814938] nix-daemon[857]: reaped child process 1042, status = succeeded5246worker2 # [ 69.949230] nix-daemon[857]: accepted connection from pid 1044, user nix-grpc-daemon (trusted)5247worker2 # [ 69.962781] nix-daemon[857]: reaped child process 1047, status = succeeded5248lb # [ 69.994653] niks3-server[1131]: 2026/09/20 12:47:03 INFO Received uploads request method=POST path=/api/pending_closures5249worker2 # [ 69.979604] nix-grpc-daemon[907]: time=2026-09-20T12:47:03.952Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5250worker2 # [ 69.981144] nix-grpc-daemon[907]: time=2026-09-20T12:47:03.954Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5251lb # [ 70.024142] niks3-server[1131]: 2026/09/20 12:47:03 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5252lb # [ 70.035239] niks3-server[1131]: 2026/09/20 12:47:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5253worker2 # [ 70.017517] nix-grpc-daemon[907]: time=2026-09-20T12:47:03.990Z level=INFO msg="Uploading 1 narinfos"5254lb # [ 70.040400] niks3-server[1131]: 2026/09/20 12:47:03 INFO Signed narinfos id=23 count=15255lb # [ 70.046662] niks3-server[1131]: 2026/09/20 12:47:03 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5256lb # [ 70.057359] niks3-server[1131]: 2026/09/20 12:47:03 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5257worker2 # [ 70.041452] nix-grpc-daemon[907]: time=2026-09-20T12:47:04.014Z level=INFO msg="Upload complete. (385ms)"5258lb # [ 70.064350] niks3-server[1131]: 2026/09/20 12:47:03 INFO Completed upload id=235259worker2 # [ 70.043728] nix-grpc-daemon[787]: ts=2026-09-20T12:47:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59868 duration_s=0 paths=1 nar_bytes_in=5605260lb # [ 70.067464] niks3-server[1131]: 2026/09/20 12:47:03 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5261worker2 # [ 70.048435] nix-grpc-daemon[787]: ts=2026-09-20T12:47:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:598685262worker2 # [ 70.054988] nix-daemon[857]: accepted connection from pid 787, user nix-grpc-daemon (trusted)5263worker2 # [ 70.057254] nix-daemon[857]: reaped child process 1037, status = succeeded5264worker2 # [ 70.062239] nix-daemon[857]: accepted connection from pid 787, user nix-grpc-daemon (trusted)5265client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5266client # uses-input-w1onlyc> worker2: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5267worker2 # [ 70.673132] nix-daemon[857]: reaped child process 1051, status = succeeded5268worker2 # [ 70.814634] nix-daemon[857]: accepted connection from pid 1066, user nix-grpc-daemon (trusted)5269worker2 # [ 70.825793] nix-daemon[857]: reaped child process 1069, status = succeeded5270lb # [ 70.863156] niks3-server[1131]: 2026/09/20 12:47:04 INFO Received uploads request method=POST path=/api/pending_closures5271worker2 # [ 71.009536] nix-grpc-daemon[907]: time=2026-09-20T12:47:04.981Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5272lb # [ 71.059768] niks3-server[1131]: 2026/09/20 12:47:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5273lb # [ 71.066864] niks3-server[1131]: 2026/09/20 12:47:04 INFO Signed narinfos id=24 count=15274worker2 # [ 71.043847] nix-grpc-daemon[907]: time=2026-09-20T12:47:05.016Z level=INFO msg="Uploading 1 narinfos"5275lb # [ 71.079204] niks3-server[1131]: 2026/09/20 12:47:04 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5276lb # [ 71.196173] niks3-server[1131]: 2026/09/20 12:47:04 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5277lb # [ 71.203641] niks3-server[1131]: 2026/09/20 12:47:05 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5278lb # [ 71.214893] niks3-server[1131]: 2026/09/20 12:47:05 INFO Completed upload id=245279worker2 # [ 71.192388] nix-grpc-daemon[907]: time=2026-09-20T12:47:05.165Z level=INFO msg="Upload complete. (524ms)"5280lb # [ 71.223195] niks3-server[1131]: 2026/09/20 12:47:05 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5281worker2 # [ 71.199623] nix-grpc-daemon[787]: ts=2026-09-20T12:47:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59868 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=11 outputs=15282worker2 # [ 71.213685] nix-daemon[857]: reaped child process 1049, status = succeeded5283client # copying 1 paths...5284worker2 # [ 71.243777] nix-grpc-daemon[787]: ts=2026-09-20T12:47:05Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:598685285client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5286worker2 # [ 71.263243] nix-grpc-daemon[787]: ts=2026-09-20T12:47:05Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:598685287worker2 # [ 71.267673] nix-grpc-daemon[787]: ts=2026-09-20T12:47:05Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:59868 duration_s=0 paths=1 nar_bytes_out=1205288client # [ 71.418491] nix-daemon[946]: reaped child process 1071, status = succeeded5289client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.17 seconds)5290(finished: subtest: an input only one worker has still reaches the builder, in 8.80 seconds)5291subtest: upload whose reference the worker lacks is completed from the cache5292client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache5293client # this derivation will be built:5294client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5295client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5296client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache, in 0.40 seconds)5297client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache5298client # this derivation will be built:5299client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5300client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5301client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache, in 0.39 seconds)5302client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5303client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.20 seconds)5304worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5305worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.22 seconds)5306worker2 # [ 72.654308] systemd[1]: Stopping Nix worker-protocol over gRPC...5307worker2 # [ 73.638268] nix-grpc-daemon[787]: ts=2026-09-20T12:47:07Z level=info event=signal_exit5308worker2 # [ 73.888443] nix-grpc-daemon[787]: ts=2026-09-20T12:47:07Z level=info event=scheduler_yield5309worker2 # [ 73.892604] nix-grpc-daemon[787]: ts=2026-09-20T12:47:07Z level=info event=scheduler_restarting addr=lb:500515310lb # [ 73.914922] niks3-server[1131]: 2026/09/20 12:47:07 INFO lead: released remote=192.168.1.4:485145311worker2 # [ 73.898370] nix-grpc-daemon[787]: ts=2026-09-20T12:47:07Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:496325312worker1 # [ 73.940901] nix-grpc-daemon[854]: ts=2026-09-20T12:47:07Z level=info event=scheduler_restarting addr=lb:500515313worker2 # [ 73.903096] nix-grpc-daemon[787]: ts=2026-09-20T12:47:07Z level=info event=worker_gone addr=192.168.1.4:500515314worker2 # [ 73.906061] nix-grpc-daemon[787]: ts=2026-09-20T12:47:07Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:496325315worker2 # [ 73.909268] nix-grpc-daemon[787]: ts=2026-09-20T12:47:07Z level=info event=worker_gone addr=192.168.1.3:500515316worker2 # [ 74.093232] nix-grpc-daemon[787]: ts=2026-09-20T12:47:08Z level=info event=scheduler_connected addr=lb:500515317worker2 # [ 74.095092] nix-grpc-daemon[787]: ts=2026-09-20T12:47:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5318worker1 # [ 74.141312] nix-grpc-daemon[854]: ts=2026-09-20T12:47:07Z level=info event=scheduler_connected addr=lb:500515319worker1 # [ 74.145207] nix-grpc-daemon[854]: ts=2026-09-20T12:47:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5320worker2 # [ 74.296200] nix-grpc-daemon[787]: ts=2026-09-20T12:47:08Z level=info event=scheduler_connected addr=lb:500515321worker2 # [ 74.299625] nix-grpc-daemon[787]: ts=2026-09-20T12:47:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5322worker1 # [ 74.345968] nix-grpc-daemon[854]: ts=2026-09-20T12:47:07Z level=info event=scheduler_connected addr=lb:500515323worker1 # [ 74.349972] nix-grpc-daemon[854]: ts=2026-09-20T12:47:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5324worker2 # [ 74.500319] nix-grpc-daemon[787]: ts=2026-09-20T12:47:08Z level=info event=scheduler_connected addr=lb:500515325worker2 # [ 74.502120] nix-grpc-daemon[787]: ts=2026-09-20T12:47:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5326worker1 # [ 74.550802] nix-grpc-daemon[854]: ts=2026-09-20T12:47:07Z level=info event=scheduler_connected addr=lb:500515327worker1 # [ 74.554874] nix-grpc-daemon[854]: ts=2026-09-20T12:47:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5328worker2 # [ 74.703174] nix-grpc-daemon[787]: ts=2026-09-20T12:47:08Z level=info event=scheduler_connected addr=lb:500515329worker2 # [ 74.704896] nix-grpc-daemon[787]: ts=2026-09-20T12:47:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5330worker1 # [ 74.755752] nix-grpc-daemon[854]: ts=2026-09-20T12:47:08Z level=info event=scheduler_connected addr=lb:500515331worker1 # [ 74.759797] nix-grpc-daemon[854]: ts=2026-09-20T12:47:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5332worker2 # [ 74.892687] nix-grpc-daemon[787]: ts=2026-09-20T12:47:08Z level=info event=scheduler_restarting5333worker2 # [ 74.916681] nix-grpc-daemon[787]: ts=2026-09-20T12:47:08Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5334worker1 # [ 74.962233] nix-grpc-daemon[854]: ts=2026-09-20T12:47:08Z level=info event=scheduler_connected addr=lb:500515335worker1 # [ 74.965513] nix-grpc-daemon[854]: ts=2026-09-20T12:47:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5336worker2 # [ 74.931509] nix-daemon[857]: reaped child process 859, status = succeeded5337worker1 # [ 75.163305] nix-grpc-daemon[854]: ts=2026-09-20T12:47:08Z level=info event=scheduler_connected addr=lb:500515338worker1 # [ 75.167393] nix-grpc-daemon[854]: ts=2026-09-20T12:47:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5339worker1 # [ 75.368535] nix-grpc-daemon[854]: ts=2026-09-20T12:47:08Z level=info event=scheduler_connected addr=lb:500515340worker1 # [ 75.372531] nix-grpc-daemon[854]: ts=2026-09-20T12:47:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5341worker1 # [ 75.573527] nix-grpc-daemon[854]: ts=2026-09-20T12:47:08Z level=info event=scheduler_connected addr=lb:500515342worker1 # [ 75.577630] nix-grpc-daemon[854]: ts=2026-09-20T12:47:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5343worker1 # [ 75.778191] nix-grpc-daemon[854]: ts=2026-09-20T12:47:09Z level=info event=scheduler_connected addr=lb:500515344worker1 # [ 75.781950] nix-grpc-daemon[854]: ts=2026-09-20T12:47:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5345worker2 # [ 75.776334] nix-grpc-daemon[787]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5346worker2 # [ 75.779799] nix-grpc-daemon[787]: E0000 00:00:1789908429.747871 830 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5347worker2 # [ 75.784484] nix-grpc-daemon[787]: E0000 00:00:1789908429.748538 830 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5348worker1 # [ 75.984359] nix-grpc-daemon[854]: ts=2026-09-20T12:47:09Z level=info event=scheduler_connected addr=lb:500515349worker1 # [ 75.987675] nix-grpc-daemon[854]: ts=2026-09-20T12:47:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5350lb # [ 76.150661] niks3-server[1131]: 2026/09/20 12:47:09 INFO lead: acquired remote=192.168.1.3:409505351worker1 # [ 76.172319] nix-grpc-daemon[854]: ts=2026-09-20T12:47:09Z level=info event=scheduler_take_over5352worker1 # [ 76.185188] nix-grpc-daemon[854]: ts=2026-09-20T12:47:09Z level=info event=scheduler_connected addr=lb:500515353worker1 # [ 76.188930] nix-grpc-daemon[854]: ts=2026-09-20T12:47:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5354worker1 # [ 76.389902] nix-grpc-daemon[854]: ts=2026-09-20T12:47:09Z level=info event=scheduler_connected addr=lb:500515355worker1 # [ 76.394119] nix-grpc-daemon[854]: ts=2026-09-20T12:47:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5356worker2 # [ 76.505570] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5357worker2 # [ 76.511171] systemd[1]: Stopped Nix worker-protocol over gRPC.5358worker2 # [ 76.513273] systemd[1]: nix-grpc-daemon.service: Consumed 1.501s CPU time over 59.102s wall clock time, 50.2M memory peak, 61.5K incoming IP traffic, 163.1K outgoing IP traffic.5359worker2 # [ 76.523756] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5360worker2 # [ 76.528215] systemd[1]: Closed Nix worker-protocol over gRPC.5361worker2 # [ 76.531886] systemd[1]: nix-grpc-daemon.socket: Consumed 289us CPU time over 1min 7.006s wall clock time, 256K memory peak, 75.5K incoming IP traffic, 62.9K outgoing IP traffic.5362lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5363lb: must succeed: curl -sf localhost:9901/clusters5364worker1 # [ 76.594971] nix-grpc-daemon[854]: ts=2026-09-20T12:47:10Z level=info event=scheduler_connected addr=lb:500515365worker1 # [ 76.598987] nix-grpc-daemon[854]: ts=2026-09-20T12:47:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5366lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5367lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.08 seconds)5368client: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25369worker1 # [ 76.799439] nix-grpc-daemon[854]: ts=2026-09-20T12:47:10Z level=info event=scheduler_connected addr=lb:500515370worker1 # [ 76.810672] nix-grpc-daemon[854]: ts=2026-09-20T12:47:10Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:505685371worker1 # [ 76.815329] nix-grpc-daemon[854]: ts=2026-09-20T12:47:10Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05372worker1 # [ 76.855222] nix-grpc-daemon[854]: ts=2026-09-20T12:47:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:396465373worker1 # [ 76.862361] nix-grpc-daemon[854]: ts=2026-09-20T12:47:10Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:396465374client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5375client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.24 seconds)5376worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5377worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.03 seconds)5378worker2 # [ 76.931783] systemd[1]: Listening on Nix worker-protocol over gRPC.5379worker1 # [ 77.035362] systemd[1]: Stopping Nix worker-protocol over gRPC...5380lb # [ 77.313662] envoy[973]: conn peer=[2001:db8:1::1]:56702 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3713 tx=1804 ms=27375381lb # [ 77.315586] envoy[973]: conn peer=[2001:db8:1::1]:56710 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=329 ms=825382lb # [ 77.317229] envoy[973]: conn peer=[2001:db8:1::1]:56712 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3744 tx=1796 ms=18165383lb # [ 77.318939] envoy[973]: conn peer=[2001:db8:1::1]:56722 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=585384lb # [ 77.320612] envoy[973]: conn peer=[2001:db8:1::4]:37672 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=62031 tx=2923 ms=572665385lb # [ 77.322402] envoy[973]: conn peer=[2001:db8:1::1]:48196 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=215386worker2 # [ 77.495984] systemd[1]: Starting scheduler-token.service...5387worker2 # [ 77.598480] systemd[1]: scheduler-token.service: Deactivated successfully.5388worker2 # [ 77.601298] systemd[1]: Finished scheduler-token.service.5389worker2 # [ 77.602703] systemd[1]: scheduler-token.service: Consumed 36ms CPU time over 102ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5390worker1 # [ 77.644851] nix-grpc-daemon[854]: ts=2026-09-20T12:47:11Z level=info event=signal_exit5391worker2 # [ 77.607803] systemd[1]: Starting Nix worker-protocol over gRPC...5392worker2 # [ 77.765471] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:11Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5393worker2 # [ 77.767275] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:11Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5394worker2 # [ 77.770582] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:11Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15395worker2 # [ 77.776970] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:11Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5396worker2 # [ 77.779921] systemd[1]: Started Nix worker-protocol over gRPC.5397worker2 # [ 77.796333] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:11Z level=info event=scheduler_connected addr=lb:500515398worker2 # [ 77.797968] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5399worker1 # [ 77.905692] nix-grpc-daemon[854]: ts=2026-09-20T12:47:11Z level=info event=scheduler_yield5400lb # [ 77.889876] niks3-server[1131]: 2026/09/20 12:47:11 INFO lead: released remote=192.168.1.3:409505401worker1 # [ 77.912204] nix-grpc-daemon[854]: ts=2026-09-20T12:47:11Z level=info event=scheduler_restarting addr=lb:500515402worker1 # [ 77.916629] nix-grpc-daemon[854]: ts=2026-09-20T12:47:11Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:505685403worker1 # [ 77.919970] nix-grpc-daemon[854]: ts=2026-09-20T12:47:11Z level=info event=worker_gone addr=192.168.1.3:500515404worker2 # [ 77.999238] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:11Z level=info event=scheduler_connected addr=lb:500515405worker2 # [ 78.003138] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5406worker1 # [ 78.113376] nix-grpc-daemon[854]: ts=2026-09-20T12:47:11Z level=info event=scheduler_connected addr=lb:500515407worker1 # [ 78.117426] nix-grpc-daemon[854]: ts=2026-09-20T12:47:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5408worker2 # [ 78.204145] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:12Z level=info event=scheduler_connected addr=lb:500515409worker2 # [ 78.207951] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5410worker1 # [ 78.318407] nix-grpc-daemon[854]: ts=2026-09-20T12:47:11Z level=info event=scheduler_connected addr=lb:500515411worker1 # [ 78.322318] nix-grpc-daemon[854]: ts=2026-09-20T12:47:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5412worker2 # [ 78.408916] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:12Z level=info event=scheduler_connected addr=lb:500515413worker2 # [ 78.412799] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5414worker1 # [ 78.523139] nix-grpc-daemon[854]: ts=2026-09-20T12:47:11Z level=info event=scheduler_connected addr=lb:500515415worker1 # [ 78.527157] nix-grpc-daemon[854]: ts=2026-09-20T12:47:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5416worker2 # [ 78.613900] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:12Z level=info event=scheduler_connected addr=lb:500515417worker2 # [ 78.617674] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5418worker1 # [ 78.728823] nix-grpc-daemon[854]: ts=2026-09-20T12:47:12Z level=info event=scheduler_connected addr=lb:500515419worker1 # [ 78.732654] nix-grpc-daemon[854]: ts=2026-09-20T12:47:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5420worker2 # [ 78.819748] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:12Z level=info event=scheduler_connected addr=lb:500515421worker2 # [ 78.822856] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5422worker1 # [ 78.907177] nix-grpc-daemon[854]: ts=2026-09-20T12:47:12Z level=info event=scheduler_restarting5423worker1 # [ 78.948228] nix-daemon[912]: reaped child process 914, status = succeeded5424worker1 # [ 78.950568] nix-grpc-daemon[854]: ts=2026-09-20T12:47:12Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5425worker2 # [ 79.021429] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:12Z level=info event=scheduler_connected addr=lb:500515426worker2 # [ 79.025256] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5427worker2 # [ 79.226220] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:13Z level=info event=scheduler_connected addr=lb:500515428worker2 # [ 79.230087] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5429worker2 # [ 79.431157] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:13Z level=info event=scheduler_connected addr=lb:500515430worker2 # [ 79.435193] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5431worker2 # [ 79.636276] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:13Z level=info event=scheduler_connected addr=lb:500515432worker2 # [ 79.640085] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5433worker1 # [ 79.783659] nix-grpc-daemon[854]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5434worker1 # [ 79.787246] nix-grpc-daemon[854]: E0000 00:00:1789908433.208350 897 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5435worker1 # [ 79.791902] nix-grpc-daemon[854]: E0000 00:00:1789908433.208964 897 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5436worker2 # [ 79.841716] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:13Z level=info event=scheduler_connected addr=lb:500515437worker2 # [ 79.845577] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5438worker2 # [ 80.043699] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:14Z level=info event=scheduler_connected addr=lb:500515439worker2 # [ 80.047911] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5440worker2 # [ 80.248853] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:14Z level=info event=scheduler_connected addr=lb:500515441worker2 # [ 80.252772] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5442worker1 # [ 80.488923] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5443worker1 # [ 80.492891] systemd[1]: Stopped Nix worker-protocol over gRPC.5444worker2 # [ 80.454116] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:14Z level=info event=scheduler_connected addr=lb:500515445worker1 # [ 80.496956] systemd[1]: nix-grpc-daemon.service: Consumed 2.731s CPU time over 1min 3.166s wall clock time, 58.7M memory peak, 86.8K incoming IP traffic, 163.4K outgoing IP traffic.5446worker2 # [ 80.458094] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5447worker1 # [ 80.508223] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5448worker1 # [ 80.511585] systemd[1]: Closed Nix worker-protocol over gRPC.5449worker1 # [ 80.514486] systemd[1]: nix-grpc-daemon.socket: Consumed 400us CPU time over 1min 11.115s wall clock time, 256K memory peak, 82.7K incoming IP traffic, 67.8K outgoing IP traffic.5450lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5451lb: must succeed: curl -sf localhost:9901/clusters5452lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5453lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.07 seconds)5454client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25455worker2 # [ 80.659137] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:14Z level=info event=scheduler_connected addr=lb:500515456worker2 # [ 80.663098] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5457worker2 # [ 80.742947] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:430925458worker2 # [ 80.754804] nix-daemon[857]: accepted connection from pid 1093, user nix-grpc-daemon (trusted)5459worker2 # [ 80.771189] nix-grpc-daemon[1093]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 86 ms (attempt 1/5)5460worker2 # [ 80.865591] nix-grpc-daemon[1093]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 154 ms (attempt 2/5)5461worker2 # [ 80.870845] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:14Z level=info event=scheduler_connected addr=lb:500515462worker2 # [ 80.873897] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5463worker2 # [ 81.020371] nix-grpc-daemon[1093]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 48 ms (attempt 3/5)5464worker2 # [ 81.066758] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:15Z level=info event=scheduler_connected addr=lb:500515465worker2 # [ 81.071315] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5466worker2 # [ 81.074952] nix-grpc-daemon[1093]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 712 ms (attempt 4/5)5467worker2 # [ 81.271931] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:15Z level=info event=scheduler_connected addr=lb:500515468worker2 # [ 81.275930] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5469worker2 # [ 81.477120] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:15Z level=info event=scheduler_connected addr=lb:500515470worker2 # [ 81.481097] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5471worker2 # [ 81.682342] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:15Z level=info event=scheduler_connected addr=lb:500515472worker2 # [ 81.686353] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5473worker2 # [ 81.784425] nix-grpc-daemon[1093]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5474client # copying 2 paths...5475client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5476client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5477worker2 # [ 81.806735] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:430925478worker2 # [ 81.816757] nix-daemon[857]: accepted connection from pid 1093, user nix-grpc-daemon (trusted)5479worker2 # [ 81.838857] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:15Z level=info event=niks3_push_started pid=11495480worker2 # [ 81.888713] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:15Z level=info event=scheduler_connected addr=lb:500515481worker2 # [ 81.890835] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5482worker2 # [ 82.057276] nix-grpc-daemon[1149]: time=2026-09-20T12:47:16.029Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""5483worker2 # [ 82.091276] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:16Z level=info event=scheduler_connected addr=lb:500515484worker2 # [ 82.093163] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5485worker2 # [ 82.231291] nix-daemon[857]: accepted connection from pid 1162, user nix-grpc-daemon (trusted)5486worker2 # [ 82.244962] nix-daemon[857]: reaped child process 1165, status = succeeded5487lb # [ 82.285348] niks3-server[1131]: 2026/09/20 12:47:16 INFO Received uploads request method=POST path=/api/pending_closures5488lb # [ 82.293911] niks3-server[1131]: 2026/09/20 12:47:16 INFO Received uploads request method=POST path=/api/pending_closures5489worker2 # [ 82.278960] nix-grpc-daemon[1149]: time=2026-09-20T12:47:16.251Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5490worker2 # [ 82.280614] nix-grpc-daemon[1149]: time=2026-09-20T12:47:16.253Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5491worker2 # [ 82.282560] nix-grpc-daemon[1149]: time=2026-09-20T12:47:16.255Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5492worker2 # [ 82.293670] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:16Z level=info event=scheduler_connected addr=lb:500515493worker2 # [ 82.295256] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5494lb # [ 82.344945] niks3-server[1131]: 2026/09/20 12:47:16 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5495lb # [ 82.353147] niks3-server[1131]: 2026/09/20 12:47:16 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5496lb # [ 82.368932] niks3-server[1131]: 2026/09/20 12:47:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5497lb # [ 82.372280] niks3-server[1131]: 2026/09/20 12:47:16 INFO Signed narinfos id=25 count=15498lb # [ 82.374134] niks3-server[1131]: 2026/09/20 12:47:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5499lb # [ 82.376106] niks3-server[1131]: 2026/09/20 12:47:16 INFO Signed narinfos id=26 count=25500lb # [ 82.378073] niks3-server[1131]: 2026/09/20 12:47:16 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5501worker2 # [ 82.356123] nix-grpc-daemon[1149]: time=2026-09-20T12:47:16.327Z level=INFO msg="Uploading 3 narinfos"5502lb # [ 82.389677] niks3-server[1131]: 2026/09/20 12:47:16 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5503lb # [ 82.397344] niks3-server[1131]: 2026/09/20 12:47:16 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5504lb # [ 82.401612] niks3-server[1131]: 2026/09/20 12:47:16 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5505lb # [ 82.406260] niks3-server[1131]: 2026/09/20 12:47:16 INFO Completed upload id=255506lb # [ 82.407698] niks3-server[1131]: 2026/09/20 12:47:16 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5507lb # [ 82.411170] niks3-server[1131]: 2026/09/20 12:47:16 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5508lb # [ 82.414855] niks3-server[1131]: 2026/09/20 12:47:16 INFO Completed upload id=265509worker2 # [ 82.392339] nix-grpc-daemon[1149]: time=2026-09-20T12:47:16.365Z level=INFO msg="Upload complete. (331ms)"5510worker2 # [ 82.397438] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43092 duration_s=0 paths=2 nar_bytes_in=3125511lb # [ 82.425114] niks3-server[1131]: 2026/09/20 12:47:16 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5512worker2 # [ 82.410340] nix-daemon[857]: reaped child process 1147, status = succeeded5513client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.88 seconds)5514worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5515worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.03 seconds)5516worker2 # [ 82.496838] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:16Z level=info event=scheduler_connected addr=lb:500515517worker2 # [ 82.501209] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5518worker1 # [ 82.560508] systemd[1]: Listening on Nix worker-protocol over gRPC.5519worker1 # [ 82.568411] systemd[1]: Starting Nix worker-protocol over gRPC...5520worker1 # [ 82.710355] nix-grpc-daemon[1272]: ts=2026-09-20T12:47:16Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5521worker1 # [ 82.712265] nix-grpc-daemon[1272]: ts=2026-09-20T12:47:16Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5522worker1 # [ 82.715796] nix-grpc-daemon[1272]: ts=2026-09-20T12:47:16Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15523worker1 # [ 82.722685] nix-grpc-daemon[1272]: ts=2026-09-20T12:47:16Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5524worker1 # [ 82.726648] systemd[1]: Started Nix worker-protocol over gRPC.5525lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5526lb: must succeed: curl -sf localhost:9901/clusters5527lb # [ 82.715136] niks3-server[1131]: 2026/09/20 12:47:16 INFO lead: acquired remote=192.168.1.3:566765528worker1 # [ 82.736662] nix-grpc-daemon[1272]: ts=2026-09-20T12:47:16Z level=info event=scheduler_take_over5529worker2 # [ 82.702678] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:16Z level=info event=scheduler_connected addr=lb:500515530worker1 # [ 82.749401] nix-grpc-daemon[1272]: ts=2026-09-20T12:47:16Z level=info event=scheduler_connected addr=lb:500515531worker2 # [ 82.707278] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5532worker1 # [ 82.751064] nix-grpc-daemon[1272]: ts=2026-09-20T12:47:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5533lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5534worker2 # [ 82.909487] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:16Z level=info event=scheduler_connected addr=lb:500515535worker1 # [ 82.952264] nix-grpc-daemon[1272]: ts=2026-09-20T12:47:16Z level=info event=scheduler_connected addr=lb:500515536worker2 # [ 82.913293] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5537worker1 # [ 82.956239] nix-grpc-daemon[1272]: ts=2026-09-20T12:47:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5538worker2 # [ 83.114257] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:17Z level=info event=scheduler_connected addr=lb:500515539worker1 # [ 83.157085] nix-grpc-daemon[1272]: ts=2026-09-20T12:47:16Z level=info event=scheduler_connected addr=lb:500515540worker2 # [ 83.118319] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5541worker1 # [ 83.160917] nix-grpc-daemon[1272]: ts=2026-09-20T12:47:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5542worker1 # [ 83.361700] nix-grpc-daemon[1272]: ts=2026-09-20T12:47:16Z level=info event=scheduler_connected addr=lb:500515543worker2 # [ 83.319405] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:17Z level=info event=scheduler_connected addr=lb:500515544worker1 # [ 83.363736] nix-grpc-daemon[1272]: ts=2026-09-20T12:47:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5545worker2 # [ 83.323328] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5546worker1 # [ 83.564377] nix-grpc-daemon[1272]: ts=2026-09-20T12:47:16Z level=info event=scheduler_connected addr=lb:500515547worker2 # [ 83.524596] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:17Z level=info event=scheduler_connected addr=lb:500515548worker1 # [ 83.568235] nix-grpc-daemon[1272]: ts=2026-09-20T12:47:16Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:462365549worker1 # [ 83.570036] nix-grpc-daemon[1272]: ts=2026-09-20T12:47:16Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05550worker1 # [ 83.574892] nix-grpc-daemon[1272]: ts=2026-09-20T12:47:17Z level=info event=oidc_jwks provider=mock keys=15551worker1 # [ 83.576241] nix-grpc-daemon[1272]: ts=2026-09-20T12:47:17Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:462365552worker1 # [ 83.577882] nix-grpc-daemon[1272]: ts=2026-09-20T12:47:17Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05553lb: must succeed: curl -sf localhost:9901/clusters5554lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5555lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.14 seconds)5556(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 12.45 seconds)5557subtest: scheduler down: the other node takes the lock and keeps it5558lb: must succeed: curl -sf localhost:9901/clusters5559lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5560lb: must succeed: curl -sf localhost:9901/clusters5561lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5562lb: must succeed: curl -sf localhost:9901/clusters5563lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5564lb: must succeed: curl -sf localhost:9901/clusters5565lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5566worker1 # [ 84.081565] systemd[1]: Stopping Nix worker-protocol over gRPC...5567worker1 # [ 84.727131] nix-grpc-daemon[1272]: ts=2026-09-20T12:47:18Z level=info event=signal_exit5568worker1 # [ 84.933692] nix-grpc-daemon[1272]: ts=2026-09-20T12:47:18Z level=info event=scheduler_yield5569lb # [ 84.917772] niks3-server[1131]: 2026/09/20 12:47:18 INFO lead: released remote=192.168.1.3:566765570worker2 # [ 84.900539] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:18Z level=info event=scheduler_restarting addr=lb:500515571worker1 # [ 84.942135] nix-grpc-daemon[1272]: ts=2026-09-20T12:47:18Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:462365572worker1 # [ 84.946752] nix-grpc-daemon[1272]: ts=2026-09-20T12:47:18Z level=info event=worker_gone addr=192.168.1.4:500515573worker1 # [ 84.949947] nix-grpc-daemon[1272]: ts=2026-09-20T12:47:18Z level=info event=scheduler_restarting addr=lb:500515574worker1 # [ 84.952943] nix-grpc-daemon[1272]: ts=2026-09-20T12:47:18Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:462365575worker1 # [ 84.956275] nix-grpc-daemon[1272]: ts=2026-09-20T12:47:18Z level=info event=worker_gone addr=192.168.1.3:500515576worker2 # [ 85.100781] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:19Z level=info event=scheduler_connected addr=lb:500515577worker2 # [ 85.104756] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:19Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5578worker1 # [ 85.146894] nix-grpc-daemon[1272]: ts=2026-09-20T12:47:18Z level=info event=scheduler_connected addr=lb:500515579worker1 # [ 85.151721] nix-grpc-daemon[1272]: ts=2026-09-20T12:47:18Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5580worker2 # [ 85.305954] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:19Z level=info event=scheduler_connected addr=lb:500515581worker2 # [ 85.309981] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:19Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5582worker1 # [ 85.352597] nix-grpc-daemon[1272]: ts=2026-09-20T12:47:18Z level=info event=scheduler_connected addr=lb:500515583worker1 # [ 85.357520] nix-grpc-daemon[1272]: ts=2026-09-20T12:47:18Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5584worker2 # [ 85.511070] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:19Z level=info event=scheduler_connected addr=lb:500515585worker2 # [ 85.514985] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5586worker1 # [ 85.558608] nix-grpc-daemon[1272]: ts=2026-09-20T12:47:18Z level=info event=scheduler_connected addr=lb:500515587worker1 # [ 85.562754] nix-grpc-daemon[1272]: ts=2026-09-20T12:47:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5588worker2 # [ 85.715682] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:19Z level=info event=scheduler_connected addr=lb:500515589worker2 # [ 85.717674] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5590worker1 # [ 85.763903] nix-grpc-daemon[1272]: ts=2026-09-20T12:47:19Z level=info event=scheduler_connected addr=lb:500515591worker1 # [ 85.765942] nix-grpc-daemon[1272]: ts=2026-09-20T12:47:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5592worker1 # [ 85.937685] nix-grpc-daemon[1272]: ts=2026-09-20T12:47:19Z level=info event=scheduler_restarting5593worker2 # [ 85.918825] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:19Z level=info event=scheduler_connected addr=lb:500515594worker2 # [ 85.920509] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5595worker2 # [ 86.121265] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:20Z level=info event=scheduler_connected addr=lb:500515596worker2 # [ 86.122978] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5597worker2 # [ 86.323732] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:20Z level=info event=scheduler_connected addr=lb:500515598worker2 # [ 86.325468] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5599worker1 # [ 86.481801] nix-grpc-daemon[1272]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5600worker1 # [ 86.483486] nix-grpc-daemon[1272]: E0000 00:00:1789908439.907674 1315 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true5601worker1 # [ 86.485436] nix-grpc-daemon[1272]: E0000 00:00:1789908439.909460 1315 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5602worker2 # [ 86.526174] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:20Z level=info event=scheduler_connected addr=lb:500515603worker2 # [ 86.527825] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5604worker1 # [ 86.753348] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5605worker1 # [ 86.756317] systemd[1]: Stopped Nix worker-protocol over gRPC.5606worker1 # [ 86.758340] systemd[1]: nix-grpc-daemon.service: Consumed 130ms CPU time over 4.183s wall clock time, 10.1M memory peak, 10.1K incoming IP traffic, 10K outgoing IP traffic.5607worker2 # [ 86.728583] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:20Z level=info event=scheduler_connected addr=lb:500515608worker1 # [ 86.769806] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5609worker2 # [ 86.730375] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5610worker1 # [ 86.774286] systemd[1]: Closed Nix worker-protocol over gRPC.5611worker1 # [ 86.777478] systemd[1]: nix-grpc-daemon.socket: Consumed 546us CPU time over 4.220s wall clock time, 256K memory peak, 16.6K incoming IP traffic, 11.5K outgoing IP traffic.5612worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5613worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.07 seconds)5614worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5615worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5616worker2 # [ 86.932404] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:20Z level=info event=scheduler_connected addr=lb:500515617worker2 # [ 86.936176] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5618worker2 # [ 87.137219] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:21Z level=info event=scheduler_connected addr=lb:500515619worker2 # [ 87.140969] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5620lb # [ 87.314540] envoy[973]: conn peer=192.168.1.3:46114 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=18092 tx=3609 ms=603545621lb # [ 87.318516] envoy[973]: conn peer=[2001:db8:1::1]:48208 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1007 tx=305 ms=16675622lb # [ 87.323589] envoy[973]: conn peer=[2001:db8:1::3]:59760 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2144 tx=513 ms=32275623worker2 # [ 87.342260] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:21Z level=info event=scheduler_connected addr=lb:500515624worker2 # [ 87.346122] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5625worker2 # [ 87.547207] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:21Z level=info event=scheduler_connected addr=lb:500515626worker2 # [ 87.551088] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5627worker2 # [ 87.752091] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:21Z level=info event=scheduler_connected addr=lb:500515628worker2 # [ 87.755943] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5629worker2 # [ 87.788242] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:21Z level=info event=scheduler_take_over5630lb # [ 87.811463] niks3-server[1131]: 2026/09/20 12:47:21 INFO lead: acquired remote=192.168.1.4:332345631worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5632worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5633worker2 # [ 87.957385] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:21Z level=info event=scheduler_connected addr=lb:500515634worker2 # [ 87.976144] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:21Z level=info event=oidc_jwks provider=mock keys=15635worker2 # [ 87.979827] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:21Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:520105636worker2 # [ 87.983803] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:21Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05637worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5638worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5639client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&25640client # don't know how to build these paths:5641client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5642worker2 # [ 89.160892] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:430925643worker2 # [ 89.166477] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:430925644worker2 # [ 89.188122] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:23Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05645worker2 # [ 89.193328] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:23Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.4:50051 clients=15646worker2 # [ 89.232195] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:430925647worker2 # [ 89.245191] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:430925648client # copying 1 paths to 'lb:50051'...5649client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5650worker2 # [ 89.257552] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:430925651worker2 # [ 89.268919] nix-daemon[857]: accepted connection from pid 1093, user nix-grpc-daemon (trusted)5652worker2 # [ 89.453191] nix-daemon[857]: accepted connection from pid 1189, user nix-grpc-daemon (trusted)5653worker2 # [ 89.465335] nix-daemon[857]: reaped child process 1192, status = succeeded5654worker2 # [ 89.637051] nix-daemon[857]: accepted connection from pid 1194, user nix-grpc-daemon (trusted)5655worker2 # [ 89.649067] nix-daemon[857]: reaped child process 1197, status = succeeded5656lb # [ 89.688262] niks3-server[1131]: 2026/09/20 12:47:23 INFO Received uploads request method=POST path=/api/pending_closures5657worker2 # [ 89.673422] nix-grpc-daemon[1149]: time=2026-09-20T12:47:23.645Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5658worker2 # [ 89.674916] nix-grpc-daemon[1149]: time=2026-09-20T12:47:23.645Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5659lb # [ 89.715259] niks3-server[1131]: 2026/09/20 12:47:23 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5660lb # [ 89.727794] niks3-server[1131]: 2026/09/20 12:47:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5661lb # [ 89.740194] niks3-server[1131]: 2026/09/20 12:47:23 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5662lb # [ 89.749074] niks3-server[1131]: 2026/09/20 12:47:23 INFO Signed narinfos id=27 count=15663worker2 # [ 89.726860] nix-grpc-daemon[1149]: time=2026-09-20T12:47:23.699Z level=INFO msg="Uploading 1 narinfos"5664lb # [ 89.760632] niks3-server[1131]: 2026/09/20 12:47:23 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5665lb # [ 89.765101] niks3-server[1131]: 2026/09/20 12:47:23 INFO Completed upload id=275666worker2 # [ 89.744098] nix-grpc-daemon[1149]: time=2026-09-20T12:47:23.716Z level=INFO msg="Upload complete. (463ms)"5667worker2 # [ 89.746445] nix-grpc-daemon[1149]: time=2026-09-20T12:47:23.717Z level=WARN msg="Failed to register uploaded object" key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo error="server returned 404: object is not pending upload\n"5668worker2 # [ 89.753143] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43092 duration_s=0 paths=1 nar_bytes_in=5205669worker2 # [ 89.759405] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:430925670worker2 # [ 89.770107] nix-daemon[857]: accepted connection from pid 1093, user nix-grpc-daemon (trusted)5671worker2 # [ 89.776941] nix-daemon[857]: reaped child process 1187, status = succeeded5672worker2 # [ 89.782097] nix-daemon[857]: accepted connection from pid 1093, user nix-grpc-daemon (trusted)5673client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5674client # farm-a> worker2: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5675client # farm-a> LOG-farm-a-on-standby5676worker2 # [ 90.406947] nix-daemon[857]: reaped child process 1201, status = succeeded5677worker2 # [ 90.571186] nix-daemon[857]: accepted connection from pid 1216, user nix-grpc-daemon (trusted)5678worker2 # [ 90.583626] nix-daemon[857]: reaped child process 1219, status = succeeded5679lb # [ 90.622283] niks3-server[1131]: 2026/09/20 12:47:24 INFO Received uploads request method=POST path=/api/pending_closures5680worker2 # [ 90.611392] nix-grpc-daemon[1149]: time=2026-09-20T12:47:24.584Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5681worker2 # [ 90.612856] nix-grpc-daemon[1149]: time=2026-09-20T12:47:24.585Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5682lb # [ 90.681214] niks3-server[1131]: 2026/09/20 12:47:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5683worker2 # [ 90.669274] nix-grpc-daemon[1149]: time=2026-09-20T12:47:24.642Z level=INFO msg="Uploading 1 narinfos"5684lb # [ 90.692687] niks3-server[1131]: 2026/09/20 12:47:24 INFO Signed narinfos id=28 count=15685lb # [ 90.697361] niks3-server[1131]: 2026/09/20 12:47:24 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5686lb # [ 90.699482] niks3-server[1131]: 2026/09/20 12:47:24 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5687lb # [ 90.707883] niks3-server[1131]: 2026/09/20 12:47:24 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5688lb # [ 90.711651] niks3-server[1131]: 2026/09/20 12:47:24 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5689lb # [ 90.716917] niks3-server[1131]: 2026/09/20 12:47:24 INFO Completed upload id=285690lb # [ 90.718973] niks3-server[1131]: 2026/09/20 12:47:24 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5691worker2 # [ 90.694923] nix-grpc-daemon[1149]: time=2026-09-20T12:47:24.667Z level=INFO msg="Upload complete. (305ms)"5692worker2 # [ 90.709298] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43092 duration_s=0 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15693worker2 # [ 90.715313] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:24Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05694worker2 # [ 90.719406] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:24Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.4:50051 clients=15695worker2 # [ 90.729071] nix-daemon[857]: reaped child process 1199, status = succeeded5696worker2 # [ 90.752937] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:430925697worker2 # [ 90.764370] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:430925698worker2 # [ 90.777511] nix-grpc-daemon[1149]: time=2026-09-20T12:47:24.750Z level=INFO msg="All 1 paths already cached"5699client # copying 1 paths to 'lb:50051'...5700client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5701worker2 # [ 90.785596] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:430925702worker2 # [ 90.796363] nix-daemon[857]: accepted connection from pid 1093, user nix-grpc-daemon (trusted)5703worker2 # [ 90.952877] nix-daemon[857]: accepted connection from pid 1225, user nix-grpc-daemon (trusted)5704worker2 # [ 90.965484] nix-daemon[857]: reaped child process 1228, status = succeeded5705worker2 # [ 91.126314] nix-daemon[857]: accepted connection from pid 1230, user nix-grpc-daemon (trusted)5706worker2 # [ 91.138811] nix-daemon[857]: reaped child process 1233, status = succeeded5707lb # [ 91.174776] niks3-server[1131]: 2026/09/20 12:47:24 INFO Received uploads request method=POST path=/api/pending_closures5708worker2 # [ 91.164825] nix-grpc-daemon[1149]: time=2026-09-20T12:47:25.137Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5709worker2 # [ 91.166167] nix-grpc-daemon[1149]: time=2026-09-20T12:47:25.137Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5710lb # [ 91.206575] niks3-server[1131]: 2026/09/20 12:47:25 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5711lb # [ 91.219554] niks3-server[1131]: 2026/09/20 12:47:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5712worker2 # [ 91.202180] nix-grpc-daemon[1149]: time=2026-09-20T12:47:25.174Z level=INFO msg="Uploading 1 narinfos"5713lb # [ 91.224927] niks3-server[1131]: 2026/09/20 12:47:25 INFO Signed narinfos id=29 count=15714lb # [ 91.231445] niks3-server[1131]: 2026/09/20 12:47:25 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5715lb # [ 91.241788] niks3-server[1131]: 2026/09/20 12:47:25 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5716lb # [ 91.249526] niks3-server[1131]: 2026/09/20 12:47:25 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5717lb # [ 91.253070] niks3-server[1131]: 2026/09/20 12:47:25 INFO Completed upload id=295718worker2 # [ 91.230635] nix-grpc-daemon[1149]: time=2026-09-20T12:47:25.203Z level=INFO msg="Upload complete. (423ms)"5719worker2 # [ 91.235178] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43092 duration_s=0 paths=1 nar_bytes_in=6405720worker2 # [ 91.240721] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:430925721worker2 # [ 91.253128] nix-daemon[857]: accepted connection from pid 1093, user nix-grpc-daemon (trusted)5722worker2 # [ 91.260770] nix-daemon[857]: reaped child process 1223, status = succeeded5723worker2 # [ 91.268416] nix-daemon[857]: accepted connection from pid 1093, user nix-grpc-daemon (trusted)5724client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5725client # farm-b> worker2: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5726client # farm-b> LOG-farm-b-on-standby5727worker2 # [ 91.881401] nix-daemon[857]: reaped child process 1237, status = succeeded5728worker2 # [ 92.026093] nix-daemon[857]: accepted connection from pid 1252, user nix-grpc-daemon (trusted)5729worker2 # [ 92.037445] nix-daemon[857]: reaped child process 1255, status = succeeded5730lb # [ 92.074671] niks3-server[1131]: 2026/09/20 12:47:25 INFO Received uploads request method=POST path=/api/pending_closures5731worker2 # [ 92.056551] nix-grpc-daemon[1149]: time=2026-09-20T12:47:26.029Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5732worker2 # [ 92.058349] nix-grpc-daemon[1149]: time=2026-09-20T12:47:26.031Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"5733lb # [ 92.094554] niks3-server[1131]: 2026/09/20 12:47:25 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5734lb # [ 92.096425] niks3-server[1131]: 2026/09/20 12:47:25 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst5735lb # [ 92.100807] niks3-server[1131]: 2026/09/20 12:47:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5736lb # [ 92.103184] niks3-server[1131]: 2026/09/20 12:47:25 INFO Signed narinfos id=30 count=15737worker2 # [ 92.080340] nix-grpc-daemon[1149]: time=2026-09-20T12:47:26.053Z level=INFO msg="Uploading 1 narinfos"5738lb # [ 92.106352] niks3-server[1131]: 2026/09/20 12:47:25 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls5739lb # [ 92.111145] niks3-server[1131]: 2026/09/20 12:47:25 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5740lb # [ 92.114126] niks3-server[1131]: 2026/09/20 12:47:25 INFO Completed upload id=305741worker2 # [ 92.092564] nix-grpc-daemon[1149]: time=2026-09-20T12:47:26.065Z level=INFO msg="Upload complete. (217ms)"5742lb # [ 92.115717] niks3-server[1131]: 2026/09/20 12:47:25 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo5743worker2 # [ 92.096155] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43092 duration_s=0 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=15744worker2 # [ 92.101686] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:26Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=05745worker2 # [ 92.103910] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:26Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.4:50051 clients=15746worker2 # [ 92.108403] nix-daemon[857]: reaped child process 1235, status = succeeded5747worker2 # [ 92.145354] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:430925748worker2 # [ 92.154678] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:430925749worker2 # [ 92.163915] nix-grpc-daemon[1149]: time=2026-09-20T12:47:26.136Z level=INFO msg="All 2 paths already cached"5750client # copying 1 paths to 'lb:50051'...5751client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...5752worker2 # [ 92.170594] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:430925753worker2 # [ 92.177362] nix-daemon[857]: accepted connection from pid 1093, user nix-grpc-daemon (trusted)5754worker2 # [ 92.324845] nix-daemon[857]: accepted connection from pid 1261, user nix-grpc-daemon (trusted)5755worker2 # [ 92.337418] nix-daemon[857]: reaped child process 1264, status = succeeded5756worker2 # [ 92.488726] nix-daemon[857]: accepted connection from pid 1266, user nix-grpc-daemon (trusted)5757worker2 # [ 92.501168] nix-daemon[857]: reaped child process 1269, status = succeeded5758lb # [ 92.539394] niks3-server[1131]: 2026/09/20 12:47:26 INFO Received uploads request method=POST path=/api/pending_closures5759worker2 # [ 92.520955] nix-grpc-daemon[1149]: time=2026-09-20T12:47:26.493Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5760worker2 # [ 92.522930] nix-grpc-daemon[1149]: time=2026-09-20T12:47:26.495Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"5761lb # [ 92.553485] niks3-server[1131]: 2026/09/20 12:47:26 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst5762lb # [ 92.557820] niks3-server[1131]: 2026/09/20 12:47:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5763lb # [ 92.559448] niks3-server[1131]: 2026/09/20 12:47:26 INFO Signed narinfos id=31 count=15764worker2 # [ 92.536898] nix-grpc-daemon[1149]: time=2026-09-20T12:47:26.509Z level=INFO msg="Uploading 1 narinfos"5765lb # [ 92.563127] niks3-server[1131]: 2026/09/20 12:47:26 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls5766lb # [ 92.568132] niks3-server[1131]: 2026/09/20 12:47:26 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5767lb # [ 92.571952] niks3-server[1131]: 2026/09/20 12:47:26 INFO Completed upload id=315768worker2 # [ 92.549125] nix-grpc-daemon[1149]: time=2026-09-20T12:47:26.522Z level=INFO msg="Upload complete. (363ms)"5769lb # [ 92.573561] niks3-server[1131]: 2026/09/20 12:47:26 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo5770worker2 # [ 92.551142] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43092 duration_s=0 paths=1 nar_bytes_in=7685771worker2 # [ 92.555072] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:430925772worker2 # [ 92.558759] nix-daemon[857]: accepted connection from pid 1093, user nix-grpc-daemon (trusted)5773worker2 # [ 92.564828] nix-daemon[857]: reaped child process 1259, status = succeeded5774client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...5775client # farm-top> worker2: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5776worker2 # [ 92.569299] nix-daemon[857]: accepted connection from pid 1093, user nix-grpc-daemon (trusted)5777client # farm-top> LOG-farm-top-on-standby5778worker2 # [ 93.584731] nix-daemon[857]: reaped child process 1273, status = succeeded5779worker2 # [ 93.719775] nix-daemon[857]: accepted connection from pid 1288, user nix-grpc-daemon (trusted)5780worker2 # [ 93.730808] nix-daemon[857]: reaped child process 1291, status = succeeded5781lb # [ 93.770443] niks3-server[1131]: 2026/09/20 12:47:27 INFO Received uploads request method=POST path=/api/pending_closures5782worker2 # [ 93.756668] nix-grpc-daemon[1149]: time=2026-09-20T12:47:27.729Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5783worker2 # [ 93.758282] nix-grpc-daemon[1149]: time=2026-09-20T12:47:27.731Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"5784lb # [ 93.801961] niks3-server[1131]: 2026/09/20 12:47:27 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5785lb # [ 93.818346] niks3-server[1131]: 2026/09/20 12:47:27 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst5786lb # [ 93.824483] niks3-server[1131]: 2026/09/20 12:47:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5787lb # [ 93.831612] niks3-server[1131]: 2026/09/20 12:47:27 INFO Signed narinfos id=32 count=15788worker2 # [ 93.812141] nix-grpc-daemon[1149]: time=2026-09-20T12:47:27.784Z level=INFO msg="Uploading 1 narinfos"5789lb # [ 93.836234] niks3-server[1131]: 2026/09/20 12:47:27 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls5790lb # [ 93.846249] niks3-server[1131]: 2026/09/20 12:47:27 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5791lb # [ 93.851628] niks3-server[1131]: 2026/09/20 12:47:27 INFO Completed upload id=325792worker2 # [ 93.829920] nix-grpc-daemon[1149]: time=2026-09-20T12:47:27.802Z level=INFO msg="Upload complete. (253ms)"5793lb # [ 93.854267] niks3-server[1131]: 2026/09/20 12:47:27 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo5794worker2 # [ 93.838403] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43092 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=15795worker2 # [ 93.848859] nix-daemon[857]: reaped child process 1271, status = succeeded5796client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&2, in 4.89 seconds)5797client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath5798client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath, in 0.20 seconds)5799worker1 # [ 94.166692] systemd[1]: Listening on Nix worker-protocol over gRPC.5800worker1 # [ 94.174341] systemd[1]: Starting Nix worker-protocol over gRPC...5801worker1 # [ 94.338889] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:27Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5802worker1 # [ 94.341249] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:27Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5803worker1 # [ 94.344784] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:27Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15804worker1 # [ 94.351624] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:27Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5805worker1 # [ 94.355156] systemd[1]: Started Nix worker-protocol over gRPC.5806lb: must succeed: curl -sf localhost:9901/clusters5807worker1 # [ 94.373298] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:27Z level=info event=scheduler_connected addr=lb:500515808worker2 # [ 94.333815] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:28Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:520105809worker2 # [ 94.335438] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:28Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05810lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5811worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5812worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5813lb: must succeed: curl -sf localhost:9901/clusters5814lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5815lb: must succeed: curl -sf localhost:9901/clusters5816lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5817worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5818worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5819(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 10.80 seconds)5820subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5821client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs5822client # [ 94.708174] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs.5823client # Running as unit: rs.service; invocation ID: 9db660357d1044c88b8ac07137a4aa445824client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs, in 0.04 seconds)5825worker1 # [ 94.912079] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:568425826worker1 # [ 94.923569] nix-daemon[912]: accepted connection from pid 1336, user nix-grpc-daemon (trusted)5827client # [ 94.936857] nix[1169]: don't know how to build these paths:5828client # [ 94.938239] nix[1169]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5829worker2 # [ 94.906692] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:28Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=05830worker2 # [ 94.910711] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:28Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.4:50051 clients=15831worker2 # [ 94.950109] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:430925832worker2 # [ 94.962117] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:430925833client # [ 95.014170] nix[1169]: copying 1 paths to 'lb:50051'...5834client # [ 95.015181] nix[1169]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...5835worker2 # [ 94.972746] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:430925836worker2 # [ 94.984383] nix-daemon[857]: accepted connection from pid 1093, user nix-grpc-daemon (trusted)5837worker2 # [ 95.148139] nix-daemon[857]: accepted connection from pid 1304, user nix-grpc-daemon (trusted)5838worker2 # [ 95.160465] nix-daemon[857]: reaped child process 1307, status = succeeded5839worker2 # [ 95.305679] nix-daemon[857]: accepted connection from pid 1309, user nix-grpc-daemon (trusted)5840worker2 # [ 95.317779] nix-daemon[857]: reaped child process 1312, status = succeeded5841lb # [ 95.355431] niks3-server[1131]: 2026/09/20 12:47:29 INFO Received uploads request method=POST path=/api/pending_closures5842worker2 # [ 95.345557] nix-grpc-daemon[1149]: time=2026-09-20T12:47:29.318Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5843worker2 # [ 95.346907] nix-grpc-daemon[1149]: time=2026-09-20T12:47:29.318Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"5844lb # [ 95.388772] niks3-server[1131]: 2026/09/20 12:47:29 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst5845lb # [ 95.395608] niks3-server[1131]: 2026/09/20 12:47:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5846worker2 # [ 95.380022] nix-grpc-daemon[1149]: time=2026-09-20T12:47:29.352Z level=INFO msg="Uploading 1 narinfos"5847lb # [ 95.403084] niks3-server[1131]: 2026/09/20 12:47:29 INFO Signed narinfos id=33 count=15848lb # [ 95.408928] niks3-server[1131]: 2026/09/20 12:47:29 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls5849lb # [ 95.419590] niks3-server[1131]: 2026/09/20 12:47:29 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5850lb # [ 95.422436] niks3-server[1131]: 2026/09/20 12:47:29 INFO Completed upload id=335851worker2 # [ 95.399743] nix-grpc-daemon[1149]: time=2026-09-20T12:47:29.372Z level=INFO msg="Upload complete. (404ms)"5852lb # [ 95.423990] niks3-server[1131]: 2026/09/20 12:47:29 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo5853worker2 # [ 95.402582] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43092 duration_s=0 paths=1 nar_bytes_in=4165854worker2 # [ 95.408367] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:430925855worker2 # [ 95.416127] nix-daemon[857]: accepted connection from pid 1093, user nix-grpc-daemon (trusted)5856worker2 # [ 95.421766] nix-daemon[857]: reaped child process 1302, status = succeeded5857client # [ 95.467199] nix[1169]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...5858worker2 # [ 95.425878] nix-daemon[857]: accepted connection from pid 1093, user nix-grpc-daemon (trusted)5859lb: must succeed: curl -sf localhost:9901/clusters5860lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5861lb: must succeed: curl -sf localhost:9901/clusters5862lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5863lb: must succeed: curl -sf localhost:9901/clusters5864lb # [ 97.315747] envoy[973]: conn peer=[2001:db8:1::1]:53038 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9547 tx=3833 ms=46875865lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5866lb: must succeed: curl -sf localhost:9901/clusters5867lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5868worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5869worker1 # [ 97.538601] systemd-journald[289]: Received client request to rotate journal, rotating.5870worker1 # [ 97.555535] systemd-journald[289]: Vacuuming done, freed 0B of archived journals from /var/log/journal/e3cf54c89198454c8407007fa4edfbed.5871worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.06 seconds)5872worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service5873worker2 # [ 97.431327] systemd[1]: Stopping Nix worker-protocol over gRPC...5874worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)5875??? Warning (UserWarning): sleep(): The 'secs' argument is deprecated. Use 'duration' instead.5876 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 5735877client: must succeed: sleep 5.05878worker2 # [ 97.793396] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:31Z level=info event=draining5879client: (finished: must succeed: sleep 5.0, in 5.03 seconds)5880worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5881worker2 # [ 102.862439] nix-daemon[857]: reaped child process 1316, status = succeeded5882worker2 # [ 103.023490] nix-daemon[857]: accepted connection from pid 1356, user nix-grpc-daemon (trusted)5883worker2 # [ 103.034290] nix-daemon[857]: reaped child process 1359, status = succeeded5884lb # [ 103.074370] niks3-server[1131]: 2026/09/20 12:47:36 INFO Received uploads request method=POST path=/api/pending_closures5885worker2 # [ 103.060424] nix-grpc-daemon[1149]: time=2026-09-20T12:47:37.033Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5886worker2 # [ 103.061987] nix-grpc-daemon[1149]: time=2026-09-20T12:47:37.034Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"5887lb # [ 103.114675] niks3-server[1131]: 2026/09/20 12:47:36 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5888lb # [ 103.122236] niks3-server[1131]: 2026/09/20 12:47:36 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5889lb # [ 103.133383] niks3-server[1131]: 2026/09/20 12:47:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5890lb # [ 103.138186] niks3-server[1131]: 2026/09/20 12:47:36 INFO Signed narinfos id=34 count=15891worker2 # [ 103.118105] nix-grpc-daemon[1149]: time=2026-09-20T12:47:37.090Z level=INFO msg="Uploading 1 narinfos"5892lb # [ 103.142916] niks3-server[1131]: 2026/09/20 12:47:36 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls5893lb # [ 103.153592] niks3-server[1131]: 2026/09/20 12:47:36 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5894lb # [ 103.158520] niks3-server[1131]: 2026/09/20 12:47:36 INFO Completed upload id=345895lb # [ 103.160931] niks3-server[1131]: 2026/09/20 12:47:36 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo5896worker2 # [ 103.137074] nix-grpc-daemon[1149]: time=2026-09-20T12:47:37.110Z level=INFO msg="Upload complete. (295ms)"5897worker2 # [ 103.144462] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43092 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=15898worker2 # [ 103.157336] nix-daemon[857]: reaped child process 1314, status = succeeded5899client # [ 103.211292] systemd[1]: rs.service: Deactivated successfully.5900client # [ 103.213102] systemd[1]: rs.service: Consumed 109ms CPU time over 8.504s wall clock time, 37.1M memory peak, 8.5K incoming IP traffic, 9K outgoing IP traffic.5901worker2 # [ 103.794312] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:37Z level=info event=signal_exit5902worker2 # [ 104.014229] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:37Z level=info event=scheduler_yield5903lb # [ 104.039827] niks3-server[1131]: 2026/09/20 12:47:37 INFO lead: released remote=192.168.1.4:332345904worker1 # [ 104.058901] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:37Z level=info event=scheduler_restarting addr=lb:500515905worker2 # [ 104.021595] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:37Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:520105906worker2 # [ 104.027746] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:37Z level=info event=worker_gone addr=192.168.1.3:500515907worker2 # [ 104.030916] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:37Z level=info event=scheduler_restarting addr=lb:500515908worker2 # [ 104.033910] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:38Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:520105909worker2 # [ 104.037215] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:38Z level=info event=worker_gone addr=192.168.1.4:500515910worker1 # [ 104.259346] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:37Z level=info event=scheduler_connected addr=lb:500515911worker1 # [ 104.263422] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5912worker2 # [ 104.228396] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:38Z level=info event=scheduler_connected addr=lb:500515913worker2 # [ 104.231620] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5914worker1 # [ 104.361964] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:37Z level=info event=scheduler_take_over5915lb # [ 104.344382] niks3-server[1131]: 2026/09/20 12:47:38 INFO lead: acquired remote=192.168.1.3:525145916worker1 # [ 104.464158] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:37Z level=info event=scheduler_connected addr=lb:500515917worker1 # [ 104.468172] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5918worker2 # [ 104.429617] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:38Z level=info event=scheduler_connected addr=lb:500515919worker2 # [ 104.431424] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5920worker1 # [ 104.669147] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:38Z level=info event=scheduler_connected addr=lb:500515921worker2 # [ 104.632446] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:38Z level=info event=scheduler_connected addr=lb:500515922worker1 # [ 104.673152] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5923worker2 # [ 104.636285] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5924worker1 # [ 104.874395] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:38Z level=info event=scheduler_connected addr=lb:500515925worker2 # [ 104.837355] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:38Z level=info event=scheduler_connected addr=lb:500515926worker1 # [ 104.878628] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5927worker2 # [ 104.841183] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5928worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.18 seconds)5929client: waiting for success: ! systemctl is-active rs5930client: (finished: waiting for success: ! systemctl is-active rs, in 0.05 seconds)5931client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5932client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.04 seconds)5933client: must fail: journalctl -u rs | grep -q 'reconnecting'5934client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.02 seconds)5935lb: must succeed: curl -sf localhost:9901/clusters5936worker2 # [ 105.019243] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:38Z level=info event=scheduler_restarting5937worker1 # [ 105.079811] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:38Z level=info event=scheduler_connected addr=lb:500515938worker1 # [ 105.083791] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5939worker2 # [ 105.051611] nix-daemon[857]: reaped child process 1142, status = succeeded5940worker2 # [ 105.054335] nix-grpc-daemon[1093]: ts=2026-09-20T12:47:39Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5941lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5942worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5943worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5944worker1 # [ 105.284779] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:38Z level=info event=scheduler_connected addr=lb:500515945worker1 # [ 105.288677] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5946worker1 # [ 105.489508] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:38Z level=info event=scheduler_connected addr=lb:500515947worker1 # [ 105.493537] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5948worker1 # [ 105.694431] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:39Z level=info event=scheduler_connected addr=lb:500515949worker1 # [ 105.698601] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5950worker2 # [ 105.849488] nix-grpc-daemon[1093]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5951worker2 # [ 105.852980] nix-grpc-daemon[1093]: E0000 00:00:1789908459.821176 1136 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5952worker1 # [ 105.899720] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:39Z level=info event=scheduler_connected addr=lb:500515953worker2 # [ 105.857785] nix-grpc-daemon[1093]: E0000 00:00:1789908459.821919 1136 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5954worker1 # [ 105.906929] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:39Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:385805955worker1 # [ 105.910970] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:39Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05956lb: must succeed: curl -sf localhost:9901/clusters5957lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5958worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5959worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5960worker2 # [ 106.489746] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5961worker2 # [ 106.492718] systemd[1]: Stopped Nix worker-protocol over gRPC.5962worker2 # [ 106.495574] systemd[1]: nix-grpc-daemon.service: Consumed 1.417s CPU time over 28.880s wall clock time, 33.2M memory peak, 63.1K incoming IP traffic, 162.4K outgoing IP traffic.5963worker2 # [ 106.510727] systemd[1]: Starting scheduler-token.service...5964worker2 # [ 106.591223] systemd[1]: scheduler-token.service: Deactivated successfully.5965worker2 # [ 106.592354] systemd[1]: Finished scheduler-token.service.5966worker2 # [ 106.593626] systemd[1]: scheduler-token.service: Consumed 24ms CPU time over 80ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5967worker2 # [ 106.599208] systemd[1]: Starting Nix worker-protocol over gRPC...5968worker2 # [ 106.763147] nix-grpc-daemon[1369]: ts=2026-09-20T12:47:40Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5969worker2 # [ 106.764826] nix-grpc-daemon[1369]: ts=2026-09-20T12:47:40Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5970worker2 # [ 106.768219] nix-grpc-daemon[1369]: ts=2026-09-20T12:47:40Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15971worker2 # [ 106.775392] nix-grpc-daemon[1369]: ts=2026-09-20T12:47:40Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5972worker2 # [ 106.778376] systemd[1]: Started Nix worker-protocol over gRPC.5973worker2 # [ 106.793724] nix-grpc-daemon[1369]: ts=2026-09-20T12:47:40Z level=info event=scheduler_connected addr=lb:500515974worker1 # [ 106.840284] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:40Z level=info event=oidc_jwks provider=mock keys=15975worker1 # [ 106.841625] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:40Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:385805976worker1 # [ 106.843212] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:40Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05977lb: must succeed: curl -sf localhost:9901/clusters5978lb # [ 107.315872] envoy[973]: conn peer=[2001:db8:1::1]:37174 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2947 tx=1351 ms=82885979lb # [ 107.320329] envoy[973]: conn peer=[2001:db8:1::4]:44000 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=40790 tx=2187 ms=272555980lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5981worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5982worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5983(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.76 seconds)5984subtest: scheduler restart mid-build: build finishes, no double build5985client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid5986client # [ 107.488882] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid.5987client # Running as unit: mid.service; invocation ID: 8a223ea1bc114951913c3fec149e49ff5988client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid, in 0.07 seconds)5989worker2 # [ 107.648169] nix-grpc-daemon[1369]: ts=2026-09-20T12:47:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:409945990worker2 # [ 107.659214] nix-daemon[857]: accepted connection from pid 1369, user nix-grpc-daemon (trusted)5991client # [ 107.714167] nix[1204]: don't know how to build these paths:5992client # [ 107.715101] nix[1204]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5993worker1 # [ 107.723093] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:41Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05994worker1 # [ 107.728526] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:41Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=15995worker1 # [ 107.768894] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:568425996worker1 # [ 107.782705] nix-grpc-daemon[1336]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 14 ms (attempt 1/5)5997worker1 # [ 107.802207] nix-grpc-daemon[1336]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 81 ms (attempt 2/5)5998worker1 # [ 107.889681] nix-grpc-daemon[1336]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 16 ms (attempt 3/5)5999worker1 # [ 107.905064] nix-grpc-daemon[1336]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 76 ms (attempt 4/5)6000worker1 # [ 107.984310] nix-grpc-daemon[1336]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6001worker1 # [ 108.003301] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:568426002client # [ 108.014232] nix[1204]: copying 1 paths to 'lb:50051'...6003worker1 # [ 108.014640] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:568426004client # [ 108.016925] nix[1204]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...6005worker1 # [ 108.028206] nix-daemon[912]: accepted connection from pid 1336, user nix-grpc-daemon (trusted)6006worker1 # [ 108.047892] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:41Z level=info event=niks3_push_started pid=14576007worker1 # [ 108.252672] nix-grpc-daemon[1457]: time=2026-09-20T12:47:41.678Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""6008worker1 # [ 108.428535] nix-daemon[912]: accepted connection from pid 1470, user nix-grpc-daemon (trusted)6009worker1 # [ 108.440392] nix-daemon[912]: reaped child process 1473, status = succeeded6010worker1 # [ 108.624420] nix-daemon[912]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6011worker1 # [ 108.643677] nix-daemon[912]: reaped child process 1481, status = succeeded6012lb # [ 108.638548] niks3-server[1131]: 2026/09/20 12:47:42 INFO Received uploads request method=POST path=/api/pending_closures6013worker1 # [ 108.664982] nix-grpc-daemon[1457]: time=2026-09-20T12:47:42.091Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6014worker1 # [ 108.668640] nix-grpc-daemon[1457]: time=2026-09-20T12:47:42.092Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"6015lb # [ 108.683643] niks3-server[1131]: 2026/09/20 12:47:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign6016lb # [ 108.693226] niks3-server[1131]: 2026/09/20 12:47:42 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst6017lb # [ 108.702586] niks3-server[1131]: 2026/09/20 12:47:42 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls6018worker1 # [ 108.729517] nix-grpc-daemon[1457]: time=2026-09-20T12:47:42.155Z level=INFO msg="Uploading 1 narinfos"6019lb # [ 108.711357] niks3-server[1131]: 2026/09/20 12:47:42 INFO Signed narinfos id=35 count=16020lb # [ 108.722281] niks3-server[1131]: 2026/09/20 12:47:42 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete6021lb # [ 108.725258] niks3-server[1131]: 2026/09/20 12:47:42 INFO Completed upload id=356022worker1 # [ 108.743266] nix-grpc-daemon[1457]: time=2026-09-20T12:47:42.169Z level=INFO msg="Upload complete. (486ms)"6023lb # [ 108.728212] niks3-server[1131]: 2026/09/20 12:47:42 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo6024worker1 # [ 108.745593] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56842 duration_s=0 paths=1 nar_bytes_in=4246025worker1 # [ 108.749222] nix-grpc-daemon[1336]: ts=2026-09-20T12:47:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:568426026worker1 # [ 108.756427] nix-daemon[912]: accepted connection from pid 1336, user nix-grpc-daemon (trusted)6027worker1 # [ 108.759866] nix-daemon[912]: reaped child process 1455, status = succeeded6028worker1 # [ 108.765039] nix-daemon[912]: accepted connection from pid 1336, user nix-grpc-daemon (trusted)6029client # [ 108.766144] nix[1204]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6030lb: must succeed: curl -sf localhost:9901/clusters6031lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6032lb: must succeed: curl -sf localhost:9901/clusters6033lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6034worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6035worker1 # [ 111.156369] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1336 (nix-grpc-daemon) on client request.6036worker1 # [ 111.160335] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6037lb # [ 111.150255] niks3-server[1131]: 2026/09/20 12:47:44 INFO lead: released remote=192.168.1.3:525146038client # [ 111.177898] nix[1204]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again6039client # [ 111.180445] nix[1204]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6040worker2 # [ 111.135503] nix-grpc-daemon[1369]: ts=2026-09-20T12:47:45Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"6041worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.07 seconds)6042client: must succeed: sleep 5.06043worker1 # [ 111.196439] nix-daemon[912]: reaped child process 1484, status = succeeded6044worker1 # [ 111.197949] nix-daemon[912]: reaped child process 1392, status = succeeded6045worker1 # [ 111.202851] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6046worker1 # [ 111.204730] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6047worker1 # [ 111.207544] systemd[1]: nix-grpc-daemon.service: Consumed 445ms CPU time over 17.028s wall clock time, 30.8M memory peak, 17.3K incoming IP traffic, 25.7K outgoing IP traffic.6048worker1 # [ 111.214237] nix-daemon[1486]: unexpected Nix daemon error: error: interrupted by the user6049worker1 # [ 111.220088] nix-daemon[912]: reaped child process 1486, status = failed with exit code 16050worker2 # [ 111.336369] nix-grpc-daemon[1369]: ts=2026-09-20T12:47:45Z level=info event=scheduler_connected addr=lb:500516051worker1 # [ 111.381434] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.6052worker1 # [ 111.391728] systemd[1]: Starting Nix worker-protocol over gRPC...6053worker1 # [ 111.578072] nix-grpc-daemon[1522]: ts=2026-09-20T12:47:45Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6054worker1 # [ 111.579739] nix-grpc-daemon[1522]: ts=2026-09-20T12:47:45Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6055worker1 # [ 111.582885] nix-grpc-daemon[1522]: ts=2026-09-20T12:47:45Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16056worker1 # [ 111.589547] nix-grpc-daemon[1522]: ts=2026-09-20T12:47:45Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6057worker1 # [ 111.593142] systemd[1]: Started Nix worker-protocol over gRPC.6058worker1 # [ 111.600582] nix-grpc-daemon[1522]: ts=2026-09-20T12:47:45Z level=info event=scheduler_take_over6059lb # [ 111.582619] niks3-server[1131]: 2026/09/20 12:47:45 INFO lead: acquired remote=192.168.1.3:497366060worker1 # [ 111.609504] nix-grpc-daemon[1522]: ts=2026-09-20T12:47:45Z level=info event=scheduler_connected addr=lb:500516061worker1 # [ 111.611212] nix-grpc-daemon[1522]: ts=2026-09-20T12:47:45Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:385966062worker1 # [ 111.613274] nix-grpc-daemon[1522]: ts=2026-09-20T12:47:45Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06063worker1 # [ 111.617403] nix-grpc-daemon[1522]: ts=2026-09-20T12:47:45Z level=info event=oidc_jwks provider=mock keys=16064worker1 # [ 111.618720] nix-grpc-daemon[1522]: ts=2026-09-20T12:47:45Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:385966065worker1 # [ 111.620319] nix-grpc-daemon[1522]: ts=2026-09-20T12:47:45Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06066worker1 # [ 111.689123] nix-grpc-daemon[1522]: ts=2026-09-20T12:47:45Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06067worker1 # [ 111.691211] nix-grpc-daemon[1522]: ts=2026-09-20T12:47:45Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=16068worker1 # [ 111.742671] nix-grpc-daemon[1522]: ts=2026-09-20T12:47:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:386106069worker1 # [ 111.752167] nix-daemon[912]: accepted connection from pid 1522, user nix-grpc-daemon (trusted)6070worker1 # [ 111.766500] nix-daemon[912]: accepted connection from pid 1522, user nix-grpc-daemon (trusted)6071client # [ 111.779610] nix[1204]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6072worker1 # [ 111.779924] nix-daemon[912]: accepted connection from pid 1522, user nix-grpc-daemon (trusted)6073client: (finished: must succeed: sleep 5.0, in 5.04 seconds)6074client: waiting for success: ! systemctl is-active mid6075worker1 # [ 116.603633] nix-grpc-daemon[1522]: ts=2026-09-20T12:47:50Z level=info event=niks3_push_started pid=16016076worker1 # [ 116.635617] nix-daemon[912]: reaped child process 1577, status = succeeded6077worker1 # [ 116.826597] nix-grpc-daemon[1601]: time=2026-09-20T12:47:50.252Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""6078worker1 # [ 117.005822] nix-daemon[912]: accepted connection from pid 1614, user nix-grpc-daemon (trusted)6079worker1 # [ 117.018737] nix-daemon[912]: reaped child process 1617, status = succeeded6080lb # [ 117.023268] niks3-server[1131]: 2026/09/20 12:47:50 INFO Received uploads request method=POST path=/api/pending_closures6081worker1 # [ 117.055144] nix-grpc-daemon[1601]: time=2026-09-20T12:47:50.481Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6082worker1 # [ 117.056772] nix-grpc-daemon[1601]: time=2026-09-20T12:47:50.482Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6083lb # [ 117.082700] niks3-server[1131]: 2026/09/20 12:47:50 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6084lb # [ 117.088170] niks3-server[1131]: 2026/09/20 12:47:50 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6085lb # [ 117.094497] niks3-server[1131]: 2026/09/20 12:47:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign6086lb # [ 117.099470] niks3-server[1131]: 2026/09/20 12:47:50 INFO Signed narinfos id=36 count=16087worker1 # [ 117.118405] nix-grpc-daemon[1601]: time=2026-09-20T12:47:50.543Z level=INFO msg="Uploading 1 narinfos"6088lb # [ 117.104440] niks3-server[1131]: 2026/09/20 12:47:50 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6089lb # [ 117.111475] niks3-server[1131]: 2026/09/20 12:47:50 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete6090lb # [ 117.117678] niks3-server[1131]: 2026/09/20 12:47:50 INFO Completed upload id=366091lb # [ 117.119040] niks3-server[1131]: 2026/09/20 12:47:50 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo6092worker1 # [ 117.136061] nix-grpc-daemon[1601]: time=2026-09-20T12:47:50.561Z level=INFO msg="Upload complete. (305ms)"6093worker1 # [ 117.142589] nix-grpc-daemon[1522]: ts=2026-09-20T12:47:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38610 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16094worker1 # [ 117.160691] nix-daemon[912]: reaped child process 1575, status = succeeded6095client # [ 117.210643] systemd[1]: mid.service: Deactivated successfully.6096client # [ 117.215200] systemd[1]: mid.service: Consumed 127ms CPU time over 9.725s wall clock time, 39.2M memory peak, 9.8K incoming IP traffic, 11.1K outgoing IP traffic.6097lb # [ 117.316152] envoy[973]: conn peer=[2001:db8:1::3]:56426 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2386 tx=618 ms=168076098lb # [ 117.320213] envoy[973]: conn peer=[2001:db8:1::1]:46548 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3891 tx=1652 ms=94916099client: (finished: waiting for success: ! systemctl is-active mid, in 1.11 seconds)6100client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6101client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.04 seconds)6102(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.19 seconds)6103subtest: interrupting the client stops the build on the worker6104client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr6105client # [ 117.657852] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr.6106client # Running as unit: intr.service; invocation ID: a2b2d0b0ea0f450cabc553773e7930c46107client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr, in 0.04 seconds)6108worker1 # [ 117.864916] nix-grpc-daemon[1522]: ts=2026-09-20T12:47:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:386106109client # [ 117.872915] nix[1239]: don't know how to build these paths:6110client # [ 117.874471] nix[1239]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6111worker1 # [ 117.883707] nix-grpc-daemon[1522]: ts=2026-09-20T12:47:51Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06112worker1 # [ 117.887897] nix-grpc-daemon[1522]: ts=2026-09-20T12:47:51Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=16113worker2 # [ 117.888773] nix-grpc-daemon[1369]: ts=2026-09-20T12:47:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409946114worker2 # [ 117.900976] nix-grpc-daemon[1369]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 30 ms (attempt 1/5)6115worker2 # [ 117.937060] nix-grpc-daemon[1369]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 57 ms (attempt 2/5)6116worker2 # [ 118.000084] nix-grpc-daemon[1369]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 313 ms (attempt 3/5)6117worker2 # [ 118.319595] nix-grpc-daemon[1369]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 790 ms (attempt 4/5)6118worker2 # [ 119.116850] nix-grpc-daemon[1369]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6119worker2 # [ 119.135450] nix-grpc-daemon[1369]: ts=2026-09-20T12:47:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409946120client # [ 119.188402] nix[1239]: copying 1 paths to 'lb:50051'...6121worker2 # [ 119.147453] nix-grpc-daemon[1369]: ts=2026-09-20T12:47:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409946122client # [ 119.191502] nix[1239]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6123worker2 # [ 119.161240] nix-daemon[857]: accepted connection from pid 1369, user nix-grpc-daemon (trusted)6124worker2 # [ 119.179885] nix-grpc-daemon[1369]: ts=2026-09-20T12:47:53Z level=info event=niks3_push_started pid=14616125worker2 # [ 119.379909] nix-grpc-daemon[1461]: time=2026-09-20T12:47:53.351Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""6126worker2 # [ 119.545045] nix-daemon[857]: accepted connection from pid 1474, user nix-grpc-daemon (trusted)6127worker2 # [ 119.557917] nix-daemon[857]: reaped child process 1477, status = succeeded6128worker2 # [ 119.732689] nix-daemon[857]: accepted connection from pid 1479, user nix-grpc-daemon (trusted)6129worker2 # [ 119.746159] nix-daemon[857]: reaped child process 1482, status = succeeded6130lb # [ 119.788065] niks3-server[1131]: 2026/09/20 12:47:53 INFO Received uploads request method=POST path=/api/pending_closures6131worker2 # [ 119.774731] nix-grpc-daemon[1461]: time=2026-09-20T12:47:53.747Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6132worker2 # [ 119.777144] nix-grpc-daemon[1461]: time=2026-09-20T12:47:53.749Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6133lb # [ 119.821933] niks3-server[1131]: 2026/09/20 12:47:53 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6134lb # [ 119.831633] niks3-server[1131]: 2026/09/20 12:47:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign6135lb # [ 119.836371] niks3-server[1131]: 2026/09/20 12:47:53 INFO Signed narinfos id=37 count=16136worker2 # [ 119.816769] nix-grpc-daemon[1461]: time=2026-09-20T12:47:53.789Z level=INFO msg="Uploading 1 narinfos"6137lb # [ 119.845353] niks3-server[1131]: 2026/09/20 12:47:53 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6138lb # [ 119.855294] niks3-server[1131]: 2026/09/20 12:47:53 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete6139lb # [ 119.862631] niks3-server[1131]: 2026/09/20 12:47:53 INFO Completed upload id=376140worker2 # [ 119.840047] nix-grpc-daemon[1461]: time=2026-09-20T12:47:53.812Z level=INFO msg="Upload complete. (460ms)"6141lb # [ 119.866579] niks3-server[1131]: 2026/09/20 12:47:53 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6142worker2 # [ 119.843849] nix-grpc-daemon[1369]: ts=2026-09-20T12:47:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40994 duration_s=0 paths=1 nar_bytes_in=4246143worker2 # [ 119.848345] nix-grpc-daemon[1369]: ts=2026-09-20T12:47:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409946144worker2 # [ 119.856066] nix-daemon[857]: accepted connection from pid 1369, user nix-grpc-daemon (trusted)6145worker2 # [ 119.858910] nix-daemon[857]: reaped child process 1459, status = succeeded6146worker2 # [ 119.864300] nix-daemon[857]: accepted connection from pid 1369, user nix-grpc-daemon (trusted)6147client # [ 119.905834] nix[1239]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6148client: must succeed: systemctl kill -s INT intr6149client # [ 121.161370] systemd[1]: intr.service: Sent signal SIGINT to main process 1239 (nix) on client request.6150client # [ 121.165349] nix[1239]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6151client: (finished: must succeed: systemctl kill -s INT intr, in 0.04 seconds)6152client # [ 121.169216] nix[1239]: error: interrupted by the user6153client # [ 121.194657] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6154client # [ 121.196501] systemd[1]: intr.service: Failed with result 'exit-code'.6155client # [ 121.198500] systemd[1]: intr.service: Consumed 115ms CPU time over 3.538s wall clock time, 39.1M memory peak, 7.8K incoming IP traffic, 8.7K outgoing IP traffic.6156worker2 # [ 121.282739] nix-daemon[857]: reaped child process 1484, status = succeeded6157worker2 # [ 121.287231] nix-daemon[1486]: unexpected Nix daemon error: error: interrupted by the user6158worker2 # [ 121.290503] nix-daemon[857]: reaped child process 1486, status = failed with exit code 16159(finished: subtest: interrupting the client stops the build on the worker, in 4.76 seconds)6160subtest: a deploy mid-build drains: the build finishes, the new generation takes over6161client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw6162client # [ 122.441791] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw.6163client # Running as unit: sw.service; invocation ID: bfe045f50f3547f392dc53a803c804ba6164client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw, in 0.07 seconds)6165client # [ 122.640161] nix[1259]: don't know how to build these paths:6166client # [ 122.641499] nix[1259]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6167worker1 # [ 122.638873] nix-grpc-daemon[1522]: ts=2026-09-20T12:47:56Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:386106168worker1 # [ 122.649719] nix-grpc-daemon[1522]: ts=2026-09-20T12:47:56Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06169worker1 # [ 122.654670] nix-grpc-daemon[1522]: ts=2026-09-20T12:47:56Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=16170worker1 # [ 122.693464] nix-grpc-daemon[1522]: ts=2026-09-20T12:47:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:386106171worker1 # [ 122.702082] nix-grpc-daemon[1522]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 15 ms (attempt 1/5)6172worker1 # [ 122.723068] nix-grpc-daemon[1522]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 3 ms (attempt 2/5)6173worker1 # [ 122.731732] nix-grpc-daemon[1522]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 108 ms (attempt 3/5)6174worker1 # [ 122.845605] nix-grpc-daemon[1522]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 251 ms (attempt 4/5)6175worker1 # [ 123.097220] nix-grpc-daemon[1522]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6176worker1 # [ 123.107939] nix-grpc-daemon[1522]: ts=2026-09-20T12:47:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:386106177client # [ 123.113690] nix[1259]: copying 1 paths to 'lb:50051'...6178worker1 # [ 123.116312] nix-grpc-daemon[1522]: ts=2026-09-20T12:47:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:386106179client # [ 123.118139] nix[1259]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6180worker1 # [ 123.120665] nix-daemon[912]: accepted connection from pid 1522, user nix-grpc-daemon (trusted)6181worker1 # [ 123.285236] nix-daemon[912]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6182worker1 # [ 123.298410] nix-daemon[912]: reaped child process 1659, status = succeeded6183worker1 # [ 123.457576] nix-daemon[912]: accepted connection from pid 1661, user nix-grpc-daemon (trusted)6184worker1 # [ 123.470332] nix-daemon[912]: reaped child process 1664, status = succeeded6185lb # [ 123.471739] niks3-server[1131]: 2026/09/20 12:47:57 INFO Received uploads request method=POST path=/api/pending_closures6186worker1 # [ 123.498114] nix-grpc-daemon[1601]: time=2026-09-20T12:47:56.923Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6187worker1 # [ 123.499650] nix-grpc-daemon[1601]: time=2026-09-20T12:47:56.925Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6188lb # [ 123.500547] niks3-server[1131]: 2026/09/20 12:47:57 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6189lb # [ 123.512581] niks3-server[1131]: 2026/09/20 12:47:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6190worker1 # [ 123.535720] nix-grpc-daemon[1601]: time=2026-09-20T12:47:56.961Z level=INFO msg="Uploading 1 narinfos"6191lb # [ 123.517358] niks3-server[1131]: 2026/09/20 12:47:57 INFO Signed narinfos id=38 count=16192lb # [ 123.524513] niks3-server[1131]: 2026/09/20 12:47:57 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6193lb # [ 123.535287] niks3-server[1131]: 2026/09/20 12:47:57 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6194lb # [ 123.542223] niks3-server[1131]: 2026/09/20 12:47:57 INFO Completed upload id=386195worker1 # [ 123.560444] nix-grpc-daemon[1601]: time=2026-09-20T12:47:56.986Z level=INFO msg="Upload complete. (433ms)"6196lb # [ 123.544468] niks3-server[1131]: 2026/09/20 12:47:57 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6197worker1 # [ 123.564248] nix-grpc-daemon[1522]: ts=2026-09-20T12:47:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38610 duration_s=0 paths=1 nar_bytes_in=4166198worker1 # [ 123.569227] nix-grpc-daemon[1522]: ts=2026-09-20T12:47:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:386106199worker1 # [ 123.578216] nix-daemon[912]: accepted connection from pid 1522, user nix-grpc-daemon (trusted)6200worker1 # [ 123.586877] nix-daemon[912]: reaped child process 1654, status = succeeded6201worker1 # [ 123.591905] nix-daemon[912]: accepted connection from pid 1522, user nix-grpc-daemon (trusted)6202client # [ 123.592481] nix[1259]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6203worker1: must succeed: systemctl reload nix-grpc-daemon.service6204worker1 # [ 126.136871] systemd[1]: Reloading Nix worker-protocol over gRPC...6205worker1 # [ 126.183073] systemd[1]: Reloaded Nix worker-protocol over gRPC.6206worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.09 seconds)6207worker1: must succeed: pgrep -f 'read -t [9]0 x'6208worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)6209client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&26210client # don't know how to build these paths:6211worker2 # [ 126.415252] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:409946212client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6213worker2 # [ 126.421131] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409946214worker1 # [ 126.481616] nix-grpc-daemon[1522]: ts=2026-09-20T12:47:59Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06215worker1 # [ 126.484157] nix-grpc-daemon[1522]: ts=2026-09-20T12:47:59Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=16216worker2 # [ 126.484185] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409946217worker2 # [ 126.492682] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409946218client # copying 1 paths to 'lb:50051'...6219worker2 # [ 126.501402] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409946220client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6221worker2 # [ 126.517947] nix-daemon[857]: accepted connection from pid 1369, user nix-grpc-daemon (trusted)6222worker1 # [ 126.595728] nix-grpc-daemon[1522]: ts=2026-09-20T12:48:00Z level=info event=draining6223worker2 # [ 126.682640] nix-daemon[857]: accepted connection from pid 1540, user nix-grpc-daemon (trusted)6224worker2 # [ 126.694604] nix-daemon[857]: reaped child process 1543, status = succeeded6225worker2 # [ 126.860187] nix-daemon[857]: accepted connection from pid 1545, user nix-grpc-daemon (trusted)6226worker2 # [ 126.873089] nix-daemon[857]: reaped child process 1548, status = succeeded6227lb # [ 126.910451] niks3-server[1131]: 2026/09/20 12:48:00 INFO Received uploads request method=POST path=/api/pending_closures6228worker2 # [ 126.897304] nix-grpc-daemon[1461]: time=2026-09-20T12:48:00.869Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6229worker2 # [ 126.898744] nix-grpc-daemon[1461]: time=2026-09-20T12:48:00.869Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6230lb # [ 126.937935] niks3-server[1131]: 2026/09/20 12:48:00 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6231lb # [ 126.945858] niks3-server[1131]: 2026/09/20 12:48:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6232lb # [ 126.950229] niks3-server[1131]: 2026/09/20 12:48:00 INFO Signed narinfos id=39 count=16233worker2 # [ 126.930619] nix-grpc-daemon[1461]: time=2026-09-20T12:48:00.903Z level=INFO msg="Uploading 1 narinfos"6234lb # [ 126.962817] niks3-server[1131]: 2026/09/20 12:48:00 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6235lb # [ 126.970112] niks3-server[1131]: 2026/09/20 12:48:00 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6236worker2 # [ 126.953664] nix-grpc-daemon[1461]: time=2026-09-20T12:48:00.926Z level=INFO msg="Upload complete. (429ms)"6237lb # [ 126.976509] niks3-server[1131]: 2026/09/20 12:48:00 INFO Completed upload id=396238lb # [ 126.978798] niks3-server[1131]: 2026/09/20 12:48:00 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6239worker2 # [ 126.956325] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40994 duration_s=0 paths=1 nar_bytes_in=5286240worker2 # [ 126.961542] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409946241worker2 # [ 126.970703] nix-daemon[857]: accepted connection from pid 1369, user nix-grpc-daemon (trusted)6242worker2 # [ 126.974381] nix-daemon[857]: reaped child process 1538, status = succeeded6243worker2 # [ 126.979408] nix-daemon[857]: accepted connection from pid 1369, user nix-grpc-daemon (trusted)6244client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6245client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6246lb # [ 127.316994] envoy[973]: conn peer=[2001:db8:1::1]:41334 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2924 tx=756 ms=33176247client # farm-a> LOG-farm-a-during-drain6248worker2 # [ 128.006774] nix-daemon[857]: reaped child process 1552, status = succeeded6249worker2 # [ 128.171801] nix-daemon[857]: accepted connection from pid 1567, user nix-grpc-daemon (trusted)6250worker2 # [ 128.184310] nix-daemon[857]: reaped child process 1570, status = succeeded6251lb # [ 128.222310] niks3-server[1131]: 2026/09/20 12:48:02 INFO Received uploads request method=POST path=/api/pending_closures6252worker2 # [ 128.209548] nix-grpc-daemon[1461]: time=2026-09-20T12:48:02.182Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6253worker2 # [ 128.212062] nix-grpc-daemon[1461]: time=2026-09-20T12:48:02.184Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6254lb # [ 128.259060] niks3-server[1131]: 2026/09/20 12:48:02 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6255lb # [ 128.274135] niks3-server[1131]: 2026/09/20 12:48:02 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6256lb # [ 128.283473] niks3-server[1131]: 2026/09/20 12:48:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6257lb # [ 128.288985] niks3-server[1131]: 2026/09/20 12:48:02 INFO Signed narinfos id=40 count=16258worker2 # [ 128.269364] nix-grpc-daemon[1461]: time=2026-09-20T12:48:02.241Z level=INFO msg="Uploading 1 narinfos"6259lb # [ 128.293091] niks3-server[1131]: 2026/09/20 12:48:02 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6260lb # [ 128.303703] niks3-server[1131]: 2026/09/20 12:48:02 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6261lb # [ 128.309457] niks3-server[1131]: 2026/09/20 12:48:02 INFO Completed upload id=406262worker2 # [ 128.286849] nix-grpc-daemon[1461]: time=2026-09-20T12:48:02.259Z level=INFO msg="Upload complete. (299ms)"6263lb # [ 128.311911] niks3-server[1131]: 2026/09/20 12:48:02 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6264worker2 # [ 128.292607] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40994 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16265worker1 # [ 128.337872] nix-grpc-daemon[1522]: ts=2026-09-20T12:48:01Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06266worker1 # [ 128.340290] nix-grpc-daemon[1522]: ts=2026-09-20T12:48:01Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=16267worker2 # [ 128.307532] nix-daemon[857]: reaped child process 1550, status = succeeded6268worker2 # [ 128.341162] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409946269worker2 # [ 128.359622] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409946270worker2 # [ 128.373484] nix-grpc-daemon[1461]: time=2026-09-20T12:48:02.346Z level=INFO msg="All 1 paths already cached"6271client # copying 1 paths to 'lb:50051'...6272worker2 # [ 128.381746] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409946273client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6274worker2 # [ 128.401339] nix-daemon[857]: accepted connection from pid 1369, user nix-grpc-daemon (trusted)6275worker2 # [ 128.574830] nix-daemon[857]: accepted connection from pid 1574, user nix-grpc-daemon (trusted)6276worker2 # [ 128.587420] nix-daemon[857]: reaped child process 1577, status = succeeded6277worker2 # [ 128.747267] nix-daemon[857]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6278worker2 # [ 128.759275] nix-daemon[857]: reaped child process 1582, status = succeeded6279lb # [ 128.798417] niks3-server[1131]: 2026/09/20 12:48:02 INFO Received uploads request method=POST path=/api/pending_closures6280worker2 # [ 128.785282] nix-grpc-daemon[1461]: time=2026-09-20T12:48:02.757Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6281worker2 # [ 128.786740] nix-grpc-daemon[1461]: time=2026-09-20T12:48:02.757Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6282lb # [ 128.826297] niks3-server[1131]: 2026/09/20 12:48:02 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6283lb # [ 128.836953] niks3-server[1131]: 2026/09/20 12:48:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6284worker2 # [ 128.819885] nix-grpc-daemon[1461]: time=2026-09-20T12:48:02.792Z level=INFO msg="Uploading 1 narinfos"6285lb # [ 128.842395] niks3-server[1131]: 2026/09/20 12:48:02 INFO Signed narinfos id=41 count=16286lb # [ 128.849186] niks3-server[1131]: 2026/09/20 12:48:02 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6287lb # [ 128.860404] niks3-server[1131]: 2026/09/20 12:48:02 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6288lb # [ 128.869964] niks3-server[1131]: 2026/09/20 12:48:02 INFO Completed upload id=416289worker2 # [ 128.849459] nix-grpc-daemon[1461]: time=2026-09-20T12:48:02.822Z level=INFO msg="Upload complete. (434ms)"6290lb # [ 128.873175] niks3-server[1131]: 2026/09/20 12:48:02 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6291worker2 # [ 128.853378] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40994 duration_s=0 paths=1 nar_bytes_in=6406292worker2 # [ 128.858285] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409946293worker2 # [ 128.868127] nix-daemon[857]: accepted connection from pid 1369, user nix-grpc-daemon (trusted)6294worker2 # [ 128.873568] nix-daemon[857]: reaped child process 1572, status = succeeded6295worker2 # [ 128.878563] nix-daemon[857]: accepted connection from pid 1369, user nix-grpc-daemon (trusted)6296client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6297client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6298client # farm-b> LOG-farm-b-during-drain6299worker2 # [ 129.703256] nix-daemon[857]: reaped child process 1586, status = succeeded6300worker2 # [ 129.844535] nix-daemon[857]: accepted connection from pid 1601, user nix-grpc-daemon (trusted)6301worker2 # [ 129.856909] nix-daemon[857]: reaped child process 1604, status = succeeded6302lb # [ 129.896073] niks3-server[1131]: 2026/09/20 12:48:03 INFO Received uploads request method=POST path=/api/pending_closures6303worker2 # [ 129.881295] nix-grpc-daemon[1461]: time=2026-09-20T12:48:03.854Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6304worker2 # [ 129.882793] nix-grpc-daemon[1461]: time=2026-09-20T12:48:03.855Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6305lb # [ 129.943236] niks3-server[1131]: 2026/09/20 12:48:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6306lb # [ 129.946190] niks3-server[1131]: 2026/09/20 12:48:03 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6307worker2 # [ 129.930985] nix-grpc-daemon[1461]: time=2026-09-20T12:48:03.903Z level=INFO msg="Uploading 1 narinfos"6308lb # [ 129.953687] niks3-server[1131]: 2026/09/20 12:48:03 INFO Signed narinfos id=42 count=16309lb # [ 129.959136] niks3-server[1131]: 2026/09/20 12:48:03 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6310lb # [ 129.976840] niks3-server[1131]: 2026/09/20 12:48:03 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6311lb # [ 129.982497] niks3-server[1131]: 2026/09/20 12:48:03 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6312lb # [ 129.984861] niks3-server[1131]: 2026/09/20 12:48:03 INFO Completed upload id=426313lb # [ 129.986088] niks3-server[1131]: 2026/09/20 12:48:03 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6314worker2 # [ 129.962442] nix-grpc-daemon[1461]: time=2026-09-20T12:48:03.935Z level=INFO msg="Upload complete. (277ms)"6315worker1 # [ 130.013947] nix-grpc-daemon[1522]: ts=2026-09-20T12:48:03Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06316worker2 # [ 129.969482] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40994 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16317worker1 # [ 130.016265] nix-grpc-daemon[1522]: ts=2026-09-20T12:48:03Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=16318worker2 # [ 129.983275] nix-daemon[857]: reaped child process 1584, status = succeeded6319worker2 # [ 130.018987] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409946320worker2 # [ 130.032276] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409946321worker2 # [ 130.045684] nix-grpc-daemon[1461]: time=2026-09-20T12:48:04.018Z level=INFO msg="All 2 paths already cached"6322client # copying 1 paths to 'lb:50051'...6323worker2 # [ 130.053793] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409946324client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6325worker2 # [ 130.074170] nix-daemon[857]: accepted connection from pid 1369, user nix-grpc-daemon (trusted)6326worker2 # [ 130.244166] nix-daemon[857]: accepted connection from pid 1608, user nix-grpc-daemon (trusted)6327worker2 # [ 130.256540] nix-daemon[857]: reaped child process 1611, status = succeeded6328worker2 # [ 130.418338] nix-daemon[857]: accepted connection from pid 1613, user nix-grpc-daemon (trusted)6329worker2 # [ 130.430752] nix-daemon[857]: reaped child process 1616, status = succeeded6330lb # [ 130.470118] niks3-server[1131]: 2026/09/20 12:48:04 INFO Received uploads request method=POST path=/api/pending_closures6331worker2 # [ 130.455801] nix-grpc-daemon[1461]: time=2026-09-20T12:48:04.428Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6332worker2 # [ 130.457718] nix-grpc-daemon[1461]: time=2026-09-20T12:48:04.430Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6333lb # [ 130.498657] niks3-server[1131]: 2026/09/20 12:48:04 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6334lb # [ 130.509442] niks3-server[1131]: 2026/09/20 12:48:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6335lb # [ 130.514470] niks3-server[1131]: 2026/09/20 12:48:04 INFO Signed narinfos id=43 count=16336worker2 # [ 130.494725] nix-grpc-daemon[1461]: time=2026-09-20T12:48:04.467Z level=INFO msg="Uploading 1 narinfos"6337lb # [ 130.521815] niks3-server[1131]: 2026/09/20 12:48:04 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6338lb # [ 130.532480] niks3-server[1131]: 2026/09/20 12:48:04 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6339lb # [ 130.538821] niks3-server[1131]: 2026/09/20 12:48:04 INFO Completed upload id=436340worker2 # [ 130.518262] nix-grpc-daemon[1461]: time=2026-09-20T12:48:04.491Z level=INFO msg="Upload complete. (431ms)"6341lb # [ 130.541861] niks3-server[1131]: 2026/09/20 12:48:04 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6342worker2 # [ 130.520972] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40994 duration_s=0 paths=1 nar_bytes_in=7686343worker2 # [ 130.526479] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409946344worker2 # [ 130.535532] nix-daemon[857]: accepted connection from pid 1369, user nix-grpc-daemon (trusted)6345worker2 # [ 130.542896] nix-daemon[857]: reaped child process 1606, status = succeeded6346client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6347worker2 # [ 130.548978] nix-daemon[857]: accepted connection from pid 1369, user nix-grpc-daemon (trusted)6348client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6349client # farm-top> LOG-farm-top-during-drain6350worker2 # [ 131.166553] nix-daemon[857]: reaped child process 1620, status = succeeded6351worker2 # [ 131.317189] nix-daemon[857]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6352worker2 # [ 131.329616] nix-daemon[857]: reaped child process 1638, status = succeeded6353lb # [ 131.369216] niks3-server[1131]: 2026/09/20 12:48:05 INFO Received uploads request method=POST path=/api/pending_closures6354worker2 # [ 131.354247] nix-grpc-daemon[1461]: time=2026-09-20T12:48:05.327Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6355worker2 # [ 131.357062] nix-grpc-daemon[1461]: time=2026-09-20T12:48:05.328Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6356lb # [ 131.402071] niks3-server[1131]: 2026/09/20 12:48:05 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6357lb # [ 131.416610] niks3-server[1131]: 2026/09/20 12:48:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6358lb # [ 131.421743] niks3-server[1131]: 2026/09/20 12:48:05 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6359worker2 # [ 131.405645] nix-grpc-daemon[1461]: time=2026-09-20T12:48:05.378Z level=INFO msg="Uploading 1 narinfos"6360lb # [ 131.428239] niks3-server[1131]: 2026/09/20 12:48:05 INFO Signed narinfos id=44 count=16361lb # [ 131.432675] niks3-server[1131]: 2026/09/20 12:48:05 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6362lb # [ 131.441397] niks3-server[1131]: 2026/09/20 12:48:05 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6363lb # [ 131.450179] niks3-server[1131]: 2026/09/20 12:48:05 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6364lb # [ 131.453286] niks3-server[1131]: 2026/09/20 12:48:05 INFO Completed upload id=446365worker2 # [ 131.431938] nix-grpc-daemon[1461]: time=2026-09-20T12:48:05.404Z level=INFO msg="Upload complete. (278ms)"6366worker2 # [ 131.439374] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40994 duration_s=0 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16367worker2 # [ 131.451874] nix-daemon[857]: reaped child process 1618, status = succeeded6368client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&2, in 5.29 seconds)6369client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath6370client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath, in 0.20 seconds)6371worker1: must succeed: pkill -f 'read -t [9]0 x'6372worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.06 seconds)6373client: waiting for success: ! systemctl is-active sw6374worker1 # [ 131.816435] nix-daemon[912]: reaped child process 1670, status = succeeded6375worker1 # [ 131.978399] nix-daemon[912]: accepted connection from pid 1718, user nix-grpc-daemon (trusted)6376worker1 # [ 131.991922] nix-daemon[912]: reaped child process 1721, status = succeeded6377lb # [ 131.990309] niks3-server[1131]: 2026/09/20 12:48:05 INFO Received uploads request method=POST path=/api/pending_closures6378worker1 # [ 132.014397] nix-grpc-daemon[1601]: time=2026-09-20T12:48:05.440Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6379worker1 # [ 132.015924] nix-grpc-daemon[1601]: time=2026-09-20T12:48:05.442Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6380lb # [ 132.011700] niks3-server[1131]: 2026/09/20 12:48:05 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6381lb # [ 132.016269] niks3-server[1131]: 2026/09/20 12:48:05 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6382lb # [ 132.026364] niks3-server[1131]: 2026/09/20 12:48:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6383lb # [ 132.030105] niks3-server[1131]: 2026/09/20 12:48:05 INFO Signed narinfos id=45 count=16384worker1 # [ 132.051249] nix-grpc-daemon[1601]: time=2026-09-20T12:48:05.476Z level=INFO msg="Uploading 1 narinfos"6385lb # [ 132.034826] niks3-server[1131]: 2026/09/20 12:48:05 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6386lb # [ 132.040147] niks3-server[1131]: 2026/09/20 12:48:05 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6387lb # [ 132.043083] niks3-server[1131]: 2026/09/20 12:48:05 INFO Completed upload id=456388worker1 # [ 132.062456] nix-grpc-daemon[1601]: time=2026-09-20T12:48:05.488Z level=INFO msg="Upload complete. (253ms)"6389lb # [ 132.045455] niks3-server[1131]: 2026/09/20 12:48:05 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6390worker1 # [ 132.067246] nix-grpc-daemon[1522]: ts=2026-09-20T12:48:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38610 duration_s=8 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16391worker1 # [ 132.073267] nix-daemon[912]: reaped child process 1666, status = succeeded6392client # [ 132.104794] systemd[1]: sw.service: Deactivated successfully.6393client # [ 132.108875] systemd[1]: sw.service: Consumed 111ms CPU time over 9.665s wall clock time, 39.1M memory peak, 8.8K incoming IP traffic, 9K outgoing IP traffic.6394worker1 # [ 132.598301] nix-grpc-daemon[1522]: ts=2026-09-20T12:48:06Z level=info event=signal_exit6395lb # [ 132.803478] niks3-server[1131]: 2026/09/20 12:48:06 INFO lead: released remote=192.168.1.3:497366396worker1 # [ 132.820285] nix-grpc-daemon[1522]: ts=2026-09-20T12:48:06Z level=info event=scheduler_yield6397worker1 # [ 132.825904] nix-grpc-daemon[1522]: ts=2026-09-20T12:48:06Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:385966398worker2 # [ 132.784830] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:06Z level=info event=scheduler_restarting addr=lb:500516399worker1 # [ 132.832890] nix-grpc-daemon[1522]: ts=2026-09-20T12:48:06Z level=info event=worker_gone addr=192.168.1.4:500516400worker1 # [ 132.836162] nix-grpc-daemon[1522]: ts=2026-09-20T12:48:06Z level=info event=scheduler_restarting addr=lb:500516401worker1 # [ 132.839251] nix-grpc-daemon[1522]: ts=2026-09-20T12:48:06Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:385966402worker1 # [ 132.842537] nix-grpc-daemon[1522]: ts=2026-09-20T12:48:06Z level=info event=worker_gone addr=192.168.1.3:500516403client: (finished: waiting for success: ! systemctl is-active sw, in 1.09 seconds)6404client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6405client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.04 seconds)6406worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6407worker2 # [ 132.985627] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:06Z level=info event=scheduler_connected addr=lb:500516408worker2 # [ 132.989638] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6409worker1 # [ 133.031084] nix-grpc-daemon[1522]: ts=2026-09-20T12:48:06Z level=info event=scheduler_connected addr=lb:500516410worker1 # [ 133.035155] nix-grpc-daemon[1522]: ts=2026-09-20T12:48:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6411worker2 # [ 133.190755] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:07Z level=info event=scheduler_connected addr=lb:500516412worker2 # [ 133.194683] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6413worker1 # [ 133.237568] nix-grpc-daemon[1522]: ts=2026-09-20T12:48:06Z level=info event=scheduler_connected addr=lb:500516414worker1 # [ 133.240750] nix-grpc-daemon[1522]: ts=2026-09-20T12:48:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6415worker2 # [ 133.395844] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:07Z level=info event=scheduler_connected addr=lb:500516416worker1 # [ 133.438209] nix-grpc-daemon[1522]: ts=2026-09-20T12:48:06Z level=info event=scheduler_connected addr=lb:500516417worker2 # [ 133.399892] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6418worker1 # [ 133.441974] nix-grpc-daemon[1522]: ts=2026-09-20T12:48:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6419worker1 # [ 133.640157] nix-grpc-daemon[1522]: ts=2026-09-20T12:48:07Z level=info event=scheduler_connected addr=lb:500516420worker2 # [ 133.601422] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:07Z level=info event=scheduler_connected addr=lb:500516421worker1 # [ 133.644359] nix-grpc-daemon[1522]: ts=2026-09-20T12:48:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6422worker2 # [ 133.604564] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6423worker1 # [ 133.824289] nix-grpc-daemon[1522]: ts=2026-09-20T12:48:07Z level=info event=scheduler_restarting6424worker1 # [ 133.833566] nix-grpc-daemon[1522]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6425worker1 # [ 133.837170] nix-grpc-daemon[1522]: E0000 00:00:1789908487.259477 1565 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true6426worker1 # [ 133.841893] nix-grpc-daemon[1522]: E0000 00:00:1789908487.259592 1565 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true6427worker2 # [ 133.804087] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:07Z level=info event=scheduler_connected addr=lb:500516428worker2 # [ 133.807703] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6429worker1 # [ 133.859142] nix-grpc-daemon[1522]: ts=2026-09-20T12:48:07Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6430worker1 # [ 133.870702] nix-daemon[912]: reaped child process 1573, status = succeeded6431worker2 # [ 134.005187] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:07Z level=info event=scheduler_connected addr=lb:500516432worker2 # [ 134.009242] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6433worker2 # [ 134.210485] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:08Z level=info event=scheduler_connected addr=lb:500516434worker2 # [ 134.214432] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6435worker2 # [ 134.415405] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:08Z level=info event=scheduler_connected addr=lb:500516436worker2 # [ 134.419215] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6437worker2 # [ 134.620280] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:08Z level=info event=scheduler_connected addr=lb:500516438worker2 # [ 134.624099] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6439worker2 # [ 134.826691] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:08Z level=info event=scheduler_connected addr=lb:500516440worker2 # [ 134.829827] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6441worker2 # [ 135.027877] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:09Z level=info event=scheduler_connected addr=lb:500516442worker2 # [ 135.032273] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6443worker2 # [ 135.233119] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:09Z level=info event=scheduler_connected addr=lb:500516444worker2 # [ 135.236884] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6445worker2 # [ 135.437909] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:09Z level=info event=scheduler_connected addr=lb:500516446worker2 # [ 135.441688] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6447worker1 # [ 135.636728] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6448worker1 # [ 135.640681] systemd[1]: nix-grpc-daemon.service: Consumed 680ms CPU time over 24.427s wall clock time, 31.5M memory peak, 25.5K incoming IP traffic, 32.9K outgoing IP traffic.6449worker1 # [ 135.656140] systemd[1]: Starting Nix worker-protocol over gRPC...6450worker2 # [ 135.642669] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:09Z level=info event=scheduler_connected addr=lb:500516451worker2 # [ 135.646458] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6452worker1 # [ 135.831911] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:09Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6453worker1 # [ 135.834148] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:09Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6454worker1 # [ 135.837698] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:09Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16455worker1 # [ 135.844821] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:09Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6456worker1 # [ 135.846480] systemd[1]: Started Nix worker-protocol over gRPC.6457worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.94 seconds)6458lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6459lb: must succeed: curl -sf localhost:9901/clusters6460worker1 # [ 135.857603] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:09Z level=info event=scheduler_take_over6461lb # [ 135.840523] niks3-server[1131]: 2026/09/20 12:48:09 INFO lead: acquired remote=192.168.1.3:476446462worker1 # [ 135.872096] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:09Z level=info event=scheduler_connected addr=lb:500516463worker1 # [ 135.873870] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6464worker2 # [ 135.848703] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:09Z level=info event=scheduler_connected addr=lb:500516465worker2 # [ 135.852073] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6466lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6467worker1 # [ 136.075064] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:09Z level=info event=scheduler_connected addr=lb:500516468worker1 # [ 136.078545] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6469worker2 # [ 136.050099] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:10Z level=info event=scheduler_connected addr=lb:500516470worker2 # [ 136.053971] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6471worker1 # [ 136.279167] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:09Z level=info event=scheduler_connected addr=lb:500516472worker1 # [ 136.281397] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6473worker2 # [ 136.255171] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:10Z level=info event=scheduler_connected addr=lb:500516474worker2 # [ 136.259114] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6475worker1 # [ 136.482131] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:09Z level=info event=scheduler_connected addr=lb:500516476worker1 # [ 136.484284] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6477worker2 # [ 136.459575] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:10Z level=info event=scheduler_connected addr=lb:500516478worker2 # [ 136.461503] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6479worker1 # [ 136.684750] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:10Z level=info event=scheduler_connected addr=lb:500516480worker1 # [ 136.688452] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:10Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:460686481worker1 # [ 136.689986] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:10Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06482worker2 # [ 136.662709] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:10Z level=info event=scheduler_connected addr=lb:500516483worker1 # [ 136.707778] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:10Z level=info event=oidc_jwks provider=mock keys=16484worker1 # [ 136.709199] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:10Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:460686485worker1 # [ 136.710831] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:10Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06486lb: must succeed: curl -sf localhost:9901/clusters6487lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6488lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.13 seconds)6489lb: must succeed: curl -sf localhost:9901/clusters6490lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6491worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6492worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6493(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 14.72 seconds)6494subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6495client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16496lb # [ 137.317631] envoy[973]: conn peer=[2001:db8:1::1]:55746 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9568 tx=3828 ms=50426497lb # [ 137.320152] envoy[973]: conn peer=[2001:db8:1::1]:41344 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2949 tx=1349 ms=94376498lb # [ 137.322131] envoy[973]: conn peer=[2001:db8:1::3]:43854 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1746 tx=539 ms=222466499worker2 # [ 137.304587] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:11Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:409946500worker1 # [ 137.354068] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460746501worker1 # [ 137.363220] nix-daemon[912]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)6502worker1 # [ 137.379643] nix-grpc-daemon[1735]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 11 ms (attempt 1/5)6503worker1 # [ 137.396419] nix-grpc-daemon[1735]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 176 ms (attempt 2/5)6504worker1 # [ 137.578207] nix-grpc-daemon[1735]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 39 ms (attempt 3/5)6505worker1 # [ 137.617439] nix-grpc-daemon[1735]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 667 ms (attempt 4/5)6506worker1 # [ 138.288495] nix-grpc-daemon[1735]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6507worker1 # [ 138.321430] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:11Z level=debug event=want client=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06508worker1 # [ 138.326506] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:11Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16509worker1 # [ 138.330834] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:11Z level=debug event=want client=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06510worker1 # [ 138.334555] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:11Z level=debug event=want client=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06511worker1 # [ 138.367749] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460746512worker1 # [ 138.377759] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460746513worker1 # [ 138.387395] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:460746514worker1 # [ 138.399342] nix-daemon[912]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)6515worker1 # [ 138.417546] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:11Z level=info event=niks3_push_started pid=17966516worker1 # [ 138.590856] nix-grpc-daemon[1796]: time=2026-09-20T12:48:12.016Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""6517worker1 # [ 138.739577] nix-daemon[912]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)6518worker1 # [ 138.751625] nix-daemon[912]: reaped child process 1812, status = succeeded6519worker1 # [ 138.896333] nix-daemon[912]: accepted connection from pid 1814, user nix-grpc-daemon (trusted)6520worker1 # [ 138.909550] nix-daemon[912]: reaped child process 1817, status = succeeded6521lb # [ 138.902737] niks3-server[1131]: 2026/09/20 12:48:12 INFO Received uploads request method=POST path=/api/pending_closures6522worker1 # [ 138.925391] nix-grpc-daemon[1796]: time=2026-09-20T12:48:12.351Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6523worker1 # [ 138.927125] nix-grpc-daemon[1796]: time=2026-09-20T12:48:12.352Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6524lb # [ 138.925420] niks3-server[1131]: 2026/09/20 12:48:12 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6525lb # [ 138.927909] niks3-server[1131]: 2026/09/20 12:48:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6526lb # [ 138.930948] niks3-server[1131]: 2026/09/20 12:48:12 INFO Signed narinfos id=46 count=16527worker1 # [ 138.949380] nix-grpc-daemon[1796]: time=2026-09-20T12:48:12.375Z level=INFO msg="Uploading 1 narinfos"6528lb # [ 138.933459] niks3-server[1131]: 2026/09/20 12:48:12 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6529lb # [ 138.939054] niks3-server[1131]: 2026/09/20 12:48:12 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6530lb # [ 138.942713] niks3-server[1131]: 2026/09/20 12:48:12 INFO Completed upload id=466531worker1 # [ 138.960848] nix-grpc-daemon[1796]: time=2026-09-20T12:48:12.386Z level=INFO msg="Upload complete. (365ms)"6532lb # [ 138.944274] niks3-server[1131]: 2026/09/20 12:48:12 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6533worker1 # [ 138.962957] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46074 duration_s=0 paths=1 nar_bytes_in=5046534worker1 # [ 138.966906] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460746535worker1 # [ 138.973455] nix-daemon[912]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)6536worker1 # [ 138.975987] nix-daemon[912]: reaped child process 1794, status = succeeded6537worker1 # [ 138.981277] nix-daemon[912]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)6538worker1 # [ 140.197516] nix-daemon[912]: reaped child process 1821, status = succeeded6539worker1 # [ 140.360218] nix-daemon[912]: accepted connection from pid 1836, user nix-grpc-daemon (trusted)6540worker1 # [ 140.373261] nix-daemon[912]: reaped child process 1839, status = succeeded6541lb # [ 140.373330] niks3-server[1131]: 2026/09/20 12:48:14 INFO Received uploads request method=POST path=/api/pending_closures6542worker1 # [ 140.399972] nix-grpc-daemon[1796]: time=2026-09-20T12:48:13.825Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6543worker1 # [ 140.401532] nix-grpc-daemon[1796]: time=2026-09-20T12:48:13.827Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6544lb # [ 140.414403] niks3-server[1131]: 2026/09/20 12:48:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6545lb # [ 140.423486] niks3-server[1131]: 2026/09/20 12:48:14 INFO Signed narinfos id=47 count=16546worker1 # [ 140.445090] nix-grpc-daemon[1796]: time=2026-09-20T12:48:13.870Z level=INFO msg="Uploading 1 narinfos"6547lb # [ 140.433698] niks3-server[1131]: 2026/09/20 12:48:14 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6548lb # [ 140.438050] niks3-server[1131]: 2026/09/20 12:48:14 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6549lb # [ 140.454216] niks3-server[1131]: 2026/09/20 12:48:14 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6550lb # [ 140.457157] niks3-server[1131]: 2026/09/20 12:48:14 INFO Completed upload id=476551lb # [ 140.458542] niks3-server[1131]: 2026/09/20 12:48:14 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6552worker1 # [ 140.476665] nix-grpc-daemon[1796]: time=2026-09-20T12:48:13.902Z level=INFO msg="Upload complete. (287ms)"6553lb # [ 140.463199] niks3-server[1131]: 2026/09/20 12:48:14 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6554worker1 # [ 140.489623] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46074 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=1 outputs=16555worker1 # [ 140.495447] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:13Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16556worker1 # [ 140.500158] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460746557worker1 # [ 140.502413] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:13Z level=debug event=want client=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06558worker1 # [ 140.508928] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460746559worker1 # [ 140.511185] nix-daemon[912]: reaped child process 1819, status = succeeded6560worker1 # [ 140.514972] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:460746561worker1 # [ 140.519154] nix-daemon[912]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)6562worker1 # [ 140.691473] nix-daemon[912]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)6563worker1 # [ 140.704634] nix-daemon[912]: reaped child process 1848, status = succeeded6564worker1 # [ 140.883396] nix-daemon[912]: accepted connection from pid 1850, user nix-grpc-daemon (trusted)6565worker1 # [ 140.896597] nix-daemon[912]: reaped child process 1853, status = succeeded6566lb # [ 140.894546] niks3-server[1131]: 2026/09/20 12:48:14 INFO Received uploads request method=POST path=/api/pending_closures6567worker1 # [ 140.923170] nix-grpc-daemon[1796]: time=2026-09-20T12:48:14.349Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6568worker1 # [ 140.925703] nix-grpc-daemon[1796]: time=2026-09-20T12:48:14.351Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6569lb # [ 140.926875] niks3-server[1131]: 2026/09/20 12:48:14 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6570lb # [ 140.936726] niks3-server[1131]: 2026/09/20 12:48:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6571worker1 # [ 140.961735] nix-grpc-daemon[1796]: time=2026-09-20T12:48:14.387Z level=INFO msg="Uploading 1 narinfos"6572lb # [ 140.943550] niks3-server[1131]: 2026/09/20 12:48:14 INFO Signed narinfos id=48 count=16573lb # [ 140.947695] niks3-server[1131]: 2026/09/20 12:48:14 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6574lb # [ 140.959366] niks3-server[1131]: 2026/09/20 12:48:14 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6575lb # [ 140.965110] niks3-server[1131]: 2026/09/20 12:48:14 INFO Completed upload id=486576worker1 # [ 140.983241] nix-grpc-daemon[1796]: time=2026-09-20T12:48:14.409Z level=INFO msg="Upload complete. (458ms)"6577lb # [ 140.969201] niks3-server[1131]: 2026/09/20 12:48:14 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6578worker1 # [ 140.988095] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46074 duration_s=0 paths=1 nar_bytes_in=5046579worker1 # [ 140.992919] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460746580worker1 # [ 141.001464] nix-daemon[912]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)6581worker1 # [ 141.007289] nix-daemon[912]: reaped child process 1843, status = succeeded6582worker1 # [ 141.012327] nix-daemon[912]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)6583worker1 # [ 142.027847] nix-daemon[912]: reaped child process 1857, status = succeeded6584worker1 # [ 142.169330] nix-daemon[912]: accepted connection from pid 1872, user nix-grpc-daemon (trusted)6585worker1 # [ 142.182359] nix-daemon[912]: reaped child process 1875, status = succeeded6586lb # [ 142.178703] niks3-server[1131]: 2026/09/20 12:48:15 INFO Received uploads request method=POST path=/api/pending_closures6587worker1 # [ 142.205182] nix-grpc-daemon[1796]: time=2026-09-20T12:48:15.630Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6588worker1 # [ 142.206843] nix-grpc-daemon[1796]: time=2026-09-20T12:48:15.632Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6589lb # [ 142.208739] niks3-server[1131]: 2026/09/20 12:48:16 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6590lb # [ 142.222073] niks3-server[1131]: 2026/09/20 12:48:16 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6591lb # [ 142.233816] niks3-server[1131]: 2026/09/20 12:48:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6592lb # [ 142.239433] niks3-server[1131]: 2026/09/20 12:48:16 INFO Signed narinfos id=49 count=16593worker1 # [ 142.260121] nix-grpc-daemon[1796]: time=2026-09-20T12:48:15.685Z level=INFO msg="Uploading 1 narinfos"6594lb # [ 142.244797] niks3-server[1131]: 2026/09/20 12:48:16 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6595lb # [ 142.253202] niks3-server[1131]: 2026/09/20 12:48:16 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6596worker1 # [ 142.277123] nix-grpc-daemon[1796]: time=2026-09-20T12:48:15.703Z level=INFO msg="Upload complete. (257ms)"6597lb # [ 142.259096] niks3-server[1131]: 2026/09/20 12:48:16 INFO Completed upload id=496598lb # [ 142.262130] niks3-server[1131]: 2026/09/20 12:48:16 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6599worker1 # [ 142.287074] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46074 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=2 outputs=16600worker1 # [ 142.291715] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:15Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16601worker1 # [ 142.297050] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460746602worker1 # [ 142.301183] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:15Z level=debug event=want client=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06603worker1 # [ 142.310112] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460746604worker1 # [ 142.312347] nix-daemon[912]: reaped child process 1855, status = succeeded6605worker1 # [ 142.316617] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:460746606worker1 # [ 142.320615] nix-daemon[912]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)6607worker1 # [ 142.489455] nix-daemon[912]: accepted connection from pid 1881, user nix-grpc-daemon (trusted)6608worker1 # [ 142.502660] nix-daemon[912]: reaped child process 1884, status = succeeded6609worker1 # [ 142.655811] nix-daemon[912]: accepted connection from pid 1886, user nix-grpc-daemon (trusted)6610worker1 # [ 142.668512] nix-daemon[912]: reaped child process 1889, status = succeeded6611lb # [ 142.663589] niks3-server[1131]: 2026/09/20 12:48:16 INFO Received uploads request method=POST path=/api/pending_closures6612worker1 # [ 142.689561] nix-grpc-daemon[1796]: time=2026-09-20T12:48:16.115Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6613worker1 # [ 142.690935] nix-grpc-daemon[1796]: time=2026-09-20T12:48:16.115Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6614lb # [ 142.688664] niks3-server[1131]: 2026/09/20 12:48:16 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6615lb # [ 142.698955] niks3-server[1131]: 2026/09/20 12:48:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6616lb # [ 142.704114] niks3-server[1131]: 2026/09/20 12:48:16 INFO Signed narinfos id=50 count=16617worker1 # [ 142.725276] nix-grpc-daemon[1796]: time=2026-09-20T12:48:16.151Z level=INFO msg="Uploading 1 narinfos"6618lb # [ 142.710751] niks3-server[1131]: 2026/09/20 12:48:16 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6619lb # [ 142.722129] niks3-server[1131]: 2026/09/20 12:48:16 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6620worker1 # [ 142.746535] nix-grpc-daemon[1796]: time=2026-09-20T12:48:16.172Z level=INFO msg="Upload complete. (420ms)"6621lb # [ 142.728626] niks3-server[1131]: 2026/09/20 12:48:16 INFO Completed upload id=506622worker1 # [ 142.748626] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46074 duration_s=0 paths=1 nar_bytes_in=5046623lb # [ 142.732125] niks3-server[1131]: 2026/09/20 12:48:16 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6624worker1 # [ 142.754225] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460746625worker1 # [ 142.764404] nix-daemon[912]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)6626worker1 # [ 142.766085] nix-daemon[912]: reaped child process 1879, status = succeeded6627worker1 # [ 142.771191] nix-daemon[912]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)6628worker1 # [ 143.787222] nix-daemon[912]: reaped child process 1893, status = succeeded6629worker1 # [ 143.948717] nix-daemon[912]: accepted connection from pid 1908, user nix-grpc-daemon (trusted)6630worker1 # [ 143.961807] nix-daemon[912]: reaped child process 1911, status = succeeded6631lb # [ 143.959329] niks3-server[1131]: 2026/09/20 12:48:17 INFO Received uploads request method=POST path=/api/pending_closures6632worker1 # [ 143.985616] nix-grpc-daemon[1796]: time=2026-09-20T12:48:17.411Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6633worker1 # [ 143.988576] nix-grpc-daemon[1796]: time=2026-09-20T12:48:17.413Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6634lb # [ 143.990370] niks3-server[1131]: 2026/09/20 12:48:17 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv6635lb # [ 144.001770] niks3-server[1131]: 2026/09/20 12:48:17 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6636lb # [ 144.013578] niks3-server[1131]: 2026/09/20 12:48:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6637lb # [ 144.018988] niks3-server[1131]: 2026/09/20 12:48:17 INFO Signed narinfos id=51 count=16638worker1 # [ 144.039857] nix-grpc-daemon[1796]: time=2026-09-20T12:48:17.465Z level=INFO msg="Uploading 1 narinfos"6639lb # [ 144.024090] niks3-server[1131]: 2026/09/20 12:48:17 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls6640lb # [ 144.034305] niks3-server[1131]: 2026/09/20 12:48:17 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6641lb # [ 144.041182] niks3-server[1131]: 2026/09/20 12:48:17 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo6642lb # [ 144.044071] niks3-server[1131]: 2026/09/20 12:48:17 INFO Completed upload id=516643worker1 # [ 144.062235] nix-grpc-daemon[1796]: time=2026-09-20T12:48:17.488Z level=INFO msg="Upload complete. (283ms)"6644worker1 # [ 144.070651] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46074 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=3 outputs=16645worker1 # [ 144.079729] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:17Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=16646worker1 # [ 144.084296] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460746647worker1 # [ 144.087834] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:17Z level=debug event=want client=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=06648worker1 # [ 144.098544] nix-daemon[912]: reaped child process 1891, status = succeeded6649worker1 # [ 144.100238] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460746650worker1 # [ 144.108029] nix-grpc-daemon[1796]: time=2026-09-20T12:48:17.534Z level=INFO msg="All 1 paths already cached"6651worker1 # [ 144.112340] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:460746652worker1 # [ 144.116346] nix-daemon[912]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)6653worker1 # [ 144.294203] nix-daemon[912]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)6654worker1 # [ 144.307750] nix-daemon[912]: reaped child process 1920, status = succeeded6655worker1 # [ 144.480652] nix-daemon[912]: accepted connection from pid 1922, user nix-grpc-daemon (trusted)6656worker1 # [ 144.494040] nix-daemon[912]: reaped child process 1925, status = succeeded6657lb # [ 144.492542] niks3-server[1131]: 2026/09/20 12:48:18 INFO Received uploads request method=POST path=/api/pending_closures6658worker1 # [ 144.519339] nix-grpc-daemon[1796]: time=2026-09-20T12:48:17.945Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6659worker1 # [ 144.521229] nix-grpc-daemon[1796]: time=2026-09-20T12:48:17.947Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"6660lb # [ 144.525140] niks3-server[1131]: 2026/09/20 12:48:18 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst6661lb # [ 144.534372] niks3-server[1131]: 2026/09/20 12:48:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6662worker1 # [ 144.558904] nix-grpc-daemon[1796]: time=2026-09-20T12:48:17.984Z level=INFO msg="Uploading 1 narinfos"6663lb # [ 144.541086] niks3-server[1131]: 2026/09/20 12:48:18 INFO Signed narinfos id=52 count=16664lb # [ 144.545364] niks3-server[1131]: 2026/09/20 12:48:18 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls6665lb # [ 144.557583] niks3-server[1131]: 2026/09/20 12:48:18 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6666lb # [ 144.563451] niks3-server[1131]: 2026/09/20 12:48:18 INFO Completed upload id=526667worker1 # [ 144.583803] nix-grpc-daemon[1796]: time=2026-09-20T12:48:18.009Z level=INFO msg="Upload complete. (461ms)"6668lb # [ 144.566464] niks3-server[1131]: 2026/09/20 12:48:18 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo6669worker1 # [ 144.588425] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46074 duration_s=0 paths=1 nar_bytes_in=6246670worker1 # [ 144.594263] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460746671worker1 # [ 144.604151] nix-daemon[912]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)6672worker1 # [ 144.610419] nix-daemon[912]: reaped child process 1915, status = succeeded6673worker1 # [ 144.614967] nix-daemon[912]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)6674worker1 # [ 145.430389] nix-daemon[912]: reaped child process 1929, status = succeeded6675worker1 # [ 145.578696] nix-daemon[912]: accepted connection from pid 1944, user nix-grpc-daemon (trusted)6676worker1 # [ 145.592071] nix-daemon[912]: reaped child process 1947, status = succeeded6677lb # [ 145.591660] niks3-server[1131]: 2026/09/20 12:48:19 INFO Received uploads request method=POST path=/api/pending_closures6678worker1 # [ 145.614589] nix-grpc-daemon[1796]: time=2026-09-20T12:48:19.040Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6679worker1 # [ 145.616268] nix-grpc-daemon[1796]: time=2026-09-20T12:48:19.042Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"6680lb # [ 145.615920] niks3-server[1131]: 2026/09/20 12:48:19 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv6681lb # [ 145.617780] niks3-server[1131]: 2026/09/20 12:48:19 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst6682lb # [ 145.622106] niks3-server[1131]: 2026/09/20 12:48:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6683lb # [ 145.623785] niks3-server[1131]: 2026/09/20 12:48:19 INFO Signed narinfos id=53 count=16684worker1 # [ 145.642193] nix-grpc-daemon[1796]: time=2026-09-20T12:48:19.068Z level=INFO msg="Uploading 1 narinfos"6685lb # [ 145.629599] niks3-server[1131]: 2026/09/20 12:48:19 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls6686lb # [ 145.632599] niks3-server[1131]: 2026/09/20 12:48:19 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6687lb # [ 145.635211] niks3-server[1131]: 2026/09/20 12:48:19 INFO Completed upload id=536688worker1 # [ 145.654586] nix-grpc-daemon[1796]: time=2026-09-20T12:48:19.080Z level=INFO msg="Upload complete. (232ms)"6689lb # [ 145.637758] niks3-server[1131]: 2026/09/20 12:48:19 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo6690worker1 # [ 145.656973] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46074 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=4 outputs=16691worker1 # [ 145.662442] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:19Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=16692worker1 # [ 145.664772] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460746693worker1 # [ 145.667811] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:19Z level=debug event=want client=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=06694worker1 # [ 145.674168] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460746695worker1 # [ 145.676402] nix-daemon[912]: reaped child process 1927, status = succeeded6696worker1 # [ 145.680504] nix-grpc-daemon[1796]: time=2026-09-20T12:48:19.106Z level=INFO msg="All 1 paths already cached"6697worker1 # [ 145.684369] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:460746698worker1 # [ 145.688515] nix-daemon[912]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)6699worker1 # [ 145.869871] nix-daemon[912]: accepted connection from pid 1953, user nix-grpc-daemon (trusted)6700worker1 # [ 145.885075] nix-daemon[912]: reaped child process 1956, status = succeeded6701worker1 # [ 146.063455] nix-daemon[912]: accepted connection from pid 1958, user nix-grpc-daemon (trusted)6702worker1 # [ 146.077038] nix-daemon[912]: reaped child process 1961, status = succeeded6703lb # [ 146.076161] niks3-server[1131]: 2026/09/20 12:48:19 INFO Received uploads request method=POST path=/api/pending_closures6704worker1 # [ 146.102315] nix-grpc-daemon[1796]: time=2026-09-20T12:48:19.528Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6705worker1 # [ 146.103721] nix-grpc-daemon[1796]: time=2026-09-20T12:48:19.528Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"6706lb # [ 146.101071] niks3-server[1131]: 2026/09/20 12:48:19 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst6707lb # [ 146.111969] niks3-server[1131]: 2026/09/20 12:48:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6708lb # [ 146.118293] niks3-server[1131]: 2026/09/20 12:48:19 INFO Signed narinfos id=54 count=16709worker1 # [ 146.139663] nix-grpc-daemon[1796]: time=2026-09-20T12:48:19.565Z level=INFO msg="Uploading 1 narinfos"6710lb # [ 146.122988] niks3-server[1131]: 2026/09/20 12:48:19 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls6711lb # [ 146.135407] niks3-server[1131]: 2026/09/20 12:48:19 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6712lb # [ 146.142789] niks3-server[1131]: 2026/09/20 12:48:19 INFO Completed upload id=546713worker1 # [ 146.160847] nix-grpc-daemon[1796]: time=2026-09-20T12:48:19.586Z level=INFO msg="Upload complete. (466ms)"6714lb # [ 146.145386] niks3-server[1131]: 2026/09/20 12:48:19 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo6715worker1 # [ 146.164153] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46074 duration_s=0 paths=1 nar_bytes_in=6246716worker1 # [ 146.170257] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460746717worker1 # [ 146.181661] nix-daemon[912]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)6718worker1 # [ 146.184821] nix-daemon[912]: reaped child process 1951, status = succeeded6719worker1 # [ 146.190095] nix-daemon[912]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)6720worker1 # [ 147.205710] nix-daemon[912]: reaped child process 1965, status = succeeded6721worker1 # [ 147.362687] nix-daemon[912]: accepted connection from pid 1980, user nix-grpc-daemon (trusted)6722worker1 # [ 147.375980] nix-daemon[912]: reaped child process 1983, status = succeeded6723lb # [ 147.378764] niks3-server[1131]: 2026/09/20 12:48:21 INFO Received uploads request method=POST path=/api/pending_closures6724worker1 # [ 147.403332] nix-grpc-daemon[1796]: time=2026-09-20T12:48:20.829Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6725worker1 # [ 147.406562] nix-grpc-daemon[1796]: time=2026-09-20T12:48:20.831Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"6726lb # [ 147.424137] niks3-server[1131]: 2026/09/20 12:48:21 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv6727lb # [ 147.429609] niks3-server[1131]: 2026/09/20 12:48:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6728worker1 # [ 147.451962] nix-grpc-daemon[1796]: time=2026-09-20T12:48:20.877Z level=INFO msg="Uploading 1 narinfos"6729lb # [ 147.434252] niks3-server[1131]: 2026/09/20 12:48:21 INFO Signed narinfos id=55 count=16730lb # [ 147.436341] niks3-server[1131]: 2026/09/20 12:48:21 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst6731lb # [ 147.447262] niks3-server[1131]: 2026/09/20 12:48:21 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls6732lb # [ 147.453864] niks3-server[1131]: 2026/09/20 12:48:21 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6733lb # [ 147.457474] niks3-server[1131]: 2026/09/20 12:48:21 INFO Completed upload id=556734lb # [ 147.459520] niks3-server[1131]: 2026/09/20 12:48:21 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo6735worker1 # [ 147.477077] nix-grpc-daemon[1796]: time=2026-09-20T12:48:20.903Z level=INFO msg="Upload complete. (279ms)"6736worker1 # [ 147.487989] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46074 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=5 outputs=16737worker1 # [ 147.496137] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:20Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=16738worker1 # [ 147.499603] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460746739worker1 # [ 147.502674] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:20Z level=debug event=want client=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=06740worker1 # [ 147.512709] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460746741worker1 # [ 147.514900] nix-daemon[912]: reaped child process 1963, status = succeeded6742worker1 # [ 147.519812] nix-grpc-daemon[1796]: time=2026-09-20T12:48:20.945Z level=INFO msg="All 1 paths already cached"6743worker1 # [ 147.524364] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:460746744worker1 # [ 147.528520] nix-daemon[912]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)6745worker1 # [ 147.695323] nix-daemon[912]: accepted connection from pid 1989, user nix-grpc-daemon (trusted)6746worker1 # [ 147.708763] nix-daemon[912]: reaped child process 1992, status = succeeded6747worker1 # [ 147.869923] nix-daemon[912]: accepted connection from pid 1994, user nix-grpc-daemon (trusted)6748worker1 # [ 147.882888] nix-daemon[912]: reaped child process 1997, status = succeeded6749lb # [ 147.882656] niks3-server[1131]: 2026/09/20 12:48:21 INFO Received uploads request method=POST path=/api/pending_closures6750worker1 # [ 147.908294] nix-grpc-daemon[1796]: time=2026-09-20T12:48:21.334Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6751worker1 # [ 147.909763] nix-grpc-daemon[1796]: time=2026-09-20T12:48:21.334Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"6752lb # [ 147.906291] niks3-server[1131]: 2026/09/20 12:48:21 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst6753lb # [ 147.918639] niks3-server[1131]: 2026/09/20 12:48:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6754worker1 # [ 147.941338] nix-grpc-daemon[1796]: time=2026-09-20T12:48:21.367Z level=INFO msg="Uploading 1 narinfos"6755lb # [ 147.923747] niks3-server[1131]: 2026/09/20 12:48:21 INFO Signed narinfos id=56 count=16756lb # [ 147.929549] niks3-server[1131]: 2026/09/20 12:48:21 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls6757lb # [ 147.941214] niks3-server[1131]: 2026/09/20 12:48:21 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6758worker1 # [ 147.965234] nix-grpc-daemon[1796]: time=2026-09-20T12:48:21.391Z level=INFO msg="Upload complete. (430ms)"6759lb # [ 147.947392] niks3-server[1131]: 2026/09/20 12:48:21 INFO Completed upload id=566760worker1 # [ 147.967377] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46074 duration_s=0 paths=1 nar_bytes_in=6246761lb # [ 147.951477] niks3-server[1131]: 2026/09/20 12:48:21 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo6762worker1 # [ 147.973429] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460746763worker1 # [ 147.978618] nix-daemon[912]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)6764worker1 # [ 147.980985] nix-daemon[912]: reaped child process 1987, status = succeeded6765worker1 # [ 147.986311] nix-daemon[912]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)6766worker1 # [ 149.000075] nix-daemon[912]: reaped child process 2001, status = succeeded6767worker1 # [ 149.152989] nix-daemon[912]: accepted connection from pid 2016, user nix-grpc-daemon (trusted)6768worker1 # [ 149.166084] nix-daemon[912]: reaped child process 2019, status = succeeded6769lb # [ 149.166213] niks3-server[1131]: 2026/09/20 12:48:22 INFO Received uploads request method=POST path=/api/pending_closures6770worker1 # [ 149.193228] nix-grpc-daemon[1796]: time=2026-09-20T12:48:22.619Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6771worker1 # [ 149.194865] nix-grpc-daemon[1796]: time=2026-09-20T12:48:22.620Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"6772lb # [ 149.203745] niks3-server[1131]: 2026/09/20 12:48:23 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv6773lb # [ 149.209291] niks3-server[1131]: 2026/09/20 12:48:23 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst6774lb # [ 149.221423] niks3-server[1131]: 2026/09/20 12:48:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6775worker1 # [ 149.244640] nix-grpc-daemon[1796]: time=2026-09-20T12:48:22.670Z level=INFO msg="Uploading 1 narinfos"6776lb # [ 149.227771] niks3-server[1131]: 2026/09/20 12:48:23 INFO Signed narinfos id=57 count=16777lb # [ 149.231684] niks3-server[1131]: 2026/09/20 12:48:23 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls6778lb # [ 149.241606] niks3-server[1131]: 2026/09/20 12:48:23 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6779worker1 # [ 149.265753] nix-grpc-daemon[1796]: time=2026-09-20T12:48:22.691Z level=INFO msg="Upload complete. (272ms)"6780lb # [ 149.247657] niks3-server[1131]: 2026/09/20 12:48:23 INFO Completed upload id=576781lb # [ 149.250936] niks3-server[1131]: 2026/09/20 12:48:23 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo6782worker1 # [ 149.269582] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46074 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=6 outputs=16783worker1 # [ 149.272710] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:22Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=16784worker1 # [ 149.275964] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460746785worker1 # [ 149.278678] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:22Z level=debug event=want client=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=06786worker1 # [ 149.285881] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460746787worker1 # [ 149.288241] nix-daemon[912]: reaped child process 1999, status = succeeded6788worker1 # [ 149.293128] nix-grpc-daemon[1796]: time=2026-09-20T12:48:22.719Z level=INFO msg="All 2 paths already cached"6789worker1 # [ 149.297848] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:460746790worker1 # [ 149.301756] nix-daemon[912]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)6791worker1 # [ 149.463233] nix-daemon[912]: accepted connection from pid 2025, user nix-grpc-daemon (trusted)6792worker1 # [ 149.477253] nix-daemon[912]: reaped child process 2028, status = succeeded6793worker1 # [ 149.641507] nix-daemon[912]: accepted connection from pid 2030, user nix-grpc-daemon (trusted)6794worker1 # [ 149.655485] nix-daemon[912]: reaped child process 2033, status = succeeded6795lb # [ 149.654168] niks3-server[1131]: 2026/09/20 12:48:23 INFO Received uploads request method=POST path=/api/pending_closures6796worker1 # [ 149.676610] nix-grpc-daemon[1796]: time=2026-09-20T12:48:23.102Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6797worker1 # [ 149.678279] nix-grpc-daemon[1796]: time=2026-09-20T12:48:23.104Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"6798lb # [ 149.680707] niks3-server[1131]: 2026/09/20 12:48:23 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst6799lb # [ 149.690554] niks3-server[1131]: 2026/09/20 12:48:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6800worker1 # [ 149.713699] nix-grpc-daemon[1796]: time=2026-09-20T12:48:23.139Z level=INFO msg="Uploading 1 narinfos"6801lb # [ 149.695729] niks3-server[1131]: 2026/09/20 12:48:23 INFO Signed narinfos id=58 count=16802lb # [ 149.699893] niks3-server[1131]: 2026/09/20 12:48:23 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls6803lb # [ 149.712855] niks3-server[1131]: 2026/09/20 12:48:23 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6804lb # [ 149.717679] niks3-server[1131]: 2026/09/20 12:48:23 INFO Completed upload id=586805worker1 # [ 149.735702] nix-grpc-daemon[1796]: time=2026-09-20T12:48:23.161Z level=INFO msg="Upload complete. (427ms)"6806lb # [ 149.720173] niks3-server[1131]: 2026/09/20 12:48:23 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo6807worker1 # [ 149.739446] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46074 duration_s=0 paths=1 nar_bytes_in=7526808worker1 # [ 149.744852] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460746809worker1 # [ 149.753665] nix-daemon[912]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)6810worker1 # [ 149.757218] nix-daemon[912]: reaped child process 2023, status = succeeded6811worker1 # [ 149.762215] nix-daemon[912]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)6812worker1 # [ 151.189463] nix-daemon[912]: reaped child process 2037, status = succeeded6813worker1 # [ 151.358322] nix-daemon[912]: accepted connection from pid 2052, user nix-grpc-daemon (trusted)6814worker1 # [ 151.371407] nix-daemon[912]: reaped child process 2055, status = succeeded6815lb # [ 151.367753] niks3-server[1131]: 2026/09/20 12:48:25 INFO Received uploads request method=POST path=/api/pending_closures6816worker1 # [ 151.391420] nix-grpc-daemon[1796]: time=2026-09-20T12:48:24.817Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6817worker1 # [ 151.393034] nix-grpc-daemon[1796]: time=2026-09-20T12:48:24.819Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"6818lb # [ 151.393325] niks3-server[1131]: 2026/09/20 12:48:25 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst6819lb # [ 151.396202] niks3-server[1131]: 2026/09/20 12:48:25 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv6820lb # [ 151.399450] niks3-server[1131]: 2026/09/20 12:48:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6821lb # [ 151.401459] niks3-server[1131]: 2026/09/20 12:48:25 INFO Signed narinfos id=59 count=16822worker1 # [ 151.419761] nix-grpc-daemon[1796]: time=2026-09-20T12:48:24.845Z level=INFO msg="Uploading 1 narinfos"6823lb # [ 151.405858] niks3-server[1131]: 2026/09/20 12:48:25 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls6824lb # [ 151.409148] niks3-server[1131]: 2026/09/20 12:48:25 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6825lb # [ 151.412565] niks3-server[1131]: 2026/09/20 12:48:25 INFO Completed upload id=596826worker1 # [ 151.430394] nix-grpc-daemon[1796]: time=2026-09-20T12:48:24.856Z level=INFO msg="Upload complete. (258ms)"6827lb # [ 151.415159] niks3-server[1131]: 2026/09/20 12:48:25 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo6828worker1 # [ 151.434369] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46074 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=7 outputs=16829worker1 # [ 151.437376] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:24Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=16830worker1 # [ 151.439796] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460746831worker1 # [ 151.446288] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460746832worker1 # [ 151.448803] nix-daemon[912]: reaped child process 2035, status = succeeded6833worker1 # [ 151.452325] nix-grpc-daemon[1796]: time=2026-09-20T12:48:24.878Z level=INFO msg="All 2 paths already cached"6834worker1 # [ 151.456056] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:460746835worker1 # [ 151.461357] nix-daemon[912]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)6836worker1 # [ 151.629329] nix-daemon[912]: accepted connection from pid 2059, user nix-grpc-daemon (trusted)6837worker1 # [ 151.642451] nix-daemon[912]: reaped child process 2062, status = succeeded6838worker1 # [ 151.809107] nix-daemon[912]: accepted connection from pid 2064, user nix-grpc-daemon (trusted)6839worker1 # [ 151.821934] nix-daemon[912]: reaped child process 2067, status = succeeded6840lb # [ 151.820761] niks3-server[1131]: 2026/09/20 12:48:25 INFO Received uploads request method=POST path=/api/pending_closures6841worker1 # [ 151.848852] nix-grpc-daemon[1796]: time=2026-09-20T12:48:25.274Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6842worker1 # [ 151.850428] nix-grpc-daemon[1796]: time=2026-09-20T12:48:25.276Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"6843lb # [ 151.855988] niks3-server[1131]: 2026/09/20 12:48:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6844lb # [ 151.860714] niks3-server[1131]: 2026/09/20 12:48:25 INFO Signed narinfos id=60 count=16845worker1 # [ 151.881920] nix-grpc-daemon[1796]: time=2026-09-20T12:48:25.308Z level=INFO msg="Uploading 1 narinfos"6846lb # [ 151.867728] niks3-server[1131]: 2026/09/20 12:48:25 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst6847lb # [ 151.879457] niks3-server[1131]: 2026/09/20 12:48:25 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls6848lb # [ 151.885453] niks3-server[1131]: 2026/09/20 12:48:25 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6849lb # [ 151.893358] niks3-server[1131]: 2026/09/20 12:48:25 INFO Completed upload id=606850worker1 # [ 151.911178] nix-grpc-daemon[1796]: time=2026-09-20T12:48:25.337Z level=INFO msg="Upload complete. (444ms)"6851lb # [ 151.896359] niks3-server[1131]: 2026/09/20 12:48:25 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo6852worker1 # [ 151.914752] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46074 duration_s=0 paths=1 nar_bytes_in=7526853worker1 # [ 151.921313] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460746854worker1 # [ 151.933067] nix-daemon[912]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)6855worker1 # [ 151.935793] nix-daemon[912]: reaped child process 2057, status = succeeded6856worker1 # [ 151.940969] nix-daemon[912]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)6857worker1 # [ 152.755249] nix-daemon[912]: reaped child process 2071, status = succeeded6858worker1 # [ 152.910500] nix-daemon[912]: accepted connection from pid 2086, user nix-grpc-daemon (trusted)6859worker1 # [ 152.923774] nix-daemon[912]: reaped child process 2089, status = succeeded6860lb # [ 152.923918] niks3-server[1131]: 2026/09/20 12:48:26 INFO Received uploads request method=POST path=/api/pending_closures6861worker1 # [ 152.946641] nix-grpc-daemon[1796]: time=2026-09-20T12:48:26.372Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6862worker1 # [ 152.948661] nix-grpc-daemon[1796]: time=2026-09-20T12:48:26.374Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"6863lb # [ 152.943303] niks3-server[1131]: 2026/09/20 12:48:26 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv6864lb # [ 152.948442] niks3-server[1131]: 2026/09/20 12:48:26 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst6865lb # [ 152.950580] niks3-server[1131]: 2026/09/20 12:48:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6866lb # [ 152.952636] niks3-server[1131]: 2026/09/20 12:48:26 INFO Signed narinfos id=61 count=16867worker1 # [ 152.972110] nix-grpc-daemon[1796]: time=2026-09-20T12:48:26.398Z level=INFO msg="Uploading 1 narinfos"6868lb # [ 152.955530] niks3-server[1131]: 2026/09/20 12:48:26 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls6869lb # [ 152.963527] niks3-server[1131]: 2026/09/20 12:48:26 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6870lb # [ 152.966445] niks3-server[1131]: 2026/09/20 12:48:26 INFO Completed upload id=616871worker1 # [ 152.984544] nix-grpc-daemon[1796]: time=2026-09-20T12:48:26.410Z level=INFO msg="Upload complete. (237ms)"6872lb # [ 152.969423] niks3-server[1131]: 2026/09/20 12:48:26 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo6873worker1 # [ 152.987868] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46074 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=8 outputs=16874worker1 # [ 152.992143] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:26Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=16875worker1 # [ 152.993911] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460746876worker1 # [ 153.000296] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460746877worker1 # [ 153.002995] nix-daemon[912]: reaped child process 2069, status = succeeded6878worker1 # [ 153.006474] nix-grpc-daemon[1796]: time=2026-09-20T12:48:26.432Z level=INFO msg="All 2 paths already cached"6879worker1 # [ 153.009741] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:460746880worker1 # [ 153.013725] nix-daemon[912]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)6881worker1 # [ 153.170355] nix-daemon[912]: accepted connection from pid 2093, user nix-grpc-daemon (trusted)6882worker1 # [ 153.183638] nix-daemon[912]: reaped child process 2096, status = succeeded6883worker1 # [ 153.328622] nix-daemon[912]: accepted connection from pid 2098, user nix-grpc-daemon (trusted)6884worker1 # [ 153.341711] nix-daemon[912]: reaped child process 2101, status = succeeded6885lb # [ 153.339523] niks3-server[1131]: 2026/09/20 12:48:27 INFO Received uploads request method=POST path=/api/pending_closures6886worker1 # [ 153.362294] nix-grpc-daemon[1796]: time=2026-09-20T12:48:26.788Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6887worker1 # [ 153.363691] nix-grpc-daemon[1796]: time=2026-09-20T12:48:26.788Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"6888lb # [ 153.353511] niks3-server[1131]: 2026/09/20 12:48:27 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst6889lb # [ 153.357065] niks3-server[1131]: 2026/09/20 12:48:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6890lb # [ 153.359988] niks3-server[1131]: 2026/09/20 12:48:27 INFO Signed narinfos id=62 count=16891worker1 # [ 153.379654] nix-grpc-daemon[1796]: time=2026-09-20T12:48:26.805Z level=INFO msg="Uploading 1 narinfos"6892lb # [ 153.363584] niks3-server[1131]: 2026/09/20 12:48:27 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls6893lb # [ 153.368819] niks3-server[1131]: 2026/09/20 12:48:27 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6894lb # [ 153.372637] niks3-server[1131]: 2026/09/20 12:48:27 INFO Completed upload id=626895worker1 # [ 153.390762] nix-grpc-daemon[1796]: time=2026-09-20T12:48:26.816Z level=INFO msg="Upload complete. (370ms)"6896lb # [ 153.374410] niks3-server[1131]: 2026/09/20 12:48:27 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo6897worker1 # [ 153.394076] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46074 duration_s=0 paths=1 nar_bytes_in=7526898worker1 # [ 153.397056] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460746899worker1 # [ 153.403304] nix-daemon[912]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)6900worker1 # [ 153.405709] nix-daemon[912]: reaped child process 2091, status = succeeded6901worker1 # [ 153.411224] nix-daemon[912]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)6902worker1 # [ 154.625316] nix-daemon[912]: reaped child process 2105, status = succeeded6903worker1 # [ 154.782270] nix-daemon[912]: accepted connection from pid 2120, user nix-grpc-daemon (trusted)6904worker1 # [ 154.795839] nix-daemon[912]: reaped child process 2123, status = succeeded6905lb # [ 154.795195] niks3-server[1131]: 2026/09/20 12:48:28 INFO Received uploads request method=POST path=/api/pending_closures6906worker1 # [ 154.822441] nix-grpc-daemon[1796]: time=2026-09-20T12:48:28.248Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6907worker1 # [ 154.823969] nix-grpc-daemon[1796]: time=2026-09-20T12:48:28.250Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"6908lb # [ 154.837305] niks3-server[1131]: 2026/09/20 12:48:28 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst6909lb # [ 154.849124] niks3-server[1131]: 2026/09/20 12:48:28 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv6910lb # [ 154.853058] niks3-server[1131]: 2026/09/20 12:48:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6911lb # [ 154.858242] niks3-server[1131]: 2026/09/20 12:48:28 INFO Signed narinfos id=63 count=16912worker1 # [ 154.878988] nix-grpc-daemon[1796]: time=2026-09-20T12:48:28.304Z level=INFO msg="Uploading 1 narinfos"6913lb # [ 154.866356] niks3-server[1131]: 2026/09/20 12:48:28 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls6914lb # [ 154.874672] niks3-server[1131]: 2026/09/20 12:48:28 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6915lb # [ 154.880875] niks3-server[1131]: 2026/09/20 12:48:28 INFO Completed upload id=636916lb # [ 154.882732] niks3-server[1131]: 2026/09/20 12:48:28 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo6917worker1 # [ 154.900355] nix-grpc-daemon[1796]: time=2026-09-20T12:48:28.326Z level=INFO msg="Upload complete. (281ms)"6918worker1 # [ 154.906441] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46074 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=9 outputs=16919worker1 # [ 154.919883] nix-daemon[912]: reaped child process 2103, status = succeeded6920client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 17.90 seconds)6921worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6922worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6923client: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&16924worker2 # [ 155.252721] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:29Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:409946925worker1 # [ 155.298588] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460746926worker1 # [ 155.315369] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:28Z level=debug event=want client=2 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=06927client: (finished: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&1, in 0.28 seconds)6928(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 18.25 seconds)6929subtest: low disk drains a worker and builds go to the other6930worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6931worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.03 seconds)6932worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6933worker1 # [ 156.855582] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:30Z level=info event=unhealthy reason=min_free6934lb # [ 157.317510] envoy[973]: conn peer=[2001:db8:1::1]:59936 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27145 tx=10700 ms=176246935lb # [ 157.322490] envoy[973]: conn peer=[2001:db8:1::1]:55664 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=878 tx=481 ms=396936worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.08 seconds)6937client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&26938worker2 # [ 157.650298] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:409946939client # don't know how to build these paths:6940client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6941worker2 # [ 157.658591] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409946942worker1 # [ 157.717903] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:31Z level=debug event=want client=3 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=06943worker1 # [ 157.719828] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:31Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=16944worker2 # [ 157.720858] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409946945worker2 # [ 157.733560] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409946946client # copying 1 paths to 'lb:50051'...6947client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...6948worker2 # [ 157.745840] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409946949worker2 # [ 157.755071] nix-daemon[857]: accepted connection from pid 1369, user nix-grpc-daemon (trusted)6950worker2 # [ 157.933093] nix-daemon[857]: accepted connection from pid 1642, user nix-grpc-daemon (trusted)6951worker2 # [ 157.945380] nix-daemon[857]: reaped child process 1645, status = succeeded6952worker2 # [ 158.097199] nix-daemon[857]: accepted connection from pid 1647, user nix-grpc-daemon (trusted)6953worker2 # [ 158.109056] nix-daemon[857]: reaped child process 1650, status = succeeded6954lb # [ 158.149355] niks3-server[1131]: 2026/09/20 12:48:31 INFO Received uploads request method=POST path=/api/pending_closures6955worker2 # [ 158.135899] nix-grpc-daemon[1461]: time=2026-09-20T12:48:32.108Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6956worker2 # [ 158.137358] nix-grpc-daemon[1461]: time=2026-09-20T12:48:32.108Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"6957lb # [ 158.183123] niks3-server[1131]: 2026/09/20 12:48:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6958worker2 # [ 158.169389] nix-grpc-daemon[1461]: time=2026-09-20T12:48:32.142Z level=INFO msg="Uploading 1 narinfos"6959lb # [ 158.192062] niks3-server[1131]: 2026/09/20 12:48:31 INFO Signed narinfos id=64 count=16960lb # [ 158.197625] niks3-server[1131]: 2026/09/20 12:48:31 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst6961lb # [ 158.204642] niks3-server[1131]: 2026/09/20 12:48:32 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls6962lb # [ 158.213412] niks3-server[1131]: 2026/09/20 12:48:32 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6963worker2 # [ 158.197512] nix-grpc-daemon[1461]: time=2026-09-20T12:48:32.170Z level=INFO msg="Upload complete. (427ms)"6964lb # [ 158.220693] niks3-server[1131]: 2026/09/20 12:48:32 INFO Completed upload id=646965lb # [ 158.223314] niks3-server[1131]: 2026/09/20 12:48:32 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo6966worker2 # [ 158.200069] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40994 duration_s=0 paths=1 nar_bytes_in=5126967worker2 # [ 158.202892] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409946968worker2 # [ 158.209057] nix-daemon[857]: accepted connection from pid 1369, user nix-grpc-daemon (trusted)6969worker2 # [ 158.212643] nix-daemon[857]: reaped child process 1640, status = succeeded6970client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...6971client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6972worker2 # [ 158.217588] nix-daemon[857]: accepted connection from pid 1369, user nix-grpc-daemon (trusted)6973client # farm-a> LOG-farm-a-drain6974worker2 # [ 158.830605] nix-daemon[857]: reaped child process 1654, status = succeeded6975worker2 # [ 158.983767] nix-daemon[857]: accepted connection from pid 1669, user nix-grpc-daemon (trusted)6976worker2 # [ 158.994611] nix-daemon[857]: reaped child process 1672, status = succeeded6977lb # [ 159.033954] niks3-server[1131]: 2026/09/20 12:48:32 INFO Received uploads request method=POST path=/api/pending_closures6978worker2 # [ 159.020142] nix-grpc-daemon[1461]: time=2026-09-20T12:48:32.993Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6979worker2 # [ 159.021777] nix-grpc-daemon[1461]: time=2026-09-20T12:48:32.994Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"6980lb # [ 159.070596] niks3-server[1131]: 2026/09/20 12:48:32 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6981lb # [ 159.084873] niks3-server[1131]: 2026/09/20 12:48:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6982worker2 # [ 159.067354] nix-grpc-daemon[1461]: time=2026-09-20T12:48:33.040Z level=INFO msg="Uploading 1 narinfos"6983lb # [ 159.090224] niks3-server[1131]: 2026/09/20 12:48:32 INFO Signed narinfos id=65 count=16984lb # [ 159.096566] niks3-server[1131]: 2026/09/20 12:48:32 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6985lb # [ 159.101299] niks3-server[1131]: 2026/09/20 12:48:32 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls6986lb # [ 159.108137] niks3-server[1131]: 2026/09/20 12:48:32 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6987lb # [ 159.112608] niks3-server[1131]: 2026/09/20 12:48:32 INFO Completed upload id=656988worker2 # [ 159.089902] nix-grpc-daemon[1461]: time=2026-09-20T12:48:33.062Z level=INFO msg="Upload complete. (266ms)"6989lb # [ 159.113896] niks3-server[1131]: 2026/09/20 12:48:32 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo6990worker2 # [ 159.092594] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40994 duration_s=0 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=10 outputs=16991worker2 # [ 159.099296] nix-daemon[857]: reaped child process 1652, status = succeeded6992worker1 # [ 159.183996] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:32Z level=debug event=want client=3 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=06993worker1 # [ 159.188470] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:32Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=16994worker2 # [ 159.186962] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409946995worker2 # [ 159.195171] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409946996worker2 # [ 159.204965] nix-grpc-daemon[1461]: time=2026-09-20T12:48:33.177Z level=INFO msg="All 1 paths already cached"6997client # copying 1 paths to 'lb:50051'...6998client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...6999worker2 # [ 159.210659] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409947000worker2 # [ 159.221158] nix-daemon[857]: accepted connection from pid 1369, user nix-grpc-daemon (trusted)7001worker2 # [ 159.398071] nix-daemon[857]: accepted connection from pid 1676, user nix-grpc-daemon (trusted)7002worker2 # [ 159.409977] nix-daemon[857]: reaped child process 1679, status = succeeded7003worker2 # [ 159.569065] nix-daemon[857]: accepted connection from pid 1681, user nix-grpc-daemon (trusted)7004worker2 # [ 159.581110] nix-daemon[857]: reaped child process 1684, status = succeeded7005lb # [ 159.621597] niks3-server[1131]: 2026/09/20 12:48:33 INFO Received uploads request method=POST path=/api/pending_closures7006worker2 # [ 159.607412] nix-grpc-daemon[1461]: time=2026-09-20T12:48:33.580Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7007worker2 # [ 159.609358] nix-grpc-daemon[1461]: time=2026-09-20T12:48:33.582Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"7008lb # [ 159.657512] niks3-server[1131]: 2026/09/20 12:48:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign7009worker2 # [ 159.638977] nix-grpc-daemon[1461]: time=2026-09-20T12:48:33.611Z level=INFO msg="Uploading 1 narinfos"7010lb # [ 159.662163] niks3-server[1131]: 2026/09/20 12:48:33 INFO Signed narinfos id=66 count=17011lb # [ 159.668703] niks3-server[1131]: 2026/09/20 12:48:33 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst7012lb # [ 159.671281] niks3-server[1131]: 2026/09/20 12:48:33 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls7013lb # [ 159.676328] niks3-server[1131]: 2026/09/20 12:48:33 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete7014lb # [ 159.680086] niks3-server[1131]: 2026/09/20 12:48:33 INFO Completed upload id=667015worker2 # [ 159.657345] nix-grpc-daemon[1461]: time=2026-09-20T12:48:33.630Z level=INFO msg="Upload complete. (424ms)"7016lb # [ 159.681590] niks3-server[1131]: 2026/09/20 12:48:33 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo7017worker2 # [ 159.659839] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40994 duration_s=0 paths=1 nar_bytes_in=6247018worker2 # [ 159.662878] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409947019worker2 # [ 159.670091] nix-daemon[857]: accepted connection from pid 1369, user nix-grpc-daemon (trusted)7020worker2 # [ 159.672441] nix-daemon[857]: reaped child process 1674, status = succeeded7021client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...7022client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7023worker2 # [ 159.677416] nix-daemon[857]: accepted connection from pid 1369, user nix-grpc-daemon (trusted)7024client # farm-b> LOG-farm-b-drain7025worker2 # [ 160.903649] nix-daemon[857]: reaped child process 1688, status = succeeded7026worker2 # [ 161.051487] nix-daemon[857]: accepted connection from pid 1703, user nix-grpc-daemon (trusted)7027worker2 # [ 161.062490] nix-daemon[857]: reaped child process 1706, status = succeeded7028lb # [ 161.097993] niks3-server[1131]: 2026/09/20 12:48:34 INFO Received uploads request method=POST path=/api/pending_closures7029worker2 # [ 161.083242] nix-grpc-daemon[1461]: time=2026-09-20T12:48:35.056Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7030worker2 # [ 161.084751] nix-grpc-daemon[1461]: time=2026-09-20T12:48:35.057Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"7031lb # [ 161.130862] niks3-server[1131]: 2026/09/20 12:48:34 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7032lb # [ 161.145521] niks3-server[1131]: 2026/09/20 12:48:34 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst7033lb # [ 161.153470] niks3-server[1131]: 2026/09/20 12:48:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign7034lb # [ 161.158354] niks3-server[1131]: 2026/09/20 12:48:34 INFO Signed narinfos id=67 count=17035worker2 # [ 161.139672] nix-grpc-daemon[1461]: time=2026-09-20T12:48:35.112Z level=INFO msg="Uploading 1 narinfos"7036lb # [ 161.166252] niks3-server[1131]: 2026/09/20 12:48:34 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls7037lb # [ 161.176118] niks3-server[1131]: 2026/09/20 12:48:34 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete7038lb # [ 161.180671] niks3-server[1131]: 2026/09/20 12:48:34 INFO Completed upload id=677039worker2 # [ 161.157985] nix-grpc-daemon[1461]: time=2026-09-20T12:48:35.130Z level=INFO msg="Upload complete. (272ms)"7040lb # [ 161.183619] niks3-server[1131]: 2026/09/20 12:48:34 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo7041worker2 # [ 161.164621] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40994 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=11 outputs=17042worker1 # [ 161.212472] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:34Z level=debug event=want client=3 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=07043worker1 # [ 161.216533] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:34Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=17044worker2 # [ 161.178251] nix-daemon[857]: reaped child process 1686, status = succeeded7045worker2 # [ 161.216408] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409947046worker2 # [ 161.228357] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409947047worker2 # [ 161.241151] nix-grpc-daemon[1461]: time=2026-09-20T12:48:35.213Z level=INFO msg="All 2 paths already cached"7048client # copying 1 paths to 'lb:50051'...7049client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...7050worker2 # [ 161.249329] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409947051worker2 # [ 161.259160] nix-daemon[857]: accepted connection from pid 1369, user nix-grpc-daemon (trusted)7052worker2 # [ 161.425854] nix-daemon[857]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)7053worker2 # [ 161.439520] nix-daemon[857]: reaped child process 1713, status = succeeded7054worker2 # [ 161.590275] nix-daemon[857]: accepted connection from pid 1715, user nix-grpc-daemon (trusted)7055worker2 # [ 161.603604] nix-daemon[857]: reaped child process 1718, status = succeeded7056lb # [ 161.641882] niks3-server[1131]: 2026/09/20 12:48:35 INFO Received uploads request method=POST path=/api/pending_closures7057worker2 # [ 161.627408] nix-grpc-daemon[1461]: time=2026-09-20T12:48:35.600Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7058worker2 # [ 161.628825] nix-grpc-daemon[1461]: time=2026-09-20T12:48:35.600Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"7059lb # [ 161.667320] niks3-server[1131]: 2026/09/20 12:48:35 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst7060lb # [ 161.678598] niks3-server[1131]: 2026/09/20 12:48:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign7061worker2 # [ 161.660821] nix-grpc-daemon[1461]: time=2026-09-20T12:48:35.633Z level=INFO msg="Uploading 1 narinfos"7062lb # [ 161.684177] niks3-server[1131]: 2026/09/20 12:48:35 INFO Signed narinfos id=68 count=17063lb # [ 161.688704] niks3-server[1131]: 2026/09/20 12:48:35 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls7064lb # [ 161.700893] niks3-server[1131]: 2026/09/20 12:48:35 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete7065worker2 # [ 161.683297] nix-grpc-daemon[1461]: time=2026-09-20T12:48:35.656Z level=INFO msg="Upload complete. (411ms)"7066lb # [ 161.706341] niks3-server[1131]: 2026/09/20 12:48:35 INFO Completed upload id=687067lb # [ 161.709699] niks3-server[1131]: 2026/09/20 12:48:35 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo7068worker2 # [ 161.686975] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40994 duration_s=0 paths=1 nar_bytes_in=7527069worker2 # [ 161.691476] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409947070worker2 # [ 161.700062] nix-daemon[857]: accepted connection from pid 1369, user nix-grpc-daemon (trusted)7071worker2 # [ 161.704171] nix-daemon[857]: reaped child process 1708, status = succeeded7072worker2 # [ 161.709120] nix-daemon[857]: accepted connection from pid 1369, user nix-grpc-daemon (trusted)7073client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...7074client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7075client # farm-top> LOG-farm-top-drain7076worker2 # [ 162.732439] nix-daemon[857]: reaped child process 1722, status = succeeded7077worker2 # [ 162.905885] nix-daemon[857]: accepted connection from pid 1737, user nix-grpc-daemon (trusted)7078worker2 # [ 162.916844] nix-daemon[857]: reaped child process 1740, status = succeeded7079lb # [ 162.956817] niks3-server[1131]: 2026/09/20 12:48:36 INFO Received uploads request method=POST path=/api/pending_closures7080worker2 # [ 162.944302] nix-grpc-daemon[1461]: time=2026-09-20T12:48:36.917Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7081worker2 # [ 162.945829] nix-grpc-daemon[1461]: time=2026-09-20T12:48:36.918Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"7082lb # [ 162.990654] niks3-server[1131]: 2026/09/20 12:48:36 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7083lb # [ 163.007136] niks3-server[1131]: 2026/09/20 12:48:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign7084worker2 # [ 162.994052] nix-grpc-daemon[1461]: time=2026-09-20T12:48:36.966Z level=INFO msg="Uploading 1 narinfos"7085lb # [ 163.016636] niks3-server[1131]: 2026/09/20 12:48:36 INFO Signed narinfos id=69 count=17086lb # [ 163.026800] niks3-server[1131]: 2026/09/20 12:48:36 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7087lb # [ 163.031877] niks3-server[1131]: 2026/09/20 12:48:36 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7088lb # [ 163.039446] niks3-server[1131]: 2026/09/20 12:48:36 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete7089lb # [ 163.042429] niks3-server[1131]: 2026/09/20 12:48:36 INFO Completed upload id=697090worker2 # [ 163.019847] nix-grpc-daemon[1461]: time=2026-09-20T12:48:36.992Z level=INFO msg="Upload complete. (301ms)"7091lb # [ 163.044728] niks3-server[1131]: 2026/09/20 12:48:36 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7092worker2 # [ 163.027218] nix-grpc-daemon[1369]: ts=2026-09-20T12:48:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40994 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=12 outputs=17093worker2 # [ 163.040269] nix-daemon[857]: reaped child process 1720, status = succeeded7094client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&2, in 5.63 seconds)7095client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath7096client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath, in 0.19 seconds)7097worker1: must succeed: rm /nix/.rw-store/fill7098worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.13 seconds)7099worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7100worker1 # [ 163.860493] nix-grpc-daemon[1735]: ts=2026-09-20T12:48:37Z level=info event=healthy reason=min_free7101worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.08 seconds)7102(finished: subtest: low disk drains a worker and builds go to the other, in 9.22 seconds)7103(finished: run the VM test script, in 165.83 seconds)7104test script finished in 166.04s7105cleanup7106kill QemuMachine (pid 50)7107client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7108kill QemuMachine (pid 49)7109lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7110kill QemuMachine (pid 52)7111worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7112kill QemuMachine (pid 51)7113worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7114(finished: cleanup, in 1.03 seconds)7115additionally exposed symbols:7116 client, lb, worker1, worker2,7117 vlan1,7118 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh