vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #255
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12client: QEMU running (pid 49)13client # Disk image does not exist, creating the virtualisation disk image...14client # Formatting '/build/vm-state-client/tmp.7D93XAP21h', fmt=raw size=107374182415client # mke2fs 1.47.4 (6-Mar-2025)16client # Discarding device blocks: 0/262144 done17client # Creating filesystem with 262144 4k blocks and 65536 inodes18client # Filesystem UUID: a5a537d1-4aae-4f89-9c4c-1cc1785f389619client # Superblock backups stored on blocks:20client # 32768, 98304, 163840, 22937621client # 22client # Allocating group tables: 0/8 done23client # Writing inode tables: 0/8 done24client # Creating journal (8192 blocks): done25client # Writing superblocks and filesystem accounting information: 0/8 done26client # 27lb # Disk image does not exist, creating the virtualisation disk image...28lb: QEMU running (pid 51)29client # Virtualisation disk image created.30lb # Formatting '/build/vm-state-lb/tmp.HtDZyMRk3x', fmt=raw size=107374182431worker1 # Disk image does not exist, creating the virtualisation disk image...32lb # mke2fs 1.47.4 (6-Mar-2025)33worker1 # Formatting '/build/vm-state-worker1/tmp.SWpOsWeJ4a', fmt=raw size=107374182434lb # Discarding device blocks: 0/262144 done35worker1 # mke2fs 1.47.4 (6-Mar-2025)36lb # Creating filesystem with 262144 4k blocks and 65536 inodes37worker1 # Discarding device blocks: 0/262144 done38lb # Filesystem UUID: 900ac49c-7780-4f91-8f54-4ef028cf157739worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes40lb # Superblock backups stored on blocks:41worker1 # Filesystem UUID: a174306b-8d82-4141-9f19-953e161b3bdf42lb # 32768, 98304, 163840, 22937643worker1 # Superblock backups stored on blocks:44lb # 45worker1 # 32768, 98304, 163840, 22937646lb # Allocating group tables: 0/8 done47worker1 # 48lb # Writing inode tables: 0/8 done49worker1 # Allocating group tables: 0/8 done50lb # Creating journal (8192 blocks): done51worker1 # Writing inode tables: 0/8 done52lb # Writing superblocks and filesystem accounting information: 0/8 done53worker1 # Creating journal (8192 blocks): done54lb # 55worker1 # Writing superblocks and filesystem accounting information: 0/8 done56lb # Virtualisation disk image created.57worker1 # 58worker1: QEMU running (pid 50)59worker1 # Virtualisation disk image created.60worker2 # Disk image does not exist, creating the virtualisation disk image...61worker2: QEMU running (pid 52)62worker2 # Formatting '/build/vm-state-worker2/tmp.Ib1uOGiqjH', fmt=raw size=107374182463worker2 # mke2fs 1.47.4 (6-Mar-2025)64worker2 # Discarding device blocks: 0/262144 done65worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes66(finished: start all VMs, in 0.24 seconds)67worker2 # Filesystem UUID: d4d6e2b3-a29c-48e2-b07f-0b80035d2e3468client: waiting for unit network-addresses-eth1.service69worker2 # Superblock backups stored on blocks:70client: waiting for the VM to finish booting71worker2 # 32768, 98304, 163840, 22937672worker2 # 73worker2 # Allocating group tables: 0/8 done74worker2 # Writing inode tables: 0/8 done75worker2 # Creating journal (8192 blocks): done76worker2 # Writing superblocks and filesystem accounting information: 0/8 done77worker2 # 78worker2 # Virtualisation disk image created.79worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)80client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)81lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)82worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)83worker1 # 84worker1 # 85worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0086worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...87client # 88worker1 # 89client # 90worker1 # 91worker1 # 92worker1 # 93client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0094worker1 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB0095client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...96client # 97client # 98client # 99client # 100client # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00101worker1 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...102worker1 # 103worker1 # 104client # Press Ctrl-B to configure iPXE (PCI 00:08.0)...105client # 106client # 107lb # 108lb # 109lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00110lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...111lb # 112lb # 113lb # 114lb # 115lb # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00116worker2 # 117worker2 # 118worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00119lb # Press Ctrl-B to configure iPXE (PCI 00:08.0)...120lb # 121lb # 122worker1 # Booting from ROM...123worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...124worker2 # 125worker2 # 126worker2 # 127worker2 # 128worker2 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00129client # Booting from ROM...130worker2 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...131worker2 # 132worker2 # 133lb # Booting from ROM...134worker2 # Booting from ROM...135worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026136worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/hvv0rrwgpqzhm46a4280mwzmbapcmqq9-nixos-system-worker1-test/init regInfo=/nix/store/vhr1b62dfjs04ywz0rz399x3gf2shwsl-closure-info/registration console=ttyS0,115200n8 console=tty0137worker1 # [ 0.000000] BIOS-provided physical RAM map:138worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable139worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved140client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026141worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved142worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable143worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved144worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved145client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test/init regInfo=/nix/store/9p1idx4s91wv7zwkal5pb5lr899hddrn-closure-info/registration console=ttyS0,115200n8 console=tty0146worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved147client # [ 0.000000] BIOS-provided physical RAM map:148worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved149client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable150worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved151client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved152worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved153client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved154worker1 # [ 0.000000] NX (Execute Disable) protection: active155client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable156worker1 # [ 0.000000] APIC: Static calls initialized157worker1 # [ 0.000000] SMBIOS 2.8 present.158client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved159client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved160worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014161worker1 # [ 0.000000] DMI: Memory slots populated: 1/1162client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved163worker1 # [ 0.000000] Hypervisor detected: KVM164client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved165worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000166client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved167worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00168worker1 # [ 0.000000] kvm-clock: using sched offset of 454703877 cycles169client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved170client # [ 0.000000] NX (Execute Disable) protection: active171client # [ 0.000000] APIC: Static calls initialized172worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns173client # [ 0.000000] SMBIOS 2.8 present.174worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor175worker1 # [ 0.000817] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000176client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014177client # [ 0.000000] DMI: Memory slots populated: 1/1178worker1 # [ 0.000844] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs179client # [ 0.000000] Hypervisor detected: KVM180worker1 # [ 0.000846] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT181client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000182worker1 # [ 0.002746] found SMP MP-table at [mem 0x000f5450-0x000f545f]183client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00184worker1 # [ 0.002758] Using GB pages for direct mapping185client # [ 0.000000] kvm-clock: using sched offset of 472812525 cycles186worker1 # [ 0.002835] RAMDISK: [mem 0x5e367000-0x5ffcffff]187worker1 # [ 0.002844] ACPI: Early table checksum verification disabled188client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns189worker1 # [ 0.002846] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )190client # [ 0.000004] tsc: Detected 2400.012 MHz processor191client # [ 0.000811] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000192worker1 # [ 0.002850] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)193client # [ 0.000838] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs194worker1 # [ 0.002854] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)195client # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT196worker1 # [ 0.002861] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)197client # [ 0.002732] found SMP MP-table at [mem 0x000f5450-0x000f545f]198worker1 # [ 0.002863] ACPI: FACS 0x000000005FFE0000 000040199client # [ 0.002743] Using GB pages for direct mapping200client # [ 0.002872] RAMDISK: [mem 0x5e366000-0x5ffcffff]201worker1 # [ 0.002865] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)202client # [ 0.002881] ACPI: Early table checksum verification disabled203worker1 # [ 0.002866] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)204client # [ 0.002883] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )205worker1 # [ 0.002868] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)206client # [ 0.002887] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)207worker1 # [ 0.002869] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)208client # [ 0.002891] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)209worker1 # [ 0.002871] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]210client # [ 0.002897] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)211client # [ 0.002899] ACPI: FACS 0x000000005FFE0000 000040212worker1 # [ 0.002872] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]213worker1 # [ 0.002872] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]214client # [ 0.002900] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)215worker1 # [ 0.002873] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]216client # [ 0.002902] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)217worker1 # [ 0.002873] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]218client # [ 0.002903] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)219worker1 # [ 0.002873] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]220client # [ 0.002905] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)221worker1 # [ 0.002874] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]222worker1 # [ 0.003097] No NUMA configuration found223client # [ 0.002906] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]224worker1 # [ 0.003098] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]225client # [ 0.002907] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]226worker1 # [ 0.003100] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]227client # [ 0.002908] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]228worker1 # [ 0.005263] Zone ranges:229worker1 # [ 0.005265] DMA [mem 0x0000000000001000-0x0000000000ffffff]230client # [ 0.002908] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]231worker1 # [ 0.005266] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]232client # [ 0.002909] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]233worker1 # [ 0.005267] Normal empty234worker1 # [ 0.005268] Device empty235client # [ 0.002909] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]236worker1 # [ 0.005268] Movable zone start for each node237worker1 # [ 0.005269] Early memory node ranges238client # [ 0.002910] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]239client # [ 0.003133] No NUMA configuration found240worker1 # [ 0.005269] node 0: [mem 0x0000000000001000-0x000000000009efff]241client # [ 0.003134] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]242worker1 # [ 0.005270] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]243lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026244client # [ 0.003136] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]245worker1 # [ 0.005271] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]246client # [ 0.005394] Zone ranges:247worker1 # [ 0.005290] On node 0, zone DMA: 1 pages in unavailable ranges248client # [ 0.005394] DMA [mem 0x0000000000001000-0x0000000000ffffff]249worker1 # [ 0.005549] On node 0, zone DMA: 97 pages in unavailable ranges250client # [ 0.005396] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]251client # [ 0.005397] Normal empty252worker1 # [ 0.031297] On node 0, zone DMA32: 40 pages in unavailable ranges253client # [ 0.005398] Device empty254worker1 # [ 0.031759] ACPI: PM-Timer IO Port: 0x608255client # [ 0.005398] Movable zone start for each node256lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/8y2rmlwiyr2iliam1yhl8i23qgaaknj8-nixos-system-lb-test/init regInfo=/nix/store/9qjmxix17wyd2infdr8gh5ii9mr60zqd-closure-info/registration console=ttyS0,115200n8 console=tty0257client # [ 0.005399] Early memory node ranges258worker1 # [ 0.031769] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])259lb # [ 0.000000] BIOS-provided physical RAM map:260client # [ 0.005399] node 0: [mem 0x0000000000001000-0x000000000009efff]261worker1 # [ 0.031795] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23262lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable263client # [ 0.005400] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]264worker1 # [ 0.031798] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)265lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved266client # [ 0.005402] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]267worker1 # [ 0.031799] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)268lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved269client # [ 0.005424] On node 0, zone DMA: 1 pages in unavailable ranges270worker1 # [ 0.031800] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)271client # [ 0.005685] On node 0, zone DMA: 97 pages in unavailable ranges272lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable273worker1 # [ 0.031801] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)274client # [ 0.032581] On node 0, zone DMA32: 40 pages in unavailable ranges275lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved276client # [ 0.033034] ACPI: PM-Timer IO Port: 0x608277worker1 # [ 0.031802] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)278lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved279client # [ 0.033045] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])280worker1 # [ 0.031804] ACPI: Using ACPI (MADT) for SMP configuration information281lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved282worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026283client # [ 0.033071] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23284worker1 # [ 0.031805] ACPI: HPET id: 0x8086a201 base: 0xfed00000285worker1 # [ 0.031809] TSC deadline timer available286lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved287client # [ 0.033074] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)288worker1 # [ 0.031813] CPU topo: Max. logical packages: 1289lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved290client # [ 0.033075] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)291worker1 # [ 0.031814] CPU topo: Max. logical dies: 1292worker1 # [ 0.031814] CPU topo: Max. dies per package: 1293client # [ 0.033076] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)294lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved295worker1 # [ 0.031819] CPU topo: Max. threads per core: 1296lb # [ 0.000000] NX (Execute Disable) protection: active297client # [ 0.033077] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)298worker1 # [ 0.031820] CPU topo: Num. cores per package: 1299lb # [ 0.000000] APIC: Static calls initialized300worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/x5qgax5h37ybfdjp7r7k01yx9abl6wij-nixos-system-worker2-test/init regInfo=/nix/store/dq71v172s5jppb7nnfi3vkq85w35rkbj-closure-info/registration console=ttyS0,115200n8 console=tty0301lb # [ 0.000000] SMBIOS 2.8 present.302worker1 # [ 0.031820] CPU topo: Num. threads per package: 1303client # [ 0.033078] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)304worker2 # [ 0.000000] BIOS-provided physical RAM map:305worker1 # [ 0.031820] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs306client # [ 0.033080] ACPI: Using ACPI (MADT) for SMP configuration information307worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable308lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014309client # [ 0.033081] ACPI: HPET id: 0x8086a201 base: 0xfed00000310lb # [ 0.000000] DMI: Memory slots populated: 1/1311worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved312worker1 # [ 0.031836] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()313client # [ 0.033085] TSC deadline timer available314lb # [ 0.000000] Hypervisor detected: KVM315client # [ 0.033089] CPU topo: Max. logical packages: 1316worker1 # [ 0.031865] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]317worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved318lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000319client # [ 0.033089] CPU topo: Max. logical dies: 1320lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00321worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable322worker1 # [ 0.031867] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]323client # [ 0.033090] CPU topo: Max. dies per package: 1324lb # [ 0.000000] kvm-clock: using sched offset of 497247723 cycles325client # [ 0.033093] CPU topo: Max. threads per core: 1326worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved327worker1 # [ 0.031868] [mem 0x60000000-0xafffffff] available for PCI devices328client # [ 0.033094] CPU topo: Num. cores per package: 1329worker1 # [ 0.031869] Booting paravirtualized kernel on KVM330worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved331client # [ 0.033094] CPU topo: Num. threads per package: 1332lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns333lb # [ 0.000006] tsc: Detected 2400.012 MHz processor334worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved335client # [ 0.033095] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs336worker1 # [ 0.031871] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns337lb # [ 0.000812] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000338worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved339client # [ 0.033112] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()340worker1 # [ 0.036317] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1341worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved342lb # [ 0.000841] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs343worker1 # [ 0.038420] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152344client # [ 0.033143] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]345worker1 # [ 0.038468] kvm-guest: PV spinlocks disabled, single CPU346worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved347lb # [ 0.000844] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT348client # [ 0.033145] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]349worker2 # [ 0.000000] NX (Execute Disable) protection: active350lb # [ 0.002745] found SMP MP-table at [mem 0x000f5450-0x000f545f]351client # [ 0.033146] [mem 0x60000000-0xafffffff] available for PCI devices352worker2 # [ 0.000000] APIC: Static calls initialized353lb # [ 0.002760] Using GB pages for direct mapping354worker2 # [ 0.000000] SMBIOS 2.8 present.355client # [ 0.033148] Booting paravirtualized kernel on KVM356lb # [ 0.002913] RAMDISK: [mem 0x5e367000-0x5ffcffff]357lb # [ 0.002923] ACPI: Early table checksum verification disabled358worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014359client # [ 0.033150] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns360lb # [ 0.002927] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )361worker2 # [ 0.000000] DMI: Memory slots populated: 1/1362worker2 # [ 0.000000] Hypervisor detected: KVM363client # [ 0.037597] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1364worker1 # [ 0.038469] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/hvv0rrwgpqzhm46a4280mwzmbapcmqq9-nixos-system-worker1-test/init regInfo=/nix/store/vhr1b62dfjs04ywz0rz399x3gf2shwsl-closure-info/registration console=ttyS0,115200n8 console=tty0365lb # [ 0.002932] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)366worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000367client # [ 0.039752] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152368worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00369client # [ 0.039796] kvm-guest: PV spinlocks disabled, single CPU370lb # [ 0.002936] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)371worker1 # [ 0.038559] Unknown kernel command line parameters "regInfo=/nix/store/vhr1b62dfjs04ywz0rz399x3gf2shwsl-closure-info/registration", will be passed to user space.372worker2 # [ 0.000000] kvm-clock: using sched offset of 524335639 cycles373worker1 # [ 0.038572] random: crng init done374lb # [ 0.002944] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)375lb # [ 0.002946] ACPI: FACS 0x000000005FFE0000 000040376worker1 # [ 0.038572] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes377worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns378worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor379worker1 # [ 0.040640] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)380lb # [ 0.002947] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)381worker2 # [ 0.000810] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000382worker1 # [ 0.041680] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)383client # [ 0.039797] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test/init regInfo=/nix/store/9p1idx4s91wv7zwkal5pb5lr899hddrn-closure-info/registration console=ttyS0,115200n8 console=tty0384lb # [ 0.002949] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)385worker2 # [ 0.000837] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs386worker1 # [ 0.041711] Fallback order for Node 0: 0387lb # [ 0.002951] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)388worker2 # [ 0.000840] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT389worker1 # [ 0.041713] Built 1 zonelists, mobility grouping on. Total pages: 393078390worker1 # [ 0.041714] Policy zone: DMA32391client # [ 0.039888] Unknown kernel command line parameters "regInfo=/nix/store/9p1idx4s91wv7zwkal5pb5lr899hddrn-closure-info/registration", will be passed to user space.392worker2 # [ 0.002727] found SMP MP-table at [mem 0x000f5450-0x000f545f]393lb # [ 0.002952] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)394client # [ 0.039901] random: crng init done395worker2 # [ 0.002739] Using GB pages for direct mapping396worker1 # [ 0.044147] mem auto-init: stack:all(zero), heap alloc:on, heap free:off397worker2 # [ 0.002816] RAMDISK: [mem 0x5e367000-0x5ffcffff]398lb # [ 0.002953] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]399client # [ 0.039901] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes400worker1 # [ 0.047179] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1401worker2 # [ 0.002825] ACPI: Early table checksum verification disabled402worker1 # [ 0.050629] allocated 3145728 bytes of page_ext403lb # [ 0.002955] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]404client # [ 0.042029] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)405worker2 # [ 0.002828] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )406worker1 # [ 0.060313] ftrace: allocating 48731 entries in 192 pages407lb # [ 0.002955] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]408client # [ 0.043105] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)409worker1 # [ 0.060316] ftrace: allocated 192 pages with 2 groups410worker2 # [ 0.002832] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)411client # [ 0.043136] Fallback order for Node 0: 0412worker1 # [ 0.061143] Dynamic Preempt: lazy413lb # [ 0.002956] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]414worker1 # [ 0.061288] rcu: Preemptible hierarchical RCU implementation.415client # [ 0.043139] Built 1 zonelists, mobility grouping on. Total pages: 393078416worker2 # [ 0.002837] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)417lb # [ 0.002956] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]418client # [ 0.043140] Policy zone: DMA32419worker1 # [ 0.061289] rcu: RCU event tracing is enabled.420worker2 # [ 0.002845] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)421lb # [ 0.002957] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]422client # [ 0.045903] mem auto-init: stack:all(zero), heap alloc:on, heap free:off423worker1 # [ 0.061289] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.424worker2 # [ 0.002847] ACPI: FACS 0x000000005FFE0000 000040425lb # [ 0.002957] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]426worker1 # [ 0.061291] Trampoline variant of Tasks RCU enabled.427client # [ 0.049058] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1428lb # [ 0.003207] No NUMA configuration found429worker1 # [ 0.061291] Rude variant of Tasks RCU enabled.430client # [ 0.052372] allocated 3145728 bytes of page_ext431worker2 # [ 0.002848] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)432worker1 # [ 0.061291] Tracing variant of Tasks RCU enabled.433client # [ 0.062225] ftrace: allocating 48731 entries in 192 pages434lb # [ 0.003208] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]435worker2 # [ 0.002850] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)436client # [ 0.062227] ftrace: allocated 192 pages with 2 groups437lb # [ 0.003211] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]438worker1 # [ 0.061292] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.439client # [ 0.063047] Dynamic Preempt: lazy440lb # [ 0.005657] Zone ranges:441worker2 # [ 0.002851] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)442worker1 # [ 0.061293] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1443client # [ 0.063232] rcu: Preemptible hierarchical RCU implementation.444lb # [ 0.005658] DMA [mem 0x0000000000001000-0x0000000000ffffff]445client # [ 0.063232] rcu: RCU event tracing is enabled.446worker2 # [ 0.002853] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)447lb # [ 0.005659] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]448worker1 # [ 0.061319] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.449lb # [ 0.005661] Normal empty450client # [ 0.063233] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.451lb # [ 0.005662] Device empty452worker2 # [ 0.002854] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]453client # [ 0.063234] Trampoline variant of Tasks RCU enabled.454worker1 # [ 0.061321] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.455lb # [ 0.005662] Movable zone start for each node456worker2 # [ 0.002855] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]457client # [ 0.063235] Rude variant of Tasks RCU enabled.458lb # [ 0.005663] Early memory node ranges459client # [ 0.063235] Tracing variant of Tasks RCU enabled.460worker1 # [ 0.061322] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.461worker2 # [ 0.002856] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]462lb # [ 0.005663] node 0: [mem 0x0000000000001000-0x000000000009efff]463client # [ 0.063236] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.464worker1 # [ 0.065664] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16465lb # [ 0.005665] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]466worker2 # [ 0.002856] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]467client # [ 0.063237] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1468worker1 # [ 0.065978] rcu: srcu_init: Setting srcu_struct sizes based on contention.469lb # [ 0.005666] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]470worker2 # [ 0.002857] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]471lb # [ 0.005687] On node 0, zone DMA: 1 pages in unavailable ranges472client # [ 0.063246] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.473worker2 # [ 0.002857] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]474worker1 # [ 0.065984] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns475lb # [ 0.005962] On node 0, zone DMA: 97 pages in unavailable ranges476worker2 # [ 0.002858] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]477client # [ 0.063248] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.478lb # [ 0.033798] On node 0, zone DMA32: 40 pages in unavailable ranges479worker2 # [ 0.003086] No NUMA configuration found480worker1 # [ 0.066086] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)481lb # [ 0.034271] ACPI: PM-Timer IO Port: 0x608482worker1 # [ 0.069638] Console: colour VGA+ 80x25483worker2 # [ 0.003087] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]484client # [ 0.063248] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.485worker1 # [ 0.069642] printk: legacy console [tty0] enabled486lb # [ 0.034284] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])487worker2 # [ 0.003089] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]488client # [ 0.067589] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16489worker1 # [ 0.099554] printk: legacy console [ttyS0] enabled490worker2 # [ 0.005402] Zone ranges:491lb # [ 0.034311] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23492client # [ 0.067875] rcu: srcu_init: Setting srcu_struct sizes based on contention.493worker1 # [ 0.200065] ACPI: Core revision 20250807494worker2 # [ 0.005404] DMA [mem 0x0000000000001000-0x0000000000ffffff]495lb # [ 0.034314] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)496worker2 # [ 0.005405] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]497lb # [ 0.034316] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)498worker2 # [ 0.005406] Normal empty499client # [ 0.067881] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns500worker2 # [ 0.005407] Device empty501worker1 # [ 0.200932] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns502lb # [ 0.034317] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)503worker2 # [ 0.005408] Movable zone start for each node504client # [ 0.067979] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)505worker2 # [ 0.005409] Early memory node ranges506worker1 # [ 0.202422] APIC: Switch to symmetric I/O mode setup507lb # [ 0.034319] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)508client # [ 0.071494] Console: colour VGA+ 80x25509worker1 # [ 0.203350] x2apic enabled510worker2 # [ 0.005409] node 0: [mem 0x0000000000001000-0x000000000009efff]511client # [ 0.071496] printk: legacy console [tty0] enabled512lb # [ 0.034319] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)513client # [ 0.100922] printk: legacy console [ttyS0] enabled514worker2 # [ 0.005410] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]515lb # [ 0.034322] ACPI: Using ACPI (MADT) for SMP configuration information516worker1 # [ 0.204087] APIC: Switched APIC routing to: physical x2apic517client # [ 0.199502] ACPI: Core revision 20250807518worker2 # [ 0.005411] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]519lb # [ 0.034323] ACPI: HPET id: 0x8086a201 base: 0xfed00000520lb # [ 0.034327] TSC deadline timer available521worker2 # [ 0.005433] On node 0, zone DMA: 1 pages in unavailable ranges522lb # [ 0.034331] CPU topo: Max. logical packages: 1523worker2 # [ 0.005709] On node 0, zone DMA: 97 pages in unavailable ranges524worker1 # [ 0.205872] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1525client # [ 0.200409] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns526lb # [ 0.034332] CPU topo: Max. logical dies: 1527worker2 # [ 0.033547] On node 0, zone DMA32: 40 pages in unavailable ranges528lb # [ 0.034333] CPU topo: Max. dies per package: 1529worker2 # [ 0.034013] ACPI: PM-Timer IO Port: 0x608530client # [ 0.202254] APIC: Switch to symmetric I/O mode setup531lb # [ 0.034336] CPU topo: Max. threads per core: 1532worker1 # [ 0.206842] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns533worker2 # [ 0.034026] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])534lb # [ 0.034337] CPU topo: Num. cores per package: 1535client # [ 0.203213] x2apic enabled536lb # [ 0.034337] CPU topo: Num. threads per package: 1537worker2 # [ 0.034051] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23538worker1 # [ 0.208469] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)539client # [ 0.203946] APIC: Switched APIC routing to: physical x2apic540lb # [ 0.034338] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs541worker2 # [ 0.034054] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)542lb # [ 0.034360] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()543worker1 # [ 0.209552] x86/cpu: User Mode Instruction Prevention (UMIP) activated544worker2 # [ 0.034056] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)545lb # [ 0.034395] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]546client # [ 0.205695] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1547worker2 # [ 0.034057] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)548worker1 # [ 0.210609] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127549worker2 # [ 0.034058] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)550lb # [ 0.034397] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]551worker1 # [ 0.211467] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0552worker2 # [ 0.034059] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)553lb # [ 0.034398] [mem 0x60000000-0xafffffff] available for PCI devices554client # [ 0.206648] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns555lb # [ 0.034400] Booting paravirtualized kernel on KVM556worker2 # [ 0.034061] ACPI: Using ACPI (MADT) for SMP configuration information557worker2 # [ 0.034062] ACPI: HPET id: 0x8086a201 base: 0xfed00000558client # [ 0.208249] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)559worker1 # [ 0.212470] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto560worker2 # [ 0.034065] TSC deadline timer available561lb # [ 0.034404] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns562worker2 # [ 0.034070] CPU topo: Max. logical packages: 1563client # [ 0.209333] x86/cpu: User Mode Instruction Prevention (UMIP) activated564lb # [ 0.038973] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1565worker1 # [ 0.213467] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl566worker2 # [ 0.034071] CPU topo: Max. logical dies: 1567worker2 # [ 0.034071] CPU topo: Max. dies per package: 1568lb # [ 0.041728] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152569client # [ 0.210380] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127570worker1 # [ 0.214467] Transient Scheduler Attacks: Vulnerable: No microcode571worker2 # [ 0.034074] CPU topo: Max. threads per core: 1572lb # [ 0.041775] kvm-guest: PV spinlocks disabled, single CPU573client # [ 0.211907] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0574worker2 # [ 0.034075] CPU topo: Num. cores per package: 1575worker1 # [ 0.215466] Spectre V2 : Mitigation: Enhanced / Automatic IBRS576worker2 # [ 0.034076] CPU topo: Num. threads per package: 1577worker1 # [ 0.216467] Speculative Return Stack Overflow: Mitigation: Safe RET578worker2 # [ 0.034076] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs579client # [ 0.212250] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto580worker2 # [ 0.034098] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()581client # [ 0.214247] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl582worker2 # [ 0.034133] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]583lb # [ 0.041777] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/8y2rmlwiyr2iliam1yhl8i23qgaaknj8-nixos-system-lb-test/init regInfo=/nix/store/9qjmxix17wyd2infdr8gh5ii9mr60zqd-closure-info/registration console=ttyS0,115200n8 console=tty0584worker2 # [ 0.034134] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]585client # [ 0.215247] Transient Scheduler Attacks: Vulnerable: No microcode586worker2 # [ 0.034136] [mem 0x60000000-0xafffffff] available for PCI devices587client # [ 0.216246] Spectre V2 : Mitigation: Enhanced / Automatic IBRS588worker1 # [ 0.217466] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization589worker2 # [ 0.034138] Booting paravirtualized kernel on KVM590lb # [ 0.041876] Unknown kernel command line parameters "regInfo=/nix/store/9qjmxix17wyd2infdr8gh5ii9mr60zqd-closure-info/registration", will be passed to user space.591lb # [ 0.041889] random: crng init done592client # [ 0.217247] Speculative Return Stack Overflow: Mitigation: Safe RET593worker1 # [ 0.219474] Spectre V2 : Enabling IBPB for BPF594lb # [ 0.041890] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes595worker2 # [ 0.034141] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns596worker1 # [ 0.220467] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier597worker2 # [ 0.038628] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1598lb # [ 0.044327] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)599worker1 # [ 0.221468] active return thunk: srso_alias_return_thunk600worker2 # [ 0.041178] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152601lb # [ 0.045730] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)602lb # [ 0.045765] Fallback order for Node 0: 0603worker2 # [ 0.041224] kvm-guest: PV spinlocks disabled, single CPU604worker1 # [ 0.222488] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'605lb # [ 0.045768] Built 1 zonelists, mobility grouping on. Total pages: 393078606client # [ 0.218247] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization607lb # [ 0.045769] Policy zone: DMA32608worker1 # [ 0.223467] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'609client # [ 0.220254] Spectre V2 : Enabling IBPB for BPF610lb # [ 0.048406] mem auto-init: stack:all(zero), heap alloc:on, heap free:off611worker1 # [ 0.224466] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'612lb # [ 0.051938] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1613client # [ 0.221248] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier614lb # [ 0.055446] allocated 3145728 bytes of page_ext615worker1 # [ 0.225467] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'616worker2 # [ 0.041226] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/x5qgax5h37ybfdjp7r7k01yx9abl6wij-nixos-system-worker2-test/init regInfo=/nix/store/dq71v172s5jppb7nnfi3vkq85w35rkbj-closure-info/registration console=ttyS0,115200n8 console=tty0617lb # [ 0.065831] ftrace: allocating 48731 entries in 192 pages618client # [ 0.222247] active return thunk: srso_alias_return_thunk619worker1 # [ 0.226466] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'620lb # [ 0.065833] ftrace: allocated 192 pages with 2 groups621lb # [ 0.066756] Dynamic Preempt: lazy622client # [ 0.223268] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'623worker2 # [ 0.041325] Unknown kernel command line parameters "regInfo=/nix/store/dq71v172s5jppb7nnfi3vkq85w35rkbj-closure-info/registration", will be passed to user space.624worker1 # [ 0.227467] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'625lb # [ 0.066940] rcu: Preemptible hierarchical RCU implementation.626worker2 # [ 0.041338] random: crng init done627lb # [ 0.066941] rcu: RCU event tracing is enabled.628client # [ 0.224246] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'629worker2 # [ 0.041339] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes630worker1 # [ 0.228467] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'631lb # [ 0.066942] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.632client # [ 0.225246] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'633lb # [ 0.066944] Trampoline variant of Tasks RCU enabled.634worker2 # [ 0.043563] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)635worker1 # [ 0.229467] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256636lb # [ 0.066944] Rude variant of Tasks RCU enabled.637client # [ 0.226246] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'638worker2 # [ 0.044680] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)639worker1 # [ 0.230467] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64640lb # [ 0.066945] Tracing variant of Tasks RCU enabled.641worker2 # [ 0.044719] Fallback order for Node 0: 0642client # [ 0.227246] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'643lb # [ 0.066945] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.644worker1 # [ 0.231467] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512645worker2 # [ 0.044723] Built 1 zonelists, mobility grouping on. Total pages: 393078646worker2 # [ 0.044724] Policy zone: DMA32647client # [ 0.228246] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'648lb # [ 0.066946] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1649worker1 # [ 0.232466] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024650worker2 # [ 0.047332] mem auto-init: stack:all(zero), heap alloc:on, heap free:off651lb # [ 0.066962] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.652client # [ 0.229246] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'653worker1 # [ 0.233466] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8654worker2 # [ 0.050860] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1655worker2 # [ 0.054615] allocated 3145728 bytes of page_ext656lb # [ 0.066964] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.657client # [ 0.230247] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256658worker2 # [ 0.064865] ftrace: allocating 48731 entries in 192 pages659worker1 # [ 0.234467] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.660client # [ 0.231246] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64661worker2 # [ 0.064867] ftrace: allocated 192 pages with 2 groups662lb # [ 0.066965] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.663worker2 # [ 0.065737] Dynamic Preempt: lazy664lb # [ 0.071427] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16665client # [ 0.232247] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512666worker2 # [ 0.065863] rcu: Preemptible hierarchical RCU implementation.667lb # [ 0.072127] rcu: srcu_init: Setting srcu_struct sizes based on contention.668worker2 # [ 0.065864] rcu: RCU event tracing is enabled.669client # [ 0.233246] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024670worker2 # [ 0.065865] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.671lb # [ 0.072133] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns672client # [ 0.234246] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8673worker2 # [ 0.065866] Trampoline variant of Tasks RCU enabled.674worker2 # [ 0.065867] Rude variant of Tasks RCU enabled.675lb # [ 0.072248] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)676worker2 # [ 0.065867] Tracing variant of Tasks RCU enabled.677client # [ 0.235246] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.678lb # [ 0.075795] Console: colour VGA+ 80x25679lb # [ 0.075798] printk: legacy console [tty0] enabled680worker2 # [ 0.065868] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.681lb # [ 0.105276] printk: legacy console [ttyS0] enabled682worker2 # [ 0.065868] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1683lb # [ 0.201790] ACPI: Core revision 20250807684worker2 # [ 0.065926] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.685lb # [ 0.202707] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns686worker2 # [ 0.065928] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.687lb # [ 0.204256] APIC: Switch to symmetric I/O mode setup688worker2 # [ 0.065929] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.689lb # [ 0.205264] x2apic enabled690worker2 # [ 0.070327] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16691lb # [ 0.206036] APIC: Switched APIC routing to: physical x2apic692worker2 # [ 0.070613] rcu: srcu_init: Setting srcu_struct sizes based on contention.693worker2 # [ 0.070619] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns694lb # [ 0.207885] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1695worker2 # [ 0.070732] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)696worker2 # [ 0.074253] Console: colour VGA+ 80x25697worker2 # [ 0.074256] printk: legacy console [tty0] enabled698lb # [ 0.208886] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns699worker2 # [ 0.103757] printk: legacy console [ttyS0] enabled700worker2 # [ 0.199773] ACPI: Core revision 20250807701lb # [ 0.210584] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)702lb # [ 0.211664] x86/cpu: User Mode Instruction Prevention (UMIP) activated703worker2 # [ 0.200693] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns704lb # [ 0.213662] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127705worker2 # [ 0.202254] APIC: Switch to symmetric I/O mode setup706worker2 # [ 0.203266] x2apic enabled707lb # [ 0.214580] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0708worker2 # [ 0.204050] APIC: Switched APIC routing to: physical x2apic709lb # [ 0.215584] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto710worker2 # [ 0.205951] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1711lb # [ 0.216580] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl712lb # [ 0.217581] Transient Scheduler Attacks: Vulnerable: No microcode713worker2 # [ 0.206963] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns714lb # [ 0.218580] Spectre V2 : Mitigation: Enhanced / Automatic IBRS715worker2 # [ 0.208657] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)716lb # [ 0.219581] Speculative Return Stack Overflow: Mitigation: Safe RET717worker2 # [ 0.209744] x86/cpu: User Mode Instruction Prevention (UMIP) activated718worker2 # [ 0.211765] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127719worker2 # [ 0.212655] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0720lb # [ 0.220581] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization721lb # [ 0.222590] Spectre V2 : Enabling IBPB for BPF722worker2 # [ 0.213659] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto723lb # [ 0.223581] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier724worker2 # [ 0.214655] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl725lb # [ 0.224581] active return thunk: srso_alias_return_thunk726worker2 # [ 0.216655] Transient Scheduler Attacks: Vulnerable: No microcode727worker1 # [ 0.268954] Freeing SMP alternatives memory: 44K728worker2 # [ 0.217655] Spectre V2 : Mitigation: Enhanced / Automatic IBRS729lb # [ 0.225603] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'730worker1 # [ 0.269470] pid_max: default: 32768 minimum: 301731worker2 # [ 0.218655] Speculative Return Stack Overflow: Mitigation: Safe RET732lb # [ 0.226581] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'733worker1 # [ 0.270552] LSM: initializing lsm=capability,landlock,yama,bpf,ima734worker1 # [ 0.271567] landlock: Up and running.735lb # [ 0.227581] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'736worker1 # [ 0.272195] Yama: becoming mindful.737lb # [ 0.229349] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'738worker1 # [ 0.273119] LSM support for eBPF active739client # [ 0.269832] Freeing SMP alternatives memory: 44K740lb # [ 0.230370] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'741client # [ 0.270249] pid_max: default: 32768 minimum: 301742worker1 # [ 0.273592] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)743worker2 # [ 0.219655] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization744lb # [ 0.231319] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'745worker2 # [ 0.221663] Spectre V2 : Enabling IBPB for BPF746client # [ 0.271332] LSM: initializing lsm=capability,landlock,yama,bpf,ima747worker1 # [ 0.274509] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)748lb # [ 0.232334] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'749client # [ 0.272347] landlock: Up and running.750worker2 # [ 0.222381] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier751client # [ 0.272940] Yama: becoming mindful.752lb # [ 0.233581] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256753worker2 # [ 0.223655] active return thunk: srso_alias_return_thunk754client # [ 0.273862] LSM support for eBPF active755lb # [ 0.234581] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64756worker1 # [ 0.277529] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)757worker2 # [ 0.224669] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'758lb # [ 0.235488] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512759client # [ 0.274368] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)760worker2 # [ 0.225655] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'761lb # [ 0.236289] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024762worker1 # [ 0.279064] Performance Events: Fam17h+ core perfctr, AMD PMU driver.763client # [ 0.275285] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)764worker1 # [ 0.279476] ... version: 2765lb # [ 0.237291] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8766worker2 # [ 0.226655] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'767worker1 # [ 0.280142] ... bit width: 48768worker2 # [ 0.227654] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'769worker1 # [ 0.280469] ... generic counters: 6770lb # [ 0.238264] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.771worker2 # [ 0.228654] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'772client # [ 0.278095] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)773worker1 # [ 0.281162] ... generic bitmap: 000000000000003f774worker1 # [ 0.281469] ... fixed-purpose counters: 0775worker2 # [ 0.229654] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'776client # [ 0.278824] Performance Events: Fam17h+ core perfctr, AMD PMU driver.777worker1 # [ 0.282189] ... fixed-purpose bitmap: 0000000000000000778client # [ 0.279251] ... version: 2779worker2 # [ 0.230655] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'780worker1 # [ 0.282469] ... value mask: 0000ffffffffffff781client # [ 0.279939] ... bit width: 48782worker2 # [ 0.231655] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256783worker1 # [ 0.283336] ... max period: 00007fffffffffff784client # [ 0.280249] ... generic counters: 6785worker2 # [ 0.232654] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64786worker1 # [ 0.284138] ... global_ctrl mask: 000000000000003f787client # [ 0.280944] ... generic bitmap: 000000000000003f788worker2 # [ 0.233654] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512789worker1 # [ 0.284565] signal: max sigframe size: 3376790client # [ 0.281259] ... fixed-purpose counters: 0791worker1 # [ 0.285337] rcu: Hierarchical SRCU implementation.792client # [ 0.281988] ... fixed-purpose bitmap: 0000000000000000793worker2 # [ 0.234654] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024794worker1 # [ 0.286018] rcu: Max phase no-delay instances is 400.795client # [ 0.282249] ... value mask: 0000ffffffffffff796worker2 # [ 0.235654] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8797client # [ 0.283115] ... max period: 00007fffffffffff798worker2 # [ 0.236654] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.799client # [ 0.283954] ... global_ctrl mask: 000000000000003f800client # [ 0.284359] signal: max sigframe size: 3376801client # [ 0.285173] rcu: Hierarchical SRCU implementation.802client # [ 0.285867] rcu: Max phase no-delay instances is 400.803worker1 # [ 0.291005] smp: Bringing up secondary CPUs ...804worker1 # [ 0.291483] smp: Brought up 1 node, 1 CPU805worker1 # [ 0.292177] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)806worker1 # [ 0.292631] Memory: 1454580K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)807client # [ 0.290704] smp: Bringing up secondary CPUs ...808worker1 # [ 0.293719] devtmpfs: initialized809client # [ 0.291262] smp: Brought up 1 node, 1 CPU810worker1 # [ 0.294568] x86/mm: Memory block size: 128MB811client # [ 0.291934] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)812worker1 # [ 0.296386] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)813client # [ 0.292409] Memory: 1454540K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109348K reserved, 0K cma-reserved)814client # [ 0.293491] devtmpfs: initialized815worker1 # [ 0.297408] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).816client # [ 0.294336] x86/mm: Memory block size: 128MB817worker1 # [ 0.298560] pinctrl core: initialized pinctrl subsystem818worker1 # [ 0.299724] PM: RTC time: 09:04:57, date: 2026-09-20819client # [ 0.296090] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)820client # [ 0.297215] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).821client # [ 0.298321] pinctrl core: initialized pinctrl subsystem822worker1 # [ 0.303121] NET: Registered PF_NETLINK/PF_ROUTE protocol family823client # [ 0.299416] PM: RTC time: 09:04:57, date: 2026-09-20824worker1 # [ 0.303957] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations825worker1 # [ 0.304492] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations826client # [ 0.303047] NET: Registered PF_NETLINK/PF_ROUTE protocol family827worker1 # [ 0.305735] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations828lb # [ 0.272479] Freeing SMP alternatives memory: 44K829worker1 # [ 0.306479] audit: initializing netlink subsys (disabled)830lb # [ 0.273136] pid_max: default: 32768 minimum: 301831client # [ 0.304471] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations832worker1 # [ 0.307608] thermal_sys: Registered thermal governor 'fair_share'833lb # [ 0.273683] LSM: initializing lsm=capability,landlock,yama,bpf,ima834client # [ 0.305275] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations835worker1 # [ 0.307610] thermal_sys: Registered thermal governor 'bang_bang'836lb # [ 0.274690] landlock: Up and running.837lb # [ 0.275581] Yama: becoming mindful.838worker1 # [ 0.308470] thermal_sys: Registered thermal governor 'step_wise'839client # [ 0.306527] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations840lb # [ 0.276681] LSM support for eBPF active841worker1 # [ 0.309406] thermal_sys: Registered thermal governor 'user_space'842client # [ 0.307259] audit: initializing netlink subsys (disabled)843lb # [ 0.277435] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)844worker1 # [ 0.310205] audit: type=2000 audit(1789895098.291:1): state=initialized audit_enabled=0 res=1845client # [ 0.308332] thermal_sys: Registered thermal governor 'fair_share'846lb # [ 0.278621] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)847client # [ 0.308334] thermal_sys: Registered thermal governor 'bang_bang'848worker1 # [ 0.311474] thermal_sys: Registered thermal governor 'power_allocator'849worker1 # [ 0.311489] cpuidle: using governor menu850client # [ 0.309250] thermal_sys: Registered thermal governor 'step_wise'851client # [ 0.310158] thermal_sys: Registered thermal governor 'user_space'852worker1 # [ 0.313715] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5853lb # [ 0.280947] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)854worker2 # [ 0.271707] Freeing SMP alternatives memory: 44K855client # [ 0.310984] audit: type=2000 audit(1789895098.300:1): state=initialized audit_enabled=0 res=1856worker2 # [ 0.272478] pid_max: default: 32768 minimum: 301857worker1 # [ 0.314711] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]858lb # [ 0.282185] Performance Events: Fam17h+ core perfctr, AMD PMU driver.859client # [ 0.312252] thermal_sys: Registered thermal governor 'power_allocator'860worker2 # [ 0.273279] LSM: initializing lsm=capability,landlock,yama,bpf,ima861lb # [ 0.282590] ... version: 2862worker1 # [ 0.315472] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry863client # [ 0.312266] cpuidle: using governor menu864lb # [ 0.283298] ... bit width: 48865worker2 # [ 0.273769] landlock: Up and running.866worker1 # [ 0.316480] PCI: Using configuration type 1 for base access867worker2 # [ 0.274655] Yama: becoming mindful.868lb # [ 0.283583] ... generic counters: 6869client # [ 0.314446] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5870worker2 # [ 0.275831] LSM support for eBPF active871lb # [ 0.284273] ... generic bitmap: 000000000000003f872worker1 # [ 0.317532] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.873lb # [ 0.284583] ... fixed-purpose counters: 0874client # [ 0.315481] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]875worker2 # [ 0.276591] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)876lb # [ 0.285276] ... fixed-purpose bitmap: 0000000000000000877client # [ 0.316252] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry878lb # [ 0.285583] ... value mask: 0000ffffffffffff879worker2 # [ 0.277696] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)880client # [ 0.317260] PCI: Using configuration type 1 for base access881lb # [ 0.286420] ... max period: 00007fffffffffff882lb # [ 0.287215] ... global_ctrl mask: 000000000000003f883client # [ 0.318372] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.884lb # [ 0.287680] signal: max sigframe size: 3376885worker2 # [ 0.280129] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)886worker1 # [ 0.322716] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages887lb # [ 0.288501] rcu: Hierarchical SRCU implementation.888worker2 # [ 0.281258] Performance Events: Fam17h+ core perfctr, AMD PMU driver.889lb # [ 0.289179] rcu: Max phase no-delay instances is 400.890worker1 # [ 0.323470] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page891worker2 # [ 0.281664] ... version: 2892worker2 # [ 0.282418] ... bit width: 48893worker2 # [ 0.282657] ... generic counters: 6894worker2 # [ 0.283395] ... generic bitmap: 000000000000003f895worker2 # [ 0.283657] ... fixed-purpose counters: 0896worker2 # [ 0.284417] ... fixed-purpose bitmap: 0000000000000000897worker1 # [ 0.328470] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages898client # [ 0.325505] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages899lb # [ 0.294242] smp: Bringing up secondary CPUs ...900worker2 # [ 0.284657] ... value mask: 0000ffffffffffff901worker1 # [ 0.329470] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page902lb # [ 0.294597] smp: Brought up 1 node, 1 CPU903client # [ 0.326251] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page904worker2 # [ 0.285541] ... max period: 00007fffffffffff905lb # [ 0.295261] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)906worker2 # [ 0.286323] ... global_ctrl mask: 000000000000003f907worker2 # [ 0.286755] signal: max sigframe size: 3376908worker2 # [ 0.287549] rcu: Hierarchical SRCU implementation.909lb # [ 0.295764] Memory: 1454556K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)910worker2 # [ 0.288233] rcu: Max phase no-delay instances is 400.911lb # [ 0.296853] devtmpfs: initialized912lb # [ 0.297651] x86/mm: Memory block size: 128MB913client # [ 0.331250] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages914client # [ 0.332249] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page915lb # [ 0.299370] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)916lb # [ 0.300484] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).917worker2 # [ 0.293250] smp: Bringing up secondary CPUs ...918lb # [ 0.301674] pinctrl core: initialized pinctrl subsystem919worker2 # [ 0.293672] smp: Brought up 1 node, 1 CPU920worker1 # [ 0.339846] ACPI: Added _OSI(Module Device)921lb # [ 0.302816] PM: RTC time: 09:04:57, date: 2026-09-20922worker1 # [ 0.340471] ACPI: Added _OSI(Processor Device)923worker2 # [ 0.294345] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)924worker1 # [ 0.341220] ACPI: Added _OSI(Processor Aggregator Device)925worker2 # [ 0.294800] Memory: 1454556K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)926worker2 # [ 0.295921] devtmpfs: initialized927client # [ 0.340851] ACPI: Added _OSI(Module Device)928lb # [ 0.306774] NET: Registered PF_NETLINK/PF_ROUTE protocol family929worker2 # [ 0.296822] x86/mm: Memory block size: 128MB930client # [ 0.341251] ACPI: Added _OSI(Processor Device)931lb # [ 0.308091] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations932client # [ 0.342006] ACPI: Added _OSI(Processor Aggregator Device)933worker2 # [ 0.298599] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)934lb # [ 0.308608] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations935worker2 # [ 0.299546] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).936lb # [ 0.309873] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations937worker2 # [ 0.300735] pinctrl core: initialized pinctrl subsystem938lb # [ 0.310593] audit: initializing netlink subsys (disabled)939worker2 # [ 0.301844] PM: RTC time: 09:04:57, date: 2026-09-20940lb # [ 0.311747] thermal_sys: Registered thermal governor 'fair_share'941worker1 # [ 0.349995] ACPI: 1 ACPI AML tables successfully acquired and loaded942lb # [ 0.311749] thermal_sys: Registered thermal governor 'bang_bang'943lb # [ 0.312585] thermal_sys: Registered thermal governor 'step_wise'944client # [ 0.348807] ACPI: 1 ACPI AML tables successfully acquired and loaded945lb # [ 0.313523] thermal_sys: Registered thermal governor 'user_space'946worker2 # [ 0.305710] NET: Registered PF_NETLINK/PF_ROUTE protocol family947worker1 # [ 0.353886] ACPI: Interpreter enabled948lb # [ 0.314330] audit: type=2000 audit(1789895098.343:1): state=initialized audit_enabled=0 res=1949worker1 # [ 0.354484] ACPI: PM: (supports S0 S3 S4 S5)950lb # [ 0.315586] thermal_sys: Registered thermal governor 'power_allocator'951worker2 # [ 0.307143] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations952worker1 # [ 0.355200] ACPI: Using IOAPIC for interrupt routing953lb # [ 0.315599] cpuidle: using governor menu954client # [ 0.352234] ACPI: Interpreter enabled955worker2 # [ 0.307682] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations956client # [ 0.352704] ACPI: PM: (supports S0 S3 S4 S5)957worker1 # [ 0.355545] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug958lb # [ 0.317812] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5959client # [ 0.353249] ACPI: Using IOAPIC for interrupt routing960worker1 # [ 0.356469] PCI: Using E820 reservations for host bridge windows961worker2 # [ 0.308950] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations962lb # [ 0.318815] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]963worker2 # [ 0.309668] audit: initializing netlink subsys (disabled)964client # [ 0.354057] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug965lb # [ 0.319587] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry966worker2 # [ 0.310773] thermal_sys: Registered thermal governor 'fair_share'967lb # [ 0.320596] PCI: Using configuration type 1 for base access968worker1 # [ 0.359551] ACPI: Enabled 2 GPEs in block 00 to 3F969worker2 # [ 0.310776] thermal_sys: Registered thermal governor 'bang_bang'970worker2 # [ 0.311658] thermal_sys: Registered thermal governor 'step_wise'971lb # [ 0.321663] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.972worker2 # [ 0.312643] thermal_sys: Registered thermal governor 'user_space'973client # [ 0.359250] PCI: Using E820 reservations for host bridge windows974worker2 # [ 0.313428] audit: type=2000 audit(1789895098.354:1): state=initialized audit_enabled=0 res=1975client # [ 0.360385] ACPI: Enabled 2 GPEs in block 00 to 3F976worker2 # [ 0.314660] thermal_sys: Registered thermal governor 'power_allocator'977worker2 # [ 0.314673] cpuidle: using governor menu978lb # [ 0.326872] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages979worker2 # [ 0.316935] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5980lb # [ 0.327585] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page981worker1 # [ 0.368142] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])982worker2 # [ 0.317897] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]983worker2 # [ 0.318660] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry984worker1 # [ 0.368475] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]985worker2 # [ 0.319683] PCI: Using configuration type 1 for base access986worker1 # [ 0.369996] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]987worker2 # [ 0.320862] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.988lb # [ 0.332585] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages989client # [ 0.369016] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])990worker1 # [ 0.370591] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]991lb # [ 0.333583] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page992worker1 # [ 0.371813] PCI host bridge to bus 0000:00993client # [ 0.370637] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]994worker1 # [ 0.372473] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]995client # [ 0.371347] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]996worker1 # [ 0.373469] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]997client # [ 0.372373] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]998worker1 # [ 0.374469] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]999worker2 # [ 0.325739] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1000client # [ 0.373599] PCI host bridge to bus 0000:001001worker2 # [ 0.326658] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1002worker1 # [ 0.375469] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1003client # [ 0.374253] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1004worker1 # [ 0.376469] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1005client # [ 0.375249] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1006worker1 # [ 0.377469] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1007client # [ 0.376249] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1008worker1 # [ 0.378469] pci_bus 0000:00: root bus resource [bus 00-ff]1009client # [ 0.377250] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1010worker2 # [ 0.331658] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1011worker1 # [ 0.379490] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1012lb # [ 0.344036] ACPI: Added _OSI(Module Device)1013client # [ 0.378249] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1014worker2 # [ 0.332657] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1015lb # [ 0.344586] ACPI: Added _OSI(Processor Device)1016worker1 # [ 0.380930] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1017lb # [ 0.345319] ACPI: Added _OSI(Processor Aggregator Device)1018client # [ 0.379249] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1019client # [ 0.380249] pci_bus 0000:00: root bus resource [bus 00-ff]1020client # [ 0.381296] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1021client # [ 0.382701] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1022worker1 # [ 0.383567] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1023worker1 # [ 0.384481] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1024worker2 # [ 0.343119] ACPI: Added _OSI(Module Device)1025lb # [ 0.354159] ACPI: 1 ACPI AML tables successfully acquired and loaded1026worker1 # [ 0.385490] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1027worker2 # [ 0.343659] ACPI: Added _OSI(Processor Device)1028worker2 # [ 0.344397] ACPI: Added _OSI(Processor Aggregator Device)1029worker1 # [ 0.386789] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1030lb # [ 0.357992] ACPI: Interpreter enabled1031client # [ 0.385323] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1032worker1 # [ 0.388172] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1033lb # [ 0.358599] ACPI: PM: (supports S0 S3 S4 S5)1034client # [ 0.386261] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1035lb # [ 0.359271] ACPI: Using IOAPIC for interrupt routing1036client # [ 0.387389] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1037lb # [ 0.359661] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1038worker1 # [ 0.390497] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1039lb # [ 0.360584] PCI: Using E820 reservations for host bridge windows1040worker1 # [ 0.391365] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1041client # [ 0.388600] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1042worker1 # [ 0.392199] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1043client # [ 0.389934] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1044worker1 # [ 0.392475] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1045lb # [ 0.363626] ACPI: Enabled 2 GPEs in block 00 to 3F1046worker2 # [ 0.353241] ACPI: 1 ACPI AML tables successfully acquired and loaded1047worker1 # [ 0.394025] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1048client # [ 0.393277] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1049client # [ 0.394163] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1050worker1 # [ 0.396477] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1051worker2 # [ 0.357068] ACPI: Interpreter enabled1052client # [ 0.395001] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1053worker1 # [ 0.397349] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1054worker2 # [ 0.357673] ACPI: PM: (supports S0 S3 S4 S5)1055worker2 # [ 0.358389] ACPI: Using IOAPIC for interrupt routing1056worker1 # [ 0.398201] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1057client # [ 0.395255] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1058worker2 # [ 0.358705] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1059worker1 # [ 0.399879] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1060client # [ 0.396789] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1061worker2 # [ 0.359658] PCI: Using E820 reservations for host bridge windows1062lb # [ 0.372187] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1063lb # [ 0.372589] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1064client # [ 0.398257] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1065worker2 # [ 0.362747] ACPI: Enabled 2 GPEs in block 00 to 3F1066worker1 # [ 0.402496] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1067client # [ 0.399139] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1068lb # [ 0.374050] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1069worker1 # [ 0.403476] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1070client # [ 0.399943] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1071lb # [ 0.374707] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1072worker1 # [ 0.404449] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1073lb # [ 0.375972] PCI host bridge to bus 0000:001074client # [ 0.400822] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1075lb # [ 0.376587] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1076worker1 # [ 0.405896] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1077lb # [ 0.377583] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1078lb # [ 0.378584] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1079worker1 # [ 0.407477] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1080client # [ 0.403257] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1081lb # [ 0.379584] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1082worker1 # [ 0.408368] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1083client # [ 0.404107] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1084worker1 # [ 0.409217] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1085lb # [ 0.380583] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1086worker2 # [ 0.371457] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1087client # [ 0.405000] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1088lb # [ 0.381583] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1089worker2 # [ 0.372425] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1090worker1 # [ 0.410015] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1091lb # [ 0.382584] pci_bus 0000:00: root bus resource [bus 00-ff]1092client # [ 0.405804] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1093lb # [ 0.383570] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1094worker2 # [ 0.374318] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1095worker1 # [ 0.411477] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1096client # [ 0.407277] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1097worker2 # [ 0.374784] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1098worker1 # [ 0.412324] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1099lb # [ 0.385049] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1100client # [ 0.408071] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1101worker2 # [ 0.376029] PCI host bridge to bus 0000:001102worker1 # [ 0.413182] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1103worker2 # [ 0.376661] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1104client # [ 0.409011] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1105worker2 # [ 0.377658] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1106worker1 # [ 0.414090] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1107worker2 # [ 0.378658] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1108client # [ 0.409849] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1109worker2 # [ 0.379657] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1110worker1 # [ 0.416378] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1111worker2 # [ 0.380657] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1112client # [ 0.411275] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1113worker1 # [ 0.417133] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1114client # [ 0.412096] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1115worker2 # [ 0.381657] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1116worker1 # [ 0.417492] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1117client # [ 0.412969] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1118worker2 # [ 0.382658] pci_bus 0000:00: root bus resource [bus 00-ff]1119lb # [ 0.387672] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1120lb # [ 0.388596] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1121worker1 # [ 0.419016] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1122worker2 # [ 0.383574] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1123client # [ 0.413834] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1124lb # [ 0.389528] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1125worker2 # [ 0.385106] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1126worker1 # [ 0.420494] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1127worker1 # [ 0.421338] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1128lb # [ 0.390947] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1129worker1 # [ 0.422219] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1130client # [ 0.415540] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1131worker1 # [ 0.423413] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1132lb # [ 0.392301] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1133client # [ 0.416870] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1134client # [ 0.417271] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1135worker1 # [ 0.424762] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1136lb # [ 0.395619] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1137client # [ 0.418862] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1138worker1 # [ 0.426207] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1139lb # [ 0.396466] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1140worker1 # [ 0.426491] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1141lb # [ 0.397315] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1142client # [ 0.420257] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1143worker2 # [ 0.387852] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1144client # [ 0.421150] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1145worker1 # [ 0.427999] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1146lb # [ 0.397590] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1147worker2 # [ 0.388713] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1148client # [ 0.421965] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1149worker2 # [ 0.389668] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1150lb # [ 0.399162] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1151worker1 # [ 0.429511] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1152client # [ 0.422255] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1153worker1 # [ 0.430470] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1154worker2 # [ 0.391010] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1155client # [ 0.423818] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1156worker1 # [ 0.431479] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1157lb # [ 0.401618] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1158worker2 # [ 0.392399] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1159lb # [ 0.402458] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1160client # [ 0.425261] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1161worker1 # [ 0.433292] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1162lb # [ 0.403317] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1163client # [ 0.426201] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1164worker1 # [ 0.435476] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1165worker2 # [ 0.395694] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1166lb # [ 0.404164] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1167client # [ 0.427688] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1168worker1 # [ 0.436334] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1169worker2 # [ 0.396511] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1170worker2 # [ 0.397358] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1171worker1 # [ 0.437204] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1172client # [ 0.429288] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1173worker2 # [ 0.397663] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1174worker1 # [ 0.438029] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1175lb # [ 0.407548] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1176client # [ 0.430159] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1177worker2 # [ 0.399203] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1178lb # [ 0.408207] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1179worker1 # [ 0.439208] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1180client # [ 0.431271] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1181lb # [ 0.408605] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1182worker1 # [ 0.439669] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1183client # [ 0.432810] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1184worker2 # [ 0.400689] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1185lb # [ 0.410163] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1186worker1 # [ 0.441013] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1187worker2 # [ 0.401539] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1188worker2 # [ 0.402395] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1189client # [ 0.434279] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1190client # [ 0.435165] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1191lb # [ 0.412533] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1192worker2 # [ 0.403242] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1193client # [ 0.436046] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1194lb # [ 0.413188] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1195client # [ 0.437695] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1196lb # [ 0.413605] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1197worker2 # [ 0.405665] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1198lb # [ 0.415158] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1199client # [ 0.439261] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1200worker2 # [ 0.406497] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1201client # [ 0.440317] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1202worker2 # [ 0.407429] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1203lb # [ 0.417492] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1204client # [ 0.441914] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1205worker2 # [ 0.409108] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1206lb # [ 0.418246] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1207lb # [ 0.418605] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1208worker1 # [ 0.441665] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1209worker2 # [ 0.411583] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1210worker1 # [ 0.443990] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1211lb # [ 0.420149] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1212worker2 # [ 0.412291] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1213worker1 # [ 0.444729] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1214worker2 # [ 0.412680] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1215worker1 # [ 0.445964] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1216lb # [ 0.423620] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1217worker2 # [ 0.414226] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1218worker1 # [ 0.446686] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1219lb # [ 0.424491] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1220lb # [ 0.425309] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1221worker1 # [ 0.447755] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1222worker2 # [ 0.415664] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1223worker1 # [ 0.448747] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1224lb # [ 0.426218] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1225worker2 # [ 0.416538] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1226worker2 # [ 0.417374] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1227client # [ 0.442451] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1228worker1 # [ 0.450251] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1229worker1 # [ 0.450476] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1230lb # [ 0.427591] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1231client # [ 0.444873] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1232worker2 # [ 0.418256] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1233lb # [ 0.428426] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1234worker1 # [ 0.451713] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1235client # [ 0.445479] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1236lb # [ 0.429311] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1237worker1 # [ 0.453062] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1238client # [ 0.446810] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1239lb # [ 0.429591] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1240worker2 # [ 0.421072] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1241client # [ 0.448210] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1242lb # [ 0.431106] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1243worker2 # [ 0.422295] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1244client # [ 0.449532] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1245worker2 # [ 0.422679] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1246worker1 # [ 0.456582] ACPI: PCI: Interrupt link LNKA configured for IRQ 101247lb # [ 0.432596] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1248worker1 # [ 0.457573] ACPI: PCI: Interrupt link LNKB configured for IRQ 101249client # [ 0.450495] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1250worker2 # [ 0.424286] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1251lb # [ 0.433504] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1252worker1 # [ 0.458570] ACPI: PCI: Interrupt link LNKC configured for IRQ 111253client # [ 0.452277] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1254worker1 # [ 0.459573] ACPI: PCI: Interrupt link LNKD configured for IRQ 111255lb # [ 0.434965] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1256worker2 # [ 0.425693] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1257client # [ 0.453143] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1258worker1 # [ 0.460567] ACPI: PCI: Interrupt link LNKE configured for IRQ 101259worker2 # [ 0.426534] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1260worker1 # [ 0.461566] ACPI: PCI: Interrupt link LNKF configured for IRQ 101261client # [ 0.454265] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1262worker2 # [ 0.427435] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1263worker1 # [ 0.462572] ACPI: PCI: Interrupt link LNKG configured for IRQ 111264client # [ 0.456266] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1265worker1 # [ 0.463573] ACPI: PCI: Interrupt link LNKH configured for IRQ 111266lb # [ 0.438069] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1267worker2 # [ 0.428528] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1268worker1 # [ 0.464508] ACPI: PCI: Interrupt link GSIA configured for IRQ 161269lb # [ 0.439189] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1270worker1 # [ 0.465484] ACPI: PCI: Interrupt link GSIB configured for IRQ 171271worker2 # [ 0.430203] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1272lb # [ 0.439606] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1273worker1 # [ 0.466418] ACPI: PCI: Interrupt link GSIC configured for IRQ 181274client # [ 0.460046] ACPI: PCI: Interrupt link LNKA configured for IRQ 101275worker1 # [ 0.467279] ACPI: PCI: Interrupt link GSID configured for IRQ 191276lb # [ 0.441164] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1277worker2 # [ 0.431697] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1278client # [ 0.461050] ACPI: PCI: Interrupt link LNKB configured for IRQ 101279worker1 # [ 0.468183] ACPI: PCI: Interrupt link GSIE configured for IRQ 201280worker2 # [ 0.432679] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1281client # [ 0.462070] ACPI: PCI: Interrupt link LNKC configured for IRQ 111282worker1 # [ 0.468483] ACPI: PCI: Interrupt link GSIF configured for IRQ 211283lb # [ 0.443271] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1284client # [ 0.463085] ACPI: PCI: Interrupt link LNKD configured for IRQ 111285worker1 # [ 0.469469] ACPI: PCI: Interrupt link GSIG configured for IRQ 221286worker2 # [ 0.434191] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1287lb # [ 0.443591] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1288worker1 # [ 0.470412] ACPI: PCI: Interrupt link GSIH configured for IRQ 231289client # [ 0.464046] ACPI: PCI: Interrupt link LNKE configured for IRQ 101290lb # [ 0.444533] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1291client # [ 0.465077] ACPI: PCI: Interrupt link LNKF configured for IRQ 101292worker2 # [ 0.436488] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1293worker1 # [ 0.472308] iommu: Default domain type: Translated1294client # [ 0.466306] ACPI: PCI: Interrupt link LNKG configured for IRQ 111295worker1 # [ 0.473103] iommu: DMA domain TLB invalidation policy: lazy mode1296client # [ 0.467351] ACPI: PCI: Interrupt link LNKH configured for IRQ 111297lb # [ 0.445971] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1298worker2 # [ 0.437282] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1299worker1 # [ 0.473737] ACPI: bus type USB registered1300client # [ 0.468292] ACPI: PCI: Interrupt link GSIA configured for IRQ 161301worker2 # [ 0.437680] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1302worker1 # [ 0.474485] usbcore: registered new interface driver usbfs1303lb # [ 0.448378] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1304client # [ 0.469264] ACPI: PCI: Interrupt link GSIB configured for IRQ 171305worker1 # [ 0.475357] usbcore: registered new interface driver hub1306client # [ 0.470264] ACPI: PCI: Interrupt link GSIC configured for IRQ 181307worker1 # [ 0.476154] usbcore: registered new device driver usb1308worker2 # [ 0.439198] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1309lb # [ 0.449400] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1310client # [ 0.471196] ACPI: PCI: Interrupt link GSID configured for IRQ 191311worker1 # [ 0.477317] NetLabel: Initializing1312lb # [ 0.451227] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1313client # [ 0.472033] ACPI: PCI: Interrupt link GSIE configured for IRQ 201314worker1 # [ 0.477881] NetLabel: domain hash size = 1281315worker2 # [ 0.440665] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1316client # [ 0.472972] ACPI: PCI: Interrupt link GSIF configured for IRQ 211317worker1 # [ 0.478469] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1318worker2 # [ 0.441499] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1319client # [ 0.473261] ACPI: PCI: Interrupt link GSIG configured for IRQ 221320worker1 # [ 0.479439] NetLabel: unlabeled traffic allowed by default1321worker2 # [ 0.442424] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1322client # [ 0.474239] ACPI: PCI: Interrupt link GSIH configured for IRQ 231323worker1 # [ 0.480157] PCI: Using ACPI for IRQ routing1324worker2 # [ 0.443241] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1325client # [ 0.476057] iommu: Default domain type: Translated1326client # [ 0.476853] iommu: DMA domain TLB invalidation policy: lazy mode1327client # [ 0.477503] ACPI: bus type USB registered1328worker2 # [ 0.445068] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1329client # [ 0.478255] usbcore: registered new interface driver usbfs1330worker2 # [ 0.445866] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1331client # [ 0.479224] usbcore: registered new interface driver hub1332client # [ 0.479935] usbcore: registered new device driver usb1333worker2 # [ 0.447313] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1334client # [ 0.481106] NetLabel: Initializing1335client # [ 0.481730] NetLabel: domain hash size = 1281336client # [ 0.482249] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1337lb # [ 0.451777] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1338client # [ 0.483231] NetLabel: unlabeled traffic allowed by default1339lb # [ 0.454337] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1340client # [ 0.484037] PCI: Using ACPI for IRQ routing1341lb # [ 0.454819] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1342lb # [ 0.456269] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1343lb # [ 0.456893] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1344lb # [ 0.457867] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1345lb # [ 0.458829] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1346worker2 # [ 0.447864] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1347worker2 # [ 0.450319] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1348lb # [ 0.460612] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1349worker2 # [ 0.450895] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1350lb # [ 0.461460] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1351lb # [ 0.462931] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1352worker2 # [ 0.452626] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1353lb # [ 0.464394] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1354worker2 # [ 0.453690] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1355worker2 # [ 0.454972] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1356worker2 # [ 0.455917] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1357lb # [ 0.468243] ACPI: PCI: Interrupt link LNKA configured for IRQ 101358lb # [ 0.468693] ACPI: PCI: Interrupt link LNKB configured for IRQ 101359worker2 # [ 0.457711] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1360lb # [ 0.469681] ACPI: PCI: Interrupt link LNKC configured for IRQ 111361worker2 # [ 0.458582] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1362lb # [ 0.470680] ACPI: PCI: Interrupt link LNKD configured for IRQ 111363lb # [ 0.471692] ACPI: PCI: Interrupt link LNKE configured for IRQ 101364worker2 # [ 0.459746] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1365lb # [ 0.472687] ACPI: PCI: Interrupt link LNKF configured for IRQ 101366worker2 # [ 0.461345] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1367lb # [ 0.473691] ACPI: PCI: Interrupt link LNKG configured for IRQ 111368lb # [ 0.474688] ACPI: PCI: Interrupt link LNKH configured for IRQ 111369lb # [ 0.475623] ACPI: PCI: Interrupt link GSIA configured for IRQ 161370lb # [ 0.476538] ACPI: PCI: Interrupt link GSIB configured for IRQ 171371worker2 # [ 0.464679] ACPI: PCI: Interrupt link LNKA configured for IRQ 101372worker2 # [ 0.465860] ACPI: PCI: Interrupt link LNKB configured for IRQ 101373lb # [ 0.477314] ACPI: PCI: Interrupt link GSIC configured for IRQ 181374worker2 # [ 0.466766] ACPI: PCI: Interrupt link LNKC configured for IRQ 111375worker2 # [ 0.467756] ACPI: PCI: Interrupt link LNKD configured for IRQ 111376worker2 # [ 0.468755] ACPI: PCI: Interrupt link LNKE configured for IRQ 101377worker1 # [ 0.523871] pci 0000:00:01.0: vgaarb: setting as boot VGA device1378worker2 # [ 0.469757] ACPI: PCI: Interrupt link LNKF configured for IRQ 101379lb # [ 0.478329] ACPI: PCI: Interrupt link GSID configured for IRQ 191380worker1 # [ 0.524465] pci 0000:00:01.0: vgaarb: bridge control possible1381lb # [ 0.478599] ACPI: PCI: Interrupt link GSIE configured for IRQ 201382worker2 # [ 0.470756] ACPI: PCI: Interrupt link LNKG configured for IRQ 111383worker1 # [ 0.524465] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1384worker1 # [ 0.524474] vgaarb: loaded1385lb # [ 0.479540] ACPI: PCI: Interrupt link GSIF configured for IRQ 211386worker2 # [ 0.471753] ACPI: PCI: Interrupt link LNKH configured for IRQ 111387lb # [ 0.480322] ACPI: PCI: Interrupt link GSIG configured for IRQ 221388worker1 # [ 0.525201] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01389lb # [ 0.480599] ACPI: PCI: Interrupt link GSIH configured for IRQ 231390worker1 # [ 0.525471] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1391worker2 # [ 0.472697] ACPI: PCI: Interrupt link GSIA configured for IRQ 161392lb # [ 0.482640] iommu: Default domain type: Translated1393lb # [ 0.483439] iommu: DMA domain TLB invalidation policy: lazy mode1394worker2 # [ 0.473682] ACPI: PCI: Interrupt link GSIB configured for IRQ 171395lb # [ 0.484549] ACPI: bus type USB registered1396worker2 # [ 0.474673] ACPI: PCI: Interrupt link GSIC configured for IRQ 181397lb # [ 0.485113] usbcore: registered new interface driver usbfs1398worker1 # [ 0.529553] clocksource: Switched to clocksource kvm-clock1399lb # [ 0.485596] usbcore: registered new interface driver hub1400worker2 # [ 0.475673] ACPI: PCI: Interrupt link GSID configured for IRQ 191401client # [ 0.527868] pci 0000:00:01.0: vgaarb: setting as boot VGA device1402lb # [ 0.486420] usbcore: registered new device driver usb1403worker2 # [ 0.477421] ACPI: PCI: Interrupt link GSIE configured for IRQ 201404client # [ 0.528245] pci 0000:00:01.0: vgaarb: bridge control possible1405worker2 # [ 0.477669] ACPI: PCI: Interrupt link GSIF configured for IRQ 211406worker1 # [ 0.532989] VFS: Disk quotas dquot_6.6.01407lb # [ 0.488223] NetLabel: Initializing1408client # [ 0.528245] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1409worker2 # [ 0.478638] ACPI: PCI: Interrupt link GSIG configured for IRQ 221410client # [ 0.528253] vgaarb: loaded1411lb # [ 0.488584] NetLabel: domain hash size = 1281412worker1 # [ 0.533671] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1413worker2 # [ 0.479384] ACPI: PCI: Interrupt link GSIH configured for IRQ 231414worker1 # [ 0.534928] pnp: PnP ACPI init1415client # [ 0.528953] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01416lb # [ 0.489342] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1417client # [ 0.529251] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1418worker1 # [ 0.535681] ACPI: IRQ 4 override to edge(!), high(!)1419lb # [ 0.489650] NetLabel: unlabeled traffic allowed by default1420worker2 # [ 0.480834] iommu: Default domain type: Translated1421lb # [ 0.490557] PCI: Using ACPI for IRQ routing1422worker1 # [ 0.536596] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1423worker2 # [ 0.481624] iommu: DMA domain TLB invalidation policy: lazy mode1424worker1 # [ 0.538099] pnp: PnP ACPI: found 6 devices1425worker2 # [ 0.482712] ACPI: bus type USB registered1426worker2 # [ 0.483440] usbcore: registered new interface driver usbfs1427client # [ 0.532330] clocksource: Switched to clocksource kvm-clock1428worker2 # [ 0.483673] usbcore: registered new interface driver hub1429worker2 # [ 0.484668] usbcore: registered new device driver usb1430worker2 # [ 0.486529] NetLabel: Initializing1431client # [ 0.535941] VFS: Disk quotas dquot_6.6.01432worker2 # [ 0.487144] NetLabel: domain hash size = 1281433client # [ 0.536661] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1434worker2 # [ 0.487657] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1435client # [ 0.538018] pnp: PnP ACPI init1436worker2 # [ 0.488677] NetLabel: unlabeled traffic allowed by default1437client # [ 0.538820] ACPI: IRQ 4 override to edge(!), high(!)1438worker1 # [ 0.545625] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1439worker2 # [ 0.489659] PCI: Using ACPI for IRQ routing1440worker1 # [ 0.547095] clocksource: Switched to clocksource acpi_pm1441client # [ 0.539831] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1442worker1 # [ 0.548083] NET: Registered PF_INET protocol family1443client # [ 0.541401] pnp: PnP ACPI: found 6 devices1444worker1 # [ 0.549257] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1445client # [ 0.548973] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1446client # [ 0.550495] clocksource: Switched to clocksource acpi_pm1447client # [ 0.551498] NET: Registered PF_INET protocol family1448client # [ 0.552712] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1449worker1 # [ 0.565356] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1450worker1 # [ 0.566952] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1451worker1 # [ 0.568312] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1452worker1 # [ 0.569735] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1453worker1 # [ 0.571057] TCP: Hash tables configured (established 16384 bind 16384)1454worker1 # [ 0.572328] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1455worker1 # [ 0.573676] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1456worker1 # [ 0.574782] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1457worker1 # [ 0.576032] NET: Registered PF_UNIX/PF_LOCAL protocol family1458worker1 # [ 0.577012] NET: Registered PF_XDP protocol family1459client # [ 0.569130] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1460worker1 # [ 0.577845] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1461client # [ 0.570681] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1462worker1 # [ 0.578864] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1463client # [ 0.572019] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1464worker1 # [ 0.579880] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1465lb # [ 0.534572] pci 0000:00:01.0: vgaarb: setting as boot VGA device1466client # [ 0.573390] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1467worker1 # [ 0.581002] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1468lb # [ 0.534580] pci 0000:00:01.0: vgaarb: bridge control possible1469client # [ 0.574596] TCP: Hash tables configured (established 16384 bind 16384)1470worker1 # [ 0.582054] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1471lb # [ 0.534580] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1472worker1 # [ 0.583102] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1473client # [ 0.575656] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1474lb # [ 0.534588] vgaarb: loaded1475client # [ 0.576861] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1476worker1 # [ 0.584734] ACPI: \_SB_.GSIA: Enabled at IRQ 161477lb # [ 0.535302] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01478client # [ 0.577950] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1479lb # [ 0.535584] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1480client # [ 0.579120] NET: Registered PF_UNIX/PF_LOCAL protocol family1481worker1 # [ 0.586828] ACPI: \_SB_.GSIB: Enabled at IRQ 171482client # [ 0.580096] NET: Registered PF_XDP protocol family1483client # [ 0.580969] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1484worker1 # [ 0.588947] ACPI: \_SB_.GSIC: Enabled at IRQ 181485client # [ 0.581986] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1486client # [ 0.582997] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1487worker2 # [ 0.533545] pci 0000:00:01.0: vgaarb: setting as boot VGA device1488worker1 # [ 0.590870] ACPI: \_SB_.GSID: Enabled at IRQ 191489lb # [ 0.539674] clocksource: Switched to clocksource kvm-clock1490client # [ 0.584072] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1491worker2 # [ 0.533653] pci 0000:00:01.0: vgaarb: bridge control possible1492worker1 # [ 0.592420] PCI: CLS 0 bytes, default 641493client # [ 0.585160] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1494worker2 # [ 0.533653] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1495worker2 # [ 0.533661] vgaarb: loaded1496client # [ 0.586237] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1497worker1 # [ 0.593335] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1498lb # [ 0.543219] VFS: Disk quotas dquot_6.6.01499worker2 # [ 0.534376] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01500worker1 # [ 0.595045] Trying to unpack rootfs image as initramfs...1501client # [ 0.587891] ACPI: \_SB_.GSIA: Enabled at IRQ 161502lb # [ 0.543924] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1503worker2 # [ 0.534657] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1504lb # [ 0.545237] pnp: PnP ACPI init1505lb # [ 0.546027] ACPI: IRQ 4 override to edge(!), high(!)1506client # [ 0.590397] ACPI: \_SB_.GSIB: Enabled at IRQ 171507lb # [ 0.546964] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1508worker2 # [ 0.537745] clocksource: Switched to clocksource kvm-clock1509lb # [ 0.548481] pnp: PnP ACPI: found 6 devices1510client # [ 0.592463] ACPI: \_SB_.GSIC: Enabled at IRQ 181511client # [ 0.594429] ACPI: \_SB_.GSID: Enabled at IRQ 191512worker2 # [ 0.541306] VFS: Disk quotas dquot_6.6.01513client # [ 0.596156] PCI: CLS 0 bytes, default 641514worker2 # [ 0.542041] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1515worker2 # [ 0.543452] pnp: PnP ACPI init1516client # [ 0.597073] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1517worker2 # [ 0.544270] ACPI: IRQ 4 override to edge(!), high(!)1518client # [ 0.598793] Trying to unpack rootfs image as initramfs...1519worker2 # [ 0.545269] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1520lb # [ 0.556264] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1521worker2 # [ 0.546824] pnp: PnP ACPI: found 6 devices1522lb # [ 0.557673] clocksource: Switched to clocksource acpi_pm1523lb # [ 0.558626] NET: Registered PF_INET protocol family1524lb # [ 0.559817] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1525worker2 # [ 0.554418] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1526worker2 # [ 0.555967] clocksource: Switched to clocksource acpi_pm1527worker2 # [ 0.556975] NET: Registered PF_INET protocol family1528worker2 # [ 0.558260] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1529lb # [ 0.576439] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1530lb # [ 0.578008] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1531lb # [ 0.579277] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1532lb # [ 0.580597] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1533lb # [ 0.581813] TCP: Hash tables configured (established 16384 bind 16384)1534lb # [ 0.582882] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1535lb # [ 0.584134] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1536lb # [ 0.585193] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1537worker2 # [ 0.574886] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1538lb # [ 0.586367] NET: Registered PF_UNIX/PF_LOCAL protocol family1539worker1 # [ 0.638089] Initialise system trusted keyrings1540lb # [ 0.587304] NET: Registered PF_XDP protocol family1541worker2 # [ 0.576437] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1542lb # [ 0.588132] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1543worker2 # [ 0.577732] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1544lb # [ 0.589112] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1545worker2 # [ 0.579133] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1546lb # [ 0.590096] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1547worker1 # [ 0.641549] workingset: timestamp_bits=40 max_order=19 bucket_order=01548worker2 # [ 0.580373] TCP: Hash tables configured (established 16384 bind 16384)1549lb # [ 0.591167] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1550lb # [ 0.592232] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1551worker2 # [ 0.581493] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1552worker2 # [ 0.582777] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1553lb # [ 0.593291] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1554worker2 # [ 0.583833] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1555lb # [ 0.594975] ACPI: \_SB_.GSIA: Enabled at IRQ 161556worker2 # [ 0.585103] NET: Registered PF_UNIX/PF_LOCAL protocol family1557worker2 # [ 0.586134] NET: Registered PF_XDP protocol family1558lb # [ 0.597284] ACPI: \_SB_.GSIB: Enabled at IRQ 171559worker2 # [ 0.587005] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1560client # [ 0.642116] Initialise system trusted keyrings1561worker2 # [ 0.588105] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1562lb # [ 0.599628] ACPI: \_SB_.GSIC: Enabled at IRQ 181563worker2 # [ 0.589130] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1564worker2 # [ 0.590262] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1565lb # [ 0.601573] ACPI: \_SB_.GSID: Enabled at IRQ 191566client # [ 0.645432] workingset: timestamp_bits=40 max_order=19 bucket_order=01567worker2 # [ 0.591401] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1568lb # [ 0.603220] PCI: CLS 0 bytes, default 641569worker2 # [ 0.592552] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1570lb # [ 0.604093] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1571worker2 # [ 0.594346] ACPI: \_SB_.GSIA: Enabled at IRQ 161572lb # [ 0.605766] Trying to unpack rootfs image as initramfs...1573worker2 # [ 0.596628] ACPI: \_SB_.GSIB: Enabled at IRQ 171574worker2 # [ 0.598685] ACPI: \_SB_.GSIC: Enabled at IRQ 181575worker2 # [ 0.600639] ACPI: \_SB_.GSID: Enabled at IRQ 191576worker1 # [ 0.662730] Key type asymmetric registered1577worker1 # [ 0.663434] Asymmetric key parser 'x509' registered1578worker2 # [ 0.602413] PCI: CLS 0 bytes, default 641579worker1 # [ 0.664363] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1580worker2 # [ 0.603346] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1581worker2 # [ 0.605149] Trying to unpack rootfs image as initramfs...1582worker1 # [ 0.669631] io scheduler mq-deadline registered1583worker1 # [ 0.670432] io scheduler kyber registered1584worker1 # [ 0.674088] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1585client # [ 0.668486] Key type asymmetric registered1586worker1 # [ 0.675390] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1587client # [ 0.669198] Asymmetric key parser 'x509' registered1588client # [ 0.670047] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1589worker1 # [ 0.679279] Linux agpgart interface v0.1031590worker1 # [ 0.680122] ACPI: bus type drm_connector registered1591client # [ 0.674547] io scheduler mq-deadline registered1592client # [ 0.675346] io scheduler kyber registered1593worker1 # [ 0.684944] usbcore: registered new interface driver usbserial_generic1594worker1 # [ 0.686062] usbserial: USB Serial support registered for generic1595client # [ 0.680538] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1596client # [ 0.681831] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1597worker1 # [ 0.689518] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1598worker1 # [ 0.689518] Please enable it if your BIOS has the CPPC option.1599worker1 # [ 0.691789] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1600client # [ 0.685668] Linux agpgart interface v0.1031601client # [ 0.686429] ACPI: bus type drm_connector registered1602worker1 # [ 0.695656] drop_monitor: Initializing network drop monitor service1603worker1 # [ 0.696819] NET: Registered PF_INET6 protocol family1604client # [ 0.689762] usbcore: registered new interface driver usbserial_generic1605client # [ 0.690840] usbserial: USB Serial support registered for generic1606worker1 # [ 0.699945] Segment Routing with IPv61607worker1 # [ 0.700653] In-situ OAM (IOAM) with IPv61608lb # [ 0.649429] Initialise system trusted keyrings1609client # [ 0.694401] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1610client # [ 0.694401] Please enable it if your BIOS has the CPPC option.1611worker1 # [ 0.703772] IPI shorthand broadcast: enabled1612lb # [ 0.652520] workingset: timestamp_bits=40 max_order=19 bucket_order=01613client # [ 0.696690] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1614client # [ 0.700540] drop_monitor: Initializing network drop monitor service1615client # [ 0.701745] NET: Registered PF_INET6 protocol family1616worker2 # [ 0.649143] Initialise system trusted keyrings1617client # [ 0.704701] Segment Routing with IPv61618worker1 # [ 0.711924] sched_clock: Marking stable (573014070, 138482067)->(784058049, -72561912)1619client # [ 0.705391] In-situ OAM (IOAM) with IPv61620worker2 # [ 0.652107] workingset: timestamp_bits=40 max_order=19 bucket_order=01621worker1 # [ 0.714584] registered taskstats version 11622worker1 # [ 0.715578] Loading compiled-in X.509 certificates1623client # [ 0.708653] IPI shorthand broadcast: enabled1624client # [ 0.716752] sched_clock: Marking stable (581015097, 135364978)->(791169714, -74789639)1625lb # [ 0.673871] Key type asymmetric registered1626lb # [ 0.674609] Asymmetric key parser 'x509' registered1627client # [ 0.719464] registered taskstats version 11628client # [ 0.720443] Loading compiled-in X.509 certificates1629lb # [ 0.677479] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1630lb # [ 0.680535] io scheduler mq-deadline registered1631lb # [ 0.681303] io scheduler kyber registered1632worker1 # [ 0.733510] Demotion targets for Node 0: null1633worker1 # [ 0.735546] Key type .fscrypt registered1634worker1 # [ 0.736265] Key type fscrypt-provisioning registered1635lb # [ 0.685046] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1636worker2 # [ 0.675180] Key type asymmetric registered1637worker2 # [ 0.675907] Asymmetric key parser 'x509' registered1638lb # [ 0.686316] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1639worker1 # [ 0.738616] ima: No TPM chip found, activating TPM-bypass!1640worker2 # [ 0.676742] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1641worker1 # [ 0.739527] ima: Allocated hash algorithm: sha11642worker1 # [ 0.740341] ima: No architecture policies found1643lb # [ 0.690288] Linux agpgart interface v0.1031644worker2 # [ 0.680206] io scheduler mq-deadline registered1645lb # [ 0.691101] ACPI: bus type drm_connector registered1646worker2 # [ 0.680955] io scheduler kyber registered1647worker1 # [ 0.745662] PM: Magic number: 10:986:701648worker1 # [ 0.746444] memory: hash matches1649client # [ 0.739395] Demotion targets for Node 0: null1650worker2 # [ 0.685070] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1651lb # [ 0.695933] usbcore: registered new interface driver usbserial_generic1652client # [ 0.740235] Key type .fscrypt registered1653client # [ 0.740905] Key type fscrypt-provisioning registered1654lb # [ 0.696969] usbserial: USB Serial support registered for generic1655worker2 # [ 0.686337] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1656client # [ 0.741819] ima: No TPM chip found, activating TPM-bypass!1657worker1 # [ 0.749968] RAS: Correctable Errors collector initialized.1658lb # [ 0.699489] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1659lb # [ 0.699489] Please enable it if your BIOS has the CPPC option.1660lb # [ 0.701620] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1661worker2 # [ 0.691863] Linux agpgart interface v0.1031662client # [ 0.746394] ima: Allocated hash algorithm: sha11663worker2 # [ 0.692607] ACPI: bus type drm_connector registered1664client # [ 0.747183] ima: No architecture policies found1665client # [ 0.748142] PM: Magic number: 10:986:701666client # [ 0.748927] memory: hash matches1667lb # [ 0.705612] drop_monitor: Initializing network drop monitor service1668worker2 # [ 0.695498] usbcore: registered new interface driver usbserial_generic1669lb # [ 0.706796] NET: Registered PF_INET6 protocol family1670worker2 # [ 0.696548] usbserial: USB Serial support registered for generic1671worker1 # [ 0.758646] clk: Disabling unused clocks1672worker1 # [ 0.759340] PM: genpd: Disabling unused power domains1673client # [ 0.752193] RAS: Correctable Errors collector initialized.1674lb # [ 0.709941] Segment Routing with IPv61675lb # [ 0.710629] In-situ OAM (IOAM) with IPv61676worker2 # [ 0.700079] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1677worker2 # [ 0.700079] Please enable it if your BIOS has the CPPC option.1678lb # [ 0.712772] IPI shorthand broadcast: enabled1679worker2 # [ 0.702246] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1680worker2 # [ 0.705187] drop_monitor: Initializing network drop monitor service1681client # [ 0.760784] clk: Disabling unused clocks1682worker2 # [ 0.706349] NET: Registered PF_INET6 protocol family1683client # [ 0.763397] PM: genpd: Disabling unused power domains1684worker2 # [ 0.709541] Segment Routing with IPv61685lb # [ 0.720968] sched_clock: Marking stable (587015552, 133446698)->(800792326, -80330076)1686worker2 # [ 0.712079] In-situ OAM (IOAM) with IPv61687worker2 # [ 0.713096] IPI shorthand broadcast: enabled1688lb # [ 0.725679] registered taskstats version 11689lb # [ 0.726714] Loading compiled-in X.509 certificates1690worker2 # [ 0.721207] sched_clock: Marking stable (588013547, 133035470)->(800472238, -79423221)1691worker2 # [ 0.726261] registered taskstats version 11692worker2 # [ 0.727240] Loading compiled-in X.509 certificates1693lb # [ 0.744476] Demotion targets for Node 0: null1694lb # [ 0.747518] Key type .fscrypt registered1695lb # [ 0.748234] Key type fscrypt-provisioning registered1696lb # [ 0.749237] ima: No TPM chip found, activating TPM-bypass!1697lb # [ 0.751479] ima: Allocated hash algorithm: sha11698lb # [ 0.752316] ima: No architecture policies found1699worker2 # [ 0.745066] Demotion targets for Node 0: null1700worker2 # [ 0.745911] Key type .fscrypt registered1701lb # [ 0.756478] PM: Magic number: 10:986:701702lb # [ 0.757247] memory: hash matches1703worker2 # [ 0.746599] Key type fscrypt-provisioning registered1704worker2 # [ 0.747559] ima: No TPM chip found, activating TPM-bypass!1705lb # [ 0.760739] RAS: Correctable Errors collector initialized.1706worker2 # [ 0.752073] ima: Allocated hash algorithm: sha11707worker2 # [ 0.752885] ima: No architecture policies found1708worker2 # [ 0.755098] PM: Magic number: 10:986:701709worker2 # [ 0.755899] memory: hash matches1710lb # [ 0.769435] clk: Disabling unused clocks1711worker2 # [ 0.759411] RAS: Correctable Errors collector initialized.1712lb # [ 0.770175] PM: genpd: Disabling unused power domains1713worker2 # [ 0.768093] clk: Disabling unused clocks1714worker2 # [ 0.768805] PM: genpd: Disabling unused power domains1715worker1 # [ 0.915516] Freeing initrd memory: 29092K1716worker1 # [ 0.918587] Freeing unused decrypted memory: 2028K1717worker1 # [ 0.921088] Freeing unused kernel image (initmem) memory: 3644K1718worker1 # [ 0.922139] Write protecting the kernel read-only data: 32768k1719worker1 # [ 0.923926] Freeing unused kernel image (text/rodata gap) memory: 1208K1720worker1 # [ 0.925359] Freeing unused kernel image (rodata/data gap) memory: 752K1721client # [ 0.924866] Freeing initrd memory: 29096K1722client # [ 0.928096] Freeing unused decrypted memory: 2028K1723client # [ 0.930777] Freeing unused kernel image (initmem) memory: 3644K1724client # [ 0.931920] Write protecting the kernel read-only data: 32768k1725client # [ 0.933812] Freeing unused kernel image (text/rodata gap) memory: 1208K1726client # [ 0.935351] Freeing unused kernel image (rodata/data gap) memory: 752K1727worker1 # [ 0.976187] x86/mm: Checked W+X mappings: passed, no W+X pages found.1728worker1 # [ 0.977248] Run /init as init process1729worker1 # [ 0.987227] systemd[1]: Inserted module 'autofs4'1730lb # [ 0.940739] Freeing initrd memory: 29092K1731client # [ 0.986445] x86/mm: Checked W+X mappings: passed, no W+X pages found.1732client # [ 0.987535] Run /init as init process1733lb # [ 0.944046] Freeing unused decrypted memory: 2028K1734worker2 # [ 0.934819] Freeing initrd memory: 29092K1735lb # [ 0.946744] Freeing unused kernel image (initmem) memory: 3644K1736lb # [ 0.947846] Write protecting the kernel read-only data: 32768k1737worker2 # [ 0.938111] Freeing unused decrypted memory: 2028K1738lb # [ 0.949744] Freeing unused kernel image (text/rodata gap) memory: 1208K1739lb # [ 0.951283] Freeing unused kernel image (rodata/data gap) memory: 752K1740worker2 # [ 0.940870] Freeing unused kernel image (initmem) memory: 3644K1741worker1 # [ 1.003351] fuse: init (API version 7.45)1742worker2 # [ 0.941917] Write protecting the kernel read-only data: 32768k1743client # [ 0.997616] systemd[1]: Inserted module 'autofs4'1744worker2 # [ 0.943803] Freeing unused kernel image (text/rodata gap) memory: 1208K1745worker2 # [ 0.945419] Freeing unused kernel image (rodata/data gap) memory: 752K1746worker1 # [ 1.009163] ACPI: \_SB_.GSIG: Enabled at IRQ 221747worker1 # [ 1.011530] ACPI: \_SB_.GSIH: Enabled at IRQ 231748worker1 # [ 1.014408] ACPI: \_SB_.GSIE: Enabled at IRQ 201749worker1 # [ 1.016662] ACPI: \_SB_.GSIF: Enabled at IRQ 211750client # [ 1.013934] fuse: init (API version 7.45)1751client # [ 1.019819] ACPI: \_SB_.GSIG: Enabled at IRQ 221752client # [ 1.022225] ACPI: \_SB_.GSIH: Enabled at IRQ 231753client # [ 1.025075] ACPI: \_SB_.GSIE: Enabled at IRQ 201754client # [ 1.027255] ACPI: \_SB_.GSIF: Enabled at IRQ 211755worker1 # [ 1.040181] systemd[1]: Successfully made /usr/ read-only.1756lb # [ 1.002348] x86/mm: Checked W+X mappings: passed, no W+X pages found.1757lb # [ 1.003468] Run /init as init process1758worker2 # [ 0.996638] x86/mm: Checked W+X mappings: passed, no W+X pages found.1759worker2 # [ 0.997725] Run /init as init process1760client # [ 1.052024] systemd[1]: Successfully made /usr/ read-only.1761lb # [ 1.014309] systemd[1]: Inserted module 'autofs4'1762worker2 # [ 1.008112] systemd[1]: Inserted module 'autofs4'1763lb # [ 1.030701] fuse: init (API version 7.45)1764worker2 # [ 1.024500] fuse: init (API version 7.45)1765lb # [ 1.036609] ACPI: \_SB_.GSIG: Enabled at IRQ 221766lb # [ 1.039475] ACPI: \_SB_.GSIH: Enabled at IRQ 231767worker2 # [ 1.030309] ACPI: \_SB_.GSIG: Enabled at IRQ 221768lb # [ 1.042836] ACPI: \_SB_.GSIE: Enabled at IRQ 201769worker2 # [ 1.033079] ACPI: \_SB_.GSIH: Enabled at IRQ 231770lb # [ 1.045308] ACPI: \_SB_.GSIF: Enabled at IRQ 211771worker2 # [ 1.036413] ACPI: \_SB_.GSIE: Enabled at IRQ 201772worker2 # [ 1.038852] ACPI: \_SB_.GSIF: Enabled at IRQ 211773lb # [ 1.074614] systemd[1]: Successfully made /usr/ read-only.1774worker2 # [ 1.068859] systemd[1]: Successfully made /usr/ read-only.1775worker1 # [ 1.376690] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1776worker1 # [ 1.388612] systemd[1]: Detected virtualization kvm.1777worker1 # [ 1.390725] systemd[1]: Detected architecture x86-64.1778worker1 # [ 1.392858] systemd[1]: Running in initrd.1779worker1 # [ 1.395309] systemd[1]: Initializing machine ID from random generator.1780client # [ 1.387417] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1781worker1 # [ 1.398231] systemd[1]: Hostname set to <worker1>.1782client # [ 1.392236] systemd[1]: Detected virtualization kvm.1783client # [ 1.393060] systemd[1]: Detected architecture x86-64.1784client # [ 1.393897] systemd[1]: Running in initrd.1785client # [ 1.394947] systemd[1]: Initializing machine ID from random generator.1786client # [ 1.396059] systemd[1]: Hostname set to <client>.1787lb # [ 1.411223] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1788lb # [ 1.423050] systemd[1]: Detected virtualization kvm.1789lb # [ 1.425183] systemd[1]: Detected architecture x86-64.1790worker2 # [ 1.405390] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1791lb # [ 1.427362] systemd[1]: Running in initrd.1792worker2 # [ 1.417230] systemd[1]: Detected virtualization kvm.1793worker2 # [ 1.419340] systemd[1]: Detected architecture x86-64.1794lb # [ 1.429910] systemd[1]: Initializing machine ID from random generator.1795worker2 # [ 1.421474] systemd[1]: Running in initrd.1796lb # [ 1.432825] systemd[1]: Hostname set to <lb>.1797worker2 # [ 1.423898] systemd[1]: Initializing machine ID from random generator.1798worker2 # [ 1.426770] systemd[1]: Hostname set to <worker2>.1799client # [ 1.581397] systemd[1]: bpf-restrict-fs: LSM BPF program attached1800worker1 # [ 1.620927] systemd[1]: bpf-restrict-fs: LSM BPF program attached1801client # [ 1.619463] systemd[1]: Queued start job for default target Initrd Default Target.1802client # [ 1.622669] systemd[1]: Created slice Slice /system/modprobe.1803client # [ 1.623855] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1804client # [ 1.625278] systemd[1]: Expecting device /dev/disk/by-label/nixos...1805client # [ 1.626458] systemd[1]: Reached target Path Units.1806client # [ 1.627335] systemd[1]: Reached target Slice Units.1807client # [ 1.628240] systemd[1]: Reached target Swaps.1808client # [ 1.629052] systemd[1]: Reached target Timer Units.1809client # [ 1.630030] systemd[1]: Listening on D-Bus System Message Bus Socket.1810client # [ 1.631245] systemd[1]: Listening on Journal Socket (/dev/log).1811client # [ 1.632451] systemd[1]: Listening on Journal Sockets.1812client # [ 1.633470] systemd[1]: Listening on udev Control Socket.1813client # [ 1.634512] systemd[1]: Listening on udev Kernel Socket.1814client # [ 1.635539] systemd[1]: Reached target Socket Units.1815client # [ 1.637312] systemd[1]: Starting Create List of Static Device Nodes...1816client # [ 1.641191] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1817client # [ 1.647600] systemd[1]: Starting Load Kernel Module configfs...1818client # [ 1.660615] systemd[1]: Starting Journal Service...1819worker1 # [ 1.675615] systemd[1]: Queued start job for default target Initrd Default Target.1820worker1 # [ 1.678843] systemd[1]: Created slice Slice /system/modprobe.1821worker1 # [ 1.680026] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1822worker1 # [ 1.681507] systemd[1]: Expecting device /dev/disk/by-label/nixos...1823worker1 # [ 1.682640] systemd[1]: Reached target Path Units.1824worker1 # [ 1.683544] systemd[1]: Reached target Slice Units.1825worker1 # [ 1.684445] systemd[1]: Reached target Swaps.1826worker1 # [ 1.685299] systemd[1]: Reached target Timer Units.1827worker1 # [ 1.686316] systemd[1]: Listening on D-Bus System Message Bus Socket.1828worker1 # [ 1.687607] systemd[1]: Listening on Journal Socket (/dev/log).1829worker1 # [ 1.688825] systemd[1]: Listening on Journal Sockets.1830worker1 # [ 1.689907] systemd[1]: Listening on udev Control Socket.1831client # [ 1.672110] systemd[1]: Starting Load Kernel Modules...1832worker1 # [ 1.690945] systemd[1]: Listening on udev Kernel Socket.1833worker1 # [ 1.691863] systemd[1]: Reached target Socket Units.1834worker1 # [ 1.693667] systemd[1]: Starting Create List of Static Device Nodes...1835worker1 # [ 1.698401] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1836client # [ 1.693177] netfs: FS-Cache loaded1837client # [ 1.698812] 9pnet: Installing 9P2000 support1838worker1 # [ 1.706578] systemd[1]: Starting Load Kernel Module configfs...1839lb # [ 1.654832] systemd[1]: bpf-restrict-fs: LSM BPF program attached1840worker2 # [ 1.651868] systemd[1]: bpf-restrict-fs: LSM BPF program attached1841client # [ 1.707483] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1842worker1 # [ 1.719584] systemd[1]: Starting Journal Service...1843client # [ 1.715462] systemd[1]: Starting Coldplug All udev Devices...1844client # [ 1.727807] systemd[1]: Finished Create List of Static Device Nodes.1845worker1 # [ 1.726561] systemd[1]: Starting Load Kernel Modules...1846client # [ 1.733995] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1847client # [ 1.740023] systemd-journald[67]: Collecting audit messages is disabled.1848worker1 # [ 1.749509] netfs: FS-Cache loaded1849client # [ 1.742764] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1850worker1 # [ 1.754125] 9pnet: Installing 9P2000 support1851client # [ 1.748545] systemd[1]: modprobe@configfs.service: Deactivated successfully.1852lb # [ 1.710324] systemd[1]: Queued start job for default target Initrd Default Target.1853client # [ 1.754756] systemd[1]: Finished Load Kernel Module configfs.1854worker1 # [ 1.761631] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1855client # [ 1.756207] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1856lb # [ 1.714861] systemd[1]: Created slice Slice /system/modprobe.1857lb # [ 1.716059] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1858lb # [ 1.717421] systemd[1]: Expecting device /dev/disk/by-label/nixos...1859worker2 # [ 1.706806] systemd[1]: Queued start job for default target Initrd Default Target.1860client # [ 1.760844] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1861lb # [ 1.718490] systemd[1]: Reached target Path Units.1862lb # [ 1.719299] systemd[1]: Reached target Slice Units.1863lb # [ 1.720140] systemd[1]: Reached target Swaps.1864worker1 # [ 1.771599] systemd[1]: Starting Coldplug All udev Devices...1865lb # [ 1.720934] systemd[1]: Reached target Timer Units.1866worker2 # [ 1.711351] systemd[1]: Created slice Slice /system/modprobe.1867lb # [ 1.721868] systemd[1]: Listening on D-Bus System Message Bus Socket.1868client # [ 1.765506] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1869lb # [ 1.723034] systemd[1]: Listening on Journal Socket (/dev/log).1870worker2 # [ 1.712506] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1871lb # [ 1.724153] systemd[1]: Listening on Journal Sockets.1872worker2 # [ 1.713946] systemd[1]: Expecting device /dev/disk/by-label/nixos...1873lb # [ 1.725161] systemd[1]: Listening on udev Control Socket.1874worker2 # [ 1.715097] systemd[1]: Reached target Path Units.1875lb # [ 1.726180] systemd[1]: Listening on udev Kernel Socket.1876worker2 # [ 1.715973] systemd[1]: Reached target Slice Units.1877client # [ 1.770452] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1878lb # [ 1.727133] systemd[1]: Reached target Socket Units.1879worker2 # [ 1.716813] systemd[1]: Reached target Swaps.1880worker2 # [ 1.717640] systemd[1]: Reached target Timer Units.1881lb # [ 1.728982] systemd[1]: Starting Create List of Static Device Nodes...1882worker2 # [ 1.718640] systemd[1]: Listening on D-Bus System Message Bus Socket.1883worker2 # [ 1.719830] systemd[1]: Listening on Journal Socket (/dev/log).1884worker2 # [ 1.721019] systemd[1]: Listening on Journal Sockets.1885worker2 # [ 1.722090] systemd[1]: Listening on udev Control Socket.1886worker1 # [ 1.783871] systemd[1]: Finished Create List of Static Device Nodes.1887lb # [ 1.733536] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1888worker2 # [ 1.723133] systemd[1]: Listening on udev Kernel Socket.1889worker2 # [ 1.724102] systemd[1]: Reached target Socket Units.1890worker2 # [ 1.725898] systemd[1]: Starting Create List of Static Device Nodes...1891worker1 # [ 1.789083] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1892worker2 # [ 1.729942] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1893lb # [ 1.741531] systemd[1]: Starting Load Kernel Module configfs...1894worker1 # [ 1.796654] systemd-journald[67]: Collecting audit messages is disabled.1895worker1 # [ 1.798208] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1896worker2 # [ 1.737240] systemd[1]: Starting Load Kernel Module configfs...1897client # [ 1.794959] systemd[1]: Finished Load Kernel Modules.1898worker1 # [ 1.802634] systemd[1]: modprobe@configfs.service: Deactivated successfully.1899lb # [ 1.755964] systemd[1]: Starting Journal Service...1900worker1 # [ 1.806060] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1901client # [ 1.802589] systemd[1]: Starting Apply Kernel Variables...1902worker1 # [ 1.811863] systemd[1]: Finished Load Kernel Module configfs.1903worker2 # [ 1.752258] systemd[1]: Starting Journal Service...1904worker1 # [ 1.815932] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1905client # [ 1.814490] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1906worker1 # [ 1.821620] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1907worker1 # [ 1.826590] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1908client # [ 1.823585] systemd[1]: Starting Create Static Device Nodes in /dev...1909lb # [ 1.769499] systemd[1]: Starting Load Kernel Modules...1910worker2 # [ 1.764119] systemd[1]: Starting Load Kernel Modules...1911lb # [ 1.790734] netfs: FS-Cache loaded1912worker2 # [ 1.783288] netfs: FS-Cache loaded1913client # [ 1.841708] systemd[1]: Finished Apply Kernel Variables.1914worker2 # [ 1.787888] 9pnet: Installing 9P2000 support1915lb # [ 1.799034] 9pnet: Installing 9P2000 support1916worker1 # [ 1.851012] systemd[1]: Finished Load Kernel Modules.1917worker2 # [ 1.795344] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1918worker1 # [ 1.858688] systemd[1]: Starting Apply Kernel Variables...1919lb # [ 1.809601] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1920client # [ 1.856751] systemd[1]: Started Journal Service.1921worker2 # [ 1.804129] systemd[1]: Starting Coldplug All udev Devices...1922client # [ 1.723220] systemd-modules-load[68]: Inserted module 'dm_mod'1923client # [ 1.724581] systemd-modules-load[68]: Inserted module 'virtio_balloon'1924lb # [ 1.817969] systemd-journald[67]: Collecting audit messages is disabled.1925client # [ 1.727279] systemd-modules-load[68]: Inserted module 'virtio_gpu'1926worker1 # [ 1.869633] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1927client # [ 1.728459] systemd[1]: Finished Create Static Device Nodes in /dev.1928lb # [ 1.820656] systemd[1]: Starting Coldplug All udev Devices...1929worker2 # [ 1.817155] systemd[1]: Finished Create List of Static Device Nodes.1930worker1 # [ 1.879559] systemd[1]: Starting Create Static Device Nodes in /dev...1931client # [ 1.738120] systemd[1]: Reached target Preparation for Local File Systems.1932client # [ 1.739122] systemd[1]: Reached target Local File Systems.1933worker2 # [ 1.821648] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1934client # [ 1.741256] systemd[1]: Starting Create System Files and Directories...1935lb # [ 1.832528] systemd[1]: Finished Create List of Static Device Nodes.1936lb # [ 1.837106] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1937client # [ 1.748136] systemd[1]: Starting Rule-based Manager for Device Events and Files...1938worker2 # [ 1.831142] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1939worker2 # [ 1.832679] systemd-journald[67]: Collecting audit messages is disabled.1940lb # [ 1.845880] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1941worker1 # [ 1.897799] systemd[1]: Finished Apply Kernel Variables.1942worker2 # [ 1.836473] systemd[1]: modprobe@configfs.service: Deactivated successfully.1943lb # [ 1.850887] systemd[1]: modprobe@configfs.service: Deactivated successfully.1944worker2 # [ 1.842433] systemd[1]: Finished Load Kernel Module configfs.1945worker2 # [ 1.844635] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1946lb # [ 1.857869] systemd[1]: Finished Load Kernel Module configfs.1947worker1 # [ 1.911044] systemd[1]: Finished Create Static Device Nodes in /dev.1948worker2 # [ 1.849516] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1949worker1 # [ 1.774165] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.914669] systemd[1]: Started Journal Service.1950lb # [ 1.862321] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1951worker1 # 1952client # [ 1.772778] systemd[1]: Finished Create System Files and Directories.1953worker1 # [ 1.777418] systemd-modules-load[68]: Inserted module 'virtio_balloon'1954lb # [ 1.864715] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1955worker1 # [ 1.778433] systemd-modules-load[68]: Inserted module 'virtio_gpu'1956worker2 # [ 1.855207] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1957worker2 # [ 1.860148] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1958lb # [ 1.871415] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1959worker1 # [ 1.788116] systemd[1]: Reached target Preparation for Local File Systems.1960lb # [ 1.875538] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1961worker1 # [ 1.790917] systemd[1]: Reached target Local File Systems.1962worker1 # [ 1.792060] systemd[1]: Starting Create System Files and Directories...1963worker1 # [ 1.796973] systemd[1]: Starting Rule-based Manager for Device Events and Files...1964client # [ 1.794978] systemd-udevd[83]: Using default interface naming scheme 'v261'.1965worker2 # [ 1.886571] systemd[1]: Finished Load Kernel Modules.1966lb # [ 1.901995] systemd[1]: Finished Load Kernel Modules.1967worker2 # [ 1.894254] systemd[1]: Starting Apply Kernel Variables...1968lb # [ 1.909925] systemd[1]: Starting Apply Kernel Variables...1969worker1 # [ 1.823486] systemd[1]: Finished Create System Files and Directories.1970client # [ 1.819609] systemd[1]: Started Rule-based Manager for Device Events and Files.1971worker2 # [ 1.904210] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1972lb # [ 1.922490] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1973worker2 # [ 1.914137] systemd[1]: Starting Create Static Device Nodes in /dev...1974worker1 # [ 1.844598] systemd-udevd[83]: Using default interface naming scheme 'v261'.1975lb # [ 1.932670] systemd[1]: Starting Create Static Device Nodes in /dev...1976worker2 # [ 1.930133] systemd[1]: Finished Apply Kernel Variables.1977lb # [ 1.807566] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.942674] systemd[1]: Started Journal Service.1978lb # 1979lb # [ 1.811332] systemd-modules-load[68]: Inserted module 'virtio_balloon'1980lb # [ 1.812406] systemd-modules-load[68]: Inserted module 'virtio_gpu'1981lb # [ 1.819960] systemd[1]: Finished Apply Kernel Variables.1982worker1 # [ 1.867717] systemd[1]: Started Rule-based Manager for Device Events and Files.1983client # [ 1.866951] systemd[1]: Finished Coldplug All udev Devices.1984worker2 # [ 1.948646] systemd[1]: Finished Create Static Device Nodes in /dev.1985client # [ 1.869223] systemd[1]: Reached target System Initialization.1986client # [ 1.870332] systemd[1]: Reached target Basic System.1987worker2 # [ 1.818286] systemd-modules-load[68]: Inserted module 'dm_mod'1988worker2 # [ 1.819544] systemd-modules-load[68]: Inserted module 'virtio_balloon'1989worker2 # [ 1.820626] systemd-modules-load[68]: Inserted module 'virtio_gpu'1990worker2 # [ 1.954613] systemd[1]: Started Journal Service.1991lb # [ 1.836202] systemd[1]: Finished Create Static Device Nodes in /dev.1992lb # [ 1.837804] systemd[1]: Reached target Preparation for Local File Systems.1993lb # [ 1.839851] systemd[1]: Reached target Local File Systems.1994worker2 # [ 1.831127] systemd[1]: Reached target Preparation for Local File Systems.1995worker2 # [ 1.832196] systemd[1]: Reached target Local File Systems.1996lb # [ 1.843065] systemd[1]: Starting Create System Files and Directories...1997worker2 # [ 1.834935] systemd[1]: Starting Create System Files and Directories...1998lb # [ 1.848079] systemd[1]: Starting Rule-based Manager for Device Events and Files...1999worker2 # [ 1.843070] systemd[1]: Starting Rule-based Manager for Device Events and Files...2000worker1 # [ 1.916141] systemd[1]: Finished Coldplug All udev Devices.
loading 1,883 hidden lines…
3884worker1 # [ 10.814491] NET: Registered PF_PACKET protocol family3885worker1 # [ 10.682313] dhcpcd[692]: eth0: offered 10.0.2.15 from 10.0.2.23886worker1 # [ 10.686268] dhcpcd[692]: eth0: probing address 10.0.2.15/243887lb # [ 10.682763] postgresql-pre-start[730]: selecting default "max_connections" ... 1003888lb # [ 10.738481] dhcpcd[704]: eth0: soliciting a DHCP lease3889lb # [ 10.887254] kvm_amd: TSC scaling supported3890lb # [ 10.888352] kvm_amd: Nested Virtualization enabled3891lb # [ 10.893140] kvm_amd: Nested Paging enabled3892lb # [ 10.900842] NET: Registered PF_PACKET protocol family3893lb # [ 10.901547] kvm_amd: LBR virtualization supported3894lb # [ 10.902058] kvm_amd: Virtual VMLOAD VMSAVE supported3895lb # [ 10.904920] kvm_amd: Virtual GIF supported3896lb # [ 10.905311] kvm_amd: Virtual NMI enabled3897lb # [ 10.785131] dhcpcd[704]: eth0: offered 10.0.2.15 from 10.0.2.23898lb # [ 10.788171] dhcpcd[704]: eth0: probing address 10.0.2.15/243899client # [ 10.830982] dhcpcd[564]: eth0: soliciting a DHCP lease3900client # [ 10.989584] NET: Registered PF_PACKET protocol family3901client # [ 10.860902] dhcpcd[564]: eth0: offered 10.0.2.15 from 10.0.2.23902client # [ 10.865370] dhcpcd[564]: eth0: probing address 10.0.2.15/243903lb # [ 10.844858] postgresql-pre-start[730]: selecting default "shared_buffers" ... 128MB3904lb # [ 10.993189] EDAC MC: Ver: 3.0.03905client # [ 10.908598] dhcpcd[564]: eth0: soliciting an IPv6 router3906client # [ 10.909772] dhcpcd[564]: eth0: Router Advertisement from fe80::23907client # [ 10.910599] dhcpcd[564]: eth0: adding address fec0::5054:ff:fe12:3456/643908client # [ 10.911744] dhcpcd[564]: eth0: adding route to fec0::/643909client # [ 10.912817] dhcpcd[564]: eth0: adding default route via fe80::23910lb # [ 10.952225] systemd-vconsole-setup[747]: Configuration of first virtual console was skipped, ignoring remaining ones.3911lb # [ 10.959230] systemd[1]: Finished Virtual Console Setup.3912worker1 # [ 11.540714] dhcpcd[692]: eth0: soliciting an IPv6 router3913worker1 # [ 11.543181] dhcpcd[692]: eth0: Router Advertisement from fe80::23914worker1 # [ 11.545623] dhcpcd[692]: eth0: adding address fec0::5054:ff:fe12:3456/643915worker1 # [ 11.547807] dhcpcd[692]: eth0: adding route to fec0::/643916worker1 # [ 11.549680] dhcpcd[692]: eth0: adding default route via fe80::23917worker2 # [ 11.869283] dhcpcd[689]: eth0: soliciting a DHCP lease3918worker2 # [ 12.027302] NET: Registered PF_PACKET protocol family3919worker2 # [ 11.900734] dhcpcd[689]: eth0: offered 10.0.2.15 from 10.0.2.23920worker2 # [ 11.907156] dhcpcd[689]: eth0: probing address 10.0.2.15/243921lb # [ 12.248314] dhcpcd[704]: eth0: soliciting an IPv6 router3922lb # [ 12.249965] dhcpcd[704]: eth0: Router Advertisement from fe80::23923lb # [ 12.250986] dhcpcd[704]: eth0: adding address fec0::5054:ff:fe12:3456/643924lb # [ 12.251955] dhcpcd[704]: eth0: adding route to fec0::/643925lb # [ 12.252772] dhcpcd[704]: eth0: adding default route via fe80::23926worker2 # [ 12.480811] dhcpcd[689]: eth0: soliciting an IPv6 router3927worker2 # [ 12.483559] dhcpcd[689]: eth0: Router Advertisement from fe80::23928worker2 # [ 12.486584] dhcpcd[689]: eth0: adding address fec0::5054:ff:fe12:3456/643929worker2 # [ 12.489290] dhcpcd[689]: eth0: adding route to fec0::/643930worker2 # [ 12.491516] dhcpcd[689]: eth0: adding default route via fe80::23931lb # [ 12.758656] postgresql-pre-start[730]: selecting default time zone ... UTC3932lb # [ 12.762349] postgresql-pre-start[730]: creating configuration files ... ok3933lb # [ 13.029821] postgresql-pre-start[730]: running bootstrap script ... ok3934lb # [ 13.583117] postgresql-pre-start[730]: performing post-bootstrap initialization ... ok3935lb # [ 13.710477] postgresql-pre-start[730]: syncing data to disk ... ok3936lb # [ 13.712249] postgresql-pre-start[730]: initdb: warning: enabling "trust" authentication for local connections3937lb # [ 13.713525] postgresql-pre-start[730]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.3938lb # [ 13.715429] postgresql-pre-start[730]: Success. You can now start the database server using:3939lb # [ 13.716487] postgresql-pre-start[730]: pg_ctl -D /var/lib/postgresql/18 -l logfile start3940lb # [ 13.835155] postgres[810]: [810] LOG: starting PostgreSQL 18.6 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit3941lb # [ 13.840793] postgres[810]: [810] LOG: listening on IPv6 address "::1", port 54323942lb # [ 13.841849] postgres[810]: [810] LOG: listening on IPv4 address "127.0.0.1", port 54323943lb # [ 13.844606] postgres[810]: [810] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"3944lb # [ 13.858807] postgres[819]: [819] LOG: database system was shut down at 2026-09-20 09:05:11 GMT3945lb # [ 13.862673] postgres[810]: [810] LOG: database system is ready to accept connections3946lb # [ 13.869036] systemd[1]: Started PostgreSQL Server.3947lb # [ 13.874925] systemd[1]: Starting PostgreSQL Setup Scripts...3948lb # [ 14.079468] postgresql-setup-start[838]: CREATE DATABASE3949lb # [ 14.123496] postgresql-setup-start[849]: CREATE ROLE3950lb # [ 14.141431] postgresql-setup-start[851]: ALTER DATABASE3951lb # [ 14.146595] systemd[1]: Finished PostgreSQL Setup Scripts.3952lb # [ 14.148329] systemd[1]: Reached target PostgreSQL.3953client # [ 15.416719] dhcpcd[564]: eth0: leased 10.0.2.15 for 86400 seconds3954client # [ 15.419126] dhcpcd[564]: eth0: adding route to 10.0.2.0/243955client # [ 15.421306] dhcpcd[564]: eth0: adding default route via 10.0.2.23956lb # [ 15.474527] dhcpcd[704]: eth0: leased 10.0.2.15 for 86400 seconds3957lb # [ 15.477202] dhcpcd[704]: eth0: adding route to 10.0.2.0/243958lb # [ 15.479435] dhcpcd[704]: eth0: adding default route via 10.0.2.23959client # [ 15.546838] systemd[1]: Started DHCP Client.3960client # [ 15.548701] systemd[1]: Reached target Multi-User System.3961client # [ 15.549777] systemd[1]: Startup finished in 855ms (kernel) + 3.822s (initrd) + 10.871s (userspace) = 15.549s.3962lb # [ 15.606342] systemd[1]: Started DHCP Client.3963lb # [ 15.607900] systemd[1]: Reached target Network is Online.3964lb # [ 15.612342] systemd[1]: Started Envoy reverse proxy.3965lb # [ 15.615709] systemd[1]: Started mock-oidc.service.3966lb # [ 15.714340] mock-oidc-server[919]: Mock OIDC Server running3967lb # [ 15.715682] mock-oidc-server[919]: OIDC Address: 192.168.1.2:80803968lb # [ 15.716776] mock-oidc-server[919]: Issue Address: 0.0.0.0:80813969lb # [ 15.717682] mock-oidc-server[919]: Issuer: http://192.168.1.2:8080/oidc3970lb # [ 15.718711] mock-oidc-server[919]: JWKS: http://192.168.1.2:8080/oidc/.well-known/jwks.json3971lb # [ 15.719884] mock-oidc-server[919]: Discovery: http://192.168.1.2:8080/oidc/.well-known/openid-configuration3972lb # [ 15.721555] mock-oidc-server[919]: Issue tokens: http://0.0.0.0:8081/issue?sub=...3973worker1 # [ 15.882560] dhcpcd[692]: eth0: leased 10.0.2.15 for 86400 seconds3974worker1 # [ 15.885227] dhcpcd[692]: eth0: adding route to 10.0.2.0/243975worker1 # [ 15.887568] dhcpcd[692]: eth0: adding default route via 10.0.2.23976worker1 # [ 16.010726] systemd[1]: Started DHCP Client.3977worker1 # [ 16.012775] systemd[1]: Reached target Multi-User System.3978worker1 # [ 16.013627] systemd[1]: Startup finished in 842ms (kernel) + 4.029s (initrd) + 11.140s (userspace) = 16.012s.3979lb # [ 16.175657] envoy[918]: [2026-09-20 09:05:13.876][918][info][main] [source/server/server.cc:467] initializing epoch 0 (base id=0, hot restart version=11.104)3980lb # [ 16.177434] envoy[918]: [2026-09-20 09:05:13.876][918][info][main] [source/server/server.cc:469] statically linked extensions:3981lb # [ 16.179075] envoy[918]: [2026-09-20 09:05:13.876][918][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata3982lb # [ 16.181173] envoy[918]: [2026-09-20 09:05:13.876][918][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http13983lb # [ 16.183315] envoy[918]: [2026-09-20 09:05:13.876][918][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality3984lb # [ 16.197794] envoy[918]: [2026-09-20 09:05:13.876][918][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities3985lb # [ 16.199640] envoy[918]: [2026-09-20 09:05:13.876][918][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor3986lb # [ 16.202024] envoy[918]: [2026-09-20 09:05:13.877][918][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options3987lb # [ 16.205929] envoy[918]: [2026-09-20 09:05:13.877][918][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap3988lb # [ 16.210160] envoy[918]: [2026-09-20 09:05:13.877][918][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default3989lb # [ 16.211817] envoy[918]: [2026-09-20 09:05:13.877][918][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default3990lb # [ 16.213486] envoy[918]: [2026-09-20 09:05:13.877][918][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation3991lb # [ 16.215698] envoy[918]: [2026-09-20 09:05:13.877][918][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port3992lb # [ 16.217400] envoy[918]: [2026-09-20 09:05:13.877][918][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche3993lb # [ 16.219300] envoy[918]: [2026-09-20 09:05:13.877][918][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts3994lb # [ 16.222040] envoy[918]: [2026-09-20 09:05:13.877][918][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator3995lb # [ 16.224387] envoy[918]: [2026-09-20 09:05:13.877][918][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value3996lb # [ 16.227804] envoy[918]: [2026-09-20 09:05:13.877][918][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain3997lb # [ 16.229587] envoy[918]: [2026-09-20 09:05:13.877][918][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed3998lb # [ 16.232106] envoy[918]: [2026-09-20 09:05:13.877][918][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file3999lb # [ 16.233881] envoy[918]: [2026-09-20 09:05:13.877][918][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json4000lb # [ 16.235389] envoy[918]: [2026-09-20 09:05:13.877][918][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction4001lb # [ 16.239336] envoy[918]: [2026-09-20 09:05:13.877][918][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based4002lb # [ 16.243061] envoy[918]: [2026-09-20 09:05:13.877][918][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name4003lb # [ 16.245186] envoy[918]: [2026-09-20 09:05:13.877][918][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls4004lb # [ 16.249812] envoy[918]: [2026-09-20 09:05:13.877][918][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default4005lb # [ 16.251717] envoy[918]: [2026-09-20 09:05:13.877][918][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config4006lb # [ 16.255042] envoy[918]: [2026-09-20 09:05:13.877][918][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe4007lb # [ 16.258401] envoy[918]: [2026-09-20 09:05:13.877][918][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff4008lb # [ 16.260600] envoy[918]: [2026-09-20 09:05:13.877][918][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua4009lb # [ 16.262094] envoy[918]: [2026-09-20 09:05:13.877][918][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic4010lb # [ 16.263850] envoy[918]: [2026-09-20 09:05:13.877][918][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest4011lb # [ 16.268458] envoy[918]: [2026-09-20 09:05:13.877][918][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response4012lb # [ 16.270366] envoy[918]: [2026-09-20 09:05:13.877][918][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd4013lb # [ 16.275038] envoy[918]: [2026-09-20 09:05:13.877][918][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr4014lb # [ 16.276824] envoy[918]: [2026-09-20 09:05:13.877][918][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter4015lb # [ 16.278586] envoy[918]: [2026-09-20 09:05:13.877][918][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override4016lb # [ 16.280951] envoy[918]: [2026-09-20 09:05:13.877][918][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector4017lb # [ 16.286541] envoy[918]: [2026-09-20 09:05:13.877][918][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default4018lb # [ 16.288143] envoy[918]: [2026-09-20 09:05:13.877][918][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory4019lb # [ 16.291331] envoy[918]: [2026-09-20 09:05:13.877][918][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule4020lb # [ 16.293966] envoy[918]: [2026-09-20 09:05:13.880][918][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats4021lb # [ 16.296411] envoy[918]: [2026-09-20 09:05:13.880][918][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router4022lb # [ 16.321610] envoy[918]: [2026-09-20 09:05:13.880][918][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router4023lb # [ 16.323341] envoy[918]: [2026-09-20 09:05:13.880][918][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso4024lb # [ 16.325308] envoy[918]: [2026-09-20 09:05:13.880][918][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor4025lb # [ 16.327906] envoy[918]: [2026-09-20 09:05:13.880][918][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy4026lb # [ 16.341934] envoy[918]: [2026-09-20 09:05:13.880][918][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation4027lb # [ 16.344749] envoy[918]: [2026-09-20 09:05:13.880][918][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme4028lb # [ 16.348458] envoy[918]: [2026-09-20 09:05:13.880][918][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service4029lb # [ 16.353417] envoy[918]: [2026-09-20 09:05:13.880][918][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert4030lb # [ 16.355675] envoy[918]: [2026-09-20 09:05:13.880][918][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth24031lb # [ 16.358353] envoy[918]: [2026-09-20 09:05:13.880][918][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable4032lb # [ 16.360736] envoy[918]: [2026-09-20 09:05:13.880][918][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection4033lb # [ 16.362852] envoy[918]: [2026-09-20 09:05:13.882][918][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override4034lb # [ 16.366240] envoy[918]: [2026-09-20 09:05:13.882][918][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple4035lb # [ 16.368878] envoy[918]: [2026-09-20 09:05:13.882][918][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit4036lb # [ 16.371976] envoy[918]: [2026-09-20 09:05:13.882][918][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid4037lb # [ 16.373853] envoy[918]: [2026-09-20 09:05:13.882][918][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router4038lb # [ 16.377431] envoy[918]: [2026-09-20 09:05:13.882][918][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default4039lb # [ 16.379593] envoy[918]: [2026-09-20 09:05:13.882][918][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re24040lb # [ 16.381483] envoy[918]: [2026-09-20 09:05:13.882][918][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal4041lb # [ 16.383534] envoy[918]: [2026-09-20 09:05:13.884][918][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4042lb # [ 16.386081] envoy[918]: [2026-09-20 09:05:13.884][918][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory4043lb # [ 16.388954] envoy[918]: [2026-09-20 09:05:13.884][918][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian24044lb # [ 16.390580] envoy[918]: [2026-09-20 09:05:13.884][918][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple4045lb # [ 16.392663] envoy[918]: [2026-09-20 09:05:13.885][918][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v84046lb # [ 16.394341] envoy[918]: [2026-09-20 09:05:13.891][918][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query4047lb # [ 16.397274] envoy[918]: [2026-09-20 09:05:13.891][918][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4048lb # [ 16.399737] envoy[918]: [2026-09-20 09:05:13.891][918][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm4049lb # [ 16.405850] envoy[918]: [2026-09-20 09:05:13.891][918][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed4050lb # [ 16.407594] envoy[918]: [2026-09-20 09:05:13.891][918][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo4051lb # [ 16.409064] envoy[918]: [2026-09-20 09:05:13.891][918][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy4052lb # [ 16.411459] envoy[918]: [2026-09-20 09:05:13.891][918][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san4053lb # [ 16.417576] envoy[918]: [2026-09-20 09:05:13.891][918][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case4054lb # [ 16.419796] envoy[918]: [2026-09-20 09:05:13.891][918][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default4055lb # [ 16.421596] envoy[918]: [2026-09-20 09:05:13.891][918][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name4056lb # [ 16.423988] envoy[918]: [2026-09-20 09:05:13.891][918][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated4057lb # [ 16.425850] envoy[918]: [2026-09-20 09:05:13.891][918][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls4058lb # [ 16.431247] envoy[918]: [2026-09-20 09:05:13.891][918][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log4059lb # [ 16.437403] envoy[918]: [2026-09-20 09:05:13.891][918][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router4060lb # [ 16.439225] envoy[918]: [2026-09-20 09:05:13.891][918][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action4061lb # [ 16.441205] envoy[918]: [2026-09-20 09:05:13.891][918][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder4062lb # [ 16.443471] envoy[918]: [2026-09-20 09:05:13.891][918][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4063lb # [ 16.445371] envoy[918]: [2026-09-20 09:05:13.891][918][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift4064lb # [ 16.448449] envoy[918]: [2026-09-20 09:05:13.891][918][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params4065lb # [ 16.457502] envoy[918]: [2026-09-20 09:05:13.891][918][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher4066lb # [ 16.459873] envoy[918]: [2026-09-20 09:05:13.891][918][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle4067lb # [ 16.461977] envoy[918]: [2026-09-20 09:05:13.891][918][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based4068lb # [ 16.463770] envoy[918]: [2026-09-20 09:05:13.891][918][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator4069lb # [ 16.466144] envoy[918]: [2026-09-20 09:05:13.891][918][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default4070lb # [ 16.468503] envoy[918]: [2026-09-20 09:05:13.891][918][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy4071lb # [ 16.470925] envoy[918]: [2026-09-20 09:05:13.891][918][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection4072lb # [ 16.475432] envoy[918]: [2026-09-20 09:05:13.891][918][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default4073lb # [ 16.476968] envoy[918]: [2026-09-20 09:05:13.892][918][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input4074lb # [ 16.478638] envoy[918]: [2026-09-20 09:05:13.892][918][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state4075lb # [ 16.481334] envoy[918]: [2026-09-20 09:05:13.892][918][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port4076lb # [ 16.489411] envoy[918]: [2026-09-20 09:05:13.892][918][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource4077lb # [ 16.492392] envoy[918]: [2026-09-20 09:05:13.892][918][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind4078lb # [ 16.494195] envoy[918]: [2026-09-20 09:05:13.892][918][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4079lb # [ 16.496757] envoy[918]: [2026-09-20 09:05:13.892][918][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin4080lb # [ 16.499800] envoy[918]: [2026-09-20 09:05:13.892][918][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher4081lb # [ 16.501648] envoy[918]: [2026-09-20 09:05:13.892][918][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header4082lb # [ 16.504296] envoy[918]: [2026-09-20 09:05:13.892][918][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default4083lb # [ 16.506098] envoy[918]: [2026-09-20 09:05:13.892][918][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input4084lb # [ 16.507953] envoy[918]: [2026-09-20 09:05:13.892][918][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter4085lb # [ 16.510326] envoy[918]: [2026-09-20 09:05:13.998][918][info][main] [source/server/server.cc:531] HTTP header map info:4086lb # [ 16.511691] envoy[918]: [2026-09-20 09:05:14.079][918][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id4087lb # [ 16.524657] envoy[918]: [2026-09-20 09:05:14.079][918][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:4088lb # [ 16.526211] envoy[918]: [2026-09-20 09:05:14.079][918][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id4089lb # [ 16.533924] envoy[918]: [2026-09-20 09:05:14.079][918][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status4090worker2 # [ 16.530699] dhcpcd[689]: eth0: leased 10.0.2.15 for 86400 seconds4091worker2 # [ 16.533590] dhcpcd[689]: eth0: adding route to 10.0.2.0/244092worker2 # [ 16.535940] dhcpcd[689]: eth0: adding default route via 10.0.2.24093lb # [ 16.565362] envoy[918]: [2026-09-20 09:05:14.267][918][info][main] [source/server/server.cc:968] runtime: {}4094lb # [ 16.572352] envoy[918]: [2026-09-20 09:05:14.274][918][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.4095lb # [ 16.574556] envoy[918]: [2026-09-20 09:05:14.276][918][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99014096lb # [ 16.577486] envoy[918]: [2026-09-20 09:05:14.279][918][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99014097lb # [ 16.580677] envoy[918]: [2026-09-20 09:05:14.282][918][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration4098lb # [ 16.582306] envoy[918]: [2026-09-20 09:05:14.283][918][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)4099lb # [ 16.583925] envoy[918]: [2026-09-20 09:05:14.284][918][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)4100lb # [ 16.631921] envoy[918]: [2026-09-20 09:05:14.333][918][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)4101worker1 # [ 16.680403] systemd[1]: Starting Nix worker-protocol over gRPC...4102worker2 # [ 16.626591] systemd[1]: Starting Nix worker-protocol over gRPC...4103lb # [ 16.654039] envoy[918]: [2026-09-20 09:05:14.355][918][info][config] [source/server/configuration_impl.cc:164] loading stats configuration4104lb # [ 16.656456] envoy[918]: [2026-09-20 09:05:14.358][918][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.4105lb # [ 16.659395] envoy[918]: [2026-09-20 09:05:14.358][918][info][main] [source/server/server.cc:1077] starting main dispatch loop4106worker2 # [ 16.672544] systemd[1]: Started DHCP Client.4107worker2 # [ 16.674494] systemd[1]: Reached target Multi-User System.4108worker1 # [ 16.881204] nix-grpc-daemon[784]: ts=2026-09-20T09:05:14Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4109worker2 # [ 16.829888] nix-grpc-daemon[770]: ts=2026-09-20T09:05:15Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4110worker1 # [ 16.891453] nix-grpc-daemon[784]: ts=2026-09-20T09:05:14Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14111worker2 # [ 16.842424] nix-grpc-daemon[770]: ts=2026-09-20T09:05:15Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14112worker1 # [ 16.923595] nix-grpc-daemon[784]: ts=2026-09-20T09:05:14Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4113worker1 # [ 16.926808] systemd[1]: Started Nix worker-protocol over gRPC.4114worker2 # [ 16.882910] nix-grpc-daemon[770]: ts=2026-09-20T09:05:15Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4115worker2 # [ 16.886346] systemd[1]: Started Nix worker-protocol over gRPC.4116worker2 # [ 16.889716] systemd[1]: Startup finished in 868ms (kernel) + 4.051s (initrd) + 11.968s (userspace) = 16.887s.4117lb # [ 16.933276] envoy[918]: [2026-09-20 09:05:14.634][918][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization4118lb # [ 16.937980] envoy[918]: [2026-09-20 09:05:14.639][918][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized4119lb # [ 16.942411] envoy[918]: [2026-09-20 09:05:14.639][918][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager4120worker1 # [ 16.995186] nix-grpc-daemon[784]: ts=2026-09-20T09:05:14Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4121lb # [ 16.946945] envoy[918]: [2026-09-20 09:05:14.643][918][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers4122worker2 # [ 16.960752] nix-grpc-daemon[770]: ts=2026-09-20T09:05:15Z level=info event=scheduler_connected addr=lb:500514123worker2 # [ 16.978479] nix-grpc-daemon[770]: ts=2026-09-20T09:05:15Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4124worker1 # [ 17.196558] nix-grpc-daemon[784]: ts=2026-09-20T09:05:14Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4125worker2 # [ 17.179780] nix-grpc-daemon[770]: ts=2026-09-20T09:05:15Z level=info event=scheduler_connected addr=lb:500514126worker2 # [ 17.194479] nix-grpc-daemon[770]: ts=2026-09-20T09:05:15Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4127worker2 # [ 17.395556] nix-grpc-daemon[770]: ts=2026-09-20T09:05:15Z level=info event=scheduler_connected addr=lb:500514128worker2 # [ 17.400935] nix-grpc-daemon[770]: ts=2026-09-20T09:05:15Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4129worker1 # [ 17.596844] nix-grpc-daemon[784]: ts=2026-09-20T09:05:14Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4130worker2 # [ 17.602108] nix-grpc-daemon[770]: ts=2026-09-20T09:05:15Z level=info event=scheduler_connected addr=lb:500514131worker2 # [ 17.606318] nix-grpc-daemon[770]: ts=2026-09-20T09:05:15Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4132worker2 # [ 17.806775] nix-grpc-daemon[770]: ts=2026-09-20T09:05:16Z level=info event=scheduler_connected addr=lb:500514133worker2 # [ 17.811809] nix-grpc-daemon[770]: ts=2026-09-20T09:05:16Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4134worker2 # [ 18.013679] nix-grpc-daemon[770]: ts=2026-09-20T09:05:16Z level=info event=scheduler_connected addr=lb:500514135worker2 # [ 18.017648] nix-grpc-daemon[770]: ts=2026-09-20T09:05:16Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4136worker2 # [ 18.218277] nix-grpc-daemon[770]: ts=2026-09-20T09:05:16Z level=info event=scheduler_connected addr=lb:500514137worker2 # [ 18.224258] nix-grpc-daemon[770]: ts=2026-09-20T09:05:16Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4138worker1 # [ 18.401631] nix-grpc-daemon[784]: ts=2026-09-20T09:05:15Z level=info event=scheduler_connected addr=lb:500514139worker1 # [ 18.407377] nix-grpc-daemon[784]: ts=2026-09-20T09:05:15Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4140worker2 # [ 18.424778] nix-grpc-daemon[770]: ts=2026-09-20T09:05:16Z level=info event=scheduler_connected addr=lb:500514141worker2 # [ 18.429809] nix-grpc-daemon[770]: ts=2026-09-20T09:05:16Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4142worker1 # [ 18.608445] nix-grpc-daemon[784]: ts=2026-09-20T09:05:15Z level=info event=scheduler_connected addr=lb:500514143worker1 # [ 18.613569] nix-grpc-daemon[784]: ts=2026-09-20T09:05:15Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4144worker2 # [ 18.630979] nix-grpc-daemon[770]: ts=2026-09-20T09:05:16Z level=info event=scheduler_connected addr=lb:500514145worker2 # [ 18.636724] nix-grpc-daemon[770]: ts=2026-09-20T09:05:16Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4146worker1 # [ 18.814681] nix-grpc-daemon[784]: ts=2026-09-20T09:05:16Z level=info event=scheduler_connected addr=lb:500514147worker1 # [ 18.819956] nix-grpc-daemon[784]: ts=2026-09-20T09:05:16Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4148worker2 # [ 18.837926] nix-grpc-daemon[770]: ts=2026-09-20T09:05:17Z level=info event=scheduler_connected addr=lb:500514149worker2 # [ 18.843849] nix-grpc-daemon[770]: ts=2026-09-20T09:05:17Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4150worker1 # [ 19.020841] nix-grpc-daemon[784]: ts=2026-09-20T09:05:16Z level=info event=scheduler_connected addr=lb:500514151worker1 # [ 19.025749] nix-grpc-daemon[784]: ts=2026-09-20T09:05:16Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4152worker2 # [ 19.046597] nix-grpc-daemon[770]: ts=2026-09-20T09:05:17Z level=info event=scheduler_connected addr=lb:500514153worker2 # [ 19.051236] nix-grpc-daemon[770]: ts=2026-09-20T09:05:17Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4154worker1 # [ 19.226905] nix-grpc-daemon[784]: ts=2026-09-20T09:05:16Z level=info event=scheduler_connected addr=lb:500514155worker1 # [ 19.231795] nix-grpc-daemon[784]: ts=2026-09-20T09:05:16Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4156worker2 # [ 19.251978] nix-grpc-daemon[770]: ts=2026-09-20T09:05:17Z level=info event=scheduler_connected addr=lb:500514157worker2 # [ 19.256931] nix-grpc-daemon[770]: ts=2026-09-20T09:05:17Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4158worker1 # [ 19.435768] nix-grpc-daemon[784]: ts=2026-09-20T09:05:16Z level=info event=scheduler_connected addr=lb:500514159worker1 # [ 19.438959] nix-grpc-daemon[784]: ts=2026-09-20T09:05:16Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4160worker2 # [ 19.457996] nix-grpc-daemon[770]: ts=2026-09-20T09:05:17Z level=info event=scheduler_connected addr=lb:500514161worker2 # [ 19.462701] nix-grpc-daemon[770]: ts=2026-09-20T09:05:17Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4162worker1 # [ 19.636894] nix-grpc-daemon[784]: ts=2026-09-20T09:05:16Z level=info event=scheduler_connected addr=lb:500514163worker1 # [ 19.641945] nix-grpc-daemon[784]: ts=2026-09-20T09:05:16Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4164worker2 # [ 19.663872] nix-grpc-daemon[770]: ts=2026-09-20T09:05:17Z level=info event=scheduler_connected addr=lb:500514165worker2 # [ 19.669166] nix-grpc-daemon[770]: ts=2026-09-20T09:05:17Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4166worker1 # [ 19.843072] nix-grpc-daemon[784]: ts=2026-09-20T09:05:17Z level=info event=scheduler_connected addr=lb:500514167worker1 # [ 19.847754] nix-grpc-daemon[784]: ts=2026-09-20T09:05:17Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4168worker2 # [ 19.870186] nix-grpc-daemon[770]: ts=2026-09-20T09:05:18Z level=info event=scheduler_connected addr=lb:500514169worker2 # [ 19.875060] nix-grpc-daemon[770]: ts=2026-09-20T09:05:18Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4170worker1 # [ 20.048825] nix-grpc-daemon[784]: ts=2026-09-20T09:05:17Z level=info event=scheduler_connected addr=lb:500514171worker1 # [ 20.054397] nix-grpc-daemon[784]: ts=2026-09-20T09:05:17Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4172worker2 # [ 20.076987] nix-grpc-daemon[770]: ts=2026-09-20T09:05:18Z level=info event=scheduler_connected addr=lb:500514173worker2 # [ 20.081606] nix-grpc-daemon[770]: ts=2026-09-20T09:05:18Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4174worker1 # [ 20.255467] nix-grpc-daemon[784]: ts=2026-09-20T09:05:17Z level=info event=scheduler_connected addr=lb:500514175worker1 # [ 20.260776] nix-grpc-daemon[784]: ts=2026-09-20T09:05:17Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4176worker2 # [ 20.282153] nix-grpc-daemon[770]: ts=2026-09-20T09:05:18Z level=info event=scheduler_connected addr=lb:500514177worker2 # [ 20.286959] nix-grpc-daemon[770]: ts=2026-09-20T09:05:18Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4178worker1 # [ 20.461764] nix-grpc-daemon[784]: ts=2026-09-20T09:05:17Z level=info event=scheduler_connected addr=lb:500514179worker1 # [ 20.464058] nix-grpc-daemon[784]: ts=2026-09-20T09:05:17Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4180worker2 # [ 20.488061] nix-grpc-daemon[770]: ts=2026-09-20T09:05:18Z level=info event=scheduler_connected addr=lb:500514181worker2 # [ 20.492942] nix-grpc-daemon[770]: ts=2026-09-20T09:05:18Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4182worker1 # [ 20.664772] nix-grpc-daemon[784]: ts=2026-09-20T09:05:17Z level=info event=scheduler_connected addr=lb:500514183worker1 # [ 20.667274] nix-grpc-daemon[784]: ts=2026-09-20T09:05:17Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4184worker2 # [ 20.694069] nix-grpc-daemon[770]: ts=2026-09-20T09:05:18Z level=info event=scheduler_connected addr=lb:500514185worker2 # [ 20.699564] nix-grpc-daemon[770]: ts=2026-09-20T09:05:18Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4186worker1 # [ 20.868364] nix-grpc-daemon[784]: ts=2026-09-20T09:05:18Z level=info event=scheduler_connected addr=lb:500514187worker1 # [ 20.873152] nix-grpc-daemon[784]: ts=2026-09-20T09:05:18Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4188worker2 # [ 20.900596] nix-grpc-daemon[770]: ts=2026-09-20T09:05:19Z level=info event=scheduler_connected addr=lb:500514189worker2 # [ 20.905425] nix-grpc-daemon[770]: ts=2026-09-20T09:05:19Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4190worker1 # [ 21.074095] nix-grpc-daemon[784]: ts=2026-09-20T09:05:18Z level=info event=scheduler_connected addr=lb:500514191worker1 # [ 21.079145] nix-grpc-daemon[784]: ts=2026-09-20T09:05:18Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4192worker2 # [ 21.108975] nix-grpc-daemon[770]: ts=2026-09-20T09:05:19Z level=info event=scheduler_connected addr=lb:500514193worker2 # [ 21.112235] nix-grpc-daemon[770]: ts=2026-09-20T09:05:19Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4194worker1 # [ 21.280146] nix-grpc-daemon[784]: ts=2026-09-20T09:05:18Z level=info event=scheduler_connected addr=lb:500514195worker1 # [ 21.285134] nix-grpc-daemon[784]: ts=2026-09-20T09:05:18Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4196worker2 # [ 21.309690] nix-grpc-daemon[770]: ts=2026-09-20T09:05:19Z level=info event=scheduler_connected addr=lb:500514197worker2 # [ 21.312193] nix-grpc-daemon[770]: ts=2026-09-20T09:05:19Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4198worker1 # [ 21.487889] nix-grpc-daemon[784]: ts=2026-09-20T09:05:18Z level=info event=scheduler_connected addr=lb:500514199worker1 # [ 21.491883] nix-grpc-daemon[784]: ts=2026-09-20T09:05:18Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4200worker2 # [ 21.513365] nix-grpc-daemon[770]: ts=2026-09-20T09:05:19Z level=info event=scheduler_connected addr=lb:500514201worker2 # [ 21.518263] nix-grpc-daemon[770]: ts=2026-09-20T09:05:19Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4202worker1 # [ 21.692451] nix-grpc-daemon[784]: ts=2026-09-20T09:05:18Z level=info event=scheduler_connected addr=lb:500514203worker1 # [ 21.695253] nix-grpc-daemon[784]: ts=2026-09-20T09:05:18Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4204worker2 # [ 21.719659] nix-grpc-daemon[770]: ts=2026-09-20T09:05:19Z level=info event=scheduler_connected addr=lb:500514205worker2 # [ 21.724747] nix-grpc-daemon[770]: ts=2026-09-20T09:05:19Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4206worker1 # [ 21.896206] nix-grpc-daemon[784]: ts=2026-09-20T09:05:19Z level=info event=scheduler_connected addr=lb:500514207worker1 # [ 21.901701] nix-grpc-daemon[784]: ts=2026-09-20T09:05:19Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4208worker2 # [ 21.925847] nix-grpc-daemon[770]: ts=2026-09-20T09:05:20Z level=info event=scheduler_connected addr=lb:500514209worker2 # [ 21.930949] nix-grpc-daemon[770]: ts=2026-09-20T09:05:20Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4210worker1 # [ 22.102842] nix-grpc-daemon[784]: ts=2026-09-20T09:05:19Z level=info event=scheduler_connected addr=lb:500514211worker1 # [ 22.107887] nix-grpc-daemon[784]: ts=2026-09-20T09:05:19Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4212worker2 # [ 22.132864] nix-grpc-daemon[770]: ts=2026-09-20T09:05:20Z level=info event=scheduler_connected addr=lb:500514213worker2 # [ 22.137740] nix-grpc-daemon[770]: ts=2026-09-20T09:05:20Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4214worker1 # [ 22.308897] nix-grpc-daemon[784]: ts=2026-09-20T09:05:19Z level=info event=scheduler_connected addr=lb:500514215worker1 # [ 22.314181] nix-grpc-daemon[784]: ts=2026-09-20T09:05:19Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4216worker2 # [ 22.338794] nix-grpc-daemon[770]: ts=2026-09-20T09:05:20Z level=info event=scheduler_connected addr=lb:500514217worker2 # [ 22.343660] nix-grpc-daemon[770]: ts=2026-09-20T09:05:20Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4218worker1 # [ 22.517772] nix-grpc-daemon[784]: ts=2026-09-20T09:05:19Z level=info event=scheduler_connected addr=lb:500514219worker1 # [ 22.520870] nix-grpc-daemon[784]: ts=2026-09-20T09:05:19Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4220worker2 # [ 22.544896] nix-grpc-daemon[770]: ts=2026-09-20T09:05:20Z level=info event=scheduler_connected addr=lb:500514221worker2 # [ 22.549826] nix-grpc-daemon[770]: ts=2026-09-20T09:05:20Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4222worker1 # [ 22.718086] nix-grpc-daemon[784]: ts=2026-09-20T09:05:19Z level=info event=scheduler_connected addr=lb:500514223worker1 # [ 22.720863] nix-grpc-daemon[784]: ts=2026-09-20T09:05:19Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4224worker2 # [ 22.751397] nix-grpc-daemon[770]: ts=2026-09-20T09:05:20Z level=info event=scheduler_connected addr=lb:500514225worker2 # [ 22.755943] nix-grpc-daemon[770]: ts=2026-09-20T09:05:20Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4226worker1 # [ 22.922166] nix-grpc-daemon[784]: ts=2026-09-20T09:05:20Z level=info event=scheduler_connected addr=lb:500514227worker1 # [ 22.927543] nix-grpc-daemon[784]: ts=2026-09-20T09:05:20Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4228worker2 # [ 22.957084] nix-grpc-daemon[770]: ts=2026-09-20T09:05:21Z level=info event=scheduler_connected addr=lb:500514229worker2 # [ 22.963581] nix-grpc-daemon[770]: ts=2026-09-20T09:05:21Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4230worker1 # [ 23.128638] nix-grpc-daemon[784]: ts=2026-09-20T09:05:20Z level=info event=scheduler_connected addr=lb:500514231worker1 # [ 23.133617] nix-grpc-daemon[784]: ts=2026-09-20T09:05:20Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4232worker2 # [ 23.165708] nix-grpc-daemon[770]: ts=2026-09-20T09:05:21Z level=info event=scheduler_connected addr=lb:500514233worker2 # [ 23.169466] nix-grpc-daemon[770]: ts=2026-09-20T09:05:21Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4234worker1 # [ 23.334801] nix-grpc-daemon[784]: ts=2026-09-20T09:05:20Z level=info event=scheduler_connected addr=lb:500514235worker1 # [ 23.339666] nix-grpc-daemon[784]: ts=2026-09-20T09:05:20Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4236worker2 # [ 23.370295] nix-grpc-daemon[770]: ts=2026-09-20T09:05:21Z level=info event=scheduler_connected addr=lb:500514237worker2 # [ 23.375227] nix-grpc-daemon[770]: ts=2026-09-20T09:05:21Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4238worker1 # [ 23.541493] nix-grpc-daemon[784]: ts=2026-09-20T09:05:20Z level=info event=scheduler_connected addr=lb:500514239worker1 # [ 23.546205] nix-grpc-daemon[784]: ts=2026-09-20T09:05:20Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4240worker2 # [ 23.576350] nix-grpc-daemon[770]: ts=2026-09-20T09:05:21Z level=info event=scheduler_connected addr=lb:500514241worker2 # [ 23.581403] nix-grpc-daemon[770]: ts=2026-09-20T09:05:21Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4242worker1 # [ 23.746930] nix-grpc-daemon[784]: ts=2026-09-20T09:05:21Z level=info event=scheduler_connected addr=lb:500514243worker1 # [ 23.749792] nix-grpc-daemon[784]: ts=2026-09-20T09:05:21Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4244worker2 # [ 23.782624] nix-grpc-daemon[770]: ts=2026-09-20T09:05:21Z level=info event=scheduler_connected addr=lb:500514245worker2 # [ 23.787500] nix-grpc-daemon[770]: ts=2026-09-20T09:05:22Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4246worker1 # [ 23.950812] nix-grpc-daemon[784]: ts=2026-09-20T09:05:21Z level=info event=scheduler_connected addr=lb:500514247worker1 # [ 23.955686] nix-grpc-daemon[784]: ts=2026-09-20T09:05:21Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4248worker2 # [ 23.988351] nix-grpc-daemon[770]: ts=2026-09-20T09:05:22Z level=info event=scheduler_connected addr=lb:500514249worker2 # [ 23.993236] nix-grpc-daemon[770]: ts=2026-09-20T09:05:22Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4250worker1 # [ 24.156867] nix-grpc-daemon[784]: ts=2026-09-20T09:05:21Z level=info event=scheduler_connected addr=lb:500514251worker1 # [ 24.161934] nix-grpc-daemon[784]: ts=2026-09-20T09:05:21Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4252worker2 # [ 24.195109] nix-grpc-daemon[770]: ts=2026-09-20T09:05:22Z level=info event=scheduler_connected addr=lb:500514253worker2 # [ 24.199978] nix-grpc-daemon[770]: ts=2026-09-20T09:05:22Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4254worker1 # [ 24.362575] nix-grpc-daemon[784]: ts=2026-09-20T09:05:21Z level=info event=scheduler_connected addr=lb:500514255worker1 # [ 24.365541] nix-grpc-daemon[784]: ts=2026-09-20T09:05:21Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4256worker2 # [ 24.400950] nix-grpc-daemon[770]: ts=2026-09-20T09:05:22Z level=info event=scheduler_connected addr=lb:500514257worker2 # [ 24.405894] nix-grpc-daemon[770]: ts=2026-09-20T09:05:22Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4258worker1 # [ 24.567724] nix-grpc-daemon[784]: ts=2026-09-20T09:05:21Z level=info event=scheduler_connected addr=lb:500514259worker1 # [ 24.572568] nix-grpc-daemon[784]: ts=2026-09-20T09:05:21Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4260worker2 # [ 24.606939] nix-grpc-daemon[770]: ts=2026-09-20T09:05:22Z level=info event=scheduler_connected addr=lb:500514261worker2 # [ 24.612067] nix-grpc-daemon[770]: ts=2026-09-20T09:05:22Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4262worker1 # [ 24.773192] nix-grpc-daemon[784]: ts=2026-09-20T09:05:22Z level=info event=scheduler_connected addr=lb:500514263worker1 # [ 24.775779] nix-grpc-daemon[784]: ts=2026-09-20T09:05:22Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4264worker2 # [ 24.813088] nix-grpc-daemon[770]: ts=2026-09-20T09:05:23Z level=info event=scheduler_connected addr=lb:500514265worker2 # [ 24.818285] nix-grpc-daemon[770]: ts=2026-09-20T09:05:23Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4266worker1 # [ 24.976503] nix-grpc-daemon[784]: ts=2026-09-20T09:05:22Z level=info event=scheduler_connected addr=lb:500514267worker1 # [ 24.978890] nix-grpc-daemon[784]: ts=2026-09-20T09:05:22Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4268worker2 # [ 25.019125] nix-grpc-daemon[770]: ts=2026-09-20T09:05:23Z level=info event=scheduler_connected addr=lb:500514269worker2 # [ 25.023800] nix-grpc-daemon[770]: ts=2026-09-20T09:05:23Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4270worker1 # [ 25.179555] nix-grpc-daemon[784]: ts=2026-09-20T09:05:22Z level=info event=scheduler_connected addr=lb:500514271worker1 # [ 25.181900] nix-grpc-daemon[784]: ts=2026-09-20T09:05:22Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4272worker2 # [ 25.225718] nix-grpc-daemon[770]: ts=2026-09-20T09:05:23Z level=info event=scheduler_connected addr=lb:500514273worker2 # [ 25.230635] nix-grpc-daemon[770]: ts=2026-09-20T09:05:23Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4274worker1 # [ 25.382669] nix-grpc-daemon[784]: ts=2026-09-20T09:05:22Z level=info event=scheduler_connected addr=lb:500514275worker1 # [ 25.386593] nix-grpc-daemon[784]: ts=2026-09-20T09:05:22Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4276worker2 # [ 25.431849] nix-grpc-daemon[770]: ts=2026-09-20T09:05:23Z level=info event=scheduler_connected addr=lb:500514277worker2 # [ 25.436639] nix-grpc-daemon[770]: ts=2026-09-20T09:05:23Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4278worker1 # [ 25.587671] nix-grpc-daemon[784]: ts=2026-09-20T09:05:22Z level=info event=scheduler_connected addr=lb:500514279worker1 # [ 25.590289] nix-grpc-daemon[784]: ts=2026-09-20T09:05:22Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4280worker2 # [ 25.637765] nix-grpc-daemon[770]: ts=2026-09-20T09:05:23Z level=info event=scheduler_connected addr=lb:500514281worker2 # [ 25.642790] nix-grpc-daemon[770]: ts=2026-09-20T09:05:23Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4282worker1 # [ 25.790821] nix-grpc-daemon[784]: ts=2026-09-20T09:05:23Z level=info event=scheduler_connected addr=lb:500514283worker1 # [ 25.793070] nix-grpc-daemon[784]: ts=2026-09-20T09:05:23Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4284worker2 # [ 25.843842] nix-grpc-daemon[770]: ts=2026-09-20T09:05:24Z level=info event=scheduler_connected addr=lb:500514285worker2 # [ 25.848731] nix-grpc-daemon[770]: ts=2026-09-20T09:05:24Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4286worker1 # [ 25.993453] nix-grpc-daemon[784]: ts=2026-09-20T09:05:23Z level=info event=scheduler_connected addr=lb:500514287worker1 # [ 25.997062] nix-grpc-daemon[784]: ts=2026-09-20T09:05:23Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4288worker2 # [ 26.049588] nix-grpc-daemon[770]: ts=2026-09-20T09:05:24Z level=info event=scheduler_connected addr=lb:500514289worker2 # [ 26.054661] nix-grpc-daemon[770]: ts=2026-09-20T09:05:24Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4290worker1 # [ 26.197824] nix-grpc-daemon[784]: ts=2026-09-20T09:05:23Z level=info event=scheduler_connected addr=lb:500514291worker1 # [ 26.200430] nix-grpc-daemon[784]: ts=2026-09-20T09:05:23Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4292worker2 # [ 26.256726] nix-grpc-daemon[770]: ts=2026-09-20T09:05:24Z level=info event=scheduler_connected addr=lb:500514293worker2 # [ 26.261745] nix-grpc-daemon[770]: ts=2026-09-20T09:05:24Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4294worker1 # [ 26.401101] nix-grpc-daemon[784]: ts=2026-09-20T09:05:23Z level=info event=scheduler_connected addr=lb:500514295worker1 # [ 26.404074] nix-grpc-daemon[784]: ts=2026-09-20T09:05:23Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4296worker2 # [ 26.462528] nix-grpc-daemon[770]: ts=2026-09-20T09:05:24Z level=info event=scheduler_connected addr=lb:500514297worker2 # [ 26.467604] nix-grpc-daemon[770]: ts=2026-09-20T09:05:24Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4298worker1 # [ 26.605124] nix-grpc-daemon[784]: ts=2026-09-20T09:05:23Z level=info event=scheduler_connected addr=lb:500514299worker1 # [ 26.607650] nix-grpc-daemon[784]: ts=2026-09-20T09:05:23Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4300worker2 # [ 26.668733] nix-grpc-daemon[770]: ts=2026-09-20T09:05:24Z level=info event=scheduler_connected addr=lb:500514301worker2 # [ 26.673993] nix-grpc-daemon[770]: ts=2026-09-20T09:05:24Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4302worker1 # [ 26.808304] nix-grpc-daemon[784]: ts=2026-09-20T09:05:24Z level=info event=scheduler_connected addr=lb:500514303worker1 # [ 26.810841] nix-grpc-daemon[784]: ts=2026-09-20T09:05:24Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4304worker2 # [ 26.875187] nix-grpc-daemon[770]: ts=2026-09-20T09:05:25Z level=info event=scheduler_connected addr=lb:500514305worker2 # [ 26.881099] nix-grpc-daemon[770]: ts=2026-09-20T09:05:25Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4306worker1 # [ 27.011333] nix-grpc-daemon[784]: ts=2026-09-20T09:05:24Z level=info event=scheduler_connected addr=lb:500514307worker1 # [ 27.014923] nix-grpc-daemon[784]: ts=2026-09-20T09:05:24Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4308lb # [ 27.094892] rustfs-bucket-start[1028]: mb s3://farm4309worker2 # [ 27.081933] nix-grpc-daemon[770]: ts=2026-09-20T09:05:25Z level=info event=scheduler_connected addr=lb:500514310lb # [ 27.100944] systemd[1]: Finished rustfs-bucket.service.4311worker2 # [ 27.088288] nix-grpc-daemon[770]: ts=2026-09-20T09:05:25Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4312lb # [ 27.107691] systemd[1]: Starting niks3 server...4313worker1 # [ 27.215997] nix-grpc-daemon[784]: ts=2026-09-20T09:05:24Z level=info event=scheduler_connected addr=lb:500514314worker1 # [ 27.220937] nix-grpc-daemon[784]: ts=2026-09-20T09:05:24Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4315worker2 # [ 27.291630] nix-grpc-daemon[770]: ts=2026-09-20T09:05:25Z level=info event=scheduler_connected addr=lb:500514316worker2 # [ 27.294761] nix-grpc-daemon[770]: ts=2026-09-20T09:05:25Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4317lb # [ 27.348111] postgres[1053]: [1053] ERROR: relation "goose_db_version" does not exist at character 364318lb # [ 27.349561] postgres[1053]: [1053] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4319worker1 # [ 27.421607] nix-grpc-daemon[784]: ts=2026-09-20T09:05:24Z level=info event=scheduler_connected addr=lb:500514320worker1 # [ 27.424653] nix-grpc-daemon[784]: ts=2026-09-20T09:05:24Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4321lb # [ 27.384619] niks3-server[1047]: 2026/09/20 09:05:25 OK 20241026095416_initial_model.sql (17.54ms)4322lb # [ 27.388785] niks3-server[1047]: 2026/09/20 09:05:25 OK 20251210153512_drop_unused_gin_index.sql (2.37ms)4323lb # [ 27.394053] niks3-server[1047]: 2026/09/20 09:05:25 OK 20251218171726_add_pins.sql (5.2ms)4324lb # [ 27.399526] niks3-server[1047]: 2026/09/20 09:05:25 OK 20260628120000_add_object_size_and_stats.sql (5.43ms)4325lb # [ 27.404342] niks3-server[1047]: 2026/09/20 09:05:25 OK 20260905000000_add_claims.sql (4.82ms)4326lb # [ 27.409196] niks3-server[1047]: 2026/09/20 09:05:25 OK 20260920000000_drop_claims.sql (4.92ms)4327lb # [ 27.410557] niks3-server[1047]: 2026/09/20 09:05:25 goose: successfully migrated database to version: 202609200000004328lb # [ 27.416669] niks3-server[1047]: 2026/09/20 09:05:25 OK 1_commit_pending_closure.sql (4.67ms)4329lb # [ 27.419447] niks3-server[1047]: 2026/09/20 09:05:25 OK 2_object_stats_trigger.sql (1.51ms)4330lb # [ 27.420733] niks3-server[1047]: 2026/09/20 09:05:25 goose: up to current file version: 24331lb # [ 27.427371] niks3-server[1047]: 2026/09/20 09:05:25 INFO Loaded signing key name=farm-test-1 path=/nix/store/c745bsy529madd2s6hz4z5is2436ni04-key4332worker2 # [ 27.492636] nix-grpc-daemon[770]: ts=2026-09-20T09:05:25Z level=info event=scheduler_connected addr=lb:500514333worker2 # [ 27.497713] nix-grpc-daemon[770]: ts=2026-09-20T09:05:25Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4334lb # [ 27.510726] niks3-server[1047]: 2026/09/20 09:05:25 INFO Read proxy enabled — serving cache objects from S34335lb # [ 27.512741] niks3-server[1047]: 2026/09/20 09:05:25 INFO Using socket-activated listener address=0.0.0.0:57514336lb # [ 27.518045] niks3-server[1047]: 2026/09/20 09:05:25 INFO systemd watchdog enabled interval=15s4337lb # [ 27.519259] systemd[1]: Started niks3 server.4338lb # [ 27.519967] systemd[1]: Reached target Multi-User System.4339lb # [ 27.522261] systemd[1]: Startup finished in 873ms (kernel) + 4.138s (initrd) + 22.509s (userspace) = 27.521s.4340lb # [ 27.524418] niks3-server[1047]: 2026/09/20 09:05:25 INFO Starting HTTP server address=0.0.0.0:57514341lb # [ 27.532150] niks3-server[1047]: 2026/09/20 09:05:25 INFO lead: acquired remote=192.168.1.4:580404342worker2 # [ 27.527986] nix-grpc-daemon[770]: ts=2026-09-20T09:05:25Z level=info event=scheduler_take_over4343worker1 # [ 27.627263] nix-grpc-daemon[784]: ts=2026-09-20T09:05:24Z level=info event=scheduler_connected addr=lb:500514344worker1 # [ 27.630929] nix-grpc-daemon[784]: ts=2026-09-20T09:05:24Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4345worker2 # [ 27.698851] nix-grpc-daemon[770]: ts=2026-09-20T09:05:25Z level=info event=scheduler_connected addr=lb:500514346worker2 # [ 27.704557] nix-grpc-daemon[770]: ts=2026-09-20T09:05:25Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04347worker1 # [ 27.831622] nix-grpc-daemon[784]: ts=2026-09-20T09:05:25Z level=info event=scheduler_connected addr=lb:500514348worker1 # [ 27.836701] nix-grpc-daemon[784]: ts=2026-09-20T09:05:25Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4349worker1 # [ 28.037596] nix-grpc-daemon[784]: ts=2026-09-20T09:05:25Z level=info event=scheduler_connected addr=lb:500514350worker2 # [ 27.982659] nix-grpc-daemon[770]: ts=2026-09-20T09:05:26Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04351lb: (finished: waiting for unit niks3.service, in 17.64 seconds)4352lb: waiting for TCP port 5751 on localhost4353lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!4354lb: (finished: waiting for TCP port 5751 on localhost, in 0.08 seconds)4355lb: waiting for unit envoy.service4356lb: (finished: waiting for unit envoy.service, in 0.05 seconds)4357lb: waiting for TCP port 50051 on localhost4358lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!4359lb # [ 28.332517] envoy[918]: conn peer=[::1]:33188 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=24360lb: (finished: waiting for TCP port 50051 on localhost, in 0.04 seconds)4361lb: waiting until exactly no endpoints of x86_64-linux are unhealthy4362lb: must succeed: curl -sf localhost:9901/clusters4363lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4364lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.09 seconds)4365subtest: one node schedules and both builders hold a WorkerSession on it4366lb: must succeed: curl -sf localhost:9901/clusters4367lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4368worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true4369worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.08 seconds)4370(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.12 seconds)4371subtest: balancer auth: client cert, bearer token, nothing, unknown CN4372worker2 # [ 28.960906] nix-grpc-daemon[770]: ts=2026-09-20T09:05:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:374864373worker2 # [ 28.973383] systemd[1]: Started Nix Daemon.4374worker2 # [ 29.193737] nix-daemon[844]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4375worker2 # [ 29.249503] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 45 ms (attempt 1/5)4376worker2 # [ 29.297042] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 22 ms (attempt 2/5)4377worker2 # [ 29.321662] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 166 ms (attempt 3/5)4378worker2 # [ 29.493602] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 117 ms (attempt 4/5)4379worker2 # [ 29.622989] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4380worker2 # [ 29.652506] nix-grpc-daemon[770]: ts=2026-09-20T09:05:27Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:374864381worker2 # [ 29.671692] nix-grpc-daemon[770]: ts=2026-09-20T09:05:27Z level=debug event=path_info_miss path=akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test error="error: path '/nix/store/akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test' is required, but there is no substituter that can build it"4382worker2 # [ 29.680487] nix-grpc-daemon[770]: ts=2026-09-20T09:05:27Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:374864383worker2 # [ 29.688830] nix-grpc-daemon[770]: ts=2026-09-20T09:05:27Z level=debug event=path_info_miss path=akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test error="error: path '/nix/store/akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test' is required, but there is no substituter that can build it"4384client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4385client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.09 seconds)4386client: must succeed: install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4387client: (finished: must succeed: install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds)4388worker2 # [ 30.067398] nix-grpc-daemon[770]: ts=2026-09-20T09:05:28Z level=info event=oidc_jwks provider=mock keys=14389worker2 # [ 30.070736] nix-grpc-daemon[770]: ts=2026-09-20T09:05:28Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:374864390worker1 # [ 30.162067] nix-grpc-daemon[784]: ts=2026-09-20T09:05:27Z level=info event=oidc_jwks provider=mock keys=14391worker1 # [ 30.167241] nix-grpc-daemon[784]: ts=2026-09-20T09:05:27Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:334584392worker1 # [ 30.178244] systemd[1]: Started Nix Daemon.4393worker1 # [ 30.389728] nix-daemon[842]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4394worker1 # [ 30.421687] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 59 ms (attempt 1/5)4395worker1 # [ 30.483128] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 54 ms (attempt 2/5)4396worker1 # [ 30.539909] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 288 ms (attempt 3/5)4397worker1 # [ 30.833960] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 276 ms (attempt 4/5)4398worker1 # [ 31.117246] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4399worker1 # [ 31.136478] nix-grpc-daemon[784]: ts=2026-09-20T09:05:28Z level=debug event=path_info_miss path=akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test error="error: path '/nix/store/akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test' is required, but there is no substituter that can build it"4400worker1 # [ 31.147346] nix-grpc-daemon[784]: ts=2026-09-20T09:05:28Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:334584401worker1 # [ 31.156510] nix-grpc-daemon[784]: ts=2026-09-20T09:05:28Z level=debug event=path_info_miss path=akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test error="error: path '/nix/store/akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test' is required, but there is no substituter that can build it"4402client: must succeed: rm -r /var/lib/nix-grpc-store4403client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.03 seconds)4404worker1 # [ 32.399391] nix-grpc-daemon[784]: ts=2026-09-20T09:05:29Z level=info event=denied method=QueryMissing cn=- role=none4405worker2 # [ 32.554467] nix-grpc-daemon[770]: ts=2026-09-20T09:05:30Z level=info event=denied method=QueryMissing cn=stranger role=none4406(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.92 seconds)4407subtest: DAG build is scheduled across workers and lands in the cache4408client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24409worker1 # [ 32.827153] nix-grpc-daemon[784]: ts=2026-09-20T09:05:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:334584410client # don't know how to build these paths:4411client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4412worker2 # [ 32.790802] nix-grpc-daemon[770]: ts=2026-09-20T09:05:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374864413worker2 # [ 32.820915] nix-grpc-daemon[770]: ts=2026-09-20T09:05:31Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04414worker2 # [ 32.825864] nix-grpc-daemon[770]: ts=2026-09-20T09:05:31Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.4:50051 clients=14415worker2 # [ 32.876917] nix-grpc-daemon[770]: ts=2026-09-20T09:05:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374864416worker2 # [ 32.900754] nix-grpc-daemon[770]: ts=2026-09-20T09:05:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374864417client # copying 1 paths to 'lb:50051'...4418worker2 # [ 32.914360] nix-grpc-daemon[770]: ts=2026-09-20T09:05:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:374864419client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4420worker2 # [ 32.942193] nix-daemon[844]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4421worker2 # [ 32.970785] nix-grpc-daemon[770]: ts=2026-09-20T09:05:31Z level=info event=niks3_push_started pid=8574422worker2 # [ 33.415066] nix-daemon[844]: accepted connection from pid 869, user nix-grpc-daemon (trusted)4423worker2 # [ 33.431802] nix-daemon[844]: reaped child process 872, status = succeeded4424worker2 # [ 33.568220] nix-daemon[844]: accepted connection from pid 875, user nix-grpc-daemon (trusted)4425worker2 # [ 33.580674] nix-daemon[844]: reaped child process 878, status = succeeded4426lb # [ 33.608566] niks3-server[1047]: 2026/09/20 09:05:31 INFO Received uploads request method=POST path=/api/pending_closures4427worker2 # [ 33.615097] nix-grpc-daemon[857]: time=2026-09-20T09:05:31.827Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4428worker2 # [ 33.618898] nix-grpc-daemon[857]: time=2026-09-20T09:05:31.831Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4429lb # [ 33.658169] niks3-server[1047]: 2026/09/20 09:05:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4430lb # [ 33.662512] niks3-server[1047]: 2026/09/20 09:05:31 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4431lb # [ 33.679554] niks3-server[1047]: 2026/09/20 09:05:31 INFO Signed narinfos id=1 count=14432worker2 # [ 33.671569] nix-grpc-daemon[857]: time=2026-09-20T09:05:31.884Z level=INFO msg="Uploading 1 narinfos"4433lb # [ 33.688188] niks3-server[1047]: 2026/09/20 09:05:31 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4434lb # [ 33.693662] niks3-server[1047]: 2026/09/20 09:05:31 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4435lb # [ 33.701637] niks3-server[1047]: 2026/09/20 09:05:31 INFO Completed upload id=14436worker2 # [ 33.691757] nix-grpc-daemon[857]: time=2026-09-20T09:05:31.904Z level=INFO msg="Upload complete. (430ms)"4437lb # [ 33.708228] niks3-server[1047]: 2026/09/20 09:05:31 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo4438worker2 # [ 33.697248] nix-grpc-daemon[770]: ts=2026-09-20T09:05:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37486 duration_s=0 paths=1 nar_bytes_in=5044439worker2 # [ 33.703198] nix-grpc-daemon[770]: ts=2026-09-20T09:05:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374864440worker2 # [ 33.712432] nix-daemon[844]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4441worker2 # [ 33.717627] nix-daemon[844]: reaped child process 855, status = succeeded4442worker2 # [ 33.720758] nix-daemon[844]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4443client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4444client # farm-a> worker2: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4445client # farm-a> LOG-farm-a-t14446worker2 # [ 34.735112] nix-daemon[844]: reaped child process 882, status = succeeded4447worker2 # [ 34.882826] nix-daemon[844]: accepted connection from pid 897, user nix-grpc-daemon (trusted)4448worker2 # [ 34.895721] nix-daemon[844]: reaped child process 900, status = succeeded4449lb # [ 34.921732] niks3-server[1047]: 2026/09/20 09:05:32 INFO Received uploads request method=POST path=/api/pending_closures4450worker2 # [ 34.924034] nix-grpc-daemon[857]: time=2026-09-20T09:05:33.137Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4451worker2 # [ 34.925694] nix-grpc-daemon[857]: time=2026-09-20T09:05:33.138Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4452lb # [ 34.971251] niks3-server[1047]: 2026/09/20 09:05:32 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4453lb # [ 34.979665] niks3-server[1047]: 2026/09/20 09:05:32 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4454lb # [ 34.996923] niks3-server[1047]: 2026/09/20 09:05:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4455lb # [ 35.002924] niks3-server[1047]: 2026/09/20 09:05:32 INFO Signed narinfos id=2 count=14456worker2 # [ 34.994725] nix-grpc-daemon[857]: time=2026-09-20T09:05:33.206Z level=INFO msg="Uploading 1 narinfos"4457lb # [ 35.008060] niks3-server[1047]: 2026/09/20 09:05:32 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4458lb # [ 35.012390] niks3-server[1047]: 2026/09/20 09:05:32 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4459lb # [ 35.018227] niks3-server[1047]: 2026/09/20 09:05:32 INFO Completed upload id=24460lb # [ 35.019897] niks3-server[1047]: 2026/09/20 09:05:32 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4461worker2 # [ 35.009662] nix-grpc-daemon[857]: time=2026-09-20T09:05:33.222Z level=INFO msg="Upload complete. (280ms)"4462worker2 # [ 35.019882] nix-grpc-daemon[770]: ts=2026-09-20T09:05:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37486 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14463worker2 # [ 35.036689] nix-daemon[844]: reaped child process 880, status = succeeded4464worker2 # [ 35.038524] nix-grpc-daemon[770]: ts=2026-09-20T09:05:33Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04465worker2 # [ 35.040686] nix-grpc-daemon[770]: ts=2026-09-20T09:05:33Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.4:50051 clients=14466worker2 # [ 35.089388] nix-grpc-daemon[770]: ts=2026-09-20T09:05:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374864467worker2 # [ 35.102840] nix-grpc-daemon[770]: ts=2026-09-20T09:05:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374864468worker2 # [ 35.115993] nix-grpc-daemon[857]: time=2026-09-20T09:05:33.329Z level=INFO msg="All 1 paths already cached"4469client # copying 1 paths to 'lb:50051'...4470worker2 # [ 35.123990] nix-grpc-daemon[770]: ts=2026-09-20T09:05:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:374864471client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4472worker2 # [ 35.144179] nix-daemon[844]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4473worker2 # [ 35.325146] nix-daemon[844]: accepted connection from pid 906, user nix-grpc-daemon (trusted)4474worker2 # [ 35.339123] nix-daemon[844]: reaped child process 909, status = succeeded4475worker2 # [ 35.483567] nix-daemon[844]: accepted connection from pid 911, user nix-grpc-daemon (trusted)4476worker2 # [ 35.497058] nix-daemon[844]: reaped child process 914, status = succeeded4477lb # [ 35.526315] niks3-server[1047]: 2026/09/20 09:05:33 INFO Received uploads request method=POST path=/api/pending_closures4478worker2 # [ 35.527713] nix-grpc-daemon[857]: time=2026-09-20T09:05:33.740Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4479worker2 # [ 35.529199] nix-grpc-daemon[857]: time=2026-09-20T09:05:33.740Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4480lb # [ 35.559113] niks3-server[1047]: 2026/09/20 09:05:33 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4481lb # [ 35.566565] niks3-server[1047]: 2026/09/20 09:05:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4482worker2 # [ 35.563700] nix-grpc-daemon[857]: time=2026-09-20T09:05:33.777Z level=INFO msg="Uploading 1 narinfos"4483lb # [ 35.573742] niks3-server[1047]: 2026/09/20 09:05:33 INFO Signed narinfos id=3 count=14484lb # [ 35.581336] niks3-server[1047]: 2026/09/20 09:05:33 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4485lb # [ 35.588436] niks3-server[1047]: 2026/09/20 09:05:33 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4486worker2 # [ 35.585884] nix-grpc-daemon[857]: time=2026-09-20T09:05:33.799Z level=INFO msg="Upload complete. (426ms)"4487lb # [ 35.596046] niks3-server[1047]: 2026/09/20 09:05:33 INFO Completed upload id=34488lb # [ 35.598608] niks3-server[1047]: 2026/09/20 09:05:33 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4489worker2 # [ 35.588062] nix-grpc-daemon[770]: ts=2026-09-20T09:05:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37486 duration_s=0 paths=1 nar_bytes_in=6244490worker2 # [ 35.590761] nix-grpc-daemon[770]: ts=2026-09-20T09:05:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374864491worker2 # [ 35.598197] nix-daemon[844]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4492worker2 # [ 35.600060] nix-daemon[844]: reaped child process 904, status = succeeded4493worker2 # [ 35.604924] nix-daemon[844]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4494client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4495client # farm-b> worker2: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4496client # farm-b> LOG-farm-b-t14497worker2 # [ 36.628175] nix-daemon[844]: reaped child process 919, status = succeeded4498lb # [ 36.640496] envoy[918]: conn peer=[2001:db8:1::1]:59606 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=715 tx=389 ms=7824499lb # [ 36.644408] envoy[918]: conn peer=[2001:db8:1::1]:59612 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2613 tx=406 ms=10644500lb # [ 36.647735] envoy[918]: conn peer=[2001:db8:1::1]:59614 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=94501lb # [ 36.650916] envoy[918]: conn peer=[2001:db8:1::1]:59628 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=84502worker2 # [ 36.788465] nix-daemon[844]: accepted connection from pid 934, user nix-grpc-daemon (trusted)4503worker2 # [ 36.799684] nix-daemon[844]: reaped child process 937, status = succeeded4504lb # [ 36.824408] niks3-server[1047]: 2026/09/20 09:05:34 INFO Received uploads request method=POST path=/api/pending_closures4505worker2 # [ 36.822729] nix-grpc-daemon[857]: time=2026-09-20T09:05:35.035Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4506worker2 # [ 36.824218] nix-grpc-daemon[857]: time=2026-09-20T09:05:35.037Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4507lb # [ 36.856161] niks3-server[1047]: 2026/09/20 09:05:34 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4508lb # [ 36.870972] niks3-server[1047]: 2026/09/20 09:05:34 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4509lb # [ 36.881944] niks3-server[1047]: 2026/09/20 09:05:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4510lb # [ 36.885922] niks3-server[1047]: 2026/09/20 09:05:34 INFO Signed narinfos id=4 count=14511worker2 # [ 36.878814] nix-grpc-daemon[857]: time=2026-09-20T09:05:35.091Z level=INFO msg="Uploading 1 narinfos"4512lb # [ 36.891281] niks3-server[1047]: 2026/09/20 09:05:34 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4513lb # [ 36.899872] niks3-server[1047]: 2026/09/20 09:05:34 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4514lb # [ 36.902748] niks3-server[1047]: 2026/09/20 09:05:34 INFO Completed upload id=44515lb # [ 36.904962] niks3-server[1047]: 2026/09/20 09:05:34 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4516worker2 # [ 36.894453] nix-grpc-daemon[857]: time=2026-09-20T09:05:35.107Z level=INFO msg="Upload complete. (280ms)"4517worker2 # [ 36.900324] nix-grpc-daemon[770]: ts=2026-09-20T09:05:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37486 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14518worker2 # [ 36.911761] nix-grpc-daemon[770]: ts=2026-09-20T09:05:35Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04519worker2 # [ 36.915696] nix-grpc-daemon[770]: ts=2026-09-20T09:05:35Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.4:50051 clients=14520worker2 # [ 36.924274] nix-daemon[844]: reaped child process 917, status = succeeded4521worker2 # [ 36.956751] nix-grpc-daemon[770]: ts=2026-09-20T09:05:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374864522worker2 # [ 36.969308] nix-grpc-daemon[770]: ts=2026-09-20T09:05:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374864523worker2 # [ 36.982613] nix-grpc-daemon[857]: time=2026-09-20T09:05:35.195Z level=INFO msg="All 2 paths already cached"4524client # copying 1 paths to 'lb:50051'...4525worker2 # [ 36.991494] nix-grpc-daemon[770]: ts=2026-09-20T09:05:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:374864526client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4527worker2 # [ 37.005169] nix-daemon[844]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4528worker2 # [ 37.180709] nix-daemon[844]: accepted connection from pid 943, user nix-grpc-daemon (trusted)4529worker2 # [ 37.193414] nix-daemon[844]: reaped child process 946, status = succeeded4530worker2 # [ 37.356075] nix-daemon[844]: accepted connection from pid 948, user nix-grpc-daemon (trusted)4531worker2 # [ 37.368081] nix-daemon[844]: reaped child process 951, status = succeeded4532lb # [ 37.395708] niks3-server[1047]: 2026/09/20 09:05:35 INFO Received uploads request method=POST path=/api/pending_closures4533worker2 # [ 37.393944] nix-grpc-daemon[857]: time=2026-09-20T09:05:35.607Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4534worker2 # [ 37.395435] nix-grpc-daemon[857]: time=2026-09-20T09:05:35.608Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4535lb # [ 37.422527] niks3-server[1047]: 2026/09/20 09:05:35 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4536lb # [ 37.434764] niks3-server[1047]: 2026/09/20 09:05:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4537worker2 # [ 37.429597] nix-grpc-daemon[857]: time=2026-09-20T09:05:35.642Z level=INFO msg="Uploading 1 narinfos"4538lb # [ 37.439993] niks3-server[1047]: 2026/09/20 09:05:35 INFO Signed narinfos id=5 count=14539lb # [ 37.446123] niks3-server[1047]: 2026/09/20 09:05:35 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4540lb # [ 37.457374] niks3-server[1047]: 2026/09/20 09:05:35 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4541lb # [ 37.462389] niks3-server[1047]: 2026/09/20 09:05:35 INFO Completed upload id=54542worker2 # [ 37.454532] nix-grpc-daemon[857]: time=2026-09-20T09:05:35.667Z level=INFO msg="Upload complete. (437ms)"4543worker2 # [ 37.456559] nix-grpc-daemon[770]: ts=2026-09-20T09:05:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37486 duration_s=0 paths=1 nar_bytes_in=7524544lb # [ 37.466320] niks3-server[1047]: 2026/09/20 09:05:35 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4545worker2 # [ 37.462326] nix-grpc-daemon[770]: ts=2026-09-20T09:05:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374864546worker2 # [ 37.467126] nix-daemon[844]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4547worker2 # [ 37.469217] nix-daemon[844]: reaped child process 941, status = succeeded4548worker2 # [ 37.474746] nix-daemon[844]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4549client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4550client # farm-top> worker2: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4551client # farm-top> LOG-farm-top-t14552worker2 # [ 37.899711] nix-daemon[844]: reaped child process 955, status = succeeded4553worker2 # [ 38.059770] nix-daemon[844]: accepted connection from pid 970, user nix-grpc-daemon (trusted)4554worker2 # [ 38.073253] nix-daemon[844]: reaped child process 973, status = succeeded4555lb # [ 38.100507] niks3-server[1047]: 2026/09/20 09:05:35 INFO Received uploads request method=POST path=/api/pending_closures4556worker2 # [ 38.099040] nix-grpc-daemon[857]: time=2026-09-20T09:05:36.312Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4557worker2 # [ 38.100582] nix-grpc-daemon[857]: time=2026-09-20T09:05:36.313Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4558lb # [ 38.136250] niks3-server[1047]: 2026/09/20 09:05:35 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4559lb # [ 38.144122] niks3-server[1047]: 2026/09/20 09:05:35 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4560lb # [ 38.156347] niks3-server[1047]: 2026/09/20 09:05:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4561lb # [ 38.161689] niks3-server[1047]: 2026/09/20 09:05:35 INFO Signed narinfos id=6 count=14562worker2 # [ 38.154541] nix-grpc-daemon[857]: time=2026-09-20T09:05:36.367Z level=INFO msg="Uploading 1 narinfos"4563lb # [ 38.166434] niks3-server[1047]: 2026/09/20 09:05:35 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4564lb # [ 38.175987] niks3-server[1047]: 2026/09/20 09:05:35 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4565lb # [ 38.181036] niks3-server[1047]: 2026/09/20 09:05:35 INFO Completed upload id=64566worker2 # [ 38.171361] nix-grpc-daemon[857]: time=2026-09-20T09:05:36.384Z level=INFO msg="Upload complete. (290ms)"4567lb # [ 38.183337] niks3-server[1047]: 2026/09/20 09:05:35 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4568worker2 # [ 38.178403] nix-grpc-daemon[770]: ts=2026-09-20T09:05:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37486 duration_s=0 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14569worker2 # [ 38.190177] nix-daemon[844]: reaped child process 953, status = succeeded4570client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 5.68 seconds)4571client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4572client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.18 seconds)4573client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4574client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.02 seconds)4575client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4576client # copying 3 paths...4577client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4578client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4579client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4580client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.24 seconds)4581(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 6.19 seconds)4582subtest: repeat is answered Cached by the scheduler without building4583worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4584worker1 # finding garbage collector roots...4585worker1 # deleting specified paths...4586worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4587worker1 # deleting unused links...4588worker1 # note: hard linking is currently saving 0.0 KiB4589worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.28 seconds)4590worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4591worker2 # finding garbage collector roots...4592worker2 # deleting specified paths...4593worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4594worker2 # deleting unused links...4595worker2 # note: hard linking is currently saving 0.0 KiB4596worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.25 seconds)4597client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24598worker2 # [ 39.490606] nix-grpc-daemon[770]: ts=2026-09-20T09:05:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:374864599client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4600client # /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4601worker2 # [ 39.509059] nix-grpc-daemon[770]: ts=2026-09-20T09:05:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374864602worker2 # [ 39.518779] nix-grpc-daemon[857]: time=2026-09-20T09:05:37.731Z level=INFO msg="All 2 paths already cached"4603worker2 # [ 39.532848] nix-grpc-daemon[770]: ts=2026-09-20T09:05:37Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14604client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 0.27 seconds)4605client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4606client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.17 seconds)4607lb: must succeed: curl -sf localhost:9901/clusters4608lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4609lb: must succeed: curl -sf localhost:9901/clusters4610lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4611worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="cached"} ' || true4612worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="cached"} ' || true, in 0.06 seconds)4613(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.21 seconds)4614subtest: inputs already in the cache are not sent to the scheduler4615lb: must succeed: curl -sf localhost:9901/clusters4616lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4617lb: must succeed: curl -sf localhost:9901/clusters4618lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4619worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4620worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4621client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&24622worker2 # [ 40.330799] nix-grpc-daemon[770]: ts=2026-09-20T09:05:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:374864623client # don't know how to build these paths:4624client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4625worker2 # [ 40.339147] nix-grpc-daemon[770]: ts=2026-09-20T09:05:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374864626worker2 # [ 40.347645] nix-grpc-daemon[857]: time=2026-09-20T09:05:38.560Z level=INFO msg="All 2 paths already cached"4627worker2 # [ 40.359248] nix-grpc-daemon[770]: ts=2026-09-20T09:05:38Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04628worker2 # [ 40.364470] nix-grpc-daemon[770]: ts=2026-09-20T09:05:38Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.4:50051 clients=14629worker2 # [ 40.403844] nix-grpc-daemon[770]: ts=2026-09-20T09:05:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374864630worker2 # [ 40.416878] nix-grpc-daemon[770]: ts=2026-09-20T09:05:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374864631worker2 # [ 40.428861] nix-grpc-daemon[857]: time=2026-09-20T09:05:38.642Z level=INFO msg="All 2 paths already cached"4632client # copying 1 paths to 'lb:50051'...4633client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4634worker2 # [ 40.436739] nix-grpc-daemon[770]: ts=2026-09-20T09:05:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:374864635worker2 # [ 40.448332] nix-daemon[844]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4636worker2 # [ 40.624040] nix-daemon[844]: accepted connection from pid 1005, user nix-grpc-daemon (trusted)4637worker2 # [ 40.637729] nix-daemon[844]: reaped child process 1008, status = succeeded4638worker2 # [ 40.780223] nix-daemon[844]: accepted connection from pid 1010, user nix-grpc-daemon (trusted)4639worker2 # [ 40.792262] nix-daemon[844]: reaped child process 1013, status = succeeded4640lb # [ 40.817639] niks3-server[1047]: 2026/09/20 09:05:38 INFO Received uploads request method=POST path=/api/pending_closures4641worker2 # [ 40.815118] nix-grpc-daemon[857]: time=2026-09-20T09:05:39.027Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4642worker2 # [ 40.816583] nix-grpc-daemon[857]: time=2026-09-20T09:05:39.027Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4643lb # [ 40.841196] niks3-server[1047]: 2026/09/20 09:05:38 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4644lb # [ 40.858176] niks3-server[1047]: 2026/09/20 09:05:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4645worker2 # [ 40.854282] nix-grpc-daemon[857]: time=2026-09-20T09:05:39.067Z level=INFO msg="Uploading 1 narinfos"4646lb # [ 40.864689] niks3-server[1047]: 2026/09/20 09:05:38 INFO Signed narinfos id=7 count=14647lb # [ 40.869764] niks3-server[1047]: 2026/09/20 09:05:38 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4648lb # [ 40.879279] niks3-server[1047]: 2026/09/20 09:05:38 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4649worker2 # [ 40.875997] nix-grpc-daemon[857]: time=2026-09-20T09:05:39.089Z level=INFO msg="Upload complete. (415ms)"4650lb # [ 40.886450] niks3-server[1047]: 2026/09/20 09:05:38 INFO Completed upload id=74651worker2 # [ 40.878306] nix-grpc-daemon[770]: ts=2026-09-20T09:05:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37486 duration_s=0 paths=1 nar_bytes_in=7524652worker2 # [ 40.881264] nix-grpc-daemon[770]: ts=2026-09-20T09:05:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374864653lb # [ 40.891187] niks3-server[1047]: 2026/09/20 09:05:38 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4654worker2 # [ 40.888495] nix-daemon[844]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4655worker2 # [ 40.890953] nix-daemon[844]: reaped child process 1003, status = succeeded4656worker2 # [ 40.896067] nix-daemon[844]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4657client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4658client # farm-top> worker2: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4659client # farm-top> LOG-farm-top-t1b4660worker2 # [ 41.709948] nix-daemon[844]: reaped child process 1017, status = succeeded4661worker2 # [ 41.857544] nix-daemon[844]: accepted connection from pid 1032, user nix-grpc-daemon (trusted)4662worker2 # [ 41.869843] nix-daemon[844]: reaped child process 1035, status = succeeded4663lb # [ 41.896955] niks3-server[1047]: 2026/09/20 09:05:39 INFO Received uploads request method=POST path=/api/pending_closures4664worker2 # [ 41.891698] nix-grpc-daemon[857]: time=2026-09-20T09:05:40.104Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4665worker2 # [ 41.893288] nix-grpc-daemon[857]: time=2026-09-20T09:05:40.106Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4666lb # [ 41.925643] niks3-server[1047]: 2026/09/20 09:05:39 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4667lb # [ 41.938658] niks3-server[1047]: 2026/09/20 09:05:39 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4668lb # [ 41.949243] niks3-server[1047]: 2026/09/20 09:05:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4669lb # [ 41.955082] niks3-server[1047]: 2026/09/20 09:05:39 INFO Signed narinfos id=8 count=14670worker2 # [ 41.948309] nix-grpc-daemon[857]: time=2026-09-20T09:05:40.161Z level=INFO msg="Uploading 1 narinfos"4671lb # [ 41.958367] niks3-server[1047]: 2026/09/20 09:05:39 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4672lb # [ 41.969922] niks3-server[1047]: 2026/09/20 09:05:39 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4673lb # [ 41.976585] niks3-server[1047]: 2026/09/20 09:05:39 INFO Completed upload id=84674worker2 # [ 41.966660] nix-grpc-daemon[857]: time=2026-09-20T09:05:40.179Z level=INFO msg="Upload complete. (263ms)"4675lb # [ 41.978094] niks3-server[1047]: 2026/09/20 09:05:39 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4676worker2 # [ 41.970278] nix-grpc-daemon[770]: ts=2026-09-20T09:05:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37486 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14677worker2 # [ 41.977047] nix-daemon[844]: reaped child process 1015, status = succeeded4678client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&2, in 1.87 seconds)4679client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath4680client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.14 seconds)4681lb: must succeed: curl -sf localhost:9901/clusters4682lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4683lb: must succeed: curl -sf localhost:9901/clusters4684lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4685worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4686worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4687(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.31 seconds)4688subtest: two clients wanting the same drv build it once4689client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4690client # [ 42.372259] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4691client # Running as unit: dedup1.service; invocation ID: 7582a37e2f7f42428263f3cb33d2b6324692client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 0.04 seconds)4693client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4694worker2 # [ 42.506901] nix-grpc-daemon[770]: ts=2026-09-20T09:05:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:374864695client # [ 42.565443] nix[893]: don't know how to build these paths:4696client # [ 42.566362] nix[893]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4697worker2 # [ 42.522195] nix-grpc-daemon[770]: ts=2026-09-20T09:05:40Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04698worker2 # [ 42.526308] nix-grpc-daemon[770]: ts=2026-09-20T09:05:40Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.3:50051 clients=14699worker1 # [ 42.590445] nix-grpc-daemon[784]: ts=2026-09-20T09:05:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334584700worker1 # [ 42.602278] nix-grpc-daemon[784]: ts=2026-09-20T09:05:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334584701client # [ 42.604736] nix[893]: copying 1 paths to 'lb:50051'...4702client # [ 42.605540] nix[893]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4703worker1 # [ 42.609504] nix-grpc-daemon[784]: ts=2026-09-20T09:05:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334584704worker1 # [ 42.615291] nix-daemon[842]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4705worker1 # [ 42.629376] nix-grpc-daemon[784]: ts=2026-09-20T09:05:39Z level=info event=niks3_push_started pid=8684706worker1 # [ 43.120735] nix-daemon[842]: accepted connection from pid 881, user nix-grpc-daemon (trusted)4707worker1 # [ 43.135645] nix-daemon[842]: reaped child process 884, status = succeeded4708worker1 # [ 43.317301] nix-daemon[842]: accepted connection from pid 886, user nix-grpc-daemon (trusted)4709worker1 # [ 43.329422] nix-daemon[842]: reaped child process 889, status = succeeded4710lb # [ 43.300902] niks3-server[1047]: 2026/09/20 09:05:41 INFO Received uploads request method=POST path=/api/pending_closures4711worker1 # [ 43.358688] nix-grpc-daemon[868]: time=2026-09-20T09:05:40.616Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4712worker1 # [ 43.362577] nix-grpc-daemon[868]: time=2026-09-20T09:05:40.620Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4713lb # [ 43.342584] niks3-server[1047]: 2026/09/20 09:05:41 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4714lb # [ 43.349438] niks3-server[1047]: 2026/09/20 09:05:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4715lb # [ 43.354836] niks3-server[1047]: 2026/09/20 09:05:41 INFO Signed narinfos id=9 count=14716worker1 # [ 43.403699] nix-grpc-daemon[868]: time=2026-09-20T09:05:40.662Z level=INFO msg="Uploading 1 narinfos"4717lb # [ 43.363284] niks3-server[1047]: 2026/09/20 09:05:41 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4718lb # [ 43.373233] niks3-server[1047]: 2026/09/20 09:05:41 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4719worker1 # [ 43.425837] nix-grpc-daemon[868]: time=2026-09-20T09:05:40.685Z level=INFO msg="Upload complete. (501ms)"4720lb # [ 43.379569] niks3-server[1047]: 2026/09/20 09:05:41 INFO Completed upload id=94721worker1 # [ 43.427899] nix-grpc-daemon[784]: ts=2026-09-20T09:05:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33458 duration_s=0 paths=1 nar_bytes_in=4324722lb # [ 43.382819] niks3-server[1047]: 2026/09/20 09:05:41 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4723worker1 # [ 43.434340] nix-grpc-daemon[784]: ts=2026-09-20T09:05:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334584724worker1 # [ 43.435973] nix-daemon[842]: reaped child process 866, status = succeeded4725worker1 # [ 43.438957] nix-daemon[842]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4726worker1 # [ 43.445514] nix-daemon[842]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4727client # [ 43.436358] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4728client # [ 43.445921] nix[893]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4729client # Running as unit: dedup2.service; invocation ID: 18add469ba8d47ca995555186689ee6f4730client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 1.08 seconds)4731worker2 # [ 43.627815] nix-grpc-daemon[770]: ts=2026-09-20T09:05:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:374864732client # [ 43.686454] nix[909]: don't know how to build these paths:4733client # [ 43.687414] nix[909]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4734worker2 # [ 43.641496] nix-grpc-daemon[770]: ts=2026-09-20T09:05:41Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04735worker1 # [ 43.703653] nix-grpc-daemon[784]: ts=2026-09-20T09:05:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334584736client # [ 43.704632] nix[909]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4737client: must succeed: sleep 24738lb # [ 46.641342] envoy[918]: conn peer=[2001:db8:1::1]:33218 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9481 tx=3797 ms=54914739lb # [ 46.645141] envoy[918]: conn peer=[2001:db8:1::1]:33232 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=604740lb # [ 46.648498] envoy[918]: conn peer=[2001:db8:1::1]:33242 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4126 tx=1713 ms=16534741client: (finished: must succeed: sleep 2, in 2.05 seconds)4742client: waiting for success: ! systemctl is-active dedup1 dedup24743worker1 # [ 48.075754] nix-daemon[842]: reaped child process 893, status = succeeded4744worker1 # [ 48.252929] nix-daemon[842]: accepted connection from pid 926, user nix-grpc-daemon (trusted)4745worker1 # [ 48.263484] nix-daemon[842]: reaped child process 929, status = succeeded4746lb # [ 48.235790] niks3-server[1047]: 2026/09/20 09:05:45 INFO Received uploads request method=POST path=/api/pending_closures4747worker1 # [ 48.290382] nix-grpc-daemon[868]: time=2026-09-20T09:05:45.549Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4748worker1 # [ 48.291983] nix-grpc-daemon[868]: time=2026-09-20T09:05:45.551Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4749lb # [ 48.282442] niks3-server[1047]: 2026/09/20 09:05:45 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4750lb # [ 48.299734] niks3-server[1047]: 2026/09/20 09:05:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4751lb # [ 48.310183] niks3-server[1047]: 2026/09/20 09:05:46 INFO Signed narinfos id=10 count=14752worker1 # [ 48.359870] nix-grpc-daemon[868]: time=2026-09-20T09:05:45.618Z level=INFO msg="Uploading 1 narinfos"4753lb # [ 48.315274] niks3-server[1047]: 2026/09/20 09:05:46 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4754lb # [ 48.318554] niks3-server[1047]: 2026/09/20 09:05:46 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4755lb # [ 48.328447] niks3-server[1047]: 2026/09/20 09:05:46 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4756lb # [ 48.331497] niks3-server[1047]: 2026/09/20 09:05:46 INFO Completed upload id=104757lb # [ 48.333926] niks3-server[1047]: 2026/09/20 09:05:46 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4758worker1 # [ 48.378987] nix-grpc-daemon[868]: time=2026-09-20T09:05:45.638Z level=INFO msg="Upload complete. (317ms)"4759worker1 # [ 48.385536] nix-grpc-daemon[784]: ts=2026-09-20T09:05:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33458 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14760worker1 # [ 48.392317] nix-grpc-daemon[784]: ts=2026-09-20T09:05:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33458 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14761worker1 # [ 48.403546] nix-daemon[842]: reaped child process 891, status = succeeded4762client # [ 48.421390] systemd[1]: dedup1.service: Deactivated successfully.4763client # [ 48.424177] systemd[1]: dedup1.service: Consumed 108ms CPU time over 6.049s wall clock time, 39.3M memory peak, 8.6K incoming IP traffic, 9.1K outgoing IP traffic.4764client # [ 48.457254] systemd[1]: dedup2.service: Deactivated successfully.4765client # [ 48.458635] systemd[1]: dedup2.service: Consumed 121ms CPU time over 5.022s wall clock time, 31.2M memory peak, 7.6K incoming IP traffic, 6.4K outgoing IP traffic.4766client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.10 seconds)4767client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4768client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)4769worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4770worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.06 seconds)4771worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4772worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.03 seconds)4773worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="attached"} ' || true4774worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="attached"} ' || true, in 0.08 seconds)4775worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="attached"} ' || true4776worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="attached"} ' || true, in 0.05 seconds)4777(finished: subtest: two clients wanting the same drv build it once, in 7.03 seconds)4778subtest: build hook: nix-daemon with builders = grpc://lb4779client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log4780client # [ 49.554928] systemd[1]: Started Nix Daemon.4781client # [ 49.688190] nix-daemon[951]: accepted connection from pid 948, user root (trusted)4782worker1 # [ 49.838774] nix-grpc-daemon[784]: ts=2026-09-20T09:05:47Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:334584783worker2 # [ 49.788872] nix-grpc-daemon[770]: ts=2026-09-20T09:05:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374864784worker2 # [ 49.804632] nix-grpc-daemon[770]: ts=2026-09-20T09:05:48Z level=debug event=want client=6 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=04785worker2 # [ 49.811520] nix-grpc-daemon[770]: ts=2026-09-20T09:05:48Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.4:50051 clients=14786worker2 # [ 49.816256] nix-grpc-daemon[770]: ts=2026-09-20T09:05:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374864787worker2 # [ 49.823571] nix-grpc-daemon[770]: ts=2026-09-20T09:05:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374864788worker2 # [ 49.834909] nix-grpc-daemon[770]: ts=2026-09-20T09:05:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:374864789worker2 # [ 49.845602] nix-daemon[844]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4790worker2 # [ 50.004549] nix-daemon[844]: accepted connection from pid 1078, user nix-grpc-daemon (trusted)4791worker2 # [ 50.018781] nix-daemon[844]: reaped child process 1081, status = succeeded4792worker2 # [ 50.167088] nix-daemon[844]: accepted connection from pid 1083, user nix-grpc-daemon (trusted)4793worker2 # [ 50.178976] nix-daemon[844]: reaped child process 1086, status = succeeded4794lb # [ 50.208666] niks3-server[1047]: 2026/09/20 09:05:47 INFO Received uploads request method=POST path=/api/pending_closures4795worker2 # [ 50.206742] nix-grpc-daemon[857]: time=2026-09-20T09:05:48.419Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4796worker2 # [ 50.208631] nix-grpc-daemon[857]: time=2026-09-20T09:05:48.421Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"4797lb # [ 50.235604] niks3-server[1047]: 2026/09/20 09:05:47 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst4798lb # [ 50.245245] niks3-server[1047]: 2026/09/20 09:05:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4799lb # [ 50.249423] niks3-server[1047]: 2026/09/20 09:05:47 INFO Signed narinfos id=11 count=14800worker2 # [ 50.245193] nix-grpc-daemon[857]: time=2026-09-20T09:05:48.458Z level=INFO msg="Uploading 1 narinfos"4801lb # [ 50.259283] niks3-server[1047]: 2026/09/20 09:05:47 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls4802lb # [ 50.269653] niks3-server[1047]: 2026/09/20 09:05:47 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4803worker2 # [ 50.265286] nix-grpc-daemon[857]: time=2026-09-20T09:05:48.478Z level=INFO msg="Upload complete. (408ms)"4804lb # [ 50.275344] niks3-server[1047]: 2026/09/20 09:05:47 INFO Completed upload id=114805worker2 # [ 50.267268] nix-grpc-daemon[770]: ts=2026-09-20T09:05:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37486 duration_s=0 paths=1 nar_bytes_in=5124806lb # [ 50.278591] niks3-server[1047]: 2026/09/20 09:05:47 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo4807worker2 # [ 50.273410] nix-grpc-daemon[770]: ts=2026-09-20T09:05:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374864808worker2 # [ 50.278120] nix-daemon[844]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4809worker2 # [ 50.280237] nix-daemon[844]: reaped child process 1076, status = succeeded4810worker2 # [ 50.285543] nix-daemon[844]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4811worker2 # [ 51.311659] nix-daemon[844]: reaped child process 1090, status = succeeded4812worker2 # [ 51.449675] nix-daemon[844]: accepted connection from pid 1105, user nix-grpc-daemon (trusted)4813worker2 # [ 51.460654] nix-daemon[844]: reaped child process 1108, status = succeeded4814lb # [ 51.483732] niks3-server[1047]: 2026/09/20 09:05:49 INFO Received uploads request method=POST path=/api/pending_closures4815worker2 # [ 51.481907] nix-grpc-daemon[857]: time=2026-09-20T09:05:49.695Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4816worker2 # [ 51.483485] nix-grpc-daemon[857]: time=2026-09-20T09:05:49.696Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"4817lb # [ 51.520329] niks3-server[1047]: 2026/09/20 09:05:49 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv4818lb # [ 51.533854] niks3-server[1047]: 2026/09/20 09:05:49 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4819lb # [ 51.542929] niks3-server[1047]: 2026/09/20 09:05:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4820lb # [ 51.548577] niks3-server[1047]: 2026/09/20 09:05:49 INFO Signed narinfos id=12 count=14821worker2 # [ 51.541636] nix-grpc-daemon[857]: time=2026-09-20T09:05:49.754Z level=INFO msg="Uploading 1 narinfos"4822lb # [ 51.551661] niks3-server[1047]: 2026/09/20 09:05:49 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls4823lb # [ 51.562886] niks3-server[1047]: 2026/09/20 09:05:49 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4824lb # [ 51.568961] niks3-server[1047]: 2026/09/20 09:05:49 INFO Completed upload id=124825lb # [ 51.570596] niks3-server[1047]: 2026/09/20 09:05:49 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo4826worker2 # [ 51.559197] nix-grpc-daemon[857]: time=2026-09-20T09:05:49.772Z level=INFO msg="Upload complete. (266ms)"4827worker1 # [ 51.625929] nix-grpc-daemon[784]: ts=2026-09-20T09:05:48Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:334584828worker2 # [ 51.566825] nix-grpc-daemon[770]: ts=2026-09-20T09:05:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37486 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=6 outputs=14829worker1 # [ 51.635806] nix-grpc-daemon[784]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...4830worker2 # [ 51.580094] nix-daemon[844]: reaped child process 1088, status = succeeded4831worker1 # [ 51.665495] nix-grpc-daemon[784]: ts=2026-09-20T09:05:48Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:334584832worker1 # [ 51.671129] nix-grpc-daemon[784]: ts=2026-09-20T09:05:48Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:33458 duration_s=0 paths=1 nar_bytes_out=1284833worker1 # [ 51.928870] nix-grpc-daemon[784]: ts=2026-09-20T09:05:49Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:334584834worker2 # [ 51.875678] nix-grpc-daemon[770]: ts=2026-09-20T09:05:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374864835worker2 # [ 51.884742] nix-grpc-daemon[857]: time=2026-09-20T09:05:50.097Z level=INFO msg="All 1 paths already cached"4836worker2 # [ 51.900908] nix-grpc-daemon[770]: ts=2026-09-20T09:05:50Z level=debug event=want client=7 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=04837worker2 # [ 51.906072] nix-grpc-daemon[770]: ts=2026-09-20T09:05:50Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.4:50051 clients=14838worker2 # [ 51.912677] nix-grpc-daemon[770]: ts=2026-09-20T09:05:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374864839worker2 # [ 51.924742] nix-grpc-daemon[770]: ts=2026-09-20T09:05:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374864840worker2 # [ 51.937952] nix-grpc-daemon[857]: time=2026-09-20T09:05:50.151Z level=INFO msg="All 1 paths already cached"4841worker2 # [ 51.945262] nix-grpc-daemon[770]: ts=2026-09-20T09:05:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:374864842worker2 # [ 51.954324] nix-daemon[844]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4843worker2 # [ 52.132677] nix-daemon[844]: accepted connection from pid 1114, user nix-grpc-daemon (trusted)4844worker2 # [ 52.146110] nix-daemon[844]: reaped child process 1117, status = succeeded4845worker2 # [ 52.291538] nix-daemon[844]: accepted connection from pid 1119, user nix-grpc-daemon (trusted)4846worker2 # [ 52.303709] nix-daemon[844]: reaped child process 1122, status = succeeded4847lb # [ 52.331104] niks3-server[1047]: 2026/09/20 09:05:50 INFO Received uploads request method=POST path=/api/pending_closures4848worker2 # [ 52.329733] nix-grpc-daemon[857]: time=2026-09-20T09:05:50.543Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4849worker2 # [ 52.331614] nix-grpc-daemon[857]: time=2026-09-20T09:05:50.544Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"4850lb # [ 52.357591] niks3-server[1047]: 2026/09/20 09:05:50 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst4851lb # [ 52.368982] niks3-server[1047]: 2026/09/20 09:05:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4852lb # [ 52.374077] niks3-server[1047]: 2026/09/20 09:05:50 INFO Signed narinfos id=13 count=14853worker2 # [ 52.367091] nix-grpc-daemon[857]: time=2026-09-20T09:05:50.580Z level=INFO msg="Uploading 1 narinfos"4854lb # [ 52.379697] niks3-server[1047]: 2026/09/20 09:05:50 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls4855lb # [ 52.391775] niks3-server[1047]: 2026/09/20 09:05:50 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4856lb # [ 52.395713] niks3-server[1047]: 2026/09/20 09:05:50 INFO Completed upload id=134857worker2 # [ 52.387945] nix-grpc-daemon[857]: time=2026-09-20T09:05:50.601Z level=INFO msg="Upload complete. (420ms)"4858worker2 # [ 52.389931] nix-grpc-daemon[770]: ts=2026-09-20T09:05:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37486 duration_s=0 paths=1 nar_bytes_in=6244859lb # [ 52.399930] niks3-server[1047]: 2026/09/20 09:05:50 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo4860worker2 # [ 52.397087] nix-daemon[844]: reaped child process 1112, status = succeeded4861worker2 # [ 52.397987] nix-grpc-daemon[770]: ts=2026-09-20T09:05:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374864862worker2 # [ 52.402189] nix-daemon[844]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4863worker2 # [ 52.407895] nix-daemon[844]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4864worker2 # [ 53.432219] nix-daemon[844]: reaped child process 1126, status = succeeded4865worker2 # [ 53.593824] nix-daemon[844]: accepted connection from pid 1141, user nix-grpc-daemon (trusted)4866worker2 # [ 53.605068] nix-daemon[844]: reaped child process 1144, status = succeeded4867lb # [ 53.629270] niks3-server[1047]: 2026/09/20 09:05:51 INFO Received uploads request method=POST path=/api/pending_closures4868worker2 # [ 53.627105] nix-grpc-daemon[857]: time=2026-09-20T09:05:51.840Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4869worker2 # [ 53.628564] nix-grpc-daemon[857]: time=2026-09-20T09:05:51.841Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"4870lb # [ 53.668345] niks3-server[1047]: 2026/09/20 09:05:51 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv4871lb # [ 53.678205] niks3-server[1047]: 2026/09/20 09:05:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4872lb # [ 53.683501] niks3-server[1047]: 2026/09/20 09:05:51 INFO Signed narinfos id=14 count=14873worker2 # [ 53.676850] nix-grpc-daemon[857]: time=2026-09-20T09:05:51.889Z level=INFO msg="Uploading 1 narinfos"4874lb # [ 53.687834] niks3-server[1047]: 2026/09/20 09:05:51 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst4875lb # [ 53.694947] niks3-server[1047]: 2026/09/20 09:05:51 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls4876lb # [ 53.702872] niks3-server[1047]: 2026/09/20 09:05:51 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4877lb # [ 53.707872] niks3-server[1047]: 2026/09/20 09:05:51 INFO Completed upload id=144878worker2 # [ 53.698247] nix-grpc-daemon[857]: time=2026-09-20T09:05:51.911Z level=INFO msg="Upload complete. (279ms)"4879lb # [ 53.709657] niks3-server[1047]: 2026/09/20 09:05:51 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo4880worker2 # [ 53.705349] nix-grpc-daemon[770]: ts=2026-09-20T09:05:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37486 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=7 outputs=14881worker2 # [ 53.712466] nix-grpc-daemon[770]: ts=2026-09-20T09:05:51Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:374864882worker2 # [ 53.721396] nix-daemon[844]: reaped child process 1124, status = succeeded4883worker1 # [ 53.782473] nix-grpc-daemon[784]: ts=2026-09-20T09:05:51Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:334584884worker1 # [ 53.791730] nix-grpc-daemon[784]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...4885worker1 # [ 53.802518] nix-grpc-daemon[784]: ts=2026-09-20T09:05:51Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:33458 duration_s=0 paths=1 nar_bytes_out=1764886worker2 # [ 54.008923] nix-grpc-daemon[770]: ts=2026-09-20T09:05:52Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:374864887worker1 # [ 54.068747] nix-grpc-daemon[784]: ts=2026-09-20T09:05:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334584888worker1 # [ 54.078114] nix-grpc-daemon[868]: time=2026-09-20T09:05:51.336Z level=INFO msg="All 2 paths already cached"4889worker2 # [ 54.036476] nix-grpc-daemon[770]: ts=2026-09-20T09:05:52Z level=debug event=want client=8 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=04890worker2 # [ 54.042432] nix-grpc-daemon[770]: ts=2026-09-20T09:05:52Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.4:50051 clients=14891worker2 # [ 54.047376] nix-grpc-daemon[770]: ts=2026-09-20T09:05:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374864892worker2 # [ 54.059146] nix-grpc-daemon[770]: ts=2026-09-20T09:05:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374864893worker2 # [ 54.069395] nix-grpc-daemon[857]: time=2026-09-20T09:05:52.282Z level=INFO msg="All 2 paths already cached"4894worker2 # [ 54.076168] nix-grpc-daemon[770]: ts=2026-09-20T09:05:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:374864895worker2 # [ 54.084391] nix-daemon[844]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4896worker2 # [ 54.259449] nix-daemon[844]: accepted connection from pid 1150, user nix-grpc-daemon (trusted)4897worker2 # [ 54.271901] nix-daemon[844]: reaped child process 1153, status = succeeded4898worker2 # [ 54.437770] nix-daemon[844]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)4899worker2 # [ 54.449612] nix-daemon[844]: reaped child process 1158, status = succeeded4900lb # [ 54.477704] niks3-server[1047]: 2026/09/20 09:05:52 INFO Received uploads request method=POST path=/api/pending_closures4901worker2 # [ 54.475612] nix-grpc-daemon[857]: time=2026-09-20T09:05:52.688Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4902worker2 # [ 54.477287] nix-grpc-daemon[857]: time=2026-09-20T09:05:52.690Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"4903lb # [ 54.510090] niks3-server[1047]: 2026/09/20 09:05:52 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst4904lb # [ 54.517266] niks3-server[1047]: 2026/09/20 09:05:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4905lb # [ 54.522211] niks3-server[1047]: 2026/09/20 09:05:52 INFO Signed narinfos id=15 count=14906worker2 # [ 54.515162] nix-grpc-daemon[857]: time=2026-09-20T09:05:52.728Z level=INFO msg="Uploading 1 narinfos"4907lb # [ 54.528603] niks3-server[1047]: 2026/09/20 09:05:52 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls4908lb # [ 54.539077] niks3-server[1047]: 2026/09/20 09:05:52 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4909lb # [ 54.545457] niks3-server[1047]: 2026/09/20 09:05:52 INFO Completed upload id=154910worker2 # [ 54.535688] nix-grpc-daemon[857]: time=2026-09-20T09:05:52.748Z level=INFO msg="Upload complete. (438ms)"4911lb # [ 54.549112] niks3-server[1047]: 2026/09/20 09:05:52 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo4912worker2 # [ 54.540276] nix-grpc-daemon[770]: ts=2026-09-20T09:05:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37486 duration_s=0 paths=1 nar_bytes_in=7524913worker2 # [ 54.546287] nix-grpc-daemon[770]: ts=2026-09-20T09:05:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374864914worker2 # [ 54.554886] nix-daemon[844]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4915worker2 # [ 54.561695] nix-daemon[844]: reaped child process 1148, status = succeeded4916worker2 # [ 54.565748] nix-daemon[844]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4917worker2 # [ 55.189202] nix-daemon[844]: reaped child process 1162, status = succeeded4918worker2 # [ 55.340073] nix-daemon[844]: accepted connection from pid 1177, user nix-grpc-daemon (trusted)4919worker2 # [ 55.350950] nix-daemon[844]: reaped child process 1180, status = succeeded4920lb # [ 55.377949] niks3-server[1047]: 2026/09/20 09:05:53 INFO Received uploads request method=POST path=/api/pending_closures4921worker2 # [ 55.372960] nix-grpc-daemon[857]: time=2026-09-20T09:05:53.586Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4922worker2 # [ 55.374495] nix-grpc-daemon[857]: time=2026-09-20T09:05:53.587Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"4923lb # [ 55.401793] niks3-server[1047]: 2026/09/20 09:05:53 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst4924lb # [ 55.404557] niks3-server[1047]: 2026/09/20 09:05:53 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv4925lb # [ 55.407974] niks3-server[1047]: 2026/09/20 09:05:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4926lb # [ 55.410371] niks3-server[1047]: 2026/09/20 09:05:53 INFO Signed narinfos id=16 count=14927worker2 # [ 55.400618] nix-grpc-daemon[857]: time=2026-09-20T09:05:53.613Z level=INFO msg="Uploading 1 narinfos"4928lb # [ 55.414168] niks3-server[1047]: 2026/09/20 09:05:53 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls4929lb # [ 55.418231] niks3-server[1047]: 2026/09/20 09:05:53 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4930lb # [ 55.421830] niks3-server[1047]: 2026/09/20 09:05:53 INFO Completed upload id=164931worker2 # [ 55.411912] nix-grpc-daemon[857]: time=2026-09-20T09:05:53.625Z level=INFO msg="Upload complete. (240ms)"4932lb # [ 55.424639] niks3-server[1047]: 2026/09/20 09:05:53 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo4933worker2 # [ 55.414840] nix-grpc-daemon[770]: ts=2026-09-20T09:05:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37486 duration_s=0 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=8 outputs=14934worker2 # [ 55.420465] nix-grpc-daemon[770]: ts=2026-09-20T09:05:53Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:374864935worker2 # [ 55.423432] nix-daemon[844]: reaped child process 1160, status = succeeded4936worker1 # [ 55.491171] nix-grpc-daemon[784]: ts=2026-09-20T09:05:52Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:334584937worker1 # [ 55.499728] nix-grpc-daemon[784]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...4938worker1 # [ 55.510335] nix-grpc-daemon[784]: ts=2026-09-20T09:05:52Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:33458 duration_s=0 paths=1 nar_bytes_out=2324939client # [ 55.614495] nix-daemon[951]: reaped child process 953, status = succeeded4940client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.27 seconds)4941client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top4942client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.03 seconds)4943client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4944client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.03 seconds)4945client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4946client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4947(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 6.36 seconds)4948subtest: an input only one worker has still reaches the builder4949client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only4950client # this derivation will be built:4951client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv4952client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...4953client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only, in 0.40 seconds)4954client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure4955client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.17 seconds)4956worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4957worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.21 seconds)4958worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only4959worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.03 seconds)4960client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&24961lb # [ 56.642117] envoy[918]: conn peer=[2001:db8:1::1]:33246 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2972 tx=1366 ms=58424962lb # [ 56.645896] envoy[918]: conn peer=[2001:db8:1::1]:47060 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1427 tx=1108 ms=47674963lb # [ 56.649401] envoy[918]: conn peer=[2001:db8:1::1]:47076 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3428 tx=1751 ms=18854964lb # [ 56.652931] envoy[918]: conn peer=[2001:db8:1::1]:47086 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=634965lb # [ 56.656333] envoy[918]: conn peer=[2001:db8:1::1]:39404 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=379 ms=724966lb # [ 56.659713] envoy[918]: conn peer=[2001:db8:1::1]:47102 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3880 tx=1856 ms=19264967lb # [ 56.663786] envoy[918]: conn peer=[2001:db8:1::1]:39414 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4298 tx=2065 ms=15004968lb # [ 56.667313] envoy[918]: conn peer=[2001:db8:1::1]:39428 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=411 ms=714969client # [ 56.727450] nix-daemon[951]: accepted connection from pid 1034, user root (trusted)4970client # this derivation will be built:4971client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv4972worker1 # [ 56.907798] nix-grpc-daemon[784]: ts=2026-09-20T09:05:54Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:334584973client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux'...4974worker2 # [ 56.858710] nix-grpc-daemon[770]: ts=2026-09-20T09:05:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374864975client # copying 1 paths...4976client # copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' to 'grpc://lb:50051'...4977worker1 # [ 56.924127] nix-grpc-daemon[784]: ts=2026-09-20T09:05:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334584978worker1 # [ 56.934911] nix-daemon[842]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4979worker1 # [ 57.117269] nix-daemon[842]: accepted connection from pid 952, user nix-grpc-daemon (trusted)4980worker1 # [ 57.129707] nix-daemon[842]: reaped child process 955, status = succeeded4981lb # [ 57.101677] niks3-server[1047]: 2026/09/20 09:05:54 INFO Received uploads request method=POST path=/api/pending_closures4982worker1 # [ 57.154492] nix-grpc-daemon[868]: time=2026-09-20T09:05:54.413Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4983worker1 # [ 57.156354] nix-grpc-daemon[868]: time=2026-09-20T09:05:54.415Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"4984lb # [ 57.129629] niks3-server[1047]: 2026/09/20 09:05:54 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4985lb # [ 57.138560] niks3-server[1047]: 2026/09/20 09:05:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4986worker1 # [ 57.190507] nix-grpc-daemon[868]: time=2026-09-20T09:05:54.449Z level=INFO msg="Uploading 1 narinfos"4987lb # [ 57.144376] niks3-server[1047]: 2026/09/20 09:05:54 INFO Signed narinfos id=17 count=14988lb # [ 57.151118] niks3-server[1047]: 2026/09/20 09:05:54 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls4989lb # [ 57.161158] niks3-server[1047]: 2026/09/20 09:05:54 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4990lb # [ 57.167893] niks3-server[1047]: 2026/09/20 09:05:54 INFO Completed upload id=174991worker1 # [ 57.214151] nix-grpc-daemon[868]: time=2026-09-20T09:05:54.473Z level=INFO msg="Upload complete. (269ms)"4992lb # [ 57.171057] niks3-server[1047]: 2026/09/20 09:05:54 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo4993worker1 # [ 57.216868] nix-grpc-daemon[784]: ts=2026-09-20T09:05:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33458 duration_s=0 paths=1 nar_bytes_in=1204994worker1 # [ 57.227497] nix-daemon[842]: reaped child process 950, status = succeeded4995worker2 # [ 57.172499] nix-grpc-daemon[770]: ts=2026-09-20T09:05:55Z level=debug event=want client=9 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=04996worker2 # [ 57.174574] nix-grpc-daemon[770]: ts=2026-09-20T09:05:55Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=14997worker1 # [ 57.273269] nix-grpc-daemon[784]: ts=2026-09-20T09:05:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334584998worker1 # [ 57.285582] nix-grpc-daemon[784]: ts=2026-09-20T09:05:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334584999client # copying 2 paths to 'lb:50051'...5000client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5001worker1 # [ 57.301616] nix-grpc-daemon[784]: ts=2026-09-20T09:05:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334585002client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5003worker1 # [ 57.314378] nix-daemon[842]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5004worker1 # [ 57.497842] nix-daemon[842]: accepted connection from pid 959, user nix-grpc-daemon (trusted)5005worker1 # [ 57.510714] nix-daemon[842]: reaped child process 962, status = succeeded5006worker1 # [ 57.679075] nix-daemon[842]: accepted connection from pid 964, user nix-grpc-daemon (trusted)5007worker1 # [ 57.690720] nix-daemon[842]: reaped child process 967, status = succeeded5008lb # [ 57.662240] niks3-server[1047]: 2026/09/20 09:05:55 INFO Received uploads request method=POST path=/api/pending_closures5009lb # [ 57.671762] niks3-server[1047]: 2026/09/20 09:05:55 INFO Received uploads request method=POST path=/api/pending_closures5010worker1 # [ 57.726249] nix-grpc-daemon[868]: time=2026-09-20T09:05:54.984Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5011worker1 # [ 57.727714] nix-grpc-daemon[868]: time=2026-09-20T09:05:54.984Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5012worker1 # [ 57.729300] nix-grpc-daemon[868]: time=2026-09-20T09:05:54.985Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5013lb # [ 57.706195] niks3-server[1047]: 2026/09/20 09:05:55 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5014lb # [ 57.718393] niks3-server[1047]: 2026/09/20 09:05:55 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5015lb # [ 57.727376] niks3-server[1047]: 2026/09/20 09:05:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5016lb # [ 57.731747] niks3-server[1047]: 2026/09/20 09:05:55 INFO Signed narinfos id=18 count=15017lb # [ 57.734080] niks3-server[1047]: 2026/09/20 09:05:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5018lb # [ 57.736461] niks3-server[1047]: 2026/09/20 09:05:55 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5019lb # [ 57.738332] niks3-server[1047]: 2026/09/20 09:05:55 INFO Signed narinfos id=19 count=25020lb # [ 57.740157] niks3-server[1047]: 2026/09/20 09:05:55 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5021worker1 # [ 57.785588] nix-grpc-daemon[868]: time=2026-09-20T09:05:55.043Z level=INFO msg="Uploading 3 narinfos"5022lb # [ 57.751678] niks3-server[1047]: 2026/09/20 09:05:55 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5023lb # [ 57.756380] niks3-server[1047]: 2026/09/20 09:05:55 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5024lb # [ 57.759086] niks3-server[1047]: 2026/09/20 09:05:55 INFO Completed upload id=185025lb # [ 57.760906] niks3-server[1047]: 2026/09/20 09:05:55 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5026lb # [ 57.763390] niks3-server[1047]: 2026/09/20 09:05:55 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5027lb # [ 57.765559] niks3-server[1047]: 2026/09/20 09:05:55 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5028lb # [ 57.767803] niks3-server[1047]: 2026/09/20 09:05:55 INFO Completed upload id=195029worker1 # [ 57.815451] nix-grpc-daemon[868]: time=2026-09-20T09:05:55.074Z level=INFO msg="Upload complete. (483ms)"5030worker1 # [ 57.821166] nix-grpc-daemon[784]: ts=2026-09-20T09:05:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33458 duration_s=0 paths=2 nar_bytes_in=9765031worker1 # [ 57.826644] nix-grpc-daemon[784]: ts=2026-09-20T09:05:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334585032worker1 # [ 57.837340] nix-daemon[842]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5033worker1 # [ 57.845047] nix-daemon[842]: reaped child process 957, status = succeeded5034client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5035client # uses-input-w1onlya> node-a: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5036worker1 # [ 57.854370] nix-daemon[842]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5037worker1 # [ 59.282539] nix-daemon[842]: reaped child process 971, status = succeeded5038worker1 # [ 59.419264] nix-daemon[842]: accepted connection from pid 986, user nix-grpc-daemon (trusted)5039worker1 # [ 59.430969] nix-daemon[842]: reaped child process 989, status = succeeded5040lb # [ 59.402321] niks3-server[1047]: 2026/09/20 09:05:57 INFO Received uploads request method=POST path=/api/pending_closures5041worker1 # [ 59.458541] nix-grpc-daemon[868]: time=2026-09-20T09:05:56.717Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5042lb # [ 59.428594] niks3-server[1047]: 2026/09/20 09:05:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5043lb # [ 59.431221] niks3-server[1047]: 2026/09/20 09:05:57 INFO Signed narinfos id=20 count=15044worker1 # [ 59.478863] nix-grpc-daemon[868]: time=2026-09-20T09:05:56.737Z level=INFO msg="Uploading 1 narinfos"5045lb # [ 59.435063] niks3-server[1047]: 2026/09/20 09:05:57 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5046lb # [ 59.441143] niks3-server[1047]: 2026/09/20 09:05:57 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5047lb # [ 59.443718] niks3-server[1047]: 2026/09/20 09:05:57 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5048lb # [ 59.447280] niks3-server[1047]: 2026/09/20 09:05:57 INFO Completed upload id=205049worker1 # [ 59.494714] nix-grpc-daemon[868]: time=2026-09-20T09:05:56.753Z level=INFO msg="Upload complete. (231ms)"5050lb # [ 59.450886] niks3-server[1047]: 2026/09/20 09:05:57 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5051worker1 # [ 59.498162] nix-grpc-daemon[784]: ts=2026-09-20T09:05:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33458 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=9 outputs=15052worker1 # [ 59.503443] nix-daemon[842]: reaped child process 969, status = succeeded5053client # copying 1 paths...5054worker1 # [ 59.542380] nix-grpc-daemon[784]: ts=2026-09-20T09:05:56Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:334585055client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5056worker1 # [ 59.565141] nix-grpc-daemon[784]: ts=2026-09-20T09:05:56Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:334585057worker1 # [ 59.568838] nix-grpc-daemon[784]: ts=2026-09-20T09:05:56Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:33458 duration_s=0 paths=1 nar_bytes_out=1205058client # [ 59.665698] nix-daemon[951]: reaped child process 1037, status = succeeded5059client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&2, in 3.15 seconds)5060client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&25061client # [ 59.842307] nix-daemon[951]: accepted connection from pid 1056, user root (trusted)5062client # this derivation will be built:5063client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5064worker2 # [ 59.968556] nix-grpc-daemon[770]: ts=2026-09-20T09:05:58Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:374865065worker1 # [ 60.027406] nix-grpc-daemon[784]: ts=2026-09-20T09:05:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334585066client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux'...5067worker1 # [ 60.032373] nix-grpc-daemon[868]: time=2026-09-20T09:05:57.291Z level=INFO msg="All 1 paths already cached"5068worker2 # [ 59.985942] nix-grpc-daemon[770]: ts=2026-09-20T09:05:58Z level=debug event=want client=10 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05069worker2 # [ 59.988299] nix-grpc-daemon[770]: ts=2026-09-20T09:05:58Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=15070worker2 # [ 59.991260] nix-grpc-daemon[770]: ts=2026-09-20T09:05:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374865071worker2 # [ 59.998777] nix-grpc-daemon[770]: ts=2026-09-20T09:05:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374865072client # copying 1 paths to 'lb:50051'...5073client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5074worker2 # [ 60.012575] nix-grpc-daemon[770]: ts=2026-09-20T09:05:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:374865075worker2 # [ 60.023622] nix-daemon[844]: accepted connection from pid 770, user nix-grpc-daemon (trusted)5076worker2 # [ 60.039891] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 64 ms (attempt 1/5)5077worker2 # [ 60.108693] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 56 ms (attempt 2/5)5078worker2 # [ 60.170599] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 316 ms (attempt 3/5)5079worker2 # [ 60.492370] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 216 ms (attempt 4/5)5080worker2 # [ 60.711225] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5081worker2 # [ 60.714158] nix-grpc-daemon[770]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5082worker2 # [ 60.864082] nix-daemon[844]: accepted connection from pid 1195, user nix-grpc-daemon (trusted)5083worker2 # [ 60.877325] nix-daemon[844]: reaped child process 1198, status = succeeded5084worker2 # [ 61.015530] nix-daemon[844]: accepted connection from pid 1200, user nix-grpc-daemon (trusted)5085worker2 # [ 61.028152] nix-daemon[844]: reaped child process 1203, status = succeeded5086lb # [ 61.055376] niks3-server[1047]: 2026/09/20 09:05:58 INFO Received uploads request method=POST path=/api/pending_closures5087worker2 # [ 61.054238] nix-grpc-daemon[857]: time=2026-09-20T09:05:59.266Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5088worker2 # [ 61.055709] nix-grpc-daemon[857]: time=2026-09-20T09:05:59.267Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5089lb # [ 61.083479] niks3-server[1047]: 2026/09/20 09:05:58 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5090lb # [ 61.095900] niks3-server[1047]: 2026/09/20 09:05:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5091worker2 # [ 61.091392] nix-grpc-daemon[857]: time=2026-09-20T09:05:59.304Z level=INFO msg="Uploading 1 narinfos"5092lb # [ 61.101559] niks3-server[1047]: 2026/09/20 09:05:58 INFO Signed narinfos id=21 count=15093lb # [ 61.108493] niks3-server[1047]: 2026/09/20 09:05:58 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5094lb # [ 61.116111] niks3-server[1047]: 2026/09/20 09:05:58 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5095lb # [ 61.122887] niks3-server[1047]: 2026/09/20 09:05:58 INFO Completed upload id=215096worker2 # [ 61.112830] nix-grpc-daemon[857]: time=2026-09-20T09:05:59.326Z level=INFO msg="Upload complete. (383ms)"5097lb # [ 61.125754] niks3-server[1047]: 2026/09/20 09:05:58 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5098worker2 # [ 61.117622] nix-grpc-daemon[770]: ts=2026-09-20T09:05:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37486 duration_s=1 paths=1 nar_bytes_in=5605099worker2 # [ 61.122566] nix-grpc-daemon[770]: ts=2026-09-20T09:05:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374865100worker2 # [ 61.132289] nix-daemon[844]: accepted connection from pid 770, user nix-grpc-daemon (trusted)5101worker2 # [ 61.135691] nix-daemon[844]: reaped child process 1189, status = succeeded5102worker2 # [ 61.146749] nix-grpc-daemon[770]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...5103worker2 # [ 61.156391] nix-daemon[844]: accepted connection from pid 770, user nix-grpc-daemon (trusted)5104client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5105client # uses-input-w1onlyb> worker2: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5106worker2 # [ 61.968784] nix-daemon[844]: reaped child process 1208, status = succeeded5107worker2 # [ 62.130571] nix-daemon[844]: accepted connection from pid 1223, user nix-grpc-daemon (trusted)5108worker2 # [ 62.141513] nix-daemon[844]: reaped child process 1226, status = succeeded5109lb # [ 62.170588] niks3-server[1047]: 2026/09/20 09:05:59 INFO Received uploads request method=POST path=/api/pending_closures5110worker2 # [ 62.169075] nix-grpc-daemon[857]: time=2026-09-20T09:06:00.382Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5111lb # [ 62.199680] niks3-server[1047]: 2026/09/20 09:05:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5112lb # [ 62.208638] niks3-server[1047]: 2026/09/20 09:05:59 INFO Signed narinfos id=22 count=15113worker2 # [ 62.201651] nix-grpc-daemon[857]: time=2026-09-20T09:06:00.414Z level=INFO msg="Uploading 1 narinfos"5114lb # [ 62.214066] niks3-server[1047]: 2026/09/20 09:05:59 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5115lb # [ 62.219820] niks3-server[1047]: 2026/09/20 09:05:59 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5116lb # [ 62.231196] niks3-server[1047]: 2026/09/20 09:05:59 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5117lb # [ 62.234496] niks3-server[1047]: 2026/09/20 09:05:59 INFO Completed upload id=225118worker2 # [ 62.226729] nix-grpc-daemon[857]: time=2026-09-20T09:06:00.439Z level=INFO msg="Upload complete. (262ms)"5119lb # [ 62.237265] niks3-server[1047]: 2026/09/20 09:05:59 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5120worker2 # [ 62.229279] nix-grpc-daemon[770]: ts=2026-09-20T09:06:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37486 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=10 outputs=15121worker2 # [ 62.236528] nix-daemon[844]: reaped child process 1205, status = succeeded5122client # copying 1 paths...5123worker1 # [ 62.333081] nix-grpc-daemon[784]: ts=2026-09-20T09:05:59Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:334585124worker1 # [ 62.340789] nix-grpc-daemon[784]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5125client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5126worker1 # [ 62.366655] nix-grpc-daemon[784]: ts=2026-09-20T09:05:59Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:334585127worker1 # [ 62.368591] nix-grpc-daemon[784]: ts=2026-09-20T09:05:59Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:33458 duration_s=0 paths=1 nar_bytes_out=1205128client # [ 62.469797] nix-daemon[951]: reaped child process 1059, status = succeeded5129client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.80 seconds)5130client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&25131client # [ 62.647033] nix-daemon[951]: accepted connection from pid 1078, user root (trusted)5132client # this derivation will be built:5133client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5134worker1 # [ 62.835846] nix-grpc-daemon[784]: ts=2026-09-20T09:06:00Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:334585135client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux'...5136worker2 # [ 62.783455] nix-grpc-daemon[770]: ts=2026-09-20T09:06:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374865137worker2 # [ 62.792960] nix-grpc-daemon[857]: time=2026-09-20T09:06:01.006Z level=INFO msg="All 1 paths already cached"5138worker2 # [ 62.807201] nix-grpc-daemon[770]: ts=2026-09-20T09:06:01Z level=debug event=want client=11 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05139worker1 # [ 62.865924] nix-grpc-daemon[784]: ts=2026-09-20T09:06:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334585140worker2 # [ 62.811803] nix-grpc-daemon[770]: ts=2026-09-20T09:06:01Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=15141worker1 # [ 62.878749] nix-grpc-daemon[784]: ts=2026-09-20T09:06:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334585142worker1 # [ 62.891666] nix-grpc-daemon[868]: time=2026-09-20T09:06:00.150Z level=INFO msg="All 1 paths already cached"5143client # copying 1 paths to 'lb:50051'...5144client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5145worker1 # [ 62.898843] nix-grpc-daemon[784]: ts=2026-09-20T09:06:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334585146worker1 # [ 62.909822] nix-daemon[842]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5147worker1 # [ 63.068207] nix-daemon[842]: accepted connection from pid 994, user nix-grpc-daemon (trusted)5148worker1 # [ 63.080969] nix-daemon[842]: reaped child process 997, status = succeeded5149worker1 # [ 63.234114] nix-daemon[842]: accepted connection from pid 999, user nix-grpc-daemon (trusted)5150worker1 # [ 63.245813] nix-daemon[842]: reaped child process 1002, status = succeeded5151lb # [ 63.214847] niks3-server[1047]: 2026/09/20 09:06:00 INFO Received uploads request method=POST path=/api/pending_closures5152worker1 # [ 63.268266] nix-grpc-daemon[868]: time=2026-09-20T09:06:00.527Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5153worker1 # [ 63.270118] nix-grpc-daemon[868]: time=2026-09-20T09:06:00.529Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5154lb # [ 63.240859] niks3-server[1047]: 2026/09/20 09:06:00 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5155lb # [ 63.250829] niks3-server[1047]: 2026/09/20 09:06:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5156lb # [ 63.255630] niks3-server[1047]: 2026/09/20 09:06:00 INFO Signed narinfos id=23 count=15157worker1 # [ 63.306833] nix-grpc-daemon[868]: time=2026-09-20T09:06:00.565Z level=INFO msg="Uploading 1 narinfos"5158lb # [ 63.264799] niks3-server[1047]: 2026/09/20 09:06:00 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5159lb # [ 63.276388] niks3-server[1047]: 2026/09/20 09:06:00 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5160worker1 # [ 63.328645] nix-grpc-daemon[868]: time=2026-09-20T09:06:00.587Z level=INFO msg="Upload complete. (407ms)"5161lb # [ 63.282607] niks3-server[1047]: 2026/09/20 09:06:00 INFO Completed upload id=235162worker1 # [ 63.331150] nix-grpc-daemon[784]: ts=2026-09-20T09:06:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33458 duration_s=0 paths=1 nar_bytes_in=5605163lb # [ 63.286258] niks3-server[1047]: 2026/09/20 09:06:00 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5164worker1 # [ 63.338123] nix-grpc-daemon[784]: ts=2026-09-20T09:06:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334585165worker1 # [ 63.345841] nix-daemon[842]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5166worker1 # [ 63.352730] nix-daemon[842]: reaped child process 992, status = succeeded5167worker1 # [ 63.357797] nix-daemon[842]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5168client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5169client # uses-input-w1onlyc> node-a: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5170worker1 # [ 63.769082] nix-daemon[842]: reaped child process 1006, status = succeeded5171worker1 # [ 63.913639] nix-daemon[842]: accepted connection from pid 1021, user nix-grpc-daemon (trusted)5172worker1 # [ 63.925137] nix-daemon[842]: reaped child process 1024, status = succeeded5173lb # [ 63.894456] niks3-server[1047]: 2026/09/20 09:06:01 INFO Received uploads request method=POST path=/api/pending_closures5174worker1 # [ 63.948671] nix-grpc-daemon[868]: time=2026-09-20T09:06:01.207Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5175lb # [ 63.926136] niks3-server[1047]: 2026/09/20 09:06:01 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5176lb # [ 63.930115] niks3-server[1047]: 2026/09/20 09:06:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5177lb # [ 63.934794] niks3-server[1047]: 2026/09/20 09:06:01 INFO Signed narinfos id=24 count=15178worker1 # [ 63.984337] nix-grpc-daemon[868]: time=2026-09-20T09:06:01.243Z level=INFO msg="Uploading 1 narinfos"5179lb # [ 63.944937] niks3-server[1047]: 2026/09/20 09:06:01 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5180lb # [ 63.954150] niks3-server[1047]: 2026/09/20 09:06:01 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5181lb # [ 63.961308] niks3-server[1047]: 2026/09/20 09:06:01 INFO Completed upload id=245182worker1 # [ 64.007265] nix-grpc-daemon[868]: time=2026-09-20T09:06:01.266Z level=INFO msg="Upload complete. (244ms)"5183lb # [ 63.964409] niks3-server[1047]: 2026/09/20 09:06:01 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5184worker1 # [ 64.011296] nix-grpc-daemon[784]: ts=2026-09-20T09:06:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33458 duration_s=0 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=11 outputs=15185worker1 # [ 64.022555] nix-daemon[842]: reaped child process 1004, status = succeeded5186client # copying 1 paths...5187worker2 # [ 64.001671] nix-grpc-daemon[770]: ts=2026-09-20T09:06:02Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:374865188worker2 # [ 64.013165] nix-grpc-daemon[770]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5189client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5190worker2 # [ 64.045366] nix-grpc-daemon[770]: ts=2026-09-20T09:06:02Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:374865191worker2 # [ 64.051431] nix-grpc-daemon[770]: ts=2026-09-20T09:06:02Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37486 duration_s=0 paths=1 nar_bytes_out=1205192client # [ 64.209758] nix-daemon[951]: reaped child process 1081, status = succeeded5193client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&2, in 1.73 seconds)5194(finished: subtest: an input only one worker has still reaches the builder, in 8.50 seconds)5195subtest: upload whose reference the worker lacks is completed from the cache5196client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache5197client # this derivation will be built:5198client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5199client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5200client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache, in 0.41 seconds)5201client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache5202client # this derivation will be built:5203client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5204client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5205client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache, in 0.38 seconds)5206client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5207client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.17 seconds)5208worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5209worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.22 seconds)5210worker2 # [ 65.379558] systemd[1]: Stopping Nix worker-protocol over gRPC...5211worker2 # [ 65.904368] nix-grpc-daemon[770]: ts=2026-09-20T09:06:04Z level=info event=signal_exit5212worker2 # [ 66.169224] nix-grpc-daemon[770]: ts=2026-09-20T09:06:04Z level=info event=scheduler_yield5213worker1 # [ 66.229318] nix-grpc-daemon[784]: ts=2026-09-20T09:06:03Z level=info event=scheduler_restarting addr=lb:500515214lb # [ 66.185353] niks3-server[1047]: 2026/09/20 09:06:03 INFO lead: released remote=192.168.1.4:580405215worker2 # [ 66.175086] nix-grpc-daemon[770]: ts=2026-09-20T09:06:04Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:344905216worker2 # [ 66.181388] nix-grpc-daemon[770]: ts=2026-09-20T09:06:04Z level=info event=worker_gone addr=192.168.1.3:500515217worker2 # [ 66.184666] nix-grpc-daemon[770]: ts=2026-09-20T09:06:04Z level=info event=scheduler_restarting addr=lb:500515218worker2 # [ 66.187991] nix-grpc-daemon[770]: ts=2026-09-20T09:06:04Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:344905219worker2 # [ 66.191618] nix-grpc-daemon[770]: ts=2026-09-20T09:06:04Z level=info event=worker_gone addr=192.168.1.4:500515220worker1 # [ 66.429117] nix-grpc-daemon[784]: ts=2026-09-20T09:06:03Z level=info event=scheduler_connected addr=lb:500515221worker2 # [ 66.382315] nix-grpc-daemon[770]: ts=2026-09-20T09:06:04Z level=info event=scheduler_connected addr=lb:500515222worker1 # [ 66.444036] nix-grpc-daemon[784]: ts=2026-09-20T09:06:03Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5223worker2 # [ 66.387458] nix-grpc-daemon[770]: ts=2026-09-20T09:06:04Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5224worker2 # [ 66.587075] nix-grpc-daemon[770]: ts=2026-09-20T09:06:04Z level=info event=scheduler_connected addr=lb:500515225worker1 # [ 66.645136] nix-grpc-daemon[784]: ts=2026-09-20T09:06:03Z level=info event=scheduler_connected addr=lb:500515226worker2 # [ 66.592933] nix-grpc-daemon[770]: ts=2026-09-20T09:06:04Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5227worker1 # [ 66.650132] nix-grpc-daemon[784]: ts=2026-09-20T09:06:03Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5228lb # [ 66.642945] envoy[918]: conn peer=[2001:db8:1::1]:39434 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4197 tx=1751 ms=27095229lb # [ 66.647431] envoy[918]: conn peer=[2001:db8:1::1]:39438 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=421 tx=328 ms=535230lb # [ 66.651478] envoy[918]: conn peer=[2001:db8:1::1]:39458 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=535231lb # [ 66.655400] envoy[918]: conn peer=[2001:db8:1::1]:39444 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3715 tx=1915 ms=23975232lb # [ 66.658866] envoy[918]: conn peer=[2001:db8:1::1]:59046 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=595233lb # [ 66.662464] envoy[918]: conn peer=[2001:db8:1::1]:59044 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3703 tx=1893 ms=13245234worker2 # [ 66.794170] nix-grpc-daemon[770]: ts=2026-09-20T09:06:05Z level=info event=scheduler_connected addr=lb:500515235worker1 # [ 66.850977] nix-grpc-daemon[784]: ts=2026-09-20T09:06:04Z level=info event=scheduler_connected addr=lb:500515236worker1 # [ 66.856245] nix-grpc-daemon[784]: ts=2026-09-20T09:06:04Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5237worker2 # [ 66.799919] nix-grpc-daemon[770]: ts=2026-09-20T09:06:05Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5238worker1 # [ 67.056875] nix-grpc-daemon[784]: ts=2026-09-20T09:06:04Z level=info event=scheduler_connected addr=lb:500515239worker2 # [ 67.001675] nix-grpc-daemon[770]: ts=2026-09-20T09:06:05Z level=info event=scheduler_connected addr=lb:500515240worker1 # [ 67.062884] nix-grpc-daemon[784]: ts=2026-09-20T09:06:04Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5241worker2 # [ 67.006741] nix-grpc-daemon[770]: ts=2026-09-20T09:06:05Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5242worker2 # [ 67.173683] nix-grpc-daemon[770]: ts=2026-09-20T09:06:05Z level=info event=scheduler_restarting5243worker2 # [ 67.199299] nix-daemon[844]: reaped child process 847, status = succeeded5244worker1 # [ 67.263890] nix-grpc-daemon[784]: ts=2026-09-20T09:06:04Z level=info event=scheduler_connected addr=lb:500515245worker2 # [ 67.210780] nix-grpc-daemon[770]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5246worker2 # [ 67.217102] nix-grpc-daemon[770]: E0000 00:00:1789895165.423498 825 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 10, started true, shutdown true5247worker2 # [ 67.229326] nix-grpc-daemon[770]: ts=2026-09-20T09:06:05Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5248worker1 # [ 67.403177] nix-grpc-daemon[784]: ts=2026-09-20T09:06:04Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: remote connection failure"5249worker2 # [ 67.367683] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5250worker2 # [ 67.371803] systemd[1]: Stopped Nix worker-protocol over gRPC.5251worker2 # [ 67.375571] systemd[1]: nix-grpc-daemon.service: Consumed 2.612s CPU time over 50.739s wall clock time, 64M memory peak, 91.4K incoming IP traffic, 179.2K outgoing IP traffic.5252worker2 # [ 67.385678] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5253worker2 # [ 67.388974] systemd[1]: Closed Nix worker-protocol over gRPC.5254worker2 # [ 67.391914] systemd[1]: nix-grpc-daemon.socket: Consumed 298us CPU time over 58.186s wall clock time, 256K memory peak, 111.6K incoming IP traffic, 86K outgoing IP traffic.5255lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5256lb: must succeed: curl -sf localhost:9901/clusters5257lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5258lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.07 seconds)5259client: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25260worker1 # [ 67.577824] nix-grpc-daemon[784]: ts=2026-09-20T09:06:04Z level=info event=scheduler_take_over5261lb # [ 67.531721] niks3-server[1047]: 2026/09/20 09:06:05 INFO lead: acquired remote=192.168.1.3:479125262worker1 # [ 67.603910] nix-grpc-daemon[784]: ts=2026-09-20T09:06:04Z level=info event=scheduler_connected addr=lb:500515263worker1 # [ 67.608316] nix-grpc-daemon[784]: ts=2026-09-20T09:06:04Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05264worker1 # [ 67.734142] nix-grpc-daemon[784]: ts=2026-09-20T09:06:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:334585265worker1 # [ 67.741212] nix-grpc-daemon[784]: ts=2026-09-20T09:06:05Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:334585266client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5267client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.24 seconds)5268worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5269worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.03 seconds)5270worker2 # [ 67.802121] systemd[1]: Listening on Nix worker-protocol over gRPC.5271worker1 # [ 67.920590] systemd[1]: Stopping Nix worker-protocol over gRPC...5272worker1 # [ 67.949124] nix-grpc-daemon[784]: ts=2026-09-20T09:06:05Z level=info event=signal_exit5273worker2 # [ 68.049663] systemd[1]: Starting Nix worker-protocol over gRPC...5274worker1 # [ 68.208445] nix-grpc-daemon[784]: ts=2026-09-20T09:06:05Z level=info event=scheduler_yield5275lb # [ 68.164264] niks3-server[1047]: 2026/09/20 09:06:05 INFO lead: released remote=192.168.1.3:479125276worker1 # [ 68.212337] nix-grpc-daemon[784]: ts=2026-09-20T09:06:05Z level=info event=scheduler_restarting addr=lb:500515277worker1 # [ 68.216805] nix-grpc-daemon[784]: ts=2026-09-20T09:06:05Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:515645278worker1 # [ 68.220140] nix-grpc-daemon[784]: ts=2026-09-20T09:06:05Z level=info event=worker_gone addr=192.168.1.3:500515279worker2 # [ 68.203564] nix-grpc-daemon[1250]: ts=2026-09-20T09:06:06Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5280worker2 # [ 68.207617] nix-grpc-daemon[1250]: ts=2026-09-20T09:06:06Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15281worker2 # [ 68.213847] nix-grpc-daemon[1250]: ts=2026-09-20T09:06:06Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5282worker2 # [ 68.217678] systemd[1]: Started Nix worker-protocol over gRPC.5283lb # [ 68.236091] niks3-server[1047]: 2026/09/20 09:06:05 INFO lead: acquired remote=192.168.1.4:517165284worker2 # [ 68.226182] nix-grpc-daemon[1250]: ts=2026-09-20T09:06:06Z level=info event=scheduler_take_over5285worker2 # [ 68.246873] nix-grpc-daemon[1250]: ts=2026-09-20T09:06:06Z level=info event=scheduler_connected addr=lb:500515286worker2 # [ 68.249961] nix-grpc-daemon[1250]: ts=2026-09-20T09:06:06Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05287worker1 # [ 68.413751] nix-grpc-daemon[784]: ts=2026-09-20T09:06:05Z level=info event=scheduler_connected addr=lb:500515288worker2 # [ 68.358590] nix-grpc-daemon[1250]: ts=2026-09-20T09:06:06Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05289worker1 # [ 69.212110] nix-grpc-daemon[784]: ts=2026-09-20T09:06:06Z level=info event=scheduler_restarting5290worker2 # [ 69.163852] nix-grpc-daemon[1250]: ts=2026-09-20T09:06:07Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:380105291worker2 # [ 69.167589] nix-grpc-daemon[1250]: ts=2026-09-20T09:06:07Z level=info event=worker_gone addr=192.168.1.3:500515292worker1 # [ 69.231746] nix-grpc-daemon[784]: ts=2026-09-20T09:06:06Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5293worker1 # [ 69.246657] nix-daemon[842]: reaped child process 844, status = succeeded5294worker1 # [ 69.373663] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5295worker1 # [ 69.376696] systemd[1]: Stopped Nix worker-protocol over gRPC.5296worker1 # [ 69.380580] systemd[1]: nix-grpc-daemon.service: Consumed 1.415s CPU time over 52.692s wall clock time, 50.5M memory peak, 52.7K incoming IP traffic, 94.9K outgoing IP traffic.5297worker1 # [ 69.391914] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5298worker1 # [ 69.395422] systemd[1]: Closed Nix worker-protocol over gRPC.5299worker1 # [ 69.397708] systemd[1]: nix-grpc-daemon.socket: Consumed 288us CPU time over 1min 271ms wall clock time, 256K memory peak, 83.5K incoming IP traffic, 71.9K outgoing IP traffic.5300lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5301lb: must succeed: curl -sf localhost:9901/clusters5302lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5303lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.07 seconds)5304client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25305worker2 # [ 69.639384] nix-grpc-daemon[1250]: ts=2026-09-20T09:06:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:380245306worker2 # [ 69.651190] nix-daemon[844]: accepted connection from pid 1250, user nix-grpc-daemon (trusted)5307worker2 # [ 69.667553] nix-grpc-daemon[1250]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 5 ms (attempt 1/5)5308worker2 # [ 69.678847] nix-grpc-daemon[1250]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 43 ms (attempt 2/5)5309worker2 # [ 69.724550] nix-grpc-daemon[1250]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 189 ms (attempt 3/5)5310worker2 # [ 69.919680] nix-grpc-daemon[1250]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 653 ms (attempt 4/5)5311worker2 # [ 70.579151] nix-grpc-daemon[1250]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5312client # copying 2 paths...5313client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5314client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5315worker2 # [ 70.601155] nix-grpc-daemon[1250]: ts=2026-09-20T09:06:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:380245316worker2 # [ 70.612173] nix-daemon[844]: accepted connection from pid 1250, user nix-grpc-daemon (trusted)5317worker2 # [ 70.635444] nix-grpc-daemon[1250]: ts=2026-09-20T09:06:08Z level=info event=niks3_push_started pid=13065318worker2 # [ 71.016537] nix-daemon[844]: accepted connection from pid 1319, user nix-grpc-daemon (trusted)5319worker2 # [ 71.030663] nix-daemon[844]: reaped child process 1322, status = succeeded5320lb # [ 71.058073] niks3-server[1047]: 2026/09/20 09:06:08 INFO Received uploads request method=POST path=/api/pending_closures5321lb # [ 71.064870] niks3-server[1047]: 2026/09/20 09:06:08 INFO Received uploads request method=POST path=/api/pending_closures5322worker2 # [ 71.058717] nix-grpc-daemon[1306]: time=2026-09-20T09:06:09.271Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5323worker2 # [ 71.062064] nix-grpc-daemon[1306]: time=2026-09-20T09:06:09.273Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5324worker2 # [ 71.063645] nix-grpc-daemon[1306]: time=2026-09-20T09:06:09.273Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5325lb # [ 71.093737] niks3-server[1047]: 2026/09/20 09:06:08 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5326lb # [ 71.107373] niks3-server[1047]: 2026/09/20 09:06:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5327lb # [ 71.110771] niks3-server[1047]: 2026/09/20 09:06:08 INFO Signed narinfos id=25 count=15328lb # [ 71.113595] niks3-server[1047]: 2026/09/20 09:06:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5329lb # [ 71.117086] niks3-server[1047]: 2026/09/20 09:06:08 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5330lb # [ 71.120330] niks3-server[1047]: 2026/09/20 09:06:08 INFO Signed narinfos id=26 count=25331worker2 # [ 71.111092] nix-grpc-daemon[1306]: time=2026-09-20T09:06:09.323Z level=INFO msg="Uploading 3 narinfos"5332lb # [ 71.123829] niks3-server[1047]: 2026/09/20 09:06:08 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5333lb # [ 71.142306] niks3-server[1047]: 2026/09/20 09:06:08 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5334lb # [ 71.145623] niks3-server[1047]: 2026/09/20 09:06:08 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5335lb # [ 71.148659] niks3-server[1047]: 2026/09/20 09:06:08 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5336lb # [ 71.150485] niks3-server[1047]: 2026/09/20 09:06:08 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5337lb # [ 71.155042] niks3-server[1047]: 2026/09/20 09:06:08 INFO Completed upload id=255338lb # [ 71.156327] niks3-server[1047]: 2026/09/20 09:06:08 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5339lb # [ 71.158607] niks3-server[1047]: 2026/09/20 09:06:08 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5340lb # [ 71.161167] niks3-server[1047]: 2026/09/20 09:06:08 INFO Completed upload id=265341worker2 # [ 71.151447] nix-grpc-daemon[1306]: time=2026-09-20T09:06:09.364Z level=INFO msg="Upload complete. (297ms)"5342worker2 # [ 71.156656] nix-grpc-daemon[1250]: ts=2026-09-20T09:06:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38024 duration_s=0 paths=2 nar_bytes_in=3125343worker2 # [ 71.169452] nix-daemon[844]: reaped child process 1304, status = succeeded5344client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.76 seconds)5345worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5346worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds)5347worker1 # [ 71.320705] systemd[1]: Listening on Nix worker-protocol over gRPC.5348worker1 # [ 71.327696] systemd[1]: Starting Nix worker-protocol over gRPC...5349worker1 # [ 71.501179] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:08Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5350worker1 # [ 71.505077] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:08Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15351worker1 # [ 71.511561] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:08Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5352worker1 # [ 71.515335] systemd[1]: Started Nix worker-protocol over gRPC.5353lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5354lb: must succeed: curl -sf localhost:9901/clusters5355worker1 # [ 71.543750] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:08Z level=info event=scheduler_connected addr=lb:500515356worker2 # [ 71.489854] nix-grpc-daemon[1250]: ts=2026-09-20T09:06:09Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05357lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5358lb: must succeed: curl -sf localhost:9901/clusters5359lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5360lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.16 seconds)5361(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 8.46 seconds)5362subtest: scheduler down: the other node takes the lock and keeps it5363lb: must succeed: curl -sf localhost:9901/clusters5364lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5365lb: must succeed: curl -sf localhost:9901/clusters5366lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5367lb: must succeed: curl -sf localhost:9901/clusters5368lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5369lb: must succeed: curl -sf localhost:9901/clusters5370lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5371worker2 # [ 72.870377] systemd[1]: Stopping Nix worker-protocol over gRPC...5372worker2 # [ 73.219815] nix-grpc-daemon[1250]: ts=2026-09-20T09:06:11Z level=info event=signal_exit5373worker2 # [ 73.226700] nix-grpc-daemon[1250]: ts=2026-09-20T09:06:11Z level=info event=scheduler_yield5374lb # [ 73.239606] niks3-server[1047]: 2026/09/20 09:06:10 INFO lead: released remote=192.168.1.4:517165375worker1 # [ 73.290189] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:10Z level=info event=scheduler_restarting addr=lb:500515376worker2 # [ 73.233854] nix-grpc-daemon[1250]: ts=2026-09-20T09:06:11Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:380105377worker2 # [ 73.239232] nix-grpc-daemon[1250]: ts=2026-09-20T09:06:11Z level=info event=worker_gone addr=192.168.1.3:500515378worker2 # [ 73.242291] nix-grpc-daemon[1250]: ts=2026-09-20T09:06:11Z level=info event=scheduler_restarting addr=lb:500515379worker2 # [ 73.247092] nix-grpc-daemon[1250]: ts=2026-09-20T09:06:11Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:380105380worker2 # [ 73.250396] nix-grpc-daemon[1250]: ts=2026-09-20T09:06:11Z level=info event=worker_gone addr=192.168.1.4:500515381worker1 # [ 73.490343] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:10Z level=info event=scheduler_connected addr=lb:500515382worker2 # [ 73.440120] nix-grpc-daemon[1250]: ts=2026-09-20T09:06:11Z level=info event=scheduler_connected addr=lb:500515383worker2 # [ 73.446705] nix-grpc-daemon[1250]: ts=2026-09-20T09:06:11Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5384worker1 # [ 73.508843] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:10Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5385worker2 # [ 73.647989] nix-grpc-daemon[1250]: ts=2026-09-20T09:06:11Z level=info event=scheduler_connected addr=lb:500515386worker1 # [ 73.710190] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:10Z level=info event=scheduler_connected addr=lb:500515387worker2 # [ 73.653623] nix-grpc-daemon[1250]: ts=2026-09-20T09:06:11Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5388worker1 # [ 73.714876] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:10Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5389worker2 # [ 73.854932] nix-grpc-daemon[1250]: ts=2026-09-20T09:06:12Z level=info event=scheduler_connected addr=lb:500515390worker1 # [ 73.915650] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:11Z level=info event=scheduler_connected addr=lb:500515391worker2 # [ 73.860976] nix-grpc-daemon[1250]: ts=2026-09-20T09:06:12Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5392worker1 # [ 73.920795] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:11Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5393worker2 # [ 74.062616] nix-grpc-daemon[1250]: ts=2026-09-20T09:06:12Z level=info event=scheduler_connected addr=lb:500515394worker1 # [ 74.121723] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:11Z level=info event=scheduler_connected addr=lb:500515395worker2 # [ 74.067668] nix-grpc-daemon[1250]: ts=2026-09-20T09:06:12Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5396worker1 # [ 74.126748] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:11Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5397worker2 # [ 74.234734] nix-grpc-daemon[1250]: ts=2026-09-20T09:06:12Z level=info event=scheduler_restarting5398worker2 # [ 74.249254] nix-grpc-daemon[1250]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5399worker2 # [ 74.252516] nix-grpc-daemon[1250]: E0000 00:00:1789895172.462296 1293 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 11, started true, shutdown true5400worker2 # [ 74.257118] nix-grpc-daemon[1250]: E0000 00:00:1789895172.462468 1293 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5401worker2 # [ 74.266910] nix-daemon[844]: reaped child process 1299, status = succeeded5402worker1 # [ 74.327523] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:11Z level=info event=scheduler_connected addr=lb:500515403worker2 # [ 74.274615] nix-grpc-daemon[1250]: E0000 00:00:1789895172.486595 1293 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 10, started true, shutdown true5404worker2 # [ 74.286597] nix-grpc-daemon[1250]: ts=2026-09-20T09:06:12Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5405worker1 # [ 76.282924] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:13Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: remote connection failure"5406worker2 # [ 76.245788] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5407worker2 # [ 76.249159] systemd[1]: Stopped Nix worker-protocol over gRPC.5408worker2 # [ 76.252984] systemd[1]: nix-grpc-daemon.service: Consumed 363ms CPU time over 8.195s wall clock time, 32.6M memory peak, 17.6K incoming IP traffic, 22.2K outgoing IP traffic.5409worker2 # [ 76.263941] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5410worker2 # [ 76.267395] systemd[1]: Closed Nix worker-protocol over gRPC.5411worker2 # [ 76.270182] systemd[1]: nix-grpc-daemon.socket: Consumed 327us CPU time over 8.468s wall clock time, 256K memory peak, 25K incoming IP traffic, 18.8K outgoing IP traffic.5412worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5413worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.06 seconds)5414worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5415worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.04 seconds)5416worker1 # [ 76.482812] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:13Z level=info event=scheduler_connected addr=lb:500515417worker1 # [ 76.485583] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:13Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5418lb # [ 76.476693] niks3-server[1047]: 2026/09/20 09:06:14 INFO lead: acquired remote=192.168.1.3:590145419worker1 # [ 76.525872] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:13Z level=info event=scheduler_take_over5420worker1 # [ 76.686708] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:13Z level=info event=scheduler_connected addr=lb:500515421lb # [ 76.643447] envoy[918]: conn peer=[2001:db8:1::4]:59534 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=12975 tx=2768 ms=502615422worker1 # [ 76.690639] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:13Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: remote connection failure"5423lb # [ 76.648641] envoy[918]: conn peer=[2001:db8:1::1]:59050 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=215424lb # [ 76.652841] envoy[918]: conn peer=[2001:db8:1::3]:54656 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=11995 tx=2398 ms=511965425lb # [ 76.656822] envoy[918]: conn peer=[2001:db8:1::1]:59056 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1009 tx=305 ms=15305426lb # [ 76.662882] envoy[918]: conn peer=[2001:db8:1::4]:42310 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1323 tx=362 ms=60425427worker1 # [ 76.891648] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:14Z level=info event=scheduler_connected addr=lb:500515428worker1 # [ 76.896858] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:14Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05429worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5430worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.07 seconds)5431client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&25432worker1 # [ 77.722795] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:583765433worker1 # [ 77.734151] nix-daemon[842]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)5434client # don't know how to build these paths:5435client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5436worker1 # [ 77.748235] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:583765437worker1 # [ 77.758818] nix-grpc-daemon[1047]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 58 ms (attempt 1/5)5438worker1 # [ 77.822674] nix-grpc-daemon[1047]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 81 ms (attempt 2/5)5439worker1 # [ 77.909428] nix-grpc-daemon[1047]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 18 ms (attempt 3/5)5440worker1 # [ 77.933389] nix-grpc-daemon[1047]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 159 ms (attempt 4/5)5441worker1 # [ 78.095563] nix-grpc-daemon[1047]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5442worker1 # [ 78.124898] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:15Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05443worker1 # [ 78.130469] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:15Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=15444worker1 # [ 78.169604] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:583765445worker1 # [ 78.183850] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:583765446client # copying 1 paths to 'lb:50051'...5447client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5448worker1 # [ 78.196834] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:583765449worker1 # [ 78.205579] nix-daemon[842]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)5450worker1 # [ 78.226283] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:15Z level=info event=niks3_push_started pid=11215451worker1 # [ 78.567159] nix-daemon[842]: accepted connection from pid 1134, user nix-grpc-daemon (trusted)5452worker1 # [ 78.579486] nix-daemon[842]: reaped child process 1137, status = succeeded5453worker1 # [ 78.746528] nix-daemon[842]: accepted connection from pid 1139, user nix-grpc-daemon (trusted)5454worker1 # [ 78.758416] nix-daemon[842]: reaped child process 1142, status = succeeded5455lb # [ 78.728736] niks3-server[1047]: 2026/09/20 09:06:16 INFO Received uploads request method=POST path=/api/pending_closures5456worker1 # [ 78.786588] nix-grpc-daemon[1121]: time=2026-09-20T09:06:16.045Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5457worker1 # [ 78.788483] nix-grpc-daemon[1121]: time=2026-09-20T09:06:16.047Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5458lb # [ 78.764124] niks3-server[1047]: 2026/09/20 09:06:16 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5459lb # [ 78.778607] niks3-server[1047]: 2026/09/20 09:06:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5460lb # [ 78.784863] niks3-server[1047]: 2026/09/20 09:06:16 INFO Signed narinfos id=27 count=15461worker1 # [ 78.834374] nix-grpc-daemon[1121]: time=2026-09-20T09:06:16.093Z level=INFO msg="Uploading 1 narinfos"5462lb # [ 78.797301] niks3-server[1047]: 2026/09/20 09:06:16 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5463lb # [ 78.803795] niks3-server[1047]: 2026/09/20 09:06:16 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5464lb # [ 78.813058] niks3-server[1047]: 2026/09/20 09:06:16 INFO Completed upload id=275465lb # [ 78.814565] niks3-server[1047]: 2026/09/20 09:06:16 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5466worker1 # [ 78.860758] nix-grpc-daemon[1121]: time=2026-09-20T09:06:16.119Z level=INFO msg="Upload complete. (446ms)"5467worker1 # [ 78.865764] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58376 duration_s=0 paths=1 nar_bytes_in=5205468worker1 # [ 78.873289] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:583765469worker1 # [ 78.882111] nix-daemon[842]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)5470worker1 # [ 78.889110] nix-daemon[842]: reaped child process 1119, status = succeeded5471client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5472client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5473worker1 # [ 78.894107] nix-daemon[842]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)5474client # farm-a> LOG-farm-a-on-standby5475worker1 # [ 80.720906] nix-daemon[842]: reaped child process 1146, status = succeeded5476worker1 # [ 80.840088] nix-daemon[842]: accepted connection from pid 1161, user nix-grpc-daemon (trusted)5477worker1 # [ 80.850513] nix-daemon[842]: reaped child process 1164, status = succeeded5478lb # [ 80.815379] niks3-server[1047]: 2026/09/20 09:06:18 INFO Received uploads request method=POST path=/api/pending_closures5479worker1 # [ 80.866972] nix-grpc-daemon[1121]: time=2026-09-20T09:06:18.125Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5480worker1 # [ 80.868468] nix-grpc-daemon[1121]: time=2026-09-20T09:06:18.127Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5481lb # [ 80.842503] niks3-server[1047]: 2026/09/20 09:06:18 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5482lb # [ 80.844846] niks3-server[1047]: 2026/09/20 09:06:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5483lb # [ 80.846822] niks3-server[1047]: 2026/09/20 09:06:18 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5484lb # [ 80.849874] niks3-server[1047]: 2026/09/20 09:06:18 INFO Signed narinfos id=28 count=15485worker1 # [ 80.897805] nix-grpc-daemon[1121]: time=2026-09-20T09:06:18.156Z level=INFO msg="Uploading 1 narinfos"5486lb # [ 80.852730] niks3-server[1047]: 2026/09/20 09:06:18 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5487lb # [ 80.858060] niks3-server[1047]: 2026/09/20 09:06:18 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5488lb # [ 80.861933] niks3-server[1047]: 2026/09/20 09:06:18 INFO Completed upload id=285489worker1 # [ 80.908158] nix-grpc-daemon[1121]: time=2026-09-20T09:06:18.167Z level=INFO msg="Upload complete. (204ms)"5490lb # [ 80.863587] niks3-server[1047]: 2026/09/20 09:06:18 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5491worker1 # [ 80.910435] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58376 duration_s=2 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15492worker1 # [ 80.920460] nix-daemon[842]: reaped child process 1144, status = succeeded5493worker1 # [ 80.923982] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:18Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05494worker1 # [ 80.925829] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:18Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=15495worker1 # [ 80.969554] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:583765496worker1 # [ 80.977087] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:583765497worker1 # [ 80.986096] nix-grpc-daemon[1121]: time=2026-09-20T09:06:18.245Z level=INFO msg="All 1 paths already cached"5498client # copying 1 paths to 'lb:50051'...5499client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5500worker1 # [ 80.992327] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:583765501worker1 # [ 80.998959] nix-daemon[842]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)5502worker1 # [ 81.146176] nix-daemon[842]: accepted connection from pid 1170, user nix-grpc-daemon (trusted)5503worker1 # [ 81.158488] nix-daemon[842]: reaped child process 1173, status = succeeded5504worker1 # [ 81.290727] nix-daemon[842]: accepted connection from pid 1175, user nix-grpc-daemon (trusted)5505worker1 # [ 81.302327] nix-daemon[842]: reaped child process 1178, status = succeeded5506lb # [ 81.267339] niks3-server[1047]: 2026/09/20 09:06:18 INFO Received uploads request method=POST path=/api/pending_closures5507worker1 # [ 81.318174] nix-grpc-daemon[1121]: time=2026-09-20T09:06:18.577Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5508worker1 # [ 81.320093] nix-grpc-daemon[1121]: time=2026-09-20T09:06:18.578Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5509lb # [ 81.283805] niks3-server[1047]: 2026/09/20 09:06:18 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5510lb # [ 81.285550] niks3-server[1047]: 2026/09/20 09:06:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5511lb # [ 81.287745] niks3-server[1047]: 2026/09/20 09:06:18 INFO Signed narinfos id=29 count=15512worker1 # [ 81.334056] nix-grpc-daemon[1121]: time=2026-09-20T09:06:18.593Z level=INFO msg="Uploading 1 narinfos"5513lb # [ 81.290901] niks3-server[1047]: 2026/09/20 09:06:18 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5514lb # [ 81.295911] niks3-server[1047]: 2026/09/20 09:06:18 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5515lb # [ 81.298990] niks3-server[1047]: 2026/09/20 09:06:19 INFO Completed upload id=295516worker1 # [ 81.346501] nix-grpc-daemon[1121]: time=2026-09-20T09:06:18.605Z level=INFO msg="Upload complete. (339ms)"5517lb # [ 81.300771] niks3-server[1047]: 2026/09/20 09:06:19 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5518worker1 # [ 81.348512] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58376 duration_s=0 paths=1 nar_bytes_in=6405519worker1 # [ 81.351272] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:583765520worker1 # [ 81.358086] nix-daemon[842]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)5521worker1 # [ 81.360305] nix-daemon[842]: reaped child process 1168, status = succeeded5522worker1 # [ 81.365427] nix-daemon[842]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)5523client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5524client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5525client # farm-b> LOG-farm-b-on-standby5526worker1 # [ 82.389334] nix-daemon[842]: reaped child process 1182, status = succeeded5527worker1 # [ 82.528776] nix-daemon[842]: accepted connection from pid 1197, user nix-grpc-daemon (trusted)5528worker1 # [ 82.539546] nix-daemon[842]: reaped child process 1200, status = succeeded5529lb # [ 82.506116] niks3-server[1047]: 2026/09/20 09:06:20 INFO Received uploads request method=POST path=/api/pending_closures5530worker1 # [ 82.557380] nix-grpc-daemon[1121]: time=2026-09-20T09:06:19.816Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5531worker1 # [ 82.558979] nix-grpc-daemon[1121]: time=2026-09-20T09:06:19.818Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"5532lb # [ 82.527147] niks3-server[1047]: 2026/09/20 09:06:20 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst5533lb # [ 82.529849] niks3-server[1047]: 2026/09/20 09:06:20 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5534lb # [ 82.534418] niks3-server[1047]: 2026/09/20 09:06:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5535lb # [ 82.536766] niks3-server[1047]: 2026/09/20 09:06:20 INFO Signed narinfos id=30 count=15536worker1 # [ 82.583260] nix-grpc-daemon[1121]: time=2026-09-20T09:06:19.842Z level=INFO msg="Uploading 1 narinfos"5537lb # [ 82.539275] niks3-server[1047]: 2026/09/20 09:06:20 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls5538lb # [ 82.544805] niks3-server[1047]: 2026/09/20 09:06:20 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5539lb # [ 82.548496] niks3-server[1047]: 2026/09/20 09:06:20 INFO Completed upload id=305540worker1 # [ 82.594813] nix-grpc-daemon[1121]: time=2026-09-20T09:06:19.854Z level=INFO msg="Upload complete. (220ms)"5541lb # [ 82.550065] niks3-server[1047]: 2026/09/20 09:06:20 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo5542worker1 # [ 82.597397] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58376 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=15543worker1 # [ 82.604594] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:19Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=05544worker1 # [ 82.606561] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:19Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=15545worker1 # [ 82.610307] nix-daemon[842]: reaped child process 1180, status = succeeded5546worker1 # [ 82.648132] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:583765547worker1 # [ 82.657539] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:583765548worker1 # [ 82.666725] nix-grpc-daemon[1121]: time=2026-09-20T09:06:19.925Z level=INFO msg="All 2 paths already cached"5549client # copying 1 paths to 'lb:50051'...5550client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...5551worker1 # [ 82.672291] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:583765552worker1 # [ 82.680937] nix-daemon[842]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)5553worker1 # [ 82.841672] nix-daemon[842]: accepted connection from pid 1206, user nix-grpc-daemon (trusted)5554worker1 # [ 82.853931] nix-daemon[842]: reaped child process 1209, status = succeeded5555worker1 # [ 83.013257] nix-daemon[842]: accepted connection from pid 1211, user nix-grpc-daemon (trusted)5556worker1 # [ 83.025154] nix-daemon[842]: reaped child process 1214, status = succeeded5557lb # [ 82.995323] niks3-server[1047]: 2026/09/20 09:06:20 INFO Received uploads request method=POST path=/api/pending_closures5558worker1 # [ 83.046379] nix-grpc-daemon[1121]: time=2026-09-20T09:06:20.305Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5559worker1 # [ 83.048254] nix-grpc-daemon[1121]: time=2026-09-20T09:06:20.307Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"5560lb # [ 83.011237] niks3-server[1047]: 2026/09/20 09:06:20 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst5561lb # [ 83.014322] niks3-server[1047]: 2026/09/20 09:06:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5562lb # [ 83.016977] niks3-server[1047]: 2026/09/20 09:06:20 INFO Signed narinfos id=31 count=15563worker1 # [ 83.063290] nix-grpc-daemon[1121]: time=2026-09-20T09:06:20.322Z level=INFO msg="Uploading 1 narinfos"5564lb # [ 83.019958] niks3-server[1047]: 2026/09/20 09:06:20 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls5565lb # [ 83.024677] niks3-server[1047]: 2026/09/20 09:06:20 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5566lb # [ 83.027583] niks3-server[1047]: 2026/09/20 09:06:20 INFO Completed upload id=315567worker1 # [ 83.075227] nix-grpc-daemon[1121]: time=2026-09-20T09:06:20.334Z level=INFO msg="Upload complete. (387ms)"5568lb # [ 83.029701] niks3-server[1047]: 2026/09/20 09:06:20 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo5569worker1 # [ 83.077217] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58376 duration_s=0 paths=1 nar_bytes_in=7685570worker1 # [ 83.079983] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:583765571worker1 # [ 83.086248] nix-daemon[842]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)5572worker1 # [ 83.088993] nix-daemon[842]: reaped child process 1204, status = succeeded5573client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...5574client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5575worker1 # [ 83.094076] nix-daemon[842]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)5576client # farm-top> LOG-farm-top-on-standby5577worker1 # [ 84.107694] nix-daemon[842]: reaped child process 1218, status = succeeded5578worker1 # [ 84.254901] nix-daemon[842]: accepted connection from pid 1233, user nix-grpc-daemon (trusted)5579worker1 # [ 84.267130] nix-daemon[842]: reaped child process 1236, status = succeeded5580lb # [ 84.237891] niks3-server[1047]: 2026/09/20 09:06:21 INFO Received uploads request method=POST path=/api/pending_closures5581worker1 # [ 84.293848] nix-grpc-daemon[1121]: time=2026-09-20T09:06:21.552Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5582worker1 # [ 84.296717] nix-grpc-daemon[1121]: time=2026-09-20T09:06:21.554Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"5583lb # [ 84.279656] niks3-server[1047]: 2026/09/20 09:06:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5584lb # [ 84.286157] niks3-server[1047]: 2026/09/20 09:06:21 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst5585lb # [ 84.294769] niks3-server[1047]: 2026/09/20 09:06:21 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5586lb # [ 84.301782] niks3-server[1047]: 2026/09/20 09:06:22 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls5587lb # [ 84.308570] niks3-server[1047]: 2026/09/20 09:06:22 INFO Signed narinfos id=32 count=15588worker1 # [ 84.355272] nix-grpc-daemon[1121]: time=2026-09-20T09:06:21.614Z level=INFO msg="Uploading 1 narinfos"5589lb # [ 84.316661] niks3-server[1047]: 2026/09/20 09:06:22 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5590lb # [ 84.322301] niks3-server[1047]: 2026/09/20 09:06:22 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo5591lb # [ 84.326155] niks3-server[1047]: 2026/09/20 09:06:22 INFO Completed upload id=325592worker1 # [ 84.372756] nix-grpc-daemon[1121]: time=2026-09-20T09:06:21.631Z level=INFO msg="Upload complete. (271ms)"5593worker1 # [ 84.380379] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58376 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=15594worker1 # [ 84.393258] nix-daemon[842]: reaped child process 1216, status = succeeded5595client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&2, in 6.90 seconds)5596client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath5597client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath, in 0.17 seconds)5598worker2 # [ 84.584512] systemd[1]: Listening on Nix worker-protocol over gRPC.5599worker2 # [ 84.592242] systemd[1]: Starting Nix worker-protocol over gRPC...5600worker2 # [ 84.779038] nix-grpc-daemon[1343]: ts=2026-09-20T09:06:22Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5601worker2 # [ 84.783128] nix-grpc-daemon[1343]: ts=2026-09-20T09:06:22Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15602worker2 # [ 84.789833] nix-grpc-daemon[1343]: ts=2026-09-20T09:06:23Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5603worker2 # [ 84.791375] systemd[1]: Started Nix worker-protocol over gRPC.5604lb: must succeed: curl -sf localhost:9901/clusters5605worker2 # [ 84.822944] nix-grpc-daemon[1343]: ts=2026-09-20T09:06:23Z level=info event=scheduler_connected addr=lb:500515606worker1 # [ 84.882209] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:22Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05607lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5608worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5609worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.07 seconds)5610lb: must succeed: curl -sf localhost:9901/clusters5611lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5612lb: must succeed: curl -sf localhost:9901/clusters5613lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5614worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5615worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.05 seconds)5616(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.49 seconds)5617subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5618client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs5619client # [ 85.216123] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs.5620client # Running as unit: rs.service; invocation ID: 8bcda22632d645088a3fb21168e094595621client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs, in 0.07 seconds)5622worker2 # [ 85.369881] nix-grpc-daemon[1343]: ts=2026-09-20T09:06:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:504225623worker2 # [ 85.381315] nix-daemon[844]: accepted connection from pid 1343, user nix-grpc-daemon (trusted)5624client # [ 85.445840] nix[1182]: don't know how to build these paths:5625client # [ 85.446732] nix[1182]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5626worker1 # [ 85.459517] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:22Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=05627worker1 # [ 85.464743] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:22Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=15628worker1 # [ 85.508305] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:583765629worker1 # [ 85.522437] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:583765630client # [ 85.529318] nix[1182]: copying 1 paths to 'lb:50051'...5631worker1 # [ 85.534558] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:583765632client # [ 85.532169] nix[1182]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...5633worker1 # [ 85.545258] nix-daemon[842]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)5634worker1 # [ 85.737607] nix-daemon[842]: accepted connection from pid 1251, user nix-grpc-daemon (trusted)5635worker1 # [ 85.749658] nix-daemon[842]: reaped child process 1254, status = succeeded5636worker1 # [ 85.909582] nix-daemon[842]: accepted connection from pid 1256, user nix-grpc-daemon (trusted)5637worker1 # [ 85.921369] nix-daemon[842]: reaped child process 1259, status = succeeded5638lb # [ 85.892493] niks3-server[1047]: 2026/09/20 09:06:23 INFO Received uploads request method=POST path=/api/pending_closures5639worker1 # [ 85.945155] nix-grpc-daemon[1121]: time=2026-09-20T09:06:23.203Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5640worker1 # [ 85.946564] nix-grpc-daemon[1121]: time=2026-09-20T09:06:23.203Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"5641lb # [ 85.909433] niks3-server[1047]: 2026/09/20 09:06:23 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst5642lb # [ 85.918148] niks3-server[1047]: 2026/09/20 09:06:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5643worker1 # [ 85.968849] nix-grpc-daemon[1121]: time=2026-09-20T09:06:23.227Z level=INFO msg="Uploading 1 narinfos"5644lb # [ 85.923187] niks3-server[1047]: 2026/09/20 09:06:23 INFO Signed narinfos id=33 count=15645lb # [ 85.929128] niks3-server[1047]: 2026/09/20 09:06:23 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls5646lb # [ 85.938143] niks3-server[1047]: 2026/09/20 09:06:23 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5647lb # [ 85.941423] niks3-server[1047]: 2026/09/20 09:06:23 INFO Completed upload id=335648worker1 # [ 85.987738] nix-grpc-daemon[1121]: time=2026-09-20T09:06:23.246Z level=INFO msg="Upload complete. (431ms)"5649lb # [ 85.943042] niks3-server[1047]: 2026/09/20 09:06:23 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo5650worker1 # [ 85.991072] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58376 duration_s=0 paths=1 nar_bytes_in=4165651worker1 # [ 85.993756] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:583765652worker1 # [ 85.998537] nix-daemon[842]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)5653worker1 # [ 86.001529] nix-daemon[842]: reaped child process 1249, status = succeeded5654worker1 # [ 86.006971] nix-daemon[842]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)5655client # [ 86.002166] nix[1182]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...5656lb # [ 86.643497] envoy[918]: conn peer=[2001:db8:1::1]:40184 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9590 tx=3834 ms=66715657lb: must succeed: curl -sf localhost:9901/clusters5658lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5659lb: must succeed: curl -sf localhost:9901/clusters5660lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5661lb: must succeed: curl -sf localhost:9901/clusters5662lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5663lb: must succeed: curl -sf localhost:9901/clusters5664lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5665worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5666worker2 # [ 88.034849] systemd-journald[286]: Received client request to rotate journal, rotating.5667worker2 # [ 88.053241] systemd-journald[286]: Vacuuming done, freed 0B of archived journals from /var/log/journal/c242a32efaa6445b8f431d3f2ec7d2c9.5668worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.07 seconds)5669worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5670worker1 # [ 88.023532] systemd[1]: Stopping Nix worker-protocol over gRPC...5671worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)5672client: must succeed: sleep 5.05673worker1 # [ 88.519703] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:25Z level=info event=draining5674client: (finished: must succeed: sleep 5.0, in 5.04 seconds)5675worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5676worker1 # [ 93.438484] nix-daemon[842]: reaped child process 1263, status = succeeded5677worker1 # [ 93.608438] nix-daemon[842]: accepted connection from pid 1303, user nix-grpc-daemon (trusted)5678worker1 # [ 93.619137] nix-daemon[842]: reaped child process 1306, status = succeeded5679lb # [ 93.592352] niks3-server[1047]: 2026/09/20 09:06:31 INFO Received uploads request method=POST path=/api/pending_closures5680worker1 # [ 93.644080] nix-grpc-daemon[1121]: time=2026-09-20T09:06:30.903Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5681worker1 # [ 93.645600] nix-grpc-daemon[1121]: time=2026-09-20T09:06:30.904Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"5682lb # [ 93.619464] niks3-server[1047]: 2026/09/20 09:06:31 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5683lb # [ 93.625536] niks3-server[1047]: 2026/09/20 09:06:31 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5684lb # [ 93.627282] niks3-server[1047]: 2026/09/20 09:06:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5685lb # [ 93.629899] niks3-server[1047]: 2026/09/20 09:06:31 INFO Signed narinfos id=34 count=15686worker1 # [ 93.677818] nix-grpc-daemon[1121]: time=2026-09-20T09:06:30.936Z level=INFO msg="Uploading 1 narinfos"5687lb # [ 93.634494] niks3-server[1047]: 2026/09/20 09:06:31 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls5688lb # [ 93.639816] niks3-server[1047]: 2026/09/20 09:06:31 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5689lb # [ 93.644044] niks3-server[1047]: 2026/09/20 09:06:31 INFO Completed upload id=345690worker1 # [ 93.690359] nix-grpc-daemon[1121]: time=2026-09-20T09:06:30.949Z level=INFO msg="Upload complete. (269ms)"5691lb # [ 93.645843] niks3-server[1047]: 2026/09/20 09:06:31 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo5692worker1 # [ 93.692666] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58376 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=15693worker1 # [ 93.700184] nix-daemon[842]: reaped child process 1261, status = succeeded5694client # [ 93.719580] systemd[1]: rs.service: Deactivated successfully.5695client # [ 93.721259] systemd[1]: rs.service: Consumed 105ms CPU time over 8.505s wall clock time, 39.1M memory peak, 8.6K incoming IP traffic, 9K outgoing IP traffic.5696worker1 # [ 94.521370] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:31Z level=info event=signal_exit5697worker1 # [ 94.746320] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:32Z level=info event=scheduler_yield5698lb # [ 94.701987] niks3-server[1047]: 2026/09/20 09:06:32 INFO lead: released remote=192.168.1.3:590145699worker1 # [ 94.749935] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:32Z level=info event=scheduler_restarting addr=lb:500515700worker2 # [ 94.694724] nix-grpc-daemon[1343]: ts=2026-09-20T09:06:32Z level=info event=scheduler_restarting addr=lb:500515701worker1 # [ 94.754181] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:32Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:583665702worker1 # [ 94.757508] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:32Z level=info event=worker_gone addr=192.168.1.4:500515703worker1 # [ 94.760393] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:32Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:583665704worker1 # [ 94.763605] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:32Z level=info event=worker_gone addr=192.168.1.3:500515705lb # [ 94.810819] niks3-server[1047]: 2026/09/20 09:06:32 INFO lead: acquired remote=192.168.1.4:363685706worker2 # [ 94.801402] nix-grpc-daemon[1343]: ts=2026-09-20T09:06:33Z level=info event=scheduler_take_over5707worker1 # [ 94.950814] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:32Z level=info event=scheduler_connected addr=lb:500515708worker2 # [ 94.895177] nix-grpc-daemon[1343]: ts=2026-09-20T09:06:33Z level=info event=scheduler_connected addr=lb:500515709worker2 # [ 94.899887] nix-grpc-daemon[1343]: ts=2026-09-20T09:06:33Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5710worker1 # [ 94.956418] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:32Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5711worker2 # [ 95.100854] nix-grpc-daemon[1343]: ts=2026-09-20T09:06:33Z level=info event=scheduler_connected addr=lb:500515712worker1 # [ 95.157232] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:32Z level=info event=scheduler_connected addr=lb:500515713worker2 # [ 95.109454] nix-grpc-daemon[1343]: ts=2026-09-20T09:06:33Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=0 running=05714worker2 # [ 95.116583] nix-grpc-daemon[1343]: ts=2026-09-20T09:06:33Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5715worker2 # [ 95.317756] nix-grpc-daemon[1343]: ts=2026-09-20T09:06:33Z level=info event=scheduler_connected addr=lb:500515716worker2 # [ 95.322688] nix-grpc-daemon[1343]: ts=2026-09-20T09:06:33Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05717worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.15 seconds)5718client: waiting for success: ! systemctl is-active rs5719client: (finished: waiting for success: ! systemctl is-active rs, in 0.05 seconds)5720client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5721client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.04 seconds)5722client: must fail: journalctl -u rs | grep -q 'reconnecting'5723client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.02 seconds)5724lb: must succeed: curl -sf localhost:9901/clusters5725lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5726worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5727worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.07 seconds)5728(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 10.54 seconds)5729subtest: scheduler restart mid-build: build finishes, no double build5730client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid5731worker2 # [ 95.696028] nix-grpc-daemon[1343]: ts=2026-09-20T09:06:33Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:503565732worker2 # [ 95.697974] nix-grpc-daemon[1343]: ts=2026-09-20T09:06:33Z level=info event=worker_gone addr=192.168.1.3:500515733worker1 # [ 95.754475] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:33Z level=info event=scheduler_restarting5734client # [ 95.752395] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid.5735worker1 # [ 95.767669] nix-grpc-daemon[1047]: ts=2026-09-20T09:06:33Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5736client # Running as unit: mid.service; invocation ID: 60a3b5b812004ed4a0b82fb69b4517345737client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid, in 0.07 seconds)5738worker1 # [ 95.784509] nix-daemon[842]: reaped child process 1112, status = succeeded5739worker2 # [ 95.901567] nix-grpc-daemon[1343]: ts=2026-09-20T09:06:34Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:504225740client # [ 95.960345] nix[1218]: don't know how to build these paths:5741client # [ 95.961394] nix[1218]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5742worker1 # [ 95.966705] nix-grpc-daemon[1047]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5743worker1 # [ 95.970170] nix-grpc-daemon[1047]: E0000 00:00:1789895193.225406 1090 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 11, started true, shutdown true5744worker1 # [ 96.118595] nix-grpc-daemon[1047]: E0000 00:00:1789895193.377526 1090 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5745worker1 # [ 96.123473] nix-grpc-daemon[1047]: E0000 00:00:1789895193.377640 1090 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 8, started true, shutdown true5746lb # [ 96.643157] envoy[918]: conn peer=[2001:db8:1::1]:39982 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2960 tx=1345 ms=82825747lb # [ 96.644803] envoy[918]: conn peer=[2001:db8:1::3]:46762 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=3396 tx=993 ms=242165748client # [ 97.027907] nix[1218]: scheduler at lb:50051: upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, reconnecting5749worker1 # [ 97.049581] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5750worker1 # [ 97.053279] systemd[1]: Stopped Nix worker-protocol over gRPC.5751worker1 # [ 97.055446] systemd[1]: nix-grpc-daemon.service: Consumed 1.270s CPU time over 25.720s wall clock time, 35.6M memory peak, 45.1K incoming IP traffic, 84.8K outgoing IP traffic.5752worker1 # [ 97.070513] systemd[1]: Starting Nix worker-protocol over gRPC...5753worker1 # [ 97.220675] nix-grpc-daemon[1321]: ts=2026-09-20T09:06:34Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5754worker1 # [ 97.224500] nix-grpc-daemon[1321]: ts=2026-09-20T09:06:34Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15755worker1 # [ 97.230522] nix-grpc-daemon[1321]: ts=2026-09-20T09:06:34Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5756worker1 # [ 97.232112] systemd[1]: Started Nix worker-protocol over gRPC.5757worker1 # [ 97.254639] nix-grpc-daemon[1321]: ts=2026-09-20T09:06:34Z level=info event=scheduler_connected addr=lb:500515758worker2 # [ 97.201811] nix-grpc-daemon[1343]: ts=2026-09-20T09:06:35Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05759worker2 # [ 97.484838] nix-grpc-daemon[1343]: ts=2026-09-20T09:06:35Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05760worker2 # [ 97.490528] nix-grpc-daemon[1343]: ts=2026-09-20T09:06:35Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=15761worker2 # [ 97.529569] nix-grpc-daemon[1343]: ts=2026-09-20T09:06:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:504225762worker2 # [ 97.541812] nix-grpc-daemon[1343]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 1 ms (attempt 1/5)5763worker2 # [ 97.548380] nix-grpc-daemon[1343]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 183 ms (attempt 2/5)5764worker2 # [ 97.737413] nix-grpc-daemon[1343]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 192 ms (attempt 3/5)5765worker2 # [ 97.935562] nix-grpc-daemon[1343]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 61 ms (attempt 4/5)5766worker2 # [ 98.003074] nix-grpc-daemon[1343]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5767worker2 # [ 98.020818] nix-grpc-daemon[1343]: ts=2026-09-20T09:06:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:504225768client # [ 98.083671] nix[1218]: copying 1 paths to 'lb:50051'...5769client # [ 98.086960] nix[1218]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...5770worker2 # [ 98.035539] nix-grpc-daemon[1343]: ts=2026-09-20T09:06:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:504225771worker2 # [ 98.047402] nix-daemon[844]: accepted connection from pid 1343, user nix-grpc-daemon (trusted)5772worker2 # [ 98.074466] nix-grpc-daemon[1343]: ts=2026-09-20T09:06:36Z level=info event=niks3_push_started pid=14695773worker2 # [ 98.416048] nix-daemon[844]: accepted connection from pid 1483, user nix-grpc-daemon (trusted)5774worker2 # [ 98.428544] nix-daemon[844]: reaped child process 1486, status = succeeded5775worker2 # [ 98.601347] nix-daemon[844]: accepted connection from pid 1488, user nix-grpc-daemon (trusted)5776worker2 # [ 98.615181] nix-daemon[844]: reaped child process 1491, status = succeeded5777lb # [ 98.644653] niks3-server[1047]: 2026/09/20 09:06:36 INFO Received uploads request method=POST path=/api/pending_closures5778worker2 # [ 98.645275] nix-grpc-daemon[1469]: time=2026-09-20T09:06:36.857Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5779worker2 # [ 98.646701] nix-grpc-daemon[1469]: time=2026-09-20T09:06:36.857Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"5780lb # [ 98.681671] niks3-server[1047]: 2026/09/20 09:06:36 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst5781lb # [ 98.688316] niks3-server[1047]: 2026/09/20 09:06:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5782lb # [ 98.690579] niks3-server[1047]: 2026/09/20 09:06:36 INFO Signed narinfos id=35 count=15783worker2 # [ 98.681268] nix-grpc-daemon[1469]: time=2026-09-20T09:06:36.894Z level=INFO msg="Uploading 1 narinfos"5784lb # [ 98.694685] niks3-server[1047]: 2026/09/20 09:06:36 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls5785lb # [ 98.699059] niks3-server[1047]: 2026/09/20 09:06:36 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5786lb # [ 98.704023] niks3-server[1047]: 2026/09/20 09:06:36 INFO Completed upload id=355787worker2 # [ 98.694047] nix-grpc-daemon[1469]: time=2026-09-20T09:06:36.907Z level=INFO msg="Upload complete. (444ms)"5788lb # [ 98.705500] niks3-server[1047]: 2026/09/20 09:06:36 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo5789worker2 # [ 98.696674] nix-grpc-daemon[1343]: ts=2026-09-20T09:06:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50422 duration_s=0 paths=1 nar_bytes_in=4245790worker2 # [ 98.699924] nix-grpc-daemon[1343]: ts=2026-09-20T09:06:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:504225791worker2 # [ 98.706894] nix-daemon[844]: accepted connection from pid 1343, user nix-grpc-daemon (trusted)5792worker2 # [ 98.713195] nix-daemon[844]: reaped child process 1464, status = succeeded5793worker2 # [ 98.716372] nix-daemon[844]: accepted connection from pid 1343, user nix-grpc-daemon (trusted)5794client # [ 98.767603] nix[1218]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5795lb: must succeed: curl -sf localhost:9901/clusters5796lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5797lb: must succeed: curl -sf localhost:9901/clusters5798lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5799worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5800worker2 # [ 100.639335] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1343 (nix-grpc-daemon) on client request.5801worker2 # [ 100.642417] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5802lb # [ 100.661104] niks3-server[1047]: 2026/09/20 09:06:38 INFO lead: released remote=192.168.1.4:363685803worker1 # [ 100.704975] nix-grpc-daemon[1321]: ts=2026-09-20T09:06:37Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5804client # [ 100.701627] nix[1218]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5805client # [ 100.707338] nix[1218]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5806worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.07 seconds)5807client: must succeed: sleep 5.05808worker2 # [ 100.679205] nix-daemon[844]: reaped child process 1493, status = succeeded5809worker2 # [ 100.680792] nix-daemon[844]: reaped child process 1401, status = succeeded5810worker2 # [ 100.682176] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5811worker2 # [ 100.683812] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5812worker2 # [ 100.686211] systemd[1]: nix-grpc-daemon.service: Consumed 458ms CPU time over 16.086s wall clock time, 33.2M memory peak, 13K incoming IP traffic, 18.2K outgoing IP traffic.5813worker2 # [ 100.693266] nix-daemon[1495]: unexpected Nix daemon error: error: interrupted by the user5814worker2 # [ 100.698447] nix-daemon[844]: reaped child process 1495, status = failed with exit code 15815worker2 # [ 100.825360] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.5816worker2 # [ 100.829566] systemd[1]: Starting Nix worker-protocol over gRPC...5817worker1 # [ 100.905394] nix-grpc-daemon[1321]: ts=2026-09-20T09:06:38Z level=info event=scheduler_connected addr=lb:500515818worker2 # [ 100.969224] nix-grpc-daemon[1531]: ts=2026-09-20T09:06:39Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5819worker2 # [ 100.971879] nix-grpc-daemon[1531]: ts=2026-09-20T09:06:39Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15820worker2 # [ 100.977727] nix-grpc-daemon[1531]: ts=2026-09-20T09:06:39Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5821worker2 # [ 100.981143] systemd[1]: Started Nix worker-protocol over gRPC.5822lb # [ 100.998096] niks3-server[1047]: 2026/09/20 09:06:38 INFO lead: acquired remote=192.168.1.4:546445823worker2 # [ 100.992305] nix-grpc-daemon[1531]: ts=2026-09-20T09:06:39Z level=info event=scheduler_take_over5824worker1 # [ 101.051693] nix-grpc-daemon[1321]: ts=2026-09-20T09:06:38Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5825worker2 # [ 101.007836] nix-grpc-daemon[1531]: ts=2026-09-20T09:06:39Z level=info event=scheduler_connected addr=lb:500515826worker2 # [ 101.009531] nix-grpc-daemon[1531]: ts=2026-09-20T09:06:39Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05827worker2 # [ 101.164639] nix-grpc-daemon[1531]: ts=2026-09-20T09:06:39Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05828worker2 # [ 101.169975] nix-grpc-daemon[1531]: ts=2026-09-20T09:06:39Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=15829worker1 # [ 101.252580] nix-grpc-daemon[1321]: ts=2026-09-20T09:06:38Z level=info event=scheduler_connected addr=lb:500515830worker2 # [ 101.197690] nix-grpc-daemon[1531]: ts=2026-09-20T09:06:39Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05831worker2 # [ 101.211148] nix-grpc-daemon[1531]: ts=2026-09-20T09:06:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:503825832worker2 # [ 101.221272] nix-daemon[844]: accepted connection from pid 1531, user nix-grpc-daemon (trusted)5833worker2 # [ 101.235916] nix-daemon[844]: accepted connection from pid 1531, user nix-grpc-daemon (trusted)5834client # [ 101.299524] nix[1218]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5835worker2 # [ 101.248679] nix-daemon[844]: accepted connection from pid 1531, user nix-grpc-daemon (trusted)5836client: (finished: must succeed: sleep 5.0, in 5.04 seconds)5837client: waiting for success: ! systemctl is-active mid5838worker2 # [ 106.067278] nix-grpc-daemon[1531]: ts=2026-09-20T09:06:44Z level=info event=niks3_push_started pid=16105839worker2 # [ 106.079820] nix-daemon[844]: reaped child process 1586, status = succeeded5840worker2 # [ 106.408310] nix-daemon[844]: accepted connection from pid 1623, user nix-grpc-daemon (trusted)5841worker2 # [ 106.422169] nix-daemon[844]: reaped child process 1626, status = succeeded5842lb # [ 106.453682] niks3-server[1047]: 2026/09/20 09:06:44 INFO Received uploads request method=POST path=/api/pending_closures5843worker2 # [ 106.450800] nix-grpc-daemon[1610]: time=2026-09-20T09:06:44.663Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5844worker2 # [ 106.452296] nix-grpc-daemon[1610]: time=2026-09-20T09:06:44.665Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"5845lb # [ 106.503923] niks3-server[1047]: 2026/09/20 09:06:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5846lb # [ 106.511828] niks3-server[1047]: 2026/09/20 09:06:44 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5847lb # [ 106.519117] niks3-server[1047]: 2026/09/20 09:06:44 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5848lb # [ 106.531240] niks3-server[1047]: 2026/09/20 09:06:44 INFO Signed narinfos id=36 count=15849worker2 # [ 106.522100] nix-grpc-daemon[1610]: time=2026-09-20T09:06:44.734Z level=INFO msg="Uploading 1 narinfos"5850lb # [ 106.539589] niks3-server[1047]: 2026/09/20 09:06:44 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls5851lb # [ 106.541048] niks3-server[1047]: 2026/09/20 09:06:44 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5852lb # [ 106.546270] niks3-server[1047]: 2026/09/20 09:06:44 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo5853lb # [ 106.551128] niks3-server[1047]: 2026/09/20 09:06:44 INFO Completed upload id=365854worker2 # [ 106.541482] nix-grpc-daemon[1610]: time=2026-09-20T09:06:44.754Z level=INFO msg="Upload complete. (305ms)"5855worker2 # [ 106.548499] nix-grpc-daemon[1531]: ts=2026-09-20T09:06:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50382 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=15856worker2 # [ 106.563891] nix-daemon[844]: reaped child process 1584, status = succeeded5857client # [ 106.643785] systemd[1]: mid.service: Deactivated successfully.5858client # [ 106.647377] systemd[1]: mid.service: Consumed 128ms CPU time over 10.894s wall clock time, 39.3M memory peak, 10K incoming IP traffic, 11K outgoing IP traffic.5859lb # [ 106.643775] envoy[918]: conn peer=[2001:db8:1::4]:58426 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1172 tx=417 ms=158315860lb # [ 106.648254] envoy[918]: conn peer=[2001:db8:1::1]:35072 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4033 tx=1786 ms=106675861client: (finished: waiting for success: ! systemctl is-active mid, in 1.10 seconds)5862client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5863client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.03 seconds)5864(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 11.45 seconds)5865subtest: interrupting the client stops the build on the worker5866client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr5867client # [ 107.184375] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr.5868client # Running as unit: intr.service; invocation ID: e813ee421d784a20bc9b71244811bb615869client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr, in 0.03 seconds)5870worker1 # [ 107.370738] nix-grpc-daemon[1321]: ts=2026-09-20T09:06:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:567865871worker1 # [ 107.381637] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)5872client # [ 107.389258] nix[1254]: don't know how to build these paths:5873client # [ 107.390166] nix[1254]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv5874worker2 # [ 107.345944] nix-grpc-daemon[1531]: ts=2026-09-20T09:06:45Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=05875worker2 # [ 107.350859] nix-grpc-daemon[1531]: ts=2026-09-20T09:06:45Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.3:50051 clients=15876worker1 # [ 107.450685] nix-grpc-daemon[1321]: ts=2026-09-20T09:06:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567865877worker1 # [ 107.461889] nix-grpc-daemon[1321]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 49 ms (attempt 1/5)5878worker1 # [ 107.516666] nix-grpc-daemon[1321]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 150 ms (attempt 2/5)5879worker1 # [ 107.672718] nix-grpc-daemon[1321]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 92 ms (attempt 3/5)5880worker1 # [ 107.770638] nix-grpc-daemon[1321]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 71 ms (attempt 4/5)5881worker1 # [ 107.847880] nix-grpc-daemon[1321]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5882worker1 # [ 107.862993] nix-grpc-daemon[1321]: ts=2026-09-20T09:06:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:567865883client # [ 107.867127] nix[1254]: copying 1 paths to 'lb:50051'...5884client # [ 107.871140] nix[1254]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...5885worker1 # [ 107.875318] nix-grpc-daemon[1321]: ts=2026-09-20T09:06:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:567865886worker1 # [ 107.885156] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)5887worker1 # [ 107.903826] nix-grpc-daemon[1321]: ts=2026-09-20T09:06:45Z level=info event=niks3_push_started pid=14055888worker1 # [ 108.206086] nix-daemon[842]: accepted connection from pid 1418, user nix-grpc-daemon (trusted)5889worker1 # [ 108.216508] nix-daemon[842]: reaped child process 1421, status = succeeded5890worker1 # [ 108.389594] nix-daemon[842]: accepted connection from pid 1423, user nix-grpc-daemon (trusted)5891worker1 # [ 108.401703] nix-daemon[842]: reaped child process 1430, status = succeeded5892lb # [ 108.371185] niks3-server[1047]: 2026/09/20 09:06:46 INFO Received uploads request method=POST path=/api/pending_closures5893worker1 # [ 108.428642] nix-grpc-daemon[1405]: time=2026-09-20T09:06:45.687Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5894worker1 # [ 108.431731] nix-grpc-daemon[1405]: time=2026-09-20T09:06:45.689Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"5895lb # [ 108.407698] niks3-server[1047]: 2026/09/20 09:06:46 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst5896lb # [ 108.416796] niks3-server[1047]: 2026/09/20 09:06:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5897lb # [ 108.422237] niks3-server[1047]: 2026/09/20 09:06:46 INFO Signed narinfos id=37 count=15898worker1 # [ 108.471893] nix-grpc-daemon[1405]: time=2026-09-20T09:06:45.730Z level=INFO msg="Uploading 1 narinfos"5899lb # [ 108.435106] niks3-server[1047]: 2026/09/20 09:06:46 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls5900lb # [ 108.442837] niks3-server[1047]: 2026/09/20 09:06:46 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5901worker1 # [ 108.496521] nix-grpc-daemon[1405]: time=2026-09-20T09:06:45.755Z level=INFO msg="Upload complete. (428ms)"5902lb # [ 108.450343] niks3-server[1047]: 2026/09/20 09:06:46 INFO Completed upload id=375903lb # [ 108.453501] niks3-server[1047]: 2026/09/20 09:06:46 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo5904worker1 # [ 108.499529] nix-grpc-daemon[1321]: ts=2026-09-20T09:06:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56786 duration_s=0 paths=1 nar_bytes_in=4245905worker1 # [ 108.505736] nix-grpc-daemon[1321]: ts=2026-09-20T09:06:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567865906worker1 # [ 108.515061] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)5907worker1 # [ 108.518943] nix-daemon[842]: reaped child process 1403, status = succeeded5908worker1 # [ 108.523973] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)5909client # [ 108.519941] nix[1254]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...5910client: must succeed: systemctl kill -s INT intr5911client # [ 109.587901] systemd[1]: intr.service: Sent signal SIGINT to main process 1254 (nix) on client request.5912worker1 # [ 109.599283] nix-grpc-daemon[1321]: ts=2026-09-20T09:06:46Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv5913client # [ 109.595326] nix[1254]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5914client: (finished: must succeed: systemctl kill -s INT intr, in 0.05 seconds)5915client # [ 109.601617] nix[1254]: error: interrupted by the user5916client # [ 109.627680] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5917client # [ 109.629341] systemd[1]: intr.service: Failed with result 'exit-code'.5918client # [ 109.631080] systemd[1]: intr.service: Consumed 102ms CPU time over 2.444s wall clock time, 39.3M memory peak, 7.7K incoming IP traffic, 8.7K outgoing IP traffic.5919worker1 # [ 109.740504] nix-daemon[842]: reaped child process 1432, status = succeeded5920worker1 # [ 109.744238] nix-daemon[1434]: unexpected Nix daemon error: error: interrupted by the user5921worker1 # [ 109.747337] nix-daemon[842]: reaped child process 1434, status = failed with exit code 15922(finished: subtest: interrupting the client stops the build on the worker, in 3.63 seconds)5923subtest: a deploy mid-build drains: the build finishes, the new generation takes over5924client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw5925client # [ 110.823395] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw.5926client # Running as unit: sw.service; invocation ID: f614aac939c64cae8c6cdcbcbd9998735927client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw, in 0.04 seconds)5928client # [ 111.008039] nix[1275]: don't know how to build these paths:5929client # [ 111.008916] nix[1275]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv5930worker2 # [ 110.954865] nix-grpc-daemon[1531]: ts=2026-09-20T09:06:49Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:503825931worker2 # [ 110.963057] nix-grpc-daemon[1531]: ts=2026-09-20T09:06:49Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=05932worker2 # [ 110.965098] nix-grpc-daemon[1531]: ts=2026-09-20T09:06:49Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.4:50051 clients=15933worker2 # [ 111.007077] nix-grpc-daemon[1531]: ts=2026-09-20T09:06:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:503825934worker2 # [ 111.014468] nix-grpc-daemon[1531]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 39 ms (attempt 1/5)5935worker2 # [ 111.056020] nix-grpc-daemon[1531]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 64 ms (attempt 2/5)5936worker2 # [ 111.122637] nix-grpc-daemon[1531]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 383 ms (attempt 3/5)5937worker2 # [ 111.508580] nix-grpc-daemon[1531]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 429 ms (attempt 4/5)5938worker2 # [ 111.938212] nix-grpc-daemon[1531]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5939worker2 # [ 111.961929] nix-grpc-daemon[1531]: ts=2026-09-20T09:06:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:503825940client # [ 112.021273] nix[1275]: copying 1 paths to 'lb:50051'...5941client # [ 112.022596] nix[1275]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...5942worker2 # [ 111.970412] nix-grpc-daemon[1531]: ts=2026-09-20T09:06:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:503825943worker2 # [ 111.980858] nix-daemon[844]: accepted connection from pid 1531, user nix-grpc-daemon (trusted)5944worker2 # [ 112.165297] nix-daemon[844]: accepted connection from pid 1665, user nix-grpc-daemon (trusted)5945worker2 # [ 112.179473] nix-daemon[844]: reaped child process 1668, status = succeeded5946worker2 # [ 112.353777] nix-daemon[844]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)5947worker2 # [ 112.368196] nix-daemon[844]: reaped child process 1673, status = succeeded5948lb # [ 112.392985] niks3-server[1047]: 2026/09/20 09:06:50 INFO Received uploads request method=POST path=/api/pending_closures5949worker2 # [ 112.395489] nix-grpc-daemon[1610]: time=2026-09-20T09:06:50.608Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5950worker2 # [ 112.397655] nix-grpc-daemon[1610]: time=2026-09-20T09:06:50.610Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"5951lb # [ 112.431463] niks3-server[1047]: 2026/09/20 09:06:50 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst5952lb # [ 112.434865] niks3-server[1047]: 2026/09/20 09:06:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5953lb # [ 112.439497] niks3-server[1047]: 2026/09/20 09:06:50 INFO Signed narinfos id=38 count=15954worker2 # [ 112.433310] nix-grpc-daemon[1610]: time=2026-09-20T09:06:50.646Z level=INFO msg="Uploading 1 narinfos"5955lb # [ 112.446090] niks3-server[1047]: 2026/09/20 09:06:50 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls5956lb # [ 112.456465] niks3-server[1047]: 2026/09/20 09:06:50 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5957lb # [ 112.460804] niks3-server[1047]: 2026/09/20 09:06:50 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo5958lb # [ 112.463622] niks3-server[1047]: 2026/09/20 09:06:50 INFO Completed upload id=385959worker2 # [ 112.453815] nix-grpc-daemon[1610]: time=2026-09-20T09:06:50.667Z level=INFO msg="Upload complete. (452ms)"5960worker2 # [ 112.458127] nix-grpc-daemon[1531]: ts=2026-09-20T09:06:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50382 duration_s=0 paths=1 nar_bytes_in=4165961worker2 # [ 112.462730] nix-grpc-daemon[1531]: ts=2026-09-20T09:06:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:503825962worker2 # [ 112.474064] nix-daemon[844]: accepted connection from pid 1531, user nix-grpc-daemon (trusted)5963worker2 # [ 112.480152] nix-daemon[844]: reaped child process 1662, status = succeeded5964client # [ 112.534598] nix[1275]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...5965worker2 # [ 112.483236] nix-daemon[844]: accepted connection from pid 1531, user nix-grpc-daemon (trusted)5966worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&25967worker2 # Checking switch inhibitors... done5968worker2 # [ 115.153791] nixos[1708]: switching to system configuration /nix/store/sk820hc3azag8pgag8zqa4r8rvx2cp9w-nixos-system-worker2-test5969worker2 # [ 115.156491] systemd[1]: Stopped target Local File Systems.5970worker2 # [ 115.159059] systemd[1]: Stopped target Remote File Systems.5971worker2 # activating the configuration...5972worker2 # setting up /etc...5973worker2 # [ 115.851432] systemd[1]: Reload requested from client PID 1708 ('.switch-to-conf') (unit backdoor.service)...5974worker2 # [ 115.853198] systemd[1]: Reloading...5975worker2 # [ 116.588943] systemd[1]: Reloading finished in 733 ms.5976worker2 # restarting sysinit-reactivation.target5977lb # [ 116.643935] envoy[918]: conn peer=[2001:db8:1::1]:35700 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2924 tx=730 ms=22535978worker2 # [ 116.651987] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....5979worker2 # [ 116.798352] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..5980worker2 # [ 116.801596] systemd[1]: Reached target Reactivate sysinit units.5981worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service5982worker2 # [ 116.807188] systemd[1]: Reloading D-Bus System Message Bus...5983worker2 # [ 116.808443] dbus-broker-launch[532]: Caught SIGHUP, trigger reload.5984worker2 # [ 116.812092] systemd[1]: Reloading Nix worker-protocol over gRPC...5985worker2 # [ 116.842657] dbus-broker-launch[532]: Looking up NSS user entry for 'systemd-timesync'...5986worker2 # [ 116.845591] dbus-broker-launch[532]: NSS returned no entry for 'systemd-timesync'5987worker2 # [ 116.848194] dbus-broker-launch[532]: Invalid user-name in /nix/store/nl02z4vnl25dc1df60lqifjnw6c89zpg-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"5988worker2 # [ 116.857611] systemd[1]: Reloaded Nix worker-protocol over gRPC.5989worker2 # [ 116.876365] systemd[1]: Reloaded D-Bus System Message Bus.5990worker2 # [ 116.880702] systemd[1]: Reached target Remote File Systems.5991worker2 # [ 116.885770] systemd[1]: Starting linger-users.service...5992worker2 # [ 116.887942] systemd[1]: Reached target Local File Systems.5993worker2 # [ 116.889174] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore5994worker2 # [ 116.895063] systemd[1]: Starting Create SUID/SGID Wrappers...5995worker2 # [ 116.895895] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.5996worker2 # [ 116.897064] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc675997worker2 # [ 116.899061] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore5998worker2 # [ 116.900419] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os5999worker2 # [ 116.908237] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6000worker2 # [ 116.915726] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os6001worker2 # [ 116.917048] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki6002worker2 # [ 116.958276] systemd[1]: linger-users.service: Deactivated successfully.6003worker2 # [ 116.959413] systemd[1]: Finished linger-users.service.6004worker2 # [ 116.986895] nix-grpc-daemon[1531]: ts=2026-09-20T09:06:55Z level=info event=draining6005worker2 # [ 117.379587] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.6006worker2 # [ 117.383277] systemd[1]: Finished Create SUID/SGID Wrappers.6007worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service6008worker2 # [ 117.640520] nixos[1708]: finished switching to system configuration /nix/store/sk820hc3azag8pgag8zqa4r8rvx2cp9w-nixos-system-worker2-test6009worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 3.23 seconds)6010worker2: must succeed: pgrep -f 'read -t [9]0 x'6011worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)6012client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&26013worker1 # [ 117.919424] nix-grpc-daemon[1321]: ts=2026-09-20T09:06:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:567866014client # don't know how to build these paths:6015client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6016worker1 # [ 117.928109] nix-grpc-daemon[1321]: ts=2026-09-20T09:06:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:567866017worker2 # [ 117.895808] nix-grpc-daemon[1531]: ts=2026-09-20T09:06:56Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06018worker2 # [ 117.897638] nix-grpc-daemon[1531]: ts=2026-09-20T09:06:56Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.3:50051 clients=16019worker1 # [ 117.997690] nix-grpc-daemon[1321]: ts=2026-09-20T09:06:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567866020worker1 # [ 118.007776] nix-grpc-daemon[1321]: ts=2026-09-20T09:06:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:567866021client # copying 1 paths to 'lb:50051'...6022worker1 # [ 118.017264] nix-grpc-daemon[1321]: ts=2026-09-20T09:06:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:567866023client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6024worker1 # [ 118.029346] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)6025worker1 # [ 118.198351] nix-daemon[842]: accepted connection from pid 1484, user nix-grpc-daemon (trusted)6026worker1 # [ 118.211070] nix-daemon[842]: reaped child process 1487, status = succeeded6027worker1 # [ 118.368089] nix-daemon[842]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6028worker1 # [ 118.378768] nix-daemon[842]: reaped child process 1492, status = succeeded6029lb # [ 118.349883] niks3-server[1047]: 2026/09/20 09:06:56 INFO Received uploads request method=POST path=/api/pending_closures6030worker1 # [ 118.407558] nix-grpc-daemon[1405]: time=2026-09-20T09:06:55.666Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6031worker1 # [ 118.409519] nix-grpc-daemon[1405]: time=2026-09-20T09:06:55.668Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6032lb # [ 118.381564] niks3-server[1047]: 2026/09/20 09:06:56 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6033lb # [ 118.390630] niks3-server[1047]: 2026/09/20 09:06:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6034lb # [ 118.394871] niks3-server[1047]: 2026/09/20 09:06:56 INFO Signed narinfos id=39 count=16035worker1 # [ 118.444286] nix-grpc-daemon[1405]: time=2026-09-20T09:06:55.703Z level=INFO msg="Uploading 1 narinfos"6036lb # [ 118.405799] niks3-server[1047]: 2026/09/20 09:06:56 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6037lb # [ 118.417112] niks3-server[1047]: 2026/09/20 09:06:56 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6038lb # [ 118.422487] niks3-server[1047]: 2026/09/20 09:06:56 INFO Completed upload id=396039worker1 # [ 118.468830] nix-grpc-daemon[1405]: time=2026-09-20T09:06:55.727Z level=INFO msg="Upload complete. (428ms)"6040lb # [ 118.426452] niks3-server[1047]: 2026/09/20 09:06:56 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6041worker1 # [ 118.471924] nix-grpc-daemon[1321]: ts=2026-09-20T09:06:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56786 duration_s=0 paths=1 nar_bytes_in=5286042worker1 # [ 118.476949] nix-grpc-daemon[1321]: ts=2026-09-20T09:06:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567866043worker1 # [ 118.485970] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)6044worker1 # [ 118.493377] nix-daemon[842]: reaped child process 1482, status = succeeded6045client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6046worker1 # [ 118.499524] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)6047client # farm-a> node-a: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6048client # farm-a> LOG-farm-a-during-drain6049worker1 # [ 119.312109] nix-daemon[842]: reaped child process 1496, status = succeeded6050worker1 # [ 119.454813] nix-daemon[842]: accepted connection from pid 1511, user nix-grpc-daemon (trusted)6051worker1 # [ 119.466744] nix-daemon[842]: reaped child process 1514, status = succeeded6052lb # [ 119.437491] niks3-server[1047]: 2026/09/20 09:06:57 INFO Received uploads request method=POST path=/api/pending_closures6053worker1 # [ 119.502097] nix-grpc-daemon[1405]: time=2026-09-20T09:06:56.761Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6054worker1 # [ 119.504093] nix-grpc-daemon[1405]: time=2026-09-20T09:06:56.762Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6055lb # [ 119.481864] niks3-server[1047]: 2026/09/20 09:06:57 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6056lb # [ 119.493896] niks3-server[1047]: 2026/09/20 09:06:57 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6057lb # [ 119.504381] niks3-server[1047]: 2026/09/20 09:06:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6058lb # [ 119.509854] niks3-server[1047]: 2026/09/20 09:06:57 INFO Signed narinfos id=40 count=16059worker1 # [ 119.558915] nix-grpc-daemon[1405]: time=2026-09-20T09:06:56.817Z level=INFO msg="Uploading 1 narinfos"6060lb # [ 119.512681] niks3-server[1047]: 2026/09/20 09:06:57 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6061lb # [ 119.520542] niks3-server[1047]: 2026/09/20 09:06:57 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6062lb # [ 119.524192] niks3-server[1047]: 2026/09/20 09:06:57 INFO Completed upload id=406063lb # [ 119.525708] niks3-server[1047]: 2026/09/20 09:06:57 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6064worker1 # [ 119.571889] nix-grpc-daemon[1405]: time=2026-09-20T09:06:56.830Z level=INFO msg="Upload complete. (266ms)"6065worker2 # [ 119.526309] nix-grpc-daemon[1531]: ts=2026-09-20T09:06:57Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06066worker1 # [ 119.578622] nix-grpc-daemon[1321]: ts=2026-09-20T09:06:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56786 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16067worker2 # [ 119.528294] nix-grpc-daemon[1531]: ts=2026-09-20T09:06:57Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.3:50051 clients=16068worker1 # [ 119.591378] nix-daemon[842]: reaped child process 1494, status = succeeded6069worker1 # [ 119.630125] nix-grpc-daemon[1321]: ts=2026-09-20T09:06:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567866070worker1 # [ 119.642347] nix-grpc-daemon[1321]: ts=2026-09-20T09:06:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:567866071worker1 # [ 119.655568] nix-grpc-daemon[1405]: time=2026-09-20T09:06:56.914Z level=INFO msg="All 1 paths already cached"6072client # copying 1 paths to 'lb:50051'...6073worker1 # [ 119.662777] nix-grpc-daemon[1321]: ts=2026-09-20T09:06:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:567866074client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6075worker1 # [ 119.682163] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)6076worker1 # [ 119.833756] nix-daemon[842]: accepted connection from pid 1518, user nix-grpc-daemon (trusted)6077worker1 # [ 119.846352] nix-daemon[842]: reaped child process 1521, status = succeeded6078worker1 # [ 119.982076] nix-daemon[842]: accepted connection from pid 1523, user nix-grpc-daemon (trusted)6079worker1 # [ 119.994687] nix-daemon[842]: reaped child process 1526, status = succeeded6080lb # [ 119.958577] niks3-server[1047]: 2026/09/20 09:06:57 INFO Received uploads request method=POST path=/api/pending_closures6081worker1 # [ 120.015248] nix-grpc-daemon[1405]: time=2026-09-20T09:06:57.274Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6082worker1 # [ 120.016641] nix-grpc-daemon[1405]: time=2026-09-20T09:06:57.274Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6083lb # [ 119.988507] niks3-server[1047]: 2026/09/20 09:06:57 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6084lb # [ 120.001308] niks3-server[1047]: 2026/09/20 09:06:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6085worker1 # [ 120.053755] nix-grpc-daemon[1405]: time=2026-09-20T09:06:57.312Z level=INFO msg="Uploading 1 narinfos"6086lb # [ 120.007354] niks3-server[1047]: 2026/09/20 09:06:57 INFO Signed narinfos id=41 count=16087lb # [ 120.010833] niks3-server[1047]: 2026/09/20 09:06:57 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6088lb # [ 120.021600] niks3-server[1047]: 2026/09/20 09:06:57 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6089lb # [ 120.024184] niks3-server[1047]: 2026/09/20 09:06:57 INFO Completed upload id=416090worker1 # [ 120.070519] nix-grpc-daemon[1405]: time=2026-09-20T09:06:57.329Z level=INFO msg="Upload complete. (376ms)"6091lb # [ 120.025786] niks3-server[1047]: 2026/09/20 09:06:57 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6092worker1 # [ 120.072613] nix-grpc-daemon[1321]: ts=2026-09-20T09:06:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56786 duration_s=0 paths=1 nar_bytes_in=6406093worker1 # [ 120.076731] nix-grpc-daemon[1321]: ts=2026-09-20T09:06:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567866094worker1 # [ 120.086650] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)6095worker1 # [ 120.088648] nix-daemon[842]: reaped child process 1516, status = succeeded6096worker1 # [ 120.093890] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)6097client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6098client # farm-b> node-a: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6099client # farm-b> LOG-farm-b-during-drain6100worker1 # [ 120.908717] nix-daemon[842]: reaped child process 1530, status = succeeded6101worker1 # [ 121.036974] nix-daemon[842]: accepted connection from pid 1545, user nix-grpc-daemon (trusted)6102worker1 # [ 121.048960] nix-daemon[842]: reaped child process 1548, status = succeeded6103lb # [ 121.014398] niks3-server[1047]: 2026/09/20 09:06:58 INFO Received uploads request method=POST path=/api/pending_closures6104worker1 # [ 121.069159] nix-grpc-daemon[1405]: time=2026-09-20T09:06:58.328Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6105worker1 # [ 121.070647] nix-grpc-daemon[1405]: time=2026-09-20T09:06:58.329Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6106lb # [ 121.047840] niks3-server[1047]: 2026/09/20 09:06:58 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6107lb # [ 121.059984] niks3-server[1047]: 2026/09/20 09:06:58 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6108lb # [ 121.071927] niks3-server[1047]: 2026/09/20 09:06:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6109worker1 # [ 121.124660] nix-grpc-daemon[1405]: time=2026-09-20T09:06:58.383Z level=INFO msg="Uploading 1 narinfos"6110lb # [ 121.078131] niks3-server[1047]: 2026/09/20 09:06:58 INFO Signed narinfos id=42 count=16111lb # [ 121.081872] niks3-server[1047]: 2026/09/20 09:06:58 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6112lb # [ 121.092117] niks3-server[1047]: 2026/09/20 09:06:58 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6113lb # [ 121.094860] niks3-server[1047]: 2026/09/20 09:06:58 INFO Completed upload id=426114worker1 # [ 121.141177] nix-grpc-daemon[1405]: time=2026-09-20T09:06:58.400Z level=INFO msg="Upload complete. (241ms)"6115lb # [ 121.096516] niks3-server[1047]: 2026/09/20 09:06:58 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6116worker1 # [ 121.143462] nix-grpc-daemon[1321]: ts=2026-09-20T09:06:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56786 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16117worker1 # [ 121.150139] nix-daemon[842]: reaped child process 1528, status = succeeded6118worker2 # [ 121.094988] nix-grpc-daemon[1531]: ts=2026-09-20T09:06:59Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06119worker2 # [ 121.096875] nix-grpc-daemon[1531]: ts=2026-09-20T09:06:59Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.3:50051 clients=16120worker1 # [ 121.194475] nix-grpc-daemon[1321]: ts=2026-09-20T09:06:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567866121worker1 # [ 121.202257] nix-grpc-daemon[1321]: ts=2026-09-20T09:06:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:567866122worker1 # [ 121.210335] nix-grpc-daemon[1405]: time=2026-09-20T09:06:58.469Z level=INFO msg="All 2 paths already cached"6123client # copying 1 paths to 'lb:50051'...6124worker1 # [ 121.215914] nix-grpc-daemon[1321]: ts=2026-09-20T09:06:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:567866125client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6126worker1 # [ 121.228350] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)6127worker1 # [ 121.373215] nix-daemon[842]: accepted connection from pid 1552, user nix-grpc-daemon (trusted)6128worker1 # [ 121.385186] nix-daemon[842]: reaped child process 1555, status = succeeded6129worker1 # [ 121.537907] nix-daemon[842]: accepted connection from pid 1557, user nix-grpc-daemon (trusted)6130worker1 # [ 121.549811] nix-daemon[842]: reaped child process 1560, status = succeeded6131lb # [ 121.517913] niks3-server[1047]: 2026/09/20 09:06:59 INFO Received uploads request method=POST path=/api/pending_closures6132worker1 # [ 121.572235] nix-grpc-daemon[1405]: time=2026-09-20T09:06:58.831Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6133worker1 # [ 121.573741] nix-grpc-daemon[1405]: time=2026-09-20T09:06:58.832Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6134lb # [ 121.545218] niks3-server[1047]: 2026/09/20 09:06:59 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6135lb # [ 121.557466] niks3-server[1047]: 2026/09/20 09:06:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6136worker1 # [ 121.608054] nix-grpc-daemon[1405]: time=2026-09-20T09:06:58.867Z level=INFO msg="Uploading 1 narinfos"6137lb # [ 121.561838] niks3-server[1047]: 2026/09/20 09:06:59 INFO Signed narinfos id=43 count=16138lb # [ 121.570581] niks3-server[1047]: 2026/09/20 09:06:59 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6139lb # [ 121.579056] niks3-server[1047]: 2026/09/20 09:06:59 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6140worker1 # [ 121.631686] nix-grpc-daemon[1405]: time=2026-09-20T09:06:58.890Z level=INFO msg="Upload complete. (398ms)"6141lb # [ 121.585445] niks3-server[1047]: 2026/09/20 09:06:59 INFO Completed upload id=436142lb # [ 121.588680] niks3-server[1047]: 2026/09/20 09:06:59 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6143worker1 # [ 121.634323] nix-grpc-daemon[1321]: ts=2026-09-20T09:06:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56786 duration_s=0 paths=1 nar_bytes_in=7686144worker1 # [ 121.640214] nix-grpc-daemon[1321]: ts=2026-09-20T09:06:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567866145worker1 # [ 121.648361] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)6146worker1 # [ 121.653212] nix-daemon[842]: reaped child process 1550, status = succeeded6147worker1 # [ 121.658205] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)6148client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6149client # farm-top> node-a: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6150client # farm-top> LOG-farm-top-during-drain6151worker1 # [ 122.872299] nix-daemon[842]: reaped child process 1564, status = succeeded6152worker1 # [ 123.018428] nix-daemon[842]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6153worker1 # [ 123.030596] nix-daemon[842]: reaped child process 1582, status = succeeded6154lb # [ 123.000202] niks3-server[1047]: 2026/09/20 09:07:00 INFO Received uploads request method=POST path=/api/pending_closures6155worker1 # [ 123.054893] nix-grpc-daemon[1405]: time=2026-09-20T09:07:00.313Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6156worker1 # [ 123.056439] nix-grpc-daemon[1405]: time=2026-09-20T09:07:00.315Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6157lb # [ 123.036638] niks3-server[1047]: 2026/09/20 09:07:00 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6158lb # [ 123.047400] niks3-server[1047]: 2026/09/20 09:07:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6159lb # [ 123.052937] niks3-server[1047]: 2026/09/20 09:07:00 INFO Signed narinfos id=44 count=16160worker1 # [ 123.102096] nix-grpc-daemon[1405]: time=2026-09-20T09:07:00.361Z level=INFO msg="Uploading 1 narinfos"6161lb # [ 123.059246] niks3-server[1047]: 2026/09/20 09:07:00 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6162lb # [ 123.070936] niks3-server[1047]: 2026/09/20 09:07:00 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6163lb # [ 123.076584] niks3-server[1047]: 2026/09/20 09:07:00 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6164lb # [ 123.080479] niks3-server[1047]: 2026/09/20 09:07:00 INFO Completed upload id=446165worker1 # [ 123.128027] nix-grpc-daemon[1405]: time=2026-09-20T09:07:00.387Z level=INFO msg="Upload complete. (263ms)"6166lb # [ 123.082167] niks3-server[1047]: 2026/09/20 09:07:00 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6167worker1 # [ 123.131351] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56786 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16168worker1 # [ 123.138459] nix-daemon[842]: reaped child process 1562, status = succeeded6169client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&2, in 5.41 seconds)6170client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath6171client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath, in 0.18 seconds)6172worker2: must succeed: pkill -f 'read -t [9]0 x'6173worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6174client: waiting for success: ! systemctl is-active sw6175worker2 # [ 123.523574] nix-daemon[844]: reaped child process 1677, status = succeeded6176worker2 # [ 123.701072] nix-daemon[844]: accepted connection from pid 1854, user nix-grpc-daemon (trusted)6177worker2 # [ 123.713305] nix-daemon[844]: reaped child process 1857, status = succeeded6178lb # [ 123.740677] niks3-server[1047]: 2026/09/20 09:07:01 INFO Received uploads request method=POST path=/api/pending_closures6179worker2 # [ 123.741269] nix-grpc-daemon[1610]: time=2026-09-20T09:07:01.954Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6180worker2 # [ 123.742753] nix-grpc-daemon[1610]: time=2026-09-20T09:07:01.956Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6181lb # [ 123.780173] niks3-server[1047]: 2026/09/20 09:07:01 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6182lb # [ 123.787134] niks3-server[1047]: 2026/09/20 09:07:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6183lb # [ 123.794134] niks3-server[1047]: 2026/09/20 09:07:01 INFO Signed narinfos id=45 count=16184worker2 # [ 123.787032] nix-grpc-daemon[1610]: time=2026-09-20T09:07:01.999Z level=INFO msg="Uploading 1 narinfos"6185lb # [ 123.798315] niks3-server[1047]: 2026/09/20 09:07:01 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6186lb # [ 123.803403] niks3-server[1047]: 2026/09/20 09:07:01 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6187lb # [ 123.809171] niks3-server[1047]: 2026/09/20 09:07:01 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6188lb # [ 123.811950] niks3-server[1047]: 2026/09/20 09:07:01 INFO Completed upload id=456189worker2 # [ 123.802123] nix-grpc-daemon[1610]: time=2026-09-20T09:07:02.015Z level=INFO msg="Upload complete. (297ms)"6190lb # [ 123.814430] niks3-server[1047]: 2026/09/20 09:07:01 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6191worker2 # [ 123.807612] nix-grpc-daemon[1531]: ts=2026-09-20T09:07:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50382 duration_s=11 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16192worker2 # [ 123.819109] nix-daemon[844]: reaped child process 1675, status = succeeded6193client # [ 123.901130] systemd[1]: sw.service: Deactivated successfully.6194client # [ 123.905232] systemd[1]: sw.service: Consumed 103ms CPU time over 13.081s wall clock time, 37.1M memory peak, 8.5K incoming IP traffic, 9K outgoing IP traffic.6195worker2 # [ 123.989822] nix-grpc-daemon[1531]: ts=2026-09-20T09:07:02Z level=info event=signal_exit6196lb # [ 124.226470] niks3-server[1047]: 2026/09/20 09:07:01 INFO lead: released remote=192.168.1.4:546446197worker2 # [ 124.215503] nix-grpc-daemon[1531]: ts=2026-09-20T09:07:02Z level=info event=scheduler_yield6198worker1 # [ 124.277670] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:01Z level=info event=scheduler_restarting addr=lb:500516199worker2 # [ 124.221433] nix-grpc-daemon[1531]: ts=2026-09-20T09:07:02Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:503726200worker2 # [ 124.226602] nix-grpc-daemon[1531]: ts=2026-09-20T09:07:02Z level=info event=worker_gone addr=192.168.1.3:500516201worker2 # [ 124.229696] nix-grpc-daemon[1531]: ts=2026-09-20T09:07:02Z level=info event=scheduler_restarting addr=lb:500516202worker2 # [ 124.232892] nix-grpc-daemon[1531]: ts=2026-09-20T09:07:02Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:503726203worker2 # [ 124.236124] nix-grpc-daemon[1531]: ts=2026-09-20T09:07:02Z level=info event=worker_gone addr=192.168.1.4:500516204client: (finished: waiting for success: ! systemctl is-active sw, in 1.08 seconds)6205client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6206worker1 # [ 124.477627] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:01Z level=info event=scheduler_connected addr=lb:500516207worker2 # [ 124.427386] nix-grpc-daemon[1531]: ts=2026-09-20T09:07:02Z level=info event=scheduler_connected addr=lb:500516208worker1 # [ 124.488584] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:01Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6209worker2 # [ 124.438734] nix-grpc-daemon[1531]: ts=2026-09-20T09:07:02Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6210client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds)6211worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6212worker1 # [ 124.689733] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:01Z level=info event=scheduler_connected addr=lb:500516213worker1 # [ 124.693842] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:01Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6214worker2 # [ 124.639703] nix-grpc-daemon[1531]: ts=2026-09-20T09:07:02Z level=info event=scheduler_connected addr=lb:500516215worker2 # [ 124.643578] nix-grpc-daemon[1531]: ts=2026-09-20T09:07:02Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6216worker1 # [ 124.894927] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:02Z level=info event=scheduler_connected addr=lb:500516217worker1 # [ 124.899264] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:02Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6218worker2 # [ 124.845279] nix-grpc-daemon[1531]: ts=2026-09-20T09:07:03Z level=info event=scheduler_connected addr=lb:500516219worker2 # [ 124.849152] nix-grpc-daemon[1531]: ts=2026-09-20T09:07:03Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6220worker1 # [ 125.100186] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:02Z level=info event=scheduler_connected addr=lb:500516221worker1 # [ 125.103912] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:02Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6222worker2 # [ 125.050238] nix-grpc-daemon[1531]: ts=2026-09-20T09:07:03Z level=info event=scheduler_connected addr=lb:500516223worker2 # [ 125.054096] nix-grpc-daemon[1531]: ts=2026-09-20T09:07:03Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6224worker2 # [ 125.220221] nix-grpc-daemon[1531]: ts=2026-09-20T09:07:03Z level=info event=scheduler_restarting6225worker2 # [ 125.243297] nix-daemon[844]: reaped child process 1582, status = succeeded6226worker1 # [ 125.305707] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:02Z level=info event=scheduler_connected addr=lb:500516227worker2 # [ 125.249881] nix-grpc-daemon[1531]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6228worker2 # [ 125.253199] nix-grpc-daemon[1531]: E0000 00:00:1789895223.463075 1574 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true6229worker2 # [ 125.270645] nix-grpc-daemon[1531]: ts=2026-09-20T09:07:03Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6230worker2 # [ 125.332813] nix-grpc-daemon[1531]: E0000 00:00:1789895223.545941 1574 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 10, started true, shutdown true6231worker2 # [ 125.337814] nix-grpc-daemon[1531]: E0000 00:00:1789895223.546090 1574 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6232lb # [ 126.644445] envoy[918]: conn peer=[2001:db8:1::1]:50130 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9555 tx=3807 ms=52286233lb # [ 126.648262] envoy[918]: conn peer=[2001:db8:1::1]:35704 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2949 tx=1353 ms=128716234lb # [ 126.651924] envoy[918]: conn peer=[2001:db8:1::4]:56122 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1721 tx=552 ms=242576235worker1 # [ 127.067103] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:04Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: remote connection failure"6236worker2 # [ 127.032811] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6237worker2 # [ 127.035543] systemd[1]: nix-grpc-daemon.service: Consumed 715ms CPU time over 26.352s wall clock time, 33.2M memory peak, 24.1K incoming IP traffic, 32.9K outgoing IP traffic.6238worker2 # [ 127.051733] systemd[1]: Starting Nix worker-protocol over gRPC...6239lb # [ 127.193362] niks3-server[1047]: 2026/09/20 09:07:04 INFO lead: acquired remote=192.168.1.3:387746240worker1 # [ 127.239760] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:04Z level=info event=scheduler_take_over6241worker1 # [ 127.267190] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:04Z level=info event=scheduler_connected addr=lb:500516242worker1 # [ 127.271626] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:04Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06243worker2 # [ 127.224317] nix-grpc-daemon[1871]: ts=2026-09-20T09:07:05Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6244worker2 # [ 127.228313] nix-grpc-daemon[1871]: ts=2026-09-20T09:07:05Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16245worker2 # [ 127.235111] nix-grpc-daemon[1871]: ts=2026-09-20T09:07:05Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6246worker2 # [ 127.239150] systemd[1]: Started Nix worker-protocol over gRPC.6247worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.80 seconds)6248lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6249lb: must succeed: curl -sf localhost:9901/clusters6250worker2 # [ 127.266408] nix-grpc-daemon[1871]: ts=2026-09-20T09:07:05Z level=info event=scheduler_connected addr=lb:500516251worker2 # [ 127.276425] nix-grpc-daemon[1871]: ts=2026-09-20T09:07:05Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6252lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6253worker2 # [ 127.477747] nix-grpc-daemon[1871]: ts=2026-09-20T09:07:05Z level=info event=scheduler_connected addr=lb:500516254worker1 # [ 127.536133] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:04Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06255lb: must succeed: curl -sf localhost:9901/clusters6256lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6257lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.14 seconds)6258lb: must succeed: curl -sf localhost:9901/clusters6259lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6260worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true6261worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.06 seconds)6262(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 17.76 seconds)6263subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6264client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16265worker2 # [ 128.714513] nix-grpc-daemon[1871]: ts=2026-09-20T09:07:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:563386266worker2 # [ 128.726607] nix-daemon[844]: accepted connection from pid 1871, user nix-grpc-daemon (trusted)6267worker1 # [ 128.797854] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:567866268worker1 # [ 128.836915] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:06Z level=debug event=want client=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06269worker1 # [ 128.842526] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:06Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16270worker1 # [ 128.846720] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:06Z level=debug event=want client=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06271worker1 # [ 128.850483] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:06Z level=debug event=want client=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06272worker1 # [ 128.880931] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567866273worker1 # [ 128.893259] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:567866274worker1 # [ 128.904187] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:567866275worker1 # [ 128.915175] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)6276worker1 # [ 129.083238] nix-daemon[842]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)6277worker1 # [ 129.095424] nix-daemon[842]: reaped child process 1596, status = succeeded6278worker1 # [ 129.252988] nix-daemon[842]: accepted connection from pid 1598, user nix-grpc-daemon (trusted)6279worker1 # [ 129.264655] nix-daemon[842]: reaped child process 1601, status = succeeded6280lb # [ 129.230857] niks3-server[1047]: 2026/09/20 09:07:06 INFO Received uploads request method=POST path=/api/pending_closures6281worker1 # [ 129.287390] nix-grpc-daemon[1405]: time=2026-09-20T09:07:06.546Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6282worker1 # [ 129.288794] nix-grpc-daemon[1405]: time=2026-09-20T09:07:06.546Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6283lb # [ 129.260378] niks3-server[1047]: 2026/09/20 09:07:06 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6284lb # [ 129.269610] niks3-server[1047]: 2026/09/20 09:07:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6285worker1 # [ 129.321191] nix-grpc-daemon[1405]: time=2026-09-20T09:07:06.580Z level=INFO msg="Uploading 1 narinfos"6286lb # [ 129.276312] niks3-server[1047]: 2026/09/20 09:07:06 INFO Signed narinfos id=46 count=16287lb # [ 129.281057] niks3-server[1047]: 2026/09/20 09:07:06 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6288lb # [ 129.292592] niks3-server[1047]: 2026/09/20 09:07:06 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6289lb # [ 129.300312] niks3-server[1047]: 2026/09/20 09:07:07 INFO Completed upload id=466290worker1 # [ 129.347650] nix-grpc-daemon[1405]: time=2026-09-20T09:07:06.605Z level=INFO msg="Upload complete. (420ms)"6291lb # [ 129.303906] niks3-server[1047]: 2026/09/20 09:07:07 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6292worker1 # [ 129.351188] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56786 duration_s=0 paths=1 nar_bytes_in=5046293worker1 # [ 129.355778] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567866294worker1 # [ 129.366108] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)6295worker1 # [ 129.370112] nix-daemon[842]: reaped child process 1591, status = succeeded6296worker1 # [ 129.375253] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)6297worker1 # [ 129.988452] nix-daemon[842]: reaped child process 1605, status = succeeded6298worker1 # [ 130.128505] nix-daemon[842]: accepted connection from pid 1620, user nix-grpc-daemon (trusted)6299worker1 # [ 130.140768] nix-daemon[842]: reaped child process 1623, status = succeeded6300lb # [ 130.108553] niks3-server[1047]: 2026/09/20 09:07:07 INFO Received uploads request method=POST path=/api/pending_closures6301worker1 # [ 130.163072] nix-grpc-daemon[1405]: time=2026-09-20T09:07:07.421Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6302worker1 # [ 130.164504] nix-grpc-daemon[1405]: time=2026-09-20T09:07:07.421Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6303lb # [ 130.147862] niks3-server[1047]: 2026/09/20 09:07:07 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6304lb # [ 130.159072] niks3-server[1047]: 2026/09/20 09:07:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6305worker1 # [ 130.214866] nix-grpc-daemon[1405]: time=2026-09-20T09:07:07.473Z level=INFO msg="Uploading 1 narinfos"6306lb # [ 130.169360] niks3-server[1047]: 2026/09/20 09:07:07 INFO Signed narinfos id=47 count=16307lb # [ 130.174475] niks3-server[1047]: 2026/09/20 09:07:07 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6308lb # [ 130.188203] niks3-server[1047]: 2026/09/20 09:07:07 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6309lb # [ 130.193520] niks3-server[1047]: 2026/09/20 09:07:07 INFO Completed upload id=476310worker1 # [ 130.239734] nix-grpc-daemon[1405]: time=2026-09-20T09:07:07.498Z level=INFO msg="Upload complete. (258ms)"6311lb # [ 130.194821] niks3-server[1047]: 2026/09/20 09:07:07 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6312worker1 # [ 130.243238] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56786 duration_s=0 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=1 outputs=16313lb # [ 130.200059] niks3-server[1047]: 2026/09/20 09:07:07 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6314worker1 # [ 130.247479] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:07Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16315worker1 # [ 130.254306] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:07Z level=debug event=want client=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06316worker1 # [ 130.256195] nix-daemon[842]: reaped child process 1603, status = succeeded6317worker1 # [ 130.291217] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567866318worker1 # [ 130.298873] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:567866319worker1 # [ 130.305882] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:567866320worker1 # [ 130.309757] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)6321worker1 # [ 130.464825] nix-daemon[842]: accepted connection from pid 1629, user nix-grpc-daemon (trusted)6322worker1 # [ 130.478089] nix-daemon[842]: reaped child process 1632, status = succeeded6323worker1 # [ 130.635294] nix-daemon[842]: accepted connection from pid 1634, user nix-grpc-daemon (trusted)6324worker1 # [ 130.647279] nix-daemon[842]: reaped child process 1637, status = succeeded6325lb # [ 130.617424] niks3-server[1047]: 2026/09/20 09:07:08 INFO Received uploads request method=POST path=/api/pending_closures6326worker1 # [ 130.675209] nix-grpc-daemon[1405]: time=2026-09-20T09:07:07.934Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6327worker1 # [ 130.676747] nix-grpc-daemon[1405]: time=2026-09-20T09:07:07.935Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6328lb # [ 130.654143] niks3-server[1047]: 2026/09/20 09:07:08 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6329lb # [ 130.658961] niks3-server[1047]: 2026/09/20 09:07:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6330lb # [ 130.664650] niks3-server[1047]: 2026/09/20 09:07:08 INFO Signed narinfos id=48 count=16331worker1 # [ 130.714334] nix-grpc-daemon[1405]: time=2026-09-20T09:07:07.973Z level=INFO msg="Uploading 1 narinfos"6332lb # [ 130.676931] niks3-server[1047]: 2026/09/20 09:07:08 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6333lb # [ 130.681920] niks3-server[1047]: 2026/09/20 09:07:08 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6334lb # [ 130.691435] niks3-server[1047]: 2026/09/20 09:07:08 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6335lb # [ 130.696609] niks3-server[1047]: 2026/09/20 09:07:08 INFO Completed upload id=486336worker1 # [ 130.742985] nix-grpc-daemon[1405]: time=2026-09-20T09:07:08.002Z level=INFO msg="Upload complete. (427ms)"6337worker1 # [ 130.747323] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56786 duration_s=0 paths=1 nar_bytes_in=5046338worker1 # [ 130.753441] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567866339worker1 # [ 130.763487] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)6340worker1 # [ 130.772733] nix-daemon[842]: reaped child process 1627, status = succeeded6341worker1 # [ 130.779943] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)6342worker1 # [ 131.792511] nix-daemon[842]: reaped child process 1641, status = succeeded6343worker1 # [ 131.937483] nix-daemon[842]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6344worker1 # [ 131.949175] nix-daemon[842]: reaped child process 1659, status = succeeded6345lb # [ 131.919910] niks3-server[1047]: 2026/09/20 09:07:09 INFO Received uploads request method=POST path=/api/pending_closures6346worker1 # [ 131.975352] nix-grpc-daemon[1405]: time=2026-09-20T09:07:09.234Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6347worker1 # [ 131.976841] nix-grpc-daemon[1405]: time=2026-09-20T09:07:09.236Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6348lb # [ 131.966760] niks3-server[1047]: 2026/09/20 09:07:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6349lb # [ 131.972127] niks3-server[1047]: 2026/09/20 09:07:09 INFO Signed narinfos id=49 count=16350worker1 # [ 132.021834] nix-grpc-daemon[1405]: time=2026-09-20T09:07:09.280Z level=INFO msg="Uploading 1 narinfos"6351lb # [ 131.977794] niks3-server[1047]: 2026/09/20 09:07:09 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6352lb # [ 131.987422] niks3-server[1047]: 2026/09/20 09:07:09 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6353lb # [ 131.996165] niks3-server[1047]: 2026/09/20 09:07:09 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6354lb # [ 132.002096] niks3-server[1047]: 2026/09/20 09:07:09 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6355lb # [ 132.006803] niks3-server[1047]: 2026/09/20 09:07:09 INFO Completed upload id=496356worker1 # [ 132.053352] nix-grpc-daemon[1405]: time=2026-09-20T09:07:09.312Z level=INFO msg="Upload complete. (267ms)"6357lb # [ 132.009413] niks3-server[1047]: 2026/09/20 09:07:09 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6358worker1 # [ 132.065918] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56786 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=2 outputs=16359worker1 # [ 132.071203] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:09Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16360worker1 # [ 132.074667] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567866361worker1 # [ 132.078392] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:09Z level=debug event=want client=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06362worker1 # [ 132.087699] nix-daemon[842]: reaped child process 1639, status = succeeded6363worker1 # [ 132.089210] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:567866364worker1 # [ 132.096876] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:567866365worker1 # [ 132.102095] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)6366worker1 # [ 132.258120] nix-daemon[842]: accepted connection from pid 1665, user nix-grpc-daemon (trusted)6367worker1 # [ 132.270984] nix-daemon[842]: reaped child process 1668, status = succeeded6368worker1 # [ 132.426206] nix-daemon[842]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)6369worker1 # [ 132.437673] nix-daemon[842]: reaped child process 1673, status = succeeded6370lb # [ 132.402160] niks3-server[1047]: 2026/09/20 09:07:10 INFO Received uploads request method=POST path=/api/pending_closures6371worker1 # [ 132.456457] nix-grpc-daemon[1405]: time=2026-09-20T09:07:09.715Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6372worker1 # [ 132.457815] nix-grpc-daemon[1405]: time=2026-09-20T09:07:09.715Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6373lb # [ 132.426229] niks3-server[1047]: 2026/09/20 09:07:10 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6374lb # [ 132.439139] niks3-server[1047]: 2026/09/20 09:07:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6375lb # [ 132.443984] niks3-server[1047]: 2026/09/20 09:07:10 INFO Signed narinfos id=50 count=16376worker1 # [ 132.493123] nix-grpc-daemon[1405]: time=2026-09-20T09:07:09.752Z level=INFO msg="Uploading 1 narinfos"6377lb # [ 132.450680] niks3-server[1047]: 2026/09/20 09:07:10 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6378lb # [ 132.460967] niks3-server[1047]: 2026/09/20 09:07:10 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6379worker1 # [ 132.512298] nix-grpc-daemon[1405]: time=2026-09-20T09:07:09.771Z level=INFO msg="Upload complete. (405ms)"6380lb # [ 132.466048] niks3-server[1047]: 2026/09/20 09:07:10 INFO Completed upload id=506381worker1 # [ 132.514225] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56786 duration_s=0 paths=1 nar_bytes_in=5046382lb # [ 132.469368] niks3-server[1047]: 2026/09/20 09:07:10 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6383worker1 # [ 132.520414] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567866384worker1 # [ 132.524115] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)6385worker1 # [ 132.526317] nix-daemon[842]: reaped child process 1663, status = succeeded6386worker1 # [ 132.531133] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)6387worker1 # [ 133.743587] nix-daemon[842]: reaped child process 1677, status = succeeded6388worker1 # [ 133.871260] nix-daemon[842]: accepted connection from pid 1692, user nix-grpc-daemon (trusted)6389worker1 # [ 133.882823] nix-daemon[842]: reaped child process 1695, status = succeeded6390lb # [ 133.850965] niks3-server[1047]: 2026/09/20 09:07:11 INFO Received uploads request method=POST path=/api/pending_closures6391worker1 # [ 133.905579] nix-grpc-daemon[1405]: time=2026-09-20T09:07:11.164Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6392worker1 # [ 133.907133] nix-grpc-daemon[1405]: time=2026-09-20T09:07:11.166Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6393lb # [ 133.892359] niks3-server[1047]: 2026/09/20 09:07:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6394worker1 # [ 133.944938] nix-grpc-daemon[1405]: time=2026-09-20T09:07:11.204Z level=INFO msg="Uploading 1 narinfos"6395lb # [ 133.898777] niks3-server[1047]: 2026/09/20 09:07:11 INFO Signed narinfos id=51 count=16396lb # [ 133.907126] niks3-server[1047]: 2026/09/20 09:07:11 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv6397lb # [ 133.913746] niks3-server[1047]: 2026/09/20 09:07:11 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6398lb # [ 133.928068] niks3-server[1047]: 2026/09/20 09:07:11 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls6399lb # [ 133.931951] niks3-server[1047]: 2026/09/20 09:07:11 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6400lb # [ 133.935888] niks3-server[1047]: 2026/09/20 09:07:11 INFO Completed upload id=516401worker1 # [ 133.982089] nix-grpc-daemon[1405]: time=2026-09-20T09:07:11.241Z level=INFO msg="Upload complete. (244ms)"6402lb # [ 133.937734] niks3-server[1047]: 2026/09/20 09:07:11 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo6403worker1 # [ 133.993487] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56786 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=3 outputs=16404worker1 # [ 134.000282] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:11Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=16405worker1 # [ 134.004348] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567866406worker1 # [ 134.007982] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:11Z level=debug event=want client=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=06407worker1 # [ 134.017528] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:567866408worker1 # [ 134.020562] nix-daemon[842]: reaped child process 1675, status = succeeded6409worker1 # [ 134.025819] nix-grpc-daemon[1405]: time=2026-09-20T09:07:11.285Z level=INFO msg="All 1 paths already cached"6410worker1 # [ 134.030931] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:567866411worker1 # [ 134.035026] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)6412worker1 # [ 134.177165] nix-daemon[842]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6413worker1 # [ 134.188911] nix-daemon[842]: reaped child process 1704, status = succeeded6414worker1 # [ 134.333267] nix-daemon[842]: accepted connection from pid 1706, user nix-grpc-daemon (trusted)6415worker1 # [ 134.344769] nix-daemon[842]: reaped child process 1709, status = succeeded6416lb # [ 134.312508] niks3-server[1047]: 2026/09/20 09:07:12 INFO Received uploads request method=POST path=/api/pending_closures6417worker1 # [ 134.364574] nix-grpc-daemon[1405]: time=2026-09-20T09:07:11.623Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6418worker1 # [ 134.366063] nix-grpc-daemon[1405]: time=2026-09-20T09:07:11.623Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"6419lb # [ 134.328438] niks3-server[1047]: 2026/09/20 09:07:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6420lb # [ 134.333384] niks3-server[1047]: 2026/09/20 09:07:12 INFO Signed narinfos id=52 count=16421worker1 # [ 134.381167] nix-grpc-daemon[1405]: time=2026-09-20T09:07:11.640Z level=INFO msg="Uploading 1 narinfos"6422lb # [ 134.336050] niks3-server[1047]: 2026/09/20 09:07:12 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst6423lb # [ 134.345044] niks3-server[1047]: 2026/09/20 09:07:12 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls6424lb # [ 134.347616] niks3-server[1047]: 2026/09/20 09:07:12 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6425lb # [ 134.350961] niks3-server[1047]: 2026/09/20 09:07:12 INFO Completed upload id=526426worker1 # [ 134.397221] nix-grpc-daemon[1405]: time=2026-09-20T09:07:11.656Z level=INFO msg="Upload complete. (356ms)"6427worker1 # [ 134.399176] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56786 duration_s=0 paths=1 nar_bytes_in=6246428lb # [ 134.354082] niks3-server[1047]: 2026/09/20 09:07:12 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo6429worker1 # [ 134.403166] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567866430worker1 # [ 134.409161] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)6431worker1 # [ 134.412934] nix-daemon[842]: reaped child process 1699, status = succeeded6432worker1 # [ 134.415829] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)6433worker1 # [ 135.843332] nix-daemon[842]: reaped child process 1713, status = succeeded6434worker1 # [ 135.955451] nix-daemon[842]: accepted connection from pid 1728, user nix-grpc-daemon (trusted)6435worker1 # [ 135.966767] nix-daemon[842]: reaped child process 1731, status = succeeded6436lb # [ 135.936145] niks3-server[1047]: 2026/09/20 09:07:13 INFO Received uploads request method=POST path=/api/pending_closures6437worker1 # [ 135.986915] nix-grpc-daemon[1405]: time=2026-09-20T09:07:13.245Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6438worker1 # [ 135.988567] nix-grpc-daemon[1405]: time=2026-09-20T09:07:13.247Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"6439lb # [ 135.959260] niks3-server[1047]: 2026/09/20 09:07:13 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst6440lb # [ 135.963498] niks3-server[1047]: 2026/09/20 09:07:13 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv6441lb # [ 135.966057] niks3-server[1047]: 2026/09/20 09:07:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6442lb # [ 135.968343] niks3-server[1047]: 2026/09/20 09:07:13 INFO Signed narinfos id=53 count=16443worker1 # [ 136.015872] nix-grpc-daemon[1405]: time=2026-09-20T09:07:13.274Z level=INFO msg="Uploading 1 narinfos"6444lb # [ 135.972688] niks3-server[1047]: 2026/09/20 09:07:13 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls6445lb # [ 135.976250] niks3-server[1047]: 2026/09/20 09:07:13 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6446lb # [ 135.980259] niks3-server[1047]: 2026/09/20 09:07:13 INFO Completed upload id=536447worker1 # [ 136.027993] nix-grpc-daemon[1405]: time=2026-09-20T09:07:13.287Z level=INFO msg="Upload complete. (204ms)"6448lb # [ 135.982540] niks3-server[1047]: 2026/09/20 09:07:13 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo6449worker1 # [ 136.031507] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56786 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=4 outputs=16450worker1 # [ 136.035633] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:13Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=16451worker1 # [ 136.037892] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567866452worker1 # [ 136.040644] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:13Z level=debug event=want client=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=06453worker1 # [ 136.046299] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:567866454worker1 # [ 136.048780] nix-daemon[842]: reaped child process 1711, status = succeeded6455worker1 # [ 136.052708] nix-grpc-daemon[1405]: time=2026-09-20T09:07:13.311Z level=INFO msg="All 1 paths already cached"6456worker1 # [ 136.056751] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:567866457worker1 # [ 136.061335] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)6458worker1 # [ 136.204930] nix-daemon[842]: accepted connection from pid 1737, user nix-grpc-daemon (trusted)6459worker1 # [ 136.218127] nix-daemon[842]: reaped child process 1740, status = succeeded6460worker1 # [ 136.372499] nix-daemon[842]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6461worker1 # [ 136.383802] nix-daemon[842]: reaped child process 1745, status = succeeded6462lb # [ 136.352940] niks3-server[1047]: 2026/09/20 09:07:14 INFO Received uploads request method=POST path=/api/pending_closures6463worker1 # [ 136.403350] nix-grpc-daemon[1405]: time=2026-09-20T09:07:13.662Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6464worker1 # [ 136.404860] nix-grpc-daemon[1405]: time=2026-09-20T09:07:13.664Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"6465lb # [ 136.369933] niks3-server[1047]: 2026/09/20 09:07:14 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst6466lb # [ 136.372843] niks3-server[1047]: 2026/09/20 09:07:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6467lb # [ 136.374889] niks3-server[1047]: 2026/09/20 09:07:14 INFO Signed narinfos id=54 count=16468worker1 # [ 136.421200] nix-grpc-daemon[1405]: time=2026-09-20T09:07:13.680Z level=INFO msg="Uploading 1 narinfos"6469lb # [ 136.377897] niks3-server[1047]: 2026/09/20 09:07:14 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls6470lb # [ 136.381796] niks3-server[1047]: 2026/09/20 09:07:14 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6471lb # [ 136.385637] niks3-server[1047]: 2026/09/20 09:07:14 INFO Completed upload id=546472worker1 # [ 136.431729] nix-grpc-daemon[1405]: time=2026-09-20T09:07:13.690Z level=INFO msg="Upload complete. (365ms)"6473lb # [ 136.387115] niks3-server[1047]: 2026/09/20 09:07:14 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo6474worker1 # [ 136.433610] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56786 duration_s=0 paths=1 nar_bytes_in=6246475worker1 # [ 136.437703] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567866476worker1 # [ 136.442971] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)6477worker1 # [ 136.445143] nix-daemon[842]: reaped child process 1735, status = succeeded6478worker1 # [ 136.450261] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)6479worker1 # [ 137.462437] nix-daemon[842]: reaped child process 1749, status = succeeded6480worker1 # [ 137.598184] nix-daemon[842]: accepted connection from pid 1764, user nix-grpc-daemon (trusted)6481worker1 # [ 137.609770] nix-daemon[842]: reaped child process 1767, status = succeeded6482lb # [ 137.579819] niks3-server[1047]: 2026/09/20 09:07:15 INFO Received uploads request method=POST path=/api/pending_closures6483worker1 # [ 137.634105] nix-grpc-daemon[1405]: time=2026-09-20T09:07:14.893Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6484worker1 # [ 137.635615] nix-grpc-daemon[1405]: time=2026-09-20T09:07:14.894Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"6485lb # [ 137.609370] niks3-server[1047]: 2026/09/20 09:07:15 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv6486lb # [ 137.624430] niks3-server[1047]: 2026/09/20 09:07:15 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst6487lb # [ 137.634659] niks3-server[1047]: 2026/09/20 09:07:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6488worker1 # [ 137.686511] nix-grpc-daemon[1405]: time=2026-09-20T09:07:14.945Z level=INFO msg="Uploading 1 narinfos"6489lb # [ 137.640152] niks3-server[1047]: 2026/09/20 09:07:15 INFO Signed narinfos id=55 count=16490lb # [ 137.643689] niks3-server[1047]: 2026/09/20 09:07:15 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls6491lb # [ 137.652354] niks3-server[1047]: 2026/09/20 09:07:15 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6492lb # [ 137.658771] niks3-server[1047]: 2026/09/20 09:07:15 INFO Completed upload id=556493worker1 # [ 137.705162] nix-grpc-daemon[1405]: time=2026-09-20T09:07:14.964Z level=INFO msg="Upload complete. (246ms)"6494lb # [ 137.661095] niks3-server[1047]: 2026/09/20 09:07:15 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo6495worker1 # [ 137.717271] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56786 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=5 outputs=16496worker1 # [ 137.721670] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:14Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=16497worker1 # [ 137.726243] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:14Z level=debug event=want client=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=06498worker1 # [ 137.734162] nix-daemon[842]: reaped child process 1747, status = succeeded6499worker1 # [ 137.753929] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567866500worker1 # [ 137.764520] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:567866501worker1 # [ 137.775897] nix-grpc-daemon[1405]: time=2026-09-20T09:07:15.035Z level=INFO msg="All 1 paths already cached"6502worker1 # [ 137.782891] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:567866503worker1 # [ 137.790381] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)6504worker1 # [ 137.942141] nix-daemon[842]: accepted connection from pid 1773, user nix-grpc-daemon (trusted)6505worker1 # [ 137.953892] nix-daemon[842]: reaped child process 1776, status = succeeded6506worker1 # [ 138.103265] nix-daemon[842]: accepted connection from pid 1778, user nix-grpc-daemon (trusted)6507worker1 # [ 138.115635] nix-daemon[842]: reaped child process 1781, status = succeeded6508lb # [ 138.084242] niks3-server[1047]: 2026/09/20 09:07:15 INFO Received uploads request method=POST path=/api/pending_closures6509worker1 # [ 138.137767] nix-grpc-daemon[1405]: time=2026-09-20T09:07:15.396Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6510worker1 # [ 138.139236] nix-grpc-daemon[1405]: time=2026-09-20T09:07:15.398Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"6511lb # [ 138.111940] niks3-server[1047]: 2026/09/20 09:07:15 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst6512lb # [ 138.122115] niks3-server[1047]: 2026/09/20 09:07:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6513worker1 # [ 138.172840] nix-grpc-daemon[1405]: time=2026-09-20T09:07:15.432Z level=INFO msg="Uploading 1 narinfos"6514lb # [ 138.127061] niks3-server[1047]: 2026/09/20 09:07:15 INFO Signed narinfos id=56 count=16515lb # [ 138.133396] niks3-server[1047]: 2026/09/20 09:07:15 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls6516lb # [ 138.144302] niks3-server[1047]: 2026/09/20 09:07:15 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6517worker1 # [ 138.195889] nix-grpc-daemon[1405]: time=2026-09-20T09:07:15.455Z level=INFO msg="Upload complete. (395ms)"6518lb # [ 138.149808] niks3-server[1047]: 2026/09/20 09:07:15 INFO Completed upload id=566519worker1 # [ 138.197840] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56786 duration_s=0 paths=1 nar_bytes_in=6246520lb # [ 138.153358] niks3-server[1047]: 2026/09/20 09:07:15 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo6521worker1 # [ 138.203661] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567866522worker1 # [ 138.213389] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)6523worker1 # [ 138.215849] nix-daemon[842]: reaped child process 1771, status = succeeded6524worker1 # [ 138.220902] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)6525worker1 # [ 139.436702] nix-daemon[842]: reaped child process 1785, status = succeeded6526worker1 # [ 139.583327] nix-daemon[842]: accepted connection from pid 1800, user nix-grpc-daemon (trusted)6527worker1 # [ 139.595076] nix-daemon[842]: reaped child process 1803, status = succeeded6528lb # [ 139.565230] niks3-server[1047]: 2026/09/20 09:07:17 INFO Received uploads request method=POST path=/api/pending_closures6529worker1 # [ 139.620279] nix-grpc-daemon[1405]: time=2026-09-20T09:07:16.878Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6530worker1 # [ 139.621691] nix-grpc-daemon[1405]: time=2026-09-20T09:07:16.878Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"6531lb # [ 139.592441] niks3-server[1047]: 2026/09/20 09:07:17 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv6532lb # [ 139.606200] niks3-server[1047]: 2026/09/20 09:07:17 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst6533lb # [ 139.618396] niks3-server[1047]: 2026/09/20 09:07:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6534lb # [ 139.624717] niks3-server[1047]: 2026/09/20 09:07:17 INFO Signed narinfos id=57 count=16535worker1 # [ 139.673744] nix-grpc-daemon[1405]: time=2026-09-20T09:07:16.932Z level=INFO msg="Uploading 1 narinfos"6536lb # [ 139.627446] niks3-server[1047]: 2026/09/20 09:07:17 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls6537lb # [ 139.638409] niks3-server[1047]: 2026/09/20 09:07:17 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6538lb # [ 139.642847] niks3-server[1047]: 2026/09/20 09:07:17 INFO Completed upload id=576539worker1 # [ 139.689347] nix-grpc-daemon[1405]: time=2026-09-20T09:07:16.948Z level=INFO msg="Upload complete. (261ms)"6540lb # [ 139.645370] niks3-server[1047]: 2026/09/20 09:07:17 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo6541worker1 # [ 139.701170] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56786 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=6 outputs=16542worker1 # [ 139.705758] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:16Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=16543worker1 # [ 139.709623] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567866544worker1 # [ 139.712994] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:16Z level=debug event=want client=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=06545worker1 # [ 139.721929] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:567866546worker1 # [ 139.724176] nix-daemon[842]: reaped child process 1783, status = succeeded6547worker1 # [ 139.729256] nix-grpc-daemon[1405]: time=2026-09-20T09:07:16.988Z level=INFO msg="All 2 paths already cached"6548worker1 # [ 139.733397] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:567866549worker1 # [ 139.738240] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)6550worker1 # [ 139.887079] nix-daemon[842]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)6551worker1 # [ 139.899158] nix-daemon[842]: reaped child process 1812, status = succeeded6552worker1 # [ 140.054344] nix-daemon[842]: accepted connection from pid 1814, user nix-grpc-daemon (trusted)6553worker1 # [ 140.065779] nix-daemon[842]: reaped child process 1817, status = succeeded6554lb # [ 140.034537] niks3-server[1047]: 2026/09/20 09:07:17 INFO Received uploads request method=POST path=/api/pending_closures6555worker1 # [ 140.085232] nix-grpc-daemon[1405]: time=2026-09-20T09:07:17.344Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6556worker1 # [ 140.086782] nix-grpc-daemon[1405]: time=2026-09-20T09:07:17.346Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"6557lb # [ 140.051523] niks3-server[1047]: 2026/09/20 09:07:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6558lb # [ 140.054898] niks3-server[1047]: 2026/09/20 09:07:17 INFO Signed narinfos id=58 count=16559worker1 # [ 140.102862] nix-grpc-daemon[1405]: time=2026-09-20T09:07:17.362Z level=INFO msg="Uploading 1 narinfos"6560lb # [ 140.057235] niks3-server[1047]: 2026/09/20 09:07:17 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst6561lb # [ 140.064428] niks3-server[1047]: 2026/09/20 09:07:17 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls6562lb # [ 140.067765] niks3-server[1047]: 2026/09/20 09:07:17 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6563lb # [ 140.071149] niks3-server[1047]: 2026/09/20 09:07:17 INFO Completed upload id=586564worker1 # [ 140.118821] nix-grpc-daemon[1405]: time=2026-09-20T09:07:17.378Z level=INFO msg="Upload complete. (375ms)"6565lb # [ 140.072909] niks3-server[1047]: 2026/09/20 09:07:17 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo6566worker1 # [ 140.120592] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56786 duration_s=0 paths=1 nar_bytes_in=7526567worker1 # [ 140.123286] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567866568worker1 # [ 140.129258] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)6569worker1 # [ 140.132542] nix-daemon[842]: reaped child process 1807, status = succeeded6570worker1 # [ 140.137198] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)6571worker1 # [ 140.960147] nix-daemon[842]: reaped child process 1821, status = succeeded6572worker1 # [ 141.100637] nix-daemon[842]: accepted connection from pid 1836, user nix-grpc-daemon (trusted)6573worker1 # [ 141.112052] nix-daemon[842]: reaped child process 1839, status = succeeded6574lb # [ 141.076519] niks3-server[1047]: 2026/09/20 09:07:18 INFO Received uploads request method=POST path=/api/pending_closures6575worker1 # [ 141.130594] nix-grpc-daemon[1405]: time=2026-09-20T09:07:18.389Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6576worker1 # [ 141.132199] nix-grpc-daemon[1405]: time=2026-09-20T09:07:18.391Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"6577lb # [ 141.117201] niks3-server[1047]: 2026/09/20 09:07:18 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst6578lb # [ 141.121988] niks3-server[1047]: 2026/09/20 09:07:18 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls6579lb # [ 141.130692] niks3-server[1047]: 2026/09/20 09:07:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6580lb # [ 141.135365] niks3-server[1047]: 2026/09/20 09:07:18 INFO Signed narinfos id=59 count=16581worker1 # [ 141.184576] nix-grpc-daemon[1405]: time=2026-09-20T09:07:18.443Z level=INFO msg="Uploading 1 narinfos"6582lb # [ 141.140549] niks3-server[1047]: 2026/09/20 09:07:18 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv6583lb # [ 141.150266] niks3-server[1047]: 2026/09/20 09:07:18 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6584lb # [ 141.155350] niks3-server[1047]: 2026/09/20 09:07:18 INFO Completed upload id=596585worker1 # [ 141.203599] nix-grpc-daemon[1405]: time=2026-09-20T09:07:18.462Z level=INFO msg="Upload complete. (258ms)"6586lb # [ 141.158512] niks3-server[1047]: 2026/09/20 09:07:18 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo6587worker1 # [ 141.211681] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56786 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=7 outputs=16588worker1 # [ 141.217240] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:18Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=16589worker1 # [ 141.226920] nix-daemon[842]: reaped child process 1819, status = succeeded6590worker1 # [ 141.252955] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567866591worker1 # [ 141.264975] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:567866592worker1 # [ 141.277377] nix-grpc-daemon[1405]: time=2026-09-20T09:07:18.536Z level=INFO msg="All 2 paths already cached"6593worker1 # [ 141.284397] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:567866594worker1 # [ 141.294152] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)6595worker1 # [ 141.455552] nix-daemon[842]: accepted connection from pid 1843, user nix-grpc-daemon (trusted)6596worker1 # [ 141.467678] nix-daemon[842]: reaped child process 1846, status = succeeded6597worker1 # [ 141.600751] nix-daemon[842]: accepted connection from pid 1848, user nix-grpc-daemon (trusted)6598worker1 # [ 141.612777] nix-daemon[842]: reaped child process 1851, status = succeeded6599lb # [ 141.576144] niks3-server[1047]: 2026/09/20 09:07:19 INFO Received uploads request method=POST path=/api/pending_closures6600worker1 # [ 141.630305] nix-grpc-daemon[1405]: time=2026-09-20T09:07:18.889Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6601worker1 # [ 141.632090] nix-grpc-daemon[1405]: time=2026-09-20T09:07:18.890Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"6602lb # [ 141.600125] niks3-server[1047]: 2026/09/20 09:07:19 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst6603lb # [ 141.611361] niks3-server[1047]: 2026/09/20 09:07:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6604lb # [ 141.615512] niks3-server[1047]: 2026/09/20 09:07:19 INFO Signed narinfos id=60 count=16605worker1 # [ 141.664838] nix-grpc-daemon[1405]: time=2026-09-20T09:07:18.923Z level=INFO msg="Uploading 1 narinfos"6606lb # [ 141.624731] niks3-server[1047]: 2026/09/20 09:07:19 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls6607lb # [ 141.635160] niks3-server[1047]: 2026/09/20 09:07:19 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6608worker1 # [ 141.688380] nix-grpc-daemon[1405]: time=2026-09-20T09:07:18.947Z level=INFO msg="Upload complete. (383ms)"6609lb # [ 141.642083] niks3-server[1047]: 2026/09/20 09:07:19 INFO Completed upload id=606610worker1 # [ 141.690752] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56786 duration_s=0 paths=1 nar_bytes_in=7526611lb # [ 141.644510] niks3-server[1047]: 2026/09/20 09:07:19 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo6612worker1 # [ 141.696399] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567866613worker1 # [ 141.700661] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)6614worker1 # [ 141.702876] nix-daemon[842]: reaped child process 1841, status = succeeded6615worker1 # [ 141.707977] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)6616worker1 # [ 142.922861] nix-daemon[842]: reaped child process 1855, status = succeeded6617worker1 # [ 143.060543] nix-daemon[842]: accepted connection from pid 1870, user nix-grpc-daemon (trusted)6618worker1 # [ 143.072782] nix-daemon[842]: reaped child process 1873, status = succeeded6619lb # [ 143.042761] niks3-server[1047]: 2026/09/20 09:07:20 INFO Received uploads request method=POST path=/api/pending_closures6620worker1 # [ 143.099258] nix-grpc-daemon[1405]: time=2026-09-20T09:07:20.356Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6621worker1 # [ 143.100752] nix-grpc-daemon[1405]: time=2026-09-20T09:07:20.356Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"6622lb # [ 143.082645] niks3-server[1047]: 2026/09/20 09:07:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6623worker1 # [ 143.135106] nix-grpc-daemon[1405]: time=2026-09-20T09:07:20.394Z level=INFO msg="Uploading 1 narinfos"6624lb # [ 143.088708] niks3-server[1047]: 2026/09/20 09:07:20 INFO Signed narinfos id=61 count=16625lb # [ 143.097649] niks3-server[1047]: 2026/09/20 09:07:20 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst6626lb # [ 143.102881] niks3-server[1047]: 2026/09/20 09:07:20 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv6627lb # [ 143.110825] niks3-server[1047]: 2026/09/20 09:07:20 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls6628lb # [ 143.116917] niks3-server[1047]: 2026/09/20 09:07:20 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6629lb # [ 143.123206] niks3-server[1047]: 2026/09/20 09:07:20 INFO Completed upload id=616630lb # [ 143.124901] niks3-server[1047]: 2026/09/20 09:07:20 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo6631worker1 # [ 143.169732] nix-grpc-daemon[1405]: time=2026-09-20T09:07:20.428Z level=INFO msg="Upload complete. (253ms)"6632worker1 # [ 143.177742] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56786 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=8 outputs=16633worker1 # [ 143.184268] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:20Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=16634worker1 # [ 143.193230] nix-daemon[842]: reaped child process 1853, status = succeeded6635worker1 # [ 143.218522] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567866636worker1 # [ 143.229971] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:567866637worker1 # [ 143.241764] nix-grpc-daemon[1405]: time=2026-09-20T09:07:20.500Z level=INFO msg="All 2 paths already cached"6638worker1 # [ 143.248602] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:567866639worker1 # [ 143.258162] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)6640worker1 # [ 143.416722] nix-daemon[842]: accepted connection from pid 1877, user nix-grpc-daemon (trusted)6641worker1 # [ 143.428676] nix-daemon[842]: reaped child process 1880, status = succeeded6642worker1 # [ 143.577500] nix-daemon[842]: accepted connection from pid 1882, user nix-grpc-daemon (trusted)6643worker1 # [ 143.589886] nix-daemon[842]: reaped child process 1885, status = succeeded6644lb # [ 143.558219] niks3-server[1047]: 2026/09/20 09:07:21 INFO Received uploads request method=POST path=/api/pending_closures6645worker1 # [ 143.613025] nix-grpc-daemon[1405]: time=2026-09-20T09:07:20.871Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6646worker1 # [ 143.614355] nix-grpc-daemon[1405]: time=2026-09-20T09:07:20.872Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"6647lb # [ 143.589146] niks3-server[1047]: 2026/09/20 09:07:21 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst6648lb # [ 143.593386] niks3-server[1047]: 2026/09/20 09:07:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6649lb # [ 143.598599] niks3-server[1047]: 2026/09/20 09:07:21 INFO Signed narinfos id=62 count=16650worker1 # [ 143.648751] nix-grpc-daemon[1405]: time=2026-09-20T09:07:20.907Z level=INFO msg="Uploading 1 narinfos"6651lb # [ 143.605849] niks3-server[1047]: 2026/09/20 09:07:21 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls6652lb # [ 143.615097] niks3-server[1047]: 2026/09/20 09:07:21 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6653lb # [ 143.621045] niks3-server[1047]: 2026/09/20 09:07:21 INFO Completed upload id=626654worker1 # [ 143.667353] nix-grpc-daemon[1405]: time=2026-09-20T09:07:20.926Z level=INFO msg="Upload complete. (398ms)"6655lb # [ 143.622755] niks3-server[1047]: 2026/09/20 09:07:21 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo6656worker1 # [ 143.670576] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56786 duration_s=0 paths=1 nar_bytes_in=7526657worker1 # [ 143.676253] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567866658worker1 # [ 143.684460] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)6659worker1 # [ 143.688795] nix-daemon[842]: reaped child process 1875, status = succeeded6660worker1 # [ 143.693752] nix-daemon[842]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)6661worker1 # [ 144.907679] nix-daemon[842]: reaped child process 1889, status = succeeded6662worker1 # [ 145.043974] nix-daemon[842]: accepted connection from pid 1904, user nix-grpc-daemon (trusted)6663worker1 # [ 145.055880] nix-daemon[842]: reaped child process 1907, status = succeeded6664lb # [ 145.024687] niks3-server[1047]: 2026/09/20 09:07:22 INFO Received uploads request method=POST path=/api/pending_closures6665worker1 # [ 145.076107] nix-grpc-daemon[1405]: time=2026-09-20T09:07:22.335Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6666worker1 # [ 145.077618] nix-grpc-daemon[1405]: time=2026-09-20T09:07:22.336Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"6667lb # [ 145.047720] niks3-server[1047]: 2026/09/20 09:07:22 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv6668lb # [ 145.050483] niks3-server[1047]: 2026/09/20 09:07:22 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst6669lb # [ 145.053770] niks3-server[1047]: 2026/09/20 09:07:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6670lb # [ 145.056234] niks3-server[1047]: 2026/09/20 09:07:22 INFO Signed narinfos id=63 count=16671worker1 # [ 145.102891] nix-grpc-daemon[1405]: time=2026-09-20T09:07:22.361Z level=INFO msg="Uploading 1 narinfos"6672lb # [ 145.059599] niks3-server[1047]: 2026/09/20 09:07:22 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls6673lb # [ 145.064834] niks3-server[1047]: 2026/09/20 09:07:22 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6674worker1 # [ 145.114153] nix-grpc-daemon[1405]: time=2026-09-20T09:07:22.373Z level=INFO msg="Upload complete. (214ms)"6675lb # [ 145.068128] niks3-server[1047]: 2026/09/20 09:07:22 INFO Completed upload id=636676lb # [ 145.070856] niks3-server[1047]: 2026/09/20 09:07:22 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo6677worker1 # [ 145.116484] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56786 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=9 outputs=16678worker1 # [ 145.124227] nix-daemon[842]: reaped child process 1887, status = succeeded6679client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 16.60 seconds)6680worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -E 'nix_grpc_build_info\{.*features="[^"]*vip[^"]*".*worker="node-a"\} 1'6681worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -E 'nix_grpc_build_info\{.*features="[^"]*vip[^"]*".*worker="node-a"\} 1', in 0.04 seconds)6682client: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&16683worker1 # [ 145.374642] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:567866684worker2 # [ 145.324438] nix-grpc-daemon[1871]: ts=2026-09-20T09:07:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:563386685worker2 # [ 145.337096] nix-grpc-daemon[1871]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 81 ms (attempt 1/5)6686worker2 # [ 145.424149] nix-grpc-daemon[1871]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 15 ms (attempt 2/5)6687worker2 # [ 145.445076] nix-grpc-daemon[1871]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 4 ms (attempt 3/5)6688worker2 # [ 145.455057] nix-grpc-daemon[1871]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 2 ms (attempt 4/5)6689worker2 # [ 145.462899] nix-grpc-daemon[1871]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6690worker1 # [ 145.540069] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:22Z level=debug event=want client=2 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=06691lb # [ 146.645731] envoy[918]: conn peer=[2001:db8:1::1]:33582 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27059 tx=10624 ms=163676692client: (finished: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&1, in 30.36 seconds)6693(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 47.00 seconds)6694subtest: low disk drains a worker and builds go to the other6695worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6696worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.98 seconds)6697worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6698lb # [ 176.646326] envoy[918]: conn peer=[2001:db8:1::1]:41608 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=912 tx=516 ms=301786699worker1 # [ 177.258952] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:54Z level=info event=unhealthy reason=min_free6700worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.12 seconds)6701client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&26702client # don't know how to build these paths:6703client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6704worker2 # [ 177.781279] nix-grpc-daemon[1871]: ts=2026-09-20T09:07:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:563386705worker2 # [ 177.786979] nix-grpc-daemon[1871]: ts=2026-09-20T09:07:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:563386706worker1 # [ 177.863051] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:55Z level=debug event=want client=3 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=06707worker1 # [ 177.866478] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:55Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=16708worker2 # [ 177.850402] nix-grpc-daemon[1871]: ts=2026-09-20T09:07:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:563386709worker2 # [ 177.863365] nix-grpc-daemon[1871]: ts=2026-09-20T09:07:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:563386710client # copying 1 paths to 'lb:50051'...6711client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...6712worker2 # [ 177.876062] nix-grpc-daemon[1871]: ts=2026-09-20T09:07:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:563386713worker2 # [ 177.888667] nix-daemon[844]: accepted connection from pid 1871, user nix-grpc-daemon (trusted)6714worker2 # [ 177.907652] nix-grpc-daemon[1871]: ts=2026-09-20T09:07:56Z level=info event=niks3_push_started pid=19276715worker2 # [ 178.223484] nix-daemon[844]: accepted connection from pid 1940, user nix-grpc-daemon (trusted)6716worker2 # [ 178.238108] nix-daemon[844]: reaped child process 1943, status = succeeded6717worker2 # [ 178.404236] nix-daemon[844]: accepted connection from pid 1945, user nix-grpc-daemon (trusted)6718worker2 # [ 178.418612] nix-daemon[844]: reaped child process 1948, status = succeeded6719lb # [ 178.444279] niks3-server[1047]: 2026/09/20 09:07:56 INFO Received uploads request method=POST path=/api/pending_closures6720worker2 # [ 178.444658] nix-grpc-daemon[1927]: time=2026-09-20T09:07:56.657Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6721worker2 # [ 178.448146] nix-grpc-daemon[1927]: time=2026-09-20T09:07:56.659Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"6722lb # [ 178.484323] niks3-server[1047]: 2026/09/20 09:07:56 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst6723lb # [ 178.490607] niks3-server[1047]: 2026/09/20 09:07:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6724lb # [ 178.496072] niks3-server[1047]: 2026/09/20 09:07:56 INFO Signed narinfos id=64 count=16725worker2 # [ 178.489399] nix-grpc-daemon[1927]: time=2026-09-20T09:07:56.702Z level=INFO msg="Uploading 1 narinfos"6726lb # [ 178.509927] niks3-server[1047]: 2026/09/20 09:07:56 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls6727lb # [ 178.516965] niks3-server[1047]: 2026/09/20 09:07:56 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6728lb # [ 178.525062] niks3-server[1047]: 2026/09/20 09:07:56 INFO Completed upload id=646729worker2 # [ 178.515385] nix-grpc-daemon[1927]: time=2026-09-20T09:07:56.728Z level=INFO msg="Upload complete. (435ms)"6730lb # [ 178.528754] niks3-server[1047]: 2026/09/20 09:07:56 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo6731worker2 # [ 178.519452] nix-grpc-daemon[1871]: ts=2026-09-20T09:07:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56338 duration_s=0 paths=1 nar_bytes_in=5126732worker2 # [ 178.525715] nix-grpc-daemon[1871]: ts=2026-09-20T09:07:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:563386733worker2 # [ 178.536475] nix-daemon[844]: accepted connection from pid 1871, user nix-grpc-daemon (trusted)6734worker2 # [ 178.541114] nix-daemon[844]: reaped child process 1925, status = succeeded6735worker2 # [ 178.546188] nix-daemon[844]: accepted connection from pid 1871, user nix-grpc-daemon (trusted)6736client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...6737client # farm-a> worker2-next: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6738client # farm-a> LOG-farm-a-drain6739worker2 # [ 179.772293] nix-daemon[844]: reaped child process 1952, status = succeeded6740worker2 # [ 179.887293] nix-daemon[844]: accepted connection from pid 1967, user nix-grpc-daemon (trusted)6741worker2 # [ 179.901972] nix-daemon[844]: reaped child process 1970, status = succeeded6742lb # [ 179.921449] niks3-server[1047]: 2026/09/20 09:07:57 INFO Received uploads request method=POST path=/api/pending_closures6743worker2 # [ 179.918349] nix-grpc-daemon[1927]: time=2026-09-20T09:07:58.130Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6744worker2 # [ 179.922372] nix-grpc-daemon[1927]: time=2026-09-20T09:07:58.130Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"6745lb # [ 179.943805] niks3-server[1047]: 2026/09/20 09:07:57 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6746lb # [ 179.946383] niks3-server[1047]: 2026/09/20 09:07:57 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6747lb # [ 179.949901] niks3-server[1047]: 2026/09/20 09:07:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6748lb # [ 179.951840] niks3-server[1047]: 2026/09/20 09:07:57 INFO Signed narinfos id=65 count=16749worker2 # [ 179.943646] nix-grpc-daemon[1927]: time=2026-09-20T09:07:58.156Z level=INFO msg="Uploading 1 narinfos"6750lb # [ 179.958776] niks3-server[1047]: 2026/09/20 09:07:57 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls6751lb # [ 179.961748] niks3-server[1047]: 2026/09/20 09:07:57 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6752lb # [ 179.966106] niks3-server[1047]: 2026/09/20 09:07:57 INFO Completed upload id=656753worker2 # [ 179.956096] nix-grpc-daemon[1927]: time=2026-09-20T09:07:58.169Z level=INFO msg="Upload complete. (201ms)"6754lb # [ 179.967656] niks3-server[1047]: 2026/09/20 09:07:57 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo6755worker2 # [ 179.958958] nix-grpc-daemon[1871]: ts=2026-09-20T09:07:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56338 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=10 outputs=16756worker2 # [ 179.966843] nix-daemon[844]: reaped child process 1950, status = succeeded6757worker1 # [ 180.024785] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:57Z level=debug event=want client=3 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=06758worker1 # [ 180.028775] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:57Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=16759worker2 # [ 180.010647] nix-grpc-daemon[1871]: ts=2026-09-20T09:07:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:563386760worker2 # [ 180.019540] nix-grpc-daemon[1871]: ts=2026-09-20T09:07:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:563386761worker2 # [ 180.029188] nix-grpc-daemon[1927]: time=2026-09-20T09:07:58.242Z level=INFO msg="All 1 paths already cached"6762client # copying 1 paths to 'lb:50051'...6763client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...6764worker2 # [ 180.035962] nix-grpc-daemon[1871]: ts=2026-09-20T09:07:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:563386765worker2 # [ 180.043415] nix-daemon[844]: accepted connection from pid 1871, user nix-grpc-daemon (trusted)6766worker2 # [ 180.186746] nix-daemon[844]: accepted connection from pid 1974, user nix-grpc-daemon (trusted)6767worker2 # [ 180.201309] nix-daemon[844]: reaped child process 1977, status = succeeded6768worker2 # [ 180.348441] nix-daemon[844]: accepted connection from pid 1979, user nix-grpc-daemon (trusted)6769worker2 # [ 180.362489] nix-daemon[844]: reaped child process 1982, status = succeeded6770lb # [ 180.386259] niks3-server[1047]: 2026/09/20 09:07:58 INFO Received uploads request method=POST path=/api/pending_closures6771worker2 # [ 180.381378] nix-grpc-daemon[1927]: time=2026-09-20T09:07:58.594Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6772worker2 # [ 180.382875] nix-grpc-daemon[1927]: time=2026-09-20T09:07:58.596Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"6773lb # [ 180.403533] niks3-server[1047]: 2026/09/20 09:07:58 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst6774lb # [ 180.407146] niks3-server[1047]: 2026/09/20 09:07:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6775lb # [ 180.409788] niks3-server[1047]: 2026/09/20 09:07:58 INFO Signed narinfos id=66 count=16776worker2 # [ 180.401081] nix-grpc-daemon[1927]: time=2026-09-20T09:07:58.614Z level=INFO msg="Uploading 1 narinfos"6777lb # [ 180.412892] niks3-server[1047]: 2026/09/20 09:07:58 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls6778lb # [ 180.418794] niks3-server[1047]: 2026/09/20 09:07:58 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6779lb # [ 180.421970] niks3-server[1047]: 2026/09/20 09:07:58 INFO Completed upload id=666780worker2 # [ 180.413165] nix-grpc-daemon[1927]: time=2026-09-20T09:07:58.626Z level=INFO msg="Upload complete. (358ms)"6781lb # [ 180.424147] niks3-server[1047]: 2026/09/20 09:07:58 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo6782worker2 # [ 180.414743] nix-grpc-daemon[1871]: ts=2026-09-20T09:07:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56338 duration_s=0 paths=1 nar_bytes_in=6246783worker2 # [ 180.420471] nix-grpc-daemon[1871]: ts=2026-09-20T09:07:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:563386784worker2 # [ 180.428378] nix-daemon[844]: accepted connection from pid 1871, user nix-grpc-daemon (trusted)6785worker2 # [ 180.431748] nix-daemon[844]: reaped child process 1972, status = succeeded6786worker2 # [ 180.437324] nix-daemon[844]: accepted connection from pid 1871, user nix-grpc-daemon (trusted)6787client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...6788client # farm-b> worker2-next: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6789client # farm-b> LOG-farm-b-drain6790worker2 # [ 181.453330] nix-daemon[844]: reaped child process 1986, status = succeeded6791worker2 # [ 181.579824] nix-daemon[844]: accepted connection from pid 2001, user nix-grpc-daemon (trusted)6792worker2 # [ 181.593769] nix-daemon[844]: reaped child process 2004, status = succeeded6793lb # [ 181.619716] niks3-server[1047]: 2026/09/20 09:07:59 INFO Received uploads request method=POST path=/api/pending_closures6794worker2 # [ 181.619373] nix-grpc-daemon[1927]: time=2026-09-20T09:07:59.832Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6795worker2 # [ 181.621391] nix-grpc-daemon[1927]: time=2026-09-20T09:07:59.834Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"6796lb # [ 181.650887] niks3-server[1047]: 2026/09/20 09:07:59 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6797lb # [ 181.666489] niks3-server[1047]: 2026/09/20 09:07:59 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst6798lb # [ 181.675701] niks3-server[1047]: 2026/09/20 09:07:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6799lb # [ 181.680500] niks3-server[1047]: 2026/09/20 09:07:59 INFO Signed narinfos id=67 count=16800worker2 # [ 181.673222] nix-grpc-daemon[1927]: time=2026-09-20T09:07:59.886Z level=INFO msg="Uploading 1 narinfos"6801lb # [ 181.687968] niks3-server[1047]: 2026/09/20 09:07:59 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls6802lb # [ 181.697654] niks3-server[1047]: 2026/09/20 09:07:59 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6803lb # [ 181.702082] niks3-server[1047]: 2026/09/20 09:07:59 INFO Completed upload id=676804worker2 # [ 181.693872] nix-grpc-daemon[1927]: time=2026-09-20T09:07:59.907Z level=INFO msg="Upload complete. (249ms)"6805lb # [ 181.705031] niks3-server[1047]: 2026/09/20 09:07:59 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo6806worker2 # [ 181.698365] nix-grpc-daemon[1871]: ts=2026-09-20T09:07:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56338 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=11 outputs=16807worker2 # [ 181.705662] nix-daemon[844]: reaped child process 1984, status = succeeded6808worker1 # [ 181.760790] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:59Z level=debug event=want client=3 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=06809worker1 # [ 181.765188] nix-grpc-daemon[1321]: ts=2026-09-20T09:07:59Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=16810worker2 # [ 181.747934] nix-grpc-daemon[1871]: ts=2026-09-20T09:07:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:563386811worker2 # [ 181.761620] nix-grpc-daemon[1871]: ts=2026-09-20T09:07:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:563386812worker2 # [ 181.774611] nix-grpc-daemon[1927]: time=2026-09-20T09:07:59.987Z level=INFO msg="All 2 paths already cached"6813client # copying 1 paths to 'lb:50051'...6814client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...6815worker2 # [ 181.783126] nix-grpc-daemon[1871]: ts=2026-09-20T09:07:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:563386816worker2 # [ 181.791640] nix-daemon[844]: accepted connection from pid 1871, user nix-grpc-daemon (trusted)6817worker2 # [ 181.967585] nix-daemon[844]: accepted connection from pid 2008, user nix-grpc-daemon (trusted)6818worker2 # [ 181.981974] nix-daemon[844]: reaped child process 2011, status = succeeded6819worker2 # [ 182.118963] nix-daemon[844]: accepted connection from pid 2013, user nix-grpc-daemon (trusted)6820worker2 # [ 182.134163] nix-daemon[844]: reaped child process 2016, status = succeeded6821lb # [ 182.158533] niks3-server[1047]: 2026/09/20 09:07:59 INFO Received uploads request method=POST path=/api/pending_closures6822worker2 # [ 182.156604] nix-grpc-daemon[1927]: time=2026-09-20T09:08:00.369Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6823worker2 # [ 182.158630] nix-grpc-daemon[1927]: time=2026-09-20T09:08:00.371Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"6824lb # [ 182.188136] niks3-server[1047]: 2026/09/20 09:07:59 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst6825lb # [ 182.199089] niks3-server[1047]: 2026/09/20 09:07:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6826lb # [ 182.203781] niks3-server[1047]: 2026/09/20 09:07:59 INFO Signed narinfos id=68 count=16827worker2 # [ 182.196907] nix-grpc-daemon[1927]: time=2026-09-20T09:08:00.410Z level=INFO msg="Uploading 1 narinfos"6828lb # [ 182.210564] niks3-server[1047]: 2026/09/20 09:07:59 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls6829lb # [ 182.219734] niks3-server[1047]: 2026/09/20 09:07:59 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6830worker2 # [ 182.216140] nix-grpc-daemon[1927]: time=2026-09-20T09:08:00.429Z level=INFO msg="Upload complete. (409ms)"6831lb # [ 182.226166] niks3-server[1047]: 2026/09/20 09:07:59 INFO Completed upload id=686832worker2 # [ 182.218425] nix-grpc-daemon[1871]: ts=2026-09-20T09:08:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56338 duration_s=0 paths=1 nar_bytes_in=7526833lb # [ 182.231294] niks3-server[1047]: 2026/09/20 09:07:59 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo6834worker2 # [ 182.221675] nix-grpc-daemon[1871]: ts=2026-09-20T09:08:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:563386835worker2 # [ 182.227847] nix-daemon[844]: accepted connection from pid 1871, user nix-grpc-daemon (trusted)6836worker2 # [ 182.232686] nix-daemon[844]: reaped child process 2006, status = succeeded6837worker2 # [ 182.237986] nix-daemon[844]: accepted connection from pid 1871, user nix-grpc-daemon (trusted)6838client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...6839client # farm-top> worker2-next: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6840client # farm-top> LOG-farm-top-drain6841worker2 # [ 182.852290] nix-daemon[844]: reaped child process 2020, status = succeeded6842worker2 # [ 182.978338] nix-daemon[844]: accepted connection from pid 2035, user nix-grpc-daemon (trusted)6843worker2 # [ 182.993679] nix-daemon[844]: reaped child process 2038, status = succeeded6844lb # [ 183.018142] niks3-server[1047]: 2026/09/20 09:08:00 INFO Received uploads request method=POST path=/api/pending_closures6845worker2 # [ 183.013210] nix-grpc-daemon[1927]: time=2026-09-20T09:08:01.226Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6846worker2 # [ 183.014671] nix-grpc-daemon[1927]: time=2026-09-20T09:08:01.227Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"6847lb # [ 183.042642] niks3-server[1047]: 2026/09/20 09:08:00 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst6848lb # [ 183.045331] niks3-server[1047]: 2026/09/20 09:08:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6849worker2 # [ 183.037309] nix-grpc-daemon[1927]: time=2026-09-20T09:08:01.250Z level=INFO msg="Uploading 1 narinfos"6850lb # [ 183.047577] niks3-server[1047]: 2026/09/20 09:08:00 INFO Signed narinfos id=69 count=16851lb # [ 183.049942] niks3-server[1047]: 2026/09/20 09:08:00 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6852lb # [ 183.054504] niks3-server[1047]: 2026/09/20 09:08:00 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls6853lb # [ 183.059190] niks3-server[1047]: 2026/09/20 09:08:00 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6854lb # [ 183.062246] niks3-server[1047]: 2026/09/20 09:08:00 INFO Completed upload id=696855worker2 # [ 183.052332] nix-grpc-daemon[1927]: time=2026-09-20T09:08:01.265Z level=INFO msg="Upload complete. (206ms)"6856lb # [ 183.065250] niks3-server[1047]: 2026/09/20 09:08:00 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo6857worker2 # [ 183.054929] nix-grpc-daemon[1871]: ts=2026-09-20T09:08:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56338 duration_s=0 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=12 outputs=16858worker2 # [ 183.062428] nix-daemon[844]: reaped child process 2018, status = succeeded6859client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&2, in 5.49 seconds)6860client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath6861client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath, in 0.18 seconds)6862worker1: must succeed: rm /nix/.rw-store/fill6863worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.11 seconds)6864worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6865worker1 # [ 184.261368] nix-grpc-daemon[1321]: ts=2026-09-20T09:08:01Z level=info event=healthy reason=min_free6866worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.09 seconds)6867(finished: subtest: low disk drains a worker and builds go to the other, in 9.02 seconds)6868(finished: run the VM test script, in 185.48 seconds)6869test script finished in 185.68s6870cleanup6871kill QemuMachine (pid 49)6872client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6873kill QemuMachine (pid 51)6874lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6875kill QemuMachine (pid 50)6876worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6877kill QemuMachine (pid 52)6878worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6879(finished: cleanup, in 1.02 seconds)6880additionally exposed symbols:6881 client, lb, worker1, worker2,6882 vlan1,6883 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh