tribuchet: building on jamie Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 1200.0 seconds run the VM test script machine: waiting for unit k3s.service machine: waiting for the VM to finish booting machine: starting vm machine: QEMU running (pid 45) machine # Disk image does not exist, creating the virtualisation disk image... machine # Formatting '/build/vm-state-machine/tmp.wKbwUPuIiA', fmt=raw size=17179869184 machine # mke2fs 1.47.4 (6-Mar-2025) machine # Discarding device blocks: 0/4194304 done machine # Creating filesystem with 4194304 4k blocks and 1048576 inodes machine # Filesystem UUID: cfcf699b-8797-4084-9fb0-6e45c5d7acd4 machine # Superblock backups stored on blocks: machine # 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, machine # 4096000 machine # machine # Allocating group tables: 0/128 done machine # Writing inode tables: 0/128 done machine # Creating journal (32768 blocks): done machine # Writing superblocks and filesystem accounting information: 0/128 done machine # machine # Virtualisation disk image created. machine # Starting virtiofs daemons... machine # [2026-09-23T04:42:26Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) machine # [2026-09-23T04:42:26Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether machine # [2026-09-23T04:42:26Z INFO virtiofsd] Waiting for vhost-user socket connection... machine # [2026-09-23T04:42:26Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) machine # [2026-09-23T04:42:26Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether machine # [2026-09-23T04:42:26Z INFO virtiofsd] Waiting for vhost-user socket connection... machine # [2026-09-23T04:42:26Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) machine # [2026-09-23T04:42:26Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether machine # [2026-09-23T04:42:26Z INFO virtiofsd] Waiting for vhost-user socket connection... machine # [2026-09-23T04:42:26Z INFO virtiofsd] Client connected, servicing requests machine # [2026-09-23T04:42:26Z INFO virtiofsd] Client connected, servicing requests machine # [2026-09-23T04:42:26Z INFO virtiofsd] Client connected, servicing requests machine # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) machine # machine # machine # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+7EFCC730+7EF2C730 CA00 machine # Press Ctrl-B to configure iPXE (PCI 00:02.0)... machine # machine # machine # machine # machine # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 7EFCC730 7EF2C730 CB00 machine # Press Ctrl-B to configure iPXE (PCI 00:05.0)... machine # machine # machine # Booting from ROM... machine # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026 machine # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/q7hgjsikzjv4i374iaff8zvjhi589aad-nixos-system-machine-test/init regInfo=/nix/store/hbw5z8arskq16721rssmhpcpg3vw1ps6-closure-info/registration console=ttyS0,115200n8 console=tty0 machine # [ 0.000000] BIOS-provided physical RAM map: machine # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable machine # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved machine # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved machine # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007ffd7fff] usable machine # [ 0.000000] BIOS-e820: [mem 0x000000007ffd8000-0x000000007fffffff] reserved machine # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved machine # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved machine # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved machine # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved machine # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000017fffffff] usable machine # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved machine # [ 0.000000] NX (Execute Disable) protection: active machine # [ 0.000000] APIC: Static calls initialized machine # [ 0.000000] SMBIOS 2.8 present. machine # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 machine # [ 0.000000] DMI: Memory slots populated: 1/1 machine # [ 0.000000] Hypervisor detected: KVM machine # [ 0.000000] last_pfn = 0x7ffd8 max_arch_pfn = 0x10000000000 machine # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 machine # [ 0.000000] kvm-clock: using sched offset of 478220872 cycles machine # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns machine # [ 0.000005] tsc: Detected 2400.012 MHz processor machine # [ 0.000812] last_pfn = 0x180000 max_arch_pfn = 0x10000000000 machine # [ 0.000839] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs machine # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT machine # [ 0.000880] last_pfn = 0x7ffd8 max_arch_pfn = 0x10000000000 machine # [ 0.002731] found SMP MP-table at [mem 0x000f5450-0x000f545f] machine # [ 0.002742] Using GB pages for direct mapping machine # [ 0.002837] RAMDISK: [mem 0x7e371000-0x7ffcffff] machine # [ 0.002844] ACPI: Early table checksum verification disabled machine # [ 0.002846] ACPI: RSDP 0x00000000000F5250 000014 (v00 BOCHS ) machine # [ 0.002850] ACPI: RSDT 0x000000007FFE2539 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) machine # [ 0.002854] ACPI: FACP 0x000000007FFE2319 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001) machine # [ 0.002861] ACPI: DSDT 0x000000007FFE0040 0022D9 (v01 BOCHS BXPC 00000001 BXPC 00000001) machine # [ 0.002863] ACPI: FACS 0x000000007FFE0000 000040 machine # [ 0.002864] ACPI: APIC 0x000000007FFE240D 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) machine # [ 0.002866] ACPI: HPET 0x000000007FFE249D 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) machine # [ 0.002867] ACPI: MCFG 0x000000007FFE24D5 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001) machine # [ 0.002869] ACPI: WAET 0x000000007FFE2511 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) machine # [ 0.002870] ACPI: Reserving FACP table memory at [mem 0x7ffe2319-0x7ffe240c] machine # [ 0.002871] ACPI: Reserving DSDT table memory at [mem 0x7ffe0040-0x7ffe2318] machine # [ 0.002872] ACPI: Reserving FACS table memory at [mem 0x7ffe0000-0x7ffe003f] machine # [ 0.002872] ACPI: Reserving APIC table memory at [mem 0x7ffe240d-0x7ffe249c] machine # [ 0.002873] ACPI: Reserving HPET table memory at [mem 0x7ffe249d-0x7ffe24d4] machine # [ 0.002873] ACPI: Reserving MCFG table memory at [mem 0x7ffe24d5-0x7ffe2510] machine # [ 0.002874] ACPI: Reserving WAET table memory at [mem 0x7ffe2511-0x7ffe2538] machine # [ 0.003090] No NUMA configuration found machine # [ 0.003091] Faking a node at [mem 0x0000000000000000-0x000000017fffffff] machine # [ 0.003093] NODE_DATA(0) allocated [mem 0x17fff8780-0x17fffdcff] machine # [ 0.005302] Zone ranges: machine # [ 0.005303] DMA [mem 0x0000000000001000-0x0000000000ffffff] machine # [ 0.005305] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] machine # [ 0.005312] Normal [mem 0x0000000100000000-0x000000017fffffff] machine # [ 0.005313] Device empty machine # [ 0.005313] Movable zone start for each node machine # [ 0.005314] Early memory node ranges machine # [ 0.005314] node 0: [mem 0x0000000000001000-0x000000000009efff] machine # [ 0.005315] node 0: [mem 0x0000000000100000-0x000000007ffd7fff] machine # [ 0.005316] node 0: [mem 0x0000000100000000-0x000000017fffffff] machine # [ 0.005317] Initmem setup node 0 [mem 0x0000000000001000-0x000000017fffffff] machine # [ 0.005345] On node 0, zone DMA: 1 pages in unavailable ranges machine # [ 0.005601] On node 0, zone DMA: 97 pages in unavailable ranges machine # [ 0.074627] On node 0, zone Normal: 40 pages in unavailable ranges machine # [ 0.075093] ACPI: PM-Timer IO Port: 0x608 machine # [ 0.075107] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) machine # [ 0.075136] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 machine # [ 0.075139] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) machine # [ 0.075141] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) machine # [ 0.075142] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) machine # [ 0.075143] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) machine # [ 0.075143] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) machine # [ 0.075146] ACPI: Using ACPI (MADT) for SMP configuration information machine # [ 0.075147] ACPI: HPET id: 0x8086a201 base: 0xfed00000 machine # [ 0.075152] TSC deadline timer available machine # [ 0.075156] CPU topo: Max. logical packages: 1 machine # [ 0.075156] CPU topo: Max. logical dies: 1 machine # [ 0.075157] CPU topo: Max. dies per package: 1 machine # [ 0.075160] CPU topo: Max. threads per core: 1 machine # [ 0.075161] CPU topo: Num. cores per package: 4 machine # [ 0.075161] CPU topo: Num. threads per package: 4 machine # [ 0.075161] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs machine # [ 0.075184] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() machine # [ 0.075199] kvm-guest: KVM setup pv remote TLB flush machine # [ 0.075202] kvm-guest: setup PV sched yield machine # [ 0.075212] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] machine # [ 0.075213] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] machine # [ 0.075214] PM: hibernation: Registered nosave memory: [mem 0x7ffd8000-0xffffffff] machine # [ 0.075216] [mem 0xc0000000-0xfed1bfff] available for PCI devices machine # [ 0.075217] Booting paravirtualized kernel on KVM machine # [ 0.075220] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns machine # [ 0.079687] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 machine # [ 0.081973] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288 machine # [ 0.082017] kvm-guest: PV spinlocks enabled machine # [ 0.082018] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) machine # [ 0.082021] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/q7hgjsikzjv4i374iaff8zvjhi589aad-nixos-system-machine-test/init regInfo=/nix/store/hbw5z8arskq16721rssmhpcpg3vw1ps6-closure-info/registration console=ttyS0,115200n8 console=tty0 machine # [ 0.082115] Unknown kernel command line parameters "regInfo=/nix/store/hbw5z8arskq16721rssmhpcpg3vw1ps6-closure-info/registration", will be passed to user space. machine # [ 0.082128] random: crng init done machine # [ 0.082129] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes machine # [ 0.086264] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) machine # [ 0.088327] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) machine # [ 0.088398] software IO TLB: area num 4. machine # [ 0.159439] Fallback order for Node 0: 0 machine # [ 0.159447] Built 1 zonelists, mobility grouping on. Total pages: 1048438 machine # [ 0.159448] Policy zone: Normal machine # [ 0.161864] mem auto-init: stack:all(zero), heap alloc:on, heap free:off machine # [ 0.168542] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 machine # [ 0.177332] allocated 8388608 bytes of page_ext machine # [ 0.186965] ftrace: allocating 48787 entries in 192 pages machine # [ 0.186968] ftrace: allocated 192 pages with 2 groups machine # [ 0.187847] Dynamic Preempt: lazy machine # [ 0.187999] rcu: Preemptible hierarchical RCU implementation. machine # [ 0.188000] rcu: RCU event tracing is enabled. machine # [ 0.188001] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4. machine # [ 0.188002] Trampoline variant of Tasks RCU enabled. machine # [ 0.188003] Rude variant of Tasks RCU enabled. machine # [ 0.188003] Tracing variant of Tasks RCU enabled. machine # [ 0.188004] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. machine # [ 0.188004] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 machine # [ 0.188022] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. machine # [ 0.188024] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. machine # [ 0.188025] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. machine # [ 0.192453] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16 machine # [ 0.192770] rcu: srcu_init: Setting srcu_struct sizes based on contention. machine # [ 0.192778] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns machine # [ 0.192889] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) machine # [ 0.196453] Console: colour VGA+ 80x25 machine # [ 0.196456] printk: legacy console [tty0] enabled machine # [ 0.228104] printk: legacy console [ttyS0] enabled machine # [ 0.340761] ACPI: Core revision 20250807 machine # [ 0.341677] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns machine # [ 0.343274] APIC: Switch to symmetric I/O mode setup machine # [ 0.344363] x2apic enabled machine # [ 0.345112] APIC: Switched APIC routing to: physical x2apic machine # [ 0.346081] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask() machine # [ 0.347395] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself() machine # [ 0.348913] kvm-guest: setup PV IPIs machine # [ 0.350519] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 machine # [ 0.351602] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns machine # [ 0.355973] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012) machine # [ 0.358057] x86/cpu: User Mode Instruction Prevention (UMIP) activated machine # [ 0.359127] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 machine # [ 0.359971] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 machine # [ 0.360974] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto machine # [ 0.362970] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl machine # [ 0.363970] Transient Scheduler Attacks: Vulnerable: No microcode machine # [ 0.364970] Spectre V2 : Mitigation: Enhanced / Automatic IBRS machine # [ 0.365970] Speculative Return Stack Overflow: Mitigation: Safe RET machine # [ 0.366970] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization machine # [ 0.367977] Spectre V2 : Enabling IBPB for BPF machine # [ 0.368971] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier machine # [ 0.369971] active return thunk: srso_alias_return_thunk machine # [ 0.370887] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' machine # [ 0.371970] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' machine # [ 0.372970] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' machine # [ 0.373970] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' machine # [ 0.374970] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' machine # [ 0.375970] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' machine # [ 0.377841] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' machine # [ 0.378970] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 machine # [ 0.379970] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 machine # [ 0.380970] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 machine # [ 0.381970] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 machine # [ 0.382970] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 machine # [ 0.383970] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format. machine # [ 0.414108] Freeing SMP alternatives memory: 44K machine # [ 0.414930] pid_max: default: 32768 minimum: 301 machine # [ 0.415674] LSM: initializing lsm=capability,landlock,yama,bpf,ima machine # [ 0.417032] landlock: Up and running. machine # [ 0.417688] Yama: becoming mindful. machine # [ 0.418183] LSM support for eBPF active machine # [ 0.419126] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear) machine # [ 0.420043] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear) machine # [ 0.423401] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) machine # [ 0.424712] Performance Events: Fam17h+ core perfctr, AMD PMU driver. machine # [ 0.424975] ... version: 2 machine # [ 0.425728] ... bit width: 48 machine # [ 0.425972] ... generic counters: 6 machine # [ 0.426734] ... generic bitmap: 000000000000003f machine # [ 0.427784] ... fixed-purpose counters: 0 machine # [ 0.428530] ... fixed-purpose bitmap: 0000000000000000 machine # [ 0.428980] ... value mask: 0000ffffffffffff machine # [ 0.429909] ... max period: 00007fffffffffff machine # [ 0.430685] ... global_ctrl mask: 000000000000003f machine # [ 0.431098] signal: max sigframe size: 3376 machine # [ 0.431910] rcu: Hierarchical SRCU implementation. machine # [ 0.432607] rcu: Max phase no-delay instances is 400. machine # [ 0.433145] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level machine # [ 0.438492] smp: Bringing up secondary CPUs ... machine # [ 0.439353] smpboot: x86: Booting SMP configuration: machine # [ 0.439980] .... node #0, CPUs: #1 #2 #3 machine # [ 0.443113] smp: Brought up 1 node, 4 CPUs machine # [ 0.444729] smpboot: Total of 4 processors activated (19200.09 BogoMIPS) machine # [ 0.445308] Memory: 3957868K/4193752K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 220856K reserved, 0K cma-reserved) machine # [ 0.447972] devtmpfs: initialized machine # [ 0.449128] x86/mm: Memory block size: 128MB machine # [ 0.452038] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear) machine # [ 0.453049] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear). machine # [ 0.454052] pinctrl core: initialized pinctrl subsystem machine # [ 0.455292] PM: RTC time: 04:42:26, date: 2026-09-23 machine # [ 0.458827] NET: Registered PF_NETLINK/PF_ROUTE protocol family machine # [ 0.460561] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations machine # [ 0.461003] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations machine # [ 0.462509] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations machine # [ 0.462984] audit: initializing netlink subsys (disabled) machine # [ 0.464102] audit: type=2000 audit(1790138547.099:1): state=initialized audit_enabled=0 res=1 machine # [ 0.464270] thermal_sys: Registered thermal governor 'fair_share' machine # [ 0.465973] thermal_sys: Registered thermal governor 'bang_bang' machine # [ 0.466973] thermal_sys: Registered thermal governor 'step_wise' machine # [ 0.467972] thermal_sys: Registered thermal governor 'user_space' machine # [ 0.468972] thermal_sys: Registered thermal governor 'power_allocator' machine # [ 0.469988] cpuidle: using governor menu machine # [ 0.472218] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 machine # [ 0.473229] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff] machine # [ 0.473977] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry machine # [ 0.474987] PCI: Using configuration type 1 for base access machine # [ 0.476088] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. machine # [ 0.479344] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages machine # [ 0.480953] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page machine # [ 0.481828] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages machine # [ 0.482892] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page machine # [ 0.487535] ACPI: Added _OSI(Module Device) machine # [ 0.489982] ACPI: Added _OSI(Processor Device) machine # [ 0.491979] ACPI: Added _OSI(Processor Aggregator Device) machine # [ 0.500684] ACPI: 1 ACPI AML tables successfully acquired and loaded machine # [ 0.507976] ACPI: Interpreter enabled machine # [ 0.510060] ACPI: PM: (supports S0 S3 S4 S5) machine # [ 0.511980] ACPI: Using IOAPIC for interrupt routing machine # [ 0.514331] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug machine # [ 0.517980] PCI: Using E820 reservations for host bridge windows machine # [ 0.520362] ACPI: Enabled 2 GPEs in block 00 to 3F machine # [ 0.533663] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) machine # [ 0.535985] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] machine # [ 0.539139] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR] machine # [ 0.541220] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability] machine # [ 0.544723] PCI host bridge to bus 0000:00 machine # [ 0.545984] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] machine # [ 0.547977] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] machine # [ 0.549977] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] machine # [ 0.552976] pci_bus 0000:00: root bus resource [mem 0x80000000-0xafffffff window] machine # [ 0.554977] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] machine # [ 0.556973] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe07ffffffff window] machine # [ 0.557973] pci_bus 0000:00: root bus resource [bus 00-ff] machine # [ 0.559138] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint machine # [ 0.561447] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint machine # [ 0.573425] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] machine # [ 0.575061] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] machine # [ 0.577048] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref] machine # [ 0.580779] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] machine # [ 0.586275] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint machine # [ 0.598018] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f] machine # [ 0.599994] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] machine # [ 0.603033] pci 0000:00:02.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref] machine # [ 0.605994] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] machine # [ 0.609325] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint machine # [ 0.620017] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f] machine # [ 0.621995] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] machine # [ 0.625033] pci 0000:00:03.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref] machine # [ 0.629090] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint machine # [ 0.637015] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f] machine # [ 0.638989] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] machine # [ 0.641020] pci 0000:00:04.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref] machine # [ 0.644132] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint machine # [ 0.652008] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f] machine # [ 0.653998] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] machine # [ 0.656036] pci 0000:00:05.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref] machine # [ 0.659994] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref] machine # [ 0.663402] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint machine # [ 0.670031] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] machine # [ 0.673034] pci 0000:00:06.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref] machine # [ 0.677376] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint machine # [ 0.686036] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] machine # [ 0.688034] pci 0000:00:07.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref] machine # [ 0.692125] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint machine # [ 0.698029] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] machine # [ 0.716791] pci 0000:00:08.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref] machine # [ 0.718722] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint machine # [ 0.722016] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] machine # [ 0.723021] pci 0000:00:09.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref] machine # [ 0.725545] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint machine # [ 0.731023] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf] machine # [ 0.731980] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] machine # [ 0.732995] pci 0000:00:0a.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref] machine # [ 0.735553] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint machine # [ 0.739013] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f] machine # [ 0.739940] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff] machine # [ 0.740826] pci 0000:00:0b.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref] machine # [ 0.743629] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint machine # [ 0.745775] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f] machine # [ 0.746899] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint machine # [ 0.748633] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf] machine # [ 0.750175] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint machine # [ 0.752715] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df] machine # [ 0.754270] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint machine # [ 0.756640] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff] machine # [ 0.758192] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint machine # [ 0.759270] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO machine # [ 0.761265] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint machine # [ 0.764570] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff] machine # [ 0.765007] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff] machine # [ 0.766295] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint machine # [ 0.768671] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f] machine # [ 0.773093] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 machine # [ 0.774107] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 machine # [ 0.776084] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 machine # [ 0.777097] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 machine # [ 0.778082] ACPI: PCI: Interrupt link LNKE configured for IRQ 10 machine # [ 0.779087] ACPI: PCI: Interrupt link LNKF configured for IRQ 10 machine # [ 0.780072] ACPI: PCI: Interrupt link LNKG configured for IRQ 11 machine # [ 0.782357] ACPI: PCI: Interrupt link LNKH configured for IRQ 11 machine # [ 0.783016] ACPI: PCI: Interrupt link GSIA configured for IRQ 16 machine # [ 0.783990] ACPI: PCI: Interrupt link GSIB configured for IRQ 17 machine # [ 0.784987] ACPI: PCI: Interrupt link GSIC configured for IRQ 18 machine # [ 0.785993] ACPI: PCI: Interrupt link GSID configured for IRQ 19 machine # [ 0.786987] ACPI: PCI: Interrupt link GSIE configured for IRQ 20 machine # [ 0.788991] ACPI: PCI: Interrupt link GSIF configured for IRQ 21 machine # [ 0.789985] ACPI: PCI: Interrupt link GSIG configured for IRQ 22 machine # [ 0.790987] ACPI: PCI: Interrupt link GSIH configured for IRQ 23 machine # [ 0.794087] iommu: Default domain type: Translated machine # [ 0.794617] iommu: DMA domain TLB invalidation policy: lazy mode machine # [ 0.796302] ACPI: bus type USB registered machine # [ 0.797034] usbcore: registered new interface driver usbfs machine # [ 0.798002] usbcore: registered new interface driver hub machine # [ 0.798986] usbcore: registered new device driver usb machine # [ 0.801307] NetLabel: Initializing machine # [ 0.801952] NetLabel: domain hash size = 128 machine # [ 0.802619] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO machine # [ 0.804085] NetLabel: unlabeled traffic allowed by default machine # [ 0.804986] PCI: Using ACPI for IRQ routing machine # [ 0.849545] pci 0000:00:01.0: vgaarb: setting as boot VGA device machine # [ 0.849969] pci 0000:00:01.0: vgaarb: bridge control possible machine # [ 0.849969] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none machine # [ 0.852979] vgaarb: loaded machine # [ 0.853776] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 machine # [ 0.854702] hpet0: 3 comparators, 64-bit 100.000000 MHz counter machine # [ 0.859078] clocksource: Switched to clocksource kvm-clock machine # [ 0.864607] VFS: Disk quotas dquot_6.6.0 machine # [ 0.865392] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) machine # [ 0.867096] pnp: PnP ACPI init machine # [ 0.867932] ACPI: IRQ 4 override to edge(!), high(!) machine # [ 0.868959] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved machine # [ 0.870648] pnp: PnP ACPI: found 5 devices machine # [ 0.880957] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns machine # [ 0.882618] clocksource: Switched to clocksource acpi_pm machine # [ 0.883743] NET: Registered PF_INET protocol family machine # [ 0.885404] IP idents hash table entries: 65536 (order: 7, 524288 bytes, linear) machine # [ 0.902255] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear) machine # [ 0.903835] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) machine # [ 0.905282] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear) machine # [ 0.908067] TCP bind hash table entries: 32768 (order: 8, 1048576 bytes, linear) machine # [ 0.909468] TCP: Hash tables configured (established 32768 bind 32768) machine # [ 0.910698] MPTCP token hash table entries: 4096 (order: 5, 98304 bytes, linear) machine # [ 0.912043] UDP hash table entries: 2048 (order: 5, 131072 bytes, linear) machine # [ 0.913269] UDP-Lite hash table entries: 2048 (order: 5, 131072 bytes, linear) machine # [ 0.914686] NET: Registered PF_UNIX/PF_LOCAL protocol family machine # [ 0.915788] NET: Registered PF_XDP protocol family machine # [ 0.916708] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] machine # [ 0.917792] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] machine # [ 0.918900] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] machine # [ 0.920090] pci_bus 0000:00: resource 7 [mem 0x80000000-0xafffffff window] machine # [ 0.921308] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window] machine # [ 0.922494] pci_bus 0000:00: resource 9 [mem 0xe0000000000-0xe07ffffffff window] machine # [ 0.924436] ACPI: \_SB_.GSIA: Enabled at IRQ 16 machine # [ 0.926641] ACPI: \_SB_.GSIB: Enabled at IRQ 17 machine # [ 0.928792] ACPI: \_SB_.GSIC: Enabled at IRQ 18 machine # [ 0.930930] ACPI: \_SB_.GSID: Enabled at IRQ 19 machine # [ 0.932762] PCI: CLS 0 bytes, default 64 machine # [ 0.933682] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) machine # [ 0.934146] Trying to unpack rootfs image as initramfs... machine # [ 0.934811] software IO TLB: mapped [mem 0x000000007a371000-0x000000007e371000] (64MB) machine # [ 0.934945] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns machine # [ 0.958668] Initialise system trusted keyrings machine # [ 0.959979] workingset: timestamp_bits=40 max_order=20 bucket_order=0 machine # [ 0.971721] Key type asymmetric registered machine # [ 0.972530] Asymmetric key parser 'x509' registered machine # [ 0.973723] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) machine # [ 0.975591] io scheduler mq-deadline registered machine # [ 0.976453] io scheduler kyber registered machine # [ 0.981010] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled machine # [ 0.982400] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A machine # [ 0.984850] Linux agpgart interface v0.103 machine # [ 0.985693] ACPI: bus type drm_connector registered machine # [ 0.989146] usbcore: registered new interface driver usbserial_generic machine # [ 0.990353] usbserial: USB Serial support registered for generic machine # [ 0.991419] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. machine # [ 0.991419] Please enable it if your BIOS has the CPPC option. machine # [ 0.993872] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled machine # [ 0.995367] drop_monitor: Initializing network drop monitor service machine # [ 0.996631] NET: Registered PF_INET6 protocol family machine # [ 0.998402] Segment Routing with IPv6 machine # [ 0.999080] In-situ OAM (IOAM) with IPv6 machine # [ 1.001819] IPI shorthand broadcast: enabled machine # [ 1.006355] sched_clock: Marking stable (851012570, 155191874)->(1027347523, -21143079) machine # [ 1.008307] registered taskstats version 1 machine # [ 1.009741] Loading compiled-in X.509 certificates machine # [ 1.020108] Demotion targets for Node 0: null machine # [ 1.021513] Key type .fscrypt registered machine # [ 1.022275] Key type fscrypt-provisioning registered machine # [ 1.023337] ima: No TPM chip found, activating TPM-bypass! machine # [ 1.024354] ima: Allocated hash algorithm: sha1 machine # [ 1.025177] ima: No architecture policies found machine # [ 1.026265] PM: Magic number: 10:852:717 machine # [ 1.027907] RAS: Correctable Errors collector initialized. machine # [ 1.032494] clk: Disabling unused clocks machine # [ 1.033265] PM: genpd: Disabling unused power domains machine # [ 1.224106] Freeing initrd memory: 29052K machine # [ 1.231604] Freeing unused decrypted memory: 2028K machine # [ 1.237992] Freeing unused kernel image (initmem) memory: 3652K machine # [ 1.240765] Write protecting the kernel read-only data: 32768k machine # [ 1.245732] Freeing unused kernel image (text/rodata gap) memory: 1184K machine # [ 1.249727] Freeing unused kernel image (rodata/data gap) memory: 720K machine # [ 1.317061] x86/mm: Checked W+X mappings: passed, no W+X pages found. machine # [ 1.318262] Run /init as init process machine # [ 1.333254] systemd[1]: Inserted module 'autofs4' machine # [ 1.359778] fuse: init (API version 7.45) machine # [ 1.368914] ACPI: \_SB_.GSIG: Enabled at IRQ 22 machine # [ 1.371409] ACPI: \_SB_.GSIH: Enabled at IRQ 23 machine # [ 1.374895] ACPI: \_SB_.GSIE: Enabled at IRQ 20 machine # [ 1.377247] ACPI: \_SB_.GSIF: Enabled at IRQ 21 machine # [ 1.382012] virtiofs virtio5: discovered new tag: nix-store machine # [ 1.383625] virtiofs virtio5: virtio_fs_setup_dax: No cache capability machine # [ 1.390797] virtiofs virtio6: discovered new tag: shared machine # [ 1.403734] virtiofs virtio6: virtio_fs_setup_dax: No cache capability machine # [ 1.407273] virtiofs virtio7: discovered new tag: xchg machine # [ 1.408772] virtiofs virtio7: virtio_fs_setup_dax: No cache capability machine # [ 1.437925] systemd[1]: Successfully made /usr/ read-only. machine # [ 1.777092] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) machine # [ 1.789738] systemd[1]: Detected virtualization kvm. machine # [ 1.792010] systemd[1]: Detected architecture x86-64. machine # [ 1.794278] systemd[1]: Running in initrd. machine # [ 1.796895] systemd[1]: Initializing machine ID from random generator. machine # [ 1.799992] systemd[1]: Hostname set to . machine # [ 1.920806] systemd[1]: bpf-restrict-fs: LSM BPF program attached machine # [ 1.982506] systemd[1]: Queued start job for default target Initrd Default Target. machine # [ 1.994838] systemd[1]: Created slice Slice /system/modprobe. machine # [ 1.996058] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. machine # [ 1.997578] systemd[1]: Expecting device /dev/disk/by-label/nixos... machine # [ 1.998730] systemd[1]: Reached target Path Units. machine # [ 1.999666] systemd[1]: Reached target Slice Units. machine # [ 2.000586] systemd[1]: Reached target Swaps. machine # [ 2.001435] systemd[1]: Reached target Timer Units. machine # [ 2.002496] systemd[1]: Listening on D-Bus System Message Bus Socket. machine # [ 2.003745] systemd[1]: Listening on Journal Socket (/dev/log). machine # [ 2.004962] systemd[1]: Listening on Journal Sockets. machine # [ 2.005967] systemd[1]: Listening on udev Control Socket. machine # [ 2.006991] systemd[1]: Listening on udev Kernel Socket. machine # [ 2.007989] systemd[1]: Reached target Socket Units. machine # [ 2.009960] systemd[1]: Starting Create List of Static Device Nodes... machine # [ 2.012380] systemd[1]: Starting Load Kernel Module configfs... machine # [ 2.015503] systemd[1]: Starting Journal Service... machine # [ 2.017420] systemd[1]: Starting Load Kernel Modules... machine # [ 2.018441] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os machine # [ 2.024455] systemd[1]: Starting Coldplug All udev Devices... machine # [ 2.029392] systemd[1]: Finished Create List of Static Device Nodes. machine # [ 2.031950] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... machine # [ 2.036599] systemd[1]: modprobe@configfs.service: Deactivated successfully. machine # [ 2.041242] systemd[1]: Finished Load Kernel Module configfs. machine # [ 2.042843] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config machine # [ 2.049806] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. machine # [ 2.052439] systemd[1]: Starting Create Static Device Nodes in /dev... machine # [ 2.055238] systemd-journald[93]: Collecting audit messages is disabled. machine # [ 2.070460] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. machine # [ 2.073033] systemd[1]: Finished Create Static Device Nodes in /dev. machine # [ 2.074350] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev machine # [ 2.074452] systemd[1]: Reached target Preparation for Local File Systems. machine # [ 2.077151] systemd[1]: Reached target Local File Systems. machine # [ 2.079252] systemd[1]: Starting Rule-based Manager for Device Events and Files... machine # [ 2.081128] systemd[1]: Finished Load Kernel Modules. machine # [ 2.083289] systemd[1]: Starting Apply Kernel Variables... machine # [ 2.096937] systemd[1]: Started Journal Service. machine # [ 1.941837] systemd-modules-load[94]: Using 4 probe threads machine # [ 1.943962] systemd-modules-load[94]: Inserted module 'virtio_balloon' machine # [ 1.945777] systemd-modules-load[94]: Inserted module 'virtio_gpu' machine # [ 1.946830] systemd-modules-load[94]: Inserted module 'dm_mod' machine # [ 1.947823] systemd[1]: Starting Create System Files and Directories... machine # [ 1.950389] systemd[1]: Finished Apply Kernel Variables. machine # [ 1.955583] systemd-udevd[104]: Using default interface naming scheme 'v261'. machine # [ 1.956798] systemd[1]: Finished Create System Files and Directories. machine # [ 1.966633] systemd[1]: Started Rule-based Manager for Device Events and Files. machine # [ 2.018986] systemd[1]: Finished Coldplug All udev Devices. machine # [ 2.020284] systemd[1]: Reached target System Initialization. machine # [ 2.021223] systemd[1]: Reached target Basic System. machine # [ 2.253067] virtio_blk virtio2: 4/0/0 default/read/poll queues machine # [ 2.269178] ehci-pci 0000:00:1d.7: EHCI Host Controller machine # [ 2.271495] ehci-pci 0000:00:1d.7: new USB bus registered, assigned bus number 1 machine # [ 2.273554] ehci-pci 0000:00:1d.7: irq 19, io mem 0xfebdb000 machine # [ 2.280233] ehci-pci 0000:00:1d.7: USB 2.0 started, EHCI 1.00 machine # [ 2.281807] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.18 machine # [ 2.283022] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 machine # [ 2.284149] usb usb1: Product: EHCI Host Controller machine # [ 2.285091] usb usb1: Manufacturer: Linux 6.18.52 ehci_hcd machine # [ 2.286905] usb usb1: SerialNumber: 0000:00:1d.7 machine # [ 2.291164] virtio_blk virtio2: [vda] 33554432 512-byte logical blocks (17.2 GB/16.0 GiB) machine # [ 2.294075] hub 1-0:1.0: USB hub found machine # [ 2.295385] hub 1-0:1.0: 6 ports detected machine # [ 2.300069] SCSI subsystem initialized machine # [ 2.306598] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 machine # [ 2.307842] uhci_hcd 0000:00:1d.0: UHCI Host Controller machine # [ 2.308691] uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 2 machine # [ 2.310167] uhci_hcd 0000:00:1d.0: detected 2 ports machine # [ 2.311033] uhci_hcd 0000:00:1d.0: irq 16, io port 0x0000c180 machine # [ 2.312414] usb usb2: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 machine # [ 2.313891] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1 machine # [ 2.315311] usb usb2: Product: UHCI Host Controller machine # [ 2.316282] usb usb2: Manufacturer: Linux 6.18.52 uhci_hcd machine # [ 2.316520] serio: i8042 KBD port at 0x60,0x64 irq 1 machine # [ 2.316543] usb usb2: SerialNumber: 0000:00:1d.0 machine # [ 2.316762] serio: i8042 AUX port at 0x60,0x64 irq 12 machine # [ 2.322755] hub 2-0:1.0: USB hub found machine # [ 2.323673] hub 2-0:1.0: 2 ports detected machine # [ 2.356426] uhci_hcd 0000:00:1d.1: UHCI Host Controller machine # [ 2.358375] uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 3 machine # [ 2.359554] uhci_hcd 0000:00:1d.1: detected 2 ports machine # [ 2.360496] uhci_hcd 0000:00:1d.1: irq 17, io port 0x0000c1a0 machine # [ 2.207773] (udev-worker)[137]: Network interface NamePolicy= disabled on kernel command line. machine # [ 2.209514] systemd[1]: Starting Virtual Console Setup... machine # [ 2.366458] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 machine # [ 2.213086] (udev-worker)[136]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. machine # [ 2.215818] (udev-worker)[136]: Network interface NamePolicy= disabled on kernel command line. machine # [ 2.372426] ahci 0000:00:1f.2: AHCI vers 0001.0000, 32 command slots, 1.5 Gbps, SATA mode machine # [ 2.373656] usb usb3: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 machine # [ 2.373798] ahci 0000:00:1f.2: 6/6 ports implemented (port mask 0x3f) machine # [ 2.374125] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1 machine # [ 2.374475] ahci 0000:00:1f.2: flags: 64bit ncq only machine # [ 2.376063] usb usb3: Product: UHCI Host Controller machine # [ 2.380577] usb usb3: Manufacturer: Linux 6.18.52 uhci_hcd machine # [ 2.381497] usb usb3: SerialNumber: 0000:00:1d.1 machine # [ 2.382395] hub 3-0:1.0: USB hub found machine # [ 2.383086] hub 3-0:1.0: 2 ports detected machine # [ 2.385294] scsi host0: ahci machine # [ 2.386860] uhci_hcd 0000:00:1d.2: UHCI Host Controller machine # [ 2.387814] uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 4 machine # [ 2.387833] uhci_hcd 0000:00:1d.2: detected 2 ports machine # [ 2.387900] scsi host1: ahci machine # [ 2.388453] scsi host2: ahci machine # [ 2.234197] systemd[1]: Found device /dev/disk/by-label/nixos. machine # [ 2.236623] systemd[1]: Reached target Initrd Root Device. machine # [ 2.237673] systemd[1]: Starting File System [ 2.392792] uhci_hcd 0000:00:1d.2: irq 18, io port 0x0000c1c0 machine # Check on /dev/disk/by-label/nixos... machine # [ 2.394637] scsi host3: ahci machine # [ 2.395643] scsi host4: ahci machine # [ 2.396288] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 machine # [ 2.396771] scsi host5: ahci machine # [ 2.397601] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1 machine # [ 2.397603] usb usb4: Product: UHCI Host Controller machine # [ 2.397605] usb usb4: Manufacturer: Linux 6.18.52 uhci_hcd machine # [ 2.398126] ata1: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc100 irq 50 lpm-pol 1 machine # [ 2.398470] usb usb4: SerialNumber: 0000:00:1d.2 machine # [ 2.398997] hub 4-0:1.0: USB hub found machine # [ 2.400039] ata2: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc180 irq 50 lpm-pol 1 machine # [ 2.400311] hub 4-0:1.0: 2 ports detected machine # [ 2.400662] ata3: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc200 irq 50 lpm-pol 1 machine # [ 2.407396] ata4: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc280 irq 50 lpm-pol 1 machine # [ 2.407404] ata5: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc300 irq 50 lpm-pol 1 machine # [ 2.407411] ata6: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc380 irq 50 lpm-pol 1 machine # [ 2.246133] systemd-vconsole-setup[144]: Configuration of first virtual console was skipped, ignoring remaining ones. machine # [ 2.261147] systemd[1]: Finished Virtual Console Setup. machine # [ 2.269080] systemd-fsck[175]: nixos: clean, 12/1048576 files, 109865/4194304 blocks machine # [ 2.272643] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. machine # [ 2.540368] usb 1-1: new high-speed USB device number 2 using ehci-pci machine # [ 2.674010] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 machine # [ 2.677052] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 machine # [ 2.679867] usb 1-1: Product: QEMU USB Tablet machine # [ 2.681590] usb 1-1: Manufacturer: QEMU machine # [ 2.683141] usb 1-1: SerialNumber: 28754-0000:00:1d.7-1 machine # [ 2.717319] hid: raw HID events driver (C) Jiri Kosina machine # [ 2.724246] ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 300) machine # [ 2.726780] ata3.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 machine # [ 2.729132] ata3.00: applying bridge limits machine # [ 2.731287] ata1: SATA link down (SStatus 0 SControl 300) machine # [ 2.733514] ata3.00: configured for UDMA/100 machine # [ 2.735600] ata5: SATA link down (SStatus 0 SControl 300) machine # [ 2.738035] ata6: SATA link down (SStatus 0 SControl 300) machine # [ 2.740507] ata4: SATA link down (SStatus 0 SControl 300) machine # [ 2.742865] ata2: SATA link down (SStatus 0 SControl 300) machine # [ 2.745574] scsi 2:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 machine # [ 2.778017] usbcore: registered new interface driver usbhid machine # [ 2.780315] usbhid: USB HID core driver machine # [ 2.791048] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:1d.7/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 machine # [ 2.795415] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:1d.7-1/input0 machine # [ 2.817799] sr 2:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray machine # [ 2.853720] cdrom: Uniform CD-ROM driver Revision: 3.20 machine # [ 2.874375] systemd[1]: Mounting /sysroot... machine # [ 3.176429] EXT4-fs (vda): mounted filesystem cfcf699b-8797-4084-9fb0-6e45c5d7acd4 r/w with ordered data mode. Quota mode: none. machine # [ 3.027711] systemd[1]: Mounted /sysroot. machine # [ 3.029473] systemd[1]: Reached target Initrd Root File System. machine # [ 3.031558] systemd[1]: Mounting /sysroot/nix/.ro-store... machine # [ 3.033462] systemd[1]: Mounting /sysroot/nix/.rw-store... machine # [ 3.035472] systemd[1]: Mounting /sysroot/run... machine # [ 3.038364] systemd[1]: Mounting /sysroot/tmp/shared... machine # [ 3.047334] systemd[1]: Mounting /sysroot/tmp/xchg... machine # [ 3.053339] systemd[1]: Starting Mountpoints Configured in the Real Root... machine # [ 3.061552] systemd[1]: Mounted /sysroot/nix/.ro-store. machine # [ 3.063438] systemd[1]: Mounted /sysroot/nix/.rw-store. machine # [ 3.064725] systemd[1]: Mounted /sysroot/tmp/shared. machine # [ 3.066163] systemd-sysroot-fstab-check[195]: /sysroot should be mounted in the initrd, will request daemon-reload. machine # [ 3.068601] systemd[1]: Mounted /sysroot/tmp/xchg. machine # [ 3.069735] systemd[1]: Mounted /sysroot/run. machine # [ 3.070746] systemd[1]: Starting rw-sysroot-nix-store.service... machine # [ 3.072257] systemd[1]: Reload requested from client PID 195 ('systemd-sysroot') (unit initrd-parse-etc.service)... machine # [ 3.074905] systemd[1]: Reloading... machine # [ 3.124468] systemd[1]: Reloading finished in 53 ms. machine # [ 3.147442] systemd-sysroot-fstab-check[195]: Requesting initrd-fs.target/start/replace... machine # [ 3.150587] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. machine # [ 3.153304] systemd[1]: Finished rw-sysroot-nix-store.service. machine # [ 3.155551] systemd-sysroot-fstab-check[195]: Requesting swap.target/start/replace... machine # [ 3.158351] systemd[1]: Starting rw-sysroot-nix-store.service... machine # [ 3.160475] systemd[1]: initrd-parse-etc.service: Deactivated successfully. machine # [ 3.162867] systemd[1]: Finished Mountpoints Configured in the Real Root. machine # [ 3.165306] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. machine # [ 3.176299] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. machine # [ 3.178810] systemd[1]: Finished rw-sysroot-nix-store.service. machine # [ 3.874506] systemd[1]: Mounting /sysroot/nix/store... machine # [ 3.927046] systemd[1]: Mounted /sysroot/nix/store. machine # [ 3.928816] systemd[1]: Reached target Initrd File Systems. machine # [ 3.930945] systemd[1]: Starting Find NixOS closure... machine # [ 3.932847] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... machine # [ 3.957886] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. machine # [ 3.960307] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. machine # [ 3.980609] systemd[1]: Finished Find NixOS closure. machine # [ 3.981895] systemd[1]: Reached target Initrd Default Target. machine # [ 3.983386] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... machine # [ 4.012873] systemd[1]: Stopped target Initrd Default Target. machine # [ 4.014553] systemd[1]: Stopped target Basic System. machine # [ 4.015878] systemd[1]: Stopped target Initrd Root Device. machine # [ 4.017395] systemd[1]: Stopped target Path Units. machine # [ 4.018689] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. machine # [ 4.020793] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. machine # [ 4.022927] systemd[1]: Stopped target Slice Units. machine # [ 4.024276] systemd[1]: Stopped target Socket Units. machine # [ 4.025606] systemd[1]: Stopped target System Initialization. machine # [ 4.027445] systemd[1]: Stopped target Swaps. machine # [ 4.029320] systemd[1]: Stopped target Timer Units. machine # [ 4.030611] systemd[1]: dbus.socket: Deactivated successfully. machine # [ 4.032233] systemd[1]: Closed D-Bus System Message Bus Socket. machine # [ 4.034469] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. machine # [ 4.036596] systemd[1]: Stopped Find NixOS closure. machine # [ 4.037945] systemd[1]: Starting rw-sysroot-nix-store.service... machine # [ 4.039540] systemd[1]: systemd-sysctl.service: Deactivated successfully. machine # [ 4.041604] systemd[1]: Stopped Apply Kernel Variables. machine # [ 4.042982] systemd[1]: systemd-modules-load.service: Deactivated successfully. machine # [ 4.044737] systemd[1]: Stopped Load Kernel Modules. machine # [ 4.045857] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. machine # [ 4.047721] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. machine # [ 4.049987] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. machine # [ 4.051702] systemd[1]: Stopped Create System Files and Directories. machine # [ 4.053156] systemd[1]: Stopped target Local File Systems. machine # [ 4.054381] systemd[1]: Stopped target Preparation for Local File Systems. machine # [ 4.055965] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. machine # [ 4.057632] systemd[1]: Stopped Coldplug All udev Devices. machine # [ 4.058831] systemd[1]: Stopping Rule-based Manager for Device Events and Files... machine # [ 4.060561] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. machine # [ 4.062244] systemd[1]: Stopped Virtual Console Setup. machine # [ 4.063181] systemd[1]: initrd-cleanup.service: Deactivated successfully. machine # [ 4.064210] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. machine # [ 4.066580] systemd[1]: systemd-udevd.service: Deactivated successfully. machine # [ 4.067591] systemd[1]: Stopped Rule-based Manager for Device Events and Files. machine # [ 4.068689] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. machine # [ 4.069733] systemd[1]: Closed udev Control Socket. machine # [ 4.070683] systemd[1]: Starting Cleanup udev Database... machine # [ 4.071959] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. machine # [ 4.073605] systemd[1]: Stopped Create Static Device Nodes in /dev. machine # [ 4.074763] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. machine # [ 4.076251] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. machine # [ 4.077526] systemd[1]: kmod-static-nodes.service: Deactivated successfully. machine # [ 4.078739] systemd[1]: Stopped Create List of Static Device Nodes. machine # [ 4.087068] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. machine # [ 4.088356] systemd[1]: Finished rw-sysroot-nix-store.service. machine # [ 4.102960] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. machine # [ 4.104786] systemd[1]: Finished Cleanup udev Database. machine # [ 4.105596] systemd[1]: Reached target Switch Root. machine # [ 4.106386] systemd[1]: Starting NixOS Activation... machine # [ 4.256858] initrd-nixos-activation-start[240]: booting system configuration /nix/store/q7hgjsikzjv4i374iaff8zvjhi589aad-nixos-system-machine-test machine # [ 4.305897] initrd-nixos-activation-start[240]: running activation script... machine # [ 4.601816] initrd-nixos-activation-start[263]: setting up /etc... machine # [ 4.758912] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. machine # [ 4.760154] systemd[1]: Finished NixOS Activation. machine # [ 4.762840] systemd[1]: Starting Switch Root... machine # [ 4.794736] systemd[1]: Switching root. machine # [ 5.005778] systemd-journald[93]: Received SIGTERM from PID 1 (systemd). machine # [ 5.492866] NET: Registered PF_VSOCK protocol family machine # [ 5.873808] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) machine # [ 5.884073] systemd[1]: Detected virtualization kvm. machine # [ 5.886023] systemd[1]: Detected architecture x86-64. machine # [ 5.888053] systemd[1]: Detected first boot. machine # [ 5.892483] systemd[1]: Initializing machine ID from random generator. machine # [ 6.030014] systemd[1]: bpf-restrict-fs: LSM BPF program attached machine # [ 6.114085] systemd[1]: Applying preset policy. machine # [ 6.326517] systemd[1]: Populated /etc with preset unit settings. machine # [ 6.561995] systemd[1]: initrd-switch-root.service: Deactivated successfully. machine # [ 6.563601] systemd[1]: Stopped initrd-switch-root.service. machine # [ 6.565999] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. machine # [ 6.568242] systemd[1]: Created slice Slice /system/getty. machine # [ 6.569651] systemd[1]: Created slice User and Session Slice. machine # [ 6.570715] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. machine # [ 6.572072] systemd[1]: Started Forward Password Requests to Wall Directory Watch. machine # [ 6.573345] systemd[1]: Expecting device /dev/hvc0... machine # [ 6.574087] systemd[1]: Expecting device /dev/ttyS0... machine # [ 6.575025] systemd[1]: Reached target Local Encrypted Volumes. machine # [ 6.575870] systemd[1]: Stopped target initrd-fs.target. machine # [ 6.576746] systemd[1]: Stopped target initrd-root-fs.target. machine # [ 6.577670] systemd[1]: Stopped target initrd-switch-root.target. machine # [ 6.578728] systemd[1]: Reached target Virtual Machines and Containers. machine # [ 6.579786] systemd[1]: Reached target Path Units. machine # [ 6.580595] systemd[1]: Reached target Remote File Systems. machine # [ 6.581462] systemd[1]: Reached target Slice Units. machine # [ 6.582257] systemd[1]: Reached target Swaps. machine # [ 6.584366] systemd[1]: Listening on Query the User Interactively for a Password. machine # [ 6.587134] systemd[1]: Listening on Process Core Dump Socket. machine # [ 6.589204] systemd[1]: Listening on Credential Encryption/Decryption. machine # [ 6.591297] systemd[1]: Listening on Factory Reset Management. machine # [ 6.592320] systemd[1]: Listening on Hostname Service Socket. machine # [ 6.595645] systemd[1]: Starting Journal Log Access Socket... machine # [ 6.597036] systemd[1]: Listening on Journal Audit Socket. machine # [ 6.599526] systemd[1]: Listening on Console Output Muting Service Socket. machine # [ 6.600819] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. machine # [ 6.602109] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os machine # [ 6.603558] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki machine # [ 6.608467] systemd[1]: Listening on Disk Repartitioning Service Socket. machine # [ 6.609615] systemd[1]: Listening on udev Control Socket. machine # [ 6.610569] systemd[1]: Listening on udev Varlink Socket. machine # [ 6.613465] systemd[1]: Mounting Huge Pages File System... machine # [ 6.615668] systemd[1]: Mounting POSIX Message Queue File System... machine # [ 6.618334] systemd[1]: Mounting Kernel Debug File System... machine # [ 6.623323] systemd[1]: Mounting Kernel Trace File System... machine # [ 6.628081] systemd[1]: Starting Create List of Static Device Nodes... machine # [ 6.636742] systemd[1]: Starting Load Kernel Module configfs... machine # [ 6.638891] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm machine # [ 6.640934] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore machine # [ 6.642674] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse machine # [ 6.645503] systemd[1]: Mounting FUSE Control File System... machine # [ 6.646530] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 machine # [ 6.654843] systemd[1]: Starting Journal Service... machine # [ 6.658211] systemd[1]: Starting Load Kernel Modules... machine # [ 6.662261] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... machine # [ 6.665616] systemd[1]: Starting Remount Root and Kernel File Systems... machine # [ 6.666728] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os machine # [ 6.673118] systemd[1]: Starting Coldplug All udev Devices... machine # [ 6.675987] systemd[1]: Listening on Journal Log Access Socket. machine # [ 6.678241] systemd[1]: Mounted Huge Pages File System. machine # [ 6.679290] systemd[1]: Mounted POSIX Message Queue File System. machine # [ 6.680433] systemd[1]: Mounted Kernel Debug File System. machine # [ 6.681521] systemd[1]: Mounted Kernel Trace File System. machine # [ 6.682984] systemd[1]: Finished Create List of Static Device Nodes. machine # [ 6.684095] systemd[1]: Mounted FUSE Control File System. machine # [ 6.690981] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... machine # [ 6.695390] EXT4-fs (vda): re-mounted cfcf699b-8797-4084-9fb0-6e45c5d7acd4. machine # [ 6.699119] systemd[1]: modprobe@configfs.service: Deactivated successfully. machine # [ 6.700630] systemd[1]: Finished Load Kernel Module configfs. machine # [ 6.702298] systemd[1]: Finished Remount Root and Kernel File Systems. machine # [ 6.703696] systemd[1]: Listening on Disk Image Download Service Socket. machine # [ 6.704837] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore machine # [ 6.705844] systemd-journald[333]: Collecting audit messages is enabled. machine # [ 6.711429] systemd[1]: Starting Load/Save OS Random Seed... machine # [ 6.712368] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os machine # [ 6.572612] systemd[1]: Queued start job for default target Multi-User System. machine # [ 6.727733] systemd[1]: Started Journal Service. machine # [ 6.574068] systemd[1]: systemd-journald.service: Deactivated successfully. machine # [ 6.575850] systemd-modules-load[334]: Using 2 probe threads machine # [ 6.577210] systemd-oomd[335]: No swap; memory pressure usage will be degraded machine # [ 6.733614] loop: module loaded machine # [ 6.579459] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. machine # [ 6.580739] systemd-modules-load[334]: Inserted module 'loop' machine # [ 6.585114] systemd[1]: Finished Load Kernel Modules. machine # [ 6.591927] systemd[1]: Starting Flush Journal to Persistent Storage... machine # [ 6.593488] systemd[1]: Starting Apply Kernel Variables... machine # [ 6.605410] systemd[1]: Finished Load/Save OS Random Seed. machine # [ 6.606292] systemd[1]: Reached target First Boot Complete. machine # [ 6.611052] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. machine # [ 6.612721] systemd[1]: Starting Create Static Device Nodes in /dev... machine # [ 6.780711] systemd-journald[333]: Received client request to flush runtime journal. machine # [ 6.669817] systemd[1]: Finished Apply Kernel Variables. machine # [ 6.670871] systemd[1]: Finished Create Static Device Nodes in /dev. machine # [ 6.671983] systemd[1]: Reached target Preparation for Local File Systems. machine # [ 6.672994] systemd[1]: Starting Rule-based Manager for Device Events and Files... machine # [ 6.674207] systemd[1]: Finished Flush Journal to Persistent Storage. machine # [ 6.692437] systemd-udevd[361]: Using default interface naming scheme 'v261'. machine # [ 6.699738] systemd[1]: Finished Coldplug All udev Devices. machine # [ 6.728188] systemd[1]: Started Rule-based Manager for Device Events and Files. machine # [ 6.782856] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs machine # [ 6.801958] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs machine # [ 6.808104] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse machine # [ 6.827575] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. machine # [ 6.832691] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. machine # [ 6.845778] (udev-worker)[368]: Network interface NamePolicy= disabled on kernel command line. machine # [ 6.847401] (udev-worker)[379]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. machine # [ 6.850119] (udev-worker)[379]: Network interface NamePolicy= disabled on kernel command line. machine # [ 6.875094] systemd[1]: Condition check resulted in Virtio network device being skipped. machine # [ 6.876847] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs machine # [ 6.878603] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore machine # [ 6.880127] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 machine # [ 6.882554] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore machine # [ 6.884245] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os machine # [ 6.885721] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os machine # [ 7.042741] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 machine # [ 7.054543] ACPI: button: Power Button [PWRF] machine # [ 7.059066] bochs-drm 0000:00:01.0: vgaarb: deactivate vga console machine # [ 7.085651] Console: switching to colour dummy device 80x25 machine # [ 7.086670] [drm] Found bochs VGA, ID 0xb0c5. machine # [ 7.086672] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. machine # [ 7.090536] lpc_ich 0000:00:1f.0: I/O space for GPIO uninitialized machine # [ 7.097572] rtc_cmos PNP0B00:00: RTC can wake from S4 machine # [ 7.099490] bochs-drm 0000:00:01.0: [drm] Registered 1 planes with drm panic machine # [ 7.100383] [drm] Initialized bochs-drm 1.0.0 for 0000:00:01.0 on minor 0 machine # [ 7.101824] parport_pc 00:02: reported by Plug and Play ACPI machine # [ 7.104062] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] machine # [ 7.106449] rtc_cmos PNP0B00:00: registered as rtc0 machine # [ 7.107098] rtc_cmos PNP0B00:00: setting system clock to 2026-09-23T04:42:33 UTC (1790138553) machine # [ 7.109821] mousedev: PS/2 mouse device common for all mice machine # [ 7.110878] systemd-journald[333]: Time jumped backwards, rotating. machine # [ 7.111302] rtc_cmos PNP0B00:00: alarms up to one day, y3k, 242 bytes nvram, hpet irqs machine # [ 7.119039] Console: switching to colour frame buffer device 160x50 machine # [ 7.123407] bochs-drm 0000:00:01.0: [drm] fb0: bochs-drmdrmfb frame buffer device machine # [ 7.138703] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:06.0/virtio4/input/input4 machine # [ 7.149862] i801_smbus 0000:00:1f.3: SMBus using PCI interrupt machine # [ 7.159626] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD machine # [ 7.187547] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6 machine # [ 7.052302] systemd[1]: Starting Virtual Console Setup... machine # [ 7.213721] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 machine # [ 7.217622] iTCO_wdt iTCO_wdt.0.auto: Found a ICH9 TCO device (Version=2, TCOBASE=0x0660) machine # [ 7.221454] iTCO_wdt iTCO_wdt.0.auto: initialized. heartbeat=30 sec (nowayout=0) machine # [ 7.075242] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. machine # [ 7.076873] systemd[1]: Stopped Virtual Console Setup. machine # [ 7.080266] systemd[1]: Starting Virtual Console Setup... machine # [ 7.090633] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. machine # [ 7.092532] systemd[1]: Stopped Virtual Console Setup. machine # [ 7.259417] ppdev: user-space parallel port driver machine # [ 7.105818] systemd[1]: Starting Virtual Console Setup... machine # [ 7.312079] kvm_amd: TSC scaling supported machine # [ 7.312662] kvm_amd: Nested Virtualization enabled machine # [ 7.313291] kvm_amd: Nested Paging enabled machine # [ 7.313719] kvm_amd: LBR virtualization supported machine # [ 7.314318] kvm_amd: Virtual VMLOAD VMSAVE supported machine # [ 7.314815] kvm_amd: Virtual GIF supported machine # [ 7.315419] kvm_amd: Virtual NMI enabled machine # [ 7.416423] EDAC MC: Ver: 3.0.0 machine # [ 7.293323] systemd-vconsole-setup[406]: Configuration of first virtual console was skipped, ignoring remaining ones. machine # [ 7.296549] systemd[1]: Finished Virtual Console Setup. machine # [ 7.409719] systemd[1]: Mounting /run/wrappers... machine # [ 7.412517] systemd[1]: Mounting Kernel Configuration File System... machine # [ 7.454203] systemd[1]: Mounted /run/wrappers. machine # [ 7.456177] systemd[1]: Mounted Kernel Configuration File System. machine # [ 7.458451] systemd[1]: Reached target Local File Systems. machine # [ 7.460519] systemd[1]: Listening on Boot Loader Control Service Socket. machine # [ 7.462924] systemd[1]: Starting register-nix-paths.service... machine # [ 7.465362] systemd[1]: Starting Create SUID/SGID Wrappers... machine # [ 7.467454] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. machine # [ 7.470488] systemd[1]: Starting Save Transient machine-id to Disk... machine # [ 7.473697] systemd[1]: Starting Create System Files and Directories... machine # [ 7.545067] systemd[1]: Finished Create System Files and Directories. machine # [ 7.547558] systemd[1]: Starting Rebuild Journal Catalog... machine # [ 7.549118] systemd[1]: Starting Record System Boot/Shutdown in UTMP... machine # [ 7.578526] systemd[1]: Finished Record System Boot/Shutdown in UTMP. machine # [ 7.593742] systemd[1]: Finished Rebuild Journal Catalog. machine # [ 7.595403] systemd[1]: Starting Update is Completed... machine # [ 7.619701] systemd[1]: Finished Update is Completed. machine # [ 7.698915] systemd[1]: Finished Save Transient machine-id to Disk. machine # [ 7.749177] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. machine # [ 7.750406] systemd[1]: Finished Create SUID/SGID Wrappers. machine # [ 7.960320] systemd[1]: Finished register-nix-paths.service. machine # [ 7.962599] systemd[1]: Reached target System Initialization. machine # [ 7.964795] systemd[1]: Started Discard unused filesystem blocks once a week. machine # [ 7.967433] systemd[1]: Started niks3 garbage collection timer. machine # [ 7.969666] systemd[1]: Started Daily Cleanup of Temporary Directories. machine # [ 7.972195] systemd[1]: Reached target Timer Units. machine # [ 7.974103] systemd[1]: Listening on D-Bus System Message Bus Socket. machine # [ 7.976585] systemd[1]: Listening on niks3 server socket. machine # [ 7.978668] systemd[1]: Listening on Nix Daemon Socket. machine # [ 7.980731] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. machine # [ 7.983773] systemd[1]: Reached target Socket Units. machine # [ 7.985898] systemd[1]: Reached target Basic System. machine # [ 7.987815] systemd[1]: Started backdoor.service. machine # [ 7.989754] systemd[1]: Starting Import lastlog data into lastlog2 database... machine # [ 7.992514] systemd[1]: Starting Name Service Cache Daemon (nsncd)... machine # [ 7.995360] systemd[1]: Starting Post-Boot Actions... machine # [ 7.998159] systemd[1]: Started Reset console on configuration changes. machine # [ 8.004396] systemd[1]: Starting resolvconf update... machine # [ 8.013316] systemd[1]: Started rustfs.service. machine # [ 8.019457] systemd[1]: Starting rustfs-bucket.service... machine # [ 8.023870] systemd[1]: Starting D-Bus System Message Bus... machine # [ 8.037516] systemd[1]: Finished Post-Boot Actions. machine # [ 8.044064] nsncd[491]: Sep 23 04:42:34.590 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" machine # connecting to host... machine # [ 8.047204] systemd[1]: Started Name Service Cache Daemon (nsncd). machine # [ 8.048760] systemd[1]: Reached target Host and Network Name Lookups. machine # [ 8.049517] systemd[1]: Reached target User and Group Name Lookups. machine # [ 8.054081] systemd[1]: Starting User Login Management... machine # [ 8.063199] systemd[1]: Finished Import lastlog data into lastlog2 database. machine: Guest shell says: b'Spawning backdoor root shell...\n' machine: connected to guest root shell machine: (connecting took 8.70 seconds) machine: (finished: waiting for the VM to finish booting, in 9.01 seconds) machine # [ 8.116101] dbus-broker-launch[500]: Looking up NSS user entry for 'systemd-timesync'... machine # [ 8.126268] dbus-broker-launch[500]: NSS returned no entry for 'systemd-timesync' machine # [ 8.127238] dbus-broker-launch[500]: Invalid user-name in /nix/store/d3mrkr0v9x5sgbb0rb2pnsl0mysb1m4x-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" machine # [ 8.128293] systemd-logind[520]: New seat seat0. machine # [ 8.134732] systemd-logind[520]: Watching system buttons on /dev/input/event2 (Power Button) machine # [ 8.136626] systemd-logind[520]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) machine # [ 8.137152] systemd-logind[520]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) machine # [ 8.137771] systemd[1]: Started User Login Management. machine # [ 8.140506] systemd[1]: Starting linger-users.service... machine # [ 8.145816] systemd[1]: Started D-Bus System Message Bus. machine # [ 8.160676] dbus-broker-launch[500]: Ready machine # [ 8.163429] systemd[1]: Stopped target Host and Network Name Lookups. machine # [ 8.165472] systemd[1]: Stopping Host and Network Name Lookups... machine # [ 8.166457] systemd[1]: Stopped target User and Group Name Lookups. machine # [ 8.167329] systemd[1]: Stopping User and Group Name Lookups... machine # [ 8.168216] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... machine # [ 8.171045] systemd[1]: nscd.service: Deactivated successfully. machine # [ 8.172448] systemd[1]: Stopped Name Service Cache Daemon (nsncd). machine # [ 8.179489] systemd[1]: Starting Name Service Cache Daemon (nsncd)... machine # [ 8.187762] systemd[1]: linger-users.service: Deactivated successfully. machine # [ 8.189569] systemd[1]: Finished linger-users.service. machine # [ 8.218392] nsncd[596]: Sep 23 04:42:34.765 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" machine # [ 8.219157] systemd[1]: Started Name Service Cache Daemon (nsncd). machine # [ 8.219725] systemd[1]: Reached target Host and Network Name Lookups. machine # [ 8.220276] systemd[1]: Reached target User and Group Name Lookups. machine # [ 8.235179] systemd[1]: Finished resolvconf update. machine # [ 8.235832] systemd[1]: Reached target Preparation for Network. machine # [ 8.236883] systemd[1]: Starting DHCP Client... machine # [ 8.238560] systemd[1]: Starting Address configuration of eth1... machine # [ 8.239938] systemd[1]: Starting Extra networking commands.... machine # [ 8.304131] network-addresses-eth1-start[626]: adding address 192.168.1.1/24... done machine # [ 8.313806] network-addresses-eth1-start[626]: adding address 2001:db8:1::1/64... done machine # [ 8.328767] dhcpcd[635]: dhcpcd-10.3.2 starting machine # [ 8.331962] systemd[1]: Finished Address configuration of eth1. machine # [ 8.335297] dhcpcd[682]: dev: loaded udev machine # [ 8.503478] 8021q: 802.1Q VLAN Support v1.8 machine # [ 8.503998] 8021q: adding VLAN 0 to HW filter on device eth1 machine # [ 8.360086] systemd[1]: Finished Extra networking commands.. machine # [ 8.361030] systemd[1]: Reached target Network. machine # [ 8.365418] systemd[1]: Starting PostgreSQL Server... machine # [ 8.367223] systemd[1]: Starting Permit User Sessions... machine # [ 8.370315] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. machine # [ 8.389191] systemd[1]: Finished Permit User Sessions. machine # [ 8.392432] systemd[1]: Started Getty on tty1. machine # [ 8.393943] systemd[1]: Reached target Login Prompts. machine # [ 8.407636] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. machine # [ 8.602077] cfg80211: Loading compiled-in X.509 certificates for regulatory database machine # [ 8.623770] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' machine # [ 8.624839] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' machine # [ 8.627759] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 machine # [ 8.629689] cfg80211: failed to load regulatory.db machine # [ 8.667118] 8021q: adding VLAN 0 to HW filter on device eth0 machine # [ 8.514032] dhcpcd[682]: eth0: waiting for carrier machine # [ 8.514898] dhcpcd[682]: eth0: carrier acquired machine # [ 8.521435] dhcpcd[682]: DUID 00:01:00:01:32:46:15:3b:52:54:00:12:34:56 machine # [ 8.522566] dhcpcd[682]: eth0: IAID 00:12:34:56 machine # [ 8.523357] dhcpcd[682]: eth0: adding address fe80::5054:ff:fe12:3456 machine # [ 8.529086] postgresql-pre-start[715]: The files belonging to this database system will be owned by user "postgres". machine # [ 8.529582] postgresql-pre-start[715]: This user must also own the server process. machine # [ 8.531346] postgresql-pre-start[715]: The database cluster will be initialized with locale "en_US.UTF-8". machine # [ 8.532171] postgresql-pre-start[715]: The default database encoding has accordingly been set to "UTF8". machine # [ 8.532663] postgresql-pre-start[715]: The default text search configuration will be set to "english". machine # [ 8.533224] postgresql-pre-start[715]: Data page checksums are enabled. machine # [ 8.533784] postgresql-pre-start[715]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok machine # [ 8.534457] postgresql-pre-start[715]: creating subdirectories ... ok machine # [ 8.534936] postgresql-pre-start[715]: selecting dynamic shared memory implementation ... posix machine # [ 8.603025] postgresql-pre-start[715]: selecting default "max_connections" ... 100 machine # [ 8.660492] postgresql-pre-start[715]: selecting default "shared_buffers" ... 128MB machine # [ 9.409445] postgresql-pre-start[715]: selecting default time zone ... UTC machine # [ 9.412226] postgresql-pre-start[715]: creating configuration files ... ok machine # [ 9.610503] postgresql-pre-start[715]: running bootstrap script ... ok machine # [ 10.088261] postgresql-pre-start[715]: performing post-bootstrap initialization ... ok machine # [ 10.194947] dhcpcd[682]: eth0: soliciting a DHCP lease machine # [ 10.369572] NET: Registered PF_PACKET protocol family machine # [ 10.231240] dhcpcd[682]: eth0: offered 10.0.2.15 from 10.0.2.2 machine # [ 10.240780] postgresql-pre-start[715]: syncing data to disk ... ok machine # [ 10.242351] postgresql-pre-start[715]: initdb: warning: enabling "trust" authentication for local connections machine # [ 10.243500] postgresql-pre-start[715]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. machine # [ 10.245047] postgresql-pre-start[715]: Success. You can now start the database server using: machine # [ 10.246271] postgresql-pre-start[715]: pg_ctl -D /var/lib/postgresql/18 -l logfile start machine # [ 10.255501] dhcpcd[682]: eth0: probing address 10.0.2.15/24 machine # [ 10.397510] postgres[749]: [749] LOG: starting PostgreSQL 18.6 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit machine # [ 10.402185] postgres[749]: [749] LOG: listening on IPv6 address "::1", port 5432 machine # [ 10.403674] postgres[749]: [749] LOG: listening on IPv4 address "127.0.0.1", port 5432 machine # [ 10.409435] postgres[749]: [749] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" machine # [ 10.426391] postgres[767]: [767] LOG: database system was shut down at 2026-09-23 04:42:36 GMT machine # [ 10.436720] postgres[749]: [749] LOG: database system is ready to accept connections machine # [ 10.441574] systemd[1]: Started PostgreSQL Server. machine # [ 10.445262] systemd[1]: Starting PostgreSQL Setup Scripts... machine # [ 10.515898] dhcpcd[682]: eth0: soliciting an IPv6 router machine # [ 10.518334] dhcpcd[682]: eth0: Router Advertisement from fe80::2 machine # [ 10.520461] dhcpcd[682]: eth0: adding address fec0::5054:ff:fe12:3456/64 machine # [ 10.522778] dhcpcd[682]: eth0: adding route to fec0::/64 machine # [ 10.524787] dhcpcd[682]: eth0: adding default route via fe80::2 machine # [ 10.668287] postgresql-setup-start[778]: CREATE DATABASE machine # [ 10.711865] postgresql-setup-start[783]: CREATE ROLE machine # [ 10.728525] postgresql-setup-start[785]: ALTER DATABASE machine # [ 10.734083] systemd[1]: Finished PostgreSQL Setup Scripts. machine # [ 10.735257] systemd[1]: Reached target PostgreSQL. machine # [ 15.659282] dhcpcd[682]: eth0: leased 10.0.2.15 for 86400 seconds machine # [ 15.661767] dhcpcd[682]: eth0: adding route to 10.0.2.0/24 machine # [ 15.663943] dhcpcd[682]: eth0: adding default route via 10.0.2.2 machine # [ 15.804340] systemd[1]: Started DHCP Client. machine # [ 15.806187] systemd[1]: Reached target Network is Online. machine # [ 15.809555] systemd[1]: Starting k3s service... machine # [ 15.892589] k3s[894]: time="2026-09-23T04:42:42Z" level=info msg="Acquiring lock file /var/lib/rancher/k3s/data/.lock" machine # [ 15.894435] k3s[894]: time="2026-09-23T04:42:42Z" level=info msg="Preparing data dir /var/lib/rancher/k3s/data/05fb32d1d5d79ab4bf8f805d07e4a7dfb491001fd5137b587aa471efd4746099" machine # [ 16.764173] rustfs-bucket-start[900]: mb s3://farm machine # [ 16.771151] systemd[1]: Finished rustfs-bucket.service. machine # [ 18.520381] k3s[894]: time="2026-09-23T04:42:45Z" level=info msg="Starting k3s 1.35.8+k3s1 (e952d68a)" machine # [ 18.609500] k3s[894]: time="2026-09-23T04:42:45Z" level=info msg="Configuring sqlite3 database connection pooling: maxIdle=20, maxOpen=0, maxLifetime=0s, maxIdleTime=2m0s" machine # [ 18.611274] k3s[894]: time="2026-09-23T04:42:45Z" level=info msg="Kine built with sqlite from github.com/mattn/go-sqlite3 version 3.53.3" machine # [ 18.611906] k3s[894]: time="2026-09-23T04:42:45Z" level=info msg="Configuring database table schema and indexes, this may take a moment..." machine # [ 18.632986] k3s[894]: time="2026-09-23T04:42:45Z" level=info msg="Database tables and indexes are up to date" machine # [ 18.634570] k3s[894]: time="2026-09-23T04:42:45Z" level=info msg="Running startup VACUUM to reclaim disk space, this may take a moment..." machine # [ 18.644657] k3s[894]: time="2026-09-23T04:42:45Z" level=info msg="Startup VACUUM completed successfully" machine # [ 18.652135] k3s[894]: time="2026-09-23T04:42:45Z" level=info msg="Kine available at unix://kine.sock" machine # [ 18.653962] k3s[894]: time="2026-09-23T04:42:45Z" level=info msg="Opening etcd client connection with endpoints [unix://kine.sock]" machine # [ 18.658361] k3s[894]: time="2026-09-23T04:42:45Z" level=info msg="Datastore connection validated successfully, proceeding with bootstrap data generation" machine # [ 18.662978] k3s[894]: time="2026-09-23T04:42:45Z" level=info msg="generated self-signed CA certificate CN=k3s-client-ca@1790138565: notBefore=2026-09-23 03:42:45.207733802 +0000 UTC notAfter=2036-09-20 03:42:45.207733802 +0000 UTC" machine # [ 18.664529] k3s[894]: time="2026-09-23T04:42:45Z" level=info msg="certificate CN=system:admin,O=system:masters signed by CN=k3s-client-ca@1790138565: notBefore=2026-09-23 03:42:45 +0000 UTC notAfter=2027-09-23 03:42:45 +0000 UTC" machine # [ 18.666151] k3s[894]: time="2026-09-23T04:42:45Z" level=info msg="certificate CN=system:k3s-supervisor,O=system:masters signed by CN=k3s-client-ca@1790138565: notBefore=2026-09-23 03:42:45 +0000 UTC notAfter=2027-09-23 03:42:45 +0000 UTC" machine # [ 18.667299] k3s[894]: time="2026-09-23T04:42:45Z" level=info msg="certificate CN=system:kube-controller-manager signed by CN=k3s-client-ca@1790138565: notBefore=2026-09-23 03:42:45 +0000 UTC notAfter=2027-09-23 03:42:45 +0000 UTC" machine # [ 18.686352] k3s[894]: time="2026-09-23T04:42:45Z" level=info msg="certificate CN=system:kube-scheduler signed by CN=k3s-client-ca@1790138565: notBefore=2026-09-23 03:42:45 +0000 UTC notAfter=2027-09-23 03:42:45 +0000 UTC" machine # [ 18.687987] k3s[894]: time="2026-09-23T04:42:45Z" level=info msg="certificate CN=system:apiserver,O=system:masters signed by CN=k3s-client-ca@1790138565: notBefore=2026-09-23 03:42:45 +0000 UTC notAfter=2027-09-23 03:42:45 +0000 UTC" machine # [ 18.689204] k3s[894]: time="2026-09-23T04:42:45Z" level=info msg="certificate CN=k3s-cloud-controller-manager signed by CN=k3s-client-ca@1790138565: notBefore=2026-09-23 03:42:45 +0000 UTC notAfter=2027-09-23 03:42:45 +0000 UTC" machine # [ 18.690164] k3s[894]: time="2026-09-23T04:42:45Z" level=info msg="generated self-signed CA certificate CN=k3s-server-ca@1790138565: notBefore=2026-09-23 03:42:45.23586112 +0000 UTC notAfter=2036-09-20 03:42:45.23586112 +0000 UTC" machine # [ 18.691134] k3s[894]: time="2026-09-23T04:42:45Z" level=info msg="certificate CN=kube-apiserver signed by CN=k3s-server-ca@1790138565: notBefore=2026-09-23 03:42:45 +0000 UTC notAfter=2027-09-23 03:42:45 +0000 UTC" machine # [ 18.691952] k3s[894]: time="2026-09-23T04:42:45Z" level=info msg="certificate CN=kube-scheduler signed by CN=k3s-server-ca@1790138565: notBefore=2026-09-23 03:42:45 +0000 UTC notAfter=2027-09-23 03:42:45 +0000 UTC" machine # [ 18.693283] k3s[894]: time="2026-09-23T04:42:45Z" level=info msg="certificate CN=kube-controller-manager signed by CN=k3s-server-ca@1790138565: notBefore=2026-09-23 03:42:45 +0000 UTC notAfter=2027-09-23 03:42:45 +0000 UTC" machine # [ 18.694226] k3s[894]: time="2026-09-23T04:42:45Z" level=info msg="generated self-signed CA certificate CN=k3s-request-header-ca@1790138565: notBefore=2026-09-23 03:42:45.237699901 +0000 UTC notAfter=2036-09-20 03:42:45.237699901 +0000 UTC" machine # [ 18.695163] k3s[894]: time="2026-09-23T04:42:45Z" level=info msg="certificate CN=system:auth-proxy signed by CN=k3s-request-header-ca@1790138565: notBefore=2026-09-23 03:42:45 +0000 UTC notAfter=2027-09-23 03:42:45 +0000 UTC" machine # [ 18.696097] k3s[894]: time="2026-09-23T04:42:45Z" level=info msg="generated self-signed CA certificate CN=etcd-server-ca@1790138565: notBefore=2026-09-23 03:42:45.238459216 +0000 UTC notAfter=2036-09-20 03:42:45.238459216 +0000 UTC" machine # [ 18.697063] k3s[894]: time="2026-09-23T04:42:45Z" level=info msg="certificate CN=etcd-client signed by CN=etcd-server-ca@1790138565: notBefore=2026-09-23 03:42:45 +0000 UTC notAfter=2027-09-23 03:42:45 +0000 UTC" machine # [ 18.698350] k3s[894]: time="2026-09-23T04:42:45Z" level=info msg="generated self-signed CA certificate CN=etcd-peer-ca@1790138565: notBefore=2026-09-23 03:42:45.239158187 +0000 UTC notAfter=2036-09-20 03:42:45.239158187 +0000 UTC" machine # [ 18.699796] k3s[894]: time="2026-09-23T04:42:45Z" level=info msg="certificate CN=etcd-peer signed by CN=etcd-peer-ca@1790138565: notBefore=2026-09-23 03:42:45 +0000 UTC notAfter=2027-09-23 03:42:45 +0000 UTC" machine # [ 18.700724] k3s[894]: time="2026-09-23T04:42:45Z" level=info msg="certificate CN=etcd-server signed by CN=etcd-server-ca@1790138565: notBefore=2026-09-23 03:42:45 +0000 UTC notAfter=2027-09-23 03:42:45 +0000 UTC" machine # [ 18.714813] k3s[894]: time="2026-09-23T04:42:45Z" level=info msg="certificate CN=k3s,O=k3s signed by CN=k3s-server-ca@1790138565: notBefore=2026-09-23 03:42:45 +0000 UTC notAfter=2027-09-23 03:42:45 +0000 UTC" machine # [ 18.715738] k3s[894]: time="2026-09-23T04:42:45Z" level=warning msg="dynamiclistener [::]:6443: no cached certificate available for preload - deferring certificate load until storage initialization or first client request" machine # [ 18.721641] k3s[894]: time="2026-09-23T04:42:45Z" level=info msg="Active TLS secret / (ver=) (count 11): map[listener.cattle.io/cn-10.0.2.15:10.0.2.15 listener.cattle.io/cn-10.43.0.1:10.43.0.1 listener.cattle.io/cn-127.0.0.1:127.0.0.1 listener.cattle.io/cn-__1-f16284:::1 listener.cattle.io/cn-fec0__7b2b_a512_d19_a042-42b93d:fec0::7b2b:a512:d19:a042 listener.cattle.io/cn-kubernetes:kubernetes listener.cattle.io/cn-kubernetes.default:kubernetes.default listener.cattle.io/cn-kubernetes.default.svc:kubernetes.default.svc listener.cattle.io/cn-kubernetes.default.svc.cluster.local:kubernetes.default.svc.cluster.local listener.cattle.io/cn-localhost:localhost listener.cattle.io/cn-machine:machine listener.cattle.io/fingerprint:SHA1=887835569D857DEA06B5F8D58D35944C7C42FA6F]" machine # [ 19.858683] k3s[894]: time="2026-09-23T04:42:46Z" level=info msg="Password verified locally for node machine" machine # [ 19.860771] k3s[894]: time="2026-09-23T04:42:46Z" level=info msg="certificate CN=machine signed by CN=k3s-server-ca@1790138565: notBefore=2026-09-23 03:42:46 +0000 UTC notAfter=2027-09-23 03:42:46 +0000 UTC" machine # [ 20.110442] k3s[894]: time="2026-09-23T04:42:46Z" level=info msg="certificate CN=system:node:machine,O=system:nodes signed by CN=k3s-client-ca@1790138565: notBefore=2026-09-23 03:42:46 +0000 UTC notAfter=2027-09-23 03:42:46 +0000 UTC" machine # [ 20.203479] k3s[894]: time="2026-09-23T04:42:46Z" level=info msg="certificate CN=system:kube-proxy signed by CN=k3s-client-ca@1790138565: notBefore=2026-09-23 03:42:46 +0000 UTC notAfter=2027-09-23 03:42:46 +0000 UTC" machine # [ 20.320245] k3s[894]: time="2026-09-23T04:42:46Z" level=info msg="certificate CN=system:k3s-controller signed by CN=k3s-client-ca@1790138565: notBefore=2026-09-23 03:42:46 +0000 UTC notAfter=2027-09-23 03:42:46 +0000 UTC" machine # [ 20.433276] k3s[894]: time="2026-09-23T04:42:46Z" level=error msg="Sending HTTP/1.1 503 response to 127.0.0.1:41520: runtime core not ready" machine # [ 20.543558] k3s[894]: time="2026-09-23T04:42:47Z" level=info msg="Module overlay was already loaded" machine # [ 20.765298] bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this. machine # [ 20.769929] Bridge firewalling registered machine # [ 20.626144] k3s[894]: time="2026-09-23T04:42:47Z" level=warning msg="Failed to load kernel module iptable_nat with modprobe" machine # [ 20.636834] k3s[894]: time="2026-09-23T04:42:47Z" level=warning msg="Failed to load kernel module iptable_filter with modprobe" machine # [ 20.647467] k3s[894]: time="2026-09-23T04:42:47Z" level=warning msg="Failed to load kernel module nft-expr-counter with modprobe" machine # [ 20.734434] k3s[894]: time="2026-09-23T04:42:47Z" level=info msg="Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_established' to 86400" machine # [ 20.735704] k3s[894]: time="2026-09-23T04:42:47Z" level=info msg="Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_close_wait' to 3600" machine # [ 20.736773] k3s[894]: time="2026-09-23T04:42:47Z" level=info msg="Set sysctl 'net/ipv4/conf/all/forwarding' to 1" machine # [ 20.744354] k3s[894]: time="2026-09-23T04:42:47Z" level=info msg="Set sysctl 'net/netfilter/nf_conntrack_max' to 131072" machine # [ 20.751268] k3s[894]: time="2026-09-23T04:42:47Z" level=info msg="Connecting to proxy" url="wss://127.0.0.1:6443/v1-k3s/connect" machine # [ 20.754926] k3s[894]: time="2026-09-23T04:42:47Z" level=info msg="Polling for API server readiness: GET /readyz failed: the server is currently unable to handle the request" machine # [ 20.755584] k3s[894]: time="2026-09-23T04:42:47Z" level=info msg="Creating k3s-cert-monitor event broadcaster" machine # [ 20.756728] k3s[894]: time="2026-09-23T04:42:47Z" level=info msg="Saving cluster bootstrap data to datastore" machine # [ 20.757873] k3s[894]: time="2026-09-23T04:42:47Z" level=info msg="Opening etcd client connection with endpoints [unix://kine.sock]" machine # [ 20.759067] k3s[894]: time="2026-09-23T04:42:47Z" level=info msg="Connection to etcd is ready" machine # [ 20.759800] k3s[894]: time="2026-09-23T04:42:47Z" level=info msg="ETCD server is now running" machine # [ 20.760960] k3s[894]: time="2026-09-23T04:42:47Z" level=info msg="Handling backend connection request [machine]" machine # [ 20.762240] k3s[894]: time="2026-09-23T04:42:47Z" level=info msg="Connected to proxy" url="wss://127.0.0.1:6443/v1-k3s/connect" machine # [ 20.763470] k3s[894]: time="2026-09-23T04:42:47Z" level=info msg="Remotedialer connected to proxy" url="wss://127.0.0.1:6443/v1-k3s/connect" machine # [ 20.764801] k3s[894]: time="2026-09-23T04:42:47Z" level=info msg="Running kube-apiserver --advertise-port=6443 --allow-privileged=true --anonymous-auth=false --api-audiences=https://kubernetes.default.svc.cluster.local,k3s --authorization-mode=Node,RBAC --bind-address=127.0.0.1 --cert-dir=/var/lib/rancher/k3s/server/tls/temporary-certs --client-ca-file=/var/lib/rancher/k3s/server/tls/client-ca.crt --egress-selector-config-file=/var/lib/rancher/k3s/server/etc/egress-selector-config.yaml --enable-admission-plugins=NodeRestriction --enable-aggregator-routing=true --enable-bootstrap-token-auth=true --etcd-servers=unix://kine.sock --kubelet-certificate-authority=/var/lib/rancher/k3s/server/tls/server-ca.crt --kubelet-client-certificate=/var/lib/rancher/k3s/server/tls/client-kube-apiserver.crt --kubelet-client-key=/var/lib/rancher/k3s/server/tls/client-kube-apiserver.key --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --profiling=false --proxy-client-cert-file=/var/lib/rancher/k3s/server/tls/client-auth-proxy.crt --proxy-client-key-file=/var/lib/rancher/k3s/server/tls/client-auth-proxy.key --requestheader-allowed-names=system:auth-proxy --requestheader-client-ca-file=/var/lib/rancher/k3s/server/tls/request-header-ca.crt --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group --requestheader-username-headers=X-Remote-User --secure-port=6444 --service-account-issuer=https://kubernetes.default.svc.cluster.local --service-account-key-file=/var/lib/rancher/k3s/server/tls/service.key --service-account-signing-key-file=/var/lib/rancher/k3s/server/tls/service.current.key --service-cluster-ip-range=10.43.0.0/16 --service-node-port-range=30000-32767 --storage-backend=etcd3 --tls-cert-file=/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.crt --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305 --tls-private-key-file=/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.key" machine # [ 20.767110] k3s[894]: time="2026-09-23T04:42:47Z" level=info msg="Running kube-scheduler --authentication-kubeconfig=/var/lib/rancher/k3s/server/cred/scheduler.kubeconfig --authorization-kubeconfig=/var/lib/rancher/k3s/server/cred/scheduler.kubeconfig --bind-address=127.0.0.1 --kubeconfig=/var/lib/rancher/k3s/server/cred/scheduler.kubeconfig --leader-elect=false --profiling=false --secure-port=10259 --tls-cert-file=/var/lib/rancher/k3s/server/tls/kube-scheduler/kube-scheduler.crt --tls-private-key-file=/var/lib/rancher/k3s/server/tls/kube-scheduler/kube-scheduler.key" machine # [ 20.768996] k3s[894]: time="2026-09-23T04:42:47Z" level=info msg="Running kube-controller-manager --allocate-node-cidrs=true --authentication-kubeconfig=/var/lib/rancher/k3s/server/cred/controller.kubeconfig --authorization-kubeconfig=/var/lib/rancher/k3s/server/cred/controller.kubeconfig --bind-address=127.0.0.1 --cluster-cidr=10.42.0.0/16 --cluster-signing-kube-apiserver-client-cert-file=/var/lib/rancher/k3s/server/tls/client-ca.nochain.crt --cluster-signing-kube-apiserver-client-key-file=/var/lib/rancher/k3s/server/tls/client-ca.key --cluster-signing-kubelet-client-cert-file=/var/lib/rancher/k3s/server/tls/client-ca.nochain.crt --cluster-signing-kubelet-client-key-file=/var/lib/rancher/k3s/server/tls/client-ca.key --cluster-signing-kubelet-serving-cert-file=/var/lib/rancher/k3s/server/tls/server-ca.nochain.crt --cluster-signing-kubelet-serving-key-file=/var/lib/rancher/k3s/server/tls/server-ca.key --cluster-signing-legacy-unknown-cert-file=/var/lib/rancher/k3s/server/tls/server-ca.nochain.crt --cluster-signing-legacy-unknown-key-file=/var/lib/rancher/k3s/server/tls/server-ca.key --configure-cloud-routes=false --controllers=*,tokencleaner,-service,-route,-cloud-node-lifecycle --kubeconfig=/var/lib/rancher/k3s/server/cred/controller.kubeconfig --leader-elect=false --profiling=false --root-ca-file=/var/lib/rancher/k3s/server/tls/server-ca.crt --secure-port=10257 --service-account-private-key-file=/var/lib/rancher/k3s/server/tls/service.current.key --service-cluster-ip-range=10.43.0.0/16 --tls-cert-file=/var/lib/rancher/k3s/server/tls/kube-controller-manager/kube-controller-manager.crt --tls-private-key-file=/var/lib/rancher/k3s/server/tls/kube-controller-manager/kube-controller-manager.key --use-service-account-credentials=true" machine # [ 20.819795] k3s[894]: time="2026-09-23T04:42:47Z" level=info msg="Running cloud-controller-manager --allocate-node-cidrs=true --authentication-kubeconfig=/var/lib/rancher/k3s/server/cred/cloud-controller.kubeconfig --authorization-kubeconfig=/var/lib/rancher/k3s/server/cred/cloud-controller.kubeconfig --bind-address=127.0.0.1 --cloud-config=/var/lib/rancher/k3s/server/etc/cloud-config.yaml --cloud-provider=k3s --cluster-cidr=10.42.0.0/16 --configure-cloud-routes=false --controllers=*,-route --kubeconfig=/var/lib/rancher/k3s/server/cred/cloud-controller.kubeconfig --leader-elect=false --leader-elect-resource-name=k3s-cloud-controller-manager --node-status-update-frequency=1m0s --profiling=false" machine # [ 20.820544] k3s[894]: time="2026-09-23T04:42:47Z" level=info msg="Server node token is available at /var/lib/rancher/k3s/server/token" machine # [ 20.820981] k3s[894]: time="2026-09-23T04:42:47Z" level=info msg="To join server node to cluster: k3s server -s https://10.0.2.15:6443 -t ${SERVER_NODE_TOKEN}" machine # [ 20.821645] k3s[894]: time="2026-09-23T04:42:47Z" level=info msg="Agent node token is available at /var/lib/rancher/k3s/server/agent-token" machine # [ 20.822172] k3s[894]: time="2026-09-23T04:42:47Z" level=info msg="To join agent node to cluster: k3s agent -s https://10.0.2.15:6443 -t ${AGENT_NODE_TOKEN}" machine # [ 20.822558] k3s[894]: time="2026-09-23T04:42:47Z" level=info msg="Wrote kubeconfig /etc/rancher/k3s/k3s.yaml" machine # [ 20.823062] k3s[894]: time="2026-09-23T04:42:47Z" level=info msg="Run: k3s kubectl" machine # [ 20.823507] k3s[894]: I0923 04:42:47.321086 894 options.go:263] external host was not specified, using 10.0.2.15 machine # [ 20.824040] k3s[894]: I0923 04:42:47.323355 894 server.go:158] Version: v1.35.8+k3s1 machine # [ 20.824516] k3s[894]: I0923 04:42:47.323390 894 server.go:160] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" machine # [ 20.824961] k3s[894]: time="2026-09-23T04:42:47Z" level=info msg="Logging containerd to /var/lib/rancher/k3s/agent/containerd/containerd.log" machine # [ 20.825655] k3s[894]: time="2026-09-23T04:42:47Z" level=info msg="Running containerd -c /var/lib/rancher/k3s/agent/etc/containerd/config.toml" machine # [ 20.829061] k3s[894]: time="2026-09-23T04:42:47Z" level=error msg="Sending HTTP/1.1 503 response to 127.0.0.1:41580: runtime core not ready" machine # [ 20.942754] k3s[894]: I0923 04:42:47.490188 894 shared_informer.go:349] "Waiting for caches to sync" controller="node_authorizer" machine # [ 20.945722] k3s[894]: I0923 04:42:47.493747 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 20.947914] k3s[894]: I0923 04:42:47.495937 894 plugins.go:157] Loaded 14 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,PodTopologyLabels,MutatingAdmissionPolicy,MutatingAdmissionWebhook. machine # [ 20.955195] k3s[894]: I0923 04:42:47.495970 894 plugins.go:160] Loaded 14 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,PodSecurity,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,ClusterTrustBundleAttest,CertificateSubjectRestriction,NodeDeclaredFeatureValidator,ValidatingAdmissionPolicy,ValidatingAdmissionWebhook,ResourceQuota. machine # [ 20.959682] k3s[894]: I0923 04:42:47.496301 894 instance.go:240] Using reconciler: lease machine # [ 20.960817] k3s[894]: I0923 04:42:47.502587 894 handler.go:304] Adding GroupVersion apiextensions.k8s.io v1 to ResourceManager machine # [ 20.962392] k3s[894]: W0923 04:42:47.502621 894 genericapiserver.go:787] Skipping API apiextensions.k8s.io/v1beta1 because it has no resources. machine # [ 20.964118] k3s[894]: I0923 04:42:47.505508 894 cidrallocator.go:198] starting ServiceCIDR Allocator Controller machine # [ 20.965441] k3s[894]: time="2026-09-23T04:42:47Z" level=info msg="Running kube-proxy --cluster-cidr=10.42.0.0/16 --conntrack-max-per-core=0 --conntrack-tcp-timeout-close-wait=0s --conntrack-tcp-timeout-established=0s --healthz-bind-address=127.0.0.1 --hostname-override=machine --kubeconfig=/var/lib/rancher/k3s/agent/kubeproxy.kubeconfig --proxy-mode=iptables" machine # [ 21.010535] k3s[894]: I0923 04:42:47.558486 894 handler.go:304] Adding GroupVersion v1 to ResourceManager machine # [ 21.012236] k3s[894]: I0923 04:42:47.558739 894 apis.go:112] API group "internal.apiserver.k8s.io" is not enabled, skipping. machine # [ 21.039073] k3s[894]: I0923 04:42:47.586502 894 apis.go:112] API group "storagemigration.k8s.io" is not enabled, skipping. machine # [ 21.076335] k3s[894]: I0923 04:42:47.624306 894 handler.go:304] Adding GroupVersion authentication.k8s.io v1 to ResourceManager machine # [ 21.077904] k3s[894]: W0923 04:42:47.624346 894 genericapiserver.go:787] Skipping API authentication.k8s.io/v1beta1 because it has no resources. machine # [ 21.079537] k3s[894]: W0923 04:42:47.624360 894 genericapiserver.go:787] Skipping API authentication.k8s.io/v1alpha1 because it has no resources. machine # [ 21.081226] k3s[894]: I0923 04:42:47.624704 894 handler.go:304] Adding GroupVersion authorization.k8s.io v1 to ResourceManager machine # [ 21.082663] k3s[894]: W0923 04:42:47.624721 894 genericapiserver.go:787] Skipping API authorization.k8s.io/v1beta1 because it has no resources. machine # [ 21.084334] k3s[894]: I0923 04:42:47.625435 894 handler.go:304] Adding GroupVersion autoscaling v2 to ResourceManager machine # [ 21.085651] k3s[894]: I0923 04:42:47.625979 894 handler.go:304] Adding GroupVersion autoscaling v1 to ResourceManager machine # [ 21.087066] k3s[894]: W0923 04:42:47.625995 894 genericapiserver.go:787] Skipping API autoscaling/v2beta1 because it has no resources. machine # [ 21.088527] k3s[894]: W0923 04:42:47.626005 894 genericapiserver.go:787] Skipping API autoscaling/v2beta2 because it has no resources. machine # [ 21.090076] k3s[894]: I0923 04:42:47.627206 894 handler.go:304] Adding GroupVersion batch v1 to ResourceManager machine # [ 21.091354] k3s[894]: W0923 04:42:47.627225 894 genericapiserver.go:787] Skipping API batch/v1beta1 because it has no resources. machine # [ 21.092773] k3s[894]: I0923 04:42:47.628003 894 handler.go:304] Adding GroupVersion certificates.k8s.io v1 to ResourceManager machine # [ 21.094277] k3s[894]: W0923 04:42:47.628021 894 genericapiserver.go:787] Skipping API certificates.k8s.io/v1beta1 because it has no resources. machine # [ 21.095838] k3s[894]: W0923 04:42:47.628033 894 genericapiserver.go:787] Skipping API certificates.k8s.io/v1alpha1 because it has no resources. machine # [ 21.097552] k3s[894]: I0923 04:42:47.628629 894 handler.go:304] Adding GroupVersion coordination.k8s.io v1 to ResourceManager machine # [ 21.099094] k3s[894]: W0923 04:42:47.628648 894 genericapiserver.go:787] Skipping API coordination.k8s.io/v1beta1 because it has no resources. machine # [ 21.100702] k3s[894]: W0923 04:42:47.628666 894 genericapiserver.go:787] Skipping API coordination.k8s.io/v1alpha2 because it has no resources. machine # [ 21.102349] k3s[894]: I0923 04:42:47.629298 894 handler.go:304] Adding GroupVersion discovery.k8s.io v1 to ResourceManager machine # [ 21.103805] k3s[894]: W0923 04:42:47.629316 894 genericapiserver.go:787] Skipping API discovery.k8s.io/v1beta1 because it has no resources. machine # [ 21.105433] k3s[894]: I0923 04:42:47.631636 894 handler.go:304] Adding GroupVersion networking.k8s.io v1 to ResourceManager machine # [ 21.106898] k3s[894]: W0923 04:42:47.631660 894 genericapiserver.go:787] Skipping API networking.k8s.io/v1beta1 because it has no resources. machine # [ 21.108539] k3s[894]: I0923 04:42:47.631989 894 handler.go:304] Adding GroupVersion node.k8s.io v1 to ResourceManager machine # [ 21.109862] k3s[894]: W0923 04:42:47.632005 894 genericapiserver.go:787] Skipping API node.k8s.io/v1beta1 because it has no resources. machine # [ 21.111422] k3s[894]: W0923 04:42:47.632016 894 genericapiserver.go:787] Skipping API node.k8s.io/v1alpha1 because it has no resources. machine # [ 21.112930] k3s[894]: I0923 04:42:47.632566 894 handler.go:304] Adding GroupVersion policy v1 to ResourceManager machine # [ 21.114235] k3s[894]: W0923 04:42:47.632583 894 genericapiserver.go:787] Skipping API policy/v1beta1 because it has no resources. machine # [ 21.115669] k3s[894]: I0923 04:42:47.633878 894 handler.go:304] Adding GroupVersion rbac.authorization.k8s.io v1 to ResourceManager machine # [ 21.117367] k3s[894]: W0923 04:42:47.633898 894 genericapiserver.go:787] Skipping API rbac.authorization.k8s.io/v1beta1 because it has no resources. machine # [ 21.119113] k3s[894]: W0923 04:42:47.633910 894 genericapiserver.go:787] Skipping API rbac.authorization.k8s.io/v1alpha1 because it has no resources. machine # [ 21.120880] k3s[894]: I0923 04:42:47.634289 894 handler.go:304] Adding GroupVersion scheduling.k8s.io v1 to ResourceManager machine # [ 21.122348] k3s[894]: W0923 04:42:47.634305 894 genericapiserver.go:787] Skipping API scheduling.k8s.io/v1beta1 because it has no resources. machine # [ 21.123887] k3s[894]: W0923 04:42:47.634317 894 genericapiserver.go:787] Skipping API scheduling.k8s.io/v1alpha1 because it has no resources. machine # [ 21.125478] k3s[894]: I0923 04:42:47.636283 894 handler.go:304] Adding GroupVersion storage.k8s.io v1 to ResourceManager machine # [ 21.126908] k3s[894]: W0923 04:42:47.636305 894 genericapiserver.go:787] Skipping API storage.k8s.io/v1beta1 because it has no resources. machine # [ 21.128541] k3s[894]: W0923 04:42:47.636317 894 genericapiserver.go:787] Skipping API storage.k8s.io/v1alpha1 because it has no resources. machine # [ 21.130132] k3s[894]: I0923 04:42:47.637409 894 handler.go:304] Adding GroupVersion flowcontrol.apiserver.k8s.io v1 to ResourceManager machine # [ 21.131670] k3s[894]: W0923 04:42:47.637427 894 genericapiserver.go:787] Skipping API flowcontrol.apiserver.k8s.io/v1beta3 because it has no resources. machine # [ 21.133407] k3s[894]: W0923 04:42:47.637440 894 genericapiserver.go:787] Skipping API flowcontrol.apiserver.k8s.io/v1beta2 because it has no resources. machine # [ 21.135180] k3s[894]: W0923 04:42:47.637450 894 genericapiserver.go:787] Skipping API flowcontrol.apiserver.k8s.io/v1beta1 because it has no resources. machine # [ 21.136816] k3s[894]: I0923 04:42:47.640075 894 handler.go:304] Adding GroupVersion apps v1 to ResourceManager machine # [ 21.138130] k3s[894]: W0923 04:42:47.640122 894 genericapiserver.go:787] Skipping API apps/v1beta2 because it has no resources. machine # [ 21.139840] k3s[894]: W0923 04:42:47.640134 894 genericapiserver.go:787] Skipping API apps/v1beta1 because it has no resources. machine # [ 21.141478] k3s[894]: I0923 04:42:47.641430 894 handler.go:304] Adding GroupVersion admissionregistration.k8s.io v1 to ResourceManager machine # [ 21.143251] k3s[894]: W0923 04:42:47.641449 894 genericapiserver.go:787] Skipping API admissionregistration.k8s.io/v1beta1 because it has no resources. machine # [ 21.145521] k3s[894]: W0923 04:42:47.641460 894 genericapiserver.go:787] Skipping API admissionregistration.k8s.io/v1alpha1 because it has no resources. machine # [ 21.147289] k3s[894]: I0923 04:42:47.641814 894 handler.go:304] Adding GroupVersion events.k8s.io v1 to ResourceManager machine # [ 21.148676] k3s[894]: W0923 04:42:47.641830 894 genericapiserver.go:787] Skipping API events.k8s.io/v1beta1 because it has no resources. machine # [ 21.150179] k3s[894]: I0923 04:42:47.643033 894 handler.go:304] Adding GroupVersion resource.k8s.io v1 to ResourceManager machine # [ 21.151604] k3s[894]: W0923 04:42:47.643053 894 genericapiserver.go:787] Skipping API resource.k8s.io/v1beta2 because it has no resources. machine # [ 21.153290] k3s[894]: W0923 04:42:47.643077 894 genericapiserver.go:787] Skipping API resource.k8s.io/v1beta1 because it has no resources. machine # [ 21.154871] k3s[894]: W0923 04:42:47.643088 894 genericapiserver.go:787] Skipping API resource.k8s.io/v1alpha3 because it has no resources. machine # [ 21.156471] k3s[894]: I0923 04:42:47.646244 894 handler.go:304] Adding GroupVersion apiregistration.k8s.io v1 to ResourceManager machine # [ 21.157908] k3s[894]: W0923 04:42:47.646296 894 genericapiserver.go:787] Skipping API apiregistration.k8s.io/v1beta1 because it has no resources. machine # [ 21.417448] k3s[894]: I0923 04:42:47.965351 894 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/var/lib/rancher/k3s/server/tls/client-ca.crt" machine # [ 21.419639] k3s[894]: I0923 04:42:47.965349 894 dynamic_cafile_content.go:161] "Starting controller" name="request-header::/var/lib/rancher/k3s/server/tls/request-header-ca.crt" machine # [ 21.421755] k3s[894]: I0923 04:42:47.965458 894 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.crt::/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.key" machine # [ 21.424624] k3s[894]: I0923 04:42:47.965691 894 secure_serving.go:211] Serving securely on 127.0.0.1:6444 machine # [ 21.425880] k3s[894]: I0923 04:42:47.965859 894 apf_controller.go:377] Starting API Priority and Fairness config controller machine # [ 21.427470] k3s[894]: I0923 04:42:47.965915 894 aggregator.go:185] waiting for initial CRD sync... machine # [ 21.428700] k3s[894]: time="2026-09-23T04:42:47Z" level=info msg="Starting legacy_token_tracking_controller" logger=k3s machine # [ 21.433076] k3s[894]: time="2026-09-23T04:42:47Z" level=info msg="Waiting for caches to sync" logger=k3s machine # [ 21.434359] k3s[894]: I0923 04:42:47.966004 894 tlsconfig.go:243] "Starting DynamicServingCertificateController" machine # [ 21.435698] k3s[894]: time="2026-09-23T04:42:47Z" level=info msg="Starting apiserver lease garbage collector" logger=k3s machine # [ 21.437172] k3s[894]: time="2026-09-23T04:42:47Z" level=info msg="Waiting for caches to sync" logger=k3s machine # [ 21.438396] k3s[894]: I0923 04:42:47.966533 894 dynamic_serving_content.go:135] "Starting controller" name="aggregator-proxy-cert::/var/lib/rancher/k3s/server/tls/client-auth-proxy.crt::/var/lib/rancher/k3s/server/tls/client-auth-proxy.key" machine # [ 21.441197] k3s[894]: I0923 04:42:47.966804 894 apiservice_controller.go:100] Starting APIServiceRegistrationController machine # [ 21.442670] k3s[894]: I0923 04:42:47.966822 894 cache.go:32] Waiting for caches to sync for APIServiceRegistrationController controller machine # [ 21.444321] k3s[894]: I0923 04:42:47.966950 894 controller.go:113] "Deleting old lease on startup" lease="kube-system/apiserver-ik4pcwfs7qgdl76byc4pomgvia" machine # [ 21.446241] k3s[894]: I0923 04:42:47.967030 894 cluster_authentication_trust_controller.go:459] Starting cluster_authentication_trust_controller controller machine # [ 21.448142] k3s[894]: time="2026-09-23T04:42:47Z" level=info msg="Waiting for caches to sync" logger=k3s machine # [ 21.449343] k3s[894]: I0923 04:42:47.967356 894 customresource_discovery_controller.go:294] Starting DiscoveryController machine # [ 21.450806] k3s[894]: I0923 04:42:47.967401 894 local_available_controller.go:156] Starting LocalAvailability controller machine # [ 21.452392] k3s[894]: I0923 04:42:47.967415 894 cache.go:32] Waiting for caches to sync for LocalAvailability controller machine # [ 21.453898] k3s[894]: I0923 04:42:47.967460 894 remote_available_controller.go:425] Starting RemoteAvailability controller machine # [ 21.455526] k3s[894]: I0923 04:42:47.967473 894 cache.go:32] Waiting for caches to sync for RemoteAvailability controller machine # [ 21.457128] k3s[894]: I0923 04:42:47.967513 894 controller.go:78] Starting OpenAPI AggregationController machine # [ 21.458421] k3s[894]: I0923 04:42:47.967549 894 controller.go:80] Starting OpenAPI V3 AggregationController machine # [ 21.459827] k3s[894]: I0923 04:42:47.967602 894 crdregistration_controller.go:114] Starting crd-autoregister controller machine # [ 21.461395] k3s[894]: I0923 04:42:47.967619 894 shared_informer.go:349] "Waiting for caches to sync" controller="crd-autoregister" machine # [ 21.463109] k3s[894]: I0923 04:42:47.967629 894 system_namespaces_controller.go:66] Starting system namespaces controller machine # [ 21.464641] k3s[894]: I0923 04:42:47.967956 894 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/var/lib/rancher/k3s/server/tls/client-ca.crt" machine # [ 21.466792] k3s[894]: I0923 04:42:47.968226 894 dynamic_cafile_content.go:161] "Starting controller" name="request-header::/var/lib/rancher/k3s/server/tls/request-header-ca.crt" machine # [ 21.468984] k3s[894]: I0923 04:42:47.968776 894 controller.go:142] Starting OpenAPI controller machine # [ 21.470279] k3s[894]: I0923 04:42:47.968843 894 controller.go:90] Starting OpenAPI V3 controller machine # [ 21.471537] k3s[894]: I0923 04:42:47.968882 894 naming_controller.go:305] Starting NamingConditionController machine # [ 21.472937] k3s[894]: I0923 04:42:47.968971 894 nonstructuralschema_controller.go:202] Starting NonStructuralSchemaConditionController machine # [ 21.474600] k3s[894]: I0923 04:42:47.969003 894 apiapproval_controller.go:196] Starting KubernetesAPIApprovalPolicyConformantConditionController machine # [ 21.476335] k3s[894]: I0923 04:42:47.969043 894 crd_finalizer.go:273] Starting CRDFinalizer machine # [ 21.477482] k3s[894]: I0923 04:42:47.969219 894 default_servicecidr_controller.go:110] Starting kubernetes-service-cidr-controller machine # [ 21.478943] k3s[894]: I0923 04:42:47.969236 894 shared_informer.go:349] "Waiting for caches to sync" controller="kubernetes-service-cidr-controller" machine # [ 21.480621] k3s[894]: I0923 04:42:47.970777 894 repairip.go:210] Starting ipallocator-repair-controller machine # [ 21.481826] k3s[894]: I0923 04:42:47.970792 894 shared_informer.go:349] "Waiting for caches to sync" controller="ipallocator-repair-controller" machine # [ 21.519067] k3s[894]: time="2026-09-23T04:42:48Z" level=info msg="Caches are synced" logger=k3s machine # [ 21.520302] k3s[894]: time="2026-09-23T04:42:48Z" level=info msg="Caches are synced" logger=k3s machine # [ 21.521454] k3s[894]: I0923 04:42:48.066941 894 cache.go:39] Caches are synced for APIServiceRegistrationController controller machine # [ 21.522939] k3s[894]: I0923 04:42:48.066946 894 apf_controller.go:382] Running API Priority and Fairness config worker machine # [ 21.524410] k3s[894]: I0923 04:42:48.067006 894 handler_discovery.go:451] Starting ResourceDiscoveryManager machine # [ 21.525790] k3s[894]: I0923 04:42:48.067013 894 apf_controller.go:385] Running API Priority and Fairness periodic rebalancing process machine # [ 21.527437] k3s[894]: time="2026-09-23T04:42:48Z" level=info msg="Caches are synced" logger=k3s machine # [ 21.528532] k3s[894]: I0923 04:42:48.067469 894 cache.go:39] Caches are synced for LocalAvailability controller machine # [ 21.529940] k3s[894]: I0923 04:42:48.067516 894 cache.go:39] Caches are synced for RemoteAvailability controller machine # [ 21.530488] k3s[894]: I0923 04:42:48.069651 894 shared_informer.go:356] "Caches are synced" controller="kubernetes-service-cidr-controller" machine # [ 21.531064] k3s[894]: I0923 04:42:48.069684 894 default_servicecidr_controller.go:169] Creating default ServiceCIDR with CIDRs: [10.43.0.0/16] machine # [ 21.531558] k3s[894]: I0923 04:42:48.071294 894 shared_informer.go:356] "Caches are synced" controller="crd-autoregister" machine # [ 21.532066] k3s[894]: I0923 04:42:48.071504 894 aggregator.go:187] initial CRD sync complete... machine # [ 21.532582] k3s[894]: I0923 04:42:48.071540 894 autoregister_controller.go:144] Starting autoregister controller machine # [ 21.533075] k3s[894]: I0923 04:42:48.071552 894 cache.go:32] Waiting for caches to sync for autoregister controller machine # [ 21.533537] k3s[894]: I0923 04:42:48.071563 894 cache.go:39] Caches are synced for autoregister controller machine # [ 21.534135] k3s[894]: I0923 04:42:48.071601 894 shared_informer.go:356] "Caches are synced" controller="ipallocator-repair-controller" machine # [ 21.534601] k3s[894]: E0923 04:42:48.080458 894 controller.go:201] "Failed to ensure lease exists, will retry" err="namespaces \"kube-system\" not found" interval="200ms" machine # [ 21.543636] k3s[894]: I0923 04:42:48.091639 894 shared_informer.go:356] "Caches are synced" controller="node_authorizer" machine # [ 21.546300] k3s[894]: I0923 04:42:48.094233 894 shared_informer.go:377] "Caches are synced" machine # [ 21.546865] k3s[894]: I0923 04:42:48.094350 894 policy_source.go:248] refreshing policies machine # [ 21.576847] k3s[894]: E0923 04:42:48.124803 894 controller.go:95] Unable to perform initial Kubernetes service initialization: namespaces "default" not found machine # [ 21.579700] k3s[894]: time="2026-09-23T04:42:48Z" level=error msg="Error while syncing ConfigMap" configmap=kube-system/kube-apiserver-legacy-service-account-token-tracking error="namespaces \"kube-system\" not found" logger=k3s/UnhandledError machine # [ 21.624149] k3s[894]: I0923 04:42:48.172008 894 controller.go:667] quota admission added evaluator for: namespaces machine # [ 21.632876] k3s[894]: I0923 04:42:48.180843 894 cidrallocator.go:302] created ClusterIP allocator for Service CIDR 10.43.0.0/16 machine # [ 21.636298] k3s[894]: I0923 04:42:48.184263 894 default_servicecidr_controller.go:231] Setting default ServiceCIDR condition Ready to True machine # [ 21.657254] k3s[894]: I0923 04:42:48.205243 894 cidrallocator.go:278] updated ClusterIP allocator for Service CIDR 10.43.0.0/16 machine # [ 21.736305] k3s[894]: I0923 04:42:48.284218 894 controller.go:667] quota admission added evaluator for: leases.coordination.k8s.io machine # [ 21.864375] k3s[894]: time="2026-09-23T04:42:48Z" level=info msg="containerd is now running" machine # [ 21.880880] k3s[894]: time="2026-09-23T04:42:48Z" level=info msg="Importing images from /var/lib/rancher/k3s/agent/images/k3s-airgap-images-amd64.tar.zst" machine # [ 22.424916] k3s[894]: I0923 04:42:48.972592 894 storage_scheduling.go:123] created PriorityClass system-node-critical with value 2000001000 machine # [ 22.429743] k3s[894]: I0923 04:42:48.977740 894 storage_scheduling.go:123] created PriorityClass system-cluster-critical with value 2000000000 machine # [ 22.430462] k3s[894]: I0923 04:42:48.977774 894 storage_scheduling.go:139] all system priority classes are created successfully or already exist. machine # [ 22.753285] k3s[894]: time="2026-09-23T04:42:49Z" level=info msg="Polling for API server readiness: GET /readyz failed: unknown" machine # [ 23.304255] k3s[894]: I0923 04:42:49.852235 894 controller.go:667] quota admission added evaluator for: roles.rbac.authorization.k8s.io machine # [ 23.348592] k3s[894]: I0923 04:42:49.896579 894 controller.go:667] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io machine # [ 23.487258] k3s[894]: I0923 04:42:50.034859 894 alloc.go:329] "allocated clusterIPs" service="default/kubernetes" clusterIPs={"IPv4":"10.43.0.1"} machine # [ 23.493868] k3s[894]: W0923 04:42:50.041812 894 lease.go:265] Resetting endpoints for master service "kubernetes" to [10.0.2.15] machine # [ 23.495964] k3s[894]: I0923 04:42:50.043914 894 controller.go:667] quota admission added evaluator for: endpoints machine # [ 23.501118] k3s[894]: I0923 04:42:50.049149 894 controller.go:667] quota admission added evaluator for: endpointslices.discovery.k8s.io machine # [ 24.765744] k3s[894]: time="2026-09-23T04:42:51Z" level=info msg="Kube API server is now running" machine # [ 24.768293] k3s[894]: time="2026-09-23T04:42:51Z" level=info msg="k3s is up and running" machine # [ 24.769705] k3s[894]: time="2026-09-23T04:42:51Z" level=info msg="Waiting for untainted node" machine # [ 24.772782] k3s[894]: time="2026-09-23T04:42:51Z" level=info msg="Creating k3s-supervisor event broadcaster" machine # [ 24.776303] systemd[1]: Started k3s service. machine # [ 24.778345] systemd[1]: Starting niks3 server... machine # [ 24.779866] k3s[894]: time="2026-09-23T04:42:51Z" level=info msg="Event occurred" apiVersion= fieldPath= kind=Node logger=k3s message="Node and Certificate Authority certificates managed by k3s are OK" object=machine reason=CertificateExpirationOK type=Normal machine # [ 24.790700] k3s[894]: I0923 04:42:51.338723 894 controllermanager.go:189] "Starting" version="v1.35.8+k3s1" machine # [ 24.791753] k3s[894]: I0923 04:42:51.338782 894 controllermanager.go:191] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" machine # [ 24.794953] k3s[894]: I0923 04:42:51.342986 894 secure_serving.go:211] Serving securely on 127.0.0.1:10257 machine # [ 24.796230] k3s[894]: I0923 04:42:51.343826 894 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/var/lib/rancher/k3s/server/tls/kube-controller-manager/kube-controller-manager.crt::/var/lib/rancher/k3s/server/tls/kube-controller-manager/kube-controller-manager.key" machine # [ 24.796860] k3s[894]: I0923 04:42:51.343993 894 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController machine # [ 24.797533] k3s[894]: I0923 04:42:51.344013 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 24.798504] k3s[894]: I0923 04:42:51.346283 894 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" machine # [ 24.799419] k3s[894]: I0923 04:42:51.346358 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 24.799846] k3s[894]: I0923 04:42:51.346398 894 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" machine # [ 24.800802] k3s[894]: I0923 04:42:51.346446 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 24.804304] k3s[894]: I0923 04:42:51.352337 894 tlsconfig.go:243] "Starting DynamicServingCertificateController" machine # [ 24.815358] k3s[894]: I0923 04:42:51.363351 894 controller_descriptor.go:99] "Controller is disabled by a feature gate" controller="device-taint-eviction-controller" requiredFeatureGates=["DynamicResourceAllocation","DRADeviceTaints"] machine # [ 24.815959] k3s[894]: I0923 04:42:51.363400 894 controllermanager.go:579] "Warning: skipping controller" controller="device-taint-eviction-controller" machine # [ 24.825887] k3s[894]: I0923 04:42:51.373657 894 controller.go:667] quota admission added evaluator for: serviceaccounts machine # [ 24.830083] k3s[894]: time="2026-09-23T04:42:51Z" level=info msg="Creating embedded CRD addons.k3s.cattle.io" machine # [ 24.830934] k3s[894]: I0923 04:42:51.378026 894 controllermanager.go:627] "Warning: controller is disabled" controller="selinux-warning-controller" machine # [ 24.836379] k3s[894]: time="2026-09-23T04:42:51Z" level=info msg="Creating embedded CRD etcdsnapshotfiles.k3s.cattle.io" machine # [ 24.846655] k3s[894]: time="2026-09-23T04:42:51Z" level=info msg="Creating embedded CRD helmchartconfigs.helm.cattle.io" machine # [ 24.853245] k3s[894]: time="2026-09-23T04:42:51Z" level=info msg="Creating embedded CRD helmcharts.helm.cattle.io" machine # [ 24.863156] k3s[894]: I0923 04:42:51.411201 894 controllermanager.go:627] "Warning: controller is disabled" controller="bootstrap-signer-controller" machine # [ 24.874150] k3s[894]: I0923 04:42:51.422154 894 handler.go:304] Adding GroupVersion k3s.cattle.io v1 to ResourceManager machine # [ 24.879278] k3s[894]: time="2026-09-23T04:42:51Z" level=info msg="Waiting for CRD etcdsnapshotfiles.k3s.cattle.io to become available" machine # [ 24.892098] k3s[894]: time="2026-09-23T04:42:51Z" level=info msg="Done waiting for CRD etcdsnapshotfiles.k3s.cattle.io to become available" machine # [ 24.892579] k3s[894]: time="2026-09-23T04:42:51Z" level=info msg="Waiting for CRD helmchartconfigs.helm.cattle.io to become available" machine # [ 24.907599] k3s[894]: I0923 04:42:51.455577 894 handler.go:304] Adding GroupVersion k3s.cattle.io v1 to ResourceManager machine # [ 24.919656] k3s[894]: I0923 04:42:51.467670 894 handler.go:304] Adding GroupVersion helm.cattle.io v1 to ResourceManager machine # [ 24.933908] k3s[894]: I0923 04:42:51.481926 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 24.938066] k3s[894]: I0923 04:42:51.485283 894 controllermanager.go:579] "Warning: skipping controller" controller="storage-version-migrator-controller" machine # [ 24.950075] k3s[894]: I0923 04:42:51.498050 894 handler.go:304] Adding GroupVersion helm.cattle.io v1 to ResourceManager machine # [ 24.996346] k3s[894]: I0923 04:42:51.544241 894 shared_informer.go:377] "Caches are synced" machine # [ 25.000072] k3s[894]: I0923 04:42:51.548027 894 shared_informer.go:377] "Caches are synced" machine # [ 25.000769] k3s[894]: I0923 04:42:51.548154 894 shared_informer.go:377] "Caches are synced" machine # [ 25.012621] k3s[894]: I0923 04:42:51.560623 894 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="csistoragecapacities.storage.k8s.io" machine # [ 25.016103] k3s[894]: I0923 04:42:51.563845 894 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="limitranges" machine # [ 25.016719] k3s[894]: I0923 04:42:51.563935 894 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="replicasets.apps" machine # [ 25.017505] k3s[894]: I0923 04:42:51.564184 894 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="controllerrevisions.apps" machine # [ 25.023677] k3s[894]: I0923 04:42:51.571640 894 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="rolebindings.rbac.authorization.k8s.io" machine # [ 25.024235] k3s[894]: I0923 04:42:51.571804 894 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="serviceaccounts" machine # [ 25.024684] k3s[894]: I0923 04:42:51.572440 894 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="jobs.batch" machine # [ 25.024989] k3s[894]: I0923 04:42:51.572484 894 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="ingresses.networking.k8s.io" machine # [ 25.025638] k3s[894]: I0923 04:42:51.572518 894 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="poddisruptionbudgets.policy" machine # [ 25.027483] k3s[894]: I0923 04:42:51.572963 894 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="roles.rbac.authorization.k8s.io" machine # [ 25.028440] k3s[894]: I0923 04:42:51.573033 894 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="leases.coordination.k8s.io" machine # [ 25.028885] k3s[894]: I0923 04:42:51.573259 894 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="deployments.apps" machine # [ 25.029609] k3s[894]: I0923 04:42:51.573315 894 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="cronjobs.batch" machine # [ 25.031764] k3s[894]: I0923 04:42:51.573362 894 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="endpointslices.discovery.k8s.io" machine # [ 25.032528] k3s[894]: I0923 04:42:51.573411 894 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="helmchartconfigs.helm.cattle.io" machine # [ 25.033096] k3s[894]: I0923 04:42:51.573513 894 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="resourceclaimtemplates.resource.k8s.io" machine # [ 25.033556] k3s[894]: I0923 04:42:51.573585 894 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="addons.k3s.cattle.io" machine # [ 25.034499] k3s[894]: I0923 04:42:51.573671 894 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="podtemplates" machine # [ 25.034982] k3s[894]: I0923 04:42:51.573714 894 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="endpoints" machine # [ 25.035679] k3s[894]: I0923 04:42:51.573743 894 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="helmcharts.helm.cattle.io" machine # [ 25.036599] k3s[894]: I0923 04:42:51.573841 894 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="statefulsets.apps" machine # [ 25.037204] k3s[894]: I0923 04:42:51.573892 894 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="daemonsets.apps" machine # [ 25.037682] k3s[894]: I0923 04:42:51.573922 894 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="horizontalpodautoscalers.autoscaling" machine # [ 25.038496] k3s[894]: I0923 04:42:51.573958 894 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="networkpolicies.networking.k8s.io" machine # [ 25.044431] k3s[894]: I0923 04:42:51.592468 894 controllermanager.go:627] "Warning: controller is disabled" controller="node-route-controller" machine: (finished: waiting for unit k3s.service, in 26.06 seconds) machine: waiting for success: kubectl get ns farm machine # [ 25.134240] k3s[894]: I0923 04:42:51.682231 894 shared_informer.go:377] "Caches are synced" machine # Error from server (NotFound): namespaces "farm" not found machine # [ 25.404223] k3s[894]: time="2026-09-23T04:42:51Z" level=info msg="Done waiting for CRD helmchartconfigs.helm.cattle.io to become available" machine # [ 25.404918] k3s[894]: time="2026-09-23T04:42:51Z" level=info msg="Waiting for CRD helmcharts.helm.cattle.io to become available" machine # [ 25.408910] k3s[894]: time="2026-09-23T04:42:51Z" level=info msg="Done waiting for CRD helmcharts.helm.cattle.io to become available" machine # [ 25.410837] k3s[894]: time="2026-09-23T04:42:51Z" level=info msg="Writing static file: /var/lib/rancher/k3s/server/static/charts/traefik-40.1.4+up40.1.0.tgz" machine # [ 25.412885] k3s[894]: time="2026-09-23T04:42:51Z" level=info msg="Writing static file: /var/lib/rancher/k3s/server/static/charts/traefik-crd-40.1.4+up40.1.0.tgz" machine # [ 25.414933] k3s[894]: time="2026-09-23T04:42:51Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/ccm.yaml" machine # [ 25.416647] k3s[894]: time="2026-09-23T04:42:51Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/coredns.yaml" machine # [ 25.418347] k3s[894]: time="2026-09-23T04:42:51Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/local-storage.yaml" machine # [ 25.420070] k3s[894]: time="2026-09-23T04:42:51Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/rolebindings.yaml" machine # [ 25.421739] k3s[894]: time="2026-09-23T04:42:51Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/runtimes.yaml" machine # [ 25.423337] k3s[894]: time="2026-09-23T04:42:51Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/traefik.yaml" machine # [ 25.448614] k3s[894]: time="2026-09-23T04:42:51Z" level=info msg="Imported docker.io/rancher/klipper-helm:v0.13.3-build20260727" machine # [ 25.454447] k3s[894]: time="2026-09-23T04:42:52Z" level=info msg="Tagged docker.io/rancher/klipper-helm@sha256:5425a2b4613d2458dbdff99218e8d487ebb59c20e7a941f061c0777070f0aac1" machine # [ 25.455177] k3s[894]: time="2026-09-23T04:42:52Z" level=info msg="Imported docker.io/rancher/klipper-lb:v0.4.17" machine # [ 25.459416] k3s[894]: time="2026-09-23T04:42:52Z" level=info msg="Tagged docker.io/rancher/klipper-lb@sha256:910944bb0bd94f060a82a56ca1ea1c577d3e49b3473a093a47a985f32e92d94a" machine # [ 25.460073] k3s[894]: time="2026-09-23T04:42:52Z" level=info msg="Imported docker.io/rancher/local-path-provisioner:v0.0.37" machine # [ 25.461263] k3s[894]: time="2026-09-23T04:42:52Z" level=info msg="Tagged docker.io/rancher/local-path-provisioner@sha256:e757967a5ec338f6a9b371c5a9688bedaa8c3578ea3dd4db329ea0084be0a86f" machine # [ 25.462181] k3s[894]: time="2026-09-23T04:42:52Z" level=info msg="Imported docker.io/rancher/mirrored-coredns-coredns:1.14.6" machine # [ 25.466129] k3s[894]: time="2026-09-23T04:42:52Z" level=info msg="Tagged docker.io/rancher/mirrored-coredns-coredns@sha256:900f9c109f7a33545d3c811516e8376df9019147b750f5ce3e254468769176ea" machine # [ 25.467416] k3s[894]: time="2026-09-23T04:42:52Z" level=info msg="Imported docker.io/rancher/mirrored-library-busybox:1.37.0" machine # [ 25.471084] k3s[894]: time="2026-09-23T04:42:52Z" level=info msg="Starting dynamiclistener CN filter node controller with SANs: [127.0.0.1 ::1 localhost machine 10.0.2.15 fec0::7b2b:a512:d19:a042 10.43.0.1 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local]" machine # [ 25.474376] k3s[894]: time="2026-09-23T04:42:52Z" level=info msg="Tunnel server egress proxy mode: agent" machine # [ 25.478116] k3s[894]: time="2026-09-23T04:42:52Z" level=info msg="Tagged docker.io/rancher/mirrored-library-busybox@sha256:101b4afd76732482eff9b95cae5f94bcf295e521fbec4e01b69c5421f3f3f3e5" machine # [ 25.480388] k3s[894]: time="2026-09-23T04:42:52Z" level=info msg="Imported docker.io/rancher/mirrored-library-traefik:3.7.8" machine # [ 25.482279] k3s[894]: time="2026-09-23T04:42:52Z" level=info msg="Tagged docker.io/rancher/mirrored-library-traefik@sha256:4299bbed850421258fc5448c2e0e6ad350981d4d335a68de11b92448aedbefe5" machine # [ 25.482830] k3s[894]: time="2026-09-23T04:42:52Z" level=info msg="Imported docker.io/rancher/mirrored-metrics-server:v0.9.0" machine # [ 25.488895] k3s[894]: time="2026-09-23T04:42:52Z" level=info msg="Tagged docker.io/rancher/mirrored-metrics-server@sha256:d9862115e7c7881280d3d75ca26bda8ffc0fc213315979575bf23ce9826205c0" machine # [ 25.490353] k3s[894]: time="2026-09-23T04:42:52Z" level=info msg="Imported docker.io/rancher/mirrored-pause:3.10.2" machine # [ 25.492817] k3s[894]: time="2026-09-23T04:42:52Z" level=info msg="Tagged docker.io/rancher/mirrored-pause@sha256:f548e0e8e3dc1896ca956272154dde3314e8cc4fde0a57577ee9fa1c63f5baf4" machine # [ 25.714341] k3s[894]: time="2026-09-23T04:42:52Z" level=info msg="Creating new TLS secret for kube-system/k3s-serving (count: 11): map[listener.cattle.io/cn-10.0.2.15:10.0.2.15 listener.cattle.io/cn-10.43.0.1:10.43.0.1 listener.cattle.io/cn-127.0.0.1:127.0.0.1 listener.cattle.io/cn-__1-f16284:::1 listener.cattle.io/cn-fec0__7b2b_a512_d19_a042-42b93d:fec0::7b2b:a512:d19:a042 listener.cattle.io/cn-kubernetes:kubernetes listener.cattle.io/cn-kubernetes.default:kubernetes.default listener.cattle.io/cn-kubernetes.default.svc:kubernetes.default.svc listener.cattle.io/cn-kubernetes.default.svc.cluster.local:kubernetes.default.svc.cluster.local listener.cattle.io/cn-localhost:localhost listener.cattle.io/cn-machine:machine listener.cattle.io/fingerprint:SHA1=887835569D857DEA06B5F8D58D35944C7C42FA6F]" machine # [ 25.726333] k3s[894]: time="2026-09-23T04:42:52Z" level=info msg="Active TLS secret kube-system/k3s-serving (ver=232) (count 11): map[listener.cattle.io/cn-10.0.2.15:10.0.2.15 listener.cattle.io/cn-10.43.0.1:10.43.0.1 listener.cattle.io/cn-127.0.0.1:127.0.0.1 listener.cattle.io/cn-__1-f16284:::1 listener.cattle.io/cn-fec0__7b2b_a512_d19_a042-42b93d:fec0::7b2b:a512:d19:a042 listener.cattle.io/cn-kubernetes:kubernetes listener.cattle.io/cn-kubernetes.default:kubernetes.default listener.cattle.io/cn-kubernetes.default.svc:kubernetes.default.svc listener.cattle.io/cn-kubernetes.default.svc.cluster.local:kubernetes.default.svc.cluster.local listener.cattle.io/cn-localhost:localhost listener.cattle.io/cn-machine:machine listener.cattle.io/fingerprint:SHA1=887835569D857DEA06B5F8D58D35944C7C42FA6F]" machine # [ 25.736301] k3s[894]: time="2026-09-23T04:42:52Z" level=info msg="Imported 8 images from /var/lib/rancher/k3s/agent/images/k3s-airgap-images-amd64.tar.zst in 3.856372248s" machine # [ 25.737293] k3s[894]: time="2026-09-23T04:42:52Z" level=info msg="Importing images from /var/lib/rancher/k3s/agent/images/nix-grpc-farm-client.tar" machine # [ 25.769605] k3s[894]: I0923 04:42:52.317318 894 node_lifecycle_controller.go:419] "Controller will reconcile labels" logger="node-lifecycle-controller" machine # Error from server (NotFound): namespaces "farm" not found machine # [ 26.660255] k3s[894]: I0923 04:42:53.208171 894 range_allocator.go:113] "No Secondary Service CIDR provided. Skipping filtering out secondary service addresses" logger="node-ipam-controller" machine # [ 26.680419] k3s[894]: time="2026-09-23T04:42:53Z" level=info msg="Starting k3s.cattle.io/v1, Kind=Addon controller" machine # [ 26.682120] k3s[894]: time="2026-09-23T04:42:53Z" level=info msg="Creating deploy event broadcaster" machine # [ 26.685727] k3s[894]: I0923 04:42:53.233704 894 controller.go:667] quota admission added evaluator for: addons.k3s.cattle.io machine # [ 26.691069] k3s[894]: time="2026-09-23T04:42:53Z" level=info msg="Creating helm-controller event broadcaster" machine # [ 26.691782] k3s[894]: time="2026-09-23T04:42:53Z" level=info msg="Starting /v1, Kind=Node controller" machine # [ 26.699251] k3s[894]: time="2026-09-23T04:42:53Z" level=info msg="Unable to set control-plane role label: nodes \"machine\" not found" machine # [ 26.700095] k3s[894]: time="2026-09-23T04:42:53Z" level=info msg="Event occurred" apiVersion=k3s.cattle.io/v1 fieldPath= kind=Addon logger=k3s/deploy message="Applying manifest at \"/var/lib/rancher/k3s/server/manifests/ccm.yaml\"" object=kube-system/ccm reason=ApplyingManifest type=Normal machine # [ 26.703121] k3s[894]: time="2026-09-23T04:42:53Z" level=info msg="Cluster dns configmap has been set successfully" machine # [ 27.179550] k3s[894]: time="2026-09-23T04:42:53Z" level=info msg="Event occurred" apiVersion=k3s.cattle.io/v1 fieldPath= kind=Addon logger=k3s/deploy message="Applied manifest at \"/var/lib/rancher/k3s/server/manifests/ccm.yaml\"" object=kube-system/ccm reason=AppliedManifest type=Normal machine # [ 27.197520] k3s[894]: time="2026-09-23T04:42:53Z" level=info msg="Event occurred" apiVersion=k3s.cattle.io/v1 fieldPath= kind=Addon logger=k3s/deploy message="Applying manifest at \"/var/lib/rancher/k3s/server/manifests/ci.yaml\"" object=kube-system/ci reason=ApplyingManifest type=Normal machine # [ 27.528877] k3s[894]: time="2026-09-23T04:42:54Z" level=info msg="Event occurred" apiVersion=k3s.cattle.io/v1 fieldPath= kind=Addon logger=k3s/deploy message="Applied manifest at \"/var/lib/rancher/k3s/server/manifests/ci.yaml\"" object=kube-system/ci reason=AppliedManifest type=Normal machine # [ 27.540472] k3s[894]: time="2026-09-23T04:42:54Z" level=info msg="Event occurred" apiVersion=k3s.cattle.io/v1 fieldPath= kind=Addon logger=k3s/deploy message="Applying manifest at \"/var/lib/rancher/k3s/server/manifests/coredns.yaml\"" object=kube-system/coredns reason=ApplyingManifest type=Normal machine: (finished: waiting for success: kubectl get ns farm, in 2.51 seconds) machine: must succeed: kubectl -n farm create secret tls farm-lb-tls --cert=/nix/store/b2prb199a55q32qawjagqrylwrml5g74-k3s-farm-certs/lb.crt --key=/nix/store/b2prb199a55q32qawjagqrylwrml5g74-k3s-farm-certs/lb.key machine # [ 27.694832] k3s[894]: time="2026-09-23T04:42:54Z" level=info msg="Unable to set control-plane role label: nodes \"machine\" not found" machine: (finished: must succeed: kubectl -n farm create secret tls farm-lb-tls --cert=/nix/store/b2prb199a55q32qawjagqrylwrml5g74-k3s-farm-certs/lb.crt --key=/nix/store/b2prb199a55q32qawjagqrylwrml5g74-k3s-farm-certs/lb.key, in 0.17 seconds) machine: must succeed: kubectl -n farm create secret tls farm-worker-tls --cert=/nix/store/b2prb199a55q32qawjagqrylwrml5g74-k3s-farm-certs/worker.crt --key=/nix/store/b2prb199a55q32qawjagqrylwrml5g74-k3s-farm-certs/worker.key machine # [ 27.869153] k3s[894]: I0923 04:42:54.416599 894 serving.go:392] Generated self-signed cert in-memory machine: (finished: must succeed: kubectl -n farm create secret tls farm-worker-tls --cert=/nix/store/b2prb199a55q32qawjagqrylwrml5g74-k3s-farm-certs/worker.crt --key=/nix/store/b2prb199a55q32qawjagqrylwrml5g74-k3s-farm-certs/worker.key, in 0.14 seconds) machine: must succeed: kubectl -n farm create secret generic farm-ca --from-file=ca.crt=/nix/store/b2prb199a55q32qawjagqrylwrml5g74-k3s-farm-certs/ca.crt machine # [ 27.961575] k3s[894]: I0923 04:42:54.509568 894 serving.go:392] Generated self-signed cert in-memory machine # [ 28.007558] k3s[894]: time="2026-09-23T04:42:54Z" level=info msg="Starting rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding controller" machine # [ 28.013239] k3s[894]: time="2026-09-23T04:42:54Z" level=info msg="Starting batch/v1, Kind=Job controller" machine # [ 28.013838] k3s[894]: time="2026-09-23T04:42:54Z" level=info msg="Starting /v1, Kind=ConfigMap controller" machine # [ 28.015050] k3s[894]: time="2026-09-23T04:42:54Z" level=info msg="Starting /v1, Kind=ServiceAccount controller" machine # [ 28.016047] k3s[894]: time="2026-09-23T04:42:54Z" level=info msg="Starting /v1, Kind=Secret controller" machine # [ 28.022087] k3s[894]: time="2026-09-23T04:42:54Z" level=info msg="Starting helm.cattle.io/v1, Kind=HelmChart controller" machine # [ 28.022643] k3s[894]: time="2026-09-23T04:42:54Z" level=info msg="Starting helm.cattle.io/v1, Kind=HelmChartConfig controller" machine # [ 28.082745] k3s[894]: I0923 04:42:54.630683 894 controllermanager.go:160] Version: v1.35.8+k3s1 machine # [ 28.085795] k3s[894]: I0923 04:42:54.633756 894 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController machine # [ 28.086410] k3s[894]: I0923 04:42:54.633765 894 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" machine # [ 28.087802] k3s[894]: I0923 04:42:54.633783 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 28.088621] k3s[894]: I0923 04:42:54.633801 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 28.089176] k3s[894]: I0923 04:42:54.633836 894 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" machine # [ 28.089617] k3s[894]: I0923 04:42:54.633854 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 28.090232] k3s[894]: I0923 04:42:54.634418 894 secure_serving.go:211] Serving securely on 127.0.0.1:10258 machine # [ 28.090651] k3s[894]: I0923 04:42:54.637300 894 tlsconfig.go:243] "Starting DynamicServingCertificateController" machine # [ 28.095691] k3s[894]: time="2026-09-23T04:42:54Z" level=info msg="Creating service-lb-controller event broadcaster" machine # [ 28.110986] k3s[894]: I0923 04:42:54.658991 894 controller.go:667] quota admission added evaluator for: deployments.apps machine: (finished: must succeed: kubectl -n farm create secret generic farm-ca --from-file=ca.crt=/nix/store/b2prb199a55q32qawjagqrylwrml5g74-k3s-farm-certs/ca.crt, in 0.16 seconds) subtest: chart deploys and workers become Ready machine: waiting for success: kubectl -n farm rollout status deployment nix-grpc-farm-lb --timeout=10s machine # [ 28.187130] k3s[894]: I0923 04:42:54.734707 894 shared_informer.go:377] "Caches are synced" machine # [ 28.187641] k3s[894]: I0923 04:42:54.734866 894 shared_informer.go:377] "Caches are synced" machine # [ 28.188073] k3s[894]: I0923 04:42:54.734981 894 shared_informer.go:377] "Caches are synced" machine # [ 28.230198] k3s[894]: I0923 04:42:54.778186 894 alloc.go:329] "allocated clusterIPs" service="kube-system/kube-dns" clusterIPs={"IPv4":"10.43.0.10"} machine # [ 28.233174] k3s[894]: time="2026-09-23T04:42:54Z" level=info msg="Event occurred" apiVersion=k3s.cattle.io/v1 fieldPath= kind=Addon logger=k3s/deploy message="Applied manifest at \"/var/lib/rancher/k3s/server/manifests/coredns.yaml\"" object=kube-system/coredns reason=AppliedManifest type=Normal machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-lb" not found machine # [ 28.432762] k3s[894]: time="2026-09-23T04:42:54Z" level=info msg="Starting /v1, Kind=Node controller" machine # [ 28.438515] k3s[894]: time="2026-09-23T04:42:54Z" level=info msg="Starting /v1, Kind=Pod controller" machine # [ 28.444062] k3s[894]: time="2026-09-23T04:42:54Z" level=info msg="Starting apps/v1, Kind=DaemonSet controller" machine # [ 28.448915] k3s[894]: time="2026-09-23T04:42:54Z" level=info msg="Starting discovery.k8s.io/v1, Kind=EndpointSlice controller" machine # [ 28.449694] k3s[894]: I0923 04:42:54.996446 894 controllermanager.go:329] Started "cloud-node-controller" machine # [ 28.450166] k3s[894]: I0923 04:42:54.996556 894 node_controller.go:176] Sending events to api server. machine # [ 28.450590] k3s[894]: I0923 04:42:54.996564 894 controllermanager.go:329] Started "cloud-node-lifecycle-controller" machine # [ 28.452519] k3s[894]: I0923 04:42:54.996637 894 node_lifecycle_controller.go:112] Sending events to api server machine # [ 28.453174] k3s[894]: I0923 04:42:54.996662 894 node_controller.go:185] Waiting for informer caches to sync machine # [ 28.453479] k3s[894]: I0923 04:42:54.996905 894 controllermanager.go:329] Started "service-lb-controller" machine # [ 28.454448] k3s[894]: W0923 04:42:54.996918 894 controllermanager.go:306] "node-route-controller" is disabled machine # [ 28.454906] k3s[894]: I0923 04:42:54.997402 894 controller.go:235] Starting service controller machine # [ 28.455522] k3s[894]: I0923 04:42:54.997433 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 28.695896] k3s[894]: time="2026-09-23T04:42:55Z" level=info msg="Unable to set control-plane role label: nodes \"machine\" not found" machine # [ 28.753498] k3s[894]: I0923 04:42:55.301167 894 shared_informer.go:377] "Caches are synced" machine # [ 29.070899] k3s[894]: I0923 04:42:55.618698 894 controllermanager.go:627] "Warning: controller is disabled" controller="cloud-node-lifecycle-controller" machine # [ 29.072105] k3s[894]: I0923 04:42:55.618734 894 controller_descriptor.go:99] "Controller is disabled by a feature gate" controller="kube-apiserver-serving-clustertrustbundle-publisher-controller" requiredFeatureGates=["ClusterTrustBundle"] machine # [ 29.072870] k3s[894]: I0923 04:42:55.618757 894 controllermanager.go:579] "Warning: skipping controller" controller="kube-apiserver-serving-clustertrustbundle-publisher-controller" machine # [ 29.073718] k3s[894]: I0923 04:42:55.618771 894 controller_descriptor.go:99] "Controller is disabled by a feature gate" controller="storageversion-garbage-collector-controller" requiredFeatureGates=["APIServerIdentity","StorageVersionAPI"] machine # [ 29.074608] k3s[894]: I0923 04:42:55.618783 894 controllermanager.go:579] "Warning: skipping controller" controller="storageversion-garbage-collector-controller" machine # [ 29.075498] k3s[894]: I0923 04:42:55.618797 894 controller_descriptor.go:99] "Controller is disabled by a feature gate" controller="podcertificaterequest-cleaner-controller" requiredFeatureGates=["PodCertificateRequest"] machine # [ 29.076539] k3s[894]: I0923 04:42:55.618807 894 controllermanager.go:579] "Warning: skipping controller" controller="podcertificaterequest-cleaner-controller" machine # [ 29.077319] k3s[894]: I0923 04:42:55.618819 894 controllermanager.go:627] "Warning: controller is disabled" controller="service-lb-controller" machine # [ 29.088414] k3s[894]: time="2026-09-23T04:42:55Z" level=info msg="Event occurred" apiVersion=k3s.cattle.io/v1 fieldPath= kind=Addon logger=k3s/deploy message="Deleting manifest at \"/var/lib/rancher/k3s/server/manifests/local-storage.yaml\"" object=kube-system/local-storage reason=DeletingManifest type=Normal machine # [ 29.101513] k3s[894]: time="2026-09-23T04:42:55Z" level=info msg="Event occurred" apiVersion=k3s.cattle.io/v1 fieldPath= kind=Addon logger=k3s/deploy message="Applying manifest at \"/var/lib/rancher/k3s/server/manifests/nix-grpc-farm.yaml\"" object=kube-system/nix-grpc-farm reason=ApplyingManifest type=Normal machine # [ 29.222308] k3s[894]: W0923 04:42:55.769840 894 probe.go:272] Flexvolume plugin directory at /usr/libexec/kubernetes/kubelet-plugins/volume/exec/ does not exist. Recreating. machine # [ 29.223747] k3s[894]: I0923 04:42:55.771761 894 controller.go:667] quota admission added evaluator for: helmcharts.helm.cattle.io machine # [ 29.228628] k3s[894]: time="2026-09-23T04:42:55Z" level=info msg="Event occurred" apiVersion=k3s.cattle.io/v1 fieldPath= kind=Addon logger=k3s/deploy message="Applied manifest at \"/var/lib/rancher/k3s/server/manifests/nix-grpc-farm.yaml\"" object=kube-system/nix-grpc-farm reason=AppliedManifest type=Normal machine # [ 29.239876] k3s[894]: time="2026-09-23T04:42:55Z" level=info msg="Event occurred" apiVersion=k3s.cattle.io/v1 fieldPath= kind=Addon logger=k3s/deploy message="Applying manifest at \"/var/lib/rancher/k3s/server/manifests/rolebindings.yaml\"" object=kube-system/rolebindings reason=ApplyingManifest type=Normal machine # [ 29.373617] k3s[894]: I0923 04:42:55.921499 894 vac_protection_controller.go:206] "Starting VAC protection controller" machine # [ 29.374116] k3s[894]: I0923 04:42:55.921534 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.382099] k3s[894]: time="2026-09-23T04:42:55Z" level=info msg="Event occurred" apiVersion=helm.cattle.io/v1 fieldPath= kind=HelmChart logger=k3s/helm-controller message="Applying HelmChart from https://%{KUBERNETES_API}%/static/charts/nix-grpc-farm.tgz using Job kube-system/helm-install-nix-grpc-farm " object=kube-system/nix-grpc-farm reason=ApplyJob type=Normal machine # [ 29.382795] k3s[894]: I0923 04:42:55.928519 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.383478] k3s[894]: I0923 04:42:55.928615 894 taint_eviction.go:283] "Starting" controller="taint-eviction-controller" machine # [ 29.383953] k3s[894]: I0923 04:42:55.928713 894 taint_eviction.go:288] "Sending events to API server" machine # [ 29.384782] k3s[894]: I0923 04:42:55.928724 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.385477] k3s[894]: I0923 04:42:55.928869 894 endpointslice_controller.go:283] "Starting endpoint slice controller" machine # [ 29.385981] k3s[894]: I0923 04:42:55.928882 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.386661] k3s[894]: I0923 04:42:55.928946 894 serviceaccounts_controller.go:117] "Starting service account controller" machine # [ 29.390114] k3s[894]: I0923 04:42:55.928959 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.390764] k3s[894]: I0923 04:42:55.929072 894 pv_controller_base.go:307] "Starting persistent volume controller" machine # [ 29.391430] k3s[894]: I0923 04:42:55.929086 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.391883] k3s[894]: I0923 04:42:55.929164 894 expand_controller.go:328] "Starting expand controller" machine # [ 29.392598] k3s[894]: I0923 04:42:55.929176 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.393183] k3s[894]: I0923 04:42:55.929272 894 servicecidrs_controller.go:136] "Starting" controller="service-cidr-controller" machine # [ 29.393787] k3s[894]: I0923 04:42:55.929287 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.394677] k3s[894]: I0923 04:42:55.929418 894 endpoints_controller.go:193] "Starting endpoint controller" machine # [ 29.395419] k3s[894]: I0923 04:42:55.929431 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.398101] k3s[894]: I0923 04:42:55.929826 894 stateful_set.go:180] "Starting stateful set controller" machine # [ 29.398669] k3s[894]: I0923 04:42:55.929840 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.399488] k3s[894]: I0923 04:42:55.929977 894 controller.go:423] "Starting resource claim controller" machine # [ 29.399986] k3s[894]: I0923 04:42:55.929990 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.400675] k3s[894]: I0923 04:42:55.930025 894 disruption.go:458] "Sending events to api server." machine # [ 29.400990] k3s[894]: I0923 04:42:55.930132 894 disruption.go:465] "Starting disruption controller" machine # [ 29.401649] k3s[894]: I0923 04:42:55.930142 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.402316] k3s[894]: I0923 04:42:55.930178 894 pvc_protection_controller.go:166] "Starting PVC protection controller" machine # [ 29.402783] k3s[894]: I0923 04:42:55.930192 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.403522] k3s[894]: I0923 04:42:55.930258 894 namespace_controller.go:202] "Starting namespace controller" machine # [ 29.403982] k3s[894]: I0923 04:42:55.930273 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.404617] k3s[894]: I0923 04:42:55.930418 894 job_controller.go:254] "Starting job controller" machine # [ 29.405193] k3s[894]: I0923 04:42:55.930431 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.405684] k3s[894]: I0923 04:42:55.930519 894 node_lifecycle_controller.go:453] "Sending events to api server" machine # [ 29.409098] k3s[894]: I0923 04:42:55.930545 894 node_lifecycle_controller.go:460] "Starting node controller" machine # [ 29.409724] k3s[894]: I0923 04:42:55.930556 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.410437] k3s[894]: I0923 04:42:55.930619 894 clusterroleaggregation_controller.go:194] "Starting ClusterRoleAggregator controller" machine # [ 29.410899] k3s[894]: I0923 04:42:55.930636 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.411562] k3s[894]: I0923 04:42:55.930703 894 pv_protection_controller.go:81] "Starting PV protection controller" machine # [ 29.412090] k3s[894]: I0923 04:42:55.930717 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.412545] k3s[894]: I0923 04:42:55.930754 894 publisher.go:107] "Starting root CA cert publisher controller" machine # [ 29.413101] k3s[894]: I0923 04:42:55.930767 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.413559] k3s[894]: I0923 04:42:55.930836 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.414102] k3s[894]: I0923 04:42:55.930995 894 node_ipam_controller.go:142] "Starting ipam controller" machine # [ 29.414557] k3s[894]: I0923 04:42:55.931008 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.415116] k3s[894]: I0923 04:42:55.931079 894 replica_set.go:241] "Starting controller" name="replicaset" machine # [ 29.415600] k3s[894]: I0923 04:42:55.931112 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.416117] k3s[894]: I0923 04:42:55.931209 894 endpointslicemirroring_controller.go:226] "Starting EndpointSliceMirroring controller" machine # [ 29.416586] k3s[894]: I0923 04:42:55.931222 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.417121] k3s[894]: I0923 04:42:55.931284 894 deployment_controller.go:172] "Starting controller" controller="deployment" machine # [ 29.417632] k3s[894]: I0923 04:42:55.931297 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.418171] k3s[894]: I0923 04:42:55.931368 894 cronjob_controllerv2.go:143] "Starting cronjob controller v2" machine # [ 29.418636] k3s[894]: I0923 04:42:55.931381 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.419199] k3s[894]: I0923 04:42:55.931437 894 certificate_controller.go:120] "Starting certificate controller" name="csrsigning-legacy-unknown" machine # [ 29.419675] k3s[894]: I0923 04:42:55.931451 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.420377] k3s[894]: I0923 04:42:55.931482 894 cleaner.go:83] "Starting CSR cleaner controller" machine # [ 29.420821] k3s[894]: I0923 04:42:55.931538 894 ttl_controller.go:127] "Starting TTL controller" machine # [ 29.421465] k3s[894]: I0923 04:42:55.931550 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.422063] k3s[894]: I0923 04:42:55.931631 894 horizontal.go:204] "Starting HPA controller" machine # [ 29.422542] k3s[894]: I0923 04:42:55.931659 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.423100] k3s[894]: I0923 04:42:55.931690 894 certificate_controller.go:120] "Starting certificate controller" name="csrapproving" machine # [ 29.423577] k3s[894]: I0923 04:42:55.931704 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.424103] k3s[894]: I0923 04:42:55.931738 894 ttlafterfinished_controller.go:112] "Starting TTL after finished controller" machine # [ 29.424577] k3s[894]: I0923 04:42:55.931794 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.425105] k3s[894]: I0923 04:42:55.931879 894 controller.go:174] "Starting ephemeral volume controller" machine # [ 29.425561] k3s[894]: I0923 04:42:55.931892 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.426117] k3s[894]: I0923 04:42:55.931995 894 legacy_serviceaccount_token_cleaner.go:103] "Starting legacy service account token cleaner controller" machine # [ 29.426559] k3s[894]: I0923 04:42:55.932008 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.427076] k3s[894]: I0923 04:42:55.932163 894 replica_set.go:241] "Starting controller" name="replicationcontroller" machine # [ 29.427560] k3s[894]: I0923 04:42:55.932177 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.428141] k3s[894]: I0923 04:42:55.932239 894 gc_controller.go:98] "Starting GC controller" machine # [ 29.428584] k3s[894]: I0923 04:42:55.932251 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.429156] k3s[894]: I0923 04:42:55.932577 894 daemon_controller.go:309] "Starting daemon sets controller" machine # [ 29.429600] k3s[894]: I0923 04:42:55.932591 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.430183] k3s[894]: I0923 04:42:55.932628 894 tokencleaner.go:117] "Starting token cleaner controller" machine # [ 29.430652] k3s[894]: I0923 04:42:55.932649 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.431203] k3s[894]: I0923 04:42:55.932720 894 attach_detach_controller.go:335] "Starting attach detach controller" machine # [ 29.431951] k3s[894]: I0923 04:42:55.932733 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.433362] k3s[894]: I0923 04:42:55.933352 894 resource_quota_controller.go:297] "Starting resource quota controller" machine # [ 29.434609] k3s[894]: I0923 04:42:55.933367 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.435617] k3s[894]: I0923 04:42:55.933495 894 certificate_controller.go:120] "Starting certificate controller" name="csrsigning-kubelet-serving" machine # [ 29.437146] k3s[894]: I0923 04:42:55.933509 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.438516] k3s[894]: I0923 04:42:55.933546 894 certificate_controller.go:120] "Starting certificate controller" name="csrsigning-kubelet-client" machine # [ 29.440039] k3s[894]: I0923 04:42:55.933558 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.441062] k3s[894]: I0923 04:42:55.933590 894 certificate_controller.go:120] "Starting certificate controller" name="csrsigning-kube-apiserver-client" machine # [ 29.442826] k3s[894]: I0923 04:42:55.933603 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.443814] k3s[894]: I0923 04:42:55.933639 894 dynamic_serving_content.go:135] "Starting controller" name="csr-controller::/var/lib/rancher/k3s/server/tls/server-ca.nochain.crt::/var/lib/rancher/k3s/server/tls/server-ca.key" machine # [ 29.446416] k3s[894]: I0923 04:42:55.933925 894 garbagecollector.go:141] "Starting controller" controller="garbagecollector" machine # [ 29.447490] k3s[894]: I0923 04:42:55.934011 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.448554] k3s[894]: I0923 04:42:55.934760 894 resource_quota_monitor.go:309] "QuotaMonitor running" machine # [ 29.449637] k3s[894]: I0923 04:42:55.934886 894 dynamic_serving_content.go:135] "Starting controller" name="csr-controller::/var/lib/rancher/k3s/server/tls/server-ca.nochain.crt::/var/lib/rancher/k3s/server/tls/server-ca.key" machine # [ 29.452241] k3s[894]: I0923 04:42:55.934956 894 dynamic_serving_content.go:135] "Starting controller" name="csr-controller::/var/lib/rancher/k3s/server/tls/client-ca.nochain.crt::/var/lib/rancher/k3s/server/tls/client-ca.key" machine # [ 29.495528] k3s[894]: I0923 04:42:55.935053 894 dynamic_serving_content.go:135] "Starting controller" name="csr-controller::/var/lib/rancher/k3s/server/tls/client-ca.nochain.crt::/var/lib/rancher/k3s/server/tls/client-ca.key" machine # [ 29.496466] k3s[894]: I0923 04:42:55.935328 894 graph_builder.go:386] "Running" component="GraphBuilder" machine # [ 29.496921] k3s[894]: I0923 04:42:55.942500 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.497656] k3s[894]: I0923 04:42:56.018481 894 controller.go:667] quota admission added evaluator for: jobs.batch machine # [ 29.498249] k3s[894]: I0923 04:42:56.030228 894 shared_informer.go:377] "Caches are synced" machine # [ 29.498352] k3s[894]: I0923 04:42:56.030700 894 shared_informer.go:377] "Caches are synced" machine # [ 29.498938] k3s[894]: I0923 04:42:56.030755 894 shared_informer.go:377] "Caches are synced" machine # [ 29.499712] k3s[894]: I0923 04:42:56.031541 894 shared_informer.go:377] "Caches are synced" machine # [ 29.500510] k3s[894]: I0923 04:42:56.031759 894 shared_informer.go:377] "Caches are synced" machine # [ 29.501705] k3s[894]: I0923 04:42:56.031855 894 shared_informer.go:377] "Caches are synced" machine # [ 29.502592] k3s[894]: I0923 04:42:56.032469 894 shared_informer.go:377] "Caches are synced" machine # [ 29.502994] k3s[894]: I0923 04:42:56.033563 894 shared_informer.go:377] "Caches are synced" machine # [ 29.505463] k3s[894]: I0923 04:42:56.033628 894 shared_informer.go:377] "Caches are synced" machine # [ 29.506418] k3s[894]: I0923 04:42:56.033688 894 shared_informer.go:377] "Caches are synced" machine # [ 29.507032] k3s[894]: I0923 04:42:56.038914 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.582330] k3s[894]: I0923 04:42:56.130303 894 shared_informer.go:377] "Caches are synced" machine # [ 29.583033] k3s[894]: I0923 04:42:56.130422 894 shared_informer.go:377] "Caches are synced" machine # [ 29.593359] k3s[894]: I0923 04:42:56.141359 894 shared_informer.go:377] "Caches are synced" machine # [ 29.594064] k3s[894]: I0923 04:42:56.141450 894 range_allocator.go:177] "Sending events to api server" machine # [ 29.594554] k3s[894]: I0923 04:42:56.141496 894 range_allocator.go:181] "Starting range CIDR allocator" machine # [ 29.594974] k3s[894]: I0923 04:42:56.141507 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.595591] k3s[894]: I0923 04:42:56.141519 894 shared_informer.go:377] "Caches are synced" machine # [ 29.596118] k3s[894]: I0923 04:42:56.141656 894 shared_informer.go:377] "Caches are synced" machine # [ 29.596566] k3s[894]: I0923 04:42:56.141762 894 shared_informer.go:377] "Caches are synced" machine # [ 29.619314] k3s[894]: time="2026-09-23T04:42:56Z" level=info msg="Event occurred" apiVersion=k3s.cattle.io/v1 fieldPath= kind=Addon logger=k3s/deploy message="Applied manifest at \"/var/lib/rancher/k3s/server/manifests/rolebindings.yaml\"" object=kube-system/rolebindings reason=AppliedManifest type=Normal machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-lb" not found machine # [ 29.634126] k3s[894]: time="2026-09-23T04:42:56Z" level=info msg="Event occurred" apiVersion=k3s.cattle.io/v1 fieldPath= kind=Addon logger=k3s/deploy message="Applying manifest at \"/var/lib/rancher/k3s/server/manifests/runtimes.yaml\"" object=kube-system/runtimes reason=ApplyingManifest type=Normal machine # [ 29.674064] k3s[894]: I0923 04:42:56.221978 894 shared_informer.go:377] "Caches are synced" machine # [ 29.677854] k3s[894]: I0923 04:42:56.224357 894 cidrallocator.go:278] updated ClusterIP allocator for Service CIDR 10.43.0.0/16 machine # [ 29.682065] k3s[894]: I0923 04:42:56.229224 894 shared_informer.go:377] "Caches are synced" machine # [ 29.683349] k3s[894]: I0923 04:42:56.229407 894 shared_informer.go:377] "Caches are synced" machine # [ 29.684538] k3s[894]: I0923 04:42:56.229453 894 shared_informer.go:377] "Caches are synced" machine # [ 29.685699] k3s[894]: I0923 04:42:56.229458 894 shared_informer.go:377] "Caches are synced" machine # [ 29.686904] k3s[894]: I0923 04:42:56.229867 894 shared_informer.go:377] "Caches are synced" machine # [ 29.688098] k3s[894]: I0923 04:42:56.230112 894 shared_informer.go:377] "Caches are synced" machine # [ 29.689275] k3s[894]: I0923 04:42:56.230291 894 shared_informer.go:377] "Caches are synced" machine # [ 29.690465] k3s[894]: I0923 04:42:56.231246 894 shared_informer.go:377] "Caches are synced" machine # [ 29.691627] k3s[894]: I0923 04:42:56.231307 894 shared_informer.go:377] "Caches are synced" machine # [ 29.692795] k3s[894]: I0923 04:42:56.231561 894 cidrallocator.go:278] updated ClusterIP allocator for Service CIDR 10.43.0.0/16 machine # [ 29.694338] k3s[894]: I0923 04:42:56.231910 894 shared_informer.go:377] "Caches are synced" machine # [ 29.695462] k3s[894]: I0923 04:42:56.231952 894 shared_informer.go:377] "Caches are synced" machine # [ 29.696624] k3s[894]: I0923 04:42:56.232004 894 shared_informer.go:377] "Caches are synced" machine # [ 29.697824] k3s[894]: I0923 04:42:56.232055 894 shared_informer.go:377] "Caches are synced" machine # [ 29.698964] k3s[894]: I0923 04:42:56.232156 894 shared_informer.go:377] "Caches are synced" machine # [ 29.700142] k3s[894]: I0923 04:42:56.232204 894 shared_informer.go:377] "Caches are synced" machine # [ 29.701268] k3s[894]: I0923 04:42:56.232377 894 shared_informer.go:377] "Caches are synced" machine # [ 29.702462] k3s[894]: I0923 04:42:56.233028 894 shared_informer.go:377] "Caches are synced" machine # [ 29.703592] k3s[894]: I0923 04:42:56.233271 894 shared_informer.go:377] "Caches are synced" machine # [ 29.705910] k3s[894]: I0923 04:42:56.234188 894 shared_informer.go:377] "Caches are synced" machine # [ 29.706504] k3s[894]: I0923 04:42:56.234267 894 shared_informer.go:377] "Caches are synced" machine # [ 29.707081] k3s[894]: I0923 04:42:56.234342 894 shared_informer.go:377] "Caches are synced" machine # [ 29.707535] k3s[894]: I0923 04:42:56.234376 894 shared_informer.go:377] "Caches are synced" machine # [ 29.708115] k3s[894]: I0923 04:42:56.234568 894 shared_informer.go:377] "Caches are synced" machine # [ 29.708540] k3s[894]: I0923 04:42:56.234637 894 shared_informer.go:377] "Caches are synced" machine # [ 29.709093] k3s[894]: I0923 04:42:56.234711 894 shared_informer.go:377] "Caches are synced" machine # [ 29.709772] k3s[894]: time="2026-09-23T04:42:56Z" level=info msg="Unable to set control-plane role label: nodes \"machine\" not found" machine # [ 29.710407] k3s[894]: I0923 04:42:56.242969 894 shared_informer.go:377] "Caches are synced" machine # [ 29.723926] k3s[894]: time="2026-09-23T04:42:56Z" level=info msg="Event occurred" apiVersion=k3s.cattle.io/v1 fieldPath= kind=Addon logger=k3s/deploy message="Applied manifest at \"/var/lib/rancher/k3s/server/manifests/runtimes.yaml\"" object=kube-system/runtimes reason=AppliedManifest type=Normal machine # [ 29.995288] k3s[894]: time="2026-09-23T04:42:56Z" level=info msg="Event occurred" apiVersion= fieldPath= kind=Node logger=k3s message="Deferred node password secret validation complete" object=machine reason=NodePasswordValidationComplete type=Normal machine # [ 29.998295] k3s[894]: time="2026-09-23T04:42:56Z" level=info msg="Event occurred" apiVersion=k3s.cattle.io/v1 fieldPath= kind=Addon logger=k3s/deploy message="Deleting manifest at \"/var/lib/rancher/k3s/server/manifests/traefik.yaml\"" object=kube-system/traefik reason=DeletingManifest type=Normal machine # [ 30.291344] k3s[894]: time="2026-09-23T04:42:56Z" level=info msg="Event occurred" apiVersion=helm.cattle.io/v1 fieldPath= kind=HelmChart logger=k3s/helm-controller message="Resumed synced Job kube-system/helm-install-nix-grpc-farm" object=kube-system/nix-grpc-farm reason=ResumeJob type=Normal machine # [ 30.324438] k3s[894]: I0923 04:42:56.872420 894 controller.go:667] quota admission added evaluator for: replicasets.apps machine # [ 30.687113] k3s[894]: I0923 04:42:57.234553 894 shared_informer.go:377] "Caches are synced" machine # [ 30.688198] k3s[894]: I0923 04:42:57.234586 894 garbagecollector.go:166] "Garbage collector: all resource monitors have synced" machine # [ 30.688682] k3s[894]: I0923 04:42:57.234597 894 garbagecollector.go:169] "Proceeding to collect garbage" machine # [ 30.691424] k3s[894]: I0923 04:42:57.239447 894 shared_informer.go:377] "Caches are synced" machine # [ 30.694432] k3s[894]: time="2026-09-23T04:42:57Z" level=info msg="Unable to set control-plane role label: nodes \"machine\" not found" machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-lb" not found machine # [ 31.534837] postgres[1204]: [1204] ERROR: relation "goose_db_version" does not exist at character 36 machine # [ 31.536229] postgres[1204]: [1204] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC machine # [ 31.558494] niks3-server[1195]: 2026/09/23 04:42:58 OK 20241026095416_initial_model.sql (12.92ms) machine # [ 31.564330] niks3-server[1195]: 2026/09/23 04:42:58 OK 20251210153512_drop_unused_gin_index.sql (3.65ms) machine # [ 31.568831] niks3-server[1195]: 2026/09/23 04:42:58 OK 20251218171726_add_pins.sql (4.44ms) machine # [ 31.573980] niks3-server[1195]: 2026/09/23 04:42:58 OK 20260628120000_add_object_size_and_stats.sql (4.95ms) machine # [ 31.577593] niks3-server[1195]: 2026/09/23 04:42:58 OK 20260905000000_add_claims.sql (3.65ms) machine # [ 31.581913] niks3-server[1195]: 2026/09/23 04:42:58 OK 20260920000000_drop_claims.sql (4.27ms) machine # [ 31.582796] niks3-server[1195]: 2026/09/23 04:42:58 goose: successfully migrated database to version: 20260920000000 machine # [ 31.588464] niks3-server[1195]: 2026/09/23 04:42:58 OK 1_commit_pending_closure.sql (3.92ms) machine # [ 31.591413] niks3-server[1195]: 2026/09/23 04:42:58 OK 2_object_stats_trigger.sql (2.91ms) machine # [ 31.592112] niks3-server[1195]: 2026/09/23 04:42:58 goose: up to current file version: 2 machine # [ 31.598096] niks3-server[1195]: 2026/09/23 04:42:58 INFO OIDC provider initialized name=kubernetes issuer=https://kubernetes.default.svc.cluster.local machine # [ 31.598745] niks3-server[1195]: 2026/09/23 04:42:58 INFO OIDC authentication enabled config=/nix/store/v0b0sqyzhrjsz48x3klf7b7vl89apzs0-niks3-oidc.json machine # [ 31.601869] niks3-server[1195]: 2026/09/23 04:42:58 INFO Loaded signing key name=farm-test-1 path=/nix/store/hk0nhypl37jvkn3gr36v3y7qx593p39i-key machine # [ 31.629339] niks3-server[1195]: 2026/09/23 04:42:58 INFO Read proxy enabled — serving cache objects from S3 machine # [ 31.630639] niks3-server[1195]: 2026/09/23 04:42:58 INFO Using socket-activated listener address=0.0.0.0:5751 machine # [ 31.631853] niks3-server[1195]: 2026/09/23 04:42:58 INFO Starting HTTP server address=0.0.0.0:5751 machine # [ 31.633146] systemd[1]: Started niks3 server. machine # [ 31.633711] niks3-server[1195]: 2026/09/23 04:42:58 INFO systemd watchdog enabled interval=15s machine # [ 31.634264] systemd[1]: Reached target Multi-User System. machine # [ 31.634751] systemd[1]: Startup finished in 1.168s (kernel) + 4.135s (initrd) + 26.327s (userspace) = 31.631s. machine # [ 31.693906] k3s[894]: time="2026-09-23T04:42:58Z" level=info msg="Unable to set control-plane role label: nodes \"machine\" not found" machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-lb" not found machine # [ 32.694200] k3s[894]: time="2026-09-23T04:42:59Z" level=info msg="Unable to set control-plane role label: nodes \"machine\" not found" machine # [ 32.995517] k3s[894]: time="2026-09-23T04:42:59Z" level=info msg="Started tunnel to 10.0.2.15:6443" machine # [ 32.998967] k3s[894]: time="2026-09-23T04:42:59Z" level=info msg="Stopped tunnel to 127.0.0.1:6443" machine # [ 33.002164] k3s[894]: time="2026-09-23T04:42:59Z" level=info msg="Connecting to proxy" url="wss://10.0.2.15:6443/v1-k3s/connect" machine # [ 33.005822] k3s[894]: time="2026-09-23T04:42:59Z" level=info msg="Proxy done" err="context canceled" url="wss://127.0.0.1:6443/v1-k3s/connect" machine # [ 33.009834] k3s[894]: time="2026-09-23T04:42:59Z" level=info msg="error in remotedialer server [400]: websocket: close 1006 (abnormal closure): unexpected EOF" machine # [ 33.014307] k3s[894]: time="2026-09-23T04:42:59Z" level=info msg="Handling backend connection request [machine]" machine # [ 33.017475] k3s[894]: time="2026-09-23T04:42:59Z" level=info msg="Connected to proxy" url="wss://10.0.2.15:6443/v1-k3s/connect" machine # [ 33.021067] k3s[894]: time="2026-09-23T04:42:59Z" level=info msg="Remotedialer connected to proxy" url="wss://10.0.2.15:6443/v1-k3s/connect" machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-lb" not found machine # [ 33.310771] k3s[894]: time="2026-09-23T04:42:59Z" level=info msg="Imported docker.io/library/nix-grpc-farm-client:0.1.0-beta.3" machine # [ 33.313220] k3s[894]: time="2026-09-23T04:42:59Z" level=info msg="Tagged docker.io/library/nix-grpc-farm-client@sha256:5460d5c00151922fa2a3280753511a9846a54ffc2d41040b640a243054fc62cf" machine # [ 33.697364] k3s[894]: time="2026-09-23T04:43:00Z" level=info msg="Unable to set control-plane role label: nodes \"machine\" not found" machine # [ 33.781385] k3s[894]: time="2026-09-23T04:43:00Z" level=info msg="Imported 1 images from /var/lib/rancher/k3s/agent/images/nix-grpc-farm-client.tar in 8.044052525s" machine # [ 33.783651] k3s[894]: time="2026-09-23T04:43:00Z" level=info msg="Importing images from /var/lib/rancher/k3s/agent/images/nix-grpc-farm-lb.tar" machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-lb" not found machine # [ 34.692965] k3s[894]: time="2026-09-23T04:43:01Z" level=info msg="Unable to set control-plane role label: nodes \"machine\" not found" machine # [ 35.327659] k3s[894]: time="2026-09-23T04:43:01Z" level=info msg="Imported docker.io/library/nix-grpc-farm-lb:0.1.0-beta.3" machine # [ 35.333544] k3s[894]: time="2026-09-23T04:43:01Z" level=info msg="Tagged docker.io/library/nix-grpc-farm-lb@sha256:18ee6724be589622e1aee9d13c5121e7b2c992ca636c935ade90c66423e18f37" machine # [ 35.392583] k3s[894]: time="2026-09-23T04:43:01Z" level=info msg="Imported 1 images from /var/lib/rancher/k3s/agent/images/nix-grpc-farm-lb.tar in 1.610749411s" machine # [ 35.394388] k3s[894]: time="2026-09-23T04:43:01Z" level=info msg="Importing images from /var/lib/rancher/k3s/agent/images/nix-grpc-farm.tar" machine # [ 35.398284] k3s[894]: time="2026-09-23T04:43:01Z" level=info msg="Running kubelet --cloud-provider=external --config-dir=/var/lib/rancher/k3s/agent/etc/kubelet.conf.d --containerd=/run/k3s/containerd/containerd.sock --eviction-hard=nodefs.available<1%,imagefs.available<1% --hostname-override=machine --kubeconfig=/var/lib/rancher/k3s/agent/kubelet.kubeconfig --node-ip=10.0.2.15,fec0::7b2b:a512:d19:a042 --node-labels= --read-only-port=0" machine # [ 35.400798] k3s[894]: Flag --containerd has been deprecated, This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed. machine # [ 35.406145] k3s[894]: I0923 04:43:01.954134 894 server.go:521] "Kubelet version" kubeletVersion="v1.35.8+k3s1" machine # [ 35.408039] k3s[894]: I0923 04:43:01.954168 894 server.go:523] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" machine # [ 35.409963] k3s[894]: I0923 04:43:01.954202 894 watchdog_linux.go:95] "Systemd watchdog is not enabled" machine # [ 35.411238] k3s[894]: I0923 04:43:01.954215 894 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." machine # [ 35.412451] k3s[894]: I0923 04:43:01.955792 894 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/var/lib/rancher/k3s/agent/client-ca.crt" machine # [ 35.413118] k3s[894]: I0923 04:43:01.959693 894 server.go:1414] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" machine # [ 35.416104] k3s[894]: I0923 04:43:01.964020 894 server.go:771] "--cgroups-per-qos enabled, but --cgroup-root was not specified. Defaulting to /" machine # [ 35.417396] k3s[894]: I0923 04:43:01.964070 894 server.go:832] "NoSwap is set due to memorySwapBehavior not specified" memorySwapBehavior="" FailSwapOn=false machine # [ 35.418327] k3s[894]: I0923 04:43:01.964272 894 container_manager_linux.go:272] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] machine # [ 35.419741] k3s[894]: I0923 04:43:01.964294 894 container_manager_linux.go:277] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"machine","RuntimeCgroupsName":"","SystemCgroupsName":"","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":false,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":null,"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.01},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.01},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":-1,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} machine # [ 35.421592] k3s[894]: I0923 04:43:01.964457 894 topology_manager.go:143] "Creating topology manager with none policy" machine # [ 35.422898] k3s[894]: I0923 04:43:01.964472 894 container_manager_linux.go:308] "Creating device plugin manager" machine # [ 35.423888] k3s[894]: I0923 04:43:01.964618 894 container_manager_linux.go:317] "Creating Dynamic Resource Allocation (DRA) manager" machine # [ 35.425307] k3s[894]: I0923 04:43:01.972201 894 state_mem.go:41] "Initialized" logger="CPUManager state memory" machine # [ 35.426176] k3s[894]: I0923 04:43:01.972844 894 kubelet.go:482] "Attempting to sync node with API server" machine # [ 35.428451] k3s[894]: I0923 04:43:01.972886 894 kubelet.go:383] "Adding static pod path" path="/var/lib/rancher/k3s/agent/pod-manifests" machine # [ 35.429450] k3s[894]: I0923 04:43:01.972946 894 kubelet.go:394] "Adding apiserver pod source" machine # [ 35.430807] k3s[894]: I0923 04:43:01.972978 894 apiserver.go:42] "Waiting for node sync before watching apiserver pods" machine # [ 35.431941] k3s[894]: I0923 04:43:01.975185 894 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="containerd" version="2.2.7-k3s1" apiVersion="v1" machine # [ 35.434811] k3s[894]: I0923 04:43:01.975959 894 kubelet.go:945] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" machine # [ 35.435946] k3s[894]: I0923 04:43:01.975985 894 kubelet.go:972] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" machine # [ 35.436803] k3s[894]: I0923 04:43:01.980898 894 server.go:1252] "Started kubelet" machine # [ 35.438047] k3s[894]: I0923 04:43:01.981143 894 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 machine # [ 35.438379] k3s[894]: I0923 04:43:01.981270 894 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 machine # [ 35.439587] k3s[894]: I0923 04:43:01.981337 894 server_v1.go:49] "podresources" method="list" useActivePods=true machine # [ 35.440666] k3s[894]: I0923 04:43:01.981499 894 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" machine # [ 35.441370] k3s[894]: I0923 04:43:01.982057 894 server.go:317] "Adding debug handlers to kubelet server" machine # [ 35.442063] k3s[894]: I0923 04:43:01.986674 894 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" machine # [ 35.442524] k3s[894]: I0923 04:43:01.987332 894 dynamic_serving_content.go:135] "Starting controller" name="kubelet-server-cert-files::/var/lib/rancher/k3s/agent/serving-kubelet.crt::/var/lib/rancher/k3s/agent/serving-kubelet.key" machine # [ 35.443079] k3s[894]: I0923 04:43:01.988491 894 volume_manager.go:311] "Starting Kubelet Volume Manager" machine # [ 35.443918] k3s[894]: I0923 04:43:01.988659 894 desired_state_of_world_populator.go:146] "Desired state populator starts to run" machine # [ 35.444566] k3s[894]: I0923 04:43:01.988710 894 reconciler.go:29] "Reconciler: start to sync state" machine # [ 35.445180] k3s[894]: E0923 04:43:01.991611 894 kubelet_node_status.go:392] "Error getting the current node from lister" err="node \"machine\" not found" machine # [ 35.445678] k3s[894]: I0923 04:43:01.992795 894 factory.go:223] Registration of the systemd container factory successfully machine # [ 35.446761] k3s[894]: I0923 04:43:01.992911 894 factory.go:221] Registration of the crio container factory failed: Get "http://%2Fvar%2Frun%2Fcrio%2Fcrio.sock/info": dial unix /var/run/crio/crio.sock: connect: no such file or directory machine # [ 35.449272] k3s[894]: E0923 04:43:01.993651 894 kubelet.go:1661] "Image garbage collection failed once. Stats initialization may not have completed yet" err="invalid capacity 0 on image filesystem" machine # [ 35.496337] k3s[894]: I0923 04:43:01.995348 894 factory.go:223] Registration of the containerd container factory successfully machine # [ 35.497163] k3s[894]: E0923 04:43:02.001464 894 nodelease.go:50] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"machine\" not found" node="machine" machine # [ 35.497851] k3s[894]: I0923 04:43:02.012006 894 cpu_manager.go:225] "Starting" policy="none" machine # [ 35.499084] k3s[894]: I0923 04:43:02.012024 894 cpu_manager.go:226] "Reconciling" reconcilePeriod="10s" machine # [ 35.499474] k3s[894]: I0923 04:43:02.012046 894 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" machine # [ 35.499907] k3s[894]: I0923 04:43:02.015693 894 policy_none.go:50] "Start" machine # [ 35.500874] k3s[894]: I0923 04:43:02.015729 894 memory_manager.go:187] "Starting memorymanager" policy="None" machine # [ 35.501544] k3s[894]: I0923 04:43:02.015746 894 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" machine # [ 35.502102] k3s[894]: I0923 04:43:02.021699 894 policy_none.go:44] "Start" machine # [ 35.502879] systemd[1]: Created slice libcontainer container kubepods.slice. machine # [ 35.519337] systemd[1]: Created slice libcontainer container kubepods-burstable.slice. machine # [ 35.526142] systemd[1]: Created slice libcontainer container kubepods-besteffort.slice. machine # [ 35.534675] k3s[894]: E0923 04:43:02.082712 894 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" machine # [ 35.535660] k3s[894]: I0923 04:43:02.083697 894 eviction_manager.go:194] "Eviction manager: starting control loop" machine # [ 35.536434] k3s[894]: I0923 04:43:02.083733 894 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" machine # [ 35.536871] k3s[894]: I0923 04:43:02.084305 894 plugin_manager.go:121] "Starting Kubelet Plugin Manager" machine # [ 35.538372] k3s[894]: E0923 04:43:02.086275 894 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="no imagefs label for configured runtime" machine # [ 35.538982] k3s[894]: E0923 04:43:02.086318 894 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"machine\" not found" machine # [ 35.572814] k3s[894]: I0923 04:43:02.120431 894 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" machine # [ 35.581114] k3s[894]: I0923 04:43:02.126500 894 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" machine # [ 35.581758] k3s[894]: I0923 04:43:02.126557 894 status_manager.go:249] "Starting to sync pod status with apiserver" machine # [ 35.582419] k3s[894]: I0923 04:43:02.126607 894 kubelet.go:2506] "Starting kubelet main sync loop" machine # [ 35.582852] k3s[894]: E0923 04:43:02.127142 894 kubelet.go:2530] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" machine # [ 35.637984] k3s[894]: I0923 04:43:02.185972 894 kubelet_node_status.go:74] "Attempting to register node" node="machine" machine # [ 35.645572] k3s[894]: I0923 04:43:02.193611 894 kubelet_node_status.go:77] "Successfully registered node" node="machine" machine # [ 35.648770] k3s[894]: E0923 04:43:02.196792 894 kubelet_node_status.go:474] "Error updating node status, will retry" err="error getting node \"machine\": node \"machine\" not found" machine # [ 35.650967] k3s[894]: time="2026-09-23T04:43:02Z" level=info msg="Adding node OwnerReference to node-password secret machine.node-password.k3s" machine # [ 35.653502] k3s[894]: I0923 04:43:02.201520 894 actual_state_of_world.go:541] "Failed to update statusUpdateNeeded field in actual state of world" logger="persistentvolume-attach-detach-controller" err="Failed to set statusUpdateNeeded to needed true, because nodeName=\"machine\" does not exist" machine # [ 35.656063] k3s[894]: I0923 04:43:02.203322 894 node_controller.go:429] Initializing node machine with cloud provider machine # [ 35.656740] k3s[894]: I0923 04:43:02.203404 894 node_controller.go:233] error syncing 'machine': failed to get instance metadata for node machine: address annotations not yet set, requeuing machine # [ 35.657372] k3s[894]: E0923 04:43:02.203426 894 node_controller.go:244] "Unhandled Error" err="error syncing 'machine': failed to get instance metadata for node machine: address annotations not yet set, requeuing" logger="UnhandledError" machine # [ 35.661392] k3s[894]: I0923 04:43:02.209261 894 node_controller.go:429] Initializing node machine with cloud provider machine # [ 35.661997] k3s[894]: I0923 04:43:02.209309 894 node_controller.go:233] error syncing 'machine': failed to get instance metadata for node machine: address annotations not yet set, requeuing machine # [ 35.662491] k3s[894]: E0923 04:43:02.209327 894 node_controller.go:244] "Unhandled Error" err="error syncing 'machine': failed to get instance metadata for node machine: address annotations not yet set, requeuing" logger="UnhandledError" machine # [ 35.671761] k3s[894]: I0923 04:43:02.219790 894 node_controller.go:429] Initializing node machine with cloud provider machine # [ 35.672404] k3s[894]: I0923 04:43:02.219856 894 node_controller.go:233] error syncing 'machine': failed to get instance metadata for node machine: address annotations not yet set, requeuing machine # [ 35.672827] k3s[894]: E0923 04:43:02.219881 894 node_controller.go:244] "Unhandled Error" err="error syncing 'machine': failed to get instance metadata for node machine: address annotations not yet set, requeuing" logger="UnhandledError" machine # [ 35.683464] k3s[894]: time="2026-09-23T04:43:02Z" level=info msg="Annotations and labels have been set successfully on node: machine" machine # [ 35.684395] k3s[894]: I0923 04:43:02.232004 894 node_controller.go:429] Initializing node machine with cloud provider machine # [ 35.686114] k3s[894]: I0923 04:43:02.233457 894 kuberuntime_manager.go:2095] "Updating runtime config through cri with podcidr" CIDR="10.42.0.0/24" machine # [ 35.686705] k3s[894]: time="2026-09-23T04:43:02Z" level=info msg="Starting flannel with backend vxlan" machine # [ 35.688193] k3s[894]: I0923 04:43:02.232811 894 range_allocator.go:433] "Set node PodCIDR" node="machine" podCIDRs=["10.42.0.0/24"] machine # [ 35.690345] k3s[894]: I0923 04:43:02.238364 894 kubelet_network.go:47] "Updating Pod CIDR" originalPodCIDR="" newPodCIDR="10.42.0.0/24" machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-lb" not found machine # [ 35.695110] k3s[894]: time="2026-09-23T04:43:02Z" level=info msg="Synced coredns NodeHosts entries for machine" machine # [ 35.702815] k3s[894]: time="2026-09-23T04:43:02Z" level=info msg="Labels and annotations have been set successfully on node: machine" machine # [ 35.715178] k3s[894]: time="2026-09-23T04:43:02Z" level=info msg="Flannel found PodCIDR assigned for node machine" machine # [ 35.715711] k3s[894]: time="2026-09-23T04:43:02Z" level=info msg="The interface eth0 with ipv4 address 10.0.2.15 will be used by flannel" machine # [ 35.718330] k3s[894]: I0923 04:43:02.266362 894 kube.go:139] Waiting 10m0s for node controller to sync machine # [ 35.718990] k3s[894]: I0923 04:43:02.266436 894 kube.go:537] Starting kube subnet manager machine # [ 35.781589] k3s[894]: I0923 04:43:02.329485 894 node_controller.go:474] Successfully initialized node machine with cloud provider machine # [ 35.782596] k3s[894]: I0923 04:43:02.329892 894 event.go:389] "Event occurred" object="machine" fieldPath="" kind="Node" apiVersion="v1" type="Normal" reason="Synced" message="Node synced successfully" machine # [ 35.789022] k3s[894]: I0923 04:43:02.336989 894 server.go:173] "Starting Kubernetes Scheduler" version="v1.35.8+k3s1" machine # [ 35.789736] k3s[894]: I0923 04:43:02.337016 894 server.go:175] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" machine # [ 35.792465] k3s[894]: I0923 04:43:02.340503 894 secure_serving.go:211] Serving securely on 127.0.0.1:10259 machine # [ 35.793162] k3s[894]: I0923 04:43:02.340519 894 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController machine # [ 35.793646] k3s[894]: I0923 04:43:02.340535 894 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/var/lib/rancher/k3s/server/tls/kube-scheduler/kube-scheduler.crt::/var/lib/rancher/k3s/server/tls/kube-scheduler/kube-scheduler.key" machine # [ 35.794179] k3s[894]: I0923 04:43:02.340554 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 35.794650] k3s[894]: I0923 04:43:02.340613 894 tlsconfig.go:243] "Starting DynamicServingCertificateController" machine # [ 35.800669] k3s[894]: I0923 04:43:02.348497 894 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" machine # [ 35.801694] k3s[894]: I0923 04:43:02.348529 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 35.802483] k3s[894]: I0923 04:43:02.348558 894 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" machine # [ 35.803932] k3s[894]: I0923 04:43:02.348575 894 shared_informer.go:370] "Waiting for caches to sync" machine # [ 35.806527] k3s[894]: I0923 04:43:02.354549 894 kubelet_node_status.go:427] "Fast updating node status as it just became ready" machine # [ 36.135292] k3s[894]: I0923 04:43:02.683000 894 server.go:218] "Successfully retrieved NodeIPs" NodeIPs=["10.0.2.15","fec0::7b2b:a512:d19:a042"] machine # [ 36.135950] k3s[894]: E0923 04:43:02.683182 894 server.go:255] "Kube-proxy configuration may be incomplete or incorrect" err="nodePortAddresses is unset; NodePort connections will be accepted on all local IPs. Consider using `--nodeport-addresses primary`" machine # [ 36.160167] k3s[894]: I0923 04:43:02.708151 894 server.go:264] "kube-proxy running in dual-stack mode" primary ipFamily="IPv4" machine # [ 36.160960] k3s[894]: I0923 04:43:02.708218 894 server_linux.go:136] "Using iptables Proxier" machine # [ 36.180695] k3s[894]: I0923 04:43:02.728665 894 proxier.go:242] "Setting route_localnet=1 to allow node-ports on localhost; to change this either disable iptables.localhostNodePorts (--iptables-localhost-nodeports) or set nodePortAddresses (--nodeport-addresses) to filter loopback addresses" ipFamily="IPv4" machine # [ 36.190029] k3s[894]: I0923 04:43:02.738023 894 server.go:529] "Version info" version="v1.35.8+k3s1" machine # [ 36.191470] k3s[894]: I0923 04:43:02.738071 894 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" machine # [ 36.193101] k3s[894]: I0923 04:43:02.739208 894 config.go:200] "Starting service config controller" machine # [ 36.194931] k3s[894]: I0923 04:43:02.739241 894 shared_informer.go:349] "Waiting for caches to sync" controller="service config" machine # [ 36.196892] k3s[894]: I0923 04:43:02.739293 894 config.go:106] "Starting endpoint slice config controller" machine # [ 36.198414] k3s[894]: I0923 04:43:02.739325 894 shared_informer.go:349] "Waiting for caches to sync" controller="endpoint slice config" machine # [ 36.200563] k3s[894]: I0923 04:43:02.739766 894 config.go:403] "Starting serviceCIDR config controller" machine # [ 36.202061] k3s[894]: I0923 04:43:02.739772 894 config.go:309] "Starting node config controller" machine # [ 36.203510] k3s[894]: I0923 04:43:02.739787 894 shared_informer.go:349] "Waiting for caches to sync" controller="serviceCIDR config" machine # [ 36.205310] k3s[894]: I0923 04:43:02.739788 894 shared_informer.go:349] "Waiting for caches to sync" controller="node config" machine # [ 36.206899] k3s[894]: I0923 04:43:02.740120 894 shared_informer.go:356] "Caches are synced" controller="node config" machine # [ 36.208521] k3s[894]: I0923 04:43:02.754131 894 shared_informer.go:377] "Caches are synced" machine # [ 36.292258] k3s[894]: I0923 04:43:02.840229 894 shared_informer.go:356] "Caches are synced" controller="service config" machine # [ 36.300640] k3s[894]: I0923 04:43:02.848665 894 shared_informer.go:377] "Caches are synced" machine # [ 36.301917] k3s[894]: I0923 04:43:02.849941 894 shared_informer.go:377] "Caches are synced" machine # [ 36.392088] k3s[894]: I0923 04:43:02.939542 894 shared_informer.go:356] "Caches are synced" controller="endpoint slice config" machine # [ 36.392944] k3s[894]: I0923 04:43:02.940335 894 shared_informer.go:356] "Caches are synced" controller="serviceCIDR config" machine # [ 36.425681] k3s[894]: I0923 04:43:02.973597 894 apiserver.go:52] "Watching apiserver" machine # [ 36.672983] systemd[1]: Created slice libcontainer container kubepods-burstable-podfa66da44_a17b_4a81_b209_12a4de9ea23d.slice. machine # [ 36.697829] systemd[1]: Created slice libcontainer container kubepods-burstable-podaa7098a2_1e14_45cc_b241_9886e60d4756.slice. machine # [ 36.718945] k3s[894]: I0923 04:43:03.266784 894 kube.go:163] Node controller sync successful machine # [ 36.719628] k3s[894]: I0923 04:43:03.266862 894 vxlan.go:128] VXLAN config: VNI=1 Port=0 GBP=false Learning=false DirectRouting=false machine # [ 36.721751] k3s[894]: I0923 04:43:03.269781 894 kube.go:704] List of node(machine) annotations: map[string]string{"alpha.kubernetes.io/provided-node-ip":"10.0.2.15,fec0::7b2b:a512:d19:a042", "k3s.io/hostname":"machine", "k3s.io/internal-ip":"10.0.2.15,fec0::7b2b:a512:d19:a042", "k3s.io/node-args":"[\"server\",\"--disable\",\"traefik\",\"--disable\",\"metrics-server\",\"--disable\",\"local-storage\",\"--kubelet-arg\",\"eviction-hard=nodefs.available\\u003c1%,imagefs.available\\u003c1%\"]", "k3s.io/node-config-hash":"EFXPOZ7ZWHRCLFNW4X4ZDPFQMSAOP3BLEIN2VE47TDRKQ4UIWKUQ====", "k3s.io/node-env":"{}", "node.alpha.kubernetes.io/ttl":"0", "volumes.kubernetes.io/controller-managed-attach-detach":"true"} machine # [ 36.741513] k3s[894]: I0923 04:43:03.289427 894 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" machine # [ 36.750527] k3s[894]: I0923 04:43:03.298020 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa66da44-a17b-4a81-b209-12a4de9ea23d-tmp\") pod \"helm-install-nix-grpc-farm-l5xxx\" (UID: \"fa66da44-a17b-4a81-b209-12a4de9ea23d\") " pod="kube-system/helm-install-nix-grpc-farm-l5xxx" machine # [ 36.751237] k3s[894]: I0923 04:43:03.298072 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"values\" (UniqueName: \"kubernetes.io/projected/fa66da44-a17b-4a81-b209-12a4de9ea23d-values\") pod \"helm-install-nix-grpc-farm-l5xxx\" (UID: \"fa66da44-a17b-4a81-b209-12a4de9ea23d\") " pod="kube-system/helm-install-nix-grpc-farm-l5xxx" machine # [ 36.751696] k3s[894]: I0923 04:43:03.298129 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q59qx\" (UniqueName: \"kubernetes.io/projected/fa66da44-a17b-4a81-b209-12a4de9ea23d-kube-api-access-q59qx\") pod \"helm-install-nix-grpc-farm-l5xxx\" (UID: \"fa66da44-a17b-4a81-b209-12a4de9ea23d\") " pod="kube-system/helm-install-nix-grpc-farm-l5xxx" machine # [ 36.753164] k3s[894]: I0923 04:43:03.298157 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"custom-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa7098a2-1e14-45cc-b241-9886e60d4756-custom-config-volume\") pod \"coredns-c5fdd76cf-mpw7c\" (UID: \"aa7098a2-1e14-45cc-b241-9886e60d4756\") " pod="kube-system/coredns-c5fdd76cf-mpw7c" machine # [ 36.753730] k3s[894]: I0923 04:43:03.298187 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klipper-helm\" (UniqueName: \"kubernetes.io/empty-dir/fa66da44-a17b-4a81-b209-12a4de9ea23d-klipper-helm\") pod \"helm-install-nix-grpc-farm-l5xxx\" (UID: \"fa66da44-a17b-4a81-b209-12a4de9ea23d\") " pod="kube-system/helm-install-nix-grpc-farm-l5xxx" machine # [ 36.754436] k3s[894]: I0923 04:43:03.298212 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klipper-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa66da44-a17b-4a81-b209-12a4de9ea23d-klipper-cache\") pod \"helm-install-nix-grpc-farm-l5xxx\" (UID: \"fa66da44-a17b-4a81-b209-12a4de9ea23d\") " pod="kube-system/helm-install-nix-grpc-farm-l5xxx" machine # [ 36.754912] k3s[894]: I0923 04:43:03.298236 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klipper-config\" (UniqueName: \"kubernetes.io/empty-dir/fa66da44-a17b-4a81-b209-12a4de9ea23d-klipper-config\") pod \"helm-install-nix-grpc-farm-l5xxx\" (UID: \"fa66da44-a17b-4a81-b209-12a4de9ea23d\") " pod="kube-system/helm-install-nix-grpc-farm-l5xxx" machine # [ 36.755642] k3s[894]: I0923 04:43:03.298264 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"content\" (UniqueName: \"kubernetes.io/configmap/fa66da44-a17b-4a81-b209-12a4de9ea23d-content\") pod \"helm-install-nix-grpc-farm-l5xxx\" (UID: \"fa66da44-a17b-4a81-b209-12a4de9ea23d\") " pod="kube-system/helm-install-nix-grpc-farm-l5xxx" machine # [ 36.756411] k3s[894]: I0923 04:43:03.298288 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/aa7098a2-1e14-45cc-b241-9886e60d4756-config-volume\") pod \"coredns-c5fdd76cf-mpw7c\" (UID: \"aa7098a2-1e14-45cc-b241-9886e60d4756\") " pod="kube-system/coredns-c5fdd76cf-mpw7c" machine # [ 36.756941] k3s[894]: I0923 04:43:03.298311 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hnt6\" (UniqueName: \"kubernetes.io/projected/aa7098a2-1e14-45cc-b241-9886e60d4756-kube-api-access-7hnt6\") pod \"coredns-c5fdd76cf-mpw7c\" (UID: \"aa7098a2-1e14-45cc-b241-9886e60d4756\") " pod="kube-system/coredns-c5fdd76cf-mpw7c" machine # [ 36.757896] (udev-worker)[1242]: Network interface NamePolicy= disabled on kernel command line. machine # [ 36.763964] k3s[894]: I0923 04:43:03.311919 894 kube.go:558] Creating the node lease for IPv4. This is the n.Spec.PodCIDRs: [10.42.0.0/24] machine # [ 36.775080] k3s[894]: time="2026-09-23T04:43:03Z" level=info msg="Tunnel authorizer set Kubelet Port 0.0.0.0:10250" machine # [ 36.795258] k3s[894]: I0923 04:43:03.343249 894 iptables.go:50] Starting flannel in iptables mode... machine # [ 36.796241] k3s[894]: time="2026-09-23T04:43:03Z" level=warning msg="no subnet found for key: FLANNEL_NETWORK in file: /run/flannel/subnet.env" machine # [ 36.796907] k3s[894]: time="2026-09-23T04:43:03Z" level=warning msg="no subnet found for key: FLANNEL_SUBNET in file: /run/flannel/subnet.env" machine # [ 36.797827] k3s[894]: time="2026-09-23T04:43:03Z" level=warning msg="no subnet found for key: FLANNEL_IPV6_NETWORK in file: /run/flannel/subnet.env" machine # [ 36.798532] k3s[894]: time="2026-09-23T04:43:03Z" level=warning msg="no subnet found for key: FLANNEL_IPV6_SUBNET in file: /run/flannel/subnet.env" machine # [ 36.799591] k3s[894]: I0923 04:43:03.343372 894 iptables.go:101] Current network or subnet (10.42.0.0/16, 10.42.0.0/24) is not equal to previous one (0.0.0.0/0, 0.0.0.0/0), trying to recycle old iptables rules machine # [ 36.841124] dhcpcd[682]: flannel.1: IAID 95:46:cb:4a machine # [ 36.841926] dhcpcd[682]: flannel.1: adding address fe80::d456:95ff:fe46:cb4a machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-lb" not found machine # [ 36.894705] k3s[894]: I0923 04:43:03.442714 894 iptables.go:111] Setting up masking rules machine # [ 36.906405] k3s[894]: I0923 04:43:03.454407 894 iptables.go:212] Changing default FORWARD chain policy to ACCEPT machine # [ 36.917074] k3s[894]: time="2026-09-23T04:43:03Z" level=info msg="Wrote flannel subnet file to /run/flannel/subnet.env" machine # [ 36.918093] k3s[894]: time="2026-09-23T04:43:03Z" level=info msg="Running flannel backend" machine # [ 36.918717] k3s[894]: I0923 04:43:03.465003 894 vxlan_network.go:68] watching for new subnet leases machine # [ 36.919520] k3s[894]: I0923 04:43:03.465076 894 vxlan_network.go:115] starting vxlan device watcher machine # [ 36.970111] k3s[894]: I0923 04:43:03.517986 894 iptables.go:358] bootstrap done machine # [ 37.009139] k3s[894]: I0923 04:43:03.557151 894 iptables.go:358] bootstrap done machine # [ 37.044110] (udev-worker)[1398]: Network interface NamePolicy= disabled on kernel command line. machine # [ 37.211707] cni0: port 1(veth9f183390) entered blocking state machine # [ 37.212522] cni0: port 1(veth9f183390) entered disabled state machine # [ 37.213413] veth9f183390: entered allmulticast mode machine # [ 37.214357] veth9f183390: entered promiscuous mode machine # [ 37.219479] cni0: port 2(veth6c12ea16) entered blocking state machine # [ 37.220321] cni0: port 2(veth6c12ea16) entered disabled state machine # [ 37.221826] veth6c12ea16: entered allmulticast mode machine # [ 37.223721] veth6c12ea16: entered promiscuous mode machine # [ 37.225450] cni0: port 1(veth9f183390) entered blocking state machine # [ 37.226221] cni0: port 1(veth9f183390) entered forwarding state machine # [ 37.237505] cni0: port 2(veth6c12ea16) entered blocking state machine # [ 37.238243] cni0: port 2(veth6c12ea16) entered forwarding state machine # [ 37.114383] dhcpcd[682]: veth9f183390: waiting for carrier machine # [ 37.125358] dhcpcd[682]: veth6c12ea16: IAID 02:e7:41:28 machine # [ 37.125949] dhcpcd[682]: veth6c12ea16: adding address fe80::3891:2ff:fee7:4128 machine # [ 37.126838] dhcpcd[682]: veth6c12ea16: carrier lost machine # [ 37.151985] dhcpcd[682]: veth6c12ea16: deleting address fe80::3891:2ff:fee7:4128 machine # [ 37.152714] dhcpcd[682]: veth9f183390: carrier acquired machine # [ 37.157705] dhcpcd[682]: veth9f183390: IAID e5:de:75:6d machine # [ 37.158225] dhcpcd[682]: veth9f183390: adding address fe80::d4f9:e5ff:fede:756d machine # [ 37.158719] dhcpcd[682]: veth6c12ea16: carrier acquired machine # [ 37.163841] dhcpcd[682]: veth6c12ea16: IAID 02:e7:41:28 machine # [ 37.164521] dhcpcd[682]: veth6c12ea16: adding address fe80::3891:2ff:fee7:4128 machine # [ 37.396459] systemd[1]: Started libcontainer container 8d70266f993b734ffa96c9d66e97381215c8e4804bec1b9647cae614669e0b13. machine # [ 37.398097] systemd[1]: Started libcontainer container efa15bc2d7412147004a75a7e6f6926b797ba7c99c4ea8ca49074a5490870e8d. machine # [ 37.856069] k3s[894]: time="2026-09-23T04:43:04Z" level=info msg="Starting network policy controller version v2.6.3-k3s1, built on 1970-01-01T01:01:01Z, go1.26.7" machine # [ 37.857023] k3s[894]: I0923 04:43:04.403605 894 network_policy_controller.go:164] Starting network policy controller machine # [ 37.978825] dhcpcd[682]: flannel.1: soliciting a DHCP lease machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-lb" not found machine # [ 38.210801] k3s[894]: I0923 04:43:04.758717 894 network_policy_controller.go:179] Starting network policy controller full sync goroutine machine # [ 38.298490] dhcpcd[682]: veth6c12ea16: soliciting a DHCP lease machine # [ 38.685941] dhcpcd[682]: veth9f183390: soliciting an IPv6 router machine # [ 38.718180] dhcpcd[682]: veth9f183390: soliciting a DHCP lease machine # [ 38.772847] systemd[1]: Started libcontainer container 8f6e5d21733772333e11d5d1660b4d76a74cb88ee7ad978cc84eeaa8a8ea908c. machine # [ 38.792250] systemd[1]: Started libcontainer container ab1282eac8cdbc3de3a49947662123ca4ffd6e10164e51643cc80b4a54faf235. machine # [ 38.863879] systemd[1]: var-lib-rancher-k3s-agent-containerd-tmpmounts-containerd\x2dmount2842816288.mount: Deactivated successfully. machine # [ 39.060250] dhcpcd[682]: flannel.1: soliciting an IPv6 router machine # [ 39.067123] dhcpcd[682]: veth6c12ea16: soliciting an IPv6 router machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-lb" not found machine # [ 39.642697] k3s[894]: I0923 04:43:06.190363 894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/coredns-c5fdd76cf-mpw7c" podStartSLOduration=9.19033941 podStartE2EDuration="9.19033941s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-23 04:42:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-23 04:43:06.172873224 +0000 UTC m=+21.280289253" watchObservedRunningTime="2026-09-23 04:43:06.19033941 +0000 UTC m=+21.297755160" machine # [ 39.651484] k3s[894]: I0923 04:43:06.199072 894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/helm-install-nix-grpc-farm-l5xxx" podStartSLOduration=10.199048618 podStartE2EDuration="10.199048618s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-23 04:42:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-23 04:43:06.190229341 +0000 UTC m=+21.297645370" watchObservedRunningTime="2026-09-23 04:43:06.199048618 +0000 UTC m=+21.306464647" machine # [ 39.687123] k3s[894]: I0923 04:43:06.234507 894 node_lifecycle_controller.go:1234] "Initializing eviction metric for zone" zone="" machine # [ 39.688777] k3s[894]: I0923 04:43:06.234621 894 node_lifecycle_controller.go:886] "Missing timestamp for Node. Assuming now as a timestamp" node="machine" machine # [ 39.692780] k3s[894]: I0923 04:43:06.234714 894 node_lifecycle_controller.go:1080] "Controller detected that zone is now in new state" zone="" newState="Normal" machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-lb" not found machine # [ 40.988578] k3s[894]: I0923 04:43:07.536244 894 alloc.go:329] "allocated clusterIPs" service="farm/nix-grpc-farm" clusterIPs={"IPv4":"10.43.228.181"} machine # [ 41.055431] systemd[1]: cri-containerd-ab1282eac8cdbc3de3a49947662123ca4ffd6e10164e51643cc80b4a54faf235.scope: Deactivated successfully. machine # [ 41.055788] systemd[1]: cri-containerd-ab1282eac8cdbc3de3a49947662123ca4ffd6e10164e51643cc80b4a54faf235.scope: Consumed 615ms CPU time over 2.262s wall clock time, 31.7M memory peak, 2.4M incoming IP traffic, 141.5K outgoing IP traffic. machine # [ 41.128139] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-ab1282eac8cdbc3de3a49947662123ca4ffd6e10164e51643cc80b4a54faf235-rootfs.mount: Deactivated successfully. machine # [ 41.184464] systemd[1]: Created slice libcontainer container kubepods-besteffort-pod7c115297_d66c_4e71_8d33_a3352d5fb986.slice. machine # [ 41.227124] systemd[1]: Created slice libcontainer container kubepods-besteffort-podfd4651b8_ec20_44c9_a82a_de134cebe717.slice. machine # [ 41.252114] systemd[1]: Created slice libcontainer container kubepods-besteffort-podae9a33b9_1018_48e2_b8c8_e295154a3817.slice. machine # [ 41.266888] systemd[1]: Created slice libcontainer container kubepods-besteffort-pod2097027f_9980_4c52_8efa_cdff6446dfaf.slice. machine # [ 41.275313] k3s[894]: I0923 04:43:07.823252 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls\" (UniqueName: \"kubernetes.io/secret/7c115297-d66c-4e71-8d33-a3352d5fb986-tls\") pod \"nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2\" (UID: \"7c115297-d66c-4e71-8d33-a3352d5fb986\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2" machine # [ 41.279978] k3s[894]: I0923 04:43:07.823293 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc\" (UniqueName: \"kubernetes.io/secret/7c115297-d66c-4e71-8d33-a3352d5fb986-oidc\") pod \"nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2\" (UID: \"7c115297-d66c-4e71-8d33-a3352d5fb986\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2" machine # [ 41.284890] k3s[894]: I0923 04:43:07.823330 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qc8w8\" (UniqueName: \"kubernetes.io/projected/7c115297-d66c-4e71-8d33-a3352d5fb986-kube-api-access-qc8w8\") pod \"nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2\" (UID: \"7c115297-d66c-4e71-8d33-a3352d5fb986\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2" machine # [ 41.289673] k3s[894]: I0923 04:43:07.823505 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nix\" (UniqueName: \"kubernetes.io/empty-dir/7c115297-d66c-4e71-8d33-a3352d5fb986-nix\") pod \"nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2\" (UID: \"7c115297-d66c-4e71-8d33-a3352d5fb986\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2" machine # [ 41.294085] k3s[894]: I0923 04:43:07.823537 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"niks3-token\" (UniqueName: \"kubernetes.io/projected/7c115297-d66c-4e71-8d33-a3352d5fb986-niks3-token\") pod \"nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2\" (UID: \"7c115297-d66c-4e71-8d33-a3352d5fb986\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2" machine # [ 41.298551] k3s[894]: I0923 04:43:07.823565 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7c115297-d66c-4e71-8d33-a3352d5fb986-ca\") pod \"nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2\" (UID: \"7c115297-d66c-4e71-8d33-a3352d5fb986\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2" machine # [ 41.302728] k3s[894]: I0923 04:43:07.823616 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"build\" (UniqueName: \"kubernetes.io/empty-dir/7c115297-d66c-4e71-8d33-a3352d5fb986-build\") pod \"nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2\" (UID: \"7c115297-d66c-4e71-8d33-a3352d5fb986\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2" machine # [ 41.307096] k3s[894]: I0923 04:43:07.823642 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nixconf\" (UniqueName: \"kubernetes.io/configmap/7c115297-d66c-4e71-8d33-a3352d5fb986-nixconf\") pod \"nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2\" (UID: \"7c115297-d66c-4e71-8d33-a3352d5fb986\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2" machine # [ 41.376230] k3s[894]: I0923 04:43:07.924215 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc\" (UniqueName: \"kubernetes.io/secret/ae9a33b9-1018-48e2-b8c8-e295154a3817-oidc\") pod \"nix-grpc-farm-scheduler-64688b4568-4m5cg\" (UID: \"ae9a33b9-1018-48e2-b8c8-e295154a3817\") " pod="farm/nix-grpc-farm-scheduler-64688b4568-4m5cg" machine # [ 41.380817] k3s[894]: I0923 04:43:07.924526 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-ca\" (UniqueName: \"kubernetes.io/secret/fd4651b8-ec20-44c9-a82a-de134cebe717-tls-ca\") pod \"nix-grpc-farm-lb-775dbb5c7c-mwvmc\" (UID: \"fd4651b8-ec20-44c9-a82a-de134cebe717\") " pod="farm/nix-grpc-farm-lb-775dbb5c7c-mwvmc" machine # [ 41.381455] k3s[894]: I0923 04:43:07.924574 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"niks3-token\" (UniqueName: \"kubernetes.io/projected/2097027f-9980-4c52-8efa-cdff6446dfaf-niks3-token\") pod \"nix-grpc-farm-worker-x86-64-linux-cfb8578f6-qlcdh\" (UID: \"2097027f-9980-4c52-8efa-cdff6446dfaf\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-qlcdh" machine # [ 41.381952] k3s[894]: I0923 04:43:07.924608 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/2097027f-9980-4c52-8efa-cdff6446dfaf-ca\") pod \"nix-grpc-farm-worker-x86-64-linux-cfb8578f6-qlcdh\" (UID: \"2097027f-9980-4c52-8efa-cdff6446dfaf\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-qlcdh" machine # [ 41.382606] k3s[894]: I0923 04:43:07.924692 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkbv5\" (UniqueName: \"kubernetes.io/projected/2097027f-9980-4c52-8efa-cdff6446dfaf-kube-api-access-xkbv5\") pod \"nix-grpc-farm-worker-x86-64-linux-cfb8578f6-qlcdh\" (UID: \"2097027f-9980-4c52-8efa-cdff6446dfaf\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-qlcdh" machine # [ 41.384027] k3s[894]: I0923 04:43:07.924784 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd4651b8-ec20-44c9-a82a-de134cebe717-tmp\") pod \"nix-grpc-farm-lb-775dbb5c7c-mwvmc\" (UID: \"fd4651b8-ec20-44c9-a82a-de134cebe717\") " pod="farm/nix-grpc-farm-lb-775dbb5c7c-mwvmc" machine # [ 41.384730] k3s[894]: I0923 04:43:07.924811 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls\" (UniqueName: \"kubernetes.io/secret/ae9a33b9-1018-48e2-b8c8-e295154a3817-tls\") pod \"nix-grpc-farm-scheduler-64688b4568-4m5cg\" (UID: \"ae9a33b9-1018-48e2-b8c8-e295154a3817\") " pod="farm/nix-grpc-farm-scheduler-64688b4568-4m5cg" machine # [ 41.386486] k3s[894]: I0923 04:43:07.924839 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gkx8\" (UniqueName: \"kubernetes.io/projected/ae9a33b9-1018-48e2-b8c8-e295154a3817-kube-api-access-4gkx8\") pod \"nix-grpc-farm-scheduler-64688b4568-4m5cg\" (UID: \"ae9a33b9-1018-48e2-b8c8-e295154a3817\") " pod="farm/nix-grpc-farm-scheduler-64688b4568-4m5cg" machine # [ 41.387047] k3s[894]: I0923 04:43:07.924866 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nix\" (UniqueName: \"kubernetes.io/empty-dir/2097027f-9980-4c52-8efa-cdff6446dfaf-nix\") pod \"nix-grpc-farm-worker-x86-64-linux-cfb8578f6-qlcdh\" (UID: \"2097027f-9980-4c52-8efa-cdff6446dfaf\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-qlcdh" machine # [ 41.387520] k3s[894]: I0923 04:43:07.924892 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc\" (UniqueName: \"kubernetes.io/secret/2097027f-9980-4c52-8efa-cdff6446dfaf-oidc\") pod \"nix-grpc-farm-worker-x86-64-linux-cfb8578f6-qlcdh\" (UID: \"2097027f-9980-4c52-8efa-cdff6446dfaf\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-qlcdh" machine # [ 41.392164] k3s[894]: I0923 04:43:07.924917 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ae9a33b9-1018-48e2-b8c8-e295154a3817-ca\") pod \"nix-grpc-farm-scheduler-64688b4568-4m5cg\" (UID: \"ae9a33b9-1018-48e2-b8c8-e295154a3817\") " pod="farm/nix-grpc-farm-scheduler-64688b4568-4m5cg" machine # [ 41.392713] k3s[894]: I0923 04:43:07.924948 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls\" (UniqueName: \"kubernetes.io/secret/2097027f-9980-4c52-8efa-cdff6446dfaf-tls\") pod \"nix-grpc-farm-worker-x86-64-linux-cfb8578f6-qlcdh\" (UID: \"2097027f-9980-4c52-8efa-cdff6446dfaf\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-qlcdh" machine # [ 41.393404] k3s[894]: I0923 04:43:07.924997 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-lb\" (UniqueName: \"kubernetes.io/secret/fd4651b8-ec20-44c9-a82a-de134cebe717-tls-lb\") pod \"nix-grpc-farm-lb-775dbb5c7c-mwvmc\" (UID: \"fd4651b8-ec20-44c9-a82a-de134cebe717\") " pod="farm/nix-grpc-farm-lb-775dbb5c7c-mwvmc" machine # [ 41.394678] k3s[894]: I0923 04:43:07.925025 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fd4651b8-ec20-44c9-a82a-de134cebe717-config\") pod \"nix-grpc-farm-lb-775dbb5c7c-mwvmc\" (UID: \"fd4651b8-ec20-44c9-a82a-de134cebe717\") " pod="farm/nix-grpc-farm-lb-775dbb5c7c-mwvmc" machine # [ 41.395371] k3s[894]: I0923 04:43:07.925082 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"niks3-token\" (UniqueName: \"kubernetes.io/projected/ae9a33b9-1018-48e2-b8c8-e295154a3817-niks3-token\") pod \"nix-grpc-farm-scheduler-64688b4568-4m5cg\" (UID: \"ae9a33b9-1018-48e2-b8c8-e295154a3817\") " pod="farm/nix-grpc-farm-scheduler-64688b4568-4m5cg" machine # [ 41.397840] k3s[894]: I0923 04:43:07.925131 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"build\" (UniqueName: \"kubernetes.io/empty-dir/2097027f-9980-4c52-8efa-cdff6446dfaf-build\") pod \"nix-grpc-farm-worker-x86-64-linux-cfb8578f6-qlcdh\" (UID: \"2097027f-9980-4c52-8efa-cdff6446dfaf\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-qlcdh" machine # [ 41.401652] k3s[894]: I0923 04:43:07.925157 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nixconf\" (UniqueName: \"kubernetes.io/configmap/2097027f-9980-4c52-8efa-cdff6446dfaf-nixconf\") pod \"nix-grpc-farm-worker-x86-64-linux-cfb8578f6-qlcdh\" (UID: \"2097027f-9980-4c52-8efa-cdff6446dfaf\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-qlcdh" machine # [ 41.703350] cni0: port 3(vethef05b6a1) entered blocking state machine # [ 41.548748] ([ 41.703955] cni0: port 3(vethef05b6a1) entered disabled state machine # udev-worker)[1995]: Network inte[ 41.704956] vethef05b6a1: entered allmulticast mode machine # rface NamePolicy= disabled on ke[ 41.705917] vethef05b6a1: entered promiscuous mode machine # rnel command line. machine # [ 41.722643] cni0: port 4(veth96611f10) entered blocking state machine # [ 41.723435] cni0: port 4(veth96611f10) entered disabled state machine # [ 41.724169] veth96611f10: entered allmulticast mode machine # [ 41.725861] veth96611f10: entered promiscuous mode machine # [ 41.577157] (udev-worker)[2143]: Network inte[ 41.732622] cni0: port 3(vethef05b6a1) entered blocking state machine # rface NamePolicy[ 41.733558] cni0: port 3(vethef05b6a1) entered forwarding state machine # = disabled on kernel command line. machine # [ 41.613378] (udev-worker)[2136]: Network inte[ 41.768833] cni0: port 5(veth962b590b) entered blocking state machine # rface NamePolicy= disabled on ke[ 41.769997] cni0: port 5(veth962b590b) entered disabled state machine # rnel command line.[ 41.771080] veth962b590b: entered allmulticast mode machine # machine # [ 41.771695] veth962b590b: entered promiscuous mode machine # [ 41.619619] dhcpcd[682]: veth[ 41.774919] cni0: port 4(veth96611f10) entered blocking state machine # ef05b6a1: waitin[ 41.775840] cni0: port 4(veth96611f10) entered forwarding state machine # g for carrier machine # [ 41.798640] cni0: port 5(veth962b590b) entered blocking state machine # [ 41.799737] cni0: port 5(veth962b590b) entered forwarding state machine # [ 41.661432] dhcpcd[682]: veth96611f10: waiting for carrier machine # [ 41.668274] (udev-worker)[2140]: Network interface NamePolicy= disabled on kernel command line. machine # [ 41.824378] cni0: port 6(veth872dc9bc) entered blocking state machine # [ 41.825803] cni0: port 6(veth872dc9bc) entered disabled state machine # [ 41.826692] veth872dc9bc: entered allmulticast mode machine # [ 41.827498] veth872dc9bc: entered promiscuous mode machine # [ 41.678673] dhcpcd[682]: veth962b590b: IAID b3:6a:76:19 machine # [ 41.679512] dhcpcd[682]: veth962b590b: adding address fe80::dcff:b3ff:fe6a:7619 machine # [ 41.680281] dhcpcd[682]: vethef05b6a1: carrier acquired machine # [ 41.685200] dhcpcd[682]: vethef05b6a1: IAID 5e:a2:ea:22 machine # [ 41.685883] dhcpcd[682]: vethef05b6a1: adding address fe80::d4c8:5eff:fea2:ea22 machine # [ 41.686554] dhcpcd[682]: veth962b590b: old hardware address: de:ff:b3:6a:76:19 machine # [ 41.687167] dhcpcd[682]: veth962b590b: new hardware address: 72:78:c6:f3:14:7b machine # [ 41.687684] dhcpcd[682]: veth962b590b: carrier lost machine # [ 41.857444] cni0: port 6(veth872dc9bc) entered blocking state machine # [ 41.858192] cni0: port 6(veth872dc9bc) entered forwarding state machine # [ 41.716519] dhcpcd[682]: veth962b590b: deleting address fe80::dcff:b3ff:fe6a:7619 machine # [ 41.730111] dhcpcd[682]: veth872dc9bc: IAID a3:d6:55:72 machine # [ 41.730817] dhcpcd[682]: veth872dc9bc: adding address fe80::8e0:a3ff:fed6:5572 machine # [ 41.731463] dhcpcd[682]: veth96611f10: carrier acquired machine # [ 41.737904] dhcpcd[682]: veth96611f10: IAID 85:43:4b:9b machine # [ 41.738804] dhcpcd[682]: veth96611f10: adding address fe80::c27:85ff:fe43:4b9b machine # [ 41.739819] dhcpcd[682]: veth962b590b: carrier acquired machine # [ 41.745827] dhcpcd[682]: veth962b590b: IAID c6:f3:14:7b machine # [ 41.746813] dhcpcd[682]: veth962b590b: adding address fe80::7078:c6ff:fef3:147b machine # [ 41.748043] dhcpcd[682]: veth872dc9bc: old hardware address: 0a:e0:a3:d6:55:72 machine # [ 41.749142] dhcpcd[682]: veth872dc9bc: new hardware address: e2:80:0d:32:11:26 machine # [ 41.749739] dhcpcd[682]: veth872dc9bc: carrier lost machine # [ 41.775492] dhcpcd[682]: veth872dc9bc: deleting address fe80::8e0:a3ff:fed6:5572 machine # [ 41.776223] dhcpcd[682]: veth872dc9bc: old hardware address: e2:80:0d:32:11:26 machine # [ 41.776660] dhcpcd[682]: veth872dc9bc: new hardware address: 0a:e0:a3:d6:55:72 machine # [ 41.778941] dhcpcd[682]: veth872dc9bc: carrier acquired machine # [ 41.780612] systemd[1]: Started libcontainer container 6568901c799c519e891b9993449d8f7d341fa9c83a3f8a37092337fa2ecd1a93. machine # [ 41.785155] dhcpcd[682]: veth872dc9bc: IAID a3:d6:55:72 machine # [ 41.786280] dhcpcd[682]: veth872dc9bc: adding address fe80::8e0:a3ff:fed6:5572 machine # [ 41.858445] systemd[1]: Started libcontainer container 64bc447511b72a9409772e189c7bb392c8b32cd7e5fabd0585d35194eb604863. machine # [ 41.861209] systemd[1]: Started libcontainer container 61edf6ac7608e1a9972941a9c82042d29c511cb14bf67c51e7570bc371ad843a. machine # [ 41.901060] k3s[894]: E0923 04:43:08.449032 894 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = failed to pull and unpack image \"docker.io/library/nix-grpc-farm:0.1.0-beta.3\": failed to resolve reference \"docker.io/library/nix-grpc-farm:0.1.0-beta.3\": failed to do request: Head \"https://registry-1.docker.io/v2/library/nix-grpc-farm/manifests/0.1.0-beta.3\": dial tcp: lookup registry-1.docker.io: no such host" image="nix-grpc-farm:0.1.0-beta.3" machine # [ 41.902090] k3s[894]: E0923 04:43:08.450138 894 kuberuntime_image.go:43] "Failed to pull image" err="failed to pull and unpack image \"docker.io/library/nix-grpc-farm:0.1.0-beta.3\": failed to resolve reference \"docker.io/library/nix-grpc-farm:0.1.0-beta.3\": failed to do request: Head \"https://registry-1.docker.io/v2/library/nix-grpc-farm/manifests/0.1.0-beta.3\": dial tcp: lookup registry-1.docker.io: no such host" image="nix-grpc-farm:0.1.0-beta.3" machine # [ 41.903776] k3s[894]: E0923 04:43:08.451781 894 kuberuntime_manager.go:1674] "Unhandled Error" err="init container seed-store start failed in pod nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2_farm(7c115297-d66c-4e71-8d33-a3352d5fb986): ErrImagePull: failed to pull and unpack image \"docker.io/library/nix-grpc-farm:0.1.0-beta.3\": failed to resolve reference \"docker.io/library/nix-grpc-farm:0.1.0-beta.3\": failed to do request: Head \"https://registry-1.docker.io/v2/library/nix-grpc-farm/manifests/0.1.0-beta.3\": dial tcp: lookup registry-1.docker.io: no such host" logger="UnhandledError" machine # [ 41.904705] k3s[894]: E0923 04:43:08.452361 894 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"seed-store\" with ErrImagePull: \"failed to pull and unpack image \\\"docker.io/library/nix-grpc-farm:0.1.0-beta.3\\\": failed to resolve reference \\\"docker.io/library/nix-grpc-farm:0.1.0-beta.3\\\": failed to do request: Head \\\"https://registry-1.docker.io/v2/library/nix-grpc-farm/manifests/0.1.0-beta.3\\\": dial tcp: lookup registry-1.docker.io: no such host\"" pod="farm/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2" podUID="7c115297-d66c-4e71-8d33-a3352d5fb986" machine # [ 41.937342] systemd[1]: Started libcontainer container 8e3f678dc869071d1f10ae4ca8c440b8c54bb9a98ca6a8b9dba1982861d2abb4. machine # [ 41.967687] k3s[894]: E0923 04:43:08.515376 894 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = failed to pull and unpack image \"docker.io/library/nix-grpc-farm:0.1.0-beta.3\": failed to resolve reference \"docker.io/library/nix-grpc-farm:0.1.0-beta.3\": failed to do request: Head \"https://registry-1.docker.io/v2/library/nix-grpc-farm/manifests/0.1.0-beta.3\": dial tcp: lookup registry-1.docker.io: no such host" image="nix-grpc-farm:0.1.0-beta.3" machine # [ 41.968535] k3s[894]: E0923 04:43:08.515420 894 kuberuntime_image.go:43] "Failed to pull image" err="failed to pull and unpack image \"docker.io/library/nix-grpc-farm:0.1.0-beta.3\": failed to resolve reference \"docker.io/library/nix-grpc-farm:0.1.0-beta.3\": failed to do request: Head \"https://registry-1.docker.io/v2/library/nix-grpc-farm/manifests/0.1.0-beta.3\": dial tcp: lookup registry-1.docker.io: no such host" image="nix-grpc-farm:0.1.0-beta.3" machine # [ 41.969071] k3s[894]: E0923 04:43:08.515592 894 kuberuntime_manager.go:1674] "Unhandled Error" err="container nix-grpc-daemon start failed in pod nix-grpc-farm-scheduler-64688b4568-4m5cg_farm(ae9a33b9-1018-48e2-b8c8-e295154a3817): ErrImagePull: failed to pull and unpack image \"docker.io/library/nix-grpc-farm:0.1.0-beta.3\": failed to resolve reference \"docker.io/library/nix-grpc-farm:0.1.0-beta.3\": failed to do request: Head \"https://registry-1.docker.io/v2/library/nix-grpc-farm/manifests/0.1.0-beta.3\": dial tcp: lookup registry-1.docker.io: no such host" logger="UnhandledError" machine # [ 41.969549] k3s[894]: E0923 04:43:08.515642 894 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nix-grpc-daemon\" with ErrImagePull: \"failed to pull and unpack image \\\"docker.io/library/nix-grpc-farm:0.1.0-beta.3\\\": failed to resolve reference \\\"docker.io/library/nix-grpc-farm:0.1.0-beta.3\\\": failed to do request: Head \\\"https://registry-1.docker.io/v2/library/nix-grpc-farm/manifests/0.1.0-beta.3\\\": dial tcp: lookup registry-1.docker.io: no such host\"" pod="farm/nix-grpc-farm-scheduler-64688b4568-4m5cg" podUID="ae9a33b9-1018-48e2-b8c8-e295154a3817" machine # [ 42.036274] k3s[894]: E0923 04:43:08.583932 894 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = failed to pull and unpack image \"docker.io/library/nix-grpc-farm:0.1.0-beta.3\": failed to resolve reference \"docker.io/library/nix-grpc-farm:0.1.0-beta.3\": failed to do request: Head \"https://registry-1.docker.io/v2/library/nix-grpc-farm/manifests/0.1.0-beta.3\": dial tcp: lookup registry-1.docker.io: no such host" image="nix-grpc-farm:0.1.0-beta.3" machine # [ 42.043887] k3s[894]: E0923 04:43:08.583989 894 kuberuntime_image.go:43] "Failed to pull image" err="failed to pull and unpack image \"docker.io/library/nix-grpc-farm:0.1.0-beta.3\": failed to resolve reference \"docker.io/library/nix-grpc-farm:0.1.0-beta.3\": failed to do request: Head \"https://registry-1.docker.io/v2/library/nix-grpc-farm/manifests/0.1.0-beta.3\": dial tcp: lookup registry-1.docker.io: no such host" image="nix-grpc-farm:0.1.0-beta.3" machine # [ 42.044591] k3s[894]: E0923 04:43:08.584230 894 kuberuntime_manager.go:1674] "Unhandled Error" err="init container seed-store start failed in pod nix-grpc-farm-worker-x86-64-linux-cfb8578f6-qlcdh_farm(2097027f-9980-4c52-8efa-cdff6446dfaf): ErrImagePull: failed to pull and unpack image \"docker.io/library/nix-grpc-farm:0.1.0-beta.3\": failed to resolve reference \"docker.io/library/nix-grpc-farm:0.1.0-beta.3\": failed to do request: Head \"https://registry-1.docker.io/v2/library/nix-grpc-farm/manifests/0.1.0-beta.3\": dial tcp: lookup registry-1.docker.io: no such host" logger="UnhandledError" machine # [ 42.045241] k3s[894]: E0923 04:43:08.584294 894 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"seed-store\" with ErrImagePull: \"failed to pull and unpack image \\\"docker.io/library/nix-grpc-farm:0.1.0-beta.3\\\": failed to resolve reference \\\"docker.io/library/nix-grpc-farm:0.1.0-beta.3\\\": failed to do request: Head \\\"https://registry-1.docker.io/v2/library/nix-grpc-farm/manifests/0.1.0-beta.3\\\": dial tcp: lookup registry-1.docker.io: no such host\"" pod="farm/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-qlcdh" podUID="2097027f-9980-4c52-8efa-cdff6446dfaf" machine # [ 42.140362] k3s[894]: time="2026-09-23T04:43:08Z" level=info msg="Imported docker.io/library/nix-grpc-farm:0.1.0-beta.3" machine # [ 42.148440] k3s[894]: time="2026-09-23T04:43:08Z" level=info msg="Tagged docker.io/library/nix-grpc-farm@sha256:f8ec27a0e6a50b3825faf7f058613969eec9bfd1cc6f9a6dac783d11bd4e5a8d" machine # [ 42.354429] dhcpcd[682]: veth872dc9bc: soliciting a DHCP lease machine # [ 42.696747] systemd[1]: cri-containerd-8d70266f993b734ffa96c9d66e97381215c8e4804bec1b9647cae614669e0b13.scope: Deactivated successfully. machine # [ 42.737889] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-8d70266f993b734ffa96c9d66e97381215c8e4804bec1b9647cae614669e0b13-rootfs.mount: Deactivated successfully. machine # [ 42.764621] dhcpcd[682]: veth962b590b: soliciting a DHCP lease machine # [ 42.869692] dhcpcd[682]: vethef05b6a1: soliciting a DHCP lease machine # [ 43.050957] cni0: port 1(veth9f183390) entered disabled state machine # [ 42.896224] dhcpcd[682]: veth9f183390: carrier lost machine # [ 43.053435] veth9f183390 (unregistering): left allmulticast mode machine # [ 43.054577] veth9f183390 (unregistering): left promiscuous mode machine # [ 43.055517] cni0: port 1(veth9f183390) entered disabled state machine # [ 42.922509] dhcpcd[682]: veth9f183390: deleting address fe80::d4f9:e5ff:fede:756d machine # [ 42.965297] dhcpcd[682]: veth9f183390: removing interface machine # [ 42.978559] dhcpcd[682]: flannel.1: probing for an IPv4LL address machine # [ 43.009630] dhcpcd[682]: vethef05b6a1: soliciting an IPv6 router machine # [ 43.094225] k3s[894]: I0923 04:43:09.641891 894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fa66da44-a17b-4a81-b209-12a4de9ea23d-values\" (UniqueName: \"kubernetes.io/projected/fa66da44-a17b-4a81-b209-12a4de9ea23d-values\") pod \"fa66da44-a17b-4a81-b209-12a4de9ea23d\" (UID: \"fa66da44-a17b-4a81-b209-12a4de9ea23d\") " machine # [ 43.095348] k3s[894]: I0923 04:43:09.641939 894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fa66da44-a17b-4a81-b209-12a4de9ea23d-content\" (UniqueName: \"kubernetes.io/configmap/fa66da44-a17b-4a81-b209-12a4de9ea23d-content\") pod \"fa66da44-a17b-4a81-b209-12a4de9ea23d\" (UID: \"fa66da44-a17b-4a81-b209-12a4de9ea23d\") " machine # [ 43.097132] k3s[894]: I0923 04:43:09.641970 894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa66da44-a17b-4a81-b209-12a4de9ea23d-klipper-helm\" (UniqueName: \"kubernetes.io/empty-dir/fa66da44-a17b-4a81-b209-12a4de9ea23d-klipper-helm\") pod \"fa66da44-a17b-4a81-b209-12a4de9ea23d\" (UID: \"fa66da44-a17b-4a81-b209-12a4de9ea23d\") " machine # [ 43.098179] k3s[894]: I0923 04:43:09.642021 894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fa66da44-a17b-4a81-b209-12a4de9ea23d-kube-api-access-q59qx\" (UniqueName: \"kubernetes.io/projected/fa66da44-a17b-4a81-b209-12a4de9ea23d-kube-api-access-q59qx\") pod \"fa66da44-a17b-4a81-b209-12a4de9ea23d\" (UID: \"fa66da44-a17b-4a81-b209-12a4de9ea23d\") " machine # [ 43.098943] k3s[894]: I0923 04:43:09.642114 894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa66da44-a17b-4a81-b209-12a4de9ea23d-tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa66da44-a17b-4a81-b209-12a4de9ea23d-tmp\") pod \"fa66da44-a17b-4a81-b209-12a4de9ea23d\" (UID: \"fa66da44-a17b-4a81-b209-12a4de9ea23d\") " machine # [ 43.099654] k3s[894]: I0923 04:43:09.642149 894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa66da44-a17b-4a81-b209-12a4de9ea23d-klipper-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa66da44-a17b-4a81-b209-12a4de9ea23d-klipper-cache\") pod \"fa66da44-a17b-4a81-b209-12a4de9ea23d\" (UID: \"fa66da44-a17b-4a81-b209-12a4de9ea23d\") " machine # [ 43.100232] k3s[894]: I0923 04:43:09.642178 894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa66da44-a17b-4a81-b209-12a4de9ea23d-klipper-config\" (UniqueName: \"kubernetes.io/empty-dir/fa66da44-a17b-4a81-b209-12a4de9ea23d-klipper-config\") pod \"fa66da44-a17b-4a81-b209-12a4de9ea23d\" (UID: \"fa66da44-a17b-4a81-b209-12a4de9ea23d\") " machine # [ 43.100608] k3s[894]: I0923 04:43:09.647837 894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa66da44-a17b-4a81-b209-12a4de9ea23d-content" pod "fa66da44-a17b-4a81-b209-12a4de9ea23d" (UID: "fa66da44-a17b-4a81-b209-12a4de9ea23d"). InnerVolumeSpecName "content". PluginName "kubernetes.io/configmap", VolumeGIDValue "" machine # [ 43.128811] systemd[1]: run-k3s-containerd-io.containerd.grpc.v1.cri-sandboxes-8d70266f993b734ffa96c9d66e97381215c8e4804bec1b9647cae614669e0b13-shm.mount: Deactivated successfully. machine # [ 43.129988] systemd[1]: run-netns-cni\x2dbc8d5355\x2d4fab\x2d28f5\x2d0667\x2d9fa9a6566d0a.mount: Deactivated successfully. machine # [ 43.131131] systemd[1]: var-lib-kubelet-pods-fa66da44\x2da17b\x2d4a81\x2db209\x2d12a4de9ea23d-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dq59qx.mount: Deactivated successfully. machine # [ 43.131799] systemd[1]: var-lib-kubelet-pods-fa66da44\x2da17b\x2d4a81\x2db209\x2d12a4de9ea23d-volumes-kubernetes.io\x7eempty\x2ddir-klipper\x2dhelm.mount: Deactivated successfully. machine # [ 43.132469] systemd[1]: var-lib-kubelet-pods-fa66da44\x2da17b\x2d4a81\x2db209\x2d12a4de9ea23d-volumes-kubernetes.io\x7eempty\x2ddir-klipper\x2dcache.mount: Deactivated successfully. machine # [ 43.133100] systemd[1]: var-lib-kubelet-pods-fa66da44\x2da17b\x2d4a81\x2db209\x2d12a4de9ea23d-volumes-kubernetes.io\x7eprojected-values.mount: Deactivated successfully. machine # [ 43.133636] systemd[1]: var-lib-kubelet-pods-fa66da44\x2da17b\x2d4a81\x2db209\x2d12a4de9ea23d-volumes-kubernetes.io\x7eempty\x2ddir-tmp.mount: Deactivated successfully. machine # [ 43.148936] k3s[894]: I0923 04:43:09.696921 894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa66da44-a17b-4a81-b209-12a4de9ea23d-tmp" pod "fa66da44-a17b-4a81-b209-12a4de9ea23d" (UID: "fa66da44-a17b-4a81-b209-12a4de9ea23d"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 43.149711] k3s[894]: I0923 04:43:09.697290 894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa66da44-a17b-4a81-b209-12a4de9ea23d-kube-api-access-q59qx" pod "fa66da44-a17b-4a81-b209-12a4de9ea23d" (UID: "fa66da44-a17b-4a81-b209-12a4de9ea23d"). InnerVolumeSpecName "kube-api-access-q59qx". PluginName "kubernetes.io/projected", VolumeGIDValue "" machine # [ 43.150398] k3s[894]: I0923 04:43:09.697583 894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa66da44-a17b-4a81-b209-12a4de9ea23d-klipper-cache" pod "fa66da44-a17b-4a81-b209-12a4de9ea23d" (UID: "fa66da44-a17b-4a81-b209-12a4de9ea23d"). InnerVolumeSpecName "klipper-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 43.151059] k3s[894]: I0923 04:43:09.697697 894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa66da44-a17b-4a81-b209-12a4de9ea23d-klipper-helm" pod "fa66da44-a17b-4a81-b209-12a4de9ea23d" (UID: "fa66da44-a17b-4a81-b209-12a4de9ea23d"). InnerVolumeSpecName "klipper-helm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 43.151584] k3s[894]: I0923 04:43:09.697932 894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa66da44-a17b-4a81-b209-12a4de9ea23d-values" pod "fa66da44-a17b-4a81-b209-12a4de9ea23d" (UID: "fa66da44-a17b-4a81-b209-12a4de9ea23d"). InnerVolumeSpecName "values". PluginName "kubernetes.io/projected", VolumeGIDValue "" machine # [ 43.174777] k3s[894]: I0923 04:43:09.722775 894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa66da44-a17b-4a81-b209-12a4de9ea23d-klipper-config" pod "fa66da44-a17b-4a81-b209-12a4de9ea23d" (UID: "fa66da44-a17b-4a81-b209-12a4de9ea23d"). InnerVolumeSpecName "klipper-config". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 43.175565] systemd[1]: var-lib-kubelet-pods-fa66da44\x2da17b\x2d4a81\x2db209\x2d12a4de9ea23d-volumes-kubernetes.io\x7eempty\x2ddir-klipper\x2dconfig.mount: Deactivated successfully. machine # [ 43.194735] k3s[894]: I0923 04:43:09.742743 894 reconciler_common.go:299] "Volume detached for volume \"values\" (UniqueName: \"kubernetes.io/projected/fa66da44-a17b-4a81-b209-12a4de9ea23d-values\") on node \"machine\" DevicePath \"\"" machine # [ 43.195504] k3s[894]: I0923 04:43:09.742776 894 reconciler_common.go:299] "Volume detached for volume \"content\" (UniqueName: \"kubernetes.io/configmap/fa66da44-a17b-4a81-b209-12a4de9ea23d-content\") on node \"machine\" DevicePath \"\"" machine # [ 43.196029] k3s[894]: I0923 04:43:09.742793 894 reconciler_common.go:299] "Volume detached for volume \"klipper-helm\" (UniqueName: \"kubernetes.io/empty-dir/fa66da44-a17b-4a81-b209-12a4de9ea23d-klipper-helm\") on node \"machine\" DevicePath \"\"" machine # [ 43.196543] k3s[894]: I0923 04:43:09.742812 894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q59qx\" (UniqueName: \"kubernetes.io/projected/fa66da44-a17b-4a81-b209-12a4de9ea23d-kube-api-access-q59qx\") on node \"machine\" DevicePath \"\"" machine # [ 43.197095] k3s[894]: I0923 04:43:09.742826 894 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa66da44-a17b-4a81-b209-12a4de9ea23d-tmp\") on node \"machine\" DevicePath \"\"" machine # [ 43.197575] k3s[894]: I0923 04:43:09.742840 894 reconciler_common.go:299] "Volume detached for volume \"klipper-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa66da44-a17b-4a81-b209-12a4de9ea23d-klipper-cache\") on node \"machine\" DevicePath \"\"" machine # [ 43.198110] k3s[894]: I0923 04:43:09.742852 894 reconciler_common.go:299] "Volume detached for volume \"klipper-config\" (UniqueName: \"kubernetes.io/empty-dir/fa66da44-a17b-4a81-b209-12a4de9ea23d-klipper-config\") on node \"machine\" DevicePath \"\"" machine # [ 43.300501] dhcpcd[682]: veth6c12ea16: probing for an IPv4LL address machine # [ 43.329624] systemd[1]: Started libcontainer container bafcae70604ed9e2b0534a3450de331e8ad933bc7cfd87c957f4e898b3e9412a. machine # [ 43.540044] dhcpcd[682]: veth96611f10: soliciting a DHCP lease machine # [ 43.593733] systemd[1]: Removed slice libcontainer container kubepods-burstable-podfa66da44_a17b_4a81_b209_12a4de9ea23d.slice. machine # [ 43.595153] systemd[1]: kubepods-burstable-podfa66da44_a17b_4a81_b209_12a4de9ea23d.slice: Consumed 646ms CPU time over 6.918s wall clock time, 32M memory peak, 2.4M incoming IP traffic, 141.6K outgoing IP traffic. machine # [ 43.669336] k3s[894]: I0923 04:43:10.217245 894 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8d70266f993b734ffa96c9d66e97381215c8e4804bec1b9647cae614669e0b13" machine # [ 43.893809] dhcpcd[682]: veth96611f10: soliciting an IPv6 router machine # [ 43.946757] dhcpcd[682]: veth872dc9bc: soliciting an IPv6 router machine # [ 43.959407] k3s[894]: I0923 04:43:10.506692 894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="farm/nix-grpc-farm-lb-775dbb5c7c-mwvmc" podStartSLOduration=3.506657545 podStartE2EDuration="3.506657545s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-23 04:43:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-23 04:43:10.237354603 +0000 UTC m=+25.344770353" watchObservedRunningTime="2026-09-23 04:43:10.506657545 +0000 UTC m=+25.614073295" machine # [ 44.129443] k3s[894]: time="2026-09-23T04:43:10Z" level=info msg="Imported 1 images from /var/lib/rancher/k3s/agent/images/nix-grpc-farm.tar in 8.736221502s" machine # [ 44.302486] dhcpcd[682]: veth962b590b: soliciting an IPv6 router machine # [ 47.355587] dhcpcd[682]: veth872dc9bc: probing for an IPv4LL address machine # [ 47.765369] dhcpcd[682]: veth962b590b: probing for an IPv4LL address machine # [ 47.869899] dhcpcd[682]: vethef05b6a1: probing for an IPv4LL address machine # [ 48.137513] dhcpcd[682]: flannel.1: using IPv4LL address 169.254.112.156 machine # [ 48.140570] dhcpcd[682]: flannel.1: adding route to 169.254.0.0/16 machine # [ 48.540908] dhcpcd[682]: veth96611f10: probing for an IPv4LL address machine # [ 48.627396] systemd[1]: var-lib-rancher-k3s-agent-containerd-tmpmounts-ovl\x2didmapped586929809.mount: Deactivated successfully. machine # [ 48.644497] systemd[1]: var-lib-rancher-k3s-agent-containerd-tmpmounts-ovl\x2didmapped872463278.mount: Deactivated successfully. machine # [ 48.748326] systemd[1]: Started libcontainer container 3f3f3687c227ed69811f8dbb6e4c94d1689928c91478ccc25b1bfcf280557a64. machine # [ 48.751209] systemd[1]: Started libcontainer container b5b9e34a98e6214056976c65b12cd02141cf948e5f30c8515747777f90dcc508. machine # [ 48.771318] systemd[1]: Started libcontainer container d7251d9ef6ff6747a218c72c14b032b14b0cdb41122d1c40e17975fae12f4053. machine # [ 49.016104] dhcpcd[682]: veth6c12ea16: using IPv4LL address 169.254.179.72 machine # [ 49.016813] dhcpcd[682]: veth6c12ea16: adding route to 169.254.0.0/16 machine # [ 49.043797] niks3-server[1195]: 2026/09/23 04:43:15 INFO lead: acquired remote=10.42.0.6:33064 machine # [ 49.624441] systemd[1]: var-lib-rancher-k3s-agent-containerd-tmpmounts-ovl\x2didmapped713355902.mount: Deactivated successfully. machine # [ 49.765661] k3s[894]: I0923 04:43:16.313338 894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="farm/nix-grpc-farm-scheduler-64688b4568-4m5cg" podStartSLOduration=8.611682625 podStartE2EDuration="9.313289952s" totalImagesPullingTime="701.607327ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-23 04:43:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-23 04:43:16.28138031 +0000 UTC m=+31.388796060" watchObservedRunningTime="2026-09-23 04:43:16.313289952 +0000 UTC m=+31.420728051" machine: (finished: waiting for success: kubectl -n farm rollout status deployment nix-grpc-farm-lb --timeout=10s, in 22.69 seconds) machine: waiting for success: kubectl -n farm rollout status deployment nix-grpc-farm-scheduler --timeout=10s machine # [ 50.835323] systemd[1]: cri-containerd-b5b9e34a98e6214056976c65b12cd02141cf948e5f30c8515747777f90dcc508.scope: Deactivated successfully. machine # [ 50.837781] systemd[1]: cri-containerd-b5b9e34a98e6214056976c65b12cd02141cf948e5f30c8515747777f90dcc508.scope: Consumed 796ms CPU time over 2.083s wall clock time, 342.1M memory peak, 78.6M read from disk, 229.9M written to disk. machine # [ 50.840906] systemd[1]: cri-containerd-d7251d9ef6ff6747a218c72c14b032b14b0cdb41122d1c40e17975fae12f4053.scope: Deactivated successfully. machine # [ 50.842562] systemd[1]: cri-containerd-d7251d9ef6ff6747a218c72c14b032b14b0cdb41122d1c40e17975fae12f4053.scope: Consumed 813ms CPU time over 2.066s wall clock time, 349.9M memory peak, 76.9M read from disk, 261.4M written to disk. machine # [ 50.895396] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-d7251d9ef6ff6747a218c72c14b032b14b0cdb41122d1c40e17975fae12f4053-rootfs.mount: Deactivated successfully. machine # [ 50.899663] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-b5b9e34a98e6214056976c65b12cd02141cf948e5f30c8515747777f90dcc508-rootfs.mount: Deactivated successfully. machine: (finished: waiting for success: kubectl -n farm rollout status deployment nix-grpc-farm-scheduler --timeout=10s, in 0.21 seconds) machine: waiting for success: kubectl -n farm rollout status deployment nix-grpc-farm-worker-x86-64-linux --timeout=10s machine # [ 51.063088] dhcpcd[682]: flannel.1: no IPv6 Routers available machine # [ 51.069977] dhcpcd[682]: veth6c12ea16: no IPv6 Routers available machine # [ 51.843354] dhcpcd[682]: veth872dc9bc: using IPv4LL address 169.254.105.236 machine # [ 51.844806] dhcpcd[682]: veth872dc9bc: adding route to 169.254.0.0/16 machine # [ 51.899305] systemd[1]: var-lib-rancher-k3s-agent-containerd-tmpmounts-ovl\x2didmapped1012098725.mount: Deactivated successfully. machine # [ 51.925286] systemd[1]: var-lib-rancher-k3s-agent-containerd-tmpmounts-ovl\x2didmapped352143939.mount: Deactivated successfully. machine # [ 51.928201] systemd[1]: var-lib-rancher-k3s-agent-containerd-tmpmounts-containerd\x2dmount2915982346.mount: Deactivated successfully. machine # [ 51.947285] systemd[1]: Started libcontainer container e1ccf679381bf12f6bee9bbfce219489c7a887f367f7ea486397cab3577c67de. machine # [ 52.036307] systemd[1]: Started libcontainer container f9ba5a63eb0462b76d644667b92ae8ac19e4e5f62719d693b31fa40b539464af. machine # [ 52.145135] systemd[1]: Started libcontainer container 26233b4eec41c971718e7c5ca4729f24a94486a947b2727beb9d499452943e09. machine # [ 52.235824] systemd[1]: Started libcontainer container 86d47e32616f448aaf7ede86c6cf800d428a00db97db587f4e636f51dbdabbe4. machine # [ 52.360817] systemd[1]: Started libcontainer container fad9579f7f7d12c5d24c198640ef7f86f922e07790acdb28e0418cc246240675. machine # [ 52.430626] dhcpcd[682]: veth962b590b: using IPv4LL address 169.254.94.91 machine # [ 52.431544] dhcpcd[682]: veth962b590b: adding route to 169.254.0.0/16 machine # [ 52.463303] systemd[1]: Started libcontainer container 3dc3130f896e4d28d430dbacb77f3c9131e199ecbc34d86b2fdc1dac1973f3e2. machine # [ 52.707721] dhcpcd[682]: vethef05b6a1: using IPv4LL address 169.254.84.172 machine # [ 52.709886] dhcpcd[682]: vethef05b6a1: adding route to 169.254.0.0/16 machine # [ 52.779760] k3s[894]: I0923 04:43:19.326970 894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="farm/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2" podStartSLOduration=9.52411479 podStartE2EDuration="12.326933585s" totalImagesPullingTime="786.395199ms" totalInitContainerRuntime="2.016423596s" isStatefulPod=false podCreationTimestamp="2026-09-23 04:43:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-23 04:43:19.325768632 +0000 UTC m=+34.433187735" watchObservedRunningTime="2026-09-23 04:43:19.326933585 +0000 UTC m=+34.434352408" machine # [ 52.813457] k3s[894]: I0923 04:43:19.361363 894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="farm/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-qlcdh" podStartSLOduration=9.708278421 podStartE2EDuration="12.361342986s" totalImagesPullingTime="630.149641ms" totalInitContainerRuntime="2.022914924s" isStatefulPod=false podCreationTimestamp="2026-09-23 04:43:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-23 04:43:19.360959418 +0000 UTC m=+34.468378520" watchObservedRunningTime="2026-09-23 04:43:19.361342986 +0000 UTC m=+34.468758457" machine # [ 52.903889] systemd[1]: tmp-ovl\x2didmapped3707764055.mount: Deactivated successfully. machine # [ 53.232454] dhcpcd[682]: veth96611f10: using IPv4LL address 169.254.48.35 machine # [ 53.234199] dhcpcd[682]: veth96611f10: adding route to 169.254.0.0/16 machine # [ 54.313795] dhcpcd[682]: segfault at 583a6b43d ip 0000583a2ee35cb3 sp 00007ffc27c3ca58 error 4 in dhcpcd[38cb3,583a2ee06000+40000] likely on CPU 1 (core 1, socket 0) machine # [ 54.318373] Code: 45 ae 01 e9 0f fe ff ff 0f 1f 80 00 00 00 00 c6 45 af 01 e9 ff fd ff ff e8 fa 05 fd ff 66 2e 0f 1f 84 00 00 00 00 00 48 8b 07 <48> 8b 80 50 03 00 00 48 85 c0 74 31 90 48 8b 00 48 85 c0 74 1e 48 machine # [ 54.179919] systemd-coredump[3475]: Process 682 (dhcpcd) of user 999 terminated abnormally with signal 11/SEGV, processing... machine # [ 54.189642] systemd[1]: Created slice Slice /system/systemd-coredump. machine # [ 54.193368] systemd[1]: Started Process Core Dump (PID 3475/UID 0). machine # [ 54.421164] systemd-coredump[3476]: Process 682 (dhcpcd) of user 999 dumped core. machine # machine # Module udev.so without build-id. machine # Module dhcpcd without build-id. machine # Stack trace of thread 682: machine # #0 0x0000583a2ee35cb3 ipv6nd_expire (dhcpcd + 0x38cb3) machine # #1 0x0000583a2ee0f06a eloop_start (dhcpcd + 0x1206a) machine # #2 0x0000583a2ee07300 main (dhcpcd + 0xa300) machine # #3 0x000072ff70930285 __libc_start_call_main (libc.so.6 + 0x2b285) machine # #4 0x000072ff70930338 __libc_start_main@@GLIBC_2.34 (libc.so.6 + 0x2b338) machine # #5 0x0000583a2ee08365 _start (dhcpcd + 0xb365) machine # ELF object binary architecture: AMD x86-64 machine # machine # [ 54.439732] systemd[1]: systemd-coredump@0-1-3475_3476-0.service: Deactivated successfully. machine # [ 54.514120] systemd[1]: dhcpcd.service: Main process exited, code=dumped, status=11/SEGV machine # [ 54.516503] systemd[1]: dhcpcd.service: Failed with result 'core-dump'. machine # [ 54.518201] systemd[1]: dhcpcd.service: Consumed 602ms CPU time over 46.277s wall clock time, 6.4M memory peak, 8K written to disk, 96B incoming IP traffic, 1.2K outgoing IP traffic. machine # [ 54.698642] systemd[1]: dhcpcd.service: Scheduled restart job, restart counter is at 1. machine # [ 54.704506] systemd[1]: Starting DHCP Client... machine # [ 54.907782] dhcpcd[3488]: dhcpcd-10.3.2 starting machine # [ 54.910623] dhcpcd[3492]: dev: loaded udev machine # [ 54.911789] dhcpcd[3492]: DUID 00:01:00:01:32:46:15:3b:52:54:00:12:34:56 machine # [ 55.025554] dhcpcd[3492]: eth0: IAID 00:12:34:56 machine # [ 55.026752] dhcpcd[3492]: flannel.1: IAID 95:46:cb:4a machine # [ 55.028134] dhcpcd[3492]: veth6c12ea16: IAID 02:e7:41:28 machine # [ 55.029473] dhcpcd[3492]: vethef05b6a1: IAID 5e:a2:ea:22 machine # [ 55.031127] dhcpcd[3492]: veth96611f10: IAID 85:43:4b:9b machine # [ 55.032453] dhcpcd[3492]: veth962b590b: IAID b3:6a:76:19 machine # [ 55.033958] dhcpcd[3492]: veth872dc9bc: IAID a3:d6:55:72 machine # [ 55.037856] dhcpcd[3492]: veth962b590b: soliciting a DHCP lease machine # [ 55.195724] dhcpcd[3492]: eth0: soliciting an IPv6 router machine # [ 55.197464] dhcpcd[3492]: eth0: Router Advertisement from fe80::2 machine # [ 55.199288] dhcpcd[3492]: eth0: adding address fec0::5054:ff:fe12:3456/64 machine # [ 55.572115] dhcpcd[3492]: veth6c12ea16: soliciting an IPv6 router machine # [ 55.600236] dhcpcd[3492]: vethef05b6a1: soliciting an IPv6 router machine # [ 55.614463] dhcpcd[3492]: flannel.1: soliciting an IPv6 router machine # [ 55.746987] dhcpcd[3492]: veth96611f10: soliciting a DHCP lease machine # [ 55.797777] dhcpcd[3492]: veth962b590b: soliciting an IPv6 router machine # [ 55.827366] dhcpcd[3492]: veth872dc9bc: soliciting a DHCP lease machine # [ 55.903782] dhcpcd[3492]: veth872dc9bc: soliciting an IPv6 router machine # [ 55.937504] dhcpcd[3492]: veth96611f10: soliciting an IPv6 router machine # [ 56.079524] dhcpcd[3492]: eth0: rebinding lease of 10.0.2.15 machine # [ 56.087850] dhcpcd[3492]: eth0: leased 10.0.2.15 for 86400 seconds machine # [ 56.149657] systemd[1]: Started DHCP Client. machine # [ 56.365246] dhcpcd[3492]: veth6c12ea16: soliciting a DHCP lease machine # [ 56.685471] dhcpcd[3492]: flannel.1: soliciting a DHCP lease machine # [ 56.888334] dhcpcd[3492]: vethef05b6a1: soliciting a DHCP lease machine: (finished: waiting for success: kubectl -n farm rollout status deployment nix-grpc-farm-worker-x86-64-linux --timeout=10s, in 5.89 seconds) machine: waiting for unit niks3.service machine: (finished: waiting for unit niks3.service, in 0.04 seconds) machine: must succeed: kubectl -n farm get pod -l app.kubernetes.io/component=lb -o jsonpath='{.items[0].status.podIP}{.status.podIP}' machine: (finished: must succeed: kubectl -n farm get pod -l app.kubernetes.io/component=lb -o jsonpath='{.items[0].status.podIP}{.status.podIP}', in 0.72 seconds) machine: waiting for success: test $(curl -sf 10.42.0.5:9901/clusters | grep -c 'health_flags::healthy') -ge 3 machine: (finished: waiting for success: test $(curl -sf 10.42.0.5:9901/clusters | grep -c 'health_flags::healthy') -ge 3, in 3.25 seconds) machine: must succeed: kubectl -n farm get pod -l app.kubernetes.io/component=scheduler -o jsonpath='{.items[0].status.podIP}{.status.podIP}' machine: (finished: must succeed: kubectl -n farm get pod -l app.kubernetes.io/component=scheduler -o jsonpath='{.items[0].status.podIP}{.status.podIP}', in 0.20 seconds) machine: waiting for success: curl -sf http://10.42.0.6:9464/metrics | grep -qx 'nix_grpc_sched{kind="workers"} 2' machine: (finished: waiting for success: curl -sf http://10.42.0.6:9464/metrics | grep -qx 'nix_grpc_sched{kind="workers"} 2', in 0.04 seconds) (finished: subtest: chart deploys and workers become Ready, in 33.06 seconds) machine: must succeed: kubectl -n farm get pods -l app.kubernetes.io/component=worker --field-selector=status.phase=Running -o jsonpath='{.items[*].metadata.name}' machine: (finished: must succeed: kubectl -n farm get pods -l app.kubernetes.io/component=worker --field-selector=status.phase=Running -o jsonpath='{.items[*].metadata.name}', in 0.15 seconds) machine: must succeed: kubectl -n ci create token builder --audience nix-grpc-farm > /tmp/builder.jwt machine: (finished: must succeed: kubectl -n ci create token builder --audience nix-grpc-farm > /tmp/builder.jwt, in 0.15 seconds) subtest: build through the balancer lands in the cache machine: must succeed: nix build -L --store 'grpc://127.0.0.1:30051?ca-cert=/nix/store/b2prb199a55q32qawjagqrylwrml5g74-k3s-farm-certs/ca.crt&token-file=/tmp/builder.jwt' --eval-store auto -f /nix/store/4m7mpr04a8dziray8jhdxrxsvaq1bzrn-job.nix --argstr tag t1 2>&1 machine # [ 64.420372] dhcpcd[3492]: veth962b590b: using IPv4LL address 169.254.94.91 machine # [ 64.422899] dhcpcd[3492]: flannel.1: adding route to 169.254.0.0/16 machine # [ 64.424465] dhcpcd[3492]: veth6c12ea16: adding route to 169.254.0.0/16 machine # [ 64.425245] dhcpcd[3492]: vethef05b6a1: adding route to 169.254.0.0/16 machine # [ 64.426774] dhcpcd[3492]: veth96611f10: adding route to 169.254.0.0/16 machine # [ 64.428361] dhcpcd[3492]: veth962b590b: adding route to 169.254.0.0/16 machine # [ 64.429509] dhcpcd[3492]: veth872dc9bc: adding route to 169.254.0.0/16 machine # [ 65.380629] dhcpcd[3492]: veth872dc9bc: using IPv4LL address 169.254.105.236 machine # [ 66.286855] dhcpcd[3492]: veth96611f10: using IPv4LL address 169.254.48.35 machine # [ 67.218244] dhcpcd[3492]: flannel.1: using IPv4LL address 169.254.112.156 machine # [ 67.271145] dhcpcd[3492]: veth6c12ea16: using IPv4LL address 169.254.179.72 machine # [ 67.428307] dhcpcd[3492]: vethef05b6a1: using IPv4LL address 169.254.84.172 machine # [ 67.575620] dhcpcd[3492]: veth6c12ea16: no IPv6 Routers available machine # [ 67.602820] dhcpcd[3492]: vethef05b6a1: no IPv6 Routers available machine # [ 67.617154] dhcpcd[3492]: flannel.1: no IPv6 Routers available machine # [ 67.801501] dhcpcd[3492]: veth962b590b: no IPv6 Routers available machine # [ 67.906113] dhcpcd[3492]: veth872dc9bc: no IPv6 Routers available machine # [ 67.939230] dhcpcd[3492]: veth96611f10: no IPv6 Routers available machine # [ 94.199029] niks3-server[1195]: 2026/09/23 04:44:00 INFO Received uploads request method=POST path=/api/pending_closures machine # [ 94.225670] niks3-server[1195]: 2026/09/23 04:44:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign machine # [ 94.229226] niks3-server[1195]: 2026/09/23 04:44:00 INFO Signed narinfos id=1 count=1 machine # [ 94.239675] niks3-server[1195]: 2026/09/23 04:44:00 INFO Registered completed upload object_key=yz1sqzb72i8v6h2ass0v76ba8cncqj9r.ls machine # [ 94.241190] niks3-server[1195]: 2026/09/23 04:44:00 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete machine # [ 94.245137] niks3-server[1195]: 2026/09/23 04:44:00 INFO Completed upload id=1 machine # [ 94.250108] niks3-server[1195]: 2026/09/23 04:44:00 INFO Registered completed upload object_key=yz1sqzb72i8v6h2ass0v76ba8cncqj9r.narinfo machine # [ 94.251114] niks3-server[1195]: 2026/09/23 04:44:00 INFO Registered completed upload object_key=nar/0nzzbas7l8d37mridr7mg162smfjvn7lhs6am4crl4ydaxicigj8.nar.zst machine # [ 118.762204] niks3-server[1195]: 2026/09/23 04:44:25 INFO Received uploads request method=POST path=/api/pending_closures machine # [ 118.788897] niks3-server[1195]: 2026/09/23 04:44:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign machine # [ 118.793717] niks3-server[1195]: 2026/09/23 04:44:25 INFO Signed narinfos id=2 count=1 machine # [ 118.795908] niks3-server[1195]: 2026/09/23 04:44:25 INFO Registered completed upload object_key=nar/1p7jh5nyz9n68k5f18ymmdpgr34mpnnch1njw7ki7b2b97qlfbjk.nar.zst machine # [ 118.798479] niks3-server[1195]: 2026/09/23 04:44:25 INFO Registered completed upload object_key=r95r891injj9ski5c4w89yxjvjy065mx.ls machine # [ 118.804780] niks3-server[1195]: 2026/09/23 04:44:25 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete machine # [ 118.807237] niks3-server[1195]: 2026/09/23 04:44:25 INFO Completed upload id=2 machine # [ 118.809555] niks3-server[1195]: 2026/09/23 04:44:25 INFO Registered completed upload object_key=r95r891injj9ski5c4w89yxjvjy065mx.narinfo machine # [ 118.880880] niks3-server[1195]: 2026/09/23 04:44:25 INFO Received uploads request method=POST path=/api/pending_closures machine # [ 118.895505] niks3-server[1195]: 2026/09/23 04:44:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign machine # [ 118.898262] niks3-server[1195]: 2026/09/23 04:44:25 INFO Signed narinfos id=3 count=1 machine # [ 118.900069] niks3-server[1195]: 2026/09/23 04:44:25 INFO Registered completed upload object_key=nar/03a4ywhyr5knmbc93z4gyml5y8fvgb431xvlmfxrwsm8mrb0qgx2.nar.zst machine # [ 118.902341] niks3-server[1195]: 2026/09/23 04:44:25 INFO Registered completed upload object_key=zfq1xw9bmycqh35l0s44lmv65z4xc7br.ls machine # [ 118.905557] niks3-server[1195]: 2026/09/23 04:44:25 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete machine # [ 118.908338] niks3-server[1195]: 2026/09/23 04:44:25 INFO Completed upload id=3 machine # [ 118.909699] niks3-server[1195]: 2026/09/23 04:44:25 INFO Registered completed upload object_key=zfq1xw9bmycqh35l0s44lmv65z4xc7br.narinfo machine # [ 144.223635] niks3-server[1195]: 2026/09/23 04:44:50 INFO Received uploads request method=POST path=/api/pending_closures machine # [ 144.244799] niks3-server[1195]: 2026/09/23 04:44:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign machine # [ 144.247733] niks3-server[1195]: 2026/09/23 04:44:50 INFO Signed narinfos id=4 count=1 machine # [ 144.251681] niks3-server[1195]: 2026/09/23 04:44:50 INFO Registered completed upload object_key=log/zfq1xw9bmycqh35l0s44lmv65z4xc7br-k3s-b.drv machine # [ 144.253616] niks3-server[1195]: 2026/09/23 04:44:50 INFO Registered completed upload object_key=mrqsh0475wxsmnsvkk55g94v6g5nrcra.ls machine # [ 144.262867] niks3-server[1195]: 2026/09/23 04:44:50 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete machine # [ 144.267814] niks3-server[1195]: 2026/09/23 04:44:50 INFO Registered completed upload object_key=nar/05qjsj5ff3a6vnxkhb6k5f3m446dcc206p7cd37w4v3mgi95d7kh.nar.zst machine # [ 144.276697] niks3-server[1195]: 2026/09/23 04:44:50 INFO Registered completed upload object_key=mrqsh0475wxsmnsvkk55g94v6g5nrcra.narinfo machine # [ 144.280755] niks3-server[1195]: 2026/09/23 04:44:50 INFO Completed upload id=4 machine # [ 144.335229] niks3-server[1195]: 2026/09/23 04:44:50 INFO Received uploads request method=POST path=/api/pending_closures machine # [ 144.346926] niks3-server[1195]: 2026/09/23 04:44:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign machine # [ 144.347826] niks3-server[1195]: 2026/09/23 04:44:50 INFO Signed narinfos id=5 count=1 machine # [ 144.352306] niks3-server[1195]: 2026/09/23 04:44:50 INFO Registered completed upload object_key=n40czam6a4ldp1mh5fdmlcy07fv0akbm.ls machine # [ 144.357608] niks3-server[1195]: 2026/09/23 04:44:50 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete machine # [ 144.360931] niks3-server[1195]: 2026/09/23 04:44:50 INFO Registered completed upload object_key=nar/0jyw88klkf3nwxpnj5m5lx3i1dv2idf3hcn6im7883hnldm3gvyq.nar.zst machine # [ 144.361895] niks3-server[1195]: 2026/09/23 04:44:50 INFO Registered completed upload object_key=n40czam6a4ldp1mh5fdmlcy07fv0akbm.narinfo machine # [ 144.364775] niks3-server[1195]: 2026/09/23 04:44:50 INFO Completed upload id=5 machine # [ 169.485931] niks3-server[1195]: 2026/09/23 04:45:16 INFO Received uploads request method=POST path=/api/pending_closures machine # [ 169.501350] niks3-server[1195]: 2026/09/23 04:45:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign machine # [ 169.505664] niks3-server[1195]: 2026/09/23 04:45:16 INFO Signed narinfos id=6 count=1 machine # [ 169.508688] niks3-server[1195]: 2026/09/23 04:45:16 INFO Registered completed upload object_key=0vg7x9rp0cg9wyzr3fn2wj6nw4l33b5c.ls machine # [ 169.513150] niks3-server[1195]: 2026/09/23 04:45:16 INFO Registered completed upload object_key=nar/1q8vp5xqws560mpf8rxk8r8jhqrzibivh13x4wxy0cfqlzbk7cr5.nar.zst machine # [ 169.518332] niks3-server[1195]: 2026/09/23 04:45:16 INFO Registered completed upload object_key=log/n40czam6a4ldp1mh5fdmlcy07fv0akbm-k3s-top.drv machine # [ 169.524393] niks3-server[1195]: 2026/09/23 04:45:16 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete machine # [ 169.528711] niks3-server[1195]: 2026/09/23 04:45:16 INFO Registered completed upload object_key=0vg7x9rp0cg9wyzr3fn2wj6nw4l33b5c.narinfo machine # [ 169.532829] niks3-server[1195]: 2026/09/23 04:45:16 INFO Completed upload id=6 machine: (finished: must succeed: nix build -L --store 'grpc://127.0.0.1:30051?ca-cert=/nix/store/b2prb199a55q32qawjagqrylwrml5g74-k3s-farm-certs/ca.crt&token-file=/tmp/builder.jwt' --eval-store auto -f /nix/store/4m7mpr04a8dziray8jhdxrxsvaq1bzrn-job.nix --argstr tag t1 2>&1, in 108.09 seconds) machine: must succeed: nix eval --raw -f /nix/store/4m7mpr04a8dziray8jhdxrxsvaq1bzrn-job.nix --argstr tag t1 outPath machine: (finished: must succeed: nix eval --raw -f /nix/store/4m7mpr04a8dziray8jhdxrxsvaq1bzrn-job.nix --argstr tag t1 outPath, in 0.14 seconds) machine: must fail: test -e /nix/store/0vg7x9rp0cg9wyzr3fn2wj6nw4l33b5c-k3s-top machine: (finished: must fail: test -e /nix/store/0vg7x9rp0cg9wyzr3fn2wj6nw4l33b5c-k3s-top, in 0.01 seconds) machine: must succeed: nix copy --from http://192.168.1.1:5751 /nix/store/0vg7x9rp0cg9wyzr3fn2wj6nw4l33b5c-k3s-top && grep k3s-top-t1 /nix/store/0vg7x9rp0cg9wyzr3fn2wj6nw4l33b5c-k3s-top machine # copying 3 paths... machine # copying path '/nix/store/r95r891injj9ski5c4w89yxjvjy065mx-k3s-a' from 'http://192.168.1.1:5751'... machine # copying path '/nix/store/mrqsh0475wxsmnsvkk55g94v6g5nrcra-k3s-b' from 'http://192.168.1.1:5751'... machine # copying path '/nix/store/0vg7x9rp0cg9wyzr3fn2wj6nw4l33b5c-k3s-top' from 'http://192.168.1.1:5751'... machine: (finished: must succeed: nix copy --from http://192.168.1.1:5751 /nix/store/0vg7x9rp0cg9wyzr3fn2wj6nw4l33b5c-k3s-top && grep k3s-top-t1 /nix/store/0vg7x9rp0cg9wyzr3fn2wj6nw4l33b5c-k3s-top, in 0.20 seconds) machine: must succeed: nix build -L --store 'grpc://127.0.0.1:30051?ca-cert=/nix/store/b2prb199a55q32qawjagqrylwrml5g74-k3s-farm-certs/ca.crt&token-file=/tmp/builder.jwt' --eval-store auto -f /nix/store/4m7mpr04a8dziray8jhdxrxsvaq1bzrn-job.nix --argstr tag t2 2>&1 machine # [ 198.275872] niks3-server[1195]: 2026/09/23 04:45:44 INFO Received uploads request method=POST path=/api/pending_closures machine # [ 198.302787] niks3-server[1195]: 2026/09/23 04:45:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign machine # [ 198.307566] niks3-server[1195]: 2026/09/23 04:45:44 INFO Signed narinfos id=7 count=1 machine # [ 198.310934] niks3-server[1195]: 2026/09/23 04:45:44 INFO Registered completed upload object_key=nar/1fr26arhri17j9bj6hyjzmmd74v730biz1gb3kqd194vjx3ccvp9.nar.zst machine # [ 198.316254] niks3-server[1195]: 2026/09/23 04:45:44 INFO Registered completed upload object_key=pbpqsh50ar10fpasdqqx8xg3lmr5j8iz.ls machine # [ 198.320479] niks3-server[1195]: 2026/09/23 04:45:44 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete machine # [ 198.326147] niks3-server[1195]: 2026/09/23 04:45:44 INFO Completed upload id=7 machine # [ 198.328122] niks3-server[1195]: 2026/09/23 04:45:44 INFO Registered completed upload object_key=pbpqsh50ar10fpasdqqx8xg3lmr5j8iz.narinfo machine # [ 223.039813] niks3-server[1195]: 2026/09/23 04:46:09 INFO Received uploads request method=POST path=/api/pending_closures machine # [ 223.062289] niks3-server[1195]: 2026/09/23 04:46:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign machine # [ 223.065946] niks3-server[1195]: 2026/09/23 04:46:09 INFO Signed narinfos id=8 count=1 machine # [ 223.070390] niks3-server[1195]: 2026/09/23 04:46:09 INFO Registered completed upload object_key=nar/1hfc6x4x25k3b0fk091d3nb0k3mympfp05fkslcx19synp3ig0zd.nar.zst machine # [ 223.073468] niks3-server[1195]: 2026/09/23 04:46:09 INFO Registered completed upload object_key=log/pbpqsh50ar10fpasdqqx8xg3lmr5j8iz-k3s-a.drv machine # [ 223.074569] niks3-server[1195]: 2026/09/23 04:46:09 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete machine # [ 223.076700] niks3-server[1195]: 2026/09/23 04:46:09 INFO Completed upload id=8 machine # [ 223.086481] niks3-server[1195]: 2026/09/23 04:46:09 INFO Registered completed upload object_key=r1wl8yi1c55jrx53153rpsk307x9c14q.narinfo machine # [ 223.096341] niks3-server[1195]: 2026/09/23 04:46:09 INFO Registered completed upload object_key=r1wl8yi1c55jrx53153rpsk307x9c14q.ls machine # [ 223.152749] niks3-server[1195]: 2026/09/23 04:46:09 INFO Received uploads request method=POST path=/api/pending_closures machine # [ 223.164897] niks3-server[1195]: 2026/09/23 04:46:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign machine # [ 223.166162] niks3-server[1195]: 2026/09/23 04:46:09 INFO Signed narinfos id=9 count=1 machine # [ 223.171165] niks3-server[1195]: 2026/09/23 04:46:09 INFO Registered completed upload object_key=nar/0zzq2vgjln4shvh57091n14sam6wh6w06vja6lzb9ppcj4ffyx1n.nar.zst machine # [ 223.174320] niks3-server[1195]: 2026/09/23 04:46:09 INFO Registered completed upload object_key=894h6wzw2l4w5f0hikj1k6q57jfnyi87.ls machine # [ 223.178939] niks3-server[1195]: 2026/09/23 04:46:09 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete machine # [ 223.181474] niks3-server[1195]: 2026/09/23 04:46:09 INFO Completed upload id=9 machine # [ 248.096458] niks3-server[1195]: 2026/09/23 04:46:34 INFO Received uploads request method=POST path=/api/pending_closures machine # [ 248.115576] niks3-server[1195]: 2026/09/23 04:46:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign machine # [ 248.116792] niks3-server[1195]: 2026/09/23 04:46:34 INFO Signed narinfos id=10 count=1 machine # [ 248.122640] niks3-server[1195]: 2026/09/23 04:46:34 INFO Registered completed upload object_key=nar/0hh3nqk95r6g32lhdmxw8cxjv694vbpx9876zkhcpg4r4qxni2ch.nar.zst machine # [ 248.124872] niks3-server[1195]: 2026/09/23 04:46:34 INFO Registered completed upload object_key=log/894h6wzw2l4w5f0hikj1k6q57jfnyi87-k3s-b.drv machine # [ 248.127627] niks3-server[1195]: 2026/09/23 04:46:34 INFO Registered completed upload object_key=zqhpzdjr9k4lbxyx13lygsbapjhj6jbs.ls machine # [ 248.129344] niks3-server[1195]: 2026/09/23 04:46:34 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete machine # [ 248.131620] niks3-server[1195]: 2026/09/23 04:46:34 INFO Completed upload id=10 machine # [ 248.134327] niks3-server[1195]: 2026/09/23 04:46:34 INFO Registered completed upload object_key=zqhpzdjr9k4lbxyx13lygsbapjhj6jbs.narinfo machine # [ 248.204830] niks3-server[1195]: 2026/09/23 04:46:34 INFO Received uploads request method=POST path=/api/pending_closures machine # [ 248.215960] niks3-server[1195]: 2026/09/23 04:46:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign machine # [ 248.217032] niks3-server[1195]: 2026/09/23 04:46:34 INFO Signed narinfos id=11 count=1 machine # [ 248.219349] niks3-server[1195]: 2026/09/23 04:46:34 INFO Registered completed upload object_key=nar/0ingamx56k31lwiyxmnxmscjj4n3qfx6sr2kpaynsrwhwz1278iw.nar.zst machine # [ 248.221262] niks3-server[1195]: 2026/09/23 04:46:34 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete machine # [ 248.224138] niks3-server[1195]: 2026/09/23 04:46:34 INFO Completed upload id=11 machine # [ 248.226036] niks3-server[1195]: 2026/09/23 04:46:34 INFO Registered completed upload object_key=12zw7pp0zj6xgpqjxdyfrj7gbr252z3g.narinfo machine # [ 273.144294] niks3-server[1195]: 2026/09/23 04:46:59 INFO Received uploads request method=POST path=/api/pending_closures machine # [ 273.159433] niks3-server[1195]: 2026/09/23 04:46:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign machine # [ 273.162255] niks3-server[1195]: 2026/09/23 04:46:59 INFO Signed narinfos id=12 count=1 machine # [ 273.165402] niks3-server[1195]: 2026/09/23 04:46:59 INFO Registered completed upload object_key=log/12zw7pp0zj6xgpqjxdyfrj7gbr252z3g-k3s-top.drv machine # [ 273.167282] niks3-server[1195]: 2026/09/23 04:46:59 INFO Registered completed upload object_key=5nkgigdq00qnxqr4vvknpjyrrz7d5rih.ls machine # [ 273.168519] niks3-server[1195]: 2026/09/23 04:46:59 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete machine # [ 273.172110] niks3-server[1195]: 2026/09/23 04:46:59 INFO Completed upload id=12 machine # [ 273.175721] niks3-server[1195]: 2026/09/23 04:46:59 INFO Registered completed upload object_key=5nkgigdq00qnxqr4vvknpjyrrz7d5rih.narinfo machine # [ 273.187652] niks3-server[1195]: 2026/09/23 04:46:59 INFO Registered completed upload object_key=nar/0v48x4sp58bvfra08axgkz7yl1n9nc4nirskyv1hwann4i7q277i.nar.zst machine: (finished: must succeed: nix build -L --store 'grpc://127.0.0.1:30051?ca-cert=/nix/store/b2prb199a55q32qawjagqrylwrml5g74-k3s-farm-certs/ca.crt&token-file=/tmp/builder.jwt' --eval-store auto -f /nix/store/4m7mpr04a8dziray8jhdxrxsvaq1bzrn-job.nix --argstr tag t2 2>&1, in 103.30 seconds) (finished: subtest: build through the balancer lands in the cache, in 211.74 seconds) subtest: a Job with the client image builds through the farm with its service account machine: must succeed: kubectl -n ci create configmap nix-grpc-farm-ca --from-file=ca.crt=/nix/store/b2prb199a55q32qawjagqrylwrml5g74-k3s-farm-certs/ca.crt machine: (finished: must succeed: kubectl -n ci create configmap nix-grpc-farm-ca --from-file=ca.crt=/nix/store/b2prb199a55q32qawjagqrylwrml5g74-k3s-farm-certs/ca.crt, in 0.14 seconds) machine: must succeed: kubectl -n ci create configmap job --from-file=job.nix=/nix/store/4m7mpr04a8dziray8jhdxrxsvaq1bzrn-job.nix --from-file=flake.nix=/nix/store/x8s8k9cql05dy59sjsy1ih3fnyg9zksh-flake.nix machine: (finished: must succeed: kubectl -n ci create configmap job --from-file=job.nix=/nix/store/4m7mpr04a8dziray8jhdxrxsvaq1bzrn-job.nix --from-file=flake.nix=/nix/store/x8s8k9cql05dy59sjsy1ih3fnyg9zksh-flake.nix, in 0.13 seconds) machine: must succeed: kubectl apply -f /nix/store/lp0q6rjq0cw1vs9npm6c3ggx088f4vva-ci-job.json machine: (finished: must succeed: kubectl apply -f /nix/store/lp0q6rjq0cw1vs9npm6c3ggx088f4vva-ci-job.json, in 0.16 seconds) machine: waiting for success: kubectl -n ci get job ci -o jsonpath='{.status.conditions[*].type}' | grep -qE 'Complete|Failed' machine # [ 273.755325] systemd[1]: Created slice libcontainer container kubepods-besteffort-pod789d9f64_d594_40cd_aa7d_afa1d0f2e5f2.slice. machine # [ 273.852387] k3s[894]: I0923 04:47:00.399982 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nix-grpc-farm\" (UniqueName: \"kubernetes.io/projected/789d9f64-d594-40cd-aa7d-afa1d0f2e5f2-nix-grpc-farm\") pod \"ci-2nddx\" (UID: \"789d9f64-d594-40cd-aa7d-afa1d0f2e5f2\") " pod="ci/ci-2nddx" machine # [ 273.853177] k3s[894]: I0923 04:47:00.400054 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q4z9\" (UniqueName: \"kubernetes.io/projected/789d9f64-d594-40cd-aa7d-afa1d0f2e5f2-kube-api-access-7q4z9\") pod \"ci-2nddx\" (UID: \"789d9f64-d594-40cd-aa7d-afa1d0f2e5f2\") " pod="ci/ci-2nddx" machine # [ 273.853785] k3s[894]: I0923 04:47:00.400126 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"job\" (UniqueName: \"kubernetes.io/configmap/789d9f64-d594-40cd-aa7d-afa1d0f2e5f2-job\") pod \"ci-2nddx\" (UID: \"789d9f64-d594-40cd-aa7d-afa1d0f2e5f2\") " pod="ci/ci-2nddx" machine # [ 275.154874] cni0: port 1(veth1a9a0450) entered blocking state machine # [ 275.157445] cni0: port 1(veth1a9a0450) entered disabled state machine # [ 275.160232] veth1a9a0450: entered allmulticast mode machine # [ 275.161028] veth1a9a0450: entered promiscuous mode machine # [ 275.174493] cni0: port 1(veth1a9a0450) entered blocking state machine # [ 275.176215] cni0: port 1(veth1a9a0450) entered forwarding state machine # [ 275.060376] (udev-worker)[4956]: Network interface NamePolicy= disabled on kernel command line. machine # [ 275.093610] dhcpcd[3492]: veth1a9a0450: IAID f1:27:cd:06 machine # [ 275.094763] dhcpcd[3492]: veth1a9a0450: adding address fe80::48bc:f1ff:fe27:cd06 machine # [ 275.122572] systemd[1]: Started libcontainer container d1d0b9310c21e6aa9796d40c4cf592fd0bf4d8c5938aecebf052fd52f6c85cde. machine # [ 275.800985] dhcpcd[3492]: veth1a9a0450: soliciting a DHCP lease machine # [ 277.296779] systemd[1]: var-lib-rancher-k3s-agent-containerd-tmpmounts-containerd\x2dmount3014286334.mount: Deactivated successfully. machine # [ 277.357324] systemd[1]: Started libcontainer container d7ba5e5e21efabd098be197ec003d9b75535811458b4ff6765d24c18d92e70dd. machine # [ 277.375253] dhcpcd[3492]: veth1a9a0450: soliciting an IPv6 router machine # [ 277.797996] k3s[894]: I0923 04:47:04.345041 894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ci/ci-2nddx" podStartSLOduration=4.344997028 podStartE2EDuration="4.344997028s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-23 04:47:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-23 04:47:04.3437952 +0000 UTC m=+259.451215978" watchObservedRunningTime="2026-09-23 04:47:04.344997028 +0000 UTC m=+259.452417248" machine # [ 278.669703] niks3-server[1195]: 2026/09/23 04:47:05 INFO Received uploads request method=POST path=/api/pending_closures machine # [ 278.732836] niks3-server[1195]: 2026/09/23 04:47:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign machine # [ 278.734444] niks3-server[1195]: 2026/09/23 04:47:05 INFO Signed narinfos id=13 count=1 machine # [ 278.737124] niks3-server[1195]: 2026/09/23 04:47:05 INFO Registered completed upload object_key=fb6zgh20gn6ix96sz5fv2gzdaffbjhws.ls machine # [ 278.739789] niks3-server[1195]: 2026/09/23 04:47:05 INFO Registered completed upload object_key=nar/111g6dyibwcf2z7jhpn72rgja0qvjgpdiv9nc06g2dp7ih0a6kyq.nar.zst machine # [ 278.751480] niks3-server[1195]: 2026/09/23 04:47:05 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete machine # [ 278.754364] niks3-server[1195]: 2026/09/23 04:47:05 INFO Completed upload id=13 machine # [ 280.802169] dhcpcd[3492]: veth1a9a0450: probing for an IPv4LL address machine # [ 285.296148] dhcpcd[3492]: veth1a9a0450: using IPv4LL address 169.254.191.60 machine # [ 285.298773] dhcpcd[3492]: veth1a9a0450: adding route to 169.254.0.0/16 machine # [ 289.379883] dhcpcd[3492]: veth1a9a0450: no IPv6 Routers available machine # [ 303.269514] niks3-server[1195]: 2026/09/23 04:47:29 INFO Received uploads request method=POST path=/api/pending_closures machine # [ 303.284855] niks3-server[1195]: 2026/09/23 04:47:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign machine # [ 303.287745] niks3-server[1195]: 2026/09/23 04:47:29 INFO Signed narinfos id=14 count=1 machine # [ 303.291176] niks3-server[1195]: 2026/09/23 04:47:29 INFO Registered completed upload object_key=gpvd12kzis7szd8w51klc50y5fsyav70.ls machine # [ 303.294291] niks3-server[1195]: 2026/09/23 04:47:29 INFO Registered completed upload object_key=nar/058kjd2vfl1r27b4d9ixvwf77d8rkr18a4v894kwx8lg0nwpajnr.nar.zst machine # [ 303.297437] niks3-server[1195]: 2026/09/23 04:47:29 INFO Registered completed upload object_key=log/fb6zgh20gn6ix96sz5fv2gzdaffbjhws-k3s-a.drv machine # [ 303.299644] niks3-server[1195]: 2026/09/23 04:47:29 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete machine # [ 303.302521] niks3-server[1195]: 2026/09/23 04:47:29 INFO Completed upload id=14 machine # [ 303.304169] niks3-server[1195]: 2026/09/23 04:47:29 INFO Registered completed upload object_key=gpvd12kzis7szd8w51klc50y5fsyav70.narinfo machine # [ 303.375694] niks3-server[1195]: 2026/09/23 04:47:29 INFO Received uploads request method=POST path=/api/pending_closures machine # [ 303.387075] niks3-server[1195]: 2026/09/23 04:47:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign machine # [ 303.387767] niks3-server[1195]: 2026/09/23 04:47:29 INFO Signed narinfos id=15 count=1 machine # [ 303.392679] niks3-server[1195]: 2026/09/23 04:47:29 INFO Registered completed upload object_key=1fq6z4j3ry021a66ia281js0yid1pi8j.ls machine # [ 303.394248] niks3-server[1195]: 2026/09/23 04:47:29 INFO Registered completed upload object_key=nar/138ynhcd5wxzz485x9qcq7xsargaklrf2li31ai69m5r46am35q6.nar.zst machine # [ 303.394865] niks3-server[1195]: 2026/09/23 04:47:29 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete machine # [ 303.398277] niks3-server[1195]: 2026/09/23 04:47:29 INFO Completed upload id=15 machine # [ 303.399894] niks3-server[1195]: 2026/09/23 04:47:29 INFO Registered completed upload object_key=1fq6z4j3ry021a66ia281js0yid1pi8j.narinfo machine # [ 310.519652] postgres[765]: [765] LOG: checkpoint starting: time machine # [ 327.916953] niks3-server[1195]: 2026/09/23 04:47:54 INFO Received uploads request method=POST path=/api/pending_closures machine # [ 327.937539] niks3-server[1195]: 2026/09/23 04:47:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign machine # [ 327.943280] niks3-server[1195]: 2026/09/23 04:47:54 INFO Signed narinfos id=16 count=1 machine # [ 327.947742] niks3-server[1195]: 2026/09/23 04:47:54 INFO Registered completed upload object_key=6d6fywl171qq44fqg0h0gzya2y3sml0p.ls machine # [ 327.952156] niks3-server[1195]: 2026/09/23 04:47:54 INFO Registered completed upload object_key=nar/0n4lbdlp8w72fs3qbavl2lwrzbrn55i2b1vyglc89cc3pi8vbb1m.nar.zst machine # [ 327.953823] niks3-server[1195]: 2026/09/23 04:47:54 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete machine # [ 327.955059] niks3-server[1195]: 2026/09/23 04:47:54 INFO Registered completed upload object_key=log/1fq6z4j3ry021a66ia281js0yid1pi8j-k3s-b.drv machine # [ 327.956575] niks3-server[1195]: 2026/09/23 04:47:54 INFO Completed upload id=16 machine # [ 327.957781] niks3-server[1195]: 2026/09/23 04:47:54 INFO Registered completed upload object_key=6d6fywl171qq44fqg0h0gzya2y3sml0p.narinfo machine # [ 328.035496] niks3-server[1195]: 2026/09/23 04:47:54 INFO Received uploads request method=POST path=/api/pending_closures machine # [ 328.045554] niks3-server[1195]: 2026/09/23 04:47:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign machine # [ 328.046465] niks3-server[1195]: 2026/09/23 04:47:54 INFO Signed narinfos id=17 count=1 machine # [ 328.050153] niks3-server[1195]: 2026/09/23 04:47:54 INFO Registered completed upload object_key=nar/1f3gvnvqsm2pm3laixkxkg44kvjb251zqhszcfpp0a00h6x66ljs.nar.zst machine # [ 328.052481] niks3-server[1195]: 2026/09/23 04:47:54 INFO Registered completed upload object_key=rccnai4l0r1y75cnyfrxd9xk26kkd7lg.ls machine # [ 328.056836] niks3-server[1195]: 2026/09/23 04:47:54 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete machine # [ 328.058356] niks3-server[1195]: 2026/09/23 04:47:54 INFO Completed upload id=17 machine # [ 352.372316] niks3-server[1195]: 2026/09/23 04:48:18 INFO Received uploads request method=POST path=/api/pending_closures machine # [ 352.390408] niks3-server[1195]: 2026/09/23 04:48:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign machine # [ 352.394681] niks3-server[1195]: 2026/09/23 04:48:18 INFO Signed narinfos id=18 count=1 machine # [ 352.397966] niks3-server[1195]: 2026/09/23 04:48:18 INFO Registered completed upload object_key=nar/1zwf0asrdv12q40fmqga5lx5330dvn9y4in4i9ficz9ij5lhvnf5.nar.zst machine # [ 352.403246] niks3-server[1195]: 2026/09/23 04:48:18 INFO Registered completed upload object_key=log/rccnai4l0r1y75cnyfrxd9xk26kkd7lg-k3s-top.drv machine # [ 352.407680] niks3-server[1195]: 2026/09/23 04:48:18 INFO Registered completed upload object_key=a7wc2s1svbimnjgrgl4hkq86pi15mpjn.ls machine # [ 352.411962] niks3-server[1195]: 2026/09/23 04:48:18 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete machine # [ 352.416512] niks3-server[1195]: 2026/09/23 04:48:18 INFO Completed upload id=18 machine # [ 352.419710] niks3-server[1195]: 2026/09/23 04:48:18 INFO Registered completed upload object_key=a7wc2s1svbimnjgrgl4hkq86pi15mpjn.narinfo machine # [ 354.150484] niks3-server[1195]: 2026/09/23 04:48:20 INFO Received uploads request method=POST path=/api/pending_closures machine # [ 354.167830] niks3-server[1195]: 2026/09/23 04:48:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign machine # [ 354.171098] niks3-server[1195]: 2026/09/23 04:48:20 INFO Signed narinfos id=19 count=1 machine # [ 354.177743] niks3-server[1195]: 2026/09/23 04:48:20 INFO Registered completed upload object_key=nar/0xiyqcsl8db7dnrq2gqnx5izavr51ma3dcdswrriaxfgj3qzk7xf.nar.zst machine # [ 354.181697] niks3-server[1195]: 2026/09/23 04:48:20 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete machine # [ 354.184466] niks3-server[1195]: 2026/09/23 04:48:20 INFO Registered completed upload object_key=fm28nk4zj5837pwx4c8iv6ds8919r62h.ls machine # [ 354.187110] niks3-server[1195]: 2026/09/23 04:48:20 INFO Completed upload id=19 machine # [ 354.194631] niks3-server[1195]: 2026/09/23 04:48:20 INFO Registered completed upload object_key=fm28nk4zj5837pwx4c8iv6ds8919r62h.narinfo machine # [ 378.895772] niks3-server[1195]: 2026/09/23 04:48:45 INFO Received uploads request method=POST path=/api/pending_closures machine # [ 378.913709] niks3-server[1195]: 2026/09/23 04:48:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign machine # [ 378.914920] niks3-server[1195]: 2026/09/23 04:48:45 INFO Registered completed upload object_key=pg3nlwkwpr2na0lv1p4pqqrk1s647g16.ls machine # [ 378.916660] niks3-server[1195]: 2026/09/23 04:48:45 INFO Registered completed upload object_key=nar/0p3l2z3riilylr2rxb1aq2blxdfxg55wbgp4gxkns93srbr4bwyq.nar.zst machine # [ 378.919891] niks3-server[1195]: 2026/09/23 04:48:45 INFO Registered completed upload object_key=log/fm28nk4zj5837pwx4c8iv6ds8919r62h-nfb-ci.drv machine # [ 378.922228] niks3-server[1195]: 2026/09/23 04:48:45 INFO Signed narinfos id=20 count=1 machine # [ 378.927768] niks3-server[1195]: 2026/09/23 04:48:45 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete machine # [ 378.930205] niks3-server[1195]: 2026/09/23 04:48:45 INFO Completed upload id=20 machine # [ 378.932190] niks3-server[1195]: 2026/09/23 04:48:45 INFO Registered completed upload object_key=pg3nlwkwpr2na0lv1p4pqqrk1s647g16.narinfo machine # [ 380.105771] systemd[1]: cri-containerd-d7ba5e5e21efabd098be197ec003d9b75535811458b4ff6765d24c18d92e70dd.scope: Deactivated successfully. machine # [ 380.109093] systemd[1]: cri-containerd-d7ba5e5e21efabd098be197ec003d9b75535811458b4ff6765d24c18d92e70dd.scope: Consumed 1.060s CPU time over 1min 42.749s wall clock time, 104.2M memory peak, 34.4M read from disk, 460K written to disk, 18.2K incoming IP traffic, 44.7K outgoing IP traffic. machine # [ 380.167836] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-d7ba5e5e21efabd098be197ec003d9b75535811458b4ff6765d24c18d92e70dd-rootfs.mount: Deactivated successfully. machine # [ 381.242679] systemd[1]: cri-containerd-d1d0b9310c21e6aa9796d40c4cf592fd0bf4d8c5938aecebf052fd52f6c85cde.scope: Deactivated successfully. machine # [ 381.280123] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-d1d0b9310c21e6aa9796d40c4cf592fd0bf4d8c5938aecebf052fd52f6c85cde-rootfs.mount: Deactivated successfully. machine # [ 381.308915] systemd[1]: run-k3s-containerd-io.containerd.grpc.v1.cri-sandboxes-d1d0b9310c21e6aa9796d40c4cf592fd0bf4d8c5938aecebf052fd52f6c85cde-shm.mount: Deactivated successfully. machine # [ 381.494916] cni0: port 1(veth1a9a0450) entered disabled state machine # [ 381.341694] dhcpcd[3492]: veth1a9a0450: carrier lost machine # [ 381.497409] veth1a9a0450 (unregistering): left allmulticast mode machine # [ 381.499415] veth1a9a0450 (unregistering): left promiscuous mode machine # [ 381.500737] cni0: port 1(veth1a9a0450) entered disabled state machine # [ 381.366154] dhcpcd[3492]: veth1a9a0450: deleting address fe80::48bc:f1ff:fe27:cd06 machine # [ 381.369220] dhcpcd[3492]: veth1a9a0450: deleting route to 169.254.0.0/16 machine # [ 381.370533] systemd[1]: run-netns-cni\x2d9b47d14e\x2def53\x2d4d82\x2d078d\x2d6cd16282c794.mount: Deactivated successfully. machine # [ 381.435359] dhcpcd[3492]: veth1a9a0450: removing interface machine # [ 381.520895] k3s[894]: I0923 04:48:48.068471 894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/789d9f64-d594-40cd-aa7d-afa1d0f2e5f2-nix-grpc-farm\" (UniqueName: \"kubernetes.io/projected/789d9f64-d594-40cd-aa7d-afa1d0f2e5f2-nix-grpc-farm\") pod \"789d9f64-d594-40cd-aa7d-afa1d0f2e5f2\" (UID: \"789d9f64-d594-40cd-aa7d-afa1d0f2e5f2\") " machine # [ 381.521798] k3s[894]: I0923 04:48:48.068545 894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/789d9f64-d594-40cd-aa7d-afa1d0f2e5f2-job\" (UniqueName: \"kubernetes.io/configmap/789d9f64-d594-40cd-aa7d-afa1d0f2e5f2-job\") pod \"789d9f64-d594-40cd-aa7d-afa1d0f2e5f2\" (UID: \"789d9f64-d594-40cd-aa7d-afa1d0f2e5f2\") " machine # [ 381.522621] k3s[894]: I0923 04:48:48.068595 894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/789d9f64-d594-40cd-aa7d-afa1d0f2e5f2-kube-api-access-7q4z9\" (UniqueName: \"kubernetes.io/projected/789d9f64-d594-40cd-aa7d-afa1d0f2e5f2-kube-api-access-7q4z9\") pod \"789d9f64-d594-40cd-aa7d-afa1d0f2e5f2\" (UID: \"789d9f64-d594-40cd-aa7d-afa1d0f2e5f2\") " machine # [ 381.523309] k3s[894]: I0923 04:48:48.069197 894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/789d9f64-d594-40cd-aa7d-afa1d0f2e5f2-job" pod "789d9f64-d594-40cd-aa7d-afa1d0f2e5f2" (UID: "789d9f64-d594-40cd-aa7d-afa1d0f2e5f2"). InnerVolumeSpecName "job". PluginName "kubernetes.io/configmap", VolumeGIDValue "" machine # [ 381.528598] systemd[1]: var-lib-kubelet-pods-789d9f64\x2dd594\x2d40cd\x2daa7d\x2dafa1d0f2e5f2-volumes-kubernetes.io\x7eprojected-nix\x2dgrpc\x2dfarm.mount: Deactivated successfully. machine # [ 381.529539] systemd[1]: var-lib-kubelet-pods-789d9f64\x2dd594\x2d40cd\x2daa7d\x2dafa1d0f2e5f2-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2d7q4z9.mount: Deactivated successfully. machine # [ 381.543228] k3s[894]: I0923 04:48:48.091203 894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/789d9f64-d594-40cd-aa7d-afa1d0f2e5f2-nix-grpc-farm" pod "789d9f64-d594-40cd-aa7d-afa1d0f2e5f2" (UID: "789d9f64-d594-40cd-aa7d-afa1d0f2e5f2"). InnerVolumeSpecName "nix-grpc-farm". PluginName "kubernetes.io/projected", VolumeGIDValue "" machine # [ 381.547372] k3s[894]: I0923 04:48:48.091264 894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/789d9f64-d594-40cd-aa7d-afa1d0f2e5f2-kube-api-access-7q4z9" pod "789d9f64-d594-40cd-aa7d-afa1d0f2e5f2" (UID: "789d9f64-d594-40cd-aa7d-afa1d0f2e5f2"). InnerVolumeSpecName "kube-api-access-7q4z9". PluginName "kubernetes.io/projected", VolumeGIDValue "" machine # [ 381.590935] systemd[1]: Removed slice libcontainer container kubepods-besteffort-pod789d9f64_d594_40cd_aa7d_afa1d0f2e5f2.slice. machine # [ 381.592749] systemd[1]: kubepods-besteffort-pod789d9f64_d594_40cd_aa7d_afa1d0f2e5f2.slice: Consumed 1.091s CPU time over 1min 47.835s wall clock time, 104.4M memory peak, 34.4M read from disk, 664K written to disk, 18.2K incoming IP traffic, 44.7K outgoing IP traffic. machine # [ 381.620825] k3s[894]: I0923 04:48:48.168808 894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7q4z9\" (UniqueName: \"kubernetes.io/projected/789d9f64-d594-40cd-aa7d-afa1d0f2e5f2-kube-api-access-7q4z9\") on node \"machine\" DevicePath \"\"" machine # [ 381.621550] k3s[894]: I0923 04:48:48.168860 894 reconciler_common.go:299] "Volume detached for volume \"nix-grpc-farm\" (UniqueName: \"kubernetes.io/projected/789d9f64-d594-40cd-aa7d-afa1d0f2e5f2-nix-grpc-farm\") on node \"machine\" DevicePath \"\"" machine # [ 381.622201] k3s[894]: I0923 04:48:48.168882 894 reconciler_common.go:299] "Volume detached for volume \"job\" (UniqueName: \"kubernetes.io/configmap/789d9f64-d594-40cd-aa7d-afa1d0f2e5f2-job\") on node \"machine\" DevicePath \"\"" machine # [ 382.232712] k3s[894]: I0923 04:48:48.780624 894 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d1d0b9310c21e6aa9796d40c4cf592fd0bf4d8c5938aecebf052fd52f6c85cde" machine: (finished: waiting for success: kubectl -n ci get job ci -o jsonpath='{.status.conditions[*].type}' | grep -qE 'Complete|Failed', in 108.89 seconds) machine: must succeed: kubectl -n ci get job ci -o jsonpath='{.status.succeeded}' | grep -qx 1 machine: (finished: must succeed: kubectl -n ci get job ci -o jsonpath='{.status.succeeded}' | grep -qx 1, in 0.20 seconds) machine # don't know how to build these paths: machine # /nix/store/rccnai4l0r1y75cnyfrxd9xk26kkd7lg-k3s-top.drv machine # copying 1 paths to 'nix-grpc-farm.farm.svc:50051'... machine # copying path '/nix/store/fb6zgh20gn6ix96sz5fv2gzdaffbjhws-k3s-a.drv' to 'nix-grpc-farm.farm.svc:50051'... machine # building '/nix/store/fb6zgh20gn6ix96sz5fv2gzdaffbjhws-k3s-a.drv' on nix-grpc-farm.farm.svc:50051... machine # k3s-a> machine/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2: building fb6zgh20gn6ix96sz5fv2gzdaffbjhws-k3s-a.drv machine # copying 1 paths to 'nix-grpc-farm.farm.svc:50051'... machine # copying path '/nix/store/1fq6z4j3ry021a66ia281js0yid1pi8j-k3s-b.drv' to 'nix-grpc-farm.farm.svc:50051'... machine # building '/nix/store/1fq6z4j3ry021a66ia281js0yid1pi8j-k3s-b.drv' on nix-grpc-farm.farm.svc:50051... machine # k3s-b> machine/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2: building 1fq6z4j3ry021a66ia281js0yid1pi8j-k3s-b.drv machine # copying 1 paths to 'nix-grpc-farm.farm.svc:50051'... machine # copying path '/nix/store/rccnai4l0r1y75cnyfrxd9xk26kkd7lg-k3s-top.drv' to 'nix-grpc-farm.farm.svc:50051'... machine # building '/nix/store/rccnai4l0r1y75cnyfrxd9xk26kkd7lg-k3s-top.drv' on nix-grpc-farm.farm.svc:50051... machine # k3s-top> machine/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2: building rccnai4l0r1y75cnyfrxd9xk26kkd7lg-k3s-top.drv machine # INFO:nix_fast_build.processes:run nix-eval-jobs --gc-roots-dir /tmp/tmpsipouop7/gcroots --force-recurse --max-memory-size 4096 --workers 4 --flake '.#checks' --check-cache-status machine # INFO:nix_fast_build.workers:builds: 0 machine # warning: Git tree '/tmp' is dirty machine # warning: Git tree '/tmp' is dirty machine # ▶ x86_64-linux.t started machine # INFO:nix_fast_build.workers:builds: 2 machine # ✔ x86_64-linux.t (27s) machine # x86_64-linux.t> machine/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-qlcdh: building fm28nk4zj5837pwx4c8iv6ds8919r62h-nfb-ci.drv machine # warning: Git tree '/tmp' is dirty machine: must succeed: nix eval --raw -f /nix/store/4m7mpr04a8dziray8jhdxrxsvaq1bzrn-job.nix --argstr tag ci outPath machine: (finished: must succeed: nix eval --raw -f /nix/store/4m7mpr04a8dziray8jhdxrxsvaq1bzrn-job.nix --argstr tag ci outPath, in 0.16 seconds) machine: must succeed: curl -sf http://192.168.1.1:5751/a7wc2s1svbimnjgrgl4hkq86pi15mpjn.narinfo > /dev/null machine: (finished: must succeed: curl -sf http://192.168.1.1:5751/a7wc2s1svbimnjgrgl4hkq86pi15mpjn.narinfo > /dev/null, in 0.04 seconds) (finished: subtest: a Job with the client image builds through the farm with its service account, in 109.99 seconds) subtest: a renewed balancer certificate is served without a rollout machine: must succeed: kubectl -n farm create secret tls farm-lb-tls --cert=/nix/store/b2prb199a55q32qawjagqrylwrml5g74-k3s-farm-certs/lb-renewed.crt --key=/nix/store/b2prb199a55q32qawjagqrylwrml5g74-k3s-farm-certs/lb-renewed.key --dry-run=client -o yaml | kubectl -n farm apply -f - machine # Warning: resource secrets/farm-lb-tls is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically. machine: (finished: must succeed: kubectl -n farm create secret tls farm-lb-tls --cert=/nix/store/b2prb199a55q32qawjagqrylwrml5g74-k3s-farm-certs/lb-renewed.crt --key=/nix/store/b2prb199a55q32qawjagqrylwrml5g74-k3s-farm-certs/lb-renewed.key --dry-run=client -o yaml | kubectl -n farm apply -f -, in 0.29 seconds) machine: must succeed: kubectl -n farm get pods -l app.kubernetes.io/component=lb -o jsonpath='{.items[0].metadata.name}' machine: (finished: must succeed: kubectl -n farm get pods -l app.kubernetes.io/component=lb -o jsonpath='{.items[0].metadata.name}', in 0.17 seconds) machine: waiting for success: kubectl -n farm exec nix-grpc-farm-lb-775dbb5c7c-mwvmc -- cat /etc/envoy/tls/lb/tls.crt | openssl x509 -noout -ext subjectAltName | grep -q renewed.example machine # [ 383.897308] k3s[894]: E0923 04:48:50.444854 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:50988: use of closed network connection machine # [ 385.170241] k3s[894]: E0923 04:48:51.717819 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:51010: use of closed network connection machine # [ 386.483028] k3s[894]: E0923 04:48:53.030371 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:51040: use of closed network connection machine # [ 387.813629] k3s[894]: E0923 04:48:54.361334 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:51078: use of closed network connection machine # [ 389.126504] k3s[894]: E0923 04:48:55.673865 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:51110: use of closed network connection machine # [ 390.426827] k3s[894]: E0923 04:48:56.974521 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:51130: use of closed network connection machine # [ 391.757417] k3s[894]: E0923 04:48:58.304647 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:51150: use of closed network connection machine # [ 393.060618] k3s[894]: E0923 04:48:59.607905 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:60054: use of closed network connection machine # [ 394.378027] k3s[894]: E0923 04:49:00.925209 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:60064: use of closed network connection machine # [ 395.666785] k3s[894]: E0923 04:49:02.214461 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:60090: use of closed network connection machine # [ 396.944465] k3s[894]: E0923 04:49:03.491969 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:60108: use of closed network connection machine # [ 398.252569] k3s[894]: E0923 04:49:04.799882 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:60146: use of closed network connection machine # [ 399.549745] k3s[894]: E0923 04:49:06.096953 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:60174: use of closed network connection machine # [ 400.854656] k3s[894]: E0923 04:49:07.402238 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:60212: use of closed network connection machine # [ 402.140466] k3s[894]: time="2026-09-23T04:49:08Z" level=warning msg="Proxy error: write failed: write tcp 127.0.0.1:34294->127.0.0.1:10250: write: broken pipe" machine # [ 402.141552] k3s[894]: E0923 04:49:08.687953 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:60248: use of closed network connection machine # [ 403.434545] k3s[894]: E0923 04:49:09.981969 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:53524: use of closed network connection machine # [ 404.747197] k3s[894]: E0923 04:49:11.294715 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:53550: use of closed network connection machine # [ 406.024939] k3s[894]: E0923 04:49:12.572443 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:53586: use of closed network connection machine # [ 407.327193] k3s[894]: E0923 04:49:13.874435 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:53618: use of closed network connection machine # [ 408.650389] k3s[894]: E0923 04:49:15.197398 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:53644: use of closed network connection machine: (finished: waiting for success: kubectl -n farm exec nix-grpc-farm-lb-775dbb5c7c-mwvmc -- cat /etc/envoy/tls/lb/tls.crt | openssl x509 -noout -ext subjectAltName | grep -q renewed.example, in 25.00 seconds) machine: waiting for success: openssl s_client -connect 127.0.0.1:30051 /dev/null | openssl x509 -noout -ext subjectAltName | grep -q renewed.example machine: (finished: waiting for success: openssl s_client -connect 127.0.0.1:30051 /dev/null | openssl x509 -noout -ext subjectAltName | grep -q renewed.example, in 0.05 seconds) machine: must succeed: nix build -L --store 'grpc://127.0.0.1:30051?ca-cert=/nix/store/b2prb199a55q32qawjagqrylwrml5g74-k3s-farm-certs/ca.crt&token-file=/tmp/builder.jwt' --eval-store auto -f /nix/store/4m7mpr04a8dziray8jhdxrxsvaq1bzrn-job.nix --argstr tag t-renewed 2>&1 machine # [ 408.900309] niks3-server[1195]: 2026/09/23 04:49:15 INFO Received uploads request method=POST path=/api/pending_closures machine # [ 408.916526] niks3-server[1195]: 2026/09/23 04:49:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign machine # [ 408.918078] niks3-server[1195]: 2026/09/23 04:49:15 INFO Signed narinfos id=21 count=1 machine # [ 408.923937] niks3-server[1195]: 2026/09/23 04:49:15 INFO Registered completed upload object_key=0hkdl0zrqkc6m0v0jc2gzmsjsynmsw31.ls machine # [ 408.926819] niks3-server[1195]: 2026/09/23 04:49:15 INFO Registered completed upload object_key=nar/0g42hgwa9jp9abmqxz90dkwxpybsznzbpsy35rm921mlk5sw56jq.nar.zst machine # [ 408.929401] niks3-server[1195]: 2026/09/23 04:49:15 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete machine # [ 408.931543] niks3-server[1195]: 2026/09/23 04:49:15 INFO Completed upload id=21 machine # [ 408.933195] niks3-server[1195]: 2026/09/23 04:49:15 INFO Registered completed upload object_key=0hkdl0zrqkc6m0v0jc2gzmsjsynmsw31.narinfo machine # [ 414.844276] postgres[765]: [765] LOG: checkpoint complete: wrote 1037 buffers (6.3%), wrote 3 SLRU buffers; 1 WAL file(s) added, 0 removed, 0 recycled; write=104.223 s, sync=0.063 s, total=104.325 s; sync files=364, longest=0.004 s, average=0.001 s; distance=4978 kB, estimate=4978 kB; lsn=0/1C72298, redo lsn=0/1C42A80 machine # [ 433.651886] niks3-server[1195]: 2026/09/23 04:49:40 INFO Received uploads request method=POST path=/api/pending_closures machine # [ 433.670818] niks3-server[1195]: 2026/09/23 04:49:40 INFO Registered completed upload object_key=log/0hkdl0zrqkc6m0v0jc2gzmsjsynmsw31-k3s-a.drv machine # [ 433.672568] niks3-server[1195]: 2026/09/23 04:49:40 INFO Registered completed upload object_key=nar/1vxmmzwi16s4iri6dcmi9v62dpq1ys7wz15b497ncik3sd67nfzg.nar.zst machine # [ 433.680154] niks3-server[1195]: 2026/09/23 04:49:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign machine # [ 433.682343] niks3-server[1195]: 2026/09/23 04:49:40 INFO Signed narinfos id=22 count=1 machine # [ 433.687232] niks3-server[1195]: 2026/09/23 04:49:40 INFO Registered completed upload object_key=x1rn55sjk6z5bdv1lzma81ydfwp93qzx.ls machine # [ 433.692283] niks3-server[1195]: 2026/09/23 04:49:40 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete machine # [ 433.696533] niks3-server[1195]: 2026/09/23 04:49:40 INFO Completed upload id=22 machine # [ 433.698299] niks3-server[1195]: 2026/09/23 04:49:40 INFO Registered completed upload object_key=x1rn55sjk6z5bdv1lzma81ydfwp93qzx.narinfo machine # [ 433.763834] niks3-server[1195]: 2026/09/23 04:49:40 INFO Received uploads request method=POST path=/api/pending_closures machine # [ 433.775153] niks3-server[1195]: 2026/09/23 04:49:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign machine # [ 433.776074] niks3-server[1195]: 2026/09/23 04:49:40 INFO Signed narinfos id=23 count=1 machine # [ 433.779438] niks3-server[1195]: 2026/09/23 04:49:40 INFO Registered completed upload object_key=nar/0cbxkyxz068xc5mivh2h67kw6y01n1x28cbs11v1a03g8m4c71zm.nar.zst machine # [ 433.780714] niks3-server[1195]: 2026/09/23 04:49:40 INFO Registered completed upload object_key=rymqqkkn6wq4pbxaiigc5qw0cnd1mjw4.ls machine # [ 433.783212] niks3-server[1195]: 2026/09/23 04:49:40 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete machine # [ 433.785449] niks3-server[1195]: 2026/09/23 04:49:40 INFO Completed upload id=23 machine # [ 433.788138] niks3-server[1195]: 2026/09/23 04:49:40 INFO Registered completed upload object_key=rymqqkkn6wq4pbxaiigc5qw0cnd1mjw4.narinfo machine # [ 458.708344] niks3-server[1195]: 2026/09/23 04:50:05 INFO Received uploads request method=POST path=/api/pending_closures machine # [ 458.729500] niks3-server[1195]: 2026/09/23 04:50:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign machine # [ 458.734234] niks3-server[1195]: 2026/09/23 04:50:05 INFO Signed narinfos id=24 count=1 machine # [ 458.737300] niks3-server[1195]: 2026/09/23 04:50:05 INFO Registered completed upload object_key=nar/0khxywxb2r78vhpc5hsmha3ymgp910z0aka7550jmaqzs333rmzp.nar.zst machine # [ 458.741436] niks3-server[1195]: 2026/09/23 04:50:05 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete machine # [ 458.745850] niks3-server[1195]: 2026/09/23 04:50:05 INFO Registered completed upload object_key=gxh64g2i0y0n37wabfxp5k8jr79mvikp.ls machine # [ 458.747816] niks3-server[1195]: 2026/09/23 04:50:05 INFO Completed upload id=24 machine # [ 458.749896] niks3-server[1195]: 2026/09/23 04:50:05 INFO Registered completed upload object_key=log/rymqqkkn6wq4pbxaiigc5qw0cnd1mjw4-k3s-b.drv machine # [ 458.753247] niks3-server[1195]: 2026/09/23 04:50:05 INFO Registered completed upload object_key=gxh64g2i0y0n37wabfxp5k8jr79mvikp.narinfo machine # [ 458.822328] niks3-server[1195]: 2026/09/23 04:50:05 INFO Received uploads request method=POST path=/api/pending_closures machine # [ 458.830935] niks3-server[1195]: 2026/09/23 04:50:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign machine # [ 458.831796] niks3-server[1195]: 2026/09/23 04:50:05 INFO Signed narinfos id=25 count=1 machine # [ 458.835505] niks3-server[1195]: 2026/09/23 04:50:05 INFO Registered completed upload object_key=jz7biafi8rj5r3r392blyazw5mdpvy0d.ls machine # [ 458.837249] niks3-server[1195]: 2026/09/23 04:50:05 INFO Registered completed upload object_key=nar/04bim5f8bvjh25xncf5x40kb529khrlzcbc92qxvapgbc0bdfv2h.nar.zst machine # [ 458.837984] niks3-server[1195]: 2026/09/23 04:50:05 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete machine # [ 458.839643] niks3-server[1195]: 2026/09/23 04:50:05 INFO Completed upload id=25 machine # [ 458.841432] niks3-server[1195]: 2026/09/23 04:50:05 INFO Registered completed upload object_key=jz7biafi8rj5r3r392blyazw5mdpvy0d.narinfo machine # [ 483.351643] niks3-server[1195]: 2026/09/23 04:50:29 INFO Received uploads request method=POST path=/api/pending_closures machine # [ 483.376187] niks3-server[1195]: 2026/09/23 04:50:29 INFO Registered completed upload object_key=a35mdcwbgm2yxm8gj2iqp3ksr0iw0iim.ls machine # [ 483.380160] niks3-server[1195]: 2026/09/23 04:50:29 INFO Registered completed upload object_key=nar/11f7f1hixry25c2yc6np9hs10a2pwwqmymmrxxbswhdjabcyvv0s.nar.zst machine # [ 483.381524] niks3-server[1195]: 2026/09/23 04:50:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign machine # [ 483.383232] niks3-server[1195]: 2026/09/23 04:50:29 INFO Signed narinfos id=26 count=1 machine # [ 483.391315] niks3-server[1195]: 2026/09/23 04:50:29 INFO Registered completed upload object_key=log/jz7biafi8rj5r3r392blyazw5mdpvy0d-k3s-top.drv machine # [ 483.396117] niks3-server[1195]: 2026/09/23 04:50:29 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete machine # [ 483.400094] niks3-server[1195]: 2026/09/23 04:50:29 INFO Completed upload id=26 machine # [ 483.402402] niks3-server[1195]: 2026/09/23 04:50:29 INFO Registered completed upload object_key=a35mdcwbgm2yxm8gj2iqp3ksr0iw0iim.narinfo machine: (finished: must succeed: nix build -L --store 'grpc://127.0.0.1:30051?ca-cert=/nix/store/b2prb199a55q32qawjagqrylwrml5g74-k3s-farm-certs/ca.crt&token-file=/tmp/builder.jwt' --eval-store auto -f /nix/store/4m7mpr04a8dziray8jhdxrxsvaq1bzrn-job.nix --argstr tag t-renewed 2>&1, in 74.72 seconds) (finished: subtest: a renewed balancer certificate is served without a rollout, in 100.23 seconds) subtest: metrics carry build_info machine: must succeed: kubectl -n farm get pod nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2 -o jsonpath='{.items[0].status.podIP}{.status.podIP}' machine: (finished: must succeed: kubectl -n farm get pod nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2 -o jsonpath='{.items[0].status.podIP}{.status.podIP}', in 0.14 seconds) machine: must succeed: curl -sf http://10.42.0.4:9464/metrics | grep -E 'nix_grpc_build_info\{.*worker="machine/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2"' machine: (finished: must succeed: curl -sf http://10.42.0.4:9464/metrics | grep -E 'nix_grpc_build_info\{.*worker="machine/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2"', in 0.03 seconds) (finished: subtest: metrics carry build_info, in 0.17 seconds) subtest: harmonia-gc frees space and the worker stays Ready machine: must succeed: kubectl -n farm exec nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2 -c gc -- sh -c 'df -B1 /nix/store | awk '"'"'NR==2 {print $4}'"'"'' machine # [ 483.775764] k3s[894]: E0923 04:50:30.323472 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:37296: use of closed network connection machine: (finished: must succeed: kubectl -n farm exec nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2 -c gc -- sh -c 'df -B1 /nix/store | awk '"'"'NR==2 {print $4}'"'"'', in 0.18 seconds) machine: must succeed: kubectl -n farm exec nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2 -c nix-daemon -- sh -c 'for i in 1 2 3; do echo $i > /tmp/g$i; nix-store --add /tmp/g$i; done' machine # [ 484.018758] k3s[894]: E0923 04:50:30.566727 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:37314: use of closed network connection machine: (finished: must succeed: kubectl -n farm exec nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2 -c nix-daemon -- sh -c 'for i in 1 2 3; do echo $i > /tmp/g$i; nix-store --add /tmp/g$i; done', in 0.24 seconds) machine: must succeed: kubectl -n farm exec nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2 -c gc -- sh -c 'fallocate -l 11440582656 /nix/fill' machine # [ 484.201373] k3s[894]: E0923 04:50:30.749271 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:37342: use of closed network connection machine: (finished: must succeed: kubectl -n farm exec nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2 -c gc -- sh -c 'fallocate -l 11440582656 /nix/fill', in 0.18 seconds) machine: waiting for success: kubectl -n farm logs nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2 -c gc --since=30s | grep -q 'store paths deleted' machine: (finished: waiting for success: kubectl -n farm logs nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2 -c gc --since=30s | grep -q 'store paths deleted', in 8.82 seconds) machine: must succeed: kubectl -n farm exec nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2 -c gc -- sh -c 'rm /nix/fill' machine # [ 493.221761] k3s[894]: E0923 04:50:39.769198 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:34596: use of closed network connection machine: (finished: must succeed: kubectl -n farm exec nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2 -c gc -- sh -c 'rm /nix/fill', in 0.20 seconds) machine: must succeed: kubectl -n farm get pod nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2 -o jsonpath='{.status.conditions[?(@.type=="Ready")].status}' machine: (finished: must succeed: kubectl -n farm get pod nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2 -o jsonpath='{.status.conditions[?(@.type=="Ready")].status}', in 0.13 seconds) machine: must succeed: kubectl -n farm exec nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2 -c gc -- sh -c 'command -v nix-daemon && command -v niks3' machine # [ 493.524629] k3s[894]: E0923 04:50:40.072486 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:34636: use of closed network connection machine: (finished: must succeed: kubectl -n farm exec nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2 -c gc -- sh -c 'command -v nix-daemon && command -v niks3', in 0.18 seconds) (finished: subtest: harmonia-gc frees space and the worker stays Ready, in 9.93 seconds) subtest: losing the balancer mid-build: the client reconnects and the build succeeds machine: must succeed: systemd-run --unit vialb nix build -L --store 'grpc://127.0.0.1:30051?ca-cert=/nix/store/b2prb199a55q32qawjagqrylwrml5g74-k3s-farm-certs/ca.crt&token-file=/tmp/builder.jwt' --eval-store auto -f /nix/store/zffvrw2ldylpxwxig3cnh0pmwsfs2fm8-slow.nix --argstr tag lb1 machine # [ 493.549497] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://127.0.0.1:30051?ca-cert=/nix/store/b2prb199a55q32qawjagqrylwrml5g74-k3s-farm-certs/ca.crt&token-file=/tmp/builder.jwt" --eval-store auto -f /nix/store/zffvrw2ldylpxwxig3cnh0pmwsfs2fm8-slow.nix --argstr tag lb1. machine # Running as unit: vialb.service; invocation ID: 804a2f64d8724130837dd5989b22c957 machine: (finished: must succeed: systemd-run --unit vialb nix build -L --store 'grpc://127.0.0.1:30051?ca-cert=/nix/store/b2prb199a55q32qawjagqrylwrml5g74-k3s-farm-certs/ca.crt&token-file=/tmp/builder.jwt' --eval-store auto -f /nix/store/zffvrw2ldylpxwxig3cnh0pmwsfs2fm8-slow.nix --argstr tag lb1, in 0.02 seconds) machine: waiting for success: journalctl -u vialb | grep -q "building '/nix/store/.*-slow-lb1.drv'" machine # [ 493.669793] nix[9510]: don't know how to build these paths: machine # [ 493.672060] nix[9510]: /nix/store/d3pzijs4zz7gwai9b8n1j50g96lqiii4-slow-lb1.drv machine # [ 493.692422] nix[9510]: copying 1 paths to '127.0.0.1:30051'... machine # [ 493.694534] nix[9510]: copying path '/nix/store/d3pzijs4zz7gwai9b8n1j50g96lqiii4-slow-lb1.drv' to '127.0.0.1:30051'... machine # [ 493.743355] niks3-server[1195]: 2026/09/23 04:50:40 INFO Received uploads request method=POST path=/api/pending_closures machine # [ 493.756560] niks3-server[1195]: 2026/09/23 04:50:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign machine # [ 493.758356] niks3-server[1195]: 2026/09/23 04:50:40 INFO Signed narinfos id=27 count=1 machine # [ 493.760125] niks3-server[1195]: 2026/09/23 04:50:40 INFO Registered completed upload object_key=nar/0kkr2vb091wkq0cb0f1q92413dsfskh76wydp8mwysv7rawns0ar.nar.zst machine # [ 493.762115] niks3-server[1195]: 2026/09/23 04:50:40 INFO Registered completed upload object_key=d3pzijs4zz7gwai9b8n1j50g96lqiii4.ls machine # [ 493.774428] niks3-server[1195]: 2026/09/23 04:50:40 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete machine # [ 493.776232] niks3-server[1195]: 2026/09/23 04:50:40 INFO Completed upload id=27 machine # [ 493.777818] niks3-server[1195]: 2026/09/23 04:50:40 INFO Registered completed upload object_key=d3pzijs4zz7gwai9b8n1j50g96lqiii4.narinfo machine # [ 493.791859] nix[9510]: building '/nix/store/d3pzijs4zz7gwai9b8n1j50g96lqiii4-slow-lb1.drv' on 127.0.0.1:30051... machine # [ 493.793065] nix[9510]: slow-lb1> machine/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-qlcdh: building d3pzijs4zz7gwai9b8n1j50g96lqiii4-slow-lb1.drv machine: (finished: waiting for success: journalctl -u vialb | grep -q "building '/nix/store/.*-slow-lb1.drv'", in 1.06 seconds) machine: must succeed: kubectl -n farm delete pod -l app.kubernetes.io/component=lb --wait=false machine: (finished: must succeed: kubectl -n farm delete pod -l app.kubernetes.io/component=lb --wait=false, in 1.07 seconds) machine: waiting for success: ! systemctl is-active vialb machine # [ 495.711030] nix[9510]: scheduler at 127.0.0.1:30051: Stream removed (Socket closed), reconnecting machine # [ 495.711654] nix[9510]: d3pzijs4zz7gwai9b8n1j50g96lqiii4-slow-lb1.drv: Stream removed (Socket closed), asking the scheduler again machine # [ 495.740072] systemd[1]: cri-containerd-bafcae70604ed9e2b0534a3450de331e8ad933bc7cfd87c957f4e898b3e9412a.scope: Deactivated successfully. machine # [ 495.740759] systemd[1]: cri-containerd-bafcae70604ed9e2b0534a3450de331e8ad933bc7cfd87c957f4e898b3e9412a.scope: Consumed 4.439s CPU time over 7min 32.410s wall clock time, 57.1M memory peak, 37.1M read from disk, 709.2K incoming IP traffic, 676.6K outgoing IP traffic. machine # [ 495.776470] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-bafcae70604ed9e2b0534a3450de331e8ad933bc7cfd87c957f4e898b3e9412a-rootfs.mount: Deactivated successfully. machine # [ 495.805228] systemd[1]: cri-containerd-61edf6ac7608e1a9972941a9c82042d29c511cb14bf67c51e7570bc371ad843a.scope: Deactivated successfully. machine # [ 495.834234] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-61edf6ac7608e1a9972941a9c82042d29c511cb14bf67c51e7570bc371ad843a-rootfs.mount: Deactivated successfully. machine # [ 495.856821] systemd[1]: run-k3s-containerd-io.containerd.grpc.v1.cri-sandboxes-61edf6ac7608e1a9972941a9c82042d29c511cb14bf67c51e7570bc371ad843a-shm.mount: Deactivated successfully. machine # [ 495.888413] d[ 496.041384] cni0: port 4(veth96611f10) entered disabled state machine # hcpcd[3492]: veth96611f10: carrier lost machine # [ 496.043673] veth96611f10 (unregistering): left allmulticast mode machine # [ 496.044795] veth96611f10 (unregistering): left promiscuous mode machine # [ 496.045822] cni0: port 4(veth96611f10) entered disabled state machine # [ 495.911231] systemd[1]: run-netns-cni\x2dec6721a3\x2dbaa5\x2d1893\x2dc193\x2dbcea8c3a9b03.mount: Deactivated successfully. machine # [ 495.921221] dhcpcd[3492]: veth96611f10: deleting route to 169.254.0.0/16 machine # [ 495.955482] k3s[894]: I0923 04:50:42.502946 894 endpointslice_controller.go:361] "Error syncing endpoint slices for service, retrying" key="farm/nix-grpc-farm" err="EndpointSlice informer cache is out of date" machine # [ 495.967871] systemd[1]: Created slice libcontainer container kubepods-besteffort-pod187d9c4b_856d_47d8_9a83_593fdd108464.slice. machine # [ 495.998286] dhcpcd[3492]: veth96611f10: removing interface machine # [ 496.032724] k3s[894]: I0923 04:50:42.580735 894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fd4651b8-ec20-44c9-a82a-de134cebe717-tls-lb\" (UniqueName: \"kubernetes.io/secret/fd4651b8-ec20-44c9-a82a-de134cebe717-tls-lb\") pod \"fd4651b8-ec20-44c9-a82a-de134cebe717\" (UID: \"fd4651b8-ec20-44c9-a82a-de134cebe717\") " machine # [ 496.033622] k3s[894]: I0923 04:50:42.580787 894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fd4651b8-ec20-44c9-a82a-de134cebe717-tls-ca\" (UniqueName: \"kubernetes.io/secret/fd4651b8-ec20-44c9-a82a-de134cebe717-tls-ca\") pod \"fd4651b8-ec20-44c9-a82a-de134cebe717\" (UID: \"fd4651b8-ec20-44c9-a82a-de134cebe717\") " machine # [ 496.034558] k3s[894]: I0923 04:50:42.580827 894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fd4651b8-ec20-44c9-a82a-de134cebe717-config\" (UniqueName: \"kubernetes.io/configmap/fd4651b8-ec20-44c9-a82a-de134cebe717-config\") pod \"fd4651b8-ec20-44c9-a82a-de134cebe717\" (UID: \"fd4651b8-ec20-44c9-a82a-de134cebe717\") " machine # [ 496.035455] k3s[894]: I0923 04:50:42.580872 894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd4651b8-ec20-44c9-a82a-de134cebe717-tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd4651b8-ec20-44c9-a82a-de134cebe717-tmp\") pod \"fd4651b8-ec20-44c9-a82a-de134cebe717\" (UID: \"fd4651b8-ec20-44c9-a82a-de134cebe717\") " machine # [ 496.036189] k3s[894]: I0923 04:50:42.580930 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-ca\" (UniqueName: \"kubernetes.io/secret/187d9c4b-856d-47d8-9a83-593fdd108464-tls-ca\") pod \"nix-grpc-farm-lb-775dbb5c7c-x86kf\" (UID: \"187d9c4b-856d-47d8-9a83-593fdd108464\") " pod="farm/nix-grpc-farm-lb-775dbb5c7c-x86kf" machine # [ 496.036781] k3s[894]: I0923 04:50:42.580961 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/187d9c4b-856d-47d8-9a83-593fdd108464-tmp\") pod \"nix-grpc-farm-lb-775dbb5c7c-x86kf\" (UID: \"187d9c4b-856d-47d8-9a83-593fdd108464\") " pod="farm/nix-grpc-farm-lb-775dbb5c7c-x86kf" machine # [ 496.037607] k3s[894]: I0923 04:50:42.580994 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/187d9c4b-856d-47d8-9a83-593fdd108464-config\") pod \"nix-grpc-farm-lb-775dbb5c7c-x86kf\" (UID: \"187d9c4b-856d-47d8-9a83-593fdd108464\") " pod="farm/nix-grpc-farm-lb-775dbb5c7c-x86kf" machine # [ 496.038481] k3s[894]: I0923 04:50:42.581021 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-lb\" (UniqueName: \"kubernetes.io/secret/187d9c4b-856d-47d8-9a83-593fdd108464-tls-lb\") pod \"nix-grpc-farm-lb-775dbb5c7c-x86kf\" (UID: \"187d9c4b-856d-47d8-9a83-593fdd108464\") " pod="farm/nix-grpc-farm-lb-775dbb5c7c-x86kf" machine # [ 496.041192] systemd[1]: var-lib-kubelet-pods-fd4651b8\x2dec20\x2d44c9\x2da82a\x2dde134cebe717-volume\x2dsubpaths-config-envoy-0.mount: Deactivated successfully. machine # [ 496.072354] k3s[894]: I0923 04:50:42.620350 894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd4651b8-ec20-44c9-a82a-de134cebe717-tmp" pod "fd4651b8-ec20-44c9-a82a-de134cebe717" (UID: "fd4651b8-ec20-44c9-a82a-de134cebe717"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 496.076492] k3s[894]: I0923 04:50:42.624511 894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd4651b8-ec20-44c9-a82a-de134cebe717-tls-ca" pod "fd4651b8-ec20-44c9-a82a-de134cebe717" (UID: "fd4651b8-ec20-44c9-a82a-de134cebe717"). InnerVolumeSpecName "tls-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 496.078515] k3s[894]: I0923 04:50:42.626537 894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd4651b8-ec20-44c9-a82a-de134cebe717-tls-lb" pod "fd4651b8-ec20-44c9-a82a-de134cebe717" (UID: "fd4651b8-ec20-44c9-a82a-de134cebe717"). InnerVolumeSpecName "tls-lb". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 496.086056] k3s[894]: I0923 04:50:42.634051 894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd4651b8-ec20-44c9-a82a-de134cebe717-config" pod "fd4651b8-ec20-44c9-a82a-de134cebe717" (UID: "fd4651b8-ec20-44c9-a82a-de134cebe717"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" machine # [ 496.133549] k3s[894]: I0923 04:50:42.681548 894 reconciler_common.go:299] "Volume detached for volume \"tls-ca\" (UniqueName: \"kubernetes.io/secret/fd4651b8-ec20-44c9-a82a-de134cebe717-tls-ca\") on node \"machine\" DevicePath \"\"" machine # [ 496.134455] k3s[894]: I0923 04:50:42.681575 894 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fd4651b8-ec20-44c9-a82a-de134cebe717-config\") on node \"machine\" DevicePath \"\"" machine # [ 496.134867] k3s[894]: I0923 04:50:42.681591 894 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd4651b8-ec20-44c9-a82a-de134cebe717-tmp\") on node \"machine\" DevicePath \"\"" machine # [ 496.135578] k3s[894]: I0923 04:50:42.681604 894 reconciler_common.go:299] "Volume detached for volume \"tls-lb\" (UniqueName: \"kubernetes.io/secret/fd4651b8-ec20-44c9-a82a-de134cebe717-tls-lb\") on node \"machine\" DevicePath \"\"" machine # [ 496.459333] cni0: port 1(vethbba6346e) entered blocking state machine # [ 496.306492] (udev-worker)[9707]: Network interface NamePolicy= disabled on kernel command line. machine # [ 496.460375] cni0: port 1(vethbba6346e) entered disabled state machine # [ 496.462503] vethbba6346e: entered allmulticast mode machine # [ 496.463786] vethbba6346e: entered promiscuous mode machine # [ 496.472642] cni0: port 1(vethbba6346e) entered blocking state machine # [ 496.474254] cni0: port 1(vethbba6346e) entered forwarding state machine # [ 496.336543] dhcpcd[3492]: vethbba6346e: IAID 8f:44:71:69 machine # [ 496.337107] dhcpcd[3492]: vethbba6346e: adding address fe80::e46f:8fff:fe44:7169 machine # [ 496.401333] systemd[1]: Started libcontainer container d954a286dd44db8590ec6202b494711ec4d4c26352adea5bf76b3ea968a60c9c. machine # [ 496.523524] systemd[1]: Started libcontainer container f3121c46c8e277cc95c5f683091fdf115be448fd59894d6bdf9858cf4f2803ec. machine # [ 496.751231] k3s[894]: I0923 04:50:43.298300 894 scope.go:122] "RemoveContainer" containerID="bafcae70604ed9e2b0534a3450de331e8ad933bc7cfd87c957f4e898b3e9412a" machine # [ 496.771430] k3s[894]: I0923 04:50:43.319388 894 scope.go:122] "RemoveContainer" containerID="bafcae70604ed9e2b0534a3450de331e8ad933bc7cfd87c957f4e898b3e9412a" machine # [ 496.774416] k3s[894]: E0923 04:50:43.322399 894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"bafcae70604ed9e2b0534a3450de331e8ad933bc7cfd87c957f4e898b3e9412a\": not found" containerID="bafcae70604ed9e2b0534a3450de331e8ad933bc7cfd87c957f4e898b3e9412a" machine # [ 496.776097] k3s[894]: I0923 04:50:43.322448 894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"bafcae70604ed9e2b0534a3450de331e8ad933bc7cfd87c957f4e898b3e9412a"} err="failed to get container status \"bafcae70604ed9e2b0534a3450de331e8ad933bc7cfd87c957f4e898b3e9412a\": rpc error: code = NotFound desc = an error occurred when try to find container \"bafcae70604ed9e2b0534a3450de331e8ad933bc7cfd87c957f4e898b3e9412a\": not found" machine # [ 496.777115] k3s[894]: I0923 04:50:43.323664 894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="farm/nix-grpc-farm-lb-775dbb5c7c-x86kf" podStartSLOduration=1.323645966 podStartE2EDuration="1.323645966s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-23 04:50:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-23 04:50:43.320815438 +0000 UTC m=+478.428234541" watchObservedRunningTime="2026-09-23 04:50:43.323645966 +0000 UTC m=+478.431061716" machine # [ 496.787986] systemd[1]: var-lib-kubelet-pods-fd4651b8\x2dec20\x2d44c9\x2da82a\x2dde134cebe717-volume\x2dsubpaths-config-envoy-2.mount: Deactivated successfully. machine # [ 496.788641] systemd[1]: var-lib-kubelet-pods-fd4651b8\x2dec20\x2d44c9\x2da82a\x2dde134cebe717-volume\x2dsubpaths-config-envoy-1.mount: Deactivated successfully. machine # [ 496.789363] systemd[1]: var-lib-kubelet-pods-fd4651b8\x2dec20\x2d44c9\x2da82a\x2dde134cebe717-volumes-kubernetes.io\x7eempty\x2ddir-tmp.mount: Deactivated successfully. machine # [ 496.789864] systemd[1]: var-lib-kubelet-pods-fd4651b8\x2dec20\x2d44c9\x2da82a\x2dde134cebe717-volumes-kubernetes.io\x7esecret-tls\x2dlb.mount: Deactivated successfully. machine # [ 496.790745] systemd[1]: var-lib-kubelet-pods-fd4651b8\x2dec20\x2d44c9\x2da82a\x2dde134cebe717-volumes-kubernetes.io\x7esecret-tls\x2dca.mount: Deactivated successfully. machine # [ 496.794826] systemd[1]: Removed slice libcontainer container kubepods-besteffort-podfd4651b8_ec20_44c9_a82a_de134cebe717.slice. machine # [ 496.795735] systemd[1]: kubepods-besteffort-podfd4651b8_ec20_44c9_a82a_de134cebe717.slice: Consumed 4.476s CPU time over 7min 35.566s wall clock time, 57.3M memory peak, 37.1M read from disk, 709.2K incoming IP traffic, 676.6K outgoing IP traffic. machine # [ 497.587710] k3s[894]: I0923 04:50:44.134877 894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fd4651b8-ec20-44c9-a82a-de134cebe717" path="/var/lib/kubelet/pods/fd4651b8-ec20-44c9-a82a-de134cebe717/volumes" machine # [ 497.762373] dhcpcd[3492]: vethbba6346e: soliciting a DHCP lease machine # [ 498.213903] nix[9510]: scheduler at 127.0.0.1:30051: failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:30051: tcp handshaker shutdown, reconnecting machine # [ 498.304393] dhcpcd[3492]: vethbba6346e: soliciting an IPv6 router machine # [ 499.268804] nix[9510]: building '/nix/store/d3pzijs4zz7gwai9b8n1j50g96lqiii4-slow-lb1.drv' on 127.0.0.1:30051... machine # [ 499.270795] nix[9510]: slow-lb1> machine/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-7hvb2: building d3pzijs4zz7gwai9b8n1j50g96lqiii4-slow-lb1.drv machine # [ 502.763260] dhcpcd[3492]: vethbba6346e: probing for an IPv4LL address machine # [ 507.848924] dhcpcd[3492]: vethbba6346e: using IPv4LL address 169.254.60.112 machine # [ 507.851122] dhcpcd[3492]: vethbba6346e: adding route to 169.254.0.0/16 machine # [ 510.308556] dhcpcd[3492]: vethbba6346e: no IPv6 Routers available machine # [ 544.224418] niks3-server[1195]: 2026/09/23 04:51:30 INFO Received uploads request method=POST path=/api/pending_closures machine # [ 544.239503] niks3-server[1195]: 2026/09/23 04:51:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign machine # [ 544.242223] niks3-server[1195]: 2026/09/23 04:51:30 INFO Signed narinfos id=28 count=1 machine # [ 544.248753] niks3-server[1195]: 2026/09/23 04:51:30 INFO Registered completed upload object_key=log/d3pzijs4zz7gwai9b8n1j50g96lqiii4-slow-lb1.drv machine # [ 544.251255] niks3-server[1195]: 2026/09/23 04:51:30 INFO Registered completed upload object_key=2804s0b4bxfggjycnhf3zz7f1g8jfsvp.ls machine # [ 544.256069] niks3-server[1195]: 2026/09/23 04:51:30 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete machine # [ 544.259432] niks3-server[1195]: 2026/09/23 04:51:30 INFO Completed upload id=28 machine # [ 544.261732] niks3-server[1195]: 2026/09/23 04:51:30 INFO Registered completed upload object_key=2804s0b4bxfggjycnhf3zz7f1g8jfsvp.narinfo machine # [ 544.297908] systemd[1]: vialb.service: Deactivated successfully. machine # [ 544.299259] systemd[1]: vialb.service: Consumed 188ms CPU time over 50.748s wall clock time, 48.4M memory peak, 4K written to disk, 8.6K incoming IP traffic, 17.3K outgoing IP traffic. machine: (finished: waiting for success: ! systemctl is-active vialb, in 48.89 seconds) machine: must succeed: systemctl show -p Result --value vialb | grep -qx success || { journalctl -u vialb >&2; false; } machine: (finished: must succeed: systemctl show -p Result --value vialb | grep -qx success || { journalctl -u vialb >&2; false; }, in 0.04 seconds) machine: waiting for success: kubectl -n farm rollout status deployment nix-grpc-farm-lb --timeout=10s machine: (finished: waiting for success: kubectl -n farm rollout status deployment nix-grpc-farm-lb --timeout=10s, in 0.26 seconds) (finished: subtest: losing the balancer mid-build: the client reconnects and the build succeeds, in 51.35 seconds) subtest: deleting a worker pod mid-build drains it: the build finishes there and publishes machine: must succeed: systemd-run --unit slow nix build -L --store 'grpc://127.0.0.1:30051?ca-cert=/nix/store/b2prb199a55q32qawjagqrylwrml5g74-k3s-farm-certs/ca.crt&token-file=/tmp/builder.jwt' --eval-store auto -f /nix/store/zffvrw2ldylpxwxig3cnh0pmwsfs2fm8-slow.nix --argstr tag s1 machine # [ 544.908256] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://127.0.0.1:30051?ca-cert=/nix/store/b2prb199a55q32qawjagqrylwrml5g74-k3s-farm-certs/ca.crt&token-file=/tmp/builder.jwt" --eval-store auto -f /nix/store/zffvrw2ldylpxwxig3cnh0pmwsfs2fm8-slow.nix --argstr tag s1. machine # Running as unit: slow.service; invocation ID: d191cfb356ea4ba0bbcc0c1b0bd92966 machine: (finished: must succeed: systemd-run --unit slow nix build -L --store 'grpc://127.0.0.1:30051?ca-cert=/nix/store/b2prb199a55q32qawjagqrylwrml5g74-k3s-farm-certs/ca.crt&token-file=/tmp/builder.jwt' --eval-store auto -f /nix/store/zffvrw2ldylpxwxig3cnh0pmwsfs2fm8-slow.nix --argstr tag s1, in 0.03 seconds) machine # [ 545.035767] nix[10491]: don't know how to build these paths: machine # [ 545.036562] nix[10491]: /nix/store/zlgfhpppbdjjaq7dv6cy36ss086nzwmz-slow-s1.drv machine # [ 545.059916] nix[10491]: copying 1 paths to '127.0.0.1:30051'... machine # [ 545.061720] nix[10491]: copying path '/nix/store/zlgfhpppbdjjaq7dv6cy36ss086nzwmz-slow-s1.drv' to '127.0.0.1:30051'... machine # [ 545.100336] niks3-server[1195]: 2026/09/23 04:51:31 INFO Received uploads request method=POST path=/api/pending_closures machine # [ 545.114106] niks3-server[1195]: 2026/09/23 04:51:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign machine # [ 545.115468] niks3-server[1195]: 2026/09/23 04:51:31 INFO Signed narinfos id=29 count=1 machine # [ 545.121354] niks3-server[1195]: 2026/09/23 04:51:31 INFO Registered completed upload object_key=nar/0wl2i9xpw0gwrx1xqdgx9ka4n503ky9qhl20pdrdlphn8d6lcx0i.nar.zst machine # [ 545.123802] niks3-server[1195]: 2026/09/23 04:51:31 INFO Registered completed upload object_key=zlgfhpppbdjjaq7dv6cy36ss086nzwmz.ls machine # [ 545.126562] niks3-server[1195]: 2026/09/23 04:51:31 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete machine # command terminated with exit code 1 machine # [ 545.132910] k3s[894]: E0923 04:51:31.679798 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:34328: use of closed network connection machine # [ 545.134513] niks3-server[1195]: 2026/09/23 04:51:31 INFO Completed upload id=29 machine # [ 545.135236] niks3-server[1195]: 2026/09/23 04:51:31 INFO Registered completed upload object_key=zlgfhpppbdjjaq7dv6cy36ss086nzwmz.narinfo machine # [ 545.144965] nix[10491]: building '/nix/store/zlgfhpppbdjjaq7dv6cy36ss086nzwmz-slow-s1.drv' on 127.0.0.1:30051... machine # [ 545.145852] nix[10491]: slow-s1> machine/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-qlcdh: building zlgfhpppbdjjaq7dv6cy36ss086nzwmz-slow-s1.drv machine # [ 545.345238] k3s[894]: E0923 04:51:31.893026 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:34358: use of closed network connection machine # command terminated with exit code 1 machine # [ 546.601686] k3s[894]: E0923 04:51:33.149286 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:34394: use of closed network connection machine # command terminated with exit code 1 machine # [ 546.794860] k3s[894]: E0923 04:51:33.342752 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:34420: use of closed network connection machine # command terminated with exit code 1 machine # command terminated with exit code 1 machine # [ 548.070629] k3s[894]: E0923 04:51:34.617797 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:34460: use of closed network connection machine # [ 548.255784] k3s[894]: E0923 04:51:34.803767 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:34488: use of closed network connection machine # command terminated with exit code 1 machine # command terminated with exit code 1 machine # [ 549.590234] k3s[894]: E0923 04:51:36.137387 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:34508: use of closed network connection machine # [ 549.779867] k3s[894]: E0923 04:51:36.327824 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:34544: use of closed network connection machine # command terminated with exit code 1 machine # [ 551.091798] k3s[894]: E0923 04:51:37.639298 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:34580: use of closed network connection machine # command terminated with exit code 1 machine # [ 551.299137] k3s[894]: E0923 04:51:37.846989 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:34600: use of closed network connection machine # command terminated with exit code 1 machine # [ 552.631855] k3s[894]: E0923 04:51:39.179543 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:34634: use of closed network connection machine # command terminated with exit code 1 machine # [ 552.840154] k3s[894]: E0923 04:51:39.387758 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:34656: use of closed network connection machine # command terminated with exit code 1 machine # command terminated with exit code 1 machine # [ 554.176935] k3s[894]: E0923 04:51:40.723957 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:44724: use of closed network connection machine # [ 554.363103] k3s[894]: E0923 04:51:40.911070 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:44754: use of closed network connection machine # command terminated with exit code 1 machine # command terminated with exit code 1 machine # [ 555.655366] k3s[894]: E0923 04:51:42.202956 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:44780: use of closed network connection machine # [ 555.849454] k3s[894]: E0923 04:51:42.397432 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:44816: use of closed network connection machine # command terminated with exit code 1 machine # [ 557.163415] k3s[894]: E0923 04:51:43.710980 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:44846: use of closed network connection machine # command terminated with exit code 1 machine # [ 557.380587] k3s[894]: E0923 04:51:43.928542 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:44876: use of closed network connection machine # command terminated with exit code 1 machine # [ 558.717403] k3s[894]: E0923 04:51:45.264756 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:44900: use of closed network connection machine # command terminated with exit code 1 machine # [ 558.903854] k3s[894]: E0923 04:51:45.451831 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:44936: use of closed network connection machine # command terminated with exit code 1 machine # command terminated with exit code 1 machine # [ 560.163550] k3s[894]: E0923 04:51:46.709946 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:44968: use of closed network connection machine # [ 560.350670] k3s[894]: E0923 04:51:46.898655 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:45004: use of closed network connection machine # command terminated with exit code 1 machine # command terminated with exit code 1 machine # [ 561.658372] k3s[894]: E0923 04:51:48.205734 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:45032: use of closed network connection machine # [ 561.848879] k3s[894]: E0923 04:51:48.396743 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:45052: use of closed network connection machine # command terminated with exit code 1 machine # command terminated with exit code 1 machine # [ 563.132619] k3s[894]: E0923 04:51:49.679847 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:44728: use of closed network connection machine # [ 563.317480] k3s[894]: E0923 04:51:49.865458 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:44754: use of closed network connection machine # command terminated with exit code 1 machine # [ 564.599228] k3s[894]: E0923 04:51:51.146792 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:44770: use of closed network connection machine # command terminated with exit code 1 machine # [ 564.787388] k3s[894]: E0923 04:51:51.335365 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:44792: use of closed network connection machine # command terminated with exit code 1 machine # [ 566.054065] k3s[894]: time="2026-09-23T04:51:52Z" level=warning msg="Proxy error: read failed: read tcp 127.0.0.1:50690->127.0.0.1:10250: read: connection reset by peer" machine # [ 566.054768] k3s[894]: E0923 04:51:52.601993 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:44818: use of closed network connection machine # command terminated with exit code 1 machine # [ 566.244118] k3s[894]: E0923 04:51:52.792007 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:44854: use of closed network connection machine # command terminated with exit code 1 machine # command terminated with exit code 1 machine # [ 567.546521] k3s[894]: E0923 04:51:54.094122 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:44900: use of closed network connection machine # [ 567.755456] k3s[894]: E0923 04:51:54.302973 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:44932: use of closed network connection machine # command terminated with exit code 1 machine # command terminated with exit code 1 machine # [ 569.039515] k3s[894]: E0923 04:51:55.586926 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:44948: use of closed network connection machine # [ 569.210698] k3s[894]: E0923 04:51:55.758692 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:44970: use of closed network connection machine # command terminated with exit code 1 machine # command terminated with exit code 1 machine # [ 570.461201] k3s[894]: E0923 04:51:57.008294 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:44994: use of closed network connection machine # [ 570.691545] k3s[894]: E0923 04:51:57.239423 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:45022: use of closed network connection machine # [ 570.868106] k3s[894]: E0923 04:51:57.416011 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:45052: use of closed network connection machine # command terminated with exit code 1 machine # [ 571.056176] k3s[894]: E0923 04:51:57.603781 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:45066: use of closed network connection machine # [ 571.056722] k3s[894]: E0923 04:51:57.603829 894 upgradeaware.go:427] Error proxying data from client to backend: readfrom tcp 127.0.0.1:34928->127.0.0.1:10010: write tcp 127.0.0.1:34928->127.0.0.1:10010: write: broken pipe machine # [ 571.057270] k3s[894]: time="2026-09-23T04:51:57Z" level=warning msg="Proxy error: write failed: write tcp 127.0.0.1:50772->127.0.0.1:10250: write: connection reset by peer" machine: must succeed: kubectl -n farm delete pod nix-grpc-farm-worker-x86-64-linux-cfb8578f6-qlcdh --wait=false machine: (finished: must succeed: kubectl -n farm delete pod nix-grpc-farm-worker-x86-64-linux-cfb8578f6-qlcdh --wait=false, in 0.15 seconds) machine: must succeed: sleep 5.0 machine # [ 571.245660] systemd[1]: cri-containerd-3dc3130f896e4d28d430dbacb77f3c9131e199ecbc34d86b2fdc1dac1973f3e2.scope: Deactivated successfully. machine # [ 571.247238] systemd[1]: cri-containerd-3dc3130f896e4d28d430dbacb77f3c9131e199ecbc34d86b2fdc1dac1973f3e2.scope: Consumed 514ms CPU time over 8min 38.781s wall clock time, 12.2M memory peak, 5.2M read from disk. machine # [ 571.279917] systemd[1]: Created slice libcontainer container kubepods-besteffort-pod409aa3d0_d1de_4f14_ba98_15e0dd1e19e9.slice. machine # [ 571.295653] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-3dc3130f896e4d28d430dbacb77f3c9131e199ecbc34d86b2fdc1dac1973f3e2-rootfs.mount: Deactivated successfully. machine # [ 571.420120] k3s[894]: I0923 04:51:57.968066 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"build\" (UniqueName: \"kubernetes.io/empty-dir/409aa3d0-d1de-4f14-ba98-15e0dd1e19e9-build\") pod \"nix-grpc-farm-worker-x86-64-linux-cfb8578f6-xm2lw\" (UID: \"409aa3d0-d1de-4f14-ba98-15e0dd1e19e9\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-xm2lw" machine # [ 571.420914] k3s[894]: I0923 04:51:57.968141 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc\" (UniqueName: \"kubernetes.io/secret/409aa3d0-d1de-4f14-ba98-15e0dd1e19e9-oidc\") pod \"nix-grpc-farm-worker-x86-64-linux-cfb8578f6-xm2lw\" (UID: \"409aa3d0-d1de-4f14-ba98-15e0dd1e19e9\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-xm2lw" machine # [ 571.421674] k3s[894]: I0923 04:51:57.968174 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"niks3-token\" (UniqueName: \"kubernetes.io/projected/409aa3d0-d1de-4f14-ba98-15e0dd1e19e9-niks3-token\") pod \"nix-grpc-farm-worker-x86-64-linux-cfb8578f6-xm2lw\" (UID: \"409aa3d0-d1de-4f14-ba98-15e0dd1e19e9\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-xm2lw" machine # [ 571.422255] k3s[894]: I0923 04:51:57.968200 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nixconf\" (UniqueName: \"kubernetes.io/configmap/409aa3d0-d1de-4f14-ba98-15e0dd1e19e9-nixconf\") pod \"nix-grpc-farm-worker-x86-64-linux-cfb8578f6-xm2lw\" (UID: \"409aa3d0-d1de-4f14-ba98-15e0dd1e19e9\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-xm2lw" machine # [ 571.422691] k3s[894]: I0923 04:51:57.968229 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5k7b\" (UniqueName: \"kubernetes.io/projected/409aa3d0-d1de-4f14-ba98-15e0dd1e19e9-kube-api-access-j5k7b\") pod \"nix-grpc-farm-worker-x86-64-linux-cfb8578f6-xm2lw\" (UID: \"409aa3d0-d1de-4f14-ba98-15e0dd1e19e9\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-xm2lw" machine # [ 571.423277] k3s[894]: I0923 04:51:57.968262 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/409aa3d0-d1de-4f14-ba98-15e0dd1e19e9-ca\") pod \"nix-grpc-farm-worker-x86-64-linux-cfb8578f6-xm2lw\" (UID: \"409aa3d0-d1de-4f14-ba98-15e0dd1e19e9\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-xm2lw" machine # [ 571.423895] k3s[894]: I0923 04:51:57.968289 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nix\" (UniqueName: \"kubernetes.io/empty-dir/409aa3d0-d1de-4f14-ba98-15e0dd1e19e9-nix\") pod \"nix-grpc-farm-worker-x86-64-linux-cfb8578f6-xm2lw\" (UID: \"409aa3d0-d1de-4f14-ba98-15e0dd1e19e9\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-xm2lw" machine # [ 571.424715] k3s[894]: I0923 04:51:57.968315 894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls\" (UniqueName: \"kubernetes.io/secret/409aa3d0-d1de-4f14-ba98-15e0dd1e19e9-tls\") pod \"nix-grpc-farm-worker-x86-64-linux-cfb8578f6-xm2lw\" (UID: \"409aa3d0-d1de-4f14-ba98-15e0dd1e19e9\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-xm2lw" machine # [ 571.779793] cni0: port 4(vethd2786ed8) entered blocking state machine # [ 571.781191] cni0: port 4(vethd2786ed8) entered disabled state machine # [ 571.782467] vethd2786ed8: entered allmulticast mode machine # [ 571.783723] vethd2786ed8: entered promiscuous mode machine # [ 571.796038] cni0: port 4(vethd2786ed8) entered blocking state machine # [ 571.797048] cni0: port 4(vethd2786ed8) entered forwarding state machine # [ 571.678099] (udev-worker)[12168]: Network interface NamePolicy= disabled on kernel command line. machine # [ 571.707268] dhcpcd[3492]: vethd2786ed8: IAID 19:f6:c3:0c machine # [ 571.708214] dhcpcd[3492]: vethd2786ed8: adding address fe80::f0bb:19ff:fef6:c30c machine # [ 571.738298] systemd[1]: Started libcontainer container 8e62ef0569a451203ff357d8f7c4fe2dbb68116eed7480b6046444f999e89e2f. machine # [ 572.029520] systemd[1]: Started libcontainer container 05ff8579972d2098e7850d62706355f1366ef3c53c61503b03e08c3376fc098a. machine # [ 572.567677] dhcpcd[3492]: vethd2786ed8: soliciting a DHCP lease machine # [ 573.162102] dhcpcd[3492]: vethd2786ed8: soliciting an IPv6 router machine # [ 574.251416] systemd[1]: cri-containerd-05ff8579972d2098e7850d62706355f1366ef3c53c61503b03e08c3376fc098a.scope: Deactivated successfully. machine # [ 574.254823] systemd[1]: cri-containerd-05ff8579972d2098e7850d62706355f1366ef3c53c61503b03e08c3376fc098a.scope: Consumed 1.025s CPU time over 2.219s wall clock time, 482.9M memory peak, 201.9M read from disk, 160.8M written to disk. machine # [ 574.326648] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-05ff8579972d2098e7850d62706355f1366ef3c53c61503b03e08c3376fc098a-rootfs.mount: Deactivated successfully. machine # [ 575.234933] systemd[1]: var-lib-rancher-k3s-agent-containerd-tmpmounts-ovl\x2didmapped48808488.mount: Deactivated successfully. machine # [ 575.330721] systemd[1]: var-lib-rancher-k3s-agent-containerd-tmpmounts-containerd\x2dmount4237607139.mount: Deactivated successfully. machine # [ 575.371179] systemd[1]: tmp-ovl\x2didmapped4103531846.mount: Deactivated successfully. machine # [ 575.399567] systemd[1]: Started libcontainer container ea937beb13358a771591adaa6e8eb363c41bb8d9666eb138e68c210e017ed156. machine # [ 575.623532] systemd[1]: Started libcontainer container d60234ed2495f7e14f0aea4c8b1b4551792b6fa9b98bbdb9a71c3bede36968bc. machine # [ 575.786297] systemd[1]: Started libcontainer container 21252b6761fb25d2292a6ee35a0ca084f6f55246631cbf7efae4e135c437634d. machine # [ 576.166783] k3s[894]: I0923 04:52:02.713630 894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="farm/nix-grpc-farm-worker-x86-64-linux-cfb8578f6-xm2lw" podStartSLOduration=3.5304887259999997 podStartE2EDuration="5.713586019s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.183097293s" isStatefulPod=false podCreationTimestamp="2026-09-23 04:51:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-23 04:52:02.713436558 +0000 UTC m=+557.820857337" watchObservedRunningTime="2026-09-23 04:52:02.713586019 +0000 UTC m=+557.821006239" machine: (finished: must succeed: sleep 5.0, in 5.03 seconds) machine # [ 576.333680] systemd[1]: var-lib-rancher-k3s-agent-containerd-tmpmounts-containerd\x2dmount1054601800.mount: Deactivated successfully. machine # [ 576.334729] systemd[1]: var-lib-rancher-k3s-agent-containerd-tmpmounts-ovl\x2didmapped1055949240.mount: Deactivated successfully. machine # [ 576.427667] k3s[894]: E0923 04:52:02.975469 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:38338: use of closed network connection machine # command terminated with exit code 1 machine # [ 576.635028] k3s[894]: E0923 04:52:03.182994 894 conn.go:339] Error on socket receive: read tcp 127.0.0.1:6443->127.0.0.1:38360: use of closed network connection machine: waiting for success: ! systemctl is-active slow machine # [ 577.569495] dhcpcd[3492]: vethd2786ed8: probing for an IPv4LL address machine # [ 582.144993] dhcpcd[3492]: vethd2786ed8: using IPv4LL address 169.254.123.29 machine # [ 582.148145] dhcpcd[3492]: vethd2786ed8: adding route to 169.254.0.0/16 machine # [ 585.167122] dhcpcd[3492]: vethd2786ed8: no IPv6 Routers available machine # [ 590.095984] niks3-server[1195]: 2026/09/23 04:52:16 INFO Received uploads request method=POST path=/api/pending_closures machine # [ 590.110566] niks3-server[1195]: 2026/09/23 04:52:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign machine # [ 590.114884] niks3-server[1195]: 2026/09/23 04:52:16 INFO Signed narinfos id=30 count=1 machine # [ 590.118400] niks3-server[1195]: 2026/09/23 04:52:16 INFO Registered completed upload object_key=log/zlgfhpppbdjjaq7dv6cy36ss086nzwmz-slow-s1.drv machine # [ 590.122961] niks3-server[1195]: 2026/09/23 04:52:16 INFO Registered completed upload object_key=yn9npyvhypx0s0v0rp739lvy4yjqph7k.ls machine # [ 590.127235] niks3-server[1195]: 2026/09/23 04:52:16 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete machine # [ 590.133105] niks3-server[1195]: 2026/09/23 04:52:16 INFO Completed upload id=30 machine # [ 590.136799] niks3-server[1195]: 2026/09/23 04:52:16 INFO Registered completed upload object_key=yn9npyvhypx0s0v0rp739lvy4yjqph7k.narinfo machine # [ 590.859981] systemd[1]: cri-containerd-86d47e32616f448aaf7ede86c6cf800d428a00db97db587f4e636f51dbdabbe4.scope: Deactivated successfully. machine # [ 590.862283] systemd[1]: cri-containerd-86d47e32616f448aaf7ede86c6cf800d428a00db97db587f4e636f51dbdabbe4.scope: Consumed 3.329s CPU time over 8min 58.623s wall clock time, 71.3M memory peak, 33.8M read from disk, 269.2K incoming IP traffic, 339.1K outgoing IP traffic. machine # [ 590.940297] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-86d47e32616f448aaf7ede86c6cf800d428a00db97db587f4e636f51dbdabbe4-rootfs.mount: Deactivated successfully. machine # [ 591.187451] systemd[1]: slow.service: Deactivated successfully. machine # [ 591.189239] systemd[1]: slow.service: Consumed 175ms CPU time over 46.279s wall clock time, 38.1M memory peak, 4K written to disk, 5.4K incoming IP traffic, 11.7K outgoing IP traffic. machine: (finished: waiting for success: ! systemctl is-active slow, in 14.59 seconds) machine: must succeed: systemctl show -p Result --value slow | grep -qx success || { journalctl -u slow >&2; false; } machine: (finished: must succeed: systemctl show -p Result --value slow | grep -qx success || { journalctl -u slow >&2; false; }, in 0.03 seconds) machine: must fail: journalctl -u slow | grep -q 'asking the scheduler again' machine: (finished: must fail: journalctl -u slow | grep -q 'asking the scheduler again', in 0.02 seconds) machine: must succeed: nix eval --raw -f /nix/store/zffvrw2ldylpxwxig3cnh0pmwsfs2fm8-slow.nix --argstr tag s1 outPath machine # [ 591.412604] systemd[1]: cri-containerd-f9ba5a63eb0462b76d644667b92ae8ac19e4e5f62719d693b31fa40b539464af.scope: Deactivated successfully. machine: (finished: must succeed: nix eval --raw -f /nix/store/zffvrw2ldylpxwxig3cnh0pmwsfs2fm8-slow.nix --argstr tag s1 outPath, in 0.13 seconds) machine: must succeed: curl -sf http://192.168.1.1:5751/yn9npyvhypx0s0v0rp739lvy4yjqph7k.narinfo > /dev/null machine # [ 591.414445] systemd[1]: cri-containerd-f9ba5a63eb0462b76d644667b92ae8ac19e4e5f62719d693b31fa40b539464af.scope: Consumed 22.119s CPU time over 8min 59.376s wall clock time, 56.1M memory peak, 39M read from disk, 1.2M written to disk, 48.4K incoming IP traffic, 32.9K outgoing IP traffic. machine: (finished: must succeed: curl -sf http://192.168.1.1:5751/yn9npyvhypx0s0v0rp739lvy4yjqph7k.narinfo > /dev/null, in 0.03 seconds) (finished: subtest: deleting a worker pod mid-build drains it: the build finishes there and publishes, in 46.56 seconds) (finished: run the VM test script, in 592.38 seconds) machine # [ 591.471729] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-f9ba5a63eb0462b76d644667b92ae8ac19e4e5f62719d693b31fa40b539464af-rootfs.mount: Deactivated successfully. test script finished in 592.41s cleanup kill QemuMachine (pid 45) machine # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14) machine # [2026-09-23T04:52:18Z INFO virtiofsd] Client disconnected, shutting down machine # [2026-09-23T04:52:18Z INFO virtiofsd] Client disconnected, shutting down machine # [2026-09-23T04:52:18Z INFO virtiofsd] Client disconnected, shutting down (finished: cleanup, in 1.28 seconds) additionally exposed symbols: machine, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh