tribuchet: building on jamie Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 1200.0 seconds run the VM test script machine: waiting for unit k3s.service machine: waiting for the VM to finish booting machine: starting vm machine: QEMU running (pid 45) machine # Disk image does not exist, creating the virtualisation disk image... machine # Formatting '/build/vm-state-machine/tmp.PaQARahDpI', fmt=raw size=17179869184 machine # mke2fs 1.47.4 (6-Mar-2025) machine # Discarding device blocks: 0/4194304 done machine # Creating filesystem with 4194304 4k blocks and 1048576 inodes machine # Filesystem UUID: 54a48cd3-4a85-413f-b473-807784e0b114 machine # Superblock backups stored on blocks: machine # 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, machine # 4096000 machine # machine # Allocating group tables: 0/128 done machine # Writing inode tables: 0/128 done machine # Creating journal (32768 blocks): done machine # Writing superblocks and filesystem accounting information: 0/128 done machine # machine # Virtualisation disk image created. machine # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) machine # machine # machine # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+7EFCC730+7EF2C730 CA00 machine # Press Ctrl-B to configure iPXE (PCI 00:02.0)... machine # machine # machine # machine # machine # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 7EFCC730 7EF2C730 CB00 machine # Press Ctrl-B to configure iPXE (PCI 00:08.0)... machine # machine # machine # Booting from ROM... machine # Probing EDD (edd=off to disable)... ok machine # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026 machine # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ahsbs2m4fkxv6mpvp82cia2apdq6kcry-nixos-system-machine-test/init regInfo=/nix/store/2r4bgqq1hlnv6nd0nz1k9af6hfcihzmi-closure-info/registration console=ttyS0,115200n8 console=tty0 machine # [ 0.000000] BIOS-provided physical RAM map: machine # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable machine # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved machine # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved machine # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007ffd7fff] usable machine # [ 0.000000] BIOS-e820: [mem 0x000000007ffd8000-0x000000007fffffff] reserved machine # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved machine # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved machine # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved machine # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved machine # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000017fffffff] usable machine # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved machine # [ 0.000000] NX (Execute Disable) protection: active machine # [ 0.000000] APIC: Static calls initialized machine # [ 0.000000] SMBIOS 2.8 present. machine # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 machine # [ 0.000000] DMI: Memory slots populated: 1/1 machine # [ 0.000000] Hypervisor detected: KVM machine # [ 0.000000] last_pfn = 0x7ffd8 max_arch_pfn = 0x10000000000 machine # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 machine # [ 0.000001] kvm-clock: using sched offset of 538890649 cycles machine # [ 0.000003] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns machine # [ 0.000007] tsc: Detected 2400.012 MHz processor machine # [ 0.000933] last_pfn = 0x180000 max_arch_pfn = 0x10000000000 machine # [ 0.000961] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs machine # [ 0.000964] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT machine # [ 0.001005] last_pfn = 0x7ffd8 max_arch_pfn = 0x10000000000 machine # [ 0.002935] found SMP MP-table at [mem 0x000f5450-0x000f545f] machine # [ 0.002952] Using GB pages for direct mapping machine # [ 0.003032] RAMDISK: [mem 0x7e367000-0x7ffcffff] machine # [ 0.003043] ACPI: Early table checksum verification disabled machine # [ 0.003047] ACPI: RSDP 0x00000000000F5250 000014 (v00 BOCHS ) machine # [ 0.003052] ACPI: RSDT 0x000000007FFE2539 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) machine # [ 0.003057] ACPI: FACP 0x000000007FFE2319 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001) machine # [ 0.003069] ACPI: DSDT 0x000000007FFE0040 0022D9 (v01 BOCHS BXPC 00000001 BXPC 00000001) machine # [ 0.003071] ACPI: FACS 0x000000007FFE0000 000040 machine # [ 0.003073] ACPI: APIC 0x000000007FFE240D 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) machine # [ 0.003074] ACPI: HPET 0x000000007FFE249D 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) machine # [ 0.003076] ACPI: MCFG 0x000000007FFE24D5 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001) machine # [ 0.003078] ACPI: WAET 0x000000007FFE2511 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) machine # [ 0.003079] ACPI: Reserving FACP table memory at [mem 0x7ffe2319-0x7ffe240c] machine # [ 0.003080] ACPI: Reserving DSDT table memory at [mem 0x7ffe0040-0x7ffe2318] machine # [ 0.003081] ACPI: Reserving FACS table memory at [mem 0x7ffe0000-0x7ffe003f] machine # [ 0.003081] ACPI: Reserving APIC table memory at [mem 0x7ffe240d-0x7ffe249c] machine # [ 0.003082] ACPI: Reserving HPET table memory at [mem 0x7ffe249d-0x7ffe24d4] machine # [ 0.003082] ACPI: Reserving MCFG table memory at [mem 0x7ffe24d5-0x7ffe2510] machine # [ 0.003083] ACPI: Reserving WAET table memory at [mem 0x7ffe2511-0x7ffe2538] machine # [ 0.003343] No NUMA configuration found machine # [ 0.003344] Faking a node at [mem 0x0000000000000000-0x000000017fffffff] machine # [ 0.003346] NODE_DATA(0) allocated [mem 0x17fffa780-0x17ffffcff] machine # [ 0.005833] Zone ranges: machine # [ 0.005834] DMA [mem 0x0000000000001000-0x0000000000ffffff] machine # [ 0.005836] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] machine # [ 0.005838] Normal [mem 0x0000000100000000-0x000000017fffffff] machine # [ 0.005839] Device empty machine # [ 0.005840] Movable zone start for each node machine # [ 0.005841] Early memory node ranges machine # [ 0.005841] node 0: [mem 0x0000000000001000-0x000000000009efff] machine # [ 0.005842] node 0: [mem 0x0000000000100000-0x000000007ffd7fff] machine # [ 0.005843] node 0: [mem 0x0000000100000000-0x000000017fffffff] machine # [ 0.005844] Initmem setup node 0 [mem 0x0000000000001000-0x000000017fffffff] machine # [ 0.005867] On node 0, zone DMA: 1 pages in unavailable ranges machine # [ 0.006160] On node 0, zone DMA: 97 pages in unavailable ranges machine # [ 0.085532] On node 0, zone Normal: 40 pages in unavailable ranges machine # [ 0.086053] ACPI: PM-Timer IO Port: 0x608 machine # [ 0.086071] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) machine # [ 0.086110] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 machine # [ 0.086113] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) machine # [ 0.086115] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) machine # [ 0.086116] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) machine # [ 0.086117] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) machine # [ 0.086118] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) machine # [ 0.086121] ACPI: Using ACPI (MADT) for SMP configuration information machine # [ 0.086122] ACPI: HPET id: 0x8086a201 base: 0xfed00000 machine # [ 0.086128] TSC deadline timer available machine # [ 0.086132] CPU topo: Max. logical packages: 1 machine # [ 0.086133] CPU topo: Max. logical dies: 1 machine # [ 0.086134] CPU topo: Max. dies per package: 1 machine # [ 0.086137] CPU topo: Max. threads per core: 1 machine # [ 0.086138] CPU topo: Num. cores per package: 4 machine # [ 0.086138] CPU topo: Num. threads per package: 4 machine # [ 0.086138] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs machine # [ 0.086165] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() machine # [ 0.086186] kvm-guest: KVM setup pv remote TLB flush machine # [ 0.086188] kvm-guest: setup PV sched yield machine # [ 0.086199] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] machine # [ 0.086201] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] machine # [ 0.086202] PM: hibernation: Registered nosave memory: [mem 0x7ffd8000-0xffffffff] machine # [ 0.086203] [mem 0xc0000000-0xfed1bfff] available for PCI devices machine # [ 0.086206] Booting paravirtualized kernel on KVM machine # [ 0.086211] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns machine # [ 0.090781] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 machine # [ 0.093249] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288 machine # [ 0.093307] kvm-guest: PV spinlocks enabled machine # [ 0.093310] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) machine # [ 0.093313] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ahsbs2m4fkxv6mpvp82cia2apdq6kcry-nixos-system-machine-test/init regInfo=/nix/store/2r4bgqq1hlnv6nd0nz1k9af6hfcihzmi-closure-info/registration console=ttyS0,115200n8 console=tty0 machine # [ 0.093417] Unknown kernel command line parameters "regInfo=/nix/store/2r4bgqq1hlnv6nd0nz1k9af6hfcihzmi-closure-info/registration", will be passed to user space. machine # [ 0.093433] random: crng init done machine # [ 0.093434] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes machine # [ 0.097981] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) machine # [ 0.100349] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) machine # [ 0.100435] software IO TLB: area num 4. machine # [ 0.180876] Fallback order for Node 0: 0 machine # [ 0.180886] Built 1 zonelists, mobility grouping on. Total pages: 1048438 machine # [ 0.180889] Policy zone: Normal machine # [ 0.183573] mem auto-init: stack:all(zero), heap alloc:on, heap free:off machine # [ 0.191531] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 machine # [ 0.201617] allocated 8388608 bytes of page_ext machine # [ 0.212709] ftrace: allocating 48731 entries in 192 pages machine # [ 0.212712] ftrace: allocated 192 pages with 2 groups machine # [ 0.213746] Dynamic Preempt: lazy machine # [ 0.214009] rcu: Preemptible hierarchical RCU implementation. machine # [ 0.214010] rcu: RCU event tracing is enabled. machine # [ 0.214011] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4. machine # [ 0.214013] Trampoline variant of Tasks RCU enabled. machine # [ 0.214014] Rude variant of Tasks RCU enabled. machine # [ 0.214014] Tracing variant of Tasks RCU enabled. machine # [ 0.214015] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. machine # [ 0.214016] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 machine # [ 0.214039] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. machine # [ 0.214041] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. machine # [ 0.214042] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. machine # [ 0.218533] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16 machine # [ 0.218880] rcu: srcu_init: Setting srcu_struct sizes based on contention. machine # [ 0.218890] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns machine # [ 0.219020] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) machine # [ 0.222661] Console: colour VGA+ 80x25 machine # [ 0.222665] printk: legacy console [tty0] enabled machine # [ 0.254696] printk: legacy console [ttyS0] enabled machine # [ 0.359885] ACPI: Core revision 20250807 machine # [ 0.360789] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns machine # [ 0.362331] APIC: Switch to symmetric I/O mode setup machine # [ 0.363310] x2apic enabled machine # [ 0.364065] APIC: Switched APIC routing to: physical x2apic machine # [ 0.364958] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask() machine # [ 0.366099] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself() machine # [ 0.367557] kvm-guest: setup PV IPIs machine # [ 0.369135] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 machine # [ 0.370142] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns machine # [ 0.374533] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012) machine # [ 0.375620] x86/cpu: User Mode Instruction Prevention (UMIP) activated machine # [ 0.376679] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 machine # [ 0.378261] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 machine # [ 0.379336] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto machine # [ 0.380530] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl machine # [ 0.381530] Transient Scheduler Attacks: Vulnerable: No microcode machine # [ 0.382529] Spectre V2 : Mitigation: Enhanced / Automatic IBRS machine # [ 0.383463] Speculative Return Stack Overflow: Mitigation: Safe RET machine # [ 0.384529] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization machine # [ 0.385542] Spectre V2 : Enabling IBPB for BPF machine # [ 0.386531] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier machine # [ 0.387531] active return thunk: srso_alias_return_thunk machine # [ 0.389259] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' machine # [ 0.390529] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' machine # [ 0.391530] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' machine # [ 0.392530] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' machine # [ 0.393529] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' machine # [ 0.394530] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' machine # [ 0.395530] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' machine # [ 0.396530] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 machine # [ 0.398368] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 machine # [ 0.399529] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 machine # [ 0.400530] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 machine # [ 0.401529] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 machine # [ 0.402541] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format. machine # [ 0.436042] Freeing SMP alternatives memory: 44K machine # [ 0.436535] pid_max: default: 32768 minimum: 301 machine # [ 0.437652] LSM: initializing lsm=capability,landlock,yama,bpf,ima machine # [ 0.438658] landlock: Up and running. machine # [ 0.439530] Yama: becoming mindful. machine # [ 0.440715] LSM support for eBPF active machine # [ 0.441612] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear) machine # [ 0.442603] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear) machine # [ 0.445732] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) machine # [ 0.447162] Performance Events: Fam17h+ core perfctr, AMD PMU driver. machine # [ 0.447541] ... version: 2 machine # [ 0.448336] ... bit width: 48 machine # [ 0.448532] ... generic counters: 6 machine # [ 0.449358] ... generic bitmap: 000000000000003f machine # [ 0.450343] ... fixed-purpose counters: 0 machine # [ 0.451116] ... fixed-purpose bitmap: 0000000000000000 machine # [ 0.451531] ... value mask: 0000ffffffffffff machine # [ 0.452494] ... max period: 00007fffffffffff machine # [ 0.453307] ... global_ctrl mask: 000000000000003f machine # [ 0.453660] signal: max sigframe size: 3376 machine # [ 0.454539] rcu: Hierarchical SRCU implementation. machine # [ 0.455460] rcu: Max phase no-delay instances is 400. machine # [ 0.456471] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level machine # [ 0.462425] smp: Bringing up secondary CPUs ... machine # [ 0.463660] smpboot: x86: Booting SMP configuration: machine # [ 0.464538] .... node #0, CPUs: #1 #2 #3 machine # [ 0.466637] smp: Brought up 1 node, 4 CPUs machine # [ 0.468532] smpboot: Total of 4 processors activated (19200.09 BogoMIPS) machine # [ 0.469810] Memory: 3957540K/4193752K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 220888K reserved, 0K cma-reserved) machine # [ 0.470936] devtmpfs: initialized machine # [ 0.472291] x86/mm: Memory block size: 128MB machine # [ 0.475581] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear) machine # [ 0.476614] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear). machine # [ 0.477645] pinctrl core: initialized pinctrl subsystem machine # [ 0.478862] PM: RTC time: 21:20:26, date: 2026-09-19 machine # [ 0.482518] NET: Registered PF_NETLINK/PF_ROUTE protocol family machine # [ 0.484181] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations machine # [ 0.484571] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations machine # [ 0.486105] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations machine # [ 0.486546] audit: initializing netlink subsys (disabled) machine # [ 0.487714] audit: type=2000 audit(1789852826.785:1): state=initialized audit_enabled=0 res=1 machine # [ 0.487842] thermal_sys: Registered thermal governor 'fair_share' machine # [ 0.488535] thermal_sys: Registered thermal governor 'bang_bang' machine # [ 0.489532] thermal_sys: Registered thermal governor 'step_wise' machine # [ 0.490533] thermal_sys: Registered thermal governor 'user_space' machine # [ 0.491534] thermal_sys: Registered thermal governor 'power_allocator' machine # [ 0.492550] cpuidle: using governor menu machine # [ 0.494768] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 machine # [ 0.495813] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff] machine # [ 0.496537] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry machine # [ 0.497552] PCI: Using configuration type 1 for base access machine # [ 0.498678] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. machine # [ 0.501666] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages machine # [ 0.503552] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page machine # [ 0.504541] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages machine # [ 0.505538] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page machine # [ 0.512163] ACPI: Added _OSI(Module Device) machine # [ 0.514543] ACPI: Added _OSI(Processor Device) machine # [ 0.516540] ACPI: Added _OSI(Processor Aggregator Device) machine # [ 0.525789] ACPI: 1 ACPI AML tables successfully acquired and loaded machine # [ 0.532017] ACPI: Interpreter enabled machine # [ 0.532644] ACPI: PM: (supports S0 S3 S4 S5) machine # [ 0.533419] ACPI: Using IOAPIC for interrupt routing machine # [ 0.534367] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug machine # [ 0.535533] PCI: Using E820 reservations for host bridge windows machine # [ 0.536788] ACPI: Enabled 2 GPEs in block 00 to 3F machine # [ 0.543620] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) machine # [ 0.544541] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] machine # [ 0.545642] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR] machine # [ 0.547675] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability] machine # [ 0.549617] PCI host bridge to bus 0000:00 machine # [ 0.550450] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] machine # [ 0.551408] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] machine # [ 0.552403] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] machine # [ 0.554479] pci_bus 0000:00: root bus resource [mem 0x80000000-0xafffffff window] machine # [ 0.555449] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] machine # [ 0.556492] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe07ffffffff window] machine # [ 0.557535] pci_bus 0000:00: root bus resource [bus 00-ff] machine # [ 0.558684] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint machine # [ 0.561137] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint machine # [ 0.582628] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] machine # [ 0.584547] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] machine # [ 0.585537] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref] machine # [ 0.586774] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] machine # [ 0.588525] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint machine # [ 0.592554] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f] machine # [ 0.593448] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] machine # [ 0.595555] pci 0000:00:02.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref] machine # [ 0.596537] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] machine # [ 0.598227] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint machine # [ 0.601559] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f] machine # [ 0.602462] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] machine # [ 0.604323] pci 0000:00:03.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref] machine # [ 0.606193] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint machine # [ 0.609555] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf] machine # [ 0.610469] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] machine # [ 0.611368] pci 0000:00:04.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref] machine # [ 0.613175] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint machine # [ 0.617541] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f] machine # [ 0.618488] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] machine # [ 0.619321] pci 0000:00:05.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref] machine # [ 0.621221] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint machine # [ 0.624562] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf] machine # [ 0.625454] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] machine # [ 0.626319] pci 0000:00:06.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref] machine # [ 0.629284] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint machine # [ 0.632537] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f] machine # [ 0.633469] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] machine # [ 0.635328] pci 0000:00:07.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref] machine # [ 0.637221] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint machine # [ 0.653730] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df] machine # [ 0.656559] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] machine # [ 0.658596] pci 0000:00:08.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref] machine # [ 0.661553] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref] machine # [ 0.667256] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint machine # [ 0.676570] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] machine # [ 0.678593] pci 0000:00:09.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref] machine # [ 0.684439] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint machine # [ 0.694995] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff] machine # [ 0.696556] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] machine # [ 0.699610] pci 0000:00:0a.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref] machine # [ 0.704043] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint machine # [ 0.712555] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff] machine # [ 0.714548] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff] machine # [ 0.715578] pci 0000:00:0b.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref] machine # [ 0.720197] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint machine # [ 0.723285] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f] machine # [ 0.724902] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint machine # [ 0.727563] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f] machine # [ 0.729016] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint machine # [ 0.731528] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f] machine # [ 0.732654] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint machine # [ 0.735542] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff] machine # [ 0.736784] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint machine # [ 0.737849] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO machine # [ 0.739799] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint machine # [ 0.743564] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f] machine # [ 0.744457] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff] machine # [ 0.745671] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint machine # [ 0.748565] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f] machine # [ 0.753332] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 machine # [ 0.754413] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 machine # [ 0.755439] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 machine # [ 0.756409] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 machine # [ 0.757425] ACPI: PCI: Interrupt link LNKE configured for IRQ 10 machine # [ 0.758404] ACPI: PCI: Interrupt link LNKF configured for IRQ 10 machine # [ 0.759491] ACPI: PCI: Interrupt link LNKG configured for IRQ 11 machine # [ 0.761551] ACPI: PCI: Interrupt link LNKH configured for IRQ 11 machine # [ 0.762574] ACPI: PCI: Interrupt link GSIA configured for IRQ 16 machine # [ 0.763551] ACPI: PCI: Interrupt link GSIB configured for IRQ 17 machine # [ 0.764551] ACPI: PCI: Interrupt link GSIC configured for IRQ 18 machine # [ 0.765549] ACPI: PCI: Interrupt link GSID configured for IRQ 19 machine # [ 0.766547] ACPI: PCI: Interrupt link GSIE configured for IRQ 20 machine # [ 0.767546] ACPI: PCI: Interrupt link GSIF configured for IRQ 21 machine # [ 0.768550] ACPI: PCI: Interrupt link GSIG configured for IRQ 22 machine # [ 0.769546] ACPI: PCI: Interrupt link GSIH configured for IRQ 23 machine # [ 0.772815] iommu: Default domain type: Translated machine # [ 0.773533] iommu: DMA domain TLB invalidation policy: lazy mode machine # [ 0.774840] ACPI: bus type USB registered machine # [ 0.775604] usbcore: registered new interface driver usbfs machine # [ 0.776547] usbcore: registered new interface driver hub machine # [ 0.777561] usbcore: registered new device driver usb machine # [ 0.780207] NetLabel: Initializing machine # [ 0.781533] NetLabel: domain hash size = 128 machine # [ 0.782276] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO machine # [ 0.782712] NetLabel: unlabeled traffic allowed by default machine # [ 0.783546] PCI: Using ACPI for IRQ routing machine # [ 0.828568] pci 0000:00:01.0: vgaarb: setting as boot VGA device machine # [ 0.829528] pci 0000:00:01.0: vgaarb: bridge control possible machine # [ 0.829528] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none machine # [ 0.831543] vgaarb: loaded machine # [ 0.832302] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 machine # [ 0.832532] hpet0: 3 comparators, 64-bit 100.000000 MHz counter machine # [ 0.837689] clocksource: Switched to clocksource kvm-clock machine # [ 0.843403] VFS: Disk quotas dquot_6.6.0 machine # [ 0.844168] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) machine # [ 0.845939] pnp: PnP ACPI init machine # [ 0.846845] ACPI: IRQ 4 override to edge(!), high(!) machine # [ 0.847850] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved machine # [ 0.849439] pnp: PnP ACPI: found 6 devices machine # [ 0.859807] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns machine # [ 0.861422] clocksource: Switched to clocksource acpi_pm machine # [ 0.862457] NET: Registered PF_INET protocol family machine # [ 0.864077] IP idents hash table entries: 65536 (order: 7, 524288 bytes, linear) machine # [ 0.881877] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear) machine # [ 0.883477] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) machine # [ 0.885063] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear) machine # [ 0.887597] TCP bind hash table entries: 32768 (order: 8, 1048576 bytes, linear) machine # [ 0.888964] TCP: Hash tables configured (established 32768 bind 32768) machine # [ 0.890341] MPTCP token hash table entries: 4096 (order: 5, 98304 bytes, linear) machine # [ 0.891700] UDP hash table entries: 2048 (order: 5, 131072 bytes, linear) machine # [ 0.892881] UDP-Lite hash table entries: 2048 (order: 5, 131072 bytes, linear) machine # [ 0.894233] NET: Registered PF_UNIX/PF_LOCAL protocol family machine # [ 0.895284] NET: Registered PF_XDP protocol family machine # [ 0.896184] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] machine # [ 0.897264] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] machine # [ 0.898338] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] machine # [ 0.899516] pci_bus 0000:00: resource 7 [mem 0x80000000-0xafffffff window] machine # [ 0.900723] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window] machine # [ 0.901901] pci_bus 0000:00: resource 9 [mem 0xe0000000000-0xe07ffffffff window] machine # [ 0.903930] ACPI: \_SB_.GSIA: Enabled at IRQ 16 machine # [ 0.906148] ACPI: \_SB_.GSIB: Enabled at IRQ 17 machine # [ 0.908349] ACPI: \_SB_.GSIC: Enabled at IRQ 18 machine # [ 0.910505] ACPI: \_SB_.GSID: Enabled at IRQ 19 machine # [ 0.912307] PCI: CLS 0 bytes, default 64 machine # [ 0.913151] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) machine # [ 0.913733] Trying to unpack rootfs image as initramfs... machine # [ 0.914311] software IO TLB: mapped [mem 0x000000007a367000-0x000000007e367000] (64MB) machine # [ 0.917253] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns machine # [ 0.938695] Initialise system trusted keyrings machine # [ 0.940142] workingset: timestamp_bits=40 max_order=20 bucket_order=0 machine # [ 0.951809] Key type asymmetric registered machine # [ 0.952603] Asymmetric key parser 'x509' registered machine # [ 0.953477] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) machine # [ 0.955142] io scheduler mq-deadline registered machine # [ 0.955980] io scheduler kyber registered machine # [ 0.961750] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled machine # [ 0.963181] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A machine # [ 0.965861] Linux agpgart interface v0.103 machine # [ 0.966676] ACPI: bus type drm_connector registered machine # [ 0.969359] usbcore: registered new interface driver usbserial_generic machine # [ 0.970452] usbserial: USB Serial support registered for generic machine # [ 0.971431] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. machine # [ 0.971431] Please enable it if your BIOS has the CPPC option. machine # [ 0.973713] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled machine # [ 0.975170] drop_monitor: Initializing network drop monitor service machine # [ 0.976423] NET: Registered PF_INET6 protocol family machine # [ 0.978445] Segment Routing with IPv6 machine # [ 0.979163] In-situ OAM (IOAM) with IPv6 machine # [ 0.982532] IPI shorthand broadcast: enabled machine # [ 0.994420] sched_clock: Marking stable (846035833, 147638252)->(1018426158, -24752073) machine # [ 0.999007] registered taskstats version 1 machine # [ 1.002027] Loading compiled-in X.509 certificates machine # [ 1.013855] Demotion targets for Node 0: null machine # [ 1.015247] Key type .fscrypt registered machine # [ 1.016030] Key type fscrypt-provisioning registered machine # [ 1.017097] ima: No TPM chip found, activating TPM-bypass! machine # [ 1.018283] ima: Allocated hash algorithm: sha1 machine # [ 1.019904] ima: No architecture policies found machine # [ 1.021154] PM: Magic number: 10:945:348 machine # [ 1.022975] RAS: Correctable Errors collector initialized. machine # [ 1.027704] clk: Disabling unused clocks machine # [ 1.028453] PM: genpd: Disabling unused power domains machine # [ 1.225423] Freeing initrd memory: 29092K machine # [ 1.229688] Freeing unused decrypted memory: 2028K machine # [ 1.233029] Freeing unused kernel image (initmem) memory: 3644K machine # [ 1.234336] Write protecting the kernel read-only data: 32768k machine # [ 1.236789] Freeing unused kernel image (text/rodata gap) memory: 1208K machine # [ 1.238753] Freeing unused kernel image (rodata/data gap) memory: 752K machine # [ 1.294776] x86/mm: Checked W+X mappings: passed, no W+X pages found. machine # [ 1.295952] Run /init as init process machine # [ 1.307400] systemd[1]: Inserted module 'autofs4' machine # [ 1.329327] fuse: init (API version 7.45) machine # [ 1.336723] ACPI: \_SB_.GSIG: Enabled at IRQ 22 machine # [ 1.339366] ACPI: \_SB_.GSIH: Enabled at IRQ 23 machine # [ 1.342782] ACPI: \_SB_.GSIE: Enabled at IRQ 20 machine # [ 1.345064] ACPI: \_SB_.GSIF: Enabled at IRQ 21 machine # [ 1.394030] systemd[1]: Successfully made /usr/ read-only. machine # [ 1.730252] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) machine # [ 1.736552] systemd[1]: Detected virtualization kvm. machine # [ 1.738352] systemd[1]: Detected architecture x86-64. machine # [ 1.740191] systemd[1]: Running in initrd. machine # [ 1.742350] systemd[1]: Initializing machine ID from random generator. machine # [ 1.745303] systemd[1]: Hostname set to . machine # [ 1.867190] systemd[1]: bpf-restrict-fs: LSM BPF program attached machine # [ 1.903442] systemd[1]: Queued start job for default target Initrd Default Target. machine # [ 1.913249] systemd[1]: Created slice Slice /system/modprobe. machine # [ 1.914480] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. machine # [ 1.915912] systemd[1]: Expecting device /dev/disk/by-label/nixos... machine # [ 1.917018] systemd[1]: Reached target Path Units. machine # [ 1.917894] systemd[1]: Reached target Slice Units. machine # [ 1.918772] systemd[1]: Reached target Swaps. machine # [ 1.919563] systemd[1]: Reached target Timer Units. machine # [ 1.920542] systemd[1]: Listening on D-Bus System Message Bus Socket. machine # [ 1.921806] systemd[1]: Listening on Journal Socket (/dev/log). machine # [ 1.922944] systemd[1]: Listening on Journal Sockets. machine # [ 1.923962] systemd[1]: Listening on udev Control Socket. machine # [ 1.925007] systemd[1]: Listening on udev Kernel Socket. machine # [ 1.925986] systemd[1]: Reached target Socket Units. machine # [ 1.927999] systemd[1]: Starting Create List of Static Device Nodes... machine # [ 1.929967] systemd[1]: Starting Load Kernel Module 9pnet_virtio... machine # [ 1.931960] systemd[1]: Starting Load Kernel Module configfs... machine # [ 1.934713] systemd[1]: Starting Journal Service... machine # [ 1.938737] systemd[1]: Starting Load Kernel Modules... machine # [ 1.940969] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os machine # [ 1.944795] systemd[1]: Starting Coldplug All udev Devices... machine # [ 1.950498] systemd[1]: modprobe@configfs.service: Deactivated successfully. machine # [ 1.952211] systemd[1]: Finished Load Kernel Module configfs. machine # [ 1.954320] systemd[1]: Finished Create List of Static Device Nodes. machine # [ 1.955952] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config machine # [ 1.958111] systemd-journald[91]: Collecting audit messages is disabled. machine # [ 1.960028] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... machine # [ 1.982597] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. machine # [ 1.989840] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev machine # [ 1.990300] netfs: FS-Cache loaded machine # [ 1.993317] systemd[1]: Finished Load Kernel Modules. machine # [ 1.995952] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. machine # [ 1.999021] systemd[1]: Starting Apply Kernel Variables... machine # [ 1.999614] 9pnet: Installing 9P2000 support machine # [ 2.004727] systemd[1]: Starting Create Static Device Nodes in /dev... machine # [ 2.007550] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. machine # [ 2.009313] systemd[1]: Finished Load Kernel Module 9pnet_virtio. machine # [ 2.012985] systemd[1]: Started Journal Service. machine # [ 1.864830] systemd-modules-load[93]: Using 4 probe threads machine # [ 1.867101] systemd-modules-load[93]: Inserted module 'virtio_balloon' machine # [ 1.868370] systemd-modules-load[93]: Inserted module 'virtio_gpu' machine # [ 1.869554] systemd-modules-load[93]: Inserted module 'dm_mod' machine # [ 1.870614] systemd[1]: Finished Apply Kernel Variables. machine # [ 1.871810] systemd[1]: Finished Create Static Device Nodes in /dev. machine # [ 1.872909] systemd[1]: Reached target Preparation for Local File Systems. machine # [ 1.874070] systemd[1]: Reached target Local File Systems. machine # [ 1.874965] systemd[1]: Starting Create System Files and Directories... machine # [ 1.876051] systemd[1]: Starting Rule-based Manager for Device Events and Files... machine # [ 1.887055] systemd[1]: Finished Create System Files and Directories. machine # [ 1.896201] systemd-udevd[108]: Using default interface naming scheme 'v261'. machine # [ 1.906604] systemd[1]: Started Rule-based Manager for Device Events and Files. machine # [ 1.950894] systemd[1]: Finished Coldplug All udev Devices. machine # [ 1.951838] systemd[1]: Reached target System Initialization. machine # [ 1.952725] systemd[1]: Reached target Basic System. machine # [ 2.173298] ehci-pci 0000:00:1d.7: EHCI Host Controller machine # [ 2.174360] ehci-pci 0000:00:1d.7: new USB bus registered, assigned bus number 1 machine # [ 2.176274] ehci-pci 0000:00:1d.7: irq 19, io mem 0xfebdb000 machine # [ 2.180573] virtio_blk virtio5: 4/0/0 default/read/poll queues machine # [ 2.182682] ehci-pci 0000:00:1d.7: USB 2.0 started, EHCI 1.00 machine # [ 2.185418] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.18 machine # [ 2.186734] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 machine # [ 2.187999] usb usb1: Product: EHCI Host Controller machine # [ 2.188821] usb usb1: Manufacturer: Linux 6.18.50 ehci_hcd machine # [ 2.189616] usb usb1: SerialNumber: 0000:00:1d.7 machine # [ 2.194839] virtio_blk virtio5: [vda] 33554432 512-byte logical blocks (17.2 GB/16.0 GiB) machine # [ 2.203323] hub 1-0:1.0: USB hub found machine # [ 2.204034] hub 1-0:1.0: 6 ports detected machine # [ 2.244186] uhci_hcd 0000:00:1d.0: UHCI Host Controller machine # [ 2.245100] uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 2 machine # [ 2.246229] uhci_hcd 0000:00:1d.0: detected 2 ports machine # [ 2.247185] uhci_hcd 0000:00:1d.0: irq 16, io port 0x0000c200 machine # [ 2.249519] usb usb2: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 machine # [ 2.251123] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1 machine # [ 2.252272] usb usb2: Product: UHCI Host Controller machine # [ 2.253086] usb usb2: Manufacturer: Linux 6.18.50 uhci_hcd machine # [ 2.254008] usb usb2: SerialNumber: 0000:00:1d.0 machine # [ 2.259530] hub 2-0:1.0: USB hub found machine # [ 2.262547] hub 2-0:1.0: 2 ports detected machine # [ 2.262888] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 machine # [ 2.265298] uhci_hcd 0000:00:1d.1: UHCI Host Controller machine # [ 2.266650] uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 3 machine # [ 2.119654] (udev-worker)[126]: Network inter[ 2.267729] serio: i8042 KBD port at 0x60,0x64 irq 1 machine # face NamePolicy= disabled on kernel command line[ 2.268010] uhci_hcd 0000:00:1d.1: detected 2 ports machine # . machine # [ 2.268180] serio: i8042 AUX port at 0x60,0x64 irq 12 machine # [ 2.269496] SCSI subsystem initialized machine # [ 2.271805] uhci_hcd 0000:00:1d.1: irq 17, io port 0x0000c220 machine # [ 2.125166] (udev-worker)[139]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. machine # [ 2.127516] (udev-worker)[139]: Network interface NamePolicy= disabled on kernel command line. machine # [ 2.277018] usb usb3: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 machine # [ 2.278208] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1 machine # [ 2.131904] systemd[1]: Starting Virtual Console Setup... machine # [ 2.135803] systemd[1]: Found device /dev/disk/by-label/nixos. machine # [ 2.137103] systemd[1]: Reached target Initrd Root Device. machine # [ 2.287722] usb usb3: Product: UHCI Host Controller machine # [ 2.288480] usb usb3: Manufacturer: Linux 6.18.50 uhci_hcd machine # [ 2.288482] usb usb3: SerialNumber: 0000:00:1d.1 machine # [ 2.290866] hub 3-0:1.0: USB hub found machine # [ 2.141849] systemd[1]: Starting File System [ 2.290919] hub 3-0:1.0: 2 ports detected machine # Check on /dev/disk/by-label/nixos...[ 2.291184] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 machine # [ 2.292752] uhci_hcd 0000:00:1d.2: UHCI Host Controller machine # machine # [ 2.296321] uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 4 machine # [ 2.296344] uhci_hcd 0000:00:1d.2: detected 2 ports machine # [ 2.296473] uhci_hcd 0000:00:1d.2: irq 18, io port 0x0000c240 machine # [ 2.302183] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 machine # [ 2.303735] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1 machine # [ 2.304129] usb usb4: Product: UHCI Host Controller machine # [ 2.306395] usb usb4: Manufacturer: Linux 6.18.50 uhci_hcd machine # [ 2.308092] usb usb4: SerialNumber: 0000:00:1d.2 machine # [ 2.161558] systemd-vconsole-setup[145]: Configuration of first virt[ 2.309461] hub 4-0:1.0: USB hub found machine # ual console was skipped, ignoring remaining ones. machine # [ 2.310612] hub 4-0:1.0: 2 ports detected machine # [ 2.168610] systemd[1]: Finished Virtual Console Setup. machine # [ 2.317218] ahci 0000:00:1f.2: AHCI vers 0001.0000, 32 command slots, 1.5 Gbps, SATA mode machine # [ 2.319719] ahci 0000:00:1f.2: 6/6 ports implemented (port mask 0x3f) machine # [ 2.320936] ahci 0000:00:1f.2: flags: 64bit ncq only machine # [ 2.176067] systemd-fsck[152]: nixos: clean, 12/1048576 files, 109865/4194304 blocks machine # [ 2.179639] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. machine # [ 2.328294] scsi host0: ahci machine # [ 2.329353] scsi host1: ahci machine # [ 2.330176] scsi host2: ahci machine # [ 2.331087] scsi host3: ahci machine # [ 2.332476] scsi host4: ahci machine # [ 2.333415] scsi host5: ahci machine # [ 2.334208] ata1: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc100 irq 47 lpm-pol 1 machine # [ 2.335563] ata2: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc180 irq 47 lpm-pol 1 machine # [ 2.336926] ata3: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc200 irq 47 lpm-pol 1 machine # [ 2.338148] ata4: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc280 irq 47 lpm-pol 1 machine # [ 2.339480] ata5: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc300 irq 47 lpm-pol 1 machine # [ 2.340855] ata6: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc380 irq 47 lpm-pol 1 machine # [ 2.478741] usb 1-1: new high-speed USB device number 2 using ehci-pci machine # [ 2.608734] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 machine # [ 2.610065] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 machine # [ 2.611183] usb 1-1: Product: QEMU USB Tablet machine # [ 2.611964] usb 1-1: Manufacturer: QEMU machine # [ 2.612683] usb 1-1: SerialNumber: 28754-0000:00:1d.7-1 machine # [ 2.630213] hid: raw HID events driver (C) Jiri Kosina machine # [ 2.668932] ata4: SATA link down (SStatus 0 SControl 300) machine # [ 2.669938] ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 300) machine # [ 2.672290] ata5: SATA link down (SStatus 0 SControl 300) machine # [ 2.674772] ata1: SATA link down (SStatus 0 SControl 300) machine # [ 2.675754] ata2: SATA link down (SStatus 0 SControl 300) machine # [ 2.676884] ata6: SATA link down (SStatus 0 SControl 300) machine # [ 2.677799] ata3.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 machine # [ 2.679917] ata3.00: applying bridge limits machine # [ 2.681839] ata3.00: configured for UDMA/100 machine # [ 2.684416] scsi 2:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 machine # [ 2.715384] usbcore: registered new interface driver usbhid machine # [ 2.716414] usbhid: USB HID core driver machine # [ 2.721847] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:1d.7/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 machine # [ 2.724048] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:1d.7-1/input0 machine # [ 2.745970] sr 2:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray machine # [ 2.779849] cdrom: Uniform CD-ROM driver Revision: 3.20 machine # [ 2.804728] systemd[1]: Mounting /sysroot... machine # [ 3.054958] EXT4-fs (vda): mounted filesystem 54a48cd3-4a85-413f-b473-807784e0b114 r/w with ordered data mode. Quota mode: none. machine # [ 2.910781] systemd[1]: Mounted /sysroot. machine # [ 2.911726] systemd[1]: Reached target Initrd Root File System. machine # [ 2.913654] systemd[1]: Mounting /sysroot/nix/.ro-store... machine # [ 2.914771] systemd[1]: Mounting /sysroot/nix/.rw-store... machine # [ 2.917213] systemd[1]: Mounting /sysroot/run... machine # [ 2.926121] systemd[1]: Mounting /sysroot/tmp/shared... machine # [ 2.928727] systemd[1]: Mounting /sysroot/tmp/xchg... machine # [ 3.080252] 9p: Installing v9fs 9p2000 file system support machine # [ 2.933577] systemd[1]: Starting Mountpoints Configured in the Real Root... machine # [ 2.941076] systemd[1]: Mounted /sysroot/nix/.ro-store. machine # [ 2.942175] systemd-sysroot-fstab-check[194]: /sysroot should be mounted in the initrd, will request daemon-reload. machine # [ 2.943515] systemd[1]: Mounted /sysroot/nix/.rw-store. machine # [ 2.944396] systemd[1]: Mounted /sysroot/run. machine # [ 2.945159] systemd[1]: Mounted /sysroot/tmp/shared. machine # [ 2.945917] systemd[1]: Mounted /sysroot/tmp/xchg. machine # [ 2.947431] systemd[1]: Starting rw-sysroot-nix-store.service... machine # [ 2.948331] systemd[1]: Reload requested from client PID 194 ('systemd-sysroot') (unit initrd-parse-etc.service)... machine # [ 2.949831] systemd[1]: Reloading... machine # [ 3.003680] systemd[1]: Reloading finished in 55 ms. machine # [ 3.023346] systemd-sysroot-fstab-check[194]: Requesting initrd-fs.target/start/replace... machine # [ 3.024735] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. machine # [ 3.025994] systemd[1]: Finished rw-sysroot-nix-store.service. machine # [ 3.026977] systemd-sysroot-fstab-check[194]: Requesting swap.target/start/replace... machine # [ 3.028253] systemd[1]: Starting rw-sysroot-nix-store.service... machine # [ 3.030144] systemd[1]: initrd-parse-etc.service: Deactivated successfully. machine # [ 3.031330] systemd[1]: Finished Mountpoints Configured in the Real Root. machine # [ 3.032462] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. machine # [ 3.050117] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. machine # [ 3.051160] systemd[1]: Finished rw-sysroot-nix-store.service. machine # [ 3.804965] systemd[1]: Mounting /sysroot/nix/store... machine # [ 3.853613] systemd[1]: Mounted /sysroot/nix/store. machine # [ 3.854822] systemd[1]: Reached target Initrd File Systems. machine # [ 3.856724] systemd[1]: Starting Find NixOS closure... machine # [ 3.858490] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... machine # [ 3.878635] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. machine # [ 3.880103] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. machine # [ 3.895528] systemd[1]: Finished Find NixOS closure. machine # [ 3.896445] systemd[1]: Reached target Initrd Default Target. machine # [ 3.897903] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... machine # [ 3.916666] systemd[1]: Stopped target Initrd Default Target. machine # [ 3.917797] systemd[1]: Stopped target Basic System. machine # [ 3.918822] systemd[1]: Stopped target Initrd Root Device. machine # [ 3.919871] systemd[1]: Stopped target Path Units. machine # [ 3.920812] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. machine # [ 3.922215] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. machine # [ 3.923638] systemd[1]: Stopped target Slice Units. machine # [ 3.924653] systemd[1]: Stopped target Socket Units. machine # [ 3.925662] systemd[1]: Stopped target System Initialization. machine # [ 3.926750] systemd[1]: Stopped target Swaps. machine # [ 3.927655] systemd[1]: Stopped target Timer Units. machine # [ 3.928751] systemd[1]: dbus.socket: Deactivated successfully. machine # [ 3.929935] systemd[1]: Closed D-Bus System Message Bus Socket. machine # [ 3.931250] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. machine # [ 3.932620] systemd[1]: Stopped Find NixOS closure. machine # [ 3.933723] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio machine # [ 3.935652] systemd[1]: Starting rw-sysroot-nix-store.service... machine # [ 3.937251] systemd[1]: systemd-sysctl.service: Deactivated successfully. machine # [ 3.938491] systemd[1]: Stopped Apply Kernel Variables. machine # [ 3.939513] systemd[1]: systemd-modules-load.service: Deactivated successfully. machine # [ 3.940866] systemd[1]: Stopped Load Kernel Modules. machine # [ 3.941845] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. machine # [ 3.943350] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. machine # [ 3.944846] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. machine # [ 3.946320] systemd[1]: Stopped Create System Files and Directories. machine # [ 3.947541] systemd[1]: Stopped target Local File Systems. machine # [ 3.948653] systemd[1]: Stopped target Preparation for Local File Systems. machine # [ 3.949950] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. machine # [ 3.951369] systemd[1]: Stopped Coldplug All udev Devices. machine # [ 3.952465] systemd[1]: Stopping Rule-based Manager for Device Events and Files... machine # [ 3.953867] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. machine # [ 3.955328] systemd[1]: Stopped Virtual Console Setup. machine # [ 3.956473] systemd[1]: initrd-cleanup.service: Deactivated successfully. machine # [ 3.957511] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. machine # [ 3.958518] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. machine # [ 3.959581] systemd[1]: Finished rw-sysroot-nix-store.service. machine # [ 3.960480] systemd[1]: systemd-udevd.service: Deactivated successfully. machine # [ 3.961476] systemd[1]: Stopped Rule-based Manager for Device Events and Files. machine # [ 3.962538] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. machine # [ 3.963651] systemd[1]: Closed udev Control Socket. machine # [ 3.964436] systemd[1]: Starting Cleanup udev Database... machine # [ 3.965271] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. machine # [ 3.966382] systemd[1]: Stopped Create Static Device Nodes in /dev. machine # [ 3.967321] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. machine # [ 3.968498] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. machine # [ 3.969587] systemd[1]: kmod-static-nodes.service: Deactivated successfully. machine # [ 3.970611] systemd[1]: Stopped Create List of Static Device Nodes. machine # [ 3.983816] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. machine # [ 3.985189] systemd[1]: Finished Cleanup udev Database. machine # [ 3.985940] systemd[1]: Reached target Switch Root. machine # [ 3.987243] systemd[1]: Starting NixOS Activation... machine # [ 4.172125] initrd-nixos-activation-start[240]: booting system configuration /nix/store/ahsbs2m4fkxv6mpvp82cia2apdq6kcry-nixos-system-machine-test machine # [ 4.250121] initrd-nixos-activation-start[240]: running activation script... machine # [ 4.699693] initrd-nixos-activation-start[264]: setting up /etc... machine # [ 4.958966] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. machine # [ 4.960203] systemd[1]: Finished NixOS Activation. machine # [ 4.960937] systemd[1]: Starting Switch Root... machine # [ 4.980318] systemd[1]: Switching root. machine # [ 5.178341] systemd-journald[91]: Received SIGTERM from PID 1 (systemd). machine # [ 5.324506] NET: Registered PF_VSOCK protocol family machine # [ 5.739170] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) machine # [ 5.743366] systemd[1]: Detected virtualization kvm. machine # [ 5.744108] systemd[1]: Detected architecture x86-64. machine # [ 5.744942] systemd[1]: Detected first boot. machine # [ 5.749358] systemd[1]: Initializing machine ID from random generator. machine # [ 5.887769] systemd[1]: bpf-restrict-fs: LSM BPF program attached machine # [ 5.994705] systemd[1]: Applying preset policy. machine # [ 6.357097] systemd[1]: Populated /etc with preset unit settings. machine # [ 6.799854] systemd[1]: initrd-switch-root.service: Deactivated successfully. machine # [ 6.801247] systemd[1]: Stopped initrd-switch-root.service. machine # [ 6.803991] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. machine # [ 6.806199] systemd[1]: Created slice Slice /system/getty. machine # [ 6.807744] systemd[1]: Created slice User and Session Slice. machine # [ 6.808652] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. machine # [ 6.809880] systemd[1]: Started Forward Password Requests to Wall Directory Watch. machine # [ 6.811012] systemd[1]: Expecting device /dev/hvc0... machine # [ 6.811767] systemd[1]: Expecting device /dev/ttyS0... machine # [ 6.812508] systemd[1]: Reached target Local Encrypted Volumes. machine # [ 6.813374] systemd[1]: Stopped target initrd-fs.target. machine # [ 6.814157] systemd[1]: Stopped target initrd-root-fs.target. machine # [ 6.815014] systemd[1]: Stopped target initrd-switch-root.target. machine # [ 6.815928] systemd[1]: Reached target Virtual Machines and Containers. machine # [ 6.816886] systemd[1]: Reached target Path Units. machine # [ 6.817638] systemd[1]: Reached target Remote File Systems. machine # [ 6.818422] systemd[1]: Reached target Slice Units. machine # [ 6.819193] systemd[1]: Reached target Swaps. machine # [ 6.822245] systemd[1]: Listening on Query the User Interactively for a Password. machine # [ 6.825892] systemd[1]: Listening on Process Core Dump Socket. machine # [ 6.828689] systemd[1]: Listening on Credential Encryption/Decryption. machine # [ 6.831256] systemd[1]: Listening on Factory Reset Management. machine # [ 6.832341] systemd[1]: Listening on Hostname Service Socket. machine # [ 6.836256] systemd[1]: Starting Journal Log Access Socket... machine # [ 6.838333] systemd[1]: Listening on Journal Audit Socket. machine # [ 6.841375] systemd[1]: Listening on Console Output Muting Service Socket. machine # [ 6.842902] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. machine # [ 6.845082] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os machine # [ 6.846531] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki machine # [ 6.855871] systemd[1]: Listening on Disk Repartitioning Service Socket. machine # [ 6.856973] systemd[1]: Listening on udev Control Socket. machine # [ 6.857935] systemd[1]: Listening on udev Varlink Socket. machine # [ 6.862422] systemd[1]: Mounting Huge Pages File System... machine # [ 6.865974] systemd[1]: Mounting POSIX Message Queue File System... machine # [ 6.870196] systemd[1]: Mounting Kernel Debug File System... machine # [ 6.876790] systemd[1]: Mounting Kernel Trace File System... machine # [ 6.883888] systemd[1]: Starting Create List of Static Device Nodes... machine # [ 6.889218] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio machine # [ 6.903428] systemd[1]: Starting Load Kernel Module configfs... machine # [ 6.906114] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm machine # [ 6.909793] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore machine # [ 6.913856] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse machine # [ 6.926060] systemd[1]: Mounting FUSE Control File System... machine # [ 6.927756] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 machine # [ 6.937603] systemd[1]: Starting Journal Service... machine # [ 6.941226] systemd[1]: Starting Load Kernel Modules... machine # [ 6.945725] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... machine # [ 6.953268] systemd[1]: Starting Remount Root and Kernel File Systems... machine # [ 6.956038] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os machine # [ 6.963885] systemd[1]: Starting Coldplug All udev Devices... machine # [ 6.968457] systemd[1]: Listening on Journal Log Access Socket. machine # [ 6.970102] systemd[1]: Mounted Huge Pages File System. machine # [ 6.972078] systemd[1]: Mounted POSIX Message Queue File System. machine # [ 6.974359] systemd[1]: Mounted Kernel Debug File System. machine # [ 6.976659] systemd[1]: Mounted Kernel Trace File System. machine # [ 6.979248] systemd[1]: Finished Create List of Static Device Nodes. machine # [ 6.981953] systemd[1]: Mounted FUSE Control File System. machine # [ 6.986726] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... machine # [ 6.996355] systemd[1]: modprobe@configfs.service: Deactivated successfully. machine # [ 6.998683] systemd-journald[334]: Collecting audit messages is enabled. machine # [ 7.001304] systemd[1]: Finished Load Kernel Module configfs. machine # [ 7.005975] systemd[1]: Mounting Kernel Configuration File System... machine # [ 7.008686] EXT4-fs (vda): re-mounted 54a48cd3-4a85-413f-b473-807784e0b114. machine # [ 7.017794] loop: module loaded machine # [ 7.018095] systemd[1]: Finished Remount Root and Kernel File Systems. machine # [ 7.020545] systemd[1]: Listening on Disk Image Download Service Socket. machine # [ 7.022137] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore machine # [ 6.874955] systemd[1]: Queued start job for default target Multi-User System. machine # [ 6.878450] systemd[1]: systemd-journald.service: Deactivated successfully. machine # [ 7.026777] systemd[1]: Starting Load/Save OS Random Seed... machine # [ 6.880284] systemd-modules-load[335]: Using 2 probe threads machine # [ 6.882450] s[ 7.028748] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os machine # ystemd-modules-load[335]: Inserted module 'loop' machine # [ 7.032143] systemd[1]: Started Journal Service. machine # [ 6.886744] systemd-oomd[337]: No swap; memory pressure usage will be degraded machine # [ 6.891490] systemd[1]: Finished Load Kernel Modules. machine # [ 6.893193] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. machine # [ 6.899063] systemd[1]: Starting Flush Journal to Persistent Storage... machine # [ 6.902487] systemd[1]: Starting Apply Kernel Variables... machine # [ 6.903694] systemd[1]: Mounted Kernel Configuration File System. machine # [ 6.923334] systemd[1]: Finished Load/Save OS Random Seed. machine # [ 6.924729] systemd[1]: Reached target First Boot Complete. machine # [ 7.079448] systemd-journald[334]: Received client request to flush runtime journal. machine # [ 6.985787] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. machine # [ 6.987690] systemd[1]: Starting Create Static Device Nodes in /dev... machine # [ 6.988970] systemd[1]: Finished Apply Kernel Variables. machine # [ 6.990067] systemd[1]: Finished Flush Journal to Persistent Storage. machine # [ 6.992869] systemd[1]: Finished Create Static Device Nodes in /dev. machine # [ 6.993854] systemd[1]: Reached target Preparation for Local File Systems. machine # [ 6.995553] systemd[1]: Starting Rule-based Manager for Device Events and Files... machine # [ 7.026754] systemd[1]: Finished Coldplug All udev Devices. machine # [ 7.068389] systemd-udevd[366]: Using default interface naming scheme 'v261'. machine # [ 7.163972] systemd[1]: Started Rule-based Manager for Device Events and Files. machine # [ 7.233781] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse machine # [ 7.257935] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. machine # [ 7.273253] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. machine # [ 7.281296] (udev-worker)[378]: Network interface NamePolicy= disabled on kernel command line. machine # [ 7.289371] (udev-worker)[382]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. machine # [ 7.291880] (udev-worker)[382]: Network interface NamePolicy= disabled on kernel command line. machine # [ 7.471596] bochs-drm 0000:00:01.0: vgaarb: deactivate vga console machine # [ 7.330057] systemd[1]: Condition check resulted in Virtio network device being skipped. machine # [ 7.331571] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore machine # [ 7.333602] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 machine # [ 7.336351] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore machine # [ 7.338781] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os machine # [ 7.341164] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os machine # [ 7.497767] Console: switching to colour dummy device 80x25 machine # [ 7.498935] [drm] Found bochs VGA, ID 0xb0c5. machine # [ 7.499696] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. machine # [ 7.502554] lpc_ich 0000:00:1f.0: I/O space for GPIO uninitialized machine # [ 7.505027] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 machine # [ 7.507722] bochs-drm 0000:00:01.0: [drm] Registered 1 planes with drm panic machine # [ 7.508457] [drm] Initialized bochs-drm 1.0.0 for 0000:00:01.0 on minor 0 machine # [ 7.510770] mousedev: PS/2 mouse device common for all mice machine # [ 7.519031] Console: switching to colour frame buffer device 160x50 machine # [ 7.520550] bochs-drm 0000:00:01.0: [drm] fb0: bochs-drmdrmfb frame buffer device machine # [ 7.521825] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:09.0/virtio7/input/input4 machine # [ 7.525184] ACPI: button: Power Button [PWRF] machine # [ 7.553053] rtc_cmos 00:04: RTC can wake from S4 machine # [ 7.555456] rtc_cmos 00:04: registered as rtc0 machine # [ 7.556956] rtc_cmos 00:04: setting system clock to 2026-09-19T21:20:33 UTC (1789852833) machine # [ 7.558489] systemd-journald[334]: Time jumped backwards, rotating. machine # [ 7.561365] rtc_cmos 00:04: alarms up to one day, y3k, 242 bytes nvram, hpet irqs machine # [ 7.578693] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6 machine # [ 7.580995] i801_smbus 0000:00:1f.3: SMBus using PCI interrupt machine # [ 7.592318] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD machine # [ 7.595292] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 machine # [ 7.655674] parport_pc 00:02: reported by Plug and Play ACPI machine # [ 7.656377] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] machine # [ 7.541652] systemd[1]: Starting Virtual Console Setup... machine # [ 7.549659] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. machine # [ 7.551465] systemd[1]: Stopped Virtual Console Setup. machine # [ 7.556490] systemd[1]: Starting Virtual Console Setup... machine # [ 7.706932] iTCO_wdt iTCO_wdt.0.auto: Found a ICH9 TCO device (Version=2, TCOBASE=0x0660) machine # [ 7.708760] ppdev: user-space parallel port driver machine # [ 7.564034] systemd[1]: systemd-vconsole-setup.service: Deact[ 7.712380] iTCO_wdt iTCO_wdt.0.auto: initialized. heartbeat=30 sec (nowayout=0) machine # ivated successfully. machine # [ 7.568860] systemd[1]: Stopped Virtual Console Setup. machine # [ 7.572339] systemd[1]: Starting Virtual Console Setup... machine # [ 7.739184] kvm_amd: TSC scaling supported machine # [ 7.740017] kvm_amd: Nested Virtualization enabled machine # [ 7.740798] kvm_amd: Nested Paging enabled machine # [ 7.741390] kvm_amd: LBR virtualization supported machine # [ 7.742220] kvm_amd: Virtual VMLOAD VMSAVE supported machine # [ 7.743070] kvm_amd: Virtual GIF supported machine # [ 7.743770] kvm_amd: Virtual NMI enabled machine # [ 7.784763] EDAC MC: Ver: 3.0.0 machine # [ 7.654598] systemd[1]: Mounting /run/wrappers... machine # [ 7.686800] systemd[1]: Mounted /run/wrappers. machine # [ 7.688566] systemd[1]: Reached target Local File Systems. machine # [ 7.692555] systemd[1]: Listening on Boot Loader Control Service Socket. machine # [ 7.696679] systemd[1]: Starting register-nix-paths.service... machine # [ 7.700738] systemd[1]: Starting Create SUID/SGID Wrappers... machine # [ 7.702421] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. machine # [ 7.717274] systemd[1]: Starting Save Transient machine-id to Disk... machine # [ 7.721272] systemd[1]: Starting Create System Files and Directories... machine # [ 7.753956] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. machine # [ 7.757423] systemd[1]: Finished Save Transient machine-id to Disk. machine # [ 7.779461] systemd[1]: Finished Create System Files and Directories. machine # [ 7.783335] systemd[1]: Starting Rebuild Journal Catalog... machine # [ 7.784792] systemd[1]: Starting Record System Boot/Shutdown in UTMP... machine # [ 7.813699] systemd[1]: Finished Record System Boot/Shutdown in UTMP. machine # [ 7.838653] systemd[1]: Finished Rebuild Journal Catalog. machine # [ 7.840904] systemd[1]: Starting Update is Completed... machine # [ 7.869811] systemd[1]: Finished Update is Completed. machine # [ 7.948879] systemd-vconsole-setup[409]: Configuration of first virtual console was skipped, ignoring remaining ones. machine # [ 7.951828] systemd[1]: Finished Virtual Console Setup. machine # [ 8.015148] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. machine # [ 8.016824] systemd[1]: Finished Create SUID/SGID Wrappers. machine # [ 8.524419] systemd[1]: Finished register-nix-paths.service. machine # [ 8.525450] systemd[1]: Reached target System Initialization. machine # [ 8.526310] systemd[1]: Started Discard unused filesystem blocks once a week. machine # [ 8.527352] systemd[1]: Started niks3 garbage collection timer. machine # [ 8.528194] systemd[1]: Started Daily Cleanup of Temporary Directories. machine # [ 8.529129] systemd[1]: Reached target Timer Units. machine # [ 8.529826] systemd[1]: Listening on D-Bus System Message Bus Socket. machine # [ 8.531443] systemd[1]: Listening on niks3 server socket. machine # [ 8.532338] systemd[1]: Listening on Nix Daemon Socket. machine # [ 8.533166] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. machine # [ 8.534328] systemd[1]: Reached target Socket Units. machine # [ 8.535056] systemd[1]: Reached target Basic System. machine # [ 8.538571] systemd[1]: Started backdoor.service. machine # [ 8.542378] systemd[1]: Starting Import lastlog data into lastlog2 database... machine # [ 8.546292] systemd[1]: Starting Name Service Cache Daemon (nsncd)... machine # [ 8.551386] systemd[1]: Starting Post-Boot Actions... machine # [ 8.555580] systemd[1]: Started Reset console on configuration changes. machine # [ 8.559241] systemd[1]: Starting resolvconf update... machine # [ 8.567354] systemd[1]: Started rustfs.service. machine # [ 8.570480] systemd[1]: Starting rustfs-bucket.service... machine # [ 8.576070] systemd[1]: Starting D-Bus System Message Bus... machine # [ 8.611598] systemd[1]: Finished Post-Boot Actions. machine # connecting to host... machine # [ 8.622915] systemd[1]: Started Name Service Cache Daemon (nsncd). machine # [ 8.624988] systemd[1]: Reached target Host and Network Name Lookups. machine # [ 8.628553] systemd[1]: Reached target User and Group Name Lookups. machine # [ 8.631245] nsncd[492]: Sep 19 21:20:34.712 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" machine # [ 8.635160] systemd[1]: Starting User Login Management... machine # [ 8.640202] systemd[1]: Finished Import lastlog data into lastlog2 database. machine: Guest shell says: b'Spawning backdoor root shell...\n' machine: connected to guest root shell machine: (connecting took 9.31 seconds) machine: (finished: waiting for the VM to finish booting, in 9.63 seconds) machine # [ 8.688325] dbus-broker-launch[499]: Looking up NSS user entry for 'systemd-timesync'... machine # [ 8.697308] systemd-logind[516]: New seat seat0. machine # [ 8.701715] systemd-logind[516]: Watching system buttons on /dev/input/event2 (Power Button) machine # [ 8.701972] systemd-logind[516]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) machine # [ 8.702919] systemd-logind[516]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) machine # [ 8.716838] systemd[1]: Started User Login Management. machine # [ 8.720270] systemd[1]: Stopped target Host and Network Name Lookups. machine # [ 8.720435] systemd[1]: Stopping Host and Network Name Lookups... machine # [ 8.720881] systemd[1]: Stopped target User and Group Name Lookups. machine # [ 8.721536] systemd[1]: Stopping User and Group Name Lookups... machine # [ 8.723091] systemd[1]: Starting linger-users.service... machine # [ 8.723740] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... machine # [ 8.729100] systemd[1]: nscd.service: Deactivated successfully. machine # [ 8.730058] systemd[1]: Stopped Name Service Cache Daemon (nsncd). machine # [ 8.734850] dbus-broker-launch[499]: NSS returned no entry for 'systemd-timesync' machine # [ 8.735082] dbus-broker-launch[499]: Invalid user-name in /nix/store/ndgnid3f88z2mgg8ra084q4cm8l9ixds-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" machine # [ 8.739353] systemd[1]: Starting Name Service Cache Daemon (nsncd)... machine # [ 8.760370] systemd[1]: Started D-Bus System Message Bus. machine # [ 8.767286] systemd[1]: linger-users.service: Deactivated successfully. machine # [ 8.768303] systemd[1]: Finished linger-users.service. machine # [ 8.781438] dbus-broker-launch[499]: Ready machine # [ 8.783817] nsncd[583]: Sep 19 21:20:34.874 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" machine # [ 8.787438] systemd[1]: Started Name Service Cache Daemon (nsncd). machine # [ 8.787590] systemd[1]: Reached target Host and Network Name Lookups. machine # [ 8.788343] systemd[1]: Reached target User and Group Name Lookups. machine # [ 8.800271] systemd[1]: Finished resolvconf update. machine # [ 8.801350] systemd[1]: Reached target Preparation for Network. machine # [ 8.804474] systemd[1]: Starting DHCP Client... machine # [ 8.806524] systemd[1]: Starting Address configuration of eth1... machine # [ 8.809092] systemd[1]: Starting Extra networking commands.... machine # [ 8.883311] network-addresses-eth1-start[614]: adding address 192.168.1.1/24... done machine # [ 8.895288] network-addresses-eth1-start[614]: adding address 2001:db8:1::1/64... done machine # [ 8.914042] systemd[1]: Finished Address configuration of eth1. machine # [ 8.921245] dhcpcd[621]: dhcpcd-10.3.2 starting machine # [ 8.930859] dhcpcd[673]: dev: loaded udev machine # [ 8.947380] systemd[1]: Finished Extra networking commands.. machine # [ 8.947962] systemd[1]: Reached target Network. machine # [ 8.952293] systemd[1]: Starting PostgreSQL Server... machine # [ 8.955811] systemd[1]: Start[ 9.103286] 8021q: 802.1Q VLAN Support v1.8 machine # ing Permit User Sessions... machine # [ 9.104325] 8021q: adding VLAN 0 to HW filter on device eth1 machine # [ 8.984682] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. machine # [ 8.994294] systemd[1]: Finished Permit User Sessions. machine # [ 8.997888] systemd[1]: Started Getty on tty1. machine # [ 9.000269] systemd[1]: Reached target Login Prompts. machine # [ 9.209950] cfg80211: Loading compiled-in X.509 certificates for regulatory database machine # [ 9.241893] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' machine # [ 9.243515] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' machine # [ 9.247171] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 machine # [ 9.248733] cfg80211: failed to load regulatory.db machine # [ 9.147386] postgresql-pre-start[700]: The files belonging to this database system will be owned by user "postgres". machine # [ 9.150050] postgresql-pre-start[700]: This user must also own the server process. machine # [ 9.302085] 8021q: adding VLAN 0 to HW filter on device eth0 machine # [ 9.156866] postgresql-pre-start[700]: The database cluster will be initialized with locale "en_US.UTF-8". machine # [ 9.157107] postgresql-pre-start[700]: The default database encoding has accordingly been set to "UTF8". machine # [ 9.157545] postgresql-pre-start[700]: The default text search configuration will be set to "english". machine # [ 9.158071] postgresql-pre-start[700]: Data page checksums are enabled. machine # [ 9.158753] postgresql-pre-start[700]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok machine # [ 9.159907] postgresql-pre-start[700]: creating subdirectories ... ok machine # [ 9.160534] postgresql-pre-start[700]: selecting dynamic shared memory implementation ... posix machine # [ 9.161114] dhcpcd[673]: eth0: waiting for carrier machine # [ 9.162138] dhcpcd[673]: eth0: carrier acquired machine # [ 9.172114] dhcpcd[673]: DUID 00:01:00:01:32:41:b9:23:52:54:00:12:34:56 machine # [ 9.172374] dhcpcd[673]: eth0: IAID 00:12:34:56 machine # [ 9.172835] dhcpcd[673]: eth0: adding address fe80::5054:ff:fe12:3456 machine # [ 9.240272] postgresql-pre-start[700]: selecting default "max_connections" ... 100 machine # [ 9.244254] dhcpcd[673]: eth0: soliciting a DHCP lease machine # [ 9.407672] NET: Registered PF_PACKET protocol family machine # [ 9.268504] dhcpcd[673]: eth0: offered 10.0.2.15 from 10.0.2.2 machine # [ 9.274212] dhcpcd[673]: eth0: probing address 10.0.2.15/24 machine # [ 9.311265] postgresql-pre-start[700]: selecting default "shared_buffers" ... 128MB machine # [ 10.916344] postgresql-pre-start[700]: selecting default time zone ... UTC machine # [ 10.919817] postgresql-pre-start[700]: creating configuration files ... ok machine # [ 11.153067] postgresql-pre-start[700]: running bootstrap script ... ok machine # [ 11.220930] dhcpcd[673]: eth0: soliciting an IPv6 router machine # [ 11.222160] dhcpcd[673]: eth0: Router Advertisement from fe80::2 machine # [ 11.223582] dhcpcd[673]: eth0: adding address fec0::5054:ff:fe12:3456/64 machine # [ 11.227945] dhcpcd[673]: eth0: adding route to fec0::/64 machine # [ 11.230141] dhcpcd[673]: eth0: adding default route via fe80::2 machine # [ 11.740936] postgresql-pre-start[700]: performing post-bootstrap initialization ... ok machine # [ 11.907177] postgresql-pre-start[700]: syncing data to disk ... ok machine # [ 11.907521] postgresql-pre-start[700]: initdb: warning: enabling "trust" authentication for local connections machine # [ 11.908478] postgresql-pre-start[700]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. machine # [ 11.911495] postgresql-pre-start[700]: Success. You can now start the database server using: machine # [ 11.913093] postgresql-pre-start[700]: pg_ctl -D /var/lib/postgresql/18 -l logfile start machine # [ 12.064773] postgres[761]: [761] LOG: starting PostgreSQL 18.6 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit machine # [ 12.071783] postgres[761]: [761] LOG: listening on IPv6 address "::1", port 5432 machine # [ 12.071941] postgres[761]: [761] LOG: listening on IPv4 address "127.0.0.1", port 5432 machine # [ 12.073652] postgres[761]: [761] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" machine # [ 12.082839] postgres[773]: [773] LOG: database system was shut down at 2026-09-19 21:20:37 GMT machine # [ 12.086894] postgres[761]: [761] LOG: database system is ready to accept connections machine # [ 12.091793] systemd[1]: Started PostgreSQL Server. machine # [ 12.096164] systemd[1]: Starting PostgreSQL Setup Scripts... machine # [ 12.338642] postgresql-setup-start[784]: CREATE DATABASE machine # [ 12.385093] postgresql-setup-start[790]: CREATE ROLE machine # [ 12.407960] postgresql-setup-start[792]: ALTER DATABASE machine # [ 12.412500] systemd[1]: Finished PostgreSQL Setup Scripts. machine # [ 12.413432] systemd[1]: Reached target PostgreSQL. machine # [ 14.541507] dhcpcd[673]: eth0: leased 10.0.2.15 for 86400 seconds machine # [ 14.541928] dhcpcd[673]: eth0: adding route to 10.0.2.0/24 machine # [ 14.542562] dhcpcd[673]: eth0: adding default route via 10.0.2.2 machine # [ 14.650871] systemd[1]: Started DHCP Client. machine # [ 14.652614] systemd[1]: Reached target Network is Online. machine # [ 14.655725] systemd[1]: Starting k3s service... machine # [ 14.785393] k3s[873]: time="2026-09-19T21:20:40Z" level=info msg="Acquiring lock file /var/lib/rancher/k3s/data/.lock" machine # [ 14.790582] k3s[873]: time="2026-09-19T21:20:40Z" level=info msg="Preparing data dir /var/lib/rancher/k3s/data/76c98da5fd74d9a243cba8d533ae287fc547ab08ccd8c5ea4811608ce7075724" machine # [ 18.437844] k3s[873]: time="2026-09-19T21:20:44Z" level=info msg="Starting k3s 1.35.8+k3s1 (e952d68a)" machine # [ 18.445804] k3s[873]: time="2026-09-19T21:20:44Z" level=info msg="Configuring sqlite3 database connection pooling: maxIdle=20, maxOpen=0, maxLifetime=0s, maxIdleTime=2m0s" machine # [ 18.450704] k3s[873]: time="2026-09-19T21:20:44Z" level=info msg="Kine built with sqlite from github.com/mattn/go-sqlite3 version 3.53.3" machine # [ 18.450798] k3s[873]: time="2026-09-19T21:20:44Z" level=info msg="Configuring database table schema and indexes, this may take a moment..." machine # [ 18.467615] k3s[873]: time="2026-09-19T21:20:44Z" level=info msg="Database tables and indexes are up to date" machine # [ 18.468294] k3s[873]: time="2026-09-19T21:20:44Z" level=info msg="Running startup VACUUM to reclaim disk space, this may take a moment..." machine # [ 18.478614] k3s[873]: time="2026-09-19T21:20:44Z" level=info msg="Startup VACUUM completed successfully" machine # [ 18.482174] k3s[873]: time="2026-09-19T21:20:44Z" level=info msg="Kine available at unix://kine.sock" machine # [ 18.485077] k3s[873]: time="2026-09-19T21:20:44Z" level=info msg="Opening etcd client connection with endpoints [unix://kine.sock]" machine # [ 18.492395] k3s[873]: time="2026-09-19T21:20:44Z" level=info msg="Datastore connection validated successfully, proceeding with bootstrap data generation" machine # [ 18.495813] k3s[873]: time="2026-09-19T21:20:44Z" level=info msg="generated self-signed CA certificate CN=k3s-client-ca@1789852844: notBefore=2026-09-19 20:20:44.583310307 +0000 UTC notAfter=2036-09-16 20:20:44.583310307 +0000 UTC" machine # [ 18.500670] k3s[873]: time="2026-09-19T21:20:44Z" level=info msg="certificate CN=system:admin,O=system:masters signed by CN=k3s-client-ca@1789852844: notBefore=2026-09-19 20:20:44 +0000 UTC notAfter=2027-09-19 20:20:44 +0000 UTC" machine # [ 18.501787] k3s[873]: time="2026-09-19T21:20:44Z" level=info msg="certificate CN=system:k3s-supervisor,O=system:masters signed by CN=k3s-client-ca@1789852844: notBefore=2026-09-19 20:20:44 +0000 UTC notAfter=2027-09-19 20:20:44 +0000 UTC" machine # [ 18.502967] k3s[873]: time="2026-09-19T21:20:44Z" level=info msg="certificate CN=system:kube-controller-manager signed by CN=k3s-client-ca@1789852844: notBefore=2026-09-19 20:20:44 +0000 UTC notAfter=2027-09-19 20:20:44 +0000 UTC" machine # [ 18.504306] k3s[873]: time="2026-09-19T21:20:44Z" level=info msg="certificate CN=system:kube-scheduler signed by CN=k3s-client-ca@1789852844: notBefore=2026-09-19 20:20:44 +0000 UTC notAfter=2027-09-19 20:20:44 +0000 UTC" machine # [ 18.505188] k3s[873]: time="2026-09-19T21:20:44Z" level=info msg="certificate CN=system:apiserver,O=system:masters signed by CN=k3s-client-ca@1789852844: notBefore=2026-09-19 20:20:44 +0000 UTC notAfter=2027-09-19 20:20:44 +0000 UTC" machine # [ 18.506156] k3s[873]: time="2026-09-19T21:20:44Z" level=info msg="certificate CN=k3s-cloud-controller-manager signed by CN=k3s-client-ca@1789852844: notBefore=2026-09-19 20:20:44 +0000 UTC notAfter=2027-09-19 20:20:44 +0000 UTC" machine # [ 18.507112] k3s[873]: time="2026-09-19T21:20:44Z" level=info msg="generated self-signed CA certificate CN=k3s-server-ca@1789852844: notBefore=2026-09-19 20:20:44.586519374 +0000 UTC notAfter=2036-09-16 20:20:44.586519374 +0000 UTC" machine # [ 18.507941] k3s[873]: time="2026-09-19T21:20:44Z" level=info msg="certificate CN=kube-apiserver signed by CN=k3s-server-ca@1789852844: notBefore=2026-09-19 20:20:44 +0000 UTC notAfter=2027-09-19 20:20:44 +0000 UTC" machine # [ 18.509209] k3s[873]: time="2026-09-19T21:20:44Z" level=info msg="certificate CN=kube-scheduler signed by CN=k3s-server-ca@1789852844: notBefore=2026-09-19 20:20:44 +0000 UTC notAfter=2027-09-19 20:20:44 +0000 UTC" machine # [ 18.510127] k3s[873]: time="2026-09-19T21:20:44Z" level=info msg="certificate CN=kube-controller-manager signed by CN=k3s-server-ca@1789852844: notBefore=2026-09-19 20:20:44 +0000 UTC notAfter=2027-09-19 20:20:44 +0000 UTC" machine # [ 18.511061] k3s[873]: time="2026-09-19T21:20:44Z" level=info msg="generated self-signed CA certificate CN=k3s-request-header-ca@1789852844: notBefore=2026-09-19 20:20:44.588163159 +0000 UTC notAfter=2036-09-16 20:20:44.588163159 +0000 UTC" machine # [ 18.511715] k3s[873]: time="2026-09-19T21:20:44Z" level=info msg="certificate CN=system:auth-proxy signed by CN=k3s-request-header-ca@1789852844: notBefore=2026-09-19 20:20:44 +0000 UTC notAfter=2027-09-19 20:20:44 +0000 UTC" machine # [ 18.512573] k3s[873]: time="2026-09-19T21:20:44Z" level=info msg="generated self-signed CA certificate CN=etcd-server-ca@1789852844: notBefore=2026-09-19 20:20:44.588846765 +0000 UTC notAfter=2036-09-16 20:20:44.588846765 +0000 UTC" machine # [ 18.513261] k3s[873]: time="2026-09-19T21:20:44Z" level=info msg="certificate CN=etcd-client signed by CN=etcd-server-ca@1789852844: notBefore=2026-09-19 20:20:44 +0000 UTC notAfter=2027-09-19 20:20:44 +0000 UTC" machine # [ 18.513865] k3s[873]: time="2026-09-19T21:20:44Z" level=info msg="generated self-signed CA certificate CN=etcd-peer-ca@1789852844: notBefore=2026-09-19 20:20:44.589568924 +0000 UTC notAfter=2036-09-16 20:20:44.589568924 +0000 UTC" machine # [ 18.514669] k3s[873]: time="2026-09-19T21:20:44Z" level=info msg="certificate CN=etcd-peer signed by CN=etcd-peer-ca@1789852844: notBefore=2026-09-19 20:20:44 +0000 UTC notAfter=2027-09-19 20:20:44 +0000 UTC" machine # [ 18.515527] k3s[873]: time="2026-09-19T21:20:44Z" level=info msg="certificate CN=etcd-server signed by CN=etcd-server-ca@1789852844: notBefore=2026-09-19 20:20:44 +0000 UTC notAfter=2027-09-19 20:20:44 +0000 UTC" machine # [ 18.588995] k3s[873]: time="2026-09-19T21:20:44Z" level=info msg="certificate CN=k3s,O=k3s signed by CN=k3s-server-ca@1789852844: notBefore=2026-09-19 20:20:44 +0000 UTC notAfter=2027-09-19 20:20:44 +0000 UTC" machine # [ 18.590124] k3s[873]: time="2026-09-19T21:20:44Z" level=warning msg="dynamiclistener [::]:6443: no cached certificate available for preload - deferring certificate load until storage initialization or first client request" machine # [ 18.592134] k3s[873]: time="2026-09-19T21:20:44Z" level=info msg="Active TLS secret / (ver=) (count 11): map[listener.cattle.io/cn-10.0.2.15:10.0.2.15 listener.cattle.io/cn-10.43.0.1:10.43.0.1 listener.cattle.io/cn-127.0.0.1:127.0.0.1 listener.cattle.io/cn-__1-f16284:::1 listener.cattle.io/cn-fec0__5fe0_fdce_6860_f263-5f27ad:fec0::5fe0:fdce:6860:f263 listener.cattle.io/cn-kubernetes:kubernetes listener.cattle.io/cn-kubernetes.default:kubernetes.default listener.cattle.io/cn-kubernetes.default.svc:kubernetes.default.svc listener.cattle.io/cn-kubernetes.default.svc.cluster.local:kubernetes.default.svc.cluster.local listener.cattle.io/cn-localhost:localhost listener.cattle.io/cn-machine:machine listener.cattle.io/fingerprint:SHA1=07CEF5D751C0D029F060778B8FB7B4C3D85C6926]" machine # [ 18.738527] rustfs-bucket-start[907]: mb s3://farm machine # [ 18.742993] systemd[1]: Finished rustfs-bucket.service. machine # [ 19.429175] k3s[873]: time="2026-09-19T21:20:45Z" level=info msg="Password verified locally for node machine" machine # [ 19.432446] k3s[873]: time="2026-09-19T21:20:45Z" level=info msg="certificate CN=machine signed by CN=k3s-server-ca@1789852844: notBefore=2026-09-19 20:20:45 +0000 UTC notAfter=2027-09-19 20:20:45 +0000 UTC" machine # [ 19.685182] k3s[873]: time="2026-09-19T21:20:45Z" level=info msg="certificate CN=system:node:machine,O=system:nodes signed by CN=k3s-client-ca@1789852844: notBefore=2026-09-19 20:20:45 +0000 UTC notAfter=2027-09-19 20:20:45 +0000 UTC" machine # [ 19.790155] k3s[873]: time="2026-09-19T21:20:45Z" level=info msg="certificate CN=system:kube-proxy signed by CN=k3s-client-ca@1789852844: notBefore=2026-09-19 20:20:45 +0000 UTC notAfter=2027-09-19 20:20:45 +0000 UTC" machine # [ 19.872743] k3s[873]: time="2026-09-19T21:20:45Z" level=info msg="certificate CN=system:k3s-controller signed by CN=k3s-client-ca@1789852844: notBefore=2026-09-19 20:20:45 +0000 UTC notAfter=2027-09-19 20:20:45 +0000 UTC" machine # [ 19.961954] k3s[873]: time="2026-09-19T21:20:46Z" level=error msg="Sending HTTP/1.1 503 response to 127.0.0.1:54834: runtime core not ready" machine # [ 20.065141] k3s[873]: time="2026-09-19T21:20:46Z" level=info msg="Module overlay was already loaded" machine # [ 20.289247] bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this. machine # [ 20.294012] Bridge firewalling registered machine # [ 20.168656] k3s[873]: time="2026-09-19T21:20:46Z" level=warning msg="Failed to load kernel module iptable_nat with modprobe" machine # [ 20.186187] k3s[873]: time="2026-09-19T21:20:46Z" level=warning msg="Failed to load kernel module iptable_filter with modprobe" machine # [ 20.200801] k3s[873]: time="2026-09-19T21:20:46Z" level=warning msg="Failed to load kernel module nft-expr-counter with modprobe" machine # [ 20.294540] k3s[873]: time="2026-09-19T21:20:46Z" level=info msg="Set sysctl 'net/ipv4/conf/all/forwarding' to 1" machine # [ 20.298198] k3s[873]: time="2026-09-19T21:20:46Z" level=info msg="Set sysctl 'net/netfilter/nf_conntrack_max' to 131072" machine # [ 20.302639] k3s[873]: time="2026-09-19T21:20:46Z" level=info msg="Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_established' to 86400" machine # [ 20.306564] k3s[873]: time="2026-09-19T21:20:46Z" level=info msg="Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_close_wait' to 3600" machine # [ 20.310204] k3s[873]: time="2026-09-19T21:20:46Z" level=info msg="Connecting to proxy" url="wss://127.0.0.1:6443/v1-k3s/connect" machine # [ 20.313799] k3s[873]: time="2026-09-19T21:20:46Z" level=info msg="Creating k3s-cert-monitor event broadcaster" machine # [ 20.316322] k3s[873]: time="2026-09-19T21:20:46Z" level=info msg="Saving cluster bootstrap data to datastore" machine # [ 20.318044] k3s[873]: time="2026-09-19T21:20:46Z" level=info msg="Handling backend connection request [machine]" machine # [ 20.319889] k3s[873]: time="2026-09-19T21:20:46Z" level=info msg="Opening etcd client connection with endpoints [unix://kine.sock]" machine # [ 20.321951] k3s[873]: time="2026-09-19T21:20:46Z" level=info msg="Connected to proxy" url="wss://127.0.0.1:6443/v1-k3s/connect" machine # [ 20.324278] k3s[873]: time="2026-09-19T21:20:46Z" level=info msg="Remotedialer connected to proxy" url="wss://127.0.0.1:6443/v1-k3s/connect" machine # [ 20.326236] k3s[873]: time="2026-09-19T21:20:46Z" level=info msg="Connection to etcd is ready" machine # [ 20.328398] k3s[873]: time="2026-09-19T21:20:46Z" level=info msg="Polling for API server readiness: GET /readyz failed: the server is currently unable to handle the request" machine # [ 20.331896] k3s[873]: time="2026-09-19T21:20:46Z" level=info msg="ETCD server is now running" machine # [ 20.334249] k3s[873]: time="2026-09-19T21:20:46Z" level=info msg="Running kube-apiserver --advertise-port=6443 --allow-privileged=true --anonymous-auth=false --api-audiences=https://kubernetes.default.svc.cluster.local,k3s --authorization-mode=Node,RBAC --bind-address=127.0.0.1 --cert-dir=/var/lib/rancher/k3s/server/tls/temporary-certs --client-ca-file=/var/lib/rancher/k3s/server/tls/client-ca.crt --egress-selector-config-file=/var/lib/rancher/k3s/server/etc/egress-selector-config.yaml --enable-admission-plugins=NodeRestriction --enable-aggregator-routing=true --enable-bootstrap-token-auth=true --etcd-servers=unix://kine.sock --kubelet-certificate-authority=/var/lib/rancher/k3s/server/tls/server-ca.crt --kubelet-client-certificate=/var/lib/rancher/k3s/server/tls/client-kube-apiserver.crt --kubelet-client-key=/var/lib/rancher/k3s/server/tls/client-kube-apiserver.key --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --profiling=false --proxy-client-cert-file=/var/lib/rancher/k3s/server/tls/client-auth-proxy.crt --proxy-client-key-file=/var/lib/rancher/k3s/server/tls/client-auth-proxy.key --requestheader-allowed-names=system:auth-proxy --requestheader-client-ca-file=/var/lib/rancher/k3s/server/tls/request-header-ca.crt --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group --requestheader-username-headers=X-Remote-User --secure-port=6444 --service-account-issuer=https://kubernetes.default.svc.cluster.local --service-account-key-file=/var/lib/rancher/k3s/server/tls/service.key --service-account-signing-key-file=/var/lib/rancher/k3s/server/tls/service.current.key --service-cluster-ip-range=10.43.0.0/16 --service-node-port-range=30000-32767 --storage-backend=etcd3 --tls-cert-file=/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.crt --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305 --tls-private-key-file=/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.key" machine # [ 20.371536] k3s[873]: time="2026-09-19T21:20:46Z" level=info msg="Running kube-scheduler --authentication-kubeconfig=/var/lib/rancher/k3s/server/cred/scheduler.kubeconfig --authorization-kubeconfig=/var/lib/rancher/k3s/server/cred/scheduler.kubeconfig --bind-address=127.0.0.1 --kubeconfig=/var/lib/rancher/k3s/server/cred/scheduler.kubeconfig --leader-elect=false --profiling=false --secure-port=10259 --tls-cert-file=/var/lib/rancher/k3s/server/tls/kube-scheduler/kube-scheduler.crt --tls-private-key-file=/var/lib/rancher/k3s/server/tls/kube-scheduler/kube-scheduler.key" machine # [ 20.378878] k3s[873]: time="2026-09-19T21:20:46Z" level=info msg="Running kube-controller-manager --allocate-node-cidrs=true --authentication-kubeconfig=/var/lib/rancher/k3s/server/cred/controller.kubeconfig --authorization-kubeconfig=/var/lib/rancher/k3s/server/cred/controller.kubeconfig --bind-address=127.0.0.1 --cluster-cidr=10.42.0.0/16 --cluster-signing-kube-apiserver-client-cert-file=/var/lib/rancher/k3s/server/tls/client-ca.nochain.crt --cluster-signing-kube-apiserver-client-key-file=/var/lib/rancher/k3s/server/tls/client-ca.key --cluster-signing-kubelet-client-cert-file=/var/lib/rancher/k3s/server/tls/client-ca.nochain.crt --cluster-signing-kubelet-client-key-file=/var/lib/rancher/k3s/server/tls/client-ca.key --cluster-signing-kubelet-serving-cert-file=/var/lib/rancher/k3s/server/tls/server-ca.nochain.crt --cluster-signing-kubelet-serving-key-file=/var/lib/rancher/k3s/server/tls/server-ca.key --cluster-signing-legacy-unknown-cert-file=/var/lib/rancher/k3s/server/tls/server-ca.nochain.crt --cluster-signing-legacy-unknown-key-file=/var/lib/rancher/k3s/server/tls/server-ca.key --configure-cloud-routes=false --controllers=*,tokencleaner,-service,-route,-cloud-node-lifecycle --kubeconfig=/var/lib/rancher/k3s/server/cred/controller.kubeconfig --leader-elect=false --profiling=false --root-ca-file=/var/lib/rancher/k3s/server/tls/server-ca.crt --secure-port=10257 --service-account-private-key-file=/var/lib/rancher/k3s/server/tls/service.current.key --service-cluster-ip-range=10.43.0.0/16 --tls-cert-file=/var/lib/rancher/k3s/server/tls/kube-controller-manager/kube-controller-manager.crt --tls-private-key-file=/var/lib/rancher/k3s/server/tls/kube-controller-manager/kube-controller-manager.key --use-service-account-credentials=true" machine # [ 20.400488] k3s[873]: I0919 21:20:46.398925 873 options.go:263] external host was not specified, using 10.0.2.15 machine # [ 20.402325] k3s[873]: time="2026-09-19T21:20:46Z" level=info msg="Running cloud-controller-manager --allocate-node-cidrs=true --authentication-kubeconfig=/var/lib/rancher/k3s/server/cred/cloud-controller.kubeconfig --authorization-kubeconfig=/var/lib/rancher/k3s/server/cred/cloud-controller.kubeconfig --bind-address=127.0.0.1 --cloud-config=/var/lib/rancher/k3s/server/etc/cloud-config.yaml --cloud-provider=k3s --cluster-cidr=10.42.0.0/16 --configure-cloud-routes=false --controllers=*,-route --kubeconfig=/var/lib/rancher/k3s/server/cred/cloud-controller.kubeconfig --leader-elect=false --leader-elect-resource-name=k3s-cloud-controller-manager --node-status-update-frequency=1m0s --profiling=false" machine # [ 20.410653] k3s[873]: time="2026-09-19T21:20:46Z" level=info msg="Server node token is available at /var/lib/rancher/k3s/server/token" machine # [ 20.412367] k3s[873]: time="2026-09-19T21:20:46Z" level=info msg="To join server node to cluster: k3s server -s https://10.0.2.15:6443 -t ${SERVER_NODE_TOKEN}" machine # [ 20.414251] k3s[873]: time="2026-09-19T21:20:46Z" level=info msg="Agent node token is available at /var/lib/rancher/k3s/server/agent-token" machine # [ 20.415974] k3s[873]: time="2026-09-19T21:20:46Z" level=info msg="To join agent node to cluster: k3s agent -s https://10.0.2.15:6443 -t ${AGENT_NODE_TOKEN}" machine # [ 20.417793] k3s[873]: time="2026-09-19T21:20:46Z" level=info msg="Wrote kubeconfig /etc/rancher/k3s/k3s.yaml" machine # [ 20.419186] k3s[873]: time="2026-09-19T21:20:46Z" level=info msg="Run: k3s kubectl" machine # [ 20.420341] k3s[873]: I0919 21:20:46.405080 873 server.go:158] Version: v1.35.8+k3s1 machine # [ 20.421515] k3s[873]: I0919 21:20:46.405112 873 server.go:160] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" machine # [ 20.423033] k3s[873]: time="2026-09-19T21:20:46Z" level=info msg="Logging containerd to /var/lib/rancher/k3s/agent/containerd/containerd.log" machine # [ 20.424730] k3s[873]: time="2026-09-19T21:20:46Z" level=info msg="Running containerd -c /var/lib/rancher/k3s/agent/etc/containerd/config.toml" machine # [ 20.426471] k3s[873]: time="2026-09-19T21:20:46Z" level=error msg="Sending HTTP/1.1 503 response to 127.0.0.1:54890: runtime core not ready" machine # [ 20.460107] k3s[873]: I0919 21:20:46.535056 873 shared_informer.go:349] "Waiting for caches to sync" controller="node_authorizer" machine # [ 20.461800] k3s[873]: I0919 21:20:46.538784 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 20.463134] k3s[873]: I0919 21:20:46.540637 873 plugins.go:157] Loaded 14 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,PodTopologyLabels,MutatingAdmissionPolicy,MutatingAdmissionWebhook. machine # [ 20.467429] k3s[873]: I0919 21:20:46.540657 873 plugins.go:160] Loaded 14 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,PodSecurity,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,ClusterTrustBundleAttest,CertificateSubjectRestriction,NodeDeclaredFeatureValidator,ValidatingAdmissionPolicy,ValidatingAdmissionWebhook,ResourceQuota. machine # [ 20.471800] k3s[873]: I0919 21:20:46.541081 873 instance.go:240] Using reconciler: lease machine # [ 20.472929] k3s[873]: I0919 21:20:46.550778 873 handler.go:304] Adding GroupVersion apiextensions.k8s.io v1 to ResourceManager machine # [ 20.474380] k3s[873]: W0919 21:20:46.550798 873 genericapiserver.go:787] Skipping API apiextensions.k8s.io/v1beta1 because it has no resources. machine # [ 20.475969] k3s[873]: I0919 21:20:46.553286 873 cidrallocator.go:198] starting ServiceCIDR Allocator Controller machine # [ 20.501079] k3s[873]: I0919 21:20:46.591614 873 handler.go:304] Adding GroupVersion v1 to ResourceManager machine # [ 20.502484] k3s[873]: I0919 21:20:46.591821 873 apis.go:112] API group "internal.apiserver.k8s.io" is not enabled, skipping. machine # [ 20.535106] k3s[873]: I0919 21:20:46.625318 873 apis.go:112] API group "storagemigration.k8s.io" is not enabled, skipping. machine # [ 20.552680] k3s[873]: time="2026-09-19T21:20:46Z" level=info msg="Running kube-proxy --cluster-cidr=10.42.0.0/16 --conntrack-max-per-core=0 --conntrack-tcp-timeout-close-wait=0s --conntrack-tcp-timeout-established=0s --healthz-bind-address=127.0.0.1 --hostname-override=machine --kubeconfig=/var/lib/rancher/k3s/agent/kubeproxy.kubeconfig --proxy-mode=iptables" machine # [ 20.570829] k3s[873]: I0919 21:20:46.661363 873 handler.go:304] Adding GroupVersion authentication.k8s.io v1 to ResourceManager machine # [ 20.573480] k3s[873]: W0919 21:20:46.661403 873 genericapiserver.go:787] Skipping API authentication.k8s.io/v1beta1 because it has no resources. machine # [ 20.576420] k3s[873]: W0919 21:20:46.661415 873 genericapiserver.go:787] Skipping API authentication.k8s.io/v1alpha1 because it has no resources. machine # [ 20.576587] k3s[873]: I0919 21:20:46.661761 873 handler.go:304] Adding GroupVersion authorization.k8s.io v1 to ResourceManager machine # [ 20.576749] k3s[873]: W0919 21:20:46.661776 873 genericapiserver.go:787] Skipping API authorization.k8s.io/v1beta1 because it has no resources. machine # [ 20.577366] k3s[873]: I0919 21:20:46.662397 873 handler.go:304] Adding GroupVersion autoscaling v2 to ResourceManager machine # [ 20.578055] k3s[873]: I0919 21:20:46.662823 873 handler.go:304] Adding GroupVersion autoscaling v1 to ResourceManager machine # [ 20.578870] k3s[873]: W0919 21:20:46.662838 873 genericapiserver.go:787] Skipping API autoscaling/v2beta1 because it has no resources. machine # [ 20.579736] k3s[873]: W0919 21:20:46.662849 873 genericapiserver.go:787] Skipping API autoscaling/v2beta2 because it has no resources. machine # [ 20.580602] k3s[873]: I0919 21:20:46.663796 873 handler.go:304] Adding GroupVersion batch v1 to ResourceManager machine # [ 20.581211] k3s[873]: W0919 21:20:46.663814 873 genericapiserver.go:787] Skipping API batch/v1beta1 because it has no resources. machine # [ 20.581701] k3s[873]: I0919 21:20:46.664376 873 handler.go:304] Adding GroupVersion certificates.k8s.io v1 to ResourceManager machine # [ 20.582447] k3s[873]: W0919 21:20:46.664394 873 genericapiserver.go:787] Skipping API certificates.k8s.io/v1beta1 because it has no resources. machine # [ 20.583340] k3s[873]: W0919 21:20:46.664404 873 genericapiserver.go:787] Skipping API certificates.k8s.io/v1alpha1 because it has no resources. machine # [ 20.583948] k3s[873]: I0919 21:20:46.664770 873 handler.go:304] Adding GroupVersion coordination.k8s.io v1 to ResourceManager machine # [ 20.584786] k3s[873]: W0919 21:20:46.664784 873 genericapiserver.go:787] Skipping API coordination.k8s.io/v1beta1 because it has no resources. machine # [ 20.585825] k3s[873]: W0919 21:20:46.664795 873 genericapiserver.go:787] Skipping API coordination.k8s.io/v1alpha2 because it has no resources. machine # [ 20.586746] k3s[873]: I0919 21:20:46.665154 873 handler.go:304] Adding GroupVersion discovery.k8s.io v1 to ResourceManager machine # [ 20.587788] k3s[873]: W0919 21:20:46.665171 873 genericapiserver.go:787] Skipping API discovery.k8s.io/v1beta1 because it has no resources. machine # [ 20.588652] k3s[873]: I0919 21:20:46.666630 873 handler.go:304] Adding GroupVersion networking.k8s.io v1 to ResourceManager machine # [ 20.589245] k3s[873]: W0919 21:20:46.666648 873 genericapiserver.go:787] Skipping API networking.k8s.io/v1beta1 because it has no resources. machine # [ 20.589941] k3s[873]: I0919 21:20:46.667002 873 handler.go:304] Adding GroupVersion node.k8s.io v1 to ResourceManager machine # [ 20.590755] k3s[873]: W0919 21:20:46.667027 873 genericapiserver.go:787] Skipping API node.k8s.io/v1beta1 because it has no resources. machine # [ 20.591591] k3s[873]: W0919 21:20:46.667038 873 genericapiserver.go:787] Skipping API node.k8s.io/v1alpha1 because it has no resources. machine # [ 20.592328] k3s[873]: I0919 21:20:46.667504 873 handler.go:304] Adding GroupVersion policy v1 to ResourceManager machine # [ 20.592885] k3s[873]: W0919 21:20:46.667520 873 genericapiserver.go:787] Skipping API policy/v1beta1 because it has no resources. machine # [ 20.593795] k3s[873]: I0919 21:20:46.668520 873 handler.go:304] Adding GroupVersion rbac.authorization.k8s.io v1 to ResourceManager machine # [ 20.594487] k3s[873]: W0919 21:20:46.668536 873 genericapiserver.go:787] Skipping API rbac.authorization.k8s.io/v1beta1 because it has no resources. machine # [ 20.594729] k3s[873]: W0919 21:20:46.668547 873 genericapiserver.go:787] Skipping API rbac.authorization.k8s.io/v1alpha1 because it has no resources. machine # [ 20.595204] k3s[873]: I0919 21:20:46.668827 873 handler.go:304] Adding GroupVersion scheduling.k8s.io v1 to ResourceManager machine # [ 20.595460] k3s[873]: W0919 21:20:46.668840 873 genericapiserver.go:787] Skipping API scheduling.k8s.io/v1beta1 because it has no resources. machine # [ 20.596467] k3s[873]: W0919 21:20:46.668850 873 genericapiserver.go:787] Skipping API scheduling.k8s.io/v1alpha1 because it has no resources. machine # [ 20.596919] k3s[873]: I0919 21:20:46.670318 873 handler.go:304] Adding GroupVersion storage.k8s.io v1 to ResourceManager machine # [ 20.597431] k3s[873]: W0919 21:20:46.670347 873 genericapiserver.go:787] Skipping API storage.k8s.io/v1beta1 because it has no resources. machine # [ 20.597647] k3s[873]: W0919 21:20:46.670358 873 genericapiserver.go:787] Skipping API storage.k8s.io/v1alpha1 because it has no resources. machine # [ 20.597888] k3s[873]: I0919 21:20:46.671145 873 handler.go:304] Adding GroupVersion flowcontrol.apiserver.k8s.io v1 to ResourceManager machine # [ 20.598578] k3s[873]: W0919 21:20:46.671164 873 genericapiserver.go:787] Skipping API flowcontrol.apiserver.k8s.io/v1beta3 because it has no resources. machine # [ 20.599250] k3s[873]: W0919 21:20:46.671186 873 genericapiserver.go:787] Skipping API flowcontrol.apiserver.k8s.io/v1beta2 because it has no resources. machine # [ 20.599469] k3s[873]: W0919 21:20:46.671196 873 genericapiserver.go:787] Skipping API flowcontrol.apiserver.k8s.io/v1beta1 because it has no resources. machine # [ 20.600184] k3s[873]: I0919 21:20:46.674876 873 handler.go:304] Adding GroupVersion apps v1 to ResourceManager machine # [ 20.600429] k3s[873]: W0919 21:20:46.674895 873 genericapiserver.go:787] Skipping API apps/v1beta2 because it has no resources. machine # [ 20.600860] k3s[873]: W0919 21:20:46.674919 873 genericapiserver.go:787] Skipping API apps/v1beta1 because it has no resources. machine # [ 20.601808] k3s[873]: I0919 21:20:46.677485 873 handler.go:304] Adding GroupVersion admissionregistration.k8s.io v1 to ResourceManager machine # [ 20.602285] k3s[873]: W0919 21:20:46.677503 873 genericapiserver.go:787] Skipping API admissionregistration.k8s.io/v1beta1 because it has no resources. machine # [ 20.602940] k3s[873]: W0919 21:20:46.677514 873 genericapiserver.go:787] Skipping API admissionregistration.k8s.io/v1alpha1 because it has no resources. machine # [ 20.604594] k3s[873]: I0919 21:20:46.678245 873 handler.go:304] Adding GroupVersion events.k8s.io v1 to ResourceManager machine # [ 20.656111] k3s[873]: W0919 21:20:46.678263 873 genericapiserver.go:787] Skipping API events.k8s.io/v1beta1 because it has no resources. machine # [ 20.656820] k3s[873]: I0919 21:20:46.681019 873 handler.go:304] Adding GroupVersion resource.k8s.io v1 to ResourceManager machine # [ 20.657664] k3s[873]: W0919 21:20:46.681037 873 genericapiserver.go:787] Skipping API resource.k8s.io/v1beta2 because it has no resources. machine # [ 20.658831] k3s[873]: W0919 21:20:46.681049 873 genericapiserver.go:787] Skipping API resource.k8s.io/v1beta1 because it has no resources. machine # [ 20.659717] k3s[873]: W0919 21:20:46.681059 873 genericapiserver.go:787] Skipping API resource.k8s.io/v1alpha3 because it has no resources. machine # [ 20.660879] k3s[873]: I0919 21:20:46.684142 873 handler.go:304] Adding GroupVersion apiregistration.k8s.io v1 to ResourceManager machine # [ 20.661639] k3s[873]: W0919 21:20:46.684160 873 genericapiserver.go:787] Skipping API apiregistration.k8s.io/v1beta1 because it has no resources. machine # [ 20.876470] k3s[873]: I0919 21:20:46.966621 873 dynamic_cafile_content.go:161] "Starting controller" name="request-header::/var/lib/rancher/k3s/server/tls/request-header-ca.crt" machine # [ 20.878626] k3s[873]: I0919 21:20:46.966827 873 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/var/lib/rancher/k3s/server/tls/client-ca.crt" machine # [ 20.880641] k3s[873]: I0919 21:20:46.966952 873 secure_serving.go:211] Serving securely on 127.0.0.1:6444 machine # [ 20.882092] k3s[873]: I0919 21:20:46.966990 873 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.crt::/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.key" machine # [ 20.885300] k3s[873]: I0919 21:20:46.967061 873 aggregator.go:185] waiting for initial CRD sync... machine # [ 20.886838] k3s[873]: I0919 21:20:46.967066 873 tlsconfig.go:243] "Starting DynamicServingCertificateController" machine # [ 20.888861] k3s[873]: I0919 21:20:46.967085 873 controller.go:78] Starting OpenAPI AggregationController machine # [ 20.890578] k3s[873]: I0919 21:20:46.967161 873 system_namespaces_controller.go:66] Starting system namespaces controller machine # [ 20.892562] k3s[873]: I0919 21:20:46.967199 873 remote_available_controller.go:425] Starting RemoteAvailability controller machine # [ 20.894646] k3s[873]: I0919 21:20:46.967220 873 cache.go:32] Waiting for caches to sync for RemoteAvailability controller machine # [ 20.896824] k3s[873]: time="2026-09-19T21:20:46Z" level=info msg="Starting apiserver lease garbage collector" logger=k3s machine # [ 20.898300] k3s[873]: time="2026-09-19T21:20:46Z" level=info msg="Waiting for caches to sync" logger=k3s machine # [ 20.899540] k3s[873]: I0919 21:20:46.967303 873 crdregistration_controller.go:114] Starting crd-autoregister controller machine # [ 20.901227] k3s[873]: I0919 21:20:46.967306 873 controller.go:113] "Deleting old lease on startup" lease="kube-system/apiserver-ik4pcwfs7qgdl76byc4pomgvia" machine # [ 20.903647] k3s[873]: I0919 21:20:46.967318 873 shared_informer.go:349] "Waiting for caches to sync" controller="crd-autoregister" machine # [ 20.905860] k3s[873]: I0919 21:20:46.967348 873 apiservice_controller.go:100] Starting APIServiceRegistrationController machine # [ 20.907805] k3s[873]: I0919 21:20:46.967360 873 cache.go:32] Waiting for caches to sync for APIServiceRegistrationController controller machine # [ 20.909931] k3s[873]: I0919 21:20:46.967412 873 controller.go:80] Starting OpenAPI V3 AggregationController machine # [ 20.911780] k3s[873]: I0919 21:20:46.967439 873 apf_controller.go:377] Starting API Priority and Fairness config controller machine # [ 20.913803] k3s[873]: I0919 21:20:46.967807 873 dynamic_serving_content.go:135] "Starting controller" name="aggregator-proxy-cert::/var/lib/rancher/k3s/server/tls/client-auth-proxy.crt::/var/lib/rancher/k3s/server/tls/client-auth-proxy.key" machine # [ 20.917425] k3s[873]: I0919 21:20:46.967893 873 local_available_controller.go:156] Starting LocalAvailability controller machine # [ 20.919506] k3s[873]: I0919 21:20:46.967913 873 cache.go:32] Waiting for caches to sync for LocalAvailability controller machine # [ 20.921580] k3s[873]: I0919 21:20:46.968096 873 cluster_authentication_trust_controller.go:459] Starting cluster_authentication_trust_controller controller machine # [ 20.924055] k3s[873]: time="2026-09-19T21:20:46Z" level=info msg="Waiting for caches to sync" logger=k3s machine # [ 20.925716] k3s[873]: I0919 21:20:46.968119 873 customresource_discovery_controller.go:294] Starting DiscoveryController machine # [ 20.927716] k3s[873]: time="2026-09-19T21:20:46Z" level=info msg="Starting legacy_token_tracking_controller" logger=k3s machine # [ 20.929661] k3s[873]: I0919 21:20:46.968280 873 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/var/lib/rancher/k3s/server/tls/client-ca.crt" machine # [ 20.932371] k3s[873]: time="2026-09-19T21:20:46Z" level=info msg="Waiting for caches to sync" logger=k3s machine # [ 20.934052] k3s[873]: I0919 21:20:46.968343 873 dynamic_cafile_content.go:161] "Starting controller" name="request-header::/var/lib/rancher/k3s/server/tls/request-header-ca.crt" machine # [ 20.936583] k3s[873]: I0919 21:20:46.968633 873 default_servicecidr_controller.go:110] Starting kubernetes-service-cidr-controller machine # [ 20.938704] k3s[873]: I0919 21:20:46.968648 873 shared_informer.go:349] "Waiting for caches to sync" controller="kubernetes-service-cidr-controller" machine # [ 20.941139] k3s[873]: I0919 21:20:46.981570 873 controller.go:142] Starting OpenAPI controller machine # [ 20.942727] k3s[873]: I0919 21:20:46.981638 873 controller.go:90] Starting OpenAPI V3 controller machine # [ 20.944366] k3s[873]: I0919 21:20:46.981695 873 naming_controller.go:305] Starting NamingConditionController machine # [ 20.945951] k3s[873]: I0919 21:20:46.981774 873 nonstructuralschema_controller.go:202] Starting NonStructuralSchemaConditionController machine # [ 20.947916] k3s[873]: I0919 21:20:46.981804 873 apiapproval_controller.go:196] Starting KubernetesAPIApprovalPolicyConformantConditionController machine # [ 20.950242] k3s[873]: I0919 21:20:46.981834 873 crd_finalizer.go:273] Starting CRDFinalizer machine # [ 20.951749] k3s[873]: I0919 21:20:46.984591 873 repairip.go:210] Starting ipallocator-repair-controller machine # [ 20.953340] k3s[873]: I0919 21:20:46.984609 873 shared_informer.go:349] "Waiting for caches to sync" controller="ipallocator-repair-controller" machine # [ 20.955486] k3s[873]: I0919 21:20:47.035377 873 shared_informer.go:356] "Caches are synced" controller="node_authorizer" machine # [ 20.957524] k3s[873]: I0919 21:20:47.039881 873 shared_informer.go:377] "Caches are synced" machine # [ 20.959049] k3s[873]: I0919 21:20:47.039928 873 policy_source.go:248] refreshing policies machine # [ 20.964637] k3s[873]: E0919 21:20:47.055219 873 controller.go:201] "Failed to ensure lease exists, will retry" err="namespaces \"kube-system\" not found" interval="200ms" machine # [ 20.980078] k3s[873]: I0919 21:20:47.067893 873 shared_informer.go:356] "Caches are synced" controller="crd-autoregister" machine # [ 20.980760] k3s[873]: I0919 21:20:47.067926 873 cache.go:39] Caches are synced for RemoteAvailability controller machine # [ 20.981663] k3s[873]: I0919 21:20:47.067968 873 cache.go:39] Caches are synced for LocalAvailability controller machine # [ 20.982293] k3s[873]: I0919 21:20:47.067974 873 apf_controller.go:382] Running API Priority and Fairness config worker machine # [ 20.983040] k3s[873]: I0919 21:20:47.067989 873 cache.go:39] Caches are synced for APIServiceRegistrationController controller machine # [ 20.983600] k3s[873]: I0919 21:20:47.067993 873 apf_controller.go:385] Running API Priority and Fairness periodic rebalancing process machine # [ 20.984177] k3s[873]: time="2026-09-19T21:20:47Z" level=info msg="Caches are synced" logger=k3s machine # [ 20.984604] k3s[873]: I0919 21:20:47.068124 873 handler_discovery.go:451] Starting ResourceDiscoveryManager machine # [ 20.985116] k3s[873]: time="2026-09-19T21:20:47Z" level=info msg="Caches are synced" logger=k3s machine # [ 20.985560] k3s[873]: time="2026-09-19T21:20:47Z" level=info msg="Caches are synced" logger=k3s machine # [ 20.986125] k3s[873]: I0919 21:20:47.068416 873 aggregator.go:187] initial CRD sync complete... machine # [ 20.986579] k3s[873]: I0919 21:20:47.068432 873 autoregister_controller.go:144] Starting autoregister controller machine # [ 20.987086] k3s[873]: I0919 21:20:47.068443 873 cache.go:32] Waiting for caches to sync for autoregister controller machine # [ 20.987528] k3s[873]: I0919 21:20:47.068454 873 cache.go:39] Caches are synced for autoregister controller machine # [ 20.988049] k3s[873]: I0919 21:20:47.068733 873 shared_informer.go:356] "Caches are synced" controller="kubernetes-service-cidr-controller" machine # [ 20.988551] k3s[873]: I0919 21:20:47.068758 873 default_servicecidr_controller.go:169] Creating default ServiceCIDR with CIDRs: [10.43.0.0/16] machine # [ 20.989055] k3s[873]: I0919 21:20:47.070325 873 controller.go:667] quota admission added evaluator for: namespaces machine # [ 21.004238] k3s[873]: I0919 21:20:47.094839 873 shared_informer.go:356] "Caches are synced" controller="ipallocator-repair-controller" machine # [ 21.007465] k3s[873]: I0919 21:20:47.098083 873 cidrallocator.go:302] created ClusterIP allocator for Service CIDR 10.43.0.0/16 machine # [ 21.007753] k3s[873]: I0919 21:20:47.098173 873 default_servicecidr_controller.go:231] Setting default ServiceCIDR condition Ready to True machine # [ 21.013638] k3s[873]: I0919 21:20:47.103825 873 cidrallocator.go:278] updated ClusterIP allocator for Service CIDR 10.43.0.0/16 machine # [ 21.013939] k3s[873]: I0919 21:20:47.104172 873 default_servicecidr_controller.go:169] Creating default ServiceCIDR with CIDRs: [10.43.0.0/16] machine # [ 21.021062] k3s[873]: I0919 21:20:47.111051 873 default_servicecidr_controller.go:203] Inconsistent ServiceCIDR status for , controller configuration: [10.43.0.0/16], ServiceCIDR configuration: []. Configure the flags to match current ServiceCIDR or manually delete it. machine # [ 21.021377] k3s[873]: time="2026-09-19T21:20:47Z" level=info msg="Event occurred" apiVersion=networking.k8s.io/v1 fieldPath= kind=ServiceCIDR logger=k3s message="The default ServiceCIDR [] does not match the controller flag configurations [10.43.0.0/16]" object= reason=KubernetesDefaultServiceCIDRInconsistent type=Warning machine # [ 21.036632] k3s[873]: I0919 21:20:47.127201 873 alloc.go:329] "allocated clusterIPs" service="default/kubernetes" clusterIPs={"IPv4":"10.43.0.1"} machine # [ 21.042187] k3s[873]: W0919 21:20:47.132798 873 lease.go:265] Resetting endpoints for master service "kubernetes" to [10.0.2.15] machine # [ 21.043712] k3s[873]: I0919 21:20:47.134054 873 controller.go:667] quota admission added evaluator for: endpoints machine # [ 21.049286] k3s[873]: I0919 21:20:47.139830 873 controller.go:667] quota admission added evaluator for: endpointslices.discovery.k8s.io machine # [ 21.054674] k3s[873]: E0919 21:20:47.145272 873 controller.go:102] Error removing old endpoints from kubernetes service: no API server IP addresses were listed in storage, refusing to erase all endpoints for the kubernetes Service machine # [ 21.172080] k3s[873]: I0919 21:20:47.262404 873 controller.go:667] quota admission added evaluator for: leases.coordination.k8s.io machine # [ 21.360693] k3s[873]: time="2026-09-19T21:20:47Z" level=info msg="containerd is now running" machine # [ 21.366373] k3s[873]: time="2026-09-19T21:20:47Z" level=info msg="Importing images from /var/lib/rancher/k3s/agent/images/k3s-airgap-images-amd64.tar.zst" machine # [ 21.883186] k3s[873]: I0919 21:20:47.973318 873 storage_scheduling.go:123] created PriorityClass system-node-critical with value 2000001000 machine # [ 21.889854] k3s[873]: I0919 21:20:47.980461 873 storage_scheduling.go:123] created PriorityClass system-cluster-critical with value 2000000000 machine # [ 21.890212] k3s[873]: I0919 21:20:47.980484 873 storage_scheduling.go:139] all system priority classes are created successfully or already exist. machine # [ 22.301765] k3s[873]: time="2026-09-19T21:20:48Z" level=info msg="Polling for API server readiness: GET /readyz failed: unknown" machine # [ 23.086398] k3s[873]: I0919 21:20:49.176085 873 controller.go:667] quota admission added evaluator for: roles.rbac.authorization.k8s.io machine # [ 23.125252] k3s[873]: I0919 21:20:49.215832 873 controller.go:667] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io machine # [ 24.305370] k3s[873]: time="2026-09-19T21:20:50Z" level=info msg="Kube API server is now running" machine # [ 24.306088] k3s[873]: time="2026-09-19T21:20:50Z" level=info msg="k3s is up and running" machine # [ 24.307102] k3s[873]: time="2026-09-19T21:20:50Z" level=info msg="Waiting for untainted node" machine # [ 24.308293] k3s[873]: time="2026-09-19T21:20:50Z" level=info msg="Creating k3s-supervisor event broadcaster" machine # [ 24.308745] k3s[873]: time="2026-09-19T21:20:50Z" level=info msg="Event occurred" apiVersion= fieldPath= kind=Node logger=k3s message="Node and Certificate Authority certificates managed by k3s are OK" object=machine reason=CertificateExpirationOK type=Normal machine # [ 24.309959] systemd[1]: Started k3s service. machine # [ 24.314520] systemd[1]: Starting niks3 server... machine # [ 24.325216] k3s[873]: I0919 21:20:50.415809 873 controllermanager.go:189] "Starting" version="v1.35.8+k3s1" machine # [ 24.327166] k3s[873]: I0919 21:20:50.415833 873 controllermanager.go:191] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" machine # [ 24.328099] k3s[873]: I0919 21:20:50.418717 873 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController machine # [ 24.329098] k3s[873]: I0919 21:20:50.418730 873 secure_serving.go:211] Serving securely on 127.0.0.1:10257 machine # [ 24.329844] k3s[873]: I0919 21:20:50.418742 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 24.330648] k3s[873]: I0919 21:20:50.418820 873 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" machine # [ 24.330809] k3s[873]: I0919 21:20:50.418836 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 24.331624] k3s[873]: I0919 21:20:50.418865 873 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" machine # [ 24.332280] k3s[873]: I0919 21:20:50.418922 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 24.333024] k3s[873]: I0919 21:20:50.419053 873 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/var/lib/rancher/k3s/server/tls/kube-controller-manager/kube-controller-manager.crt::/var/lib/rancher/k3s/server/tls/kube-controller-manager/kube-controller-manager.key" machine # [ 24.333845] k3s[873]: I0919 21:20:50.419156 873 tlsconfig.go:243] "Starting DynamicServingCertificateController" machine # [ 24.334626] k3s[873]: I0919 21:20:50.424218 873 controller_descriptor.go:99] "Controller is disabled by a feature gate" controller="kube-apiserver-serving-clustertrustbundle-publisher-controller" requiredFeatureGates=["ClusterTrustBundle"] machine # [ 24.335465] k3s[873]: I0919 21:20:50.424255 873 controllermanager.go:579] "Warning: skipping controller" controller="kube-apiserver-serving-clustertrustbundle-publisher-controller" machine # [ 24.339520] k3s[873]: I0919 21:20:50.430095 873 controller.go:667] quota admission added evaluator for: serviceaccounts machine # [ 24.342531] k3s[873]: I0919 21:20:50.433135 873 controller_descriptor.go:99] "Controller is disabled by a feature gate" controller="device-taint-eviction-controller" requiredFeatureGates=["DynamicResourceAllocation","DRADeviceTaints"] machine # [ 24.343452] k3s[873]: I0919 21:20:50.433157 873 controllermanager.go:579] "Warning: skipping controller" controller="device-taint-eviction-controller" machine # [ 24.349403] k3s[873]: W0919 21:20:50.440015 873 probe.go:272] Flexvolume plugin directory at /usr/libexec/kubernetes/kubelet-plugins/volume/exec/ does not exist. Recreating. machine # [ 24.400121] k3s[873]: time="2026-09-19T21:20:50Z" level=info msg="Creating embedded CRD addons.k3s.cattle.io" machine # [ 24.405360] k3s[873]: time="2026-09-19T21:20:50Z" level=info msg="Creating embedded CRD etcdsnapshotfiles.k3s.cattle.io" machine # [ 24.416085] k3s[873]: time="2026-09-19T21:20:50Z" level=info msg="Creating embedded CRD helmchartconfigs.helm.cattle.io" machine # [ 24.424296] k3s[873]: time="2026-09-19T21:20:50Z" level=info msg="Creating embedded CRD helmcharts.helm.cattle.io" machine # [ 24.426155] k3s[873]: I0919 21:20:50.515602 873 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="leases.coordination.k8s.io" machine # [ 24.429158] k3s[873]: I0919 21:20:50.515717 873 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="statefulsets.apps" machine # [ 24.431810] k3s[873]: I0919 21:20:50.515746 873 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="podtemplates" machine # [ 24.434337] k3s[873]: I0919 21:20:50.515796 873 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="replicasets.apps" machine # [ 24.436679] k3s[873]: I0919 21:20:50.515836 873 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="controllerrevisions.apps" machine # [ 24.439309] k3s[873]: I0919 21:20:50.515867 873 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="resourceclaimtemplates.resource.k8s.io" machine # [ 24.442137] k3s[873]: I0919 21:20:50.515913 873 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="horizontalpodautoscalers.autoscaling" machine # [ 24.445472] k3s[873]: I0919 21:20:50.515947 873 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="roles.rbac.authorization.k8s.io" machine # [ 24.448468] k3s[873]: I0919 21:20:50.515976 873 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="deployments.apps" machine # [ 24.450857] k3s[873]: I0919 21:20:50.516004 873 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="cronjobs.batch" machine # [ 24.453453] k3s[873]: I0919 21:20:50.516052 873 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="endpoints" machine # [ 24.456292] k3s[873]: I0919 21:20:50.516102 873 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="endpointslices.discovery.k8s.io" machine # [ 24.459100] k3s[873]: I0919 21:20:50.516131 873 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="serviceaccounts" machine # [ 24.461637] k3s[873]: I0919 21:20:50.516164 873 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="jobs.batch" machine # [ 24.464102] k3s[873]: I0919 21:20:50.516195 873 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="limitranges" machine # [ 24.466845] k3s[873]: I0919 21:20:50.516260 873 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="poddisruptionbudgets.policy" machine # [ 24.469976] k3s[873]: I0919 21:20:50.516288 873 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="rolebindings.rbac.authorization.k8s.io" machine # [ 24.473052] k3s[873]: I0919 21:20:50.516337 873 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="daemonsets.apps" machine # [ 24.475564] k3s[873]: I0919 21:20:50.516367 873 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="networkpolicies.networking.k8s.io" machine # [ 24.478606] k3s[873]: I0919 21:20:50.516394 873 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="ingresses.networking.k8s.io" machine # [ 24.478759] k3s[873]: I0919 21:20:50.516428 873 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" logger="resourcequota-controller" resource="csistoragecapacities.storage.k8s.io" machine # [ 24.480066] k3s[873]: I0919 21:20:50.518869 873 shared_informer.go:377] "Caches are synced" machine # [ 24.481077] k3s[873]: I0919 21:20:50.518927 873 shared_informer.go:377] "Caches are synced" machine # [ 24.481819] k3s[873]: I0919 21:20:50.518982 873 shared_informer.go:377] "Caches are synced" machine # [ 24.482594] k3s[873]: I0919 21:20:50.541178 873 handler.go:304] Adding GroupVersion k3s.cattle.io v1 to ResourceManager machine # [ 24.483505] k3s[873]: I0919 21:20:50.545384 873 handler.go:304] Adding GroupVersion k3s.cattle.io v1 to ResourceManager machine # [ 24.484117] k3s[873]: time="2026-09-19T21:20:50Z" level=info msg="Waiting for CRD etcdsnapshotfiles.k3s.cattle.io to become available" machine # [ 24.484918] k3s[873]: time="2026-09-19T21:20:50Z" level=info msg="Done waiting for CRD etcdsnapshotfiles.k3s.cattle.io to become available" machine # [ 24.485530] k3s[873]: time="2026-09-19T21:20:50Z" level=info msg="Waiting for CRD helmchartconfigs.helm.cattle.io to become available" machine # [ 24.486548] k3s[873]: I0919 21:20:50.561068 873 handler.go:304] Adding GroupVersion helm.cattle.io v1 to ResourceManager machine # [ 24.487250] k3s[873]: time="2026-09-19T21:20:50Z" level=info msg="Done waiting for CRD helmchartconfigs.helm.cattle.io to become available" machine # [ 24.487885] k3s[873]: time="2026-09-19T21:20:50Z" level=info msg="Waiting for CRD helmcharts.helm.cattle.io to become available" machine # [ 24.506886] k3s[873]: I0919 21:20:50.597450 873 handler.go:304] Adding GroupVersion helm.cattle.io v1 to ResourceManager machine # [ 24.510722] k3s[873]: I0919 21:20:50.601294 873 node_lifecycle_controller.go:419] "Controller will reconcile labels" logger="node-lifecycle-controller" machine # [ 24.511158] k3s[873]: I0919 21:20:50.601340 873 controllermanager.go:627] "Warning: controller is disabled" controller="node-route-controller" machine # [ 24.511647] k3s[873]: I0919 21:20:50.601353 873 controllermanager.go:627] "Warning: controller is disabled" controller="cloud-node-lifecycle-controller" machine # [ 24.521045] k3s[873]: I0919 21:20:50.611625 873 shared_informer.go:370] "Waiting for caches to sync" machine: (finished: waiting for unit k3s.service, in 25.57 seconds) ??? Warning (UserWarning): wait_until_succeeds(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead. File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 39 machine: waiting for success: kubectl get ns farm ??? Warning (UserWarning): execute(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead. File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 39 machine # [ 24.722179] k3s[873]: I0919 21:20:50.812129 873 shared_informer.go:377] "Caches are synced" machine # Error from server (NotFound): namespaces "farm" not found machine # [ 24.988926] k3s[873]: time="2026-09-19T21:20:51Z" level=info msg="Done waiting for CRD helmcharts.helm.cattle.io to become available" machine # [ 24.991185] k3s[873]: time="2026-09-19T21:20:51Z" level=info msg="Writing static file: /var/lib/rancher/k3s/server/static/charts/traefik-40.1.4+up40.1.0.tgz" machine # [ 24.993434] k3s[873]: time="2026-09-19T21:20:51Z" level=info msg="Writing static file: /var/lib/rancher/k3s/server/static/charts/traefik-crd-40.1.4+up40.1.0.tgz" machine # [ 24.995861] k3s[873]: time="2026-09-19T21:20:51Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/ccm.yaml" machine # [ 24.997914] k3s[873]: time="2026-09-19T21:20:51Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/coredns.yaml" machine # [ 24.999923] k3s[873]: time="2026-09-19T21:20:51Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/local-storage.yaml" machine # [ 25.002080] k3s[873]: time="2026-09-19T21:20:51Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/rolebindings.yaml" machine # [ 25.004280] k3s[873]: time="2026-09-19T21:20:51Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/runtimes.yaml" machine # [ 25.006386] k3s[873]: time="2026-09-19T21:20:51Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/traefik.yaml" machine # [ 25.090313] k3s[873]: time="2026-09-19T21:20:51Z" level=info msg="Starting dynamiclistener CN filter node controller with SANs: [127.0.0.1 ::1 localhost machine 10.0.2.15 fec0::5fe0:fdce:6860:f263 10.43.0.1 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local]" machine # [ 25.090558] k3s[873]: time="2026-09-19T21:20:51Z" level=info msg="Tunnel server egress proxy mode: agent" machine # [ 25.439097] k3s[873]: I0919 21:20:51.529254 873 controllermanager.go:627] "Warning: controller is disabled" controller="service-lb-controller" machine # [ 25.588687] k3s[873]: time="2026-09-19T21:20:51Z" level=info msg="Creating new TLS secret for kube-system/k3s-serving (count: 11): map[listener.cattle.io/cn-10.0.2.15:10.0.2.15 listener.cattle.io/cn-10.43.0.1:10.43.0.1 listener.cattle.io/cn-127.0.0.1:127.0.0.1 listener.cattle.io/cn-__1-f16284:::1 listener.cattle.io/cn-fec0__5fe0_fdce_6860_f263-5f27ad:fec0::5fe0:fdce:6860:f263 listener.cattle.io/cn-kubernetes:kubernetes listener.cattle.io/cn-kubernetes.default:kubernetes.default listener.cattle.io/cn-kubernetes.default.svc:kubernetes.default.svc listener.cattle.io/cn-kubernetes.default.svc.cluster.local:kubernetes.default.svc.cluster.local listener.cattle.io/cn-localhost:localhost listener.cattle.io/cn-machine:machine listener.cattle.io/fingerprint:SHA1=07CEF5D751C0D029F060778B8FB7B4C3D85C6926]" machine # [ 25.605479] k3s[873]: time="2026-09-19T21:20:51Z" level=info msg="Active TLS secret kube-system/k3s-serving (ver=238) (count 11): map[listener.cattle.io/cn-10.0.2.15:10.0.2.15 listener.cattle.io/cn-10.43.0.1:10.43.0.1 listener.cattle.io/cn-127.0.0.1:127.0.0.1 listener.cattle.io/cn-__1-f16284:::1 listener.cattle.io/cn-fec0__5fe0_fdce_6860_f263-5f27ad:fec0::5fe0:fdce:6860:f263 listener.cattle.io/cn-kubernetes:kubernetes listener.cattle.io/cn-kubernetes.default:kubernetes.default listener.cattle.io/cn-kubernetes.default.svc:kubernetes.default.svc listener.cattle.io/cn-kubernetes.default.svc.cluster.local:kubernetes.default.svc.cluster.local listener.cattle.io/cn-localhost:localhost listener.cattle.io/cn-machine:machine listener.cattle.io/fingerprint:SHA1=07CEF5D751C0D029F060778B8FB7B4C3D85C6926]" machine # [ 25.976400] k3s[873]: time="2026-09-19T21:20:52Z" level=info msg="Imported docker.io/rancher/klipper-helm:v0.13.3-build20260727" machine # [ 25.979065] k3s[873]: time="2026-09-19T21:20:52Z" level=info msg="Tagged docker.io/rancher/klipper-helm@sha256:5425a2b4613d2458dbdff99218e8d487ebb59c20e7a941f061c0777070f0aac1" machine # [ 25.979620] k3s[873]: time="2026-09-19T21:20:52Z" level=info msg="Imported docker.io/rancher/klipper-lb:v0.4.17" machine # [ 25.983519] k3s[873]: time="2026-09-19T21:20:52Z" level=info msg="Tagged docker.io/rancher/klipper-lb@sha256:910944bb0bd94f060a82a56ca1ea1c577d3e49b3473a093a47a985f32e92d94a" machine # [ 25.983944] k3s[873]: time="2026-09-19T21:20:52Z" level=info msg="Imported docker.io/rancher/local-path-provisioner:v0.0.37" machine # [ 25.988043] k3s[873]: time="2026-09-19T21:20:52Z" level=info msg="Tagged docker.io/rancher/local-path-provisioner@sha256:e757967a5ec338f6a9b371c5a9688bedaa8c3578ea3dd4db329ea0084be0a86f" machine # [ 25.988905] k3s[873]: time="2026-09-19T21:20:52Z" level=info msg="Imported docker.io/rancher/mirrored-coredns-coredns:1.14.6" machine # [ 25.993419] k3s[873]: time="2026-09-19T21:20:52Z" level=info msg="Tagged docker.io/rancher/mirrored-coredns-coredns@sha256:900f9c109f7a33545d3c811516e8376df9019147b750f5ce3e254468769176ea" machine # [ 25.993710] k3s[873]: time="2026-09-19T21:20:52Z" level=info msg="Imported docker.io/rancher/mirrored-library-busybox:1.37.0" machine # [ 25.997560] k3s[873]: time="2026-09-19T21:20:52Z" level=info msg="Tagged docker.io/rancher/mirrored-library-busybox@sha256:101b4afd76732482eff9b95cae5f94bcf295e521fbec4e01b69c5421f3f3f3e5" machine # [ 25.998224] k3s[873]: time="2026-09-19T21:20:52Z" level=info msg="Imported docker.io/rancher/mirrored-library-traefik:3.7.8" machine # [ 26.001753] k3s[873]: time="2026-09-19T21:20:52Z" level=info msg="Tagged docker.io/rancher/mirrored-library-traefik@sha256:4299bbed850421258fc5448c2e0e6ad350981d4d335a68de11b92448aedbefe5" machine # [ 26.002183] k3s[873]: time="2026-09-19T21:20:52Z" level=info msg="Imported docker.io/rancher/mirrored-metrics-server:v0.9.0" machine # [ 26.008557] k3s[873]: time="2026-09-19T21:20:52Z" level=info msg="Tagged docker.io/rancher/mirrored-metrics-server@sha256:d9862115e7c7881280d3d75ca26bda8ffc0fc213315979575bf23ce9826205c0" machine # [ 26.009022] k3s[873]: time="2026-09-19T21:20:52Z" level=info msg="Imported docker.io/rancher/mirrored-pause:3.10.2" machine # [ 26.014817] k3s[873]: time="2026-09-19T21:20:52Z" level=info msg="Tagged docker.io/rancher/mirrored-pause@sha256:f548e0e8e3dc1896ca956272154dde3314e8cc4fde0a57577ee9fa1c63f5baf4" machine # Error from server (NotFound): namespaces "farm" not found machine # [ 26.188606] k3s[873]: I0919 21:20:52.279157 873 controllermanager.go:627] "Warning: controller is disabled" controller="selinux-warning-controller" machine # [ 26.197411] k3s[873]: time="2026-09-19T21:20:52Z" level=info msg="Starting k3s.cattle.io/v1, Kind=Addon controller" machine # [ 26.197743] k3s[873]: time="2026-09-19T21:20:52Z" level=info msg="Creating deploy event broadcaster" machine # [ 26.202052] k3s[873]: I0919 21:20:52.292635 873 controller.go:667] quota admission added evaluator for: addons.k3s.cattle.io machine # [ 26.206600] k3s[873]: time="2026-09-19T21:20:52Z" level=info msg="Starting /v1, Kind=Node controller" machine # [ 26.207350] k3s[873]: time="2026-09-19T21:20:52Z" level=info msg="Event occurred" apiVersion=k3s.cattle.io/v1 fieldPath= kind=Addon logger=k3s/deploy message="Applying manifest at \"/var/lib/rancher/k3s/server/manifests/ccm.yaml\"" object=kube-system/ccm reason=ApplyingManifest type=Normal machine # [ 26.211042] k3s[873]: time="2026-09-19T21:20:52Z" level=info msg="Unable to set control-plane role label: nodes \"machine\" not found" machine # [ 26.213801] k3s[873]: time="2026-09-19T21:20:52Z" level=info msg="Creating helm-controller event broadcaster" machine # [ 26.219071] k3s[873]: time="2026-09-19T21:20:52Z" level=info msg="Cluster dns configmap has been set successfully" machine # [ 26.266490] k3s[873]: time="2026-09-19T21:20:52Z" level=info msg="Imported 8 images from /var/lib/rancher/k3s/agent/images/k3s-airgap-images-amd64.tar.zst in 4.900803314s" machine # [ 26.269374] k3s[873]: time="2026-09-19T21:20:52Z" level=info msg="Importing images from /var/lib/rancher/k3s/agent/images/nix-grpc-farm-lb.tar" machine # [ 26.640222] k3s[873]: I0919 21:20:52.730134 873 controller_descriptor.go:99] "Controller is disabled by a feature gate" controller="podcertificaterequest-cleaner-controller" requiredFeatureGates=["PodCertificateRequest"] machine # [ 26.643719] k3s[873]: I0919 21:20:52.730190 873 controllermanager.go:579] "Warning: skipping controller" controller="podcertificaterequest-cleaner-controller" machine # [ 26.648293] k3s[873]: time="2026-09-19T21:20:52Z" level=info msg="Event occurred" apiVersion=k3s.cattle.io/v1 fieldPath= kind=Addon logger=k3s/deploy message="Applied manifest at \"/var/lib/rancher/k3s/server/manifests/ccm.yaml\"" object=kube-system/ccm reason=AppliedManifest type=Normal machine # [ 26.660366] k3s[873]: time="2026-09-19T21:20:52Z" level=info msg="Event occurred" apiVersion=k3s.cattle.io/v1 fieldPath= kind=Addon logger=k3s/deploy message="Applying manifest at \"/var/lib/rancher/k3s/server/manifests/ci.yaml\"" object=kube-system/ci reason=ApplyingManifest type=Normal machine # [ 26.939624] k3s[873]: I0919 21:20:53.030101 873 controller_descriptor.go:99] "Controller is disabled by a feature gate" controller="storageversion-garbage-collector-controller" requiredFeatureGates=["APIServerIdentity","StorageVersionAPI"] machine # [ 26.943523] k3s[873]: I0919 21:20:53.030146 873 controllermanager.go:579] "Warning: skipping controller" controller="storageversion-garbage-collector-controller" machine # [ 26.946341] k3s[873]: time="2026-09-19T21:20:53Z" level=info msg="Event occurred" apiVersion=k3s.cattle.io/v1 fieldPath= kind=Addon logger=k3s/deploy message="Applied manifest at \"/var/lib/rancher/k3s/server/manifests/ci.yaml\"" object=kube-system/ci reason=AppliedManifest type=Normal machine # [ 26.953319] k3s[873]: time="2026-09-19T21:20:53Z" level=info msg="Event occurred" apiVersion=k3s.cattle.io/v1 fieldPath= kind=Addon logger=k3s/deploy message="Applying manifest at \"/var/lib/rancher/k3s/server/manifests/coredns.yaml\"" object=kube-system/coredns reason=ApplyingManifest type=Normal machine # [ 27.210379] k3s[873]: time="2026-09-19T21:20:53Z" level=info msg="Unable to set control-plane role label: nodes \"machine\" not found" machine: (finished: waiting for success: kubectl get ns farm, in 2.71 seconds) machine: must succeed: kubectl -n farm create secret tls farm-lb-tls --cert=/nix/store/kp4hvipjm36bm67lbwnmka0j153vc80m-k3s-farm-certs/lb.crt --key=/nix/store/kp4hvipjm36bm67lbwnmka0j153vc80m-k3s-farm-certs/lb.key machine # [ 27.364185] k3s[873]: I0919 21:20:53.454721 873 serving.go:392] Generated self-signed cert in-memory machine # [ 27.426099] k3s[873]: time="2026-09-19T21:20:53Z" level=info msg="Starting rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding controller" machine # [ 27.432055] k3s[873]: time="2026-09-19T21:20:53Z" level=info msg="Starting batch/v1, Kind=Job controller" machine # [ 27.433956] k3s[873]: time="2026-09-19T21:20:53Z" level=info msg="Starting /v1, Kind=Secret controller" machine # [ 27.435738] k3s[873]: time="2026-09-19T21:20:53Z" level=info msg="Starting /v1, Kind=ConfigMap controller" machine # [ 27.437885] k3s[873]: time="2026-09-19T21:20:53Z" level=info msg="Starting /v1, Kind=ServiceAccount controller" machine # [ 27.439533] k3s[873]: time="2026-09-19T21:20:53Z" level=info msg="Starting helm.cattle.io/v1, Kind=HelmChart controller" machine # [ 27.441335] k3s[873]: time="2026-09-19T21:20:53Z" level=info msg="Starting helm.cattle.io/v1, Kind=HelmChartConfig controller" machine # [ 27.503637] k3s[873]: I0919 21:20:53.594137 873 serving.go:392] Generated self-signed cert in-memory machine: (finished: must succeed: kubectl -n farm create secret tls farm-lb-tls --cert=/nix/store/kp4hvipjm36bm67lbwnmka0j153vc80m-k3s-farm-certs/lb.crt --key=/nix/store/kp4hvipjm36bm67lbwnmka0j153vc80m-k3s-farm-certs/lb.key, in 0.24 seconds) machine: must succeed: kubectl -n farm create secret tls farm-worker-tls --cert=/nix/store/kp4hvipjm36bm67lbwnmka0j153vc80m-k3s-farm-certs/worker.crt --key=/nix/store/kp4hvipjm36bm67lbwnmka0j153vc80m-k3s-farm-certs/worker.key machine # [ 27.641881] k3s[873]: I0919 21:20:53.732033 873 controllermanager.go:160] Version: v1.35.8+k3s1 machine # [ 27.644875] k3s[873]: I0919 21:20:53.735491 873 secure_serving.go:211] Serving securely on 127.0.0.1:10258 machine # [ 27.646242] k3s[873]: I0919 21:20:53.736011 873 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController machine # [ 27.647706] k3s[873]: I0919 21:20:53.736043 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 27.649370] k3s[873]: I0919 21:20:53.736088 873 tlsconfig.go:243] "Starting DynamicServingCertificateController" machine # [ 27.650958] k3s[873]: I0919 21:20:53.736191 873 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" machine # [ 27.653518] k3s[873]: I0919 21:20:53.736230 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 27.654947] k3s[873]: I0919 21:20:53.736268 873 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" machine # [ 27.657431] k3s[873]: I0919 21:20:53.736289 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 27.658920] k3s[873]: time="2026-09-19T21:20:53Z" level=info msg="Creating service-lb-controller event broadcaster" machine # [ 27.666248] k3s[873]: I0919 21:20:53.756816 873 controller.go:667] quota admission added evaluator for: deployments.apps machine # [ 27.688759] k3s[873]: I0919 21:20:53.779131 873 controllermanager.go:627] "Warning: controller is disabled" controller="bootstrap-signer-controller" machine # [ 27.689333] k3s[873]: I0919 21:20:53.779160 873 controllermanager.go:579] "Warning: skipping controller" controller="storage-version-migrator-controller" machine: (finished: must succeed: kubectl -n farm create secret tls farm-worker-tls --cert=/nix/store/kp4hvipjm36bm67lbwnmka0j153vc80m-k3s-farm-certs/worker.crt --key=/nix/store/kp4hvipjm36bm67lbwnmka0j153vc80m-k3s-farm-certs/worker.key, in 0.20 seconds) machine: must succeed: kubectl -n farm create secret generic farm-ca --from-file=ca.crt=/nix/store/kp4hvipjm36bm67lbwnmka0j153vc80m-k3s-farm-certs/ca.crt machine # [ 27.758300] k3s[873]: I0919 21:20:53.848768 873 alloc.go:329] "allocated clusterIPs" service="kube-system/kube-dns" clusterIPs={"IPv4":"10.43.0.10"} machine # [ 27.760579] k3s[873]: time="2026-09-19T21:20:53Z" level=info msg="Event occurred" apiVersion=k3s.cattle.io/v1 fieldPath= kind=Addon logger=k3s/deploy message="Applied manifest at \"/var/lib/rancher/k3s/server/manifests/coredns.yaml\"" object=kube-system/coredns reason=AppliedManifest type=Normal machine # [ 27.845600] k3s[873]: I0919 21:20:53.936124 873 shared_informer.go:377] "Caches are synced" machine # [ 27.850065] k3s[873]: I0919 21:20:53.938740 873 shared_informer.go:377] "Caches are synced" machine # [ 27.851284] k3s[873]: I0919 21:20:53.938859 873 shared_informer.go:377] "Caches are synced" machine: (finished: must succeed: kubectl -n farm create secret generic farm-ca --from-file=ca.crt=/nix/store/kp4hvipjm36bm67lbwnmka0j153vc80m-k3s-farm-certs/ca.crt, in 0.22 seconds) subtest: chart deploys and workers become Ready machine: waiting for success: kubectl -n farm rollout status deployment nix-grpc-farm-lb --timeout=10s machine # [ 28.058960] k3s[873]: time="2026-09-19T21:20:54Z" level=info msg="Starting /v1, Kind=Node controller" machine # [ 28.068665] k3s[873]: time="2026-09-19T21:20:54Z" level=info msg="Starting /v1, Kind=Pod controller" machine # [ 28.075691] k3s[873]: time="2026-09-19T21:20:54Z" level=info msg="Starting apps/v1, Kind=DaemonSet controller" machine # [ 28.080286] k3s[873]: time="2026-09-19T21:20:54Z" level=info msg="Starting discovery.k8s.io/v1, Kind=EndpointSlice controller" machine # [ 28.081986] k3s[873]: I0919 21:20:54.171474 873 controllermanager.go:329] Started "service-lb-controller" machine # [ 28.083642] k3s[873]: W0919 21:20:54.171501 873 controllermanager.go:306] "node-route-controller" is disabled machine # [ 28.085377] k3s[873]: I0919 21:20:54.171736 873 controller.go:235] Starting service controller machine # [ 28.086696] k3s[873]: I0919 21:20:54.171803 873 controllermanager.go:329] Started "cloud-node-controller" machine # [ 28.089699] k3s[873]: I0919 21:20:54.171774 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 28.091046] k3s[873]: I0919 21:20:54.171961 873 controllermanager.go:329] Started "cloud-node-lifecycle-controller" machine # [ 28.092484] k3s[873]: I0919 21:20:54.172057 873 node_controller.go:176] Sending events to api server. machine # [ 28.093786] k3s[873]: I0919 21:20:54.172155 873 node_controller.go:185] Waiting for informer caches to sync machine # [ 28.095180] k3s[873]: I0919 21:20:54.172349 873 node_lifecycle_controller.go:112] Sending events to api server machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-lb" not found machine # [ 28.210544] k3s[873]: time="2026-09-19T21:20:54Z" level=info msg="Unable to set control-plane role label: nodes \"machine\" not found" machine # [ 28.382240] k3s[873]: I0919 21:20:54.472709 873 shared_informer.go:377] "Caches are synced" machine # [ 28.489736] k3s[873]: time="2026-09-19T21:20:54Z" level=info msg="Event occurred" apiVersion=k3s.cattle.io/v1 fieldPath= kind=Addon logger=k3s/deploy message="Deleting manifest at \"/var/lib/rancher/k3s/server/manifests/local-storage.yaml\"" object=kube-system/local-storage reason=DeletingManifest type=Normal machine # [ 28.499666] k3s[873]: time="2026-09-19T21:20:54Z" level=info msg="Event occurred" apiVersion=k3s.cattle.io/v1 fieldPath= kind=Addon logger=k3s/deploy message="Applying manifest at \"/var/lib/rancher/k3s/server/manifests/nix-grpc-farm.yaml\"" object=kube-system/nix-grpc-farm reason=ApplyingManifest type=Normal machine # [ 28.641912] k3s[873]: I0919 21:20:54.731968 873 controller.go:667] quota admission added evaluator for: helmcharts.helm.cattle.io machine # [ 28.646185] k3s[873]: time="2026-09-19T21:20:54Z" level=info msg="Event occurred" apiVersion=k3s.cattle.io/v1 fieldPath= kind=Addon logger=k3s/deploy message="Applied manifest at \"/var/lib/rancher/k3s/server/manifests/nix-grpc-farm.yaml\"" object=kube-system/nix-grpc-farm reason=AppliedManifest type=Normal machine # [ 28.658771] k3s[873]: time="2026-09-19T21:20:54Z" level=info msg="Event occurred" apiVersion=k3s.cattle.io/v1 fieldPath= kind=Addon logger=k3s/deploy message="Applying manifest at \"/var/lib/rancher/k3s/server/manifests/rolebindings.yaml\"" object=kube-system/rolebindings reason=ApplyingManifest type=Normal machine # [ 29.211865] k3s[873]: time="2026-09-19T21:20:55Z" level=info msg="Unable to set control-plane role label: nodes \"machine\" not found" machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-lb" not found machine # [ 29.493850] k3s[873]: I0919 21:20:55.584340 873 range_allocator.go:113] "No Secondary Service CIDR provided. Skipping filtering out secondary service addresses" logger="node-ipam-controller" machine # [ 29.498088] k3s[873]: time="2026-09-19T21:20:55Z" level=info msg="Event occurred" apiVersion=helm.cattle.io/v1 fieldPath= kind=HelmChart logger=k3s/helm-controller message="Applying HelmChart from https://%{KUBERNETES_API}%/static/charts/nix-grpc-farm.tgz using Job kube-system/helm-install-nix-grpc-farm " object=kube-system/nix-grpc-farm reason=ApplyJob type=Normal machine # [ 29.501711] k3s[873]: I0919 21:20:55.592263 873 node_ipam_controller.go:142] "Starting ipam controller" machine # [ 29.502542] k3s[873]: I0919 21:20:55.592292 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.503081] k3s[873]: I0919 21:20:55.593330 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.503536] k3s[873]: I0919 21:20:55.593425 873 controller.go:423] "Starting resource claim controller" machine # [ 29.505099] k3s[873]: I0919 21:20:55.593439 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.505732] k3s[873]: I0919 21:20:55.593544 873 attach_detach_controller.go:335] "Starting attach detach controller" machine # [ 29.506441] k3s[873]: I0919 21:20:55.593560 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.506936] k3s[873]: I0919 21:20:55.593590 873 expand_controller.go:328] "Starting expand controller" machine # [ 29.507618] k3s[873]: I0919 21:20:55.593606 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.511104] k3s[873]: I0919 21:20:55.593643 873 pvc_protection_controller.go:166] "Starting PVC protection controller" machine # [ 29.511782] k3s[873]: I0919 21:20:55.593656 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.512491] k3s[873]: I0919 21:20:55.593706 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.513239] k3s[873]: I0919 21:20:55.594195 873 job_controller.go:254] "Starting job controller" machine # [ 29.513762] k3s[873]: I0919 21:20:55.594211 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.514475] k3s[873]: I0919 21:20:55.594262 873 node_lifecycle_controller.go:453] "Sending events to api server" machine # [ 29.517704] k3s[873]: I0919 21:20:55.594313 873 node_lifecycle_controller.go:460] "Starting node controller" machine # [ 29.518405] k3s[873]: I0919 21:20:55.594324 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.519043] k3s[873]: I0919 21:20:55.594367 873 vac_protection_controller.go:206] "Starting VAC protection controller" machine # [ 29.519852] k3s[873]: I0919 21:20:55.594384 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.520519] k3s[873]: I0919 21:20:55.594420 873 legacy_serviceaccount_token_cleaner.go:103] "Starting legacy service account token cleaner controller" machine # [ 29.521555] k3s[873]: I0919 21:20:55.594432 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.522194] k3s[873]: I0919 21:20:55.594504 873 replica_set.go:241] "Starting controller" name="replicationcontroller" machine # [ 29.522740] k3s[873]: I0919 21:20:55.594517 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.523502] k3s[873]: I0919 21:20:55.594586 873 replica_set.go:241] "Starting controller" name="replicaset" machine # [ 29.524138] k3s[873]: I0919 21:20:55.594601 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.524715] k3s[873]: I0919 21:20:55.594640 873 disruption.go:458] "Sending events to api server." machine # [ 29.525503] k3s[873]: I0919 21:20:55.594693 873 disruption.go:465] "Starting disruption controller" machine # [ 29.526657] k3s[873]: I0919 21:20:55.594704 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.527372] k3s[873]: I0919 21:20:55.594743 873 cleaner.go:83] "Starting CSR cleaner controller" machine # [ 29.528551] k3s[873]: I0919 21:20:55.594829 873 pv_controller_base.go:307] "Starting persistent volume controller" machine # [ 29.529858] k3s[873]: I0919 21:20:55.594846 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.530561] k3s[873]: I0919 21:20:55.594877 873 pv_protection_controller.go:81] "Starting PV protection controller" machine # [ 29.531219] k3s[873]: I0919 21:20:55.594895 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.531688] k3s[873]: I0919 21:20:55.594933 873 ttlafterfinished_controller.go:112] "Starting TTL after finished controller" machine # [ 29.532237] k3s[873]: I0919 21:20:55.594946 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.532733] k3s[873]: I0919 21:20:55.594988 873 clusterroleaggregation_controller.go:194] "Starting ClusterRoleAggregator controller" machine # [ 29.533493] k3s[873]: I0919 21:20:55.595015 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.534300] k3s[873]: I0919 21:20:55.595069 873 controller.go:174] "Starting ephemeral volume controller" machine # [ 29.535161] k3s[873]: I0919 21:20:55.595085 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.535789] k3s[873]: I0919 21:20:55.595191 873 endpoints_controller.go:193] "Starting endpoint controller" machine # [ 29.536624] k3s[873]: I0919 21:20:55.595207 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.537259] k3s[873]: I0919 21:20:55.595242 873 serviceaccounts_controller.go:117] "Starting service account controller" machine # [ 29.537872] k3s[873]: I0919 21:20:55.595255 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.539092] k3s[873]: I0919 21:20:55.595316 873 deployment_controller.go:172] "Starting controller" controller="deployment" machine # [ 29.539432] k3s[873]: I0919 21:20:55.595378 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.540624] k3s[873]: I0919 21:20:55.595421 873 tokencleaner.go:117] "Starting token cleaner controller" machine # [ 29.541574] k3s[873]: I0919 21:20:55.595435 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.542196] k3s[873]: I0919 21:20:55.595476 873 publisher.go:107] "Starting root CA cert publisher controller" machine # [ 29.542782] k3s[873]: I0919 21:20:55.595496 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.543561] k3s[873]: I0919 21:20:55.595526 873 taint_eviction.go:283] "Starting" controller="taint-eviction-controller" machine # [ 29.544367] k3s[873]: I0919 21:20:55.595593 873 taint_eviction.go:288] "Sending events to API server" machine # [ 29.545212] k3s[873]: I0919 21:20:55.595604 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.545960] k3s[873]: I0919 21:20:55.595646 873 namespace_controller.go:202] "Starting namespace controller" machine # [ 29.546966] k3s[873]: I0919 21:20:55.595660 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.548073] k3s[873]: I0919 21:20:55.595807 873 daemon_controller.go:309] "Starting daemon sets controller" machine # [ 29.549182] k3s[873]: I0919 21:20:55.595822 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.550223] k3s[873]: I0919 21:20:55.595887 873 stateful_set.go:180] "Starting stateful set controller" machine # [ 29.550944] k3s[873]: I0919 21:20:55.595917 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.551535] k3s[873]: I0919 21:20:55.595947 873 ttl_controller.go:127] "Starting TTL controller" machine # [ 29.552097] k3s[873]: I0919 21:20:55.595963 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.552503] k3s[873]: I0919 21:20:55.596056 873 endpointslicemirroring_controller.go:226] "Starting EndpointSliceMirroring controller" machine # [ 29.553310] k3s[873]: I0919 21:20:55.596068 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.554291] k3s[873]: I0919 21:20:55.596103 873 gc_controller.go:98] "Starting GC controller" machine # [ 29.554778] k3s[873]: I0919 21:20:55.596119 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.555464] k3s[873]: I0919 21:20:55.596183 873 cronjob_controllerv2.go:143] "Starting cronjob controller v2" machine # [ 29.556028] k3s[873]: I0919 21:20:55.596207 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.556555] k3s[873]: I0919 21:20:55.596269 873 certificate_controller.go:120] "Starting certificate controller" name="csrsigning-legacy-unknown" machine # [ 29.557048] k3s[873]: I0919 21:20:55.596283 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.557539] k3s[873]: I0919 21:20:55.596355 873 servicecidrs_controller.go:136] "Starting" controller="service-cidr-controller" machine # [ 29.558062] k3s[873]: I0919 21:20:55.596369 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.558557] k3s[873]: I0919 21:20:55.596506 873 endpointslice_controller.go:283] "Starting endpoint slice controller" machine # [ 29.559070] k3s[873]: I0919 21:20:55.596525 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.559542] k3s[873]: I0919 21:20:55.596577 873 horizontal.go:204] "Starting HPA controller" machine # [ 29.560069] k3s[873]: I0919 21:20:55.596590 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.560584] k3s[873]: I0919 21:20:55.596617 873 certificate_controller.go:120] "Starting certificate controller" name="csrapproving" machine # [ 29.561137] k3s[873]: I0919 21:20:55.596631 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.561645] k3s[873]: I0919 21:20:55.597455 873 resource_quota_controller.go:297] "Starting resource quota controller" machine # [ 29.562357] k3s[873]: I0919 21:20:55.597473 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.562865] k3s[873]: I0919 21:20:55.597548 873 garbagecollector.go:141] "Starting controller" controller="garbagecollector" machine # [ 29.563378] k3s[873]: I0919 21:20:55.597581 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.563909] k3s[873]: I0919 21:20:55.597769 873 certificate_controller.go:120] "Starting certificate controller" name="csrsigning-kubelet-serving" machine # [ 29.564531] k3s[873]: I0919 21:20:55.597782 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.565078] k3s[873]: I0919 21:20:55.597819 873 certificate_controller.go:120] "Starting certificate controller" name="csrsigning-kubelet-client" machine # [ 29.566872] k3s[873]: I0919 21:20:55.597838 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.567885] k3s[873]: I0919 21:20:55.597882 873 certificate_controller.go:120] "Starting certificate controller" name="csrsigning-kube-apiserver-client" machine # [ 29.569634] k3s[873]: I0919 21:20:55.597895 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.570801] k3s[873]: I0919 21:20:55.597938 873 dynamic_serving_content.go:135] "Starting controller" name="csr-controller::/var/lib/rancher/k3s/server/tls/server-ca.nochain.crt::/var/lib/rancher/k3s/server/tls/server-ca.key" machine # [ 29.573471] k3s[873]: I0919 21:20:55.598110 873 resource_quota_monitor.go:309] "QuotaMonitor running" machine # [ 29.574592] k3s[873]: I0919 21:20:55.598238 873 graph_builder.go:386] "Running" component="GraphBuilder" machine # [ 29.575626] k3s[873]: I0919 21:20:55.598273 873 dynamic_serving_content.go:135] "Starting controller" name="csr-controller::/var/lib/rancher/k3s/server/tls/server-ca.nochain.crt::/var/lib/rancher/k3s/server/tls/server-ca.key" machine # [ 29.578060] k3s[873]: I0919 21:20:55.598391 873 dynamic_serving_content.go:135] "Starting controller" name="csr-controller::/var/lib/rancher/k3s/server/tls/client-ca.nochain.crt::/var/lib/rancher/k3s/server/tls/client-ca.key" machine # [ 29.580662] k3s[873]: I0919 21:20:55.598468 873 dynamic_serving_content.go:135] "Starting controller" name="csr-controller::/var/lib/rancher/k3s/server/tls/client-ca.nochain.crt::/var/lib/rancher/k3s/server/tls/client-ca.key" machine # [ 29.631066] k3s[873]: I0919 21:20:55.721624 873 shared_informer.go:377] "Caches are synced" machine # [ 29.632579] k3s[873]: I0919 21:20:55.723187 873 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" resource="helmcharts.helm.cattle.io" machine # [ 29.633222] k3s[873]: I0919 21:20:55.723235 873 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" resource="addons.k3s.cattle.io" machine # [ 29.642438] k3s[873]: I0919 21:20:55.723266 873 resource_quota_monitor.go:229] "QuotaMonitor created object count evaluator" resource="helmchartconfigs.helm.cattle.io" machine # [ 29.642580] k3s[873]: I0919 21:20:55.729864 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.656357] k3s[873]: I0919 21:20:55.746957 873 controller.go:667] quota admission added evaluator for: jobs.batch machine # [ 29.658931] k3s[873]: I0919 21:20:55.749538 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 29.662898] k3s[873]: time="2026-09-19T21:20:55Z" level=info msg="Event occurred" apiVersion= fieldPath= kind=Node logger=k3s message="Deferred node password secret validation complete" object=machine reason=NodePasswordValidationComplete type=Normal machine # [ 29.703089] k3s[873]: I0919 21:20:55.793428 873 shared_informer.go:377] "Caches are synced" machine # [ 29.703961] k3s[873]: I0919 21:20:55.794488 873 shared_informer.go:377] "Caches are synced" machine # [ 29.704876] k3s[873]: I0919 21:20:55.795208 873 shared_informer.go:377] "Caches are synced" machine # [ 29.705609] k3s[873]: I0919 21:20:55.795887 873 shared_informer.go:377] "Caches are synced" machine # [ 29.706563] k3s[873]: I0919 21:20:55.796494 873 shared_informer.go:377] "Caches are synced" machine # [ 30.155245] k3s[873]: time="2026-09-19T21:20:56Z" level=info msg="Imported docker.io/library/nix-grpc-farm-lb:0.1.0" machine # [ 30.157032] k3s[873]: time="2026-09-19T21:20:56Z" level=info msg="Tagged docker.io/library/nix-grpc-farm-lb@sha256:a3fb275e15335967c368c203214cabcce2886a8867fcd31c07e4effd2ad9a5ec" machine # [ 30.202425] k3s[873]: time="2026-09-19T21:20:56Z" level=info msg="Imported 1 images from /var/lib/rancher/k3s/agent/images/nix-grpc-farm-lb.tar in 3.934691141s" machine # [ 30.203596] k3s[873]: time="2026-09-19T21:20:56Z" level=info msg="Importing images from /var/lib/rancher/k3s/agent/images/nix-grpc-farm.tar" machine # [ 30.210115] k3s[873]: time="2026-09-19T21:20:56Z" level=info msg="Unable to set control-plane role label: nodes \"machine\" not found" machine # [ 30.379430] k3s[873]: time="2026-09-19T21:20:56Z" level=info msg="Running kubelet --cloud-provider=external --config-dir=/var/lib/rancher/k3s/agent/etc/kubelet.conf.d --containerd=/run/k3s/containerd/containerd.sock --eviction-hard=nodefs.available<1%,imagefs.available<1% --hostname-override=machine --kubeconfig=/var/lib/rancher/k3s/agent/kubelet.kubeconfig --node-ip=10.0.2.15,fec0::5fe0:fdce:6860:f263 --node-labels= --read-only-port=0" machine # [ 30.380179] k3s[873]: Flag --containerd has been deprecated, This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed. machine # [ 30.392767] k3s[873]: I0919 21:20:56.483341 873 server.go:521] "Kubelet version" kubeletVersion="v1.35.8+k3s1" machine # [ 30.393133] k3s[873]: I0919 21:20:56.483371 873 server.go:523] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" machine # [ 30.393610] k3s[873]: I0919 21:20:56.483429 873 watchdog_linux.go:95] "Systemd watchdog is not enabled" machine # [ 30.394070] k3s[873]: I0919 21:20:56.483445 873 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." machine # [ 30.394531] k3s[873]: I0919 21:20:56.484950 873 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/var/lib/rancher/k3s/agent/client-ca.crt" machine # [ 30.403817] k3s[873]: I0919 21:20:56.494428 873 server.go:1414] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" machine # [ 30.409486] k3s[873]: I0919 21:20:56.500056 873 server.go:771] "--cgroups-per-qos enabled, but --cgroup-root was not specified. Defaulting to /" machine # [ 30.409925] k3s[873]: I0919 21:20:56.500092 873 server.go:832] "NoSwap is set due to memorySwapBehavior not specified" memorySwapBehavior="" FailSwapOn=false machine # [ 30.410645] k3s[873]: I0919 21:20:56.500325 873 container_manager_linux.go:272] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] machine # [ 30.411088] k3s[873]: I0919 21:20:56.500351 873 container_manager_linux.go:277] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"machine","RuntimeCgroupsName":"","SystemCgroupsName":"","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":false,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":null,"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.01},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.01},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":-1,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} machine # [ 30.411735] k3s[873]: I0919 21:20:56.500538 873 topology_manager.go:143] "Creating topology manager with none policy" machine # [ 30.412637] k3s[873]: I0919 21:20:56.500552 873 container_manager_linux.go:308] "Creating device plugin manager" machine # [ 30.413479] k3s[873]: I0919 21:20:56.500632 873 container_manager_linux.go:317] "Creating Dynamic Resource Allocation (DRA) manager" machine # [ 30.414166] k3s[873]: I0919 21:20:56.503616 873 state_mem.go:41] "Initialized" logger="CPUManager state memory" machine # [ 30.414666] k3s[873]: I0919 21:20:56.503930 873 kubelet.go:482] "Attempting to sync node with API server" machine # [ 30.414975] k3s[873]: I0919 21:20:56.503953 873 kubelet.go:383] "Adding static pod path" path="/var/lib/rancher/k3s/agent/pod-manifests" machine # [ 30.415626] k3s[873]: I0919 21:20:56.503984 873 kubelet.go:394] "Adding apiserver pod source" machine # [ 30.416145] k3s[873]: I0919 21:20:56.504003 873 apiserver.go:42] "Waiting for node sync before watching apiserver pods" machine # [ 30.417968] k3s[873]: I0919 21:20:56.508470 873 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="containerd" version="2.2.7-k3s1" apiVersion="v1" machine # [ 30.419364] k3s[873]: I0919 21:20:56.509972 873 kubelet.go:945] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" machine # [ 30.420019] k3s[873]: I0919 21:20:56.510008 873 kubelet.go:972] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" machine # [ 30.421391] k3s[873]: I0919 21:20:56.512007 873 server.go:1252] "Started kubelet" machine # [ 30.423423] k3s[873]: I0919 21:20:56.513227 873 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 machine # [ 30.424174] k3s[873]: I0919 21:20:56.513292 873 server_v1.go:49] "podresources" method="list" useActivePods=true machine # [ 30.424903] k3s[873]: I0919 21:20:56.513452 873 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" machine # [ 30.425734] k3s[873]: I0919 21:20:56.513586 873 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 machine # [ 30.426426] k3s[873]: I0919 21:20:56.514583 873 server.go:317] "Adding debug handlers to kubelet server" machine # [ 30.426864] k3s[873]: I0919 21:20:56.515225 873 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" machine # [ 30.427564] k3s[873]: I0919 21:20:56.516608 873 dynamic_serving_content.go:135] "Starting controller" name="kubelet-server-cert-files::/var/lib/rancher/k3s/agent/serving-kubelet.crt::/var/lib/rancher/k3s/agent/serving-kubelet.key" machine # [ 30.428072] k3s[873]: I0919 21:20:56.517229 873 volume_manager.go:311] "Starting Kubelet Volume Manager" machine # [ 30.428509] k3s[873]: E0919 21:20:56.517510 873 kubelet_node_status.go:392] "Error getting the current node from lister" err="node \"machine\" not found" machine # [ 30.433679] k3s[873]: I0919 21:20:56.524280 873 desired_state_of_world_populator.go:146] "Desired state populator starts to run" machine # [ 30.434338] k3s[873]: I0919 21:20:56.524353 873 reconciler.go:29] "Reconciler: start to sync state" machine # [ 30.446821] k3s[873]: I0919 21:20:56.537414 873 factory.go:223] Registration of the containerd container factory successfully machine # [ 30.447822] k3s[873]: I0919 21:20:56.537436 873 factory.go:223] Registration of the systemd container factory successfully machine # [ 30.448178] k3s[873]: I0919 21:20:56.537553 873 factory.go:221] Registration of the crio container factory failed: Get "http://%2Fvar%2Frun%2Fcrio%2Fcrio.sock/info": dial unix /var/run/crio/crio.sock: connect: no such file or directory machine # [ 30.449256] k3s[873]: E0919 21:20:56.539857 873 kubelet.go:1661] "Image garbage collection failed once. Stats initialization may not have completed yet" err="invalid capacity 0 on image filesystem" machine # [ 30.472874] k3s[873]: E0919 21:20:56.563411 873 nodelease.go:50] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"machine\" not found" node="machine" machine # [ 30.474542] k3s[873]: I0919 21:20:56.565162 873 cpu_manager.go:225] "Starting" policy="none" machine # [ 30.475705] k3s[873]: I0919 21:20:56.565183 873 cpu_manager.go:226] "Reconciling" reconcilePeriod="10s" machine # [ 30.476574] k3s[873]: I0919 21:20:56.565206 873 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" machine # [ 30.479303] k3s[873]: I0919 21:20:56.569739 873 policy_none.go:50] "Start" machine # [ 30.480039] k3s[873]: I0919 21:20:56.569763 873 memory_manager.go:187] "Starting memorymanager" policy="None" machine # [ 30.480619] k3s[873]: I0919 21:20:56.569780 873 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" machine # [ 30.484357] k3s[873]: I0919 21:20:56.574749 873 policy_none.go:44] "Start" machine # [ 30.493934] systemd[1]: Created slice libcontainer container kubepods.slice. machine # [ 30.498198] k3s[873]: time="2026-09-19T21:20:56Z" level=info msg="Event occurred" apiVersion=k3s.cattle.io/v1 fieldPath= kind=Addon logger=k3s/deploy message="Applied manifest at \"/var/lib/rancher/k3s/server/manifests/rolebindings.yaml\"" object=kube-system/rolebindings reason=AppliedManifest type=Normal machine # [ 30.501753] k3s[873]: I0919 21:20:56.592371 873 shared_informer.go:377] "Caches are synced" machine # [ 30.505108] k3s[873]: I0919 21:20:56.595720 873 shared_informer.go:377] "Caches are synced" machine # [ 30.505776] k3s[873]: I0919 21:20:56.596307 873 shared_informer.go:377] "Caches are synced" machine # [ 30.506845] k3s[873]: I0919 21:20:56.596388 873 shared_informer.go:377] "Caches are synced" machine # [ 30.507831] k3s[873]: I0919 21:20:56.597272 873 shared_informer.go:377] "Caches are synced" machine # [ 30.509926] k3s[873]: I0919 21:20:56.600553 873 shared_informer.go:377] "Caches are synced" machine # [ 30.512073] k3s[873]: I0919 21:20:56.601467 873 range_allocator.go:177] "Sending events to api server" machine # [ 30.512402] k3s[873]: I0919 21:20:56.601530 873 range_allocator.go:181] "Starting range CIDR allocator" machine # [ 30.512892] k3s[873]: I0919 21:20:56.601545 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 30.513651] k3s[873]: I0919 21:20:56.601557 873 shared_informer.go:377] "Caches are synced" machine # [ 30.514441] k3s[873]: I0919 21:20:56.601651 873 shared_informer.go:377] "Caches are synced" machine # [ 30.515064] k3s[873]: I0919 21:20:56.601780 873 shared_informer.go:377] "Caches are synced" machine # [ 30.518103] k3s[873]: I0919 21:20:56.601940 873 shared_informer.go:377] "Caches are synced" machine # [ 30.519161] k3s[873]: I0919 21:20:56.602002 873 shared_informer.go:377] "Caches are synced" machine # [ 30.521190] k3s[873]: I0919 21:20:56.602100 873 shared_informer.go:377] "Caches are synced" machine # [ 30.521908] k3s[873]: I0919 21:20:56.602199 873 shared_informer.go:377] "Caches are synced" machine # [ 30.522630] k3s[873]: I0919 21:20:56.602299 873 shared_informer.go:377] "Caches are synced" machine # [ 30.523236] k3s[873]: I0919 21:20:56.602354 873 shared_informer.go:377] "Caches are synced" machine # [ 30.523879] k3s[873]: I0919 21:20:56.602623 873 shared_informer.go:377] "Caches are synced" machine # [ 30.525115] k3s[873]: I0919 21:20:56.602696 873 shared_informer.go:377] "Caches are synced" machine # [ 30.525940] k3s[873]: I0919 21:20:56.608451 873 shared_informer.go:377] "Caches are synced" machine # [ 30.526724] k3s[873]: I0919 21:20:56.608652 873 shared_informer.go:377] "Caches are synced" machine # [ 30.527471] k3s[873]: I0919 21:20:56.609599 873 shared_informer.go:377] "Caches are synced" machine # [ 30.527901] k3s[873]: I0919 21:20:56.609713 873 shared_informer.go:377] "Caches are synced" machine # [ 30.528574] k3s[873]: I0919 21:20:56.615055 873 shared_informer.go:377] "Caches are synced" machine # [ 30.529136] k3s[873]: I0919 21:20:56.615129 873 shared_informer.go:377] "Caches are synced" machine # [ 30.529597] k3s[873]: I0919 21:20:56.615228 873 shared_informer.go:377] "Caches are synced" machine # [ 30.530171] k3s[873]: I0919 21:20:56.615723 873 shared_informer.go:377] "Caches are synced" machine # [ 30.530644] systemd[1]: Created slice libcontainer container kubepods-burstable.slice. machine # [ 30.531068] k3s[873]: I0919 21:20:56.617261 873 shared_informer.go:377] "Caches are synced" machine # [ 30.531480] k3s[873]: I0919 21:20:56.617370 873 shared_informer.go:377] "Caches are synced" machine # [ 30.531999] k3s[873]: I0919 21:20:56.617505 873 shared_informer.go:377] "Caches are synced" machine # [ 30.532505] k3s[873]: E0919 21:20:56.617561 873 kubelet_node_status.go:392] "Error getting the current node from lister" err="node \"machine\" not found" machine # [ 30.532986] k3s[873]: I0919 21:20:56.617631 873 shared_informer.go:377] "Caches are synced" machine # [ 30.533627] k3s[873]: I0919 21:20:56.618026 873 shared_informer.go:377] "Caches are synced" machine # [ 30.534137] k3s[873]: I0919 21:20:56.618129 873 shared_informer.go:377] "Caches are synced" machine # [ 30.534651] k3s[873]: I0919 21:20:56.620364 873 shared_informer.go:377] "Caches are synced" machine # [ 30.535165] k3s[873]: I0919 21:20:56.620592 873 shared_informer.go:377] "Caches are synced" machine # [ 30.535826] k3s[873]: I0919 21:20:56.620710 873 shared_informer.go:377] "Caches are synced" machine # [ 30.536459] k3s[873]: I0919 21:20:56.620840 873 shared_informer.go:377] "Caches are synced" machine # [ 30.553772] k3s[873]: I0919 21:20:56.644368 873 shared_informer.go:377] "Caches are synced" machine # [ 30.555436] systemd[1]: Created slice libcontainer container kubepods-besteffort.slice. machine # [ 30.559970] k3s[873]: I0919 21:20:56.647981 873 shared_informer.go:377] "Caches are synced" machine # [ 30.568078] k3s[873]: E0919 21:20:56.658064 873 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" machine # [ 30.568492] k3s[873]: I0919 21:20:56.658201 873 eviction_manager.go:194] "Eviction manager: starting control loop" machine # [ 30.569508] k3s[873]: I0919 21:20:56.658231 873 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" machine # [ 30.574137] k3s[873]: I0919 21:20:56.662343 873 plugin_manager.go:121] "Starting Kubelet Plugin Manager" machine # [ 30.580158] k3s[873]: E0919 21:20:56.670767 873 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="no imagefs label for configured runtime" machine # [ 30.580799] k3s[873]: E0919 21:20:56.671429 873 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"machine\" not found" machine # [ 30.641671] k3s[873]: I0919 21:20:56.732235 873 controller.go:667] quota admission added evaluator for: replicasets.apps machine # [ 30.655991] k3s[873]: time="2026-09-19T21:20:56Z" level=info msg="Event occurred" apiVersion=k3s.cattle.io/v1 fieldPath= kind=Addon logger=k3s/deploy message="Applying manifest at \"/var/lib/rancher/k3s/server/manifests/runtimes.yaml\"" object=kube-system/runtimes reason=ApplyingManifest type=Normal machine # [ 30.669153] k3s[873]: I0919 21:20:56.758726 873 cidrallocator.go:278] updated ClusterIP allocator for Service CIDR 10.43.0.0/16 machine # [ 30.677070] k3s[873]: I0919 21:20:56.767634 873 kubelet_node_status.go:74] "Attempting to register node" node="machine" machine # [ 30.718395] k3s[873]: I0919 21:20:56.808959 873 cidrallocator.go:278] updated ClusterIP allocator for Service CIDR 10.43.0.0/16 machine # [ 30.729306] k3s[873]: I0919 21:20:56.819838 873 kubelet_node_status.go:77] "Successfully registered node" node="machine" machine # [ 30.729883] k3s[873]: E0919 21:20:56.819875 873 kubelet_node_status.go:474] "Error updating node status, will retry" err="error getting node \"machine\": node \"machine\" not found" machine # [ 30.732516] k3s[873]: I0919 21:20:56.823133 873 node_controller.go:429] Initializing node machine with cloud provider machine # [ 30.732958] k3s[873]: I0919 21:20:56.823591 873 node_controller.go:233] error syncing 'machine': failed to get instance metadata for node machine: address annotations not yet set, requeuing machine # [ 30.734077] k3s[873]: E0919 21:20:56.823814 873 node_controller.go:244] "Unhandled Error" err="error syncing 'machine': failed to get instance metadata for node machine: address annotations not yet set, requeuing" logger="UnhandledError" machine # [ 30.734928] k3s[873]: I0919 21:20:56.824325 873 actual_state_of_world.go:541] "Failed to update statusUpdateNeeded field in actual state of world" logger="persistentvolume-attach-detach-controller" err="Failed to set statusUpdateNeeded to needed true, because nodeName=\"machine\" does not exist" machine # [ 30.735483] k3s[873]: time="2026-09-19T21:20:56Z" level=info msg="Adding node OwnerReference to node-password secret machine.node-password.k3s" machine # [ 30.736546] k3s[873]: time="2026-09-19T21:20:56Z" level=info msg="Event occurred" apiVersion=helm.cattle.io/v1 fieldPath= kind=HelmChart logger=k3s/helm-controller message="Resumed synced Job kube-system/helm-install-nix-grpc-farm" object=kube-system/nix-grpc-farm reason=ResumeJob type=Normal machine # [ 30.744715] k3s[873]: I0919 21:20:56.835178 873 node_controller.go:429] Initializing node machine with cloud provider machine # [ 30.751661] k3s[873]: I0919 21:20:56.835306 873 node_controller.go:233] error syncing 'machine': failed to get instance metadata for node machine: address annotations not yet set, requeuing machine # [ 30.752780] k3s[873]: E0919 21:20:56.835755 873 node_controller.go:244] "Unhandled Error" err="error syncing 'machine': failed to get instance metadata for node machine: address annotations not yet set, requeuing" logger="UnhandledError" machine # [ 30.757829] k3s[873]: I0919 21:20:56.848411 873 node_controller.go:429] Initializing node machine with cloud provider machine # [ 30.758532] k3s[873]: I0919 21:20:56.848475 873 node_controller.go:233] error syncing 'machine': failed to get instance metadata for node machine: address annotations not yet set, requeuing machine # [ 30.759067] k3s[873]: E0919 21:20:56.848497 873 node_controller.go:244] "Unhandled Error" err="error syncing 'machine': failed to get instance metadata for node machine: address annotations not yet set, requeuing" logger="UnhandledError" machine # [ 30.768518] k3s[873]: I0919 21:20:56.858861 873 node_controller.go:429] Initializing node machine with cloud provider machine # [ 30.771464] k3s[873]: I0919 21:20:56.862057 873 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" machine # [ 30.779242] k3s[873]: time="2026-09-19T21:20:56Z" level=info msg="Annotations and labels have been set successfully on node: machine" machine # [ 30.783431] k3s[873]: time="2026-09-19T21:20:56Z" level=info msg="Starting flannel with backend vxlan" machine # [ 30.791238] k3s[873]: I0919 21:20:56.881763 873 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" machine # [ 30.791564] k3s[873]: I0919 21:20:56.881807 873 status_manager.go:249] "Starting to sync pod status with apiserver" machine # [ 30.795109] k3s[873]: I0919 21:20:56.881847 873 kubelet.go:2506] "Starting kubelet main sync loop" machine # [ 30.795465] k3s[873]: E0919 21:20:56.881916 873 kubelet.go:2530] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" machine # [ 30.795982] k3s[873]: I0919 21:20:56.886516 873 range_allocator.go:433] "Set node PodCIDR" node="machine" podCIDRs=["10.42.0.0/24"] machine # [ 30.806250] k3s[873]: I0919 21:20:56.896855 873 kuberuntime_manager.go:2095] "Updating runtime config through cri with podcidr" CIDR="10.42.0.0/24" machine # [ 30.807730] k3s[873]: I0919 21:20:56.898139 873 shared_informer.go:377] "Caches are synced" machine # [ 30.809522] k3s[873]: I0919 21:20:56.898302 873 garbagecollector.go:166] "Garbage collector: all resource monitors have synced" machine # [ 30.811344] k3s[873]: I0919 21:20:56.898312 873 garbagecollector.go:169] "Proceeding to collect garbage" machine # [ 30.812966] k3s[873]: time="2026-09-19T21:20:56Z" level=info msg="Synced coredns NodeHosts entries for machine" machine # [ 30.817331] k3s[873]: I0919 21:20:56.907926 873 kubelet_network.go:47] "Updating Pod CIDR" originalPodCIDR="" newPodCIDR="10.42.0.0/24" machine # [ 30.858082] k3s[873]: time="2026-09-19T21:20:56Z" level=info msg="Flannel found PodCIDR assigned for node machine" machine # [ 30.861661] k3s[873]: I0919 21:20:56.952278 873 shared_informer.go:377] "Caches are synced" machine # [ 30.863736] k3s[873]: time="2026-09-19T21:20:56Z" level=info msg="The interface eth0 with ipv4 address 10.0.2.15 will be used by flannel" machine # [ 30.864422] k3s[873]: I0919 21:20:56.954989 873 kube.go:139] Waiting 10m0s for node controller to sync machine # [ 30.865079] k3s[873]: I0919 21:20:56.955095 873 kube.go:537] Starting kube subnet manager machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-lb" not found machine # [ 30.901563] k3s[873]: I0919 21:20:56.991995 873 node_controller.go:474] Successfully initialized node machine with cloud provider machine # [ 30.901860] k3s[873]: I0919 21:20:56.992366 873 event.go:389] "Event occurred" object="machine" fieldPath="" kind="Node" apiVersion="v1" type="Normal" reason="Synced" message="Node synced successfully" machine # [ 30.909248] k3s[873]: time="2026-09-19T21:20:56Z" level=info msg="Event occurred" apiVersion=k3s.cattle.io/v1 fieldPath= kind=Addon logger=k3s/deploy message="Applied manifest at \"/var/lib/rancher/k3s/server/manifests/runtimes.yaml\"" object=kube-system/runtimes reason=AppliedManifest type=Normal machine # [ 30.917165] k3s[873]: I0919 21:20:57.007734 873 server.go:173] "Starting Kubernetes Scheduler" version="v1.35.8+k3s1" machine # [ 30.917587] k3s[873]: I0919 21:20:57.007772 873 server.go:175] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" machine # [ 30.927357] k3s[873]: I0919 21:20:57.017875 873 secure_serving.go:211] Serving securely on 127.0.0.1:10259 machine # [ 30.927501] k3s[873]: I0919 21:20:57.017917 873 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" machine # [ 30.928205] k3s[873]: I0919 21:20:57.017986 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 30.928851] k3s[873]: I0919 21:20:57.018000 873 tlsconfig.go:243] "Starting DynamicServingCertificateController" machine # [ 30.929617] k3s[873]: I0919 21:20:57.017878 873 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController machine # [ 30.933096] k3s[873]: I0919 21:20:57.018069 873 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" machine # [ 30.933921] k3s[873]: I0919 21:20:57.018081 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 30.934762] k3s[873]: I0919 21:20:57.017979 873 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/var/lib/rancher/k3s/server/tls/kube-scheduler/kube-scheduler.crt::/var/lib/rancher/k3s/server/tls/kube-scheduler/kube-scheduler.key" machine # [ 30.935601] k3s[873]: I0919 21:20:57.018086 873 shared_informer.go:370] "Waiting for caches to sync" machine # [ 30.937479] k3s[873]: I0919 21:20:57.028090 873 kubelet_node_status.go:427] "Fast updating node status as it just became ready" machine # [ 30.949647] k3s[873]: time="2026-09-19T21:20:57Z" level=info msg="Event occurred" apiVersion=k3s.cattle.io/v1 fieldPath= kind=Addon logger=k3s/deploy message="Deleting manifest at \"/var/lib/rancher/k3s/server/manifests/traefik.yaml\"" object=kube-system/traefik reason=DeletingManifest type=Normal machine # [ 30.951492] k3s[873]: I0919 21:20:57.042049 873 topologycache.go:237] "Can't get CPU or zone information for node" node="machine" machine # [ 31.028559] k3s[873]: I0919 21:20:57.119065 873 shared_informer.go:377] "Caches are synced" machine # [ 31.028974] k3s[873]: I0919 21:20:57.119097 873 shared_informer.go:377] "Caches are synced" machine # [ 31.032554] k3s[873]: I0919 21:20:57.123135 873 shared_informer.go:377] "Caches are synced" machine # [ 31.219983] k3s[873]: time="2026-09-19T21:20:57Z" level=info msg="Labels and annotations have been set successfully on node: machine" machine # [ 31.313321] k3s[873]: time="2026-09-19T21:20:57Z" level=info msg="Tunnel authorizer set Kubelet Port 0.0.0.0:10250" machine # [ 31.414087] k3s[873]: I0919 21:20:57.504642 873 apiserver.go:52] "Watching apiserver" machine # [ 31.532276] systemd[1]: Created slice libcontainer container kubepods-burstable-pod083c1ce1_596a_4328_ac2d_08dec14e5dea.slice. machine # [ 31.535477] k3s[873]: I0919 21:20:57.625594 873 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" machine # [ 31.547500] k3s[873]: I0919 21:20:57.638089 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fn9w5\" (UniqueName: \"kubernetes.io/projected/083c1ce1-596a-4328-ac2d-08dec14e5dea-kube-api-access-fn9w5\") pod \"coredns-c5fdd76cf-rjgdg\" (UID: \"083c1ce1-596a-4328-ac2d-08dec14e5dea\") " pod="kube-system/coredns-c5fdd76cf-rjgdg" machine # [ 31.554390] k3s[873]: I0919 21:20:57.638142 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klipper-cache\" (UniqueName: \"kubernetes.io/empty-dir/9583a273-8094-4a85-92fd-83dea302013f-klipper-cache\") pod \"helm-install-nix-grpc-farm-c8xfg\" (UID: \"9583a273-8094-4a85-92fd-83dea302013f\") " pod="kube-system/helm-install-nix-grpc-farm-c8xfg" machine # [ 31.561760] k3s[873]: I0919 21:20:57.638175 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9583a273-8094-4a85-92fd-83dea302013f-tmp\") pod \"helm-install-nix-grpc-farm-c8xfg\" (UID: \"9583a273-8094-4a85-92fd-83dea302013f\") " pod="kube-system/helm-install-nix-grpc-farm-c8xfg" machine # [ 31.568088] k3s[873]: I0919 21:20:57.638219 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"values\" (UniqueName: \"kubernetes.io/projected/9583a273-8094-4a85-92fd-83dea302013f-values\") pod \"helm-install-nix-grpc-farm-c8xfg\" (UID: \"9583a273-8094-4a85-92fd-83dea302013f\") " pod="kube-system/helm-install-nix-grpc-farm-c8xfg" machine # [ 31.574853] k3s[873]: I0919 21:20:57.638326 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/083c1ce1-596a-4328-ac2d-08dec14e5dea-config-volume\") pod \"coredns-c5fdd76cf-rjgdg\" (UID: \"083c1ce1-596a-4328-ac2d-08dec14e5dea\") " pod="kube-system/coredns-c5fdd76cf-rjgdg" machine # [ 31.581556] k3s[873]: I0919 21:20:57.638396 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"custom-config-volume\" (UniqueName: \"kubernetes.io/configmap/083c1ce1-596a-4328-ac2d-08dec14e5dea-custom-config-volume\") pod \"coredns-c5fdd76cf-rjgdg\" (UID: \"083c1ce1-596a-4328-ac2d-08dec14e5dea\") " pod="kube-system/coredns-c5fdd76cf-rjgdg" machine # [ 31.588116] k3s[873]: I0919 21:20:57.638423 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klipper-helm\" (UniqueName: \"kubernetes.io/empty-dir/9583a273-8094-4a85-92fd-83dea302013f-klipper-helm\") pod \"helm-install-nix-grpc-farm-c8xfg\" (UID: \"9583a273-8094-4a85-92fd-83dea302013f\") " pod="kube-system/helm-install-nix-grpc-farm-c8xfg" machine # [ 31.593666] k3s[873]: I0919 21:20:57.638484 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klipper-config\" (UniqueName: \"kubernetes.io/empty-dir/9583a273-8094-4a85-92fd-83dea302013f-klipper-config\") pod \"helm-install-nix-grpc-farm-c8xfg\" (UID: \"9583a273-8094-4a85-92fd-83dea302013f\") " pod="kube-system/helm-install-nix-grpc-farm-c8xfg" machine # [ 31.599477] k3s[873]: I0919 21:20:57.638509 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"content\" (UniqueName: \"kubernetes.io/configmap/9583a273-8094-4a85-92fd-83dea302013f-content\") pod \"helm-install-nix-grpc-farm-c8xfg\" (UID: \"9583a273-8094-4a85-92fd-83dea302013f\") " pod="kube-system/helm-install-nix-grpc-farm-c8xfg" machine # [ 31.605511] k3s[873]: I0919 21:20:57.638542 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phf8p\" (UniqueName: \"kubernetes.io/projected/9583a273-8094-4a85-92fd-83dea302013f-kube-api-access-phf8p\") pod \"helm-install-nix-grpc-farm-c8xfg\" (UID: \"9583a273-8094-4a85-92fd-83dea302013f\") " pod="kube-system/helm-install-nix-grpc-farm-c8xfg" machine # [ 31.610741] systemd[1]: Created slice libcontainer container kubepods-burstable-pod9583a273_8094_4a85_92fd_83dea302013f.slice. machine # [ 31.611214] postgres[1211]: [1211] ERROR: relation "goose_db_version" does not exist at character 36 machine # [ 31.611662] postgres[1211]: [1211] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC machine # [ 31.637272] niks3-server[1202]: 2026/09/19 21:20:57 OK 20241026095416_initial_model.sql (10.97ms) machine # [ 31.640724] niks3-server[1202]: 2026/09/19 21:20:57 OK 20251210153512_drop_unused_gin_index.sql (3.54ms) machine # [ 31.644314] niks3-server[1202]: 2026/09/19 21:20:57 OK 20251218171726_add_pins.sql (3.52ms) machine # [ 31.650131] niks3-server[1202]: 2026/09/19 21:20:57 OK 20260628120000_add_object_size_and_stats.sql (5.64ms) machine # [ 31.654427] niks3-server[1202]: 2026/09/19 21:20:57 OK 20260905000000_add_claims.sql (4.37ms) machine # [ 31.654783] niks3-server[1202]: 2026/09/19 21:20:57 goose: successfully migrated database to version: 20260905000000 machine # [ 31.663572] niks3-server[1202]: 2026/09/19 21:20:57 OK 1_commit_pending_closure.sql (8.89ms) machine # [ 31.666779] niks3-server[1202]: 2026/09/19 21:20:57 OK 2_object_stats_trigger.sql (3.29ms) machine # [ 31.667090] niks3-server[1202]: 2026/09/19 21:20:57 goose: up to current file version: 2 machine # [ 31.688966] niks3-server[1202]: 2026/09/19 21:20:57 INFO OIDC provider initialized name=kubernetes issuer=https://kubernetes.default.svc.cluster.local machine # [ 31.691497] niks3-server[1202]: 2026/09/19 21:20:57 INFO OIDC authentication enabled config=/nix/store/2xcn0lgmca14rg9xzcfqj65j9hs85hn6-niks3-oidc.json machine # [ 31.692077] niks3-server[1202]: 2026/09/19 21:20:57 INFO Loaded signing key name=farm-test-1 path=/nix/store/c745bsy529madd2s6hz4z5is2436ni04-key machine # [ 31.722190] k3s[873]: I0919 21:20:57.812283 873 server.go:218] "Successfully retrieved NodeIPs" NodeIPs=["10.0.2.15","fec0::5fe0:fdce:6860:f263"] machine # [ 31.722624] k3s[873]: E0919 21:20:57.812493 873 server.go:255] "Kube-proxy configuration may be incomplete or incorrect" err="nodePortAddresses is unset; NodePort connections will be accepted on all local IPs. Consider using `--nodeport-addresses primary`" machine # [ 31.752704] k3s[873]: I0919 21:20:57.843246 873 server.go:264] "kube-proxy running in dual-stack mode" primary ipFamily="IPv4" machine # [ 31.753477] k3s[873]: I0919 21:20:57.843302 873 server_linux.go:136] "Using iptables Proxier" machine # [ 31.766981] niks3-server[1202]: 2026/09/19 21:20:57 INFO Read proxy enabled — serving cache objects from S3 machine # [ 31.769473] niks3-server[1202]: 2026/09/19 21:20:57 INFO Using socket-activated listener address=0.0.0.0:5751 machine # [ 31.771938] niks3-server[1202]: 2026/09/19 21:20:57 INFO systemd watchdog enabled interval=15s machine # [ 31.772349] systemd[1]: Started niks3 server. machine # [ 31.773115] systemd[1]: Reached target Multi-User System. machine # [ 31.773629] systemd[1]: Startup finished in 1.152s (kernel) + 3.974s (initrd) + 26.646s (userspace) = 31.773s. machine # [ 31.775459] niks3-server[1202]: 2026/09/19 21:20:57 INFO Starting HTTP server address=0.0.0.0:5751 machine # [ 31.792296] k3s[873]: I0919 21:20:57.882832 873 proxier.go:242] "Setting route_localnet=1 to allow node-ports on localhost; to change this either disable iptables.localhostNodePorts (--iptables-localhost-nodeports) or set nodePortAddresses (--nodeport-addresses) to filter loopback addresses" ipFamily="IPv4" machine # [ 31.811056] k3s[873]: I0919 21:20:57.901623 873 server.go:529] "Version info" version="v1.35.8+k3s1" machine # [ 31.811543] k3s[873]: I0919 21:20:57.901654 873 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" machine # [ 31.816151] k3s[873]: I0919 21:20:57.906736 873 config.go:200] "Starting service config controller" machine # [ 31.816642] k3s[873]: I0919 21:20:57.906765 873 shared_informer.go:349] "Waiting for caches to sync" controller="service config" machine # [ 31.817140] k3s[873]: I0919 21:20:57.906787 873 config.go:106] "Starting endpoint slice config controller" machine # [ 31.817624] k3s[873]: I0919 21:20:57.906805 873 shared_informer.go:349] "Waiting for caches to sync" controller="endpoint slice config" machine # [ 31.818089] k3s[873]: I0919 21:20:57.906840 873 config.go:403] "Starting serviceCIDR config controller" machine # [ 31.818534] k3s[873]: I0919 21:20:57.906853 873 shared_informer.go:349] "Waiting for caches to sync" controller="serviceCIDR config" machine # [ 31.819100] k3s[873]: I0919 21:20:57.907010 873 config.go:309] "Starting node config controller" machine # [ 31.819542] k3s[873]: I0919 21:20:57.907029 873 shared_informer.go:349] "Waiting for caches to sync" controller="node config" machine # [ 31.865525] k3s[873]: I0919 21:20:57.956098 873 kube.go:163] Node controller sync successful machine # [ 31.865770] k3s[873]: I0919 21:20:57.956170 873 vxlan.go:128] VXLAN config: VNI=1 Port=0 GBP=false Learning=false DirectRouting=false machine # [ 31.868821] k3s[873]: I0919 21:20:57.959424 873 kube.go:704] List of node(machine) annotations: map[string]string{"alpha.kubernetes.io/provided-node-ip":"10.0.2.15,fec0::5fe0:fdce:6860:f263", "k3s.io/hostname":"machine", "k3s.io/internal-ip":"10.0.2.15,fec0::5fe0:fdce:6860:f263", "k3s.io/node-args":"[\"server\",\"--disable\",\"traefik\",\"--disable\",\"metrics-server\",\"--disable\",\"local-storage\",\"--kubelet-arg\",\"eviction-hard=nodefs.available\\u003c1%,imagefs.available\\u003c1%\"]", "k3s.io/node-config-hash":"EFXPOZ7ZWHRCLFNW4X4ZDPFQMSAOP3BLEIN2VE47TDRKQ4UIWKUQ====", "k3s.io/node-env":"{}", "node.alpha.kubernetes.io/ttl":"0", "volumes.kubernetes.io/controller-managed-attach-detach":"true"} machine # [ 31.907993] (udev-worker)[1210]: Network interface NamePolicy= disabled on kernel command line. machine # [ 31.917315] k3s[873]: I0919 21:20:58.007916 873 shared_informer.go:356] "Caches are synced" controller="node config" machine # [ 31.919759] k3s[873]: I0919 21:20:58.010368 873 iptables.go:50] Starting flannel in iptables mode... machine # [ 31.920204] k3s[873]: time="2026-09-19T21:20:58Z" level=warning msg="no subnet found for key: FLANNEL_NETWORK in file: /run/flannel/subnet.env" machine # [ 31.921694] k3s[873]: time="2026-09-19T21:20:58Z" level=warning msg="no subnet found for key: FLANNEL_SUBNET in file: /run/flannel/subnet.env" machine # [ 31.922910] k3s[873]: time="2026-09-19T21:20:58Z" level=warning msg="no subnet found for key: FLANNEL_IPV6_NETWORK in file: /run/flannel/subnet.env" machine # [ 31.923467] k3s[873]: time="2026-09-19T21:20:58Z" level=warning msg="no subnet found for key: FLANNEL_IPV6_SUBNET in file: /run/flannel/subnet.env" machine # [ 31.926880] k3s[873]: I0919 21:20:58.010477 873 iptables.go:101] Current network or subnet (10.42.0.0/16, 10.42.0.0/24) is not equal to previous one (0.0.0.0/0, 0.0.0.0/0), trying to recycle old iptables rules machine # [ 31.927887] k3s[873]: I0919 21:20:58.013647 873 kube.go:558] Creating the node lease for IPv4. This is the n.Spec.PodCIDRs: [10.42.0.0/24] machine # [ 31.958527] dhcpcd[673]: flannel.1: IAID 2e:3b:95:3e machine # [ 31.959114] dhcpcd[673]: flannel.1: adding address fe80::8bb:2eff:fe3b:953e machine # [ 32.016703] k3s[873]: I0919 21:20:58.106934 873 shared_informer.go:356] "Caches are synced" controller="serviceCIDR config" machine # [ 32.016905] k3s[873]: I0919 21:20:58.106988 873 shared_informer.go:356] "Caches are synced" controller="endpoint slice config" machine # [ 32.017667] k3s[873]: I0919 21:20:58.107123 873 shared_informer.go:356] "Caches are synced" controller="service config" machine # [ 32.029671] k3s[873]: E0919 21:20:58.120247 873 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"1403be1b9b1119aedb5707cced0a9e60ddf4f7d77d91730265ca68e7f3ae282e\": plugin type=\"flannel\" failed (add): loadFlannelSubnetEnv failed: open /run/flannel/subnet.env: no such file or directory" machine # [ 32.030089] k3s[873]: E0919 21:20:58.120357 873 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"1403be1b9b1119aedb5707cced0a9e60ddf4f7d77d91730265ca68e7f3ae282e\": plugin type=\"flannel\" failed (add): loadFlannelSubnetEnv failed: open /run/flannel/subnet.env: no such file or directory" pod="kube-system/coredns-c5fdd76cf-rjgdg" machine # [ 32.030281] k3s[873]: E0919 21:20:58.120390 873 kuberuntime_manager.go:1568] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"1403be1b9b1119aedb5707cced0a9e60ddf4f7d77d91730265ca68e7f3ae282e\": plugin type=\"flannel\" failed (add): loadFlannelSubnetEnv failed: open /run/flannel/subnet.env: no such file or directory" pod="kube-system/coredns-c5fdd76cf-rjgdg" machine # [ 32.031025] k3s[873]: E0919 21:20:58.120464 873 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"coredns-c5fdd76cf-rjgdg_kube-system(083c1ce1-596a-4328-ac2d-08dec14e5dea)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"coredns-c5fdd76cf-rjgdg_kube-system(083c1ce1-596a-4328-ac2d-08dec14e5dea)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"1403be1b9b1119aedb5707cced0a9e60ddf4f7d77d91730265ca68e7f3ae282e\\\": plugin type=\\\"flannel\\\" failed (add): loadFlannelSubnetEnv failed: open /run/flannel/subnet.env: no such file or directory\"" pod="kube-system/coredns-c5fdd76cf-rjgdg" podUID="083c1ce1-596a-4328-ac2d-08dec14e5dea" machine # [ 32.042539] k3s[873]: E0919 21:20:58.133017 873 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"0b9925086407381ed3656f9fc34c52dba3b1045c6270fec7629076b26a641258\": plugin type=\"flannel\" failed (add): loadFlannelSubnetEnv failed: open /run/flannel/subnet.env: no such file or directory" machine # [ 32.043722] k3s[873]: E0919 21:20:58.133087 873 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"0b9925086407381ed3656f9fc34c52dba3b1045c6270fec7629076b26a641258\": plugin type=\"flannel\" failed (add): loadFlannelSubnetEnv failed: open /run/flannel/subnet.env: no such file or directory" pod="kube-system/helm-install-nix-grpc-farm-c8xfg" machine # [ 32.044194] k3s[873]: E0919 21:20:58.133113 873 kuberuntime_manager.go:1568] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"0b9925086407381ed3656f9fc34c52dba3b1045c6270fec7629076b26a641258\": plugin type=\"flannel\" failed (add): loadFlannelSubnetEnv failed: open /run/flannel/subnet.env: no such file or directory" pod="kube-system/helm-install-nix-grpc-farm-c8xfg" machine # [ 32.044837] k3s[873]: E0919 21:20:58.133186 873 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"helm-install-nix-grpc-farm-c8xfg_kube-system(9583a273-8094-4a85-92fd-83dea302013f)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"helm-install-nix-grpc-farm-c8xfg_kube-system(9583a273-8094-4a85-92fd-83dea302013f)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"0b9925086407381ed3656f9fc34c52dba3b1045c6270fec7629076b26a641258\\\": plugin type=\\\"flannel\\\" failed (add): loadFlannelSubnetEnv failed: open /run/flannel/subnet.env: no such file or directory\"" pod="kube-system/helm-install-nix-grpc-farm-c8xfg" podUID="9583a273-8094-4a85-92fd-83dea302013f" machine # [ 32.055351] k3s[873]: I0919 21:20:58.145948 873 iptables.go:111] Setting up masking rules machine # [ 32.096531] k3s[873]: I0919 21:20:58.187083 873 iptables.go:212] Changing default FORWARD chain policy to ACCEPT machine # [ 32.117729] k3s[873]: time="2026-09-19T21:20:58Z" level=info msg="Wrote flannel subnet file to /run/flannel/subnet.env" machine # [ 32.118951] k3s[873]: time="2026-09-19T21:20:58Z" level=info msg="Running flannel backend" machine # [ 32.119339] k3s[873]: I0919 21:20:58.207750 873 vxlan_network.go:68] watching for new subnet leases machine # [ 32.119518] k3s[873]: I0919 21:20:58.207775 873 vxlan_network.go:115] starting vxlan device watcher machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-lb" not found machine # [ 32.188392] k3s[873]: I0919 21:20:58.278960 873 iptables.go:358] bootstrap done machine # [ 32.262373] k3s[873]: I0919 21:20:58.352927 873 iptables.go:358] bootstrap done machine # [ 32.663209] systemd[1]: run-netns-cni\x2d0b62db1b\x2d4917\x2d24a3\x2df42f\x2d639cebce31ab.mount: Deactivated successfully. machine # [ 32.663502] systemd[1]: run-netns-cni\x2d5068a586\x2d45a0\x2d9ef0\x2dc4bb\x2dd04ac2413054.mount: Deactivated successfully. machine # [ 32.845900] dhcpcd[673]: flannel.1: soliciting a DHCP lease machine # [ 33.042967] k3s[873]: time="2026-09-19T21:20:59Z" level=info msg="Starting network policy controller version v2.6.3-k3s1, built on 1970-01-01T01:01:01Z, go1.26.7" machine # [ 33.043222] k3s[873]: I0919 21:20:59.133149 873 network_policy_controller.go:164] Starting network policy controller machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-lb" not found machine # [ 33.340938] k3s[873]: I0919 21:20:59.431429 873 network_policy_controller.go:179] Starting network policy controller full sync goroutine machine # [ 33.420784] dhcpcd[673]: flannel.1: soliciting an IPv6 router machine # [ 34.495437] k3s[873]: time="2026-09-19T21:21:00Z" level=info msg="Started tunnel to 10.0.2.15:6443" machine # [ 34.495934] k3s[873]: time="2026-09-19T21:21:00Z" level=info msg="Stopped tunnel to 127.0.0.1:6443" machine # [ 34.496513] k3s[873]: time="2026-09-19T21:21:00Z" level=info msg="Connecting to proxy" url="wss://10.0.2.15:6443/v1-k3s/connect" machine # [ 34.497027] k3s[873]: time="2026-09-19T21:21:00Z" level=info msg="Proxy done" err="context canceled" url="wss://127.0.0.1:6443/v1-k3s/connect" machine # [ 34.498339] k3s[873]: time="2026-09-19T21:21:00Z" level=info msg="Handling backend connection request [machine]" machine # [ 34.501247] k3s[873]: time="2026-09-19T21:21:00Z" level=info msg="Connected to proxy" url="wss://10.0.2.15:6443/v1-k3s/connect" machine # [ 34.502137] k3s[873]: time="2026-09-19T21:21:00Z" level=info msg="Remotedialer connected to proxy" url="wss://10.0.2.15:6443/v1-k3s/connect" machine # [ 34.505369] k3s[873]: time="2026-09-19T21:21:00Z" level=info msg="error in remotedialer server [400]: websocket: close 1006 (abnormal closure): unexpected EOF" machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-lb" not found machine # [ 35.512478] k3s[873]: I0919 21:21:01.602578 873 node_lifecycle_controller.go:1234] "Initializing eviction metric for zone" zone="" machine # [ 35.514609] k3s[873]: I0919 21:21:01.602761 873 node_lifecycle_controller.go:886] "Missing timestamp for Node. Assuming now as a timestamp" node="machine" machine # [ 35.516936] k3s[873]: I0919 21:21:01.602910 873 node_lifecycle_controller.go:1080] "Controller detected that zone is now in new state" zone="" newState="Normal" machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-lb" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-lb" not found machine # [ 37.742356] k3s[873]: time="2026-09-19T21:21:03Z" level=info msg="Imported docker.io/library/nix-grpc-farm:0.1.0" machine # [ 37.744874] k3s[873]: time="2026-09-19T21:21:03Z" level=info msg="Tagged docker.io/library/nix-grpc-farm@sha256:9a533f04743023ae7556413ade537bb1cdd529e9406b9f64f04541b0cc864db4" machine # [ 37.847115] dhcpcd[673]: flannel.1: probing for an IPv4LL address machine # [ 38.028194] k3s[873]: time="2026-09-19T21:21:04Z" level=info msg="Imported 1 images from /var/lib/rancher/k3s/agent/images/nix-grpc-farm.tar in 7.824995076s" machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-lb" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-lb" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-lb" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-lb" not found machine # [ 42.641926] dhcpcd[673]: flannel.1: using IPv4LL address 169.254.196.202 machine # [ 42.645931] dhcpcd[673]: flannel.1: adding route to 169.254.0.0/16 machine # [ 42.826252] (udev-worker)[1674]: Network interface NamePolicy= disabled on kernel command line. machine # [ 42.852911] (udev-worker)[1722]: Network interface NamePolicy= disabled on kernel command line. machine # [ 43.002528] cni0: port 1(vethdded1014) entered blocking state machine # [ 43.003251] cni0: port 1(vethdded1014) entered disabled state machine # [ 43.004914] vethdded1014: entered allmulticast mode machine # [ 43.006137] vethdded1014: entered promiscuous mode machine # [ 43.023726] cni0: port 1(vethdded1014) entered blocking state machine # [ 43.024530] cni0: port 1(vethdded1014) entered forwarding state machine # [ 42.918587] dhcpcd[673]: vethdded1014: waiting for carrier machine # [ 42.919168] dhcpcd[673]: vethdded1014: carrier acquired machine # [ 42.926941] dhcpcd[673]: vethdded1014: IAID c4:bd:7e:14 machine # [ 42.927663] dhcpcd[673]: vethdded1014: adding address fe80::d446:c4ff:febd:7e14 machine # [ 42.938684] systemd[1]: var-lib-rancher-k3s-agent-containerd-tmpmounts-containerd\x2dmount412317463.mount: Deactivated successfully. machine # [ 43.035507] systemd[1]: Started libcontainer container a92e45ce458e984c927d4092d116b712da4f299ad24d066fb7eb19f48b870187. machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-lb" not found machine # [ 43.462211] dhcpcd[673]: vethdded1014: soliciting a DHCP lease machine # [ 43.832642] systemd[1]: var-lib-rancher-k3s-agent-containerd-tmpmounts-containerd\x2dmount482372361.mount: Deactivated successfully. machine # [ 43.880598] systemd[1]: Started libcontainer container c5895602f8750aa441c8575d02afb2a07893b786386790206c46258a187cfe53. machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-lb" not found machine # [ 44.855409] k3s[873]: I0919 21:21:10.945180 873 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/coredns-c5fdd76cf-rjgdg" podStartSLOduration=14.945153779 podStartE2EDuration="14.945153779s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 21:20:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 21:21:10.943651912 +0000 UTC m=+26.527496649" watchObservedRunningTime="2026-09-19 21:21:10.945153779 +0000 UTC m=+26.528998516" machine # [ 45.004046] dhcpcd[673]: vethdded1014: soliciting an IPv6 router machine # [ 45.423466] dhcpcd[673]: flannel.1: no IPv6 Routers available machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-lb" not found machine # [ 46.166809] cni0: port 2(veth51f330eb) entered blocking state machine # [ 46.168407] cni0: port 2(veth51f330eb) entered disabled state machine # [ 46.169995] veth51f330eb: entered allmulticast mode machine # [ 46.171759] veth51f330eb: entered promiscuous mode machine # [ 46.026897] (udev-worker)[1893]: Network interface NamePolicy= disabled on kernel command line. machine # [ 46.187992] cni0: port 2(veth51f330eb) entered blocking state machine # [ 46.190100] cni0: port 2(veth51f330eb) entered forwarding state machine # [ 46.078040] dhcpcd[673]: veth51f330eb: IAID ca:71:73:bd machine # [ 46.078310] dhcpcd[673]: veth51f330eb: adding address fe80::b4aa:caff:fe71:73bd machine # [ 46.160614] systemd[1]: Started libcontainer container 3db3a25b6932a0d09916b583597cc1cd0f2a7004f9fde1d1f643a404d293b0f6. machine # [ 46.370359] systemd[1]: var-lib-rancher-k3s-agent-containerd-tmpmounts-containerd\x2dmount1328181328.mount: Deactivated successfully. machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-lb" not found machine # [ 47.176658] systemd[1]: Started libcontainer container 3dd039beaa1a5bf9e46c743205ecae542e6c265cdfe0195f3df439a007e79a20. machine # [ 47.511180] dhcpcd[673]: veth51f330eb: soliciting a DHCP lease machine # [ 47.874595] k3s[873]: E0919 21:21:13.964611 873 status.go:71] "Unhandled Error" err="apiserver received an error that is not an metav1.Status: &errors.errorString{s:\"failed to create typed patch object (farm/nix-grpc-farm-scheduler; /v1, Kind=Service): .spec.selector.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0}\"}: failed to create typed patch object (farm/nix-grpc-farm-scheduler; /v1, Kind=Service): .spec.selector.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0}" logger="UnhandledError" machine # [ 47.891793] k3s[873]: I0919 21:21:13.982351 873 alloc.go:329] "allocated clusterIPs" service="farm/nix-grpc-farm" clusterIPs={"IPv4":"10.43.241.207"} machine # [ 47.901499] k3s[873]: E0919 21:21:13.992071 873 status.go:71] "Unhandled Error" err=< machine # [ 47.901873] k3s[873]: apiserver received an error that is not an metav1.Status: &errors.errorString{s:"failed to create typed patch object (farm/nix-grpc-farm-scheduler; apps/v1, Kind=Deployment): errors:\n .spec.template.metadata.labels.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0}\n .spec.selector.matchLabels.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0}"}: failed to create typed patch object (farm/nix-grpc-farm-scheduler; apps/v1, Kind=Deployment): errors: machine # [ 47.903548] k3s[873]: .spec.template.metadata.labels.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0} machine # [ 47.904991] k3s[873]: .spec.selector.matchLabels.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0} machine # [ 47.906291] k3s[873]: > logger="UnhandledError" machine # [ 47.997800] systemd[1]: cri-containerd-3dd039beaa1a5bf9e46c743205ecae542e6c265cdfe0195f3df439a007e79a20.scope: Deactivated successfully. machine # [ 47.998448] systemd[1]: cri-containerd-3dd039beaa1a5bf9e46c743205ecae542e6c265cdfe0195f3df439a007e79a20.scope: Consumed 632ms CPU time over 820ms wall clock time, 30.6M memory peak, 2.4M incoming IP traffic, 125.1K outgoing IP traffic. machine # [ 48.075188] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-3dd039beaa1a5bf9e46c743205ecae542e6c265cdfe0195f3df439a007e79a20-rootfs.mount: Deactivated successfully. machine # [ 48.344126] dhcpcd[673]: veth51f330eb: soliciting an IPv6 router machine # [ 48.462641] dhcpcd[673]: vethdded1014: probing for an IPv4LL address machine # [ 48.516147] k3s[873]: I0919 21:21:14.606479 873 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/helm-install-nix-grpc-farm-c8xfg" podStartSLOduration=18.606436986 podStartE2EDuration="18.606436986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 21:20:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 21:21:13.949537576 +0000 UTC m=+29.533382313" watchObservedRunningTime="2026-09-19 21:21:14.606436986 +0000 UTC m=+30.190287031" machine # [ 48.664559] systemd[1]: Created slice libcontainer container kubepods-besteffort-pod60ff9873_7c18_45d4_82a7_9263602acdbb.slice. machine # [ 48.700481] systemd[1]: Created slice libcontainer container kubepods-besteffort-pod6dc8f60f_caca_4d20_b10f_d00881a93074.slice. machine # [ 48.724956] systemd[1]: Created slice libcontainer container kubepods-besteffort-pod25f2f7ac_a091_4b45_8d23_efa63777d014.slice. machine # [ 48.766294] k3s[873]: I0919 21:21:14.856835 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/60ff9873-7c18-45d4-82a7-9263602acdbb-ca\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-qlcmg\" (UID: \"60ff9873-7c18-45d4-82a7-9263602acdbb\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-qlcmg" machine # [ 48.766530] k3s[873]: I0919 21:21:14.856935 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc\" (UniqueName: \"kubernetes.io/secret/60ff9873-7c18-45d4-82a7-9263602acdbb-oidc\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-qlcmg\" (UID: \"60ff9873-7c18-45d4-82a7-9263602acdbb\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-qlcmg" machine # [ 48.798417] k3s[873]: I0919 21:21:14.888945 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6dc8f60f-caca-4d20-b10f-d00881a93074-config\") pod \"nix-grpc-farm-lb-688d8d547b-r4w5t\" (UID: \"6dc8f60f-caca-4d20-b10f-d00881a93074\") " pod="farm/nix-grpc-farm-lb-688d8d547b-r4w5t" machine # [ 48.798612] k3s[873]: I0919 21:21:14.889104 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"niks3-token\" (UniqueName: \"kubernetes.io/projected/25f2f7ac-a091-4b45-8d23-efa63777d014-niks3-token\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-lxl7f\" (UID: \"25f2f7ac-a091-4b45-8d23-efa63777d014\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-lxl7f" machine # [ 48.821702] k3s[873]: I0919 21:21:14.912282 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"niks3-token\" (UniqueName: \"kubernetes.io/projected/60ff9873-7c18-45d4-82a7-9263602acdbb-niks3-token\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-qlcmg\" (UID: \"60ff9873-7c18-45d4-82a7-9263602acdbb\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-qlcmg" machine # [ 48.821906] k3s[873]: I0919 21:21:14.912364 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8mgc\" (UniqueName: \"kubernetes.io/projected/60ff9873-7c18-45d4-82a7-9263602acdbb-kube-api-access-d8mgc\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-qlcmg\" (UID: \"60ff9873-7c18-45d4-82a7-9263602acdbb\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-qlcmg" machine # [ 48.823606] k3s[873]: I0919 21:21:14.912494 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-ca\" (UniqueName: \"kubernetes.io/secret/6dc8f60f-caca-4d20-b10f-d00881a93074-tls-ca\") pod \"nix-grpc-farm-lb-688d8d547b-r4w5t\" (UID: \"6dc8f60f-caca-4d20-b10f-d00881a93074\") " pod="farm/nix-grpc-farm-lb-688d8d547b-r4w5t" machine # [ 48.825121] k3s[873]: I0919 21:21:14.912558 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggxl9\" (UniqueName: \"kubernetes.io/projected/25f2f7ac-a091-4b45-8d23-efa63777d014-kube-api-access-ggxl9\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-lxl7f\" (UID: \"25f2f7ac-a091-4b45-8d23-efa63777d014\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-lxl7f" machine # [ 48.827236] k3s[873]: I0919 21:21:14.912660 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls\" (UniqueName: \"kubernetes.io/secret/25f2f7ac-a091-4b45-8d23-efa63777d014-tls\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-lxl7f\" (UID: \"25f2f7ac-a091-4b45-8d23-efa63777d014\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-lxl7f" machine # [ 48.828593] k3s[873]: I0919 21:21:14.912735 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nixconf\" (UniqueName: \"kubernetes.io/configmap/60ff9873-7c18-45d4-82a7-9263602acdbb-nixconf\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-qlcmg\" (UID: \"60ff9873-7c18-45d4-82a7-9263602acdbb\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-qlcmg" machine # [ 48.829817] k3s[873]: I0919 21:21:14.912811 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nix\" (UniqueName: \"kubernetes.io/empty-dir/60ff9873-7c18-45d4-82a7-9263602acdbb-nix\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-qlcmg\" (UID: \"60ff9873-7c18-45d4-82a7-9263602acdbb\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-qlcmg" machine # [ 48.831708] k3s[873]: I0919 21:21:14.912841 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls\" (UniqueName: \"kubernetes.io/secret/60ff9873-7c18-45d4-82a7-9263602acdbb-tls\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-qlcmg\" (UID: \"60ff9873-7c18-45d4-82a7-9263602acdbb\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-qlcmg" machine # [ 48.832428] k3s[873]: I0919 21:21:14.912870 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"build\" (UniqueName: \"kubernetes.io/empty-dir/25f2f7ac-a091-4b45-8d23-efa63777d014-build\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-lxl7f\" (UID: \"25f2f7ac-a091-4b45-8d23-efa63777d014\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-lxl7f" machine # [ 48.833490] k3s[873]: I0919 21:21:14.912915 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/25f2f7ac-a091-4b45-8d23-efa63777d014-ca\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-lxl7f\" (UID: \"25f2f7ac-a091-4b45-8d23-efa63777d014\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-lxl7f" machine # [ 48.834575] k3s[873]: I0919 21:21:14.913024 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"build\" (UniqueName: \"kubernetes.io/empty-dir/60ff9873-7c18-45d4-82a7-9263602acdbb-build\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-qlcmg\" (UID: \"60ff9873-7c18-45d4-82a7-9263602acdbb\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-qlcmg" machine # [ 48.835236] k3s[873]: I0919 21:21:14.913061 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nix\" (UniqueName: \"kubernetes.io/empty-dir/25f2f7ac-a091-4b45-8d23-efa63777d014-nix\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-lxl7f\" (UID: \"25f2f7ac-a091-4b45-8d23-efa63777d014\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-lxl7f" machine # [ 48.893912] k3s[873]: I0919 21:21:14.913091 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6dc8f60f-caca-4d20-b10f-d00881a93074-tmp\") pod \"nix-grpc-farm-lb-688d8d547b-r4w5t\" (UID: \"6dc8f60f-caca-4d20-b10f-d00881a93074\") " pod="farm/nix-grpc-farm-lb-688d8d547b-r4w5t" machine # [ 48.894389] k3s[873]: I0919 21:21:14.913121 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nixconf\" (UniqueName: \"kubernetes.io/configmap/25f2f7ac-a091-4b45-8d23-efa63777d014-nixconf\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-lxl7f\" (UID: \"25f2f7ac-a091-4b45-8d23-efa63777d014\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-lxl7f" machine # [ 48.895201] k3s[873]: I0919 21:21:14.913239 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc\" (UniqueName: \"kubernetes.io/secret/25f2f7ac-a091-4b45-8d23-efa63777d014-oidc\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-lxl7f\" (UID: \"25f2f7ac-a091-4b45-8d23-efa63777d014\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-lxl7f" machine # [ 48.895770] k3s[873]: I0919 21:21:14.913270 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-lb\" (UniqueName: \"kubernetes.io/secret/6dc8f60f-caca-4d20-b10f-d00881a93074-tls-lb\") pod \"nix-grpc-farm-lb-688d8d547b-r4w5t\" (UID: \"6dc8f60f-caca-4d20-b10f-d00881a93074\") " pod="farm/nix-grpc-farm-lb-688d8d547b-r4w5t" machine # [ 48.896880] k3s[873]: I0919 21:21:14.947349 873 scope.go:122] "RemoveContainer" containerID="3dd039beaa1a5bf9e46c743205ecae542e6c265cdfe0195f3df439a007e79a20" machine # [ 48.976348] systemd[1]: Started libcontainer container acbe59e06b50b8d33864d22fabd682df10b29b6e5b3869561ce0d0ecfeb8876f. machine # [ 49.165747] cni0: port 3(veth03230c89) entered blocking state machine # [ 49.167242] cni0: port 3(veth03230c89) entered disabled state machine # [ 49.168294] veth03230c89: entered allmulticast mode machine # [ 49.169160] veth03230c89: entered promiscuous mode machine # [ 49.183369] cni0: port 3(veth03230c89) entered blocking state machine # [ 49.184107] cni0: port 3(veth03230c89) entered forwarding state machine # [ 49.205066] cni0: port 4(vethd329d3b7) entered blocking state machine # [ 49.206302] cni0: port 4(vethd329d3b7) entered disabled state machine # [ 49.207206] vethd329d3b7: entered allmulticast mode machine # [ 49.207976] vethd329d3b7: entered promiscuous mode machine # [ 49.236522] cni0: port 4(vethd329d3b7) entered blocking state machine # [ 49.237384] cni0: port 4(vethd329d3b7) entered forwarding state machine # [ 49.241097] cni0: port 5(veth66585ebe) entered blocking state machine # [ 49.243229] cni0: port 5(veth66585ebe) entered disabled state machine # [ 49.245044] veth66585ebe: entered allmulticast mode machine # [ 49.246492] veth66585ebe: entered promiscuous mode machine # [ 49.115058] (udev-worker)[2346]: Network interface NamePolicy= disabled on kernel command line. machine # [ 49.120493] (udev-worker)[2354]: Network interface NamePolicy= disabled on kernel command line. machine # [ 49.123931] (udev-worker)[2355]: Network interface NamePolicy= disabled on kernel command line. machine # [ 49.276392] cni0: port 5(veth66585ebe) entered blocking state machine # [ 49.277024] cni0: port 5(veth66585ebe) entered forwarding state machine # [ 49.175282] dhcpcd[673]: veth03230c89: IAID cc:4f:03:e4 machine # [ 49.175689] dhcpcd[673]: veth03230c89: adding address fe80::4c4d:ccff:fe4f:3e4 machine # [ 49.201362] dhcpcd[673]: veth66585ebe: IAID c5:89:46:e4 machine # [ 49.201674] dhcpcd[673]: veth66585ebe: adding address fe80::9ce0:c5ff:fe89:46e4 machine # [ 49.226445] dhcpcd[673]: vethd329d3b7: IAID a8:a4:d1:4d machine # [ 49.226979] dhcpcd[673]: vethd329d3b7: adding address fe80::5cde:a8ff:fea4:d14d machine # [ 49.242640] systemd[1]: Started libcontainer container 1c27627b20769404a4bf5515a1dad2481a7af72a25c05938e475ef9f1d46575d. machine # [ 49.271559] systemd[1]: Started libcontainer container 3640714b93fea4a7b2684b2b4304234f50a39708cfff671224d2c74bcdbc56bb. machine # [ 49.313104] systemd[1]: Started libcontainer container bcb2791eb5d2554ee32631ebd16ae41333bac33236f68654596aa746b078c6f6. machine # [ 49.607725] systemd[1]: cri-containerd-acbe59e06b50b8d33864d22fabd682df10b29b6e5b3869561ce0d0ecfeb8876f.scope: Deactivated successfully. machine # [ 49.612840] systemd[1]: cri-containerd-acbe59e06b50b8d33864d22fabd682df10b29b6e5b3869561ce0d0ecfeb8876f.scope: Consumed 281ms CPU time over 631ms wall clock time, 17.3M memory peak, 425.4K incoming IP traffic, 18.7K outgoing IP traffic. machine # [ 49.772532] dhcpcd[673]: veth66585ebe: soliciting a DHCP lease machine # [ 49.870769] k3s[873]: I0919 21:21:15.961078 873 scope.go:122] "RemoveContainer" containerID="3dd039beaa1a5bf9e46c743205ecae542e6c265cdfe0195f3df439a007e79a20" machine # [ 50.077916] dhcpcd[673]: vethd329d3b7: soliciting a DHCP lease machine # [ 50.250890] systemd[1]: Started libcontainer container 3a03c71aeaedd7de915ca4e3700adc600f7daf5bce78f39ec3fb6960270348ba. machine # [ 50.476889] dhcpcd[673]: veth03230c89: soliciting a DHCP lease machine # [ 50.779696] dhcpcd[673]: vethd329d3b7: soliciting an IPv6 router machine # [ 50.905408] k3s[873]: I0919 21:21:16.993604 873 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="farm/nix-grpc-farm-lb-688d8d547b-r4w5t" podStartSLOduration=2.993561606 podStartE2EDuration="2.993561606s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 21:21:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 21:21:16.993123562 +0000 UTC m=+32.576972769" watchObservedRunningTime="2026-09-19 21:21:16.993561606 +0000 UTC m=+32.577410814" machine # [ 50.906245] dhcpcd[673]: veth66585ebe: soliciting an IPv6 router machine # [ 50.936313] systemd[1]: cri-containerd-3db3a25b6932a0d09916b583597cc1cd0f2a7004f9fde1d1f643a404d293b0f6.scope: Deactivated successfully. machine # [ 50.979075] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-3db3a25b6932a0d09916b583597cc1cd0f2a7004f9fde1d1f643a404d293b0f6-rootfs.mount: Deactivated successfully. machine # [ 51.148486] systemd[1]: run-k3s-containerd-io.containerd.grpc.v1.cri-sandboxes-3db3a25b6932a0d09916b583597cc1cd0f2a7004f9fde1d1f643a404d293b0f6-shm.mount: Deactivated successfully. machine # [ 51.353633] cni0: port 2(veth51f330eb) entered disabled state machine # [ 51.207777] dhcpcd[673]: veth51f330eb: carrier lost machine # [ 51.357876] veth51f330eb (unregistering): left allmulticast mode machine # [ 51.359089] veth51f330eb (unregistering): left promiscuous mode machine # [ 51.360446] cni0: port 2(veth51f330eb) entered disabled state machine # [ 51.238307] systemd[1]: run-netns-cni\x2d2526a4f7\x2d0d9e\x2d7f0e\x2d89c8\x2dcbcdd065e75c.mount: Deactivated successfully. machine # [ 51.258286] dhcpcd[673]: veth51f330eb: deleting address fe80::b4aa:caff:fe71:73bd machine # [ 51.348310] dhcpcd[673]: veth03230c89: soliciting an IPv6 router machine # [ 51.349253] dhcpcd[673]: veth51f330eb: removing interface machine # [ 51.350579] k3s[873]: I0919 21:21:17.441192 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9583a273-8094-4a85-92fd-83dea302013f-kube-api-access-phf8p\" (UniqueName: \"kubernetes.io/projected/9583a273-8094-4a85-92fd-83dea302013f-kube-api-access-phf8p\") pod \"9583a273-8094-4a85-92fd-83dea302013f\" (UID: \"9583a273-8094-4a85-92fd-83dea302013f\") " machine # [ 51.351343] k3s[873]: I0919 21:21:17.441951 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9583a273-8094-4a85-92fd-83dea302013f-values\" (UniqueName: \"kubernetes.io/projected/9583a273-8094-4a85-92fd-83dea302013f-values\") pod \"9583a273-8094-4a85-92fd-83dea302013f\" (UID: \"9583a273-8094-4a85-92fd-83dea302013f\") " machine # [ 51.352969] k3s[873]: I0919 21:21:17.442934 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9583a273-8094-4a85-92fd-83dea302013f-tmp\" (UniqueName: \"kubernetes.io/empty-dir/9583a273-8094-4a85-92fd-83dea302013f-tmp\") pod \"9583a273-8094-4a85-92fd-83dea302013f\" (UID: \"9583a273-8094-4a85-92fd-83dea302013f\") " machine # [ 51.354574] k3s[873]: I0919 21:21:17.442971 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9583a273-8094-4a85-92fd-83dea302013f-content\" (UniqueName: \"kubernetes.io/configmap/9583a273-8094-4a85-92fd-83dea302013f-content\") pod \"9583a273-8094-4a85-92fd-83dea302013f\" (UID: \"9583a273-8094-4a85-92fd-83dea302013f\") " machine # [ 51.355406] k3s[873]: I0919 21:21:17.443010 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9583a273-8094-4a85-92fd-83dea302013f-klipper-cache\" (UniqueName: \"kubernetes.io/empty-dir/9583a273-8094-4a85-92fd-83dea302013f-klipper-cache\") pod \"9583a273-8094-4a85-92fd-83dea302013f\" (UID: \"9583a273-8094-4a85-92fd-83dea302013f\") " machine # [ 51.357051] k3s[873]: I0919 21:21:17.443054 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9583a273-8094-4a85-92fd-83dea302013f-klipper-helm\" (UniqueName: \"kubernetes.io/empty-dir/9583a273-8094-4a85-92fd-83dea302013f-klipper-helm\") pod \"9583a273-8094-4a85-92fd-83dea302013f\" (UID: \"9583a273-8094-4a85-92fd-83dea302013f\") " machine # [ 51.357625] k3s[873]: I0919 21:21:17.443087 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9583a273-8094-4a85-92fd-83dea302013f-klipper-config\" (UniqueName: \"kubernetes.io/empty-dir/9583a273-8094-4a85-92fd-83dea302013f-klipper-config\") pod \"9583a273-8094-4a85-92fd-83dea302013f\" (UID: \"9583a273-8094-4a85-92fd-83dea302013f\") " machine # [ 51.358282] k3s[873]: I0919 21:21:17.443666 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9583a273-8094-4a85-92fd-83dea302013f-content" pod "9583a273-8094-4a85-92fd-83dea302013f" (UID: "9583a273-8094-4a85-92fd-83dea302013f"). InnerVolumeSpecName "content". PluginName "kubernetes.io/configmap", VolumeGIDValue "" machine # [ 51.369860] systemd[1]: var-lib-kubelet-pods-9583a273\x2d8094\x2d4a85\x2d92fd\x2d83dea302013f-volumes-kubernetes.io\x7eempty\x2ddir-tmp.mount: Deactivated successfully. machine # [ 51.372700] systemd[1]: var-lib-kubelet-pods-9583a273\x2d8094\x2d4a85\x2d92fd\x2d83dea302013f-volumes-kubernetes.io\x7eempty\x2ddir-klipper\x2dhelm.mount: Deactivated successfully. machine # [ 51.397840] k3s[873]: I0919 21:21:17.488276 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9583a273-8094-4a85-92fd-83dea302013f-values" pod "9583a273-8094-4a85-92fd-83dea302013f" (UID: "9583a273-8094-4a85-92fd-83dea302013f"). InnerVolumeSpecName "values". PluginName "kubernetes.io/projected", VolumeGIDValue "" machine # [ 51.403406] k3s[873]: I0919 21:21:17.488276 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9583a273-8094-4a85-92fd-83dea302013f-klipper-config" pod "9583a273-8094-4a85-92fd-83dea302013f" (UID: "9583a273-8094-4a85-92fd-83dea302013f"). InnerVolumeSpecName "klipper-config". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 51.409373] k3s[873]: I0919 21:21:17.489063 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9583a273-8094-4a85-92fd-83dea302013f-klipper-helm" pod "9583a273-8094-4a85-92fd-83dea302013f" (UID: "9583a273-8094-4a85-92fd-83dea302013f"). InnerVolumeSpecName "klipper-helm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 51.409705] k3s[873]: I0919 21:21:17.489356 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9583a273-8094-4a85-92fd-83dea302013f-klipper-cache" pod "9583a273-8094-4a85-92fd-83dea302013f" (UID: "9583a273-8094-4a85-92fd-83dea302013f"). InnerVolumeSpecName "klipper-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 51.410487] k3s[873]: I0919 21:21:17.489496 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9583a273-8094-4a85-92fd-83dea302013f-tmp" pod "9583a273-8094-4a85-92fd-83dea302013f" (UID: "9583a273-8094-4a85-92fd-83dea302013f"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 51.436112] k3s[873]: I0919 21:21:17.525180 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9583a273-8094-4a85-92fd-83dea302013f-kube-api-access-phf8p" pod "9583a273-8094-4a85-92fd-83dea302013f" (UID: "9583a273-8094-4a85-92fd-83dea302013f"). InnerVolumeSpecName "kube-api-access-phf8p". PluginName "kubernetes.io/projected", VolumeGIDValue "" machine # [ 51.453382] k3s[873]: I0919 21:21:17.543946 873 reconciler_common.go:299] "Volume detached for volume \"klipper-cache\" (UniqueName: \"kubernetes.io/empty-dir/9583a273-8094-4a85-92fd-83dea302013f-klipper-cache\") on node \"machine\" DevicePath \"\"" machine # [ 51.453697] k3s[873]: I0919 21:21:17.543995 873 reconciler_common.go:299] "Volume detached for volume \"klipper-helm\" (UniqueName: \"kubernetes.io/empty-dir/9583a273-8094-4a85-92fd-83dea302013f-klipper-helm\") on node \"machine\" DevicePath \"\"" machine # [ 51.454226] k3s[873]: I0919 21:21:17.544009 873 reconciler_common.go:299] "Volume detached for volume \"klipper-config\" (UniqueName: \"kubernetes.io/empty-dir/9583a273-8094-4a85-92fd-83dea302013f-klipper-config\") on node \"machine\" DevicePath \"\"" machine # [ 51.454676] k3s[873]: I0919 21:21:17.544027 873 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phf8p\" (UniqueName: \"kubernetes.io/projected/9583a273-8094-4a85-92fd-83dea302013f-kube-api-access-phf8p\") on node \"machine\" DevicePath \"\"" machine # [ 51.455243] k3s[873]: I0919 21:21:17.544043 873 reconciler_common.go:299] "Volume detached for volume \"values\" (UniqueName: \"kubernetes.io/projected/9583a273-8094-4a85-92fd-83dea302013f-values\") on node \"machine\" DevicePath \"\"" machine # [ 51.455756] k3s[873]: I0919 21:21:17.544056 873 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9583a273-8094-4a85-92fd-83dea302013f-tmp\") on node \"machine\" DevicePath \"\"" machine # [ 51.456419] k3s[873]: I0919 21:21:17.544069 873 reconciler_common.go:299] "Volume detached for volume \"content\" (UniqueName: \"kubernetes.io/configmap/9583a273-8094-4a85-92fd-83dea302013f-content\") on node \"machine\" DevicePath \"\"" machine # [ 51.895742] k3s[873]: I0919 21:21:17.986273 873 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3db3a25b6932a0d09916b583597cc1cd0f2a7004f9fde1d1f643a404d293b0f6" machine # [ 51.910155] systemd[1]: Removed slice libcontainer container kubepods-burstable-pod9583a273_8094_4a85_92fd_83dea302013f.slice. machine # [ 51.910582] systemd[1]: kubepods-burstable-pod9583a273_8094_4a85_92fd_83dea302013f.slice: Consumed 947ms CPU time over 20.356s wall clock time, 31.4M memory peak, 2.8M incoming IP traffic, 143.9K outgoing IP traffic. machine # [ 51.979374] systemd[1]: var-lib-kubelet-pods-9583a273\x2d8094\x2d4a85\x2d92fd\x2d83dea302013f-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dphf8p.mount: Deactivated successfully. machine # [ 51.982870] systemd[1]: var-lib-kubelet-pods-9583a273\x2d8094\x2d4a85\x2d92fd\x2d83dea302013f-volumes-kubernetes.io\x7eprojected-values.mount: Deactivated successfully. machine # [ 51.986149] systemd[1]: var-lib-kubelet-pods-9583a273\x2d8094\x2d4a85\x2d92fd\x2d83dea302013f-volumes-kubernetes.io\x7eempty\x2ddir-klipper\x2dconfig.mount: Deactivated successfully. machine # [ 51.988504] systemd[1]: var-lib-kubelet-pods-9583a273\x2d8094\x2d4a85\x2d92fd\x2d83dea302013f-volumes-kubernetes.io\x7eempty\x2ddir-klipper\x2dcache.mount: Deactivated successfully. machine # [ 52.515891] k3s[873]: time="2026-09-19T21:21:18Z" level=info msg="Event occurred" apiVersion=helm.cattle.io/v1 fieldPath= kind=HelmChart logger=k3s/helm-controller message="Suspended Job kube-system/helm-install-nix-grpc-farm for delete" object=kube-system/nix-grpc-farm reason=SuspendJob type=Normal machine # [ 52.521220] k3s[873]: time="2026-09-19T21:21:18Z" level=info msg="Event occurred" apiVersion=helm.cattle.io/v1 fieldPath= kind=HelmChart logger=k3s/helm-controller message="Applying HelmChart from https://%{KUBERNETES_API}%/static/charts/nix-grpc-farm.tgz using Job kube-system/helm-install-nix-grpc-farm " object=kube-system/nix-grpc-farm reason=ApplyJob type=Normal machine # [ 52.531123] k3s[873]: time="2026-09-19T21:21:18Z" level=error msg="error syncing 'kube-system/nix-grpc-farm': handler helm-controller-chart-registration: DesiredSet - Replace Wait batch/v1, Kind=Job kube-system/helm-install-nix-grpc-farm for helm-controller-chart-registration kube-system/nix-grpc-farm, requeuing" machine # [ 52.795974] k3s[873]: I0919 21:21:18.886430 873 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9583a273-8094-4a85-92fd-83dea302013f" path="/var/lib/kubelet/pods/9583a273-8094-4a85-92fd-83dea302013f/volumes" machine # [ 52.911074] k3s[873]: time="2026-09-19T21:21:19Z" level=info msg="Event occurred" apiVersion=helm.cattle.io/v1 fieldPath= kind=HelmChart logger=k3s/helm-controller message="Applying HelmChart from https://%{KUBERNETES_API}%/static/charts/nix-grpc-farm.tgz using Job kube-system/helm-install-nix-grpc-farm " object=kube-system/nix-grpc-farm reason=ApplyJob type=Normal machine # [ 53.287972] k3s[873]: time="2026-09-19T21:21:19Z" level=info msg="Event occurred" apiVersion=helm.cattle.io/v1 fieldPath= kind=HelmChart logger=k3s/helm-controller message="Resumed synced Job kube-system/helm-install-nix-grpc-farm" object=kube-system/nix-grpc-farm reason=ResumeJob type=Normal machine # [ 53.330666] systemd[1]: Created slice libcontainer container kubepods-burstable-pod6d037147_1d29_4e02_b2ab_2f08bc2a576c.slice. machine # [ 53.440024] systemd[1]: var-lib-rancher-k3s-agent-containerd-tmpmounts-containerd\x2dmount2851700305.mount: Deactivated successfully. machine # [ 53.445443] systemd[1]: var-lib-rancher-k3s-agent-containerd-tmpmounts-containerd\x2dmount2856492313.mount: Deactivated successfully. machine # [ 53.467603] k3s[873]: I0919 21:21:19.558183 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klipper-helm\" (UniqueName: \"kubernetes.io/empty-dir/6d037147-1d29-4e02-b2ab-2f08bc2a576c-klipper-helm\") pod \"helm-install-nix-grpc-farm-zcg5w\" (UID: \"6d037147-1d29-4e02-b2ab-2f08bc2a576c\") " pod="kube-system/helm-install-nix-grpc-farm-zcg5w" machine # [ 53.476827] k3s[873]: I0919 21:21:19.558232 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klipper-config\" (UniqueName: \"kubernetes.io/empty-dir/6d037147-1d29-4e02-b2ab-2f08bc2a576c-klipper-config\") pod \"helm-install-nix-grpc-farm-zcg5w\" (UID: \"6d037147-1d29-4e02-b2ab-2f08bc2a576c\") " pod="kube-system/helm-install-nix-grpc-farm-zcg5w" machine # [ 53.483818] k3s[873]: I0919 21:21:19.558261 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d037147-1d29-4e02-b2ab-2f08bc2a576c-tmp\") pod \"helm-install-nix-grpc-farm-zcg5w\" (UID: \"6d037147-1d29-4e02-b2ab-2f08bc2a576c\") " pod="kube-system/helm-install-nix-grpc-farm-zcg5w" machine # [ 53.491363] k3s[873]: I0919 21:21:19.558413 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"content\" (UniqueName: \"kubernetes.io/configmap/6d037147-1d29-4e02-b2ab-2f08bc2a576c-content\") pod \"helm-install-nix-grpc-farm-zcg5w\" (UID: \"6d037147-1d29-4e02-b2ab-2f08bc2a576c\") " pod="kube-system/helm-install-nix-grpc-farm-zcg5w" machine # [ 53.498376] k3s[873]: I0919 21:21:19.558564 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klipper-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d037147-1d29-4e02-b2ab-2f08bc2a576c-klipper-cache\") pod \"helm-install-nix-grpc-farm-zcg5w\" (UID: \"6d037147-1d29-4e02-b2ab-2f08bc2a576c\") " pod="kube-system/helm-install-nix-grpc-farm-zcg5w" machine # [ 53.505959] k3s[873]: I0919 21:21:19.558642 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96g8m\" (UniqueName: \"kubernetes.io/projected/6d037147-1d29-4e02-b2ab-2f08bc2a576c-kube-api-access-96g8m\") pod \"helm-install-nix-grpc-farm-zcg5w\" (UID: \"6d037147-1d29-4e02-b2ab-2f08bc2a576c\") " pod="kube-system/helm-install-nix-grpc-farm-zcg5w" machine # [ 53.513325] k3s[873]: I0919 21:21:19.558721 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"values\" (UniqueName: \"kubernetes.io/projected/6d037147-1d29-4e02-b2ab-2f08bc2a576c-values\") pod \"helm-install-nix-grpc-farm-zcg5w\" (UID: \"6d037147-1d29-4e02-b2ab-2f08bc2a576c\") " pod="kube-system/helm-install-nix-grpc-farm-zcg5w" machine # [ 53.522559] systemd[1]: Started libcontainer container 97beef8239fd728a6210f42e4d02e4c40d45d0e319491b30511a6ffdf7858a11. machine # [ 53.611762] systemd[1]: Started libcontainer container 793c5107f72c6c9db564477d1a8454b8cabf5ec18c5ac9c66f8a33144f6a85b6. machine # [ 53.681460] dhcpcd[673]: vethdded1014: using IPv4LL address 169.254.234.42 machine # [ 53.682256] dhcpcd[673]: vethdded1014: adding route to 169.254.0.0/16 machine # [ 53.843342] cni0: port 2(veth3ce56192) entered blocking state machine # [ 53.845446] cni0: port 2(veth3ce56192) entered disabled state machine # [ 53.846065] veth3ce56192: entered allmulticast mode machine # [ 53.846806] veth3ce56192: entered promiscuous mode machine # [ 53.866922] cni0: port 2(veth3ce56192) entered blocking state machine # [ 53.867643] cni0: port 2(veth3ce56192) entered forwarding state machine # [ 53.807154] (udev-worker)[2954]: Network interface NamePolicy= disabled on kernel command line. machine # [ 53.859148] dhcpcd[673]: veth3ce56192: IAID 32:c4:e4:f4 machine # [ 53.859603] dhcpcd[673]: veth3ce56192: adding address fe80::2c2e:32ff:fec4:e4f4 machine # [ 54.046640] systemd[1]: Started libcontainer container 71c179323a283435e549a0b82de3afba35dd6cd7fa80f6e339929305f5ea19e4. machine # [ 54.244391] systemd[1]: cri-containerd-97beef8239fd728a6210f42e4d02e4c40d45d0e319491b30511a6ffdf7858a11.scope: Deactivated successfully. machine # [ 54.244659] systemd[1]: cri-containerd-97beef8239fd728a6210f42e4d02e4c40d45d0e319491b30511a6ffdf7858a11.scope: Consumed 559ms CPU time over 720ms wall clock time, 285.2M memory peak, 32K read from disk, 221.3M written to disk. machine # [ 54.278408] systemd[1]: Started libcontainer container 55f4b790498984e296d0abfd180da62e8fafb625d150d3a71861715a3bcee531. machine # [ 54.423510] systemd[1]: cri-containerd-793c5107f72c6c9db564477d1a8454b8cabf5ec18c5ac9c66f8a33144f6a85b6.scope: Deactivated successfully. machine # [ 54.424159] systemd[1]: cri-containerd-793c5107f72c6c9db564477d1a8454b8cabf5ec18c5ac9c66f8a33144f6a85b6.scope: Consumed 587ms CPU time over 811ms wall clock time, 290.2M memory peak, 36K read from disk, 264.4M written to disk. machine # [ 54.450095] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-97beef8239fd728a6210f42e4d02e4c40d45d0e319491b30511a6ffdf7858a11-rootfs.mount: Deactivated successfully. machine # [ 54.506566] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-793c5107f72c6c9db564477d1a8454b8cabf5ec18c5ac9c66f8a33144f6a85b6-rootfs.mount: Deactivated successfully. machine # [ 54.773597] dhcpcd[673]: veth66585ebe: probing for an IPv4LL address machine # error: object has been deleted machine # [ 54.948252] systemd[1]: cri-containerd-3a03c71aeaedd7de915ca4e3700adc600f7daf5bce78f39ec3fb6960270348ba.scope: Deactivated successfully. machine # [ 54.950400] systemd[1]: cri-containerd-3a03c71aeaedd7de915ca4e3700adc600f7daf5bce78f39ec3fb6960270348ba.scope: Consumed 123ms CPU time over 4.698s wall clock time, 15.2M memory peak, 3.9K incoming IP traffic, 4.9K outgoing IP traffic. machine # [ 54.993823] k3s[873]: I0919 21:21:21.082159 873 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/helm-install-nix-grpc-farm-zcg5w" podStartSLOduration=2.082123979 podStartE2EDuration="2.082123979s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 21:21:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 21:21:21.067018987 +0000 UTC m=+36.650863724" watchObservedRunningTime="2026-09-19 21:21:21.082123979 +0000 UTC m=+36.665968437" machine # [ 55.042451] systemd[1]: Started libcontainer container 5a44891c07ab9df573bfd747dde766f80400ac7aafa9567b3f9ff7670301ed51. machine # [ 55.063207] systemd[1]: Started libcontainer container 98e6c45ffc9787574eed8892d739db6a99981cf79a86b755016c5aafa5d534bb. machine # [ 55.063453] systemd[1]: cri-containerd-3640714b93fea4a7b2684b2b4304234f50a39708cfff671224d2c74bcdbc56bb.scope: Deactivated successfully. machine # [ 55.079208] dhcpcd[673]: vethd329d3b7: probing for an IPv4LL address machine # [ 55.092425] k3s[873]: E0919 21:21:21.182935 873 configmap.go:193] Couldn't get configMap farm/nix-grpc-farm-worker-x86-64-linux: configmap "nix-grpc-farm-worker-x86-64-linux" not found machine # [ 55.092885] k3s[873]: E0919 21:21:21.183515 873 secret.go:189] Couldn't get secret farm/nix-grpc-farm-oidc: secret "nix-grpc-farm-oidc" not found machine # [ 55.094186] k3s[873]: E0919 21:21:21.184499 873 configmap.go:193] Couldn't get configMap farm/nix-grpc-farm-worker-x86-64-linux: configmap "nix-grpc-farm-worker-x86-64-linux" not found machine # [ 55.094758] k3s[873]: E0919 21:21:21.184597 873 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25f2f7ac-a091-4b45-8d23-efa63777d014-oidc podName:25f2f7ac-a091-4b45-8d23-efa63777d014 nodeName:}" failed. No retries permitted until 2026-09-19 21:21:21.684566887 +0000 UTC m=+37.268411345 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "oidc" (UniqueName: "kubernetes.io/secret/25f2f7ac-a091-4b45-8d23-efa63777d014-oidc") pod "nix-grpc-farm-worker-x86-64-linux-5cdbdb595-lxl7f" (UID: "25f2f7ac-a091-4b45-8d23-efa63777d014") : secret "nix-grpc-farm-oidc" not found machine # [ 55.095597] k3s[873]: E0919 21:21:21.184644 873 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/60ff9873-7c18-45d4-82a7-9263602acdbb-nixconf podName:60ff9873-7c18-45d4-82a7-9263602acdbb nodeName:}" failed. No retries permitted until 2026-09-19 21:21:21.684632259 +0000 UTC m=+37.268476437 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "nixconf" (UniqueName: "kubernetes.io/configmap/60ff9873-7c18-45d4-82a7-9263602acdbb-nixconf") pod "nix-grpc-farm-worker-x86-64-linux-5cdbdb595-qlcmg" (UID: "60ff9873-7c18-45d4-82a7-9263602acdbb") : configmap "nix-grpc-farm-worker-x86-64-linux" not found machine # [ 55.096170] k3s[873]: E0919 21:21:21.183653 873 secret.go:189] Couldn't get secret farm/nix-grpc-farm-oidc: secret "nix-grpc-farm-oidc" not found machine # [ 55.097295] k3s[873]: E0919 21:21:21.184721 873 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60ff9873-7c18-45d4-82a7-9263602acdbb-oidc podName:60ff9873-7c18-45d4-82a7-9263602acdbb nodeName:}" failed. No retries permitted until 2026-09-19 21:21:21.684709364 +0000 UTC m=+37.268553822 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "oidc" (UniqueName: "kubernetes.io/secret/60ff9873-7c18-45d4-82a7-9263602acdbb-oidc") pod "nix-grpc-farm-worker-x86-64-linux-5cdbdb595-qlcmg" (UID: "60ff9873-7c18-45d4-82a7-9263602acdbb") : secret "nix-grpc-farm-oidc" not found machine # [ 55.098550] k3s[873]: E0919 21:21:21.186074 873 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/25f2f7ac-a091-4b45-8d23-efa63777d014-nixconf podName:25f2f7ac-a091-4b45-8d23-efa63777d014 nodeName:}" failed. No retries permitted until 2026-09-19 21:21:21.686053389 +0000 UTC m=+37.269898406 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "nixconf" (UniqueName: "kubernetes.io/configmap/25f2f7ac-a091-4b45-8d23-efa63777d014-nixconf") pod "nix-grpc-farm-worker-x86-64-linux-5cdbdb595-lxl7f" (UID: "25f2f7ac-a091-4b45-8d23-efa63777d014") : configmap "nix-grpc-farm-worker-x86-64-linux" not found machine # [ 55.434106] cni0: port 4(vethd329d3b7) entered disabled state machine # [ 55.286894] dhcpcd[673]: vethd329d3b7: carrier lost machine # [ 55.289108] systemd[1]: Started libcontainer container 437ba7[ 55.437045] vethd329d3b7 (unregistering): left allmulticast mode machine # a736f8ea8a4384acab8a15546a6d8720bfce0a457ab9f302bc4c5c512c. machine # [ 55.439571] vethd329d3b7 (unregistering): left promiscuous mode machine # [ 55.441783] cni0: port 4(vethd329d3b7) entered disabled state machine # [ 55.309920] systemd[1]: Started libcontainer container 0fe734dda2d1972b0181decd0c781a4332e29db73eaf48e3a3bc74a9693006f4. machine # [ 55.341611] dhcpcd[673]: vethd329d3b7: deleting address fe80::5cde:a8ff:fea4:d14d machine # [ 55.430503] dhcpcd[673]: vethd329d3b7: removing interface machine # [ 55.455404] systemd[1]: var-lib-rancher-k3s-agent-containerd-tmpmounts-containerd\x2dmount2156069973.mount: Deactivated successfully. machine # [ 55.456047] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-3a03c71aeaedd7de915ca4e3700adc600f7daf5bce78f39ec3fb6960270348ba-rootfs.mount: Deactivated successfully. machine # [ 55.462463] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-3640714b93fea4a7b2684b2b4304234f50a39708cfff671224d2c74bcdbc56bb-rootfs.mount: Deactivated successfully. machine # [ 55.463923] systemd[1]: run-k3s-containerd-io.containerd.grpc.v1.cri-sandboxes-3640714b93fea4a7b2684b2b4304234f50a39708cfff671224d2c74bcdbc56bb-shm.mount: Deactivated successfully. machine # [ 55.464756] systemd[1]: run-netns-cni\x2d5a723d27\x2d7ec0\x2d8cb7\x2d9a96\x2d5c42acaf70e1.mount: Deactivated successfully. machine # [ 55.467738] dhcpcd[673]: veth3ce56192: soliciting a DHCP lease machine # [ 55.476685] dhcpcd[673]: veth03230c89: probing for an IPv4LL address machine # [ 55.494396] systemd[1]: Started libcontainer container b9fe605bc08a3583d8bec5e4d011058900346272fc99203bcb02d07750e59b20. machine # [ 55.497544] k3s[873]: I0919 21:21:21.587306 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6dc8f60f-caca-4d20-b10f-d00881a93074-tmp\" (UniqueName: \"kubernetes.io/empty-dir/6dc8f60f-caca-4d20-b10f-d00881a93074-tmp\") pod \"6dc8f60f-caca-4d20-b10f-d00881a93074\" (UID: \"6dc8f60f-caca-4d20-b10f-d00881a93074\") " machine # [ 55.509088] k3s[873]: I0919 21:21:21.587364 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6dc8f60f-caca-4d20-b10f-d00881a93074-tls-ca\" (UniqueName: \"kubernetes.io/secret/6dc8f60f-caca-4d20-b10f-d00881a93074-tls-ca\") pod \"6dc8f60f-caca-4d20-b10f-d00881a93074\" (UID: \"6dc8f60f-caca-4d20-b10f-d00881a93074\") " machine # [ 55.518375] k3s[873]: I0919 21:21:21.587399 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6dc8f60f-caca-4d20-b10f-d00881a93074-config\" (UniqueName: \"kubernetes.io/configmap/6dc8f60f-caca-4d20-b10f-d00881a93074-config\") pod \"6dc8f60f-caca-4d20-b10f-d00881a93074\" (UID: \"6dc8f60f-caca-4d20-b10f-d00881a93074\") " machine # [ 55.523135] k3s[873]: I0919 21:21:21.587432 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6dc8f60f-caca-4d20-b10f-d00881a93074-tls-lb\" (UniqueName: \"kubernetes.io/secret/6dc8f60f-caca-4d20-b10f-d00881a93074-tls-lb\") pod \"6dc8f60f-caca-4d20-b10f-d00881a93074\" (UID: \"6dc8f60f-caca-4d20-b10f-d00881a93074\") " machine # [ 55.528042] systemd[1]: Started libcontainer container 19944217fa890aec1acc35879e3500db1cc3f58eadc2bade0b1f11a91d1ccadc. machine # [ 55.530320] k3s[873]: I0919 21:21:21.609353 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dc8f60f-caca-4d20-b10f-d00881a93074-tmp" pod "6dc8f60f-caca-4d20-b10f-d00881a93074" (UID: "6dc8f60f-caca-4d20-b10f-d00881a93074"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 55.535351] k3s[873]: I0919 21:21:21.610102 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6dc8f60f-caca-4d20-b10f-d00881a93074-config" pod "6dc8f60f-caca-4d20-b10f-d00881a93074" (UID: "6dc8f60f-caca-4d20-b10f-d00881a93074"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" machine # [ 55.540608] k3s[873]: I0919 21:21:21.619610 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6dc8f60f-caca-4d20-b10f-d00881a93074-tls-lb" pod "6dc8f60f-caca-4d20-b10f-d00881a93074" (UID: "6dc8f60f-caca-4d20-b10f-d00881a93074"). InnerVolumeSpecName "tls-lb". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 55.545556] systemd[1]: var-lib-kubelet-pods-6dc8f60f\x2dcaca\x2d4d20\x2db10f\x2dd00881a93074-volumes-kubernetes.io\x7eempty\x2ddir-tmp.mount: Deactivated successfully. machine # [ 55.546109] k3s[873]: I0919 21:21:21.630148 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6dc8f60f-caca-4d20-b10f-d00881a93074-tls-ca" pod "6dc8f60f-caca-4d20-b10f-d00881a93074" (UID: "6dc8f60f-caca-4d20-b10f-d00881a93074"). InnerVolumeSpecName "tls-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 55.546637] systemd[1]: var-lib-kubelet-pods-6dc8f60f\x2dcaca\x2d4d20\x2db10f\x2dd00881a93074-volume\x2dsubpaths-config-envoy-0.mount: Deactivated successfully. machine # [ 55.597841] k3s[873]: I0919 21:21:21.687976 873 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6dc8f60f-caca-4d20-b10f-d00881a93074-tmp\") on node \"machine\" DevicePath \"\"" machine # [ 55.598307] k3s[873]: I0919 21:21:21.688013 873 reconciler_common.go:299] "Volume detached for volume \"tls-ca\" (UniqueName: \"kubernetes.io/secret/6dc8f60f-caca-4d20-b10f-d00881a93074-tls-ca\") on node \"machine\" DevicePath \"\"" machine # [ 55.598722] k3s[873]: I0919 21:21:21.688027 873 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6dc8f60f-caca-4d20-b10f-d00881a93074-config\") on node \"machine\" DevicePath \"\"" machine # [ 55.599415] k3s[873]: I0919 21:21:21.688047 873 reconciler_common.go:299] "Volume detached for volume \"tls-lb\" (UniqueName: \"kubernetes.io/secret/6dc8f60f-caca-4d20-b10f-d00881a93074-tls-lb\") on node \"machine\" DevicePath \"\"" machine # [ 55.599865] k3s[873]: E0919 21:21:21.688085 873 secret.go:189] Couldn't get secret farm/nix-grpc-farm-oidc: secret "nix-grpc-farm-oidc" not found machine # [ 55.600720] k3s[873]: E0919 21:21:21.688156 873 configmap.go:193] Couldn't get configMap farm/nix-grpc-farm-worker-x86-64-linux: configmap "nix-grpc-farm-worker-x86-64-linux" not found machine # [ 55.601372] k3s[873]: E0919 21:21:21.688173 873 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25f2f7ac-a091-4b45-8d23-efa63777d014-oidc podName:25f2f7ac-a091-4b45-8d23-efa63777d014 nodeName:}" failed. No retries permitted until 2026-09-19 21:21:22.688151002 +0000 UTC m=+38.271995460 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "oidc" (UniqueName: "kubernetes.io/secret/25f2f7ac-a091-4b45-8d23-efa63777d014-oidc") pod "nix-grpc-farm-worker-x86-64-linux-5cdbdb595-lxl7f" (UID: "25f2f7ac-a091-4b45-8d23-efa63777d014") : secret "nix-grpc-farm-oidc" not found machine # [ 55.602087] k3s[873]: E0919 21:21:21.688233 873 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/60ff9873-7c18-45d4-82a7-9263602acdbb-nixconf podName:60ff9873-7c18-45d4-82a7-9263602acdbb nodeName:}" failed. No retries permitted until 2026-09-19 21:21:22.688212183 +0000 UTC m=+38.272056641 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "nixconf" (UniqueName: "kubernetes.io/configmap/60ff9873-7c18-45d4-82a7-9263602acdbb-nixconf") pod "nix-grpc-farm-worker-x86-64-linux-5cdbdb595-qlcmg" (UID: "60ff9873-7c18-45d4-82a7-9263602acdbb") : configmap "nix-grpc-farm-worker-x86-64-linux" not found machine # [ 55.602681] k3s[873]: E0919 21:21:21.688237 873 configmap.go:193] Couldn't get configMap farm/nix-grpc-farm-worker-x86-64-linux: configmap "nix-grpc-farm-worker-x86-64-linux" not found machine # [ 55.603557] k3s[873]: E0919 21:21:21.688281 873 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/25f2f7ac-a091-4b45-8d23-efa63777d014-nixconf podName:25f2f7ac-a091-4b45-8d23-efa63777d014 nodeName:}" failed. No retries permitted until 2026-09-19 21:21:22.68827057 +0000 UTC m=+38.272114749 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "nixconf" (UniqueName: "kubernetes.io/configmap/25f2f7ac-a091-4b45-8d23-efa63777d014-nixconf") pod "nix-grpc-farm-worker-x86-64-linux-5cdbdb595-lxl7f" (UID: "25f2f7ac-a091-4b45-8d23-efa63777d014") : configmap "nix-grpc-farm-worker-x86-64-linux" not found machine # [ 55.604145] k3s[873]: E0919 21:21:21.688332 873 secret.go:189] Couldn't get secret farm/nix-grpc-farm-oidc: secret "nix-grpc-farm-oidc" not found machine # [ 55.604771] k3s[873]: E0919 21:21:21.688379 873 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60ff9873-7c18-45d4-82a7-9263602acdbb-oidc podName:60ff9873-7c18-45d4-82a7-9263602acdbb nodeName:}" failed. No retries permitted until 2026-09-19 21:21:22.688368348 +0000 UTC m=+38.272212806 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "oidc" (UniqueName: "kubernetes.io/secret/60ff9873-7c18-45d4-82a7-9263602acdbb-oidc") pod "nix-grpc-farm-worker-x86-64-linux-5cdbdb595-qlcmg" (UID: "60ff9873-7c18-45d4-82a7-9263602acdbb") : secret "nix-grpc-farm-oidc" not found machine # [ 55.763449] k3s[873]: E0919 21:21:21.853996 873 status.go:71] "Unhandled Error" err="apiserver received an error that is not an metav1.Status: &errors.errorString{s:\"failed to create typed patch object (farm/nix-grpc-farm-scheduler; /v1, Kind=Service): .spec.selector.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0}\"}: failed to create typed patch object (farm/nix-grpc-farm-scheduler; /v1, Kind=Service): .spec.selector.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0}" logger="UnhandledError" machine # [ 55.770613] k3s[873]: I0919 21:21:21.861193 873 alloc.go:329] "allocated clusterIPs" service="farm/nix-grpc-farm" clusterIPs={"IPv4":"10.43.76.206"} machine # [ 55.781935] k3s[873]: E0919 21:21:21.872486 873 status.go:71] "Unhandled Error" err=< machine # [ 55.782637] k3s[873]: apiserver received an error that is not an metav1.Status: &errors.errorString{s:"failed to create typed patch object (farm/nix-grpc-farm-scheduler; apps/v1, Kind=Deployment): errors:\n .spec.selector.matchLabels.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0}\n .spec.template.metadata.labels.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0}"}: failed to create typed patch object (farm/nix-grpc-farm-scheduler; apps/v1, Kind=Deployment): errors: machine # [ 55.782943] k3s[873]: .spec.selector.matchLabels.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0} machine # [ 55.783379] k3s[873]: .spec.template.metadata.labels.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0} machine # [ 55.784329] k3s[873]: > logger="UnhandledError" machine # [ 55.828917] k3s[873]: E0919 21:21:21.918756 873 status_manager.go:1045] "Failed to get status for pod" err="pods \"nix-grpc-farm-lb-688d8d547b-j7c7c\" is forbidden: User \"system:node:machine\" cannot get resource \"pods\" in API group \"\" in the namespace \"farm\": no relationship found between node 'machine' and this object" podUID="905b88b7-2056-415a-bf62-9e1a86f7376d" pod="farm/nix-grpc-farm-lb-688d8d547b-j7c7c" machine # [ 55.829184] systemd[1]: cri-containerd-55f4b790498984e296d0abfd180da62e8fafb625d150d3a71861715a3bcee531.scope: Deactivated successfully. machine # [ 55.829899] systemd[1]: cri-containerd-55f4b790498984e296d0abfd180da62e8fafb625d150d3a71861715a3bcee531.scope: Consumed 864ms CPU time over 1.550s wall clock time, 37.2M memory peak, 3.3M incoming IP traffic, 198.2K outgoing IP traffic. machine # [ 55.844074] systemd[1]: Created slice libcontainer container kubepods-besteffort-pod905b88b7_2056_415a_bf62_9e1a86f7376d.slice. machine # [ 55.858585] systemd[1]: Created slice libcontainer container kubepods-besteffort-pod8213252c_bf3a_4ce0_bf80_5872e309ad9f.slice. machine # [ 55.878632] systemd[1]: Created slice libcontainer container kubepods-besteffort-podaaf0adb9_7e3a_4c6f_b115_3b5de76ef59f.slice. machine # [ 55.954774] k3s[873]: I0919 21:21:22.045159 873 scope.go:122] "RemoveContainer" containerID="3a03c71aeaedd7de915ca4e3700adc600f7daf5bce78f39ec3fb6960270348ba" machine # [ 55.971955] systemd[1]: Removed slice libcontainer container kubepods-besteffort-pod6dc8f60f_caca_4d20_b10f_d00881a93074.slice. machine # [ 55.972616] systemd[1]: kubepods-besteffort-pod6dc8f60f_caca_4d20_b10f_d00881a93074.slice: Consumed 161ms CPU time over 7.270s wall clock time, 15.5M memory peak, 3.9K incoming IP traffic, 4.9K outgoing IP traffic. machine # [ 55.995885] k3s[873]: I0919 21:21:22.086043 873 scope.go:122] "RemoveContainer" containerID="3a03c71aeaedd7de915ca4e3700adc600f7daf5bce78f39ec3fb6960270348ba" machine # [ 55.999739] k3s[873]: E0919 21:21:22.090296 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"3a03c71aeaedd7de915ca4e3700adc600f7daf5bce78f39ec3fb6960270348ba\": not found" containerID="3a03c71aeaedd7de915ca4e3700adc600f7daf5bce78f39ec3fb6960270348ba" machine # [ 56.000136] k3s[873]: I0919 21:21:22.090356 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"3a03c71aeaedd7de915ca4e3700adc600f7daf5bce78f39ec3fb6960270348ba"} err="failed to get container status \"3a03c71aeaedd7de915ca4e3700adc600f7daf5bce78f39ec3fb6960270348ba\": rpc error: code = NotFound desc = an error occurred when try to find container \"3a03c71aeaedd7de915ca4e3700adc600f7daf5bce78f39ec3fb6960270348ba\": not found" machine # [ 56.000552] k3s[873]: I0919 21:21:22.090596 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"build\" (UniqueName: \"kubernetes.io/empty-dir/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-build\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-nz2rf\" (UID: \"aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-nz2rf" machine # [ 56.001598] k3s[873]: I0919 21:21:22.090631 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/905b88b7-2056-415a-bf62-9e1a86f7376d-tmp\") pod \"nix-grpc-farm-lb-688d8d547b-j7c7c\" (UID: \"905b88b7-2056-415a-bf62-9e1a86f7376d\") " pod="farm/nix-grpc-farm-lb-688d8d547b-j7c7c" machine # [ 56.004113] k3s[873]: I0919 21:21:22.090657 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-ca\" (UniqueName: \"kubernetes.io/secret/905b88b7-2056-415a-bf62-9e1a86f7376d-tls-ca\") pod \"nix-grpc-farm-lb-688d8d547b-j7c7c\" (UID: \"905b88b7-2056-415a-bf62-9e1a86f7376d\") " pod="farm/nix-grpc-farm-lb-688d8d547b-j7c7c" machine # [ 56.005087] k3s[873]: I0919 21:21:22.092294 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"niks3-token\" (UniqueName: \"kubernetes.io/projected/8213252c-bf3a-4ce0-bf80-5872e309ad9f-niks3-token\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-98fbp\" (UID: \"8213252c-bf3a-4ce0-bf80-5872e309ad9f\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-98fbp" machine # [ 56.005890] k3s[873]: I0919 21:21:22.092662 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls\" (UniqueName: \"kubernetes.io/secret/8213252c-bf3a-4ce0-bf80-5872e309ad9f-tls\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-98fbp\" (UID: \"8213252c-bf3a-4ce0-bf80-5872e309ad9f\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-98fbp" machine # [ 56.006571] k3s[873]: I0919 21:21:22.092732 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nixconf\" (UniqueName: \"kubernetes.io/configmap/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-nixconf\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-nz2rf\" (UID: \"aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-nz2rf" machine # [ 56.010869] k3s[873]: I0919 21:21:22.092763 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nix\" (UniqueName: \"kubernetes.io/empty-dir/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-nix\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-nz2rf\" (UID: \"aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-nz2rf" machine # [ 56.011893] k3s[873]: I0919 21:21:22.092830 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls\" (UniqueName: \"kubernetes.io/secret/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-tls\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-nz2rf\" (UID: \"aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-nz2rf" machine # [ 56.012537] k3s[873]: I0919 21:21:22.092946 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49x9m\" (UniqueName: \"kubernetes.io/projected/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-kube-api-access-49x9m\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-nz2rf\" (UID: \"aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-nz2rf" machine # [ 56.014350] k3s[873]: I0919 21:21:22.093018 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-lb\" (UniqueName: \"kubernetes.io/secret/905b88b7-2056-415a-bf62-9e1a86f7376d-tls-lb\") pod \"nix-grpc-farm-lb-688d8d547b-j7c7c\" (UID: \"905b88b7-2056-415a-bf62-9e1a86f7376d\") " pod="farm/nix-grpc-farm-lb-688d8d547b-j7c7c" machine # [ 56.015066] k3s[873]: I0919 21:21:22.093045 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shnnc\" (UniqueName: \"kubernetes.io/projected/8213252c-bf3a-4ce0-bf80-5872e309ad9f-kube-api-access-shnnc\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-98fbp\" (UID: \"8213252c-bf3a-4ce0-bf80-5872e309ad9f\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-98fbp" machine # [ 56.018891] k3s[873]: I0919 21:21:22.093100 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"niks3-token\" (UniqueName: \"kubernetes.io/projected/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-niks3-token\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-nz2rf\" (UID: \"aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-nz2rf" machine # [ 56.024185] k3s[873]: I0919 21:21:22.093176 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nix\" (UniqueName: \"kubernetes.io/empty-dir/8213252c-bf3a-4ce0-bf80-5872e309ad9f-nix\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-98fbp\" (UID: \"8213252c-bf3a-4ce0-bf80-5872e309ad9f\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-98fbp" machine # [ 56.076331] k3s[873]: I0919 21:21:22.093202 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"build\" (UniqueName: \"kubernetes.io/empty-dir/8213252c-bf3a-4ce0-bf80-5872e309ad9f-build\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-98fbp\" (UID: \"8213252c-bf3a-4ce0-bf80-5872e309ad9f\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-98fbp" machine # [ 56.076946] k3s[873]: I0919 21:21:22.093226 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/8213252c-bf3a-4ce0-bf80-5872e309ad9f-ca\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-98fbp\" (UID: \"8213252c-bf3a-4ce0-bf80-5872e309ad9f\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-98fbp" machine # [ 56.077610] k3s[873]: I0919 21:21:22.093282 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc\" (UniqueName: \"kubernetes.io/secret/8213252c-bf3a-4ce0-bf80-5872e309ad9f-oidc\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-98fbp\" (UID: \"8213252c-bf3a-4ce0-bf80-5872e309ad9f\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-98fbp" machine # [ 56.079613] k3s[873]: I0919 21:21:22.093325 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/905b88b7-2056-415a-bf62-9e1a86f7376d-config\") pod \"nix-grpc-farm-lb-688d8d547b-j7c7c\" (UID: \"905b88b7-2056-415a-bf62-9e1a86f7376d\") " pod="farm/nix-grpc-farm-lb-688d8d547b-j7c7c" machine # [ 56.080558] k3s[873]: I0919 21:21:22.093474 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nixconf\" (UniqueName: \"kubernetes.io/configmap/8213252c-bf3a-4ce0-bf80-5872e309ad9f-nixconf\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-98fbp\" (UID: \"8213252c-bf3a-4ce0-bf80-5872e309ad9f\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-98fbp" machine # [ 56.081181] k3s[873]: I0919 21:21:22.093503 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-ca\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-nz2rf\" (UID: \"aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-nz2rf" machine # [ 56.081616] k3s[873]: I0919 21:21:22.093562 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc\" (UniqueName: \"kubernetes.io/secret/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-oidc\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-nz2rf\" (UID: \"aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-nz2rf" machine # [ 56.083086] k3s[873]: I0919 21:21:22.171131 873 scope.go:122] "RemoveContainer" containerID="55f4b790498984e296d0abfd180da62e8fafb625d150d3a71861715a3bcee531" machine # [ 56.086585] k3s[873]: I0919 21:21:22.177157 873 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-lxl7f" podStartSLOduration=7.403846992 podStartE2EDuration="8.177137731s" totalImagesPullingTime="0s" totalInitContainerRuntime="773.290739ms" isStatefulPod=false podCreationTimestamp="2026-09-19 21:21:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 21:21:22.086584043 +0000 UTC m=+37.670428780" watchObservedRunningTime="2026-09-19 21:21:22.177137731 +0000 UTC m=+37.760982189" machine # [ 56.128470] systemd[1]: cri-containerd-b9fe605bc08a3583d8bec5e4d011058900346272fc99203bcb02d07750e59b20.scope: Deactivated successfully. machine # [ 56.184126] systemd[1]: cri-containerd-19944217fa890aec1acc35879e3500db1cc3f58eadc2bade0b1f11a91d1ccadc.scope: Deactivated successfully. machine # [ 56.196863] dhcpcd[673]: veth3ce56192: soliciting an IPv6 router machine # [ 56.216115] systemd[1]: cri-containerd-5a44891c07ab9df573bfd747dde766f80400ac7aafa9567b3f9ff7670301ed51.scope: Deactivated successfully. machine # [ 56.216572] systemd[1]: cri-containerd-5a44891c07ab9df573bfd747dde766f80400ac7aafa9567b3f9ff7670301ed51.scope: Consumed 108ms CPU time over 1.172s wall clock time, 10.7M memory peak, 1.6M read from disk. machine # [ 56.229285] k3s[873]: I0919 21:21:22.319807 873 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-qlcmg" podStartSLOduration=7.641908253 podStartE2EDuration="8.319777368s" totalImagesPullingTime="0s" totalInitContainerRuntime="677.869115ms" isStatefulPod=false podCreationTimestamp="2026-09-19 21:21:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 21:21:22.244821193 +0000 UTC m=+37.828665651" watchObservedRunningTime="2026-09-19 21:21:22.319777368 +0000 UTC m=+37.903621826" machine # [ 56.243382] systemd[1]: Started libcontainer container 1b940306e86be7c1ea430586108c4d8105721acde29fc5373cf6ae9179d0853f. machine # [ 56.281245] systemd[1]: cri-containerd-98e6c45ffc9787574eed8892d739db6a99981cf79a86b755016c5aafa5d534bb.scope: Deactivated successfully. machine # [ 56.282160] systemd[1]: cri-containerd-98e6c45ffc9787574eed8892d739db6a99981cf79a86b755016c5aafa5d534bb.scope: Consumed 88ms CPU time over 1.218s wall clock time, 24.5M memory peak, 14.6M read from disk. machine # [ 56.453155] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-19944217fa890aec1acc35879e3500db1cc3f58eadc2bade0b1f11a91d1ccadc-rootfs.mount: Deactivated successfully. machine # [ 56.454729] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-b9fe605bc08a3583d8bec5e4d011058900346272fc99203bcb02d07750e59b20-rootfs.mount: Deactivated successfully. machine # [ 56.455564] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-98e6c45ffc9787574eed8892d739db6a99981cf79a86b755016c5aafa5d534bb-rootfs.mount: Deactivated successfully. machine # [ 56.457799] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-5a44891c07ab9df573bfd747dde766f80400ac7aafa9567b3f9ff7670301ed51-rootfs.mount: Deactivated successfully. machine # [ 56.458877] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-55f4b790498984e296d0abfd180da62e8fafb625d150d3a71861715a3bcee531-rootfs.mount: Deactivated successfully. machine # [ 56.459599] systemd[1]: var-lib-kubelet-pods-6dc8f60f\x2dcaca\x2d4d20\x2db10f\x2dd00881a93074-volumes-kubernetes.io\x7esecret-tls\x2dlb.mount: Deactivated successfully. machine # [ 56.460685] systemd[1]: var-lib-kubelet-pods-6dc8f60f\x2dcaca\x2d4d20\x2db10f\x2dd00881a93074-volumes-kubernetes.io\x7esecret-tls\x2dca.mount: Deactivated successfully. machine # [ 56.570697] (udev-worker)[2962]: Network interface NamePolicy= disabled on kernel command line. machine # [ 56.728771] cni0: port 4(vethf781df3a) entered blocking state machine # [ 56.730209] cni0: port 4(vethf781df3a) entered disabled state machine # [ 56.731623] vethf781df3a: entered allmulticast mode machine # [ 56.733059] vethf781df3a: entered promiscuous mode machine # [ 56.734529] cni0: port 6(veth199282f3) entered blocking state machine # [ 56.736145] cni0: port 6(veth199282f3) entered disabled state machine # [ 56.737490] veth199282f3: entered allmulticast mode machine # [ 56.738723] veth199282f3: entered promiscuous mode machine # [ 56.598752] (udev-worker)[2960]: Network interface NamePolicy= disabled on kernel command line. machine # [ 56.747887] cni0: port 7(veth8d576c36) entered blocking state machine # [ 56.751087] cni0: port 7(veth8d576c36) entered disabled state machine # [ 56.754037] veth8d576c36: entered allmulticast mode machine # [ 56.755765] veth8d576c36: entered promiscuous mode machine # [ 56.762258] cni0: port 4(vethf781df3a) entered blocking state machine # [ 56.763283] cni0: port 4(vethf781df3a) entered forwarding state machine # [ 56.766291] cni0: port 6(veth199282f3) entered blocking state machine # [ 56.767652] cni0: port 6(veth199282f3) entered forwarding state machine # [ 56.787490] cni0: port 7(veth8d576c36) entered blocking state machine # [ 56.788576] cni0: port 7(veth8d576c36) entered forwarding state machine # [ 56.676545] dhcpcd[673]: vethf781df3a: waiting for carrier machine # [ 56.704627] dhcpcd[673]: veth199282f3: IAID 01:b9:5e:3c machine # [ 56.705049] dhcpcd[673]: veth199282f3: adding address fe80::e4cf:1ff:feb9:5e3c machine # [ 56.729521] dhcpcd[673]: veth8d576c36: IAID b9:aa:7a:95 machine # [ 56.729745] dhcpcd[673]: veth8d576c36: adding address fe80::78b8:b9ff:feaa:7a95 machine # [ 56.730603] dhcpcd[673]: veth8d576c36: carrier lost machine # [ 56.786438] dhcpcd[673]: veth8d576c36: deleting address fe80::78b8:b9ff:feaa:7a95 machine # [ 56.788403] dhcpcd[673]: vethf781df3a: carrier acquired machine # [ 56.797840] k3s[873]: I0919 21:21:22.888431 873 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6dc8f60f-caca-4d20-b10f-d00881a93074" path="/var/lib/kubelet/pods/6dc8f60f-caca-4d20-b10f-d00881a93074/volumes" machine # [ 56.804089] dhcpcd[673]: vethf781df3a: IAID 04:c1:f5:41 machine # [ 56.806845] dhcpcd[673]: vethf781df3a: adding address fe80::408e:4ff:fec1:f541 machine # [ 56.808411] dhcpcd[673]: veth8d576c36: carrier acquired machine # [ 56.810417] systemd[1]: Started libcontainer container 4817213430471689792d913034415f3aa589c81b05558f32eca49ba49a1a6407. machine # [ 56.816044] systemd[1]: Started libcontainer container e575d61ca29ea2b0f80c0232693e3327015ed49af632c629315081600eb9ab7b. machine # [ 56.820685] dhcpcd[673]: veth8d576c36: IAID b9:aa:7a:95 machine # [ 56.822097] dhcpcd[673]: veth8d576c36: adding address fe80::78b8:b9ff:feaa:7a95 machine # [ 56.839346] systemd[1]: Started libcontainer container 1f681b5bd67093519df4bac0f6e7b1359c2f5747762000c5b8b9254ba201a445. machine # error: object has been deleted machine # [ 56.987437] systemd[1]: Started libcontainer container d703341d71355fc788740992c9f3e569e743a2efe87e3fc729a087cd34f6f5e3. machine # [ 57.000959] systemd[1]: Started libcontainer container 3cb63c4299487150834276786f400583c95bcb60c9c6f96e1b60c941aaa9ac9b. machine # [ 57.007651] dhcpcd[673]: vethdded1014: no IPv6 Routers available machine # [ 57.012532] systemd[1]: Started libcontainer container 5721a1ae2b7029b2e1ae1879748350c5fdd22e6504f8c5fecfadbda3d9fb42b3. machine # [ 57.172359] dhcpcd[673]: veth199282f3: soliciting a DHCP lease machine # [ 57.184151] systemd[1]: cri-containerd-d703341d71355fc788740992c9f3e569e743a2efe87e3fc729a087cd34f6f5e3.scope: Deactivated successfully. machine # [ 57.185879] systemd[1]: cri-containerd-d703341d71355fc788740992c9f3e569e743a2efe87e3fc729a087cd34f6f5e3.scope: Consumed 82ms CPU time over 195ms wall clock time, 43.8M memory peak, 16K read from disk, 37.7M written to disk. machine # [ 57.225123] systemd[1]: cri-containerd-3cb63c4299487150834276786f400583c95bcb60c9c6f96e1b60c941aaa9ac9b.scope: Deactivated successfully. machine # [ 57.225913] systemd[1]: cri-containerd-3cb63c4299487150834276786f400583c95bcb60c9c6f96e1b60c941aaa9ac9b.scope: Consumed 103ms CPU time over 223ms wall clock time, 67.8M memory peak, 12K read from disk, 26.8M written to disk. machine # [ 57.452645] systemd[1]: cri-containerd-e575d61ca29ea2b0f80c0232693e3327015ed49af632c629315081600eb9ab7b.scope: Deactivated successfully. machine # [ 57.458850] systemd[1]: cri-containerd-1f681b5bd67093519df4bac0f6e7b1359c2f5747762000c5b8b9254ba201a445.scope: Deactivated successfully. machine # [ 57.461548] systemd[1]: cri-containerd-437ba7a736f8ea8a4384acab8a15546a6d8720bfce0a457ab9f302bc4c5c512c.scope: Deactivated successfully. machine # [ 57.465615] systemd[1]: cri-containerd-437ba7a736f8ea8a4384acab8a15546a6d8720bfce0a457ab9f302bc4c5c512c.scope: Consumed 86ms CPU time over 2.172s wall clock time, 14.8M memory peak, 7.4M read from disk, 1.1K incoming IP traffic, 890B outgoing IP traffic. machine # [ 57.490930] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-1f681b5bd67093519df4bac0f6e7b1359c2f5747762000c5b8b9254ba201a445-rootfs.mount: Deactivated successfully. machine # [ 57.499363] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-e575d61ca29ea2b0f80c0232693e3327015ed49af632c629315081600eb9ab7b-rootfs.mount: Deactivated successfully. machine # [ 57.526634] systemd[1]: cri-containerd-0fe734dda2d1972b0181decd0c781a4332e29db73eaf48e3a3bc74a9693006f4.scope: Deactivated successfully. machine # [ 57.527346] systemd[1]: cri-containerd-0fe734dda2d1972b0181decd0c781a4332e29db73eaf48e3a3bc74a9693006f4.scope: Consumed 90ms CPU time over 2.216s wall clock time, 32.4M memory peak, 23.7M read from disk, 1.1K incoming IP traffic, 890B outgoing IP traffic. machine # [ 57.550380] dhcpcd[673]: veth8d576c36: soliciting a DHCP lease machine # [ 57.552679] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-437ba7a736f8ea8a4384acab8a15546a6d8720bfce0a457ab9f302bc4c5c512c-rootfs.mount: Deactivated successfully. machine # [ 57.557527] systemd[1]: run-k3s-containerd-io.containerd.grpc.v1.cri-sandboxes-1f681b5bd67093519df4bac0f6e7b1359c2f5747762000c5b8b9254ba201a445-shm.mount: Deactivated successfully. machine # [ 57.604975] systemd[1]: cri-containerd-bcb2791eb5d2554ee32631ebd16ae41333bac33236f68654596aa746b078c6f6.scope: Deactivated successfully. machine # [ 57.808885] cni0: port 4(vethf781df3a) entered disabled state machine # [ 57.663046] dhcpcd[673]: vethf781df3a: carrier lost machine # [ 57.812179] vethf781df3a (unregistering): left allmulticast mode machine # [ 57.813401] vethf781df3a (unregistering): left promiscuous mode machine # [ 57.815322] cni0: port 4(vethf781df3a) entered disabled state machine # [ 57.819106] cni0: port 7(veth8d576c36) entered disabled state machine # [ 57.821968] veth8d576c36 (unregistering): left allmulticast mode machine # [ 57.824410] veth8d576c36 (unregistering): left promiscuous mode machine # [ 57.827811] cni0: port 7(veth8d576c36) entered disabled state machine # [ 57.680365] systemd[1]: cri-containerd-1c27627b20769404a4bf5515a1dad2481a7af72a25c05938e475ef9f1d46575d.scope: Deactivated successfully. machine # [ 57.738210] dhcpcd[673]: vethf781df3a: deleting address fe80::408e:4ff:fec1:f541 machine # [ 57.740104] dhcpcd[673]: dhcp_readbpf: unexpected event 0x0100 machine # [ 57.741332] dhcpcd[673]: dhcp_readbpf: veth8d576c36: Network is down machine # [ 57.743345] dhcpcd[673]: veth8d576c36: carrier lost machine # [ 57.820653] dhcpcd[673]: veth8d576c36: deleting address fe80::78b8:b9ff:feaa:7a95 machine # [ 57.998912] cni0: port 5(veth66585ebe) entered disabled state machine # [ 58.001532] veth66585ebe (unregistering): left allmulticast mode machine # [ 58.002930] veth66585ebe (unregistering): left promiscuous mode machine # [ 58.005264] cni0: port 5(veth66585ebe) entered disabled state machine # [ 57.875979] k3s[873]: I0919 21:21:23.966238 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-kube-api-access-49x9m\" (UniqueName: \"kubernetes.io/projected/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-kube-api-access-49x9m\") pod \"aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f\" (UID: \"aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f\") " machine # [ 57.876213] k3s[873]: I0919 21:21:23.966300 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-ca\" (UniqueName: \"kubernetes.io/secret/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-ca\") pod \"aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f\" (UID: \"aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f\") " machine # [ 57.876378] k3s[873]: I0919 21:21:23.966333 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8213252c-bf3a-4ce0-bf80-5872e309ad9f-nixconf\" (UniqueName: \"kubernetes.io/configmap/8213252c-bf3a-4ce0-bf80-5872e309ad9f-nixconf\") pod \"8213252c-bf3a-4ce0-bf80-5872e309ad9f\" (UID: \"8213252c-bf3a-4ce0-bf80-5872e309ad9f\") " machine # [ 57.877066] k3s[873]: I0919 21:21:23.966367 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-nixconf\" (UniqueName: \"kubernetes.io/configmap/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-nixconf\") pod \"aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f\" (UID: \"aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f\") " machine # [ 57.877611] k3s[873]: I0919 21:21:23.966402 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8213252c-bf3a-4ce0-bf80-5872e309ad9f-nix\" (UniqueName: \"kubernetes.io/empty-dir/8213252c-bf3a-4ce0-bf80-5872e309ad9f-nix\") pod \"8213252c-bf3a-4ce0-bf80-5872e309ad9f\" (UID: \"8213252c-bf3a-4ce0-bf80-5872e309ad9f\") " machine # [ 57.878538] k3s[873]: I0919 21:21:23.966459 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8213252c-bf3a-4ce0-bf80-5872e309ad9f-niks3-token\" (UniqueName: \"kubernetes.io/projected/8213252c-bf3a-4ce0-bf80-5872e309ad9f-niks3-token\") pod \"8213252c-bf3a-4ce0-bf80-5872e309ad9f\" (UID: \"8213252c-bf3a-4ce0-bf80-5872e309ad9f\") " machine # [ 57.879524] k3s[873]: I0919 21:21:23.966497 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8213252c-bf3a-4ce0-bf80-5872e309ad9f-ca\" (UniqueName: \"kubernetes.io/secret/8213252c-bf3a-4ce0-bf80-5872e309ad9f-ca\") pod \"8213252c-bf3a-4ce0-bf80-5872e309ad9f\" (UID: \"8213252c-bf3a-4ce0-bf80-5872e309ad9f\") " machine # [ 57.880691] k3s[873]: I0919 21:21:23.966527 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-nix\" (UniqueName: \"kubernetes.io/empty-dir/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-nix\") pod \"aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f\" (UID: \"aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f\") " machine # [ 57.881890] k3s[873]: I0919 21:21:23.966579 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-tls\" (UniqueName: \"kubernetes.io/secret/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-tls\") pod \"aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f\" (UID: \"aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f\") " machine # [ 57.883080] k3s[873]: I0919 21:21:23.966610 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-oidc\" (UniqueName: \"kubernetes.io/secret/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-oidc\") pod \"aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f\" (UID: \"aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f\") " machine # [ 57.883755] k3s[873]: I0919 21:21:23.966641 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-build\" (UniqueName: \"kubernetes.io/empty-dir/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-build\") pod \"aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f\" (UID: \"aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f\") " machine # [ 57.884610] k3s[873]: I0919 21:21:23.966696 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8213252c-bf3a-4ce0-bf80-5872e309ad9f-kube-api-access-shnnc\" (UniqueName: \"kubernetes.io/projected/8213252c-bf3a-4ce0-bf80-5872e309ad9f-kube-api-access-shnnc\") pod \"8213252c-bf3a-4ce0-bf80-5872e309ad9f\" (UID: \"8213252c-bf3a-4ce0-bf80-5872e309ad9f\") " machine # [ 57.904661] k3s[873]: I0919 21:21:23.995250 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8213252c-bf3a-4ce0-bf80-5872e309ad9f-oidc\" (UniqueName: \"kubernetes.io/secret/8213252c-bf3a-4ce0-bf80-5872e309ad9f-oidc\") pod \"8213252c-bf3a-4ce0-bf80-5872e309ad9f\" (UID: \"8213252c-bf3a-4ce0-bf80-5872e309ad9f\") " machine # [ 57.906055] k3s[873]: I0919 21:21:23.996157 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-niks3-token\" (UniqueName: \"kubernetes.io/projected/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-niks3-token\") pod \"aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f\" (UID: \"aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f\") " machine # [ 57.906652] k3s[873]: I0919 21:21:23.996202 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8213252c-bf3a-4ce0-bf80-5872e309ad9f-build\" (UniqueName: \"kubernetes.io/empty-dir/8213252c-bf3a-4ce0-bf80-5872e309ad9f-build\") pod \"8213252c-bf3a-4ce0-bf80-5872e309ad9f\" (UID: \"8213252c-bf3a-4ce0-bf80-5872e309ad9f\") " machine # [ 57.907136] k3s[873]: I0919 21:21:23.996242 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8213252c-bf3a-4ce0-bf80-5872e309ad9f-tls\" (UniqueName: \"kubernetes.io/secret/8213252c-bf3a-4ce0-bf80-5872e309ad9f-tls\") pod \"8213252c-bf3a-4ce0-bf80-5872e309ad9f\" (UID: \"8213252c-bf3a-4ce0-bf80-5872e309ad9f\") " machine # [ 57.907581] k3s[873]: I0919 21:21:23.967939 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8213252c-bf3a-4ce0-bf80-5872e309ad9f-nixconf" pod "8213252c-bf3a-4ce0-bf80-5872e309ad9f" (UID: "8213252c-bf3a-4ce0-bf80-5872e309ad9f"). InnerVolumeSpecName "nixconf". PluginName "kubernetes.io/configmap", VolumeGIDValue "" machine # [ 57.908097] k3s[873]: I0919 21:21:23.968172 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-nixconf" pod "aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f" (UID: "aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f"). InnerVolumeSpecName "nixconf". PluginName "kubernetes.io/configmap", VolumeGIDValue "" machine # [ 57.908552] k3s[873]: I0919 21:21:23.969883 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-build" pod "aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f" (UID: "aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f"). InnerVolumeSpecName "build". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 57.911660] k3s[873]: I0919 21:21:23.995035 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8213252c-bf3a-4ce0-bf80-5872e309ad9f-nix" pod "8213252c-bf3a-4ce0-bf80-5872e309ad9f" (UID: "8213252c-bf3a-4ce0-bf80-5872e309ad9f"). InnerVolumeSpecName "nix". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 57.926239] dhcpcd[673]: vethf781df3a: removing interface machine # [ 57.974707] dhcpcd[673]: dhcp_readbpf: unexpected event 0x0100 machine # [ 57.974932] dhcpcd[673]: dhcp_readbpf: veth66585ebe: Network is down machine # [ 57.976697] k3s[873]: I0919 21:21:24.066508 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-oidc" pod "aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f" (UID: "aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f"). InnerVolumeSpecName "oidc". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 57.977081] k3s[873]: I0919 21:21:24.066750 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8213252c-bf3a-4ce0-bf80-5872e309ad9f-niks3-token" pod "8213252c-bf3a-4ce0-bf80-5872e309ad9f" (UID: "8213252c-bf3a-4ce0-bf80-5872e309ad9f"). InnerVolumeSpecName "niks3-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" machine # [ 57.978790] k3s[873]: I0919 21:21:24.066859 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-ca" pod "aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f" (UID: "aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f"). InnerVolumeSpecName "ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 57.980157] k3s[873]: I0919 21:21:24.070723 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-tls" pod "aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f" (UID: "aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f"). InnerVolumeSpecName "tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 57.981044] k3s[873]: I0919 21:21:24.071362 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8213252c-bf3a-4ce0-bf80-5872e309ad9f-build" pod "8213252c-bf3a-4ce0-bf80-5872e309ad9f" (UID: "8213252c-bf3a-4ce0-bf80-5872e309ad9f"). InnerVolumeSpecName "build". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 57.985625] k3s[873]: I0919 21:21:24.076175 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8213252c-bf3a-4ce0-bf80-5872e309ad9f-ca" pod "8213252c-bf3a-4ce0-bf80-5872e309ad9f" (UID: "8213252c-bf3a-4ce0-bf80-5872e309ad9f"). InnerVolumeSpecName "ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 58.008516] k3s[873]: I0919 21:21:24.098026 873 reconciler_common.go:299] "Volume detached for volume \"ca\" (UniqueName: \"kubernetes.io/secret/8213252c-bf3a-4ce0-bf80-5872e309ad9f-ca\") on node \"machine\" DevicePath \"\"" machine # [ 58.008991] k3s[873]: I0919 21:21:24.098056 873 reconciler_common.go:299] "Volume detached for volume \"tls\" (UniqueName: \"kubernetes.io/secret/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-tls\") on node \"machine\" DevicePath \"\"" machine # [ 58.009671] k3s[873]: I0919 21:21:24.098072 873 reconciler_common.go:299] "Volume detached for volume \"oidc\" (UniqueName: \"kubernetes.io/secret/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-oidc\") on node \"machine\" DevicePath \"\"" machine # [ 58.011224] k3s[873]: I0919 21:21:24.098710 873 reconciler_common.go:299] "Volume detached for volume \"build\" (UniqueName: \"kubernetes.io/empty-dir/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-build\") on node \"machine\" DevicePath \"\"" machine # [ 58.012076] k3s[873]: I0919 21:21:24.098729 873 reconciler_common.go:299] "Volume detached for volume \"build\" (UniqueName: \"kubernetes.io/empty-dir/8213252c-bf3a-4ce0-bf80-5872e309ad9f-build\") on node \"machine\" DevicePath \"\"" machine # [ 58.013615] k3s[873]: I0919 21:21:24.098742 873 reconciler_common.go:299] "Volume detached for volume \"ca\" (UniqueName: \"kubernetes.io/secret/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-ca\") on node \"machine\" DevicePath \"\"" machine # [ 58.014698] k3s[873]: I0919 21:21:24.098755 873 reconciler_common.go:299] "Volume detached for volume \"nixconf\" (UniqueName: \"kubernetes.io/configmap/8213252c-bf3a-4ce0-bf80-5872e309ad9f-nixconf\") on node \"machine\" DevicePath \"\"" machine # [ 58.015385] k3s[873]: I0919 21:21:24.098916 873 reconciler_common.go:299] "Volume detached for volume \"nixconf\" (UniqueName: \"kubernetes.io/configmap/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-nixconf\") on node \"machine\" DevicePath \"\"" machine # [ 58.016051] k3s[873]: I0919 21:21:24.098931 873 reconciler_common.go:299] "Volume detached for volume \"nix\" (UniqueName: \"kubernetes.io/empty-dir/8213252c-bf3a-4ce0-bf80-5872e309ad9f-nix\") on node \"machine\" DevicePath \"\"" machine # [ 58.016833] k3s[873]: I0919 21:21:24.098947 873 reconciler_common.go:299] "Volume detached for volume \"niks3-token\" (UniqueName: \"kubernetes.io/projected/8213252c-bf3a-4ce0-bf80-5872e309ad9f-niks3-token\") on node \"machine\" DevicePath \"\"" machine # [ 58.028931] k3s[873]: I0919 21:21:24.119458 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-nix" pod "aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f" (UID: "aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f"). InnerVolumeSpecName "nix". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 58.045072] k3s[873]: I0919 21:21:24.135634 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-niks3-token" pod "aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f" (UID: "aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f"). InnerVolumeSpecName "niks3-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" machine # [ 58.045530] k3s[873]: I0919 21:21:24.135777 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8213252c-bf3a-4ce0-bf80-5872e309ad9f-tls" pod "8213252c-bf3a-4ce0-bf80-5872e309ad9f" (UID: "8213252c-bf3a-4ce0-bf80-5872e309ad9f"). InnerVolumeSpecName "tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 58.047413] k3s[873]: I0919 21:21:24.137990 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8213252c-bf3a-4ce0-bf80-5872e309ad9f-kube-api-access-shnnc" pod "8213252c-bf3a-4ce0-bf80-5872e309ad9f" (UID: "8213252c-bf3a-4ce0-bf80-5872e309ad9f"). InnerVolumeSpecName "kube-api-access-shnnc". PluginName "kubernetes.io/projected", VolumeGIDValue "" machine # [ 58.049417] k3s[873]: I0919 21:21:24.139879 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-kube-api-access-49x9m" pod "aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f" (UID: "aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f"). InnerVolumeSpecName "kube-api-access-49x9m". PluginName "kubernetes.io/projected", VolumeGIDValue "" machine # [ 58.060798] dhcpcd[673]: arp_read: unexpected event 0x0100 machine # [ 58.061921] dhcpcd[673]: arp_read: veth66585ebe: Network is down machine # [ 58.069584] k3s[873]: I0919 21:21:24.159938 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8213252c-bf3a-4ce0-bf80-5872e309ad9f-oidc" pod "8213252c-bf3a-4ce0-bf80-5872e309ad9f" (UID: "8213252c-bf3a-4ce0-bf80-5872e309ad9f"). InnerVolumeSpecName "oidc". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 58.075304] dhcpcd[673]: veth8d576c36: removing interface machine # [ 58.079987] k3s[873]: E0919 21:21:24.170555 873 status.go:71] "Unhandled Error" err="apiserver received an error that is not an metav1.Status: &errors.errorString{s:\"failed to create typed patch object (farm/nix-grpc-farm-scheduler; /v1, Kind=Service): .spec.selector.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0}\"}: failed to create typed patch object (farm/nix-grpc-farm-scheduler; /v1, Kind=Service): .spec.selector.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0}" logger="UnhandledError" machine # [ 58.092079] k3s[873]: I0919 21:21:24.182354 873 alloc.go:329] "allocated clusterIPs" service="farm/nix-grpc-farm" clusterIPs={"IPv4":"10.43.138.45"} machine # [ 58.250495] cni0: port 3(veth03230c89) entered disabled state machine # [ 58.253758] veth03230c89 (unregistering): left allmulticast mode machine # [ 58.255169] veth03230c89 (unregistering): left promiscuous mode machine # [ 58.256414] cni0: port 3(veth03230c89) entered disabled state machine # [ 58.110307] k3s[873]: E0919 21:21:24.200038 873 status.go:71] "Unhandled Error" err=< machine # [ 58.111137] k3s[873]: apiserver received an error that is not an metav1.Status: &errors.errorString{s:"failed to create typed patch object (farm/nix-grpc-farm-scheduler; apps/v1, Kind=Deployment): errors:\n .spec.selector.matchLabels.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0}\n .spec.template.metadata.labels.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0}"}: failed to create typed patch object (farm/nix-grpc-farm-scheduler; apps/v1, Kind=Deployment): errors: machine # [ 58.111827] k3s[873]: .spec.selector.matchLabels.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0} machine # [ 58.113075] k3s[873]: .spec.template.metadata.labels.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0} machine # [ 58.113758] k3s[873]: > logger="UnhandledError" machine # [ 58.114450] k3s[873]: I0919 21:21:24.204623 873 reconciler_common.go:299] "Volume detached for volume \"nix\" (UniqueName: \"kubernetes.io/empty-dir/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-nix\") on node \"machine\" DevicePath \"\"" machine # [ 58.114885] k3s[873]: I0919 21:21:24.204645 873 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shnnc\" (UniqueName: \"kubernetes.io/projected/8213252c-bf3a-4ce0-bf80-5872e309ad9f-kube-api-access-shnnc\") on node \"machine\" DevicePath \"\"" machine # [ 58.115535] k3s[873]: I0919 21:21:24.204659 873 reconciler_common.go:299] "Volume detached for volume \"oidc\" (UniqueName: \"kubernetes.io/secret/8213252c-bf3a-4ce0-bf80-5872e309ad9f-oidc\") on node \"machine\" DevicePath \"\"" machine # [ 58.115968] k3s[873]: I0919 21:21:24.204694 873 reconciler_common.go:299] "Volume detached for volume \"niks3-token\" (UniqueName: \"kubernetes.io/projected/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-niks3-token\") on node \"machine\" DevicePath \"\"" machine # [ 58.119604] k3s[873]: I0919 21:21:24.204708 873 reconciler_common.go:299] "Volume detached for volume \"tls\" (UniqueName: \"kubernetes.io/secret/8213252c-bf3a-4ce0-bf80-5872e309ad9f-tls\") on node \"machine\" DevicePath \"\"" machine # [ 58.120068] k3s[873]: I0919 21:21:24.204721 873 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49x9m\" (UniqueName: \"kubernetes.io/projected/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f-kube-api-access-49x9m\") on node \"machine\" DevicePath \"\"" machine # [ 58.141294] dhcpcd[673]: veth66585ebe: removing interface machine # [ 58.141555] dhcpcd[673]: veth66585ebe: deleting address fe80::9ce0:c5ff:fe89:46e4 machine # [ 58.156435] systemd[1]: cri-containerd-1b940306e86be7c1ea430586108c4d8105721acde29fc5373cf6ae9179d0853f.scope: Deactivated successfully. machine # [ 58.157639] systemd[1]: cri-containerd-1b940306e86be7c1ea430586108c4d8105721acde29fc5373cf6ae9179d0853f.scope: Consumed 884ms CPU time over 1.913s wall clock time, 35.1M memory peak, 3.3M incoming IP traffic, 199.2K outgoing IP traffic. machine # [ 58.182112] k3s[873]: I0919 21:21:24.270822 873 scope.go:122] "RemoveContainer" containerID="b9fe605bc08a3583d8bec5e4d011058900346272fc99203bcb02d07750e59b20" machine # [ 58.198171] dhcpcd[673]: dhcp_readbpf: unexpected event 0x0100 machine # [ 58.198677] dhcpcd[673]: dhcp_readbpf: veth03230c89: Network is down machine # [ 58.201189] dhcpcd[673]: arp_read: unexpected event 0x0100 machine # [ 58.203210] dhcpcd[673]: arp_read: veth03230c89: Network is down machine # [ 58.205318] dhcpcd[673]: veth03230c89: removing interface machine # [ 58.205666] dhcpcd[673]: veth03230c89: deleting address fe80::4c4d:ccff:fe4f:3e4 machine # [ 58.216521] k3s[873]: I0919 21:21:24.307091 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/25f2f7ac-a091-4b45-8d23-efa63777d014-kube-api-access-ggxl9\" (UniqueName: \"kubernetes.io/projected/25f2f7ac-a091-4b45-8d23-efa63777d014-kube-api-access-ggxl9\") pod \"25f2f7ac-a091-4b45-8d23-efa63777d014\" (UID: \"25f2f7ac-a091-4b45-8d23-efa63777d014\") " machine # [ 58.216972] k3s[873]: I0919 21:21:24.307155 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/25f2f7ac-a091-4b45-8d23-efa63777d014-nixconf\" (UniqueName: \"kubernetes.io/configmap/25f2f7ac-a091-4b45-8d23-efa63777d014-nixconf\") pod \"25f2f7ac-a091-4b45-8d23-efa63777d014\" (UID: \"25f2f7ac-a091-4b45-8d23-efa63777d014\") " machine # [ 58.218289] k3s[873]: I0919 21:21:24.307191 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25f2f7ac-a091-4b45-8d23-efa63777d014-build\" (UniqueName: \"kubernetes.io/empty-dir/25f2f7ac-a091-4b45-8d23-efa63777d014-build\") pod \"25f2f7ac-a091-4b45-8d23-efa63777d014\" (UID: \"25f2f7ac-a091-4b45-8d23-efa63777d014\") " machine # [ 58.219460] k3s[873]: I0919 21:21:24.307219 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25f2f7ac-a091-4b45-8d23-efa63777d014-nix\" (UniqueName: \"kubernetes.io/empty-dir/25f2f7ac-a091-4b45-8d23-efa63777d014-nix\") pod \"25f2f7ac-a091-4b45-8d23-efa63777d014\" (UID: \"25f2f7ac-a091-4b45-8d23-efa63777d014\") " machine # [ 58.220068] k3s[873]: I0919 21:21:24.310551 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25f2f7ac-a091-4b45-8d23-efa63777d014-build" pod "25f2f7ac-a091-4b45-8d23-efa63777d014" (UID: "25f2f7ac-a091-4b45-8d23-efa63777d014"). InnerVolumeSpecName "build". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 58.223731] k3s[873]: I0919 21:21:24.314310 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25f2f7ac-a091-4b45-8d23-efa63777d014-nixconf" pod "25f2f7ac-a091-4b45-8d23-efa63777d014" (UID: "25f2f7ac-a091-4b45-8d23-efa63777d014"). InnerVolumeSpecName "nixconf". PluginName "kubernetes.io/configmap", VolumeGIDValue "" machine # [ 58.251892] k3s[873]: I0919 21:21:24.342435 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25f2f7ac-a091-4b45-8d23-efa63777d014-kube-api-access-ggxl9" pod "25f2f7ac-a091-4b45-8d23-efa63777d014" (UID: "25f2f7ac-a091-4b45-8d23-efa63777d014"). InnerVolumeSpecName "kube-api-access-ggxl9". PluginName "kubernetes.io/projected", VolumeGIDValue "" machine # [ 58.256109] k3s[873]: I0919 21:21:24.346714 873 scope.go:122] "RemoveContainer" containerID="437ba7a736f8ea8a4384acab8a15546a6d8720bfce0a457ab9f302bc4c5c512c" machine # [ 58.258927] systemd[1]: Removed slice libcontainer container kubepods-besteffort-pod8213252c_bf3a_4ce0_bf80_5872e309ad9f.slice. machine # [ 58.260180] systemd[1]: kubepods-besteffort-pod8213252c_bf3a_4ce0_bf80_5872e309ad9f.slice: Consumed 96ms CPU time over 2.399s wall clock time, 44.2M memory peak, 16K read from disk, 37.7M written to disk. machine # [ 58.261671] k3s[873]: I0919 21:21:24.307248 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/25f2f7ac-a091-4b45-8d23-efa63777d014-niks3-token\" (UniqueName: \"kubernetes.io/projected/25f2f7ac-a091-4b45-8d23-efa63777d014-niks3-token\") pod \"25f2f7ac-a091-4b45-8d23-efa63777d014\" (UID: \"25f2f7ac-a091-4b45-8d23-efa63777d014\") " machine # [ 58.262997] k3s[873]: I0919 21:21:24.351194 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/25f2f7ac-a091-4b45-8d23-efa63777d014-ca\" (UniqueName: \"kubernetes.io/secret/25f2f7ac-a091-4b45-8d23-efa63777d014-ca\") pod \"25f2f7ac-a091-4b45-8d23-efa63777d014\" (UID: \"25f2f7ac-a091-4b45-8d23-efa63777d014\") " machine # [ 58.264473] k3s[873]: I0919 21:21:24.351233 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/25f2f7ac-a091-4b45-8d23-efa63777d014-oidc\" (UniqueName: \"kubernetes.io/secret/25f2f7ac-a091-4b45-8d23-efa63777d014-oidc\") pod \"25f2f7ac-a091-4b45-8d23-efa63777d014\" (UID: \"25f2f7ac-a091-4b45-8d23-efa63777d014\") " machine # [ 58.265511] k3s[873]: I0919 21:21:24.351266 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/25f2f7ac-a091-4b45-8d23-efa63777d014-tls\" (UniqueName: \"kubernetes.io/secret/25f2f7ac-a091-4b45-8d23-efa63777d014-tls\") pod \"25f2f7ac-a091-4b45-8d23-efa63777d014\" (UID: \"25f2f7ac-a091-4b45-8d23-efa63777d014\") " machine # [ 58.266642] k3s[873]: I0919 21:21:24.351357 873 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ggxl9\" (UniqueName: \"kubernetes.io/projected/25f2f7ac-a091-4b45-8d23-efa63777d014-kube-api-access-ggxl9\") on node \"machine\" DevicePath \"\"" machine # [ 58.267985] k3s[873]: I0919 21:21:24.351372 873 reconciler_common.go:299] "Volume detached for volume \"nixconf\" (UniqueName: \"kubernetes.io/configmap/25f2f7ac-a091-4b45-8d23-efa63777d014-nixconf\") on node \"machine\" DevicePath \"\"" machine # [ 58.269400] k3s[873]: I0919 21:21:24.351384 873 reconciler_common.go:299] "Volume detached for volume \"build\" (UniqueName: \"kubernetes.io/empty-dir/25f2f7ac-a091-4b45-8d23-efa63777d014-build\") on node \"machine\" DevicePath \"\"" machine # [ 58.303497] k3s[873]: I0919 21:21:24.394018 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25f2f7ac-a091-4b45-8d23-efa63777d014-oidc" pod "25f2f7ac-a091-4b45-8d23-efa63777d014" (UID: "25f2f7ac-a091-4b45-8d23-efa63777d014"). InnerVolumeSpecName "oidc". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 58.312767] k3s[873]: I0919 21:21:24.402983 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25f2f7ac-a091-4b45-8d23-efa63777d014-ca" pod "25f2f7ac-a091-4b45-8d23-efa63777d014" (UID: "25f2f7ac-a091-4b45-8d23-efa63777d014"). InnerVolumeSpecName "ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 58.314469] k3s[873]: I0919 21:21:24.405073 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25f2f7ac-a091-4b45-8d23-efa63777d014-niks3-token" pod "25f2f7ac-a091-4b45-8d23-efa63777d014" (UID: "25f2f7ac-a091-4b45-8d23-efa63777d014"). InnerVolumeSpecName "niks3-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" machine # [ 58.318370] k3s[873]: I0919 21:21:24.408842 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25f2f7ac-a091-4b45-8d23-efa63777d014-tls" pod "25f2f7ac-a091-4b45-8d23-efa63777d014" (UID: "25f2f7ac-a091-4b45-8d23-efa63777d014"). InnerVolumeSpecName "tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 58.333829] k3s[873]: I0919 21:21:24.424336 873 scope.go:122] "RemoveContainer" containerID="5a44891c07ab9df573bfd747dde766f80400ac7aafa9567b3f9ff7670301ed51" machine # [ 58.352259] systemd[1]: Created slice libcontainer container kubepods-besteffort-pod586d96c1_2512_46a6_99e0_896b90a7f605.slice. machine # [ 58.362221] k3s[873]: I0919 21:21:24.452808 873 reconciler_common.go:299] "Volume detached for volume \"niks3-token\" (UniqueName: \"kubernetes.io/projected/25f2f7ac-a091-4b45-8d23-efa63777d014-niks3-token\") on node \"machine\" DevicePath \"\"" machine # [ 58.365297] k3s[873]: I0919 21:21:24.455914 873 reconciler_common.go:299] "Volume detached for volume \"ca\" (UniqueName: \"kubernetes.io/secret/25f2f7ac-a091-4b45-8d23-efa63777d014-ca\") on node \"machine\" DevicePath \"\"" machine # [ 58.369076] systemd[1]: cri-containerd-5721a1ae2b7029b2e1ae1879748350c5fdd22e6504f8c5fecfadbda3d9fb42b3.scope: Deactivated successfully. machine # [ 58.370979] systemd[1]: cri-containerd-5721a1ae2b7029b2e1ae1879748350c5fdd22e6504f8c5fecfadbda3d9fb42b3.scope: Consumed 102ms CPU time over 1.356s wall clock time, 14.6M memory peak, 2.8K incoming IP traffic, 3K outgoing IP traffic. machine # [ 58.378459] k3s[873]: I0919 21:21:24.466735 873 reconciler_common.go:299] "Volume detached for volume \"oidc\" (UniqueName: \"kubernetes.io/secret/25f2f7ac-a091-4b45-8d23-efa63777d014-oidc\") on node \"machine\" DevicePath \"\"" machine # [ 58.384199] k3s[873]: I0919 21:21:24.466924 873 reconciler_common.go:299] "Volume detached for volume \"tls\" (UniqueName: \"kubernetes.io/secret/25f2f7ac-a091-4b45-8d23-efa63777d014-tls\") on node \"machine\" DevicePath \"\"" machine # [ 58.386885] k3s[873]: I0919 21:21:24.474328 873 scope.go:122] "RemoveContainer" containerID="793c5107f72c6c9db564477d1a8454b8cabf5ec18c5ac9c66f8a33144f6a85b6" machine # [ 58.388759] systemd[1]: Created slice libcontainer container kubepods-besteffort-pod119b9bfe_8f78_4a41_9061_7e2df29c5a85.slice. machine # [ 58.395907] k3s[873]: I0919 21:21:24.486323 873 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="farm/nix-grpc-farm-lb-688d8d547b-j7c7c" podStartSLOduration=3.486296297 podStartE2EDuration="3.486296297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 21:21:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 21:21:24.485354557 +0000 UTC m=+40.069375574" watchObservedRunningTime="2026-09-19 21:21:24.486296297 +0000 UTC m=+40.070141034" machine # [ 58.401261] systemd[1]: Created slice libcontainer container kubepods-besteffort-pod092f1a79_f45e_4ae4_90b6_babb0b28134c.slice. machine # [ 58.409205] k3s[873]: I0919 21:21:24.499786 873 scope.go:122] "RemoveContainer" containerID="b9fe605bc08a3583d8bec5e4d011058900346272fc99203bcb02d07750e59b20" machine # [ 58.414059] k3s[873]: E0919 21:21:24.504626 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"b9fe605bc08a3583d8bec5e4d011058900346272fc99203bcb02d07750e59b20\": not found" containerID="b9fe605bc08a3583d8bec5e4d011058900346272fc99203bcb02d07750e59b20" machine # [ 58.415136] k3s[873]: I0919 21:21:24.505030 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"b9fe605bc08a3583d8bec5e4d011058900346272fc99203bcb02d07750e59b20"} err="failed to get container status \"b9fe605bc08a3583d8bec5e4d011058900346272fc99203bcb02d07750e59b20\": rpc error: code = NotFound desc = an error occurred when try to find container \"b9fe605bc08a3583d8bec5e4d011058900346272fc99203bcb02d07750e59b20\": not found" machine # [ 58.415655] k3s[873]: I0919 21:21:24.505071 873 scope.go:122] "RemoveContainer" containerID="437ba7a736f8ea8a4384acab8a15546a6d8720bfce0a457ab9f302bc4c5c512c" machine # [ 58.421540] k3s[873]: E0919 21:21:24.512109 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"437ba7a736f8ea8a4384acab8a15546a6d8720bfce0a457ab9f302bc4c5c512c\": not found" containerID="437ba7a736f8ea8a4384acab8a15546a6d8720bfce0a457ab9f302bc4c5c512c" machine # [ 58.423805] systemd[1]: Removed slice libcontainer container kubepods-besteffort-podaaf0adb9_7e3a_4c6f_b115_3b5de76ef59f.slice. machine # [ 58.424714] systemd[1]: kubepods-besteffort-podaaf0adb9_7e3a_4c6f_b115_3b5de76ef59f.slice: Consumed 120ms CPU time over 2.543s wall clock time, 68.1M memory peak, 12K read from disk, 60.7M written to disk. machine # [ 58.425509] k3s[873]: I0919 21:21:24.515887 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"437ba7a736f8ea8a4384acab8a15546a6d8720bfce0a457ab9f302bc4c5c512c"} err="failed to get container status \"437ba7a736f8ea8a4384acab8a15546a6d8720bfce0a457ab9f302bc4c5c512c\": rpc error: code = NotFound desc = an error occurred when try to find container \"437ba7a736f8ea8a4384acab8a15546a6d8720bfce0a457ab9f302bc4c5c512c\": not found" machine # [ 58.426050] k3s[873]: I0919 21:21:24.515936 873 scope.go:122] "RemoveContainer" containerID="5a44891c07ab9df573bfd747dde766f80400ac7aafa9567b3f9ff7670301ed51" machine # [ 58.427706] k3s[873]: E0919 21:21:24.518311 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"5a44891c07ab9df573bfd747dde766f80400ac7aafa9567b3f9ff7670301ed51\": not found" containerID="5a44891c07ab9df573bfd747dde766f80400ac7aafa9567b3f9ff7670301ed51" machine # [ 58.441787] systemd[1]: run-k3s-containerd-io.containerd.grpc.v1.cri-sandboxes-e575d61ca29ea2b0f80c0232693e3327015ed49af632c629315081600eb9ab7b-shm.mount: Deactivated successfully. machine # [ 58.442461] systemd[1]: run-netns-cni\x2d92a3c9fd\x2d2b00\x2d4b6f\x2db87e\x2d1b8160755353.mount: Deactivated successfully. machine # [ 58.443059] systemd[1]: run-netns-cni\x2dc1705635\x2d7281\x2dc770\x2d1ce2\x2d964fc203929b.mount: Deactivated successfully. machine # [ 58.443601] systemd[1]: var-lib-kubelet-pods-8213252c\x2dbf3a\x2d4ce0\x2dbf80\x2d5872e309ad9f-volumes-kubernetes.io\x7eprojected-niks3\x2dtoken.mount: Deactivated successfully. machine # [ 58.444135] systemd[1]: var-lib-kubelet-pods-8213252c\x2dbf3a\x2d4ce0\x2dbf80\x2d5872e309ad9f-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dshnnc.mount: Deactivated successfully. machine # [ 58.444781] k3s[873]: I0919 21:21:24.534643 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"5a44891c07ab9df573bfd747dde766f80400ac7aafa9567b3f9ff7670301ed51"} err="failed to get container status \"5a44891c07ab9df573bfd747dde766f80400ac7aafa9567b3f9ff7670301ed51\": rpc error: code = NotFound desc = an error occurred when try to find container \"5a44891c07ab9df573bfd747dde766f80400ac7aafa9567b3f9ff7670301ed51\": not found" machine # [ 58.445442] k3s[873]: I0919 21:21:24.534729 873 scope.go:122] "RemoveContainer" containerID="793c5107f72c6c9db564477d1a8454b8cabf5ec18c5ac9c66f8a33144f6a85b6" machine # [ 58.445892] systemd[1]: var-lib-kubelet-pods-aaf0adb9\x2d7e3a\x2d4c6f\x2db115\x2d3b5de76ef59f-volumes-kubernetes.io\x7eprojected-niks3\x2dtoken.mount: Deactivated successfully. machine # [ 58.446411] k3s[873]: E0919 21:21:24.536492 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"793c5107f72c6c9db564477d1a8454b8cabf5ec18c5ac9c66f8a33144f6a85b6\": not found" containerID="793c5107f72c6c9db564477d1a8454b8cabf5ec18c5ac9c66f8a33144f6a85b6" machine # [ 58.446846] k3s[873]: I0919 21:21:24.536532 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"793c5107f72c6c9db564477d1a8454b8cabf5ec18c5ac9c66f8a33144f6a85b6"} err="failed to get container status \"793c5107f72c6c9db564477d1a8454b8cabf5ec18c5ac9c66f8a33144f6a85b6\": rpc error: code = NotFound desc = an error occurred when try to find container \"793c5107f72c6c9db564477d1a8454b8cabf5ec18c5ac9c66f8a33144f6a85b6\": not found" machine # [ 58.447464] k3s[873]: I0919 21:21:24.536567 873 scope.go:122] "RemoveContainer" containerID="d703341d71355fc788740992c9f3e569e743a2efe87e3fc729a087cd34f6f5e3" machine # [ 58.447938] systemd[1]: var-lib-kubelet-pods-aaf0adb9\x2d7e3a\x2d4c6f\x2db115\x2d3b5de76ef59f-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2d49x9m.mount: Deactivated successfully. machine # [ 58.448658] systemd[1]: var-lib-kubelet-pods-8213252c\x2dbf3a\x2d4ce0\x2dbf80\x2d5872e309ad9f-volumes-kubernetes.io\x7esecret-tls.mount: Deactivated successfully. machine # [ 58.449156] systemd[1]: var-lib-kubelet-pods-8213252c\x2dbf3a\x2d4ce0\x2dbf80\x2d5872e309ad9f-volumes-kubernetes.io\x7esecret-ca.mount: Deactivated successfully. machine # [ 58.449642] systemd[1]: var-lib-kubelet-pods-aaf0adb9\x2d7e3a\x2d4c6f\x2db115\x2d3b5de76ef59f-volumes-kubernetes.io\x7esecret-tls.mount: Deactivated successfully. machine # [ 58.450193] systemd[1]: var-lib-kubelet-pods-8213252c\x2dbf3a\x2d4ce0\x2dbf80\x2d5872e309ad9f-volumes-kubernetes.io\x7esecret-oidc.mount: Deactivated successfully. machine # [ 58.450690] systemd[1]: var-lib-kubelet-pods-aaf0adb9\x2d7e3a\x2d4c6f\x2db115\x2d3b5de76ef59f-volumes-kubernetes.io\x7esecret-oidc.mount: Deactivated successfully. machine # [ 58.451349] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-1b940306e86be7c1ea430586108c4d8105721acde29fc5373cf6ae9179d0853f-rootfs.mount: Deactivated successfully. machine # [ 58.451802] systemd[1]: var-lib-kubelet-pods-aaf0adb9\x2d7e3a\x2d4c6f\x2db115\x2d3b5de76ef59f-volumes-kubernetes.io\x7esecret-ca.mount: Deactivated successfully. machine # [ 58.452489] systemd[1]: var-lib-kubelet-pods-25f2f7ac\x2da091\x2d4b45\x2d8d23\x2defa63777d014-volume\x2dsubpaths-nix-gc-0.mount: Deactivated successfully. machine # [ 58.452977] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-0fe734dda2d1972b0181decd0c781a4332e29db73eaf48e3a3bc74a9693006f4-rootfs.mount: Deactivated successfully. machine # [ 58.453689] systemd[1]: var-lib-kubelet-pods-25f2f7ac\x2da091\x2d4b45\x2d8d23\x2defa63777d014-volume\x2dsubpaths-nix-nix\x2dgrpc\x2ddaemon-0.mount: Deactivated successfully. machine # [ 58.454434] systemd[1]: var-lib-kubelet-pods-25f2f7ac\x2da091\x2d4b45\x2d8d23\x2defa63777d014-volume\x2dsubpaths-nix-nix\x2ddaemon-0.mount: Deactivated successfully. machine # [ 58.457450] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-bcb2791eb5d2554ee32631ebd16ae41333bac33236f68654596aa746b078c6f6-rootfs.mount: Deactivated successfully. machine # [ 58.458208] systemd[1]: run-k3s-containerd-io.containerd.grpc.v1.cri-sandboxes-bcb2791eb5d2554ee32631ebd16ae41333bac33236f68654596aa746b078c6f6-shm.mount: Deactivated successfully. machine # [ 58.458915] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-1c27627b20769404a4bf5515a1dad2481a7af72a25c05938e475ef9f1d46575d-rootfs.mount: Deactivated successfully. machine # [ 58.459635] systemd[1]: run-k3s-containerd-io.containerd.grpc.v1.cri-sandboxes-1c27627b20769404a4bf5515a1dad2481a7af72a25c05938e475ef9f1d46575d-shm.mount: Deactivated successfully. machine # [ 58.460131] systemd[1]: run-netns-cni\x2d4710ed75\x2d6120\x2d540d\x2d1dfb\x2d57fa095f76f6.mount: Deactivated successfully. machine # [ 58.460582] systemd[1]: run-netns-cni\x2dcadd9502\x2d6eae\x2dfb64\x2d3b7d\x2d6e702ac1696f.mount: Deactivated successfully. machine # [ 58.461086] systemd[1]: var-lib-kubelet-pods-25f2f7ac\x2da091\x2d4b45\x2d8d23\x2defa63777d014-volumes-kubernetes.io\x7eprojected-niks3\x2dtoken.mount: Deactivated successfully. machine # [ 58.461859] systemd[1]: var-lib-kubelet-pods-25f2f7ac\x2da091\x2d4b45\x2d8d23\x2defa63777d014-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dggxl9.mount: Deactivated successfully. machine # [ 58.463965] systemd[1]: var-lib-kubelet-pods-25f2f7ac\x2da091\x2d4b45\x2d8d23\x2defa63777d014-volumes-kubernetes.io\x7esecret-ca.mount: Deactivated successfully. machine # [ 58.465740] systemd[1]: var-lib-kubelet-pods-25f2f7ac\x2da091\x2d4b45\x2d8d23\x2defa63777d014-volumes-kubernetes.io\x7esecret-tls.mount: Deactivated successfully. machine # [ 58.468449] k3s[873]: I0919 21:21:24.555127 873 scope.go:122] "RemoveContainer" containerID="d703341d71355fc788740992c9f3e569e743a2efe87e3fc729a087cd34f6f5e3" machine # [ 58.469404] k3s[873]: E0919 21:21:24.558232 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"d703341d71355fc788740992c9f3e569e743a2efe87e3fc729a087cd34f6f5e3\": not found" containerID="d703341d71355fc788740992c9f3e569e743a2efe87e3fc729a087cd34f6f5e3" machine # [ 58.474523] k3s[873]: I0919 21:21:24.558281 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"d703341d71355fc788740992c9f3e569e743a2efe87e3fc729a087cd34f6f5e3"} err="failed to get container status \"d703341d71355fc788740992c9f3e569e743a2efe87e3fc729a087cd34f6f5e3\": rpc error: code = NotFound desc = an error occurred when try to find container \"d703341d71355fc788740992c9f3e569e743a2efe87e3fc729a087cd34f6f5e3\": not found" machine # [ 58.478515] k3s[873]: I0919 21:21:24.558347 873 scope.go:122] "RemoveContainer" containerID="19944217fa890aec1acc35879e3500db1cc3f58eadc2bade0b1f11a91d1ccadc" machine # [ 58.529979] systemd[1]: var-lib-kubelet-pods-25f2f7ac\x2da091\x2d4b45\x2d8d23\x2defa63777d014-volumes-kubernetes.io\x7esecret-oidc.mount: Deactivated successfully. machine # [ 58.530721] k3s[873]: I0919 21:21:24.567527 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/60ff9873-7c18-45d4-82a7-9263602acdbb-tls\" (UniqueName: \"kubernetes.io/secret/60ff9873-7c18-45d4-82a7-9263602acdbb-tls\") pod \"60ff9873-7c18-45d4-82a7-9263602acdbb\" (UID: \"60ff9873-7c18-45d4-82a7-9263602acdbb\") " machine # [ 58.531376] k3s[873]: I0919 21:21:24.567611 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/60ff9873-7c18-45d4-82a7-9263602acdbb-kube-api-access-d8mgc\" (UniqueName: \"kubernetes.io/projected/60ff9873-7c18-45d4-82a7-9263602acdbb-kube-api-access-d8mgc\") pod \"60ff9873-7c18-45d4-82a7-9263602acdbb\" (UID: \"60ff9873-7c18-45d4-82a7-9263602acdbb\") " machine # [ 58.531777] k3s[873]: I0919 21:21:24.567652 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/60ff9873-7c18-45d4-82a7-9263602acdbb-nixconf\" (UniqueName: \"kubernetes.io/configmap/60ff9873-7c18-45d4-82a7-9263602acdbb-nixconf\") pod \"60ff9873-7c18-45d4-82a7-9263602acdbb\" (UID: \"60ff9873-7c18-45d4-82a7-9263602acdbb\") " machine # [ 58.532463] k3s[873]: I0919 21:21:24.567729 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60ff9873-7c18-45d4-82a7-9263602acdbb-nix\" (UniqueName: \"kubernetes.io/empty-dir/60ff9873-7c18-45d4-82a7-9263602acdbb-nix\") pod \"60ff9873-7c18-45d4-82a7-9263602acdbb\" (UID: \"60ff9873-7c18-45d4-82a7-9263602acdbb\") " machine # [ 58.532961] k3s[873]: I0919 21:21:24.567756 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/60ff9873-7c18-45d4-82a7-9263602acdbb-ca\" (UniqueName: \"kubernetes.io/secret/60ff9873-7c18-45d4-82a7-9263602acdbb-ca\") pod \"60ff9873-7c18-45d4-82a7-9263602acdbb\" (UID: \"60ff9873-7c18-45d4-82a7-9263602acdbb\") " machine # [ 58.534599] k3s[873]: I0919 21:21:24.567816 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/60ff9873-7c18-45d4-82a7-9263602acdbb-oidc\" (UniqueName: \"kubernetes.io/secret/60ff9873-7c18-45d4-82a7-9263602acdbb-oidc\") pod \"60ff9873-7c18-45d4-82a7-9263602acdbb\" (UID: \"60ff9873-7c18-45d4-82a7-9263602acdbb\") " machine # [ 58.535125] k3s[873]: I0919 21:21:24.567912 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/60ff9873-7c18-45d4-82a7-9263602acdbb-niks3-token\" (UniqueName: \"kubernetes.io/projected/60ff9873-7c18-45d4-82a7-9263602acdbb-niks3-token\") pod \"60ff9873-7c18-45d4-82a7-9263602acdbb\" (UID: \"60ff9873-7c18-45d4-82a7-9263602acdbb\") " machine # [ 58.535914] k3s[873]: I0919 21:21:24.567941 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60ff9873-7c18-45d4-82a7-9263602acdbb-build\" (UniqueName: \"kubernetes.io/empty-dir/60ff9873-7c18-45d4-82a7-9263602acdbb-build\") pod \"60ff9873-7c18-45d4-82a7-9263602acdbb\" (UID: \"60ff9873-7c18-45d4-82a7-9263602acdbb\") " machine # [ 58.536956] k3s[873]: I0919 21:21:24.568031 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"build\" (UniqueName: \"kubernetes.io/empty-dir/586d96c1-2512-46a6-99e0-896b90a7f605-build\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-6m2dg\" (UID: \"586d96c1-2512-46a6-99e0-896b90a7f605\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-6m2dg" machine # [ 58.537675] k3s[873]: I0919 21:21:24.568141 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls\" (UniqueName: \"kubernetes.io/secret/119b9bfe-8f78-4a41-9061-7e2df29c5a85-tls\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-ctxr7\" (UID: \"119b9bfe-8f78-4a41-9061-7e2df29c5a85\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-ctxr7" machine # [ 58.538169] k3s[873]: I0919 21:21:24.568203 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"niks3-token\" (UniqueName: \"kubernetes.io/projected/586d96c1-2512-46a6-99e0-896b90a7f605-niks3-token\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-6m2dg\" (UID: \"586d96c1-2512-46a6-99e0-896b90a7f605\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-6m2dg" machine # [ 58.538989] k3s[873]: I0919 21:21:24.568234 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/586d96c1-2512-46a6-99e0-896b90a7f605-ca\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-6m2dg\" (UID: \"586d96c1-2512-46a6-99e0-896b90a7f605\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-6m2dg" machine # [ 58.595365] k3s[873]: I0919 21:21:24.568301 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc\" (UniqueName: \"kubernetes.io/secret/586d96c1-2512-46a6-99e0-896b90a7f605-oidc\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-6m2dg\" (UID: \"586d96c1-2512-46a6-99e0-896b90a7f605\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-6m2dg" machine # [ 58.597083] k3s[873]: I0919 21:21:24.568327 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/119b9bfe-8f78-4a41-9061-7e2df29c5a85-ca\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-ctxr7\" (UID: \"119b9bfe-8f78-4a41-9061-7e2df29c5a85\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-ctxr7" machine # [ 58.597731] k3s[873]: I0919 21:21:24.568387 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/092f1a79-f45e-4ae4-90b6-babb0b28134c-config\") pod \"nix-grpc-farm-lb-688d8d547b-6k5gq\" (UID: \"092f1a79-f45e-4ae4-90b6-babb0b28134c\") " pod="farm/nix-grpc-farm-lb-688d8d547b-6k5gq" machine # [ 58.598761] k3s[873]: I0919 21:21:24.568479 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"niks3-token\" (UniqueName: \"kubernetes.io/projected/119b9bfe-8f78-4a41-9061-7e2df29c5a85-niks3-token\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-ctxr7\" (UID: \"119b9bfe-8f78-4a41-9061-7e2df29c5a85\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-ctxr7" machine # [ 58.600176] systemd[1]: var-lib-kubelet-pods-60ff9873\x2d7c18\x2d45d4\x2d82a7\x2d9263602acdbb-volumes-kubernetes.io\x7esecret-tls.mount: Deactivated successfully. machine # [ 58.601916] k3s[873]: I0919 21:21:24.568507 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nix\" (UniqueName: \"kubernetes.io/empty-dir/586d96c1-2512-46a6-99e0-896b90a7f605-nix\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-6m2dg\" (UID: \"586d96c1-2512-46a6-99e0-896b90a7f605\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-6m2dg" machine # [ 58.602921] k3s[873]: I0919 21:21:24.568563 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nixconf\" (UniqueName: \"kubernetes.io/configmap/586d96c1-2512-46a6-99e0-896b90a7f605-nixconf\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-6m2dg\" (UID: \"586d96c1-2512-46a6-99e0-896b90a7f605\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-6m2dg" machine # [ 58.603705] k3s[873]: I0919 21:21:24.568589 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"build\" (UniqueName: \"kubernetes.io/empty-dir/119b9bfe-8f78-4a41-9061-7e2df29c5a85-build\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-ctxr7\" (UID: \"119b9bfe-8f78-4a41-9061-7e2df29c5a85\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-ctxr7" machine # [ 58.604994] k3s[873]: I0919 21:21:24.568657 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc\" (UniqueName: \"kubernetes.io/secret/119b9bfe-8f78-4a41-9061-7e2df29c5a85-oidc\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-ctxr7\" (UID: \"119b9bfe-8f78-4a41-9061-7e2df29c5a85\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-ctxr7" machine # [ 58.605685] k3s[873]: I0919 21:21:24.568729 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nix\" (UniqueName: \"kubernetes.io/empty-dir/119b9bfe-8f78-4a41-9061-7e2df29c5a85-nix\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-ctxr7\" (UID: \"119b9bfe-8f78-4a41-9061-7e2df29c5a85\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-ctxr7" machine # [ 58.606467] k3s[873]: I0919 21:21:24.568757 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nixconf\" (UniqueName: \"kubernetes.io/configmap/119b9bfe-8f78-4a41-9061-7e2df29c5a85-nixconf\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-ctxr7\" (UID: \"119b9bfe-8f78-4a41-9061-7e2df29c5a85\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-ctxr7" machine # [ 58.607802] k3s[873]: I0919 21:21:24.569019 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fms9r\" (UniqueName: \"kubernetes.io/projected/119b9bfe-8f78-4a41-9061-7e2df29c5a85-kube-api-access-fms9r\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-ctxr7\" (UID: \"119b9bfe-8f78-4a41-9061-7e2df29c5a85\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-ctxr7" machine # [ 58.609937] k3s[873]: I0919 21:21:24.569044 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-lb\" (UniqueName: \"kubernetes.io/secret/092f1a79-f45e-4ae4-90b6-babb0b28134c-tls-lb\") pod \"nix-grpc-farm-lb-688d8d547b-6k5gq\" (UID: \"092f1a79-f45e-4ae4-90b6-babb0b28134c\") " pod="farm/nix-grpc-farm-lb-688d8d547b-6k5gq" machine # [ 58.669060] k3s[873]: I0919 21:21:24.569069 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/092f1a79-f45e-4ae4-90b6-babb0b28134c-tmp\") pod \"nix-grpc-farm-lb-688d8d547b-6k5gq\" (UID: \"092f1a79-f45e-4ae4-90b6-babb0b28134c\") " pod="farm/nix-grpc-farm-lb-688d8d547b-6k5gq" machine # [ 58.669957] k3s[873]: I0919 21:21:24.569096 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxktc\" (UniqueName: \"kubernetes.io/projected/586d96c1-2512-46a6-99e0-896b90a7f605-kube-api-access-fxktc\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-6m2dg\" (UID: \"586d96c1-2512-46a6-99e0-896b90a7f605\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-6m2dg" machine # [ 58.670826] k3s[873]: I0919 21:21:24.569122 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-ca\" (UniqueName: \"kubernetes.io/secret/092f1a79-f45e-4ae4-90b6-babb0b28134c-tls-ca\") pod \"nix-grpc-farm-lb-688d8d547b-6k5gq\" (UID: \"092f1a79-f45e-4ae4-90b6-babb0b28134c\") " pod="farm/nix-grpc-farm-lb-688d8d547b-6k5gq" machine # [ 58.671696] k3s[873]: I0919 21:21:24.569147 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls\" (UniqueName: \"kubernetes.io/secret/586d96c1-2512-46a6-99e0-896b90a7f605-tls\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-6m2dg\" (UID: \"586d96c1-2512-46a6-99e0-896b90a7f605\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-6m2dg" machine # [ 58.672534] k3s[873]: I0919 21:21:24.590258 873 scope.go:122] "RemoveContainer" containerID="0fe734dda2d1972b0181decd0c781a4332e29db73eaf48e3a3bc74a9693006f4" machine # [ 58.674180] k3s[873]: I0919 21:21:24.591160 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60ff9873-7c18-45d4-82a7-9263602acdbb-nixconf" pod "60ff9873-7c18-45d4-82a7-9263602acdbb" (UID: "60ff9873-7c18-45d4-82a7-9263602acdbb"). InnerVolumeSpecName "nixconf". PluginName "kubernetes.io/configmap", VolumeGIDValue "" machine # [ 58.674751] k3s[873]: I0919 21:21:24.593428 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60ff9873-7c18-45d4-82a7-9263602acdbb-build" pod "60ff9873-7c18-45d4-82a7-9263602acdbb" (UID: "60ff9873-7c18-45d4-82a7-9263602acdbb"). InnerVolumeSpecName "build". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 58.675530] k3s[873]: I0919 21:21:24.619489 873 scope.go:122] "RemoveContainer" containerID="98e6c45ffc9787574eed8892d739db6a99981cf79a86b755016c5aafa5d534bb" machine # [ 58.676175] k3s[873]: I0919 21:21:24.631579 873 scope.go:122] "RemoveContainer" containerID="97beef8239fd728a6210f42e4d02e4c40d45d0e319491b30511a6ffdf7858a11" machine # [ 58.676788] k3s[873]: I0919 21:21:24.643290 873 scope.go:122] "RemoveContainer" containerID="19944217fa890aec1acc35879e3500db1cc3f58eadc2bade0b1f11a91d1ccadc" machine # [ 58.677699] systemd[1]: var-lib-kubelet-pods-60ff9873\x2d7c18\x2d45d4\x2d82a7\x2d9263602acdbb-volume\x2dsubpaths-nix-gc-0.mount: Deactivated successfully. machine # [ 58.678799] k3s[873]: E0919 21:21:24.644156 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"19944217fa890aec1acc35879e3500db1cc3f58eadc2bade0b1f11a91d1ccadc\": not found" containerID="19944217fa890aec1acc35879e3500db1cc3f58eadc2bade0b1f11a91d1ccadc" machine # [ 58.679499] k3s[873]: I0919 21:21:24.644192 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"19944217fa890aec1acc35879e3500db1cc3f58eadc2bade0b1f11a91d1ccadc"} err="failed to get container status \"19944217fa890aec1acc35879e3500db1cc3f58eadc2bade0b1f11a91d1ccadc\": rpc error: code = NotFound desc = an error occurred when try to find container \"19944217fa890aec1acc35879e3500db1cc3f58eadc2bade0b1f11a91d1ccadc\": not found" machine # [ 58.680490] k3s[873]: I0919 21:21:24.644225 873 scope.go:122] "RemoveContainer" containerID="0fe734dda2d1972b0181decd0c781a4332e29db73eaf48e3a3bc74a9693006f4" machine # [ 58.681121] k3s[873]: E0919 21:21:24.645328 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"0fe734dda2d1972b0181decd0c781a4332e29db73eaf48e3a3bc74a9693006f4\": not found" containerID="0fe734dda2d1972b0181decd0c781a4332e29db73eaf48e3a3bc74a9693006f4" machine # [ 58.681690] k3s[873]: I0919 21:21:24.645376 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"0fe734dda2d1972b0181decd0c781a4332e29db73eaf48e3a3bc74a9693006f4"} err="failed to get container status \"0fe734dda2d1972b0181decd0c781a4332e29db73eaf48e3a3bc74a9693006f4\": rpc error: code = NotFound desc = an error occurred when try to find container \"0fe734dda2d1972b0181decd0c781a4332e29db73eaf48e3a3bc74a9693006f4\": not found" machine # [ 58.737869] k3s[873]: I0919 21:21:24.645415 873 scope.go:122] "RemoveContainer" containerID="98e6c45ffc9787574eed8892d739db6a99981cf79a86b755016c5aafa5d534bb" machine # [ 58.739144] k3s[873]: E0919 21:21:24.646022 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"98e6c45ffc9787574eed8892d739db6a99981cf79a86b755016c5aafa5d534bb\": not found" containerID="98e6c45ffc9787574eed8892d739db6a99981cf79a86b755016c5aafa5d534bb" machine # [ 58.739847] k3s[873]: I0919 21:21:24.646054 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"98e6c45ffc9787574eed8892d739db6a99981cf79a86b755016c5aafa5d534bb"} err="failed to get container status \"98e6c45ffc9787574eed8892d739db6a99981cf79a86b755016c5aafa5d534bb\": rpc error: code = NotFound desc = an error occurred when try to find container \"98e6c45ffc9787574eed8892d739db6a99981cf79a86b755016c5aafa5d534bb\": not found" machine # [ 58.740818] k3s[873]: I0919 21:21:24.646079 873 scope.go:122] "RemoveContainer" containerID="97beef8239fd728a6210f42e4d02e4c40d45d0e319491b30511a6ffdf7858a11" machine # [ 58.741985] k3s[873]: E0919 21:21:24.647702 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"97beef8239fd728a6210f42e4d02e4c40d45d0e319491b30511a6ffdf7858a11\": not found" containerID="97beef8239fd728a6210f42e4d02e4c40d45d0e319491b30511a6ffdf7858a11" machine # [ 58.743225] k3s[873]: I0919 21:21:24.647726 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"97beef8239fd728a6210f42e4d02e4c40d45d0e319491b30511a6ffdf7858a11"} err="failed to get container status \"97beef8239fd728a6210f42e4d02e4c40d45d0e319491b30511a6ffdf7858a11\": rpc error: code = NotFound desc = an error occurred when try to find container \"97beef8239fd728a6210f42e4d02e4c40d45d0e319491b30511a6ffdf7858a11\": not found" machine # [ 58.743695] k3s[873]: I0919 21:21:24.647745 873 scope.go:122] "RemoveContainer" containerID="3cb63c4299487150834276786f400583c95bcb60c9c6f96e1b60c941aaa9ac9b" machine # [ 58.744886] k3s[873]: I0919 21:21:24.660411 873 scope.go:122] "RemoveContainer" containerID="3cb63c4299487150834276786f400583c95bcb60c9c6f96e1b60c941aaa9ac9b" machine # [ 58.745464] k3s[873]: E0919 21:21:24.661602 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"3cb63c4299487150834276786f400583c95bcb60c9c6f96e1b60c941aaa9ac9b\": not found" containerID="3cb63c4299487150834276786f400583c95bcb60c9c6f96e1b60c941aaa9ac9b" machine # [ 58.746703] k3s[873]: I0919 21:21:24.661644 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"3cb63c4299487150834276786f400583c95bcb60c9c6f96e1b60c941aaa9ac9b"} err="failed to get container status \"3cb63c4299487150834276786f400583c95bcb60c9c6f96e1b60c941aaa9ac9b\": rpc error: code = NotFound desc = an error occurred when try to find container \"3cb63c4299487150834276786f400583c95bcb60c9c6f96e1b60c941aaa9ac9b\": not found" machine # [ 58.747541] k3s[873]: I0919 21:21:24.669699 873 reconciler_common.go:299] "Volume detached for volume \"nixconf\" (UniqueName: \"kubernetes.io/configmap/60ff9873-7c18-45d4-82a7-9263602acdbb-nixconf\") on node \"machine\" DevicePath \"\"" machine # [ 58.748033] k3s[873]: I0919 21:21:24.669724 873 reconciler_common.go:299] "Volume detached for volume \"build\" (UniqueName: \"kubernetes.io/empty-dir/60ff9873-7c18-45d4-82a7-9263602acdbb-build\") on node \"machine\" DevicePath \"\"" machine # [ 58.748802] systemd[1]: var-lib-kubelet-pods-60ff9873\x2d7c18\x2d45d4\x2d82a7\x2d9263602acdbb-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dd8mgc.mount: Deactivated successfully. machine # [ 58.749497] k3s[873]: I0919 21:21:24.839072 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25f2f7ac-a091-4b45-8d23-efa63777d014-nix" pod "25f2f7ac-a091-4b45-8d23-efa63777d014" (UID: "25f2f7ac-a091-4b45-8d23-efa63777d014"). InnerVolumeSpecName "nix". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 58.750307] systemd[1]: var-lib-kubelet-pods-60ff9873\x2d7c18\x2d45d4\x2d82a7\x2d9263602acdbb-volumes-kubernetes.io\x7esecret-oidc.mount: Deactivated successfully. machine # [ 58.780753] k3s[873]: I0919 21:21:24.871247 873 reconciler_common.go:299] "Volume detached for volume \"nix\" (UniqueName: \"kubernetes.io/empty-dir/25f2f7ac-a091-4b45-8d23-efa63777d014-nix\") on node \"machine\" DevicePath \"\"" machine # [ 58.787808] systemd[1]: Removed slice libcontainer container kubepods-besteffort-pod25f2f7ac_a091_4b45_8d23_efa63777d014.slice. machine # [ 58.789632] systemd[1]: kubepods-besteffort-pod25f2f7ac_a091_4b45_8d23_efa63777d014.slice: Consumed 819ms CPU time over 10.061s wall clock time, 305.6M memory peak, 9.1M read from disk, 283.8M written to disk, 1.1K incoming IP traffic, 890B outgoing IP traffic. machine # [ 58.796119] k3s[873]: I0919 21:21:24.886533 873 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8213252c-bf3a-4ce0-bf80-5872e309ad9f" path="/var/lib/kubelet/pods/8213252c-bf3a-4ce0-bf80-5872e309ad9f/volumes" machine # [ 58.798970] k3s[873]: I0919 21:21:24.889567 873 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f" path="/var/lib/kubelet/pods/aaf0adb9-7e3a-4c6f-b115-3b5de76ef59f/volumes" machine # [ 58.966346] cni0: port 3(vethf99efaf0) entered blocking state machine # [ 58.969249] cni0: port 3(vethf99efaf0) entered disabled state machine # [ 58.970972] vethf99efaf0: entered allmulticast mode machine # [ 58.824858] (udev-worker)[4624]: Network inte[ 58.972526] vethf99efaf0: entered promiscuous mode machine # rface NamePolicy= disabled on kernel command line. machine # [ 58.827112] k3s[873]: I0919 21:21:24.912255 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60ff9873-7c18-45d4-82a7-9263602acdbb-oidc" pod "60ff9873-7c18-45d4-82a7-9263602acdbb" (UID: "60ff9873-7c18-45d4-82a7-9263602acdbb"). InnerVolumeSpecName "oidc". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 58.831754] k3s[873]: I0919 21:21:24.912338 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60ff9873-7c18-45d4-82a7-9263602acdbb-ca" pod "60ff9873-7c18-45d4-82a7-9263602acdbb" (UID: "60ff9873-7c18-45d4-82a7-9263602acdbb"). InnerVolumeSpecName "ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 58.837950] k3s[873]: I0919 21:21:24.912455 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60ff9873-7c18-45d4-82a7-9263602acdbb-niks3-token" pod "60ff9873-7c18-45d4-82a7-9263602acdbb" (UID: "60ff9873-7c18-45d4-82a7-9263602acdbb"). InnerVolumeSpecName "niks3-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" machine # [ 58.843173] k3s[873]: I0919 21:21:24.912563 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60ff9873-7c18-45d4-82a7-9263602acdbb-tls" pod "60ff9873-7c18-45d4-82a7-9263602acdbb" (UID: "60ff9873-7c18-45d4-82a7-9263602acdbb"). InnerVolumeSpecName "tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 58.848896] (udev-worker)[4627]: Network interface NamePolicy= disabled on kernel command line. machine # [ 58.850919] k3s[873]: I0919 21:21:24.917939 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/pr[ 59.000680] cni0: port 3(vethf99efaf0) entered blocking state machine # ojected/60ff9873-7c18-45d4-82a7-[ 59.001965] cni0: port 3(vethf99efaf0) entered forwarding state machine # 9263602acdbb-kube-api-access-d8mgc" pod "60ff9873-7c18-45d4-82a7-9263602acdbb" (UID: "60ff9873-7c18-45d4-82a7-9263602acdbb"). InnerVolumeSpecName "kube-api-access-d8mgc". PluginName "kubernetes.io/projected", VolumeGIDValue "" machine # [ 58.859502] (udev-worker)[4623]: Network interface NamePolicy= disabled on kernel command lin[ 59.007144] cni0: port 4(veth959ef3ec) entered blocking state machine # e. machine # [ 59.009929] cni0: port 4(veth959ef3ec) entered disabled state machine # [ 59.012061] veth959ef3ec: entered allmulticast mode machine # [ 59.013639] veth959ef3ec: entered promiscuous mode machine # [ 59.017336] cni0: port 5(veth7e34664d) entered blocking state machine # [ 59.019744] cni0: port 5(veth7e34664d) entered disabled state machine # [ 59.021792] veth7e34664d: entered allmulticast mode machine # [ 59.023116] veth7e34664d: entered promiscuous mode machine # [ 58.882063] k3s[873]: I0919 21:21:24.972329 873 reconciler_common.go:299] "Volume detached for volume \"tls\" (UniqueName: \"kubernetes.io/secret/60ff9873-7c18-45d4-82a7-9263602acdbb-tl[ 59.031131] cni0: port 4(veth959ef3ec) entered blocking state machine # s\") on node \"machine\" DeviceP[ 59.032282] cni0: port 4(veth959ef3ec) entered forwarding state machine # ath \"\"" machine # [ 58.882208] k3s[873]: I0919 21:21:24.972384 873 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8mgc\" (UniqueName: \"kubernetes.io/projected/60ff9873-7c18-45d4-82a7-9263602acdbb-kube-api-access-d8mgc\") on node \"machine\" DevicePath \"\"" machine # [ 58.882730] k3s[873]: I0919 21:21:24.972403 873 reconciler_common.go:299] "Volume detached for volume \"ca\" (UniqueName: \"kubernetes.io/secret/60ff9873-7c18-45d4-82a7-9263602acdbb-ca\") on node \"machine\" DevicePath \"\"" machine # [ 58.883210] k3s[873]: I0919 21:21:24.972420 873 reconciler_common.go:299] "Volume detached for volume \"oidc\" (UniqueName: \"kubernetes.io/secret/60ff9873-7c18-45d4-82a7-9263602acdbb-oidc\") on node \"machine\" DevicePath \"\"" machine # [ 58.883711] k3s[873]: I0919 21:21:24.972441 873 reconciler_common.go:299] "Volume detached for volume \"niks3-token\" (UniqueName: \"kubernetes.io/projected/60ff9873-7c18-45d4-82a7-9263602acdbb-niks3-token\") on node \"machine\" DevicePath \"\"" machine # [ 59.057230] cni0: port 5(veth7e34664d) entered blocking state machine # [ 59.058419] cni0: port 5(veth7e34664d) entered forwarding state machine # [ 58.927338] dhcpcd[673]: vethf99efaf0: IAID 97:8b:ff:f9 machine # [ 58.928823] dhcpcd[673]: vethf99efaf0: adding address fe80::f88f:97ff:fe8b:fff9 machine # [ 58.970849] dhcpcd[673]: veth959ef3ec: IAID ba:93:b0:84 machine # [ 58.972425] dhcpcd[673]: veth959ef3ec: adding address fe80::34b4:baff:fe93:b084 machine # [ 58.979953] dhcpcd[673]: veth7e34664d: IAID b4:1b:17:6d machine # [ 58.981344] dhcpcd[673]: veth7e34664d: adding address fe80::c843:b4ff:fe1b:176d machine # [ 58.982774] dhcpcd[673]: veth7e34664d: carrier lost machine # [ 59.009388] systemd[1]: cri-containerd-4817213430471689792d913034415f3aa589c81b05558f32eca49ba49a1a6407.scope: Deactivated successfully. machine # [ 59.023577] dhcpcd[673]: veth7e34664d: deleting address fe80::c843:b4ff:fe1b:176d machine # [ 59.025519] dhcpcd[673]: veth959ef3ec: carrier lost machine # [ 59.071139] dhcpcd[673]: veth959ef3ec: deleting address fe80::34b4:baff:fe93:b084 machine # [ 59.073089] dhcpcd[673]: veth959ef3ec: carrier acquired machine # [ 59.078673] k3s[873]: I0919 21:21:25.169227 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60ff9873-7c18-45d4-82a7-9263602acdbb-nix" pod "60ff9873-7c18-45d4-82a7-9263602acdbb" (UID: "60ff9873-7c18-45d4-82a7-9263602acdbb"). InnerVolumeSpecName "nix". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 59.083727] k3s[873]: I0919 21:21:25.174328 873 reconciler_common.go:299] "Volume detached for volume \"nix\" (UniqueName: \"kubernetes.io/empty-dir/60ff9873-7c18-45d4-82a7-9263602acdbb-nix\") on node \"machine\" DevicePath \"\"" machine # [ 59.084172] dhcpcd[673]: veth959ef3ec: IAID ba:93:b0:84 machine # [ 59.084785] dhcpcd[673]: veth959ef3ec: adding address fe80::34b4:baff:fe93:b084 machine # [ 59.085320] dhcpcd[673]: veth7e34664d: carrier acquired machine # [ 59.096652] dhcpcd[673]: veth7e34664d: IAID b4:1b:17:6d machine # [ 59.097848] dhcpcd[673]: veth7e34664d: adding address fe80::c843:b4ff:fe1b:176d machine # [ 59.314285] cni0: port 6(veth199282f3) entered disabled state machine # [ 59.167285] dhcpcd[673]: veth199282f3: carrier lost machine # [ 59.316874] veth199282f3 (unregistering): left allmulticast mode machine # [ 59.318161] veth199282f3 (unregistering): left promiscuous mode machine # [ 59.319458] cni0: port 6(veth199282f3) entered disabled state machine # [ 59.207216] dhcpcd[673]: veth199282f3: deleting address fe80::e4cf:1ff:feb9:5e3c machine # [ 59.258365] dhcpcd[673]: veth199282f3: removing interface machine # [ 59.428698] systemd[1]: Removed slice libcontainer container kubepods-besteffort-pod60ff9873_7c18_45d4_82a7_9263602acdbb.slice. machine # [ 59.433163] systemd[1]: kubepods-besteffort-pod60ff9873_7c18_45d4_82a7_9263602acdbb.slice: Consumed 773ms CPU time over 10.763s wall clock time, 285.4M memory peak, 38.3M read from disk, 283.8M written to disk, 1.1K incoming IP traffic, 890B outgoing IP traffic. machine # [ 59.435304] k3s[873]: I0919 21:21:25.524931 873 scope.go:122] "RemoveContainer" containerID="55f4b790498984e296d0abfd180da62e8fafb625d150d3a71861715a3bcee531" machine # [ 59.441368] k3s[873]: I0919 21:21:25.526619 873 scope.go:122] "RemoveContainer" containerID="1b940306e86be7c1ea430586108c4d8105721acde29fc5373cf6ae9179d0853f" machine # [ 59.443668] k3s[873]: E0919 21:21:25.527251 873 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"helm\" with CrashLoopBackOff: \"back-off 10s restarting failed container=helm pod=helm-install-nix-grpc-farm-zcg5w_kube-system(6d037147-1d29-4e02-b2ab-2f08bc2a576c)\"" pod="kube-system/helm-install-nix-grpc-farm-zcg5w" podUID="6d037147-1d29-4e02-b2ab-2f08bc2a576c" machine # [ 59.475879] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-5721a1ae2b7029b2e1ae1879748350c5fdd22e6504f8c5fecfadbda3d9fb42b3-rootfs.mount: Deactivated successfully. machine # [ 59.479324] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-4817213430471689792d913034415f3aa589c81b05558f32eca49ba49a1a6407-rootfs.mount: Deactivated successfully. machine # [ 59.482832] systemd[1]: run-k3s-containerd-io.containerd.grpc.v1.cri-sandboxes-4817213430471689792d913034415f3aa589c81b05558f32eca49ba49a1a6407-shm.mount: Deactivated successfully. machine # [ 59.485466] systemd[1]: run-netns-cni\x2d9730eac6\x2d6a94\x2d8404\x2dab49\x2d1dabb563bc9a.mount: Deactivated successfully. machine # [ 59.487570] systemd[1]: var-lib-kubelet-pods-60ff9873\x2d7c18\x2d45d4\x2d82a7\x2d9263602acdbb-volume\x2dsubpaths-nix-nix\x2dgrpc\x2ddaemon-0.mount: Deactivated successfully. machine # [ 59.490086] systemd[1]: var-lib-kubelet-pods-60ff9873\x2d7c18\x2d45d4\x2d82a7\x2d9263602acdbb-volume\x2dsubpaths-nix-nix\x2ddaemon-0.mount: Deactivated successfully. machine # [ 59.492436] systemd[1]: var-lib-kubelet-pods-60ff9873\x2d7c18\x2d45d4\x2d82a7\x2d9263602acdbb-volumes-kubernetes.io\x7eprojected-niks3\x2dtoken.mount: Deactivated successfully. machine # [ 59.494824] systemd[1]: var-lib-kubelet-pods-60ff9873\x2d7c18\x2d45d4\x2d82a7\x2d9263602acdbb-volumes-kubernetes.io\x7esecret-ca.mount: Deactivated successfully. machine # [ 59.496931] k3s[873]: I0919 21:21:25.587448 873 scope.go:122] "RemoveContainer" containerID="5721a1ae2b7029b2e1ae1879748350c5fdd22e6504f8c5fecfadbda3d9fb42b3" machine # [ 59.510607] k3s[873]: I0919 21:21:25.601182 873 scope.go:122] "RemoveContainer" containerID="5721a1ae2b7029b2e1ae1879748350c5fdd22e6504f8c5fecfadbda3d9fb42b3" machine # [ 59.512543] k3s[873]: E0919 21:21:25.602495 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"5721a1ae2b7029b2e1ae1879748350c5fdd22e6504f8c5fecfadbda3d9fb42b3\": not found" containerID="5721a1ae2b7029b2e1ae1879748350c5fdd22e6504f8c5fecfadbda3d9fb42b3" machine # [ 59.517302] k3s[873]: I0919 21:21:25.602540 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"5721a1ae2b7029b2e1ae1879748350c5fdd22e6504f8c5fecfadbda3d9fb42b3"} err="failed to get container status \"5721a1ae2b7029b2e1ae1879748350c5fdd22e6504f8c5fecfadbda3d9fb42b3\": rpc error: code = NotFound desc = an error occurred when try to find container \"5721a1ae2b7029b2e1ae1879748350c5fdd22e6504f8c5fecfadbda3d9fb42b3\": not found" machine # [ 59.561535] systemd[1]: Started libcontainer container 17e462c37879905c86009df2f6562ccb4719332819d0125f9e21a35695c3a79d. machine # [ 59.563995] systemd[1]: Started libcontainer container d6e0a2df34faa36683dc2acc8692f4ee6d363609a92a6501fc74a7ddbeb39cb5. machine # [ 59.567877] systemd[1]: Started libcontainer container c2afc9fd2a69142e84b98f5bcd8290981bbf5ae28ec278b784e52c1418159d53. machine # [ 59.587181] k3s[873]: I0919 21:21:25.677765 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/905b88b7-2056-415a-bf62-9e1a86f7376d-tls-lb\" (UniqueName: \"kubernetes.io/secret/905b88b7-2056-415a-bf62-9e1a86f7376d-tls-lb\") pod \"905b88b7-2056-415a-bf62-9e1a86f7376d\" (UID: \"905b88b7-2056-415a-bf62-9e1a86f7376d\") " machine # [ 59.591782] k3s[873]: I0919 21:21:25.677825 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/905b88b7-2056-415a-bf62-9e1a86f7376d-tmp\" (UniqueName: \"kubernetes.io/empty-dir/905b88b7-2056-415a-bf62-9e1a86f7376d-tmp\") pod \"905b88b7-2056-415a-bf62-9e1a86f7376d\" (UID: \"905b88b7-2056-415a-bf62-9e1a86f7376d\") " machine # [ 59.597466] k3s[873]: I0919 21:21:25.677868 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/905b88b7-2056-415a-bf62-9e1a86f7376d-config\" (UniqueName: \"kubernetes.io/configmap/905b88b7-2056-415a-bf62-9e1a86f7376d-config\") pod \"905b88b7-2056-415a-bf62-9e1a86f7376d\" (UID: \"905b88b7-2056-415a-bf62-9e1a86f7376d\") " machine # [ 59.601851] k3s[873]: I0919 21:21:25.677915 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/905b88b7-2056-415a-bf62-9e1a86f7376d-tls-ca\" (UniqueName: \"kubernetes.io/secret/905b88b7-2056-415a-bf62-9e1a86f7376d-tls-ca\") pod \"905b88b7-2056-415a-bf62-9e1a86f7376d\" (UID: \"905b88b7-2056-415a-bf62-9e1a86f7376d\") " machine # [ 59.611543] k3s[873]: I0919 21:21:25.702068 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/905b88b7-2056-415a-bf62-9e1a86f7376d-tmp" pod "905b88b7-2056-415a-bf62-9e1a86f7376d" (UID: "905b88b7-2056-415a-bf62-9e1a86f7376d"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 59.615569] k3s[873]: I0919 21:21:25.703447 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/905b88b7-2056-415a-bf62-9e1a86f7376d-config" pod "905b88b7-2056-415a-bf62-9e1a86f7376d" (UID: "905b88b7-2056-415a-bf62-9e1a86f7376d"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" machine # [ 59.621184] k3s[873]: I0919 21:21:25.703792 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/905b88b7-2056-415a-bf62-9e1a86f7376d-tls-lb" pod "905b88b7-2056-415a-bf62-9e1a86f7376d" (UID: "905b88b7-2056-415a-bf62-9e1a86f7376d"). InnerVolumeSpecName "tls-lb". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 59.625201] k3s[873]: I0919 21:21:25.706267 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/905b88b7-2056-415a-bf62-9e1a86f7376d-tls-ca" pod "905b88b7-2056-415a-bf62-9e1a86f7376d" (UID: "905b88b7-2056-415a-bf62-9e1a86f7376d"). InnerVolumeSpecName "tls-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 59.648439] dhcpcd[673]: veth7e34664d: soliciting a DHCP lease machine # [ 59.689170] k3s[873]: I0919 21:21:25.779139 873 reconciler_common.go:299] "Volume detached for volume \"tls-ca\" (UniqueName: \"kubernetes.io/secret/905b88b7-2056-415a-bf62-9e1a86f7376d-tls-ca\") on node \"machine\" DevicePath \"\"" machine # [ 59.689677] k3s[873]: I0919 21:21:25.779170 873 reconciler_common.go:299] "Volume detached for volume \"tls-lb\" (UniqueName: \"kubernetes.io/secret/905b88b7-2056-415a-bf62-9e1a86f7376d-tls-lb\") on node \"machine\" DevicePath \"\"" machine # [ 59.690290] k3s[873]: I0919 21:21:25.779182 873 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/905b88b7-2056-415a-bf62-9e1a86f7376d-tmp\") on node \"machine\" DevicePath \"\"" machine # [ 59.690851] k3s[873]: I0919 21:21:25.779197 873 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/905b88b7-2056-415a-bf62-9e1a86f7376d-config\") on node \"machine\" DevicePath \"\"" machine # [ 59.739647] systemd[1]: Started libcontainer container eecef8c188fddf9f78b4dba9edf9ec1d0176d715fac5df3b7594c6ee0a934ea4. machine # [ 59.745514] systemd[1]: Started libcontainer container 1f4073b56af6205458e8ad7904383923bb02fac5947d93b909bf88a24ef0ab17. machine # [ 59.768679] systemd[1]: Started libcontainer container c17a33a880488b897fc343e396a9ed15941ec7f46c4af8499925fd7cede5c577. machine # [ 59.776599] systemd[1]: Removed slice libcontainer container kubepods-besteffort-pod905b88b7_2056_415a_bf62_9e1a86f7376d.slice. machine # [ 59.776782] systemd[1]: kubepods-besteffort-pod905b88b7_2056_415a_bf62_9e1a86f7376d.slice: Consumed 135ms CPU time over 3.931s wall clock time, 14.9M memory peak, 2.8K incoming IP traffic, 3K outgoing IP traffic. machine # [ 60.465054] systemd[1]: var-lib-kubelet-pods-905b88b7\x2d2056\x2d415a\x2dbf62\x2d9e1a86f7376d-volume\x2dsubpaths-config-envoy-0.mount: Deactivated successfully. machine # [ 60.465788] systemd[1]: var-lib-kubelet-pods-905b88b7\x2d2056\x2d415a\x2dbf62\x2d9e1a86f7376d-volumes-kubernetes.io\x7esecret-tls\x2dlb.mount: Deactivated successfully. machine # [ 60.467676] systemd[1]: var-lib-kubelet-pods-905b88b7\x2d2056\x2d415a\x2dbf62\x2d9e1a86f7376d-volumes-kubernetes.io\x7esecret-tls\x2dca.mount: Deactivated successfully. machine # [ 60.470165] systemd[1]: var-lib-kubelet-pods-905b88b7\x2d2056\x2d415a\x2dbf62\x2d9e1a86f7376d-volumes-kubernetes.io\x7eempty\x2ddir-tmp.mount: Deactivated successfully. machine # [ 60.471302] dhcpcd[673]: veth3ce56192: probing for an IPv4LL address machine # [ 60.528321] systemd[1]: cri-containerd-eecef8c188fddf9f78b4dba9edf9ec1d0176d715fac5df3b7594c6ee0a934ea4.scope: Deactivated successfully. machine # [ 60.528694] systemd[1]: cri-containerd-eecef8c188fddf9f78b4dba9edf9ec1d0176d715fac5df3b7594c6ee0a934ea4.scope: Consumed 527ms CPU time over 788ms wall clock time, 308.9M memory peak, 80K read from disk, 145.8M written to disk. machine # [ 60.562321] systemd[1]: cri-containerd-1f4073b56af6205458e8ad7904383923bb02fac5947d93b909bf88a24ef0ab17.scope: Deactivated successfully. machine # [ 60.563918] systemd[1]: cri-containerd-1f4073b56af6205458e8ad7904383923bb02fac5947d93b909bf88a24ef0ab17.scope: Consumed 532ms CPU time over 817ms wall clock time, 308.8M memory peak, 100K read from disk, 158.7M written to disk. machine # [ 60.577065] k3s[873]: I0919 21:21:26.666848 873 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="farm/nix-grpc-farm-lb-688d8d547b-6k5gq" podStartSLOduration=2.666553284 podStartE2EDuration="2.666553284s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 21:21:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 21:21:26.66535816 +0000 UTC m=+42.249215190" watchObservedRunningTime="2026-09-19 21:21:26.666553284 +0000 UTC m=+42.250668447" machine # [ 60.646579] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-eecef8c188fddf9f78b4dba9edf9ec1d0176d715fac5df3b7594c6ee0a934ea4-rootfs.mount: Deactivated successfully. machine # [ 60.670607] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-1f4073b56af6205458e8ad7904383923bb02fac5947d93b909bf88a24ef0ab17-rootfs.mount: Deactivated successfully. machine # [ 60.750576] dhcpcd[673]: vethf99efaf0: soliciting a DHCP lease machine # [ 60.794389] dhcpcd[673]: vethf99efaf0: soliciting an IPv6 router machine # [ 60.797654] k3s[873]: I0919 21:21:26.888231 873 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="25f2f7ac-a091-4b45-8d23-efa63777d014" path="/var/lib/kubelet/pods/25f2f7ac-a091-4b45-8d23-efa63777d014/volumes" machine # [ 60.798637] k3s[873]: I0919 21:21:26.889257 873 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="60ff9873-7c18-45d4-82a7-9263602acdbb" path="/var/lib/kubelet/pods/60ff9873-7c18-45d4-82a7-9263602acdbb/volumes" machine # [ 60.800201] k3s[873]: I0919 21:21:26.890816 873 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="905b88b7-2056-415a-bf62-9e1a86f7376d" path="/var/lib/kubelet/pods/905b88b7-2056-415a-bf62-9e1a86f7376d/volumes" machine # [ 60.973528] dhcpcd[673]: veth959ef3ec: soliciting a DHCP lease machine # [ 61.237712] dhcpcd[673]: veth7e34664d: soliciting an IPv6 router machine # [ 61.441400] dhcpcd[673]: veth959ef3ec: soliciting an IPv6 router machine # [ 61.735626] systemd[1]: var-lib-rancher-k3s-agent-containerd-tmpmounts-containerd\x2dmount4262733398.mount: Deactivated successfully. machine # [ 61.791550] systemd[1]: Started libcontainer container 3a7bbd4f1c81ef5888b0c5f51620e082100da74fde6e0b3b98ec518e78f23979. machine # [ 61.795263] systemd[1]: Started libcontainer container 52a3a76a1104d26dfa13587d87d299663e3528f525af01557afcc88b6191300e. machine # [ 61.970334] systemd[1]: Started libcontainer container 4b879e3b41a4ba18a45761add1567ccc239051125dd77d8925423ba08604c659. machine # [ 61.972451] systemd[1]: Started libcontainer container 6385b2c3bd9aec029cef3c5d85fb1d732d4415793015d40b4afcfd7ac14a310c. machine # [ 62.100375] systemd[1]: Started libcontainer container b1d0716c5bf5b2c7957dff26421b87f642b389f83fa1b70d2fcf1306ceb19171. machine # [ 62.105972] systemd[1]: Started libcontainer container ca7c7d668571350bc0a583a0509fc69ee36a2ae291db636e21c4a9faacb589b0. machine # [ 62.565190] k3s[873]: I0919 21:21:28.654835 873 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-ctxr7" podStartSLOduration=3.91020548 podStartE2EDuration="4.654791454s" totalImagesPullingTime="0s" totalInitContainerRuntime="744.585974ms" isStatefulPod=false podCreationTimestamp="2026-09-19 21:21:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 21:21:28.653207733 +0000 UTC m=+44.237057220" watchObservedRunningTime="2026-09-19 21:21:28.654791454 +0000 UTC m=+44.238640661" machine # [ 62.584681] k3s[873]: I0919 21:21:28.674335 873 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-6m2dg" podStartSLOduration=3.901363574 podStartE2EDuration="4.674297285s" totalImagesPullingTime="0s" totalInitContainerRuntime="772.933711ms" isStatefulPod=false podCreationTimestamp="2026-09-19 21:21:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 21:21:28.67394333 +0000 UTC m=+44.257792816" watchObservedRunningTime="2026-09-19 21:21:28.674297285 +0000 UTC m=+44.258146213" machine # [ 64.649628] dhcpcd[673]: veth7e34664d: probing for an IPv4LL address machine # [ 65.215631] dhcpcd[673]: veth3ce56192: using IPv4LL address 169.254.229.31 machine # [ 65.218188] dhcpcd[673]: veth3ce56192: adding route to 169.254.0.0/16 machine # [ 65.752526] dhcpcd[673]: vethf99efaf0: probing for an IPv4LL address machine # [ 65.974929] dhcpcd[673]: veth959ef3ec: probing for an IPv4LL address machine: (finished: waiting for success: kubectl -n farm rollout status deployment nix-grpc-farm-lb --timeout=10s, in 38.69 seconds) machine: waiting for success: kubectl -n farm rollout status deployment nix-grpc-farm-scheduler --timeout=10s machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 68.198671] dhcpcd[673]: veth3ce56192: no IPv6 Routers available machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 69.733989] dhcpcd[673]: veth7e34664d: using IPv4LL address 169.254.41.149 machine # [ 69.734541] dhcpcd[673]: veth7e34664d: adding route to 169.254.0.0/16 machine # [ 70.711339] dhcpcd[673]: vethf99efaf0: using IPv4LL address 169.254.14.49 machine # [ 70.711647] dhcpcd[673]: vethf99efaf0: adding route to 169.254.0.0/16 machine # [ 70.865069] dhcpcd[673]: veth959ef3ec: using IPv4LL address 169.254.246.47 machine # [ 70.865296] dhcpcd[673]: veth959ef3ec: adding route to 169.254.0.0/16 machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 72.793879] k3s[873]: I0919 21:21:38.883652 873 scope.go:122] "RemoveContainer" containerID="1b940306e86be7c1ea430586108c4d8105721acde29fc5373cf6ae9179d0853f" machine # [ 72.800473] dhcpcd[673]: vethf99efaf0: no IPv6 Routers available machine # [ 72.832643] systemd[1]: var-lib-rancher-k3s-agent-containerd-tmpmounts-containerd\x2dmount1697042120.mount: Deactivated successfully. machine # [ 72.886562] systemd[1]: Started libcontainer container 20dcd11150271900e2cc11cc4135789ea0c630872f741af5896478df3fdc5b6d. machine # [ 73.288156] dhcpcd[673]: veth7e34664d: no IPv6 Routers available machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 73.444265] dhcpcd[673]: veth959ef3ec: no IPv6 Routers available machine # [ 73.951021] dhcpcd[673]: segfault at 5912dfec1 ip 00005912cd45ccb3 sp 00007ffcdf001df8 error 4 in dhcpcd[38cb3,5912cd42d000+40000] likely on CPU 2 (core 2, socket 0) machine # [ 73.956488] Code: 45 ae 01 e9 0f fe ff ff 0f 1f 80 00 00 00 00 c6 45 af 01 e9 ff fd ff ff e8 fa 05 fd ff 66 2e 0f 1f 84 00 00 00 00 00 48 8b 07 <48> 8b 80 50 03 00 00 48 85 c0 74 31 90 48 8b 00 48 85 c0 74 1e 48 machine # [ 73.834507] systemd-coredump[6014]: Process 673 (dhcpcd) of user 999 terminated abnormally with signal 11/SEGV, processing... machine # [ 73.849503] systemd[1]: Created slice Slice /system/systemd-coredump. machine # [ 73.853882] systemd[1]: Started Process Core Dump (PID 6014/UID 0). machine # [ 74.062457] systemd[1]: cri-containerd-c17a33a880488b897fc343e396a9ed15941ec7f46c4af8499925fd7cede5c577.scope: Deactivated successfully. machine # [ 74.068673] systemd[1]: cri-containerd-c17a33a880488b897fc343e396a9ed15941ec7f46c4af8499925fd7cede5c577.scope: Consumed 199ms CPU time over 14.294s wall clock time, 15.6M memory peak, 19.2K incoming IP traffic, 15.8K outgoing IP traffic. machine # [ 74.072887] systemd[1]: cri-containerd-ca7c7d668571350bc0a583a0509fc69ee36a2ae291db636e21c4a9faacb589b0.scope: Deactivated successfully. machine # [ 74.092942] systemd[1]: cri-containerd-b1d0716c5bf5b2c7957dff26421b87f642b389f83fa1b70d2fcf1306ceb19171.scope: Deactivated successfully. machine # [ 74.128068] systemd[1]: cri-containerd-3a7bbd4f1c81ef5888b0c5f51620e082100da74fde6e0b3b98ec518e78f23979.scope: Deactivated successfully. machine # [ 74.132646] systemd-coredump[6015]: Process 673 (dhcpcd) of user 999 dumped core. machine # machine # Module udev.so without build-id. machine # Module dhcpcd without build-id. machine # Stack trace of thread 673: machine # #0 0x00005912cd45ccb3 ipv6nd_expire (dhcpcd + 0x38cb3) machine # #1 0x00005912cd43606a eloop_start (dhcpcd + 0x1206a) machine # #2 0x00005912cd42e300 main (dhcpcd + 0xa300) machine # #3 0x0000773feb980285 __libc_start_call_main (libc.so.6 + 0x2b285) machine # #4 0x0000773feb980338 __libc_start_main@@GLIBC_2.34 (libc.so.6 + 0x2b338) machine # #5 0x00005912cd42f365 _start (dhcpcd + 0xb365) machine # ELF object binary architecture: AMD x86-64 machine # machine # [ 74.149447] systemd[1]: systemd-coredump@0-1-6014_6015-0.service: Deactivated successfully. machine # [ 74.157763] systemd[1]: cri-containerd-52a3a76a1104d26dfa13587d87d299663e3528f525af01557afcc88b6191300e.scope: Deactivated successfully. machine # [ 74.159925] systemd[1]: cri-containerd-52a3a76a1104d26dfa13587d87d299663e3528f525af01557afcc88b6191300e.scope: Consumed 90ms CPU time over 12.357s wall clock time, 31.8M memory peak, 22.3M read from disk. machine # [ 74.168084] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-c17a33a880488b897fc343e396a9ed15941ec7f46c4af8499925fd7cede5c577-rootfs.mount: Deactivated successfully. machine # [ 74.172195] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-ca7c7d668571350bc0a583a0509fc69ee36a2ae291db636e21c4a9faacb589b0-rootfs.mount: Deactivated successfully. machine # [ 74.181074] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-b1d0716c5bf5b2c7957dff26421b87f642b389f83fa1b70d2fcf1306ceb19171-rootfs.mount: Deactivated successfully. machine # [ 74.197337] systemd[1]: dhcpcd.service: Main process exited, code=dumped, status=11/SEGV machine # [ 74.200458] systemd[1]: dhcpcd.service: Failed with result 'core-dump'. machine # [ 74.202545] systemd[1]: dhcpcd.service: Consumed 789ms CPU time over 1min 5.395s wall clock time, 6M memory peak, 8K written to disk, 396B incoming IP traffic, 1.7K outgoing IP traffic. machine # [ 74.220908] systemd[1]: cri-containerd-c2afc9fd2a69142e84b98f5bcd8290981bbf5ae28ec278b784e52c1418159d53.scope: Deactivated successfully. machine # [ 74.227415] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-3a7bbd4f1c81ef5888b0c5f51620e082100da74fde6e0b3b98ec518e78f23979-rootfs.mount: Deactivated successfully. machine # [ 74.251606] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-52a3a76a1104d26dfa13587d87d299663e3528f525af01557afcc88b6191300e-rootfs.mount: Deactivated successfully. machine # [ 74.510058] cni0: port 4(veth959ef3ec) entered disabled state machine # [ 74.512817] veth959ef3ec (unregistering): left allmulticast mode machine # [ 74.514586] veth959ef3ec (unregistering): left promiscuous mode machine # [ 74.515862] cni0: port 4(veth959ef3ec) entered disabled state machine # [ 74.384650] systemd[1]: dhcpcd.service: Scheduled restart job, restart counter is at 1. machine # [ 74.390368] systemd[1]: Starting DHCP Client... machine # [ 74.513197] k3s[873]: I0919 21:21:40.603415 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/092f1a79-f45e-4ae4-90b6-babb0b28134c-tls-lb\" (UniqueName: \"kubernetes.io/secret/092f1a79-f45e-4ae4-90b6-babb0b28134c-tls-lb\") pod \"092f1a79-f45e-4ae4-90b6-babb0b28134c\" (UID: \"092f1a79-f45e-4ae4-90b6-babb0b28134c\") " machine # [ 74.521580] k3s[873]: I0919 21:21:40.603531 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/092f1a79-f45e-4ae4-90b6-babb0b28134c-config\" (UniqueName: \"kubernetes.io/configmap/092f1a79-f45e-4ae4-90b6-babb0b28134c-config\") pod \"092f1a79-f45e-4ae4-90b6-babb0b28134c\" (UID: \"092f1a79-f45e-4ae4-90b6-babb0b28134c\") " machine # [ 74.529477] k3s[873]: I0919 21:21:40.603602 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/092f1a79-f45e-4ae4-90b6-babb0b28134c-tmp\" (UniqueName: \"kubernetes.io/empty-dir/092f1a79-f45e-4ae4-90b6-babb0b28134c-tmp\") pod \"092f1a79-f45e-4ae4-90b6-babb0b28134c\" (UID: \"092f1a79-f45e-4ae4-90b6-babb0b28134c\") " machine # [ 74.535630] k3s[873]: I0919 21:21:40.603639 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/092f1a79-f45e-4ae4-90b6-babb0b28134c-tls-ca\" (UniqueName: \"kubernetes.io/secret/092f1a79-f45e-4ae4-90b6-babb0b28134c-tls-ca\") pod \"092f1a79-f45e-4ae4-90b6-babb0b28134c\" (UID: \"092f1a79-f45e-4ae4-90b6-babb0b28134c\") " machine # [ 74.542659] k3s[873]: I0919 21:21:40.620311 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/092f1a79-f45e-4ae4-90b6-babb0b28134c-tls-lb" pod "092f1a79-f45e-4ae4-90b6-babb0b28134c" (UID: "092f1a79-f45e-4ae4-90b6-babb0b28134c"). InnerVolumeSpecName "tls-lb". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 74.548369] k3s[873]: I0919 21:21:40.634863 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/092f1a79-f45e-4ae4-90b6-babb0b28134c-tls-ca" pod "092f1a79-f45e-4ae4-90b6-babb0b28134c" (UID: "092f1a79-f45e-4ae4-90b6-babb0b28134c"). InnerVolumeSpecName "tls-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 74.553189] k3s[873]: I0919 21:21:40.636617 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/092f1a79-f45e-4ae4-90b6-babb0b28134c-config" pod "092f1a79-f45e-4ae4-90b6-babb0b28134c" (UID: "092f1a79-f45e-4ae4-90b6-babb0b28134c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" machine # [ 74.559287] k3s[873]: I0919 21:21:40.639446 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/092f1a79-f45e-4ae4-90b6-babb0b28134c-tmp" pod "092f1a79-f45e-4ae4-90b6-babb0b28134c" (UID: "092f1a79-f45e-4ae4-90b6-babb0b28134c"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 74.574213] dhcpcd[6298]: dhcpcd-10.3.2 starting machine # [ 74.577906] dhcpcd[6337]: dev: loaded udev machine # [ 74.579834] dhcpcd[6337]: DUID 00:01:00:01:32:41:b9:23:52:54:00:12:34:56 machine # [ 74.613369] k3s[873]: I0919 21:21:40.703963 873 reconciler_common.go:299] "Volume detached for volume \"tls-lb\" (UniqueName: \"kubernetes.io/secret/092f1a79-f45e-4ae4-90b6-babb0b28134c-tls-lb\") on node \"machine\" DevicePath \"\"" machine # [ 74.618021] k3s[873]: I0919 21:21:40.704721 873 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/092f1a79-f45e-4ae4-90b6-babb0b28134c-config\") on node \"machine\" DevicePath \"\"" machine # [ 74.622645] k3s[873]: I0919 21:21:40.704757 873 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/092f1a79-f45e-4ae4-90b6-babb0b28134c-tmp\") on node \"machine\" DevicePath \"\"" machine # [ 74.628413] k3s[873]: I0919 21:21:40.704772 873 reconciler_common.go:299] "Volume detached for volume \"tls-ca\" (UniqueName: \"kubernetes.io/secret/092f1a79-f45e-4ae4-90b6-babb0b28134c-tls-ca\") on node \"machine\" DevicePath \"\"" machine # [ 74.632977] k3s[873]: I0919 21:21:40.718414 873 scope.go:122] "RemoveContainer" containerID="c17a33a880488b897fc343e396a9ed15941ec7f46c4af8499925fd7cede5c577" machine # [ 74.637731] systemd[1]: Removed slice libcontainer container kubepods-besteffort-pod092f1a79_f45e_4ae4_90b6_babb0b28134c.slice. machine # [ 74.640239] systemd[1]: kubepods-besteffort-pod092f1a79_f45e_4ae4_90b6_babb0b28134c.slice: Consumed 233ms CPU time over 16.235s wall clock time, 15.8M memory peak, 19.2K incoming IP traffic, 15.8K outgoing IP traffic. machine # [ 74.643986] k3s[873]: I0919 21:21:40.734215 873 scope.go:122] "RemoveContainer" containerID="c17a33a880488b897fc343e396a9ed15941ec7f46c4af8499925fd7cede5c577" machine # [ 74.646963] k3s[873]: E0919 21:21:40.735087 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"c17a33a880488b897fc343e396a9ed15941ec7f46c4af8499925fd7cede5c577\": not found" containerID="c17a33a880488b897fc343e396a9ed15941ec7f46c4af8499925fd7cede5c577" machine # [ 74.647403] k3s[873]: I0919 21:21:40.735124 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"c17a33a880488b897fc343e396a9ed15941ec7f46c4af8499925fd7cede5c577"} err="failed to get container status \"c17a33a880488b897fc343e396a9ed15941ec7f46c4af8499925fd7cede5c577\": rpc error: code = NotFound desc = an error occurred when try to find container \"c17a33a880488b897fc343e396a9ed15941ec7f46c4af8499925fd7cede5c577\": not found" machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 74.760125] dhcpcd[6337]: eth0: IAID 00:12:34:56 machine # [ 74.760384] dhcpcd[6337]: flannel.1: IAID 2e:3b:95:3e machine # [ 74.760537] dhcpcd[6337]: vethdded1014: IAID c4:bd:7e:14 machine # [ 74.761263] dhcpcd[6337]: veth3ce56192: IAID 32:c4:e4:f4 machine # [ 74.761882] dhcpcd[6337]: vethf99efaf0: IAID 97:8b:ff:f9 machine # [ 74.762632] dhcpcd[6337]: veth7e34664d: IAID b4:1b:17:6d machine # [ 74.795754] k3s[873]: I0919 21:21:40.885945 873 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="092f1a79-f45e-4ae4-90b6-babb0b28134c" path="/var/lib/kubelet/pods/092f1a79-f45e-4ae4-90b6-babb0b28134c/volumes" machine # [ 74.939394] dhcpcd[6337]: eth0: rebinding lease of 10.0.2.15 machine # [ 74.948323] dhcpcd[6337]: eth0: leased 10.0.2.15 for 86400 seconds machine # [ 75.002313] k3s[873]: E0919 21:21:41.092806 873 status.go:71] "Unhandled Error" err="apiserver received an error that is not an metav1.Status: &errors.errorString{s:\"failed to create typed patch object (farm/nix-grpc-farm-scheduler; /v1, Kind=Service): .spec.selector.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0}\"}: failed to create typed patch object (farm/nix-grpc-farm-scheduler; /v1, Kind=Service): .spec.selector.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0}" logger="UnhandledError" machine # [ 75.010320] k3s[873]: I0919 21:21:41.100938 873 alloc.go:329] "allocated clusterIPs" service="farm/nix-grpc-farm" clusterIPs={"IPv4":"10.43.221.65"} machine # [ 75.013086] systemd[1]: Started DHCP Client. machine # [ 75.024159] k3s[873]: E0919 21:21:41.114250 873 status.go:71] "Unhandled Error" err=< machine # [ 75.025908] k3s[873]: apiserver received an error that is not an metav1.Status: &errors.errorString{s:"failed to create typed patch object (farm/nix-grpc-farm-scheduler; apps/v1, Kind=Deployment): errors:\n .spec.selector.matchLabels.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0}\n .spec.template.metadata.labels.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0}"}: failed to create typed patch object (farm/nix-grpc-farm-scheduler; apps/v1, Kind=Deployment): errors: machine # [ 75.035222] k3s[873]: .spec.selector.matchLabels.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0} machine # [ 75.037601] k3s[873]: .spec.template.metadata.labels.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0} machine # [ 75.043346] k3s[873]: > logger="UnhandledError" machine # [ 75.057088] systemd[1]: cri-containerd-20dcd11150271900e2cc11cc4135789ea0c630872f741af5896478df3fdc5b6d.scope: Deactivated successfully. machine # [ 75.059253] systemd[1]: cri-containerd-20dcd11150271900e2cc11cc4135789ea0c630872f741af5896478df3fdc5b6d.scope: Consumed 1.004s CPU time over 2.169s wall clock time, 34.1M memory peak, 3.3M incoming IP traffic, 199.3K outgoing IP traffic. machine # [ 75.070370] dhcpcd[6337]: vethdded1014: soliciting a DHCP lease machine # [ 75.095036] systemd[1]: Created slice libcontainer container kubepods-besteffort-poda13ff034_93af_44be_893a_1c0700eda93f.slice. machine # [ 75.115994] systemd[1]: Created slice libcontainer container kubepods-besteffort-pod14aec9b1_6bcd_41a9_9fbe_48013fa6a384.slice. machine # [ 75.134397] systemd[1]: Created slice libcontainer container kubepods-besteffort-podee902c0f_4a9d_47d1_b6f2_3971baa9c8f7.slice. machine # [ 75.159622] dhcpcd[6337]: vethdded1014: soliciting an IPv6 router machine # [ 75.168451] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-20dcd11150271900e2cc11cc4135789ea0c630872f741af5896478df3fdc5b6d-rootfs.mount: Deactivated successfully. machine # [ 75.168684] systemd[1]: var-lib-kubelet-pods-092f1a79\x2df45e\x2d4ae4\x2d90b6\x2dbabb0b28134c-volume\x2dsubpaths-config-envoy-0.mount: Deactivated successfully. machine # [ 75.169480] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-c2afc9fd2a69142e84b98f5bcd8290981bbf5ae28ec278b784e52c1418159d53-rootfs.mount: Deactivated successfully. machine # [ 75.169948] systemd[1]: run-k3s-containerd-io.containerd.grpc.v1.cri-sandboxes-c2afc9fd2a69142e84b98f5bcd8290981bbf5ae28ec278b784e52c1418159d53-shm.mount: Deactivated successfully. machine # [ 75.170645] systemd[1]: run-netns-cni\x2d9da32475\x2dbbab\x2d9fcb\x2dc212\x2dcec718e98848.mount: Deactivated successfully. machine # [ 75.173138] systemd[1]: var-lib-kubelet-pods-092f1a79\x2df45e\x2d4ae4\x2d90b6\x2dbabb0b28134c-volumes-kubernetes.io\x7esecret-tls\x2dca.mount: Deactivated successfully. machine # [ 75.173881] systemd[1]: var-lib-kubelet-pods-092f1a79\x2df45e\x2d4ae4\x2d90b6\x2dbabb0b28134c-volumes-kubernetes.io\x7eempty\x2ddir-tmp.mount: Deactivated successfully. machine # [ 75.174561] systemd[1]: var-lib-kubelet-pods-092f1a79\x2df45e\x2d4ae4\x2d90b6\x2dbabb0b28134c-volumes-kubernetes.io\x7esecret-tls\x2dlb.mount: Deactivated successfully. machine # [ 75.235437] k3s[873]: I0919 21:21:41.325958 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nixconf\" (UniqueName: \"kubernetes.io/configmap/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-nixconf\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-p5rbt\" (UID: \"14aec9b1-6bcd-41a9-9fbe-48013fa6a384\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-p5rbt" machine # [ 75.242508] k3s[873]: I0919 21:21:41.326021 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nixconf\" (UniqueName: \"kubernetes.io/configmap/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-nixconf\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-kz6rx\" (UID: \"ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-kz6rx" machine # [ 75.249098] k3s[873]: I0919 21:21:41.326051 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"niks3-token\" (UniqueName: \"kubernetes.io/projected/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-niks3-token\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-kz6rx\" (UID: \"ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-kz6rx" machine # [ 75.256182] k3s[873]: I0919 21:21:41.326100 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"niks3-token\" (UniqueName: \"kubernetes.io/projected/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-niks3-token\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-p5rbt\" (UID: \"14aec9b1-6bcd-41a9-9fbe-48013fa6a384\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-p5rbt" machine # [ 75.262435] k3s[873]: I0919 21:21:41.326330 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"build\" (UniqueName: \"kubernetes.io/empty-dir/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-build\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-kz6rx\" (UID: \"ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-kz6rx" machine # [ 75.269367] k3s[873]: I0919 21:21:41.326456 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-ca\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-kz6rx\" (UID: \"ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-kz6rx" machine # [ 75.276262] k3s[873]: I0919 21:21:41.326561 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc\" (UniqueName: \"kubernetes.io/secret/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-oidc\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-kz6rx\" (UID: \"ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-kz6rx" machine # [ 75.283171] k3s[873]: I0919 21:21:41.326663 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrngq\" (UniqueName: \"kubernetes.io/projected/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-kube-api-access-qrngq\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-p5rbt\" (UID: \"14aec9b1-6bcd-41a9-9fbe-48013fa6a384\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-p5rbt" machine # [ 75.290627] k3s[873]: I0919 21:21:41.326716 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a13ff034-93af-44be-893a-1c0700eda93f-config\") pod \"nix-grpc-farm-lb-688d8d547b-p6tbh\" (UID: \"a13ff034-93af-44be-893a-1c0700eda93f\") " pod="farm/nix-grpc-farm-lb-688d8d547b-p6tbh" machine # [ 75.297225] k3s[873]: I0919 21:21:41.326746 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-ca\" (UniqueName: \"kubernetes.io/secret/a13ff034-93af-44be-893a-1c0700eda93f-tls-ca\") pod \"nix-grpc-farm-lb-688d8d547b-p6tbh\" (UID: \"a13ff034-93af-44be-893a-1c0700eda93f\") " pod="farm/nix-grpc-farm-lb-688d8d547b-p6tbh" machine # [ 75.301354] k3s[873]: I0919 21:21:41.327028 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc\" (UniqueName: \"kubernetes.io/secret/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-oidc\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-p5rbt\" (UID: \"14aec9b1-6bcd-41a9-9fbe-48013fa6a384\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-p5rbt" machine # [ 75.305689] k3s[873]: I0919 21:21:41.327101 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nix\" (UniqueName: \"kubernetes.io/empty-dir/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-nix\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-p5rbt\" (UID: \"14aec9b1-6bcd-41a9-9fbe-48013fa6a384\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-p5rbt" machine # [ 75.309894] k3s[873]: I0919 21:21:41.327165 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-ca\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-p5rbt\" (UID: \"14aec9b1-6bcd-41a9-9fbe-48013fa6a384\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-p5rbt" machine # [ 75.313965] k3s[873]: I0919 21:21:41.327191 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nix\" (UniqueName: \"kubernetes.io/empty-dir/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-nix\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-kz6rx\" (UID: \"ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-kz6rx" machine # [ 75.318092] k3s[873]: I0919 21:21:41.327217 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a13ff034-93af-44be-893a-1c0700eda93f-tmp\") pod \"nix-grpc-farm-lb-688d8d547b-p6tbh\" (UID: \"a13ff034-93af-44be-893a-1c0700eda93f\") " pod="farm/nix-grpc-farm-lb-688d8d547b-p6tbh" machine # [ 75.321873] k3s[873]: I0919 21:21:41.327241 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-lb\" (UniqueName: \"kubernetes.io/secret/a13ff034-93af-44be-893a-1c0700eda93f-tls-lb\") pod \"nix-grpc-farm-lb-688d8d547b-p6tbh\" (UID: \"a13ff034-93af-44be-893a-1c0700eda93f\") " pod="farm/nix-grpc-farm-lb-688d8d547b-p6tbh" machine # [ 75.325995] k3s[873]: I0919 21:21:41.327284 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"build\" (UniqueName: \"kubernetes.io/empty-dir/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-build\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-p5rbt\" (UID: \"14aec9b1-6bcd-41a9-9fbe-48013fa6a384\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-p5rbt" machine # [ 75.330736] k3s[873]: I0919 21:21:41.327318 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls\" (UniqueName: \"kubernetes.io/secret/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-tls\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-p5rbt\" (UID: \"14aec9b1-6bcd-41a9-9fbe-48013fa6a384\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-p5rbt" machine # [ 75.335041] k3s[873]: I0919 21:21:41.327388 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls\" (UniqueName: \"kubernetes.io/secret/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-tls\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-kz6rx\" (UID: \"ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-kz6rx" machine # [ 75.340870] k3s[873]: I0919 21:21:41.327454 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgwkg\" (UniqueName: \"kubernetes.io/projected/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-kube-api-access-xgwkg\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-kz6rx\" (UID: \"ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-kz6rx" machine # [ 75.348851] dhcpcd[6337]: veth7e34664d: soliciting a DHCP lease machine # [ 75.350474] dhcpcd[6337]: veth3ce56192: soliciting an IPv6 router machine # [ 75.409817] dhcpcd[6337]: flannel.1: soliciting a DHCP lease machine # [ 75.426715] dhcpcd[6337]: flannel.1: soliciting an IPv6 router machine # [ 75.625840] cni0: port 4(veth931cfdd2) entered blocking state machine # [ 75.628697] cni0: port 4(veth931cfdd2) entered disabled state machine # [ 75.630552] veth931cfdd2: entered allmulticast mode machine # [ 75.632049] veth931cfdd2: entered promiscuous mode machine # [ 75.487769] (udev-worker)[6300]: Network interface NamePolicy= disabled on kernel command line. machine # [ 75.493996] (udev-worker)[6301]: Network interface NamePolicy[ 75.641420] cni0: port 6(veth3bd36ce0) entered blocking state machine # = disabled on kernel command line. machine # [ 75.644676] cni0: port 6(veth3bd36ce0) entered disabled state machine # [ 75.646330] veth3bd36ce0: entered allmulticast mode machine # [ 75.648768] veth3bd36ce0: entered promiscuous mode machine # [ 75.653997] cni0: port 7(vethbd58bc26) entered blocking state machine # [ 75.656537] cni0: port 7(vethbd58bc26) entered disabled state machine # [ 75.658520] vethbd58bc26: entered allmulticast mode machine # [ 75.660447] vethbd58bc26: entered promiscuous mode machine # [ 75.517277] (udev-worker)[6531]: Network interface NamePolicy= disabled on kernel command line. machine # [ 75.665372] cni0: port 4(veth931cfdd2) entered blocking state machine # [ 75.667354] cni0: port 4(veth931cfdd2) entered forwarding state machine # [ 75.674980] cni0: port 6(veth3bd36ce0) entered blocking state machine # [ 75.676389] cni0: port 6(veth3bd36ce0) entered forwarding state machine # [ 75.683125] cni0: port 7(vethbd58bc26) entered blocking state machine # [ 75.684640] cni0: port 7(vethbd58bc26) entered forwarding state machine # [ 75.581335] dhcpcd[6337]: veth931cfdd2: waiting for carrier machine # [ 75.581760] dhcpcd[6337]: vethf99efaf0: soliciting an IPv6 router machine # [ 75.601350] dhcpcd[6337]: veth3bd36ce0: IAID 8a:ff:c8:fc machine # [ 75.601799] dhcpcd[6337]: veth3bd36ce0: adding address fe80::60f6:8aff:feff:c8fc machine # [ 75.626209] dhcpcd[6337]: vethbd58bc26: IAID a2:f2:5b:34 machine # [ 75.626461] dhcpcd[6337]: vethbd58bc26: adding address fe80::545c:a2ff:fef2:5b34 machine # [ 75.626927] dhcpcd[6337]: vethbd58bc26: carrier lost machine # [ 75.638608] k3s[873]: I0919 21:21:41.728869 873 scope.go:122] "RemoveContainer" containerID="1b940306e86be7c1ea430586108c4d8105721acde29fc5373cf6ae9179d0853f" machine # [ 75.646610] k3s[873]: I0919 21:21:41.737220 873 scope.go:122] "RemoveContainer" containerID="20dcd11150271900e2cc11cc4135789ea0c630872f741af5896478df3fdc5b6d" machine # [ 75.648952] k3s[873]: E0919 21:21:41.737523 873 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"helm\" with CrashLoopBackOff: \"back-off 20s restarting failed container=helm pod=helm-install-nix-grpc-farm-zcg5w_kube-system(6d037147-1d29-4e02-b2ab-2f08bc2a576c)\"" pod="kube-system/helm-install-nix-grpc-farm-zcg5w" podUID="6d037147-1d29-4e02-b2ab-2f08bc2a576c" machine # [ 75.679321] dhcpcd[6337]: vethbd58bc26: deleting address fe80::545c:a2ff:fef2:5b34 machine # [ 75.682183] dhcpcd[6337]: veth931cfdd2: carrier acquired machine # [ 75.688729] systemd[1]: Started libcontainer container 842ab80e30eb70b0c51302731a2bea11fe7fcc0f6abda588848c1c69a0591870. machine # [ 75.691587] dhcpcd[6337]: veth931cfdd2: IAID dd:23:c8:d4 machine # [ 75.692327] dhcpcd[6337]: veth931cfdd2: adding address fe80::64d2:ddff:fe23:c8d4 machine # [ 75.692654] dhcpcd[6337]: veth7e34664d: soliciting an IPv6 router machine # [ 75.693496] dhcpcd[6337]: vethbd58bc26: carrier acquired machine # [ 75.697658] systemd[1]: Started libcontainer container 93c05483a319f07b846d4d517b82537da07bd51a556d5e936975872570d979df. machine # [ 75.705857] systemd[1]: Started libcontainer container 0886d9c6e21f1943ae6b21c18cd0c8be5c704363b59a6a6afe6ec88fcf36e21e. machine # [ 75.709963] dhcpcd[6337]: vethbd58bc26: IAID a2:f2:5b:34 machine # [ 75.710739] dhcpcd[6337]: vethbd58bc26: adding address fe80::545c:a2ff:fef2:5b34 machine # [ 75.719099] dhcpcd[6337]: veth3bd36ce0: soliciting a DHCP lease machine # [ 75.741461] dhcpcd[6337]: eth0: soliciting an IPv6 router machine # [ 75.768627] dhcpcd[6337]: eth0: Router Advertisement from fe80::2 machine # [ 75.769042] dhcpcd[6337]: eth0: adding address fec0::5054:ff:fe12:3456/64 machine # [ 75.832600] dhcpcd[6337]: veth3ce56192: soliciting a DHCP lease machine # [ 75.890753] systemd[1]: Started libcontainer container ef71cca7acd75a946500d9b8aecc57e84a9dd3bf5f73da02d2a4f7b91eb0372a. machine # [ 75.893447] systemd[1]: Started libcontainer container 2ca001180238662d79248477aab8b8fd022667f15dcebcbb65a7cc43ea506e36. machine # [ 75.918291] systemd[1]: Started libcontainer container e401edb9f53e2cd04d0f3d3fedfd1000c5ef5785f2541cc6cfdc683452fab367. machine # [ 75.923288] dhcpcd[6337]: vethf99efaf0: soliciting a DHCP lease machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 76.088894] systemd[1]: cri-containerd-4b879e3b41a4ba18a45761add1567ccc239051125dd77d8925423ba08604c659.scope: Deactivated successfully. machine # [ 76.092679] systemd[1]: cri-containerd-4b879e3b41a4ba18a45761add1567ccc239051125dd77d8925423ba08604c659.scope: Consumed 142ms CPU time over 14.121s wall clock time, 11.5M memory peak, 2.4M read from disk, 8.9K incoming IP traffic, 9.2K outgoing IP traffic. machine # [ 76.132430] systemd[1]: cri-containerd-6385b2c3bd9aec029cef3c5d85fb1d732d4415793015d40b4afcfd7ac14a310c.scope: Deactivated successfully. machine # [ 76.134182] systemd[1]: cri-containerd-6385b2c3bd9aec029cef3c5d85fb1d732d4415793015d40b4afcfd7ac14a310c.scope: Consumed 157ms CPU time over 14.158s wall clock time, 33.4M memory peak, 25.8M read from disk, 8.9K incoming IP traffic, 9.6K outgoing IP traffic. machine # [ 76.184029] dhcpcd[6337]: veth931cfdd2: soliciting a DHCP lease machine # [ 76.200653] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-4b879e3b41a4ba18a45761add1567ccc239051125dd77d8925423ba08604c659-rootfs.mount: Deactivated successfully. machine # [ 76.218246] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-6385b2c3bd9aec029cef3c5d85fb1d732d4415793015d40b4afcfd7ac14a310c-rootfs.mount: Deactivated successfully. machine # [ 76.562123] systemd[1]: cri-containerd-d6e0a2df34faa36683dc2acc8692f4ee6d363609a92a6501fc74a7ddbeb39cb5.scope: Deactivated successfully. machine # [ 76.565830] systemd[1]: cri-containerd-17e462c37879905c86009df2f6562ccb4719332819d0125f9e21a35695c3a79d.scope: Deactivated successfully. machine # [ 76.618969] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-d6e0a2df34faa36683dc2acc8692f4ee6d363609a92a6501fc74a7ddbeb39cb5-rootfs.mount: Deactivated successfully. machine # [ 76.625803] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-17e462c37879905c86009df2f6562ccb4719332819d0125f9e21a35695c3a79d-rootfs.mount: Deactivated successfully. machine # [ 76.748889] systemd[1]: cri-containerd-ef71cca7acd75a946500d9b8aecc57e84a9dd3bf5f73da02d2a4f7b91eb0372a.scope: Deactivated successfully. machine # [ 76.749654] systemd[1]: cri-containerd-ef71cca7acd75a946500d9b8aecc57e84a9dd3bf5f73da02d2a4f7b91eb0372a.scope: Consumed 588ms CPU time over 857ms wall clock time, 309.8M memory peak, 116K read from disk, 275.8M written to disk. machine # [ 76.776568] systemd[1]: cri-containerd-2ca001180238662d79248477aab8b8fd022667f15dcebcbb65a7cc43ea506e36.scope: Deactivated successfully. machine # [ 76.777914] systemd[1]: cri-containerd-2ca001180238662d79248477aab8b8fd022667f15dcebcbb65a7cc43ea506e36.scope: Consumed 594ms CPU time over 883ms wall clock time, 312.1M memory peak, 112K read from disk, 275.8M written to disk. machine # [ 77.049776] cni0: port 3(vethf99efaf0) entered disabled state machine # [ 76.903087] dhcpcd[6337]: vethf99efaf0: carrier lost machine # [ 77.052431] vethf99efaf0 (unregistering): left allmulticast mode machine # [ 77.054437] vethf99efaf0 (unregistering): left promiscuous mode machine # [ 77.056031] cni0: port 3(vethf99efaf0) entered disabled state machine # [ 77.059767] cni0: port 5(veth7e34664d) entered disabled state machine # [ 77.063447] veth7e34664d (unregistering): left allmulticast mode machine # [ 77.065467] veth7e34664d (unregistering): left promiscuous mode machine # [ 77.067531] cni0: port 5(veth7e34664d) entered disabled state machine # [ 76.964081] k3s[873]: I0919 21:21:43.054219 873 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="farm/nix-grpc-farm-lb-688d8d547b-p6tbh" podStartSLOduration=2.054190817 podStartE2EDuration="2.054190817s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 21:21:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 21:21:42.82440435 +0000 UTC m=+58.408248808" watchObservedRunningTime="2026-09-19 21:21:43.054190817 +0000 UTC m=+58.638035555" machine # [ 77.057119] k3s[873]: I0919 21:21:43.147633 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/586d96c1-2512-46a6-99e0-896b90a7f605-nixconf\" (UniqueName: \"kubernetes.io/configmap/586d96c1-2512-46a6-99e0-896b90a7f605-nixconf\") pod \"586d96c1-2512-46a6-99e0-896b90a7f605\" (UID: \"586d96c1-2512-46a6-99e0-896b90a7f605\") " machine # [ 77.057363] k3s[873]: I0919 21:21:43.147726 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/119b9bfe-8f78-4a41-9061-7e2df29c5a85-nixconf\" (UniqueName: \"kubernetes.io/configmap/119b9bfe-8f78-4a41-9061-7e2df29c5a85-nixconf\") pod \"119b9bfe-8f78-4a41-9061-7e2df29c5a85\" (UID: \"119b9bfe-8f78-4a41-9061-7e2df29c5a85\") " machine # [ 77.058136] k3s[873]: I0919 21:21:43.147758 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/586d96c1-2512-46a6-99e0-896b90a7f605-niks3-token\" (UniqueName: \"kubernetes.io/projected/586d96c1-2512-46a6-99e0-896b90a7f605-niks3-token\") pod \"586d96c1-2512-46a6-99e0-896b90a7f605\" (UID: \"586d96c1-2512-46a6-99e0-896b90a7f605\") " machine # [ 77.058739] k3s[873]: I0919 21:21:43.147801 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/586d96c1-2512-46a6-99e0-896b90a7f605-ca\" (UniqueName: \"kubernetes.io/secret/586d96c1-2512-46a6-99e0-896b90a7f605-ca\") pod \"586d96c1-2512-46a6-99e0-896b90a7f605\" (UID: \"586d96c1-2512-46a6-99e0-896b90a7f605\") " machine # [ 77.059660] k3s[873]: I0919 21:21:43.147840 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/586d96c1-2512-46a6-99e0-896b90a7f605-build\" (UniqueName: \"kubernetes.io/empty-dir/586d96c1-2512-46a6-99e0-896b90a7f605-build\") pod \"586d96c1-2512-46a6-99e0-896b90a7f605\" (UID: \"586d96c1-2512-46a6-99e0-896b90a7f605\") " machine # [ 77.060965] k3s[873]: I0919 21:21:43.147882 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/119b9bfe-8f78-4a41-9061-7e2df29c5a85-build\" (UniqueName: \"kubernetes.io/empty-dir/119b9bfe-8f78-4a41-9061-7e2df29c5a85-build\") pod \"119b9bfe-8f78-4a41-9061-7e2df29c5a85\" (UID: \"119b9bfe-8f78-4a41-9061-7e2df29c5a85\") " machine # [ 77.061883] k3s[873]: I0919 21:21:43.147919 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/119b9bfe-8f78-4a41-9061-7e2df29c5a85-oidc\" (UniqueName: \"kubernetes.io/secret/119b9bfe-8f78-4a41-9061-7e2df29c5a85-oidc\") pod \"119b9bfe-8f78-4a41-9061-7e2df29c5a85\" (UID: \"119b9bfe-8f78-4a41-9061-7e2df29c5a85\") " machine # [ 77.063114] k3s[873]: I0919 21:21:43.147951 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/119b9bfe-8f78-4a41-9061-7e2df29c5a85-kube-api-access-fms9r\" (UniqueName: \"kubernetes.io/projected/119b9bfe-8f78-4a41-9061-7e2df29c5a85-kube-api-access-fms9r\") pod \"119b9bfe-8f78-4a41-9061-7e2df29c5a85\" (UID: \"119b9bfe-8f78-4a41-9061-7e2df29c5a85\") " machine # [ 77.063734] k3s[873]: I0919 21:21:43.147981 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/119b9bfe-8f78-4a41-9061-7e2df29c5a85-tls\" (UniqueName: \"kubernetes.io/secret/119b9bfe-8f78-4a41-9061-7e2df29c5a85-tls\") pod \"119b9bfe-8f78-4a41-9061-7e2df29c5a85\" (UID: \"119b9bfe-8f78-4a41-9061-7e2df29c5a85\") " machine # [ 77.064487] k3s[873]: I0919 21:21:43.148011 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/586d96c1-2512-46a6-99e0-896b90a7f605-tls\" (UniqueName: \"kubernetes.io/secret/586d96c1-2512-46a6-99e0-896b90a7f605-tls\") pod \"586d96c1-2512-46a6-99e0-896b90a7f605\" (UID: \"586d96c1-2512-46a6-99e0-896b90a7f605\") " machine # [ 77.064860] k3s[873]: I0919 21:21:43.148037 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/119b9bfe-8f78-4a41-9061-7e2df29c5a85-nix\" (UniqueName: \"kubernetes.io/empty-dir/119b9bfe-8f78-4a41-9061-7e2df29c5a85-nix\") pod \"119b9bfe-8f78-4a41-9061-7e2df29c5a85\" (UID: \"119b9bfe-8f78-4a41-9061-7e2df29c5a85\") " machine # [ 77.065379] k3s[873]: I0919 21:21:43.148068 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/586d96c1-2512-46a6-99e0-896b90a7f605-oidc\" (UniqueName: \"kubernetes.io/secret/586d96c1-2512-46a6-99e0-896b90a7f605-oidc\") pod \"586d96c1-2512-46a6-99e0-896b90a7f605\" (UID: \"586d96c1-2512-46a6-99e0-896b90a7f605\") " machine # [ 77.067804] k3s[873]: I0919 21:21:43.148093 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/119b9bfe-8f78-4a41-9061-7e2df29c5a85-ca\" (UniqueName: \"kubernetes.io/secret/119b9bfe-8f78-4a41-9061-7e2df29c5a85-ca\") pod \"119b9bfe-8f78-4a41-9061-7e2df29c5a85\" (UID: \"119b9bfe-8f78-4a41-9061-7e2df29c5a85\") " machine # [ 77.069558] k3s[873]: I0919 21:21:43.148119 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/119b9bfe-8f78-4a41-9061-7e2df29c5a85-niks3-token\" (UniqueName: \"kubernetes.io/projected/119b9bfe-8f78-4a41-9061-7e2df29c5a85-niks3-token\") pod \"119b9bfe-8f78-4a41-9061-7e2df29c5a85\" (UID: \"119b9bfe-8f78-4a41-9061-7e2df29c5a85\") " machine # [ 77.115259] k3s[873]: I0919 21:21:43.148119 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/586d96c1-2512-46a6-99e0-896b90a7f605-nixconf" pod "586d96c1-2512-46a6-99e0-896b90a7f605" (UID: "586d96c1-2512-46a6-99e0-896b90a7f605"). InnerVolumeSpecName "nixconf". PluginName "kubernetes.io/configmap", VolumeGIDValue "" machine # [ 77.115879] k3s[873]: I0919 21:21:43.148148 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/586d96c1-2512-46a6-99e0-896b90a7f605-nix\" (UniqueName: \"kubernetes.io/empty-dir/586d96c1-2512-46a6-99e0-896b90a7f605-nix\") pod \"586d96c1-2512-46a6-99e0-896b90a7f605\" (UID: \"586d96c1-2512-46a6-99e0-896b90a7f605\") " machine # [ 77.116534] k3s[873]: I0919 21:21:43.148175 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/586d96c1-2512-46a6-99e0-896b90a7f605-kube-api-access-fxktc\" (UniqueName: \"kubernetes.io/projected/586d96c1-2512-46a6-99e0-896b90a7f605-kube-api-access-fxktc\") pod \"586d96c1-2512-46a6-99e0-896b90a7f605\" (UID: \"586d96c1-2512-46a6-99e0-896b90a7f605\") " machine # [ 77.117030] k3s[873]: I0919 21:21:43.148222 873 reconciler_common.go:299] "Volume detached for volume \"nixconf\" (UniqueName: \"kubernetes.io/configmap/586d96c1-2512-46a6-99e0-896b90a7f605-nixconf\") on node \"machine\" DevicePath \"\"" machine # [ 77.117502] k3s[873]: I0919 21:21:43.173033 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/119b9bfe-8f78-4a41-9061-7e2df29c5a85-nixconf" pod "119b9bfe-8f78-4a41-9061-7e2df29c5a85" (UID: "119b9bfe-8f78-4a41-9061-7e2df29c5a85"). InnerVolumeSpecName "nixconf". PluginName "kubernetes.io/configmap", VolumeGIDValue "" machine # [ 77.139174] k3s[873]: I0919 21:21:43.229494 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/119b9bfe-8f78-4a41-9061-7e2df29c5a85-build" pod "119b9bfe-8f78-4a41-9061-7e2df29c5a85" (UID: "119b9bfe-8f78-4a41-9061-7e2df29c5a85"). InnerVolumeSpecName "build". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 77.142923] k3s[873]: I0919 21:21:43.233490 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/586d96c1-2512-46a6-99e0-896b90a7f605-build" pod "586d96c1-2512-46a6-99e0-896b90a7f605" (UID: "586d96c1-2512-46a6-99e0-896b90a7f605"). InnerVolumeSpecName "build". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 77.144110] k3s[873]: I0919 21:21:43.234619 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/586d96c1-2512-46a6-99e0-896b90a7f605-ca" pod "586d96c1-2512-46a6-99e0-896b90a7f605" (UID: "586d96c1-2512-46a6-99e0-896b90a7f605"). InnerVolumeSpecName "ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 77.146135] k3s[873]: I0919 21:21:43.236708 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/119b9bfe-8f78-4a41-9061-7e2df29c5a85-niks3-token" pod "119b9bfe-8f78-4a41-9061-7e2df29c5a85" (UID: "119b9bfe-8f78-4a41-9061-7e2df29c5a85"). InnerVolumeSpecName "niks3-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" machine # [ 77.147761] k3s[873]: I0919 21:21:43.237240 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/119b9bfe-8f78-4a41-9061-7e2df29c5a85-tls" pod "119b9bfe-8f78-4a41-9061-7e2df29c5a85" (UID: "119b9bfe-8f78-4a41-9061-7e2df29c5a85"). InnerVolumeSpecName "tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 77.149068] k3s[873]: I0919 21:21:43.237294 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/586d96c1-2512-46a6-99e0-896b90a7f605-niks3-token" pod "586d96c1-2512-46a6-99e0-896b90a7f605" (UID: "586d96c1-2512-46a6-99e0-896b90a7f605"). InnerVolumeSpecName "niks3-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" machine # [ 77.153034] k3s[873]: I0919 21:21:43.242641 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/586d96c1-2512-46a6-99e0-896b90a7f605-kube-api-access-fxktc" pod "586d96c1-2512-46a6-99e0-896b90a7f605" (UID: "586d96c1-2512-46a6-99e0-896b90a7f605"). InnerVolumeSpecName "kube-api-access-fxktc". PluginName "kubernetes.io/projected", VolumeGIDValue "" machine # [ 77.153683] k3s[873]: I0919 21:21:43.242295 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/119b9bfe-8f78-4a41-9061-7e2df29c5a85-kube-api-access-fms9r" pod "119b9bfe-8f78-4a41-9061-7e2df29c5a85" (UID: "119b9bfe-8f78-4a41-9061-7e2df29c5a85"). InnerVolumeSpecName "kube-api-access-fms9r". PluginName "kubernetes.io/projected", VolumeGIDValue "" machine # [ 77.155127] k3s[873]: I0919 21:21:43.245233 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/119b9bfe-8f78-4a41-9061-7e2df29c5a85-ca" pod "119b9bfe-8f78-4a41-9061-7e2df29c5a85" (UID: "119b9bfe-8f78-4a41-9061-7e2df29c5a85"). InnerVolumeSpecName "ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 77.158906] k3s[873]: I0919 21:21:43.249215 873 reconciler_common.go:299] "Volume detached for volume \"niks3-token\" (UniqueName: \"kubernetes.io/projected/586d96c1-2512-46a6-99e0-896b90a7f605-niks3-token\") on node \"machine\" DevicePath \"\"" machine # [ 77.160083] k3s[873]: I0919 21:21:43.249284 873 reconciler_common.go:299] "Volume detached for volume \"ca\" (UniqueName: \"kubernetes.io/secret/586d96c1-2512-46a6-99e0-896b90a7f605-ca\") on node \"machine\" DevicePath \"\"" machine # [ 77.160676] k3s[873]: I0919 21:21:43.249298 873 reconciler_common.go:299] "Volume detached for volume \"build\" (UniqueName: \"kubernetes.io/empty-dir/586d96c1-2512-46a6-99e0-896b90a7f605-build\") on node \"machine\" DevicePath \"\"" machine # [ 77.161716] k3s[873]: I0919 21:21:43.249313 873 reconciler_common.go:299] "Volume detached for volume \"build\" (UniqueName: \"kubernetes.io/empty-dir/119b9bfe-8f78-4a41-9061-7e2df29c5a85-build\") on node \"machine\" DevicePath \"\"" machine # [ 77.162804] k3s[873]: I0919 21:21:43.249339 873 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fms9r\" (UniqueName: \"kubernetes.io/projected/119b9bfe-8f78-4a41-9061-7e2df29c5a85-kube-api-access-fms9r\") on node \"machine\" DevicePath \"\"" machine # [ 77.163274] k3s[873]: I0919 21:21:43.249354 873 reconciler_common.go:299] "Volume detached for volume \"tls\" (UniqueName: \"kubernetes.io/secret/119b9bfe-8f78-4a41-9061-7e2df29c5a85-tls\") on node \"machine\" DevicePath \"\"" machine # [ 77.163899] k3s[873]: I0919 21:21:43.249366 873 reconciler_common.go:299] "Volume detached for volume \"ca\" (UniqueName: \"kubernetes.io/secret/119b9bfe-8f78-4a41-9061-7e2df29c5a85-ca\") on node \"machine\" DevicePath \"\"" machine # [ 77.164660] k3s[873]: I0919 21:21:43.249380 873 reconciler_common.go:299] "Volume detached for volume \"niks3-token\" (UniqueName: \"kubernetes.io/projected/119b9bfe-8f78-4a41-9061-7e2df29c5a85-niks3-token\") on node \"machine\" DevicePath \"\"" machine # [ 77.165350] k3s[873]: I0919 21:21:43.249393 873 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxktc\" (UniqueName: \"kubernetes.io/projected/586d96c1-2512-46a6-99e0-896b90a7f605-kube-api-access-fxktc\") on node \"machine\" DevicePath \"\"" machine # [ 77.165632] k3s[873]: I0919 21:21:43.249405 873 reconciler_common.go:299] "Volume detached for volume \"nixconf\" (UniqueName: \"kubernetes.io/configmap/119b9bfe-8f78-4a41-9061-7e2df29c5a85-nixconf\") on node \"machine\" DevicePath \"\"" machine # [ 77.169993] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-2ca001180238662d79248477aab8b8fd022667f15dcebcbb65a7cc43ea506e36-rootfs.mount: Deactivated successfully. machine # [ 77.172794] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-ef71cca7acd75a946500d9b8aecc57e84a9dd3bf5f73da02d2a4f7b91eb0372a-rootfs.mount: Deactivated successfully. machine # [ 77.173498] systemd[1]: var-lib-kubelet-pods-586d96c1\x2d2512\x2d46a6\x2d99e0\x2d896b90a7f605-volume\x2dsubpaths-nix-gc-0.mount: Deactivated successfully. machine # [ 77.174071] systemd[1]: var-lib-kubelet-pods-119b9bfe\x2d8f78\x2d4a41\x2d9061\x2d7e2df29c5a85-volume\x2dsubpaths-nix-gc-0.mount: Deactivated successfully. machine # [ 77.174502] systemd[1]: run-k3s-containerd-io.containerd.grpc.v1.cri-sandboxes-d6e0a2df34faa36683dc2acc8692f4ee6d363609a92a6501fc74a7ddbeb39cb5-shm.mount: Deactivated successfully. machine # [ 77.225764] systemd[1]: run-k3s-containerd-io.containerd.grpc.v1.cri-sandboxes-17e462c37879905c86009df2f6562ccb4719332819d0125f9e21a35695c3a79d-shm.mount: Deactivated successfully. machine # [ 77.226562] systemd[1]: run-netns-cni\x2dedc5722c\x2d9903\x2df239\x2d92a4\x2da09e40b54596.mount: Deactivated successfully. machine # [ 77.227039] systemd[1]: run-netns-cni\x2ddba39b00\x2df598\x2d1375\x2d9cf7\x2d212f38266f23.mount: Deactivated successfully. machine # [ 77.227525] systemd[1]: var-lib-kubelet-pods-119b9bfe\x2d8f78\x2d4a41\x2d9061\x2d7e2df29c5a85-volumes-kubernetes.io\x7eprojected-niks3\x2dtoken.mount: Deactivated successfully. machine # [ 77.228191] systemd[1]: var-lib-kubelet-pods-586d96c1\x2d2512\x2d46a6\x2d99e0\x2d896b90a7f605-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dfxktc.mount: Deactivated successfully. machine # [ 77.228597] systemd[1]: var-lib-kubelet-pods-119b9bfe\x2d8f78\x2d4a41\x2d9061\x2d7e2df29c5a85-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dfms9r.mount: Deactivated successfully. machine # [ 77.229338] systemd[1]: var-lib-kubelet-pods-586d96c1\x2d2512\x2d46a6\x2d99e0\x2d896b90a7f605-volumes-kubernetes.io\x7eprojected-niks3\x2dtoken.mount: Deactivated successfully. machine # [ 77.229935] systemd[1]: var-lib-kubelet-pods-119b9bfe\x2d8f78\x2d4a41\x2d9061\x2d7e2df29c5a85-volumes-kubernetes.io\x7esecret-oidc.mount: Deactivated successfully. machine # [ 77.230792] systemd[1]: var-lib-kubelet-pods-586d96c1\x2d2512\x2d46a6\x2d99e0\x2d896b90a7f605-volumes-kubernetes.io\x7esecret-tls.mount: Deactivated successfully. machine # [ 77.231436] systemd[1]: var-lib-kubelet-pods-119b9bfe\x2d8f78\x2d4a41\x2d9061\x2d7e2df29c5a85-volumes-kubernetes.io\x7esecret-ca.mount: Deactivated successfully. machine # [ 77.231824] systemd[1]: var-lib-kubelet-pods-586d96c1\x2d2512\x2d46a6\x2d99e0\x2d896b90a7f605-volumes-kubernetes.io\x7esecret-oidc.mount: Deactivated successfully. machine # [ 77.232567] systemd[1]: var-lib-kubelet-pods-586d96c1\x2d2512\x2d46a6\x2d99e0\x2d896b90a7f605-volumes-kubernetes.io\x7esecret-ca.mount: Deactivated successfully. machine # [ 77.233057] systemd[1]: var-lib-kubelet-pods-119b9bfe\x2d8f78\x2d4a41\x2d9061\x2d7e2df29c5a85-volumes-kubernetes.io\x7esecret-tls.mount: Deactivated successfully. machine # [ 77.254132] k3s[873]: I0919 21:21:43.344576 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/586d96c1-2512-46a6-99e0-896b90a7f605-oidc" pod "586d96c1-2512-46a6-99e0-896b90a7f605" (UID: "586d96c1-2512-46a6-99e0-896b90a7f605"). InnerVolumeSpecName "oidc". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 77.260639] k3s[873]: I0919 21:21:43.344918 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/586d96c1-2512-46a6-99e0-896b90a7f605-tls" pod "586d96c1-2512-46a6-99e0-896b90a7f605" (UID: "586d96c1-2512-46a6-99e0-896b90a7f605"). InnerVolumeSpecName "tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 77.265338] k3s[873]: I0919 21:21:43.345391 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/119b9bfe-8f78-4a41-9061-7e2df29c5a85-oidc" pod "119b9bfe-8f78-4a41-9061-7e2df29c5a85" (UID: "119b9bfe-8f78-4a41-9061-7e2df29c5a85"). InnerVolumeSpecName "oidc". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 77.271740] k3s[873]: I0919 21:21:43.353968 873 reconciler_common.go:299] "Volume detached for volume \"oidc\" (UniqueName: \"kubernetes.io/secret/119b9bfe-8f78-4a41-9061-7e2df29c5a85-oidc\") on node \"machine\" DevicePath \"\"" machine # [ 77.275339] k3s[873]: I0919 21:21:43.353991 873 reconciler_common.go:299] "Volume detached for volume \"tls\" (UniqueName: \"kubernetes.io/secret/586d96c1-2512-46a6-99e0-896b90a7f605-tls\") on node \"machine\" DevicePath \"\"" machine # [ 77.279353] k3s[873]: I0919 21:21:43.354004 873 reconciler_common.go:299] "Volume detached for volume \"oidc\" (UniqueName: \"kubernetes.io/secret/586d96c1-2512-46a6-99e0-896b90a7f605-oidc\") on node \"machine\" DevicePath \"\"" machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 77.284320] systemd[1]: var-lib-kubelet-pods-586d96c1\x2d2512\x2d46a6\x2d99e0\x2d896b90a7f605-volume\x2dsubpaths-nix-nix\x2ddaemon-0.mount: Deactivated successfully. machine # [ 77.286271] systemd[1]: var-lib-kubelet-pods-119b9bfe\x2d8f78\x2d4a41\x2d9061\x2d7e2df29c5a85-volume\x2dsubpaths-nix-nix\x2ddaemon-0.mount: Deactivated successfully. machine # [ 77.289104] systemd[1]: var-lib-kubelet-pods-586d96c1\x2d2512\x2d46a6\x2d99e0\x2d896b90a7f605-volume\x2dsubpaths-nix-nix\x2dgrpc\x2ddaemon-0.mount: Deactivated successfully. machine # [ 77.291868] systemd[1]: var-lib-kubelet-pods-119b9bfe\x2d8f78\x2d4a41\x2d9061\x2d7e2df29c5a85-volume\x2dsubpaths-nix-nix\x2dgrpc\x2ddaemon-0.mount: Deactivated successfully. machine # [ 77.300487] dhcpcd[6337]: vethbd58bc26: soliciting a DHCP lease machine # [ 77.302135] dhcpcd[6337]: dhcp_readbpf: unexpected event 0x0100 machine # [ 77.303833] dhcpcd[6337]: dhcp_readbpf: veth7e34664d: Network is down machine # [ 77.306289] dhcpcd[6337]: vethf99efaf0: removing interface machine # [ 77.329858] dhcpcd[6337]: veth7e34664d: removing interface machine # [ 77.478626] dhcpcd[6337]: vethbd58bc26: soliciting an IPv6 router machine # [ 77.480979] k3s[873]: I0919 21:21:43.571553 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/119b9bfe-8f78-4a41-9061-7e2df29c5a85-nix" pod "119b9bfe-8f78-4a41-9061-7e2df29c5a85" (UID: "119b9bfe-8f78-4a41-9061-7e2df29c5a85"). InnerVolumeSpecName "nix". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 77.487565] k3s[873]: I0919 21:21:43.578123 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/586d96c1-2512-46a6-99e0-896b90a7f605-nix" pod "586d96c1-2512-46a6-99e0-896b90a7f605" (UID: "586d96c1-2512-46a6-99e0-896b90a7f605"). InnerVolumeSpecName "nix". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 77.565909] k3s[873]: I0919 21:21:43.656448 873 reconciler_common.go:299] "Volume detached for volume \"nix\" (UniqueName: \"kubernetes.io/empty-dir/119b9bfe-8f78-4a41-9061-7e2df29c5a85-nix\") on node \"machine\" DevicePath \"\"" machine # [ 77.566367] k3s[873]: I0919 21:21:43.656500 873 reconciler_common.go:299] "Volume detached for volume \"nix\" (UniqueName: \"kubernetes.io/empty-dir/586d96c1-2512-46a6-99e0-896b90a7f605-nix\") on node \"machine\" DevicePath \"\"" machine # [ 77.625952] dhcpcd[6337]: veth3bd36ce0: soliciting an IPv6 router machine # [ 77.735986] k3s[873]: I0919 21:21:43.826344 873 scope.go:122] "RemoveContainer" containerID="ca7c7d668571350bc0a583a0509fc69ee36a2ae291db636e21c4a9faacb589b0" machine # [ 77.755342] k3s[873]: I0919 21:21:43.845814 873 scope.go:122] "RemoveContainer" containerID="4b879e3b41a4ba18a45761add1567ccc239051125dd77d8925423ba08604c659" machine # [ 77.763056] systemd[1]: Removed slice libcontainer container kubepods-besteffort-pod119b9bfe_8f78_4a41_9061_7e2df29c5a85.slice. machine # [ 77.767081] systemd[1]: kubepods-besteffort-pod119b9bfe_8f78_4a41_9061_7e2df29c5a85.slice: Consumed 794ms CPU time over 19.375s wall clock time, 322.7M memory peak, 2.5M read from disk, 283.8M written to disk, 8.9K incoming IP traffic, 9.2K outgoing IP traffic. machine # [ 77.782094] k3s[873]: I0919 21:21:43.872123 873 scope.go:122] "RemoveContainer" containerID="3a7bbd4f1c81ef5888b0c5f51620e082100da74fde6e0b3b98ec518e78f23979" machine # [ 77.785620] systemd[1]: Removed slice libcontainer container kubepods-besteffort-pod586d96c1_2512_46a6_99e0_896b90a7f605.slice. machine # [ 77.786409] systemd[1]: kubepods-besteffort-pod586d96c1_2512_46a6_99e0_896b90a7f605.slice: Consumed 823ms CPU time over 19.432s wall clock time, 309M memory peak, 48.2M read from disk, 283.8M written to disk, 8.9K incoming IP traffic, 9.6K outgoing IP traffic. machine # [ 77.800326] k3s[873]: I0919 21:21:43.890794 873 scope.go:122] "RemoveContainer" containerID="eecef8c188fddf9f78b4dba9edf9ec1d0176d715fac5df3b7594c6ee0a934ea4" machine # [ 77.811145] k3s[873]: I0919 21:21:43.901732 873 scope.go:122] "RemoveContainer" containerID="b1d0716c5bf5b2c7957dff26421b87f642b389f83fa1b70d2fcf1306ceb19171" machine # [ 77.823442] k3s[873]: I0919 21:21:43.913801 873 scope.go:122] "RemoveContainer" containerID="6385b2c3bd9aec029cef3c5d85fb1d732d4415793015d40b4afcfd7ac14a310c" machine # [ 77.849325] k3s[873]: I0919 21:21:43.939799 873 scope.go:122] "RemoveContainer" containerID="52a3a76a1104d26dfa13587d87d299663e3528f525af01557afcc88b6191300e" machine # [ 77.861780] k3s[873]: I0919 21:21:43.952365 873 scope.go:122] "RemoveContainer" containerID="1f4073b56af6205458e8ad7904383923bb02fac5947d93b909bf88a24ef0ab17" machine # [ 77.895348] systemd[1]: Started libcontainer container 9b3a3fbed84ac80bd2aa2cfb50436443a107baa984bd335887fa0cc46391b566. machine # [ 77.896874] systemd[1]: Started libcontainer container be6a4add840fdb41707c5a7ca7395a86099a959898408c9924e96458d390efd3. machine # [ 77.989599] dhcpcd[6337]: veth931cfdd2: soliciting an IPv6 router machine # [ 78.068380] systemd[1]: Started libcontainer container ea4f4a1497f3701457603cc65c862dade042a76ec1d091744b857918b0391260. machine # [ 78.072075] systemd[1]: Started libcontainer container f38efc5b0fa33e210961b12d0c47bef60d77aa6dd54bc82927a194d818f2ec13. machine # [ 78.243646] systemd[1]: Started libcontainer container 6b78b1e411112a33a981d74b7f493b1fe70f517bab7639aaad29f194bb734cda. machine # [ 78.246082] systemd[1]: Started libcontainer container f814918b1fe064957a4c252acea7f865a9befa76969c4de37362e980d4ac0922. machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 78.797075] k3s[873]: I0919 21:21:44.886050 873 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="119b9bfe-8f78-4a41-9061-7e2df29c5a85" path="/var/lib/kubelet/pods/119b9bfe-8f78-4a41-9061-7e2df29c5a85/volumes" machine # [ 78.797302] k3s[873]: I0919 21:21:44.886811 873 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="586d96c1-2512-46a6-99e0-896b90a7f605" path="/var/lib/kubelet/pods/586d96c1-2512-46a6-99e0-896b90a7f605/volumes" machine # [ 78.845173] k3s[873]: I0919 21:21:44.935650 873 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-p5rbt" podStartSLOduration=3.113234914 podStartE2EDuration="3.935621278s" totalImagesPullingTime="0s" totalInitContainerRuntime="822.386364ms" isStatefulPod=false podCreationTimestamp="2026-09-19 21:21:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 21:21:44.91322765 +0000 UTC m=+60.497072388" watchObservedRunningTime="2026-09-19 21:21:44.935621278 +0000 UTC m=+60.519466295" machine # [ 78.853635] k3s[873]: I0919 21:21:44.944211 873 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-kz6rx" podStartSLOduration=3.138672784 podStartE2EDuration="3.944196391s" totalImagesPullingTime="0s" totalInitContainerRuntime="805.523607ms" isStatefulPod=false podCreationTimestamp="2026-09-19 21:21:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 21:21:44.935056402 +0000 UTC m=+60.518900860" watchObservedRunningTime="2026-09-19 21:21:44.944196391 +0000 UTC m=+60.528041407" machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 80.720248] dhcpcd[6337]: veth3bd36ce0: probing for an IPv4LL address machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 81.184534] dhcpcd[6337]: veth931cfdd2: probing for an IPv4LL address machine # [ 82.296856] dhcpcd[6337]: vethbd58bc26: probing for an IPv4LL address machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 84.813535] dhcpcd[6337]: vethdded1014: using IPv4LL address 169.254.234.42 machine # [ 84.813762] dhcpcd[6337]: flannel.1: adding route to 169.254.0.0/16 machine # [ 84.814048] dhcpcd[6337]: vethdded1014: adding route to 169.254.0.0/16 machine # [ 84.814657] dhcpcd[6337]: veth3ce56192: adding route to 169.254.0.0/16 machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 85.175841] dhcpcd[6337]: flannel.1: using IPv4LL address 169.254.196.202 machine # [ 85.509423] dhcpcd[6337]: veth3bd36ce0: using IPv4LL address 169.254.102.120 machine # [ 85.510500] dhcpcd[6337]: veth3bd36ce0: adding route to 169.254.0.0/16 machine # [ 85.776556] dhcpcd[6337]: veth3ce56192: using IPv4LL address 169.254.229.31 machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 86.321533] dhcpcd[6337]: veth931cfdd2: using IPv4LL address 169.254.114.175 machine # [ 86.325391] dhcpcd[6337]: veth931cfdd2: adding route to 169.254.0.0/16 machine # [ 87.162943] dhcpcd[6337]: vethdded1014: no IPv6 Routers available machine # [ 87.247219] dhcpcd[6337]: vethbd58bc26: using IPv4LL address 169.254.236.208 machine # [ 87.247992] dhcpcd[6337]: vethbd58bc26: adding route to 169.254.0.0/16 machine # [ 87.298304] dhcpcd[6337]: veth3ce56192: no IPv6 Routers available machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 87.428273] dhcpcd[6337]: flannel.1: no IPv6 Routers available machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 89.481492] dhcpcd[6337]: vethbd58bc26: no IPv6 Routers available machine # [ 89.628581] dhcpcd[6337]: veth3bd36ce0: no IPv6 Routers available machine # [ 89.992069] dhcpcd[6337]: veth931cfdd2: no IPv6 Routers available machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 90.448573] k3s[873]: I0919 21:21:56.538265 873 scope.go:122] "RemoveContainer" containerID="acbe59e06b50b8d33864d22fabd682df10b29b6e5b3869561ce0d0ecfeb8876f" machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 99.794320] k3s[873]: I0919 21:22:05.883752 873 scope.go:122] "RemoveContainer" containerID="20dcd11150271900e2cc11cc4135789ea0c630872f741af5896478df3fdc5b6d" machine # [ 99.880798] systemd[1]: Started libcontainer container 7f3a53e9a3bfa29293dd2d147c25a7e4ddd91eb40fbf2ad330e19127df7876e5. machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 100.976585] systemd[1]: cri-containerd-e401edb9f53e2cd04d0f3d3fedfd1000c5ef5785f2541cc6cfdc683452fab367.scope: Deactivated successfully. machine # [ 100.980361] systemd[1]: cri-containerd-e401edb9f53e2cd04d0f3d3fedfd1000c5ef5785f2541cc6cfdc683452fab367.scope: Consumed 259ms CPU time over 25.057s wall clock time, 15.9M memory peak, 32K incoming IP traffic, 27.5K outgoing IP traffic. machine # [ 101.003093] systemd[1]: cri-containerd-6b78b1e411112a33a981d74b7f493b1fe70f517bab7639aaad29f194bb734cda.scope: Deactivated successfully. machine # [ 101.005728] systemd[1]: cri-containerd-f814918b1fe064957a4c252acea7f865a9befa76969c4de37362e980d4ac0922.scope: Deactivated successfully. machine # [ 101.006393] systemd[1]: cri-containerd-f814918b1fe064957a4c252acea7f865a9befa76969c4de37362e980d4ac0922.scope: Consumed 33ms CPU time over 22.759s wall clock time, 3.1M memory peak, 1.1M read from disk. machine # [ 101.039778] systemd[1]: cri-containerd-9b3a3fbed84ac80bd2aa2cfb50436443a107baa984bd335887fa0cc46391b566.scope: Deactivated successfully. machine # [ 101.045935] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-e401edb9f53e2cd04d0f3d3fedfd1000c5ef5785f2541cc6cfdc683452fab367-rootfs.mount: Deactivated successfully. machine # [ 101.052586] systemd[1]: cri-containerd-be6a4add840fdb41707c5a7ca7395a86099a959898408c9924e96458d390efd3.scope: Deactivated successfully. machine # [ 101.056742] systemd[1]: cri-containerd-be6a4add840fdb41707c5a7ca7395a86099a959898408c9924e96458d390efd3.scope: Consumed 80ms CPU time over 23.153s wall clock time, 28M memory peak, 18.3M read from disk. machine # [ 101.064762] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-f814918b1fe064957a4c252acea7f865a9befa76969c4de37362e980d4ac0922-rootfs.mount: Deactivated successfully. machine # [ 101.065384] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-6b78b1e411112a33a981d74b7f493b1fe70f517bab7639aaad29f194bb734cda-rootfs.mount: Deactivated successfully. machine # [ 101.093809] systemd[1]: cri-containerd-93c05483a319f07b846d4d517b82537da07bd51a556d5e936975872570d979df.scope: Deactivated successfully. machine # [ 101.115101] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-9b3a3fbed84ac80bd2aa2cfb50436443a107baa984bd335887fa0cc46391b566-rootfs.mount: Deactivated successfully. machine # [ 101.122313] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-be6a4add840fdb41707c5a7ca7395a86099a959898408c9924e96458d390efd3-rootfs.mount: Deactivated successfully. machine # [ 101.365357] cni0: port 4(veth931cfdd2) entered disabled state machine # [ 101.218582] dhcpcd[6337]: veth931cfdd2: carrier lost machine # [ 101.369279] veth931cfdd2 (unregistering): left allmulticast mode machine # [ 101.371336] veth931cfdd2 (unregistering): left promiscuous mode machine # [ 101.372823] cni0: port 4(veth931cfdd2) entered disabled state machine # [ 101.268625] dhcpcd[6337]: veth931cfdd2: deleting address fe80::64d2:ddff:fe23:c8d4 machine # [ 101.272170] dhcpcd[6337]: veth931cfdd2: deleting route to 169.254.0.0/16 machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 101.350541] k3s[873]: I0919 21:22:07.440781 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a13ff034-93af-44be-893a-1c0700eda93f-tls-ca\" (UniqueName: \"kubernetes.io/secret/a13ff034-93af-44be-893a-1c0700eda93f-tls-ca\") pod \"a13ff034-93af-44be-893a-1c0700eda93f\" (UID: \"a13ff034-93af-44be-893a-1c0700eda93f\") " machine # [ 101.350750] k3s[873]: I0919 21:22:07.440829 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a13ff034-93af-44be-893a-1c0700eda93f-config\" (UniqueName: \"kubernetes.io/configmap/a13ff034-93af-44be-893a-1c0700eda93f-config\") pod \"a13ff034-93af-44be-893a-1c0700eda93f\" (UID: \"a13ff034-93af-44be-893a-1c0700eda93f\") " machine # [ 101.351459] k3s[873]: I0919 21:22:07.440867 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a13ff034-93af-44be-893a-1c0700eda93f-tmp\" (UniqueName: \"kubernetes.io/empty-dir/a13ff034-93af-44be-893a-1c0700eda93f-tmp\") pod \"a13ff034-93af-44be-893a-1c0700eda93f\" (UID: \"a13ff034-93af-44be-893a-1c0700eda93f\") " machine # [ 101.351962] k3s[873]: I0919 21:22:07.440920 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a13ff034-93af-44be-893a-1c0700eda93f-tls-lb\" (UniqueName: \"kubernetes.io/secret/a13ff034-93af-44be-893a-1c0700eda93f-tls-lb\") pod \"a13ff034-93af-44be-893a-1c0700eda93f\" (UID: \"a13ff034-93af-44be-893a-1c0700eda93f\") " machine # [ 101.371114] k3s[873]: I0919 21:22:07.460892 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a13ff034-93af-44be-893a-1c0700eda93f-tls-lb" pod "a13ff034-93af-44be-893a-1c0700eda93f" (UID: "a13ff034-93af-44be-893a-1c0700eda93f"). InnerVolumeSpecName "tls-lb". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 101.371273] k3s[873]: I0919 21:22:07.460970 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a13ff034-93af-44be-893a-1c0700eda93f-tmp" pod "a13ff034-93af-44be-893a-1c0700eda93f" (UID: "a13ff034-93af-44be-893a-1c0700eda93f"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 101.371733] k3s[873]: I0919 21:22:07.461193 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a13ff034-93af-44be-893a-1c0700eda93f-tls-ca" pod "a13ff034-93af-44be-893a-1c0700eda93f" (UID: "a13ff034-93af-44be-893a-1c0700eda93f"). InnerVolumeSpecName "tls-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 101.372851] k3s[873]: I0919 21:22:07.461982 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a13ff034-93af-44be-893a-1c0700eda93f-config" pod "a13ff034-93af-44be-893a-1c0700eda93f" (UID: "a13ff034-93af-44be-893a-1c0700eda93f"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" machine # [ 101.407288] dhcpcd[6337]: veth931cfdd2: removing interface machine # [ 101.451303] k3s[873]: I0919 21:22:07.541854 873 reconciler_common.go:299] "Volume detached for volume \"tls-ca\" (UniqueName: \"kubernetes.io/secret/a13ff034-93af-44be-893a-1c0700eda93f-tls-ca\") on node \"machine\" DevicePath \"\"" machine # [ 101.451537] k3s[873]: I0919 21:22:07.541889 873 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a13ff034-93af-44be-893a-1c0700eda93f-config\") on node \"machine\" DevicePath \"\"" machine # [ 101.451946] k3s[873]: I0919 21:22:07.541911 873 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a13ff034-93af-44be-893a-1c0700eda93f-tmp\") on node \"machine\" DevicePath \"\"" machine # [ 101.452704] k3s[873]: I0919 21:22:07.541926 873 reconciler_common.go:299] "Volume detached for volume \"tls-lb\" (UniqueName: \"kubernetes.io/secret/a13ff034-93af-44be-893a-1c0700eda93f-tls-lb\") on node \"machine\" DevicePath \"\"" machine # [ 101.670950] k3s[873]: E0919 21:22:07.761437 873 status.go:71] "Unhandled Error" err="apiserver received an error that is not an metav1.Status: &errors.errorString{s:\"failed to create typed patch object (farm/nix-grpc-farm-scheduler; /v1, Kind=Service): .spec.selector.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0}\"}: failed to create typed patch object (farm/nix-grpc-farm-scheduler; /v1, Kind=Service): .spec.selector.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0}" logger="UnhandledError" machine # [ 101.675482] k3s[873]: I0919 21:22:07.766016 873 alloc.go:329] "allocated clusterIPs" service="farm/nix-grpc-farm" clusterIPs={"IPv4":"10.43.73.65"} machine # [ 101.683952] k3s[873]: E0919 21:22:07.774559 873 status.go:71] "Unhandled Error" err=< machine # [ 101.684422] k3s[873]: apiserver received an error that is not an metav1.Status: &errors.errorString{s:"failed to create typed patch object (farm/nix-grpc-farm-scheduler; apps/v1, Kind=Deployment): errors:\n .spec.selector.matchLabels.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0}\n .spec.template.metadata.labels.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0}"}: failed to create typed patch object (farm/nix-grpc-farm-scheduler; apps/v1, Kind=Deployment): errors: machine # [ 101.685081] k3s[873]: .spec.selector.matchLabels.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0} machine # [ 101.685683] k3s[873]: .spec.template.metadata.labels.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0} machine # [ 101.686335] k3s[873]: > logger="UnhandledError" machine # [ 101.711816] systemd[1]: cri-containerd-7f3a53e9a3bfa29293dd2d147c25a7e4ddd91eb40fbf2ad330e19127df7876e5.scope: Deactivated successfully. machine # [ 101.712834] systemd[1]: cri-containerd-7f3a53e9a3bfa29293dd2d147c25a7e4ddd91eb40fbf2ad330e19127df7876e5.scope: Consumed 773ms CPU time over 1.830s wall clock time, 32.1M memory peak, 3.3M incoming IP traffic, 197.8K outgoing IP traffic. machine # [ 101.910874] k3s[873]: I0919 21:22:08.001384 873 scope.go:122] "RemoveContainer" containerID="20dcd11150271900e2cc11cc4135789ea0c630872f741af5896478df3fdc5b6d" machine # [ 101.915821] k3s[873]: I0919 21:22:08.006317 873 scope.go:122] "RemoveContainer" containerID="7f3a53e9a3bfa29293dd2d147c25a7e4ddd91eb40fbf2ad330e19127df7876e5" machine # [ 101.921247] k3s[873]: E0919 21:22:08.010814 873 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"helm\" with CrashLoopBackOff: \"back-off 40s restarting failed container=helm pod=helm-install-nix-grpc-farm-zcg5w_kube-system(6d037147-1d29-4e02-b2ab-2f08bc2a576c)\"" pod="kube-system/helm-install-nix-grpc-farm-zcg5w" podUID="6d037147-1d29-4e02-b2ab-2f08bc2a576c" machine # [ 101.973215] k3s[873]: I0919 21:22:08.063794 873 scope.go:122] "RemoveContainer" containerID="e401edb9f53e2cd04d0f3d3fedfd1000c5ef5785f2541cc6cfdc683452fab367" machine # [ 101.976109] systemd[1]: Created slice libcontainer container kubepods-besteffort-podeb4a3576_135a_4de5_8386_b0da1ef01e63.slice. machine # [ 101.984424] systemd[1]: Created slice libcontainer container kubepods-besteffort-poda1838066_8e10_430d_8c6f_82315f17e552.slice. machine # [ 101.991431] systemd[1]: Removed slice libcontainer container kubepods-besteffort-poda13ff034_93af_44be_893a_1c0700eda93f.slice. machine # [ 101.992118] systemd[1]: kubepods-besteffort-poda13ff034_93af_44be_893a_1c0700eda93f.slice: Consumed 325ms CPU time over 26.895s wall clock time, 16.2M memory peak, 32K incoming IP traffic, 27.5K outgoing IP traffic. machine # [ 101.994993] k3s[873]: I0919 21:22:08.085393 873 scope.go:122] "RemoveContainer" containerID="e401edb9f53e2cd04d0f3d3fedfd1000c5ef5785f2541cc6cfdc683452fab367" machine # [ 101.995818] k3s[873]: E0919 21:22:08.086382 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"e401edb9f53e2cd04d0f3d3fedfd1000c5ef5785f2541cc6cfdc683452fab367\": not found" containerID="e401edb9f53e2cd04d0f3d3fedfd1000c5ef5785f2541cc6cfdc683452fab367" machine # [ 101.996512] k3s[873]: I0919 21:22:08.086414 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"e401edb9f53e2cd04d0f3d3fedfd1000c5ef5785f2541cc6cfdc683452fab367"} err="failed to get container status \"e401edb9f53e2cd04d0f3d3fedfd1000c5ef5785f2541cc6cfdc683452fab367\": rpc error: code = NotFound desc = an error occurred when try to find container \"e401edb9f53e2cd04d0f3d3fedfd1000c5ef5785f2541cc6cfdc683452fab367\": not found" machine # [ 101.999822] systemd[1]: Created slice libcontainer container kubepods-besteffort-pod658947de_798a_4627_a442_01fff1c59e29.slice. machine # [ 102.045866] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-7f3a53e9a3bfa29293dd2d147c25a7e4ddd91eb40fbf2ad330e19127df7876e5-rootfs.mount: Deactivated successfully. machine # [ 102.046130] systemd[1]: var-lib-kubelet-pods-a13ff034\x2d93af\x2d44be\x2d893a\x2d1c0700eda93f-volume\x2dsubpaths-config-envoy-0.mount: Deactivated successfully. machine # [ 102.046814] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-93c05483a319f07b846d4d517b82537da07bd51a556d5e936975872570d979df-rootfs.mount: Deactivated successfully. machine # [ 102.047491] systemd[1]: run-k3s-containerd-io.containerd.grpc.v1.cri-sandboxes-93c05483a319f07b846d4d517b82537da07bd51a556d5e936975872570d979df-shm.mount: Deactivated successfully. machine # [ 102.047981] systemd[1]: run-netns-cni\x2d47adf5bc\x2d1ae4\x2dc8cd\x2d78dc\x2d6c88273c697f.mount: Deactivated successfully. machine # [ 102.049017] systemd[1]: var-lib-kubelet-pods-a13ff034\x2d93af\x2d44be\x2d893a\x2d1c0700eda93f-volumes-kubernetes.io\x7esecret-tls\x2dlb.mount: Deactivated successfully. machine # [ 102.049809] systemd[1]: var-lib-kubelet-pods-a13ff034\x2d93af\x2d44be\x2d893a\x2d1c0700eda93f-volumes-kubernetes.io\x7eempty\x2ddir-tmp.mount: Deactivated successfully. machine # [ 102.050943] systemd[1]: var-lib-kubelet-pods-a13ff034\x2d93af\x2d44be\x2d893a\x2d1c0700eda93f-volumes-kubernetes.io\x7esecret-tls\x2dca.mount: Deactivated successfully. machine # [ 102.056406] k3s[873]: I0919 21:22:08.146964 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"build\" (UniqueName: \"kubernetes.io/empty-dir/eb4a3576-135a-4de5-8386-b0da1ef01e63-build\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-sp9dx\" (UID: \"eb4a3576-135a-4de5-8386-b0da1ef01e63\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-sp9dx" machine # [ 102.056628] k3s[873]: I0919 21:22:08.147003 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"build\" (UniqueName: \"kubernetes.io/empty-dir/658947de-798a-4627-a442-01fff1c59e29-build\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-bvc9w\" (UID: \"658947de-798a-4627-a442-01fff1c59e29\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-bvc9w" machine # [ 102.057223] k3s[873]: I0919 21:22:08.147054 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"niks3-token\" (UniqueName: \"kubernetes.io/projected/658947de-798a-4627-a442-01fff1c59e29-niks3-token\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-bvc9w\" (UID: \"658947de-798a-4627-a442-01fff1c59e29\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-bvc9w" machine # [ 102.057849] k3s[873]: I0919 21:22:08.147121 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-lb\" (UniqueName: \"kubernetes.io/secret/a1838066-8e10-430d-8c6f-82315f17e552-tls-lb\") pod \"nix-grpc-farm-lb-688d8d547b-9mcjk\" (UID: \"a1838066-8e10-430d-8c6f-82315f17e552\") " pod="farm/nix-grpc-farm-lb-688d8d547b-9mcjk" machine # [ 102.058806] k3s[873]: I0919 21:22:08.147188 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls\" (UniqueName: \"kubernetes.io/secret/eb4a3576-135a-4de5-8386-b0da1ef01e63-tls\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-sp9dx\" (UID: \"eb4a3576-135a-4de5-8386-b0da1ef01e63\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-sp9dx" machine # [ 102.059911] k3s[873]: I0919 21:22:08.147326 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls\" (UniqueName: \"kubernetes.io/secret/658947de-798a-4627-a442-01fff1c59e29-tls\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-bvc9w\" (UID: \"658947de-798a-4627-a442-01fff1c59e29\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-bvc9w" machine # [ 102.060875] k3s[873]: I0919 21:22:08.147361 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ms2mt\" (UniqueName: \"kubernetes.io/projected/658947de-798a-4627-a442-01fff1c59e29-kube-api-access-ms2mt\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-bvc9w\" (UID: \"658947de-798a-4627-a442-01fff1c59e29\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-bvc9w" machine # [ 102.061759] k3s[873]: I0919 21:22:08.147413 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a1838066-8e10-430d-8c6f-82315f17e552-config\") pod \"nix-grpc-farm-lb-688d8d547b-9mcjk\" (UID: \"a1838066-8e10-430d-8c6f-82315f17e552\") " pod="farm/nix-grpc-farm-lb-688d8d547b-9mcjk" machine # [ 102.062737] k3s[873]: I0919 21:22:08.147457 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc\" (UniqueName: \"kubernetes.io/secret/eb4a3576-135a-4de5-8386-b0da1ef01e63-oidc\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-sp9dx\" (UID: \"eb4a3576-135a-4de5-8386-b0da1ef01e63\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-sp9dx" machine # [ 102.063776] k3s[873]: I0919 21:22:08.147513 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-ca\" (UniqueName: \"kubernetes.io/secret/a1838066-8e10-430d-8c6f-82315f17e552-tls-ca\") pod \"nix-grpc-farm-lb-688d8d547b-9mcjk\" (UID: \"a1838066-8e10-430d-8c6f-82315f17e552\") " pod="farm/nix-grpc-farm-lb-688d8d547b-9mcjk" machine # [ 102.066395] k3s[873]: I0919 21:22:08.147540 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nix\" (UniqueName: \"kubernetes.io/empty-dir/eb4a3576-135a-4de5-8386-b0da1ef01e63-nix\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-sp9dx\" (UID: \"eb4a3576-135a-4de5-8386-b0da1ef01e63\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-sp9dx" machine # [ 102.072888] k3s[873]: I0919 21:22:08.147569 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nixconf\" (UniqueName: \"kubernetes.io/configmap/eb4a3576-135a-4de5-8386-b0da1ef01e63-nixconf\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-sp9dx\" (UID: \"eb4a3576-135a-4de5-8386-b0da1ef01e63\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-sp9dx" machine # [ 102.130549] k3s[873]: I0919 21:22:08.147592 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nix\" (UniqueName: \"kubernetes.io/empty-dir/658947de-798a-4627-a442-01fff1c59e29-nix\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-bvc9w\" (UID: \"658947de-798a-4627-a442-01fff1c59e29\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-bvc9w" machine # [ 102.131641] k3s[873]: I0919 21:22:08.147631 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nixconf\" (UniqueName: \"kubernetes.io/configmap/658947de-798a-4627-a442-01fff1c59e29-nixconf\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-bvc9w\" (UID: \"658947de-798a-4627-a442-01fff1c59e29\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-bvc9w" machine # [ 102.132618] k3s[873]: I0919 21:22:08.147702 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/658947de-798a-4627-a442-01fff1c59e29-ca\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-bvc9w\" (UID: \"658947de-798a-4627-a442-01fff1c59e29\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-bvc9w" machine # [ 102.133619] k3s[873]: I0919 21:22:08.147752 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc\" (UniqueName: \"kubernetes.io/secret/658947de-798a-4627-a442-01fff1c59e29-oidc\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-bvc9w\" (UID: \"658947de-798a-4627-a442-01fff1c59e29\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-bvc9w" machine # [ 102.134662] k3s[873]: I0919 21:22:08.147809 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a1838066-8e10-430d-8c6f-82315f17e552-tmp\") pod \"nix-grpc-farm-lb-688d8d547b-9mcjk\" (UID: \"a1838066-8e10-430d-8c6f-82315f17e552\") " pod="farm/nix-grpc-farm-lb-688d8d547b-9mcjk" machine # [ 102.135676] k3s[873]: I0919 21:22:08.147978 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/eb4a3576-135a-4de5-8386-b0da1ef01e63-ca\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-sp9dx\" (UID: \"eb4a3576-135a-4de5-8386-b0da1ef01e63\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-sp9dx" machine # [ 102.136713] k3s[873]: I0919 21:22:08.148046 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"niks3-token\" (UniqueName: \"kubernetes.io/projected/eb4a3576-135a-4de5-8386-b0da1ef01e63-niks3-token\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-sp9dx\" (UID: \"eb4a3576-135a-4de5-8386-b0da1ef01e63\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-sp9dx" machine # [ 102.137720] k3s[873]: I0919 21:22:08.148076 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsl7n\" (UniqueName: \"kubernetes.io/projected/eb4a3576-135a-4de5-8386-b0da1ef01e63-kube-api-access-bsl7n\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-sp9dx\" (UID: \"eb4a3576-135a-4de5-8386-b0da1ef01e63\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-sp9dx" machine # [ 102.455497] cni0: port 3(veth9158045b) entered blocking state machine # [ 102.458050] cni0: port 3(veth9158045b) entered disabled state machine # [ 102.460049] veth9158045b: entered allmulticast mode machine # [ 102.461913] veth9158045b: entered promiscuous mode machine # [ 102.317366] (udev-worker)[8881]: Network interface NamePolicy= disabled on kernel command line. machine # [ 102.480071] cni0: port 4(veth81c92383) entered blocking state machine # [ 102.482394] cni0: port 4(veth81c92383) entered disabled state machine # [ 102.484726] veth81c92383: entered allmulticast mode machine # [ 102.486603] veth81c92383: entered promiscuous mode machine # [ 102.341682] (udev-worker)[8882]: Network interface NamePolicy= disabled on kernel command line.[ 102.491388] cni0: port 3(veth9158045b) entered blocking state machine # [ 102.492960] cni0: port 3(veth9158045b) entered forwarding state machine # machine # [ 102.357467] (udev-worker)[9067]: Network interface NamePolicy= disabled on ke[ 102.505678] cni0: port 5(vethfb2eae5e) entered blocking state machine # rnel command line. machine # [ 102.509359] cni0: port 5(vethfb2eae5e) entered disabled state machine # [ 102.511131] vethfb2eae5e: entered allmulticast mode machine # [ 102.513452] vethfb2eae5e: entered promiscuous mode machine # [ 102.519416] cni0: port 4(veth81c92383) entered blocking state machine # [ 102.520857] cni0: port 4(veth81c92383) entered forwarding state machine # [ 102.378043] dhcpcd[6337]: veth9158045b: IAID 63:72:c5:02 machine # [ 102.380437] dhcpcd[6337]: veth9158045b: adding address fe80::84f2:63ff:fe72:c502 machine # [ 102.538551] cni0: port 5(vethfb2eae5e) entered blocking state machine # [ 102.539592] cni0: port 5(vethfb2eae5e) entered forwarding state machine # [ 102.402155] dhcpcd[6337]: veth81c92383: IAID 68:a1:20:25 machine # [ 102.403852] dhcpcd[6337]: veth81c92383: adding address fe80::181b:68ff:fea1:2025 machine # [ 102.404994] dhcpcd[6337]: veth81c92383: carrier lost machine # [ 102.446554] dhcpcd[6337]: veth81c92383: deleting address fe80::181b:68ff:fea1:2025 machine # [ 102.448549] dhcpcd[6337]: veth9158045b: soliciting a DHCP lease machine # [ 102.474346] dhcpcd[6337]: vethfb2eae5e: IAID bc:2d:51:be machine # [ 102.474621] dhcpcd[6337]: vethfb2eae5e: adding address fe80::3cff:bcff:fe2d:51be machine # [ 102.475374] dhcpcd[6337]: vethfb2eae5e: old hardware address: 3e:ff:bc:2d:51:be machine # [ 102.475873] dhcpcd[6337]: vethfb2eae5e: new hardware address: 3a:04:b2:1b:d8:a6 machine # [ 102.476059] dhcpcd[6337]: vethfb2eae5e: carrier lost machine # [ 102.494373] systemd[1]: Started libcontainer container 0aef947c62b0e194d5b5aeca7f1c43e38a8382b251b9de9908fbf749baa252d1. machine # [ 102.522835] dhcpcd[6337]: vethfb2eae5e: deleting address fe80::3cff:bcff:fe2d:51be machine # [ 102.523700] dhcpcd[6337]: veth81c92383: carrier acquired machine # [ 102.535449] dhcpcd[6337]: veth81c92383: IAID 68:a1:20:25 machine # [ 102.535761] dhcpcd[6337]: veth81c92383: adding address fe80::181b:68ff:fea1:2025 machine # [ 102.536259] dhcpcd[6337]: vethfb2eae5e: carrier acquired machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 102.548075] dhcpcd[6337]: vethfb2eae5e: IAID b2:1b:d8:a6 machine # [ 102.548286] dhcpcd[6337]: vethfb2eae5e: adding address fe80::3804:b2ff:fe1b:d8a6 machine # [ 102.555896] systemd[1]: Started libcontainer container ff9e08cbda1b9fcf9a6162c76ad2a4340948727430d7755398e7ebafa5c70e88. machine # [ 102.559496] systemd[1]: Started libcontainer container 1af82520a49c872b1e17ab513962ee1ff69c760b94c6325f2ead93641f3c76d0. machine # [ 102.635474] systemd[1]: Started libcontainer container 0d22b669f678b552053934b9cc1112b738f2f2b89fa8f973edfefbc8b523d0d6. machine # [ 102.670318] systemd[1]: Started libcontainer container eeef823f106441635eb627f73d876e38f208e6590417f75ccea1b70d1ca1ee88. machine # [ 102.713772] systemd[1]: Started libcontainer container f613bf199062a0f34bfc0e67f722da77c1b52b3e4192cf8539ac4fa2d32e679d. machine # [ 102.851342] k3s[873]: I0919 21:22:08.941516 873 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a13ff034-93af-44be-893a-1c0700eda93f" path="/var/lib/kubelet/pods/a13ff034-93af-44be-893a-1c0700eda93f/volumes" machine # [ 102.937830] dhcpcd[6337]: vethfb2eae5e: soliciting a DHCP lease machine # [ 103.006111] k3s[873]: I0919 21:22:09.091652 873 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="farm/nix-grpc-farm-lb-688d8d547b-9mcjk" podStartSLOduration=1.091627137 podStartE2EDuration="1.091627137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 21:22:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 21:22:09.08791633 +0000 UTC m=+84.671761067" watchObservedRunningTime="2026-09-19 21:22:09.091627137 +0000 UTC m=+84.675471595" machine # [ 103.256072] systemd[1]: cri-containerd-0d22b669f678b552053934b9cc1112b738f2f2b89fa8f973edfefbc8b523d0d6.scope: Deactivated successfully. machine # [ 103.256888] systemd[1]: cri-containerd-0d22b669f678b552053934b9cc1112b738f2f2b89fa8f973edfefbc8b523d0d6.scope: Consumed 503ms CPU time over 620ms wall clock time, 304.2M memory peak, 372K read from disk, 83M written to disk. machine # [ 103.271929] systemd[1]: cri-containerd-eeef823f106441635eb627f73d876e38f208e6590417f75ccea1b70d1ca1ee88.scope: Deactivated successfully. machine # [ 103.272353] systemd[1]: cri-containerd-eeef823f106441635eb627f73d876e38f208e6590417f75ccea1b70d1ca1ee88.scope: Consumed 504ms CPU time over 600ms wall clock time, 305.9M memory peak, 644K read from disk, 227M written to disk. machine # [ 103.337501] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-0d22b669f678b552053934b9cc1112b738f2f2b89fa8f973edfefbc8b523d0d6-rootfs.mount: Deactivated successfully. machine # [ 103.341284] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-eeef823f106441635eb627f73d876e38f208e6590417f75ccea1b70d1ca1ee88-rootfs.mount: Deactivated successfully. machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 103.918642] dhcpcd[6337]: veth9158045b: soliciting an IPv6 router machine # [ 104.072807] systemd[1]: var-lib-rancher-k3s-agent-containerd-tmpmounts-containerd\x2dmount1296773715.mount: Deactivated successfully. machine # [ 104.081063] systemd[1]: var-lib-rancher-k3s-agent-containerd-tmpmounts-containerd\x2dmount652346800.mount: Deactivated successfully. machine # [ 104.120468] systemd[1]: Started libcontainer container 8f31687a726dad45f215df2346ae43e303eeea80cd0e24cd5de7a3e9fb18a546. machine # [ 104.124738] systemd[1]: Started libcontainer container e10566537825b7410ec27bb4bd88db4a17a0668bb97a743b7b58db32bb6c40eb. machine # [ 104.273417] systemd[1]: Started libcontainer container 3ade944e74e46ec0b6d412ac35a2b03882f265c4a4d4afb3ba7c4cfd012e3ef8. machine # [ 104.275036] systemd[1]: Started libcontainer container efbcb2c7f00e526fc1d543c20dde4263a13402244f747249c01a23ef4e44841c. machine # [ 104.352583] dhcpcd[6337]: veth81c92383: soliciting a DHCP lease machine # [ 104.437365] systemd[1]: Started libcontainer container f80462ee20289be5052b6f1e51ba74c2111059fb08622ec1edc82423f743fe4b. machine # [ 104.452409] systemd[1]: Started libcontainer container 4f3aca8d093f0ffecf31405aca382497c938a05fd3f43628a45b46e0a3b7ef88. machine # [ 104.882527] dhcpcd[6337]: vethfb2eae5e: soliciting an IPv6 router machine # [ 104.959407] dhcpcd[6337]: veth81c92383: soliciting an IPv6 router machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 105.037494] k3s[873]: I0919 21:22:11.127646 873 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-bvc9w" podStartSLOduration=2.561774879 podStartE2EDuration="3.127614392s" totalImagesPullingTime="0s" totalInitContainerRuntime="565.839513ms" isStatefulPod=false podCreationTimestamp="2026-09-19 21:22:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 21:22:11.126755065 +0000 UTC m=+86.710600082" watchObservedRunningTime="2026-09-19 21:22:11.127614392 +0000 UTC m=+86.711461923" machine # [ 105.064293] k3s[873]: I0919 21:22:11.154844 873 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-sp9dx" podStartSLOduration=2.562241978 podStartE2EDuration="3.15482651s" totalImagesPullingTime="0s" totalInitContainerRuntime="592.584532ms" isStatefulPod=false podCreationTimestamp="2026-09-19 21:22:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 21:22:11.154788795 +0000 UTC m=+86.738636606" watchObservedRunningTime="2026-09-19 21:22:11.15482651 +0000 UTC m=+86.738670968" machine # [ 106.232868] systemd[1]: cri-containerd-ea4f4a1497f3701457603cc65c862dade042a76ec1d091744b857918b0391260.scope: Deactivated successfully. machine # [ 106.234772] systemd[1]: cri-containerd-ea4f4a1497f3701457603cc65c862dade042a76ec1d091744b857918b0391260.scope: Consumed 213ms CPU time over 28.163s wall clock time, 19.8M memory peak, 11.8M read from disk, 15.7K incoming IP traffic, 15K outgoing IP traffic. machine # [ 106.251133] systemd[1]: cri-containerd-f38efc5b0fa33e210961b12d0c47bef60d77aa6dd54bc82927a194d818f2ec13.scope: Deactivated successfully. machine # [ 106.252122] systemd[1]: cri-containerd-f38efc5b0fa33e210961b12d0c47bef60d77aa6dd54bc82927a194d818f2ec13.scope: Consumed 213ms CPU time over 28.179s wall clock time, 35.3M memory peak, 27.7M read from disk, 15.4K incoming IP traffic, 15.5K outgoing IP traffic. machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 106.302223] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-ea4f4a1497f3701457603cc65c862dade042a76ec1d091744b857918b0391260-rootfs.mount: Deactivated successfully. machine # [ 106.311642] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-f38efc5b0fa33e210961b12d0c47bef60d77aa6dd54bc82927a194d818f2ec13-rootfs.mount: Deactivated successfully. machine # [ 106.334756] systemd[1]: cri-containerd-0886d9c6e21f1943ae6b21c18cd0c8be5c704363b59a6a6afe6ec88fcf36e21e.scope: Deactivated successfully. machine # [ 106.338319] systemd[1]: cri-containerd-842ab80e30eb70b0c51302731a2bea11fe7fcc0f6abda588848c1c69a0591870.scope: Deactivated successfully. machine # [ 106.376870] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-0886d9c6e21f1943ae6b21c18cd0c8be5c704363b59a6a6afe6ec88fcf36e21e-rootfs.mount: Deactivated successfully. machine # [ 106.377205] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-842ab80e30eb70b0c51302731a2bea11fe7fcc0f6abda588848c1c69a0591870-rootfs.mount: Deactivated successfully. machine # [ 106.414581] systemd[1]: run-k3s-containerd-io.containerd.grpc.v1.cri-sandboxes-842ab80e30eb70b0c51302731a2bea11fe7fcc0f6abda588848c1c69a0591870-shm.mount: Deactivated successfully. machine # [ 106.415153] systemd[1]: run-k3s-containerd-io.containerd.grpc.v1.cri-sandboxes-0886d9c6e21f1943ae6b21c18cd0c8be5c704363b59a6a6afe6ec88fcf36e21e-shm.mount: Deactivated successfully. machine # [ 106.617027] cni0: port 6(veth3bd36ce0) entered disabled state machine # [ 106.471753] dhcpcd[6337]: veth3bd36ce0: carri[ 106.619333] veth3bd36ce0 (unregistering): left allmulticast mode machine # er lost machine # [ 106.621026] veth3bd36ce0 (unregistering): left promiscuous mode machine # [ 106.622573] cni0: port 6(veth3bd36ce0) entered disabled state machine # [ 106.625521] cni0: port 7(vethbd58bc26) entered disabled state machine # [ 106.627547] vethbd58bc26 (unregistering): left allmulticast mode machine # [ 106.630025] vethbd58bc26 (unregistering): left promiscuous mode machine # [ 106.631995] cni0: port 7(vethbd58bc26) entered disabled state machine # [ 106.499415] systemd[1]: run-netns-cni\x2db1fcacdb\x2d0f14\x2dc47a\x2d5c0c\x2d171026de1a4f.mount: Deactivated successfully. machine # [ 106.530302] dhcpcd[6337]: veth3bd36ce0: deleting address fe80::60f6:8aff:feff:c8fc machine # [ 106.536338] dhcpcd[6337]: veth3bd36ce0: deleting route to 169.254.0.0/16 machine # [ 106.599736] k3s[873]: I0919 21:22:12.689889 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-nixconf\" (UniqueName: \"kubernetes.io/configmap/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-nixconf\") pod \"14aec9b1-6bcd-41a9-9fbe-48013fa6a384\" (UID: \"14aec9b1-6bcd-41a9-9fbe-48013fa6a384\") " machine # [ 106.600089] k3s[873]: I0919 21:22:12.689943 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-build\" (UniqueName: \"kubernetes.io/empty-dir/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-build\") pod \"14aec9b1-6bcd-41a9-9fbe-48013fa6a384\" (UID: \"14aec9b1-6bcd-41a9-9fbe-48013fa6a384\") " machine # [ 106.601379] k3s[873]: I0919 21:22:12.689984 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-ca\" (UniqueName: \"kubernetes.io/secret/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-ca\") pod \"ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7\" (UID: \"ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7\") " machine # [ 106.602978] k3s[873]: I0919 21:22:12.690013 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-oidc\" (UniqueName: \"kubernetes.io/secret/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-oidc\") pod \"ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7\" (UID: \"ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7\") " machine # [ 106.604455] k3s[873]: I0919 21:22:12.690045 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-ca\" (UniqueName: \"kubernetes.io/secret/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-ca\") pod \"14aec9b1-6bcd-41a9-9fbe-48013fa6a384\" (UID: \"14aec9b1-6bcd-41a9-9fbe-48013fa6a384\") " machine # [ 106.606366] k3s[873]: I0919 21:22:12.690077 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-nixconf\" (UniqueName: \"kubernetes.io/configmap/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-nixconf\") pod \"ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7\" (UID: \"ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7\") " machine # [ 106.607910] k3s[873]: I0919 21:22:12.690109 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-oidc\" (UniqueName: \"kubernetes.io/secret/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-oidc\") pod \"14aec9b1-6bcd-41a9-9fbe-48013fa6a384\" (UID: \"14aec9b1-6bcd-41a9-9fbe-48013fa6a384\") " machine # [ 106.608517] k3s[873]: I0919 21:22:12.690135 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-nix\" (UniqueName: \"kubernetes.io/empty-dir/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-nix\") pod \"14aec9b1-6bcd-41a9-9fbe-48013fa6a384\" (UID: \"14aec9b1-6bcd-41a9-9fbe-48013fa6a384\") " machine # [ 106.609200] k3s[873]: I0919 21:22:12.690165 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-nix\" (UniqueName: \"kubernetes.io/empty-dir/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-nix\") pod \"ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7\" (UID: \"ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7\") " machine # [ 106.610261] k3s[873]: I0919 21:22:12.690201 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-kube-api-access-qrngq\" (UniqueName: \"kubernetes.io/projected/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-kube-api-access-qrngq\") pod \"14aec9b1-6bcd-41a9-9fbe-48013fa6a384\" (UID: \"14aec9b1-6bcd-41a9-9fbe-48013fa6a384\") " machine # [ 106.611100] k3s[873]: I0919 21:22:12.690237 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-niks3-token\" (UniqueName: \"kubernetes.io/projected/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-niks3-token\") pod \"ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7\" (UID: \"ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7\") " machine # [ 106.612142] k3s[873]: I0919 21:22:12.690268 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-niks3-token\" (UniqueName: \"kubernetes.io/projected/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-niks3-token\") pod \"14aec9b1-6bcd-41a9-9fbe-48013fa6a384\" (UID: \"14aec9b1-6bcd-41a9-9fbe-48013fa6a384\") " machine # [ 106.612987] k3s[873]: I0919 21:22:12.690293 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-tls\" (UniqueName: \"kubernetes.io/secret/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-tls\") pod \"14aec9b1-6bcd-41a9-9fbe-48013fa6a384\" (UID: \"14aec9b1-6bcd-41a9-9fbe-48013fa6a384\") " machine # [ 106.614132] k3s[873]: I0919 21:22:12.690323 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-build\" (UniqueName: \"kubernetes.io/empty-dir/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-build\") pod \"ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7\" (UID: \"ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7\") " machine # [ 106.618906] k3s[873]: I0919 21:22:12.690350 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-tls\" (UniqueName: \"kubernetes.io/secret/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-tls\") pod \"ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7\" (UID: \"ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7\") " machine # [ 106.622945] k3s[873]: I0919 21:22:12.690383 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-kube-api-access-xgwkg\" (UniqueName: \"kubernetes.io/projected/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-kube-api-access-xgwkg\") pod \"ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7\" (UID: \"ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7\") " machine # [ 106.682196] k3s[873]: I0919 21:22:12.772730 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-nixconf" pod "14aec9b1-6bcd-41a9-9fbe-48013fa6a384" (UID: "14aec9b1-6bcd-41a9-9fbe-48013fa6a384"). InnerVolumeSpecName "nixconf". PluginName "kubernetes.io/configmap", VolumeGIDValue "" machine # [ 106.682662] k3s[873]: I0919 21:22:12.772938 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-build" pod "14aec9b1-6bcd-41a9-9fbe-48013fa6a384" (UID: "14aec9b1-6bcd-41a9-9fbe-48013fa6a384"). InnerVolumeSpecName "build". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 106.691933] k3s[873]: I0919 21:22:12.782247 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-build" pod "ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7" (UID: "ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7"). InnerVolumeSpecName "build". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 106.692771] k3s[873]: I0919 21:22:12.782274 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-nixconf" pod "ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7" (UID: "ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7"). InnerVolumeSpecName "nixconf". PluginName "kubernetes.io/configmap", VolumeGIDValue "" machine # [ 106.693944] k3s[873]: I0919 21:22:12.782611 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-niks3-token" pod "14aec9b1-6bcd-41a9-9fbe-48013fa6a384" (UID: "14aec9b1-6bcd-41a9-9fbe-48013fa6a384"). InnerVolumeSpecName "niks3-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" machine # [ 106.700202] k3s[873]: I0919 21:22:12.790803 873 reconciler_common.go:299] "Volume detached for volume \"nixconf\" (UniqueName: \"kubernetes.io/configmap/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-nixconf\") on node \"machine\" DevicePath \"\"" machine # [ 106.701839] k3s[873]: I0919 21:22:12.790835 873 reconciler_common.go:299] "Volume detached for volume \"niks3-token\" (UniqueName: \"kubernetes.io/projected/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-niks3-token\") on node \"machine\" DevicePath \"\"" machine # [ 106.702737] k3s[873]: I0919 21:22:12.790851 873 reconciler_common.go:299] "Volume detached for volume \"build\" (UniqueName: \"kubernetes.io/empty-dir/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-build\") on node \"machine\" DevicePath \"\"" machine # [ 106.704215] k3s[873]: I0919 21:22:12.790904 873 reconciler_common.go:299] "Volume detached for volume \"nixconf\" (UniqueName: \"kubernetes.io/configmap/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-nixconf\") on node \"machine\" DevicePath \"\"" machine # [ 106.705785] k3s[873]: I0919 21:22:12.790918 873 reconciler_common.go:299] "Volume detached for volume \"build\" (UniqueName: \"kubernetes.io/empty-dir/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-build\") on node \"machine\" DevicePath \"\"" machine # [ 106.726620] k3s[873]: I0919 21:22:12.817183 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-niks3-token" pod "ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7" (UID: "ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7"). InnerVolumeSpecName "niks3-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" machine # [ 106.727391] k3s[873]: I0919 21:22:12.817191 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-ca" pod "ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7" (UID: "ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7"). InnerVolumeSpecName "ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 106.728104] k3s[873]: I0919 21:22:12.817231 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-tls" pod "14aec9b1-6bcd-41a9-9fbe-48013fa6a384" (UID: "14aec9b1-6bcd-41a9-9fbe-48013fa6a384"). InnerVolumeSpecName "tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 106.728942] k3s[873]: I0919 21:22:12.817399 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-kube-api-access-xgwkg" pod "ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7" (UID: "ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7"). InnerVolumeSpecName "kube-api-access-xgwkg". PluginName "kubernetes.io/projected", VolumeGIDValue "" machine # [ 106.729823] k3s[873]: I0919 21:22:12.818051 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-oidc" pod "ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7" (UID: "ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7"). InnerVolumeSpecName "oidc". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 106.737489] k3s[873]: I0919 21:22:12.828052 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-oidc" pod "14aec9b1-6bcd-41a9-9fbe-48013fa6a384" (UID: "14aec9b1-6bcd-41a9-9fbe-48013fa6a384"). InnerVolumeSpecName "oidc". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 106.738608] k3s[873]: I0919 21:22:12.828140 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-tls" pod "ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7" (UID: "ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7"). InnerVolumeSpecName "tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 106.760290] k3s[873]: I0919 21:22:12.850864 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-kube-api-access-qrngq" pod "14aec9b1-6bcd-41a9-9fbe-48013fa6a384" (UID: "14aec9b1-6bcd-41a9-9fbe-48013fa6a384"). InnerVolumeSpecName "kube-api-access-qrngq". PluginName "kubernetes.io/projected", VolumeGIDValue "" machine # [ 106.766923] k3s[873]: I0919 21:22:12.857513 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-ca" pod "14aec9b1-6bcd-41a9-9fbe-48013fa6a384" (UID: "14aec9b1-6bcd-41a9-9fbe-48013fa6a384"). InnerVolumeSpecName "ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 106.800951] k3s[873]: I0919 21:22:12.891507 873 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qrngq\" (UniqueName: \"kubernetes.io/projected/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-kube-api-access-qrngq\") on node \"machine\" DevicePath \"\"" machine # [ 106.801757] k3s[873]: I0919 21:22:12.891545 873 reconciler_common.go:299] "Volume detached for volume \"niks3-token\" (UniqueName: \"kubernetes.io/projected/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-niks3-token\") on node \"machine\" DevicePath \"\"" machine # [ 106.802854] k3s[873]: I0919 21:22:12.891563 873 reconciler_common.go:299] "Volume detached for volume \"tls\" (UniqueName: \"kubernetes.io/secret/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-tls\") on node \"machine\" DevicePath \"\"" machine # [ 106.803605] k3s[873]: I0919 21:22:12.891576 873 reconciler_common.go:299] "Volume detached for volume \"tls\" (UniqueName: \"kubernetes.io/secret/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-tls\") on node \"machine\" DevicePath \"\"" machine # [ 106.804371] k3s[873]: I0919 21:22:12.891588 873 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgwkg\" (UniqueName: \"kubernetes.io/projected/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-kube-api-access-xgwkg\") on node \"machine\" DevicePath \"\"" machine # [ 106.805222] k3s[873]: I0919 21:22:12.891603 873 reconciler_common.go:299] "Volume detached for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-ca\") on node \"machine\" DevicePath \"\"" machine # [ 106.807373] k3s[873]: I0919 21:22:12.891616 873 reconciler_common.go:299] "Volume detached for volume \"oidc\" (UniqueName: \"kubernetes.io/secret/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-oidc\") on node \"machine\" DevicePath \"\"" machine # [ 106.808188] k3s[873]: I0919 21:22:12.891629 873 reconciler_common.go:299] "Volume detached for volume \"ca\" (UniqueName: \"kubernetes.io/secret/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-ca\") on node \"machine\" DevicePath \"\"" machine # [ 106.808884] k3s[873]: I0919 21:22:12.891641 873 reconciler_common.go:299] "Volume detached for volume \"oidc\" (UniqueName: \"kubernetes.io/secret/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-oidc\") on node \"machine\" DevicePath \"\"" machine # [ 106.833374] dhcpcd[6337]: dhcp_readbpf: unexpected event 0x0100 machine # [ 106.834807] dhcpcd[6337]: dhcp_readbpf: vethbd58bc26: Network is down machine # [ 106.838166] dhcpcd[6337]: arp_read: unexpected event 0x0100 machine # [ 106.838346] dhcpcd[6337]: arp_read: vethbd58bc26: Network is down machine # [ 106.844347] dhcpcd[6337]: veth3bd36ce0: removing interface machine # [ 106.871886] dhcpcd[6337]: vethbd58bc26: removing interface machine # [ 106.872586] dhcpcd[6337]: vethbd58bc26: deleting address fe80::545c:a2ff:fef2:5b34 machine # [ 106.874160] dhcpcd[6337]: vethbd58bc26: deleting route to 169.254.0.0/16 machine # [ 106.947312] k3s[873]: I0919 21:22:13.037826 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-nix" pod "14aec9b1-6bcd-41a9-9fbe-48013fa6a384" (UID: "14aec9b1-6bcd-41a9-9fbe-48013fa6a384"). InnerVolumeSpecName "nix". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 107.002290] k3s[873]: I0919 21:22:13.092807 873 reconciler_common.go:299] "Volume detached for volume \"nix\" (UniqueName: \"kubernetes.io/empty-dir/14aec9b1-6bcd-41a9-9fbe-48013fa6a384-nix\") on node \"machine\" DevicePath \"\"" machine # [ 107.043531] k3s[873]: I0919 21:22:13.133599 873 scope.go:122] "RemoveContainer" containerID="6b78b1e411112a33a981d74b7f493b1fe70f517bab7639aaad29f194bb734cda" machine # [ 107.051833] systemd[1]: Removed slice libcontainer container kubepods-besteffort-pod14aec9b1_6bcd_41a9_9fbe_48013fa6a384.slice. machine # [ 107.053761] systemd[1]: kubepods-besteffort-pod14aec9b1_6bcd_41a9_9fbe_48013fa6a384.slice: Consumed 930ms CPU time over 31.934s wall clock time, 325.7M memory peak, 11.9M read from disk, 283.8M written to disk, 15.7K incoming IP traffic, 15K outgoing IP traffic. machine # [ 107.057381] k3s[873]: I0919 21:22:13.146039 873 scope.go:122] "RemoveContainer" containerID="ea4f4a1497f3701457603cc65c862dade042a76ec1d091744b857918b0391260" machine # [ 107.072670] k3s[873]: I0919 21:22:13.163250 873 scope.go:122] "RemoveContainer" containerID="9b3a3fbed84ac80bd2aa2cfb50436443a107baa984bd335887fa0cc46391b566" machine # [ 107.076853] k3s[873]: I0919 21:22:13.167438 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-nix" pod "ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7" (UID: "ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7"). InnerVolumeSpecName "nix". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 107.082606] k3s[873]: I0919 21:22:13.173189 873 scope.go:122] "RemoveContainer" containerID="2ca001180238662d79248477aab8b8fd022667f15dcebcbb65a7cc43ea506e36" machine # [ 107.089878] k3s[873]: I0919 21:22:13.180477 873 scope.go:122] "RemoveContainer" containerID="6b78b1e411112a33a981d74b7f493b1fe70f517bab7639aaad29f194bb734cda" machine # [ 107.090802] k3s[873]: E0919 21:22:13.181404 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"6b78b1e411112a33a981d74b7f493b1fe70f517bab7639aaad29f194bb734cda\": not found" containerID="6b78b1e411112a33a981d74b7f493b1fe70f517bab7639aaad29f194bb734cda" machine # [ 107.091548] k3s[873]: I0919 21:22:13.181446 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"6b78b1e411112a33a981d74b7f493b1fe70f517bab7639aaad29f194bb734cda"} err="failed to get container status \"6b78b1e411112a33a981d74b7f493b1fe70f517bab7639aaad29f194bb734cda\": rpc error: code = NotFound desc = an error occurred when try to find container \"6b78b1e411112a33a981d74b7f493b1fe70f517bab7639aaad29f194bb734cda\": not found" machine # [ 107.092040] k3s[873]: I0919 21:22:13.181473 873 scope.go:122] "RemoveContainer" containerID="ea4f4a1497f3701457603cc65c862dade042a76ec1d091744b857918b0391260" machine # [ 107.092491] k3s[873]: E0919 21:22:13.182155 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"ea4f4a1497f3701457603cc65c862dade042a76ec1d091744b857918b0391260\": not found" containerID="ea4f4a1497f3701457603cc65c862dade042a76ec1d091744b857918b0391260" machine # [ 107.092993] k3s[873]: I0919 21:22:13.182210 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"ea4f4a1497f3701457603cc65c862dade042a76ec1d091744b857918b0391260"} err="failed to get container status \"ea4f4a1497f3701457603cc65c862dade042a76ec1d091744b857918b0391260\": rpc error: code = NotFound desc = an error occurred when try to find container \"ea4f4a1497f3701457603cc65c862dade042a76ec1d091744b857918b0391260\": not found" machine # [ 107.093627] k3s[873]: I0919 21:22:13.182254 873 scope.go:122] "RemoveContainer" containerID="9b3a3fbed84ac80bd2aa2cfb50436443a107baa984bd335887fa0cc46391b566" machine # [ 107.094164] k3s[873]: E0919 21:22:13.182849 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"9b3a3fbed84ac80bd2aa2cfb50436443a107baa984bd335887fa0cc46391b566\": not found" containerID="9b3a3fbed84ac80bd2aa2cfb50436443a107baa984bd335887fa0cc46391b566" machine # [ 107.094622] k3s[873]: I0919 21:22:13.182877 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"9b3a3fbed84ac80bd2aa2cfb50436443a107baa984bd335887fa0cc46391b566"} err="failed to get container status \"9b3a3fbed84ac80bd2aa2cfb50436443a107baa984bd335887fa0cc46391b566\": rpc error: code = NotFound desc = an error occurred when try to find container \"9b3a3fbed84ac80bd2aa2cfb50436443a107baa984bd335887fa0cc46391b566\": not found" machine # [ 107.095478] k3s[873]: I0919 21:22:13.182907 873 scope.go:122] "RemoveContainer" containerID="2ca001180238662d79248477aab8b8fd022667f15dcebcbb65a7cc43ea506e36" machine # [ 107.096057] k3s[873]: E0919 21:22:13.183534 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"2ca001180238662d79248477aab8b8fd022667f15dcebcbb65a7cc43ea506e36\": not found" containerID="2ca001180238662d79248477aab8b8fd022667f15dcebcbb65a7cc43ea506e36" machine # [ 107.096530] k3s[873]: I0919 21:22:13.183554 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"2ca001180238662d79248477aab8b8fd022667f15dcebcbb65a7cc43ea506e36"} err="failed to get container status \"2ca001180238662d79248477aab8b8fd022667f15dcebcbb65a7cc43ea506e36\": rpc error: code = NotFound desc = an error occurred when try to find container \"2ca001180238662d79248477aab8b8fd022667f15dcebcbb65a7cc43ea506e36\": not found" machine # [ 107.097156] k3s[873]: I0919 21:22:13.183570 873 scope.go:122] "RemoveContainer" containerID="f814918b1fe064957a4c252acea7f865a9befa76969c4de37362e980d4ac0922" machine # [ 107.101351] k3s[873]: I0919 21:22:13.191806 873 scope.go:122] "RemoveContainer" containerID="f38efc5b0fa33e210961b12d0c47bef60d77aa6dd54bc82927a194d818f2ec13" machine # [ 107.102551] k3s[873]: I0919 21:22:13.193057 873 reconciler_common.go:299] "Volume detached for volume \"nix\" (UniqueName: \"kubernetes.io/empty-dir/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7-nix\") on node \"machine\" DevicePath \"\"" machine # [ 107.111511] k3s[873]: I0919 21:22:13.202125 873 scope.go:122] "RemoveContainer" containerID="be6a4add840fdb41707c5a7ca7395a86099a959898408c9924e96458d390efd3" machine # [ 107.120553] k3s[873]: I0919 21:22:13.211138 873 scope.go:122] "RemoveContainer" containerID="ef71cca7acd75a946500d9b8aecc57e84a9dd3bf5f73da02d2a4f7b91eb0372a" machine # [ 107.128441] k3s[873]: I0919 21:22:13.219043 873 scope.go:122] "RemoveContainer" containerID="f814918b1fe064957a4c252acea7f865a9befa76969c4de37362e980d4ac0922" machine # [ 107.129430] k3s[873]: E0919 21:22:13.220039 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"f814918b1fe064957a4c252acea7f865a9befa76969c4de37362e980d4ac0922\": not found" containerID="f814918b1fe064957a4c252acea7f865a9befa76969c4de37362e980d4ac0922" machine # [ 107.130037] k3s[873]: I0919 21:22:13.220075 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"f814918b1fe064957a4c252acea7f865a9befa76969c4de37362e980d4ac0922"} err="failed to get container status \"f814918b1fe064957a4c252acea7f865a9befa76969c4de37362e980d4ac0922\": rpc error: code = NotFound desc = an error occurred when try to find container \"f814918b1fe064957a4c252acea7f865a9befa76969c4de37362e980d4ac0922\": not found" machine # [ 107.130523] k3s[873]: I0919 21:22:13.220099 873 scope.go:122] "RemoveContainer" containerID="f38efc5b0fa33e210961b12d0c47bef60d77aa6dd54bc82927a194d818f2ec13" machine # [ 107.130972] k3s[873]: E0919 21:22:13.220642 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"f38efc5b0fa33e210961b12d0c47bef60d77aa6dd54bc82927a194d818f2ec13\": not found" containerID="f38efc5b0fa33e210961b12d0c47bef60d77aa6dd54bc82927a194d818f2ec13" machine # [ 107.131665] k3s[873]: I0919 21:22:13.220702 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"f38efc5b0fa33e210961b12d0c47bef60d77aa6dd54bc82927a194d818f2ec13"} err="failed to get container status \"f38efc5b0fa33e210961b12d0c47bef60d77aa6dd54bc82927a194d818f2ec13\": rpc error: code = NotFound desc = an error occurred when try to find container \"f38efc5b0fa33e210961b12d0c47bef60d77aa6dd54bc82927a194d818f2ec13\": not found" machine # [ 107.132206] k3s[873]: I0919 21:22:13.220731 873 scope.go:122] "RemoveContainer" containerID="be6a4add840fdb41707c5a7ca7395a86099a959898408c9924e96458d390efd3" machine # [ 107.132668] k3s[873]: E0919 21:22:13.221245 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"be6a4add840fdb41707c5a7ca7395a86099a959898408c9924e96458d390efd3\": not found" containerID="be6a4add840fdb41707c5a7ca7395a86099a959898408c9924e96458d390efd3" machine # [ 107.133186] k3s[873]: I0919 21:22:13.221267 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"be6a4add840fdb41707c5a7ca7395a86099a959898408c9924e96458d390efd3"} err="failed to get container status \"be6a4add840fdb41707c5a7ca7395a86099a959898408c9924e96458d390efd3\": rpc error: code = NotFound desc = an error occurred when try to find container \"be6a4add840fdb41707c5a7ca7395a86099a959898408c9924e96458d390efd3\": not found" machine # [ 107.133619] k3s[873]: I0919 21:22:13.221286 873 scope.go:122] "RemoveContainer" containerID="ef71cca7acd75a946500d9b8aecc57e84a9dd3bf5f73da02d2a4f7b91eb0372a" machine # [ 107.134180] k3s[873]: E0919 21:22:13.221913 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"ef71cca7acd75a946500d9b8aecc57e84a9dd3bf5f73da02d2a4f7b91eb0372a\": not found" containerID="ef71cca7acd75a946500d9b8aecc57e84a9dd3bf5f73da02d2a4f7b91eb0372a" machine # [ 107.176186] k3s[873]: I0919 21:22:13.221939 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"ef71cca7acd75a946500d9b8aecc57e84a9dd3bf5f73da02d2a4f7b91eb0372a"} err="failed to get container status \"ef71cca7acd75a946500d9b8aecc57e84a9dd3bf5f73da02d2a4f7b91eb0372a\": rpc error: code = NotFound desc = an error occurred when try to find container \"ef71cca7acd75a946500d9b8aecc57e84a9dd3bf5f73da02d2a4f7b91eb0372a\": not found" machine # [ 107.303793] systemd[1]: var-lib-kubelet-pods-ee902c0f\x2d4a9d\x2d47d1\x2db6f2\x2d3971baa9c8f7-volume\x2dsubpaths-nix-gc-0.mount: Deactivated successfully. machine # [ 107.310294] systemd[1]: var-lib-kubelet-pods-14aec9b1\x2d6bcd\x2d41a9\x2d9fbe\x2d48013fa6a384-volume\x2dsubpaths-nix-gc-0.mount: Deactivated successfully. machine # [ 107.310547] systemd[1]: var-lib-kubelet-pods-14aec9b1\x2d6bcd\x2d41a9\x2d9fbe\x2d48013fa6a384-volume\x2dsubpaths-nix-nix\x2dgrpc\x2ddaemon-0.mount: Deactivated successfully. machine # [ 107.311595] systemd[1]: var-lib-kubelet-pods-ee902c0f\x2d4a9d\x2d47d1\x2db6f2\x2d3971baa9c8f7-volume\x2dsubpaths-nix-nix\x2dgrpc\x2ddaemon-0.mount: Deactivated successfully. machine # [ 107.312069] systemd[1]: var-lib-kubelet-pods-ee902c0f\x2d4a9d\x2d47d1\x2db6f2\x2d3971baa9c8f7-volume\x2dsubpaths-nix-nix\x2ddaemon-0.mount: Deactivated successfully. machine # [ 107.312640] systemd[1]: var-lib-kubelet-pods-14aec9b1\x2d6bcd\x2d41a9\x2d9fbe\x2d48013fa6a384-volume\x2dsubpaths-nix-nix\x2ddaemon-0.mount: Deactivated successfully. machine # [ 107.313915] systemd[1]: run-netns-cni\x2d745cf538\x2d8b38\x2decf2\x2dc252\x2d907ba11c5b08.mount: Deactivated successfully. machine # [ 107.315457] systemd[1]: var-lib-kubelet-pods-14aec9b1\x2d6bcd\x2d41a9\x2d9fbe\x2d48013fa6a384-volumes-kubernetes.io\x7eprojected-niks3\x2dtoken.mount: Deactivated successfully. machine # [ 107.315936] systemd[1]: var-lib-kubelet-pods-ee902c0f\x2d4a9d\x2d47d1\x2db6f2\x2d3971baa9c8f7-volumes-kubernetes.io\x7esecret-tls.mount: Deactivated successfully. machine # [ 107.316753] systemd[1]: var-lib-kubelet-pods-14aec9b1\x2d6bcd\x2d41a9\x2d9fbe\x2d48013fa6a384-volumes-kubernetes.io\x7esecret-tls.mount: Deactivated successfully. machine # [ 107.317720] systemd[1]: var-lib-kubelet-pods-ee902c0f\x2d4a9d\x2d47d1\x2db6f2\x2d3971baa9c8f7-volumes-kubernetes.io\x7eprojected-niks3\x2dtoken.mount: Deactivated successfully. machine # [ 107.318091] systemd[1]: var-lib-kubelet-pods-ee902c0f\x2d4a9d\x2d47d1\x2db6f2\x2d3971baa9c8f7-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dxgwkg.mount: Deactivated successfully. machine # [ 107.318876] systemd[1]: var-lib-kubelet-pods-14aec9b1\x2d6bcd\x2d41a9\x2d9fbe\x2d48013fa6a384-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dqrngq.mount: Deactivated successfully. machine # [ 107.319526] systemd[1]: var-lib-kubelet-pods-14aec9b1\x2d6bcd\x2d41a9\x2d9fbe\x2d48013fa6a384-volumes-kubernetes.io\x7esecret-ca.mount: Deactivated successfully. machine # [ 107.320086] systemd[1]: var-lib-kubelet-pods-ee902c0f\x2d4a9d\x2d47d1\x2db6f2\x2d3971baa9c8f7-volumes-kubernetes.io\x7esecret-oidc.mount: Deactivated successfully. machine # [ 107.320545] systemd[1]: var-lib-kubelet-pods-14aec9b1\x2d6bcd\x2d41a9\x2d9fbe\x2d48013fa6a384-volumes-kubernetes.io\x7esecret-oidc.mount: Deactivated successfully. machine # [ 107.321071] systemd[1]: var-lib-kubelet-pods-ee902c0f\x2d4a9d\x2d47d1\x2db6f2\x2d3971baa9c8f7-volumes-kubernetes.io\x7esecret-ca.mount: Deactivated successfully. machine # [ 107.359137] systemd[1]: Removed slice libcontainer container kubepods-besteffort-podee902c0f_4a9d_47d1_b6f2_3971baa9c8f7.slice. machine # [ 107.359722] systemd[1]: kubepods-besteffort-podee902c0f_4a9d_47d1_b6f2_3971baa9c8f7.slice: Consumed 932ms CPU time over 32.223s wall clock time, 310.1M memory peak, 47.4M read from disk, 283.8M written to disk, 15.4K incoming IP traffic, 15.5K outgoing IP traffic. machine # [ 107.449526] dhcpcd[6337]: veth9158045b: probing for an IPv4LL address machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 107.938830] dhcpcd[6337]: vethfb2eae5e: probing for an IPv4LL address machine # [ 108.798326] k3s[873]: I0919 21:22:14.887350 873 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="14aec9b1-6bcd-41a9-9fbe-48013fa6a384" path="/var/lib/kubelet/pods/14aec9b1-6bcd-41a9-9fbe-48013fa6a384/volumes" machine # [ 108.798761] k3s[873]: I0919 21:22:14.888099 873 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7" path="/var/lib/kubelet/pods/ee902c0f-4a9d-47d1-b6f2-3971baa9c8f7/volumes" machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 109.354088] dhcpcd[6337]: veth81c92383: probing for an IPv4LL address machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 112.387948] dhcpcd[6337]: veth9158045b: using IPv4LL address 169.254.64.136 machine # [ 112.390378] dhcpcd[6337]: veth9158045b: adding route to 169.254.0.0/16 machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 112.707154] dhcpcd[6337]: vethfb2eae5e: using IPv4LL address 169.254.247.24 machine # [ 112.708250] dhcpcd[6337]: vethfb2eae5e: adding route to 169.254.0.0/16 machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 114.377983] dhcpcd[6337]: veth81c92383: using IPv4LL address 169.254.190.31 machine # [ 114.378580] dhcpcd[6337]: veth81c92383: adding route to 169.254.0.0/16 machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 115.921331] dhcpcd[6337]: veth9158045b: no IPv6 Routers available machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 116.885396] dhcpcd[6337]: vethfb2eae5e: no IPv6 Routers available machine # [ 116.961894] dhcpcd[6337]: veth81c92383: no IPv6 Routers available machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 159.793528] k3s[873]: I0919 21:23:05.883331 873 scope.go:122] "RemoveContainer" containerID="7f3a53e9a3bfa29293dd2d147c25a7e4ddd91eb40fbf2ad330e19127df7876e5" machine # [ 159.872654] systemd[1]: Started libcontainer container 5bf4831cbfac88ce0fe1326052a91a4950f2e503cb310b35394b6f8bd55995a0. machine # [ 160.659339] systemd[1]: cri-containerd-f613bf199062a0f34bfc0e67f722da77c1b52b3e4192cf8539ac4fa2d32e679d.scope: Deactivated successfully. machine # [ 160.660065] systemd[1]: cri-containerd-f613bf199062a0f34bfc0e67f722da77c1b52b3e4192cf8539ac4fa2d32e679d.scope: Consumed 459ms CPU time over 57.945s wall clock time, 15.8M memory peak, 62.5K incoming IP traffic, 58.2K outgoing IP traffic. machine # [ 160.706188] systemd[1]: cri-containerd-f80462ee20289be5052b6f1e51ba74c2111059fb08622ec1edc82423f743fe4b.scope: Deactivated successfully. machine # [ 160.723986] systemd[1]: cri-containerd-4f3aca8d093f0ffecf31405aca382497c938a05fd3f43628a45b46e0a3b7ef88.scope: Deactivated successfully. machine # [ 160.782134] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-f613bf199062a0f34bfc0e67f722da77c1b52b3e4192cf8539ac4fa2d32e679d-rootfs.mount: Deactivated successfully. machine # [ 160.812778] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-f80462ee20289be5052b6f1e51ba74c2111059fb08622ec1edc82423f743fe4b-rootfs.mount: Deactivated successfully. machine # [ 160.822689] systemd[1]: cri-containerd-e10566537825b7410ec27bb4bd88db4a17a0668bb97a743b7b58db32bb6c40eb.scope: Deactivated successfully. machine # [ 160.828287] systemd[1]: cri-containerd-8f31687a726dad45f215df2346ae43e303eeea80cd0e24cd5de7a3e9fb18a546.scope: Deactivated successfully. machine # [ 160.831613] systemd[1]: cri-containerd-8f31687a726dad45f215df2346ae43e303eeea80cd0e24cd5de7a3e9fb18a546.scope: Consumed 123ms CPU time over 56.706s wall clock time, 27.9M memory peak, 14.5M read from disk, 32K written to disk. machine # [ 160.850475] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-4f3aca8d093f0ffecf31405aca382497c938a05fd3f43628a45b46e0a3b7ef88-rootfs.mount: Deactivated successfully. machine # [ 160.853460] systemd[1]: cri-containerd-ff9e08cbda1b9fcf9a6162c76ad2a4340948727430d7755398e7ebafa5c70e88.scope: Deactivated successfully. machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 160.904853] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-ff9e08cbda1b9fcf9a6162c76ad2a4340948727430d7755398e7ebafa5c70e88-rootfs.mount: Deactivated successfully. machine # [ 160.911123] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-e10566537825b7410ec27bb4bd88db4a17a0668bb97a743b7b58db32bb6c40eb-rootfs.mount: Deactivated successfully. machine # [ 160.911688] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-8f31687a726dad45f215df2346ae43e303eeea80cd0e24cd5de7a3e9fb18a546-rootfs.mount: Deactivated successfully. machine # [ 160.943153] systemd[1]: run-k3s-containerd-io.containerd.grpc.v1.cri-sandboxes-ff9e08cbda1b9fcf9a6162c76ad2a4340948727430d7755398e7ebafa5c70e88-shm.mount: Deactivated successfully. machine # [ 161.144895] cni0: port 4(veth81c92383) entered disabled state machine # [ 160.998470] dhcpcd[6337]: veth81c92383: carrier lost machine # [ 161.147556] veth81c92383 (unregistering): left allmulticast mode machine # [ 161.149114] veth81c92383 (unregistering): left promiscuous mode machine # [ 161.150317] cni0: port 4(veth81c92383) entered disabled state machine # [ 161.040176] dhcpcd[6337]: veth81c92383: deleting address fe80::181b:68ff:fea1:2025 machine # [ 161.044530] dhcpcd[6337]: veth81c92383: deleting route to 169.254.0.0/16 machine # [ 161.120266] k3s[873]: I0919 21:23:07.210474 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a1838066-8e10-430d-8c6f-82315f17e552-tls-ca\" (UniqueName: \"kubernetes.io/secret/a1838066-8e10-430d-8c6f-82315f17e552-tls-ca\") pod \"a1838066-8e10-430d-8c6f-82315f17e552\" (UID: \"a1838066-8e10-430d-8c6f-82315f17e552\") " machine # [ 161.129302] k3s[873]: I0919 21:23:07.210548 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a1838066-8e10-430d-8c6f-82315f17e552-tls-lb\" (UniqueName: \"kubernetes.io/secret/a1838066-8e10-430d-8c6f-82315f17e552-tls-lb\") pod \"a1838066-8e10-430d-8c6f-82315f17e552\" (UID: \"a1838066-8e10-430d-8c6f-82315f17e552\") " machine # [ 161.136329] k3s[873]: I0919 21:23:07.210594 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1838066-8e10-430d-8c6f-82315f17e552-tmp\" (UniqueName: \"kubernetes.io/empty-dir/a1838066-8e10-430d-8c6f-82315f17e552-tmp\") pod \"a1838066-8e10-430d-8c6f-82315f17e552\" (UID: \"a1838066-8e10-430d-8c6f-82315f17e552\") " machine # [ 161.142776] k3s[873]: I0919 21:23:07.210631 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a1838066-8e10-430d-8c6f-82315f17e552-config\" (UniqueName: \"kubernetes.io/configmap/a1838066-8e10-430d-8c6f-82315f17e552-config\") pod \"a1838066-8e10-430d-8c6f-82315f17e552\" (UID: \"a1838066-8e10-430d-8c6f-82315f17e552\") " machine # [ 161.149835] k3s[873]: I0919 21:23:07.220557 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1838066-8e10-430d-8c6f-82315f17e552-tmp" pod "a1838066-8e10-430d-8c6f-82315f17e552" (UID: "a1838066-8e10-430d-8c6f-82315f17e552"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 161.156083] k3s[873]: I0919 21:23:07.220615 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1838066-8e10-430d-8c6f-82315f17e552-tls-lb" pod "a1838066-8e10-430d-8c6f-82315f17e552" (UID: "a1838066-8e10-430d-8c6f-82315f17e552"). InnerVolumeSpecName "tls-lb". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 161.162396] k3s[873]: I0919 21:23:07.221013 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1838066-8e10-430d-8c6f-82315f17e552-tls-ca" pod "a1838066-8e10-430d-8c6f-82315f17e552" (UID: "a1838066-8e10-430d-8c6f-82315f17e552"). InnerVolumeSpecName "tls-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 161.169468] k3s[873]: I0919 21:23:07.228632 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1838066-8e10-430d-8c6f-82315f17e552-config" pod "a1838066-8e10-430d-8c6f-82315f17e552" (UID: "a1838066-8e10-430d-8c6f-82315f17e552"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" machine # [ 161.178363] dhcpcd[6337]: veth81c92383: removing interface machine # [ 161.220546] k3s[873]: I0919 21:23:07.311101 873 reconciler_common.go:299] "Volume detached for volume \"tls-ca\" (UniqueName: \"kubernetes.io/secret/a1838066-8e10-430d-8c6f-82315f17e552-tls-ca\") on node \"machine\" DevicePath \"\"" machine # [ 161.220772] k3s[873]: I0919 21:23:07.311134 873 reconciler_common.go:299] "Volume detached for volume \"tls-lb\" (UniqueName: \"kubernetes.io/secret/a1838066-8e10-430d-8c6f-82315f17e552-tls-lb\") on node \"machine\" DevicePath \"\"" machine # [ 161.220991] k3s[873]: I0919 21:23:07.311146 873 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a1838066-8e10-430d-8c6f-82315f17e552-tmp\") on node \"machine\" DevicePath \"\"" machine # [ 161.222086] k3s[873]: I0919 21:23:07.311160 873 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a1838066-8e10-430d-8c6f-82315f17e552-config\") on node \"machine\" DevicePath \"\"" machine # [ 161.262606] k3s[873]: I0919 21:23:07.353133 873 scope.go:122] "RemoveContainer" containerID="f613bf199062a0f34bfc0e67f722da77c1b52b3e4192cf8539ac4fa2d32e679d" machine # [ 161.271058] systemd[1]: Removed slice libcontainer container kubepods-besteffort-poda1838066_8e10_430d_8c6f_82315f17e552.slice. machine # [ 161.272895] systemd[1]: kubepods-besteffort-poda1838066_8e10_430d_8c6f_82315f17e552.slice: Consumed 490ms CPU time over 59.285s wall clock time, 16M memory peak, 62.5K incoming IP traffic, 58.2K outgoing IP traffic. machine # [ 161.279298] k3s[873]: I0919 21:23:07.369849 873 scope.go:122] "RemoveContainer" containerID="f613bf199062a0f34bfc0e67f722da77c1b52b3e4192cf8539ac4fa2d32e679d" machine # [ 161.281345] k3s[873]: E0919 21:23:07.371954 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"f613bf199062a0f34bfc0e67f722da77c1b52b3e4192cf8539ac4fa2d32e679d\": not found" containerID="f613bf199062a0f34bfc0e67f722da77c1b52b3e4192cf8539ac4fa2d32e679d" machine # [ 161.286602] k3s[873]: I0919 21:23:07.372005 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"f613bf199062a0f34bfc0e67f722da77c1b52b3e4192cf8539ac4fa2d32e679d"} err="failed to get container status \"f613bf199062a0f34bfc0e67f722da77c1b52b3e4192cf8539ac4fa2d32e679d\": rpc error: code = NotFound desc = an error occurred when try to find container \"f613bf199062a0f34bfc0e67f722da77c1b52b3e4192cf8539ac4fa2d32e679d\": not found" machine # [ 161.363588] k3s[873]: E0919 21:23:07.454137 873 status.go:71] "Unhandled Error" err="apiserver received an error that is not an metav1.Status: &errors.errorString{s:\"failed to create typed patch object (farm/nix-grpc-farm-scheduler; /v1, Kind=Service): .spec.selector.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0}\"}: failed to create typed patch object (farm/nix-grpc-farm-scheduler; /v1, Kind=Service): .spec.selector.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0}" logger="UnhandledError" machine # [ 161.372955] k3s[873]: I0919 21:23:07.463549 873 alloc.go:329] "allocated clusterIPs" service="farm/nix-grpc-farm" clusterIPs={"IPv4":"10.43.146.59"} machine # [ 161.380036] k3s[873]: E0919 21:23:07.470607 873 status.go:71] "Unhandled Error" err=< machine # [ 161.381207] k3s[873]: apiserver received an error that is not an metav1.Status: &errors.errorString{s:"failed to create typed patch object (farm/nix-grpc-farm-scheduler; apps/v1, Kind=Deployment): errors:\n .spec.selector.matchLabels.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0}\n .spec.template.metadata.labels.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0}"}: failed to create typed patch object (farm/nix-grpc-farm-scheduler; apps/v1, Kind=Deployment): errors: machine # [ 161.387349] k3s[873]: .spec.selector.matchLabels.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0} machine # [ 161.389459] k3s[873]: .spec.template.metadata.labels.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0} machine # [ 161.391253] k3s[873]: > logger="UnhandledError" machine # [ 161.406411] systemd[1]: cri-containerd-5bf4831cbfac88ce0fe1326052a91a4950f2e503cb310b35394b6f8bd55995a0.scope: Deactivated successfully. machine # [ 161.408584] systemd[1]: cri-containerd-5bf4831cbfac88ce0fe1326052a91a4950f2e503cb310b35394b6f8bd55995a0.scope: Consumed 858ms CPU time over 1.534s wall clock time, 30.9M memory peak, 3.3M incoming IP traffic, 197.6K outgoing IP traffic. machine # [ 161.527747] systemd[1]: Created slice libcontainer container kubepods-besteffort-podaf333d0b_795c_4aba_88c4_d8deaf56c0a3.slice. machine # [ 161.534817] systemd[1]: Created slice libcontainer container kubepods-besteffort-pod3504816e_e1c5_458d_9b6f_2017cb4702a9.slice. machine # [ 161.542022] systemd[1]: Created slice libcontainer container kubepods-besteffort-pod4f78420d_d22a_43a9_93c0_97f172907985.slice. machine # [ 161.622147] k3s[873]: I0919 21:23:07.712703 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af333d0b-795c-4aba-88c4-d8deaf56c0a3-tmp\") pod \"nix-grpc-farm-lb-688d8d547b-hxxtx\" (UID: \"af333d0b-795c-4aba-88c4-d8deaf56c0a3\") " pod="farm/nix-grpc-farm-lb-688d8d547b-hxxtx" machine # [ 161.622440] k3s[873]: I0919 21:23:07.712756 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls\" (UniqueName: \"kubernetes.io/secret/3504816e-e1c5-458d-9b6f-2017cb4702a9-tls\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-hnbx5\" (UID: \"3504816e-e1c5-458d-9b6f-2017cb4702a9\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-hnbx5" machine # [ 161.623097] k3s[873]: I0919 21:23:07.712791 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtpdj\" (UniqueName: \"kubernetes.io/projected/3504816e-e1c5-458d-9b6f-2017cb4702a9-kube-api-access-jtpdj\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-hnbx5\" (UID: \"3504816e-e1c5-458d-9b6f-2017cb4702a9\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-hnbx5" machine # [ 161.623809] k3s[873]: I0919 21:23:07.713077 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc\" (UniqueName: \"kubernetes.io/secret/3504816e-e1c5-458d-9b6f-2017cb4702a9-oidc\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-hnbx5\" (UID: \"3504816e-e1c5-458d-9b6f-2017cb4702a9\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-hnbx5" machine # [ 161.624686] k3s[873]: I0919 21:23:07.713217 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nix\" (UniqueName: \"kubernetes.io/empty-dir/4f78420d-d22a-43a9-93c0-97f172907985-nix\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-9f9l9\" (UID: \"4f78420d-d22a-43a9-93c0-97f172907985\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-9f9l9" machine # [ 161.625252] k3s[873]: I0919 21:23:07.713262 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc\" (UniqueName: \"kubernetes.io/secret/4f78420d-d22a-43a9-93c0-97f172907985-oidc\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-9f9l9\" (UID: \"4f78420d-d22a-43a9-93c0-97f172907985\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-9f9l9" machine # [ 161.626322] k3s[873]: I0919 21:23:07.713370 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-ca\" (UniqueName: \"kubernetes.io/secret/af333d0b-795c-4aba-88c4-d8deaf56c0a3-tls-ca\") pod \"nix-grpc-farm-lb-688d8d547b-hxxtx\" (UID: \"af333d0b-795c-4aba-88c4-d8deaf56c0a3\") " pod="farm/nix-grpc-farm-lb-688d8d547b-hxxtx" machine # [ 161.626948] k3s[873]: I0919 21:23:07.713400 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nixconf\" (UniqueName: \"kubernetes.io/configmap/4f78420d-d22a-43a9-93c0-97f172907985-nixconf\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-9f9l9\" (UID: \"4f78420d-d22a-43a9-93c0-97f172907985\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-9f9l9" machine # [ 161.628209] k3s[873]: I0919 21:23:07.713427 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"niks3-token\" (UniqueName: \"kubernetes.io/projected/4f78420d-d22a-43a9-93c0-97f172907985-niks3-token\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-9f9l9\" (UID: \"4f78420d-d22a-43a9-93c0-97f172907985\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-9f9l9" machine # [ 161.629599] k3s[873]: I0919 21:23:07.713452 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/4f78420d-d22a-43a9-93c0-97f172907985-ca\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-9f9l9\" (UID: \"4f78420d-d22a-43a9-93c0-97f172907985\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-9f9l9" machine # [ 161.630251] k3s[873]: I0919 21:23:07.713482 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/af333d0b-795c-4aba-88c4-d8deaf56c0a3-config\") pod \"nix-grpc-farm-lb-688d8d547b-hxxtx\" (UID: \"af333d0b-795c-4aba-88c4-d8deaf56c0a3\") " pod="farm/nix-grpc-farm-lb-688d8d547b-hxxtx" machine # [ 161.630805] k3s[873]: I0919 21:23:07.713510 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tt5nz\" (UniqueName: \"kubernetes.io/projected/4f78420d-d22a-43a9-93c0-97f172907985-kube-api-access-tt5nz\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-9f9l9\" (UID: \"4f78420d-d22a-43a9-93c0-97f172907985\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-9f9l9" machine # [ 161.693160] k3s[873]: I0919 21:23:07.713542 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-lb\" (UniqueName: \"kubernetes.io/secret/af333d0b-795c-4aba-88c4-d8deaf56c0a3-tls-lb\") pod \"nix-grpc-farm-lb-688d8d547b-hxxtx\" (UID: \"af333d0b-795c-4aba-88c4-d8deaf56c0a3\") " pod="farm/nix-grpc-farm-lb-688d8d547b-hxxtx" machine # [ 161.693783] k3s[873]: I0919 21:23:07.713572 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nix\" (UniqueName: \"kubernetes.io/empty-dir/3504816e-e1c5-458d-9b6f-2017cb4702a9-nix\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-hnbx5\" (UID: \"3504816e-e1c5-458d-9b6f-2017cb4702a9\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-hnbx5" machine # [ 161.694421] k3s[873]: I0919 21:23:07.713599 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3504816e-e1c5-458d-9b6f-2017cb4702a9-ca\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-hnbx5\" (UID: \"3504816e-e1c5-458d-9b6f-2017cb4702a9\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-hnbx5" machine # [ 161.694867] k3s[873]: I0919 21:23:07.713624 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"build\" (UniqueName: \"kubernetes.io/empty-dir/4f78420d-d22a-43a9-93c0-97f172907985-build\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-9f9l9\" (UID: \"4f78420d-d22a-43a9-93c0-97f172907985\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-9f9l9" machine # [ 161.695509] k3s[873]: I0919 21:23:07.713647 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"niks3-token\" (UniqueName: \"kubernetes.io/projected/3504816e-e1c5-458d-9b6f-2017cb4702a9-niks3-token\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-hnbx5\" (UID: \"3504816e-e1c5-458d-9b6f-2017cb4702a9\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-hnbx5" machine # [ 161.695986] k3s[873]: I0919 21:23:07.713709 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls\" (UniqueName: \"kubernetes.io/secret/4f78420d-d22a-43a9-93c0-97f172907985-tls\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-9f9l9\" (UID: \"4f78420d-d22a-43a9-93c0-97f172907985\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-9f9l9" machine # [ 161.696581] k3s[873]: I0919 21:23:07.713743 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"build\" (UniqueName: \"kubernetes.io/empty-dir/3504816e-e1c5-458d-9b6f-2017cb4702a9-build\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-hnbx5\" (UID: \"3504816e-e1c5-458d-9b6f-2017cb4702a9\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-hnbx5" machine # [ 161.697083] k3s[873]: I0919 21:23:07.713770 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nixconf\" (UniqueName: \"kubernetes.io/configmap/3504816e-e1c5-458d-9b6f-2017cb4702a9-nixconf\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-hnbx5\" (UID: \"3504816e-e1c5-458d-9b6f-2017cb4702a9\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-hnbx5" machine # [ 161.861892] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-5bf4831cbfac88ce0fe1326052a91a4950f2e503cb310b35394b6f8bd55995a0-rootfs.mount: Deactivated successfully. machine # [ 161.864082] systemd[1]: var-lib-kubelet-pods-a1838066\x2d8e10\x2d430d\x2d8c6f\x2d82315f17e552-volume\x2dsubpaths-config-envoy-0.mount: Deactivated successfully. machine # [ 161.864607] systemd[1]: run-netns-cni\x2d7d296ea2\x2d412c\x2dadae\x2dad35\x2dda745a7e14b3.mount: Deactivated successfully. machine # [ 161.865914] systemd[1]: var-lib-kubelet-pods-a1838066\x2d8e10\x2d430d\x2d8c6f\x2d82315f17e552-volumes-kubernetes.io\x7esecret-tls\x2dca.mount: Deactivated successfully. machine # [ 161.867145] systemd[1]: var-lib-kubelet-pods-a1838066\x2d8e10\x2d430d\x2d8c6f\x2d82315f17e552-volumes-kubernetes.io\x7eempty\x2ddir-tmp.mount: Deactivated successfully. machine # [ 161.868149] systemd[1]: var-lib-kubelet-pods-a1838066\x2d8e10\x2d430d\x2d8c6f\x2d82315f17e552-volumes-kubernetes.io\x7esecret-tls\x2dlb.mount: Deactivated successfully. machine # [ 162.027182] cni0: port 4(vethfaa8ef55) entered blocking state machine # [ 162.028163] cni0: port 4(vethfaa8ef55) entered disabled state machine # [ 162.029482] vethfaa8ef55: entered allmulticast mode machine # [ 162.030839] vethfaa8ef55: entered promiscuous mode machine # [ 161.886333] (udev-worker)[11612]: Network interface NamePolicy= disabled on kernel command line. machine # [ 161.889204] (udev-worker)[11611]: Network interface NamePolicy= disabled on kernel command line. machine # [ 162.038123] cni0: port 6(veth698edbd7) entered blocking state machine # [ 162.040969] cni0: port 6(veth698edbd7) entered disabled state machine # [ 162.043297] veth698edbd7: entered allmulticast mode machine # [ 162.044910] veth698edbd7: entered promiscuous mode machine # [ 162.049004] cni0: port 4(vethfaa8ef55) entered blocking state machine # [ 162.051178] cni0: port 4(vethfaa8ef55) entered forwarding state machine # [ 162.056766] cni0: port 6(veth698edbd7) entered blocking state machine # [ 162.058245] cni0: port 6(veth698edbd7) entered forwarding state machine # [ 161.940412] dhcpcd[6337]: vethfaa8ef55: IAID 6a:ce:5e:60 machine # [ 161.943155] dhcpcd[6337]: vethfaa8ef55: adding address fe80::287a:6aff:fece:5e60 machine # [ 161.961312] dhcpcd[6337]: veth698edbd7: IAID 53:7d:ab:d5 machine # [ 161.962605] dhcpcd[6337]: veth698edbd7: adding address fe80::503f:53ff:fe7d:abd5 machine # [ 162.026663] systemd[1]: Started libcontainer container 29e32b42add63a0f56229212de8cb170fec22da80fdae00761885c321758fb72. machine # [ 162.043411] systemd[1]: Started libcontainer container 49f8629f60d74e6179f08b7d7c4728215cf493d812b5f402421f00f75c52d452. machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 162.146432] systemd[1]: Started libcontainer container 5dd3e9ea82cae64396f53e7727f5f3de0ceb812dc70007717319c395042c7cee. machine # [ 162.157337] systemd[1]: Started libcontainer container 917a846d68e28b0cb235106fcc38e3b333714223d7b316501c86f5fbdc4dc865. machine # [ 162.162086] (udev-worker)[11790]: Network interface NamePolicy= disabled on kernel command line. machine # [ 162.312410] cni0: port 7(vethe45bd26c) entered blocking state machine # [ 162.314223] cni0: port 7(vethe45bd26c) entered disabled state machine # [ 162.315777] vethe45bd26c: entered allmulticast mode machine # [ 162.317285] vethe45bd26c: entered promiscuous mode machine # [ 162.340055] cni0: port 7(vethe45bd26c) entered blocking state machine # [ 162.341480] cni0: port 7(vethe45bd26c) entered forwarding state machine # [ 162.256060] dhcpcd[6337]: vethe45bd26c: waiting for carrier machine # [ 162.256697] dhcpcd[6337]: vethe45bd26c: carrier acquired machine # [ 162.268149] dhcpcd[6337]: vethe45bd26c: IAID c2:96:66:55 machine # [ 162.269535] dhcpcd[6337]: vethe45bd26c: adding address fe80::ecc9:c2ff:fe96:6655 machine # [ 162.316224] k3s[873]: I0919 21:23:08.406446 873 scope.go:122] "RemoveContainer" containerID="7f3a53e9a3bfa29293dd2d147c25a7e4ddd91eb40fbf2ad330e19127df7876e5" machine # [ 162.318816] k3s[873]: I0919 21:23:08.406937 873 scope.go:122] "RemoveContainer" containerID="5bf4831cbfac88ce0fe1326052a91a4950f2e503cb310b35394b6f8bd55995a0" machine # [ 162.320686] k3s[873]: E0919 21:23:08.407174 873 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"helm\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=helm pod=helm-install-nix-grpc-farm-zcg5w_kube-system(6d037147-1d29-4e02-b2ab-2f08bc2a576c)\"" pod="kube-system/helm-install-nix-grpc-farm-zcg5w" podUID="6d037147-1d29-4e02-b2ab-2f08bc2a576c" machine # [ 162.337103] systemd[1]: Started libcontainer container a6ddb3ed35defde8888f9061c2aec09c363f13ee66293544bcabb52190898f32. machine # [ 162.446080] systemd[1]: cri-containerd-efbcb2c7f00e526fc1d543c20dde4263a13402244f747249c01a23ef4e44841c.scope: Deactivated successfully. machine # [ 162.449327] systemd[1]: cri-containerd-efbcb2c7f00e526fc1d543c20dde4263a13402244f747249c01a23ef4e44841c.scope: Consumed 301ms CPU time over 58.173s wall clock time, 11.8M memory peak, 2.4M read from disk, 27.5K incoming IP traffic, 24.9K outgoing IP traffic. machine # [ 162.455927] systemd[1]: cri-containerd-3ade944e74e46ec0b6d412ac35a2b03882f265c4a4d4afb3ba7c4cfd012e3ef8.scope: Deactivated successfully. machine # [ 162.457882] systemd[1]: cri-containerd-3ade944e74e46ec0b6d412ac35a2b03882f265c4a4d4afb3ba7c4cfd012e3ef8.scope: Consumed 319ms CPU time over 58.184s wall clock time, 13.1M memory peak, 4M read from disk, 27.6K incoming IP traffic, 25.1K outgoing IP traffic. machine # [ 162.718198] systemd[1]: cri-containerd-0aef947c62b0e194d5b5aeca7f1c43e38a8382b251b9de9908fbf749baa252d1.scope: Deactivated successfully. machine # [ 162.731793] systemd[1]: cri-containerd-1af82520a49c872b1e17ab513962ee1ff69c760b94c6325f2ead93641f3c76d0.scope: Deactivated successfully. machine # [ 162.749383] systemd[1]: Started libcontainer container eac12c189fbcf16d1c00cefa7622d6a73f466369fd8159c6e33e59dcfe945471. machine # [ 162.764522] systemd[1]: cri-containerd-917a846d68e28b0cb235106fcc38e3b333714223d7b316501c86f5fbdc4dc865.scope: Deactivated successfully. machine # [ 162.769431] systemd[1]: cri-containerd-917a846d68e28b0cb235106fcc38e3b333714223d7b316501c86f5fbdc4dc865.scope: Consumed 492ms CPU time over 607ms wall clock time, 306.7M memory peak, 266.3M written to disk. machine # [ 162.817672] systemd[1]: cri-containerd-5dd3e9ea82cae64396f53e7727f5f3de0ceb812dc70007717319c395042c7cee.scope: Deactivated successfully. machine # [ 162.818155] systemd[1]: cri-containerd-5dd3e9ea82cae64396f53e7727f5f3de0ceb812dc70007717319c395042c7cee.scope: Consumed 515ms CPU time over 670ms wall clock time, 306M memory peak, 270.2M written to disk. machine # [ 162.823176] k3s[873]: I0919 21:23:08.913437 873 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a1838066-8e10-430d-8c6f-82315f17e552" path="/var/lib/kubelet/pods/a1838066-8e10-430d-8c6f-82315f17e552/volumes" machine # [ 162.862681] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-efbcb2c7f00e526fc1d543c20dde4263a13402244f747249c01a23ef4e44841c-rootfs.mount: Deactivated successfully. machine # [ 162.862974] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-3ade944e74e46ec0b6d412ac35a2b03882f265c4a4d4afb3ba7c4cfd012e3ef8-rootfs.mount: Deactivated successfully. machine # [ 162.863854] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-0aef947c62b0e194d5b5aeca7f1c43e38a8382b251b9de9908fbf749baa252d1-rootfs.mount: Deactivated successfully. machine # [ 162.865458] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-1af82520a49c872b1e17ab513962ee1ff69c760b94c6325f2ead93641f3c76d0-rootfs.mount: Deactivated successfully. machine # [ 162.901267] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-917a846d68e28b0cb235106fcc38e3b333714223d7b316501c86f5fbdc4dc865-rootfs.mount: Deactivated successfully. machine # [ 162.908086] systemd[1]: run-k3s-containerd-io.containerd.grpc.v1.cri-sandboxes-0aef947c62b0e194d5b5aeca7f1c43e38a8382b251b9de9908fbf749baa252d1-shm.mount: Deactivated successfully. machine # [ 162.923357] systemd[1]: run-k3s-containerd-io.containerd.grpc.v1.cri-sandboxes-1af82520a49c872b1e17ab513962ee1ff69c760b94c6325f2ead93641f3c76d0-shm.mount: Deactivated successfully. machine # [ 162.985367] d[ 163.132094] cni0: port 3(veth9158045b) entered disabled state machine # hcpcd[6337]: veth9158045b: carrier lost machine # [ 163.135224] veth9158045b (unregistering): left allmulticast mode machine # [ 163.136897] veth9158045b (unregistering): left promiscuous mode machine # [ 163.138056] cni0: port 3(veth9158045b) entered disabled state machine # [ 163.145801] cni0: port 5(vethfb2eae5e) entered disabled state machine # [ 163.148521] vethfb2eae5e (unregistering): left allmulticast mode machine # [ 163.150590] vethfb2eae5e (unregistering): left promiscuous mode machine # [ 163.152558] cni0: port 5(vethfb2eae5e) entered disabled state machine # [ 163.037971] k3s[873]: I0919 21:23:09.128524 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/eb4a3576-135a-4de5-8386-b0da1ef01e63-tls\" (UniqueName: \"kubernetes.io/secret/eb4a3576-135a-4de5-8386-b0da1ef01e63-tls\") pod \"eb4a3576-135a-4de5-8386-b0da1ef01e63\" (UID: \"eb4a3576-135a-4de5-8386-b0da1ef01e63\") " machine # [ 163.046873] k3s[873]: I0919 21:23:09.128579 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eb4a3576-135a-4de5-8386-b0da1ef01e63-nixconf\" (UniqueName: \"kubernetes.io/configmap/eb4a3576-135a-4de5-8386-b0da1ef01e63-nixconf\") pod \"eb4a3576-135a-4de5-8386-b0da1ef01e63\" (UID: \"eb4a3576-135a-4de5-8386-b0da1ef01e63\") " machine # [ 163.053792] k3s[873]: I0919 21:23:09.128612 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb4a3576-135a-4de5-8386-b0da1ef01e63-niks3-token\" (UniqueName: \"kubernetes.io/projected/eb4a3576-135a-4de5-8386-b0da1ef01e63-niks3-token\") pod \"eb4a3576-135a-4de5-8386-b0da1ef01e63\" (UID: \"eb4a3576-135a-4de5-8386-b0da1ef01e63\") " machine # [ 163.061164] k3s[873]: I0919 21:23:09.128644 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb4a3576-135a-4de5-8386-b0da1ef01e63-kube-api-access-bsl7n\" (UniqueName: \"kubernetes.io/projected/eb4a3576-135a-4de5-8386-b0da1ef01e63-kube-api-access-bsl7n\") pod \"eb4a3576-135a-4de5-8386-b0da1ef01e63\" (UID: \"eb4a3576-135a-4de5-8386-b0da1ef01e63\") " machine # [ 163.068792] k3s[873]: I0919 21:23:09.128701 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb4a3576-135a-4de5-8386-b0da1ef01e63-build\" (UniqueName: \"kubernetes.io/empty-dir/eb4a3576-135a-4de5-8386-b0da1ef01e63-build\") pod \"eb4a3576-135a-4de5-8386-b0da1ef01e63\" (UID: \"eb4a3576-135a-4de5-8386-b0da1ef01e63\") " machine # [ 163.075318] k3s[873]: I0919 21:23:09.128731 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/eb4a3576-135a-4de5-8386-b0da1ef01e63-oidc\" (UniqueName: \"kubernetes.io/secret/eb4a3576-135a-4de5-8386-b0da1ef01e63-oidc\") pod \"eb4a3576-135a-4de5-8386-b0da1ef01e63\" (UID: \"eb4a3576-135a-4de5-8386-b0da1ef01e63\") " machine # [ 163.083695] k3s[873]: I0919 21:23:09.128758 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb4a3576-135a-4de5-8386-b0da1ef01e63-nix\" (UniqueName: \"kubernetes.io/empty-dir/eb4a3576-135a-4de5-8386-b0da1ef01e63-nix\") pod \"eb4a3576-135a-4de5-8386-b0da1ef01e63\" (UID: \"eb4a3576-135a-4de5-8386-b0da1ef01e63\") " machine # [ 163.091854] k3s[873]: I0919 21:23:09.128783 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/eb4a3576-135a-4de5-8386-b0da1ef01e63-ca\" (UniqueName: \"kubernetes.io/secret/eb4a3576-135a-4de5-8386-b0da1ef01e63-ca\") pod \"eb4a3576-135a-4de5-8386-b0da1ef01e63\" (UID: \"eb4a3576-135a-4de5-8386-b0da1ef01e63\") " machine # [ 163.098600] k3s[873]: I0919 21:23:09.131937 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb4a3576-135a-4de5-8386-b0da1ef01e63-build" pod "eb4a3576-135a-4de5-8386-b0da1ef01e63" (UID: "eb4a3576-135a-4de5-8386-b0da1ef01e63"). InnerVolumeSpecName "build". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 163.105783] k3s[873]: I0919 21:23:09.133562 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb4a3576-135a-4de5-8386-b0da1ef01e63-nixconf" pod "eb4a3576-135a-4de5-8386-b0da1ef01e63" (UID: "eb4a3576-135a-4de5-8386-b0da1ef01e63"). InnerVolumeSpecName "nixconf". PluginName "kubernetes.io/configmap", VolumeGIDValue "" machine # [ 163.112300] k3s[873]: I0919 21:23:09.144760 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb4a3576-135a-4de5-8386-b0da1ef01e63-ca" pod "eb4a3576-135a-4de5-8386-b0da1ef01e63" (UID: "eb4a3576-135a-4de5-8386-b0da1ef01e63"). InnerVolumeSpecName "ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 163.112543] k3s[873]: I0919 21:23:09.146477 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb4a3576-135a-4de5-8386-b0da1ef01e63-tls" pod "eb4a3576-135a-4de5-8386-b0da1ef01e63" (UID: "eb4a3576-135a-4de5-8386-b0da1ef01e63"). InnerVolumeSpecName "tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 163.113111] dhcpcd[6337]: veth9158045b: deleting address fe80::84f2:63ff:fe72:c502 machine # [ 163.113665] k3s[873]: I0919 21:23:09.152751 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb4a3576-135a-4de5-8386-b0da1ef01e63-kube-api-access-bsl7n" pod "eb4a3576-135a-4de5-8386-b0da1ef01e63" (UID: "eb4a3576-135a-4de5-8386-b0da1ef01e63"). InnerVolumeSpecName "kube-api-access-bsl7n". PluginName "kubernetes.io/projected", VolumeGIDValue "" machine # [ 163.114532] k3s[873]: I0919 21:23:09.157600 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb4a3576-135a-4de5-8386-b0da1ef01e63-oidc" pod "eb4a3576-135a-4de5-8386-b0da1ef01e63" (UID: "eb4a3576-135a-4de5-8386-b0da1ef01e63"). InnerVolumeSpecName "oidc". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 163.115885] k3s[873]: I0919 21:23:09.157701 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb4a3576-135a-4de5-8386-b0da1ef01e63-niks3-token" pod "eb4a3576-135a-4de5-8386-b0da1ef01e63" (UID: "eb4a3576-135a-4de5-8386-b0da1ef01e63"). InnerVolumeSpecName "niks3-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" machine # [ 163.116785] dhcpcd[6337]: veth9158045b: deleting route to 169.254.0.0/16 machine # [ 163.139527] k3s[873]: I0919 21:23:09.229441 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/658947de-798a-4627-a442-01fff1c59e29-nixconf\" (UniqueName: \"kubernetes.io/configmap/658947de-798a-4627-a442-01fff1c59e29-nixconf\") pod \"658947de-798a-4627-a442-01fff1c59e29\" (UID: \"658947de-798a-4627-a442-01fff1c59e29\") " machine # [ 163.140448] k3s[873]: I0919 21:23:09.229496 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/658947de-798a-4627-a442-01fff1c59e29-build\" (UniqueName: \"kubernetes.io/empty-dir/658947de-798a-4627-a442-01fff1c59e29-build\") pod \"658947de-798a-4627-a442-01fff1c59e29\" (UID: \"658947de-798a-4627-a442-01fff1c59e29\") " machine # [ 163.140860] k3s[873]: I0919 21:23:09.229527 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/658947de-798a-4627-a442-01fff1c59e29-tls\" (UniqueName: \"kubernetes.io/secret/658947de-798a-4627-a442-01fff1c59e29-tls\") pod \"658947de-798a-4627-a442-01fff1c59e29\" (UID: \"658947de-798a-4627-a442-01fff1c59e29\") " machine # [ 163.141663] k3s[873]: I0919 21:23:09.229910 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/658947de-798a-4627-a442-01fff1c59e29-kube-api-access-ms2mt\" (UniqueName: \"kubernetes.io/projected/658947de-798a-4627-a442-01fff1c59e29-kube-api-access-ms2mt\") pod \"658947de-798a-4627-a442-01fff1c59e29\" (UID: \"658947de-798a-4627-a442-01fff1c59e29\") " machine # [ 163.144173] k3s[873]: I0919 21:23:09.229945 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/658947de-798a-4627-a442-01fff1c59e29-ca\" (UniqueName: \"kubernetes.io/secret/658947de-798a-4627-a442-01fff1c59e29-ca\") pod \"658947de-798a-4627-a442-01fff1c59e29\" (UID: \"658947de-798a-4627-a442-01fff1c59e29\") " machine # [ 163.144256] k3s[873]: I0919 21:23:09.229986 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/658947de-798a-4627-a442-01fff1c59e29-nix\" (UniqueName: \"kubernetes.io/empty-dir/658947de-798a-4627-a442-01fff1c59e29-nix\") pod \"658947de-798a-4627-a442-01fff1c59e29\" (UID: \"658947de-798a-4627-a442-01fff1c59e29\") " machine # [ 163.144801] k3s[873]: I0919 21:23:09.230014 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/658947de-798a-4627-a442-01fff1c59e29-oidc\" (UniqueName: \"kubernetes.io/secret/658947de-798a-4627-a442-01fff1c59e29-oidc\") pod \"658947de-798a-4627-a442-01fff1c59e29\" (UID: \"658947de-798a-4627-a442-01fff1c59e29\") " machine # [ 163.147614] k3s[873]: I0919 21:23:09.230051 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/658947de-798a-4627-a442-01fff1c59e29-niks3-token\" (UniqueName: \"kubernetes.io/projected/658947de-798a-4627-a442-01fff1c59e29-niks3-token\") pod \"658947de-798a-4627-a442-01fff1c59e29\" (UID: \"658947de-798a-4627-a442-01fff1c59e29\") " machine # [ 163.150025] k3s[873]: I0919 21:23:09.230100 873 reconciler_common.go:299] "Volume detached for volume \"build\" (UniqueName: \"kubernetes.io/empty-dir/eb4a3576-135a-4de5-8386-b0da1ef01e63-build\") on node \"machine\" DevicePath \"\"" machine # [ 163.150118] k3s[873]: I0919 21:23:09.230115 873 reconciler_common.go:299] "Volume detached for volume \"oidc\" (UniqueName: \"kubernetes.io/secret/eb4a3576-135a-4de5-8386-b0da1ef01e63-oidc\") on node \"machine\" DevicePath \"\"" machine # [ 163.150197] k3s[873]: I0919 21:23:09.230126 873 reconciler_common.go:299] "Volume detached for volume \"ca\" (UniqueName: \"kubernetes.io/secret/eb4a3576-135a-4de5-8386-b0da1ef01e63-ca\") on node \"machine\" DevicePath \"\"" machine # [ 163.150277] k3s[873]: I0919 21:23:09.230138 873 reconciler_common.go:299] "Volume detached for volume \"tls\" (UniqueName: \"kubernetes.io/secret/eb4a3576-135a-4de5-8386-b0da1ef01e63-tls\") on node \"machine\" DevicePath \"\"" machine # [ 163.150344] k3s[873]: I0919 21:23:09.230150 873 reconciler_common.go:299] "Volume detached for volume \"nixconf\" (UniqueName: \"kubernetes.io/configmap/eb4a3576-135a-4de5-8386-b0da1ef01e63-nixconf\") on node \"machine\" DevicePath \"\"" machine # [ 163.225680] k3s[873]: I0919 21:23:09.230164 873 reconciler_common.go:299] "Volume detached for volume \"niks3-token\" (UniqueName: \"kubernetes.io/projected/eb4a3576-135a-4de5-8386-b0da1ef01e63-niks3-token\") on node \"machine\" DevicePath \"\"" machine # [ 163.226751] k3s[873]: I0919 21:23:09.230178 873 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bsl7n\" (UniqueName: \"kubernetes.io/projected/eb4a3576-135a-4de5-8386-b0da1ef01e63-kube-api-access-bsl7n\") on node \"machine\" DevicePath \"\"" machine # [ 163.227979] k3s[873]: I0919 21:23:09.229941 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/658947de-798a-4627-a442-01fff1c59e29-nixconf" pod "658947de-798a-4627-a442-01fff1c59e29" (UID: "658947de-798a-4627-a442-01fff1c59e29"). InnerVolumeSpecName "nixconf". PluginName "kubernetes.io/configmap", VolumeGIDValue "" machine # [ 163.230112] k3s[873]: I0919 21:23:09.230404 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/658947de-798a-4627-a442-01fff1c59e29-build" pod "658947de-798a-4627-a442-01fff1c59e29" (UID: "658947de-798a-4627-a442-01fff1c59e29"). InnerVolumeSpecName "build". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 163.240804] k3s[873]: I0919 21:23:09.331349 873 reconciler_common.go:299] "Volume detached for volume \"nixconf\" (UniqueName: \"kubernetes.io/configmap/658947de-798a-4627-a442-01fff1c59e29-nixconf\") on node \"machine\" DevicePath \"\"" machine # [ 163.241596] k3s[873]: I0919 21:23:09.331376 873 reconciler_common.go:299] "Volume detached for volume \"build\" (UniqueName: \"kubernetes.io/empty-dir/658947de-798a-4627-a442-01fff1c59e29-build\") on node \"machine\" DevicePath \"\"" machine # [ 163.255864] k3s[873]: I0919 21:23:09.346429 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/658947de-798a-4627-a442-01fff1c59e29-ca" pod "658947de-798a-4627-a442-01fff1c59e29" (UID: "658947de-798a-4627-a442-01fff1c59e29"). InnerVolumeSpecName "ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 163.268074] k3s[873]: I0919 21:23:09.356640 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/658947de-798a-4627-a442-01fff1c59e29-kube-api-access-ms2mt" pod "658947de-798a-4627-a442-01fff1c59e29" (UID: "658947de-798a-4627-a442-01fff1c59e29"). InnerVolumeSpecName "kube-api-access-ms2mt". PluginName "kubernetes.io/projected", VolumeGIDValue "" machine # [ 163.268466] k3s[873]: I0919 21:23:09.356816 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/658947de-798a-4627-a442-01fff1c59e29-niks3-token" pod "658947de-798a-4627-a442-01fff1c59e29" (UID: "658947de-798a-4627-a442-01fff1c59e29"). InnerVolumeSpecName "niks3-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" machine # [ 163.268932] k3s[873]: I0919 21:23:09.356913 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/658947de-798a-4627-a442-01fff1c59e29-tls" pod "658947de-798a-4627-a442-01fff1c59e29" (UID: "658947de-798a-4627-a442-01fff1c59e29"). InnerVolumeSpecName "tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 163.269755] k3s[873]: I0919 21:23:09.357263 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/658947de-798a-4627-a442-01fff1c59e29-oidc" pod "658947de-798a-4627-a442-01fff1c59e29" (UID: "658947de-798a-4627-a442-01fff1c59e29"). InnerVolumeSpecName "oidc". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 163.270783] k3s[873]: I0919 21:23:09.360983 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb4a3576-135a-4de5-8386-b0da1ef01e63-nix" pod "eb4a3576-135a-4de5-8386-b0da1ef01e63" (UID: "eb4a3576-135a-4de5-8386-b0da1ef01e63"). InnerVolumeSpecName "nix". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 163.319558] dhcpcd[6337]: vethfb2eae5e: if_getmtu: No such device machine # [ 163.320836] dhcpcd[6337]: vethfaa8ef55: soliciting a DHCP lease machine # [ 163.323430] dhcpcd[6337]: veth9158045b: removing interface machine # [ 163.341461] k3s[873]: I0919 21:23:09.431780 873 reconciler_common.go:299] "Volume detached for volume \"niks3-token\" (UniqueName: \"kubernetes.io/projected/658947de-798a-4627-a442-01fff1c59e29-niks3-token\") on node \"machine\" DevicePath \"\"" machine # [ 163.344658] k3s[873]: I0919 21:23:09.431817 873 reconciler_common.go:299] "Volume detached for volume \"nix\" (UniqueName: \"kubernetes.io/empty-dir/eb4a3576-135a-4de5-8386-b0da1ef01e63-nix\") on node \"machine\" DevicePath \"\"" machine # [ 163.349268] k3s[873]: I0919 21:23:09.431833 873 reconciler_common.go:299] "Volume detached for volume \"tls\" (UniqueName: \"kubernetes.io/secret/658947de-798a-4627-a442-01fff1c59e29-tls\") on node \"machine\" DevicePath \"\"" machine # [ 163.354359] k3s[873]: I0919 21:23:09.431848 873 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ms2mt\" (UniqueName: \"kubernetes.io/projected/658947de-798a-4627-a442-01fff1c59e29-kube-api-access-ms2mt\") on node \"machine\" DevicePath \"\"" machine # [ 163.355170] k3s[873]: I0919 21:23:09.431862 873 reconciler_common.go:299] "Volume detached for volume \"ca\" (UniqueName: \"kubernetes.io/secret/658947de-798a-4627-a442-01fff1c59e29-ca\") on node \"machine\" DevicePath \"\"" machine # [ 163.355978] k3s[873]: I0919 21:23:09.431874 873 reconciler_common.go:299] "Volume detached for volume \"oidc\" (UniqueName: \"kubernetes.io/secret/658947de-798a-4627-a442-01fff1c59e29-oidc\") on node \"machine\" DevicePath \"\"" machine # [ 163.365825] dhcpcd[6337]: dhcp_readbpf: unexpected event 0x0100 machine # [ 163.367380] dhcpcd[6337]: dhcp_readbpf: vethfb2eae5e: Network is down machine # [ 163.370159] dhcpcd[6337]: arp_read: unexpected event 0x0100 machine # [ 163.371373] dhcpcd[6337]: arp_read: vethfb2eae5e: Network is down machine # [ 163.377305] dhcpcd[6337]: vethfb2eae5e: removing interface machine # [ 163.377765] dhcpcd[6337]: vethfb2eae5e: deleting address fe80::3804:b2ff:fe1b:d8a6 machine # [ 163.378775] dhcpcd[6337]: vethfb2eae5e: deleting route to 169.254.0.0/16 machine # [ 163.454086] k3s[873]: I0919 21:23:09.544505 873 scope.go:122] "RemoveContainer" containerID="f80462ee20289be5052b6f1e51ba74c2111059fb08622ec1edc82423f743fe4b" machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 163.480544] k3s[873]: I0919 21:23:09.571112 873 scope.go:122] "RemoveContainer" containerID="3ade944e74e46ec0b6d412ac35a2b03882f265c4a4d4afb3ba7c4cfd012e3ef8" machine # [ 163.500286] k3s[873]: I0919 21:23:09.590838 873 scope.go:122] "RemoveContainer" containerID="8f31687a726dad45f215df2346ae43e303eeea80cd0e24cd5de7a3e9fb18a546" machine # [ 163.510539] systemd[1]: Removed slice libcontainer container kubepods-besteffort-podeb4a3576_135a_4de5_8386_b0da1ef01e63.slice. machine # [ 163.511748] systemd[1]: kubepods-besteffort-podeb4a3576_135a_4de5_8386_b0da1ef01e63.slice: Consumed 998ms CPU time over 1min 1.533s wall clock time, 321.1M memory peak, 2.8M read from disk, 283.9M written to disk, 27.5K incoming IP traffic, 24.9K outgoing IP traffic. machine # [ 163.550089] k3s[873]: I0919 21:23:09.640576 873 scope.go:122] "RemoveContainer" containerID="eeef823f106441635eb627f73d876e38f208e6590417f75ccea1b70d1ca1ee88" machine # [ 163.552625] k3s[873]: I0919 21:23:09.643225 873 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="farm/nix-grpc-farm-lb-688d8d547b-hxxtx" podStartSLOduration=2.643207009 podStartE2EDuration="2.643207009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 21:23:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 21:23:09.612125964 +0000 UTC m=+145.195970702" watchObservedRunningTime="2026-09-19 21:23:09.643207009 +0000 UTC m=+145.227051747" machine # [ 163.575771] k3s[873]: I0919 21:23:09.666342 873 scope.go:122] "RemoveContainer" containerID="f80462ee20289be5052b6f1e51ba74c2111059fb08622ec1edc82423f743fe4b" machine # [ 163.577615] k3s[873]: E0919 21:23:09.668166 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"f80462ee20289be5052b6f1e51ba74c2111059fb08622ec1edc82423f743fe4b\": not found" containerID="f80462ee20289be5052b6f1e51ba74c2111059fb08622ec1edc82423f743fe4b" machine # [ 163.578430] k3s[873]: I0919 21:23:09.668307 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"f80462ee20289be5052b6f1e51ba74c2111059fb08622ec1edc82423f743fe4b"} err="failed to get container status \"f80462ee20289be5052b6f1e51ba74c2111059fb08622ec1edc82423f743fe4b\": rpc error: code = NotFound desc = an error occurred when try to find container \"f80462ee20289be5052b6f1e51ba74c2111059fb08622ec1edc82423f743fe4b\": not found" machine # [ 163.578894] k3s[873]: I0919 21:23:09.668338 873 scope.go:122] "RemoveContainer" containerID="3ade944e74e46ec0b6d412ac35a2b03882f265c4a4d4afb3ba7c4cfd012e3ef8" machine # [ 163.581316] k3s[873]: E0919 21:23:09.671934 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"3ade944e74e46ec0b6d412ac35a2b03882f265c4a4d4afb3ba7c4cfd012e3ef8\": not found" containerID="3ade944e74e46ec0b6d412ac35a2b03882f265c4a4d4afb3ba7c4cfd012e3ef8" machine # [ 163.582190] k3s[873]: I0919 21:23:09.672661 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"3ade944e74e46ec0b6d412ac35a2b03882f265c4a4d4afb3ba7c4cfd012e3ef8"} err="failed to get container status \"3ade944e74e46ec0b6d412ac35a2b03882f265c4a4d4afb3ba7c4cfd012e3ef8\": rpc error: code = NotFound desc = an error occurred when try to find container \"3ade944e74e46ec0b6d412ac35a2b03882f265c4a4d4afb3ba7c4cfd012e3ef8\": not found" machine # [ 163.582796] k3s[873]: I0919 21:23:09.672715 873 scope.go:122] "RemoveContainer" containerID="8f31687a726dad45f215df2346ae43e303eeea80cd0e24cd5de7a3e9fb18a546" machine # [ 163.583035] k3s[873]: E0919 21:23:09.673163 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"8f31687a726dad45f215df2346ae43e303eeea80cd0e24cd5de7a3e9fb18a546\": not found" containerID="8f31687a726dad45f215df2346ae43e303eeea80cd0e24cd5de7a3e9fb18a546" machine # [ 163.583981] k3s[873]: I0919 21:23:09.673190 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"8f31687a726dad45f215df2346ae43e303eeea80cd0e24cd5de7a3e9fb18a546"} err="failed to get container status \"8f31687a726dad45f215df2346ae43e303eeea80cd0e24cd5de7a3e9fb18a546\": rpc error: code = NotFound desc = an error occurred when try to find container \"8f31687a726dad45f215df2346ae43e303eeea80cd0e24cd5de7a3e9fb18a546\": not found" machine # [ 163.584798] k3s[873]: I0919 21:23:09.673214 873 scope.go:122] "RemoveContainer" containerID="eeef823f106441635eb627f73d876e38f208e6590417f75ccea1b70d1ca1ee88" machine # [ 163.585490] k3s[873]: E0919 21:23:09.673927 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"eeef823f106441635eb627f73d876e38f208e6590417f75ccea1b70d1ca1ee88\": not found" containerID="eeef823f106441635eb627f73d876e38f208e6590417f75ccea1b70d1ca1ee88" machine # [ 163.585932] k3s[873]: I0919 21:23:09.673947 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"eeef823f106441635eb627f73d876e38f208e6590417f75ccea1b70d1ca1ee88"} err="failed to get container status \"eeef823f106441635eb627f73d876e38f208e6590417f75ccea1b70d1ca1ee88\": rpc error: code = NotFound desc = an error occurred when try to find container \"eeef823f106441635eb627f73d876e38f208e6590417f75ccea1b70d1ca1ee88\": not found" machine # [ 163.587684] k3s[873]: I0919 21:23:09.673964 873 scope.go:122] "RemoveContainer" containerID="4f3aca8d093f0ffecf31405aca382497c938a05fd3f43628a45b46e0a3b7ef88" machine # [ 163.594187] k3s[873]: I0919 21:23:09.684078 873 scope.go:122] "RemoveContainer" containerID="efbcb2c7f00e526fc1d543c20dde4263a13402244f747249c01a23ef4e44841c" machine # [ 163.627679] k3s[873]: I0919 21:23:09.718254 873 scope.go:122] "RemoveContainer" containerID="e10566537825b7410ec27bb4bd88db4a17a0668bb97a743b7b58db32bb6c40eb" machine # [ 163.639450] systemd[1]: Started libcontainer container f723ffb0ffdf00b4ed03f3d5d0b8480677e418bc8325d2a2ba48a242f47f62e4. machine # [ 163.642417] k3s[873]: I0919 21:23:09.730492 873 scope.go:122] "RemoveContainer" containerID="0d22b669f678b552053934b9cc1112b738f2f2b89fa8f973edfefbc8b523d0d6" machine # [ 163.653498] k3s[873]: I0919 21:23:09.743955 873 scope.go:122] "RemoveContainer" containerID="4f3aca8d093f0ffecf31405aca382497c938a05fd3f43628a45b46e0a3b7ef88" machine # [ 163.656947] k3s[873]: E0919 21:23:09.747451 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"4f3aca8d093f0ffecf31405aca382497c938a05fd3f43628a45b46e0a3b7ef88\": not found" containerID="4f3aca8d093f0ffecf31405aca382497c938a05fd3f43628a45b46e0a3b7ef88" machine # [ 163.657660] k3s[873]: I0919 21:23:09.747493 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"4f3aca8d093f0ffecf31405aca382497c938a05fd3f43628a45b46e0a3b7ef88"} err="failed to get container status \"4f3aca8d093f0ffecf31405aca382497c938a05fd3f43628a45b46e0a3b7ef88\": rpc error: code = NotFound desc = an error occurred when try to find container \"4f3aca8d093f0ffecf31405aca382497c938a05fd3f43628a45b46e0a3b7ef88\": not found" machine # [ 163.658575] k3s[873]: I0919 21:23:09.747523 873 scope.go:122] "RemoveContainer" containerID="efbcb2c7f00e526fc1d543c20dde4263a13402244f747249c01a23ef4e44841c" machine # [ 163.671816] k3s[873]: I0919 21:23:09.762388 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/658947de-798a-4627-a442-01fff1c59e29-nix" pod "658947de-798a-4627-a442-01fff1c59e29" (UID: "658947de-798a-4627-a442-01fff1c59e29"). InnerVolumeSpecName "nix". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 163.681984] k3s[873]: E0919 21:23:09.772577 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"efbcb2c7f00e526fc1d543c20dde4263a13402244f747249c01a23ef4e44841c\": not found" containerID="efbcb2c7f00e526fc1d543c20dde4263a13402244f747249c01a23ef4e44841c" machine # [ 163.682366] k3s[873]: I0919 21:23:09.772622 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"efbcb2c7f00e526fc1d543c20dde4263a13402244f747249c01a23ef4e44841c"} err="failed to get container status \"efbcb2c7f00e526fc1d543c20dde4263a13402244f747249c01a23ef4e44841c\": rpc error: code = NotFound desc = an error occurred when try to find container \"efbcb2c7f00e526fc1d543c20dde4263a13402244f747249c01a23ef4e44841c\": not found" machine # [ 163.682818] k3s[873]: I0919 21:23:09.772658 873 scope.go:122] "RemoveContainer" containerID="e10566537825b7410ec27bb4bd88db4a17a0668bb97a743b7b58db32bb6c40eb" machine # [ 163.694925] systemd[1]: Started libcontainer container 958ccc32ed8090c615b6ac3092471d8722ac7c8f642f3c89930574729fc602df. machine # [ 163.695339] k3s[873]: E0919 21:23:09.785372 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"e10566537825b7410ec27bb4bd88db4a17a0668bb97a743b7b58db32bb6c40eb\": not found" containerID="e10566537825b7410ec27bb4bd88db4a17a0668bb97a743b7b58db32bb6c40eb" machine # [ 163.696278] k3s[873]: I0919 21:23:09.785416 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"e10566537825b7410ec27bb4bd88db4a17a0668bb97a743b7b58db32bb6c40eb"} err="failed to get container status \"e10566537825b7410ec27bb4bd88db4a17a0668bb97a743b7b58db32bb6c40eb\": rpc error: code = NotFound desc = an error occurred when try to find container \"e10566537825b7410ec27bb4bd88db4a17a0668bb97a743b7b58db32bb6c40eb\": not found" machine # [ 163.696928] k3s[873]: I0919 21:23:09.785444 873 scope.go:122] "RemoveContainer" containerID="0d22b669f678b552053934b9cc1112b738f2f2b89fa8f973edfefbc8b523d0d6" machine # [ 163.698686] k3s[873]: E0919 21:23:09.788716 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"0d22b669f678b552053934b9cc1112b738f2f2b89fa8f973edfefbc8b523d0d6\": not found" containerID="0d22b669f678b552053934b9cc1112b738f2f2b89fa8f973edfefbc8b523d0d6" machine # [ 163.699265] k3s[873]: I0919 21:23:09.788745 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"0d22b669f678b552053934b9cc1112b738f2f2b89fa8f973edfefbc8b523d0d6"} err="failed to get container status \"0d22b669f678b552053934b9cc1112b738f2f2b89fa8f973edfefbc8b523d0d6\": rpc error: code = NotFound desc = an error occurred when try to find container \"0d22b669f678b552053934b9cc1112b738f2f2b89fa8f973edfefbc8b523d0d6\": not found" machine # [ 163.747089] k3s[873]: I0919 21:23:09.837521 873 reconciler_common.go:299] "Volume detached for volume \"nix\" (UniqueName: \"kubernetes.io/empty-dir/658947de-798a-4627-a442-01fff1c59e29-nix\") on node \"machine\" DevicePath \"\"" machine # [ 163.766457] systemd[1]: Removed slice libcontainer container kubepods-besteffort-pod658947de_798a_4627_a442_01fff1c59e29.slice. machine # [ 163.766800] systemd[1]: kubepods-besteffort-pod658947de_798a_4627_a442_01fff1c59e29.slice: Consumed 1.007s CPU time over 1min 1.765s wall clock time, 318.9M memory peak, 19.2M read from disk, 283.9M written to disk, 27.6K incoming IP traffic, 25.1K outgoing IP traffic. machine # [ 163.848211] dhcpcd[6337]: veth698edbd7: soliciting a DHCP lease machine # [ 163.848457] systemd[1]: Started libcontainer container 06fc36d18e9c836e8e351d31c937b07ff7ef390f09bdd99fc5958c9004959311. machine # [ 163.866954] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-5dd3e9ea82cae64396f53e7727f5f3de0ceb812dc70007717319c395042c7cee-rootfs.mount: Deactivated successfully. machine # [ 163.868916] systemd[1]: var-lib-kubelet-pods-eb4a3576\x2d135a\x2d4de5\x2d8386\x2db0da1ef01e63-volume\x2dsubpaths-nix-gc-0.mount: Deactivated successfully. machine # [ 163.869486] systemd[1]: var-lib-kubelet-pods-658947de\x2d798a\x2d4627\x2da442\x2d01fff1c59e29-volume\x2dsubpaths-nix-gc-0.mount: Deactivated successfully. machine # [ 163.870303] systemd[1]: var-lib-kubelet-pods-eb4a3576\x2d135a\x2d4de5\x2d8386\x2db0da1ef01e63-volume\x2dsubpaths-nix-nix\x2dgrpc\x2ddaemon-0.mount: Deactivated successfully. machine # [ 163.870739] systemd[1]: var-lib-kubelet-pods-658947de\x2d798a\x2d4627\x2da442\x2d01fff1c59e29-volume\x2dsubpaths-nix-nix\x2dgrpc\x2ddaemon-0.mount: Deactivated successfully. machine # [ 163.871530] systemd[1]: var-lib-kubelet-pods-eb4a3576\x2d135a\x2d4de5\x2d8386\x2db0da1ef01e63-volume\x2dsubpaths-nix-nix\x2ddaemon-0.mount: Deactivated successfully. machine # [ 163.872096] systemd[1]: var-lib-kubelet-pods-658947de\x2d798a\x2d4627\x2da442\x2d01fff1c59e29-volume\x2dsubpaths-nix-nix\x2ddaemon-0.mount: Deactivated successfully. machine # [ 163.872583] systemd[1]: run-netns-cni\x2d397296f3\x2d1ec4\x2d2f34\x2d9ef4\x2df9b7550985f4.mount: Deactivated successfully. machine # [ 163.873156] systemd[1]: run-netns-cni\x2d936efcf9\x2d8b5c\x2db95d\x2de7bc\x2d13d1d375dc71.mount: Deactivated successfully. machine # [ 163.873659] systemd[1]: var-lib-kubelet-pods-658947de\x2d798a\x2d4627\x2da442\x2d01fff1c59e29-volumes-kubernetes.io\x7esecret-ca.mount: Deactivated successfully. machine # [ 163.874145] systemd[1]: var-lib-kubelet-pods-eb4a3576\x2d135a\x2d4de5\x2d8386\x2db0da1ef01e63-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dbsl7n.mount: Deactivated successfully. machine # [ 163.874628] systemd[1]: var-lib-kubelet-pods-658947de\x2d798a\x2d4627\x2da442\x2d01fff1c59e29-volumes-kubernetes.io\x7eprojected-niks3\x2dtoken.mount: Deactivated successfully. machine # [ 163.875146] systemd[1]: var-lib-kubelet-pods-658947de\x2d798a\x2d4627\x2da442\x2d01fff1c59e29-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dms2mt.mount: Deactivated successfully. machine # [ 163.875641] systemd[1]: var-lib-kubelet-pods-eb4a3576\x2d135a\x2d4de5\x2d8386\x2db0da1ef01e63-volumes-kubernetes.io\x7eprojected-niks3\x2dtoken.mount: Deactivated successfully. machine # [ 163.876161] systemd[1]: var-lib-kubelet-pods-eb4a3576\x2d135a\x2d4de5\x2d8386\x2db0da1ef01e63-volumes-kubernetes.io\x7esecret-oidc.mount: Deactivated successfully. machine # [ 163.876637] systemd[1]: var-lib-kubelet-pods-658947de\x2d798a\x2d4627\x2da442\x2d01fff1c59e29-volumes-kubernetes.io\x7esecret-oidc.mount: Deactivated successfully. machine # [ 163.877193] systemd[1]: var-lib-kubelet-pods-658947de\x2d798a\x2d4627\x2da442\x2d01fff1c59e29-volumes-kubernetes.io\x7esecret-tls.mount: Deactivated successfully. machine # [ 163.877642] systemd[1]: var-lib-kubelet-pods-eb4a3576\x2d135a\x2d4de5\x2d8386\x2db0da1ef01e63-volumes-kubernetes.io\x7esecret-ca.mount: Deactivated successfully. machine # [ 163.878172] systemd[1]: var-lib-kubelet-pods-eb4a3576\x2d135a\x2d4de5\x2d8386\x2db0da1ef01e63-volumes-kubernetes.io\x7esecret-tls.mount: Deactivated successfully. machine # [ 163.912526] systemd[1]: Started libcontainer container a9cb0bdf0309fc52a22b11d06edaad03d62941154cb728cd2f1f76c0218d6027. machine # [ 164.028538] systemd[1]: Started libcontainer container 4df2f49f563d6302870767ed7f976ef2090119204bc4fddafeba0f11ed9a58a4. machine # [ 164.044485] systemd[1]: Started libcontainer container 8eddec7e5d7fc060debb21b996be5e7c9b8aa8597d36f43d8060c6d9d62dcca8. machine # [ 164.114764] dhcpcd[6337]: vethe45bd26c: soliciting a DHCP lease machine # [ 164.362504] dhcpcd[6337]: veth698edbd7: soliciting an IPv6 router machine # [ 164.501165] dhcpcd[6337]: vethe45bd26c: soliciting an IPv6 router machine # [ 164.534950] k3s[873]: I0919 21:23:10.625192 873 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-9f9l9" podStartSLOduration=2.993706584 podStartE2EDuration="3.625150804s" totalImagesPullingTime="0s" totalInitContainerRuntime="631.44422ms" isStatefulPod=false podCreationTimestamp="2026-09-19 21:23:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 21:23:10.62321648 +0000 UTC m=+146.207065966" watchObservedRunningTime="2026-09-19 21:23:10.625150804 +0000 UTC m=+146.209000011" machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 164.752505] dhcpcd[6337]: vethfaa8ef55: soliciting an IPv6 router machine # [ 164.795893] k3s[873]: I0919 21:23:10.886466 873 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="658947de-798a-4627-a442-01fff1c59e29" path="/var/lib/kubelet/pods/658947de-798a-4627-a442-01fff1c59e29/volumes" machine # [ 164.799381] k3s[873]: I0919 21:23:10.887317 873 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eb4a3576-135a-4de5-8386-b0da1ef01e63" path="/var/lib/kubelet/pods/eb4a3576-135a-4de5-8386-b0da1ef01e63/volumes" machine # [ 164.853061] systemd[1]: var-lib-rancher-k3s-agent-containerd-tmpmounts-containerd\x2dmount2772598605.mount: Deactivated successfully. machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 166.859179] k3s[873]: I0919 21:23:12.949049 873 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-hnbx5" podStartSLOduration=5.389806367 podStartE2EDuration="5.949003137s" totalImagesPullingTime="0s" totalInitContainerRuntime="559.19677ms" isStatefulPod=false podCreationTimestamp="2026-09-19 21:23:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 21:23:10.647480737 +0000 UTC m=+146.231328547" watchObservedRunningTime="2026-09-19 21:23:12.949003137 +0000 UTC m=+148.532852065" machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 168.321049] dhcpcd[6337]: vethfaa8ef55: probing for an IPv4LL address machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 168.849357] dhcpcd[6337]: veth698edbd7: probing for an IPv4LL address machine # [ 169.116274] dhcpcd[6337]: vethe45bd26c: probing for an IPv4LL address machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 173.262520] dhcpcd[6337]: veth698edbd7: using IPv4LL address 169.254.194.119 machine # [ 173.263058] dhcpcd[6337]: veth698edbd7: adding route to 169.254.0.0/16 machine # [ 173.371161] dhcpcd[6337]: vethfaa8ef55: using IPv4LL address 169.254.136.81 machine # [ 173.371632] dhcpcd[6337]: vethfaa8ef55: adding route to 169.254.0.0/16 machine # [ 173.563592] dhcpcd[6337]: vethe45bd26c: using IPv4LL address 169.254.88.232 machine # [ 173.563933] dhcpcd[6337]: vethe45bd26c: adding route to 169.254.0.0/16 machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 176.366251] dhcpcd[6337]: veth698edbd7: no IPv6 Routers available machine # [ 176.504188] dhcpcd[6337]: vethe45bd26c: no IPv6 Routers available machine # [ 176.754442] dhcpcd[6337]: vethfaa8ef55: no IPv6 Routers available machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 234.794793] k3s[873]: I0919 21:24:20.883581 873 scope.go:122] "RemoveContainer" containerID="5bf4831cbfac88ce0fe1326052a91a4950f2e503cb310b35394b6f8bd55995a0" machine # [ 234.795866] k3s[873]: E0919 21:24:20.883978 873 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"helm\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=helm pod=helm-install-nix-grpc-farm-zcg5w_kube-system(6d037147-1d29-4e02-b2ab-2f08bc2a576c)\"" pod="kube-system/helm-install-nix-grpc-farm-zcg5w" podUID="6d037147-1d29-4e02-b2ab-2f08bc2a576c" machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 243.793328] k3s[873]: I0919 21:24:29.883500 873 scope.go:122] "RemoveContainer" containerID="5bf4831cbfac88ce0fe1326052a91a4950f2e503cb310b35394b6f8bd55995a0" machine # [ 243.855508] systemd[1]: Started libcontainer container b7cbd85f6018dc9bb360423473e25291d7eef45f0a03e58ca776b2366de6ba50. machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 244.902401] systemd[1]: cri-containerd-8eddec7e5d7fc060debb21b996be5e7c9b8aa8597d36f43d8060c6d9d62dcca8.scope: Deactivated successfully. machine # [ 244.905522] systemd[1]: cri-containerd-8eddec7e5d7fc060debb21b996be5e7c9b8aa8597d36f43d8060c6d9d62dcca8.scope: Consumed 82ms CPU time over 1min 20.857s wall clock time, 3.6M memory peak, 1.1M read from disk. machine # [ 244.909888] systemd[1]: cri-containerd-eac12c189fbcf16d1c00cefa7622d6a73f466369fd8159c6e33e59dcfe945471.scope: Deactivated successfully. machine # [ 244.913371] systemd[1]: cri-containerd-eac12c189fbcf16d1c00cefa7622d6a73f466369fd8159c6e33e59dcfe945471.scope: Consumed 619ms CPU time over 1min 22.161s wall clock time, 15.9M memory peak, 85.6K incoming IP traffic, 80.9K outgoing IP traffic. machine # [ 244.927986] systemd[1]: cri-containerd-4df2f49f563d6302870767ed7f976ef2090119204bc4fddafeba0f11ed9a58a4.scope: Deactivated successfully. machine # [ 244.940839] systemd[1]: cri-containerd-f723ffb0ffdf00b4ed03f3d5d0b8480677e418bc8325d2a2ba48a242f47f62e4.scope: Deactivated successfully. machine # [ 244.960286] systemd[1]: cri-containerd-958ccc32ed8090c615b6ac3092471d8722ac7c8f642f3c89930574729fc602df.scope: Deactivated successfully. machine # [ 244.962298] systemd[1]: cri-containerd-958ccc32ed8090c615b6ac3092471d8722ac7c8f642f3c89930574729fc602df.scope: Consumed 200ms CPU time over 1min 21.265s wall clock time, 47.5M memory peak, 33.5M read from disk, 32K written to disk. machine # [ 244.975285] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-8eddec7e5d7fc060debb21b996be5e7c9b8aa8597d36f43d8060c6d9d62dcca8-rootfs.mount: Deactivated successfully. machine # [ 244.985734] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-eac12c189fbcf16d1c00cefa7622d6a73f466369fd8159c6e33e59dcfe945471-rootfs.mount: Deactivated successfully. machine # [ 245.006504] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-4df2f49f563d6302870767ed7f976ef2090119204bc4fddafeba0f11ed9a58a4-rootfs.mount: Deactivated successfully. machine # [ 245.030053] systemd[1]: cri-containerd-a6ddb3ed35defde8888f9061c2aec09c363f13ee66293544bcabb52190898f32.scope: Deactivated successfully. machine # [ 245.052561] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-f723ffb0ffdf00b4ed03f3d5d0b8480677e418bc8325d2a2ba48a242f47f62e4-rootfs.mount: Deactivated successfully. machine # [ 245.066189] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-958ccc32ed8090c615b6ac3092471d8722ac7c8f642f3c89930574729fc602df-rootfs.mount: Deactivated successfully. machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 245.329559] cni0: port 7(vethe45bd26c) entered disabled state machine # [ 245.183570] dhcpcd[6337]: vethe45bd26c: carrier lost machine # [ 245.334188] vethe45bd26c (unregistering): left allmulticast mode machine # [ 245.335837] vethe45bd26c (unregistering): left promiscuous mode machine # [ 245.336956] cni0: port 7(vethe45bd26c) entered disabled state machine # [ 245.234993] dhcpcd[6337]: vethe45bd26c: deleting address fe80::ecc9:c2ff:fe96:6655 machine # [ 245.239265] dhcpcd[6337]: vethe45bd26c: deleting route to 169.254.0.0/16 machine # [ 245.271714] k3s[873]: I0919 21:24:31.362037 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/af333d0b-795c-4aba-88c4-d8deaf56c0a3-tls-lb\" (UniqueName: \"kubernetes.io/secret/af333d0b-795c-4aba-88c4-d8deaf56c0a3-tls-lb\") pod \"af333d0b-795c-4aba-88c4-d8deaf56c0a3\" (UID: \"af333d0b-795c-4aba-88c4-d8deaf56c0a3\") " machine # [ 245.278040] k3s[873]: I0919 21:24:31.367536 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/af333d0b-795c-4aba-88c4-d8deaf56c0a3-config\" (UniqueName: \"kubernetes.io/configmap/af333d0b-795c-4aba-88c4-d8deaf56c0a3-config\") pod \"af333d0b-795c-4aba-88c4-d8deaf56c0a3\" (UID: \"af333d0b-795c-4aba-88c4-d8deaf56c0a3\") " machine # [ 245.284198] k3s[873]: I0919 21:24:31.367577 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af333d0b-795c-4aba-88c4-d8deaf56c0a3-tmp\" (UniqueName: \"kubernetes.io/empty-dir/af333d0b-795c-4aba-88c4-d8deaf56c0a3-tmp\") pod \"af333d0b-795c-4aba-88c4-d8deaf56c0a3\" (UID: \"af333d0b-795c-4aba-88c4-d8deaf56c0a3\") " machine # [ 245.291870] k3s[873]: I0919 21:24:31.367608 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/af333d0b-795c-4aba-88c4-d8deaf56c0a3-tls-ca\" (UniqueName: \"kubernetes.io/secret/af333d0b-795c-4aba-88c4-d8deaf56c0a3-tls-ca\") pod \"af333d0b-795c-4aba-88c4-d8deaf56c0a3\" (UID: \"af333d0b-795c-4aba-88c4-d8deaf56c0a3\") " machine # [ 245.297980] k3s[873]: I0919 21:24:31.379022 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af333d0b-795c-4aba-88c4-d8deaf56c0a3-tls-lb" pod "af333d0b-795c-4aba-88c4-d8deaf56c0a3" (UID: "af333d0b-795c-4aba-88c4-d8deaf56c0a3"). InnerVolumeSpecName "tls-lb". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 245.305117] k3s[873]: I0919 21:24:31.383109 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af333d0b-795c-4aba-88c4-d8deaf56c0a3-tls-ca" pod "af333d0b-795c-4aba-88c4-d8deaf56c0a3" (UID: "af333d0b-795c-4aba-88c4-d8deaf56c0a3"). InnerVolumeSpecName "tls-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 245.311197] k3s[873]: I0919 21:24:31.388856 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af333d0b-795c-4aba-88c4-d8deaf56c0a3-tmp" pod "af333d0b-795c-4aba-88c4-d8deaf56c0a3" (UID: "af333d0b-795c-4aba-88c4-d8deaf56c0a3"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 245.318305] k3s[873]: I0919 21:24:31.389191 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af333d0b-795c-4aba-88c4-d8deaf56c0a3-config" pod "af333d0b-795c-4aba-88c4-d8deaf56c0a3" (UID: "af333d0b-795c-4aba-88c4-d8deaf56c0a3"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" machine # [ 245.378459] k3s[873]: I0919 21:24:31.469015 873 reconciler_common.go:299] "Volume detached for volume \"tls-lb\" (UniqueName: \"kubernetes.io/secret/af333d0b-795c-4aba-88c4-d8deaf56c0a3-tls-lb\") on node \"machine\" DevicePath \"\"" machine # [ 245.383995] k3s[873]: I0919 21:24:31.469472 873 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/af333d0b-795c-4aba-88c4-d8deaf56c0a3-config\") on node \"machine\" DevicePath \"\"" machine # [ 245.384366] k3s[873]: I0919 21:24:31.469498 873 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af333d0b-795c-4aba-88c4-d8deaf56c0a3-tmp\") on node \"machine\" DevicePath \"\"" machine # [ 245.384819] k3s[873]: I0919 21:24:31.469553 873 reconciler_common.go:299] "Volume detached for volume \"tls-ca\" (UniqueName: \"kubernetes.io/secret/af333d0b-795c-4aba-88c4-d8deaf56c0a3-tls-ca\") on node \"machine\" DevicePath \"\"" machine # [ 245.392764] dhcpcd[6337]: vethe45bd26c: removing interface machine # [ 245.619109] k3s[873]: E0919 21:24:31.709434 873 status.go:71] "Unhandled Error" err="apiserver received an error that is not an metav1.Status: &errors.errorString{s:\"failed to create typed patch object (farm/nix-grpc-farm-scheduler; /v1, Kind=Service): .spec.selector.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0}\"}: failed to create typed patch object (farm/nix-grpc-farm-scheduler; /v1, Kind=Service): .spec.selector.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0}" logger="UnhandledError" machine # [ 245.623726] k3s[873]: I0919 21:24:31.714331 873 alloc.go:329] "allocated clusterIPs" service="farm/nix-grpc-farm" clusterIPs={"IPv4":"10.43.37.210"} machine # [ 245.628737] k3s[873]: E0919 21:24:31.719315 873 repairip.go:372] "Unhandled Error" err="the ClusterIP [IPv4]: 10.43.37.210 for Service farm/nix-grpc-farm is not allocated; repairing" logger="UnhandledError" machine # [ 245.633583] k3s[873]: E0919 21:24:31.724183 873 status.go:71] "Unhandled Error" err=< machine # [ 245.633864] k3s[873]: apiserver received an error that is not an metav1.Status: &errors.errorString{s:"failed to create typed patch object (farm/nix-grpc-farm-scheduler; apps/v1, Kind=Deployment): errors:\n .spec.selector.matchLabels.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0}\n .spec.template.metadata.labels.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0}"}: failed to create typed patch object (farm/nix-grpc-farm-scheduler; apps/v1, Kind=Deployment): errors: machine # [ 245.634734] k3s[873]: .spec.selector.matchLabels.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0} machine # [ 245.635595] k3s[873]: .spec.template.metadata.labels.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0} machine # [ 245.639077] k3s[873]: > logger="UnhandledError" machine # [ 245.660724] systemd[1]: cri-containerd-b7cbd85f6018dc9bb360423473e25291d7eef45f0a03e58ca776b2366de6ba50.scope: Deactivated successfully. machine # [ 245.661630] systemd[1]: cri-containerd-b7cbd85f6018dc9bb360423473e25291d7eef45f0a03e58ca776b2366de6ba50.scope: Consumed 815ms CPU time over 1.805s wall clock time, 31M memory peak, 3.3M incoming IP traffic, 196.4K outgoing IP traffic. machine # [ 245.827623] k3s[873]: I0919 21:24:31.918141 873 scope.go:122] "RemoveContainer" containerID="eac12c189fbcf16d1c00cefa7622d6a73f466369fd8159c6e33e59dcfe945471" machine # [ 245.834487] systemd[1]: Removed slice libcontainer container kubepods-besteffort-podaf333d0b_795c_4aba_88c4_d8deaf56c0a3.slice. machine # [ 245.835138] systemd[1]: kubepods-besteffort-podaf333d0b_795c_4aba_88c4_d8deaf56c0a3.slice: Consumed 656ms CPU time over 1min 24.306s wall clock time, 16.2M memory peak, 85.6K incoming IP traffic, 80.9K outgoing IP traffic. machine # [ 245.842632] k3s[873]: I0919 21:24:31.933243 873 scope.go:122] "RemoveContainer" containerID="eac12c189fbcf16d1c00cefa7622d6a73f466369fd8159c6e33e59dcfe945471" machine # [ 245.843663] k3s[873]: E0919 21:24:31.934205 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"eac12c189fbcf16d1c00cefa7622d6a73f466369fd8159c6e33e59dcfe945471\": not found" containerID="eac12c189fbcf16d1c00cefa7622d6a73f466369fd8159c6e33e59dcfe945471" machine # [ 245.844569] k3s[873]: I0919 21:24:31.934254 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"eac12c189fbcf16d1c00cefa7622d6a73f466369fd8159c6e33e59dcfe945471"} err="failed to get container status \"eac12c189fbcf16d1c00cefa7622d6a73f466369fd8159c6e33e59dcfe945471\": rpc error: code = NotFound desc = an error occurred when try to find container \"eac12c189fbcf16d1c00cefa7622d6a73f466369fd8159c6e33e59dcfe945471\": not found" machine # [ 245.850475] k3s[873]: I0919 21:24:31.941077 873 scope.go:122] "RemoveContainer" containerID="5bf4831cbfac88ce0fe1326052a91a4950f2e503cb310b35394b6f8bd55995a0" machine # [ 245.851212] k3s[873]: I0919 21:24:31.941440 873 scope.go:122] "RemoveContainer" containerID="b7cbd85f6018dc9bb360423473e25291d7eef45f0a03e58ca776b2366de6ba50" machine # [ 245.851608] k3s[873]: E0919 21:24:31.941616 873 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"helm\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=helm pod=helm-install-nix-grpc-farm-zcg5w_kube-system(6d037147-1d29-4e02-b2ab-2f08bc2a576c)\"" pod="kube-system/helm-install-nix-grpc-farm-zcg5w" podUID="6d037147-1d29-4e02-b2ab-2f08bc2a576c" machine # [ 245.908272] systemd[1]: Created slice libcontainer container kubepods-besteffort-pode187fb7d_78e0_482b_b7fd_27eab0e7c263.slice. machine # [ 245.918818] systemd[1]: Created slice libcontainer container kubepods-besteffort-pod58fe4722_bc16_4cce_9a7d_93dd290f88a9.slice. machine # [ 245.927108] systemd[1]: Created slice libcontainer container kubepods-besteffort-pod58f09918_6c02_42ea_a7cd_7a2ebf47c912.slice. machine # [ 245.975368] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-b7cbd85f6018dc9bb360423473e25291d7eef45f0a03e58ca776b2366de6ba50-rootfs.mount: Deactivated successfully. machine # [ 245.975608] systemd[1]: var-lib-kubelet-pods-af333d0b\x2d795c\x2d4aba\x2d88c4\x2dd8deaf56c0a3-volume\x2dsubpaths-config-envoy-0.mount: Deactivated successfully. machine # [ 245.976207] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-a6ddb3ed35defde8888f9061c2aec09c363f13ee66293544bcabb52190898f32-rootfs.mount: Deactivated successfully. machine # [ 245.976617] systemd[1]: run-k3s-containerd-io.containerd.grpc.v1.cri-sandboxes-a6ddb3ed35defde8888f9061c2aec09c363f13ee66293544bcabb52190898f32-shm.mount: Deactivated successfully. machine # [ 245.977176] systemd[1]: run-netns-cni\x2db35e4bc0\x2d1e25\x2d7f59\x2d006c\x2d96bf6dbfeff3.mount: Deactivated successfully. machine # [ 245.977634] systemd[1]: var-lib-kubelet-pods-af333d0b\x2d795c\x2d4aba\x2d88c4\x2dd8deaf56c0a3-volumes-kubernetes.io\x7esecret-tls\x2dlb.mount: Deactivated successfully. machine # [ 245.978160] systemd[1]: var-lib-kubelet-pods-af333d0b\x2d795c\x2d4aba\x2d88c4\x2dd8deaf56c0a3-volumes-kubernetes.io\x7esecret-tls\x2dca.mount: Deactivated successfully. machine # [ 245.978657] systemd[1]: var-lib-kubelet-pods-af333d0b\x2d795c\x2d4aba\x2d88c4\x2dd8deaf56c0a3-volumes-kubernetes.io\x7eempty\x2ddir-tmp.mount: Deactivated successfully. machine # [ 245.986948] k3s[873]: I0919 21:24:32.077490 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls\" (UniqueName: \"kubernetes.io/secret/e187fb7d-78e0-482b-b7fd-27eab0e7c263-tls\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-hdwfw\" (UID: \"e187fb7d-78e0-482b-b7fd-27eab0e7c263\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-hdwfw" machine # [ 245.987742] k3s[873]: I0919 21:24:32.077538 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc\" (UniqueName: \"kubernetes.io/secret/e187fb7d-78e0-482b-b7fd-27eab0e7c263-oidc\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-hdwfw\" (UID: \"e187fb7d-78e0-482b-b7fd-27eab0e7c263\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-hdwfw" machine # [ 245.988493] k3s[873]: I0919 21:24:32.077569 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/58fe4722-bc16-4cce-9a7d-93dd290f88a9-config\") pod \"nix-grpc-farm-lb-688d8d547b-2gfc4\" (UID: \"58fe4722-bc16-4cce-9a7d-93dd290f88a9\") " pod="farm/nix-grpc-farm-lb-688d8d547b-2gfc4" machine # [ 245.989207] k3s[873]: I0919 21:24:32.077616 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nix\" (UniqueName: \"kubernetes.io/empty-dir/e187fb7d-78e0-482b-b7fd-27eab0e7c263-nix\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-hdwfw\" (UID: \"e187fb7d-78e0-482b-b7fd-27eab0e7c263\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-hdwfw" machine # [ 245.989649] k3s[873]: I0919 21:24:32.077644 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/58fe4722-bc16-4cce-9a7d-93dd290f88a9-tmp\") pod \"nix-grpc-farm-lb-688d8d547b-2gfc4\" (UID: \"58fe4722-bc16-4cce-9a7d-93dd290f88a9\") " pod="farm/nix-grpc-farm-lb-688d8d547b-2gfc4" machine # [ 245.990244] k3s[873]: I0919 21:24:32.077668 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-lb\" (UniqueName: \"kubernetes.io/secret/58fe4722-bc16-4cce-9a7d-93dd290f88a9-tls-lb\") pod \"nix-grpc-farm-lb-688d8d547b-2gfc4\" (UID: \"58fe4722-bc16-4cce-9a7d-93dd290f88a9\") " pod="farm/nix-grpc-farm-lb-688d8d547b-2gfc4" machine # [ 245.990697] k3s[873]: I0919 21:24:32.077714 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"niks3-token\" (UniqueName: \"kubernetes.io/projected/58f09918-6c02-42ea-a7cd-7a2ebf47c912-niks3-token\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-4tgln\" (UID: \"58f09918-6c02-42ea-a7cd-7a2ebf47c912\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-4tgln" machine # [ 245.991367] k3s[873]: I0919 21:24:32.077758 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"build\" (UniqueName: \"kubernetes.io/empty-dir/58f09918-6c02-42ea-a7cd-7a2ebf47c912-build\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-4tgln\" (UID: \"58f09918-6c02-42ea-a7cd-7a2ebf47c912\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-4tgln" machine # [ 245.991816] k3s[873]: I0919 21:24:32.077784 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls\" (UniqueName: \"kubernetes.io/secret/58f09918-6c02-42ea-a7cd-7a2ebf47c912-tls\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-4tgln\" (UID: \"58f09918-6c02-42ea-a7cd-7a2ebf47c912\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-4tgln" machine # [ 245.991984] k3s[873]: I0919 21:24:32.077813 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/58f09918-6c02-42ea-a7cd-7a2ebf47c912-ca\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-4tgln\" (UID: \"58f09918-6c02-42ea-a7cd-7a2ebf47c912\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-4tgln" machine # [ 245.992651] k3s[873]: I0919 21:24:32.077841 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gh2kl\" (UniqueName: \"kubernetes.io/projected/58f09918-6c02-42ea-a7cd-7a2ebf47c912-kube-api-access-gh2kl\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-4tgln\" (UID: \"58f09918-6c02-42ea-a7cd-7a2ebf47c912\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-4tgln" machine # [ 245.993936] k3s[873]: I0919 21:24:32.077869 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-ca\" (UniqueName: \"kubernetes.io/secret/58fe4722-bc16-4cce-9a7d-93dd290f88a9-tls-ca\") pod \"nix-grpc-farm-lb-688d8d547b-2gfc4\" (UID: \"58fe4722-bc16-4cce-9a7d-93dd290f88a9\") " pod="farm/nix-grpc-farm-lb-688d8d547b-2gfc4" machine # [ 245.997892] k3s[873]: I0919 21:24:32.077895 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nixconf\" (UniqueName: \"kubernetes.io/configmap/58f09918-6c02-42ea-a7cd-7a2ebf47c912-nixconf\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-4tgln\" (UID: \"58f09918-6c02-42ea-a7cd-7a2ebf47c912\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-4tgln" machine # [ 246.038986] k3s[873]: I0919 21:24:32.077934 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"build\" (UniqueName: \"kubernetes.io/empty-dir/e187fb7d-78e0-482b-b7fd-27eab0e7c263-build\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-hdwfw\" (UID: \"e187fb7d-78e0-482b-b7fd-27eab0e7c263\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-hdwfw" machine # [ 246.039569] k3s[873]: I0919 21:24:32.077963 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"niks3-token\" (UniqueName: \"kubernetes.io/projected/e187fb7d-78e0-482b-b7fd-27eab0e7c263-niks3-token\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-hdwfw\" (UID: \"e187fb7d-78e0-482b-b7fd-27eab0e7c263\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-hdwfw" machine # [ 246.040113] k3s[873]: I0919 21:24:32.077992 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nix\" (UniqueName: \"kubernetes.io/empty-dir/58f09918-6c02-42ea-a7cd-7a2ebf47c912-nix\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-4tgln\" (UID: \"58f09918-6c02-42ea-a7cd-7a2ebf47c912\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-4tgln" machine # [ 246.040581] k3s[873]: I0919 21:24:32.078015 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc\" (UniqueName: \"kubernetes.io/secret/58f09918-6c02-42ea-a7cd-7a2ebf47c912-oidc\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-4tgln\" (UID: \"58f09918-6c02-42ea-a7cd-7a2ebf47c912\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-4tgln" machine # [ 246.041124] k3s[873]: I0919 21:24:32.078041 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e187fb7d-78e0-482b-b7fd-27eab0e7c263-ca\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-hdwfw\" (UID: \"e187fb7d-78e0-482b-b7fd-27eab0e7c263\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-hdwfw" machine # [ 246.041791] k3s[873]: I0919 21:24:32.078067 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgk42\" (UniqueName: \"kubernetes.io/projected/e187fb7d-78e0-482b-b7fd-27eab0e7c263-kube-api-access-qgk42\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-hdwfw\" (UID: \"e187fb7d-78e0-482b-b7fd-27eab0e7c263\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-hdwfw" machine # [ 246.042451] k3s[873]: I0919 21:24:32.078116 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nixconf\" (UniqueName: \"kubernetes.io/configmap/e187fb7d-78e0-482b-b7fd-27eab0e7c263-nixconf\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-hdwfw\" (UID: \"e187fb7d-78e0-482b-b7fd-27eab0e7c263\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-hdwfw" machine # [ 246.242712] (udev-worker)[14697]: Network int[ 246.389903] cni0: port 3(vethe862d05c) entered blocking state machine # erface NamePolic[ 246.391831] cni0: port 3(vethe862d05c) entered disabled state machine # y= disabled on kernel command line. machine # [ 246.393995] vethe862d05c: entered allmulticast mode machine # [ 246.395777] vethe862d05c: entered promiscuous mode machine # [ 246.408986] cni0: port 3(vethe862d05c) entered blocking state machine # [ 246.410133] cni0: port 3(vethe862d05c) entered forwarding state machine # [ 246.277488] (udev-worker)[148[ 246.424512] cni0: port 5(vethe0f40db3) entered blocking state machine # 70]: Network interface NamePolicy= disabled on kernel command li[ 246.426239] cni0: port 5(vethe0f40db3) entered disabled state machine # ne. machine # [ 246.427695] vethe0f40db3: entered allmulticast mode machine # [ 246.429071] vethe0f40db3: entered promiscuous mode machine # [ 246.292964] dhcpcd[6337]: vethe862d05c: IAID 90:95:cc:f2 machine # [ 246.293697] dhcpcd[6337]: vethe8[ 246.440310] cni0: port 5(vethe0f40db3) entered blocking state machine # 62d05c: adding address fe80::600[ 246.442094] cni0: port 5(vethe0f40db3) entered forwarding state machine # d:90ff:fe95:ccf2 machine # [ 246.325032] dhcpcd[6337]: vethe0f40db3: IAID 25:2a:57:39 machine # [ 246.325550] dhcpcd[6337]: vethe0f40db3: adding address fe80::282d:25ff:fe2a:5739 machine # [ 246.374354] systemd[1]: Started libcontainer container 1267dc040a94a4fc9ed98b204dc6438c7dd8d2395a8cc14c272757b3dfb22150. machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 246.416428] systemd[1]: Started libcontainer container 9208df284202020f16ef557ff0b0a79ccfcfa74324981539f7edba6dc2d85f57. machine # [ 246.482378] systemd[1]: Started libcontainer container db8b1e86a79df4116d33f5d6e8be3a0006c7ff8863fee4ed8f05f9831cdbfde7. machine # [ 246.517515] systemd[1]: Started libcontainer container b4401b02b5bb37e7066c398f3c9ab50e0e33913a13e8d341bc7654df283b8a26. machine # [ 246.702440] cni0: port 7(vethaf65f9ab) entered blocking state machine # [ 246.704280] cni0: port 7(vethaf65f9ab) entered disabled state machine # [ 246.706075] vethaf65f9ab: entered allmulticast mode machine # [ 246.707251] vethaf65f9ab: entered promiscuous mode machine # [ 246.723480] cni0: port 7(vethaf65f9ab) entered blocking state machine # [ 246.724748] cni0: port 7(vethaf65f9ab) entered forwarding state machine # [ 246.605477] dhcpcd[6337]: vethaf65f9ab: IAID 69:b0:c7:77 machine # [ 246.605679] dhcpcd[6337]: vethaf65f9ab: adding address fe80::b4d8:69ff:feb0:c777 machine # [ 246.694164] systemd[1]: Started libcontainer container 8cb216c6dac205cb93634eeeda31a3445793e2c5640127981d5e6eaac270b5d1. machine # [ 246.797078] k3s[873]: I0919 21:24:32.887239 873 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="af333d0b-795c-4aba-88c4-d8deaf56c0a3" path="/var/lib/kubelet/pods/af333d0b-795c-4aba-88c4-d8deaf56c0a3/volumes" machine # [ 246.931582] systemd[1]: Started libcontainer container 549007675375dccc8addbbe0a60e06243cc8da6fecb3d29c53d1c0cfcd195605. machine # [ 247.032962] systemd[1]: cri-containerd-db8b1e86a79df4116d33f5d6e8be3a0006c7ff8863fee4ed8f05f9831cdbfde7.scope: Deactivated successfully. machine # [ 247.033790] systemd[1]: cri-containerd-db8b1e86a79df4116d33f5d6e8be3a0006c7ff8863fee4ed8f05f9831cdbfde7.scope: Consumed 483ms CPU time over 550ms wall clock time, 305.5M memory peak, 156.3M written to disk. machine # [ 247.102935] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-db8b1e86a79df4116d33f5d6e8be3a0006c7ff8863fee4ed8f05f9831cdbfde7-rootfs.mount: Deactivated successfully. machine # [ 247.106240] systemd[1]: cri-containerd-b4401b02b5bb37e7066c398f3c9ab50e0e33913a13e8d341bc7654df283b8a26.scope: Deactivated successfully. machine # [ 247.110264] systemd[1]: cri-containerd-b4401b02b5bb37e7066c398f3c9ab50e0e33913a13e8d341bc7654df283b8a26.scope: Consumed 496ms CPU time over 588ms wall clock time, 300.8M memory peak, 4K read from disk, 159.5M written to disk. machine # [ 247.190419] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-b4401b02b5bb37e7066c398f3c9ab50e0e33913a13e8d341bc7654df283b8a26-rootfs.mount: Deactivated successfully. machine # [ 247.535824] dhcpcd[6337]: vethe0f40db3: soliciting an IPv6 router machine # [ 247.546073] dhcpcd[6337]: vethe862d05c: soliciting a DHCP lease machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 247.670969] dhcpcd[6337]: vethe0f40db3: soliciting a DHCP lease machine # [ 247.818827] dhcpcd[6337]: vethaf65f9ab: soliciting a DHCP lease machine # [ 247.932419] k3s[873]: I0919 21:24:34.021966 873 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="farm/nix-grpc-farm-lb-688d8d547b-2gfc4" podStartSLOduration=3.021945638 podStartE2EDuration="3.021945638s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 21:24:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 21:24:34.020134514 +0000 UTC m=+229.603983721" watchObservedRunningTime="2026-09-19 21:24:34.021945638 +0000 UTC m=+229.605790096" machine # [ 248.020638] systemd[1]: Started libcontainer container 2c3ec07dbb77166c06c735c47592a510da9d58987c4cf62ed1a483042d531285. machine # [ 248.032679] systemd[1]: Started libcontainer container 6351c37c4fc718b0471e384515a99ebf82c2c17831bb85ed543a7b0eaa70c343. machine # [ 248.080939] systemd[1]: cri-containerd-06fc36d18e9c836e8e351d31c937b07ff7ef390f09bdd99fc5958c9004959311.scope: Deactivated successfully. machine # [ 248.082378] systemd[1]: cri-containerd-06fc36d18e9c836e8e351d31c937b07ff7ef390f09bdd99fc5958c9004959311.scope: Consumed 423ms CPU time over 1min 24.232s wall clock time, 15.4M memory peak, 7.9M read from disk, 35K incoming IP traffic, 31.1K outgoing IP traffic. machine # [ 248.149738] systemd[1]: cri-containerd-a9cb0bdf0309fc52a22b11d06edaad03d62941154cb728cd2f1f76c0218d6027.scope: Deactivated successfully. machine # [ 248.150359] systemd[1]: cri-containerd-a9cb0bdf0309fc52a22b11d06edaad03d62941154cb728cd2f1f76c0218d6027.scope: Consumed 428ms CPU time over 1min 24.237s wall clock time, 39.9M memory peak, 30.8M read from disk, 35.8K incoming IP traffic, 32.1K outgoing IP traffic. machine # [ 248.192665] systemd[1]: cri-containerd-29e32b42add63a0f56229212de8cb170fec22da80fdae00761885c321758fb72.scope: Deactivated successfully. machine # [ 248.213336] systemd[1]: Started libcontainer container d55c241872ed8542a8300861700b666022c77b0412c1c7cc4a30b5860448a006. machine # [ 248.239524] systemd[1]: Started libcontainer container 1fad4be34f8c04e4063583129e70de393080b173541dc8a3034254c4670f21b6. machine # [ 248.272889] systemd[1]: cri-containerd-49f8629f60d74e6179f08b7d7c4728215cf493d812b5f402421f00f75c52d452.scope: Deactivated successfully. machine # [ 248.373060] dhcpcd[6337]: vethaf65f9ab: soliciting an IPv6 router machine # [ 248.531980] cni0: port 4(vethfaa8ef55) entered disabled state machine # [ 248.388075] dhcpcd[6337]: vethfaa8ef55: carrier lost machine # [ 248.535938] vethfaa8ef55 (unregistering): left allmulticast mode machine # [ 248.537772] vethfaa8ef55 (unregistering): left promiscuous mode machine # [ 248.538968] cni0: port 4(vethfaa8ef55) entered disabled state machine # [ 248.411737] systemd[1]: Started libcontainer container 1d5df292fc04e81c9ee953630b9728f287a6c32223613ff42cb63afc62e2757a. machine # [ 248.437270] systemd[1]: Started libcontainer container 2b614dc6d0de68ea75e051e57ba8eb5ac1b4b37a3933d978abbe72caef4c2f2d. machine # [ 248.439785] dhcpcd[6337]: vethfaa8ef55: deleting address fe80::287a:6aff:fece:5e60 machine # [ 248.445159] dhcpcd[6337]: vethfaa8ef55: deleting route to 169.254.0.0/16 machine # [ 248.640675] cni0: port 6(veth698edbd7) entered disabled state machine # [ 248.644150] veth698edbd7 (unregistering): left allmulticast mode machine # [ 248.646107] veth698edbd7 (unregistering): left promiscuous mode machine # [ 248.648205] cni0: port 6(veth698edbd7) entered disabled state machine # [ 248.520107] k3s[873]: I0919 21:24:34.610693 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4f78420d-d22a-43a9-93c0-97f172907985-ca\" (UniqueName: \"kubernetes.io/secret/4f78420d-d22a-43a9-93c0-97f172907985-ca\") pod \"4f78420d-d22a-43a9-93c0-97f172907985\" (UID: \"4f78420d-d22a-43a9-93c0-97f172907985\") " machine # [ 248.520339] k3s[873]: I0919 21:24:34.610767 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f78420d-d22a-43a9-93c0-97f172907985-nix\" (UniqueName: \"kubernetes.io/empty-dir/4f78420d-d22a-43a9-93c0-97f172907985-nix\") pod \"4f78420d-d22a-43a9-93c0-97f172907985\" (UID: \"4f78420d-d22a-43a9-93c0-97f172907985\") " machine # [ 248.520922] k3s[873]: I0919 21:24:34.610805 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4f78420d-d22a-43a9-93c0-97f172907985-kube-api-access-tt5nz\" (UniqueName: \"kubernetes.io/projected/4f78420d-d22a-43a9-93c0-97f172907985-kube-api-access-tt5nz\") pod \"4f78420d-d22a-43a9-93c0-97f172907985\" (UID: \"4f78420d-d22a-43a9-93c0-97f172907985\") " machine # [ 248.522384] k3s[873]: I0919 21:24:34.611619 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4f78420d-d22a-43a9-93c0-97f172907985-oidc\" (UniqueName: \"kubernetes.io/secret/4f78420d-d22a-43a9-93c0-97f172907985-oidc\") pod \"4f78420d-d22a-43a9-93c0-97f172907985\" (UID: \"4f78420d-d22a-43a9-93c0-97f172907985\") " machine # [ 248.522913] k3s[873]: I0919 21:24:34.611880 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4f78420d-d22a-43a9-93c0-97f172907985-tls\" (UniqueName: \"kubernetes.io/secret/4f78420d-d22a-43a9-93c0-97f172907985-tls\") pod \"4f78420d-d22a-43a9-93c0-97f172907985\" (UID: \"4f78420d-d22a-43a9-93c0-97f172907985\") " machine # [ 248.522983] k3s[873]: I0919 21:24:34.611927 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4f78420d-d22a-43a9-93c0-97f172907985-nixconf\" (UniqueName: \"kubernetes.io/configmap/4f78420d-d22a-43a9-93c0-97f172907985-nixconf\") pod \"4f78420d-d22a-43a9-93c0-97f172907985\" (UID: \"4f78420d-d22a-43a9-93c0-97f172907985\") " machine # [ 248.523079] k3s[873]: I0919 21:24:34.611959 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4f78420d-d22a-43a9-93c0-97f172907985-niks3-token\" (UniqueName: \"kubernetes.io/projected/4f78420d-d22a-43a9-93c0-97f172907985-niks3-token\") pod \"4f78420d-d22a-43a9-93c0-97f172907985\" (UID: \"4f78420d-d22a-43a9-93c0-97f172907985\") " machine # [ 248.523147] k3s[873]: I0919 21:24:34.611987 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f78420d-d22a-43a9-93c0-97f172907985-build\" (UniqueName: \"kubernetes.io/empty-dir/4f78420d-d22a-43a9-93c0-97f172907985-build\") pod \"4f78420d-d22a-43a9-93c0-97f172907985\" (UID: \"4f78420d-d22a-43a9-93c0-97f172907985\") " machine # [ 248.523215] k3s[873]: I0919 21:24:34.612232 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f78420d-d22a-43a9-93c0-97f172907985-build" pod "4f78420d-d22a-43a9-93c0-97f172907985" (UID: "4f78420d-d22a-43a9-93c0-97f172907985"). InnerVolumeSpecName "build". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 248.524705] k3s[873]: I0919 21:24:34.615320 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f78420d-d22a-43a9-93c0-97f172907985-nixconf" pod "4f78420d-d22a-43a9-93c0-97f172907985" (UID: "4f78420d-d22a-43a9-93c0-97f172907985"). InnerVolumeSpecName "nixconf". PluginName "kubernetes.io/configmap", VolumeGIDValue "" machine # [ 248.585751] k3s[873]: I0919 21:24:34.676322 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f78420d-d22a-43a9-93c0-97f172907985-tls" pod "4f78420d-d22a-43a9-93c0-97f172907985" (UID: "4f78420d-d22a-43a9-93c0-97f172907985"). InnerVolumeSpecName "tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 248.585966] k3s[873]: I0919 21:24:34.676461 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f78420d-d22a-43a9-93c0-97f172907985-ca" pod "4f78420d-d22a-43a9-93c0-97f172907985" (UID: "4f78420d-d22a-43a9-93c0-97f172907985"). InnerVolumeSpecName "ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 248.586910] k3s[873]: I0919 21:24:34.676715 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f78420d-d22a-43a9-93c0-97f172907985-oidc" pod "4f78420d-d22a-43a9-93c0-97f172907985" (UID: "4f78420d-d22a-43a9-93c0-97f172907985"). InnerVolumeSpecName "oidc". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 248.588888] k3s[873]: I0919 21:24:34.676358 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f78420d-d22a-43a9-93c0-97f172907985-niks3-token" pod "4f78420d-d22a-43a9-93c0-97f172907985" (UID: "4f78420d-d22a-43a9-93c0-97f172907985"). InnerVolumeSpecName "niks3-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" machine # [ 248.589420] k3s[873]: I0919 21:24:34.678366 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f78420d-d22a-43a9-93c0-97f172907985-kube-api-access-tt5nz" pod "4f78420d-d22a-43a9-93c0-97f172907985" (UID: "4f78420d-d22a-43a9-93c0-97f172907985"). InnerVolumeSpecName "kube-api-access-tt5nz". PluginName "kubernetes.io/projected", VolumeGIDValue "" machine # [ 248.621982] k3s[873]: I0919 21:24:34.712503 873 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tt5nz\" (UniqueName: \"kubernetes.io/projected/4f78420d-d22a-43a9-93c0-97f172907985-kube-api-access-tt5nz\") on node \"machine\" DevicePath \"\"" machine # [ 248.622410] k3s[873]: I0919 21:24:34.712532 873 reconciler_common.go:299] "Volume detached for volume \"oidc\" (UniqueName: \"kubernetes.io/secret/4f78420d-d22a-43a9-93c0-97f172907985-oidc\") on node \"machine\" DevicePath \"\"" machine # [ 248.623164] k3s[873]: I0919 21:24:34.712545 873 reconciler_common.go:299] "Volume detached for volume \"tls\" (UniqueName: \"kubernetes.io/secret/4f78420d-d22a-43a9-93c0-97f172907985-tls\") on node \"machine\" DevicePath \"\"" machine # [ 248.623856] k3s[873]: I0919 21:24:34.713080 873 reconciler_common.go:299] "Volume detached for volume \"nixconf\" (UniqueName: \"kubernetes.io/configmap/4f78420d-d22a-43a9-93c0-97f172907985-nixconf\") on node \"machine\" DevicePath \"\"" machine # [ 248.625337] k3s[873]: I0919 21:24:34.713097 873 reconciler_common.go:299] "Volume detached for volume \"niks3-token\" (UniqueName: \"kubernetes.io/projected/4f78420d-d22a-43a9-93c0-97f172907985-niks3-token\") on node \"machine\" DevicePath \"\"" machine # [ 248.625884] k3s[873]: I0919 21:24:34.713109 873 reconciler_common.go:299] "Volume detached for volume \"build\" (UniqueName: \"kubernetes.io/empty-dir/4f78420d-d22a-43a9-93c0-97f172907985-build\") on node \"machine\" DevicePath \"\"" machine # [ 248.626533] k3s[873]: I0919 21:24:34.713121 873 reconciler_common.go:299] "Volume detached for volume \"ca\" (UniqueName: \"kubernetes.io/secret/4f78420d-d22a-43a9-93c0-97f172907985-ca\") on node \"machine\" DevicePath \"\"" machine # [ 248.661452] dhcpcd[6337]: dhcp_readbpf: unexpected event 0x0100 machine # [ 248.662624] dhcpcd[6337]: dhcp_readbpf: veth698edbd7: Network is down machine # [ 248.666563] dhcpcd[6337]: arp_read: unexpected event 0x0100 machine # [ 248.667613] dhcpcd[6337]: arp_read: veth698edbd7: Network is down machine # [ 248.678118] dhcpcd[6337]: vethfaa8ef55: removing interface machine # [ 248.705945] dhcpcd[6337]: veth698edbd7: removing interface machine # [ 248.707678] dhcpcd[6337]: veth698edbd7: deleting address fe80::503f:53ff:fe7d:abd5 machine # [ 248.709465] dhcpcd[6337]: veth698edbd7: deleting route to 169.254.0.0/16 machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 248.823865] k3s[873]: I0919 21:24:34.914415 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f78420d-d22a-43a9-93c0-97f172907985-nix" pod "4f78420d-d22a-43a9-93c0-97f172907985" (UID: "4f78420d-d22a-43a9-93c0-97f172907985"). InnerVolumeSpecName "nix". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 248.832102] k3s[873]: I0919 21:24:34.914652 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3504816e-e1c5-458d-9b6f-2017cb4702a9-niks3-token\" (UniqueName: \"kubernetes.io/projected/3504816e-e1c5-458d-9b6f-2017cb4702a9-niks3-token\") pod \"3504816e-e1c5-458d-9b6f-2017cb4702a9\" (UID: \"3504816e-e1c5-458d-9b6f-2017cb4702a9\") " machine # [ 248.832480] k3s[873]: I0919 21:24:34.914921 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3504816e-e1c5-458d-9b6f-2017cb4702a9-kube-api-access-jtpdj\" (UniqueName: \"kubernetes.io/projected/3504816e-e1c5-458d-9b6f-2017cb4702a9-kube-api-access-jtpdj\") pod \"3504816e-e1c5-458d-9b6f-2017cb4702a9\" (UID: \"3504816e-e1c5-458d-9b6f-2017cb4702a9\") " machine # [ 248.832955] k3s[873]: I0919 21:24:34.914963 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3504816e-e1c5-458d-9b6f-2017cb4702a9-nixconf\" (UniqueName: \"kubernetes.io/configmap/3504816e-e1c5-458d-9b6f-2017cb4702a9-nixconf\") pod \"3504816e-e1c5-458d-9b6f-2017cb4702a9\" (UID: \"3504816e-e1c5-458d-9b6f-2017cb4702a9\") " machine # [ 248.834222] k3s[873]: I0919 21:24:34.915008 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3504816e-e1c5-458d-9b6f-2017cb4702a9-tls\" (UniqueName: \"kubernetes.io/secret/3504816e-e1c5-458d-9b6f-2017cb4702a9-tls\") pod \"3504816e-e1c5-458d-9b6f-2017cb4702a9\" (UID: \"3504816e-e1c5-458d-9b6f-2017cb4702a9\") " machine # [ 248.834443] k3s[873]: I0919 21:24:34.915036 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3504816e-e1c5-458d-9b6f-2017cb4702a9-ca\" (UniqueName: \"kubernetes.io/secret/3504816e-e1c5-458d-9b6f-2017cb4702a9-ca\") pod \"3504816e-e1c5-458d-9b6f-2017cb4702a9\" (UID: \"3504816e-e1c5-458d-9b6f-2017cb4702a9\") " machine # [ 248.835895] k3s[873]: I0919 21:24:34.915072 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3504816e-e1c5-458d-9b6f-2017cb4702a9-build\" (UniqueName: \"kubernetes.io/empty-dir/3504816e-e1c5-458d-9b6f-2017cb4702a9-build\") pod \"3504816e-e1c5-458d-9b6f-2017cb4702a9\" (UID: \"3504816e-e1c5-458d-9b6f-2017cb4702a9\") " machine # [ 248.836526] k3s[873]: I0919 21:24:34.915103 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3504816e-e1c5-458d-9b6f-2017cb4702a9-oidc\" (UniqueName: \"kubernetes.io/secret/3504816e-e1c5-458d-9b6f-2017cb4702a9-oidc\") pod \"3504816e-e1c5-458d-9b6f-2017cb4702a9\" (UID: \"3504816e-e1c5-458d-9b6f-2017cb4702a9\") " machine # [ 248.836982] k3s[873]: I0919 21:24:34.915127 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3504816e-e1c5-458d-9b6f-2017cb4702a9-nix\" (UniqueName: \"kubernetes.io/empty-dir/3504816e-e1c5-458d-9b6f-2017cb4702a9-nix\") pod \"3504816e-e1c5-458d-9b6f-2017cb4702a9\" (UID: \"3504816e-e1c5-458d-9b6f-2017cb4702a9\") " machine # [ 248.837889] k3s[873]: I0919 21:24:34.915166 873 reconciler_common.go:299] "Volume detached for volume \"nix\" (UniqueName: \"kubernetes.io/empty-dir/4f78420d-d22a-43a9-93c0-97f172907985-nix\") on node \"machine\" DevicePath \"\"" machine # [ 248.838735] k3s[873]: I0919 21:24:34.918086 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3504816e-e1c5-458d-9b6f-2017cb4702a9-nixconf" pod "3504816e-e1c5-458d-9b6f-2017cb4702a9" (UID: "3504816e-e1c5-458d-9b6f-2017cb4702a9"). InnerVolumeSpecName "nixconf". PluginName "kubernetes.io/configmap", VolumeGIDValue "" machine # [ 248.839524] k3s[873]: I0919 21:24:34.918484 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3504816e-e1c5-458d-9b6f-2017cb4702a9-build" pod "3504816e-e1c5-458d-9b6f-2017cb4702a9" (UID: "3504816e-e1c5-458d-9b6f-2017cb4702a9"). InnerVolumeSpecName "build". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 248.840301] k3s[873]: I0919 21:24:34.923847 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3504816e-e1c5-458d-9b6f-2017cb4702a9-kube-api-access-jtpdj" pod "3504816e-e1c5-458d-9b6f-2017cb4702a9" (UID: "3504816e-e1c5-458d-9b6f-2017cb4702a9"). InnerVolumeSpecName "kube-api-access-jtpdj". PluginName "kubernetes.io/projected", VolumeGIDValue "" machine # [ 248.896186] k3s[873]: I0919 21:24:34.986691 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3504816e-e1c5-458d-9b6f-2017cb4702a9-tls" pod "3504816e-e1c5-458d-9b6f-2017cb4702a9" (UID: "3504816e-e1c5-458d-9b6f-2017cb4702a9"). InnerVolumeSpecName "tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 248.896643] k3s[873]: I0919 21:24:34.986848 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3504816e-e1c5-458d-9b6f-2017cb4702a9-ca" pod "3504816e-e1c5-458d-9b6f-2017cb4702a9" (UID: "3504816e-e1c5-458d-9b6f-2017cb4702a9"). InnerVolumeSpecName "ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 248.897358] k3s[873]: I0919 21:24:34.986834 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3504816e-e1c5-458d-9b6f-2017cb4702a9-niks3-token" pod "3504816e-e1c5-458d-9b6f-2017cb4702a9" (UID: "3504816e-e1c5-458d-9b6f-2017cb4702a9"). InnerVolumeSpecName "niks3-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" machine # [ 248.908607] k3s[873]: I0919 21:24:34.999212 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3504816e-e1c5-458d-9b6f-2017cb4702a9-oidc" pod "3504816e-e1c5-458d-9b6f-2017cb4702a9" (UID: "3504816e-e1c5-458d-9b6f-2017cb4702a9"). InnerVolumeSpecName "oidc". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 248.924797] k3s[873]: I0919 21:24:35.015353 873 reconciler_common.go:299] "Volume detached for volume \"niks3-token\" (UniqueName: \"kubernetes.io/projected/3504816e-e1c5-458d-9b6f-2017cb4702a9-niks3-token\") on node \"machine\" DevicePath \"\"" machine # [ 248.928372] k3s[873]: I0919 21:24:35.015381 873 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtpdj\" (UniqueName: \"kubernetes.io/projected/3504816e-e1c5-458d-9b6f-2017cb4702a9-kube-api-access-jtpdj\") on node \"machine\" DevicePath \"\"" machine # [ 248.932462] k3s[873]: I0919 21:24:35.015397 873 reconciler_common.go:299] "Volume detached for volume \"nixconf\" (UniqueName: \"kubernetes.io/configmap/3504816e-e1c5-458d-9b6f-2017cb4702a9-nixconf\") on node \"machine\" DevicePath \"\"" machine # [ 248.935658] k3s[873]: I0919 21:24:35.015409 873 reconciler_common.go:299] "Volume detached for volume \"tls\" (UniqueName: \"kubernetes.io/secret/3504816e-e1c5-458d-9b6f-2017cb4702a9-tls\") on node \"machine\" DevicePath \"\"" machine # [ 248.939117] k3s[873]: I0919 21:24:35.015423 873 reconciler_common.go:299] "Volume detached for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3504816e-e1c5-458d-9b6f-2017cb4702a9-ca\") on node \"machine\" DevicePath \"\"" machine # [ 248.942061] k3s[873]: I0919 21:24:35.015435 873 reconciler_common.go:299] "Volume detached for volume \"build\" (UniqueName: \"kubernetes.io/empty-dir/3504816e-e1c5-458d-9b6f-2017cb4702a9-build\") on node \"machine\" DevicePath \"\"" machine # [ 248.945671] k3s[873]: I0919 21:24:35.015447 873 reconciler_common.go:299] "Volume detached for volume \"oidc\" (UniqueName: \"kubernetes.io/secret/3504816e-e1c5-458d-9b6f-2017cb4702a9-oidc\") on node \"machine\" DevicePath \"\"" machine # [ 248.949054] k3s[873]: I0919 21:24:35.020960 873 scope.go:122] "RemoveContainer" containerID="8eddec7e5d7fc060debb21b996be5e7c9b8aa8597d36f43d8060c6d9d62dcca8" machine # [ 248.950885] k3s[873]: I0919 21:24:35.030949 873 scope.go:122] "RemoveContainer" containerID="a9cb0bdf0309fc52a22b11d06edaad03d62941154cb728cd2f1f76c0218d6027" machine # [ 248.952675] k3s[873]: I0919 21:24:35.034705 873 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-hdwfw" podStartSLOduration=3.510526449 podStartE2EDuration="4.034660941s" totalImagesPullingTime="0s" totalInitContainerRuntime="524.134492ms" isStatefulPod=false podCreationTimestamp="2026-09-19 21:24:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 21:24:35.033626452 +0000 UTC m=+230.617471190" watchObservedRunningTime="2026-09-19 21:24:35.034660941 +0000 UTC m=+230.618505679" machine # [ 248.959225] dhcpcd[6337]: vethe862d05c: soliciting an IPv6 router machine # [ 248.960115] k3s[873]: I0919 21:24:35.043246 873 scope.go:122] "RemoveContainer" containerID="958ccc32ed8090c615b6ac3092471d8722ac7c8f642f3c89930574729fc602df" machine # [ 248.964123] systemd[1]: Removed slice libcontainer container kubepods-besteffort-pod4f78420d_d22a_43a9_93c0_97f172907985.slice. machine # [ 248.966143] systemd[1]: kubepods-besteffort-pod4f78420d_d22a_43a9_93c0_97f172907985.slice: Consumed 1.201s CPU time over 1min 27.420s wall clock time, 323.9M memory peak, 7.9M read from disk, 283.9M written to disk, 35K incoming IP traffic, 31.1K outgoing IP traffic. machine # [ 248.970027] k3s[873]: I0919 21:24:35.060632 873 scope.go:122] "RemoveContainer" containerID="917a846d68e28b0cb235106fcc38e3b333714223d7b316501c86f5fbdc4dc865" machine # [ 248.984733] k3s[873]: I0919 21:24:35.072050 873 scope.go:122] "RemoveContainer" containerID="8eddec7e5d7fc060debb21b996be5e7c9b8aa8597d36f43d8060c6d9d62dcca8" machine # [ 248.986860] k3s[873]: E0919 21:24:35.074329 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"8eddec7e5d7fc060debb21b996be5e7c9b8aa8597d36f43d8060c6d9d62dcca8\": not found" containerID="8eddec7e5d7fc060debb21b996be5e7c9b8aa8597d36f43d8060c6d9d62dcca8" machine # [ 248.991195] k3s[873]: I0919 21:24:35.074360 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"8eddec7e5d7fc060debb21b996be5e7c9b8aa8597d36f43d8060c6d9d62dcca8"} err="failed to get container status \"8eddec7e5d7fc060debb21b996be5e7c9b8aa8597d36f43d8060c6d9d62dcca8\": rpc error: code = NotFound desc = an error occurred when try to find container \"8eddec7e5d7fc060debb21b996be5e7c9b8aa8597d36f43d8060c6d9d62dcca8\": not found" machine # [ 248.998679] k3s[873]: I0919 21:24:35.074385 873 scope.go:122] "RemoveContainer" containerID="a9cb0bdf0309fc52a22b11d06edaad03d62941154cb728cd2f1f76c0218d6027" machine # [ 249.000583] k3s[873]: I0919 21:24:35.074959 873 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-4tgln" podStartSLOduration=3.515663694 podStartE2EDuration="4.074943435s" totalImagesPullingTime="0s" totalInitContainerRuntime="559.279741ms" isStatefulPod=false podCreationTimestamp="2026-09-19 21:24:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 21:24:35.056809008 +0000 UTC m=+230.640653745" watchObservedRunningTime="2026-09-19 21:24:35.074943435 +0000 UTC m=+230.658787893" machine # [ 249.010927] k3s[873]: E0919 21:24:35.076281 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"a9cb0bdf0309fc52a22b11d06edaad03d62941154cb728cd2f1f76c0218d6027\": not found" containerID="a9cb0bdf0309fc52a22b11d06edaad03d62941154cb728cd2f1f76c0218d6027" machine # [ 249.016768] k3s[873]: I0919 21:24:35.076316 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"a9cb0bdf0309fc52a22b11d06edaad03d62941154cb728cd2f1f76c0218d6027"} err="failed to get container status \"a9cb0bdf0309fc52a22b11d06edaad03d62941154cb728cd2f1f76c0218d6027\": rpc error: code = NotFound desc = an error occurred when try to find container \"a9cb0bdf0309fc52a22b11d06edaad03d62941154cb728cd2f1f76c0218d6027\": not found" machine # [ 249.024785] k3s[873]: I0919 21:24:35.076346 873 scope.go:122] "RemoveContainer" containerID="958ccc32ed8090c615b6ac3092471d8722ac7c8f642f3c89930574729fc602df" machine # [ 249.025123] k3s[873]: E0919 21:24:35.076979 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"958ccc32ed8090c615b6ac3092471d8722ac7c8f642f3c89930574729fc602df\": not found" containerID="958ccc32ed8090c615b6ac3092471d8722ac7c8f642f3c89930574729fc602df" machine # [ 249.025949] k3s[873]: I0919 21:24:35.076997 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"958ccc32ed8090c615b6ac3092471d8722ac7c8f642f3c89930574729fc602df"} err="failed to get container status \"958ccc32ed8090c615b6ac3092471d8722ac7c8f642f3c89930574729fc602df\": rpc error: code = NotFound desc = an error occurred when try to find container \"958ccc32ed8090c615b6ac3092471d8722ac7c8f642f3c89930574729fc602df\": not found" machine # [ 249.026579] k3s[873]: I0919 21:24:35.077016 873 scope.go:122] "RemoveContainer" containerID="917a846d68e28b0cb235106fcc38e3b333714223d7b316501c86f5fbdc4dc865" machine # [ 249.028136] k3s[873]: E0919 21:24:35.077507 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"917a846d68e28b0cb235106fcc38e3b333714223d7b316501c86f5fbdc4dc865\": not found" containerID="917a846d68e28b0cb235106fcc38e3b333714223d7b316501c86f5fbdc4dc865" machine # [ 249.029061] k3s[873]: I0919 21:24:35.077535 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"917a846d68e28b0cb235106fcc38e3b333714223d7b316501c86f5fbdc4dc865"} err="failed to get container status \"917a846d68e28b0cb235106fcc38e3b333714223d7b316501c86f5fbdc4dc865\": rpc error: code = NotFound desc = an error occurred when try to find container \"917a846d68e28b0cb235106fcc38e3b333714223d7b316501c86f5fbdc4dc865\": not found" machine # [ 249.030114] k3s[873]: I0919 21:24:35.077558 873 scope.go:122] "RemoveContainer" containerID="4df2f49f563d6302870767ed7f976ef2090119204bc4fddafeba0f11ed9a58a4" machine # [ 249.030950] k3s[873]: I0919 21:24:35.091031 873 scope.go:122] "RemoveContainer" containerID="06fc36d18e9c836e8e351d31c937b07ff7ef390f09bdd99fc5958c9004959311" machine # [ 249.032144] k3s[873]: I0919 21:24:35.098962 873 scope.go:122] "RemoveContainer" containerID="f723ffb0ffdf00b4ed03f3d5d0b8480677e418bc8325d2a2ba48a242f47f62e4" machine # [ 249.032893] systemd[1]: var-lib-kubelet-pods-3504816e\x2de1c5\x2d458d\x2d9b6f\x2d2017cb4702a9-volume\x2dsubpaths-nix-gc-0.mount: Deactivated successfully. machine # [ 249.033812] k3s[873]: I0919 21:24:35.106879 873 scope.go:122] "RemoveContainer" containerID="5dd3e9ea82cae64396f53e7727f5f3de0ceb812dc70007717319c395042c7cee" machine # [ 249.033991] k3s[873]: I0919 21:24:35.113937 873 scope.go:122] "RemoveContainer" containerID="4df2f49f563d6302870767ed7f976ef2090119204bc4fddafeba0f11ed9a58a4" machine # [ 249.035335] k3s[873]: E0919 21:24:35.114412 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"4df2f49f563d6302870767ed7f976ef2090119204bc4fddafeba0f11ed9a58a4\": not found" containerID="4df2f49f563d6302870767ed7f976ef2090119204bc4fddafeba0f11ed9a58a4" machine # [ 249.036505] k3s[873]: I0919 21:24:35.114447 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"4df2f49f563d6302870767ed7f976ef2090119204bc4fddafeba0f11ed9a58a4"} err="failed to get container status \"4df2f49f563d6302870767ed7f976ef2090119204bc4fddafeba0f11ed9a58a4\": rpc error: code = NotFound desc = an error occurred when try to find container \"4df2f49f563d6302870767ed7f976ef2090119204bc4fddafeba0f11ed9a58a4\": not found" machine # [ 249.036978] k3s[873]: I0919 21:24:35.114469 873 scope.go:122] "RemoveContainer" containerID="06fc36d18e9c836e8e351d31c937b07ff7ef390f09bdd99fc5958c9004959311" machine # [ 249.038253] k3s[873]: E0919 21:24:35.117766 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"06fc36d18e9c836e8e351d31c937b07ff7ef390f09bdd99fc5958c9004959311\": not found" containerID="06fc36d18e9c836e8e351d31c937b07ff7ef390f09bdd99fc5958c9004959311" machine # [ 249.039135] k3s[873]: I0919 21:24:35.117812 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"06fc36d18e9c836e8e351d31c937b07ff7ef390f09bdd99fc5958c9004959311"} err="failed to get container status \"06fc36d18e9c836e8e351d31c937b07ff7ef390f09bdd99fc5958c9004959311\": rpc error: code = NotFound desc = an error occurred when try to find container \"06fc36d18e9c836e8e351d31c937b07ff7ef390f09bdd99fc5958c9004959311\": not found" machine # [ 249.040280] k3s[873]: I0919 21:24:35.117843 873 scope.go:122] "RemoveContainer" containerID="f723ffb0ffdf00b4ed03f3d5d0b8480677e418bc8325d2a2ba48a242f47f62e4" machine # [ 249.040944] k3s[873]: E0919 21:24:35.119233 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"f723ffb0ffdf00b4ed03f3d5d0b8480677e418bc8325d2a2ba48a242f47f62e4\": not found" containerID="f723ffb0ffdf00b4ed03f3d5d0b8480677e418bc8325d2a2ba48a242f47f62e4" machine # [ 249.102776] k3s[873]: I0919 21:24:35.119261 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"f723ffb0ffdf00b4ed03f3d5d0b8480677e418bc8325d2a2ba48a242f47f62e4"} err="failed to get container status \"f723ffb0ffdf00b4ed03f3d5d0b8480677e418bc8325d2a2ba48a242f47f62e4\": rpc error: code = NotFound desc = an error occurred when try to find container \"f723ffb0ffdf00b4ed03f3d5d0b8480677e418bc8325d2a2ba48a242f47f62e4\": not found" machine # [ 249.104085] k3s[873]: I0919 21:24:35.119284 873 scope.go:122] "RemoveContainer" containerID="5dd3e9ea82cae64396f53e7727f5f3de0ceb812dc70007717319c395042c7cee" machine # [ 249.104737] k3s[873]: E0919 21:24:35.120216 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"5dd3e9ea82cae64396f53e7727f5f3de0ceb812dc70007717319c395042c7cee\": not found" containerID="5dd3e9ea82cae64396f53e7727f5f3de0ceb812dc70007717319c395042c7cee" machine # [ 249.105550] k3s[873]: I0919 21:24:35.120236 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"5dd3e9ea82cae64396f53e7727f5f3de0ceb812dc70007717319c395042c7cee"} err="failed to get container status \"5dd3e9ea82cae64396f53e7727f5f3de0ceb812dc70007717319c395042c7cee\": rpc error: code = NotFound desc = an error occurred when try to find container \"5dd3e9ea82cae64396f53e7727f5f3de0ceb812dc70007717319c395042c7cee\": not found" machine # [ 249.106710] systemd[1]: var-lib-kubelet-pods-4f78420d\x2dd22a\x2d43a9\x2d93c0\x2d97f172907985-volume\x2dsubpaths-nix-gc-0.mount: Deactivated successfully. machine # [ 249.108132] k3s[873]: I0919 21:24:35.141353 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3504816e-e1c5-458d-9b6f-2017cb4702a9-nix" pod "3504816e-e1c5-458d-9b6f-2017cb4702a9" (UID: "3504816e-e1c5-458d-9b6f-2017cb4702a9"). InnerVolumeSpecName "nix". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 249.108918] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-a9cb0bdf0309fc52a22b11d06edaad03d62941154cb728cd2f1f76c0218d6027-rootfs.mount: Deactivated successfully. machine # [ 249.110460] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-06fc36d18e9c836e8e351d31c937b07ff7ef390f09bdd99fc5958c9004959311-rootfs.mount: Deactivated successfully. machine # [ 249.111059] systemd[1]: var-lib-kubelet-pods-3504816e\x2de1c5\x2d458d\x2d9b6f\x2d2017cb4702a9-volume\x2dsubpaths-nix-nix\x2dgrpc\x2ddaemon-0.mount: Deactivated successfully. machine # [ 249.111943] systemd[1]: var-lib-kubelet-pods-4f78420d\x2dd22a\x2d43a9\x2d93c0\x2d97f172907985-volume\x2dsubpaths-nix-nix\x2dgrpc\x2ddaemon-0.mount: Deactivated successfully. machine # [ 249.113366] systemd[1]: var-lib-kubelet-pods-3504816e\x2de1c5\x2d458d\x2d9b6f\x2d2017cb4702a9-volume\x2dsubpaths-nix-nix\x2ddaemon-0.mount: Deactivated successfully. machine # [ 249.114341] systemd[1]: var-lib-kubelet-pods-4f78420d\x2dd22a\x2d43a9\x2d93c0\x2d97f172907985-volume\x2dsubpaths-nix-nix\x2ddaemon-0.mount: Deactivated successfully. machine # [ 249.115409] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-49f8629f60d74e6179f08b7d7c4728215cf493d812b5f402421f00f75c52d452-rootfs.mount: Deactivated successfully. machine # [ 249.116633] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-29e32b42add63a0f56229212de8cb170fec22da80fdae00761885c321758fb72-rootfs.mount: Deactivated successfully. machine # [ 249.117646] systemd[1]: run-k3s-containerd-io.containerd.grpc.v1.cri-sandboxes-49f8629f60d74e6179f08b7d7c4728215cf493d812b5f402421f00f75c52d452-shm.mount: Deactivated successfully. machine # [ 249.117864] systemd[1]: run-k3s-containerd-io.containerd.grpc.v1.cri-sandboxes-29e32b42add63a0f56229212de8cb170fec22da80fdae00761885c321758fb72-shm.mount: Deactivated successfully. machine # [ 249.118867] systemd[1]: run-netns-cni\x2de50a5952\x2d35e9\x2d19d0\x2d5f34\x2d97c20dbc96dc.mount: Deactivated successfully. machine # [ 249.119576] systemd[1]: run-netns-cni\x2dd9c88910\x2dcdad\x2dcde8\x2ddfba\x2db7af587d4ce5.mount: Deactivated successfully. machine # [ 249.119825] systemd[1]: var-lib-kubelet-pods-3504816e\x2de1c5\x2d458d\x2d9b6f\x2d2017cb4702a9-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2djtpdj.mount: Deactivated successfully. machine # [ 249.120812] systemd[1]: var-lib-kubelet-pods-4f78420d\x2dd22a\x2d43a9\x2d93c0\x2d97f172907985-volumes-kubernetes.io\x7eprojected-niks3\x2dtoken.mount: Deactivated successfully. machine # [ 249.121719] systemd[1]: var-lib-kubelet-pods-4f78420d\x2dd22a\x2d43a9\x2d93c0\x2d97f172907985-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dtt5nz.mount: Deactivated successfully. machine # [ 249.122361] systemd[1]: var-lib-kubelet-pods-3504816e\x2de1c5\x2d458d\x2d9b6f\x2d2017cb4702a9-volumes-kubernetes.io\x7eprojected-niks3\x2dtoken.mount: Deactivated successfully. machine # [ 249.123025] systemd[1]: var-lib-kubelet-pods-3504816e\x2de1c5\x2d458d\x2d9b6f\x2d2017cb4702a9-volumes-kubernetes.io\x7esecret-oidc.mount: Deactivated successfully. machine # [ 249.124131] systemd[1]: var-lib-kubelet-pods-3504816e\x2de1c5\x2d458d\x2d9b6f\x2d2017cb4702a9-volumes-kubernetes.io\x7esecret-tls.mount: Deactivated successfully. machine # [ 249.125499] systemd[1]: var-lib-kubelet-pods-4f78420d\x2dd22a\x2d43a9\x2d93c0\x2d97f172907985-volumes-kubernetes.io\x7esecret-ca.mount: Deactivated successfully. machine # [ 249.126544] systemd[1]: var-lib-kubelet-pods-4f78420d\x2dd22a\x2d43a9\x2d93c0\x2d97f172907985-volumes-kubernetes.io\x7esecret-tls.mount: Deactivated successfully. machine # [ 249.127533] k3s[873]: I0919 21:24:35.217062 873 reconciler_common.go:299] "Volume detached for volume \"nix\" (UniqueName: \"kubernetes.io/empty-dir/3504816e-e1c5-458d-9b6f-2017cb4702a9-nix\") on node \"machine\" DevicePath \"\"" machine # [ 249.186805] systemd[1]: var-lib-kubelet-pods-4f78420d\x2dd22a\x2d43a9\x2d93c0\x2d97f172907985-volumes-kubernetes.io\x7esecret-oidc.mount: Deactivated successfully. machine # [ 249.187654] systemd[1]: var-lib-kubelet-pods-3504816e\x2de1c5\x2d458d\x2d9b6f\x2d2017cb4702a9-volumes-kubernetes.io\x7esecret-ca.mount: Deactivated successfully. machine # [ 249.238090] systemd[1]: Removed slice libcontainer container kubepods-besteffort-pod3504816e_e1c5_458d_9b6f_2017cb4702a9.slice. machine # [ 249.238377] systemd[1]: kubepods-besteffort-pod3504816e_e1c5_458d_9b6f_2017cb4702a9.slice: Consumed 1.218s CPU time over 1min 27.703s wall clock time, 306.9M memory peak, 65.5M read from disk, 283.9M written to disk, 35.8K incoming IP traffic, 32.1K outgoing IP traffic. machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 250.800166] k3s[873]: I0919 21:24:36.889933 873 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3504816e-e1c5-458d-9b6f-2017cb4702a9" path="/var/lib/kubelet/pods/3504816e-e1c5-458d-9b6f-2017cb4702a9/volumes" machine # [ 250.800981] k3s[873]: I0919 21:24:36.891522 873 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4f78420d-d22a-43a9-93c0-97f172907985" path="/var/lib/kubelet/pods/4f78420d-d22a-43a9-93c0-97f172907985/volumes" machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 252.545661] dhcpcd[6337]: vethe862d05c: probing for an IPv4LL address machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 252.671332] dhcpcd[6337]: vethe0f40db3: probing for an IPv4LL address machine # [ 252.819888] dhcpcd[6337]: vethaf65f9ab: probing for an IPv4LL address machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 256.963202] dhcpcd[6337]: vethe0f40db3: using IPv4LL address 169.254.115.228 machine # [ 256.963482] dhcpcd[6337]: vethe0f40db3: adding route to 169.254.0.0/16 machine # [ 257.304956] dhcpcd[6337]: vethaf65f9ab: using IPv4LL address 169.254.98.251 machine # [ 257.305772] dhcpcd[6337]: vethaf65f9ab: adding route to 169.254.0.0/16 machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 257.990878] dhcpcd[6337]: vethe862d05c: using IPv4LL address 169.254.32.62 machine # [ 257.992467] dhcpcd[6337]: vethe862d05c: adding route to 169.254.0.0/16 machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 259.539278] dhcpcd[6337]: vethe0f40db3: no IPv6 Routers available machine # [ 260.375506] dhcpcd[6337]: vethaf65f9ab: no IPv6 Routers available machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 260.939399] dhcpcd[6337]: vethe862d05c: no IPv6 Routers available machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 312.160917] postgres[771]: [771] LOG: checkpoint starting: time machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 319.793797] k3s[873]: I0919 21:25:45.883583 873 scope.go:122] "RemoveContainer" containerID="b7cbd85f6018dc9bb360423473e25291d7eef45f0a03e58ca776b2366de6ba50" machine # [ 319.798849] k3s[873]: E0919 21:25:45.884061 873 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"helm\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=helm pod=helm-install-nix-grpc-farm-zcg5w_kube-system(6d037147-1d29-4e02-b2ab-2f08bc2a576c)\"" pod="kube-system/helm-install-nix-grpc-farm-zcg5w" podUID="6d037147-1d29-4e02-b2ab-2f08bc2a576c" machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 406.795783] k3s[873]: I0919 21:27:12.885371 873 scope.go:122] "RemoveContainer" containerID="b7cbd85f6018dc9bb360423473e25291d7eef45f0a03e58ca776b2366de6ba50" machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 406.863880] systemd[1]: Started libcontainer container 007d47a685973e2a9413ce9e05b2306fe3096d78ac9d0114cad13819d726af6d. machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 408.712178] systemd[1]: cri-containerd-549007675375dccc8addbbe0a60e06243cc8da6fecb3d29c53d1c0cfcd195605.scope: Deactivated successfully. machine # [ 408.714139] systemd[1]: cri-containerd-549007675375dccc8addbbe0a60e06243cc8da6fecb3d29c53d1c0cfcd195605.scope: Consumed 1.119s CPU time over 2min 41.780s wall clock time, 15.9M memory peak, 152.9K incoming IP traffic, 148.7K outgoing IP traffic. machine # [ 408.725241] systemd[1]: cri-containerd-2b614dc6d0de68ea75e051e57ba8eb5ac1b4b37a3933d978abbe72caef4c2f2d.scope: Deactivated successfully. machine # [ 408.725649] systemd[1]: cri-containerd-1d5df292fc04e81c9ee953630b9728f287a6c32223613ff42cb63afc62e2757a.scope: Deactivated successfully. machine # [ 408.761278] systemd[1]: cri-containerd-6351c37c4fc718b0471e384515a99ebf82c2c17831bb85ed543a7b0eaa70c343.scope: Deactivated successfully. machine # [ 408.764506] systemd[1]: cri-containerd-2c3ec07dbb77166c06c735c47592a510da9d58987c4cf62ed1a483042d531285.scope: Deactivated successfully. machine # [ 408.764989] systemd[1]: cri-containerd-2c3ec07dbb77166c06c735c47592a510da9d58987c4cf62ed1a483042d531285.scope: Consumed 358ms CPU time over 2min 40.741s wall clock time, 33.1M memory peak, 19.1M read from disk, 64K written to disk. machine # [ 408.780957] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-2b614dc6d0de68ea75e051e57ba8eb5ac1b4b37a3933d978abbe72caef4c2f2d-rootfs.mount: Deactivated successfully. machine # [ 408.788676] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-549007675375dccc8addbbe0a60e06243cc8da6fecb3d29c53d1c0cfcd195605-rootfs.mount: Deactivated successfully. machine # [ 408.791749] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-1d5df292fc04e81c9ee953630b9728f287a6c32223613ff42cb63afc62e2757a-rootfs.mount: Deactivated successfully. machine # [ 408.817073] systemd[1]: cri-containerd-8cb216c6dac205cb93634eeeda31a3445793e2c5640127981d5e6eaac270b5d1.scope: Deactivated successfully. machine # [ 408.843951] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-6351c37c4fc718b0471e384515a99ebf82c2c17831bb85ed543a7b0eaa70c343-rootfs.mount: Deactivated successfully. machine # [ 408.847312] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-2c3ec07dbb77166c06c735c47592a510da9d58987c4cf62ed1a483042d531285-rootfs.mount: Deactivated successfully. machine # [ 409.115015] cni0: port 7(vethaf65f9ab) entered disabled state machine # [ 408.971386] dhcpcd[6337]: vethaf65f9ab: carrier lost machine # [ 409.119389] vethaf65f9ab (unregistering): left allmulticast mode machine # [ 409.121355] vethaf65f9ab (unregistering): left promiscuous mode machine # [ 409.122472] cni0: port 7(vethaf65f9ab) entered disabled state machine # [ 409.037799] dhcpcd[6337]: vethaf65f9ab: deleting address fe80::b4d8:69ff:feb0:c777 machine # [ 409.043252] dhcpcd[6337]: vethaf65f9ab: deleting route to 169.254.0.0/16 machine # [ 409.093915] k3s[873]: I0919 21:27:15.183890 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/58fe4722-bc16-4cce-9a7d-93dd290f88a9-tls-ca\" (UniqueName: \"kubernetes.io/secret/58fe4722-bc16-4cce-9a7d-93dd290f88a9-tls-ca\") pod \"58fe4722-bc16-4cce-9a7d-93dd290f88a9\" (UID: \"58fe4722-bc16-4cce-9a7d-93dd290f88a9\") " machine # [ 409.101890] k3s[873]: I0919 21:27:15.183948 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/58fe4722-bc16-4cce-9a7d-93dd290f88a9-tmp\" (UniqueName: \"kubernetes.io/empty-dir/58fe4722-bc16-4cce-9a7d-93dd290f88a9-tmp\") pod \"58fe4722-bc16-4cce-9a7d-93dd290f88a9\" (UID: \"58fe4722-bc16-4cce-9a7d-93dd290f88a9\") " machine # [ 409.110199] k3s[873]: I0919 21:27:15.183982 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/58fe4722-bc16-4cce-9a7d-93dd290f88a9-config\" (UniqueName: \"kubernetes.io/configmap/58fe4722-bc16-4cce-9a7d-93dd290f88a9-config\") pod \"58fe4722-bc16-4cce-9a7d-93dd290f88a9\" (UID: \"58fe4722-bc16-4cce-9a7d-93dd290f88a9\") " machine # [ 409.111330] k3s[873]: I0919 21:27:15.184009 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/58fe4722-bc16-4cce-9a7d-93dd290f88a9-tls-lb\" (UniqueName: \"kubernetes.io/secret/58fe4722-bc16-4cce-9a7d-93dd290f88a9-tls-lb\") pod \"58fe4722-bc16-4cce-9a7d-93dd290f88a9\" (UID: \"58fe4722-bc16-4cce-9a7d-93dd290f88a9\") " machine # [ 409.112046] k3s[873]: I0919 21:27:15.193173 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58fe4722-bc16-4cce-9a7d-93dd290f88a9-tmp" pod "58fe4722-bc16-4cce-9a7d-93dd290f88a9" (UID: "58fe4722-bc16-4cce-9a7d-93dd290f88a9"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 409.112630] k3s[873]: I0919 21:27:15.199976 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58fe4722-bc16-4cce-9a7d-93dd290f88a9-tls-lb" pod "58fe4722-bc16-4cce-9a7d-93dd290f88a9" (UID: "58fe4722-bc16-4cce-9a7d-93dd290f88a9"). InnerVolumeSpecName "tls-lb". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 409.130583] k3s[873]: I0919 21:27:15.220951 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58fe4722-bc16-4cce-9a7d-93dd290f88a9-tls-ca" pod "58fe4722-bc16-4cce-9a7d-93dd290f88a9" (UID: "58fe4722-bc16-4cce-9a7d-93dd290f88a9"). InnerVolumeSpecName "tls-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 409.138268] k3s[873]: I0919 21:27:15.225443 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58fe4722-bc16-4cce-9a7d-93dd290f88a9-config" pod "58fe4722-bc16-4cce-9a7d-93dd290f88a9" (UID: "58fe4722-bc16-4cce-9a7d-93dd290f88a9"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" machine # [ 409.194575] k3s[873]: I0919 21:27:15.284847 873 reconciler_common.go:299] "Volume detached for volume \"tls-ca\" (UniqueName: \"kubernetes.io/secret/58fe4722-bc16-4cce-9a7d-93dd290f88a9-tls-ca\") on node \"machine\" DevicePath \"\"" machine # [ 409.194777] k3s[873]: I0919 21:27:15.284886 873 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/58fe4722-bc16-4cce-9a7d-93dd290f88a9-tmp\") on node \"machine\" DevicePath \"\"" machine # [ 409.195408] k3s[873]: I0919 21:27:15.284907 873 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/58fe4722-bc16-4cce-9a7d-93dd290f88a9-config\") on node \"machine\" DevicePath \"\"" machine # [ 409.195859] k3s[873]: I0919 21:27:15.284920 873 reconciler_common.go:299] "Volume detached for volume \"tls-lb\" (UniqueName: \"kubernetes.io/secret/58fe4722-bc16-4cce-9a7d-93dd290f88a9-tls-lb\") on node \"machine\" DevicePath \"\"" machine # [ 409.212823] dhcpcd[6337]: vethaf65f9ab: removing interface machine # [ 409.288445] k3s[873]: E0919 21:27:15.378025 873 status.go:71] "Unhandled Error" err="apiserver received an error that is not an metav1.Status: &errors.errorString{s:\"failed to create typed patch object (farm/nix-grpc-farm-scheduler; /v1, Kind=Service): .spec.selector.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0}\"}: failed to create typed patch object (farm/nix-grpc-farm-scheduler; /v1, Kind=Service): .spec.selector.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0}" logger="UnhandledError" machine # [ 409.300129] k3s[873]: I0919 21:27:15.390487 873 alloc.go:329] "allocated clusterIPs" service="farm/nix-grpc-farm" clusterIPs={"IPv4":"10.43.210.157"} machine # [ 409.306991] k3s[873]: E0919 21:27:15.397598 873 status.go:71] "Unhandled Error" err=< machine # [ 409.307482] k3s[873]: apiserver received an error that is not an metav1.Status: &errors.errorString{s:"failed to create typed patch object (farm/nix-grpc-farm-scheduler; apps/v1, Kind=Deployment): errors:\n .spec.selector.matchLabels.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0}\n .spec.template.metadata.labels.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0}"}: failed to create typed patch object (farm/nix-grpc-farm-scheduler; apps/v1, Kind=Deployment): errors: machine # [ 409.308455] k3s[873]: .spec.selector.matchLabels.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0} machine # [ 409.310088] k3s[873]: .spec.template.metadata.labels.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0} machine # [ 409.310802] k3s[873]: > logger="UnhandledError" machine # [ 409.342627] systemd[1]: cri-containerd-007d47a685973e2a9413ce9e05b2306fe3096d78ac9d0114cad13819d726af6d.scope: Deactivated successfully. machine # [ 409.345545] systemd[1]: cri-containerd-007d47a685973e2a9413ce9e05b2306fe3096d78ac9d0114cad13819d726af6d.scope: Consumed 891ms CPU time over 2.479s wall clock time, 35.8M memory peak, 3.3M incoming IP traffic, 197.2K outgoing IP traffic. machine # [ 409.365642] systemd[1]: Created slice libcontainer container kubepods-besteffort-pod13c0092e_52ba_4681_a2ec_db61612ccf73.slice. machine # [ 409.383604] systemd[1]: Created slice libcontainer container kubepods-besteffort-podc1842c8c_7248_498f_9e8d_e23e96fec00e.slice. machine # [ 409.394663] k3s[873]: I0919 21:27:15.485256 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/13c0092e-52ba-4681-a2ec-db61612ccf73-tmp\") pod \"nix-grpc-farm-lb-688d8d547b-955kq\" (UID: \"13c0092e-52ba-4681-a2ec-db61612ccf73\") " pod="farm/nix-grpc-farm-lb-688d8d547b-955kq" machine # [ 409.394991] k3s[873]: I0919 21:27:15.485300 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"build\" (UniqueName: \"kubernetes.io/empty-dir/c1842c8c-7248-498f-9e8d-e23e96fec00e-build\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn\" (UID: \"c1842c8c-7248-498f-9e8d-e23e96fec00e\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn" machine # [ 409.395763] k3s[873]: I0919 21:27:15.485328 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nixconf\" (UniqueName: \"kubernetes.io/configmap/c1842c8c-7248-498f-9e8d-e23e96fec00e-nixconf\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn\" (UID: \"c1842c8c-7248-498f-9e8d-e23e96fec00e\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn" machine # [ 409.395974] k3s[873]: I0919 21:27:15.485352 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-lb\" (UniqueName: \"kubernetes.io/secret/13c0092e-52ba-4681-a2ec-db61612ccf73-tls-lb\") pod \"nix-grpc-farm-lb-688d8d547b-955kq\" (UID: \"13c0092e-52ba-4681-a2ec-db61612ccf73\") " pod="farm/nix-grpc-farm-lb-688d8d547b-955kq" machine # [ 409.396797] k3s[873]: I0919 21:27:15.485378 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-ca\" (UniqueName: \"kubernetes.io/secret/13c0092e-52ba-4681-a2ec-db61612ccf73-tls-ca\") pod \"nix-grpc-farm-lb-688d8d547b-955kq\" (UID: \"13c0092e-52ba-4681-a2ec-db61612ccf73\") " pod="farm/nix-grpc-farm-lb-688d8d547b-955kq" machine # [ 409.398194] k3s[873]: I0919 21:27:15.485405 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc\" (UniqueName: \"kubernetes.io/secret/c1842c8c-7248-498f-9e8d-e23e96fec00e-oidc\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn\" (UID: \"c1842c8c-7248-498f-9e8d-e23e96fec00e\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn" machine # [ 409.398877] k3s[873]: I0919 21:27:15.485432 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"niks3-token\" (UniqueName: \"kubernetes.io/projected/c1842c8c-7248-498f-9e8d-e23e96fec00e-niks3-token\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn\" (UID: \"c1842c8c-7248-498f-9e8d-e23e96fec00e\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn" machine # [ 409.399841] k3s[873]: I0919 21:27:15.485458 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls\" (UniqueName: \"kubernetes.io/secret/c1842c8c-7248-498f-9e8d-e23e96fec00e-tls\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn\" (UID: \"c1842c8c-7248-498f-9e8d-e23e96fec00e\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn" machine # [ 409.400409] k3s[873]: I0919 21:27:15.485486 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c1842c8c-7248-498f-9e8d-e23e96fec00e-ca\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn\" (UID: \"c1842c8c-7248-498f-9e8d-e23e96fec00e\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn" machine # [ 409.400963] k3s[873]: I0919 21:27:15.485515 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/13c0092e-52ba-4681-a2ec-db61612ccf73-config\") pod \"nix-grpc-farm-lb-688d8d547b-955kq\" (UID: \"13c0092e-52ba-4681-a2ec-db61612ccf73\") " pod="farm/nix-grpc-farm-lb-688d8d547b-955kq" machine # [ 409.401907] k3s[873]: I0919 21:27:15.485546 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nix\" (UniqueName: \"kubernetes.io/empty-dir/c1842c8c-7248-498f-9e8d-e23e96fec00e-nix\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn\" (UID: \"c1842c8c-7248-498f-9e8d-e23e96fec00e\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn" machine # [ 409.403061] k3s[873]: I0919 21:27:15.485572 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmjvn\" (UniqueName: \"kubernetes.io/projected/c1842c8c-7248-498f-9e8d-e23e96fec00e-kube-api-access-rmjvn\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn\" (UID: \"c1842c8c-7248-498f-9e8d-e23e96fec00e\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn" machine # [ 409.457139] systemd[1]: Created slice libcontainer container kubepods-besteffort-poda4b7c451_15a9_4252_a426_1db055a87b22.slice. machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 409.496082] k3s[873]: I0919 21:27:15.586644 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nix\" (UniqueName: \"kubernetes.io/empty-dir/a4b7c451-15a9-4252-a426-1db055a87b22-nix\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6\" (UID: \"a4b7c451-15a9-4252-a426-1db055a87b22\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6" machine # [ 409.496484] k3s[873]: I0919 21:27:15.587114 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls\" (UniqueName: \"kubernetes.io/secret/a4b7c451-15a9-4252-a426-1db055a87b22-tls\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6\" (UID: \"a4b7c451-15a9-4252-a426-1db055a87b22\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6" machine # [ 409.496829] k3s[873]: I0919 21:27:15.587465 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc\" (UniqueName: \"kubernetes.io/secret/a4b7c451-15a9-4252-a426-1db055a87b22-oidc\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6\" (UID: \"a4b7c451-15a9-4252-a426-1db055a87b22\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6" machine # [ 409.497808] k3s[873]: I0919 21:27:15.588441 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a4b7c451-15a9-4252-a426-1db055a87b22-ca\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6\" (UID: \"a4b7c451-15a9-4252-a426-1db055a87b22\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6" machine # [ 409.499320] k3s[873]: I0919 21:27:15.589230 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-td2fn\" (UniqueName: \"kubernetes.io/projected/a4b7c451-15a9-4252-a426-1db055a87b22-kube-api-access-td2fn\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6\" (UID: \"a4b7c451-15a9-4252-a426-1db055a87b22\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6" machine # [ 409.499638] k3s[873]: I0919 21:27:15.589263 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nixconf\" (UniqueName: \"kubernetes.io/configmap/a4b7c451-15a9-4252-a426-1db055a87b22-nixconf\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6\" (UID: \"a4b7c451-15a9-4252-a426-1db055a87b22\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6" machine # [ 409.501090] k3s[873]: I0919 21:27:15.589328 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"build\" (UniqueName: \"kubernetes.io/empty-dir/a4b7c451-15a9-4252-a426-1db055a87b22-build\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6\" (UID: \"a4b7c451-15a9-4252-a426-1db055a87b22\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6" machine # [ 409.501406] k3s[873]: I0919 21:27:15.589355 873 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"niks3-token\" (UniqueName: \"kubernetes.io/projected/a4b7c451-15a9-4252-a426-1db055a87b22-niks3-token\") pod \"nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6\" (UID: \"a4b7c451-15a9-4252-a426-1db055a87b22\") " pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6" machine # [ 409.542940] k3s[873]: I0919 21:27:15.633245 873 scope.go:122] "RemoveContainer" containerID="b7cbd85f6018dc9bb360423473e25291d7eef45f0a03e58ca776b2366de6ba50" machine # [ 409.543856] k3s[873]: I0919 21:27:15.633647 873 scope.go:122] "RemoveContainer" containerID="007d47a685973e2a9413ce9e05b2306fe3096d78ac9d0114cad13819d726af6d" machine # [ 409.548045] k3s[873]: E0919 21:27:15.638503 873 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"helm\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=helm pod=helm-install-nix-grpc-farm-zcg5w_kube-system(6d037147-1d29-4e02-b2ab-2f08bc2a576c)\"" pod="kube-system/helm-install-nix-grpc-farm-zcg5w" podUID="6d037147-1d29-4e02-b2ab-2f08bc2a576c" machine # [ 409.553695] k3s[873]: I0919 21:27:15.644220 873 scope.go:122] "RemoveContainer" containerID="549007675375dccc8addbbe0a60e06243cc8da6fecb3d29c53d1c0cfcd195605" machine # [ 409.563500] systemd[1]: Removed slice libcontainer container kubepods-besteffort-pod58fe4722_bc16_4cce_9a7d_93dd290f88a9.slice. machine # [ 409.563978] systemd[1]: kubepods-besteffort-pod58fe4722_bc16_4cce_9a7d_93dd290f88a9.slice: Consumed 1.153s CPU time over 2min 43.643s wall clock time, 16.1M memory peak, 152.9K incoming IP traffic, 148.7K outgoing IP traffic. machine # [ 409.568321] k3s[873]: I0919 21:27:15.658832 873 scope.go:122] "RemoveContainer" containerID="549007675375dccc8addbbe0a60e06243cc8da6fecb3d29c53d1c0cfcd195605" machine # [ 409.569606] k3s[873]: E0919 21:27:15.660116 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"549007675375dccc8addbbe0a60e06243cc8da6fecb3d29c53d1c0cfcd195605\": not found" containerID="549007675375dccc8addbbe0a60e06243cc8da6fecb3d29c53d1c0cfcd195605" machine # [ 409.570028] k3s[873]: I0919 21:27:15.660170 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"549007675375dccc8addbbe0a60e06243cc8da6fecb3d29c53d1c0cfcd195605"} err="failed to get container status \"549007675375dccc8addbbe0a60e06243cc8da6fecb3d29c53d1c0cfcd195605\": rpc error: code = NotFound desc = an error occurred when try to find container \"549007675375dccc8addbbe0a60e06243cc8da6fecb3d29c53d1c0cfcd195605\": not found" machine # [ 409.843875] cni0: port 4(vethc8ce4203) entered blocking state machine # [ 409.845247] cni0: port 4(vethc8ce4203) entered disabled state machine # [ 409.847198] vethc8ce4203: entered allmulticast mode machine # [ 409.849092] vethc8ce4203: entered promiscuous mode machine # [ 409.706659] (udev-worker)[19268]: Network interface NamePolicy= disabled on kernel command line. machine # [ 409.865609] cni0: port 4(vethc8ce4203) entered blocking state machine # [ 409.866650] cni0: port 4(vethc8ce4203) entered forwarding state machine # [ 409.877199] cni0: port 6(vethe1ddc4e5) entered blocking state machine # [ 409.879455] cni0: port 6(vethe1ddc4e5) entered disabled state machine # [ 409.881608] vethe1ddc4e5: entered allmulticast mode machine # [ 409.883248] vethe1ddc4e5: entered promiscuous mode machine # [ 409.741314] (udev-worker)[19430]: Network interface NamePolicy= disabled on kernel command line. machine # [ 409.897781] cni0: port 6(vethe1ddc4e5) entered blocking state machine # [ 409.899110] cni0: port 6(vethe1ddc4e5) entered forwarding state machine # [ 409.754916] dhcpcd[6337]: vethc8ce4203: IAID 07:2d:e5:43 machine # [ 409.757607] dhcpcd[6337]: vethc8ce4203: adding address fe80::7466:7ff:fe2d:e543 machine # [ 409.772598] (udev-worker)[19421]: Network interface NamePolicy= disabled on kernel command line. machine # [ 409.920837] cni0: port 7(vethfebecc9a) entered blocking state machine # [ 409.923002] cni0: port 7(vethfebecc9a) entered disabled state machine # [ 409.924674] vethfebecc9a: entered allmulticast mode machine # [ 409.925897] vethfebecc9a: entered promiscuous mode machine # [ 409.785832] dhcpcd[6337]: vethe1ddc4e5: IAID 8d:2c:9d:98 machine # [ 409.787354] dhcpcd[6337]: vethe1ddc4e5: adding address fe80::c0b5:8dff:fe2c:9d98 machine # [ 409.940842] cni0: port 7(vethfebecc9a) entered blocking state machine # [ 409.942179] cni0: port 7(vethfebecc9a) entered forwarding state machine # [ 409.797305] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-007d47a685973e2a9413ce9e05b2306fe3096d78ac9d0114cad13819d726af6d-rootfs.mount: Deactivated successfully. machine # [ 409.803118] systemd[1]: var-lib-kubelet-pods-58fe4722\x2dbc16\x2d4cce\x2d9a7d\x2d93dd290f88a9-volume\x2dsubpaths-config-envoy-0.mount: Deactivated successfully. machine # [ 409.807144] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-8cb216c6dac205cb93634eeeda31a3445793e2c5640127981d5e6eaac270b5d1-rootfs.mount: Deactivated successfully. machine # [ 409.812524] systemd[1]: run-k3s-containerd-io.containerd.grpc.v1.cri-sandboxes-8cb216c6dac205cb93634eeeda31a3445793e2c5640127981d5e6eaac270b5d1-shm.mount: Deactivated successfully. machine # [ 409.813350] systemd[1]: run-netns-cni\x2d50441055\x2d3f94\x2db2cc\x2def52\x2db9e63be137d0.mount: Deactivated successfully. machine # [ 409.813514] systemd[1]: var-lib-kubelet-pods-58fe4722\x2dbc16\x2d4cce\x2d9a7d\x2d93dd290f88a9-volumes-kubernetes.io\x7esecret-tls\x2dlb.mount: Deactivated successfully. machine # [ 409.814641] systemd[1]: var-lib-kubelet-pods-58fe4722\x2dbc16\x2d4cce\x2d9a7d\x2d93dd290f88a9-volumes-kubernetes.io\x7esecret-tls\x2dca.mount: Deactivated successfully. machine # [ 409.815136] systemd[1]: var-lib-kubelet-pods-58fe4722\x2dbc16\x2d4cce\x2d9a7d\x2d93dd290f88a9-volumes-kubernetes.io\x7eempty\x2ddir-tmp.mount: Deactivated successfully. machine # [ 409.820962] dhcpcd[6337]: vethfebecc9a: IAID f3:0a:77:6b machine # [ 409.821751] dhcpcd[6337]: vethfebecc9a: adding address fe80::2095:f3ff:fe0a:776b machine # [ 409.879687] systemd[1]: Started libcontainer container 98c22703502fd19571fe840900554ccbe13632675d835e460690ef49f6d8b087. machine # [ 409.913333] systemd[1]: Started libcontainer container 37ad862e28cc78bfbd35c0ad5666cc5d42f336de5e790c1947fd5d5f5b4e94f4. machine # [ 409.916477] systemd[1]: Started libcontainer container 7064d27080c661cc82b44cfd3ebce49d22ac2df61ef46e931274ef6aebf74f1f. machine # [ 410.035338] systemd[1]: Started libcontainer container 757e9628040798fdb8d9e4b0f352ae214af4aae62c0c2820b9163fa7a46c6c2f. machine # [ 410.052329] systemd[1]: Started libcontainer container b84d5e2f4e5d6532e8a1e5ed472a0a15eb50fdc8ccd76a2323274c4526715685. machine # [ 410.055658] systemd[1]: Started libcontainer container 3fbdd82cd7b4b49ffc803b3ce00dc4b8d653de6b6c51c0f0c179bf7498064f5e. machine # [ 410.369349] dhcpcd[6337]: vethfebecc9a: soliciting a DHCP lease machine # [ 410.391569] dhcpcd[6337]: vethe1ddc4e5: soliciting a DHCP lease machine # [ 410.657682] k3s[873]: I0919 21:27:16.747853 873 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="farm/nix-grpc-farm-lb-688d8d547b-955kq" podStartSLOduration=1.747828937 podStartE2EDuration="1.747828937s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 21:27:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 21:27:16.723794039 +0000 UTC m=+392.307638776" watchObservedRunningTime="2026-09-19 21:27:16.747828937 +0000 UTC m=+392.331673395" machine # [ 410.701460] systemd[1]: cri-containerd-3fbdd82cd7b4b49ffc803b3ce00dc4b8d653de6b6c51c0f0c179bf7498064f5e.scope: Deactivated successfully. machine # [ 410.705178] systemd[1]: cri-containerd-3fbdd82cd7b4b49ffc803b3ce00dc4b8d653de6b6c51c0f0c179bf7498064f5e.scope: Consumed 593ms CPU time over 645ms wall clock time, 301.9M memory peak, 150M written to disk. machine # [ 410.714096] systemd[1]: cri-containerd-b84d5e2f4e5d6532e8a1e5ed472a0a15eb50fdc8ccd76a2323274c4526715685.scope: Deactivated successfully. machine # [ 410.714589] systemd[1]: cri-containerd-b84d5e2f4e5d6532e8a1e5ed472a0a15eb50fdc8ccd76a2323274c4526715685.scope: Consumed 571ms CPU time over 658ms wall clock time, 301.5M memory peak, 159.9M written to disk. machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 410.799691] k3s[873]: I0919 21:27:16.889869 873 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="58fe4722-bc16-4cce-9a7d-93dd290f88a9" path="/var/lib/kubelet/pods/58fe4722-bc16-4cce-9a7d-93dd290f88a9/volumes" machine # [ 410.804376] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-3fbdd82cd7b4b49ffc803b3ce00dc4b8d653de6b6c51c0f0c179bf7498064f5e-rootfs.mount: Deactivated successfully. machine # [ 410.807555] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-b84d5e2f4e5d6532e8a1e5ed472a0a15eb50fdc8ccd76a2323274c4526715685-rootfs.mount: Deactivated successfully. machine # [ 411.319648] dhcpcd[6337]: vethc8ce4203: soliciting a DHCP lease machine # [ 411.586634] dhcpcd[6337]: vethc8ce4203: soliciting an IPv6 router machine # [ 411.708392] systemd[1]: Started libcontainer container fac5adda407e5e1d90c79cfcfdd32baa19308a1ac62b38c116521b67e84da943. machine # [ 411.716344] systemd[1]: Started libcontainer container c443b6a0e3a939f02df4b0c6aa08cb872e8ad27c124e59cfb35be7dda5cc082e. machine # [ 411.802509] dhcpcd[6337]: vethfebecc9a: soliciting an IPv6 router machine # [ 411.824329] systemd[1]: var-lib-rancher-k3s-agent-containerd-tmpmounts-containerd\x2dmount3477753855.mount: Deactivated successfully. machine # [ 411.870382] systemd[1]: Started libcontainer container a7542ef9f30848cf5fa30f15b458887f85287c9b10338e60c94551e2840e9856. machine # [ 411.883251] systemd[1]: Started libcontainer container 94e1350d8e3c804f92496cfbb1b53d22eca256fa459379e5d37c6640bf78941c. machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 412.041411] systemd[1]: Started libcontainer container 2f9b093b2520ae5fdc02332eec2d00943ca7c48ccf62b3e1cc31b07b69826994. machine # [ 412.045568] systemd[1]: Started libcontainer container 89d04a83b5d430a03812de6c8a477af8342c9a9df505cb4c2e5b5d108d230b1e. machine # [ 412.073272] dhcpcd[6337]: vethe1ddc4e5: soliciting an IPv6 router machine # [ 412.491886] systemd[1]: cri-containerd-d55c241872ed8542a8300861700b666022c77b0412c1c7cc4a30b5860448a006.scope: Deactivated successfully. machine # [ 412.493732] systemd[1]: cri-containerd-d55c241872ed8542a8300861700b666022c77b0412c1c7cc4a30b5860448a006.scope: Consumed 709ms CPU time over 2min 44.278s wall clock time, 13.5M memory peak, 5.6M read from disk, 58.7K incoming IP traffic, 50.2K outgoing IP traffic. machine # [ 412.545536] systemd[1]: cri-containerd-1fad4be34f8c04e4063583129e70de393080b173541dc8a3034254c4670f21b6.scope: Deactivated successfully. machine # [ 412.547339] systemd[1]: cri-containerd-1fad4be34f8c04e4063583129e70de393080b173541dc8a3034254c4670f21b6.scope: Consumed 716ms CPU time over 2min 44.306s wall clock time, 33M memory peak, 25.4M read from disk, 58.8K incoming IP traffic, 50K outgoing IP traffic. machine # [ 412.623306] systemd[1]: cri-containerd-9208df284202020f16ef557ff0b0a79ccfcfa74324981539f7edba6dc2d85f57.scope: Deactivated successfully. machine # [ 412.656419] k3s[873]: I0919 21:27:18.746330 873 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6" podStartSLOduration=3.140651489 podStartE2EDuration="3.746310029s" totalImagesPullingTime="0s" totalInitContainerRuntime="605.65854ms" isStatefulPod=false podCreationTimestamp="2026-09-19 21:27:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 21:27:18.745908023 +0000 UTC m=+394.329752760" watchObservedRunningTime="2026-09-19 21:27:18.746310029 +0000 UTC m=+394.330154487" machine # [ 412.678179] k3s[873]: I0919 21:27:18.767534 873 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn" podStartSLOduration=3.138074065 podStartE2EDuration="3.767511606s" totalImagesPullingTime="0s" totalInitContainerRuntime="629.437541ms" isStatefulPod=false podCreationTimestamp="2026-09-19 21:27:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 21:27:18.765298755 +0000 UTC m=+394.349143493" watchObservedRunningTime="2026-09-19 21:27:18.767511606 +0000 UTC m=+394.351356064" machine # [ 412.687030] systemd[1]: cri-containerd-1267dc040a94a4fc9ed98b204dc6438c7dd8d2395a8cc14c272757b3dfb22150.scope: Deactivated successfully. machine # [ 412.793573] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-1fad4be34f8c04e4063583129e70de393080b173541dc8a3034254c4670f21b6-rootfs.mount: Deactivated successfully. machine # [ 412.793900] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-d55c241872ed8542a8300861700b666022c77b0412c1c7cc4a30b5860448a006-rootfs.mount: Deactivated successfully. machine # [ 412.794751] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-9208df284202020f16ef557ff0b0a79ccfcfa74324981539f7edba6dc2d85f57-rootfs.mount: Deactivated successfully. machine # [ 412.795707] systemd[1]: run-k3s-containerd-io.containerd.grpc.v1.cri-sandboxes-9208df284202020f16ef557ff0b0a79ccfcfa74324981539f7edba6dc2d85f57-shm.mount: Deactivated successfully. machine # [ 412.806351] systemd[1]: run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-1267dc040a94a4fc9ed98b204dc64[ 412.953141] cni0: port 5(vethe0f40db3) entered disabled state machine # 38c7dd8d2395a8cc14c272757b3dfb22150-rootfs.mount: Deactivated successfully. machine # [ 412.806955] systemd[1]: run-k3s-containerd-io.containerd.grpc.v1.cri-sandboxes-1267dc040a94a4fc9ed98b204dc6438c7dd8d2395a8cc14c272757b3dfb22150-shm.mount: Deactivated successfully. machine # [ 412.809083] dhcpcd[6337]: vethe0f40db3: carrier lost machine # [ 412.960310] vethe0f40db3 (unregistering): left allmulticast mode machine # [ 412.962503] vethe0f40db3 (unregistering): left promiscuous mode machine # [ 412.964396] cni0: port 5(vethe0f40db3) entered disabled state machine # [ 412.843343] systemd[1]: run-netns-cni\x2d794a7179\x2d8c44\x2d3fa1\x2de0d1\x2dd29b687d9557.mount: Deactivated successfully. machine # [ 413.000473] cni0: port 3(vethe862d05c) entered disabled state machine # [ 413.002815] vethe862d05c (unregistering): left allmulticast mode machine # [ 413.004533] vethe862d05c (unregistering): left promiscuous mode machine # [ 413.006068] cni0: port 3(vethe862d05c) entered disabled state machine # [ 412.872747] dhcpcd[6337]: vethe0f40db3: deleting address fe80::282d:25ff:fe2a:5739 machine # [ 412.877547] dhcpcd[6337]: vethe0f40db3: deleting route to 169.254.0.0/16 machine # [ 412.882650] systemd[1]: run-netns-cni\x2d8bd80a14\x2df963\x2da752\x2d2a3a\x2d9b3787d4ae8d.mount: Deactivated successfully. machine # [ 412.936420] k3s[873]: I0919 21:27:19.026176 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/58f09918-6c02-42ea-a7cd-7a2ebf47c912-kube-api-access-gh2kl\" (UniqueName: \"kubernetes.io/projected/58f09918-6c02-42ea-a7cd-7a2ebf47c912-kube-api-access-gh2kl\") pod \"58f09918-6c02-42ea-a7cd-7a2ebf47c912\" (UID: \"58f09918-6c02-42ea-a7cd-7a2ebf47c912\") " machine # [ 412.936670] k3s[873]: I0919 21:27:19.026232 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/58f09918-6c02-42ea-a7cd-7a2ebf47c912-oidc\" (UniqueName: \"kubernetes.io/secret/58f09918-6c02-42ea-a7cd-7a2ebf47c912-oidc\") pod \"58f09918-6c02-42ea-a7cd-7a2ebf47c912\" (UID: \"58f09918-6c02-42ea-a7cd-7a2ebf47c912\") " machine # [ 412.937325] k3s[873]: I0919 21:27:19.026266 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/58f09918-6c02-42ea-a7cd-7a2ebf47c912-niks3-token\" (UniqueName: \"kubernetes.io/projected/58f09918-6c02-42ea-a7cd-7a2ebf47c912-niks3-token\") pod \"58f09918-6c02-42ea-a7cd-7a2ebf47c912\" (UID: \"58f09918-6c02-42ea-a7cd-7a2ebf47c912\") " machine # [ 412.938034] k3s[873]: I0919 21:27:19.026297 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e187fb7d-78e0-482b-b7fd-27eab0e7c263-nix\" (UniqueName: \"kubernetes.io/empty-dir/e187fb7d-78e0-482b-b7fd-27eab0e7c263-nix\") pod \"e187fb7d-78e0-482b-b7fd-27eab0e7c263\" (UID: \"e187fb7d-78e0-482b-b7fd-27eab0e7c263\") " machine # [ 412.939272] k3s[873]: I0919 21:27:19.026329 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/58f09918-6c02-42ea-a7cd-7a2ebf47c912-nix\" (UniqueName: \"kubernetes.io/empty-dir/58f09918-6c02-42ea-a7cd-7a2ebf47c912-nix\") pod \"58f09918-6c02-42ea-a7cd-7a2ebf47c912\" (UID: \"58f09918-6c02-42ea-a7cd-7a2ebf47c912\") " machine # [ 412.939827] k3s[873]: I0919 21:27:19.026358 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e187fb7d-78e0-482b-b7fd-27eab0e7c263-nixconf\" (UniqueName: \"kubernetes.io/configmap/e187fb7d-78e0-482b-b7fd-27eab0e7c263-nixconf\") pod \"e187fb7d-78e0-482b-b7fd-27eab0e7c263\" (UID: \"e187fb7d-78e0-482b-b7fd-27eab0e7c263\") " machine # [ 412.941169] k3s[873]: I0919 21:27:19.026384 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e187fb7d-78e0-482b-b7fd-27eab0e7c263-niks3-token\" (UniqueName: \"kubernetes.io/projected/e187fb7d-78e0-482b-b7fd-27eab0e7c263-niks3-token\") pod \"e187fb7d-78e0-482b-b7fd-27eab0e7c263\" (UID: \"e187fb7d-78e0-482b-b7fd-27eab0e7c263\") " machine # [ 412.941951] k3s[873]: I0919 21:27:19.026411 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/58f09918-6c02-42ea-a7cd-7a2ebf47c912-ca\" (UniqueName: \"kubernetes.io/secret/58f09918-6c02-42ea-a7cd-7a2ebf47c912-ca\") pod \"58f09918-6c02-42ea-a7cd-7a2ebf47c912\" (UID: \"58f09918-6c02-42ea-a7cd-7a2ebf47c912\") " machine # [ 412.942766] k3s[873]: I0919 21:27:19.026443 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/58f09918-6c02-42ea-a7cd-7a2ebf47c912-nixconf\" (UniqueName: \"kubernetes.io/configmap/58f09918-6c02-42ea-a7cd-7a2ebf47c912-nixconf\") pod \"58f09918-6c02-42ea-a7cd-7a2ebf47c912\" (UID: \"58f09918-6c02-42ea-a7cd-7a2ebf47c912\") " machine # [ 412.943865] k3s[873]: I0919 21:27:19.026471 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e187fb7d-78e0-482b-b7fd-27eab0e7c263-ca\" (UniqueName: \"kubernetes.io/secret/e187fb7d-78e0-482b-b7fd-27eab0e7c263-ca\") pod \"e187fb7d-78e0-482b-b7fd-27eab0e7c263\" (UID: \"e187fb7d-78e0-482b-b7fd-27eab0e7c263\") " machine # [ 412.944496] k3s[873]: I0919 21:27:19.026498 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e187fb7d-78e0-482b-b7fd-27eab0e7c263-oidc\" (UniqueName: \"kubernetes.io/secret/e187fb7d-78e0-482b-b7fd-27eab0e7c263-oidc\") pod \"e187fb7d-78e0-482b-b7fd-27eab0e7c263\" (UID: \"e187fb7d-78e0-482b-b7fd-27eab0e7c263\") " machine # [ 412.944810] k3s[873]: I0919 21:27:19.026523 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/58f09918-6c02-42ea-a7cd-7a2ebf47c912-tls\" (UniqueName: \"kubernetes.io/secret/58f09918-6c02-42ea-a7cd-7a2ebf47c912-tls\") pod \"58f09918-6c02-42ea-a7cd-7a2ebf47c912\" (UID: \"58f09918-6c02-42ea-a7cd-7a2ebf47c912\") " machine # [ 412.945877] k3s[873]: I0919 21:27:19.026547 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e187fb7d-78e0-482b-b7fd-27eab0e7c263-build\" (UniqueName: \"kubernetes.io/empty-dir/e187fb7d-78e0-482b-b7fd-27eab0e7c263-build\") pod \"e187fb7d-78e0-482b-b7fd-27eab0e7c263\" (UID: \"e187fb7d-78e0-482b-b7fd-27eab0e7c263\") " machine # [ 412.992767] k3s[873]: I0919 21:27:19.026572 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e187fb7d-78e0-482b-b7fd-27eab0e7c263-tls\" (UniqueName: \"kubernetes.io/secret/e187fb7d-78e0-482b-b7fd-27eab0e7c263-tls\") pod \"e187fb7d-78e0-482b-b7fd-27eab0e7c263\" (UID: \"e187fb7d-78e0-482b-b7fd-27eab0e7c263\") " machine # [ 412.993872] k3s[873]: I0919 21:27:19.026599 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/58f09918-6c02-42ea-a7cd-7a2ebf47c912-build\" (UniqueName: \"kubernetes.io/empty-dir/58f09918-6c02-42ea-a7cd-7a2ebf47c912-build\") pod \"58f09918-6c02-42ea-a7cd-7a2ebf47c912\" (UID: \"58f09918-6c02-42ea-a7cd-7a2ebf47c912\") " machine # [ 412.995199] k3s[873]: I0919 21:27:19.026625 873 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e187fb7d-78e0-482b-b7fd-27eab0e7c263-kube-api-access-qgk42\" (UniqueName: \"kubernetes.io/projected/e187fb7d-78e0-482b-b7fd-27eab0e7c263-kube-api-access-qgk42\") pod \"e187fb7d-78e0-482b-b7fd-27eab0e7c263\" (UID: \"e187fb7d-78e0-482b-b7fd-27eab0e7c263\") " machine # [ 412.995885] k3s[873]: I0919 21:27:19.029943 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e187fb7d-78e0-482b-b7fd-27eab0e7c263-build" pod "e187fb7d-78e0-482b-b7fd-27eab0e7c263" (UID: "e187fb7d-78e0-482b-b7fd-27eab0e7c263"). InnerVolumeSpecName "build". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 412.995972] k3s[873]: I0919 21:27:19.030393 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58f09918-6c02-42ea-a7cd-7a2ebf47c912-build" pod "58f09918-6c02-42ea-a7cd-7a2ebf47c912" (UID: "58f09918-6c02-42ea-a7cd-7a2ebf47c912"). InnerVolumeSpecName "build". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 412.996404] k3s[873]: I0919 21:27:19.031608 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58f09918-6c02-42ea-a7cd-7a2ebf47c912-nixconf" pod "58f09918-6c02-42ea-a7cd-7a2ebf47c912" (UID: "58f09918-6c02-42ea-a7cd-7a2ebf47c912"). InnerVolumeSpecName "nixconf". PluginName "kubernetes.io/configmap", VolumeGIDValue "" machine # [ 412.998641] k3s[873]: I0919 21:27:19.034881 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e187fb7d-78e0-482b-b7fd-27eab0e7c263-nixconf" pod "e187fb7d-78e0-482b-b7fd-27eab0e7c263" (UID: "e187fb7d-78e0-482b-b7fd-27eab0e7c263"). InnerVolumeSpecName "nixconf". PluginName "kubernetes.io/configmap", VolumeGIDValue "" machine # [ 412.999072] systemd[1]: var-lib-kubelet-pods-e187fb7d\x2d78e0\x2d482b\x2db7fd\x2d27eab0e7c263-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dqgk42.mount: Deactivated successfully. machine # [ 412.999323] systemd[1]: var-lib-kubelet-pods-58f09918\x2d6c02\x2d42ea\x2da7cd\x2d7a2ebf47c912-volumes-kubernetes.io\x7eprojected-niks3\x2dtoken.mount: Deactivated successfully. machine # [ 412.999657] systemd[1]: var-lib-kubelet-pods-e187fb7d\x2d78e0\x2d482b\x2db7fd\x2d27eab0e7c263-volumes-kubernetes.io\x7esecret-oidc.mount: Deactivated successfully. machine # [ 413.014768] k3s[873]: I0919 21:27:19.105315 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e187fb7d-78e0-482b-b7fd-27eab0e7c263-kube-api-access-qgk42" pod "e187fb7d-78e0-482b-b7fd-27eab0e7c263" (UID: "e187fb7d-78e0-482b-b7fd-27eab0e7c263"). InnerVolumeSpecName "kube-api-access-qgk42". PluginName "kubernetes.io/projected", VolumeGIDValue "" machine # [ 413.040169] k3s[873]: I0919 21:27:19.130765 873 reconciler_common.go:299] "Volume detached for volume \"build\" (UniqueName: \"kubernetes.io/empty-dir/58f09918-6c02-42ea-a7cd-7a2ebf47c912-build\") on node \"machine\" DevicePath \"\"" machine # [ 413.040941] k3s[873]: I0919 21:27:19.131174 873 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgk42\" (UniqueName: \"kubernetes.io/projected/e187fb7d-78e0-482b-b7fd-27eab0e7c263-kube-api-access-qgk42\") on node \"machine\" DevicePath \"\"" machine # [ 413.041936] k3s[873]: I0919 21:27:19.131195 873 reconciler_common.go:299] "Volume detached for volume \"nixconf\" (UniqueName: \"kubernetes.io/configmap/e187fb7d-78e0-482b-b7fd-27eab0e7c263-nixconf\") on node \"machine\" DevicePath \"\"" machine # [ 413.043356] k3s[873]: I0919 21:27:19.131208 873 reconciler_common.go:299] "Volume detached for volume \"nixconf\" (UniqueName: \"kubernetes.io/configmap/58f09918-6c02-42ea-a7cd-7a2ebf47c912-nixconf\") on node \"machine\" DevicePath \"\"" machine # [ 413.044658] k3s[873]: I0919 21:27:19.131220 873 reconciler_common.go:299] "Volume detached for volume \"build\" (UniqueName: \"kubernetes.io/empty-dir/e187fb7d-78e0-482b-b7fd-27eab0e7c263-build\") on node \"machine\" DevicePath \"\"" machine # [ 413.046411] k3s[873]: I0919 21:27:19.131341 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58f09918-6c02-42ea-a7cd-7a2ebf47c912-niks3-token" pod "58f09918-6c02-42ea-a7cd-7a2ebf47c912" (UID: "58f09918-6c02-42ea-a7cd-7a2ebf47c912"). InnerVolumeSpecName "niks3-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" machine # [ 413.048446] k3s[873]: I0919 21:27:19.131456 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58f09918-6c02-42ea-a7cd-7a2ebf47c912-ca" pod "58f09918-6c02-42ea-a7cd-7a2ebf47c912" (UID: "58f09918-6c02-42ea-a7cd-7a2ebf47c912"). InnerVolumeSpecName "ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 413.049649] k3s[873]: I0919 21:27:19.131571 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58f09918-6c02-42ea-a7cd-7a2ebf47c912-kube-api-access-gh2kl" pod "58f09918-6c02-42ea-a7cd-7a2ebf47c912" (UID: "58f09918-6c02-42ea-a7cd-7a2ebf47c912"). InnerVolumeSpecName "kube-api-access-gh2kl". PluginName "kubernetes.io/projected", VolumeGIDValue "" machine # [ 413.050937] k3s[873]: I0919 21:27:19.132138 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e187fb7d-78e0-482b-b7fd-27eab0e7c263-oidc" pod "e187fb7d-78e0-482b-b7fd-27eab0e7c263" (UID: "e187fb7d-78e0-482b-b7fd-27eab0e7c263"). InnerVolumeSpecName "oidc". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 413.052396] k3s[873]: I0919 21:27:19.132593 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e187fb7d-78e0-482b-b7fd-27eab0e7c263-niks3-token" pod "e187fb7d-78e0-482b-b7fd-27eab0e7c263" (UID: "e187fb7d-78e0-482b-b7fd-27eab0e7c263"). InnerVolumeSpecName "niks3-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" machine # [ 413.054684] k3s[873]: I0919 21:27:19.132777 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e187fb7d-78e0-482b-b7fd-27eab0e7c263-ca" pod "e187fb7d-78e0-482b-b7fd-27eab0e7c263" (UID: "e187fb7d-78e0-482b-b7fd-27eab0e7c263"). InnerVolumeSpecName "ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 413.055846] k3s[873]: I0919 21:27:19.134883 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58f09918-6c02-42ea-a7cd-7a2ebf47c912-oidc" pod "58f09918-6c02-42ea-a7cd-7a2ebf47c912" (UID: "58f09918-6c02-42ea-a7cd-7a2ebf47c912"). InnerVolumeSpecName "oidc". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 413.057151] k3s[873]: I0919 21:27:19.136773 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e187fb7d-78e0-482b-b7fd-27eab0e7c263-tls" pod "e187fb7d-78e0-482b-b7fd-27eab0e7c263" (UID: "e187fb7d-78e0-482b-b7fd-27eab0e7c263"). InnerVolumeSpecName "tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 413.125306] k3s[873]: I0919 21:27:19.131065 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58f09918-6c02-42ea-a7cd-7a2ebf47c912-tls" pod "58f09918-6c02-42ea-a7cd-7a2ebf47c912" (UID: "58f09918-6c02-42ea-a7cd-7a2ebf47c912"). InnerVolumeSpecName "tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" machine # [ 413.140870] k3s[873]: I0919 21:27:19.231426 873 reconciler_common.go:299] "Volume detached for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e187fb7d-78e0-482b-b7fd-27eab0e7c263-ca\") on node \"machine\" DevicePath \"\"" machine # [ 413.141253] k3s[873]: I0919 21:27:19.231473 873 reconciler_common.go:299] "Volume detached for volume \"oidc\" (UniqueName: \"kubernetes.io/secret/e187fb7d-78e0-482b-b7fd-27eab0e7c263-oidc\") on node \"machine\" DevicePath \"\"" machine # [ 413.141628] k3s[873]: I0919 21:27:19.231494 873 reconciler_common.go:299] "Volume detached for volume \"tls\" (UniqueName: \"kubernetes.io/secret/58f09918-6c02-42ea-a7cd-7a2ebf47c912-tls\") on node \"machine\" DevicePath \"\"" machine # [ 413.142121] k3s[873]: I0919 21:27:19.231506 873 reconciler_common.go:299] "Volume detached for volume \"tls\" (UniqueName: \"kubernetes.io/secret/e187fb7d-78e0-482b-b7fd-27eab0e7c263-tls\") on node \"machine\" DevicePath \"\"" machine # [ 413.142616] k3s[873]: I0919 21:27:19.231523 873 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gh2kl\" (UniqueName: \"kubernetes.io/projected/58f09918-6c02-42ea-a7cd-7a2ebf47c912-kube-api-access-gh2kl\") on node \"machine\" DevicePath \"\"" machine # [ 413.143900] k3s[873]: I0919 21:27:19.231535 873 reconciler_common.go:299] "Volume detached for volume \"oidc\" (UniqueName: \"kubernetes.io/secret/58f09918-6c02-42ea-a7cd-7a2ebf47c912-oidc\") on node \"machine\" DevicePath \"\"" machine # [ 413.145115] k3s[873]: I0919 21:27:19.231553 873 reconciler_common.go:299] "Volume detached for volume \"niks3-token\" (UniqueName: \"kubernetes.io/projected/58f09918-6c02-42ea-a7cd-7a2ebf47c912-niks3-token\") on node \"machine\" DevicePath \"\"" machine # [ 413.145697] k3s[873]: I0919 21:27:19.231565 873 reconciler_common.go:299] "Volume detached for volume \"niks3-token\" (UniqueName: \"kubernetes.io/projected/e187fb7d-78e0-482b-b7fd-27eab0e7c263-niks3-token\") on node \"machine\" DevicePath \"\"" machine # [ 413.163099] dhcpcd[6337]: dhcp_readbpf: unexpected event 0x0100 machine # [ 413.163592] k3s[873]: I0919 21:27:19.231577 873 reconciler_common.go:299] "Volume detached for volume \"ca\" (UniqueName: \"kubernetes.io/secret/58f09918-6c02-42ea-a7cd-7a2ebf47c912-ca\") on node \"machine\" DevicePath \"\"" machine # [ 413.163983] dhcpcd[6337]: dhcp_readbpf: vethe862d05c: Network is down machine # [ 413.181215] dhcpcd[6337]: arp_read: unexpected event 0x0100 machine # [ 413.181423] dhcpcd[6337]: arp_read: vethe862d05c: Network is down machine # [ 413.185593] dhcpcd[6337]: vethe0f40db3: removing interface machine # [ 413.215476] dhcpcd[6337]: vethe862d05c: removing interface machine # [ 413.215664] dhcpcd[6337]: vethe862d05c: deleting address fe80::600d:90ff:fe95:ccf2 machine # [ 413.217370] dhcpcd[6337]: vethe862d05c: deleting route to 169.254.0.0/16 machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 413.302180] k3s[873]: I0919 21:27:19.392567 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e187fb7d-78e0-482b-b7fd-27eab0e7c263-nix" pod "e187fb7d-78e0-482b-b7fd-27eab0e7c263" (UID: "e187fb7d-78e0-482b-b7fd-27eab0e7c263"). InnerVolumeSpecName "nix". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 413.335935] k3s[873]: I0919 21:27:19.426485 873 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58f09918-6c02-42ea-a7cd-7a2ebf47c912-nix" pod "58f09918-6c02-42ea-a7cd-7a2ebf47c912" (UID: "58f09918-6c02-42ea-a7cd-7a2ebf47c912"). InnerVolumeSpecName "nix". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" machine # [ 413.365020] k3s[873]: I0919 21:27:19.455571 873 reconciler_common.go:299] "Volume detached for volume \"nix\" (UniqueName: \"kubernetes.io/empty-dir/e187fb7d-78e0-482b-b7fd-27eab0e7c263-nix\") on node \"machine\" DevicePath \"\"" machine # [ 413.365171] k3s[873]: I0919 21:27:19.455605 873 reconciler_common.go:299] "Volume detached for volume \"nix\" (UniqueName: \"kubernetes.io/empty-dir/58f09918-6c02-42ea-a7cd-7a2ebf47c912-nix\") on node \"machine\" DevicePath \"\"" machine # [ 413.669224] k3s[873]: I0919 21:27:19.759252 873 scope.go:122] "RemoveContainer" containerID="1d5df292fc04e81c9ee953630b9728f287a6c32223613ff42cb63afc62e2757a" machine # [ 413.685363] k3s[873]: I0919 21:27:19.775793 873 scope.go:122] "RemoveContainer" containerID="d55c241872ed8542a8300861700b666022c77b0412c1c7cc4a30b5860448a006" machine # [ 413.690878] systemd[1]: Removed slice libcontainer container kubepods-besteffort-pod58f09918_6c02_42ea_a7cd_7a2ebf47c912.slice. machine # [ 413.692870] systemd[1]: kubepods-besteffort-pod58f09918_6c02_42ea_a7cd_7a2ebf47c912.slice: Consumed 1.658s CPU time over 2min 47.759s wall clock time, 320.3M memory peak, 5.6M read from disk, 283.9M written to disk, 58.7K incoming IP traffic, 50.2K outgoing IP traffic. machine # [ 413.695499] systemd[1]: Removed slice libcontainer container kubepods-besteffort-pode187fb7d_78e0_482b_b7fd_27eab0e7c263.slice. machine # [ 413.696369] systemd[1]: kubepods-besteffort-pode187fb7d_78e0_482b_b7fd_27eab0e7c263.slice: Consumed 1.696s CPU time over 2min 47.786s wall clock time, 305.8M memory peak, 44.5M read from disk, 283.9M written to disk, 58.8K incoming IP traffic, 50K outgoing IP traffic. machine # [ 413.701347] k3s[873]: I0919 21:27:19.791804 873 scope.go:122] "RemoveContainer" containerID="6351c37c4fc718b0471e384515a99ebf82c2c17831bb85ed543a7b0eaa70c343" machine # [ 413.713825] k3s[873]: I0919 21:27:19.804005 873 scope.go:122] "RemoveContainer" containerID="b4401b02b5bb37e7066c398f3c9ab50e0e33913a13e8d341bc7654df283b8a26" machine # [ 413.722737] k3s[873]: I0919 21:27:19.813302 873 scope.go:122] "RemoveContainer" containerID="2b614dc6d0de68ea75e051e57ba8eb5ac1b4b37a3933d978abbe72caef4c2f2d" machine # [ 413.732891] k3s[873]: I0919 21:27:19.823472 873 scope.go:122] "RemoveContainer" containerID="1fad4be34f8c04e4063583129e70de393080b173541dc8a3034254c4670f21b6" machine # [ 413.740616] k3s[873]: I0919 21:27:19.831225 873 scope.go:122] "RemoveContainer" containerID="2c3ec07dbb77166c06c735c47592a510da9d58987c4cf62ed1a483042d531285" machine # [ 413.749051] k3s[873]: I0919 21:27:19.839580 873 scope.go:122] "RemoveContainer" containerID="db8b1e86a79df4116d33f5d6e8be3a0006c7ff8863fee4ed8f05f9831cdbfde7" machine # [ 413.756033] k3s[873]: I0919 21:27:19.846559 873 scope.go:122] "RemoveContainer" containerID="2b614dc6d0de68ea75e051e57ba8eb5ac1b4b37a3933d978abbe72caef4c2f2d" machine # [ 413.756701] k3s[873]: E0919 21:27:19.847271 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"2b614dc6d0de68ea75e051e57ba8eb5ac1b4b37a3933d978abbe72caef4c2f2d\": not found" containerID="2b614dc6d0de68ea75e051e57ba8eb5ac1b4b37a3933d978abbe72caef4c2f2d" machine # [ 413.757577] k3s[873]: I0919 21:27:19.847311 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"2b614dc6d0de68ea75e051e57ba8eb5ac1b4b37a3933d978abbe72caef4c2f2d"} err="failed to get container status \"2b614dc6d0de68ea75e051e57ba8eb5ac1b4b37a3933d978abbe72caef4c2f2d\": rpc error: code = NotFound desc = an error occurred when try to find container \"2b614dc6d0de68ea75e051e57ba8eb5ac1b4b37a3933d978abbe72caef4c2f2d\": not found" machine # [ 413.758968] k3s[873]: I0919 21:27:19.847346 873 scope.go:122] "RemoveContainer" containerID="1fad4be34f8c04e4063583129e70de393080b173541dc8a3034254c4670f21b6" machine # [ 413.760089] k3s[873]: E0919 21:27:19.847853 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"1fad4be34f8c04e4063583129e70de393080b173541dc8a3034254c4670f21b6\": not found" containerID="1fad4be34f8c04e4063583129e70de393080b173541dc8a3034254c4670f21b6" machine # [ 413.761448] k3s[873]: I0919 21:27:19.847873 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"1fad4be34f8c04e4063583129e70de393080b173541dc8a3034254c4670f21b6"} err="failed to get container status \"1fad4be34f8c04e4063583129e70de393080b173541dc8a3034254c4670f21b6\": rpc error: code = NotFound desc = an error occurred when try to find container \"1fad4be34f8c04e4063583129e70de393080b173541dc8a3034254c4670f21b6\": not found" machine # [ 413.762233] k3s[873]: I0919 21:27:19.847889 873 scope.go:122] "RemoveContainer" containerID="2c3ec07dbb77166c06c735c47592a510da9d58987c4cf62ed1a483042d531285" machine # [ 413.762869] k3s[873]: E0919 21:27:19.848453 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"2c3ec07dbb77166c06c735c47592a510da9d58987c4cf62ed1a483042d531285\": not found" containerID="2c3ec07dbb77166c06c735c47592a510da9d58987c4cf62ed1a483042d531285" machine # [ 413.764086] k3s[873]: I0919 21:27:19.848491 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"2c3ec07dbb77166c06c735c47592a510da9d58987c4cf62ed1a483042d531285"} err="failed to get container status \"2c3ec07dbb77166c06c735c47592a510da9d58987c4cf62ed1a483042d531285\": rpc error: code = NotFound desc = an error occurred when try to find container \"2c3ec07dbb77166c06c735c47592a510da9d58987c4cf62ed1a483042d531285\": not found" machine # [ 413.764859] k3s[873]: I0919 21:27:19.848522 873 scope.go:122] "RemoveContainer" containerID="db8b1e86a79df4116d33f5d6e8be3a0006c7ff8863fee4ed8f05f9831cdbfde7" machine # [ 413.766060] k3s[873]: E0919 21:27:19.848979 873 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"db8b1e86a79df4116d33f5d6e8be3a0006c7ff8863fee4ed8f05f9831cdbfde7\": not found" containerID="db8b1e86a79df4116d33f5d6e8be3a0006c7ff8863fee4ed8f05f9831cdbfde7" machine # [ 413.766688] k3s[873]: I0919 21:27:19.849002 873 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"db8b1e86a79df4116d33f5d6e8be3a0006c7ff8863fee4ed8f05f9831cdbfde7"} err="failed to get container status \"db8b1e86a79df4116d33f5d6e8be3a0006c7ff8863fee4ed8f05f9831cdbfde7\": rpc error: code = NotFound desc = an error occurred when try to find container \"db8b1e86a79df4116d33f5d6e8be3a0006c7ff8863fee4ed8f05f9831cdbfde7\": not found" machine # [ 413.783290] systemd[1]: var-lib-kubelet-pods-e187fb7d\x2d78e0\x2d482b\x2db7fd\x2d27eab0e7c263-volume\x2dsubpaths-nix-gc-0.mount: Deactivated successfully. machine # [ 413.784883] systemd[1]: var-lib-kubelet-pods-58f09918\x2d6c02\x2d42ea\x2da7cd\x2d7a2ebf47c912-volume\x2dsubpaths-nix-gc-0.mount: Deactivated successfully. machine # [ 413.785538] systemd[1]: var-lib-kubelet-pods-58f09918\x2d6c02\x2d42ea\x2da7cd\x2d7a2ebf47c912-volume\x2dsubpaths-nix-nix\x2dgrpc\x2ddaemon-0.mount: Deactivated successfully. machine # [ 413.786755] systemd[1]: var-lib-kubelet-pods-e187fb7d\x2d78e0\x2d482b\x2db7fd\x2d27eab0e7c263-volume\x2dsubpaths-nix-nix\x2dgrpc\x2ddaemon-0.mount: Deactivated successfully. machine # [ 413.787867] systemd[1]: var-lib-kubelet-pods-58f09918\x2d6c02\x2d42ea\x2da7cd\x2d7a2ebf47c912-volume\x2dsubpaths-nix-nix\x2ddaemon-0.mount: Deactivated successfully. machine # [ 413.789233] systemd[1]: var-lib-kubelet-pods-e187fb7d\x2d78e0\x2d482b\x2db7fd\x2d27eab0e7c263-volume\x2dsubpaths-nix-nix\x2ddaemon-0.mount: Deactivated successfully. machine # [ 413.790095] systemd[1]: var-lib-kubelet-pods-e187fb7d\x2d78e0\x2d482b\x2db7fd\x2d27eab0e7c263-volumes-kubernetes.io\x7eprojected-niks3\x2dtoken.mount: Deactivated successfully. machine # [ 413.790639] systemd[1]: var-lib-kubelet-pods-58f09918\x2d6c02\x2d42ea\x2da7cd\x2d7a2ebf47c912-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dgh2kl.mount: Deactivated successfully. machine # [ 413.791591] systemd[1]: var-lib-kubelet-pods-58f09918\x2d6c02\x2d42ea\x2da7cd\x2d7a2ebf47c912-volumes-kubernetes.io\x7esecret-ca.mount: Deactivated successfully. machine # [ 413.792377] systemd[1]: var-lib-kubelet-pods-e187fb7d\x2d78e0\x2d482b\x2db7fd\x2d27eab0e7c263-volumes-kubernetes.io\x7esecret-tls.mount: Deactivated successfully. machine # [ 413.793219] systemd[1]: var-lib-kubelet-pods-e187fb7d\x2d78e0\x2d482b\x2db7fd\x2d27eab0e7c263-volumes-kubernetes.io\x7esecret-ca.mount: Deactivated successfully. machine # [ 413.794433] systemd[1]: var-lib-kubelet-pods-58f09918\x2d6c02\x2d42ea\x2da7cd\x2d7a2ebf47c912-volumes-kubernetes.io\x7esecret-tls.mount: Deactivated successfully. machine # [ 413.795293] systemd[1]: var-lib-kubelet-pods-58f09918\x2d6c02\x2d42ea\x2da7cd\x2d7a2ebf47c912-volumes-kubernetes.io\x7esecret-oidc.mount: Deactivated successfully. machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 414.680591] postgres[771]: [771] LOG: checkpoint complete: wrote 1021 buffers (6.2%), wrote 3 SLRU buffers; 1 WAL file(s) added, 0 removed, 0 recycled; write=102.432 s, sync=0.049 s, total=102.520 s; sync files=363, longest=0.011 s, average=0.001 s; distance=4901 kB, estimate=4901 kB; lsn=0/1C2F680, redo lsn=0/1C2F5F0 machine # [ 414.800148] k3s[873]: I0919 21:27:20.889922 873 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="58f09918-6c02-42ea-a7cd-7a2ebf47c912" path="/var/lib/kubelet/pods/58f09918-6c02-42ea-a7cd-7a2ebf47c912/volumes" machine # [ 414.807164] k3s[873]: I0919 21:27:20.897604 873 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e187fb7d-78e0-482b-b7fd-27eab0e7c263" path="/var/lib/kubelet/pods/e187fb7d-78e0-482b-b7fd-27eab0e7c263/volumes" machine # [ 415.370242] dhcpcd[6337]: vethfebecc9a: probing for an IPv4LL address machine # [ 415.392691] dhcpcd[6337]: vethe1ddc4e5: probing for an IPv4LL address machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 416.320431] dhcpcd[6337]: vethc8ce4203: probing for an IPv4LL address machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 420.218595] dhcpcd[6337]: vethe1ddc4e5: using IPv4LL address 169.254.37.0 machine # [ 420.219116] dhcpcd[6337]: vethe1ddc4e5: adding route to 169.254.0.0/16 machine # [ 420.527777] dhcpcd[6337]: vethfebecc9a: using IPv4LL address 169.254.249.70 machine # [ 420.529007] dhcpcd[6337]: vethfebecc9a: adding route to 169.254.0.0/16 machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 421.080366] dhcpcd[6337]: vethc8ce4203: using IPv4LL address 169.254.96.18 machine # [ 421.080844] dhcpcd[6337]: vethc8ce4203: adding route to 169.254.0.0/16 machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # [ 423.589350] dhcpcd[6337]: vethc8ce4203: no IPv6 Routers available machine # [ 423.805628] dhcpcd[6337]: vethfebecc9a: no IPv6 Routers available machine # [ 424.076254] dhcpcd[6337]: vethe1ddc4e5: no IPv6 Routers available machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # Error from server (NotFound): deployments.apps "nix-grpc-farm-scheduler" not found machine # NAMESPACE NAME READY STATUS RESTARTS AGE machine # farm nix-grpc-farm-lb-688d8d547b-955kq 1/1 Running 0 78s machine # farm nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6 3/3 Running 0 78s machine # farm nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn 3/3 Running 0 78s machine # kube-system coredns-c5fdd76cf-rjgdg 1/1 Running 0 7m37s machine # kube-system helm-install-nix-grpc-farm-zcg5w 0/1 Error 6 (4m2s ago) 7m14s machine # [ 487.793577] k3s[873]: I0919 21:28:33.883288 873 scope.go:122] "RemoveContainer" containerID="007d47a685973e2a9413ce9e05b2306fe3096d78ac9d0114cad13819d726af6d" machine # [ 487.794632] k3s[873]: E0919 21:28:33.883483 873 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"helm\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=helm pod=helm-install-nix-grpc-farm-zcg5w_kube-system(6d037147-1d29-4e02-b2ab-2f08bc2a576c)\"" pod="kube-system/helm-install-nix-grpc-farm-zcg5w" podUID="6d037147-1d29-4e02-b2ab-2f08bc2a576c" machine # farm 79s Normal Killing pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-hdwfw Stopping container nix-grpc-daemon machine # farm 79s Normal Killing pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-4tgln Stopping container gc machine # farm 79s Normal Killing pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-4tgln Stopping container nix-daemon machine # farm 79s Normal Killing pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-4tgln Stopping container nix-grpc-daemon machine # farm 78s Normal ScalingReplicaSet deployment/nix-grpc-farm-lb Scaled up replica set nix-grpc-farm-lb-688d8d547b from 0 to 1 machine # farm 78s Normal SuccessfulCreate replicaset/nix-grpc-farm-lb-688d8d547b Created pod: nix-grpc-farm-lb-688d8d547b-955kq machine # farm 78s Normal SuccessfulCreate replicaset/nix-grpc-farm-worker-x86-64-linux-5cdbdb595 Created pod: nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6 machine # farm 78s Normal SuccessfulCreate replicaset/nix-grpc-farm-worker-x86-64-linux-5cdbdb595 Created pod: nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn machine # farm 78s Normal ScalingReplicaSet deployment/nix-grpc-farm-worker-x86-64-linux Scaled up replica set nix-grpc-farm-worker-x86-64-linux-5cdbdb595 from 0 to 2 machine # kube-system 78s Warning BackOff pod/helm-install-nix-grpc-farm-zcg5w Back-off restarting failed container helm in pod helm-install-nix-grpc-farm-zcg5w_kube-system(6d037147-1d29-4e02-b2ab-2f08bc2a576c) machine # farm 77s Normal Created pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn Container created machine # farm 77s Normal Pulled pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6 Container image "nix-grpc-farm:0.1.0" already present on machine and can be accessed by the pod machine # farm 77s Normal Started pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn Container started machine # farm 77s Normal Created pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6 Container created machine # farm 77s Normal Started pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6 Container started machine # farm 77s Normal Pulled pod/nix-grpc-farm-lb-688d8d547b-955kq Container image "nix-grpc-farm-lb:0.1.0" already present on machine and can be accessed by the pod machine # farm 77s Normal Created pod/nix-grpc-farm-lb-688d8d547b-955kq Container created machine # farm 77s Normal Started pod/nix-grpc-farm-lb-688d8d547b-955kq Container started machine # farm 77s Normal Pulled pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn Container image "nix-grpc-farm:0.1.0" already present on machine and can be accessed by the pod machine # farm 76s Normal Pulled pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn Container image "nix-grpc-farm:0.1.0" already present on machine and can be accessed by the pod machine # farm 76s Normal Created pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6 Container created machine # farm 76s Normal Started pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn Container started machine # farm 76s Normal Pulled pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn Container image "nix-grpc-farm:0.1.0" already present on machine and can be accessed by the pod machine # farm 76s Normal Created pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6 Container created machine # farm 76s Normal Pulled pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6 Container image "nix-grpc-farm:0.1.0" already present on machine and can be accessed by the pod machine # farm 76s Normal Started pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6 Container started machine # farm 76s Warning Unhealthy pod/nix-grpc-farm-lb-688d8d547b-955kq Readiness probe failed: HTTP probe failed with statuscode: 503 machine # farm 76s Normal Created pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn Container created machine # farm 76s Normal Pulled pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6 Container image "nix-grpc-farm:0.1.0" already present on machine and can be accessed by the pod machine # farm 76s Normal Created pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn Container created machine # farm 75s Normal Started pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6 Container started machine # farm 75s Normal Created pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6 Container created machine # farm 75s Normal Pulled pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6 Container image "nix-grpc-farm:0.1.0" already present on machine and can be accessed by the pod machine # farm 75s Warning Unhealthy pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-4tgln Readiness probe failed: dial tcp 10.42.0.24:50051: connect: connection refused machine # farm 75s Normal Started pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn Container started machine # farm 75s Normal Pulled pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn Container image "nix-grpc-farm:0.1.0" already present on machine and can be accessed by the pod machine # farm 75s Normal Started pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6 Container started machine # farm 75s Normal Created pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn Container created machine # farm 75s Normal Started pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn Container started machine # farm 75s Warning Unhealthy pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-hdwfw Readiness probe failed: dial tcp 10.42.0.23:50051: connect: connection refused machine # + [[ -z 1 ]] machine # + check_revision 1 machine # + set -e machine # + REVISION=1 machine # + [[ -n 1 ]] machine # + [[ 1 != 1 ]] machine # + set +e machine # + [[ failed =~ ^(pending-install|pending-upgrade|pending-rollback|superseded|uninstalling)$ ]] machine # + [[ failed =~ ^deployed$ ]] machine # + [[ failed =~ ^(deleted|failed|null|unknown)$ ]] machine # + case "${FAILURE_POLICY:-reinstall}" in machine # + echo 'Uninstalling failed helm chart' machine # + helm uninstall nix-grpc-farm --namespace farm --wait machine # release "nix-grpc-farm" uninstalled machine # + echo Deleted machine # ++ get_ssa_args '' '' auto false machine # ++ set -e machine # ++ REVISION= machine # ++ APPLY_METHOD= machine # ++ SERVER_SIDE=auto machine # ++ FORCE_CONFLICTS=false machine # ++ [[ -z '' ]] machine # ++ [[ auto =~ ^auto$ ]] machine # ++ SERVER_SIDE=true machine # ++ [[ false =~ ^true$ ]] machine # ++ [[ -n true ]] machine # ++ echo -n '--server-side=true ' machine # ++ [[ -n false ]] machine # ++ echo -n '--force-conflicts=false ' machine # ++ echo machine # ++ set +e machine # + SSA_ARGS='--server-side=true --force-conflicts=false ' machine # + echo 'Installing helm chart' machine # + helm install --namespace farm nix-grpc-farm https://10.43.0.1:443/static/charts/nix-grpc-farm.tgz --labels helmcharts.helm.cattle.io/configHash=68C1C1B485DA72929DFAF7B1476413DFE982F4E7429B88269B82ADF71F73F2A --server-side=true --force-conflicts=false --values /config/values-0-000-HelmChart-ValuesContent.yaml machine # Deleted machine # Error: INSTALLATION FAILED: failed to create typed patch object (farm/nix-grpc-farm-scheduler; /v1, Kind=Service): .spec.selector.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0} machine # failed to create typed patch object (farm/nix-grpc-farm-scheduler; apps/v1, Kind=Deployment): errors: machine # .spec.selector.matchLabels.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0} machine # .spec.template.metadata.labels.app.kubernetes.io/instance-rank: expected string, got &value.valueUnstructured{Value:0} machine # + exit machine # Host Port: machine # Command: machine # /bin/sh machine # -ec machine # Args: machine # while :; do machine # harmonia-gc --ensure-free 2G --no-vacuum || echo "gc: exit $?" >&2 machine # sleep 10 machine # done machine # machine # State: Running machine # Started: Sat, 19 Sep 2026 21:27:18 +0000 machine # Ready: True machine # Restart Count: 0 machine # Environment: machine # Mounts: machine # /etc/nix/nix.conf.d from nixconf (rw) machine # /nix from nix (rw,path="nix") machine # /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-rmjvn (ro) machine # Conditions: machine # Type Status machine # PodReadyToStartContainers True machine # Initialized True machine # Ready True machine # ContainersReady True machine # PodScheduled True machine # Volumes: machine # nix: machine # Type: EmptyDir (a temporary directory that shares a pod's lifetime) machine # Medium: machine # SizeLimit: machine # build: machine # Type: EmptyDir (a temporary directory that shares a pod's lifetime) machine # Medium: machine # SizeLimit: machine # nixconf: machine # Type: ConfigMap (a volume populated by a ConfigMap) machine # Name: nix-grpc-farm-worker-x86-64-linux machine # Optional: false machine # niks3-token: machine # Type: Projected (a volume that contains injected data from multiple sources) machine # TokenExpirationSeconds: 3600 machine # tls: machine # Type: Secret (a volume populated by a Secret) machine # SecretName: farm-worker-tls machine # Optional: false machine # ca: machine # Type: Secret (a volume populated by a Secret) machine # SecretName: farm-ca machine # Optional: false machine # oidc: machine # Type: Secret (a volume populated by a Secret) machine # SecretName: nix-grpc-farm-oidc machine # Optional: false machine # kube-api-access-rmjvn: machine # Type: Projected (a volume that contains injected data from multiple sources) machine # TokenExpirationSeconds: 3607 machine # ConfigMapName: kube-root-ca.crt machine # Optional: false machine # DownwardAPI: true machine # QoS Class: BestEffort machine # Node-Selectors: machine # Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s machine # node.kubernetes.io/unreachable:NoExecute op=Exists for 300s machine # Events: machine # Type Reason Age From Message machine # ---- ------ ---- ---- ------- machine # Normal Scheduled 79s default-scheduler Successfully assigned farm/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn to machine machine # Normal Pulled 78s kubelet spec.initContainers{seed-store}: Container image "nix-grpc-farm:0.1.0" already present on machine and can be accessed by the pod machine # Normal Created 78s kubelet spec.initContainers{seed-store}: Container created machine # Normal Started 78s kubelet spec.initContainers{seed-store}: Container started machine # Normal Pulled 77s kubelet spec.containers{nix-daemon}: Container image "nix-grpc-farm:0.1.0" already present on machine and can be accessed by the pod machine # Normal Created 77s kubelet spec.containers{nix-daemon}: Container created machine # Normal Started 77s kubelet spec.containers{nix-daemon}: Container started machine # Normal Pulled 77s kubelet spec.containers{nix-grpc-daemon}: Container image "nix-grpc-farm:0.1.0" already present on machine and can be accessed by the pod machine # Normal Created 77s kubelet spec.containers{nix-grpc-daemon}: Container created machine # Normal Started 76s kubelet spec.containers{nix-grpc-daemon}: Container started machine # Normal Pulled 76s kubelet spec.containers{gc}: Container image "nix-grpc-farm:0.1.0" already present on machine and can be accessed by the pod machine # Normal Created 76s kubelet spec.containers{gc}: Container created machine # Normal Started 76s kubelet spec.containers{gc}: Container started machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/nix-daemon] accepted connection from pid 66, user root (trusted) machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/nix-daemon] accepted connection from pid 97, user root (trusted) machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/nix-grpc-daemon] ts=2026-09-19T21:27:18Z level=info event=oidc_provider name=kubernetes issuer=https://kubernetes.default.svc.cluster.local machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/nix-grpc-daemon] ts=2026-09-19T21:27:18Z level=info event=roles builder=1 scheduler=0 scheduler_addr=nix-grpc-farm-scheduler.farm.svc:50051 advertise=10.42.0.28:50051 niks3=http://192.168.1.1:5751 max_jobs=2 machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/nix-grpc-daemon] ts=2026-09-19T21:27:18Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/nix-grpc-daemon] ts=2026-09-19T21:27:19Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/nix-grpc-daemon] ts=2026-09-19T21:27:19Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/nix-grpc-daemon] ts=2026-09-19T21:27:19Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/nix-grpc-daemon] ts=2026-09-19T21:27:20Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/nix-grpc-daemon] ts=2026-09-19T21:27:22Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/nix-grpc-daemon] ts=2026-09-19T21:27:25Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/nix-grpc-daemon] ts=2026-09-19T21:27:30Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/nix-grpc-daemon] ts=2026-09-19T21:27:35Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/nix-grpc-daemon] ts=2026-09-19T21:27:40Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/nix-grpc-daemon] ts=2026-09-19T21:27:45Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/nix-grpc-daemon] ts=2026-09-19T21:27:50Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/nix-grpc-daemon] ts=2026-09-19T21:27:55Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/nix-grpc-daemon] ts=2026-09-19T21:28:00Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/nix-grpc-daemon] ts=2026-09-19T21:28:05Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/nix-grpc-daemon] ts=2026-09-19T21:28:10Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/nix-grpc-daemon] ts=2026-09-19T21:28:15Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/nix-grpc-daemon] ts=2026-09-19T21:28:20Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/nix-grpc-daemon] ts=2026-09-19T21:28:25Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/nix-grpc-daemon] ts=2026-09-19T21:28:30Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/gc] 2026-09-19T21:27:18.179646Z WARN harmonia_gc: no private mount namespace: setting up a private mount namespace: EPERM: Operation not permitted machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/gc] 11.99 GiB already free, nothing to do machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/gc] 2026-09-19T21:27:28.185467Z WARN harmonia_gc: no private mount namespace: setting up a private mount namespace: EPERM: Operation not permitted machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/gc] 12.54 GiB already free, nothing to do machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/gc] 2026-09-19T21:27:38.193156Z WARN harmonia_gc: no private mount namespace: setting up a private mount namespace: EPERM: Operation not permitted machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/gc] 12.54 GiB already free, nothing to do machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/gc] 2026-09-19T21:27:48.200657Z WARN harmonia_gc: no private mount namespace: setting up a private mount namespace: EPERM: Operation not permitted machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/gc] 12.54 GiB already free, nothing to do machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/gc] 2026-09-19T21:27:58.207993Z WARN harmonia_gc: no private mount namespace: setting up a private mount namespace: EPERM: Operation not permitted machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/gc] 12.54 GiB already free, nothing to do machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/gc] 2026-09-19T21:28:08.215591Z WARN harmonia_gc: no private mount namespace: setting up a private mount namespace: EPERM: Operation not permitted machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/gc] 12.54 GiB already free, nothing to do machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/gc] 2026-09-19T21:28:18.222952Z WARN harmonia_gc: no private mount namespace: setting up a private mount namespace: EPERM: Operation not permitted machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/gc] 12.54 GiB already free, nothing to do machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/gc] 2026-09-19T21:28:28.229140Z WARN harmonia_gc: no private mount namespace: setting up a private mount namespace: EPERM: Operation not permitted machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-dmtv6/gc] 12.54 GiB already free, nothing to do machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/nix-daemon] accepted connection from pid 66, user root (trusted) machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/nix-daemon] accepted connection from pid 97, user root (trusted) machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/nix-grpc-daemon] ts=2026-09-19T21:27:18Z level=info event=oidc_provider name=kubernetes issuer=https://kubernetes.default.svc.cluster.local machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/nix-grpc-daemon] ts=2026-09-19T21:27:18Z level=info event=roles builder=1 scheduler=0 scheduler_addr=nix-grpc-farm-scheduler.farm.svc:50051 advertise=10.42.0.27:50051 niks3=http://192.168.1.1:5751 max_jobs=2 machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/nix-grpc-daemon] ts=2026-09-19T21:27:18Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/nix-grpc-daemon] ts=2026-09-19T21:27:19Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/nix-grpc-daemon] ts=2026-09-19T21:27:19Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/nix-grpc-daemon] ts=2026-09-19T21:27:19Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/nix-grpc-daemon] ts=2026-09-19T21:27:20Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/nix-grpc-daemon] ts=2026-09-19T21:27:22Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/nix-grpc-daemon] ts=2026-09-19T21:27:25Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/nix-grpc-daemon] ts=2026-09-19T21:27:30Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/nix-grpc-daemon] ts=2026-09-19T21:27:35Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/nix-grpc-daemon] ts=2026-09-19T21:27:40Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/nix-grpc-daemon] ts=2026-09-19T21:27:45Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/nix-grpc-daemon] ts=2026-09-19T21:27:50Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/nix-grpc-daemon] ts=2026-09-19T21:27:55Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/nix-grpc-daemon] ts=2026-09-19T21:28:00Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/nix-grpc-daemon] ts=2026-09-19T21:28:05Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/nix-grpc-daemon] ts=2026-09-19T21:28:10Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/nix-grpc-daemon] ts=2026-09-19T21:28:15Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/nix-grpc-daemon] ts=2026-09-19T21:28:20Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/nix-grpc-daemon] ts=2026-09-19T21:28:25Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/nix-grpc-daemon] ts=2026-09-19T21:28:30Z level=info event=scheduler_disconnected addr=nix-grpc-farm-scheduler.farm.svc:50051 error="errors resolving nix-grpc-farm-scheduler.farm.svc:50051: [field:hostname lookup error:address lookup failed for nix-grpc-farm-scheduler.farm.svc:50051: DNS query cancelled]" machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/gc] 2026-09-19T21:27:18.184411Z WARN harmonia_gc: no private mount namespace: setting up a private mount namespace: EPERM: Operation not permitted machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/gc] 11.99 GiB already free, nothing to do machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/gc] 2026-09-19T21:27:28.191080Z WARN harmonia_gc: no private mount namespace: setting up a private mount namespace: EPERM: Operation not permitted machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/gc] 12.54 GiB already free, nothing to do machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/gc] 2026-09-19T21:27:38.200848Z WARN harmonia_gc: no private mount namespace: setting up a private mount namespace: EPERM: Operation not permitted machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/gc] 12.54 GiB already free, nothing to do machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/gc] 2026-09-19T21:27:48.207256Z WARN harmonia_gc: no private mount namespace: setting up a private mount namespace: EPERM: Operation not permitted machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/gc] 12.54 GiB already free, nothing to do machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/gc] 2026-09-19T21:27:58.214428Z WARN harmonia_gc: no private mount namespace: setting up a private mount namespace: EPERM: Operation not permitted machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/gc] 12.54 GiB already free, nothing to do machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/gc] 2026-09-19T21:28:08.221814Z WARN harmonia_gc: no private mount namespace: setting up a private mount namespace: EPERM: Operation not permitted machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/gc] 12.54 GiB already free, nothing to do machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/gc] 2026-09-19T21:28:18.229154Z WARN harmonia_gc: no private mount namespace: setting up a private mount namespace: EPERM: Operation not permitted machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/gc] 12.54 GiB already free, nothing to do machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/gc] 2026-09-19T21:28:28.235282Z WARN harmonia_gc: no private mount namespace: setting up a private mount namespace: EPERM: Operation not permitted machine # [pod/nix-grpc-farm-worker-x86-64-linux-5cdbdb595-v2mbn/gc] 12.54 GiB already free, nothing to do machine # No resources found in farm namespace. !!! Test "chart deploys and workers become Ready" failed with error: "action timed out after 420.55 seconds (timeout=420.0)" !!! Traceback (most recent call last): !!! File "", line 19, in !!! machine.wait_until_succeeds(kubectl + "rollout status deployment nix-grpc-farm-scheduler --timeout=10s", timeout=420) !!! !!! RequestedAssertionFailed: action timed out after 420.55 seconds (timeout=420.0) cleanup kill QemuMachine (pid 45) machine # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14) (finished: cleanup, in 1.18 seconds) additionally exposed symbols: machine, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh